# Quickstart: LLM Validation v2

## Prerequisites

1. An API Key from OpenAI, OpenRouter, Kilo, or LiteLLM.
2. At least one environment configured in ss-tools (pointing to a Superset instance).
3. (Optional) Playwright browsers installed for Path A (screenshot) validation:
   ```bash
   playwright install chromium
   ```

## Configuration

### LLM Provider

1. Navigate to **Settings > LLM Providers**.
2. Click **"Add Provider"**.
3. Select type (e.g., `OpenRouter`).
4. Enter your API Key and Base URL (e.g., `https://openrouter.ai/api/v1`).
5. Set a default model (e.g., `gpt-4o` for multimodal, `gpt-4o-mini` for text-only).
6. Check **"Is Multimodal"** if the model supports image input (required for Path A).
7. Click **Save** and **Test Connection**.

> **Note**: Provider bindings for Documentation, Git Commits, and Chat are configured in Settings. Dashboard Validation provider is selected **per-task** in the task creation form — not in global settings.

### Validation Prompt (Optional)

The default validation prompt covers standard dashboard health checks. To customize:
1. When creating a task, edit the **Prompt Template** field.
2. Path A uses `dashboard_validation_prompt_multimodal` (image-focused).
3. Path B uses `dashboard_validation_prompt_text` (topology-focused).

## Usage

### Create a Validation Task

1. Go to **Validation Tasks** (`/validation-tasks`).
2. Click **"+ New Task"**.
3. **Step 1 — Name & Environment**: Enter task name (e.g., "Daily KXD Health Check"), select Superset environment.
4. **Step 2 — Sources**: Add dashboards to validate:
   - **From List**: Check dashboards from the environment list.
   - **From URL**: Paste a full Superset dashboard URL (with filters/tabs). System parses URL and shows dashboard title, active filters, and tabs.
5. **Step 3 — Analysis Options**:
   - Select **LLM Provider** (filtered: multimodal-only when Screenshot enabled).
   - Toggle **Screenshot** (Playwright):
     - **ON** (Path A): Captures per-tab screenshots, sends to multimodal LLM. Catches visual bugs, rendering errors.
     - **OFF** (Path B): Text-only analysis via Superset API. Checks dataset health, KXD connectivity, chart parameters, execution logs. Lighter, faster, catches data-layer errors.
   - Toggle **Logs**: Include Superset execution logs in LLM context.
   - Toggle **Execute Chart Data** (Path B only): Runs actual chart queries to verify data returns. Off by default (load-aware).
   - Edit **Prompt Template** (optional): Customize what the LLM checks.
6. **Step 4 — Schedule**:
   - **Manual**: Run on demand via "Run Now" button.
   - **Scheduled**: Cron expression (e.g., `0 8 * * 1-5` for weekdays at 8 AM).
7. Click **Create Task**.

### View Results

1. Go to **Validation Tasks** → see all tasks with last run status.
2. Click a task → see per-dashboard breakdown with PASS/WARN/FAIL badges.
3. Click a dashboard run → **Full Report**:
   - **Path A**: Tab selector + per-tab screenshots + issues table.
   - **Path B**: Dataset health table (backend, status, errors) + chart data results + issues table.
   - Both paths: Summary, execution logs sent to LLM, task execution logs, timing breakdown.

### Dashboard Hub (changes)

- The per-row "Validate" action is **removed**.
- The LLM validation status column is **removed**.
- Each dashboard row now shows a **colored dot indicator** next to the title (🟢 PASS / 🟡 WARN / 🔴 FAIL / ⚪ UNKNOWN) reflecting the most recent validation across all tasks. Click the dot to see the last 3 runs with links to full reports.
- Use the **«+ Create Validation Task»** button in the toolbar to create tasks with pre-selected dashboard IDs.
- The **dashboard detail page** (`/dashboards/{id}`) now includes a «Validation History» section showing the last 10 validation runs for this dashboard.

### Dataset Documentation

*(unchanged from v1)*

1. Go to **Tools > Documentation**.
2. Select a dataset.
3. Click **"Generate Docs"**.
4. Review and apply generated descriptions.

### Git Commit Generation

*(unchanged from v1)*

1. Stage files in the Git plugin.
2. Click the **"✨" (Magic)** button.
3. Review and edit the suggested message.

## Path Selection Guide

| Scenario | Choose |
|----------|--------|
| Checking if dashboards render correctly after Superset upgrade | Path A (Screenshot ON) |
| Checking if KXD/Trino data sources are healthy and returning data | Path B (Screenshot OFF) |
| Both visual and data-layer issues | Create two tasks — one Path A, one Path B |
| Lightweight daily health check with 20 dashboards | Path B, no chart data execution |
| Deep investigation of specific dashboard with filters | Path A with URL source (preserves filters) |