ss-tools/specs/017-llm-analysis-plugin/data-model.md

# Data Model: LLM Analysis Plugin v2

**Feature**: `017-llm-analysis-plugin`
**Updated**: 2026-06-07 — v2 implemented: ValidationPolicy expansion, ValidationSource, dual-path execution models

## Entities

### LLMProviderConfig
Stores configuration for different LLM providers.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| id | UUID | Yes | Unique identifier |
| provider_type | Enum | Yes | `openai`, `openrouter`, `kilo`, `litellm` |
| name | String | Yes | Display name (e.g., "Production GPT-4") |
| base_url | String | Yes | API Endpoint URL |
| api_key | String | Yes | Encrypted API Key (AES-256) |
| default_model | String | Yes | Model identifier (e.g., `gpt-4o`) |
| is_active | Boolean | Yes | Whether this provider is currently enabled |
| is_multimodal | Boolean | Yes | Whether model supports image input (required for Path A) |

### ValidationPolicy (v2 — expanded)

Stores configuration for a scheduled validation task. Uses ORM relationship to `ValidationSource` table — no JSON column denormalization.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| id | UUID | Yes | Unique identifier |
| name | String | Yes | Human-readable task name |
| description | String | No | Optional description |
| environment_id | String | Yes | Target Superset environment |
| sources | relationship → list[ValidationSource] | Yes | ORM relationship only. Dashboards to validate (1..N). Migrated from v1 `dashboard_ids` list via `ValidationSource` rows |
| source_snapshot | JSON | No | Read-only denormalized copy of sources at last edit time — for audit/diff only, never used as authoritative state |
| provider_id | String | Yes | LLM provider to use (per-task, not global) |
| prompt_template | String | No | Custom prompt; `null` = use path-specific default: `dashboard_validation_prompt_multimodal` for Path A, `dashboard_validation_prompt_text` for Path B. Must NOT fall back to v1 legacy `dashboard_validation_prompt` |
| screenshot_enabled | Boolean | Yes | `true` = Path A (Playwright + multimodal LLM), `false` = Path B (text-only) |
| logs_enabled | Boolean | Yes | Fetch Superset execution logs for LLM context |
| execute_chart_data | Boolean | Yes | Path B only: execute `POST /api/v1/chart/data` for each chart (default: false) |
| llm_batch_size | Integer | No | Path B only: dashboards per LLM call (default: 1 — full isolation, one call per dashboard). Values >1 are experimental. Path A always uses 1 (image token constraints). |
| policy_dashboard_concurrency_limit | Integer | No | Max dashboards validated in parallel within one policy run (default: 3). Global worker limit controlled by `global_validation_worker_limit` config |
| cron_expression | String | No | Cron schedule; null = manual trigger only |
| is_active | Boolean | Yes | Whether schedule is currently active |
| notification_channels | list[String] | No | Email/Pulse channel IDs for failure notifications |
| created_at | DateTime | Yes | Creation timestamp |
| updated_at | DateTime | Yes | Last modification timestamp |

### ValidationSource (new)

Represents a single dashboard to validate within a task — either by numeric ID or by full Superset URL.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| id | UUID | Yes | Unique identifier |
| policy_id | UUID | Yes | FK → ValidationPolicy |
| type | Enum | Yes | `dashboard_id` or `dashboard_url` |
| value | String | Yes | Numeric dashboard ID or full Superset URL |
| parsed_context | JSON | No | For `dashboard_url`: result of `SupersetContextExtractor.parse_superset_link()` — contains dashboard_id, native_filters, activeTabs, anchor |
| status | Enum | Yes | `valid`, `partial_recovery` (SupersetContextExtractor returned partial match), `stale` (permalink expired, URL parse warning), `invalid` (dashboard deleted or not found), `permission_denied`, `parse_failed` |
| resolved_dashboard_id | String | No | Resolved numeric dashboard ID (cached after URL parse for display) |
| title | String | No | Resolved dashboard title (cached for display) |
| last_error | String | No | Last resolution error message |
| last_checked_at | DateTime | No | When the source was last validated/resolved |

### Migration from v1 dashboard_ids:

```python
# Existing dashboard_ids: list[str] → ValidationSource rows (relational only, no JSON denormalization)
for dashboard_id in old_policy.dashboard_ids:
    db.add(ValidationSource(
        policy_id=policy.id,
        type="dashboard_id",
        value=dashboard_id,
        is_valid=True,
    ))
# Optional: populate source_snapshot JSON for audit trail
policy.source_snapshot = {"dashboard_ids": list(old_policy.dashboard_ids)}
```

### ValidationRun (new — groups records from one execution)

A single task execution validates ALL dashboards in a policy. `ValidationRun` groups these per-dashboard `ValidationRecord`s into one logical unit.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| id | UUID | Yes | Unique run identifier |
| policy_id | UUID | Yes | FK → ValidationPolicy |
| task_id | String | No | Associated Task Manager task ID (one run = one task) |
| started_at | DateTime | Yes | When the run started |
| finished_at | DateTime | No | When all dashboards finished (null while running) |
| trigger | Enum | Yes | `manual` or `scheduled` |
| status | Enum | Yes | `completed` (all dashboards done), `partial` (some sources skipped/invalid), `failed` (execution error before any dashboard) |
| dashboard_count | Integer | Yes | Total dashboards attempted |
| pass_count | Integer | Yes | PASS results |
| warn_count | Integer | Yes | WARN results |
| fail_count | Integer | Yes | FAIL results |
| unknown_count | Integer | Yes | UNKNOWN results (LLM errors, timeouts) |

### ValidationRecord (v2 — expanded, linked to run)

Stores the outcome of one dashboard validation within a task run. Linked to `ValidationRun` via `run_id`.

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| id | UUID | Yes | Unique identifier |
| run_id | UUID | Yes | FK → ValidationRun — groups per-dashboard results from one task execution |
| task_id | String | No | Deprecated in favor of run_id; kept for v1 backward compat |
| policy_id | UUID | No | FK → ValidationPolicy |
| source_id | UUID | No | FK → ValidationSource |
| dashboard_id | String | Yes | Resolved dashboard ID |
| status | Enum | Yes | `PASS`, `WARN`, `FAIL`, `UNKNOWN` |
| execution_path | Enum | Yes | `screenshot` (Path A) or `text_only` (Path B) |
| summary | String | Yes | LLM-generated summary |
| issues | JSON | Yes | `[{severity, message, location}]` |
| dataset_health | JSON | No | Path B only: per-dataset status `[{dataset_id, name, database, backend, healthy, error, affected_charts}]` |
| chart_data_results | JSON | No | Path B with `execute_chart_data=true`: `[{chart_id, executed, duration_ms, row_count, error}]` |
| screenshot_paths | list[String] | No | Path A: paths to archived WebP screenshots (one per tab). PNG/JPEG intermediates deleted after conversion |
| tab_screenshots | JSON | No | Path A: per-tab mapping `[{tab_name, webp_path, errored}]` |
| logs_sent_to_llm | list[String] | No | Redacted Superset execution logs sent to LLM |
| token_usage | JSON | No | LLM token usage `{prompt_tokens, completion_tokens, total_tokens}` (when provider reports it) |
| raw_response | Text | No | Full LLM response for debugging. Retention: 30 days per FR-029a |
| timings | JSON | No | Phase timings: `{dashboard_fetch_ms, dataset_fetch_ms, chart_fetch_ms, chart_data_exec_ms, screenshot_ms, llm_call_ms, total_ms}` |
| created_at | DateTime | Yes | Record creation timestamp |

### Path B — Dataset Health Check (runtime model, not persisted separately)

Built during Path B execution from API responses:

```python
class DatasetHealth:
    dataset_id: int
    dataset_name: str           # e.g. "finance.kpi_revenue"
    database_name: str          # e.g. "PostgreSQL / analytics_prod"
    backend: str                # e.g. "postgresql", "trino"
    kind: str                   # "virtual" or "physical"
    is_accessible: bool         # HTTP 200 from GET /api/v1/dataset/{id}
    error: str | None           # Connection / timeout error
    affected_chart_ids: list[int]  # Charts using this dataset
```

### Chart Execution Result (runtime model, Path B + execute_chart_data)

```python
class ChartDataResult:
    chart_id: int
    chart_name: str
    viz_type: str
    executed: bool              # False if skipped
    duration_ms: int | None
    row_count: int | None
    error: str | None           # Timeout / query error
```

## API Contracts

See `contracts/` directory.

### Key Interactions — v2

1. **Configure Provider**: `POST /api/llm/providers` (unchanged)
2. **Create Validation Task**: `POST /api/validation-tasks` — creates `ValidationPolicy` + `ValidationSource`s
3. **List Tasks**: `GET /api/validation-tasks` — returns all policies with aggregated status
4. **Get Task Detail**: `GET /api/validation-tasks/{policy_id}` — returns policy + per-source status
5. **Trigger Run**: `POST /api/validation-tasks/{policy_id}/run` — manual trigger (also fires on cron)
6. **Get Run History**: `GET /api/validation-tasks/{policy_id}/runs` — paginated `ValidationRecord` list
7. **Get Run Detail**: `GET /api/validation-tasks/{policy_id}/runs/{record_id}` — full report
8. **Parse URL**: `POST /api/validation-tasks/parse-url` — returns `{dashboard_id, title, filters, tabs}` from URL
9. **Generate Documentation**: `POST /api/tasks` with `plugin_id=llm_documentation` (unchanged)
10. **Generate Commit Message**: `POST /api/git/generate-message` (unchanged)