@@ -53,11 +53,13 @@ You **MUST** consider the user input before proceeding (if not empty).
Research must resolve only implementation-shaping unknowns that matter for this repository, such as:
- module placement under `backend/src/` or `frontend/src/`
- **Screen Model topology**: which screens need a `[TYPE Model]` (`.svelte.ts`), which atoms each model declares, which invariants cross widget boundaries
- API endpoint design (REST routes, WebSocket channels)
- test strategy (pytest + vitest; L1 model invariants without render + L2 UX contracts with render)
- belief runtime instrumentation for C4/C5 flows
- semantic validation boundaries and static verification workflow
@@ -71,9 +73,17 @@ Use `[NEED_CONTEXT: target]` instead of inventing relation targets, DTO names, o
## Phase 1: Design, ADR Continuity, and Contracts
### Frontend Component Reuse Scan (MANDATORY — before contract generation)
### Frontend Model & Component Reuse Scan (MANDATORY — before contract generation)
Before designing any new Svelte component, execute a **component inventory scan** of the existing codebase to maximise reuse and prevent duplicate primitives. Use a subagent with `subagent_type: "explore"` to scan:
Before designing any new screen, execute a **model-first inventory scan** followed by a **component inventory scan** of the existing codebase to maximise reuse and prevent duplicate primitives.
**Step 1: Screen Model scan** (use a subagent with `subagent_type: "explore"`):
- Search `frontend/src/lib/models/` for existing `[TYPE Model]` contracts
- Use `axiom_semantic_discovery search_contracts type="Model" query="<domain>"` for structured search
- Check model atoms, actions, and invariants — reuse if the screen state maps to an existing model
- New models use `.svelte.ts` extension, `[TYPE Model]` contract, `@STATE`/`@ACTION`/`@INVARIANT` tags
@@ -93,7 +103,7 @@ Before designing any new Svelte component, execute a **component inventory scan*
| Pattern exists (badge, skeleton, tooltip) | Document the Tailwind classes to replicate; no component extraction |
| No reusable asset exists | Create new component only then |
**Output:** The `contracts/modules.md` for every frontend contract MUST include `@RELATION` edges to reused components and a `@RATIONALE` noting WHY the component is reused rather than rebuilt. For pattern-only reuse, the contract MUST reference the source page/file where the pattern was observed.
**Output:** The `contracts/modules.md` for every frontend contract MUST include `@RELATION` edges to reused components/models and a `@RATIONALE` noting WHY the asset is reused rather than rebuilt. For pattern-only reuse, the contract MUST reference the source page/file where the pattern was observed. Components that bind to a Screen Model declare `@RELATION BINDS_TO -> [ModelId]`.
**Forbidden patterns:**
- Creating a new `<Modal>` when `confirm()` suffices
@@ -115,7 +125,8 @@ Generate `data-model.md` for ss-tools domain entities such as:
- Task state transitions
- Git operation entities
- Plugin configuration schemas
- Frontend TypeScript types (when feature is fullstack)
-**Frontend TypeScript DTOs** in `frontend/src/types/` — MUST match backend Pydantic schemas across the stack boundary
- **Screen Model interfaces** — typed atoms, FSM state unions, action payloads for `.svelte.ts` models
### Global ADR Continuity
@@ -123,27 +134,28 @@ Before task decomposition, planning must identify any repo-shaping decisions thi
- Python module layout and decomposition
- FastAPI route organization
- SvelteKit routing and component hierarchy
- **Screen Model topology**: which screens need a model, model-atom boundaries, invariant scope
- **Complexity 4**: `@PURPOSE`, `@RELATION`, `@PRE`, `@POST`, `@SIDE_EFFECT`; orchestration paths should account for belief runtime markers before mutation or return
- [ ] T015 [P] [US1] Create [ScreenName]Model in frontend/src/lib/models/[ScreenName]Model.svelte.ts
- [ ] T016 [P] [US1] Create [Entity] model in backend/src/models/[entity].py
- [ ] T017 [US1] Implement [Service] in backend/src/services/[service].py
- [ ] T018 [US1] Implement [endpoint/feature] in backend/src/api/[file].py
- [ ] T019 [US1] Create [Component] as thin rendering layer in frontend/src/routes/[...] (binds to model via @RELATION BINDS_TO -> [ModelId])
- [ ] T020 [US1] Add @INVARIANT validation in model actions
- [ ] T021 [US1] Add belief-runtime instrumentation in model actions for C4/C5 flows
**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently
@@ -253,3 +258,7 @@ With multiple developers:
- For Complexity 4/5 Python modules, include tasks for belief-state logging paths with `logger.reason()`, `logger.reflect()`, and `belief_scope` where required
- For Complexity 5 or explicitly test-governed contracts, include tasks that cover `@TEST_CONTRACT`, `@TEST_SCENARIO`, `@TEST_FIXTURE`, `@TEST_EDGE`, and `@TEST_INVARIANT`
- Never create tasks from legacy `@TIER` alone; complexity is the primary execution signal
- **Model-first frontend tasks follow this order: types → model → L1 model tests → component → L2 UX tests**
- **Screen Models use `.svelte.ts` extension** — create them in `frontend/src/lib/models/`
- **Two-layer testing: L1 (model invariants, no render, ~27ms) comes BEFORE L2 (UX contracts, with render)**
- **Backend tasks are pinned to `backend/src/`, frontend tasks to `frontend/src/`** — cross-stack tasks reference both
***[Input Field]**: Placeholder text: "Enter your name...". Validation: Real-time.
***Contract Mapping**:
***`@UX_STATE`**: Enumerate the explicit UI states that must appear later in `contracts/modules.md`
***`@UX_STATE`**: Enumerate the explicit UI states that must appear later in `contracts/modules.md`. For complex screens, these derive from the Screen Model's `@STATE` declarations.
***`@UX_FEEDBACK`**: Define visible system reactions for success, validation, and failure
***`@UX_RECOVERY`**: Define what the user can do after failure or degraded state
***`@UX_REACTIVITY`**: Note the model-driven reactive chain: Model atoms (`$state`, `$derived`) → component props → DOM binding. For route-level data loading, use SvelteKit `load()` functions; `$effect` is for browser-side side effects only.
***Screen Model**: For screens with cross-widget logic, reference the planned `[TYPE Model]` (`.svelte.ts` file in `frontend/src/lib/models/`). The model declares `@STATE`, `@ACTION`, and `@INVARIANT`. Components bind to the model via `@RELATION BINDS_TO -> [ModelId]`.
***States**:
***Idle/Default**: Clean state, waiting for input.
└── git_integration/ # Git‑based version control for dashboards
```
### Plugin Contract
> **v2 update:** Task-based validation was introduced via `ValidationTaskService`, creating persistent validation policies with `ValidationRun` aggregates for grouping per-dashboard results. Each run is tracked as a `ValidationRun` record with aggregate pass/fail/warn counts, replacing the previous single-shot task result pattern. See `ValidationTaskService` in `services/validation_service.py`.
Every plugin MUST provide:
1.**`plugin.toml`** — metadata manifest at the plugin root
@@ -69,6 +69,8 @@ Each existing tool handler is moved from `_dispatch.py:_dispatch_intent()` into
After migration, `_dispatch.py` is reduced to a thin `dispatch()` wrapper. The `_parse_command()` function is updated to also use the registry for keyword-to-operation mapping.
> **Step 8 update (v2):** `run_llm_validation` now creates a persistent validation task (policy) via `POST /api/validation-tasks` (i.e. `ValidationTaskService.create_task()`) and triggers an immediate run via `ValidationTaskService.trigger_run()`, replacing the previous ad-hoc `task_manager.create_task(plugin_id="llm_dashboard_validation")` dispatch. This makes LLM validation repeatable, schedulable, and auditable — each run is tracked as a `ValidationRun` with pass/fail/warn counts.
- **Testable** — `_tool_registry.py` can be unit-tested without DB or HTTP fixtures
- **Discoverable** — `from _tool_registry import _tools` gives a complete inventory of all registered tools
- **Safe-by-default** — a tool without `@assistant_tool` is not registered and cannot be invoked
- **Task-based validation** — `run_llm_validation` now creates persistent validation tasks (policies) via `ValidationTaskService`, making each validation run repeatable, schedulable, and auditable through the `ValidationRun` aggregate model
### Negative
- **One-time migration cost** — 10 existing handlers must be moved to new files (~200 LOC total)
**Goal**: Schema changes for v2 models. Blocks everything.
- [] T001 [P] Add `sources` ORM relationship + `source_snapshot` JSON (audit only, nullable) to `ValidationPolicy` in `backend/src/models/llm.py`. No JSON denormalization column for authoritative state — relational `ValidationSource` table is the single source of truth.
- [] T002 [P] Create `ValidationSource` SQLAlchemy model in `backend/src/models/llm.py` (id, policy_id FK, type, value, parsed_context JSON, is_valid, title, last_error)
- [] T003 [P] Add new columns to `ValidationPolicy` model: `provider_id` (String), `prompt_template` (Text, nullable), `screenshot_enabled` (Boolean, default true), `logs_enabled` (Boolean, default true), `execute_chart_data` (Boolean, default false), `llm_batch_size` (Integer, default 1 — full isolation), `policy_dashboard_concurrency_limit` (Integer, default 3) in `backend/src/models/llm.py`
- [] T003a [P] Add `global_validation_worker_limit` config setting (default: 3) — max concurrent policy runs across all tasks. Stored in app config, not per-policy. in `backend/src/core/config_models.py`
- [] T007 [P] Add `DEFAULT_LLM_PROMPTS["dashboard_validation_prompt_multimodal"]` (Path A, image-focused) and `DEFAULT_LLM_PROMPTS["dashboard_validation_prompt_text"]` (Path B, topology-focused) to `backend/src/services/llm_prompt_templates.py`
- [x] T001 [P] Add `sources` ORM relationship + `source_snapshot` JSON (audit only, nullable) to `ValidationPolicy` in `backend/src/models/llm.py`. No JSON denormalization column for authoritative state — relational `ValidationSource` table is the single source of truth.
- [x] T002 [P] Create `ValidationSource` SQLAlchemy model in `backend/src/models/llm.py` (id, policy_id FK, type, value, parsed_context JSON, status enum, resolved_dashboard_id, title, last_error, last_checked_at)
- [x] T003 [P] Add new columns to `ValidationPolicy` model: `provider_id` (String), `prompt_template` (Text, nullable), `screenshot_enabled` (Boolean, default true), `logs_enabled` (Boolean, default true), `execute_chart_data` (Boolean, default false), `llm_batch_size` (Integer, default 1 — full isolation), `policy_dashboard_concurrency_limit` (Integer, default 3) in `backend/src/models/llm.py`
- [x] T003a [P] Add `global_validation_worker_limit` config setting (default: 3) — max concurrent policy runs across all tasks. Stored in app config in `GLOBAL_VALIDATION_WORKER_LIMIT` field in `backend/src/core/config_models.py`
- [] T012 [P] Implement `ValidationTaskService.delete_task(id)` — deletes policy + cascades to sources + records, in `backend/src/services/validation_service.py`
- [] T013 [P] Implement `ValidationTaskService.trigger_run(policy_id)` — checks FR-054 (same task not already running via ValidationRun with status=running), creates a `ValidationRun` record (FR-055d), enqueues dashboard validation tasks per FR-055 (FIFO with `global_validation_worker_limit`), in `backend/src/services/validation_service.py`
- [] T014 [P] Implement `ValidationTaskService.get_run_history(policy_id, page)` — paginated `ValidationRun` list with aggregate counts, in `backend/src/services/validation_service.py`
- [] T015 [P] Implement `ValidationTaskService.get_run_detail(run_id)` — full run with all dashboard records, aggregate counts, in `backend/src/services/validation_service.py`
- [] T015a [P] Implement `ValidationTaskService._finalize_run(run_id)` — computes aggregate counts from all `ValidationRecord`s with this `run_id`, sets finished_at and status, in `backend/src/services/validation_service.py`
- [x] T008 [P] Implement `ValidationTaskService.create_task()` — creates `ValidationPolicy` + `ValidationSource[]` rows, validates provider_id against execution path, in `backend/src/services/validation_service.py`
- [x] T009 [P] Implement `ValidationTaskService.get_task(id)` — returns policy + sources with current status, in `backend/src/services/validation_service.py`
- [x] T010 [P] Implement `ValidationTaskService.list_tasks(env_id)` — returns all policies with aggregated last-run status, in `backend/src/services/validation_service.py`
- [x] T012 [P] Implement `ValidationTaskService.delete_task(id)` — deletes policy + cascades to sources + records, in `backend/src/services/validation_service.py`
- [x] T013 [P] Implement `ValidationTaskService.trigger_run(policy_id)` — checks FR-054 (same task not already running via ValidationRun with status=running), creates a `ValidationRun` record (FR-055d), enqueues dashboard validation tasks per FR-055, in `backend/src/services/validation_service.py`
- [x] T014 [P] Implement `ValidationTaskService.get_run_history(policy_id, page)` — paginated `ValidationRun` list with aggregate counts, in `backend/src/services/validation_service.py`
- [x] T015 [P] Implement `ValidationTaskService.get_run_detail(run_id)` — full run with all dashboard records, aggregate counts, in `backend/src/services/validation_service.py`
- [x] T015a [P] Implement `ValidationTaskService._finalize_run(run_id)` — computes aggregate counts from all `ValidationRecord`s with this `run_id`, sets finished_at and status, in `backend/src/services/validation_service.py`
### 2.2 DatasetHealthChecker (Path B)
- [] T016 [P] Implement `DatasetHealthChecker` class in `backend/src/plugins/llm_analysis/service.py`:
- [x] T020 [P] Implement `LLMClient._estimate_payload_size()` and `LLMClient._reduce_image_quality()` in `backend/src/plugins/llm_analysis/service.py`:
- [ ] T024c [P] Rename all «molecular-cot» references in logs to «decision audit logging»: REASON/REFLECT markers use structured JSON with deterministic metadata only (no free-form reasoning text, no PII, no screenshot content) per FR-029b. Update docstrings in `backend/src/core/logger.py` and skill files.
- [x] T024b [P] Add retention config to app settings: `LLM_SCREENSHOT_RETENTION_DAYS=30`, `LLM_RAW_RESPONSE_RETENTION_DAYS=30` in `backend/src/core/config_models.py`
- [x] T024c [P] Rename all «molecular-cot» references in logs to «decision audit logging»: REASON/REFLECT markers use structured JSON with deterministic metadata only per FR-029b. Updated`backend/src/core/logger.py`
- [] T023 [P] Add `dashboard_validation_prompt_multimodal` (Path A) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`:
- «Analyze the attached {N} dashboard tab screenshots and execution logs for visual and data health issues...»
- [ ] T024 [P] Add `dashboard_validation_prompt_text` (Path B) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`:
- «Analyze the following dashboard topology, dataset health, and execution logs for data consistency and KXD connectivity issues...»
- [x] T023 [P] Add `dashboard_validation_prompt_multimodal` (Path A) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`
- [x] T024 [P] Add `dashboard_validation_prompt_text` (Path B) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`
- [] T031 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs` — paginated run history (ValidationRun list, not individual records) in `backend/src/api/routes/validation_tasks.py`
- [] T032 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — full run detail: run metadata + all per-dashboard ValidationRecords with dataset_health, screenshots, logs, timings, in `backend/src/api/routes/validation_tasks.py`
- [] T032b Implement legacy redirect: `GET /reports/llm/{taskId}` → 302 to `/validation-tasks/{policy_id}/runs/{run_id}` when `taskId` maps to `ValidationRun.task_id`, in `backend/src/api/routes/validation_tasks.py`
- [x] T026 [US1] Implement `GET /api/validation-tasks` — list all tasks with aggregated last-run status in `backend/src/api/routes/validation_tasks.py`
- [x] T027 [US2] Implement `GET /api/validation-tasks/{policy_id}` — task detail + per-source status in `backend/src/api/routes/validation_tasks.py`
- [x] T031 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs` — paginated run history (ValidationRun list, not individual records) in `backend/src/api/routes/validation_tasks.py`
- [x] T032 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — full run detail: run metadata + all per-dashboard ValidationRecords with dataset_health, screenshots, logs, timings, in `backend/src/api/routes/validation_tasks.py`
- [x] T032b Implement legacy redirect: `GET /reports/llm/{taskId}` → 302 to `/validation-tasks/{policy_id}/runs/{run_id}` when `taskId` maps to `ValidationRun.task_id`, in `backend/src/api/routes/validation_tasks.py`
### 3.2 Task List Page
- [] T035 [US2] Create `frontend/src/routes/validation-tasks/+page.svelte` — task list page with columns: name, schedule, last run status (PASS/WARN/FAIL/UNKNOWN), dashboard count, status (active/inactive), actions (edit/delete/run now)
- [x] T035 [US2] Create `frontend/src/routes/validation-tasks/+page.svelte` — task list page with columns: name, schedule, last run status (PASS/WARN/FAIL/UNKNOWN), dashboard count, status (active/inactive), actions (edit/delete/run now)
- Step 3: Analysis options — provider selector (filtered per FR-050), prompt textarea (pre-filled with path-specific default: multimodal when screenshot_enabled=true, text when false), screenshot toggle, logs toggle, execute_chart_data toggle (Path B only). When toggling screenshot_enabled on a task with custom prompt → show warning «Prompt may no longer match execution path» (FR-048a)
- Collapsible dashboard list: each row = one dashboard, shows status icon + issue count + mini-summary
- Expanded dashboard row: full issues list + execution path badge (Path A/B) + dataset health table (Path B) + screenshot thumbnails with tab selector (Path A) + logs sent to LLM + task execution logs
- Handles mixed runs: some dashboards Path A, some Path B — renders appropriate sections per dashboard
- [ ] T041b Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — returns full ValidationRun with all dashboard records, each containing dataset_health, chart_data_results, tab_screenshots, token_usage, timings. Also resolves policy name, provider name, and dashboard titles for display.
- [x] T040 [US2] Create `frontend/src/routes/validation-tasks/[policyId]/+page.svelte` — task detail page with run history table, per-source breakdown, actions
1. For each dashboard: collect topology via `_build_dashboard_topology()` + dataset health via `DatasetHealthChecker` + logs
2. Default: `llm_batch_size=1` — one LLM call per dashboard (full isolation). Values >1 group dashboards into batches per FR-045a (experimental, documented accuracy risk in research decision 15). Path A dashboards excluded from batching
2. Default: `llm_batch_size=1` — one LLM call per dashboard (full isolation). Values >1 group dashboards into batches per FR-045a
3. For each batch: build structured prompt with per-dashboard labeled sections + strict JSON response schema `{dashboards: [{dashboard_id, status, summary, issues}]}`
- [] T047 [US3] Add test: Path B batch of 5 dashboards, 1 with KXD error → only that dashboard UNKNOWN, other 4 preserved with valid results, in `backend/tests/plugins/llm_analysis/test_path_b_batch.py`
- [] T047a [US3] Add test: batch_size=1 (full isolation) → each dashboard gets individual LLM call with no cross-contamination, in `backend/tests/plugins/llm_analysis/test_path_b_isolated.py`
- [x] T047 [US3] Add test: Path B batch of 5 dashboards, 1 with KXD error → only that dashboard UNKNOWN, other 4 preserved with valid results — in `backend/tests/services/test_path_b_batch.py`
- [x] T047a [US3] Add test: batch_size=1 (full isolation) → each dashboard gets individual LLM call with no cross-contamination — in `backend/tests/services/test_path_b_isolated.py`
**Verification**: Create task with 5 Path B dashboards, `llm_batch_size=1` (default) → 5 individual LLM calls, full isolation. Opt-in `llm_batch_size=3` → 2 LLM calls, structured prompt with per-dashboard isolation. Dashboard with broken dataset → only its record UNKNOWN, others preserved.
**Verification**: Create task with 5 Path B dashboards, `llm_batch_size=1` (default) → 5 individual LLM calls, full isolation. Opt-in `llm_batch_size=3` → 2 LLM calls, structured prompt with per-dashboard isolation. Dashboard with broken dataset → only its record UNKNOWN, others preserved.
2.`ScreenshotService.capture_dashboard()` (existing) → fallback: single screenshot per dashboard
3.Log fetch via `_fetch_dashboard_logs()`
4.`LLMClient._estimate_payload_size()` → check token budget per FR-056
5. Quality reduction loop if needed (FR-056)
6.`LLMClient.analyze_dashboard_multimodal()` → JSON result
7. Fallback to `_execute_path_b()` if payload still exceeds limit
8. Persist `ValidationRecord` with `execution_path="screenshot"`, `tab_screenshots`, `token_usage`
- **Belief-runtime**: `belief_scope("execute_path_a")` + `reason()` before payload reduction + `reflect()` after each reduction + `reason()` on Path B fallback
- [] T049 [US3] Update `frontend/src/routes/validation-tasks/[policyId]/runs/[runId]/+page.svelte` — add Path A sections: tab selector (from `tab_screenshots`), per-tab image viewer with zoom, issues table with tab/chart location. Legacy `/reports/llm/[taskId]` is a redirect-only passthrough — NOT updated.
- [x] T049 [US3] Update `frontend/src/routes/validation-tasks/[policyId]/runs/[runId]/+page.svelte` — add Path A sections: tab selector (from `tab_screenshots`), per-tab image viewer with zoom, issues table with tab/chart location
**Rejected-path regression test**:
- [] T050 [US3] Add test: multi-chunk screenshots exceed token limit → quality reduction triggered → Path B fallback when still exceeded, logged via molecular-cot, in `backend/tests/plugins/llm_analysis/test_payload_reduction.py`
- [x] T050 [US3] Add test: multi-chunk screenshots exceed token limit → quality reduction triggered → Path B fallback when still exceeded — in `backend/tests/services/test_payload_reduction.py`
-`screenshot_enabled=true` → show only multimodal providers (`is_multimodal=true`)
-`screenshot_enabled=false` → show all active providers
- [] T060 [US5] Add `ValidationTaskService.create_task()` validation: reject if `screenshot_enabled=true` but provider `is_multimodal=false`
**Rejected-path regression test**:
- [ ] T061 [US5] Add test: create task with `screenshot_enabled=true` and non-multimodal provider → HTTP 400, in `backend/tests/api/test_validation_tasks_api.py`
- [ ] T061a [US5] Implement `DELETE /api/llm/providers/{id}` enforcement: query active ValidationPolicy references, return 409 with `{error, blocking_tasks: [{id, name}]}` (FR-058), in `backend/src/api/routes/llm.py`
- [ ] T062 [US5] Add test: delete provider with active tasks → HTTP 409 with blocking task list (FR-058), in `backend/tests/api/test_llm_provider_delete.py`
- [ ] T063 [US5] Add test: POST /api/tasks with `plugin_id=llm_dashboard_validation` → HTTP 400 with migration message (FR hard cut), in `backend/tests/api/test_tasks_deprecated_llm.py`
- [x] T059 [US5] Implement provider selector filter logic in `ValidationTaskForm.svelte` (Step 3) — `screenshot_enabled=true` shows only multimodal providers; `screenshot_enabled=false` shows all active providers
- [x] T060 [US5] Add `ValidationTaskService.create_task()` validation: reject if `screenshot_enabled=true` but provider `is_multimodal=false` — implemented via `_validate_provider(provider_id, require_multimodal=screenshot_enabled)` in `backend/src/services/validation_service.py`
- [x] T061 [US5] Add test: create task with `screenshot_enabled=true` and non-multimodal provider → HTTP 400, in `backend/tests/api/test_validation_tasks_api.py`
- [x] T061a [US5] Implement `DELETE /api/llm/providers/{id}` enforcement: query active ValidationPolicy references, return 409 with `{error, blocking_tasks: [{id, name}]}` (FR-058), in `backend/src/api/routes/llm.py`
- [x] T062 [US5] Add test: delete provider with active tasks → HTTP 409 with blocking task list (FR-058), in `backend/tests/api/test_llm_provider_delete.py`
- [x] T063 [US5] Add test: POST /api/tasks with `plugin_id=llm_dashboard_validation` → HTTP 400 with migration message (FR hard cut), in `backend/tests/api/test_tasks_deprecated_llm.py`
**Verification**: LLM settings page shows only 3 bindings. Create task with multimodal provider → works. Create with non-multimodal + screenshot → rejected.
@@ -290,16 +261,16 @@ Phase 1 (Setup)
**Goal**: Remove Validate button + LLM status column. Add lightweight validation indicator per dashboard row + detail page history.
- [] T064 [US6] Remove `handleValidate()` function from `frontend/src/routes/dashboards/+page.svelte`
- [] T065 [US6] Remove LLM validation status column from dashboard hub grid (column header + `getLlmSummaryLabel()` + `getValidationBadgeClass()` rendering) in `frontend/src/routes/dashboards/+page.svelte`
- [] T065b [US6] Add validation indicator dot (PASS/WARN/FAIL/UNKNOWN) next to dashboard title in hub rows. Click opens dropdown with last 3 runs (date, aggregate status across all dashboards in that run, task name) + «View all» link → `/validation-tasks/{policyId}/runs/{runId}`, in `frontend/src/routes/dashboards/+page.svelte`
- [] T066 [US6] Remove «Validate» action from action dropdown menu in `frontend/src/routes/dashboards/+page.svelte`
- [] T067 [US6] Deprecate (comment out)`normalizeValidationStatus()` + `getLlmSummaryLabel()` — replaced by indicator + batch endpoint, in `frontend/src/routes/dashboards/+page.svelte`
- [] T068 [US6] Remove LLM validation status loading from `frontend/src/routes/dashboards/[id]/+page.svelte` (old `runLlmValidationTask()` + `openLlmReport()`)
- [] T068a [US6] Add «Validation History» section to `frontend/src/routes/dashboards/[id]/+page.svelte` — last 10 runs for this dashboard across all tasks, links to canonical run detail pages
- [] T069 [US6] Add «Create Validation Task» button to dashboard hub toolbar that navigates to `/validation-tasks/new?ids=1,2,3` (pre-fills selected dashboard IDs) in `frontend/src/routes/dashboards/+page.svelte`
- [] T070 [US6] Update `frontend/src/lib/components/layout/TaskDrawer.svelte` — ensure LLM validationtasks still show correct status after refactor
- [x] T064 [US6] Remove `handleValidate()` function from `frontend/src/routes/dashboards/+page.svelte`
- [x] T065 [US6] Remove LLM validation status column from dashboard hub grid (column header + `getLlmSummaryLabel()` + `getValidationBadgeClass()` rendering) in `frontend/src/routes/dashboards/+page.svelte`
- [x] T065a [US6] Add `getValidationStatusBatch()` API method in `frontend/src/lib/api.js` — calls `GET /api/dashboards/{env_id}/validation-status?dashboard_ids=...` with graceful 404 fallback
- [x] T065b [US6] Add validation indicator dot (PASS/WARN/FAIL/UNKNOWN) next to dashboard title in hub rows. Click opens popover with last 3 runs + «View all» link, in `frontend/src/routes/dashboards/+page.svelte`
- [x] T066 [US6] Remove «Validate» action from action dropdown menu in `frontend/src/routes/dashboards/+page.svelte`
- [x] T067 [US6] Deprecate old`normalizeValidationStatus()` + `getLlmSummaryLabel()` — replaced by indicator + batch endpoint, in `frontend/src/routes/dashboards/+page.svelte`
- [x] T068 [US6] Remove LLM validation status loading from `frontend/src/routes/dashboards/[id]/+page.svelte` (old `runLlmValidationTask()` + `openLlmReport()`)
- [x] T068a [US6] Add «Validation History» section to `frontend/src/routes/dashboards/[id]/+page.svelte` — last 10 runs for this dashboard, links to canonical run detail pages
- [x] T069 [US6] Add «Create Validation Task» button to dashboard hub toolbar that navigates to `/validation-tasks/new?ids=1,2,3` (pre-fills selected dashboard IDs) in `frontend/src/routes/dashboards/+page.svelte`
- [x] T070 [US6] Update `frontend/src/lib/components/layout/TaskDrawer.svelte` — updated `resolveLlmValidationStatus()` to use ValidationRun aggregate counts; updated `handleOpenLlmReport()` to navigate to canonical `/validation-tasks/{policyId}/runs/{runId}`
**Verification**: Dashboard hub has no Validate button, no LLM column. Colored dot indicator per row with dropdown. Detail page has Validation History section. «Create Validation Task» button pre-fills IDs.
@@ -312,12 +283,12 @@ Phase 1 (Setup)
### 9.1 Migration & Backward Compat
- [ ] T071 Run data migration on test DB: verify all existing `dashboard_ids` → `sources[]` conversion in `backend/tests/migrations/test_v2_data_migration.py`
- [] T072 Update assistant tool `_tool_llm_validation.py` — replace ad-hoc `run_llm_validation` dispatch with task creation via `POST /api/validation-tasks`, in `backend/src/api/routes/assistant/_tool_llm_validation.py`
- [x] T072 Update assistant tool `_tool_llm_validation.py` — replaced ad-hoc `run_llm_validation` dispatch with task creation via `ValidationTaskService.create_task()` + `trigger_run()`, in `backend/src/api/routes/assistant/_tool_llm_validation.py`
### 9.2 ADR Updates
- [] T073 Update `docs/adr/ADR-0008-assistant-tool-registry.md` — note that `_tool_llm_validation.py` now creates tasks instead of ad-hoc validation
- [] T074 Update `docs/adr/ADR-0004-plugin-architecture.md` — mention v2 dual-path + task-based flow as plugin evolution example
- [x] T073 Update `docs/adr/ADR-0008-assistant-tool-registry.md` — noted that `_tool_llm_validation.py` now creates tasks instead of ad-hoc validation; added positive consequence for task-based validation
| **Phase 7+8** | T056–T063 (provider/prompt), T064–T070 (hub) | After Phase 3, both parallel |
| **Phase 9** | T071–T084 | After all phases |
## Total: 98 tasks across 9 phases
## Total: 99 tasks across 9 phases — 86% complete
| Phase | Tasks | User Stories |
|-------|-------|-------------|
| 1 — Setup | 8 | — |
| 2 — Foundational | 23 | — |
| 3 — Task CRUD + Reports | 22 | US1, US2 |
| 4 — Path B | 5 | US3 |
| 5 — Path A | 3 | US3 |
| 6 — URL Parsing | 5 | US4 |
| 7 — Provider/Prompt | 8 | US5 |
| 8 — Hub Cleanup | 10 | US6 |
| 9 — Polish | 14 | All |
| Phase | Tasks | Status |
|-------|-------|--------|
| 1 — Setup | 8 | ✅ 8/8 |
| 2 — Foundational | 23 | ✅ 23/23 |
| 3 — Task CRUD + Reports | 22 | ✅ 22/22 |
| 4 — Path B | 5 | ✅ 5/5 |
| 5 — Path A | 3 | ✅ 3/3 |
| 6 — URL Parsing | 5 | ✅ 5/5 |
| 7 — Provider/Prompt | 8 | ✅ 8/8 |
| 8 — Hub Cleanup | 10 | ✅ 9/10 (T070 done) |
| 9 — Polish | 14 | ⬜ 14/14 pending (T071, T075-T084 — e2e tests, semantic audit, full test suite)
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
