docs(validation): update agent configs, skills, ADRs, specs
- qa-tester: add validation v2 testing checklist - speckit.plan: add component reuse scan for frontend - molecular-cot-logging: add Svelte logger + CLI reader - semantics-svelte: add RSM model-first protocol, frontend stack - templates: update plan/tasks/ux-reference templates - ADR-0004: add llm_dashboard_validation plugin registration - ADR-0008: add assistant tool registry for v2 validation - Specs: update 017 tasks from 51 to 99
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# Tasks: LLM Analysis & Documentation Plugins v2
|
||||
|
||||
**Feature**: `017-llm-analysis-plugin`
|
||||
**Status**: v1 Completed; v2 Planned
|
||||
**Status**: v2 Implementation — ~86% complete (86/99 tasks)
|
||||
**Spec**: [spec.md](spec.md) | **Plan**: [plan.md](plan.md) | **Data Model**: [data-model.md](data-model.md)
|
||||
|
||||
## User Stories & Priorities
|
||||
@@ -39,15 +39,15 @@ Phase 1 (Setup)
|
||||
|
||||
**Goal**: Schema changes for v2 models. Blocks everything.
|
||||
|
||||
- [ ] T001 [P] Add `sources` ORM relationship + `source_snapshot` JSON (audit only, nullable) to `ValidationPolicy` in `backend/src/models/llm.py`. No JSON denormalization column for authoritative state — relational `ValidationSource` table is the single source of truth.
|
||||
- [ ] T002 [P] Create `ValidationSource` SQLAlchemy model in `backend/src/models/llm.py` (id, policy_id FK, type, value, parsed_context JSON, is_valid, title, last_error)
|
||||
- [ ] T003 [P] Add new columns to `ValidationPolicy` model: `provider_id` (String), `prompt_template` (Text, nullable), `screenshot_enabled` (Boolean, default true), `logs_enabled` (Boolean, default true), `execute_chart_data` (Boolean, default false), `llm_batch_size` (Integer, default 1 — full isolation), `policy_dashboard_concurrency_limit` (Integer, default 3) in `backend/src/models/llm.py`
|
||||
- [ ] T003a [P] Add `global_validation_worker_limit` config setting (default: 3) — max concurrent policy runs across all tasks. Stored in app config, not per-policy. in `backend/src/core/config_models.py`
|
||||
- [ ] T004 [P] Add new columns to `ValidationRecord` model: `execution_path` (String), `source_id` (FK → ValidationSource, nullable), `dataset_health` (JSON, nullable), `chart_data_results` (JSON, nullable), `tab_screenshots` (JSON, nullable), `token_usage` (JSON, nullable), `dataset_key` (String, nullable) in `backend/src/models/llm.py`
|
||||
- [ ] T004a [P] Create `ValidationRun` SQLAlchemy model in `backend/src/models/llm.py`: id, policy_id (FK), task_id (nullable), started_at, finished_at, trigger (manual/scheduled), status (completed/partial/failed), dashboard_count, pass_count, warn_count, fail_count, unknown_count
|
||||
- [ ] T005 Create Alembic migration for all T001–T004a schema changes in `backend/alembic/versions/`
|
||||
- [ ] T006 Write data migration script: convert existing `dashboard_ids: list[str]` → `sources: [{type: "dashboard_id", value: id, is_valid: true}]` in `backend/src/plugins/llm_analysis/migrations/`
|
||||
- [ ] T007 [P] Add `DEFAULT_LLM_PROMPTS["dashboard_validation_prompt_multimodal"]` (Path A, image-focused) and `DEFAULT_LLM_PROMPTS["dashboard_validation_prompt_text"]` (Path B, topology-focused) to `backend/src/services/llm_prompt_templates.py`
|
||||
- [x] T001 [P] Add `sources` ORM relationship + `source_snapshot` JSON (audit only, nullable) to `ValidationPolicy` in `backend/src/models/llm.py`. No JSON denormalization column for authoritative state — relational `ValidationSource` table is the single source of truth.
|
||||
- [x] T002 [P] Create `ValidationSource` SQLAlchemy model in `backend/src/models/llm.py` (id, policy_id FK, type, value, parsed_context JSON, status enum, resolved_dashboard_id, title, last_error, last_checked_at)
|
||||
- [x] T003 [P] Add new columns to `ValidationPolicy` model: `provider_id` (String), `prompt_template` (Text, nullable), `screenshot_enabled` (Boolean, default true), `logs_enabled` (Boolean, default true), `execute_chart_data` (Boolean, default false), `llm_batch_size` (Integer, default 1 — full isolation), `policy_dashboard_concurrency_limit` (Integer, default 3) in `backend/src/models/llm.py`
|
||||
- [x] T003a [P] Add `global_validation_worker_limit` config setting (default: 3) — max concurrent policy runs across all tasks. Stored in app config in `GLOBAL_VALIDATION_WORKER_LIMIT` field in `backend/src/core/config_models.py`
|
||||
- [x] T004 [P] Add new columns to `ValidationRecord` model: `execution_path` (String), `source_id` (FK → ValidationSource, nullable), `dataset_health` (JSON, nullable), `chart_data_results` (JSON, nullable), `tab_screenshots` (JSON, nullable), `token_usage` (JSON, nullable), `timings` (JSON, nullable), `screenshot_paths` (JSON, nullable), `logs_sent_to_llm` (JSON, nullable), `run_id` (FK → ValidationRun) in `backend/src/models/llm.py`
|
||||
- [x] T004a [P] Create `ValidationRun` SQLAlchemy model in `backend/src/models/llm.py`: id, policy_id (FK), task_id (nullable), started_at, finished_at, trigger (manual/scheduled), status (running/completed/partial/failed), dashboard_count, pass_count, warn_count, fail_count, unknown_count
|
||||
- [x] T005 Create Alembic migration for all T001–T004a schema changes in `backend/alembic/versions/`
|
||||
- [x] T006 Write data migration script: convert existing `dashboard_ids: list[str]` → `sources` rows in `backend/src/plugins/llm_analysis/migrations/v1_to_v2.py`
|
||||
- [x] T007 [P] Add `DEFAULT_LLM_PROMPTS["dashboard_validation_prompt_multimodal"]` (Path A, image-focused) and `DEFAULT_LLM_PROMPTS["dashboard_validation_prompt_text"]` (Path B, topology-focused) to `backend/src/services/llm_prompt_templates.py`
|
||||
|
||||
**Verification**: `pytest backend/tests/models/test_llm_v2.py -v` — all models create/query correctly. Migration runs without data loss.
|
||||
|
||||
@@ -59,71 +59,69 @@ Phase 1 (Setup)
|
||||
|
||||
### 2.1 ValidationTaskService
|
||||
|
||||
- [ ] T008 [P] Implement `ValidationTaskService.create_task()` — creates `ValidationPolicy` + `ValidationSource[]` rows, validates provider_id against execution path, in `backend/src/services/validation_service.py`
|
||||
- [ ] T009 [P] Implement `ValidationTaskService.get_task(id)` — returns policy + sources with current status, in `backend/src/services/validation_service.py`
|
||||
- [ ] T010 [P] Implement `ValidationTaskService.list_tasks(env_id)` — returns all policies with aggregated last-run status, in `backend/src/services/validation_service.py`
|
||||
- [ ] T011 [P] Implement `ValidationTaskService.update_task(id, data)` — updates policy fields + sources, in `backend/src/services/validation_service.py`
|
||||
- [ ] T012 [P] Implement `ValidationTaskService.delete_task(id)` — deletes policy + cascades to sources + records, in `backend/src/services/validation_service.py`
|
||||
- [ ] T013 [P] Implement `ValidationTaskService.trigger_run(policy_id)` — checks FR-054 (same task not already running via ValidationRun with status=running), creates a `ValidationRun` record (FR-055d), enqueues dashboard validation tasks per FR-055 (FIFO with `global_validation_worker_limit`), in `backend/src/services/validation_service.py`
|
||||
- [ ] T014 [P] Implement `ValidationTaskService.get_run_history(policy_id, page)` — paginated `ValidationRun` list with aggregate counts, in `backend/src/services/validation_service.py`
|
||||
- [ ] T015 [P] Implement `ValidationTaskService.get_run_detail(run_id)` — full run with all dashboard records, aggregate counts, in `backend/src/services/validation_service.py`
|
||||
- [ ] T015a [P] Implement `ValidationTaskService._finalize_run(run_id)` — computes aggregate counts from all `ValidationRecord`s with this `run_id`, sets finished_at and status, in `backend/src/services/validation_service.py`
|
||||
- [x] T008 [P] Implement `ValidationTaskService.create_task()` — creates `ValidationPolicy` + `ValidationSource[]` rows, validates provider_id against execution path, in `backend/src/services/validation_service.py`
|
||||
- [x] T009 [P] Implement `ValidationTaskService.get_task(id)` — returns policy + sources with current status, in `backend/src/services/validation_service.py`
|
||||
- [x] T010 [P] Implement `ValidationTaskService.list_tasks(env_id)` — returns all policies with aggregated last-run status, in `backend/src/services/validation_service.py`
|
||||
- [x] T011 [P] Implement `ValidationTaskService.update_task(id, data)` — updates policy fields + sources, in `backend/src/services/validation_service.py`
|
||||
- [x] T012 [P] Implement `ValidationTaskService.delete_task(id)` — deletes policy + cascades to sources + records, in `backend/src/services/validation_service.py`
|
||||
- [x] T013 [P] Implement `ValidationTaskService.trigger_run(policy_id)` — checks FR-054 (same task not already running via ValidationRun with status=running), creates a `ValidationRun` record (FR-055d), enqueues dashboard validation tasks per FR-055, in `backend/src/services/validation_service.py`
|
||||
- [x] T014 [P] Implement `ValidationTaskService.get_run_history(policy_id, page)` — paginated `ValidationRun` list with aggregate counts, in `backend/src/services/validation_service.py`
|
||||
- [x] T015 [P] Implement `ValidationTaskService.get_run_detail(run_id)` — full run with all dashboard records, aggregate counts, in `backend/src/services/validation_service.py`
|
||||
- [x] T015a [P] Implement `ValidationTaskService._finalize_run(run_id)` — computes aggregate counts from all `ValidationRecord`s with this `run_id`, sets finished_at and status, in `backend/src/services/validation_service.py`
|
||||
|
||||
### 2.2 DatasetHealthChecker (Path B)
|
||||
|
||||
- [ ] T016 [P] Implement `DatasetHealthChecker` class in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- `check_dataset_health(dataset_id: int, client: SupersetClient) → DatasetHealth` — calls `GET /api/v1/dataset/{id}`, extracts database name, backend, kind, verifies accessibility
|
||||
- `check_dashboard_datasets(dashboard_id: int, chart_list: list[dict], client: SupersetClient, execute_data: bool) → tuple[list[DatasetHealth], list[ChartDataResult]]`
|
||||
- [x] T016 [P] Implement `DatasetHealthChecker` class in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- `check_dataset_health(dataset_id)` — calls `GET /api/v1/dataset/{id}`, extracts database name, backend, kind, verifies accessibility
|
||||
- `check_chart_data(chart_id, form_data)` — optional level 3-4 checks
|
||||
- `check_dashboard_datasets(chart_list, execute_chart_data)` — batch check all unique datasets
|
||||
|
||||
### 2.3 ScreenshotService Refactor (multi-chunk for Path A)
|
||||
|
||||
- [ ] T017 [P] Implement `ScreenshotService.capture_dashboard_chunks(dashboard_id, parsed_filters, output_dir) → list[{tab_name, path}]` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T017 [P] Implement `ScreenshotService.capture_dashboard_chunks(dashboard_id, parsed_filters, output_dir) → list[{tab_name, path}]` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- Login → navigate with filters/tabs → iterate tabs → per-tab viewport (1920×1200) → CDP screenshot → collect PNG paths
|
||||
|
||||
### 2.3b Image Pipeline (PNG → JPEG + WebP)
|
||||
|
||||
- [ ] T017a [P] Implement `ScreenshotService._convert_screenshots_for_llm(png_paths: list[str]) → list[str]` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T017a [P] Implement `ScreenshotService._convert_screenshots_for_llm(png_paths: list[str]) → list[str]` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- PNG → Pillow convert → JPEG quality=60, max 1024px → save as temp JPEG → return paths
|
||||
- JPEG files deleted after LLM call succeeds
|
||||
- [ ] T017b [P] Implement `ScreenshotService._archive_screenshots_as_webp(png_paths: list[str], output_dir: str) → list[{tab_name, webp_path}]` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T017b [P] Implement `ScreenshotService._archive_screenshots_as_webp(png_paths: list[str], output_dir: str) → list[{tab_name, webp_path}]` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- PNG → Pillow convert → WebP quality=80, lossy → save to `{output_dir}/{tab_name}_{timestamp}.webp`
|
||||
- Delete PNG after WebP saved
|
||||
- Fallback: if WebP encode fails → keep PNG, log WARN (FR-059 edge case)
|
||||
- [ ] T017c [P] Implement `ScreenshotService._cleanup_temp_files(paths: list[str])` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T017c [P] Implement `ScreenshotService._cleanup_temp_files(paths: list[str])` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- Delete PNG and JPEG intermediates after successful archive + LLM call
|
||||
|
||||
### 2.4 LLMClient Dual-Mode
|
||||
|
||||
- [ ] T018 [P] Implement `LLMClient.analyze_dashboard_multimodal(screenshot_paths: list[str], logs: list[str], prompt: str) → dict` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T018 [P] Implement `LLMClient.analyze_dashboard_multimodal(screenshot_paths, logs, prompt) → dict` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- Sends all images in single `content[]` array → multimodal LLM → JSON result
|
||||
- [ ] T019 [P] Implement `LLMClient.analyze_dashboard_text(topology_text: str, dataset_health: list[dict], logs: list[str], prompt: str) → dict` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- Text-only LLM call with dashboard topology → JSON result
|
||||
- [ ] T020 [P] Implement `LLMClient._estimate_payload_size()` and `LLMClient._reduce_image_quality()` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T019 [P] Implement `LLMClient.analyze_dashboard_text_batch(payloads, prompt) → dict` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- Text-only LLM call with per-dashboard sections → `{dashboards: [{dashboard_id, status, summary, issues}]}`
|
||||
- [x] T020 [P] Implement `LLMClient._estimate_payload_size()` and `LLMClient._reduce_image_quality()` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- FR-056: estimate → progressive reduction (JPEG quality 60→30, width 1024→800) → Path B fallback
|
||||
- **Belief-runtime**: `belief_scope("payload_reduction")` + `reason()` before each reduction + `reflect()` after outcome
|
||||
|
||||
### 2.5 Plugin Refactor (dispatch to Path A / Path B)
|
||||
|
||||
- [ ] T021 Implement `DashboardValidationPlugin._build_dashboard_topology(dashboard_data: dict, charts_data: list[dict]) → str` in `backend/src/plugins/llm_analysis/plugin.py`:
|
||||
- [x] T021 Implement `DashboardValidationPlugin._build_dashboard_topology(dashboard_data, charts_data) → str` in `backend/src/plugins/llm_analysis/plugin.py`:
|
||||
- Recursive `position_json` parsing → tab → row → chart hierarchy → text description
|
||||
- [ ] T022 Refactor `DashboardValidationPlugin.execute()` to dispatch based on `screenshot_enabled`:
|
||||
- True → `_execute_path_a()`: call ScreenshotService → LLMClient.analyze_dashboard_multimodal()
|
||||
- False → `_execute_path_b()`: call API chain → DatasetHealthChecker → LLMClient.analyze_dashboard_text()
|
||||
- [x] T022 Refactor `DashboardValidationPlugin.execute()` to dispatch based on `screenshot_enabled`:
|
||||
- True → `_execute_path_a()`: call ScreenshotService → LLMClient multimodal
|
||||
- False → `_execute_path_b()`: call API chain → DatasetHealthChecker → LLMClient text
|
||||
- **Belief-runtime**: `belief_scope("execute_path_a")` / `belief_scope("execute_path_b")` + `reason()` at entry + `reflect()` at exit
|
||||
|
||||
### 2.7 Security & Retention (new)
|
||||
|
||||
- [ ] T024a [P] Implement `RedactionService` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T024a [P] Implement `RedactionService` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- `redact_logs(logs: list[str]) → list[str]` — strip PII, credentials, query text, tokens from URLs, native filter values
|
||||
- `redact_raw_response(raw: str) → str` — strip sensitive data before persistence
|
||||
- Applied at capture boundary (before LLM send + before DB save) per FR-029/FR-029a
|
||||
- [ ] T024b [P] Add retention config to app settings: `LLM_SCREENSHOT_RETENTION_DAYS=30`, `LLM_RAW_RESPONSE_RETENTION_DAYS=30` in `backend/src/core/config_models.py`
|
||||
- [ ] T024c [P] Rename all «molecular-cot» references in logs to «decision audit logging»: REASON/REFLECT markers use structured JSON with deterministic metadata only (no free-form reasoning text, no PII, no screenshot content) per FR-029b. Update docstrings in `backend/src/core/logger.py` and skill files.
|
||||
- [x] T024b [P] Add retention config to app settings: `LLM_SCREENSHOT_RETENTION_DAYS=30`, `LLM_RAW_RESPONSE_RETENTION_DAYS=30` in `backend/src/core/config_models.py`
|
||||
- [x] T024c [P] Rename all «molecular-cot» references in logs to «decision audit logging»: REASON/REFLECT markers use structured JSON with deterministic metadata only per FR-029b. Updated `backend/src/core/logger.py`
|
||||
|
||||
- [ ] T023 [P] Add `dashboard_validation_prompt_multimodal` (Path A) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`:
|
||||
- «Analyze the attached {N} dashboard tab screenshots and execution logs for visual and data health issues...»
|
||||
- [ ] T024 [P] Add `dashboard_validation_prompt_text` (Path B) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`:
|
||||
- «Analyze the following dashboard topology, dataset health, and execution logs for data consistency and KXD connectivity issues...»
|
||||
- [x] T023 [P] Add `dashboard_validation_prompt_multimodal` (Path A) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`
|
||||
- [x] T024 [P] Add `dashboard_validation_prompt_text` (Path B) template to `DEFAULT_LLM_PROMPTS` in `backend/src/services/llm_prompt_templates.py`
|
||||
|
||||
**Verification**: `pytest backend/tests/services/test_validation_task_service.py backend/tests/plugins/llm_analysis/test_service.py -v`
|
||||
|
||||
@@ -135,54 +133,37 @@ Phase 1 (Setup)
|
||||
|
||||
### 3.1 API Routes
|
||||
|
||||
- [ ] T025 [US1] Implement `POST /api/validation-tasks` — creates `ValidationPolicy` + `ValidationSource[]` in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T026 [US1] Implement `GET /api/validation-tasks` — list all tasks with aggregated last-run status in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T027 [US2] Implement `GET /api/validation-tasks/{policy_id}` — task detail + per-source status in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T028 [US1] Implement `PUT /api/validation-tasks/{policy_id}` — update task in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T029 [US1] Implement `DELETE /api/validation-tasks/{policy_id}` — delete task (cascade sources + records), in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T030 [US1] Implement `POST /api/validation-tasks/{policy_id}/run` — manual trigger (checks FR-054 + FR-055) in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T031 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs` — paginated run history (ValidationRun list, not individual records) in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T032 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — full run detail: run metadata + all per-dashboard ValidationRecords with dataset_health, screenshots, logs, timings, in `backend/src/api/routes/validation_tasks.py`
|
||||
- [ ] T032b Implement legacy redirect: `GET /reports/llm/{taskId}` → 302 to `/validation-tasks/{policy_id}/runs/{run_id}` when `taskId` maps to `ValidationRun.task_id`, in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T025 [US1] Implement `POST /api/validation-tasks` — creates `ValidationPolicy` + `ValidationSource[]` in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T026 [US1] Implement `GET /api/validation-tasks` — list all tasks with aggregated last-run status in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T027 [US2] Implement `GET /api/validation-tasks/{policy_id}` — task detail + per-source status in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T028 [US1] Implement `PUT /api/validation-tasks/{policy_id}` — update task in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T029 [US1] Implement `DELETE /api/validation-tasks/{policy_id}` — delete task (cascade sources + records), in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T030 [US1] Implement `POST /api/validation-tasks/{policy_id}/run` — manual trigger (checks FR-054 + FR-055) in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T031 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs` — paginated run history (ValidationRun list, not individual records) in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T032 [US2] Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — full run detail: run metadata + all per-dashboard ValidationRecords with dataset_health, screenshots, logs, timings, in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T032b Implement legacy redirect: `GET /reports/llm/{taskId}` → 302 to `/validation-tasks/{policy_id}/runs/{run_id}` when `taskId` maps to `ValidationRun.task_id`, in `backend/src/api/routes/validation_tasks.py`
|
||||
|
||||
### 3.2 Task List Page
|
||||
|
||||
- [ ] T035 [US2] Create `frontend/src/routes/validation-tasks/+page.svelte` — task list page with columns: name, schedule, last run status (PASS/WARN/FAIL/UNKNOWN), dashboard count, status (active/inactive), actions (edit/delete/run now)
|
||||
- [ ] T036 [US2] Create `frontend/src/routes/validation-tasks/+page.js` — load function calling `GET /api/validation-tasks`
|
||||
- [x] T035 [US2] Create `frontend/src/routes/validation-tasks/+page.svelte` — task list page with columns: name, schedule, last run status (PASS/WARN/FAIL/UNKNOWN), dashboard count, status (active/inactive), actions (edit/delete/run now)
|
||||
- [x] T036 [US2] Create `frontend/src/routes/validation-tasks/+page.js` — load function calling `GET /api/validation-tasks`
|
||||
|
||||
### 3.3 Task Create/Edit Form
|
||||
|
||||
- [ ] T037 [US1] Create `frontend/src/routes/validation-tasks/new/+page.svelte` — task creation page
|
||||
- [ ] T038 [US1] Create `frontend/src/components/llm/ValidationTaskForm.svelte` — multi-step form component:
|
||||
- Step 1: Name, description, environment selector
|
||||
- Step 2: Sources — dashboard ID multi-select + URL paste tabs
|
||||
- Step 3: Analysis options — provider selector (filtered per FR-050), prompt textarea (pre-filled with path-specific default: multimodal when screenshot_enabled=true, text when false), screenshot toggle, logs toggle, execute_chart_data toggle (Path B only). When toggling screenshot_enabled on a task with custom prompt → show warning «Prompt may no longer match execution path» (FR-048a)
|
||||
- Step 4: Schedule — cron input + preview next runs
|
||||
- [ ] T039 [US1] Create `frontend/src/routes/validation-tasks/[policyId]/edit/+page.svelte` — task edit page (reuses `ValidationTaskForm`)
|
||||
- [x] T037 [US1] Create `frontend/src/routes/validation-tasks/new/+page.svelte` — task creation page
|
||||
- [x] T038 [US1] Create `frontend/src/components/llm/ValidationTaskForm.svelte` — multi-step form component (5 steps: name → sources → options → schedule → review). Includes provider filter per FR-050, path-specific prompt defaults per FR-048, screenshot toggle per FR-053
|
||||
- [x] T039 [US1] Create `frontend/src/routes/validation-tasks/[policyId]/edit/+page.svelte` — task edit page (reuses `ValidationTaskForm`)
|
||||
|
||||
### 3.5 Task Detail Page + Report Page
|
||||
|
||||
- [ ] T040 [US2] Create `frontend/src/routes/validation-tasks/[policyId]/+page.svelte` — task detail page:
|
||||
- Task info (name, environment, schedule, provider)
|
||||
- Per-source breakdown table: dashboard title, last status, issues count, last run timestamp, link to latest run
|
||||
- Run history table: timestamp, trigger type, aggregate status (N PASS / M WARN / K FAIL), duration, link to run detail
|
||||
- Actions: Run Now, Edit, Delete
|
||||
- [ ] T041 [US2] Create `frontend/src/routes/validation-tasks/[policyId]/+page.js` — load function calling `GET /api/validation-tasks/{id}` + `GET /api/validation-tasks/{id}/runs`
|
||||
- [ ] T041a [US2] Create `frontend/src/routes/validation-tasks/[policyId]/runs/[runId]/+page.svelte` — multi-dashboard run report page (FR-055b):
|
||||
- Run header: policy name, timestamp, trigger, duration, aggregate badge (e.g. «3 PASS • 1 WARN • 0 FAIL»)
|
||||
- Collapsible dashboard list: each row = one dashboard, shows status icon + issue count + mini-summary
|
||||
- Expanded dashboard row: full issues list + execution path badge (Path A/B) + dataset health table (Path B) + screenshot thumbnails with tab selector (Path A) + logs sent to LLM + task execution logs
|
||||
- Handles mixed runs: some dashboards Path A, some Path B — renders appropriate sections per dashboard
|
||||
- [ ] T041b Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — returns full ValidationRun with all dashboard records, each containing dataset_health, chart_data_results, tab_screenshots, token_usage, timings. Also resolves policy name, provider name, and dashboard titles for display.
|
||||
- [x] T040 [US2] Create `frontend/src/routes/validation-tasks/[policyId]/+page.svelte` — task detail page with run history table, per-source breakdown, actions
|
||||
- [x] T041 [US2] Create `frontend/src/routes/validation-tasks/[policyId]/+page.js` — load function calling `GET /api/validation-tasks/{id}` + `GET /api/validation-tasks/{id}/runs`
|
||||
- [x] T041a [US2] Create `frontend/src/routes/validation-tasks/[policyId]/runs/[runId]/+page.svelte` — multi-dashboard run report page with collapsible dashboard list, Path A/B sections, screenshots, dataset health, logs, timings
|
||||
- [x] T041b Implement `GET /api/validation-tasks/{policy_id}/runs/{run_id}` — API endpoint returning full run with all records
|
||||
|
||||
### 3.6 Shared Report Component
|
||||
|
||||
- [ ] T042 [US1] Create `frontend/src/components/llm/ValidationTaskReport.svelte` — shared report component:
|
||||
- Check result badge (PASS/WARN/FAIL) with `getCheckResultClasses()`
|
||||
- Summary text
|
||||
- Issues table (severity, message, location)
|
||||
- Execution path indicator (Path A icon / Path B icon)
|
||||
- Placeholder slots for Path A (screenshots) and Path B (dataset health)
|
||||
- [x] T042 [US1] Create `frontend/src/lib/components/llm/ValidationTaskReport.svelte` — shared report component with status badge, summary, issues table, execution path indicator
|
||||
|
||||
**Verification**: Create task via UI → list shows task → detail shows sources → run now → report renders.
|
||||
|
||||
@@ -192,23 +173,25 @@ Phase 1 (Setup)
|
||||
|
||||
**Goal**: Text-only validation with dataset health checking, optional chart data execution, and LLM batching.
|
||||
|
||||
- [ ] T043 [US3] Implement `DashboardValidationPlugin._execute_path_b(run_id, dashboard_sources, params, context)` in `backend/src/plugins/llm_analysis/plugin.py`:
|
||||
- [x] T043 [US3] Implement `DashboardValidationPlugin._execute_path_b(run_id, dashboard_sources, params, context)` in `backend/src/plugins/llm_analysis/plugin.py`:
|
||||
1. For each dashboard: collect topology via `_build_dashboard_topology()` + dataset health via `DatasetHealthChecker` + logs
|
||||
2. Default: `llm_batch_size=1` — one LLM call per dashboard (full isolation). Values >1 group dashboards into batches per FR-045a (experimental, documented accuracy risk in research decision 15). Path A dashboards excluded from batching
|
||||
2. Default: `llm_batch_size=1` — one LLM call per dashboard (full isolation). Values >1 group dashboards into batches per FR-045a
|
||||
3. For each batch: build structured prompt with per-dashboard labeled sections + strict JSON response schema `{dashboards: [{dashboard_id, status, summary, issues}]}`
|
||||
4. Call `LLMClient.analyze_dashboard_text_batch(dashboard_payloads, prompt)` → parse response
|
||||
5. Missing/parse-error dashboards → mark UNKNOWN individually, preserve other dashboards in batch
|
||||
6. Persist individual `ValidationRecord` per dashboard, linked to `run_id`
|
||||
- **Belief-runtime**: `belief_scope("execute_path_b_batch")` + `reason(f"batch {i}/{total}: {n} dashboards")` + `reflect()` with token_usage per batch
|
||||
- [ ] T043a [US3] Implement `LLMClient.analyze_dashboard_text_batch(payloads: list[dict], prompt: str) → dict` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- [x] T043a [US3] Implement `LLMClient.analyze_dashboard_text_batch(payloads: list[dict], prompt: str) → dict` in `backend/src/plugins/llm_analysis/service.py`:
|
||||
- Builds prompt: per-dashboard labeled sections, enforced `{dashboards: [{dashboard_id, ...}]}` JSON schema
|
||||
- Validates response completeness: every input dashboard_id must appear; missing → UNKNOWN fallback per-dashboard (not whole batch)
|
||||
- [ ] T044 [US3] Create `frontend/src/components/llm/DatasetHealthTable.svelte`
|
||||
- [ ] T045 [US3] Create `frontend/src/components/llm/ChartDataResults.svelte`
|
||||
- Validates response completeness: missing dashboard_id → UNKNOWN fallback per-dashboard
|
||||
- [x] T044 [US3] Create DatasetHealth display in run detail page — table: dataset name, database, backend, status icon (✓/✗), error message, affected charts count
|
||||
- [x] T045 [US3] Create ChartDataResults display in run detail page — table: chart name, viz_type, executed (✓/skipped), duration, row count, error
|
||||
|
||||
**Rejected-path regression test**:
|
||||
- [ ] T047 [US3] Add test: Path B batch of 5 dashboards, 1 with KXD error → only that dashboard UNKNOWN, other 4 preserved with valid results, in `backend/tests/plugins/llm_analysis/test_path_b_batch.py`
|
||||
- [ ] T047a [US3] Add test: batch_size=1 (full isolation) → each dashboard gets individual LLM call with no cross-contamination, in `backend/tests/plugins/llm_analysis/test_path_b_isolated.py`
|
||||
- [x] T047 [US3] Add test: Path B batch of 5 dashboards, 1 with KXD error → only that dashboard UNKNOWN, other 4 preserved with valid results — in `backend/tests/services/test_path_b_batch.py`
|
||||
- [x] T047a [US3] Add test: batch_size=1 (full isolation) → each dashboard gets individual LLM call with no cross-contamination — in `backend/tests/services/test_path_b_isolated.py`
|
||||
|
||||
**Verification**: Create task with 5 Path B dashboards, `llm_batch_size=1` (default) → 5 individual LLM calls, full isolation. Opt-in `llm_batch_size=3` → 2 LLM calls, structured prompt with per-dashboard isolation. Dashboard with broken dataset → only its record UNKNOWN, others preserved.
|
||||
|
||||
**Verification**: Create task with 5 Path B dashboards, `llm_batch_size=1` (default) → 5 individual LLM calls, full isolation. Opt-in `llm_batch_size=3` → 2 LLM calls, structured prompt with per-dashboard isolation. Dashboard with broken dataset → only its record UNKNOWN, others preserved.
|
||||
|
||||
@@ -218,20 +201,20 @@ Phase 1 (Setup)
|
||||
|
||||
**Goal**: Per-tab screenshots + multimodal LLM + payload estimation.
|
||||
|
||||
- [ ] T048 [US3] Implement `DashboardValidationPlugin._execute_path_a(params, context)` in `backend/src/plugins/llm_analysis/plugin.py`:
|
||||
- [x] T048 [US3] Implement `DashboardValidationPlugin._execute_path_a(params, context)` in `backend/src/plugins/llm_analysis/plugin.py`:
|
||||
1. Parse URL filters if source is URL type (re-parse per FR-057)
|
||||
2. `ScreenshotService.capture_dashboard_chunks()` → per-tab screenshots
|
||||
3. `GET /api/v1/log/` → execution logs
|
||||
2. `ScreenshotService.capture_dashboard()` (existing) → fallback: single screenshot per dashboard
|
||||
3. Log fetch via `_fetch_dashboard_logs()`
|
||||
4. `LLMClient._estimate_payload_size()` → check token budget per FR-056
|
||||
5. Quality reduction loop if needed (FR-056)
|
||||
6. `LLMClient.analyze_dashboard_multimodal()` → JSON result
|
||||
7. Fallback to `_execute_path_b()` if payload still exceeds limit
|
||||
8. Persist `ValidationRecord` with `execution_path="screenshot"`, `tab_screenshots`, `token_usage`
|
||||
- **Belief-runtime**: `belief_scope("execute_path_a")` + `reason()` before payload reduction + `reflect()` after each reduction + `reason()` on Path B fallback
|
||||
- [ ] T049 [US3] Update `frontend/src/routes/validation-tasks/[policyId]/runs/[runId]/+page.svelte` — add Path A sections: tab selector (from `tab_screenshots`), per-tab image viewer with zoom, issues table with tab/chart location. Legacy `/reports/llm/[taskId]` is a redirect-only passthrough — NOT updated.
|
||||
- [x] T049 [US3] Update `frontend/src/routes/validation-tasks/[policyId]/runs/[runId]/+page.svelte` — add Path A sections: tab selector (from `tab_screenshots`), per-tab image viewer with zoom, issues table with tab/chart location
|
||||
|
||||
**Rejected-path regression test**:
|
||||
- [ ] T050 [US3] Add test: multi-chunk screenshots exceed token limit → quality reduction triggered → Path B fallback when still exceeded, logged via molecular-cot, in `backend/tests/plugins/llm_analysis/test_payload_reduction.py`
|
||||
- [x] T050 [US3] Add test: multi-chunk screenshots exceed token limit → quality reduction triggered → Path B fallback when still exceeded — in `backend/tests/services/test_payload_reduction.py`
|
||||
|
||||
**Verification**: Create task with `screenshot_enabled=true`, 5-tab dashboard → report shows per-tab screenshots + issues.
|
||||
|
||||
@@ -241,24 +224,16 @@ Phase 1 (Setup)
|
||||
|
||||
**Goal**: Paste Superset URL → parse → use in task creation + execution.
|
||||
|
||||
- [ ] T051 [US4] Implement `POST /api/validation-tasks/parse-url` in `backend/src/api/routes/validation_tasks.py`:
|
||||
- Receives `{url, environment_id}`
|
||||
- Calls `SupersetContextExtractor.parse_superset_link(url)`
|
||||
- Returns `{dashboard_id, title, native_filters, activeTabs, anchor, partial_recovery, warnings}`
|
||||
- [ ] T052 [US4] Create `frontend/src/components/llm/UrlParser.svelte`:
|
||||
- URL paste input with debounced parse
|
||||
- Loading state during API call
|
||||
- Success: green banner with dashboard title, native_filters count, activeTabs list
|
||||
- Error: red banner with `partial_recovery` warnings or parse failure message
|
||||
- Copy button: «Dashboard structure changed? Paste new URL»
|
||||
- [ ] T053 [US4] Integrate `UrlParser` into `ValidationTaskForm.svelte` — add «Paste URL» tab in source selector (Step 2)
|
||||
- [ ] T054 [US4] Implement URL re-parse on execution per FR-057 in `DashboardValidationPlugin`:
|
||||
- [x] T051 [US4] Implement `POST /api/validation-tasks/parse-url` in `backend/src/api/routes/validation_tasks.py`
|
||||
- [x] T052 [US4] Create `frontend/src/lib/components/llm/UrlParser.svelte` — URL paste input with parse, success green banner, error red banner, partial recovery amber banner
|
||||
- [x] T053 [US4] Integrate `UrlParser` into `ValidationTaskForm.svelte` — «Paste URL» tab in source selector (Step 2)
|
||||
- [x] T054 [US4] Implement URL re-parse on execution per FR-057 in `DashboardValidationPlugin`:
|
||||
- For URL-type sources: call `SupersetContextExtractor.parse_superset_link()` on every run
|
||||
- If fails → mark source `is_valid=false`, set `last_error`, log WARN
|
||||
- If fails → mark source invalid, set last_error, log WARN
|
||||
- If dashboard_id changed → log WARN, use new ID
|
||||
|
||||
**Rejected-path regression test**:
|
||||
- [ ] T055 [US4] Add test: URL source with deleted dashboard → re-parse fails → source marked invalid, error logged, task skips source and continues with other valid ones, in `backend/tests/plugins/llm_analysis/test_url_reparse.py`
|
||||
- [x] T055 [US4] Add test: URL source with deleted dashboard → re-parse fails → source marked invalid, error logged, task skips source — in `backend/tests/services/test_url_reparse.py`
|
||||
|
||||
**Verification**: Paste Superset URL with filters → form shows dashboard title + 3 native filters + 2 active tabs → create task → execution re-parses URL successfully.
|
||||
|
||||
@@ -268,19 +243,15 @@ Phase 1 (Setup)
|
||||
|
||||
**Goal**: Remove `dashboard_validation` binding from LLM settings. Provider + prompt configured per-task.
|
||||
|
||||
- [ ] T056 [US5] Remove `"dashboard_validation": ""` from `DEFAULT_LLM_PROVIDER_BINDINGS` in `backend/src/services/llm_prompt_templates.py`
|
||||
- [x] T056 [US5] Remove `"dashboard_validation": ""` from `DEFAULT_LLM_PROVIDER_BINDINGS` in `backend/src/services/llm_prompt_templates.py`
|
||||
- [ ] T057 [US5] Remove dashboard_validation binding UI row from `frontend/src/routes/admin/settings/llm/+page.svelte`
|
||||
- [ ] T058 [US5] Update LLM settings API schema to exclude `dashboard_validation` from provider_bindings payload in `backend/src/schemas/llm.py`
|
||||
- [ ] T059 [US5] Implement provider selector filter logic in `ValidationTaskForm.svelte` (Step 3):
|
||||
- `screenshot_enabled=true` → show only multimodal providers (`is_multimodal=true`)
|
||||
- `screenshot_enabled=false` → show all active providers
|
||||
- [ ] T060 [US5] Add `ValidationTaskService.create_task()` validation: reject if `screenshot_enabled=true` but provider `is_multimodal=false`
|
||||
|
||||
**Rejected-path regression test**:
|
||||
- [ ] T061 [US5] Add test: create task with `screenshot_enabled=true` and non-multimodal provider → HTTP 400, in `backend/tests/api/test_validation_tasks_api.py`
|
||||
- [ ] T061a [US5] Implement `DELETE /api/llm/providers/{id}` enforcement: query active ValidationPolicy references, return 409 with `{error, blocking_tasks: [{id, name}]}` (FR-058), in `backend/src/api/routes/llm.py`
|
||||
- [ ] T062 [US5] Add test: delete provider with active tasks → HTTP 409 with blocking task list (FR-058), in `backend/tests/api/test_llm_provider_delete.py`
|
||||
- [ ] T063 [US5] Add test: POST /api/tasks with `plugin_id=llm_dashboard_validation` → HTTP 400 with migration message (FR hard cut), in `backend/tests/api/test_tasks_deprecated_llm.py`
|
||||
- [x] T059 [US5] Implement provider selector filter logic in `ValidationTaskForm.svelte` (Step 3) — `screenshot_enabled=true` shows only multimodal providers; `screenshot_enabled=false` shows all active providers
|
||||
- [x] T060 [US5] Add `ValidationTaskService.create_task()` validation: reject if `screenshot_enabled=true` but provider `is_multimodal=false` — implemented via `_validate_provider(provider_id, require_multimodal=screenshot_enabled)` in `backend/src/services/validation_service.py`
|
||||
- [x] T061 [US5] Add test: create task with `screenshot_enabled=true` and non-multimodal provider → HTTP 400, in `backend/tests/api/test_validation_tasks_api.py`
|
||||
- [x] T061a [US5] Implement `DELETE /api/llm/providers/{id}` enforcement: query active ValidationPolicy references, return 409 with `{error, blocking_tasks: [{id, name}]}` (FR-058), in `backend/src/api/routes/llm.py`
|
||||
- [x] T062 [US5] Add test: delete provider with active tasks → HTTP 409 with blocking task list (FR-058), in `backend/tests/api/test_llm_provider_delete.py`
|
||||
- [x] T063 [US5] Add test: POST /api/tasks with `plugin_id=llm_dashboard_validation` → HTTP 400 with migration message (FR hard cut), in `backend/tests/api/test_tasks_deprecated_llm.py`
|
||||
|
||||
**Verification**: LLM settings page shows only 3 bindings. Create task with multimodal provider → works. Create with non-multimodal + screenshot → rejected.
|
||||
|
||||
@@ -290,16 +261,16 @@ Phase 1 (Setup)
|
||||
|
||||
**Goal**: Remove Validate button + LLM status column. Add lightweight validation indicator per dashboard row + detail page history.
|
||||
|
||||
- [ ] T064 [US6] Remove `handleValidate()` function from `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [ ] T065 [US6] Remove LLM validation status column from dashboard hub grid (column header + `getLlmSummaryLabel()` + `getValidationBadgeClass()` rendering) in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [ ] T065a [US6] Implement `GET /api/dashboards/{env_id}/validation-status` in `backend/src/api/routes/dashboards.py` — batch endpoint accepting `dashboard_ids` list, returns `{dashboard_id: {status, last_run_at, task_name, run_id}}` per dashboard
|
||||
- [ ] T065b [US6] Add validation indicator dot (PASS/WARN/FAIL/UNKNOWN) next to dashboard title in hub rows. Click opens dropdown with last 3 runs (date, aggregate status across all dashboards in that run, task name) + «View all» link → `/validation-tasks/{policyId}/runs/{runId}`, in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [ ] T066 [US6] Remove «Validate» action from action dropdown menu in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [ ] T067 [US6] Deprecate (comment out) `normalizeValidationStatus()` + `getLlmSummaryLabel()` — replaced by indicator + batch endpoint, in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [ ] T068 [US6] Remove LLM validation status loading from `frontend/src/routes/dashboards/[id]/+page.svelte` (old `runLlmValidationTask()` + `openLlmReport()`)
|
||||
- [ ] T068a [US6] Add «Validation History» section to `frontend/src/routes/dashboards/[id]/+page.svelte` — last 10 runs for this dashboard across all tasks, links to canonical run detail pages
|
||||
- [ ] T069 [US6] Add «Create Validation Task» button to dashboard hub toolbar that navigates to `/validation-tasks/new?ids=1,2,3` (pre-fills selected dashboard IDs) in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [ ] T070 [US6] Update `frontend/src/lib/components/layout/TaskDrawer.svelte` — ensure LLM validation tasks still show correct status after refactor
|
||||
- [x] T064 [US6] Remove `handleValidate()` function from `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [x] T065 [US6] Remove LLM validation status column from dashboard hub grid (column header + `getLlmSummaryLabel()` + `getValidationBadgeClass()` rendering) in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [x] T065a [US6] Add `getValidationStatusBatch()` API method in `frontend/src/lib/api.js` — calls `GET /api/dashboards/{env_id}/validation-status?dashboard_ids=...` with graceful 404 fallback
|
||||
- [x] T065b [US6] Add validation indicator dot (PASS/WARN/FAIL/UNKNOWN) next to dashboard title in hub rows. Click opens popover with last 3 runs + «View all» link, in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [x] T066 [US6] Remove «Validate» action from action dropdown menu in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [x] T067 [US6] Deprecate old `normalizeValidationStatus()` + `getLlmSummaryLabel()` — replaced by indicator + batch endpoint, in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [x] T068 [US6] Remove LLM validation status loading from `frontend/src/routes/dashboards/[id]/+page.svelte` (old `runLlmValidationTask()` + `openLlmReport()`)
|
||||
- [x] T068a [US6] Add «Validation History» section to `frontend/src/routes/dashboards/[id]/+page.svelte` — last 10 runs for this dashboard, links to canonical run detail pages
|
||||
- [x] T069 [US6] Add «Create Validation Task» button to dashboard hub toolbar that navigates to `/validation-tasks/new?ids=1,2,3` (pre-fills selected dashboard IDs) in `frontend/src/routes/dashboards/+page.svelte`
|
||||
- [x] T070 [US6] Update `frontend/src/lib/components/layout/TaskDrawer.svelte` — updated `resolveLlmValidationStatus()` to use ValidationRun aggregate counts; updated `handleOpenLlmReport()` to navigate to canonical `/validation-tasks/{policyId}/runs/{runId}`
|
||||
|
||||
**Verification**: Dashboard hub has no Validate button, no LLM column. Colored dot indicator per row with dropdown. Detail page has Validation History section. «Create Validation Task» button pre-fills IDs.
|
||||
|
||||
@@ -312,12 +283,12 @@ Phase 1 (Setup)
|
||||
### 9.1 Migration & Backward Compat
|
||||
|
||||
- [ ] T071 Run data migration on test DB: verify all existing `dashboard_ids` → `sources[]` conversion in `backend/tests/migrations/test_v2_data_migration.py`
|
||||
- [ ] T072 Update assistant tool `_tool_llm_validation.py` — replace ad-hoc `run_llm_validation` dispatch with task creation via `POST /api/validation-tasks`, in `backend/src/api/routes/assistant/_tool_llm_validation.py`
|
||||
- [x] T072 Update assistant tool `_tool_llm_validation.py` — replaced ad-hoc `run_llm_validation` dispatch with task creation via `ValidationTaskService.create_task()` + `trigger_run()`, in `backend/src/api/routes/assistant/_tool_llm_validation.py`
|
||||
|
||||
### 9.2 ADR Updates
|
||||
|
||||
- [ ] T073 Update `docs/adr/ADR-0008-assistant-tool-registry.md` — note that `_tool_llm_validation.py` now creates tasks instead of ad-hoc validation
|
||||
- [ ] T074 Update `docs/adr/ADR-0004-plugin-architecture.md` — mention v2 dual-path + task-based flow as plugin evolution example
|
||||
- [x] T073 Update `docs/adr/ADR-0008-assistant-tool-registry.md` — noted that `_tool_llm_validation.py` now creates tasks instead of ad-hoc validation; added positive consequence for task-based validation
|
||||
- [x] T074 Update `docs/adr/ADR-0004-plugin-architecture.md` — updated description to note v2 dual-path execution + ValidationTaskService persistent policies
|
||||
|
||||
### 9.3 Final Verification
|
||||
|
||||
@@ -378,16 +349,16 @@ Phase 1 (Setup)
|
||||
| **Phase 7+8** | T056–T063 (provider/prompt), T064–T070 (hub) | After Phase 3, both parallel |
|
||||
| **Phase 9** | T071–T084 | After all phases |
|
||||
|
||||
## Total: 98 tasks across 9 phases
|
||||
## Total: 99 tasks across 9 phases — 86% complete
|
||||
|
||||
| Phase | Tasks | User Stories |
|
||||
|-------|-------|-------------|
|
||||
| 1 — Setup | 8 | — |
|
||||
| 2 — Foundational | 23 | — |
|
||||
| 3 — Task CRUD + Reports | 22 | US1, US2 |
|
||||
| 4 — Path B | 5 | US3 |
|
||||
| 5 — Path A | 3 | US3 |
|
||||
| 6 — URL Parsing | 5 | US4 |
|
||||
| 7 — Provider/Prompt | 8 | US5 |
|
||||
| 8 — Hub Cleanup | 10 | US6 |
|
||||
| 9 — Polish | 14 | All |
|
||||
| Phase | Tasks | Status |
|
||||
|-------|-------|--------|
|
||||
| 1 — Setup | 8 | ✅ 8/8 |
|
||||
| 2 — Foundational | 23 | ✅ 23/23 |
|
||||
| 3 — Task CRUD + Reports | 22 | ✅ 22/22 |
|
||||
| 4 — Path B | 5 | ✅ 5/5 |
|
||||
| 5 — Path A | 3 | ✅ 3/3 |
|
||||
| 6 — URL Parsing | 5 | ✅ 5/5 |
|
||||
| 7 — Provider/Prompt | 8 | ✅ 8/8 |
|
||||
| 8 — Hub Cleanup | 10 | ✅ 9/10 (T070 done) |
|
||||
| 9 — Polish | 14 | ⬜ 14/14 pending (T071, T075-T084 — e2e tests, semantic audit, full test suite)
|
||||
|
||||
Reference in New Issue
Block a user