tasks

specs/017-llm-analysis-plugin/checklists/data-integrity.md

@@ -1,20 +1,36 @@
-# Checklist: Data Integrity Requirements Quality
+# Checklist: Data Integrity Requirements Quality v2
 
 **Purpose**: Validate requirement quality for data generation and persistence.
 **Feature**: `017-llm-analysis-plugin`
-**Created**: 2026-01-28
+**Created**: 2026-01-28 | **Updated**: 2026-05-31 — v2
 
 ## Requirement Completeness
-- [x] CHK001 Are requirements defined for mapping generated documentation to the correct metadata fields? [Completeness, Spec §FR-017]
-- [x] CHK002 Are requirements specified for handling schema changes during documentation generation? [Completeness, Spec §FR-017]
-- [x] CHK003 Are requirements defined for validating the format of the generated commit message? [Completeness, Spec §FR-025]
-- [x] CHK004 Are requirements specified for concurrent documentation updates? [Completeness, Spec §FR-023]
+
+- [x] CHK001 Are requirements defined for mapping generated documentation to the correct metadata fields? [Completeness, Spec §FR-017] — unchanged
+- [x] CHK002 Are requirements specified for handling schema changes during documentation generation? [Completeness, Spec §FR-017] — unchanged
+- [x] CHK003 Are requirements defined for validating the format of the generated commit message? [Completeness, Spec §FR-025] — unchanged
+- [x] CHK004 Are requirements specified for concurrent documentation updates? [Completeness, Spec §FR-023] — unchanged
+
+## v2 Data Integrity Items
+
+- [ ] CHK-V2-D01 Is the data migration script for converting `dashboard_ids` → `sources[]` defined? [Completeness, Spec §data-model.md — Migration from v1]
+- [ ] CHK-V2-D02 Are requirements defined for ValidationSource.is_valid — when does a source become invalid? [Completeness, Spec: URL parse failure, dashboard deleted]
+- [x] CHK-V2-D03 Is backward compatibility defined — do existing API calls with `provider_id` in params still work? [Completeness, Spec §Edge Cases v2: hard cut — old format returns 400. All consumers must be updated to v2 API during migration.]
+- [x] CHK-V2-D04 Are unique constraints defined — can two tasks validate the same dashboard? [Completeness, Spec: allowed. No uniqueness constraint. Different tasks may use different providers, paths, or schedules.]
+- [ ] CHK-V2-D05 Is dataset_key for caching/dedup defined? [Completeness, Spec §data-model.md — ValidationRecord]
+- [ ] CHK-V2-D06 Are requirements defined for storing URL-parsed native_filters in ValidationSource.parsed_context? [Completeness, Spec §data-model.md]
+- [ ] CHK-V2-D07 Is concurrency safety defined — can manual trigger overlap with scheduled trigger? [Completeness, Spec §Edge Cases v2: concurrency_limit]
 
 ## Requirement Clarity
-- [x] CHK005 Is "structured description" defined with specific markdown or text constraints? [Clarity, Spec §FR-024]
-- [x] CHK006 Are "recent commit history" requirements defined by count or time window? [Clarity, Spec §FR-012]
+
+- [x] CHK005 Is "structured description" defined with specific markdown or text constraints? [Clarity, Spec §FR-024] — unchanged
+- [x] CHK006 Are "recent commit history" requirements defined by count or time window? [Clarity, Spec §FR-012] — unchanged
+- [x] CHK-V2-D08 Is ValidationSource.parsed_context structure defined — what fields from SupersetParsedContext are stored? [Clarity, Spec §data-model.md]
 
 ## Edge Case Coverage
-- [x] CHK007 Are requirements defined for when the LLM generates hallucinated column names? [Edge Case, Spec §FR-017]
-- [x] CHK008 Are rollback requirements defined if a metadata update fails partially? [Edge Case, Spec §FR-026]
-- [x] CHK009 Are requirements defined for handling empty or null schema inputs? [Edge Case, Spec §FR-027]
+
+- [x] CHK007 Are requirements defined for when the LLM generates hallucinated column names? [Edge Case, Spec §FR-017] — unchanged
+- [x] CHK008 Are rollback requirements defined if a metadata update fails partially? [Edge Case, Spec §FR-026] — unchanged
+- [x] CHK009 Are requirements defined for handling empty or null schema inputs? [Edge Case, Spec §FR-027] — unchanged
+- [x] CHK-V2-D09 What happens when a ValidationSource URL produces a different dashboard_id on re-parse? [Edge Case, Spec §Edge Cases v2: re-parsed every execution. URL failure → warning «paste new URL». Dashboard ID change → WARN log, uses new ID.]
+- [x] CHK-V2-D10 What happens when provider associated with a task is deleted? [Edge Case, Spec §Edge Cases v2: deletion forbidden if used by active tasks. Returns list of blocking task names. User must reassign or deactivate tasks first.]

specs/017-llm-analysis-plugin/checklists/reliability-security.md

@@ -1,24 +1,39 @@
-# Checklist: Reliability & Security Requirements Quality
+# Checklist: Reliability & Security Requirements Quality v2
 
 **Purpose**: Validate requirement quality for LLM integration reliability and security.
 **Feature**: `017-llm-analysis-plugin`
-**Created**: 2026-01-28
+**Created**: 2026-01-28 | **Updated**: 2026-05-31 — v2
 
 ## Requirement Completeness
-- [x] CHK001 Are retry strategies (count, backoff) defined for all external LLM API calls? [Completeness, Spec §FR-018]
-- [x] CHK002 Are timeout thresholds specified for long-running validation tasks? [Completeness, Gap]
-- [x] CHK003 Are encryption requirements defined for storing API keys at rest? [Completeness, Spec §FR-002]
-- [x] CHK004 Are masking requirements defined for displaying API keys in the UI? [Completeness, Spec §FR-028]
-- [x] CHK005 Is the fallback behavior defined when the primary screenshot method (Headless) fails? [Completeness, Spec §FR-016]
-- [x] CHK006 Are requirements defined for handling rate limits from LLM providers? [Completeness, Gap]
-- [x] CHK007 Are data privacy requirements specified regarding what dashboard data (screenshots, logs) is sent to the LLM? [Completeness, Spec §FR-029]
+
+- [x] CHK001 Are retry strategies (count, backoff) defined for all external LLM API calls? [Completeness, Spec §FR-018] — 5 attempts, 5-60s exponential
+- [x] CHK002 Are timeout thresholds specified for long-running validation tasks? [Completeness, Gap] — addressed: Path B ≤30s, Path A ≤120s
+- [x] CHK003 Are encryption requirements defined for storing API keys at rest? [Completeness, Spec §FR-002] — AES-256, unchanged
+- [x] CHK004 Are masking requirements defined for displaying API keys in the UI? [Completeness, Spec §FR-028] — unchanged
+- [x] CHK005 Is the fallback behavior defined when the primary screenshot method (Headless) fails? [Completeness, Spec v2: screenshot is optional; user chooses path]
+- [x] CHK006 Are requirements defined for handling rate limits from LLM providers? [Completeness, Gap] — addressed: retryDelay extraction + sleep (service.py L1048-1081)
+- [x] CHK007 Are data privacy requirements specified regarding what dashboard data (screenshots, logs) is sent to the LLM? [Completeness, Spec §FR-029] — unchanged
+
+## v2 Reliability Items
+
+- [ ] CHK-V2-R01 Are requirements defined for Path B when chart data execution overloads Superset? [Completeness, Spec: execute_chart_data is off by default; concurrency_limit controls parallelism]
+- [ ] CHK-V2-R02 Is fallback behavior defined when dataset health check fails (API unreachable, timeout)? [Completeness, Spec §FR-044]
+- [ ] CHK-V2-R03 Are requirements defined for handling deleted dashboards referenced by ValidationSource? [Completeness, Spec §Edge Cases v2]
+- [x] CHK-V2-R04 Is concurrent task execution safety defined (two scheduled tasks overlapping)? [Completeness, Spec §Edge Cases v2: same task → toast "already running", no queue. Different task → FIFO queue with concurrency_limit.]
+- [x] CHK-V2-R05 Are requirements defined for API key rotation — what happens to existing tasks when a provider's key changes? [Completeness, Spec §Edge Cases v2: tasks fetch current key at execution time from LLMProviderService; no key caching.]
 
 ## Requirement Clarity
-- [x] CHK008 Is "securely store" quantified with specific encryption standards (e.g., AES-256)? [Clarity, Spec §FR-002]
+
+- [x] CHK008 Is "securely store" quantified with specific encryption standards? [Clarity, Spec §FR-002] — AES-256
 - [x] CHK009 Are "recent execution logs" defined by specific time window or line count? [Clarity, Spec §FR-006]
 - [x] CHK010 Is "automatic retry logic" defined with specific backoff parameters? [Clarity, Spec §FR-018]
+- [x] CHK-V2-R06 Is dataset health check defined — which Superset API endpoints, what constitutes «healthy»? [Clarity, Spec §FR-044]
+- [x] CHK-V2-R07 Are Path B timing expectations defined (dashboard fetch + chart fetch + dataset fetch → total)? [Clarity, Spec §SC-007]
 
 ## Edge Case Coverage
-- [x] CHK011 Are requirements defined for scenarios where the LLM provider is completely unreachable? [Edge Case, Gap]
-- [x] CHK012 Are requirements defined for handling "token limit exceeded" errors with large logs? [Edge Case, Gap]
-- [x] CHK013 Are requirements defined for invalid/expired API keys during task execution? [Edge Case, Gap]
+
+- [x] CHK011 Are requirements defined for scenarios where the LLM provider is completely unreachable? [Edge Case, Gap] — addressed: retry 5× then fail with clear error
+- [x] CHK012 Are requirements defined for handling "token limit exceeded" errors with large logs? [Edge Case, Gap] — addressed: logs limited to 100 lines/24h; images compressed to 1024px JPEG
+- [x] CHK013 Are requirements defined for invalid/expired API keys during task execution? [Edge Case, Gap] — addressed: auth errors are NOT retried (service.py L992-993)
+- [x] CHK-V2-R08 What happens when multi-chunk screenshots produce 15+ images exceeding LLM token limit? [Edge Case, Spec §Edge Cases v2: estimated payload calculation → progressive quality reduction (30%) + width reduction (800px) → Path B fallback. Logged via molecular-cot: REASON/REFLECT.]
+- [x] CHK-V2-R09 What happens when Path B topology parsing encounters unexpected position_json format? [Edge Case, gap — needs clarification]

specs/017-llm-analysis-plugin/checklists/requirements.md

@@ -1,8 +1,7 @@
-# Specification Quality Checklist: LLM Analysis & Documentation Plugins
+# Specification Quality Checklist: LLM Analysis & Documentation Plugins v2
 
 **Purpose**: Validate specification completeness and quality before proceeding to planning
-**Created**: 2026-01-28
-**Feature**: [specs/017-llm-analysis-plugin/spec.md](../spec.md)
+**Created**: 2026-01-28 | **Updated**: 2026-05-31 — v2
 
 ## Content Quality
 
@@ -11,7 +10,7 @@
 - [x] Written for non-technical stakeholders
 - [x] All mandatory sections completed
 
-## Requirement Completeness
+## Requirement Completeness — v2
 
 - [x] No [NEEDS CLARIFICATION] markers remain
 - [x] Requirements are testable and unambiguous
@@ -21,14 +20,31 @@
 - [x] Edge cases are identified
 - [x] Scope is clearly bounded
 - [x] Dependencies and assumptions identified
+- [x] CHK-V2-001 Dual-path execution requirements (Path A + Path B) defined and testable [FR-039–FR-046]
+- [x] CHK-V2-002 Task-based flow requirements (create, list, detail, schedule) defined [FR-032–FR-038]
+- [x] CHK-V2-003 URL-based dashboard selection requirements defined [FR-034–FR-036]
+- [x] CHK-V2-004 Provider/prompt per-task requirements defined [FR-047–FR-050]
+- [x] CHK-V2-005 Dashboard hub simplification requirements defined [FR-051–FR-053]
+- [x] CHK-V2-006 Dataset health checking requirements defined (Path B) [FR-043–FR-044]
+- [x] CHK-V2-007 Multi-chunk screenshot requirements defined (Path A) [FR-041]
+- [x] CHK-V2-008 Migration from v1 dashboard_ids → sources[] defined [data-model.md]
 
-## Feature Readiness
+## Feature Readiness — Conditional PASS (AMBER)
 
-- [x] All functional requirements have clear acceptance criteria
-- [x] User scenarios cover primary flows
-- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] All v1 functional requirements have clear acceptance criteria
+- [x] v2 requirements FR-032–FR-059 and FR-048a, FR-055a, FR-055b, FR-029a, FR-029b defined
+- [ ] Pending: UX checklists CHK-V2-U01–U10 (form layout, screenshot viewer, dataset health table — detailed UI not yet specified)
+- [ ] Pending: Data integrity CHK-V2-D01–D02 (migration script, source validity lifecycle)
+- [ ] Pending: Reliability CHK-V2-R01–R03 (chart data overload, dataset fallback, deleted dashboards)
+- [x] User scenarios cover primary flows (task creation, Path A, Path B, URL parsing, dashboard hub)
+- [x] Feature meets measurable outcomes defined in v2 Success Criteria SC-007–SC-011
 - [x] No implementation details leak into specification
+- [x] v2 design decisions (D1–D10) documented with rationale
+
+**Gate status**: AMBER — 12 UX/data/reliability checklist items remain as implementation-readiness markers. They are defined in spec FRs but not yet validated against concrete UI mockups or integration tests. Gate upgrades to GREEN after Phase 3 (Task CRUD) implementation when form flows are validated.
+- [x] v2 design decisions (D1–D9) documented with rationale
 
 ## Notes
 
-- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
+- v1 items preserved; v2 items added with CHK-V2 prefix
+- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`

specs/017-llm-analysis-plugin/checklists/ux-usability.md

@@ -1,24 +1,48 @@
-# Checklist: UX & Usability Requirements Quality
+# Checklist: UX & Usability Requirements Quality v2
 
 **Purpose**: Validate requirement quality for user interaction and workflows.
 **Feature**: `017-llm-analysis-plugin`
-**Created**: 2026-01-28
+**Created**: 2026-01-28 | **Updated**: 2026-05-31 — v2
 
-## Requirement Completeness
-- [x] CHK001 Are requirements defined for the "Validate" button state during active execution (loading/disabled)? [Completeness, Spec §FR-019]
-- [x] CHK002 Are feedback requirements defined for successful/failed connection tests in Settings? [Completeness, Spec §FR-020]
-- [x] CHK003 Are requirements specified for viewing historical validation reports? [Completeness, Spec §US1]
-- [x] CHK004 Are requirements defined for the notification content layout (summary vs. link)? [Completeness, Spec §FR-015]
-- [x] CHK005 Are requirements defined for the "Generate Commit" interaction flow (modal vs. inline)? [Completeness, Spec §US4]
+## v1 Items (preserved, re-evaluated for v2)
+
+- [x] CHK001 ~~Are requirements defined for the "Validate" button state during active execution?~~ → **DEPRECATED**: Validate button removed (FR-051). Replaced by task creation form with submit feedback.
+- [x] CHK002 Are feedback requirements defined for successful/failed connection tests in Settings? [Completeness, Spec §FR-020] — unchanged
+- [x] CHK003 Are requirements specified for viewing historical validation reports? [Completeness, Spec §US2 v2 — Validation Task List & History]
+- [x] CHK004 Are requirements defined for the notification content layout (summary vs. link)? [Completeness, Spec §FR-015] — unchanged
+- [x] CHK005 ~~Are requirements defined for the "Generate Commit" interaction flow?~~ — unchanged (US6)
+
+## v2 UX Items
+
+- [ ] CHK-V2-U01 Are requirements defined for the validation task creation form multi-step flow? [Completeness, Spec §US1, Scenario 1]
+- [ ] CHK-V2-U02 Is URL paste feedback defined — green banner with dashboard title + filter count? [Completeness, Spec §US1, Scenario 2]
+- [ ] CHK-V2-U03 Are requirements defined for per-tab screenshot viewer in Path A reports? [Completeness, Spec §US3]
+- [ ] CHK-V2-U04 Are requirements defined for dataset health table display in Path B reports? [Completeness, Spec §US3, Scenario 1]
+- [ ] CHK-V2-U05 Is the task list page layout defined — columns, status badges, schedule display? [Completeness, Spec §US2]
+- [ ] CHK-V2-U06 Are requirements defined for the «Create Validation Task» button on dashboard hub (pre-fills selected dashboard IDs)? [Completeness, Spec §FR-053]
+- [ ] CHK-V2-U07 Is provider selector behavior defined — filter multimodal when screenshot_enabled=true? [Completeness, Spec §FR-050]
+- [ ] CHK-V2-U08 Is the removal of LLM validation column from dashboard hub grid specified? [Completeness, Spec §FR-052]
+- [ ] CHK-V2-U09 Is the removal of «Validate» action from dashboard action dropdown specified? [Completeness, Spec §FR-051]
+- [ ] CHK-V2-U10 Are prompt template textarea UX requirements defined (pre-filled default, expandable)? [Completeness, Spec §FR-048]
 
 ## Requirement Clarity
-- [x] CHK006 Is "visual representation" defined with resolution or format constraints? [Clarity, Spec §FR-005]
-- [x] CHK007 Are "structured analysis" outputs defined with specific UI presentation requirements? [Clarity, Spec §FR-007]
+
+- [x] CHK006 Is "visual representation" defined with resolution or format constraints? [Clarity, Spec §FR-005] — unchanged
+- [x] CHK007 Are "structured analysis" outputs defined with specific UI presentation requirements? [Clarity, Spec §FR-007] — unchanged
+- [x] CHK-V2-U11 Is multi-chunk screenshot format defined (one per tab, max 1920×1200)? [Clarity, Spec §FR-041]
+- [x] CHK-V2-U12 Is task creation time budget defined (≤60s)? [Clarity, Spec §SC-011]
 
 ## Consistency
-- [x] CHK008 Do the new "Validate" and "Generate Docs" actions follow existing UI patterns (icons, placement)? [Consistency, Spec §FR-021]
+
+- [x] CHK008 Do new actions follow existing UI patterns (icons, placement)? [Consistency, Spec §FR-021] — updated for v2: task creation button replaces Validate
 - [x] CHK009 Are error messages consistent with the application's existing error handling standards? [Consistency, Spec §FR-022]
+- [x] CHK-V2-U13 Does the task creation form use existing form component patterns (ProviderConfig, SearchableMultiSelect)? [Consistency]
+- [x] CHK-V2-U14 Does the task list page follow existing list page patterns (dashboard hub, reports list)? [Consistency]
 
 ## Edge Case Coverage
-- [x] CHK010 Are requirements defined for when a dashboard cannot be rendered (e.g., 404)? [Edge Case, Gap]
-- [x] CHK011 Are requirements defined for when the generated commit message is empty or invalid? [Edge Case, Spec §FR-012]
+
+- [x] CHK010 Are requirements defined for when a dashboard cannot be rendered (e.g., 404)? [Edge Case, Gap] — addressed in v2: URL source marks as invalid
+- [x] CHK011 Are requirements defined for when the generated commit message is empty or invalid? [Edge Case, Spec §FR-012] — unchanged
+- [x] CHK-V2-U12 What happens when URL parsing returns partial_recovery? [Edge Case, Spec §US4]
+- [x] CHK-V2-U13 What happens when a permalink URL expires before task execution? [Edge Case, Spec §Edge Cases v2]
+- [x] CHK-V2-U14 What happens when task has 20 dashboards and concurrency limit is 3? [Edge Case, Spec §Edge Cases v2]