#region DashboardScenarioModel.Tasks [C:3] [TYPE ADR] [SEMANTICS tasks,scenario,implementation] @BRIEF Ordered TDD backlog for deterministic scenario graph, safe draft-pack compilation, capture/VLM/disposition, and full verification gates. **Prerequisites**: plan.md, spec.md (required); contracts/modules.md, contracts/openapi.yaml, traceability.md, prototype/manifest.md (present). **Tests**: Write tests FIRST (fail before implementation) for every C3+ contract per constitution VII. Frontend tasks are N/A — 038 is DTO-only; UI rendering is owned by 039. ## Format: `- [ ] T### [P] [USx] Description with exact file path` ## Phase 1 — Setup (Shared Infrastructure) - [ ] T001 Transcribe checklist-catalog.md into a versioned declarative resource in `backend/src/services/dashboard_testing/scenario/catalog_v1.yaml` - [ ] T002 [P] Write catalog completeness tests for B01–B09, C01–C07, T01–T03 in `backend/tests/services/dashboard_testing/scenario/test_catalog.py` - [ ] T003 [P] Create valid 18-step scenario plus invalid cycle/missing-ref/duplicate-output/raw-baseline/SQL fixtures under `specs/038-dashboard-scenario-model/fixtures/` - [ ] T004 [P] Materialize fixtures from `specs/038-dashboard-scenario-model/fixtures/` into `backend/tests/fixtures/dashboard_scenarios/` (copy as-is, no adaptation) - [ ] T005 Implement Pydantic models in `backend/src/services/dashboard_testing/scenario/models.py` matching `contracts/dashboard-test-scenario.schema.json` **Checkpoint**: Catalog loads 19 cases; schema round-trips against golden fixtures. ## Phase 2 — US1 Compile Scenario Graph - [ ] T006 [US1] Write failing capability mapping tests in `backend/tests/services/dashboard_testing/scenario/test_capability_mapper.py` - [ ] T007 [US1] Implement `backend/src/services/dashboard_testing/scenario/checklist_catalog.py` validation and `backend/src/services/dashboard_testing/scenario/capability_mapper.py` with complete classification - [ ] T008 [US1] Cover unavailable XLSX, missing selector/test data, unsafe mutation context, cross-dashboard absence, and T01–T03 no-SQL fallbacks @TEST_EDGE: xlsx_unavailable→manual/unsupported, technical_without_dataset_fields→human checkpoint (no SQL) - [ ] T009 [US1] Write failing deterministic compiler tests in `backend/tests/services/dashboard_testing/scenario/test_compiler.py` - [ ] T010 [US1] Implement registered tool/action and step-template catalogs under `backend/src/services/dashboard_testing/scenario/templates/` - [ ] T011 [US1] Implement `backend/src/services/dashboard_testing/scenario/compiler.py` with stable ids, phase order, refs, coverage, and fingerprints @PRE: intent, query model, catalog, baseline summary, parameters have valid fingerprints @POST: same canonical inputs/compiler version yield byte-identical graph and stable ids/order @DATA_CONTRACT: CompileScenarioRequest → DashboardTestScenario @TEST_EDGE: missing_selector→NEEDS_SELECTOR + save blocker, missing_baseline→NEEDS_BASELINE (no embedded numeric truth) - [ ] T012 [US1] Prove repeated compile and shuffled input order produce identical graph bytes @INVARIANT: Deterministic_Graph → VERIFIED_BY: repeated_compile, shuffled_input_order **Checkpoint**: Valid fixture compiles to stable graph and classifies all 19 cases. ## Phase 3 — US2 Validate Safety and Completeness - [ ] T013 [US2] Write failing full invalid-fixture matrix in `backend/tests/services/dashboard_testing/scenario/test_validator.py` - [ ] T014 [US2] Implement schema, ref producer/consumer, duplicate, dependency, and cycle checks in `backend/src/services/dashboard_testing/scenario/validator.py` @POST: valid is true only with zero errors/blockers; findings stably ordered and actionable @TEST_EDGE: cycle→error contains cycle path, duplicate_output→both producer ids reported - [ ] T015 [US2] Implement parameter, selector, baseline, tool/action, path, SQL/code, raw-expected, and coverage checks @TEST_EDGE: raw_metric_expected→forbidden baseline literal error, unreachable_step→warning/error per coverage - [ ] T016 [US2] Return deterministic all-findings output with JSON pointers and recovery options - [ ] T017 [US2] Add property tests generating small DAG/cycle/ref variations without mirroring validator logic in `backend/tests/services/dashboard_testing/scenario/test_validator_properties.py` - [ ] T017b [P] [US2] Add belief-runtime instrumentation tests for ScenarioGraph.Validator.Validate in `backend/tests/services/dashboard_testing/scenario/test_validator_belief.py` @POST: REASON logged before mutation boundary; REFLECT after; belief_scope wraps validator run **Checkpoint**: Invalid fixture matrix passes; no SQL/raw-baseline/cycle escapes. ## Phase 4 — US3 Checklist Coverage and Serialization - [ ] T018 [US3] Write JSON/YAML golden tests in `backend/tests/services/dashboard_testing/scenario/test_serializer.py` - [ ] T019 [US3] Implement `backend/src/services/dashboard_testing/scenario/serializer.py` and revision hash exclusions @POST: key/order/decimal/date/newline rules stable across runs; JSON and YAML equal domain data @TEST_EDGE: shuffled_dicts→identical bytes, timestamp_display_field→excluded from revision identity - [ ] T020 [US3] Validate JSON Schema and Pydantic round-trip for all golden fixtures - [ ] T021 [US3] Add catalog-version and compiler-version fingerprints to scenario inputs **Checkpoint**: Byte-stable snapshots across runs. ## Phase 5 — US4 Parameters and Human Checkpoints - [ ] T022 [US4] Write failing typed resolution/stale revision tests in `backend/tests/services/dashboard_testing/scenario/test_resolver.py` - [ ] T023 [US4] Implement `backend/src/services/dashboard_testing/scenario/resolver.py` for parameter, selector, manual conversion, and remove-step operations @PRE: base revision hash matches; changes target declared unresolved items @POST: unrelated step ids/order unchanged; new parent/revision hashes link revisions @TEST_EDGE: stale_base_revision→409, invalid_parameter_type→422, unrelated_graph_change→invariant failure - [ ] T024 [US4] Enforce immutable revisions and unchanged unrelated step ids/order - [ ] T025 [US4] Cover safe-environment/test-data requirements for mutating PDF cases **Checkpoint**: Resolution produces linked immutable revisions; unrelated structure stable. ## Phase 6 — Safe Draft Pack - [ ] T026 Write failing template registry, preview-only, path, SQL, shell, and injected-code tests in `backend/tests/services/dashboard_testing/scenario/test_pack_compiler.py` @TEST_INVARIANT: No_LLM_To_Code → VERIFIED_BY: injected_code_field, template_registry_only @TEST_EDGE: unknown_template→blocked, path_traversal→blocked before artifact registration - [ ] T027 [P] Create versioned templates for scenario.yaml, runner.plan.json, report_template.md, evidence_manifest.json, and bounded browser/XLSX modules under `backend/src/services/dashboard_testing/scenario/pack_templates/v1/` - [ ] T028 Implement `backend/src/services/dashboard_testing/scenario/pack_compiler.py`; accept only registered template ids and structured inputs @PRE: scenario validation result available; template ids registered; target paths safe @POST: outputs match ArtifactPlan, contain no LLM executable bodies, and are registered as 036 drafts @SIDE_EFFECT: renders bounded templates; calls AgentRuns.Artifacts.Register @INVARIANT: errors/unresolved required inputs make pack preview_only - [ ] T029 [P] Register outputs through 036 AgentRuns.Artifacts.Register and emit generate/validate progress - [ ] T030 Ensure invalid/unresolved graphs produce preview_only with repeated blockers **Checkpoint**: Valid graph → save_eligible pack; invalid/injected → preview_only/blocked. ## Phase 7 — API, Agent, Quality - [ ] T031 Add `backend/src/api/routes/dashboard_scenarios.py` matching `contracts/openapi.yaml` and register router - [ ] T032 Write RBAC/contract/revision tests in `backend/tests/api/test_dashboard_scenarios.py` @TEST_EDGE: 401→UNAUTHORIZED, 403→FORBIDDEN (per scope), 409 stale→STALE_REVISION, 422→VALIDATION_ERROR, 429→RATE_LIMITED - [ ] T033 [P] Add thin compile/validate/resolve/generate tools in `agent/src/ss_tools/agent/tools.py` - [ ] T034 Verify agent schemas cannot carry code, SQL, raw expected metrics, custom tools, or artifact paths - [ ] T035 Run quickstart, JSON/OpenAPI schema validation, scoped/full backend tests, and ruff - [ ] T036 Audit all 19 cases, direct SQL/code bans, contract anchors, ATTN_1–4, and unresolved relations **Checkpoint**: API + agent tools match openapi.yaml; RBAC enforced. ## Phase 8 — Screenshot Capture, VLM Analysis, and Human Disposition (AGSCN-FR-010..012) - [ ] T037 [P] Write failing capture spec validation tests in `backend/tests/services/dashboard_testing/scenario/test_capture.py` - [ ] T038 [P] Create `backend/src/services/dashboard_testing/scenario/capture_profile.py` — load and validate CaptureProfile from `contracts/capture-profile.schema.json` - [ ] T039 [P] Implement ScenarioGraph.Capture.Dispatch: accept CaptureSpec from step, call AgentRuns.Evidence.Adapter, register screenshot artifacts, emit evidence_captured @POST: ScreenshotEvidence DraftArtifact registered; evidence_captured event emitted @TEST_EDGE: capture_timeout→step inconclusive; no artifact registered, masking_applied→original + masked artifacts - [ ] T040 [P] Write failing VLM analysis tests in `backend/tests/services/dashboard_testing/scenario/test_vlm.py` for typed findings, provenance, stale prompt rejection @TEST_EDGE: stale_prompt→422 STALE_PROMPT, vlm_timeout→inconclusive, empty_response→empty findings + inconclusive - [ ] T041 Implement `backend/src/services/dashboard_testing/scenario/vlm.py`: submit masked screenshot to VLM provider, parse typed VlmFinding[], validate model_provenance, persist raw response under redaction @POST: returns typed VlmFinding[] with model/prompt provenance; raw response stored under redaction @SIDE_EFFECT: external VLM API call; raw response persisted as separate DraftArtifact @INVARIANT: VLM findings advisory; never alter metric baseline truth; stale prompts block analysis @REJECTED: embedding VLM findings directly as assertion results (observations, not deterministic pass/fail) - [ ] T042 Create registered VLM prompt template v1 under `backend/src/services/dashboard_testing/scenario/prompt_templates/v1/` with versioned hash - [ ] T043 [P] Write failing human disposition tests in `backend/tests/services/dashboard_testing/scenario/test_disposition.py` for confirm/dismiss/inconclusive, double-disposition rejection @TEST_EDGE: double_disposition→409, confirm_requires_comment→422 - [ ] T044 Implement `backend/src/services/dashboard_testing/scenario/disposition.py`: record immutable disposition per finding id, enforce confirm requires non-blank comment, emit audit event @POST: each finding disposition set exactly once; step transitions per policy @SIDE_EFFECT: audit record of disposition decision @INVARIANT: disposition never alters scenario graph structure or step ordering - [ ] T045 Add VlmFinding and HumanDisposition DTOs to `contracts/openapi.yaml` response schemas (verify round-trip) - [ ] T046 Wire capture/VLM/disposition into the scenario step execution loop: screenshot step → capture → analysis step → VLM call → human step → disposition - [ ] T047 Audit: VLM findings are advisory, never alter metric baseline truth; disposition never changes graph structure; stale prompts block analysis **Checkpoint**: Capture/VLM/disposition flow verified end-to-end; typed findings auditable. ## Phase 9 — Polish & Cross-Cutting Verification - [ ] T048 [P] **Prototype validation**: verify every @UX_STATE in `contracts/ux/scenario-graph-ux.md` reachable via `specs/038-dashboard-scenario-model/prototype/index.html` state switcher; responsive on mobile viewport - [ ] T049 [P] **OpenAPI drift check**: verify operationId uniqueness (7), $ref resolution, example coverage, and RBAC scopes in `contracts/openapi.yaml` against implemented endpoints in `backend/src/api/routes/dashboard_scenarios.py` - [ ] T050 [P] **Belief runtime audit (C4/C5)**: `axiom_audit({operation="audit_belief_runtime"})` + `axiom_audit({operation="audit_belief_protocol"})` — confirm Compiler/Validator/Mapper/PackCompiler/Vlm/Capture/Resolver contracts have @RATIONALE/@REJECTED and REASON/REFLECT/EXPLORE markers - [ ] T051 [P] **Attention compliance audit**: verify ATTN_1–4 per `semantics-core` §VIII across `contracts/modules.md` - [ ] T052 [P] **Semantic index rebuild**: `axiom_search({operation="rebuild", rebuild_mode="full"})` — 0 parse warnings required - [ ] T053 [P] **Orphan audit**: `axiom_search({operation="workspace_health"})` — confirm no new orphans from this feature - [ ] T054 [P] **Traceability coverage gate**: verify `traceability.md` rows all map to real task IDs, contracts, and operationIds - [ ] T055 [P] Run quickstart.md validation and `make test-related F=backend/src/services/dashboard_testing/scenario/` for regression scope - [ ] T056 Run `/speckit.validate` — confirm PASS before `/speckit.implement` ## Dependencies T001–T005 → US1 → US2; US3 follows compiler; US4 follows validator; draft pack follows validation/resolution; Phase 8 depends on 036 Phase 8 (screenshot evidence artifacts) and 037 Phase 7 (visual baseline infrastructure). 039 begins only after ScenarioResponse and DraftPack fixtures are stable. Frontend tasks N/A — 038 is DTO-only. #endregion DashboardScenarioModel.Tasks