ss-tools/specs/038-dashboard-scenario-model/tasks.md

#region DashboardScenarioModel.Tasks [C:3] [TYPE ADR] [SEMANTICS tasks,scenario,implementation]
@BRIEF Ordered TDD backlog for deterministic scenario graph, safe draft-pack compilation, capture/VLM/disposition, and full verification gates.

**Prerequisites**: plan.md, spec.md (required); contracts/modules.md, contracts/openapi.yaml, traceability.md, prototype/manifest.md (present).
**Tests**: Write tests FIRST (fail before implementation) for every C3+ contract per constitution VII. Frontend tasks are N/A — 038 is DTO-only; UI rendering is owned by 039.

## Format: `- [ ] T### [P] [USx] Description with exact file path`

## Phase 1 — Setup (Shared Infrastructure)

- [ ] T001 Transcribe checklist-catalog.md into a versioned declarative resource in `backend/src/services/dashboard_testing/scenario/catalog_v1.yaml`
- [ ] T002 [P] Write catalog completeness tests for B01–B09, C01–C07, T01–T03 in `backend/tests/services/dashboard_testing/scenario/test_catalog.py`
- [ ] T003 [P] Create valid 18-step scenario plus invalid cycle/missing-ref/duplicate-output/raw-baseline/SQL fixtures under `specs/038-dashboard-scenario-model/fixtures/`
- [ ] T004 [P] Materialize fixtures from `specs/038-dashboard-scenario-model/fixtures/` into `backend/tests/fixtures/dashboard_scenarios/` (copy as-is, no adaptation)
- [ ] T005 Implement Pydantic models in `backend/src/services/dashboard_testing/scenario/models.py` matching `contracts/dashboard-test-scenario.schema.json`

**Checkpoint**: Catalog loads 19 cases; schema round-trips against golden fixtures.

## Phase 2 — US1 Compile Scenario Graph

- [ ] T006 [US1] Write failing capability mapping tests in `backend/tests/services/dashboard_testing/scenario/test_capability_mapper.py`
- [ ] T007 [US1] Implement `backend/src/services/dashboard_testing/scenario/checklist_catalog.py` validation and `backend/src/services/dashboard_testing/scenario/capability_mapper.py` with complete classification
- [ ] T008 [US1] Cover unavailable XLSX, missing selector/test data, unsafe mutation context, cross-dashboard absence, and T01–T03 no-SQL fallbacks
      @TEST_EDGE: xlsx_unavailable→manual/unsupported, technical_without_dataset_fields→human checkpoint (no SQL)
- [ ] T009 [US1] Write failing deterministic compiler tests in `backend/tests/services/dashboard_testing/scenario/test_compiler.py`
- [ ] T010 [US1] Implement registered tool/action and step-template catalogs under `backend/src/services/dashboard_testing/scenario/templates/`
- [ ] T011 [US1] Implement `backend/src/services/dashboard_testing/scenario/compiler.py` with stable ids, phase order, refs, coverage, and fingerprints
      @PRE: intent, query model, catalog, baseline summary, parameters have valid fingerprints
      @POST: same canonical inputs/compiler version yield byte-identical graph and stable ids/order
      @DATA_CONTRACT: CompileScenarioRequest → DashboardTestScenario
      @TEST_EDGE: missing_selector→NEEDS_SELECTOR + save blocker, missing_baseline→NEEDS_BASELINE (no embedded numeric truth)
- [ ] T012 [US1] Prove repeated compile and shuffled input order produce identical graph bytes
      @INVARIANT: Deterministic_Graph → VERIFIED_BY: repeated_compile, shuffled_input_order

**Checkpoint**: Valid fixture compiles to stable graph and classifies all 19 cases.

## Phase 3 — US2 Validate Safety and Completeness

- [ ] T013 [US2] Write failing full invalid-fixture matrix in `backend/tests/services/dashboard_testing/scenario/test_validator.py`
- [ ] T014 [US2] Implement schema, ref producer/consumer, duplicate, dependency, and cycle checks in `backend/src/services/dashboard_testing/scenario/validator.py`
      @POST: valid is true only with zero errors/blockers; findings stably ordered and actionable
      @TEST_EDGE: cycle→error contains cycle path, duplicate_output→both producer ids reported
- [ ] T015 [US2] Implement parameter, selector, baseline, tool/action, path, SQL/code, raw-expected, and coverage checks
      @TEST_EDGE: raw_metric_expected→forbidden baseline literal error, unreachable_step→warning/error per coverage
- [ ] T016 [US2] Return deterministic all-findings output with JSON pointers and recovery options
- [ ] T017 [US2] Add property tests generating small DAG/cycle/ref variations without mirroring validator logic in `backend/tests/services/dashboard_testing/scenario/test_validator_properties.py`
- [ ] T017b [P] [US2] Add belief-runtime instrumentation tests for ScenarioGraph.Validator.Validate in `backend/tests/services/dashboard_testing/scenario/test_validator_belief.py`
      @POST: REASON logged before mutation boundary; REFLECT after; belief_scope wraps validator run

**Checkpoint**: Invalid fixture matrix passes; no SQL/raw-baseline/cycle escapes.

## Phase 4 — US3 Checklist Coverage and Serialization

- [ ] T018 [US3] Write JSON/YAML golden tests in `backend/tests/services/dashboard_testing/scenario/test_serializer.py`
- [ ] T019 [US3] Implement `backend/src/services/dashboard_testing/scenario/serializer.py` and revision hash exclusions
      @POST: key/order/decimal/date/newline rules stable across runs; JSON and YAML equal domain data
      @TEST_EDGE: shuffled_dicts→identical bytes, timestamp_display_field→excluded from revision identity
- [ ] T020 [US3] Validate JSON Schema and Pydantic round-trip for all golden fixtures
- [ ] T021 [US3] Add catalog-version and compiler-version fingerprints to scenario inputs

**Checkpoint**: Byte-stable snapshots across runs.

## Phase 5 — US4 Parameters and Human Checkpoints

- [ ] T022 [US4] Write failing typed resolution/stale revision tests in `backend/tests/services/dashboard_testing/scenario/test_resolver.py`
- [ ] T023 [US4] Implement `backend/src/services/dashboard_testing/scenario/resolver.py` for parameter, selector, manual conversion, and remove-step operations
      @PRE: base revision hash matches; changes target declared unresolved items
      @POST: unrelated step ids/order unchanged; new parent/revision hashes link revisions
      @TEST_EDGE: stale_base_revision→409, invalid_parameter_type→422, unrelated_graph_change→invariant failure
- [ ] T024 [US4] Enforce immutable revisions and unchanged unrelated step ids/order
- [ ] T025 [US4] Cover safe-environment/test-data requirements for mutating PDF cases

**Checkpoint**: Resolution produces linked immutable revisions; unrelated structure stable.

## Phase 6 — Safe Draft Pack

- [ ] T026 Write failing template registry, preview-only, path, SQL, shell, and injected-code tests in `backend/tests/services/dashboard_testing/scenario/test_pack_compiler.py`
      @TEST_INVARIANT: No_LLM_To_Code → VERIFIED_BY: injected_code_field, template_registry_only
      @TEST_EDGE: unknown_template→blocked, path_traversal→blocked before artifact registration
- [ ] T027 [P] Create versioned templates for scenario.yaml, runner.plan.json, report_template.md, evidence_manifest.json, and bounded browser/XLSX modules under `backend/src/services/dashboard_testing/scenario/pack_templates/v1/`
- [ ] T028 Implement `backend/src/services/dashboard_testing/scenario/pack_compiler.py`; accept only registered template ids and structured inputs
      @PRE: scenario validation result available; template ids registered; target paths safe
      @POST: outputs match ArtifactPlan, contain no LLM executable bodies, and are registered as 036 drafts
      @SIDE_EFFECT: renders bounded templates; calls AgentRuns.Artifacts.Register
      @INVARIANT: errors/unresolved required inputs make pack preview_only
- [ ] T029 [P] Register outputs through 036 AgentRuns.Artifacts.Register and emit generate/validate progress
- [ ] T030 Ensure invalid/unresolved graphs produce preview_only with repeated blockers

**Checkpoint**: Valid graph → save_eligible pack; invalid/injected → preview_only/blocked.

## Phase 7 — API, Agent, Quality

- [ ] T031 Add `backend/src/api/routes/dashboard_scenarios.py` matching `contracts/openapi.yaml` and register router
- [ ] T032 Write RBAC/contract/revision tests in `backend/tests/api/test_dashboard_scenarios.py`
      @TEST_EDGE: 401→UNAUTHORIZED, 403→FORBIDDEN (per scope), 409 stale→STALE_REVISION, 422→VALIDATION_ERROR, 429→RATE_LIMITED
- [ ] T033 [P] Add thin compile/validate/resolve/generate tools in `agent/src/ss_tools/agent/tools.py`
- [ ] T034 Verify agent schemas cannot carry code, SQL, raw expected metrics, custom tools, or artifact paths
- [ ] T035 Run quickstart, JSON/OpenAPI schema validation, scoped/full backend tests, and ruff
- [ ] T036 Audit all 19 cases, direct SQL/code bans, contract anchors, ATTN_1–4, and unresolved relations

**Checkpoint**: API + agent tools match openapi.yaml; RBAC enforced.

## Phase 8 — Screenshot Capture, VLM Analysis, and Human Disposition (AGSCN-FR-010..012)

- [ ] T037 [P] Write failing capture spec validation tests in `backend/tests/services/dashboard_testing/scenario/test_capture.py`
- [ ] T038 [P] Create `backend/src/services/dashboard_testing/scenario/capture_profile.py` — load and validate CaptureProfile from `contracts/capture-profile.schema.json`
- [ ] T039 [P] Implement ScenarioGraph.Capture.Dispatch: accept CaptureSpec from step, call AgentRuns.Evidence.Adapter, register screenshot artifacts, emit evidence_captured
      @POST: ScreenshotEvidence DraftArtifact registered; evidence_captured event emitted
      @TEST_EDGE: capture_timeout→step inconclusive; no artifact registered, masking_applied→original + masked artifacts
- [ ] T040 [P] Write failing VLM analysis tests in `backend/tests/services/dashboard_testing/scenario/test_vlm.py` for typed findings, provenance, stale prompt rejection
      @TEST_EDGE: stale_prompt→422 STALE_PROMPT, vlm_timeout→inconclusive, empty_response→empty findings + inconclusive
- [ ] T041 Implement `backend/src/services/dashboard_testing/scenario/vlm.py`: submit masked screenshot to VLM provider, parse typed VlmFinding[], validate model_provenance, persist raw response under redaction
      @POST: returns typed VlmFinding[] with model/prompt provenance; raw response stored under redaction
      @SIDE_EFFECT: external VLM API call; raw response persisted as separate DraftArtifact
      @INVARIANT: VLM findings advisory; never alter metric baseline truth; stale prompts block analysis
      @REJECTED: embedding VLM findings directly as assertion results (observations, not deterministic pass/fail)
- [ ] T042 Create registered VLM prompt template v1 under `backend/src/services/dashboard_testing/scenario/prompt_templates/v1/` with versioned hash
- [ ] T043 [P] Write failing human disposition tests in `backend/tests/services/dashboard_testing/scenario/test_disposition.py` for confirm/dismiss/inconclusive, double-disposition rejection
      @TEST_EDGE: double_disposition→409, confirm_requires_comment→422
- [ ] T044 Implement `backend/src/services/dashboard_testing/scenario/disposition.py`: record immutable disposition per finding id, enforce confirm requires non-blank comment, emit audit event
      @POST: each finding disposition set exactly once; step transitions per policy
      @SIDE_EFFECT: audit record of disposition decision
      @INVARIANT: disposition never alters scenario graph structure or step ordering
- [ ] T045 Add VlmFinding and HumanDisposition DTOs to `contracts/openapi.yaml` response schemas (verify round-trip)
- [ ] T046 Wire capture/VLM/disposition into the scenario step execution loop: screenshot step → capture → analysis step → VLM call → human step → disposition
- [ ] T047 Audit: VLM findings are advisory, never alter metric baseline truth; disposition never changes graph structure; stale prompts block analysis

**Checkpoint**: Capture/VLM/disposition flow verified end-to-end; typed findings auditable.

## Phase 9 — Polish & Cross-Cutting Verification

- [ ] T048 [P] **Prototype validation**: verify every @UX_STATE in `contracts/ux/scenario-graph-ux.md` reachable via `specs/038-dashboard-scenario-model/prototype/index.html` state switcher; responsive on mobile viewport
- [ ] T049 [P] **OpenAPI drift check**: verify operationId uniqueness (7), $ref resolution, example coverage, and RBAC scopes in `contracts/openapi.yaml` against implemented endpoints in `backend/src/api/routes/dashboard_scenarios.py`
- [ ] T050 [P] **Belief runtime audit (C4/C5)**: `axiom_audit({operation="audit_belief_runtime"})` + `axiom_audit({operation="audit_belief_protocol"})` — confirm Compiler/Validator/Mapper/PackCompiler/Vlm/Capture/Resolver contracts have @RATIONALE/@REJECTED and REASON/REFLECT/EXPLORE markers
- [ ] T051 [P] **Attention compliance audit**: verify ATTN_1–4 per `semantics-core` §VIII across `contracts/modules.md`
- [ ] T052 [P] **Semantic index rebuild**: `axiom_search({operation="rebuild", rebuild_mode="full"})` — 0 parse warnings required
- [ ] T053 [P] **Orphan audit**: `axiom_search({operation="workspace_health"})` — confirm no new orphans from this feature
- [ ] T054 [P] **Traceability coverage gate**: verify `traceability.md` rows all map to real task IDs, contracts, and operationIds
- [ ] T055 [P] Run quickstart.md validation and `make test-related F=backend/src/services/dashboard_testing/scenario/` for regression scope
- [ ] T056 Run `/speckit.validate` — confirm PASS before `/speckit.implement`

## Dependencies

T001–T005 → US1 → US2; US3 follows compiler; US4 follows validator; draft pack follows validation/resolution; Phase 8 depends on 036 Phase 8 (screenshot evidence artifacts) and 037 Phase 7 (visual baseline infrastructure). 039 begins only after ScenarioResponse and DraftPack fixtures are stable. Frontend tasks N/A — 038 is DTO-only.

#endregion DashboardScenarioModel.Tasks