busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
2eca5b514ba5e7a93bdcd0d1cbaa8d37745f4086
ss-tools/specs/038-dashboard-scenario-model/tasks.md

5.4 KiB
Raw Blame History

#region DashboardScenarioModel.Tasks [C:3] [TYPE ADR] [SEMANTICS tasks,scenario,implementation] @BRIEF Ordered TDD backlog for deterministic scenario graph and safe draft-pack compilation.

Phase 1 — Catalog, Schemas, Fixtures

  • T001 Transcribe checklist-catalog.md into a versioned declarative resource in backend/src/services/dashboard_testing/scenario/catalog_v1.yaml.
  • T002 Write catalog completeness tests for B01–B09, C01–C07, T01–T03 in backend/tests/services/dashboard_testing/scenario/test_catalog.py.
  • T003 Create valid 18-step scenario plus invalid cycle/missing-ref/duplicate-output/raw-baseline/SQL fixtures under specs/038-dashboard-scenario-model/fixtures/.
  • T004 Materialize fixtures to backend/tests/fixtures/dashboard_scenarios/.
  • T005 Implement Pydantic models in backend/src/services/dashboard_testing/scenario/models.py matching dashboard-test-scenario.schema.json.

Phase 2 — US1 Compile Scenario Graph

  • T006 [US1] Write failing capability mapping tests in backend/tests/services/dashboard_testing/scenario/test_capability_mapper.py.
  • T007 [US1] Implement backend/src/services/dashboard_testing/scenario/checklist_catalog.py validation and backend/src/services/dashboard_testing/scenario/capability_mapper.py with complete classification.
  • T008 [US1] Cover unavailable XLSX, missing selector/test data, unsafe mutation context, cross-dashboard absence, and T01–T03 no-SQL fallbacks.
  • T009 [US1] Write failing deterministic compiler tests in backend/tests/services/dashboard_testing/scenario/test_compiler.py.
  • T010 [US1] Implement registered tool/action and step-template catalogs under backend/src/services/dashboard_testing/scenario/templates/.
  • T011 [US1] Implement backend/src/services/dashboard_testing/scenario/compiler.py with stable ids, phase order, refs, coverage, and fingerprints.
  • T012 [US1] Prove repeated compile and shuffled input order produce identical graph bytes.

Checkpoint: Valid fixture compiles to stable graph and classifies all 19 cases.

Phase 3 — US2 Validate Safety and Completeness

  • T013 [US2] Write failing full invalid-fixture matrix in backend/tests/services/dashboard_testing/scenario/test_validator.py.
  • T014 [US2] Implement schema, ref producer/consumer, duplicate, dependency, and cycle checks in backend/src/services/dashboard_testing/scenario/validator.py.
  • T015 [US2] Implement parameter, selector, baseline, tool/action, path, SQL/code, raw-expected, and coverage checks.
  • T016 [US2] Return deterministic all-findings output with JSON pointers and recovery options.
  • T017 [US2] Add property tests generating small DAG/cycle/ref variations without mirroring validator logic in backend/tests/services/dashboard_testing/scenario/test_validator_properties.py.

Phase 4 — US3 Checklist Coverage and Serialization

  • T018 [US3] Write JSON/YAML golden tests in backend/tests/services/dashboard_testing/scenario/test_serializer.py.
  • T019 [US3] Implement backend/src/services/dashboard_testing/scenario/serializer.py and revision hash exclusions.
  • T020 [US3] Validate JSON Schema and Pydantic round-trip for all golden fixtures.
  • T021 [US3] Add catalog-version and compiler-version fingerprints to scenario inputs.

Phase 5 — US4 Parameters and Human Checkpoints

  • T022 [US4] Write failing typed resolution/stale revision tests in backend/tests/services/dashboard_testing/scenario/test_resolver.py.
  • T023 [US4] Implement backend/src/services/dashboard_testing/scenario/resolver.py for parameter, selector, manual conversion, and remove-step operations.
  • T024 [US4] Enforce immutable revisions and unchanged unrelated step ids/order.
  • T025 [US4] Cover safe-environment/test-data requirements for mutating PDF cases.

Phase 6 — Safe Draft Pack

  • T026 Write failing template registry, preview-only, path, SQL, shell, and injected-code tests in backend/tests/services/dashboard_testing/scenario/test_pack_compiler.py.
  • T027 Create versioned templates for scenario.yaml, runner.plan.json, report_template.md, evidence_manifest.json, and bounded browser/XLSX modules under backend/src/services/dashboard_testing/scenario/pack_templates/v1/.
  • T028 Implement backend/src/services/dashboard_testing/scenario/pack_compiler.py; accept only registered template ids and structured inputs.
  • T029 Register outputs through 036 AgentRuns.Artifacts.Register and emit generate/validate progress.
  • T030 Ensure invalid/unresolved graphs produce preview_only with repeated blockers.

Phase 7 — API, Agent, Quality

  • T031 Add backend/src/api/routes/dashboard_scenarios.py matching scenario-api.openapi.yaml and register router.
  • T032 Write RBAC/contract/revision tests in backend/tests/api/test_dashboard_scenarios.py.
  • T033 Add thin compile/validate/resolve/generate tools in agent/src/ss_tools/agent/tools.py.
  • T034 Verify agent schemas cannot carry code, SQL, raw expected metrics, custom tools, or artifact paths.
  • T035 Run quickstart, JSON/OpenAPI schema validation, scoped/full backend tests, and ruff.
  • T036 Audit all 19 cases, direct SQL/code bans, contract anchors, ATTN_1–4, and unresolved relations.

Dependencies

T001–T005 → US1 → US2; US3 can follow compiler, US4 follows validator, draft pack follows validation/resolution. 039 begins only after ScenarioResponse and DraftPack fixtures are stable.

#endregion DashboardScenarioModel.Tasks