busya
d874a4dca6
Add prototype/openapi/validate/resume commands, wire edge-failure matrix into UX, enforce traceability + validation gates, mandate C4/C5 belief-runtime verification. Rework 038-dashboard-scenario-model artifacts: applicability, structured edge cases, 24-class UX state matrix, interactive HTML prototype, standardized OpenAPI 3.1 (7 ops), full RTM with coverage gate, 56-task backlog, and PASS validation report.
152 lines
9.4 KiB
Markdown
152 lines
9.4 KiB
Markdown
# Implementation Plan: Dashboard Scenario Model
|
||
|
||
**Branch**: `038-dashboard-scenario-model` | **Date**: 2026-07-31 | **Spec**: [spec.md](./spec.md) | **Status**: Reworked per new speckit flow
|
||
|
||
## Summary
|
||
|
||
Implement a backend Pydantic ScenarioGraph domain and deterministic compiler that maps the 19 normalized PDF checklist cases plus dashboard/baseline capabilities into a reviewable DAG. Validate all refs, cycles, tools, selectors, parameters, and baseline rules before compiling a draft pack through versioned safe templates and registering it with 036. Supply scenario/validation/coverage/parameter/artifact DTOs to 039 (scenario preview UI) and typed capture/VLM/disposition semantics (AGSCN-FR-010..012).
|
||
|
||
## Technical Context
|
||
|
||
**Language/Version**: Python 3.13+ (backend), TypeScript DTOs (frontend, consumed by 039)
|
||
**Primary Dependencies**: Pydantic 2, FastAPI, deterministic JSON/YAML serialization, existing 036/037 services
|
||
**Storage**: Immutable request/graph/draft hashes; draft bytes via 036; no new DB table required for MVP beyond AgentRun event/artifact links
|
||
**Testing**: pytest property/unit/contract, JSON Schema validation, golden snapshots; L1 (model, no render) + L2 (component/UX with render) per edge matrix ownership
|
||
**Frontend Architecture**: TypeScript-first DTOs in `frontend/src/types/` matching Pydantic schemas; UI rendering owned by 039, 038 supplies DTO contracts
|
||
**Performance Goals**: validate 100-step graph under 100ms; resolve parameters under 200ms; byte-stable snapshots
|
||
**Constraints**: DAG only; no raw baseline numbers; no SQL; no executable LLM output; all checklist cases classified; VLM findings advisory; disposition auditable
|
||
**Scale**: 19 source cases, up to 100 steps, 50 parameters, 200 refs
|
||
|
||
## Constitution Check
|
||
|
||
| Principle | Result |
|
||
|---|---|
|
||
| I. Semantic Contract First | PASS — compiler, validator, serializer, pack generator, capture, VLM, disposition contracted (C3–C5) |
|
||
| II. Decision Memory | PASS — `@RATIONALE`/`@REJECTED` on compiler/validator/pack/VLM (direct-code, silent-repair, raw-VLM, one-size-fits-all rejected) |
|
||
| III. External Orchestrator | PASS — reads 037 query models/baselines; never calls Superset directly; writes drafts via 036 |
|
||
| IV. Module Discipline | PASS — models/catalog/mapping/compiler/validator/serializer/templates/pack separated; no oversized compiler |
|
||
| V. RBAC Enforcement | PASS — scenario.compile/resolve/draft/execute scopes per ADR-0005; default-allow forbidden |
|
||
| VI. Svelte 5 Runes Only | PASS — UI is DTO-only for 039; no legacy Svelte introduced |
|
||
| VII. Test-Driven for C3+ | PASS — invalid fixture matrix and rejected direct-code/SQL/VLM paths written first |
|
||
| VIII. Attention-Optimized | PASS — ScenarioGraph.* IDs, shared `[SEMANTICS scenario,...]`, ATTN_1–4 gate applied |
|
||
|
||
## Project Structure
|
||
|
||
### Documentation (this feature)
|
||
|
||
```text
|
||
specs/038-dashboard-scenario-model/
|
||
├── spec.md # Reworked: applicability, structured edge cases, Clarifications
|
||
├── ux_reference.md # Reworked: edge/failure state matrix (24 classes)
|
||
├── research.md # Phase 0 decisions (deterministic compiler boundary)
|
||
├── data-model.md # Canonical graph/step/ref/parameter/validation models
|
||
├── quickstart.md # Verification commands and exit gates
|
||
├── traceability.md # REQUIRED RTM: Story → Screen+State → Model → operationId → Contract → Task → Test
|
||
├── checklist-catalog.md # Normalized 19-case catalog from the research PDF
|
||
├── checklists/ # Requirements-quality checklists
|
||
├── contracts/
|
||
│ ├── modules.md # C3+ contracts (compiler, validator, resolver, pack, capture, VLM, disposition)
|
||
│ ├── openapi.yaml # OpenAPI 3.1 — 7 operations, envelopes, RBAC, examples
|
||
│ ├── openapi-traceability.md # operationId → spec → data-model → UX drift map
|
||
│ ├── dashboard-test-scenario.schema.json
|
||
│ ├── capture-profile.schema.json
|
||
│ └── ux/
|
||
│ ├── api-ux.md # UI mapping for compile/validate/resolve/draft-pack
|
||
│ ├── scenario-graph-ux.md # Per-screen UX contract + edge/failure matrix
|
||
│ ├── decisions.md # Final UX decisions
|
||
│ └── alternatives.md # Design space explored
|
||
├── prototype/
|
||
│ ├── index.html # Interactive HTML prototype (10 states, state switcher, responsive)
|
||
│ └── manifest.md # State coverage + screen↔story traceability
|
||
└── validation.md # Pre-implementation gate (/speckit.validate output)
|
||
```
|
||
|
||
### Source Code (repository root)
|
||
|
||
```text
|
||
backend/src/
|
||
├── api/routes/dashboard_scenarios.py # compile/validate/resolve/draft-pack/capture/vlm/disposition
|
||
├── schemas/dashboard_scenario.py # Pydantic request/response schemas (mirror openapi.yaml)
|
||
└── services/dashboard_testing/scenario/
|
||
├── models.py # DashboardTestScenario, ScenarioStep, ScenarioParameter, ScenarioRef
|
||
├── checklist_catalog.py
|
||
├── capability_mapper.py
|
||
├── compiler.py
|
||
├── validator.py
|
||
├── serializer.py
|
||
├── resolver.py
|
||
├── pack_compiler.py
|
||
├── capture_profile.py
|
||
├── vlm.py # typed VlmFinding, provenance, stale-prompt guard
|
||
├── disposition.py # typed human disposition, double-disposition guard
|
||
├── templates/ # step-template catalog (declarative)
|
||
├── pack_templates/v1/ # scenario.yaml, runner.plan.json, report_template.md, evidence_manifest.json
|
||
└── prompt_templates/v1/ # registered VLM prompt template + hash
|
||
|
||
backend/tests/services/dashboard_testing/scenario/
|
||
backend/tests/api/test_dashboard_scenarios.py
|
||
frontend/src/types/ # DTOs matching Pydantic schemas (consumed by 039)
|
||
agent/src/ss_tools/agent/tools.py # thin compile/validate/resolve/generate tools
|
||
```
|
||
|
||
**Structure Decision**: ScenarioGraph is a bounded intermediate boundary between agent intent and generated artifacts; all generation flows through registered templates.
|
||
|
||
## Semantic Contract Guidance
|
||
|
||
### Attention Compliance Gate (MANDATORY — validated)
|
||
|
||
| Rule | Check | Why |
|
||
|------|-------|-----|
|
||
| **ATTN_1** | First anchor line packs `[C:N] [TYPE] [SEMANTICS]` on ONE line | CSA 4× pooling |
|
||
| **ATTN_2** | IDs hierarchical: `ScenarioGraph.Compiler.Compile`, `ScenarioGraph.Validator.Validate` | HCA 128× |
|
||
| **ATTN_3** | All scenario contracts share `[SEMANTICS scenario,...]` primary keyword | DSA grouping |
|
||
| **ATTN_4** | Contract ≤150 lines, module ≤400 lines | Sliding window |
|
||
|
||
### Function-Level Contracts (C3+)
|
||
|
||
Full `#region` headers for compiler, validator, resolver, serializer, pack_compiler, capture, vlm, disposition are defined in `contracts/modules.md` with `@PRE`/`@POST`/`@SIDE_EFFECT`/`@DATA_CONTRACT`/`@TEST_EDGE`. Cross-stack DTOs (VlmFinding, HumanDisposition, DraftPack) have matching `@RELATION` edges across the Pydantic ↔ TypeScript boundary.
|
||
|
||
## Decision Memory (Global ADR Continuity)
|
||
|
||
| ADR / Guardrail | Present in Plan | Propagated to Contracts | Propagated to Tasks | Verifying Tasks Exist | Rejected Path Protected |
|
||
|-----------------|:---:|:---:|:---:|:---:|:---:|
|
||
| ADR-0001 module layout | ✅ | ✅ | ✅ | T001–T005 | ✅ |
|
||
| ADR-0002 semantic protocol | ✅ | ✅ | ✅ | T036 | ✅ |
|
||
| ADR-0005 RBAC scopes | ✅ | ✅ (openapi security) | ✅ | T032 | ✅ |
|
||
| 037 no-direct-SQL invariant | ✅ | ✅ (validator) | ✅ | T015, T034 | ✅ |
|
||
| Direct LLM-to-code rejected | ✅ | ✅ (compiler @REJECTED) | ✅ | T026, T034 | ✅ |
|
||
| Raw baseline literals rejected | ✅ | ✅ (validator) | ✅ | T003, T014 | ✅ |
|
||
| VLM prose-as-state rejected | ✅ | ✅ (vlm @REJECTED) | ✅ | T040–T041 | ✅ |
|
||
| One-size-fits-all scripts rejected | ✅ | ✅ (mapper @REJECTED) | ✅ | T008 | ✅ |
|
||
|
||
## Delivery Phases
|
||
|
||
1. Machine schema, Pydantic models, normalized checklist catalog and fixtures.
|
||
2. Capability mapping and deterministic graph compiler.
|
||
3. Full validator and canonical serializer.
|
||
4. Parameter/selector/manual resolution with immutable revisions.
|
||
5. Safe template-based draft-pack compiler and 036 registration.
|
||
6. Screenshot capture, VLM analysis, human disposition (AGSCN-FR-010..012).
|
||
7. REST/agent tools, DTOs for 039, and regression gates (incl. `/speckit.validate` PASS).
|
||
|
||
## API and Schema
|
||
|
||
- `contracts/openapi.yaml` is the canonical REST contract (7 operations, RBAC scopes, error envelopes).
|
||
- `contracts/dashboard-test-scenario.schema.json` and `contracts/capture-profile.schema.json` are the canonical graph/capture interchange contracts.
|
||
- `contracts/openapi-traceability.md` maps every operationId to spec/data-model/UX.
|
||
|
||
## Traceability
|
||
|
||
`traceability.md` is REQUIRED and maps Story/Requirement → UX Screen+State → Screen Model → API operationId → Contract → Task → Test, with explicit N/A rationale and a coverage gate (see `traceability.md`).
|
||
|
||
## Cross-Spec Boundary
|
||
|
||
- Reads 037 query models/baseline summaries; never calls Superset directly.
|
||
- Writes drafts and progress through 036.
|
||
- Supplies scenario, validation, coverage, parameter, and artifact manifest DTOs to 039.
|
||
- Screenshot capture/VLM/disposition depend on 036 Phase 8 (screenshot evidence artifacts) and 037 Phase 7 (visual baseline infrastructure).
|
||
|
||
## Complexity Tracking
|
||
|
||
No exception planned. Checklist data remains declarative and versioned; do not turn the 19 cases into one large conditional compiler function. VLM findings are advisory — never asserted as deterministic truth.
|