busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
d874a4dca6bee2c72a07cbc129eb1e66816783a0
ss-tools/specs/038-dashboard-scenario-model/quickstart.md
busya d874a4dca6 feat(speckit): workflow architecture upgrades + rework 038 spec per new flow 
Add prototype/openapi/validate/resume commands, wire edge-failure matrix into UX,
enforce traceability + validation gates, mandate C4/C5 belief-runtime verification.
Rework 038-dashboard-scenario-model artifacts: applicability, structured edge cases,
24-class UX state matrix, interactive HTML prototype, standardized OpenAPI 3.1 (7 ops),
full RTM with coverage gate, 56-task backlog, and PASS validation report.
2026-07-31 11:25:43 +03:00

2.8 KiB
Raw Blame History

Quickstart: Dashboard Scenario Model

Prerequisites

036 draft registration and 037 query-model/baseline summary contracts must pass. /speckit.validate must report PASS before /speckit.implement.

Test Order

# Tier 1: Fast unit tests (<120s, no Docker)
make test-unit              # backend SQLite unit tests
make test-frontend          # frontend vitest (039 DTO consumption, if present)

# Tier 1 alt: Smart selection — only tests linked to scenario contracts
make test-related F=backend/src/services/dashboard_testing/scenario/

# Tier 2: Integration (Docker required)
make test-integration       # capture/VLM/disposition bridges via 036/037

# Scoped scenario suite
cd backend && python -m pytest tests/services/dashboard_testing/scenario -v
cd backend && python -m pytest tests/api/test_dashboard_scenarios.py -v

# Linting
make lint

# OpenAPI schema validation (in-repo, no new deps)
python3 -c "import yaml; d=yaml.safe_load(open('specs/038-dashboard-scenario-model/contracts/openapi.yaml')); assert d['openapi'].startswith('3.1'); print('openapi-ok')"

Independent Validation

  1. Load the catalog and assert exactly 19 unique ids.
  2. Map a full-capability dashboard; every case has one classification.
  3. Map a no-XLSX/no-dataset-field dashboard; C04–C06 and T01–T03 retain rationale with no SQL.
  4. Compile the same canonical inputs repeatedly and with shuffled input maps; bytes/hashes must match.
  5. Validate invalid fixtures: cycle, missing ref, duplicate output, raw metric expected, stale baseline, unknown selector/tool, unresolved required parameter.
  6. Resolve one parameter; unrelated step ids/order must remain unchanged and parent hash must link.
  7. Compile a valid graph to draft pack through registered templates.
  8. Attempt executable-code, shell, SQL, custom path, and unknown-template injection; all must block before draft registration.
  9. Prototype: every @UX_STATE in contracts/ux/scenario-graph-ux.md reachable via prototype/index.html state switcher (see prototype/manifest.md).
  10. VLM: typed findings + provenance; stale prompt → 422 STALE_PROMPT; raw response stored under redaction.
  11. Disposition: confirm/dismiss/inconclusive; double disposition → 409; graph structure unchanged.

Exit Gates

  • JSON Schema and Pydantic round-trip.
  • 100% of 19 cases classified in fixtures; at least 80% meet spec classification criterion.
  • Byte-stable JSON/YAML snapshots.
  • No raw baseline numbers in executable assertions.
  • No LLM executable body or SQL surface.
  • contracts/openapi.yaml valid: 7 unique operationIds, security on all mutating ops.
  • Belief runtime audit (C4/C5): axiom_audit({operation="audit_belief_runtime"}) + axiom_audit({operation="audit_belief_protocol"}) PASS.
  • /speckit.validate verdict: PASS.
  • Unit/API, ruff, schema, and semantic audits pass.