busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
master
ss-tools/specs/040-dashboard-load-testing/traceability.md
busya ac95beb1a0 docs(specs): record 036-041 MVP runtime gaps as open closure phases 
Fact-check the dashboard-testing spec packages against actual code and
amend the full speckit document set (spec, plan, research, tasks,
traceability, quickstart) so documented status matches reality:

- 037: discrete metric tools work, but deploy hooks do not create
  VerificationRun and GET read-API endpoints are missing (T080-T081)
- 038: compiler/validator work; VLM _default_submit and capture dispatch
  remain runtime stubs that never call LLMClient/ScreenshotService
  (T057-T059)
- 039: UI components exist, but api/dashboard-testing.ts is unbound and
  pipeline views are not wired to pages; depends on 037 read-API
  (T054-T058)
- 040: run_load_run never invokes RunnerPool, so load runs execute zero
  Superset requests; must wire 037 executor (T075-T079)
- 041: backend index works, but no /lineage frontend and lineage_index
  stays opt-in (T045-T048)
- 036: confirmed operational, relations to reused plugin modules fixed

19 open closure tasks total; region pairs balanced.
2026-08-07 12:56:40 +07:00

4.0 KiB
Raw Permalink Blame History

#region DashboardLoadTesting.Traceability [C:3] [TYPE ADR] [SEMANTICS traceability,load-testing,requirements] @BRIEF Requirement-to-contract-to-test matrix for 040 load testing.

Requirement Contract Verification
LOAD-FR-001 LoadTesting.Api.Routes + SupersetClient.ChartData.Execute malicious SQL/raw-context schema tests
LOAD-FR-002, LOAD-FR-016, LOAD-FR-017 LoadTesting.Capacity.Resolve, LoadTesting.RunnerPool capacity, worker registry, one TaskManager task, multi-run env cap
LOAD-FR-003 LoadTesting.RunnerPool + baseline read-only boundary catalog hash before/after
LOAD-FR-004 LoadTesting.Breaker.Evaluate rolling window, error/p99 abort, drain
LOAD-FR-005, LOAD-FR-014 LoadTesting.Api.Routes + 036 ApprovalGate PROD deny/approval/hash mutation/RBAC
LOAD-FR-006 LoadTesting.RunnerPool + ConsistencyFinding identical coordinate digest divergence; row-order edge
LOAD-FR-007, LOAD-FR-010 LoadTesting.Matrix.Expand closed axes, NEEDS_CONTEXT, deterministic cartesian/seeded sample
LOAD-FR-008, LOAD-FR-011 LoadTesting.RunnerPool + LoadRunModel reconnect, ramp/steady/drain terminal FSM
LOAD-FR-009, LOAD-FR-013 041 pinned BlastRadiusReport configure/gate/results same fingerprint; schedule overlap
LOAD-FR-012, LOAD-FR-019 LoadTesting.Cache.MapResponse + LoadExecution authoritative response metadata mapping
LOAD-FR-015 LoadRunComparison complete vs partial comparison
LOAD-FR-018 LoadTesting.Result.BoundedProcess raw digest, row/sample cap, no full 037 normalization, GIL fixture

Story Coverage

Story Independent checkpoint
US1 Profile quickstart 1–2; matrix/cap preview and blast-radius warning
US2 Execution quickstart 3, 6, 10; worker cap, drain, recovery
US3 Safety quickstart 7–9; breaker, PROD gate, baseline isolation
US4 Results quickstart 4–5, 7; cache/metrics/consistency
US5 Comparison quickstart 10; cache metadata and partial-run deltas

Cross-Spec Relations

  • Upstream: 036 ApprovalGate, TaskManager run/recovery; 037 QueryModel, Filters, ChartData.Execute, error taxonomy; 041 pinned dependents read model.
  • Downstream: 039 can link LoadRun results from dashboard/release verification surfaces; 041 provides blast-radius snapshot.
  • Repository integration prerequisite: client_registry.get_client() must pass its existing per-env semaphore to AsyncAPIClient before load capacity claims are trusted.

Amendment (041-dataset-lineage-blast-radius, LIN-FR-013)

The 041 pinned blast-radius read model is now available: GET /api/lineage/datasets/{dataset_id}/dependents?env_id=&fingerprint= serves {fingerprint, stale_index, stale_notice, env_scope, dependents} from the lineage index snapshot (R6). 040 load profiles and PROD gates consume this shape; fingerprint mismatch returns stale_notice (re-fetch, never silent drift). Producer: Api.Lineage.Dependents + Services.Lineage.Indexer.

Amendment (2026-08-07 MVP audit — Phase 9 runtime closure)

Факт-чекинг кода выявил, что run_load_run в backend/src/plugins/load_testing.py не вызывает RunnerPool — фазы проходят без реальных Superset-запросов. Добавлены задачи Phase 9 (T075–T079):

Requirement Contract Tasks Test
LOAD-FR-001, LOAD-FR-002 LoadTesting.RunnerPool + SupersetClient.ChartData.Execute T075, T076 lifecycle asserts executor invoked per item with real chart-data context
LOAD-FR-002 (shared admission) LoadTesting.Capacity.Resolve + client_registry T077 concurrent ordinary + load request never exceed env cap
LOAD-FR-004 LoadTesting.Breaker.Evaluate T078 breaker abort mid-drain preserves partial LoadExecutions
LOAD-FR-008, LOAD-FR-012, LOAD-FR-015 LoadTesting.Result.BoundedProcess + LoadExecution store T079 /status & /compare return real latency/cache/consistency data

#endregion DashboardLoadTesting.Traceability