busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs(specs): complete dashboard testing contracts

Browse Source
This commit is contained in:
busya
2026-07-23 14:08:09 +03:00
parent eeb3a05e42
commit fb6327e92b
28 changed files with 2164 additions and 139 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
specs/036-agent-test-stabilization/contracts/agent-pipeline.md Normal file
View File

@@ -0,0 +1,46 @@
#region AgentTestStabilization.AgentPipeline [C:4] [TYPE ADR] [SEMANTICS contracts,agent-run,pipeline,tools]
@BRIEF Agent-side tracker, context validation, scenario tool filtering, and verification trigger contracts.
@RELATION DEPENDS_ON -> [AgentTestStabilization.DataModel]
@RELATION DEPENDS_ON -> [AgentRuns.Api]
# #region AgentRuns.Tracker [C:4] [TYPE Module] [SEMANTICS agent-run,agent,http,events]
# @defgroup AgentRuns Agent-side typed client for backend run lifecycle APIs.
# @LAYER Service
# @RELATION DEPENDS_ON -> [AgentRuns.Api]
# @INVARIANT Every call carries service JWT and current user JWT; logs redact both.
# @RATIONALE Small client isolates retry/idempotency behavior from the large Gradio handler.
# #endregion AgentRuns.Tracker
# #region AgentRuns.Context.ValidateV2 [C:3] [TYPE Function] [SEMANTICS agent-run,context,intent,validation]
# @ingroup AgentRuns
# @BRIEF Validate UIContext v1/v2 and scenario-intent cross-field rules.
# @POST Returns normalized context; v1 ordinary chat remains compatible.
# @TEST_EDGE v1_without_intent -> accepted; v2_scenario_dashboard -> accepted; scenario_dataset -> rejected; unknown_intent -> rejected.
# #endregion AgentRuns.Context.ValidateV2
# #region AgentRuns.ToolPipeline.Scenario [C:4] [TYPE Function] [SEMANTICS agent-run,tools,scenario,no-sql]
# @ingroup AgentRuns
# @BRIEF Restrict scenario-intent tools to dashboard inspection, scenario, artifact, and approval operations.
# @PRE RBAC filtering has already run.
# @POST Tool list excludes arbitrary SQL operations.
# @SIDE_EFFECT Emits pipeline_result audit metadata.
# @INVARIANT No direct SQL tool reaches build_dashboard_test_scenario.
# @TEST_INVARIANT No_Direct_SQL -> VERIFIED_BY: scenario_tool_list, replayed_sql_call.
# @TEST_EDGE hallucinated_sql_tool_call -> invocation guard rejects.
# @REJECTED Relying only on system prompt to avoid SQL — tool availability is enforceable.
# #endregion AgentRuns.ToolPipeline.Scenario
# #region AgentRuns.Verification.CreateRun [C:4] [TYPE Function] [SEMANTICS agent-run,verification,release,dashboard-testing]
# @ingroup AgentRuns
# @BRIEF Create an AgentRun triggered by release pipeline or scheduler, linked to VerificationRun.
# @PRE Trigger valid; actor has dashboard testing execute; release triggers have DashboardRelease.
# @POST AgentRun and linked VerificationRun are created atomically.
# @SIDE_EFFECT Writes AgentRun and VerificationRun.
# @DATA_CONTRACT CreateAgentRunRequest -> AgentRunSnapshot + VerificationRun
# @RELATION DEPENDS_ON -> [AgentRuns.Service.Create]
# @TEST_EDGE scheduled without release -> run without release_id.
# @RATIONALE Trigger enum makes verification scope explicit and recoverable.
# @REJECTED Creating verification outside AgentRun lifecycle — recovery and gate semantics would diverge.
# #endregion AgentRuns.Verification.CreateRun
#endregion AgentTestStabilization.AgentPipeline

24
specs/036-agent-test-stabilization/contracts/evidence.md Normal file
View File

@@ -0,0 +1,24 @@
#region AgentTestStabilization.EvidenceContracts [C:4] [TYPE ADR] [SEMANTICS contracts,agent-run,evidence,screenshot]
@BRIEF Screenshot evidence adapter contract for masked artifacts and VLM analysis handoff.
@RELATION DEPENDS_ON -> [AgentTestStabilization.DataModel]
@RELATION DEPENDS_ON -> [AgentRuns.Artifacts.Register]
@RATIONALE Existing ScreenshotService capture behavior is reused through an adapter so scenario evidence has one masking and readiness implementation.
@REJECTED Rebuilding screenshot capture inside scenario execution — rejected because it would fork Playwright/CDP behavior and weaken evidence auditability.
# #region AgentRuns.Evidence.Adapter [C:5] [TYPE Module] [SEMANTICS agent-run,evidence,screenshot,capture,llm]
# @ingroup AgentRuns
# @BRIEF Bridge existing ScreenshotService/VLM pipeline to AgentRun/DraftArtifact.
# @RATIONALE Existing capture+VLM pipeline handles tab traversal, stabilization, and image conversion; the adapter registers it without duplicating behavior.
# @REJECTED Rebuilding screenshot capture inside AgentRun lifecycle — rejected because scenario runs should consume the established capture contract.
# @LAYER Service
# @RELATION DEPENDS_ON -> [AgentRuns.Artifacts.Register]
# @RELATION DEPENDS_ON -> [Spec.LlmAnalysisPlugin.ScreenshotService]
# @DATA_CONTRACT CaptureSpec + AgentRun -> DraftArtifact[]
# @INVARIANT Original capture never reaches external LLM; only masked derivative does.
# @INVARIANT Every capture carries capture_meta for reproducibility.
# @TEST_EDGE no_mask_selectors -> original and masked artifacts remain separate.
# @TEST_EDGE vlm_failure -> evidence remains valid WARN; analysis becomes inconclusive.
# @TEST_EDGE missing_capture_meta -> 422.
# #endregion AgentRuns.Evidence.Adapter
#endregion AgentTestStabilization.EvidenceContracts

40
specs/036-agent-test-stabilization/contracts/frontend-models.md Normal file
View File

@@ -0,0 +1,40 @@
#region AgentTestStabilization.FrontendModels [C:4] [TYPE ADR] [SEMANTICS contracts,agent-run,frontend,ux]
@BRIEF Frontend run model and draft/recovery components for durable AgentRuns.
@RELATION DEPENDS_ON -> [AgentTestStabilization.DataModel]
@RELATION DEPENDS_ON -> [AgentRuns.Api]
// #region AgentRuns.Model [C:5] [TYPE Model] [SEMANTICS agent-run,model,recovery,draft,approval]
// @defgroup AgentRuns Frontend run projection composed by AgentChat.Model.
// @STATE absent | starting | running | waiting_input | waiting_approval | completed | failed | cancelled | disconnected
// @ACTION applyMetadata(meta) — accept typed event when sequence is newer.
// @ACTION recover(runId) — fetch authoritative snapshot after reload/drop.
// @ACTION decideGate(gateId, decision, reason?) — submit HITL decision through existing resume path.
// @INVARIANT Stage state derives from structured metadata/snapshot only, never assistant prose.
// @INVARIANT Drafts are keyed by artifact id and cannot cross run boundaries.
// @RELATION DEPENDS_ON -> [AgentRuns.Api]
// @RELATION BINDS_TO -> [AgentRuns.RunPanel]
// @RATIONALE A composed submodel prevents further uncontrolled growth of AgentChat.Model.
// @REJECTED Duplicate inline run atoms in components — creates divergent recovery state.
// #endregion AgentRuns.Model
<!-- #region AgentRuns.RunPanel [C:4] [TYPE Component] [SEMANTICS agent-run,ui,progress,recovery] -->
<!-- @ingroup AgentRuns -->
<!-- @RELATION BINDS_TO -> [AgentRuns.Model] -->
<!-- @UX_STATE running -> Context header, run id, ordered progress, draft count. -->
<!-- @UX_STATE disconnected -> Last known state plus Recover action; no implied execution restart. -->
<!-- @UX_STATE failed -> Error code/detail and retry-from-dashboard guidance. -->
<!-- @UX_FEEDBACK aria-live polite announces stage transitions once. -->
<!-- @UX_RECOVERY Recover snapshot, retry safe analysis, or return to dashboard. -->
<!-- @UX_TEST reconnect -> snapshot restores stage and drafts without duplicated messages. -->
<!-- #endregion AgentRuns.RunPanel -->
<!-- #region AgentRuns.DraftList [C:4] [TYPE Component] [SEMANTICS agent-run,ui,draft,artifact] -->
<!-- @ingroup AgentRuns -->
<!-- @RELATION BINDS_TO -> [AgentRuns.Model] -->
<!-- @UX_STATE empty | ready | warning | invalid | loading_preview -->
<!-- @UX_FEEDBACK Warnings and invalid markers are visible next to each intended path. -->
<!-- @UX_RECOVERY Preview/download draft remain side-effect free; invalid draft cannot request save. -->
<!-- @UX_TEST invalid_artifact -> Request save disabled and warning linked by aria-describedby. -->
<!-- #endregion AgentRuns.DraftList -->
#endregion AgentTestStabilization.FrontendModels

109
specs/036-agent-test-stabilization/contracts/modules.md
View File

@@ -50,6 +50,7 @@
# @POST Draft is retrievable by opaque id; target repository remains unchanged.
# @SIDE_EFFECT Writes draft storage and DraftArtifact record.
# @DATA_CONTRACT RegisterDraftRequest + bytes -> DraftArtifactRef
# @RATIONALE Opaque draft references separate previewable generated content from repository state and prevent path disclosure before HITL approval.
# @INVARIANT content_ref and filesystem path are never returned to clients.
# @TEST_EDGE traversal_path -> 422 before write.
# @TEST_EDGE digest_mismatch -> draft bytes removed and 422.
@@ -65,6 +66,8 @@
# @SIDE_EFFECT Writes ApprovalGate and approval_requested event.
# @DATA_CONTRACT ApprovalRequest -> ApprovalGateView
# @INVARIANT Unauthorized calls return permission_denied semantics and create no gate.
# @RATIONALE Gate creation binds exact draft ownership and operation inputs before any decision can be recorded; this keeps the later consume step auditable.
# @REJECTED Creating a reusable unbound confirmation token — rejected because it could be replayed against another run, artifact, or baseline candidate.
# @TEST_EDGE foreign_artifact -> 403.
# @TEST_EDGE missing_baseline_reason_requirement -> gate created with reason_required=true.
# #endregion AgentRuns.Approvals.Request
@@ -76,6 +79,8 @@
# @POST Gate becomes CONFIRMED or DENIED exactly once and decision event is appended.
# @SIDE_EFFECT Writes decision audit; denial returns run to RUNNING or CANCELLED according to operation policy.
# @DATA_CONTRACT ApprovalDecisionRequest -> ApprovalGateView
# @RATIONALE Decisions are immutable audit facts; recording exactly one decision prevents ambiguous approval state under retries or multiple tabs.
# @REJECTED Last-write-wins gate decisions — rejected because a late denial/confirmation could silently override the actor's earlier decision.
# @TEST_EDGE blank_required_reason -> 422.
# @TEST_EDGE repeated_decision -> 409.
# @TEST_EDGE expired_gate -> EXPIRED and no confirmation.
@@ -98,74 +103,6 @@
## Standalone Agent
# #region AgentRuns.Tracker [C:4] [TYPE Module] [SEMANTICS agent-run,agent,http,events]
# @defgroup AgentRuns Agent-side typed client for backend run lifecycle APIs.
# @LAYER Integration
# @RELATION DEPENDS_ON -> [AgentRuns.Api]
# @INVARIANT Every call carries service JWT and current user JWT; logs redact both.
# @RATIONALE A small client isolates retry/idempotency behavior from the already-large Gradio handler.
# #endregion AgentRuns.Tracker
# #region AgentRuns.Context.ValidateV2 [C:3] [TYPE Function] [SEMANTICS agent-run,context,intent,validation]
# @ingroup AgentRuns
# @BRIEF Validate UIContext v1/v2 and scenario-intent cross-field rules.
# @POST Returns normalized context; v1 ordinary chat remains byte-shape compatible.
# @TEST_EDGE v1_without_intent -> accepted.
# @TEST_EDGE v2_scenario_dashboard -> accepted.
# @TEST_EDGE scenario_dataset -> rejected.
# @TEST_EDGE unknown_intent -> rejected.
# #endregion AgentRuns.Context.ValidateV2
# #region AgentRuns.ToolPipeline.Scenario [C:4] [TYPE Function] [SEMANTICS agent-run,tools,scenario,no-sql]
# @ingroup AgentRuns
# @BRIEF Restrict scenario-intent tools to dashboard inspection, scenario, artifact, and approval operations.
# @PRE RBAC filtering has already run.
# @POST Tool list excludes superset_execute_sql and every arbitrary-SQL operation.
# @SIDE_EFFECT Emits pipeline_result audit metadata.
# @INVARIANT No direct SQL tool is reachable for build_dashboard_test_scenario.
# @TEST_INVARIANT No_Direct_SQL -> VERIFIED_BY: scenario_tool_list, replayed_sql_call.
# @TEST_EDGE hallucinated_sql_tool_call -> invocation guard rejects.
# @REJECTED Relying only on the system prompt to avoid SQL — tool availability is the enforceable boundary.
# #endregion AgentRuns.ToolPipeline.Scenario
# #region AgentRuns.Verification.CreateRun [C:4] [TYPE Function] [SEMANTICS agent-run,verification,release,dashboard-testing]
# @ingroup AgentRuns
# @BRIEF Create an AgentRun triggered by the release pipeline or scheduler, linked to a VerificationRun (037).
# @PRE trigger is valid; actor has dashboard:testing EXECUTE; for release triggers, DashboardRelease exists.
# @POST AgentRun created with trigger recorded; VerificationRun created and linked.
# @SIDE_EFFECT Writes AgentRun + VerificationRun in one transaction.
# @DATA_CONTRACT CreateAgentRunRequest(with trigger) -> AgentRunSnapshot + VerificationRun
# @RELATION DEPENDS_ON -> [AgentRuns.Service.Create]
# @TEST_EDGE scheduled_trigger_no_release -> creates run without release_id.
# @TEST_EDGE deploy_to_preprod_trigger -> creates run + VerificationRun linked to PREPROD deployment.
# @TEST_EDGE etl_completed_trigger -> creates run scoped to affected datasets.
# @RATIONALE Different triggers require different verification scopes; the trigger enum makes dispatch explicit.
# @REJECTED Creating verification runs outside the AgentRun lifecycle — they share recovery, events, and gate infrastructure.
# #endregion AgentRuns.Verification.CreateRun
# #region AgentRuns.Evidence.Adapter [C:5] [TYPE Module] [SEMANTICS agent-run,evidence,screenshot,capture,llm]
# @ingroup AgentRuns
# @BRIEF Bridge between existing ScreenshotService/VLM pipeline and the new AgentRun/DraftArtifact substrate.
# @RATIONALE The existing `run_llm_validation` capture+VLM pipeline is production-grade; duplicating it inside the scenario run would create divergent capture behavior. The adapter registers captures as 036 DraftArtifacts and connects VLM findings to step outputs.
# @REJECTED Rebuilding screenshot capture inside the scenario run lifecycle — the existing Playwright/CDP pipeline handles tab traversal, chart stabilization, and image conversion; scenario runs should consume it, not fork it.
# @LAYER Integration
# @RELATION DEPENDS_ON -> [AgentRuns.Artifacts.Register]
# @RELATION DEPENDS_ON -> [Spec.LlmAnalysisPlugin.ScreenshotService]
#
# Capture contract:
# - ScreenshotService captures per CaptureProfile (viewport, readiness, masking).
# - Adapter stores original and masked derivative as separate DraftArtifacts.
# - Adapter emits evidence_captured event with artifact refs.
# - VLM analysis results are registered as typed scenario step output, not raw text.
#
# @DATA_CONTRACT CaptureSpec + AgentRun → DraftArtifact[]
# @INVARIANT Original capture is never transmitted externally; only the masked derivative reaches LLM.
# @INVARIANT Every capture carries capture_meta for reproducibility.
# @TEST_EDGE capture_no_mask_selectors -> masked derivative identical to original but still stored separately.
# @TEST_EDGE vlm_call_fails -> evidence artifact remains valid with WARN status; analysis step becomes inconclusive.
# @TEST_EDGE capture_meta_missing_for_screenshot_kind -> 422.
# #endregion AgentRuns.Evidence.Adapter
# #region AgentRuns.Gradio.Emit [C:4] [TYPE Function] [SEMANTICS agent-run,gradio,stream,event]
# @ingroup AgentRuns
# @BRIEF Persist then emit typed run metadata through the existing Gradio stream.
@@ -176,40 +113,4 @@
# @TEST_EDGE backend_event_failure -> emit recoverable error, never unpersisted progress.
# #endregion AgentRuns.Gradio.Emit
## Frontend
// #region AgentRuns.Model [C:5] [TYPE Model] [SEMANTICS agent-run,model,recovery,draft,approval]
// @defgroup AgentRuns Frontend run projection composed by AgentChat.Model.
// @STATE absent | starting | running | waiting_input | waiting_approval | completed | failed | cancelled | disconnected
// @ACTION applyMetadata(meta) — accept typed event when sequence is newer.
// @ACTION recover(runId) — fetch authoritative snapshot after reload/drop.
// @ACTION decideGate(gateId, decision, reason?) — submit HITL decision through existing resume path.
// @INVARIANT Stage state derives from structured metadata/snapshot only, never assistant prose.
// @INVARIANT Drafts are keyed by artifact id and cannot cross run boundaries.
// @RELATION DEPENDS_ON -> [AgentRuns.Api]
// @RELATION BINDS_TO -> [AgentRuns.RunPanel]
// @RATIONALE A composed submodel prevents further uncontrolled growth of AgentChat.Model.
// @REJECTED Duplicate inline run atoms in components — creates divergent recovery state.
// #endregion AgentRuns.Model
<!-- #region AgentRuns.RunPanel [C:4] [TYPE Component] [SEMANTICS agent-run,ui,progress,recovery] -->
<!-- @ingroup AgentRuns -->
<!-- @RELATION BINDS_TO -> [AgentRuns.Model] -->
<!-- @UX_STATE running -> Context header, run id, ordered progress, draft count. -->
<!-- @UX_STATE disconnected -> Last known state plus Recover action; no implied execution restart. -->
<!-- @UX_STATE failed -> Error code/detail and retry-from-dashboard guidance. -->
<!-- @UX_FEEDBACK aria-live polite announces stage transitions once. -->
<!-- @UX_RECOVERY Recover snapshot, retry safe analysis, or return to dashboard. -->
<!-- @UX_TEST reconnect -> snapshot restores stage and drafts without duplicated messages. -->
<!-- #endregion AgentRuns.RunPanel -->
<!-- #region AgentRuns.DraftList [C:4] [TYPE Component] [SEMANTICS agent-run,ui,draft,artifact] -->
<!-- @ingroup AgentRuns -->
<!-- @RELATION BINDS_TO -> [AgentRuns.Model] -->
<!-- @UX_STATE empty | ready | warning | invalid | loading_preview -->
<!-- @UX_FEEDBACK Warnings and invalid markers are visible next to each intended path. -->
<!-- @UX_RECOVERY Preview/download draft remain side-effect free; invalid draft cannot request save. -->
<!-- @UX_TEST invalid_artifact -> Request save disabled and warning linked by aria-describedby. -->
<!-- #endregion AgentRuns.DraftList -->
#endregion AgentTestStabilization.Modules

14
specs/037-superset-baseline-engine/contracts/modules.md
View File

@@ -22,6 +22,8 @@
# @POST Returns stable sorted model, per-resource warnings, capabilities, and fingerprint; missing metadata is never invented.
# @SIDE_EFFECT Async GET calls through SupersetClient.
# @DATA_CONTRACT InspectRequest -> DashboardQueryModel
# @RATIONALE The query model is built from authoritative Superset metadata so scenario and baseline decisions share the same chart/dataset/filter semantics as the dashboard.
# @REJECTED Inferring missing chart or dataset metadata from names or agent prose — rejected because fabricated metadata can produce false executable tests.
# @RELATION DEPENDS_ON -> [Spec.TranslateRequestsHttpx.SupersetClient]
# @TEST_EDGE inaccessible_chart -> warning plus executable=false.
# @TEST_EDGE scoped_filter -> only target charts list filter id.
@@ -35,6 +37,8 @@
# @POST Filters are typed, sorted, scoped, and hashed; invalid targets/operators produce structured validation errors.
# @SIDE_EFFECT None.
# @DATA_CONTRACT FilterInput[] + DashboardQueryModel -> NormalizedFilterContext
# @RATIONALE A canonical filter context gives UI, Superset execution, XLSX comparison, and baseline lookup one stable identity.
# @REJECTED Passing display-formatted filter values directly to execution — rejected because locale and scope differences would create non-reproducible queries.
# @INVARIANT Locale formatting never enters filters_hash.
# @TEST_EDGE locale_decimal -> canonical decimal string.
# @TEST_EDGE filter_outside_chart_scope -> 422.
@@ -63,6 +67,8 @@
# @POST Equivalent locale/display variants normalize identically; ambiguity returns inconclusive reason.
# @SIDE_EFFECT None.
# @DATA_CONTRACT SupersetRawResult + ResultDescriptor -> NormalizedValue
# @RATIONALE Normalization isolates presentation differences before comparison while retaining source metadata for provenance and investigation.
# @REJECTED Comparing rendered strings or binary floats — rejected because locale, timezone, and rounding differences create false diffs.
# @INVARIANT Numeric canonicalization uses Decimal/string, never binary float equality.
# @TEST_EDGE localized_number -> canonical decimal.
# @TEST_EDGE timezone_datetime -> UTC ISO-8601.
@@ -76,12 +82,20 @@
# @POST Returns pass/fail/inconclusive/stale with deterministic diff and no mutation.
# @SIDE_EFFECT None.
# @DATA_CONTRACT NormalizedValue + BaselineEntry -> ComparisonResult
# @RATIONALE Comparison is read-only so a verification run can report truth without silently changing the approved expectation.
# @REJECTED Auto-updating a stale or missing baseline during comparison — rejected because it converts a regression signal into an unreviewed truth change.
# @INVARIANT Missing/stale/unsupported baselines cannot return pass.
# @TEST_EDGE relative_expected_zero -> explicit absolute fallback or inconclusive.
# @TEST_EDGE table_duplicate_keys -> inconclusive unless multiset policy.
# @TEST_EDGE stale_query_fingerprint -> stale_baseline.
# #endregion BaselineEngine.Comparison.Compare
# #region BaselineEngine.Catalog.Load_MOVED [C:1] [TYPE Tombstone] [SEMANTICS baseline,moved]
# @DEPRECATED Lifecycle contract moved to contracts/lifecycle.md.
# @STATUS DEPRECATED -> REPLACED_BY: [BaselineEngine.Catalog.Load]
# @REPLACED_BY BaselineEngine.Catalog.Load
# #endregion BaselineEngine.Catalog.Load_MOVED
# #region BaselineEngine.Catalog.Load [C:4] [TYPE Function] [SEMANTICS baseline,catalog,yaml,validation]
# @ingroup BaselineEngine
# @BRIEF Load and validate reviewable baseline YAML from a resolved repository.

6
specs/038-dashboard-scenario-model/contracts/modules.md
View File

@@ -1,5 +1,7 @@
#region DashboardScenarioModel.Modules [C:5] [TYPE ADR] [SEMANTICS contracts,scenario,graph,validator,compiler]
@BRIEF C3+ contracts for checklist mapping, deterministic graph compilation, validation, resolution, serialization, and safe draft packs.
@RATIONALE The scenario graph is the reviewable intermediate boundary between agent intent and generated artifacts; it centralizes safety, determinism, and capability coverage.
@REJECTED Direct agent-to-script generation — rejected because missing refs, unsafe actions, and baseline truth would be discovered only after artifact generation or runtime.
@RELATION DEPENDS_ON -> [DashboardScenarioModel.DataModel]
@RELATION DEPENDS_ON -> [DashboardScenarioModel.ChecklistCatalog]
@RELATION DEPENDS_ON -> [SupersetBaselineEngine.Modules]
@@ -34,6 +36,8 @@
# @POST Every catalog case is automated, human_checkpoint, unsupported, or needs_context with rationale.
# @SIDE_EFFECT None.
# @DATA_CONTRACT ChecklistCase[] + DashboardCapabilities -> CapabilityMapping[]
# @RATIONALE Capability mapping keeps checklist intent reusable while allowing each dashboard to receive only safe, applicable step templates.
# @REJECTED One-size-fits-all scripts and user-facing low-level tool selection — rejected because capabilities, safety, and available evidence vary per dashboard.
# @INVARIANT No case is dropped and no tool is selected outside its registered capabilities.
# @TEST_EDGE xlsx_unavailable -> C04–C06 manual/unsupported with rationale.
# @TEST_EDGE technical_without_dataset_fields -> human checkpoint, no SQL.
@@ -61,6 +65,8 @@
# @POST Valid is true only with zero errors/blockers; findings are stably ordered and actionable.
# @SIDE_EFFECT None.
# @DATA_CONTRACT DashboardTestScenario -> ScenarioValidationResult
# @RATIONALE Validation is a hard safety boundary between agent-produced intent and artifact generation; deterministic findings give the user a recoverable explanation instead of a runtime surprise.
# @REJECTED Silent graph repair or best-effort artifact generation — rejected because auto-fixing refs, cycles, or unsafe actions can change business intent without review.
# @INVARIANT Cycles, missing/duplicate refs, unregistered tools, SQL, raw metric truth, and path traversal block compilation.
# @TEST_EDGE cycle -> error contains cycle path.
# @TEST_EDGE duplicate_output -> both producer ids reported.

38
specs/039-dashboard-scenario-ui/contracts/modules.md
View File

@@ -1,5 +1,7 @@
#region DashboardScenarioUi.Modules [C:5] [TYPE ADR] [SEMANTICS contracts,dashboard-testing,scenario,ux]
@BRIEF Model, API client, entry action, workspace, parameter, baseline, artifact, and confirmation UI contracts.
@RATIONALE 039 owns presentation and interaction state while 036–038 own run, graph, and artifact semantics; this boundary prevents UI components from reimplementing backend truth.
@REJECTED Component-local fetching and business state — rejected because independent panels could render different revisions or bypass the shared HITL gate.
@RELATION DEPENDS_ON -> [DashboardScenarioUi.DataModel]
@RELATION DEPENDS_ON -> [AgentRuns.Model]
@RELATION DEPENDS_ON -> [ScenarioGraph.Api]
@@ -109,7 +111,7 @@
<!-- @RELATION DEPENDS_ON -> [AgentRuns.DraftList] -->
<!-- @BRIEF View captured screenshots, review VLM findings, and record dispositions. -->
<!-- @UX_STATE empty | loading | ready | reviewing_finding. -->
<!-- @UX_LAYOUT Screenshot viewer (main) + finding list (sidebar) + finding detail card (overlay or inline below viewer). -->
<!-- Layout: Screenshot viewer (main) + finding list (sidebar) + finding detail card (overlay or inline below viewer). -->
<!-- @UX_FEEDBACK Selected finding highlights region on screenshot via overlay; provenance footer shows model, prompt version, hash, timestamp. -->
<!-- @UX_RECOVERY Confirm/dismiss/inconclusive are typed buttons; no free-text state machine. -->
<!-- @UX_TEST three_findings_different_severity -> all visible; confirmed/dismissed/inconclusive states distinguishable. -->
@@ -117,40 +119,6 @@
<!-- @INVARIANT Disposition never changes scenario graph; it records audit state only. -->
<!-- #endregion DashboardTesting.EvidencePanel -->
<!-- #region ReleaseVerification.StatusBadge [C:2] [TYPE Component] [SEMANTICS verification,status,badge,release] -->
<!-- @defgroup ReleaseVerification Pipeline-driven verification views. No AgentRun, no WorkspaceModel, no conversation. -->
<!-- @BRIEF Compact pass/warn/fail/blocked badge bound to VerificationRun.overall_status. -->
<!-- @LAYER Presentation — pure projection of 037 VerificationRun data. -->
<!-- @RELATION DEPENDS_ON -> [DashboardTesting.ApiClient] -->
<!-- @UX_STATE pass (green check) | warn (amber ⚠) | fail (red ✕) | blocked (red ⛔) | pending (grey …) -->
<!-- @UX_FEEDBACK Tooltip shows summary text, trigger, and timestamp. -->
<!-- @INVARIANT No AgentRun or WorkspaceModel dependency. Color + icon + text. -->
<!-- #endregion ReleaseVerification.StatusBadge -->
<!-- #region ReleaseVerification.StructureDiffPanel [C:4] [TYPE Component] [SEMANTICS verification,structure,diff,release] -->
<!-- @ingroup ReleaseVerification -->
<!-- @BRIEF Display structural changes between releases grouped by severity with confirm/dismiss controls. -->
<!-- @LAYER Page-level — used on PREPROD deployment tab and release detail page. -->
<!-- @RELATION DEPENDS_ON -> [DashboardTesting.ApiClient] -->
<!-- @UX_STATE loading | ready | reviewing. -->
<!-- @UX_LAYOUT Three severity groups (critical/warning/info), each collapsible. Per-change: target, kind, before/after, rationale, affected artifacts, confirm/dismiss. -->
<!-- @UX_FEEDBACK CRITICAL changes must be resolved (confirmed or dismissed) before pipeline gate advances. Disposition is audit-only, not graph-modifying. -->
<!-- @UX_TEST filter_scope_change_critical -> red border, confirm/dismiss required, Validate button disabled until all critical resolved. -->
<!-- @INVARIANT Confirm/dismiss actions write to ReleaseVerificationState.structureDispositions (local audit), never to scenario graph or baseline entries. -->
<!-- #endregion ReleaseVerification.StructureDiffPanel -->
<!-- #region ReleaseVerification.HistoryList [C:3] [TYPE Component] [SEMANTICS verification,history,list,release] -->
<!-- @ingroup ReleaseVerification -->
<!-- @BRIEF Chronological list of VerificationRun entries for a dashboard. -->
<!-- @LAYER Page-level — used on dashboard detail page. -->
<!-- @RELATION DEPENDS_ON -> [DashboardTesting.ApiClient] -->
<!-- @UX_STATE empty | loading | ready. -->
<!-- @UX_LAYOUT Table: date, trigger (icon + label), environment, overall_status (badge), summary, link to full run. -->
<!-- @UX_FEEDBACK Scheduled runs show next-run countdown; ETL-triggered runs show ETL event reference; manual runs show user avatar. -->
<!-- @UX_TEST three_runs_different_triggers -> correct trigger icons and status badges. -->
<!-- @INVARIANT Read-only projection. No mutation actions. -->
<!-- #endregion ReleaseVerification.HistoryList -->
<!-- #region DashboardTesting.ConfirmationBinding [C:4] [TYPE Component] [SEMANTICS dashboard-testing,confirmation,hitl] -->
<!-- @ingroup DashboardTesting -->
<!-- @RELATION DEPENDS_ON -> [AgentChat.ConfirmationCard] -->

42
specs/039-dashboard-scenario-ui/contracts/release-verification-modules.md Normal file
View File

@@ -0,0 +1,42 @@
#region DashboardScenarioUi.ReleaseVerificationModules [C:4] [TYPE ADR] [SEMANTICS contracts,verification,release,pipeline,ux]
@BRIEF Pipeline verification components split from DashboardScenarioUi.Modules to preserve bounded ATTN_4 contracts.
@RELATION DEPENDS_ON -> [DashboardScenarioUi.DataModel]
@RELATION DEPENDS_ON -> [DashboardTesting.ApiClient]
@RATIONALE Pipeline views consume VerificationRun data without AgentRun/Workspace state and therefore form a separate component boundary.
@REJECTED Keeping agent workspace and pipeline components in one aggregate contract — rejected because it exceeded the 150-line visibility boundary and blurred runtime ownership.
<!-- #region ReleaseVerification.StatusBadge [C:2] [TYPE Component] [SEMANTICS verification,status,badge,release] -->
<!-- @defgroup ReleaseVerification Pipeline-driven verification views. No AgentRun, no WorkspaceModel, no conversation. -->
<!-- @BRIEF Compact pass/warn/fail/blocked badge bound to VerificationRun.overall_status. -->
<!-- @LAYER UI -->
<!-- @RELATION DEPENDS_ON -> [DashboardTesting.ApiClient] -->
<!-- @UX_STATE pass (green check) | warn (amber warning) | fail (red failure) | blocked (red block) | pending (grey pending) -->
<!-- @UX_FEEDBACK Tooltip shows summary text, trigger, and timestamp. -->
<!-- @INVARIANT No AgentRun or WorkspaceModel dependency. Color + icon + text. -->
<!-- #endregion ReleaseVerification.StatusBadge -->
<!-- #region ReleaseVerification.StructureDiffPanel [C:4] [TYPE Component] [SEMANTICS verification,structure,diff,release] -->
<!-- @ingroup ReleaseVerification -->
<!-- @BRIEF Display structural changes between releases grouped by severity with confirm/dismiss controls. -->
<!-- @LAYER UI -->
<!-- @RELATION DEPENDS_ON -> [DashboardTesting.ApiClient] -->
<!-- @UX_STATE loading | ready | reviewing. -->
<!-- Layout: Three severity groups (critical/warning/info), each collapsible. Per-change: target, kind, before/after, rationale, affected artifacts, confirm/dismiss. -->
<!-- @UX_FEEDBACK CRITICAL changes must be resolved before pipeline gate advances; disposition is audit-only. -->
<!-- @UX_TEST filter_scope_change_critical -> red border, confirm/dismiss required, Validate disabled until resolved. -->
<!-- @INVARIANT Dispositions never mutate scenario graph or baseline entries. -->
<!-- #endregion ReleaseVerification.StructureDiffPanel -->
<!-- #region ReleaseVerification.HistoryList [C:3] [TYPE Component] [SEMANTICS verification,history,list,release] -->
<!-- @ingroup ReleaseVerification -->
<!-- @BRIEF Chronological list of VerificationRun entries for a dashboard. -->
<!-- @LAYER UI -->
<!-- @RELATION DEPENDS_ON -> [DashboardTesting.ApiClient] -->
<!-- @UX_STATE empty | loading | ready. -->
<!-- Layout: date, trigger, environment, status badge, summary, full-run link. -->
<!-- @UX_FEEDBACK Scheduled runs show next-run countdown; ETL runs show event reference; manual runs show actor. -->
<!-- @UX_TEST three_runs_different_triggers -> correct trigger icons and status badges. -->
<!-- @INVARIANT Read-only projection; no mutation actions. -->
<!-- #endregion ReleaseVerification.HistoryList -->
#endregion DashboardScenarioUi.ReleaseVerificationModules

45
specs/040-dashboard-load-testing/checklists/requirements.md Normal file
View File

@@ -0,0 +1,45 @@
# Requirements Checklist: Dashboard Load Testing
**Purpose**: Validate spec.md completeness, clarity, and testability before `/speckit.plan`.
**Created**: 2026-07-22
**Feature**: [spec.md](../spec.md)
## Content Quality
- [x] CHK001 Spec is user/operator-focused; no framework/library implementation leakage (executor referenced only as "037 Superset-native path" contract boundary)
- [x] CHK002 All five stories are independently testable with a stated Independent Test
- [x] CHK003 Priorities assigned P1/P2 with justification per story
- [x] CHK004 Key entities defined without schema/implementation detail
## Requirement Quality
- [x] CHK005 Every FR is testable (cap enforcement, zero baseline mutation, deterministic expansion, gate blocking)
- [x] CHK006 FR IDs hierarchical and stable (`LOAD-FR-001..015`)
- [x] CHK007 No `[NEEDS CLARIFICATION]` markers remain
- [x] CHK008 Rejected paths recorded in header `@REJECTED` (AgentRun reuse, baseline pollution, unbounded concurrency)
## Dependency Traceability
- [x] CHK009 037 dependencies explicit: execution path, normalization, error taxonomy (LOAD-FR-001/006/012)
- [x] CHK010 036 dependencies explicit: approval gate semantics, run recovery, permission_denied (LOAD-FR-005/008/014)
- [x] CHK011 Blast-radius requirement (LOAD-FR-009) consistent with gap analysis BR-1 (reverse dataset→dashboard index)
- [x] CHK012 No conflict with 038/039 scopes (no AgentRun, no scenario graph, no baseline writes)
## Safety & Risk
- [x] CHK013 Circuit breaker mandatory with typed abort (LOAD-FR-004, US3)
- [x] CHK014 PROD gate + RBAC split `execute` vs `prod` (LOAD-FR-005/014)
- [x] CHK015 Baseline isolation verifiable via catalog hash (LOAD-FR-003, SC-003)
- [x] CHK016 Schedule overlap with deployment windows of dependent dashboards (LOAD-FR-013)
## Success Criteria
- [x] CHK017 All SC measurable (100% cap enforcement, zero mutations, byte-determinism, 0 false positives)
- [x] CHK018 SC-004 determinism matches 038 determinism convention
## Clarifications Integrated (2026-07-22)
- [x] CHK019 ~~max_concurrent defaults~~ — CLOSED: worker pool + env semaphore; prod=5, default=10, ceiling=25, per-env override clamped (LOAD-FR-002/016/017)
- [x] CHK022 GIL analysis accepted: async single-loop; bounded normalization in load path (hash+count+sample), full normalization stays in 037 (LOAD-FR-018); ProcessPool offload deferred pending fixture evidence
- [x] CHK020 ~~Cache-state source~~ — CLOSED after Superset source audit: authoritative chart-data response body fields (`is_cached`, `cache_key`, `cached_dttm`, `queried_dttm`, `cache_timeout`); latency inference rejected (LOAD-FR-012/019)
- [x] CHK021 ~~Blast-radius reverse index~~ — CLOSED by 041 LIN-FR-013: pinned read model, fingerprint query-param

136
specs/040-dashboard-load-testing/contracts/modules.md Normal file
View File

@@ -0,0 +1,136 @@
#region DashboardLoadTesting.Modules [C:4] [TYPE ADR] [SEMANTICS contracts,modules,load-testing,workers]
@BRIEF GRACE contracts for profile validation, variation expansion, worker pool execution, timing, cache provenance, aggregation, and API/UI surfaces.
@RELATION DEPENDS_ON -> [DashboardLoadTesting.DataModel]
@RELATION DEPENDS_ON -> [DashboardLoadTesting.Research]
@RELATION DEPENDS_ON -> [DatasetLineageBlastRadius.Spec]
@RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec]
@RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
## Backend Contracts
```python
# #region LoadTesting.Profile.Validate [C:4] [TYPE Function] [SEMANTICS load-testing,profile,validation]
# @ingroup LoadTesting
# @BRIEF Validate load profile against authoritative dashboard model, environment policy, and 041 blast-radius snapshot.
# @PRE DashboardQueryModel and pinned 041 fingerprint are readable; actor has execute permission.
# @POST Returns validated profile, effective cap, matrix preview, warnings, and gate requirements; no execution dispatched.
# @SIDE_EFFECT Read-only metadata calls.
# @DATA_CONTRACT ProfileInput + DashboardQueryModel + BlastRadiusReport -> ValidatedLoadProfile
# @TEST_EDGE requested concurrency over cap -> clamp notice; unknown variation value -> NEEDS_CONTEXT.
# @TEST_EDGE stale fingerprint -> refresh/reconfirm required.
# @REJECTED Client-supplied cap override -> server policy wins.
# #endregion LoadTesting.Profile.Validate
# #region LoadTesting.Matrix.Expand [C:4] [TYPE Function] [SEMANTICS load-testing,matrix,deterministic]
# @ingroup LoadTesting
# @BRIEF Expand closed variation axes cartesian up to cap, then seeded reservoir-sample.
# @PRE Axes are filters, viewport, role, or time_range; values are normalized and validated.
# @POST Returns stable variation ids, theoretical count, selected count, seed, and total execution estimate.
# @SIDE_EFFECT None.
# @INVARIANT Same inputs and seed yield byte-identical matrix; unordered input axes cannot change output.
# @DATA_CONTRACT NormalizedAxes + Seed + MaxVariations -> VariationMatrix
# @TEST_EDGE unknown axis -> 422; product over cap -> deterministic selected subset.
# #endregion LoadTesting.Matrix.Expand
# #region LoadTesting.RunnerPool [C:5] [TYPE Module] [SEMANTICS load-testing,workers,concurrency]
# @defgroup LoadTesting Bounded dashboard load-testing execution domain.
# @BRIEF Runs one queued LoadRun with async workers and environment-level capacity coordination.
# @PRE Validated profile, approved PROD gate when required, pinned blast-radius fingerprint.
# @POST Worker count never exceeds effective cap; terminal status and partial results are durable; one TaskManager task owns run.
# @SIDE_EFFECT Superset chart-data calls, batched DB writes, progress events, TaskManager task state.
# @INVARIANT Acquire load semaphore before request reaches shared client semaphore; never manually re-acquire shared semaphore.
# @INVARIANT Breaker/stop prevents new queue intake; in-flight requests drain within deadline.
# @DATA_CONTRACT ValidatedLoadProfile -> LoadRun + LoadExecution[] + LoadRunAggregate
# @RELATION CALLS -> [SupersetClient.ChartData.Execute]
# @RELATION DEPENDS_ON -> [Core.Manager.CreateTask]
# @RELATION DEPENDS_ON -> [Core.ClientRegistry.GetSemaphore]
# @RATIONALE Async workers directly model executions; HTTP pool alone hides queue pressure.
# @REJECTED One TaskManager task per execution -> task/event flood. Thread/process pool -> unnecessary for I/O-dominant v1.
# #endregion LoadTesting.RunnerPool
# #region LoadTesting.RunnerPool.ExecuteWorker [C:5] [TYPE Function] [SEMANTICS load-testing,worker,timing]
# @ingroup LoadTesting
# @BRIEF Take queue items, record queue/resource/upstream timing, execute chart data, and persist bounded result metadata.
# @PRE Worker owns a queue item and breaker is not tripped; shared request path is available.
# @POST One LoadExecution is produced with disjoint timings, cache metadata, digest/count/sample, or typed error.
# @SIDE_EFFECT Network I/O; batch persistence; progress aggregation.
# @RATIONALE One worker owns one queue item at a time so queue wait, resource wait, and upstream latency remain disjoint and measurable.
# @REJECTED Full 037 normalization inside the worker — rejected because large table payloads can hold the GIL and distort measured Superset latency; load path uses bounded digest/count/sample processing.
# @TEST_EDGE cancellation before take -> item remains unexecuted; timeout -> error taxonomy + no synthetic percentile sample.
# @TEST_EDGE large table -> processing respects 100-row/256KiB bound and does not call full 037 normalization.
# #endregion LoadTesting.RunnerPool.ExecuteWorker
# #region LoadTesting.Capacity.Resolve [C:4] [TYPE Function] [SEMANTICS load-testing,capacity,environment]
# @ingroup LoadTesting
# @BRIEF Resolve effective cap from stage, profile, per-env override, absolute ceiling, pool size, and reserved slots.
# @PRE Environment has valid stage and connection_pool_size >= 1.
# @POST Returns cap and clamp reasons; cap >= 1 or rejects an environment with no safe load slot.
# @SIDE_EFFECT None.
# @INVARIANT PROD default=5, DEV/PREPROD default=10, absolute ceiling=25; reserve 5 shared slots.
# @TEST_EDGE pool_size=5 -> load cap rejected/disabled rather than starving ordinary traffic.
# #endregion LoadTesting.Capacity.Resolve
# #region LoadTesting.Cache.MapResponse [C:3] [TYPE Function] [SEMANTICS load-testing,cache,superset]
# @ingroup LoadTesting
# @BRIEF Map Superset chart-data response metadata to authoritative cache state.
# @PRE Response result includes raw fields when supported; request force flag is known.
# @POST Returns hit/miss/bypassed/disabled/unknown plus preserved raw cache metadata.
# @SIDE_EFFECT None.
# @INVARIANT Latency is never used to infer cache state.
# @TEST_EDGE is_cached=null + force=false + cached_dttm=null -> miss; force=true -> bypassed; absent fields -> unknown.
# #endregion LoadTesting.Cache.MapResponse
# #region LoadTesting.Result.BoundedProcess [C:4] [TYPE Function] [SEMANTICS load-testing,result,bounded,gil]
# @ingroup LoadTesting
# @BRIEF Hash raw response, count rows, and retain bounded sample without full verification normalization.
# @PRE Raw response available; sample limits configured.
# @POST Digest represents raw payload; sample ≤100 rows and ≤256KiB; no full 037 normalization invoked.
# @SIDE_EFFECT CPU work on event loop bounded by payload policy.
# @INVARIANT ProcessPool fallback requires explicit escalation after >10% measured distortion.
# #endregion LoadTesting.Result.BoundedProcess
# #region LoadTesting.Breaker.Evaluate [C:4] [TYPE Function] [SEMANTICS load-testing,circuit-breaker,metrics]
# @ingroup LoadTesting
# @BRIEF Evaluate rolling completed window for error rate and upstream p99 thresholds.
# @PRE At least minimum sample window or breaker remains armed without latency decision.
# @POST Tripped result closes intake and enters drain; trigger metric and threshold are durable.
# @SIDE_EFFECT Updates run circuit state and emits terminal progress.
# @TEST_EDGE <20 samples -> no p99 abort; error threshold breach -> abort within one evaluation window.
# #endregion LoadTesting.Breaker.Evaluate
# #region LoadTesting.Api.Routes [C:4] [TYPE Module] [SEMANTICS load-testing,api,rbac]
# @ingroup LoadTesting
# @BRIEF REST routes for profile validation, matrix preview, start/stop, status, results, and comparison.
# @LAYER API
# @INVARIANT DEV/PREPROD needs dashboard:loadtest:execute; PROD additionally needs dashboard:loadtest:prod + 036 gate.
# @DATA_CONTRACT DTOs are extra-forbid; no raw SQL, endpoint, arbitrary query context, or repository path.
# @RELATION BINDS_TO -> [EXT:frontend:frontend/src/lib/types/load-testing.ts]
# #endregion LoadTesting.Api.Routes
```
## Frontend Screen Models
```typescript
// #region Dashboards.LoadProfileModel [C:4] [TYPE Model] [SEMANTICS load-testing,profile,ux]
// @BRIEF Model-first editor for profile validation, variation matrix, cap notices, and blast-radius snapshot.
// @STATE idle|validating|matrix_ready|gate_required|start_error
// @ACTION validate(), expandMatrix(), start()
// @UX_STATE invalid variation -> inline NEEDS_CONTEXT; stale fingerprint -> refresh/reconfirm.
// @UX_RECOVERY Fix/drop variation, refresh 041 snapshot, retry start with same revision.
// @RELATION BINDS_TO -> [LoadTesting.Api.Routes]
// #endregion Dashboards.LoadProfileModel
// #region Dashboards.LoadRunModel [C:5] [TYPE Model] [SEMANTICS load-testing,run,progress]
// @BRIEF Recoverable live run state keyed by load_run_id: phase, workers, queue, metrics, findings, partial results.
// @STATE queued|ramping|steady|draining|completed|stopped_by_user|circuit_breaker_abort|failed
// @ACTION stop(), reconnect(), compare()
// @UX_FEEDBACK Worker busy/idle, queue depth, breaker state, p50/p95/p99 upstream latency, cache counts.
// @RATIONALE A dedicated run model keeps high-volume worker progress and partial results outside ordinary AgentRun/chat state while retaining reconnect semantics.
// @REJECTED Reusing AgentRunModel or deriving status from chat events — rejected because load executions are non-conversational and can outlive the browser stream.
// @UX_RECOVERY Browser disconnect reloads authoritative snapshot; abort preserves partial results.
// @INVARIANT Ordinary chat and 036 AgentRun state never initialize this model.
// @RELATION BINDS_TO -> [LoadTesting.Api.Routes]
// #endregion Dashboards.LoadRunModel
```
#endregion DashboardLoadTesting.Modules

140
specs/040-dashboard-load-testing/data-model.md Normal file
View File

@@ -0,0 +1,140 @@
#region DashboardLoadTesting.DataModel [C:3] [TYPE ADR] [SEMANTICS data-model,load-testing,workers,metrics]
@BRIEF Phase 1 data model for load profiles, deterministic variations, worker runs, execution timing, cache provenance, and blast-radius snapshots.
**Feature**: 040-dashboard-load-testing | **Date**: 2026-07-22
## LoadProfile
| Field | Type | Invariant |
|---|---|---|
| id | UUID | stable reusable profile id |
| dashboard_id | int | authoritative dashboard |
| environment_id | str | target environment |
| revision | int | increments on profile mutation |
| concurrency_requested | int | positive; clamped to effective cap |
| execution_mode | enum(iterations,duration) | exactly one mode |
| iterations | int? | required for iterations |
| duration_seconds | int? | required for duration |
| ramp_steps | list[int] | monotonic, ends at effective target |
| variation_axes | JSONB | closed axes only: filters, viewport, role, time_range |
| circuit_breaker | JSONB | error threshold, p99 multiplier, min samples, window |
| matrix_seed | int | deterministic expansion seed |
| max_variations | int | default 500; server bounded |
| enabled | bool | schedule control |
| created_by | str | audit owner |
## LoadVariation
Expanded immutable coordinate stored with a run:
```json
{
"variation_id": "sha256(canonical_coordinates)",
"filters": {"region": "north"},
"viewport": {"width": 1366, "height": 768, "device_scale_factor": 1},
"role": "analyst",
"time_range": {"from": "2026-01-01", "to": "2026-01-31"}
}
```
Unknown axes fail validation. Above-cap matrices use seeded reservoir sampling; selected count and theoretical count are both retained.
## LoadRun
| Field | Type | Notes |
|---|---|---|
| id | UUID | `load_run_id` |
| profile_id | UUID | profile revision binding |
| profile_revision | int | immutable gate input |
| environment_id | str | |
| dashboard_id | int | |
| status | enum(queued,ramping,steady,draining,completed,stopped_by_user,circuit_breaker_abort,failed) | terminal status immutable |
| phase | enum(ramp,steady,drain,terminal) | |
| effective_concurrency | int | after cap/reserve/clamp |
| matrix_seed | int | |
| theoretical_variations | int | |
| selected_variations | int | |
| total_executions | int | selected variations × charts × iterations/duration policy |
| blast_radius_fingerprint | str | 041 pinned snapshot |
| blast_radius_report | JSONB | dependent dashboard projection at start |
| approval_gate_id | UUID? | required for PROD |
| task_id | UUID | exactly one TaskManager task |
| circuit_state | enum(inactive,armed,tripped) | |
| started_at/finished_at | datetime? | |
| stop_reason | str? | typed terminal reason |
## LoadExecution
| Field | Type | Notes |
|---|---|---|
| id | UUID | |
| run_id | UUID FK | |
| variation_id | str | immutable coordinate |
| chart_id | int | |
| filters_hash | str | 037 canonical hash |
| outcome | enum(success,error,timeout,cancelled) | |
| error_taxonomy | str? | 037 categories |
| queue_wait_ms | int | enqueue → worker take |
| resource_wait_ms | int | worker take → shared request admission |
| upstream_latency_ms | int | admission → response/error |
| end_to_end_ms | int | sum of three timings |
| response_sha256 | str? | digest of raw response bytes |
| row_count | int? | bounded metadata |
| sample_rows | JSONB? | first N rows, bounded to 100 / 256 KiB |
| cache_state | enum(hit,miss,bypassed,disabled,unknown) | Superset body mapping |
| is_cached | bool? | raw Superset field |
| cache_key | str? | raw field |
| cached_dttm/queried_dttm | datetime? | raw fields |
| cache_timeout | int? | raw field |
| cache_state_source | str | `superset_response` when body fields exist |
| worker_id | str | registry observability |
| started_at/finished_at | datetime | monotonic timings also emitted |
## LoadRunAggregate
Per run and per chart/variation aggregation:
```json
{
"chart_id": 42,
"variation_id": "...",
"success_count": 19,
"error_count": 1,
"p50_upstream_ms": 240,
"p90_upstream_ms": 520,
"p95_upstream_ms": 670,
"p99_upstream_ms": 920,
"p95_queue_wait_ms": 18,
"p95_resource_wait_ms": 12,
"throughput_per_second": 4.8,
"cache_counts": {"hit": 15, "miss": 5}
}
```
Percentiles use nearest-rank over successful samples only. Error rate is separate.
## ConsistencyFinding
`{id, run_id, chart_id, coordinate_key, first_response_sha256, divergent_response_sha256, execution_ids, severity, classification="flakiness", created_at}`.
A row-order-only difference is normalized before comparison when the chart declares unordered table semantics; otherwise raw response digest divergence is reported.
## CircuitBreakerPolicy
`{error_rate_threshold=0.25, p99_multiplier=3.0, min_samples=20, window_size=100, absolute_p99_ms?}`. A latency threshold without a comparison reference requires explicit `absolute_p99_ms`.
## BlastRadiusReport
Pinned 041 projection: `{fingerprint, stale_index, datasets: [{dataset_id, dependent_dashboards}], dependent_dashboard_count, probe_coverage}`. A run stores the snapshot it was approved against; later index changes do not rewrite it.
## Invariants
1. Effective worker concurrency never exceeds environment cap, profile cap, absolute ceiling, or reserved-slot-adjusted shared pool.
2. One LoadRun has exactly one TaskManager task; executions are not tasks.
3. Queue/resource/upstream timings are disjoint; performance percentiles use upstream latency only.
4. Cache state comes from Superset response metadata; latency inference is forbidden.
5. Load executions never mutate 037 baseline catalogs, candidates, source hashes, or immutability status.
6. Terminal status is immutable; partial executions remain queryable after stop or breaker abort.
7. `response_sha256` is calculated before bounded JSON/sample processing where raw response bytes are available.
#endregion DashboardLoadTesting.DataModel

135
specs/040-dashboard-load-testing/plan.md Normal file
View File

@@ -0,0 +1,135 @@
# Implementation Plan: Dashboard Load Testing
**Branch**: `040-dashboard-load-testing` | **Date**: 2026-07-22 | **Spec**: [spec.md](spec.md)
**Input**: Clarified feature specification from `/specs/040-dashboard-load-testing/spec.md`
## Summary
Build a dashboard load-testing surface around bounded async workers, not raw HTTP connection count. A LoadRun is one TaskManager task with a per-run execution queue; an environment capacity registry and semaphore coordinate all active runs. The runner uses the 037 Superset-native chart-data path, preserves Superset cache metadata from the JSON response, computes queue/resource/upstream timings separately, uses bounded result processing to avoid GIL distortion, and stores a pinned 041 blast-radius snapshot for configuration and PROD approval. Load runs are strictly isolated from baseline mutation.
## Technical Context
**Language/Version**: Python 3.13+ backend; TypeScript frontend with Svelte 5 runes-only
**Primary Dependencies**: FastAPI 0.126, SQLAlchemy 2.0.45, APScheduler 3.11.2 for scheduled entry boundaries, httpx via existing AsyncAPIClient, Pydantic 2.x; SvelteKit 2.49, Svelte 5.56, Vite, Tailwind; no new runtime dependency
**Storage**: PostgreSQL 16; new load profile/run/execution/aggregate/finding tables plus existing TaskManager persistence; no baseline catalog writes
**Testing**: pytest unit/contract/integration tests; Vitest L1 Screen Model tests and L2 UX tests; Playwright for worker monitor/profile flow; ruff and frontend lint
**Target Platform**: Linux Docker deployment, modern desktop browsers
**Project Type**: FastAPI REST/WebSocket backend + SvelteKit SPA frontend
**Frontend Architecture**: `.svelte.ts` Screen Models, `$state/$derived/$effect`, typed DTOs, components bind to models; ordinary AgentChat and 036 AgentRun remain separate
**Performance Goals**: upstream p95/p99 measured independently from queue/resource wait; progress events ≤4 Hz; result flush ≤50 rows or 1s; no more than effective worker cap in flight; profile/matrix preview <200ms after metadata available
**Constraints**: PROD default cap=5, DEV/PREPROD=10, absolute ceiling=25, reserve 5 shared pool slots; direct SQL/raw query context forbidden; bounded sample 100 rows/256 KiB; 041 fingerprint required for blast-radius binding; RBAC default-deny; no implementation phase in this planning request
**Scale/Scope**: 100 dashboards/environment, 300 charts, 25 maximum load workers per environment, 500 selected variation combinations per profile by default
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* — **PASS with one explicit implementation prerequisite**
| Principle | Status | Evidence / Guardrail |
|---|---|---|
| I. Semantic Contract First | ✅ | C3–C5 contracts in `contracts/modules.md`, hierarchical `LoadTesting.*` IDs and cross-stack edges |
| II. Decision Memory | ✅ | Spec `@RATIONALE/@REJECTED`; research R1–R10; HTTP pool, per-execution tasks, latency inference, full normalization rejected |
| III. External Orchestrator | ✅ | 037 Superset-native adapter remains sole execution boundary; no Superset plugin or SQL path |
| IV. Module Discipline | ✅ | Runner, capacity, cache, bounded processing, breaker separated; target each module <400 LOC and CC≤10 |
| V. RBAC Enforcement | ✅ | `dashboard:loadtest:execute` and `dashboard:loadtest:prod`; gate hash includes profile/cap/fingerprint |
| VI. Svelte 5 Runes Only | ✅ | LoadProfileModel/LoadRunModel `.svelte.ts`, no legacy stores in new surface |
| VII. Test-Driven C3+ | ✅ | Quickstart is falsifiable; C4/C5 contracts require rejection-path tests and GIL/cache/cap edge tests |
| VIII. Attention Optimization | ✅ | Anchor density, dotted IDs, shared `load-testing` semantic tags, bounded contract blocks |
| Repository prerequisite | ⚠️ MUST FIX DURING IMPLEMENTATION | `client_registry.get_client()` creates per-env semaphore but does not pass it to `AsyncAPIClient`; without this, `connection_pool_size` is declarative only. This is a foundational task, not a silent assumption. |
**Post-Phase-1 re-check**: PASS — the prerequisite is explicitly represented in traceability and must be verified before load execution.
## Project Structure
### Documentation
```text
specs/040-dashboard-load-testing/
├── spec.md
├── ux_reference.md
├── checklists/requirements.md
├── plan.md
├── research.md
├── data-model.md
├── contracts/modules.md
├── quickstart.md
└── traceability.md
```
### Source Code (implementation target; not changed in this planning phase)
```text
backend/src/
├── models/load_testing.py
├── schemas/load_testing.py
├── services/load_testing/
│ ├── capacity.py
│ ├── profile.py
│ ├── matrix.py
│ ├── runner_pool.py
│ ├── timing.py
│ ├── cache_metadata.py
│ ├── bounded_result.py
│ ├── breaker.py
│ ├── aggregates.py
│ └── comparison.py
├── api/routes/load_testing.py
├── core/utils/client_registry.py # pass existing semaphore into AsyncAPIClient
└── plugins/load_testing.py # one TaskManager task per LoadRun
frontend/src/
├── lib/types/load-testing.ts
├── lib/api/load-testing.ts
├── lib/models/Dashboards.LoadProfileModel.svelte.ts
├── lib/models/Dashboards.LoadRunModel.svelte.ts
├── lib/components/dashboards/load-testing/
└── routes/load-runs/[id]/
```
**Structure Decision**: Backend domain under `services/load_testing/`, separate from 037 correctness engine. `core/utils/client_registry.py` gets only the semaphore wiring prerequisite; no shared-client replacement because auth/CSRF session continuity is an existing invariant. Frontend is a dedicated workspace/monitor, not Agent Workspace.
## Phase 0 Outputs
| Artifact | Decision |
|---|---|
| `research.md` R1 | Async worker queue + env capacity registry; one TaskManager task per run |
| R2 | Effective cap derives from stage, override, pool size minus 5 reserved slots, ceiling 25; existing semaphore wiring prerequisite |
| R3 | Separate queue/resource/upstream timing; upstream percentiles only |
| R4 | Superset response-body cache metadata authoritative |
| R5 | Raw digest + bounded sample/count; no full 037 normalization in load path |
| R6 | Batched persistence and ≤4 Hz progress |
| R7 | Minimum-20 rolling breaker window; error/p99 thresholds |
| R8 | Deterministic matrix and seeded sampling |
| R9 | Pinned 041 blast-radius snapshot |
| R10 | Stage/RBAC/approval gate hash semantics |
## Phase 1 Outputs
| Artifact | Coverage |
|---|---|
| `data-model.md` | Profile, variation, run, execution, aggregates, findings, breaker, blast radius, invariants |
| `contracts/modules.md` | Backend C3–C5 contracts and frontend Screen Models with UX states |
| `quickstart.md` | 10-step falsifiable verification sequence |
| `traceability.md` | LOAD-FR-001..019 to contracts and tests; cross-spec dependencies |
## Complexity Tracking
No constitution violations. The repository semaphore wiring is an explicit prerequisite, not a justification for bypassing the constitution.
## ADR Continuity
- ADR-0001: `services/load_testing/`, `models/load_testing.py`, `api/routes/load_testing.py` follow module layout.
- ADR-0003: all Superset interaction remains in existing client boundary.
- ADR-0005: new permissions are explicit and default-deny; PROD uses 036 gate.
- ADR-0006: new UI uses Svelte 5 Runes and model-first components.
- ADR-0011: async worker execution stays on FastAPI event loop; no `asyncio.run()` and no blocking `AsyncJobRunner.run()` from the event-loop thread.
- 036/037/041 continuity: one TaskManager run, 037 chart-data semantics, 041 fingerprint pinning; no baseline mutation.
## Verification Gates (implementation phase only)
1. Backend load-testing unit/contract tests and existing client-registry regressions.
2. `python -m ruff check .`.
3. Frontend Vitest/L2 tests, lint, build, and Playwright profile/run monitor flow.
4. Quickstart steps 1–10.
5. Semantic contract audit and runtime instrumentation audit for C4/C5 runner/breaker flows.
#endregion DashboardLoadTesting.Plan

35
specs/040-dashboard-load-testing/quickstart.md Normal file
View File

@@ -0,0 +1,35 @@
#region DashboardLoadTesting.Quickstart [C:2] [TYPE ADR] [SEMANTICS quickstart,load-testing,verification]
@BRIEF Falsifiable verification sequence for 040 without implementing application code.
**Feature**: 040-dashboard-load-testing | **Date**: 2026-07-22
## Fixture Setup
1. Dashboard fixture: 2 charts, one scalar and one bounded table; 2 filter values; 2 viewport values; 2 repeated identical coordinates.
2. Environment fixtures: DEV pool=20, PREPROD pool=20, PROD pool=10; one environment pool=5 edge.
3. Superset response fixtures: `is_cached=true`, ordinary source (`is_cached=null`), `force=true`, `cache_timeout=-1`, absent/inconsistent metadata, error taxonomy.
4. 041 fixtures: shared dataset with 3 dependent dashboards and pinned fingerprint.
## Verification Sequence
1. **Profile and matrix**: validate profile → effective cap, exact request estimate, deterministic matrix; shuffled axes yield same bytes; over-cap uses seeded sample.
2. **Capacity**: DEV/PREPROD default cap=10, PROD=5, ceiling=25, pool reserve=5; pool=5 rejects load slot; two active runs share env capacity and total workers never exceed cap.
3. **Worker timing**: inject queue delay, semaphore delay, and upstream delay; assert `queue_wait_ms`, `resource_wait_ms`, `upstream_latency_ms` are disjoint and upstream p99 excludes waits.
4. **Cache mapping**: assert body fixtures map hit/miss/bypassed/disabled/unknown exactly; latency changes alone never change state.
5. **Bounded processing**: 10,000-row response → digest over raw bytes, row count, sample ≤100 rows/256KiB; full 037 normalization is not called.
6. **Stop/drain**: stop run → no new queue items, in-flight drains, status `stopped_by_user`, partial executions queryable.
7. **Circuit breaker**: after ≥20 completed samples, error rate >25% or upstream p99 > configured threshold → intake closes, drain starts, `circuit_breaker_abort` with trigger details.
8. **PROD gate**: unapproved/changed profile or 041 fingerprint dispatches zero requests; approved hash starts exactly one TaskManager task.
9. **Baseline isolation**: catalog hash before/after load run identical; no candidate/source hash/immutability mutation.
10. **Recovery and comparison**: disconnect/reconnect by `load_run_id` restores workers/queue/metrics; partial run compares against complete prior run.
## Commands (implementation phase only)
```bash
cd backend && source .venv/bin/activate && python -m pytest tests/services/load_testing/ -v
cd backend && python -m ruff check .
cd frontend && npm run test
cd frontend && npm run lint
```
#endregion DashboardLoadTesting.Quickstart

87
specs/040-dashboard-load-testing/research.md Normal file
View File

@@ -0,0 +1,87 @@
#region DashboardLoadTesting.Research [C:3] [TYPE ADR] [SEMANTICS research,load-testing,workers,cache,gil]
@BRIEF Phase 0 decisions for load execution: worker runtime, environment capacity, timing model, cache provenance, bounded result processing, persistence, and 041 integration.
**Feature**: 040-dashboard-load-testing | **Date**: 2026-07-22
## R1 — Async Worker Runtime
- **Decision**: One `LoadRun` is registered as one TaskManager task. The load plugin starts N asyncio workers reading a per-run FIFO `asyncio.Queue[LoadExecutionSpec]`. An environment-scoped `LoadCapacityRegistry` owns the load semaphore shared by all active runs. Workers check cancellation and circuit-breaker state before taking the next item; in-flight requests drain within a bounded timeout.
- **Rationale**: The workload is 95–99% upstream I/O. Async workers represent execution concurrency directly, expose queue wait, and support deterministic drain. TaskManager already creates/tracks one asyncio task per plugin run and provides cancellation/events.
- **Alternatives Considered**: (a) HTTP connection count as concurrency — rejected: indirect, hides queue wait and circuit-breaker boundaries. (b) TaskManager task per execution — rejected: floods Task Center/Reports and destroys run-level lifecycle. (c) threads/process workers — rejected for v1: mostly I/O; GIL mitigation is bounded result processing (R5).
- **Impact**: New C5 `LoadTesting.RunnerPool`; no APScheduler or AsyncJobRunner for manual starts. Scheduled load callbacks may dispatch one TaskManager run through existing scheduler boundaries later.
## R2 — Effective Capacity and Existing Client Semaphore Gap
- **Decision**: Effective run concurrency = `min(profile_requested, env.load_test_max_concurrent, max(1, env.connection_pool_size - env.load_test_reserved_slots), 25)`. Defaults: PROD=5; DEV/PREPROD=10; absolute ceiling=25; `load_test_reserved_slots=5`. Worker acquires **load semaphore first**, then request travels through the shared client semaphore; one global acquisition order prevents deadlock.
- **Repository Finding**: `Environment` already has `stage`, `is_production`, `connection_pool_size` (default 20), and `connection_pool_timeout`. `client_registry.get_client()` creates a semaphore but currently constructs `AsyncAPIClient(...)` without passing `semaphore=semaphore`; therefore the documented per-env limit is not active.
- **Rationale**: Load traffic must not occupy all shared Superset capacity and freeze ordinary UI/API operations. Capacity derives from the existing environment pool rather than an unrelated number.
- **Alternatives Considered**: Separate httpx client/pool — rejected: loses shared auth/CSRF cookies; explicitly rejected by `AsyncAPIClient` contract. Acquiring shared semaphore manually in load runner — rejected: once registry wiring is fixed, this double-acquires the same semaphore and can deadlock.
- **Impact**: Foundational prerequisite: pass registry semaphore into `AsyncAPIClient`; regression test proves ordinary calls and load calls share the same limit. Add config fields `load_test_max_concurrent` (optional derived default) and `load_test_reserved_slots=5`.
## R3 — Timing Model and Percentiles
- **Decision**: Record three non-overlapping monotonic timings per execution: `queue_wait_ms` (enqueued → worker takes), `resource_wait_ms` (worker takes → upstream request admitted), `upstream_latency_ms` (admitted → response/error). `end_to_end_ms` is their sum. Dashboard performance percentiles use `upstream_latency_ms`; orchestration health reports queue/resource wait separately. Percentiles use nearest-rank over successful samples; errors/timeouts have dedicated rates and are not converted into synthetic latency values.
- **Rationale**: Combining waits with HTTP latency would measure our orchestrator under contention rather than Superset. Circuit breaker p99 must reflect upstream degradation; queue saturation has its own trigger/diagnostic.
- **Alternatives Considered**: End-to-end p99 only — rejected: cannot attribute queue pressure vs Superset slowdown. Treat timeout as latency=timeout — rejected: biases percentile and double-counts error semantics.
- **Impact**: `LatencySummary` stores sample_count, p50/p90/p95/p99 per timing dimension; breaker latency threshold evaluates upstream p99 only after minimum sample count.
## R4 — Authoritative Cache State from Superset
- **Decision**: Parse `/api/v1/chart/data` JSON body fields per query: `is_cached`, `cache_key`, `cached_dttm`, `queried_dttm`, `cache_timeout`. Mapping: hit iff `is_cached=true`; bypassed iff request.force=true; disabled iff `cache_timeout=-1`; miss iff `is_cached=null && cached_dttm=null && force=false`; unknown for absent/inconsistent fields. Persist raw fields plus `cache_state_source="superset_response"`.
- **Evidence**: Audited `/home/busya/dev/superset/superset/common/query_context_processor.py` (returns all fields), `charts/data/api.py` (serializes query results), `charts/schemas.py` (response schema), `common/utils/query_cache_manager.py` (hit semantics), and Superset integration tests (force/source execution produces `is_cached=None`).
- **Rationale**: This is an authoritative execution signal from Superset, unlike latency inference.
- **Alternatives Considered**: HTTP headers — rejected: chart-data uses body metadata. Probe inference from latency delta — rejected: DB buffer cache, connection reuse, and network jitter confound it.
- **Impact**: 037 chart-data adapter must preserve these fields rather than normalize them away.
## R5 — GIL and Bounded Result Processing
- **Decision**: Load path computes SHA-256 while consuming the raw response bytes, extracts row count/cache metadata, and retains only a bounded first-N-row sample for display. It does not run full 037 value normalization. Consistency compares digest for identical `(chart, filters_hash, role, time_range)` coordinates. Default sample cap: 100 rows and 256 KiB serialized sample.
- **Rationale**: At cap 25 and 0.5–3s upstream latency, workers are predominantly I/O-bound. The relevant GIL risk is large table JSON parsing/normalization (10k+ rows, 200–500ms continuous hold). Bounded processing keeps event-loop distortion low and ensures reported latency belongs to Superset.
- **Alternatives Considered**: ProcessPool normalization — deferred; IPC cost distorts small responses. Full 037 normalization — rejected for load path; belongs to correctness verification. Hashing post-parsed normalized JSON — rejected: incurs full parse and may hide byte-level divergence.
- **Impact**: If fixture instrumentation shows event-loop lag >10% of upstream latency at ceiling load, emit `<ESCALATION>` for a process/streaming parser ADR; do not silently add ProcessPool.
## R6 — Persistence and Event Granularity
- **Decision**: Persist `LoadProfile`, `LoadRun`, `LoadExecution`, and `ConsistencyFinding` in PostgreSQL. Workers buffer execution records and flush bounded batches (≤50 or ≤1s), while aggregate progress events are emitted at ≤4 Hz. Store digest/count/sample/cache/timing, never full large payload. Partial results survive stop/breaker/failure.
- **Rationale**: Per-execution commit/event at concurrency 25 creates database/WebSocket amplification. Batching preserves auditability without turning the orchestrator DB into the bottleneck.
- **Alternatives Considered**: Full response blobs — rejected: unbounded storage/GIL cost and duplicates Superset. In-memory results only — rejected: reconnect and cross-run comparison require durability.
- **Impact**: C5 repository contract owns atomic batch insert + run aggregate update.
## R7 — Circuit Breaker Semantics
- **Decision**: Evaluate on a rolling completed-execution window with minimum 20 samples. Defaults: error-rate threshold 25%; upstream p99 multiplier 3× the selected comparison run/profile reference; if no reference exists, latency breaker is armed only with an explicit absolute threshold. Breach closes queue intake, transitions to draining, preserves partial results, records triggering window/metric.
- **Rationale**: p99 on tiny samples is unstable; implicit latency reference would create false aborts. Error breaker works from the first minimum window.
- **Alternatives Considered**: Break on first timeout — rejected: transient failures are expected load data. Use end-to-end p99 — rejected by R3 attribution model.
- **Impact**: UI must show breaker `inactive|armed|tripped` and why latency breaker may be inactive.
## R8 — Variation Matrix Expansion
- **Decision**: Closed axes: filters, viewport, role, time_range. Normalize axis ordering and values, then cartesian-expand if total ≤ profile cap (default 500 combinations). Above cap, deterministic seeded reservoir sampling without materializing the full product. Each coordinate gets a stable variation id from canonical JSON hash.
- **Rationale**: Determinism supports reproducible comparisons; bounded sampling prevents combinatorial memory/request explosion.
- **Alternatives Considered**: Always cartesian — rejected: unbounded expansion. Random sampling without seed — rejected: cross-run comparison becomes invalid.
- **Impact**: Preview shows theoretical combination count, selected count, seed, chart count, total requests.
## R9 — 041 Blast-Radius Read Model
- **Decision**: Profile validation calls 041 dependents endpoint with pinned fingerprint. `LoadRun` persists that fingerprint and the resolved `BlastRadiusReport`. Start rejects a changed fingerprint with a refresh/reconfirm response, especially before PROD approval. Mid-run index updates do not mutate the run.
- **Rationale**: Configure-time and gate-time impact must name the same dependent dashboards; otherwise user approval is not bound to actual impact.
- **Impact**: 040 cannot be implementation-complete before 041 LIN-FR-013 read contract exists; fixtures unblock independent domain tests.
## R10 — PROD Gate and RBAC
- **Decision**: Stage source of truth is `Environment.stage` (`PROD`) with `is_production` treated as backward-compatible alias; either marks PROD. PREPROD/DEV require `dashboard:loadtest:execute`; PROD additionally requires `dashboard:loadtest:prod` and 036 approval gate reason. Gate hash binds profile revision, effective cap, estimated request count, environment id, and 041 fingerprint.
- **Rationale**: Existing configuration already exposes stage; dual interpretation avoids silently under-classifying legacy `is_production=true` environments.
- **Impact**: Start-after-approval revalidates all hash inputs; mutation/stale gate dispatches zero requests.
## Resolved Risks
| Risk | Mitigation |
|---|---|
| Load starves ordinary Superset calls | Effective cap reserves 5 shared slots; registry semaphore wiring fixed (R2) |
| Double semaphore acquisition deadlock | Single order: load semaphore then request-internal shared semaphore; never manually acquire shared semaphore |
| Event loop distortion on large tables | Raw digest + bounded sample; no full normalization (R5) |
| Cache state misclassification | Authoritative response body + force provenance (R4) |
| Per-execution DB/event amplification | Batches ≤50/1s; progress ≤4 Hz (R6) |
| Stale blast-radius approval | Fingerprint bound into gate hash; start revalidates (R9/R10) |
#endregion DashboardLoadTesting.Research

152
specs/040-dashboard-load-testing/spec.md Normal file
View File

@@ -0,0 +1,152 @@
#region DashboardLoadTesting.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,load-testing,dashboard-testing,variations]
@BRIEF Load testing interface for dashboards: bounded parallel execution of Superset-native checks across declarative variation axes, with circuit breaker, latency metrics, consistency detection, and dataset blast-radius awareness.
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0001]
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0003]
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0005]
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0006]
@RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
@RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec]
@RATIONALE Dashboard correctness (037) proves a query returns the right value once; it does not prove the dashboard survives concurrent load, nor that repeated identical executions return identical results. A dedicated load surface reuses the 037 Superset-native executor so load fidelity matches production execution semantics — a separate load stack would diverge exactly like the rejected direct-SQL path.
@REJECTED Treating load runs as 036 AgentRun instances — rejected because load execution is non-conversational, fan-out by design, and would pollute run/recovery semantics with thousands of pseudo-conversations.
@REJECTED Writing load results into the 037 baseline catalog — rejected because load executions measure latency and consistency, not truth; polluting baselines with load samples would corrupt immutability detection.
@REJECTED Unbounded client-declared concurrency — rejected because a single misconfigured run could saturate the Superset/KXD connection pool and degrade production for all users.
## Navigation (DSA Indexer keywords)
@SEMANTICS: spec, requirements, feature, load-testing, concurrency, variations, circuit-breaker, latency, blast-radius, dashboard-testing
**Feature Branch**: `040-dashboard-load-testing`
**Created**: 2026-07-22 | **Status**: Draft
**Input**: "Интерфейс для нагрузочного тестирования дашбордов — параллельный запуск нескольких проверок на одном дашборде + вариации. Учитывает blast-radius: общий dataset, cache warming одного дашборда влияет на latency зависимых дашбордов."
## Clarifications
### Session 2026-07-22
- **Q1 (execution model)**: Worker pool, not HTTP-connection cap. Per-run async worker queue + env-level semaphore (sum of workers across active runs per env ≤ `max_concurrent_per_env`). Workers check circuit breaker between executions → clean drain. Run registered as ONE TaskManager task (observable in Task Center); executions are not tasks. Pool-wait = explicit queue-wait metric. → LOAD-FR-002/016/017 added.
- **Q2 (concurrency caps)**: prod=5, default (dev/preprod)=10, absolute ceiling=25 (not client-overridable); per-env override `load_test_max_concurrent` in environment config, clamped to ceiling. Ramp defaults: prod 1→2→3→5; others 1→3→5→10. → LOAD-FR-002 amended.
- **Q3 (GIL analysis)**: Async single-loop model kept — at cap 25 and Superset latency 0.5–3s, workers are ~95–99% I/O-bound (~40% of one core CPU). GIL contention risk exists only for large table charts (10k+ rows, ~200–500ms continuous parse+normalize). Mitigation B1+B2: load path uses **bounded normalization** (response sha256 hash + row count + first-N-row sample for display; hash suffices for consistency checks); full 037 normalization stays in the verification path. ProcessPool offload (B3) deferred unless fixtures show >10% GIL distortion. → LOAD-FR-018 added.
- **Q4 (cache state, source-audited)**: Authoritative source is Superset `/api/v1/chart/data` JSON body (`result[].is_cached`, `cache_key`, `cached_dttm`, `queried_dttm`, `cache_timeout`) — verified against `/home/busya/dev/superset/superset/common/query_context_processor.py`, `charts/data/api.py`, and `charts/schemas.py`. State mapping: `hit` if is_cached=true; `bypassed` if request.force=true; `disabled` if cache_timeout=-1; `miss` if is_cached=null + cached_dttm=null + force=false; `unknown` if fields absent/inconsistent. Latency-based probe inference rejected (DB buffer cache/network jitter are confounders). → LOAD-FR-012 amended, LOAD-FR-019 added.
## User Scenarios
### Story 1 — Configure Load Profile With Variations (P1)
**Why P1**: Analysts must declare what to load (dashboard, charts), how hard (concurrency, ramp), and across which axes (filters, viewport, role) before any execution.
**Independent Test**: Build a profile for a fixture dashboard with 2 variation axes and verify the preview shows exact request count, chart coverage, and blast-radius warning listing dependent dashboards sharing datasets.
**Acceptance**:
1. **Given** a dashboard and environment **When** a load profile is configured **Then** the user selects concurrency target (bounded by server cap), execution mode (iterations or duration), ramp-up strategy, and variation axes (filters, viewport, role, time range).
2. **Given** filter variations are declared **When** the profile is validated **Then** every variation value is checked against authoritative Superset filter metadata; unknown values produce `NEEDS_CONTEXT` markers, never invented values.
3. **Given** the dashboard's charts share datasets with other dashboards **When** the profile preview renders **Then** a blast-radius panel lists dependent dashboard count and warns that cache warming/clearing affects them.
---
### Story 2 — Controlled Parallel Execution (P1)
**Why P1**: Load execution must be bounded, observable, and stoppable — never a fire-and-forget request flood.
**Independent Test**: Run a fixture load profile with concurrency=5 and verify in-flight count never exceeds the cap, ramp-up is staged, and live progress shows running/completed/failed per variation.
**Acceptance**:
1. **Given** a started load run **When** execution proceeds **Then** in-flight requests never exceed the server-enforced concurrency cap for that environment, and ramp-up increases load in declared steps.
2. **Given** a running load run **When** the user requests stop **Then** in-flight requests complete or time out within a bounded drain window, no new requests are dispatched, and the run terminates with a `stopped_by_user` status.
3. **Given** executions hit Superset **When** responses return **Then** every result is recorded with chart id, variation coordinates, latency, and typed error taxonomy (403/404/422/5xx/timeout) inherited from 037.
---
### Story 3 — Circuit Breaker and PROD Safety Gate (P1)
**Why P1**: A load run against degraded infrastructure must self-terminate before it amplifies an incident; PROD runs require explicit human approval.
**Independent Test**: Simulate error-rate exceeding threshold mid-run and verify automatic abort with typed terminal reason; verify PROD runs cannot start without a 036-style approval gate.
**Acceptance**:
1. **Given** error rate exceeds the declared threshold OR p99 latency exceeds N× the profile baseline **When** the circuit breaker evaluates **Then** the run aborts with terminal status `circuit_breaker_abort` including the triggering metric and threshold.
2. **Given** a load profile targets a PROD-classified environment **When** start is requested **Then** an approval gate shows concurrency ceiling, estimated request volume, blast-radius dependents, and requires a user-supplied reason.
3. **Given** the user denies the PROD gate **When** denial is submitted **Then** no request is dispatched and the denial is recorded in the run audit trail.
---
### Story 4 — Results: Latency, Errors, Consistency (P2)
**Why P2**: Load value is in the analysis — percentiles per chart, error breakdown, and detection of non-deterministic results under parallelism.
**Independent Test**: Complete a fixture run with injected latency variance and one consistency violation; verify percentiles, error taxonomy counts, and the violation are reported without touching baseline state.
**Acceptance**:
1. **Given** a completed run **When** results render **Then** per-chart p50/p90/p95/p99 latency, throughput, and error counts by taxonomy category are shown.
2. **Given** two executions in one run share identical (chart, normalized filters) coordinates **When** their normalized results diverge **Then** a `consistency_violation` finding is recorded with both result hashes — flagged as flakiness/race, never as a baseline event.
3. **Given** a load run completes **When** the 037 baseline catalog is inspected **Then** no baseline entry, candidate, source_response_hash, or immutability status has changed.
---
### Story 5 — Blast-Radius and Cross-Run Comparison (P2)
**Why P2**: Cache effects cross dashboard boundaries; analysts need to compare runs and see shared-dataset impact explicitly.
**Independent Test**: Run the same profile twice (cold vs warm cache) and a dependent-dashboard probe; verify the comparison view shows latency delta and attributes the shift to shared-dataset cache state.
**Acceptance**:
1. **Given** two runs of the same profile **When** comparison opens **Then** per-chart latency deltas are shown with cache-state annotation (cold/warm/unknown) derived from response metadata.
2. **Given** dashboards share a dataset with the tested dashboard **When** results render **Then** the blast-radius panel links each dependent dashboard and marks whether it was probed during the run window.
3. **Given** a run targets a dashboard whose datasets serve PROD dashboards **When** scheduling is configured **Then** the schedule policy warns about cache-interference windows, not just absolute concurrency.
---
### Edge Cases
- Superset connection pool exhaustion in the environment → executor queues within cap, records `pool_wait` latency separately, never bypasses the cap.
- Variation references a filter value that exists in PREPROD but not PROD → validation marks the variation `NEEDS_CONTEXT` per environment; the run may proceed with the valid subset only.
- Circuit breaker trips during ramp-up → partial results are preserved and the run is analyzable; abort is not data loss.
- Identical chart+filters executed concurrently return different row ordering → normalization (037) canonicalizes before consistency comparison; ordering alone is not a violation.
- User closes the browser mid-run → the run continues server-side; reconnecting shows live status by run id (recovery semantics from 036 US2).
- Scheduled load run overlaps a deployment window for a blast-radius-dependent dashboard → schedule policy blocks or warning-gates the overlap.
## Requirements
### Functional
- **LOAD-FR-001**: All load executions MUST go through the 037 Superset-native chart/dataset execution path; direct SQL, generated SQL, and raw query_context injection are forbidden (inherits 037 `@REJECTED`).
- **LOAD-FR-002**: Concurrency MUST be enforced via a **worker pool** model: per-run async worker queue plus an env-level semaphore guaranteeing the sum of workers across all active runs on one environment never exceeds `max_concurrent_per_env`. Defaults: PROD-classified=5, dev/preprod=10, absolute ceiling=25 (not client-overridable); per-env override `load_test_max_concurrent` clamped to ceiling. Client values above the cap are clamped with a visible notice. Ramp defaults: PROD 1→2→3→5; others 1→3→5→10.
- **LOAD-FR-016**: Each load run MUST be registered as exactly ONE TaskManager task (Task Center observability); individual executions are queue items, never tasks (no Reports flooding). Workers MUST check the circuit breaker between executions — in-flight completes, new executions are not taken (clean drain semantics).
- **LOAD-FR-017**: Worker state MUST be observable per run: workers busy/idle, current execution per worker, queue depth — powering the live progress of LOAD-FR-008.
- **LOAD-FR-018**: Load-path result processing MUST use bounded normalization: full-response sha256 hash + row count + first-N-row sample for display. Consistency checks (LOAD-FR-006) operate on hashes; full 037 normalization remains in the verification path only. This bounds continuous GIL hold per execution (large table charts) and keeps measured latency attributable to Superset, not to the load pipeline.
- **LOAD-FR-003**: Load executions MUST NOT create, update, or invalidate 037 baseline entries, candidates, `source_response_hash`, immutability status, or visual baselines. Load artifacts live in a separate result store.
- **LOAD-FR-004**: Every load run MUST have a circuit breaker with declared thresholds (error-rate %, p99 multiplier); breach aborts the run with typed terminal status and preserved partial results.
- **LOAD-FR-005**: Starting a run against a PROD-classified environment MUST require an approval gate (036 semantics) showing concurrency ceiling, estimated request volume, blast-radius dependents, and a mandatory reason. Denial dispatches nothing.
- **LOAD-FR-006**: Executions sharing identical (chart id, normalized filter context) within one run MUST be compared after 037 normalization; divergence MUST produce a `consistency_violation` finding, classified as flakiness — never as baseline drift.
- **LOAD-FR-007**: Variation axes MUST be declarative and closed: `filters`, `viewport`, `role`, `time_range`. Unknown axes or values MUST fail validation; missing filter values MUST surface as `NEEDS_CONTEXT`, never invented.
- **LOAD-FR-008**: Every load run MUST expose a stable `load_run_id` with live progress (per-variation running/completed/failed), recoverable after disconnect, with typed terminal statuses: `completed`, `stopped_by_user`, `circuit_breaker_abort`, `failed`.
- **LOAD-FR-009**: The system MUST resolve the reverse index dataset → dependent dashboards for the tested dashboard and display it as a blast-radius panel at configure time, at the PROD gate, and in results.
- **LOAD-FR-010**: Variation expansion MUST be deterministic for the same (dashboard query model, axes, seed): cartesian for ≤ declared cap, seeded sampling above it; the expanded matrix MUST be previewable before start.
- **LOAD-FR-011**: Ramp-up MUST be staged (declared steps to target concurrency); steady state and drain phases MUST be distinguishable in progress and metrics.
- **LOAD-FR-012**: Result records MUST include per-execution provenance: environment, dashboard, chart, variation coordinates, normalized filters hash, HTTP latency, queue-wait time, cache-state metadata, and Superset error taxonomy. Cache-state metadata comes authoritatively from `/api/v1/chart/data` response body: `is_cached`, `cache_key`, `cached_dttm`, `queried_dttm`, `cache_timeout`; mapped to `hit|miss|bypassed|disabled|unknown` with `cache_state_source="superset_response"`.
- **LOAD-FR-019**: Cache-state mapping MUST be: `hit` iff `is_cached=true`; `bypassed` iff request `force=true`; `disabled` iff `cache_timeout=-1`; `miss` iff `is_cached=null` AND `cached_dttm=null` AND force=false; `unknown` when fields are absent/inconsistent. Latency-based cache inference is forbidden because DB buffer cache, connection reuse, and network jitter are confounders.
- **LOAD-FR-013**: Scheduled load runs MUST evaluate overlap against deployment/maintenance windows of blast-radius-dependent dashboards and block or warning-gate conflicts per policy.
- **LOAD-FR-014**: RBAC MUST distinguish `dashboard:loadtest:execute` (PREPROD/staging) from `dashboard:loadtest:prod` (PROD-classified); unauthorized actors see `permission_denied`, never a confirm control (036 gate semantics).
- **LOAD-FR-015**: Comparison of two runs of the same profile MUST show per-chart latency deltas and consistency-finding deltas; comparison MUST NOT require both runs to be complete (partial-current vs baseline-run allowed).
### Key Entities
- **LoadProfile**: Declarative configuration — dashboard, environment, concurrency target, ramp steps, execution mode (iterations|duration), variation axes with values, circuit-breaker thresholds, schedule policy. Versioned and reusable.
- **LoadVariation**: One expanded coordinate set from the variation matrix — concrete filter values, viewport, role, time range. Immutable once the run starts.
- **LoadRun**: Recoverable execution instance of a profile: id, status, phase (ramp|steady|drain|terminal), per-variation progress, circuit-breaker state, approval linkage for PROD.
- **LoadExecution**: Single Superset-native request record: variation coordinates, chart, HTTP latency, queue-wait, bounded result hash/count/sample, typed outcome, and authoritative cache metadata (`is_cached`, `cache_key`, `cached_dttm`, `queried_dttm`, `cache_timeout`, mapped `cache_state`). Never writes to baseline state.
- **ConsistencyViolation**: Finding that identical (chart, filters) executions diverged post-normalization; carries both result hashes; classified flakiness, not baseline drift.
- **BlastRadiusReport**: Reverse-index projection — dataset ids used by the tested dashboard, dependent dashboards per dataset, cache-interference warning level, probe coverage.
- **CircuitBreakerPolicy**: Thresholds (error-rate %, p99 multiplier), evaluation window, abort semantics, preserved-partial guarantee.
- **LoadRunComparison**: Delta view of two runs — per-chart latency shift, cache-state attribution, consistency-finding delta.
- **LoadRunnerPool**: Per-environment worker pool — async workers executing queue items, env-level semaphore enforcing `max_concurrent_per_env` across all active runs, worker registry (busy/idle, current execution, queue depth), breaker-checked between executions.
- **ExecutionQueue**: Per-run FIFO of LoadExecution items; pool-wait measured as queue-residence time; drain = stop enqueue + bounded wait for in-flight.
## Success Criteria
- **SC-001**: In 100% of fixture runs, in-flight requests never exceed the configured cap, including during ramp-up and drain.
- **SC-002**: Circuit breaker aborts within one evaluation window of threshold breach in 100% of fault-injection tests; partial results remain fully queryable.
- **SC-003**: Zero mutations to the 037 baseline catalog across the entire load-test fixture suite (verified by catalog hash before/after).
- **SC-004**: Variation matrix expansion is byte-deterministic across repeated expansions and shuffled axis ordering.
- **SC-005**: Blast-radius panel lists 100% of dependent dashboards for fixture shared datasets at configure time and in results.
- **SC-006**: PROD gate blocks 100% of unapproved PROD starts; denial produces zero dispatched requests.
- **SC-007**: Consistency violation detection catches 100% of injected divergent-result fixtures and produces zero false positives on ordering-only differences.
#endregion DashboardLoadTesting.Spec

214
specs/040-dashboard-load-testing/tasks.md Normal file
View File

@@ -0,0 +1,214 @@
#region DashboardLoadTesting.Tasks [C:3] [TYPE ADR] [SEMANTICS tasks,load-testing,implementation]
@BRIEF Ordered TDD backlog for dashboard load testing with bounded async workers, deterministic variations, Superset cache metadata, and 041 blast-radius snapshots.
**Input**: all documents in specs/040-dashboard-load-testing/
**Prerequisites**: clarified spec, plan, research R1–R10, data-model, contracts/modules.md, quickstart, 041 LIN-FR-013 read contract
## Phase 1 — Setup: Fixtures, DTOs, Permissions
- [ ] T001 Create canonical dashboard/chart/query response fixtures under specs/040-dashboard-load-testing/fixtures/: scalar chart, bounded table chart, two filters, two viewports, repeated identical coordinates, Superset cache hit/miss/bypassed/disabled/unknown responses, and 041 shared-dataset blast-radius snapshot.
- [ ] T002 [P] Create canonical environment policy fixtures under specs/040-dashboard-load-testing/fixtures/environments/: DEV/PREPROD pool=20, PROD pool=10, pool=5 edge, stage/is_production combinations, per-env override, and reserve slots.
- [ ] T003 Materialize canonical fixtures into backend/tests/fixtures/load_testing/ and frontend/src/lib/models/__fixtures__/load-testing/.
- [ ] T004 [P] Define extra-forbid Pydantic DTOs in backend/src/schemas/load_testing.py for profiles, matrix preview, runs, executions, aggregates, findings, cache metadata, and typed errors.
- [ ] T005 [P] Define matching TypeScript DTOs in frontend/src/lib/types/load-testing.ts with cross-stack field parity and no SQL/raw endpoint/query_context fields.
- [ ] T006 [P] Register `dashboard:loadtest:execute` and `dashboard:loadtest:prod` in backend/src/services/rbac_permission_catalog.py; add default-deny role mappings and permission fixtures.
- [ ] T007 Add load testing navigation/action labels and state copy to frontend/src/lib/i18n/locales/ru/load-testing.json and frontend/src/lib/i18n/locales/en/load-testing.json; register locale resources without deriving state from localized strings.
## Phase 2 — Foundational: Persistence, Client Capacity, Task Boundary
- [ ] T008 Write failing migration/model tests in backend/tests/models/test_load_testing.py for LoadProfile revisioning, immutable LoadVariation coordinates, terminal LoadRun status, LoadExecution provenance, aggregate uniqueness, and ConsistencyFinding linkage.
- [ ] T009 Add ORM models in backend/src/models/load_testing.py and Alembic migration under backend/alembic/versions/ for profile/run/variation/execution/aggregate/finding records and indexes.
- [ ] T010 Write failing client-registry tests in backend/tests/core/test_client_registry_load_capacity.py proving Environment.connection_pool_size semaphore is passed into AsyncAPIClient, shared ordinary/load calls respect the same semaphore, and shutdown closes the shared client.
- [ ] T011 Fix backend/src/core/utils/client_registry.py to pass the existing per-environment semaphore into AsyncAPIClient; preserve shared auth/CSRF client behavior and do not create a second client/pool.
@PRE Registry client is initialized for the environment.
@POST AsyncAPIClient acquires/releases the shared semaphore around every request.
@INVARIANT No double-acquire path is introduced in the load runner.
@TEST_EDGE pool slot exhaustion waits then releases; shutdown leaves no live client.
- [ ] T012 Write failing TaskManager boundary tests in backend/tests/services/load_testing/test_task_boundary.py proving one LoadRun creates exactly one TaskManager task, executions are not tasks, cancellation reaches the run, and task events remain aggregate-level.
- [ ] T013 Add load testing plugin/task entrypoint in backend/src/plugins/load_testing.py; create one observable TaskManager task per LoadRun and connect lifecycle cancellation/status persistence.
- [ ] T014 [P] Add belief-runtime test helpers and instrumentation fixtures under backend/tests/services/load_testing/conftest.py for `belief_scope`, `logger.reason`, `logger.reflect`, and terminal event coverage.
**Checkpoint**: Models migrate; shared client semaphore is active; one run maps to one TaskManager task; no user story execution yet.
## Phase 3 — User Story 1: Configure Load Profile and Variations (P1)
**Goal**: Validate profile inputs, resolve effective capacity, expand deterministic variations, and preview 041 blast radius without dispatching requests.
**Independent Test**: Fixture profile returns exact matrix/request estimate, cap notice, and dependent-dashboard snapshot; invalid axes/values are rejected with recovery markers.
### Tests First
- [ ] T015 [P] [US1] Write failing profile validation tests in backend/tests/services/load_testing/test_profile.py for closed axes, typed filters, required iterations/duration, circuit-breaker bounds, max variation cap, and no raw SQL/query context.
- [ ] T016 [P] [US1] Write failing capacity tests in backend/tests/services/load_testing/test_capacity.py for DEV/PREPROD=10, PROD=5, ceiling=25, reserve=5, per-env override clamping, pool=5 rejection, and multi-run environment sharing.
- [ ] T017 [P] [US1] Write failing matrix tests in backend/tests/services/load_testing/test_matrix.py for canonical axis ordering, stable variation ids, cartesian expansion, seeded reservoir sampling, theoretical vs selected counts, and unknown-axis rejection.
- [ ] T018 [P] [US1] Write failing 041 integration contract tests in backend/tests/services/load_testing/test_blast_radius_contract.py for fingerprint pinning, dependent-dashboard projection, stale snapshot warning, and changed fingerprint requiring refresh.
- [ ] T019 [P] [US1] Write failing L1 model tests in frontend/src/lib/models/__tests__/Dashboards.LoadProfileModel.test.ts for idle/validating/matrix_ready/gate_required/start_error states, cap notices, NEEDS_CONTEXT, and pinned blast-radius snapshot.
- [ ] T020 [P] [US1] Write failing profile editor L2 tests in frontend/src/lib/components/dashboards/load-testing/__tests__/LoadProfileEditor.ux.test.ts for accessible controls, variation errors, matrix preview, cap clamp, and dependent-dashboard warning.
### Implementation
- [ ] T021 [US1] Implement backend/src/services/load_testing/capacity.py.
@PRE Environment stage and pool configuration are valid.
@POST Returns effective cap and clamp/rejection reasons; never exceeds profile, environment, reserve-adjusted pool, or ceiling.
@INVARIANT PROD=5, DEV/PREPROD=10, ceiling=25, reserve=5.
- [ ] T022 [US1] Implement backend/src/services/load_testing/matrix.py.
@PRE Axes are closed and normalized.
@POST Returns byte-deterministic variation matrix and request estimate; over-cap expansion is seeded and bounded.
@TEST_EDGE shuffled input ordering produces identical bytes.
- [ ] T023 [US1] Implement backend/src/services/load_testing/profile.py to validate profile against 037 DashboardQueryModel and 041 pinned dependents response; no request dispatch.
- [ ] T024 [US1] Implement backend/src/api/routes/load_testing.py profile validation and matrix preview endpoints with RBAC and typed errors.
- [ ] T025 [US1] Implement frontend/src/lib/models/Dashboards.LoadProfileModel.svelte.ts and frontend/src/lib/components/dashboards/load-testing/LoadProfileEditor.svelte bound to typed API DTOs.
- [ ] T026 [US1] Add dashboard entry action and load-testing route integration in frontend/src/routes/dashboards/[id]/components/DashboardHeader.svelte and frontend/src/routes/load-testing/.
**Checkpoint**: Profile preview works independently; zero Superset execution requests are dispatched during configuration.
## Phase 4 — User Story 2: Controlled Parallel Execution (P1)
**Goal**: Execute one validated matrix through bounded workers with clean ramp, steady, stop, drain, recovery, and aggregate progress.
**Independent Test**: Concurrency=5 fixture never exceeds cap; queue/resource/upstream timings are disjoint; stop drains and preserves partial results.
### Tests First
- [ ] T027 [P] [US2] Write failing worker-pool tests in backend/tests/services/load_testing/test_runner_pool.py for worker cap, per-run FIFO, shared env capacity across runs, one worker registry, and no execution TaskManager tasks.
- [ ] T028 [P] [US2] Write failing timing tests in backend/tests/services/load_testing/test_timing.py injecting queue/resource/upstream delays; assert separate timing fields and upstream-only percentile inputs.
- [ ] T029 [P] [US2] Write failing lifecycle tests in backend/tests/services/load_testing/test_run_lifecycle.py for ramp steps, steady state, user stop, bounded drain, browser-independent continuation, terminal immutability, and partial result durability.
- [ ] T030 [P] [US2] Write failing persistence batch tests in backend/tests/services/load_testing/test_persistence.py for ≤50/1s flush thresholds, aggregate updates, reconnect snapshot, and no full response blob storage.
- [ ] T031 [P] [US2] Write failing L1 model tests in frontend/src/lib/models/__tests__/Dashboards.LoadRunModel.test.ts for queued/ramping/steady/draining/terminal states, worker registry, queue depth, reconnect, and partial results.
- [ ] T032 [P] [US2] Write failing L2 monitor tests in frontend/src/lib/components/dashboards/load-testing/__tests__/LoadRunMonitor.ux.test.ts for worker status, phase badges, stop action, drain feedback, error counts, and keyboard accessibility.
### Implementation
- [ ] T033 [US2] Implement backend/src/services/load_testing/timing.py with monotonic queue/resource/upstream clocks and nearest-rank percentile aggregation.
- [ ] T034 [US2] Implement backend/src/services/load_testing/runner_pool.py.
@PRE Validated profile, gate if required, pinned 041 fingerprint, effective capacity.
@POST In-flight workers never exceed cap; one LoadRun terminal state and durable partial results.
@SIDE_EFFECT Superset chart-data calls, batched DB writes, aggregate progress events.
@INVARIANT Acquire load capacity before entering shared client request; breaker/stop prevents new queue intake.
@TEST_EDGE cancellation before take leaves item unexecuted; in-flight timeout drains without new dispatch.
- [ ] T035 [US2] Implement backend/src/services/load_testing/persistence.py with bounded execution batches and aggregate snapshots.
- [ ] T036 [US2] Implement backend/src/api/routes/load_testing.py start/status/stop/results endpoints and reconnect by load_run_id.
- [ ] T037 [US2] Implement frontend/src/lib/models/Dashboards.LoadRunModel.svelte.ts and frontend/src/lib/components/dashboards/load-testing/LoadRunMonitor.svelte.
- [ ] T038 [US2] Add WebSocket or existing task-event subscription binding for aggregate progress at ≤4 Hz; never emit one UI event per execution.
**Checkpoint**: Worker run survives browser disconnect, stops cleanly, and reports partial results without exceeding cap.
## Phase 5 — User Story 3: Circuit Breaker and PROD Safety Gate (P1)
**Goal**: Abort degraded runs safely and prevent unapproved PROD dispatch.
**Independent Test**: Fault injection trips breaker after minimum window; denied or stale PROD gate dispatches zero requests.
### Tests First
- [ ] T039 [P] [US3] Write failing breaker tests in backend/tests/services/load_testing/test_breaker.py for minimum 20 samples, 25% error threshold, p99 multiplier, absolute p99 fallback, trigger metadata, drain, and preserved partial results.
- [ ] T040 [P] [US3] Write failing gate/RBAC tests in backend/tests/services/load_testing/test_gate.py for DEV/PREPROD permission, PROD permission, reason requirement, profile/cap/fingerprint hash binding, denial, expiry, replay, and payload mutation.
- [ ] T041 [P] [US3] Write failing frontend gate tests in frontend/src/lib/components/dashboards/load-testing/__tests__/LoadApprovalGate.ux.test.ts for impact summary, reason, deny, stale fingerprint, and no confirmation control on permission denial.
### Implementation
- [ ] T042 [US3] Implement backend/src/services/load_testing/breaker.py.
@PRE Completed execution window has policy and required sample count.
@POST Tripped breaker closes intake, starts drain, records metric/threshold, and preserves partial results.
@TEST_EDGE fewer than 20 samples never triggers p99 abort.
- [ ] T043 [US3] Implement backend/src/services/load_testing/gates.py using 036 ApprovalGate semantics; bind profile revision, effective cap, request estimate, environment, and 041 fingerprint.
- [ ] T044 [US3] Add PROD gate and permission guards to backend/src/api/routes/load_testing.py; revalidate hash immediately before dispatch.
- [ ] T045 [US3] Add breaker badge, approval card, and recovery states to frontend/src/lib/components/dashboards/load-testing/.
- [ ] T046 [US3] Add rejected-path regression tests proving unapproved PROD, stale gate, and arbitrary-SQL payloads dispatch zero requests.
**Checkpoint**: Circuit breaker and PROD safety are independently verifiable; no unauthorized request reaches Superset.
## Phase 6 — User Story 4: Results, Cache, and Consistency (P2)
**Goal**: Report upstream latency, typed errors, authoritative cache state, bounded result metadata, and consistency findings without baseline mutation.
**Independent Test**: Fixture run returns percentiles, cache states, divergent digest finding, and unchanged baseline catalog hash.
### Tests First
- [ ] T047 [P] [US4] Write failing cache mapping tests in backend/tests/services/load_testing/test_cache_metadata.py for Superset body hit/miss/bypassed/disabled/unknown and inconsistent metadata; latency changes must not affect mapping.
- [ ] T048 [P] [US4] Write failing bounded-result tests in backend/tests/services/load_testing/test_bounded_result.py for raw SHA-256, 10k-row response, 100-row/256KiB cap, row count, and no call to full 037 normalization.
- [ ] T049 [P] [US4] Write failing consistency tests in backend/tests/services/load_testing/test_consistency.py for identical coordinate digest divergence, row-order-only allowance, and finding classification=flakiness.
- [ ] T050 [P] [US4] Write failing baseline-isolation tests in backend/tests/services/load_testing/test_baseline_isolation.py asserting catalog hash, source_response_hash, candidates, and immutability status are unchanged.
- [ ] T051 [P] [US4] Write failing results UI tests in frontend/src/lib/components/dashboards/load-testing/__tests__/LoadResults.ux.test.ts for p50/p90/p95/p99, cache counts, errors, consistency finding detail, and bounded samples.
### Implementation
- [ ] T052 [US4] Implement backend/src/services/load_testing/cache_metadata.py using Superset response-body fields: `is_cached`, `cache_key`, `cached_dttm`, `queried_dttm`, `cache_timeout`.
@POST Maps to hit/miss/bypassed/disabled/unknown with `cache_state_source="superset_response"`.
@INVARIANT Latency inference forbidden.
- [ ] T053 [US4] Implement backend/src/services/load_testing/bounded_result.py.
@POST Raw digest, row count, sample ≤100 rows/256KiB; full 037 normalization not invoked.
@TEST_EDGE GIL/event-loop distortion >10% requires escalation, not silent ProcessPool introduction.
- [ ] T054 [US4] Implement backend/src/services/load_testing/aggregates.py and consistency finding persistence; use digest-based comparison and separate error rates.
- [ ] T055 [US4] Implement frontend/src/lib/components/dashboards/load-testing/LoadResults.svelte and LoadComparison.svelte.
- [ ] T056 [US4] Add baseline read-only boundary assertions to backend/src/services/load_testing/ and expose explicit “baselines not modified” result state.
**Checkpoint**: Results and cache provenance are queryable; baseline catalog remains byte-identical.
## Phase 7 — User Story 5: Blast-Radius Comparison and Scheduling (P2)
**Goal**: Compare runs and expose shared-dataset impact with the 041 pinned read model; warn about overlap windows.
**Independent Test**: Two same-profile runs plus dependent-dashboard probe show latency deltas, cache metadata, and probe coverage bound to fingerprints.
### Tests First
- [ ] T057 [P] [US5] Write failing comparison tests in backend/tests/services/load_testing/test_comparison.py for complete/partial runs, per-chart deltas, cache-state attribution, and consistency-finding deltas.
- [ ] T058 [P] [US5] Write failing blast-radius scheduling tests in backend/tests/services/load_testing/test_schedule_policy.py for dependent deployment/maintenance overlap, fingerprint pinning, and warning/block policy.
- [ ] T059 [P] [US5] Write failing comparison UI tests in frontend/src/lib/components/dashboards/load-testing/__tests__/LoadComparison.ux.test.ts for run selector, latency deltas, cache annotations, dependent dashboards, and probe coverage.
### Implementation
- [ ] T060 [US5] Implement backend/src/services/load_testing/comparison.py with partial-run-compatible aggregates and cache-state provenance.
- [ ] T061 [US5] Implement backend/src/services/load_testing/schedule_policy.py; query 041 dependent dashboard windows and block/warning conflicts per policy.
- [ ] T062 [US5] Add comparison and blast-radius endpoints to backend/src/api/routes/load_testing.py.
- [ ] T063 [US5] Complete frontend/src/lib/components/dashboards/load-testing/LoadComparison.svelte and blast-radius result panels.
**Checkpoint**: Same-profile comparison works with partial current runs; blast-radius scope remains pinned and visible.
## Phase 8 — Integration, Accessibility, and Quality Gates
- [ ] T064 Add frontend/src/lib/models/__tests__/Dashboards.LoadModels.integration.test.ts covering profile → gate → run → reconnect → results.
- [ ] T065 Add frontend/e2e/tests/dashboard-load-testing.e2e.js covering dashboard entry → matrix preview → approval/deny → worker monitor → stop/reconnect → results.
- [ ] T066 Add backend/tests/integration/test_dashboard_load_testing_superset.py using Superset chart-data fixtures or Testcontainers; verify response cache metadata is preserved.
- [ ] T067 Add backend/tests/integration/test_load_testing_client_capacity.py proving shared semaphore wiring, reserve slots, multiple runs, and ordinary request fairness.
- [ ] T068 Add frontend responsive and accessibility tests for 1366×768 and narrow viewport profile/monitor/results screens.
- [ ] T069 Run quickstart.md steps 1–10 and record results.
- [ ] T070 Run backend load-testing tests, existing client-registry/migration/task regressions, and `python -m ruff check .`.
- [ ] T071 Run frontend tests, lint, build, and Playwright.
- [ ] T072 [P] Audit rejected paths: direct SQL, HTTP-only cap, per-execution TaskManager tasks, latency cache inference, full normalization in load path, baseline mutation, unapproved PROD dispatch.
- [ ] T073 [P] Audit ATTN_1–4, exact anchor pairs, C4/C5 `@RATIONALE/@REJECTED`, belief-runtime markers, unresolved relations, and 041 fingerprint integration.
- [ ] T074 [P] Update 041/039 traceability notes with LoadRun/FleetReport consumer links after contracts are implemented.
## Dependencies
T001–T007 → T008–T014 foundational → US1 → US2 → US3/US4; US5 follows US4 and 041 read-model availability. Phase 8 follows all stories. Tests precede implementation within each story. T011 must pass before claiming any environment capacity result. T047/T052 require the 037 chart-data adapter to preserve cache metadata.
## Parallel Opportunities
- T002 ∥ T004 ∥ T005 ∥ T006 ∥ T007
- T008 ∥ T010 ∥ T012 ∥ T014
- T015 ∥ T016 ∥ T017 ∥ T018 ∥ T019 ∥ T020
- T027 ∥ T028 ∥ T029 ∥ T030 ∥ T031 ∥ T032
- T039 ∥ T040 ∥ T041
- T047 ∥ T048 ∥ T049 ∥ T050 ∥ T051
- T057 ∥ T058 ∥ T059
- T064 ∥ T066 ∥ T067 ∥ T068
## Story Verification Criteria
| Story | Verification |
|---|---|
| US1 | Matrix/cap/041 preview; no dispatch; deterministic bytes; quickstart 1–2 |
| US2 | Worker cap, disjoint timings, stop/drain/reconnect; quickstart 3, 6, 10 |
| US3 | Breaker and PROD gate; zero dispatch on deny/stale; quickstart 7–8 |
| US4 | Cache body mapping, bounded digest/sample, consistency, baseline hash; quickstart 4–5, 9 |
| US5 | Partial comparison, cache attribution, dependent-dashboard probe coverage, schedule overlap |
## Rejected-Path Coverage
- Direct SQL/raw query context: T046, T072.
- HTTP pool as sole concurrency control: T016, T027, T067, T072.
- One TaskManager task per execution: T012, T027, T072.
- Latency-based cache inference: T047, T072.
- Full 037 normalization in load path: T048, T053, T072.
- Baseline/catalog mutation: T050, T056, T072.
- Unapproved PROD dispatch: T040, T046, T072.
#endregion DashboardLoadTesting.Tasks

35
specs/040-dashboard-load-testing/traceability.md Normal file
View File

@@ -0,0 +1,35 @@
#region DashboardLoadTesting.Traceability [C:3] [TYPE ADR] [SEMANTICS traceability,load-testing,requirements]
@BRIEF Requirement-to-contract-to-test matrix for 040 load testing.
| Requirement | Contract | Verification |
|---|---|---|
| LOAD-FR-001 | LoadTesting.Api.Routes + SupersetClient.ChartData.Execute | malicious SQL/raw-context schema tests |
| LOAD-FR-002, LOAD-FR-016, LOAD-FR-017 | LoadTesting.Capacity.Resolve, LoadTesting.RunnerPool | capacity, worker registry, one TaskManager task, multi-run env cap |
| LOAD-FR-003 | LoadTesting.RunnerPool + baseline read-only boundary | catalog hash before/after |
| LOAD-FR-004 | LoadTesting.Breaker.Evaluate | rolling window, error/p99 abort, drain |
| LOAD-FR-005, LOAD-FR-014 | LoadTesting.Api.Routes + 036 ApprovalGate | PROD deny/approval/hash mutation/RBAC |
| LOAD-FR-006 | LoadTesting.RunnerPool + ConsistencyFinding | identical coordinate digest divergence; row-order edge |
| LOAD-FR-007, LOAD-FR-010 | LoadTesting.Matrix.Expand | closed axes, NEEDS_CONTEXT, deterministic cartesian/seeded sample |
| LOAD-FR-008, LOAD-FR-011 | LoadTesting.RunnerPool + LoadRunModel | reconnect, ramp/steady/drain terminal FSM |
| LOAD-FR-009, LOAD-FR-013 | 041 pinned BlastRadiusReport | configure/gate/results same fingerprint; schedule overlap |
| LOAD-FR-012, LOAD-FR-019 | LoadTesting.Cache.MapResponse + LoadExecution | authoritative response metadata mapping |
| LOAD-FR-015 | LoadRunComparison | complete vs partial comparison |
| LOAD-FR-018 | LoadTesting.Result.BoundedProcess | raw digest, row/sample cap, no full 037 normalization, GIL fixture |
## Story Coverage
| Story | Independent checkpoint |
|---|---|
| US1 Profile | quickstart 1–2; matrix/cap preview and blast-radius warning |
| US2 Execution | quickstart 3, 6, 10; worker cap, drain, recovery |
| US3 Safety | quickstart 7–9; breaker, PROD gate, baseline isolation |
| US4 Results | quickstart 4–5, 7; cache/metrics/consistency |
| US5 Comparison | quickstart 10; cache metadata and partial-run deltas |
## Cross-Spec Relations
- Upstream: 036 ApprovalGate, TaskManager run/recovery; 037 QueryModel, Filters, ChartData.Execute, error taxonomy; 041 pinned dependents read model.
- Downstream: 039 can link LoadRun results from dashboard/release verification surfaces; 041 provides blast-radius snapshot.
- Repository integration prerequisite: `client_registry.get_client()` must pass its existing per-env semaphore to `AsyncAPIClient` before load capacity claims are trusted.
#endregion DashboardLoadTesting.Traceability

82
specs/040-dashboard-load-testing/ux_reference.md Normal file
View File

@@ -0,0 +1,82 @@
#region DashboardLoadTesting.UxReference [C:3] [TYPE ADR] [SEMANTICS ux,reference,load-testing,dashboard-testing]
@BRIEF UX interaction reference for dashboard load testing: persona, happy path, states, errors, recovery. Drives @UX_* tags in Phase 1 contracts.
**Feature Branch**: `040-dashboard-load-testing`
**Created**: 2026-07-22 | **Status**: Draft
## 1. User Persona & Context
* **Who is the user?**: BI engineer or DevOps engineer validating that a dashboard survives realistic concurrent load — before a release, after infrastructure changes, or on a schedule.
* **What is their goal?**: Answer three questions quickly: (1) how fast is each chart under N parallel users, (2) does the dashboard error or degrade under load, (3) do identical requests return identical results. They also need to know *who else is affected* — the dashboards sharing datasets with the tested one.
* **Context**: Web UI on desktop. Starts from a dashboard page («Нагрузочное тестирование» action) or the load-testing hub. For PROD runs, expects an explicit approval step with full impact disclosure.
## 2. The "Happy Path" Narrative
The user opens a dashboard and clicks «Нагрузочное тестирование». The profile form opens pre-filled: charts from the dashboard query model, concurrency slider clamped to the environment cap, axes for filter/viewport variations. The blast-radius panel immediately shows «3 других дашборда используют те же датасеты» — the user understands the cache impact before starting. The expanded matrix preview reads «120 запросов × 8 чартов, оценка ~4 мин». They press Start; the run ramps visibly (0→5→10 in-flight), live tiles fill per variation, and the circuit breaker badge stays green. On completion, percentiles per chart render with one consistency finding flagged — clickable, showing both divergent result hashes. The baseline catalog badge confirms «baselines не затронуты».
## 3. Interface Mockups
### UI Layout & Flow
**Screen**: Load Profile Editor (`/dashboards/[id]/load/new`)
* **Layout**: Three-column — left: profile form; center: variation matrix preview; right: blast-radius panel.
* **Key Elements**:
* **Concurrency slider**: Min 1, max = server cap for the environment; values above cap clamp with notice «Снижено до лимита среды (N)».
* **Axes selector**: Checkboxes — Фильтры / Viewport / Роль / Период. Each axis expands to typed value inputs validated against Superset metadata.
* **Matrix preview**: «N комбинаций × M чартов = K запросов» with deterministic seed shown; above cap → «сэмплирование, seed=42».
* **Blast-radius panel**: Dataset list → dependent dashboards with links; warning icon when any dependent is PROD-classified.
* **Circuit breaker thresholds**: Error-rate % and p99 multiplier inputs with safe defaults pre-filled.
* **Start button**: Primary; for PROD environments opens the approval gate instead of immediate start.
* **Contract Mapping**:
* **`@UX_STATE`**: `idle → validating → matrix_ready → starting → ramping → steady → draining → terminal(completed|stopped_by_user|circuit_breaker_abort|failed)`.
* **`@UX_FEEDBACK`**: Clamp notice, NEEDS_CONTEXT markers on invalid variation values, breaker badge state (green/armed/tripped).
* **`@UX_RECOVERY`**: Invalid variation → fix inline or drop variation; start failure → retry with same profile; abort → results remain analyzable.
* **`@UX_REACTIVITY`**: Screen model atoms (`profile`, `matrix`, `blastRadius`, `runStatus`) `$derived` into panels; live run updates via the 036 recovery channel keyed by `load_run_id`.
* **Screen Model**: `Dashboards.LoadProfileModel.svelte.ts` (editor FSM + matrix derivation) and `Dashboards.LoadRunModel.svelte.ts` (live execution state). Components bind via `@RELATION BINDS_TO`.
**Screen**: Load Run Monitor (`/load-runs/[id]`)
* **Layout**: Header (status, phase, breaker badge, Stop button) → per-variation progress grid → live latency sparkline per chart → findings strip (consistency violations, error taxonomy counts).
* **States**:
* **Ramping**: Phase badge «Разгон N→M»; in-flight counter visible.
* **Steady**: Tiles update per execution; pool-wait shown separately from query latency.
* **Draining**: «Завершение in-flight, новые не отправляются».
* **Terminal**: Summary cards persist; partial results labeled «частичные — прервано» when aborted.
**Screen**: Results & Comparison (`/load-runs/[id]/results`)
* **Key Elements**: Percentile table per chart (p50/p90/p95/p99), error taxonomy breakdown, consistency findings list with hash-pair detail, cache-state annotation per chart (cold/warm), comparison selector against prior run of same profile, blast-radius probe coverage.
## 4. The "Error" Experience
**Philosophy**: Load failures are expected data, not UI exceptions. Every failure is classified, preserved, and analyzable.
### Scenario A: Invalid variation value
* **User Action**: Enters filter value «2024-Q9» that doesn't exist in the target environment.
* **System Response**: (UI) Value chip turns amber, marked `NEEDS_CONTEXT` with «значение отсутствует в среде PROD»; run may start with valid subset only — excluded variations listed explicitly.
* **Recovery**: Fix the value inline (matrix re-expands deterministically) or drop the axis row; no page reload.
### Scenario B: Circuit breaker abort
* **System Response**: Breaker badge flips to red «Прервано: error-rate 34% > 25%»; phase moves to draining; terminal banner explains the trigger and preserved partial results.
* **Recovery**: «Показать частичные результаты» (default), «Повторить с меньшей concurrency» (profile pre-filled with halved value), or export findings.
### Scenario C: PROD gate denial / permission denied
* **System Response**: For denial — audit-recorded cancellation card, zero requests dispatched. For unauthorized — `permission_denied` recovery panel explaining required role (`dashboard:loadtest:prod`), no confirm control rendered.
* **Recovery**: Request access link; or switch environment to PREPROD and start without gate (if policy allows).
### Scenario D: Browser disconnect mid-run
* **System Response**: Run continues server-side. On return, status restores by `load_run_id` — phase, per-variation progress, findings to date.
* **Recovery**: Automatic; no user action needed.
## 5. Tone & Voice
* **Style**: Concise, technical, quantitative. Numbers first («120 запросов, p99 1.8s»), prose second.
* **Terminology**: «Нагрузочный профиль», «вариация», «матрица», «circuit breaker» (kept in English in RU locale), «blast-radius» explained as «затронутые дашборды». Never «SQL» as an option; copy states Superset-native execution explicitly.
* **Safety copy**: PROD gate must state impact in concrete terms: «до N параллельных запросов к среде X; K дашбордов делят датасеты — кэш будет прогрет».
#endregion DashboardLoadTesting.UxReference

60
specs/041-dataset-lineage-blast-radius/checklists/requirements.md Normal file
View File

@@ -0,0 +1,60 @@
# Requirements Checklist: Dataset Lineage & Blast-Radius
**Purpose**: Validate spec.md completeness, clarity, and testability before `/speckit.plan`.
**Created**: 2026-07-22
**Feature**: [spec.md](../spec.md)
## Content Quality
- [x] CHK001 Spec is user/operator-focused; storage/index implementation deferred to plan
- [x] CHK002 All five stories independently testable with stated Independent Tests
- [x] CHK003 Priorities assigned (US1–US3 P1, US4–US5 P2) with justification
- [x] CHK004 Key entities defined without schema detail
## Gap Coverage (BR-1..BR-5 from 036–039 analysis)
- [x] CHK005 BR-1 (reverse index dataset→dashboards) → US1, LIN-FR-001..003, SC-001
- [x] CHK006 BR-2 (cross-dashboard StructureDiff / schema-change classification) → US2, LIN-FR-004..006, SC-002
- [x] CHK007 BR-3 (baseline invalidation single-release scoped) → US3, LIN-FR-007..008, SC-003/004
- [x] CHK008 BR-4 (no fleet-wide trigger) → US5, LIN-FR-010..011, SC-005
- [x] CHK009 BR-5 (C07 too narrow / dataset deprecation absent) → US4, LIN-FR-009, SC-006
## Requirement Quality
- [x] CHK010 Every FR testable (deterministic fingerprints, zero catalog writes, severity ordering, typed errors)
- [x] CHK011 FR IDs hierarchical and stable (`LIN-FR-001..015`)
- [x] CHK012 No `[NEEDS CLARIFICATION]` markers remain
- [x] CHK013 Rejected paths recorded in header `@REJECTED` (QueryModel extension, query-time derivation, auto-invalidation)
## Dependency Traceability
- [x] CHK014 037 dependencies explicit: StructureDiff orthogonality (LIN-FR-006), immutability rules (LIN-FR-007), release pinning (LIN-FR-008)
- [x] CHK015 036 dependencies explicit: approval-gate semantics for PROD fan-out (LIN-FR-014), permission_denied UX
- [x] CHK016 040 integration explicit: blast-radius read model served from pinned index snapshot (LIN-FR-013)
- [x] CHK017 039 integration explicit: FleetReport consumable without AgentRun (LIN-FR-011)
- [x] CHK018 Trigger enum extension `dataset_updated` declared (LIN-FR-011) — requires amendment note to 036/037 at plan time
## Safety & Risk
- [x] CHK019 No silent mutation: propagation marks/reports only; catalog hash invariant verified (SC-003)
- [x] CHK020 Stale-index handling: last-good retained, typed provenance, never empty-as-current (LIN-FR-003, SC-007)
- [x] CHK021 Cycle safety: topological fan-out, typed `lineage_cycle` finding, zero hangs (LIN-FR-015, SC-005)
- [x] CHK022 Environment scoping: cross-env propagation warns `env_mismatch`, never assumes (LIN-FR-012)
## Success Criteria
- [x] CHK023 All SC measurable (100% dependents, 100% classification agreement, zero writes, ordering, zero hangs)
## Clarifications Integrated (2026-07-22)
- [x] CHK028 Graph model corrected: two-level dataset←chart←dashboard, no SQL-source lineage, cycles removed (LIN-FR-015 rewritten)
- [x] CHK029 Usage extraction = C+2 hybrid: structured params exact per viz_type + sqlglot SQL-expression analysis with `unresolved_refs` findings and per-expression conservative fallback (LIN-FR-016/017, SC-008)
- [x] CHK030 Severity Matrix v1 accepted as defaults (11 rules incl. widening=info, rename=warning+hint) — LIN-FR-004 amended
- [x] CHK031 Refresh rides existing `IdMappingService.sync_environment` cycle (migration_sync_cron + sync-now); no dedicated scheduler; post-sync hook bounded to changed uuids; opt-in `lineage_index_enabled` default false (LIN-FR-002/018/019)
## Open Items (deferred to /speckit.plan)
- [ ] CHK024 Index storage model (edges table alongside ResourceMapping vs derived cache) — plan-time research; cadence CLOSED via CHK031 (rides sync_environment cycle)
- [x] CHK025 ~~Classification rules versioning scheme~~ — CLOSED: Severity Matrix v1 accepted; rule changes require version bump + new golden fixtures (clarification Q3)
- [ ] CHK026 Amendment mechanics for 036/037 trigger enum + 040 read-model contract — plan-time coordination
- [x] CHK027 ~~Derived-dataset (SQL-query datasets) lineage source~~ — CLOSED: out of scope, graph is two-level binding graph (clarification Q1)

148
specs/041-dataset-lineage-blast-radius/contracts/modules.md Normal file
View File

@@ -0,0 +1,148 @@
#region DatasetLineageBlastRadius.Modules [C:4] [TYPE ADR] [SEMANTICS contracts,modules,lineage,blast-radius]
@BRIEF GRACE module/function contracts for 041: indexer, extractor, severity rules, propagation, deprecation, fan-out, API, frontend models.
@RELATION DEPENDS_ON -> [DatasetLineageBlastRadius.DataModel]
@RELATION DEPENDS_ON -> [DatasetLineageBlastRadius.Research]
## Backend Modules
```
// #region Services.Lineage.Indexer [C:5] [TYPE Module] [SEMANTICS lineage,indexer,sync]
// @defgroup Lineage Dataset lineage and blast-radius domain.
// @BRIEF Builds and maintains DatasetUsageEdge set per environment; post-sync hook entry point.
// @LAYER Service
// @PRE IdMappingService.sync_environment completed and committed changed ResourceMapping rows.
// @POST Edge set rebuilt for changed uuids only; snapshot fingerprint flipped atomically; stale_index retained on failure.
// @SIDE_EFFECT DB writes (edges, snapshot); Superset detail calls bounded to changed uuids.
// @INVARIANT Detail fetches NEVER exceed the changed-uuid set (cost bound).
// @INVARIANT Snapshot is never deleted on failure — stale_index=true with typed provenance (LIN-FR-003).
// @DATA_CONTRACT Input[environment_id, changed: dict[ResourceType, set[str]]] -> Output[{rebuilt: int, impacts: int}]
// @RELATION DEPENDS_ON -> [Core.MappingService.IdMappingService]
// @RELATION CALLS -> [Services.Lineage.UsageExtractor]
// @RELATION CALLS -> [Services.Lineage.SchemaDiff]
// @RATIONALE Post-commit hook placement keeps network I/O out of the identity-sync transaction (R3).
// @REJECTED Separate APScheduler lineage job — rejected: duplicate scheduler config and split identity/lineage atomicity.
// #region Services.Lineage.Indexer.OnSync [C:4] [TYPE Function] [SEMANTICS lineage,sync-hook]
// @ingroup Lineage
// @BRIEF Post-commit hook invoked by sync_environment with changed-uuid sets.
// @SIDE_EFFECT Superset detail fetches for changed uuids; DB edge rebuild; snapshot update.
// @RELATION CALLED_BY -> [Core.MappingService.SyncEnvironment]
// #endregion Services.Lineage.Indexer.OnSync
// #region Services.Lineage.Indexer.RefreshTargets [C:4] [TYPE Function] [SEMANTICS lineage,write-hook]
// @ingroup Lineage
// @BRIEF Targeted refresh for own mutating operations (migration deploy, dataset create/update).
// @PRE Caller supplies explicit dataset/chart/dashboard ids affected by the mutation.
// @SIDE_EFFECT Immediate edge rebuild for targets without waiting for sync cycle (LIN-FR-018).
// @RELATION CALLED_BY -> [Core.Datasets.SupersetClientUpdateDataset]
// #endregion Services.Lineage.Indexer.RefreshTargets
// #endregion Services.Lineage.Indexer
// #region Services.Lineage.UsageExtractor [C:4] [TYPE Module] [SEMANTICS lineage,extraction,viz]
// @ingroup Lineage
// @BRIEF Extracts consumed columns/metrics from chart params per viz_type; delegates SQL expressions.
// @LAYER Service
// @POST Returns {consumed_columns, consumed_metrics, projection_confidence, unresolved_refs}.
// @INVARIANT Unknown viz_type -> conservative (never guessed exact).
// @RELATION CALLS -> [Services.Lineage.SqlExpressionExtractor]
// @DATA_CONTRACT Input[chart_params: dict, dataset_columns, dataset_metrics] -> Output[UsageEdge projection]
// #endregion Services.Lineage.UsageExtractor
// #region Services.Lineage.SqlExpressionExtractor [C:4] [TYPE Module] [SEMANTICS lineage,sql,sqlparse]
// @ingroup Lineage
// @BRIEF sqlparse-based column-ref extractor for sqlExpression ad-hoc metrics/columns/WHERE (C+2).
// @LAYER Service
// @POST {matched, unresolved, confidence}; parse failure -> conservative for that expression only.
// @INVARIANT Regex-only extraction forbidden (LIN-FR-016); Jinja spans pre-stripped per SqlTableExtractor pattern.
// @RELATION DEPENDS_ON -> [Services.SqlTableExtractor.SqlTableExtractorModule]
// @RATIONALE sqlparse already in-repo (R2); sqlglot rejected as new dependency for marginal gain.
// #endregion Services.Lineage.SqlExpressionExtractor
// #region Services.Lineage.SeverityRulesV1 [C:3] [TYPE Module] [SEMANTICS lineage,severity,rules]
// @ingroup Lineage
// @BRIEF Pure-function Severity Matrix v1: (change_kind, before, after, consumption) -> severity.
// @POST Deterministic; matrix version "v1" embedded in every DatasetImpactRecord (R4).
// @INVARIANT column_added and type widening -> info, NEVER stale baselines (LIN-FR-005).
// #endregion Services.Lineage.SeverityRulesV1
// #region Services.Lineage.SchemaDiff [C:4] [TYPE Module] [SEMANTICS lineage,schema,diff]
// @ingroup Lineage
// @BRIEF Computes dataset schema diffs between observations; projects DatasetImpactRecord per dependent.
// @SIDE_EFFECT Appends impact records (append-only, R4); never mutates baseline catalogs (LIN-FR-006/007).
// @RELATION CALLS -> [Services.Lineage.SeverityRulesV1]
// @DATA_CONTRACT Input[before_schema, after_schema, edges] -> Output[DatasetImpactRecord?]
// #endregion Services.Lineage.SchemaDiff
// #region Services.Lineage.Propagation [C:4] [TYPE Module] [SEMANTICS lineage,staleness,propagation]
// @ingroup Lineage
// @BRIEF Report-time staleness marking for dependent baseline catalogs per release pin.
// @POST Stale/immutability_violation surfaced with dataset-change provenance; zero catalog writes (SC-003).
// @INVARIANT Closed-period divergence -> immutability_violation, never stale (LIN-FR-007, 037 AGBASE-FR-012).
// @INVARIANT Each pinned release evaluated independently (LIN-FR-008).
// @RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec]
// #endregion Services.Lineage.Propagation
// #region Services.Lineage.Deprecation [C:4] [TYPE Module] [SEMANTICS lineage,deprecation]
// @ingroup Lineage
// @BRIEF Deprecation lifecycle: successor, grace window, escalation recompute in sync cycle (LIN-FR-019).
// @SIDE_EFFECT DB writes on mark/migrate; expired state produces typed dataset_deprecated errors.
// @INVARIANT Escalation state derived from (now, deprecated_at, grace_window_days) — stored column is cache (R7).
// #endregion Services.Lineage.Deprecation
// #region Services.Lineage.Fanout [C:5] [TYPE Module] [SEMANTICS lineage,fanout,verification]
// @ingroup Lineage
// @BRIEF Coordinated multi-dashboard verification plans from dataset_updated trigger.
// @LAYER Service
// @PRE DatasetImpactRecord exists with max_severity >= warning.
// @POST One FanoutPlan; per-dashboard VerificationRuns linked by fanout_plan_id, severity-ordered; inspection-only entries with reason.
// @SIDE_EFFECT Creates VerificationRun rows (036/037 model); PROD-classified fan-out requires 036 approval gate (LIN-FR-014).
// @INVARIANT Plan is the only multi-dashboard unit; runs remain per-dashboard records (R8).
// @DATA_CONTRACT Input[dataset_impact_id] -> Output[FanoutPlanDTO + FleetReportDTO]
// @RATIONALE A coordinated plan preserves one dataset-change cause while retaining per-dashboard VerificationRuns for existing 036/037 recovery and reporting semantics.
// @REJECTED Independent ad-hoc verification calls per dashboard — rejected because ordering, approval scope, and fleet aggregation would be lost.
// @RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
// #endregion Services.Lineage.Fanout
// #region Api.Lineage.Routes [C:4] [TYPE Module] [SEMANTICS lineage,api,rest]
// @ingroup Lineage
// @BRIEF REST surface: dependents read model, impacts, deprecation manage, fanout trigger/report.
// @LAYER API
// @INVARIANT RBAC: READ for reads; dataset:lineage:refresh, dataset:deprecation:manage, dataset:fanout:trigger for mutations (LIN-FR-014).
// @DATA_CONTRACT DependentsResponse honors ?fingerprint= pinning; mismatch -> stale_notice (R6).
// @RELATION BINDS_TO -> [EXT:frontend:frontend/src/lib/types/lineage.ts]
// #endregion Api.Lineage.Routes
```
## Frontend Models
```
// #region Datasets.LineageModel [C:4] [TYPE Model] [SEMANTICS lineage,dataset,dependents]
// @BRIEF Screen model for dataset lineage panel: index freshness, dependents, impact badges.
// @STATE index: DependentsResponse | null; staleIndex: boolean; impacts: ImpactRecordDTO[]
// @ACTION load(datasetId, envId, fingerprint?) -> pinned snapshot load; retryRefresh() (role-gated)
// @UX_STATE loading -> ready -> stale_index -> refresh_failed
// @UX_RECOVERY Refresh failure keeps last-good rows with staleness marker; retry available.
// @RELATION BINDS_TO -> [EXT:backend:/api/v1/lineage/datasets/{id}/dependents]
// #endregion Datasets.LineageModel
// #region Datasets.DeprecationModel [C:4] [TYPE Model] [SEMANTICS lineage,deprecation,fsm]
// @BRIEF Deprecation manager FSM: mark, window countdown, migration tracking, expired blocking.
// @STATE record: DeprecationDTO | null; escalationState: noticed|warning|expired_blocked
// @ACTION markDeprecated(successorUuid, graceDays); recordMigration(dashboardUuid)
// @UX_FEEDBACK Countdown chip; escalation color shift; expired -> blocking banner naming successor.
// @RELATION DEPENDS_ON -> [Datasets.LineageModel]
// #endregion Datasets.DeprecationModel
// #region Dashboards.LineageBadgeModel [C:3] [TYPE Model] [SEMANTICS lineage,badge,dashboard]
// @BRIEF Shared-dataset badge on dashboard pages: "делит датасеты с N дашбордами" + impact severity.
// @RELATION BINDS_TO -> [EXT:backend:/api/v1/lineage/dashboards/{id}/shared]
// #endregion Dashboards.LineageBadgeModel
```
## Cross-Stack Edges
- `Api.Lineage.Routes` `@RELATION BINDS_TO -> frontend/src/lib/types/lineage.ts` (DTO mirror)
- `Services.Lineage.Fanout` `@RELATION DEPENDS_ON -> AgentTestStabilization.Spec` (gate semantics) and `@RELATION DEPENDS_ON -> SupersetBaselineEngine.Spec` (VerificationRun)
- Amendment notes (R5): 036/traceability.md + 037/traceability.md gain one-line `dataset_updated` trigger note at implementation time
#endregion DatasetLineageBlastRadius.Modules

101
specs/041-dataset-lineage-blast-radius/data-model.md Normal file
View File

@@ -0,0 +1,101 @@
#region DatasetLineageBlastRadius.DataModel [C:3] [TYPE ADR] [SEMANTICS data-model,lineage,orm,dto]
@BRIEF Phase 1 data model: ORM tables, DTO shapes, and invariants for dataset lineage index, impacts, deprecation, and fan-out.
**Feature**: 041-dataset-lineage-blast-radius | **Date**: 2026-07-22
## ORM Tables (backend/src/models/lineage.py — new)
### DatasetUsageEdge
| Column | Type | Notes |
|---|---|---|
| id | str (uuid) PK | |
| environment_id | str FK | env scope (LIN-FR-012) |
| dataset_uuid | str | from ResourceMapping (identity stable across re-ids) |
| chart_uuid | str | |
| dashboard_uuid | str | |
| dashboard_title | str | denormalized for panel rendering |
| consumed_columns | JSONB list[str] | exact set (structured params + matched SQL refs) |
| consumed_metrics | JSONB list[str] | |
| projection_confidence | enum(exact, conservative) | LIN-FR-001 |
| unresolved_refs | JSONB list[str] | SQL refs matching nothing (LIN-FR-016) |
| edge_fingerprint | str | sha256 of edge content |
Unique: `(environment_id, dataset_uuid, chart_uuid, dashboard_uuid)`.
### LineageIndexSnapshot
| Column | Type | Notes |
|---|---|---|
| environment_id | str PK | one current snapshot per env |
| index_fingerprint | str | sha256 over sorted edge_fingerprints |
| built_at | datetime | |
| stale_index | bool | LIN-FR-003 |
| last_error | JSONB | typed provenance {kind, message, at} |
### DatasetSchemaObservation
| Column | Type | Notes |
|---|---|---|
| id | str PK | |
| environment_id, dataset_uuid | str | unique together with observed_at ordering |
| schema_hash | str | sha256 of canonical schema (columns+types+metrics+nullability) |
| schema_payload | JSONB | canonical schema |
| observed_at | datetime | |
### DatasetImpactRecord
| Column | Type | Notes |
|---|---|---|
| id | str PK | |
| environment_id, dataset_uuid | str | |
| diff | JSONB | classified changes per Severity Matrix v1 |
| rules_version | str | e.g. "v1" (R4) |
| max_severity | enum(critical, warning, info) | |
| affected_dashboards | JSONB | per-dashboard: charts, baseline entries by release pin, scenario step refs |
| observation_pair | JSONB | {before_id, after_id} |
| created_at | datetime | monotonic sequence per dataset (edge case: racing changes) |
### DatasetDeprecation
| Column | Type | Notes |
|---|---|---|
| environment_id, dataset_uuid | str PK | |
| successor_dataset_uuid | str | LIN-FR-009 |
| deprecated_at | datetime | |
| grace_window_days | int | |
| escalation_state | enum(noticed, warning, expired_blocked) | derived cache (R7) |
| expired_at | datetime null | |
| dependent_status | JSONB | per-dashboard migrated/remaining |
### FanoutPlan
| Column | Type | Notes |
|---|---|---|
| id | str PK | plan id linked by runs (LIN-FR-010) |
| environment_id | str | |
| dataset_impact_id | str FK → DatasetImpactRecord | |
| trigger | str | `dataset_updated` |
| status | enum(planning, running, completed, failed) | |
| entries | JSONB | ordered: dashboard, severity, inspection_only+reason |
| created_at | datetime | |
### VerificationRun amendment (036/037 model)
+ `fanout_plan_id` str FK null (additive; null = ordinary run)
+ `trigger` enum gains `dataset_updated` (R5)
## DTOs (backend/src/schemas/lineage.py, Pydantic extra-forbid)
- `DependentsResponse {fingerprint, stale_index, env_scope, dependents: [{dashboard_id, title, chart_count, projection_confidence}]}` — 040 read model (R6)
- `ImpactRecordDTO {id, dataset, rules_version, max_severity, diff, affected_dashboards, created_at}`
- `DeprecationDTO {dataset, successor, deprecated_at, grace_window_days, escalation_state, dependent_status}`
- `FanoutPlanDTO {id, trigger, status, entries, fleet_summary?}`
- `FleetReportDTO {plan_id, per_dashboard: [{dashboard, status, provenance, unresolved_impacts}]}` — 039-consumable (LIN-FR-011)
## Frontend DTOs (frontend/src/lib/types/lineage.ts)
Mirror of backend schemas, generated/hand-authored 1:1; cross-stack `@RELATION` edges declared in contracts/modules.md.
## Invariants
1. Edge rows are disposable: full rebuild = delete+insert within one transaction; snapshot fingerprint flips atomically.
2. Snapshot is never deleted on refresh failure — `stale_index=true` + provenance (LIN-FR-003).
3. Impact records are append-only; classification never mutates history (R4).
4. `escalation_state` is a derived cache; source of truth = `(now, deprecated_at, grace_window_days)`.
5. No table here references baseline catalog files — propagation is report-time marking (LIN-FR-007).
#endregion DatasetLineageBlastRadius.DataModel

113
specs/041-dataset-lineage-blast-radius/plan.md Normal file
View File

@@ -0,0 +1,113 @@
# Implementation Plan: Dataset Lineage & Blast-Radius
**Branch**: `041-dataset-lineage-blast-radius` | **Date**: 2026-07-22 | **Spec**: [spec.md](spec.md)
**Input**: Feature specification from `/specs/041-dataset-lineage-blast-radius/spec.md` (clarified 2026-07-22)
## Summary
Build a dataset→dashboard reverse index riding the existing `IdMappingService.sync_environment` cycle (no new scheduler), with C+2 usage extraction (structured chart params + sqlparse SQL-expression analysis), Severity Matrix v1 classification, report-time staleness propagation with zero baseline-catalog writes, deprecation lifecycle, and coordinated `dataset_updated` verification fan-out. Closes blast-radius gaps BR-1..BR-5 identified across specs 036–039; provides the pinned read model consumed by 040 load testing.
## Technical Context
**Language/Version**: Python 3.13+ (backend), TypeScript (frontend Svelte 5 runes-only)
**Primary Dependencies**: FastAPI 0.126, SQLAlchemy 2.0.45, APScheduler 3.11.2 (existing, via migration sync), sqlparse ≥0.5 (existing — R2, no new deps); SvelteKit 2.49 / Svelte 5.56, Tailwind (frontend)
**Storage**: PostgreSQL 16 — 5 new tables + 1 additive FK (data-model.md); Alembic migration
**Testing**: pytest (backend: indexer/extractor/diff/propagation/fanout suites), vitest L1 (LineageModel/DeprecationModel) + L2 UX (@testing-library/svelte)
**Target Platform**: Linux server (Docker), modern browsers
**Project Type**: web application (FastAPI REST backend, SvelteKit SPA frontend)
**Frontend Architecture**: model-first Screen Models (`.svelte.ts`), runes-only, typed DTOs mirroring Pydantic
**Performance Goals**: dependents read model <200ms p95 from snapshot table; incremental sync hook adds ≤ changed-set detail calls (0–20 typical, never O(fleet))
**Constraints**: RBAC enforced (LIN-FR-014); zero writes to 037 baseline catalogs (SC-003); opt-in `lineage_index_enabled` default false; no new runtime dependencies
**Scale/Scope**: ~100 dashboards/env, ~300 charts, ~80 datasets; sync cycle 30min default via existing `migration_sync_cron`
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* — **PASS** (initial, 2026-07-22)
| Principle | Status | Evidence |
|---|---|---|
| I. Semantic Contract First | ✅ | contracts/modules.md: all C3+ with anchors, hierarchical IDs (`Services.Lineage.*`), shared `@SEMANTICS lineage` |
| II. Decision Memory | ✅ | 5 `@REJECTED` across spec header + contracts (QueryModel extension, query-time derivation, auto-invalidation, sqlglot, separate scheduler); R1–R8 rationale records |
| III. External Orchestrator | ✅ | Reads Superset metadata via existing SupersetClient boundary; no Superset-side plugin |
| IV. Module Discipline | ✅ | 8 backend modules, each <400 LOC by design (extractor/diff/propagation separated); functions CC≤10 via pure-function severity rules |
| V. RBAC Enforcement | ✅ | LIN-FR-014: three new granular permissions + default-deny; PROD fan-out via 036 gate |
| VI. Svelte 5 Runes Only | ✅ | Models `.svelte.ts` with `$state`; no stores/legacy syntax |
| VII. Test-Driven C3+ | ✅ | quickstart.md defines falsifiable sequence incl. explicit `@REJECTED`-path tests (regex-extraction ban, auto-invalidation ban via SC-003) |
| VIII. Attention Optimization | ✅ | ATTN_1 one-line anchors; ATTN_2 `Services.Lineage.*`; ATTN_3 shared `lineage` keyword; ATTN_4 contracts ≤150 lines |
**Post-Phase-1 re-check**: PASS — no violations introduced; no Complexity Tracking entries required.
## Project Structure
### Documentation (this feature)
```text
specs/041-dataset-lineage-blast-radius/
├── plan.md # This file
├── research.md # Phase 0 — R1..R8 decisions
├── data-model.md # Phase 1 — ORM + DTOs + invariants
├── quickstart.md # Phase 1 — falsifiable verification sequence
├── traceability.md # RTM (below)
├── checklists/requirements.md
├── contracts/
│ └── modules.md # GRACE module/function contracts
└── tasks.md # Phase 2 output (/speckit.tasks)
```
### Source Code (repository root)
```text
backend/
├── src/
│ ├── models/lineage.py # NEW: 5 ORM tables (data-model.md)
│ ├── schemas/lineage.py # NEW: Pydantic DTOs (extra-forbid)
│ ├── services/lineage/
│ │ ├── indexer.py # OnSync hook + RefreshTargets write-hook
│ │ ├── usage_extractor.py # per-viz_type structured extraction
│ │ ├── sql_expression_extractor.py # sqlparse C+2 analysis
│ │ ├── severity_rules_v1.py # pure-function matrix
│ │ ├── schema_diff.py # observation diff + impact projection
│ │ ├── propagation.py # report-time staleness marking
│ │ ├── deprecation.py # lifecycle + escalation recompute
│ │ └── fanout.py # plan + VerificationRun spawning
│ ├── api/routes/lineage.py # REST surface
│ └── core/mapping_service.py # AMEND: post-commit OnSync hook call
├── alembic/versions/ # NEW migration (5 tables + VerificationRun.fanout_plan_id)
└── tests/
├── services/lineage/ # unit + property tests per module
├── api/test_lineage.py # RBAC/contract tests
└── fixtures/lineage/ # 3 dashboards, 2 shared datasets, SQL-expression charts
frontend/
├── src/lib/models/
│ ├── Datasets.LineageModel.svelte.ts # NEW
│ ├── Datasets.DeprecationModel.svelte.ts # NEW
│ └── Dashboards.LineageBadgeModel.svelte.ts # NEW
├── src/lib/types/lineage.ts # NEW: DTO mirror
└── src/lib/components/datasets/lineage/ # NEW: panel components (039-adjacent)
```
**Structure Decision**: Backend-heavy feature; frontend limited to read panels + deprecation manager (fits ADR-0001 layout). `core/mapping_service.py` amendment is a single post-commit hook call — no refactor of identity sync.
## Semantic Contract Guidance
Applied per template §Attention Compliance Gate — all contracts in `contracts/modules.md` validated against ATTN_1–4 (see Constitution Check). Cross-stack edges declared: `Api.Lineage.Routes BINDS_TO frontend/src/lib/types/lineage.ts`; fan-out edges to 036/037 spec nodes; amendment notes for trigger enum recorded in R5.
## Complexity Tracking
No constitution violations — section intentionally empty.
## Phase Outputs
| Phase | Artifact | Status |
|---|---|---|
| 0 | research.md | ✅ R1 storage, R2 sqlparse, R3 hook placement, R4 rules versioning, R5 enum amendment, R6 040 read model, R7 deprecation, R8 fanout persistence |
| 1 | data-model.md, contracts/modules.md, quickstart.md | ✅ |
| 2 | tasks.md | ⏭ next: `/speckit.tasks` |
## ADR Continuity
- ADR-0001 module layout: new `services/lineage/` package, routes in `api/routes/lineage.py` — compliant
- ADR-0003 orchestrator: metadata reads via existing SupersetClient — compliant
- ADR-0005 RBAC: new permissions registered in `rbac_permission_catalog.py` at implementation
- No ADR amendments required; R5 spec-text notes for 036/037 deferred to implementation (branch isolation)

30
specs/041-dataset-lineage-blast-radius/quickstart.md Normal file
View File

@@ -0,0 +1,30 @@
#region DatasetLineageBlastRadius.Quickstart [C:2] [TYPE ADR] [SEMANTICS quickstart,lineage,verification]
@BRIEF Smallest falsifiable verification path for 041 (per Verifiable Edit Loop).
**Feature**: 041-dataset-lineage-blast-radius | **Date**: 2026-07-22
## Fixture Setup
1. Seed 3 fixture dashboards sharing 2 datasets (D1 shared by Dash-A/Dash-B, D2 only Dash-C) in `backend/tests/fixtures/lineage/`.
2. One chart with structured params consuming `amount, status`; one chart with `sqlExpression: "SUM(amount)/NULLIF(cnt,0)"`; one chart with unparseable expression.
## Verification Sequence
1. **Index build**: run `Indexer.OnSync(env, changed=all)` → assert edges: D1→{A,B}, D2→{C}; fingerprint byte-identical across 2 runs and shuffled input (SC-001).
2. **SQL extraction**: expression chart → `matched={amount,cnt}`, confidence exact; unknown-ref fixture → `unresolved_refs` finding (SC-008).
3. **Schema diff**: apply fixture `column_removed(amount)` → Dash-A/B records critical, Dash-C none; `column_added` → info, zero baseline marks (LIN-FR-005).
4. **Propagation zero-write**: hash baseline catalogs before/after propagation → identical (SC-003); closed-period retroactive change → `immutability_violation`, not stale (SC-004).
5. **Deprecation**: mark D1 deprecated (successor D2, window 0) → `dataset_deprecated` error names successor (SC-006).
6. **Fan-out**: `dataset_updated` on D1 → one plan; runs ordered critical-first; Dash-C absent (no D1 usage); fleet report aggregated (SC-005).
7. **Sync integration**: `sync_environment(incremental=True)` with 1 changed chart → exactly 1 detail fetch; failure mid-hook → snapshot `stale_index=true`, last good served (SC-007).
8. **RBAC**: unauthenticated fan-out trigger → `permission_denied`, no confirm control (LIN-FR-014).
## Commands
```bash
cd backend && source .venv/bin/activate && python -m pytest tests/services/lineage/ -v
cd backend && python -m ruff check .
cd frontend && npm run test -- lineage
```
#endregion DatasetLineageBlastRadius.Quickstart

66
specs/041-dataset-lineage-blast-radius/research.md Normal file
View File

@@ -0,0 +1,66 @@
#region DatasetLineageBlastRadius.Research [C:3] [TYPE ADR] [SEMANTICS research,lineage,blast-radius,storage,sqlglot]
@BRIEF Phase 0 research: resolves all material unknowns for dataset lineage — storage model, SQL-expression parsing, sync-hook placement, severity rules versioning, trigger-enum amendments, and 040 read-model contract.
**Feature**: 041-dataset-lineage-blast-radius | **Date**: 2026-07-22
## R1 — Index Storage Model (CHK024)
- **Decision**: Dedicated `DatasetUsageEdge` ORM table (backend/src/models/lineage.py) alongside `ResourceMapping`; edge rows keyed `(environment_id, dataset_uuid, chart_uuid, dashboard_uuid)` with consumed columns/metrics as JSONB + `projection_confidence` + `unresolved_refs` JSONB. Separate `LineageIndexSnapshot` table stores per-env `index_fingerprint`, `built_at`, `stale_index` flag, error provenance.
- **Rationale**: `ResourceMapping` is an uuid↔id identity table with sync cadence semantics — bolting usage detail onto it would couple identity sync failures to lineage availability and bloat the hot migration path. A dedicated table allows independent rebuild (full edge wipe + rebuild from current mappings) without touching migration state. JSONB for consumed sets avoids a join-table for data that is only ever read whole.
- **Alternatives Considered**: (a) Derived in-memory cache rebuilt per request — rejected: cold-start O(fleet) per consumer, no snapshot pinning across restarts, violates LIN-FR-002. (b) Extend ResourceMapping with JSONB usage column — rejected: couples migration sync to lineage rebuild; stale-deletion semantics differ (identity row deleted vs edge recomputed).
- **Impact**: Alembic migration adds 2 tables. `stale_index` = snapshot row flag, not edge deletion — LIN-FR-003 satisfied by retaining last good snapshot.
## R2 — SQL-Expression Parsing: sqlparse, not sqlglot (Q2 implementation)
- **Decision**: Use **sqlparse** (already in backend/requirements-backend.txt:64) with a new `Services.Lineage.SqlExpressionExtractor` modeled on the existing `Services.SqlTableExtractor` three-phase pattern (Jinja span detection → SQL spans → token analysis). Column refs extracted from DML tokens; matched against dataset columns/metrics from authoritative metadata. Parse failure or non-DML ambiguity → expression marked conservative.
- **Rationale**: sqlglot is NOT a project dependency — adding it for one feature violates minimal-dependency discipline and pulls a 1MB+ parser for a task sqlparse already handles in-repo (SqlTableExtractor proves the pattern incl. Jinja). Chart SQL expressions are dialect-simple (aggregations, CASE, arithmetic) — full sqlglot dialect resolution is overkill.
- **Alternatives Considered**: (a) sqlglot — rejected: new dependency, marginal accuracy gain over sqlparse for Superset-generated expressions; can be adopted later behind the same extractor contract if fixture failure rate justifies it. (b) Regex-only — forbidden by LIN-FR-016.
- **Impact**: No new dependencies. Extractor contract: `extract(sql_expression, known_columns, known_metrics) -> {matched: set[str], unresolved: set[str], confidence: exact|conservative}`.
## R3 — Sync-Hook Placement (Q4 implementation)
- **Decision**: Hook into `IdMappingService.sync_environment` completion: after `self.db.commit()`, emit changed-uuid sets per resource type to `Services.Lineage.Indexer.OnSync(environment_id, changed: dict[ResourceType, set[str]])`. Detail fetches (chart params, dataset schema) via SupersetClient ONLY for changed uuids. Write-hook: `SupersetClient.update_dataset/create_dataset` and migration deploy completion call `Indexer.RefreshTargets(environment_id, dataset_ids=[...])` directly.
- **Rationale**: sync_environment already computes incremental change sets (since_dttm diff) — the changed-uuid information exists there and nowhere else; hooking at commit boundary guarantees lineage sees exactly what identity-sync persisted. Detail-call bounding to changed set keeps Superset load at O(changed), not O(fleet).
- **Alternatives Considered**: (a) Separate APScheduler job `lineage_refresh_{env_id}` — rejected by user: duplicate scheduler config, splits atomicity (mappings say one thing, lineage another until next run). (b) Refresh inside the same DB transaction — rejected: detail fetches are network I/O inside a DB transaction = lock holding; must be post-commit.
- **Impact**: `sync_environment` gains one out-param/callback; transaction boundary unchanged. Task visibility: sync-now already surfaces per-env results; lineage hook appends `lineage: {rebuilt: N, impacts: M}` to that result envelope.
## R4 — Severity Rules Versioning (CHK025 closed at clarify, mechanics here)
- **Decision**: `severity_rules_v1.py` module with pure functions `(change_kind, before, after, consumption) -> severity`; matrix version constant embedded in `DatasetImpactRecord.rules_version`. Rule change = new module `severity_rules_v2.py` + dispatcher; old records keep their version; golden fixtures pinned per version.
- **Rationale**: Pure-function modules keep classification deterministic and property-testable; embedding version in each record preserves audit trail when rules evolve.
- **Impact**: No DB migration on rule bump — version is data. Re-classification of historical records is an explicit re-index operation, never implicit.
## R5 — Trigger Enum Amendment Mechanics (CHK026)
- **Decision**: The `trigger` field on AgentRun/VerificationRun (036 spec, `VerificationRun` entity 037) gains value `dataset_updated`. Amendment executed as **additive enum extension in 041's own migration** with a backward-compatible default; 036/037 spec texts receive a one-line amendment note in their traceability.md (not a spec rewrite) when 041 implements.
- **Rationale**: Enum values are data-level, not schema-level — adding a value is backward compatible (old code sees unknown trigger as opaque string). Editing 036/037 spec files now would violate branch isolation; amendment notes land at implementation time.
- **Impact**: 041 contracts declare `dataset_updated` as the producer; 037 VerificationRun consumer treats unknown triggers as displayable strings (already true per 039 AGUI-FR-016 read-only projection).
## R6 — 040 Read-Model Contract (LIN-FR-013)
- **Decision**: `GET /api/v1/lineage/datasets/{dataset_id}/dependents?env_id=...&fingerprint=...` returns pinned-snapshot view: `{fingerprint, stale_index, dependents: [{dashboard_id, title, chart_count, projection_confidence}], env_scope}`. 040 load profiles and PROD gates pass the fingerprint they validated against; mismatch → re-fetch notice, never silent drift.
- **Rationale**: Fingerprint-as-query-param makes pinning explicit and stateless; server never guesses which snapshot the caller holds.
- **Impact**: One read endpoint; 040 contracts bind to this shape.
## R7 — Deprecation Storage & Escalation (US4 mechanics)
- **Decision**: `DatasetDeprecation` ORM table: `(environment_id, dataset_uuid, successor_dataset_uuid, deprecated_at, grace_window_days, escalation_state, expired_at)` + per-dependent migration status JSONB. Escalation recomputed in sync cycle (LIN-FR-019) by pure function `(now, deprecated_at, grace_window_days, migrated_ratio) -> state`.
- **Rationale**: Escalation as derived state (not stored transitions) eliminates timer-drift bugs; stored column is a cache of last computed value for query convenience.
- **Impact**: One more table in the same Alembic migration.
## R8 — Fan-Out Plan Persistence (US5 mechanics)
- **Decision**: `FanoutPlan` ORM table `(id, environment_id, dataset_impact_id, created_at, trigger, status)`; per-dashboard runs recorded as VerificationRun rows with `fanout_plan_id` FK (nullable — null = ordinary run). Inspection-only entries are plan rows without spawned runs, reason stored.
- **Rationale**: Reuses 036/037 VerificationRun as the execution record — FleetReport is a query over plan + runs, no parallel run model.
- **Impact**: VerificationRun gains one nullable FK column (additive, backward compatible).
## Resolved Risks
| Risk | Mitigation |
|---|---|
| Superset list API lacks params/schema → detail-call storm | Bounded to changed uuids only (R3); full rebuild is explicit operator action with cost warning |
| sqlparse mis-parses exotic dialect | Conservative fallback per expression (LIN-FR-016); fixture suite covers ClickHouse/Postgres expression shapes |
| Post-commit hook failure leaves lineage behind mappings | Snapshot marked `stale_index` with provenance (LIN-FR-003); next sync cycle retries changed set (idempotent rebuild) |
| Enum amendment races with 036/037 implementation | Additive-only, opaque-string tolerance already required (R5) |
#endregion DatasetLineageBlastRadius.Research

168
specs/041-dataset-lineage-blast-radius/spec.md Normal file
View File

@@ -0,0 +1,168 @@
#region DatasetLineageBlastRadius.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,lineage,blast-radius,dataset,dashboard-testing]
@BRIEF Dataset lineage and blast-radius analysis: reverse index dataset→dashboards, cross-dashboard schema-change impact classification, staleness propagation across releases, deprecation lifecycle, and verification fan-out.
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0001]
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0003]
@RELATION DEPENDS_ON -> [Doc.Adr.ADR0005]
@RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec]
@RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
@RELATION DEPENDS_ON -> [Core.MappingService.IdMappingService]
@RATIONALE Specs 036–040 model every artifact per-dashboard: DashboardQueryModel is built for one dashboard, StructureDiff compares two releases of one dashboard, baselines pin to one dashboard's release, VerificationRun triggers fire for one dashboard. But datasets are shared N:M across dashboards — a schema change, deprecation, or data regression in one dataset silently breaks every dependent dashboard, and no current artifact can name them. The reverse index dataset→dashboards is the missing primitive that makes impact analysis computable instead of tribal knowledge.
@REJECTED Extending DashboardQueryModel with reverse lookups — rejected because QueryModel is a per-dashboard projection; embedding cross-dashboard lineage would couple every inspection to fleet-wide state and break its deterministic, single-dashboard fingerprint semantics (037 AGBASE-FR-001).
@REJECTED Deriving lineage at query time by scanning all dashboards per request — rejected because it is O(fleet) per lookup, non-deterministic under concurrent dashboard edits, and unusable inside hot paths (load-profile preview, PROD gates, release checks).
@REJECTED Propagating staleness by auto-invalidating dependent baselines — rejected because a dataset schema change does not always invalidate downstream truth (additive columns are safe); blind invalidation destroys approved baselines and violates the 037 immutability model. Impact must be classified and surfaced, not silently enforced.
## Navigation (DSA Indexer keywords)
@SEMANTICS: spec, requirements, feature, lineage, blast-radius, dataset, schema-change, staleness-propagation, deprecation, fan-out, dashboard-testing
**Feature Branch**: `041-dataset-lineage-blast-radius`
**Created**: 2026-07-22 | **Status**: Draft
**Input**: "Dataset lineage и blast-radius анализ: обратный индекс датасет→дашборды, cross-dashboard structure diff, классификация влияния изменений схемы датасета, распространение staleness между релизами, fan-out триггеры верификации зависимых дашбордов. Закрывает gaps BR-1..BR-5 из анализа спек 036–039."
## Clarifications
### Session 2026-07-22
- **Q1 (graph model)**: Blast-radius graph is two-level dataset ← chart ← dashboard — NOT data-lineage of dataset SQL sources. No SQL parsing of `dataset.sql`; cycles/topological ordering removed (impossible by construction). → LIN-FR-015 rewritten.
- **Q2 (usage extraction)**: C+2 hybrid — structured chart params extracted per viz_type (exact); custom SQL expressions (`sqlExpression` ad-hoc metrics/columns/WHERE) parsed via sqlglot: column refs matched against dataset columns/metrics → exact edges; unmatched refs → `unresolved_refs` finding; parse failure → conservative per-expression only (not whole chart). `projection_confidence: exact|conservative` per chart. → LIN-FR-001 rewritten, LIN-FR-016/017 added.
- **Q3 (severity matrix)**: Classification rules v1 accepted as defaults — see Severity Matrix v1 section. → LIN-FR-004 amended.
- **Q4 (refresh cadence)**: No separate lineage cron — index refresh rides the existing **Cross-Environment ID Synchronization** cycle (`IdMappingService.sync_environment`, `migration_sync_cron` + Migration Settings UI + sync-now). A post-sync hook rebuilds edges only for changed uuids (list API lacks params/schema → detail calls bounded to changed set). Opt-in via `lineage_index_enabled` (default false). Write-hook: our own mutating operations (migration deploy, `update_dataset`) trigger targeted refresh of affected ids without waiting for cron. Deprecation escalation states are recomputed in the same cycle. → LIN-FR-002 rewritten, LIN-FR-018/019 added.
## User Scenarios
### Story 1 — Dataset Usage Index (P1)
**Why P1**: Every blast-radius feature — 040 load panels, PROD gates, deprecation warnings — depends on answering «какие дашборды используют этот датасет?» deterministically and fast.
**Independent Test**: Build the index from fixture Superset metadata (3 dashboards sharing 2 datasets) and verify reverse lookup returns exact dependent dashboard sets, chart-level usage detail, and a deterministic index fingerprint.
**Acceptance**:
1. **Given** authoritative Superset dashboard/chart/dataset metadata **When** the index builds **Then** every dataset maps to its dependent dashboards with per-chart usage detail (which chart, which columns/metrics consumed).
2. **Given** a dashboard is added, removed, or its charts re-pointed to another dataset **When** the index refreshes **Then** reverse mappings reflect the change and the index fingerprint changes deterministically.
3. **Given** metadata is inaccessible (Superset 403/timeout) **When** refresh runs **Then** the previous index is retained with a `stale_index` marker and typed error provenance — never a partial or empty index silently served.
---
### Story 2 — Cross-Dashboard Schema Change Impact (P1)
**Why P1**: A dataset schema change is a fleet event, not a single-dashboard event; each dependent dashboard must get a classified impact, not a surprise failure at next verification.
**Independent Test**: Apply fixture schema changes (column removed, column type changed, column added) to a shared dataset and verify each dependent dashboard receives a classified `DatasetImpactRecord` with severity and affected artifacts.
**Acceptance**:
1. **Given** a dataset's schema changes between two observations **When** the diff is computed **Then** changes classify by kind (`column_removed`, `type_changed`, `column_added`, `metric_removed`, `dataset_deleted`) and severity (`critical`, `warning`, `info`).
2. **Given** a classified change **When** impact projects to dependent dashboards **Then** each dashboard record names affected charts, affected baseline entries (by release pin), and affected scenario steps (038 refs) — additive-only changes classify `info` and never mark baselines stale.
3. **Given** a dependent dashboard's 037 StructureDiff exists between its own releases **When** a dataset change occurs **Then** the dataset impact record links to (but does not mutate) that dashboard's per-release StructureDiff — cross-dashboard impact and per-dashboard diff stay orthogonal.
---
### Story 3 — Staleness Propagation Across Releases (P1)
**Why P1**: 037 pins baselines to one dashboard's release; when a dataset changes, dependent dashboards on *other* releases need their baseline staleness re-evaluated without violating immutability rules.
**Independent Test**: Change a shared dataset after dashboards A and B approved baselines on different releases; verify affected entries in both catalogs surface `stale_baseline` (or `immutability_violation` for closed periods) with dataset-change provenance.
**Acceptance**:
1. **Given** a dataset change classified `critical` or `warning` **When** dependent baseline catalogs load **Then** affected entries are marked or reported stale with provenance pointing to the dataset impact record — catalog files themselves are not rewritten by the propagation.
2. **Given** a closed-period baseline entry whose dataset data changed retroactively **When** `source_response_hash` diverges **Then** propagation surfaces `immutability_violation` per 037 AGBASE-FR-012 — never downgraded to a stale warning.
3. **Given** dashboards pin different `release_version` for the same dataset **When** propagation evaluates **Then** each release's affected entries are reported independently; one release's re-approval never auto-heals another release's staleness.
---
### Story 4 — Dataset Deprecation Lifecycle (P2)
**Why P2**: Datasets are replaced, not deleted cleanly; dependents need a managed migration path with deadlines and escalation, not a 404 at verification time.
**Independent Test**: Mark a fixture dataset deprecated with a successor and grace window; verify dependents receive deprecation notices, escalating warnings as the window closes, and a final blocking state at expiry.
**Acceptance**:
1. **Given** a dataset is marked deprecated with `successor_dataset_id` and grace window **When** dependents are enumerated **Then** every dependent dashboard owner-visible surface shows the deprecation, successor, and remaining window.
2. **Given** the grace window closes **When** verification or scenario generation touches the deprecated dataset **Then** operations warning-gate, then block at expiry with a typed `dataset_deprecated` error naming the successor.
3. **Given** a dependent dashboard migrates its charts to the successor **When** the index refreshes **Then** the deprecation record tracks migrated vs remaining dependents until zero remain.
---
### Story 5 — Verification Fan-Out for Dependent Dashboards (P2)
**Why P1 value, P2 cost**: After a dataset change, affected dashboards need re-verification prioritized by impact severity — a fleet-level trigger the current per-dashboard VerificationRun model cannot express.
**Independent Test**: Trigger `dataset_updated` on a fixture dataset with 3 dependents at different severities; verify fan-out creates one coordinated plan, per-dashboard VerificationRuns link to it, and results aggregate into a fleet report.
**Acceptance**:
1. **Given** a `dataset_updated` trigger with a classified impact record **When** fan-out executes **Then** dependent dashboards receive verification runs ordered by impact severity (critical first), each linked to the shared fan-out plan id.
2. **Given** a dependent dashboard lacks a current release or approved baselines **When** fan-out plans **Then** it is scheduled as inspection-only (query model refresh) rather than comparison, with the reason recorded.
3. **Given** fan-out completes **When** results aggregate **Then** a fleet report shows per-dashboard status, dataset-change provenance, and unresolved impacts; the report is consumable by 039 pipeline views without an AgentRun.
---
### Edge Cases
- Dataset used by zero dashboards → index returns empty dependents; deprecation and fan-out complete trivially without errors.
- Chart SQL expression references a non-existent column → `unresolved_refs` finding at index build; the chart is not silently treated as exact-confidence.
- Dataset deleted without deprecation → `dataset_deleted` critical impact on all dependents; verification fan-out marks comparisons `inconclusive`, never invents metadata (037 US1 acceptance 3 semantics).
- Two dataset changes race (schema diff computing while another lands) → impact records are versioned per observation; consumers see a monotonic sequence, never a merged ambiguous diff.
- Dependent dashboard's environment differs from the dataset-change observation environment → impact records carry environment scope; cross-environment propagation is reported as `env_mismatch` warning, not assumed valid.
- Index refresh during an active 040 load run → load run pins the index snapshot it validated against; mid-run index updates do not alter its blast-radius panel retroactively.
- Sync cycle runs while `lineage_index_enabled=false` → mappings sync proceeds unchanged; lineage hook is inert (zero Superset calls from the lineage side).
- Migration deploy re-points charts to another dataset → targeted write-hook refresh fires immediately; blast-radius panels reflect the new binding without waiting for the sync cycle.
## Requirements
### Functional
- **LIN-FR-001**: The system MUST maintain a dataset→dashboard reverse index built from authoritative Superset metadata (dataset ← chart ← dashboard bindings), with per-chart usage detail: consumed columns/metrics extracted from structured chart params per viz_type (exact), plus SQL-expression analysis for custom ad-hoc metrics/columns. Every chart edge carries `projection_confidence: exact|conservative`. Index fingerprint MUST be deterministic.
- **LIN-FR-016**: Custom SQL expressions (`sqlExpression` in ad-hoc metrics/columns/WHERE clauses) MUST be parsed (sqlglot): matched column/metric refs produce exact edges; refs matching no dataset column/metric produce an `unresolved_refs` finding per chart (possible pre-existing bug); parse failure marks only that expression conservative, never the whole chart. Regex-only extraction is forbidden.
- **LIN-FR-002**: The index MUST be incrementally refreshable and content-addressed; consumers MUST be able to pin a snapshot (index fingerprint) for the duration of an operation (load run, release check, fan-out plan). Index refresh MUST ride the existing Cross-Environment ID Synchronization cycle (`IdMappingService.sync_environment` with its `migration_sync_cron`, incremental mode, and stale-deletion) — a dedicated lineage scheduler is forbidden. A post-sync hook rebuilds edges only for changed uuids; list-API payloads lack chart params and dataset schema, so detail calls MUST be bounded to the changed set. The feature is opt-in via `lineage_index_enabled` setting (default false).
- **LIN-FR-018**: Own mutating operations (migration deploy, dataset create/update via SupersetClient) MUST trigger a targeted refresh of the affected dataset/chart/dashboard ids without waiting for the next sync cycle; external edits made directly in Superset UI are picked up by the incremental sync cycle.
- **LIN-FR-019**: Deprecation escalation states (`noticed → warning → expired_blocked`) MUST be recomputed within the same sync cycle from the grace window — no separate deprecation timer.
- **LIN-FR-003**: On refresh failure the system MUST retain the last good index marked `stale_index` with typed error provenance; serving an empty or partial index as current is forbidden.
- **LIN-FR-004**: The system MUST compute dataset schema diffs between observations, classifying changes by kind and severity per **Severity Matrix v1** (deterministic, versioned; rule changes require matrix version bump and new golden fixtures).
### Severity Matrix v1 (accepted 2026-07-22)
| Change | Severity | Rationale |
|---|---|---|
| `dataset_deleted` | critical | All dependents break |
| `column_removed` (consumed by exact-confidence charts) | critical | Queries fail |
| `column_removed` (not consumed / conservative-only) | warning | May hide in SQL expressions |
| `metric_removed` | critical | Metrics are always consumed by reference |
| `type_changed` breaking (varchar→int, date→varchar, any→bool) | critical | Cast errors / semantics change |
| `type_changed` narrowing (bigint→int, decimal(12,2)→(10,2), varchar(200)→(50)) | warning | May truncate data |
| `type_changed` widening (int→bigint, decimal(10,2)→(12,2), varchar(50)→(200)) | info | Backward compatible; may shift 037 `source_response_hash` |
| `column_added` | info | Additive; MUST NOT mark baselines stale (LIN-FR-005) |
| nullability `NOT NULL → nullable` | info | Relaxation |
| nullability `nullable → NOT NULL` | warning | May alter filtering of null rows |
| `column_renamed` (heuristic: removed+added, same type, positional proximity) | warning + rename-hint | Hint shown in UX; human confirms — never auto-applied |
- **LIN-FR-005**: For each classified dataset change the system MUST project a `DatasetImpactRecord` per dependent dashboard naming affected charts, affected baseline entries (by release pin), and affected 038 scenario step refs — additive-only changes MUST classify `info` and MUST NOT mark baselines stale.
- **LIN-FR-006**: Cross-dashboard impact records MUST remain orthogonal to per-dashboard 037 StructureDiff; impact records link to but never mutate StructureDiff, baseline catalogs, or scenario graphs.
- **LIN-FR-007**: Staleness propagation MUST surface affected baseline entries in dependent catalogs as stale (or `immutability_violation` for closed periods per 037 AGBASE-FR-012) with dataset-change provenance; propagation MUST NOT rewrite catalog files, auto-approve, or auto-update expected values.
- **LIN-FR-008**: Propagation MUST evaluate each pinned release independently; re-approval in one release MUST NOT clear staleness in another release sharing the dataset.
- **LIN-FR-009**: The system MUST support dataset deprecation with `successor_dataset_id`, grace window, escalating warning states, and a terminal blocking state at expiry producing typed `dataset_deprecated` errors; migration progress (migrated vs remaining dependents) MUST be tracked per deprecation record.
- **LIN-FR-010**: A `dataset_updated` trigger MUST create one coordinated fan-out plan; per-dashboard verification runs link to the plan id, are ordered by impact severity, and dashboards without current baselines are scheduled inspection-only with recorded reason.
- **LIN-FR-011**: Fan-out results MUST aggregate into a fleet report (per-dashboard status, provenance, unresolved impacts) consumable by 039 pipeline views without an AgentRun; the trigger value `dataset_updated` MUST be added to the 036/037 VerificationRun trigger enum.
- **LIN-FR-012**: All impact, deprecation, and fan-out records MUST carry environment scope; cross-environment propagation MUST surface `env_mismatch` warnings rather than assume validity.
- **LIN-FR-013**: The blast-radius read model exposed to 040 load profiles and PROD gates MUST be served from this index (pinned snapshot), guaranteeing configure-time and gate-time consumers see identical dependents.
- **LIN-FR-014**: RBAC MUST gate mutations: `dataset:lineage:refresh` (index rebuild), `dataset:deprecation:manage` (mark/migrate), `dataset:fanout:trigger` (PROD-classified fan-out requires 036 approval-gate semantics with reason).
- **LIN-FR-015**: The blast-radius graph is two-level (dataset ← chart ← dashboard) by construction; fan-out MUST order by impact severity only. Chained SQL-source lineage and cycle detection are out of scope.
- **LIN-FR-017**: Impact projection MUST respect `projection_confidence`: a schema change marks exact-confidence charts affected only when the changed column/metric is in their consumed set; conservative-confidence charts (or expressions) are marked affected on any schema change to their dataset, with the conservative basis shown to the user.
### Key Entities
- **DatasetUsageIndex**: Content-addressed reverse index — dataset id → dependent dashboards with per-chart usage detail, index fingerprint, observation timestamp, environment scope, `stale_index` marker.
- **DatasetSchemaDiff**: Classified change set between two dataset observations — kind, severity, versioned classification rules, observation pair provenance.
- **DatasetImpactRecord**: Per-dependent-dashboard projection of a schema diff — affected charts, affected baseline entries by release pin, affected scenario step refs, severity, links (non-mutating) to StructureDiff.
- **DeprecationRecord**: Dataset deprecation lifecycle — successor id, grace window, escalation state, migrated vs remaining dependents, terminal blocking state.
- **FanoutPlan**: Coordinated multi-dashboard verification plan from a `dataset_updated` trigger — plan id, ordered per-dashboard runs (severity order), inspection-only entries with reasons, environment scope.
- **FleetReport**: Aggregated fan-out outcome — per-dashboard status, dataset-change provenance, unresolved impacts, consumable by 039 pipeline views.
- **ChartUsageEdge**: One binding in the index — dataset id, chart id, dashboard id, consumed columns/metrics (exact set), `projection_confidence` (exact|conservative), `unresolved_refs` when SQL expressions reference unknown columns.
## Success Criteria
- **SC-001**: Reverse lookups return 100% of dependent dashboards for fixture shared datasets with byte-identical fingerprints across repeated builds and shuffled metadata ordering.
- **SC-002**: Schema-diff classification reaches 100% agreement with labeled fixture changes (removed/type-changed/added/metric-removed/deleted) with deterministic severity assignment.
- **SC-003**: Zero writes to 037 baseline catalogs across the entire propagation fixture suite (verified by catalog hash before/after); staleness surfaces only as report/load-time marking.
- **SC-004**: Closed-period retroactive data change surfaces `immutability_violation` (never `stale_baseline`) in 100% of fixture cases.
- **SC-005**: Fan-out ordering respects severity (critical before warning before info) in 100% of fixture plans.
- **SC-008**: SQL-expression extraction on fixture charts: 100% of matched refs produce exact edges; 100% of unmatched refs produce `unresolved_refs` findings; parse failures never flip whole-chart confidence to conservative when structured params are exact.
- **SC-006**: Deprecation blocks operations at expiry with typed `dataset_deprecated` naming the successor in 100% of fixture cases; zero 404-style raw failures reach users.
- **SC-007**: Index refresh failure retains last good index with `stale_index` in 100% of fault-injection cases; no empty/partial index is ever served as current.
#endregion DatasetLineageBlastRadius.Spec

124
specs/041-dataset-lineage-blast-radius/tasks.md Normal file
View File

@@ -0,0 +1,124 @@
#region DatasetLineageBlastRadius.Tasks [C:3] [TYPE ADR] [SEMANTICS tasks,lineage,implementation]
@BRIEF Ordered TDD backlog for dataset lineage and blast-radius (041).
**Input**: all documents in specs/041-dataset-lineage-blast-radius/
**Prerequisites**: spec (clarified), research R1–R8, data-model, contracts/modules.md, quickstart
## Phase 1 — Fixtures, DTO Foundation, Migration
- [ ] T001 Create canonical fixtures under specs/041-dataset-lineage-blast-radius/fixtures/: 3 dashboards (A,B share D1; C uses D2), charts with structured params (`amount,status`), SQL-expression chart (`SUM(amount)/NULLIF(cnt,0)`), unparseable-expression chart, unknown-ref chart.
- [ ] T002 [P] Create schema-change fixture set per Severity Matrix v1 rows (deleted, column_removed used/unused, metric_removed, type breaking/narrowing/widening, column_added, nullability both ways, rename heuristic) under specs/041-dataset-lineage-blast-radius/fixtures/schema_changes/.
- [ ] T003 Materialize fixtures into backend/tests/fixtures/lineage/.
- [ ] T004 Implement Pydantic DTOs (extra-forbid) in backend/src/schemas/lineage.py per data-model.md.
- [ ] T005 Add ORM models in backend/src/models/lineage.py (DatasetUsageEdge, LineageIndexSnapshot, DatasetSchemaObservation, DatasetImpactRecord, DatasetDeprecation, FanoutPlan) + additive VerificationRun.fanout_plan_id; Alembic migration under backend/alembic/versions/.
- [ ] T006 [P] Register RBAC permissions `dataset:lineage:refresh`, `dataset:deprecation:manage`, `dataset:fanout:trigger` in backend/src/services/rbac_permission_catalog.py.
- [ ] T007 [P] Add `lineage_index_enabled` setting (default false) to config models + Migration Settings UI toggle in frontend/src/routes/settings/MigrationSettings.svelte.
**Checkpoint**: Migration upgrades/downgrades; fixtures load; permissions seeded.
## Phase 2 — US1 Dataset Usage Index (P1) 🎯 MVP
- [ ] T008 [US1] Write failing extraction tests in backend/tests/services/lineage/test_usage_extractor.py: per-viz_type structured params (table, big_number, time-series), unknown viz → conservative.
- [ ] T009 [US1] Write failing SQL-expression tests in backend/tests/services/lineage/test_sql_expression_extractor.py: matched refs → exact, unknown refs → `unresolved_refs`, parse failure → per-expression conservative, Jinja spans stripped (inherits SqlTableExtractor pattern).
@POST {matched, unresolved, confidence}; regex-only extraction forbidden (LIN-FR-016)
@TEST_EDGE unparseable expression → conservative for that expression only, chart keeps structured exactness (SC-008)
- [ ] T010 [US1] Implement backend/src/services/lineage/sql_expression_extractor.py using sqlparse (R2).
- [ ] T011 [US1] Implement backend/src/services/lineage/usage_extractor.py with per-viz extractors + confidence rollup.
- [ ] T012 [US1] Write failing indexer tests in backend/tests/services/lineage/test_indexer.py: deterministic fingerprint across repeated builds and shuffled input (SC-001), changed-set-only detail fetches, stale_index retention on failure (LIN-FR-003).
- [ ] T013 [US1] Implement backend/src/services/lineage/indexer.py: OnSync post-commit hook (bounded detail calls), edge rebuild transaction, snapshot flip, RefreshTargets write-hook.
@PRE IdMappingService.sync_environment committed changed rows
@POST Edge set rebuilt for changed uuids only; snapshot fingerprint flipped atomically
@SIDE_EFFECT DB writes; Superset detail calls ≤ changed set (LIN-FR-002 invariant)
@TEST_EDGE mid-hook Superset failure → stale_index=true, last good served (SC-007)
- [ ] T014 [US1] Amend backend/src/core/mapping_service.py: emit changed-uuid sets to Indexer.OnSync after commit (single hook call, no refactor; R3).
- [ ] T015 [US1] Wire write-hook calls: backend/src/core/superset_client/_datasets.py (update/create) and migration deploy completion → Indexer.RefreshTargets (LIN-FR-018).
- [ ] T016 [US1] Add belief-runtime instrumentation (`belief_scope`, `logger.reason/reflect`) to indexer C4/C5 flows per molecular-cot-logging.
**Checkpoint**: quickstart steps 1, 2, 7 pass; `lineage_index_enabled=false` → zero lineage Superset calls.
## Phase 3 — US2 Cross-Dashboard Schema Change Impact (P1)
- [ ] T017 [US2] Write failing severity-rules tests in backend/tests/services/lineage/test_severity_rules.py: all 11 matrix rows golden-labeled (SC-002); property tests for type-widening/narrowing classification.
- [ ] T018 [US2] Implement backend/src/services/lineage/severity_rules_v1.py as pure functions with `rules_version="v1"` (R4).
- [ ] T019 [US2] Write failing schema-diff tests in backend/tests/services/lineage/test_schema_diff.py: exact-confidence projection only on consumed sets, conservative charts marked on any change (LIN-FR-017), additive=info zero stale (LIN-FR-005), append-only records, racing-change monotonic sequence.
- [ ] T020 [US2] Implement backend/src/services/lineage/schema_diff.py: observation persistence, diff computation, impact projection per dependent dashboard (charts, baseline entries by release pin, 038 step refs).
@POST DatasetImpactRecord with classified diff, rules_version, affected_dashboards; append-only (R4)
@SIDE_EFFECT DB inserts only; NEVER mutates StructureDiff/baseline/scenario (LIN-FR-006)
- [ ] T021 [US2] Implement schema observation capture in indexer detail-fetch path (DatasetSchemaObservation rows).
**Checkpoint**: quickstart step 3 passes; SC-002 at 100% on fixture matrix.
## Phase 4 — US3 Staleness Propagation (P1)
- [ ] T022 [US3] Write failing propagation tests in backend/tests/services/lineage/test_propagation.py: catalog hash before/after identical (SC-003), closed-period divergence → `immutability_violation` never stale (SC-004), per-release independence (LIN-FR-008), env_mismatch warning (LIN-FR-012).
- [ ] T023 [US3] Implement backend/src/services/lineage/propagation.py: report-time marking reading DatasetImpactRecord + dependent catalogs by release pin.
@INVARIANT Zero writes to baseline catalog files (SC-003)
@TEST_EDGE closed-period source_response_hash divergence → immutability_violation (037 AGBASE-FR-012)
- [ ] T024 [US3] Integrate propagation surface into 037 catalog-load path as read-time overlay (amendment point; no catalog writer changes).
**Checkpoint**: quickstart step 4 passes.
## Phase 5 — US4 Dataset Deprecation Lifecycle (P2)
- [ ] T025 [US4] Write failing deprecation tests in backend/tests/services/lineage/test_deprecation.py: mark with successor+window, escalation recompute (noticed→warning→expired_blocked) as pure function (R7), expired → typed `dataset_deprecated` naming successor (SC-006), migration tracking per dependent.
- [ ] T026 [US4] Implement backend/src/services/lineage/deprecation.py: mark/migrate actions, escalation recompute invoked in sync cycle (LIN-FR-019).
- [ ] T027 [US4] Add deprecation gates to 037 verification path and 038 scenario compile path: deprecated+expired dataset → typed error, not raw 404.
- [ ] T028 [P] [US4] Write failing L1 model tests in frontend/src/lib/models/__tests__/Datasets.DeprecationModel.test.ts (FSM transitions, countdown derivation).
- [ ] T029 [P] [US4] Write failing L1 model tests in frontend/src/lib/models/__tests__/Datasets.LineageModel.test.ts (pinned snapshot load, stale marker, retry).
- [ ] T030 [US4] Implement frontend/src/lib/types/lineage.ts DTO mirror + Datasets.LineageModel.svelte.ts + Datasets.DeprecationModel.svelte.ts.
- [ ] T031 [US4] Implement lineage panel + deprecation manager components in frontend/src/lib/components/datasets/lineage/ with L2 UX tests (freshness badge, escalation colors, expired banner naming successor).
**Checkpoint**: quickstart step 5 passes; frontend L1/L2 green.
## Phase 6 — US5 Verification Fan-Out (P2)
- [ ] T032 [US5] Write failing fanout tests in backend/tests/services/lineage/test_fanout.py: single plan per trigger, severity ordering critical-first (SC-005), inspection-only entries with reason, plan linkage via fanout_plan_id (R8), fleet report aggregation.
- [ ] T033 [US5] Implement backend/src/services/lineage/fanout.py: plan creation from DatasetImpactRecord, VerificationRun spawning, PROD gate via 036 approval semantics (LIN-FR-014).
@PRE DatasetImpactRecord with max_severity >= warning
@POST One FanoutPlan; runs severity-ordered; inspection-only without baselines with recorded reason
@SIDE_EFFECT VerificationRun rows with fanout_plan_id; `dataset_updated` trigger value (R5)
- [ ] T034 [US5] Add `dataset_updated` trigger enum value + amendment notes in specs/036-agent-test-stabilization/traceability.md and specs/037-superset-baseline-engine/traceability.md (R5).
- [ ] T035 [US5] Add Dashboards.LineageBadgeModel.svelte.ts + shared-dataset badge on dashboard page (impact severity, "делит датасеты с N дашбордами").
**Checkpoint**: quickstart step 6 passes; fleet report consumable without AgentRun (LIN-FR-011).
## Phase 7 — API, 040 Read Model, Integration
- [ ] T036 Implement backend/src/api/routes/lineage.py: dependents (fingerprint pinning, stale_notice per R6), impacts list, deprecation manage, fanout trigger/report; register router.
@DATA_CONTRACT DependentsResponse honors ?fingerprint= pinning; mismatch → stale_notice
@TEST_EDGE unknown fingerprint → fresh snapshot + notice, never silent drift
- [ ] T037 Write API contract/RBAC tests in backend/tests/api/test_lineage.py: all mutations default-deny, permission_denied without confirm control (quickstart 8).
- [ ] T038 [P] Verify 040 blast-radius consumer contract: dependents endpoint serves pinned snapshot shape from R6 (contract test with 040 fixture).
- [ ] T039 [P] Add rejected-path regression tests: regex-only extraction attempt fails (LIN-FR-016), auto-invalidation attempt on catalogs fails (SC-003), separate-scheduler registration absent.
## Phase 8 — Polish & Quality Gates
- [ ] T040 Run quickstart.md full sequence end-to-end.
- [ ] T041 Run backend pytest (lineage suites + existing mapping/migration regressions), ruff check.
- [ ] T042 Run frontend test/lint/build for lineage components + settings toggle.
- [ ] T043 [P] Attention compliance audit ATTN_1–4 across new contracts; semantic index rebuild; orphan audit.
- [ ] T044 [P] Update 040-dashboard-load-testing spec traceability: LIN-FR-013 read model marked available.
## Dependencies
T001–T007 → US1 (index is prerequisite for all) → US2 → US3; US4/US5 parallelizable after US2; API/integration after all stories. Tests precede implementation within each phase. 036/037 amendments (T034) land only with fanout implementation.
## Parallel Opportunities
- T002 ∥ T006 ∥ T007 (fixtures ∥ RBAC ∥ settings)
- T008 ∥ T009 (extractor test files)
- T028 ∥ T029 (frontend L1 models)
- US4 ∥ US5 after Phase 3 (deprecation ∥ fanout, different modules)
- T038 ∥ T039 (contract ∥ regression tests)
## Story Verification Criteria
| Story | Verified by |
|---|---|
| US1 | quickstart 1, 2, 7; SC-001, SC-007, SC-008 |
| US2 | quickstart 3; SC-002 |
| US3 | quickstart 4; SC-003, SC-004 |
| US4 | quickstart 5; SC-006 |
| US5 | quickstart 6; SC-005; LIN-FR-011 |
#endregion DatasetLineageBlastRadius.Tasks

36
specs/041-dataset-lineage-blast-radius/traceability.md Normal file
View File

@@ -0,0 +1,36 @@
#region DatasetLineageBlastRadius.Traceability [C:3] [TYPE ADR] [SEMANTICS traceability,lineage,requirements]
@BRIEF Requirement-to-contract-to-test matrix for feature 041.
| Requirement | Contract | Verification |
|---|---|---|
| LIN-FR-001, LIN-FR-002 | Services.Lineage.Indexer | index build determinism, snapshot pinning, changed-set bound (quickstart 1, 7) |
| LIN-FR-003 | LineageIndexSnapshot retention | refresh-failure fault injection (quickstart 7) |
| LIN-FR-004 | Services.Lineage.SeverityRulesV1 | golden fixtures per matrix row, 100% label agreement (SC-002) |
| LIN-FR-005, LIN-FR-017 | Services.Lineage.SchemaDiff | exact vs conservative projection; additive=info zero stale (quickstart 3) |
| LIN-FR-006 | SchemaDiff append-only | impact records never mutate StructureDiff/baseline/scenario (code audit + tests) |
| LIN-FR-007, LIN-FR-008 | Services.Lineage.Propagation | catalog hash before/after identical (SC-003); per-release independence |
| LIN-FR-009, LIN-FR-019 | Services.Lineage.Deprecation | escalation recompute, expired typed error naming successor (SC-006) |
| LIN-FR-010, LIN-FR-011, LIN-FR-015 | Services.Lineage.Fanout | single plan, severity ordering, inspection-only reasons, fleet report (SC-005) |
| LIN-FR-012 | env_id scoping on all tables/DTOs | cross-env propagation → env_mismatch warning |
| LIN-FR-013 | Api.Lineage.Routes dependents endpoint | fingerprint pinning, stale_notice on mismatch (R6) |
| LIN-FR-014 | RBAC catalog + route guards | permission_denied for all mutations, no confirm control (quickstart 8) |
| LIN-FR-016 | Services.Lineage.SqlExpressionExtractor | matched/unresolved/conservative fixture matrix (SC-008) |
| LIN-FR-018 | Indexer.RefreshTargets | write-hook after update_dataset/migration deploy (quickstart edge) |
## Story Coverage
| Story | Independent checkpoint |
|---|---|
| US1 Index | Deterministic fingerprint, exact dependents, stale retention |
| US2 Impact | Classification 100% agreement, per-dashboard projection, orthogonality |
| US3 Propagation | Zero catalog writes, immutability_violation for closed periods, per-release independence |
| US4 Deprecation | Escalation lifecycle, typed expiry errors, migration tracking |
| US5 Fan-out | One plan, severity order, inspection-only, fleet report for 039 |
## Upstream/Downstream
- Depends on: `Core.MappingService.IdMappingService` (sync cycle), 036 (gate semantics, VerificationRun), 037 (baseline immutability, StructureDiff orthogonality)
- Downstream: 040 consumes dependents read model (LIN-FR-013); 039 pipeline views consume FleetReport (LIN-FR-011)
- Amendments at implementation: `dataset_updated` trigger value notes in 036/037 traceability (R5)
#endregion DatasetLineageBlastRadius.Traceability

77
specs/041-dataset-lineage-blast-radius/ux_reference.md Normal file
View File

@@ -0,0 +1,77 @@
#region DatasetLineageBlastRadius.UxReference [C:3] [TYPE ADR] [SEMANTICS ux,reference,lineage,blast-radius,dataset]
@BRIEF UX interaction reference for dataset lineage and blast-radius surfaces: persona, flows, states, recovery. Drives @UX_* tags in Phase 1 contracts.
**Feature Branch**: `041-dataset-lineage-blast-radius`
**Created**: 2026-07-22 | **Status**: Draft
## 1. User Persona & Context
* **Who is the user?**: Two personas share these surfaces. (1) BI engineer owning a dashboard who needs to know «сломается ли мой дашборд, если изменят этот датасет?». (2) Data/platform engineer owning a dataset who needs to know «кого я затрону, если изменю или выведу из эксплуатации этот датасет?».
* **What is their goal?**: Make impact visible *before* the change: named dependent dashboards, classified severity, affected baselines and scenario steps — and a managed path (deprecation + fan-out re-verification) instead of silent breakage discovered at the next release check.
* **Context**: Web UI on desktop. Entry points: dataset detail page («Используется в N дашбордах»), dashboard page (shared-dataset badge), deprecation manager, fleet report after fan-out. Read-only for most users; mutations (deprecation, fan-out trigger, index refresh) gated by role.
## 2. The "Happy Path" Narrative
The data engineer opens a dataset page and sees the lineage panel: «Используется в 4 дашбордах, 11 чартах» — expandable to per-chart column usage. They mark the dataset deprecated, pick the successor from a searchable list, set a 14-day grace window, and the panel immediately lists every dependent with migration status «0 из 4 перенесено». A week later a schema diff lands on the successor: each dependent dashboard shows a classified impact card («critical: колонка `amount` удалена — затронуты 2 чарта, 3 baseline-записи release 1.4»). They trigger fan-out: one plan, four verification runs ordered critical-first, and a fleet report aggregates results — two dashboards pass, one needs baseline re-approval, one was inspection-only (no baselines yet). Nothing was silently rewritten: baseline catalogs show staleness *markers* with provenance, and the immutability badge on a closed-period entry is intact.
## 3. Interface Mockups
### UI Layout & Flow
**Screen**: Dataset Lineage Panel (on dataset detail page)
* **Layout**: Header (dataset, environment, index freshness badge) → dependents table → per-chart usage expansion → actions (gated).
* **Key Elements**:
* **Dependents table**: Dashboard title (link), chart count, used columns/metrics summary, release pin(s) with baselines, impact severity badge when an active impact record exists.
* **Index freshness**: «Индекс обновлён 5 мин назад» or amber `stale_index` «данные могут быть устаревшими — последняя ошибка: timeout».
* **«Отметить устаревшим»** (gated `dataset:deprecation:manage`): opens deprecation form — successor selector, grace window, initial notice preview.
* **Contract Mapping**:
* **`@UX_STATE`**: `loading → ready → stale_index → refresh_failed`; deprecation sub-FSM `none → noticed → warning → expired_blocked`.
* **`@UX_FEEDBACK`**: Severity badges (critical/warning/info), freshness badge, migration progress bar «2 из 4».
* **`@UX_RECOVERY`**: Refresh failure → «Повторить обновление» (gated) with last-good data retained; expired dataset in a dependent → error card names successor and links migration guidance.
* **`@UX_REACTIVITY`**: Screen model atoms (`index`, `impacts`, `deprecation`) `$derived` into panels; pinning: consumers hold the index fingerprint they opened with.
* **Screen Model**: `Datasets.LineageModel.svelte.ts` (index + impacts projection) and `Datasets.DeprecationModel.svelte.ts` (deprecation FSM). Components bind via `@RELATION BINDS_TO`.
**Screen**: Cross-Dashboard Impact View (per dataset change)
* **Layout**: Change summary (kind, severity, observation pair) → per-dependent cards grouped by severity → each card: affected charts, affected baseline entries by release, affected scenario step refs, link to the dashboard's own StructureDiff (orthogonal, read-only link).
**Screen**: Deprecation Manager
* **Layout**: Active deprecations list → per-record detail: successor, window countdown, escalation state, migrated vs remaining dependents with per-dashboard migration checklist.
**Screen**: Fleet Report (post fan-out)
* **Layout**: Plan header (trigger `dataset_updated`, impact record link, environment) → ordered run list (severity, status badge, inspection-only reason where applicable) → aggregate summary (pass/warn/fail/unresolved impacts) → export.
## 4. The "Error" Experience
**Philosophy**: Lineage is advisory infrastructure — it must degrade loudly, never fabricate. A stale or partial index is worse than no index if served as current.
### Scenario A: Index refresh failure
* **System Response**: (UI) Freshness badge flips amber `stale_index` with typed error («Superset 403 — нет доступа к metadata»); dependents table shows last-good data with the staleness marker on every row.
* **Recovery**: «Повторить обновление» (role-gated); if the failure persists, escalation link to environment settings. No row ever disappears silently.
### Scenario B: Expired deprecated dataset blocks an operation
* **System Response**: Verification or scenario generation returns typed `dataset_deprecated` error card: «Датасет выведен из эксплуатации 3 дня назад. Преемник: `sales_v2`. 2 дашборда ещё не перенесены.»
* **Recovery**: Link to deprecation record with migration checklist; for the dashboard owner — link to chart re-pointing guidance. Raw Superset 404s never reach the user unwrapped.
### Scenario C: Lineage cycle detected
* **System Response**: Fan-out planning halts the affected subtree with a `lineage_cycle` finding showing the cycle path (`ds_a → ds_b → ds_a`); unaffected dependents proceed.
* **Recovery**: Finding links the involved datasets for manual resolution; re-plan available after fix.
### Scenario D: Permission denied on mutations
* **System Response**: `permission_denied` recovery panel naming the required role (`dataset:deprecation:manage` / `dataset:fanout:trigger`); no confirm control rendered (036 gate semantics).
* **Recovery**: Request-access link; read-only lineage data remains fully visible.
## 5. Tone & Voice
* **Style**: Concrete and accountable. Always name the affected parties («затронуты 2 чарта, 3 baseline-записи release 1.4»), never vague («могут быть проблемы»).
* **Terminology**: «Зависимые дашборды», «записи влияния», «преемник датасета», «fan-out» (kept in English in RU locale), «fleet report» → «сводный отчёт». Staleness is always framed as «требует пересмотра», never «устарело автоматически» — propagation surfaces, humans decide.
* **Safety copy**: Deprecation and fan-out actions state scope upfront: «это действие уведомит владельцев N дашбордов и создаст M верификационных запусков».
#endregion DatasetLineageBlastRadius.UxReference