busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor(agent): extract agent+shared into standalone packages with full GRACE semantic markup

Browse Source 
- Move agent code from backend/src/agent/ to agent/src/ss_tools/agent/
- Extract shared stdlib-only utilities to shared/src/ss_tools/shared/
- Add #region/#endregion contracts to all ~140 functions (INV_1 compliance)
- Update docker files, entrypoint, build scripts for new package layout
- Backend now imports ss_tools.shared._llm_health (no gradio/langchain deps)
- Add specs for 036-039 feature plans
This commit is contained in:
busya
2026-07-07 15:18:24 +03:00
parent ce368429f7
commit b95df37cd5
75 changed files with 2711 additions and 1861 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
specs/036-agent-test-stabilization/checklists/requirements.md Normal file
View File

@@ -0,0 +1,26 @@
# Requirements Checklist: 036 Agent Test Stabilization
**Purpose**: Validate specification quality and readiness for planning.
**Created**: 2026-07-07
**Feature**: specs/036-agent-test-stabilization/spec.md
## Spec Completeness
- [x] CHK001 User stories are independently testable.
- [x] CHK002 Functional requirements avoid implementation code while defining observable behavior.
- [x] CHK003 No unresolved `[NEEDS CLARIFICATION]` markers remain.
- [x] CHK004 Edge cases include stale context, dropped stream, permission denial, and backwards compatibility.
## Constitution Coverage
- [x] CHK005 Semantic contract and ADR relations are present in spec header.
- [x] CHK006 Decision memory records why stabilization precedes scenario generation.
- [x] CHK007 External orchestrator boundary is preserved.
- [x] CHK008 RBAC/HITL requirements are explicit for writes and approvals.
- [x] CHK009 Svelte UX remains model-driven and structured-event based.
## Readiness for Plan
- [x] CHK010 Required outputs are limited to agent runtime stabilization artifacts.
- [x] CHK011 Feature explicitly excludes Superset query baseline engine and scenario generation logic.
- [x] CHK012 Success criteria are measurable by backend/frontend tests and one live smoke flow.

107
specs/036-agent-test-stabilization/spec.md Normal file
View File

@@ -0,0 +1,107 @@
#region AgentTestStabilization.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,agent,test-scenarios,stabilization]
@BRIEF Stabilize the existing Gradio/LangGraph agent runtime so it can safely support long-running dashboard test scenario generation.
@RELATION DEPENDS_ON -> [ADR-0001]
@RELATION DEPENDS_ON -> [ADR-0003]
@RELATION DEPENDS_ON -> [ADR-0005]
@RELATION DEPENDS_ON -> [ADR-0006]
@RELATION DEPENDS_ON -> [ADR-0008]
@RATIONALE Dashboard test scenario generation produces durable artifacts and baseline approval flows, so the agent must be a recoverable execution surface rather than a best-effort chat stream.
@REJECTED Starting dashboard scenario generation directly on top of unstabilized chat behavior — rejected because failures would be indistinguishable across Gradio transport, context passing, artifact preview, and HITL gates.
## Navigation (DSA Indexer keywords)
@SEMANTICS: spec, requirements, feature, agent, gradio, langgraph, scenario, artifact, hitl, progress, dashboard-testing
**Feature Branch**: `036-agent-test-stabilization`
**Created**: 2026-07-07 | **Status**: Draft
**Input**: "Stabilize the existing Gradio LangGraph agent runtime so it can support long running dashboard test scenario generation with stable dashboard UIContext, explicit scenario intent, structured progress events, draft artifact preview, recoverable agent run identifiers, and HITL confirmation for repository writes or baseline approvals."
## User Scenarios
### Story 1 — Dashboard Context Run Start (P1)
**Why P1**: Dashboard scenario generation must start from a precise dashboard context, environment, and intent without relying on user prose.
**Independent Test**: Open `/agent` from a dashboard route with query context and verify the agent run exposes the exact dashboard object, env, route, and scenario intent in debug/status metadata.
**Acceptance**:
1. **Given** a user clicks "Создать сценарий тестирования" on a dashboard **When** `/agent` opens **Then** `UIContext` contains `objectType=dashboard`, `objectId`, `objectName`, `envId`, `route`, `contextVersion`, and `intent=build_dashboard_test_scenario`.
2. **Given** the user opens `/agent` from two different dashboards in sequence **When** each run starts **Then** the second run uses the new dashboard context and never reuses stale `objectId` or `envId`.
3. **Given** malformed or oversized context is supplied **When** the agent run starts **Then** invalid context is rejected or discarded with visible recovery and audit metadata, not injected into the agent prompt.
---
### Story 2 — Recoverable Long Agent Runs (P1)
**Why P1**: Scenario generation may inspect Superset, produce draft artifacts, and wait for user approval; it cannot be a fragile one-shot response.
**Independent Test**: Start a simulated long scenario run, disconnect/reload `/agent`, and verify the user can see run status and draft artifact references by `run_id`.
**Acceptance**:
1. **Given** a scenario run starts **When** the backend accepts the request **Then** the UI receives a stable `agent_run_id` correlated with conversation id and dashboard context.
2. **Given** the Gradio stream drops during a run **When** the user reconnects **Then** the latest run status, progress stage, and recoverable draft references are visible.
3. **Given** no first observable event arrives within the configured timeout **When** timeout elapses **Then** the UI shows actionable recovery instead of endless thinking.
---
### Story 3 — Structured Progress and Draft Artifacts (P2)
**Why P2**: The UI must render scenario progress and artifact preview from structured events, not parse natural-language chat text.
**Independent Test**: Run a fixture-based scenario generation stub and verify progress stages and draft artifacts render from metadata events only.
**Acceptance**:
1. **Given** a run progresses through context, inspect, scenario, parameters, generate, validate, and save stages **When** events stream **Then** the UI updates each stage from structured metadata.
2. **Given** draft artifacts are produced **When** the agent reports them **Then** the user can preview artifact names, types, warnings, and validation status before any repository write.
3. **Given** an artifact contains warnings **When** preview renders **Then** warnings are visible and block silent save.
---
### Story 4 — HITL Write and Approval Gates (P2)
**Why P2**: Generated files and approved baselines affect future regression truth and must never be committed by the agent without human approval.
**Independent Test**: Trigger a repository-write or baseline-approval action and verify `confirm_required` appears with target path, operation type, risks, and denial path.
**Acceptance**:
1. **Given** the agent proposes saving generated artifacts **When** save is requested **Then** a confirmation card lists target paths, file count, risk level, and warnings before the write.
2. **Given** the agent proposes approving a baseline candidate **When** approval is requested **Then** confirmation requires a user-visible reason and shows provenance.
3. **Given** the user denies a write or approval **When** denial is submitted **Then** no artifact is written or approved, and the chat records an explicit cancellation outcome.
---
### Edge Cases
- JWT expires mid-run → next protected operation stops with session-expired recovery and does not retry privileged actions.
- User lacks permission to save artifacts or approve baselines → `permission_denied` appears instead of `confirm_required`.
- Multiple tabs start runs for the same dashboard → each run has an independent `agent_run_id`; artifact preview does not cross-contaminate.
- Existing 033/035 agent chat functions remain available → dashboard test intent does not break ordinary chat, tool cards, or confirmation flows.
## Requirements
### Functional
- **AGSTAB-FR-001**: `/agent` MUST accept and validate dashboard `UIContext` with explicit `intent=build_dashboard_test_scenario` while preserving backward compatibility with existing 035 `uicontext` payload position.
- **AGSTAB-FR-002**: Every long-running agent operation for dashboard testing MUST expose a stable `agent_run_id` correlated with conversation id, user id, dashboard context, and current progress stage.
- **AGSTAB-FR-003**: The agent runtime MUST emit structured progress metadata for scenario-oriented stages; UI MUST NOT infer these stages by parsing prose.
- **AGSTAB-FR-004**: Draft artifacts MUST be previewable before repository writes and must include type, name, intended path, validation status, warnings, and producing run id.
- **AGSTAB-FR-005**: Repository writes and baseline approvals MUST require HITL confirmation; approval must capture a user-supplied reason.
- **AGSTAB-FR-006**: Permission-denied operations MUST emit `permission_denied` metadata and never present a confirm button for unauthorized actions.
- **AGSTAB-FR-007**: The existing Gradio/LangGraph chat, streaming, confirmation, context, and guardrail flows from 033/035 MUST remain compatible.
- **AGSTAB-FR-008**: A live smoke test MUST verify context → stream → tool event → draft artifact preview → confirmation/denial recovery.
### Key Entities
- **DashboardScenarioUIContext**: Versioned context payload identifying the dashboard, environment, source route, and scenario-generation intent.
- **AgentRun**: Recoverable execution unit for long-running agent work; owns progress stage, events, draft artifact references, and terminal status.
- **ScenarioProgressEvent**: Structured metadata event representing a stage or step in scenario generation.
- **DraftArtifactRef**: Previewable reference to generated content that is not yet persisted as an approved repository artifact.
- **ApprovalGate**: HITL confirmation envelope for repository writes and baseline approvals.
## Success Criteria
- **SC-001**: 100% of dashboard scenario runs in test fixtures expose a valid `agent_run_id` before the first tool action.
- **SC-002**: Context handoff from dashboard page to `/agent` is verified for at least three dashboard ids with no stale-object reuse.
- **SC-003**: Structured progress events render all required stages without text parsing in frontend tests.
- **SC-004**: Repository write and baseline approval attempts are blocked until confirmation in 100% of permissioned test cases.
- **SC-005**: Existing 035 agent context/guardrails tests continue to pass unchanged or with documented fixture updates only.
#endregion AgentTestStabilization.Spec

85
specs/036-agent-test-stabilization/ux_reference.md Normal file
View File

@@ -0,0 +1,85 @@
#region AgentTestStabilization.UxReference [C:3] [TYPE ADR] [SEMANTICS ux,reference,agent,dashboard-testing]
@BRIEF UX reference for stabilized agent execution before dashboard test scenario generation.
**Feature Branch**: `036-agent-test-stabilization`
**Created**: 2026-07-07 | **Status**: Draft
## 1. User Persona & Context
* **Who is the user?**: QA engineer, dashboard owner, or analyst preparing an agent-generated dashboard test scenario.
* **What is their goal?**: Start a reliable, recoverable agent run from a dashboard and preview generated outputs before any durable write.
* **Context**: Browser-based Svelte `/agent` workspace launched from a dashboard page with Superset environment selected.
## 2. Happy Path Narrative
The user clicks "Создать сценарий тестирования" on a dashboard. `/agent` opens with the dashboard context already visible: dashboard name, id, environment, and intent. The agent starts a recoverable run, streams structured progress stages, produces draft artifacts, and shows a confirmation card before saving anything. If the user denies, the run ends cleanly with no side effects.
## 3. Interface Mockups
### Agent Context Header
```text
┌─────────────────────────────────────────────────────────────────────────────┐
│ 🧪 Dashboard Test Scenario Agent ● connected │
├─────────────────────────────────────────────────────────────────────────────┤
│ Context: FI-0080 | dashboard_id=42 | env=ss-dev │
│ Intent: build_dashboard_test_scenario | run_id: ag-run-20260707-001 │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Progress Strip
```text
[Context ✓] → [Inspect …] → [Scenario] → [Parameters] → [Generate] → [Validate] → [Save]
```
### Draft Artifact Preview
```text
┌──────────────────────── Draft artifacts ────────────────────────────────────┐
│ Run: ag-run-20260707-001 │
│ ✓ scenario.yaml valid │
│ ⚠ baseline-candidates.yaml needs approval reason │
│ ✓ generated-preview.json valid │
│ │
│ [Preview] [Download draft] [Request save] │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Confirmation Gate
```text
┌──────────────────────── Confirmation required ──────────────────────────────┐
│ Save generated artifacts │
│ Target: tests/generated/dashboards/fi_0080/ │
│ Files: 3 │
│ Risk: repository write │
│ Warnings: 1 baseline candidate remains draft │
│ │
│ [Confirm save] [Deny] │
└─────────────────────────────────────────────────────────────────────────────┘
```
## 4. Error Experience
### Scenario A: Stale or Invalid Context
* **System Response**: Context card turns warning-tone and lists invalid fields.
* **Recovery**: User can return to dashboard, retry context handoff, or continue in non-scenario chat mode.
### Scenario B: Stream Drops During Run
* **System Response**: UI shows `run_id`, last known stage, and reconnect actions.
* **Recovery**: User reconnects and resumes viewing draft status; no write happens during disconnected state.
### Scenario C: Unauthorized Approval
* **System Response**: Permission-denied card with required role and no confirm button.
* **Recovery**: User can download draft only if allowed, or request approval from an authorized role.
## 5. Tone & Voice
* **Style**: Technical, explicit, side-effect aware.
* **Terminology**: Use "run", "draft artifact", "confirmation", "approval reason", and "dashboard context" consistently.
#endregion AgentTestStabilization.UxReference

25
specs/037-superset-baseline-engine/checklists/requirements.md Normal file
View File

@@ -0,0 +1,25 @@
# Requirements Checklist: 037 Superset Baseline Engine
**Purpose**: Validate specification quality and no-SQL baseline scope.
**Created**: 2026-07-07
**Feature**: specs/037-superset-baseline-engine/spec.md
## Spec Completeness
- [x] CHK001 User stories cover inspection, Superset-native execution, normalization, comparison, and baseline lifecycle.
- [x] CHK002 Requirements explicitly reject direct SQL execution and SQL-based baseline provenance.
- [x] CHK003 No unresolved `[NEEDS CLARIFICATION]` markers remain.
- [x] CHK004 Edge cases include Superset API errors, stale fingerprints, empty results, and locale formatting.
## Constitution Coverage
- [x] CHK005 ADR relations preserve external orchestrator and RBAC boundaries.
- [x] CHK006 Decision memory explains Superset-native execution over SQL.
- [x] CHK007 Approved baseline lifecycle requires reviewable artifacts and HITL approval.
- [x] CHK008 Requirements are implementation-free but measurable.
## Readiness for Plan
- [x] CHK009 Query model, normalized filters, query context, result, baseline, and comparison entities are defined.
- [x] CHK010 Success criteria can be tested with deterministic fixtures.
- [x] CHK011 Scope excludes scenario graph UI and generated Playwright/XLSX execution details.

110
specs/037-superset-baseline-engine/spec.md Normal file
View File

@@ -0,0 +1,110 @@
#region SupersetBaselineEngine.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,superset,baseline,dashboard-testing]
@BRIEF Superset-native query and baseline engine for dashboard testing without direct SQL execution.
@RELATION DEPENDS_ON -> [ADR-0001]
@RELATION DEPENDS_ON -> [ADR-0003]
@RELATION DEPENDS_ON -> [ADR-0005]
@RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
@RATIONALE Dashboard test assertions must validate the same Superset-side chart or dataset execution path that powers dashboards, not a parallel SQL path that can diverge from Superset filter/query semantics.
@REJECTED Direct SQL execution for dashboard test truth — rejected because the target requirement is stable Superset dataset/chart execution and filter fidelity, not separate database querying.
## Navigation (DSA Indexer keywords)
@SEMANTICS: spec, requirements, feature, superset, baseline, chart-data, dataset, filters, normalization, dashboard-testing
**Feature Branch**: `037-superset-baseline-engine`
**Created**: 2026-07-07 | **Status**: Draft
**Input**: "Create a Superset-native query and baseline engine for dashboard testing. The system must inspect dashboard query models, map dashboard native filters into Superset chart or dataset query context, execute Superset-side chart or dataset queries without direct SQL, normalize returned metric and table values, compare them with approved baseline catalog entries, and create draft baseline candidates requiring human approval."
## User Scenarios
### Story 1 — Inspect Dashboard Query Model (P1)
**Why P1**: Scenario generation must know charts, datasets, metrics, filters, and filter scopes before it can build reliable tests.
**Independent Test**: Provide a dashboard fixture and verify the engine returns structured query model data for charts, datasets, metrics, native filters, and capabilities.
**Acceptance**:
1. **Given** a dashboard id and environment **When** the query model is inspected **Then** the result lists dashboard title, charts, datasets, metric labels/keys, native filters, filter targets, and export capabilities.
2. **Given** a native filter applies only to selected charts **When** inspection runs **Then** filter scope is represented so unrelated charts are not queried with invalid filters.
3. **Given** a chart or dataset is inaccessible **When** inspection runs **Then** the engine reports a structured warning or permission error without inventing missing metadata.
---
### Story 2 — Execute Superset Query Context (P1)
**Why P1**: Baseline validation depends on executing Superset chart/dataset queries through Superset-native APIs, never through direct SQL.
**Independent Test**: Execute a fixture chart query with normalized dashboard filters and verify returned data is traceable to chart id, dataset id, metric, filters hash, and environment.
**Acceptance**:
1. **Given** a chart metric and normalized filter context **When** execution is requested **Then** the engine calls the Superset-native chart/dataset execution path and returns structured values.
2. **Given** dashboard filters are applied **When** query context is built **Then** Superset receives semantically equivalent filters to the UI native filter state.
3. **Given** a request would require arbitrary SQL text from the agent **When** execution is attempted **Then** the request is rejected as unsupported.
---
### Story 3 — Normalize and Compare Results (P1)
**Why P1**: UI, Superset API, and XLSX values must be comparable despite formatting differences.
**Independent Test**: Feed formatted decimal, date, percent, big-number, and table-row fixtures and verify normalized values compare consistently with tolerance rules.
**Acceptance**:
1. **Given** Superset returns a metric value **When** normalization runs **Then** the value is represented with type, raw value, normalized value, label, and source metadata.
2. **Given** an approved baseline exists **When** an actual value is compared **Then** exact, absolute tolerance, relative tolerance, range, or row-set comparison rules are applied according to baseline policy.
3. **Given** actual data cannot be normalized **When** comparison is requested **Then** the result is `inconclusive` with an explanation, not a false pass.
---
### Story 4 — Baseline Candidate Lifecycle (P2)
**Why P2**: The agent may discover candidate reference values but must not silently define the truth.
**Independent Test**: Run discovery for a metric without baseline and verify a draft candidate is produced; approving it requires HITL from feature 036.
**Acceptance**:
1. **Given** no approved baseline exists for metric+filters **When** discovery runs **Then** a draft baseline candidate is created with provenance and source values.
2. **Given** a dashboard/chart/dataset/filter fingerprint changes **When** existing baselines are loaded **Then** affected baselines are marked or reported as stale.
3. **Given** UI/API/XLSX sources disagree **When** a candidate is created **Then** candidate approval is blocked or warning-gated until a user reviews the discrepancy.
---
### Edge Cases
- Superset returns empty result → compare result distinguishes expected empty, unexpected empty, and inconclusive.
- Filter value is not available in the target environment → query execution fails with recoverable validation error.
- Dashboard or chart metadata changes after baseline approval → fingerprint mismatch surfaces stale baseline warning.
- Superset API returns 403/404/422/5xx/timeout → taxonomy is preserved in comparison/report output.
- Date and number formats differ between locales → normalization uses canonical decimal/date representation.
## Requirements
### Functional
- **AGBASE-FR-001**: The engine MUST inspect dashboard query models into structured charts, datasets, metrics, native filters, filter scopes, and capabilities.
- **AGBASE-FR-002**: The engine MUST execute Superset-native chart/dataset query contexts without accepting arbitrary SQL text from the agent.
- **AGBASE-FR-003**: Dashboard native filters MUST be normalized into query filters with target columns, operators, values, scopes, and a deterministic filter hash.
- **AGBASE-FR-004**: Superset result normalization MUST support scalar metrics, big-number charts, table rows, dates, decimals, percentages, empty values, and raw/source metadata.
- **AGBASE-FR-005**: Baseline catalog entries MUST identify dashboard id, chart id or dataset id, metric/result key, normalized filters, expected value, tolerance, lifecycle status, fingerprints, and provenance.
- **AGBASE-FR-006**: Approved baselines MUST be stored as reviewable artifacts; observed runtime values and evidence remain separate from approved expectations.
- **AGBASE-FR-007**: The system MUST create draft baseline candidates only; approval is delegated to the HITL gate from 036.
- **AGBASE-FR-008**: Comparison output MUST include source, actual value, expected value, diff, status, tolerance rule, and stale/inconclusive warnings.
- **AGBASE-FR-009**: Direct SQL execution, generated SQL assertions, and SQL-based baseline provenance are explicitly out of scope for this feature.
### Key Entities
- **DashboardQueryModel**: Structured representation of charts, datasets, metrics, native filters, scopes, and execution capabilities for a dashboard.
- **NormalizedFilterContext**: Canonical filter state shared by UI, Superset query execution, scenario graph, XLSX comparison, and baseline lookup.
- **SupersetQueryContext**: Executable Superset-native query request derived from chart/dataset metadata plus normalized filters.
- **NormalizedSupersetResult**: Canonical metric/table result with type metadata and provenance.
- **BaselineEntry**: Approved or draft expected value definition for a metric/table result under a normalized filter context.
- **BaselineCandidate**: Draft expected value discovered from Superset execution and awaiting human approval.
- **ComparisonResult**: Result of comparing actual normalized values to baseline expectations.
## Success Criteria
- **SC-001**: Dashboard query model inspection returns complete chart/filter/dataset mappings for fixture dashboards with 100% deterministic JSON snapshots.
- **SC-002**: Superset-native execution validates at least scalar metric and table chart fixtures without direct SQL.
- **SC-003**: Decimal/date/percent normalization compares equivalent API/UI/XLSX formatted values without false diffs in fixture tests.
- **SC-004**: Baseline candidates are never marked approved without HITL approval metadata.
- **SC-005**: Fingerprint changes surface stale baseline warnings in 100% of changed chart/dataset/filter fixture cases.
#endregion SupersetBaselineEngine.Spec

79
specs/037-superset-baseline-engine/ux_reference.md Normal file
View File

@@ -0,0 +1,79 @@
#region SupersetBaselineEngine.UxReference [C:3] [TYPE ADR] [SEMANTICS ux,reference,superset,baseline]
@BRIEF UX reference for Superset-native baseline execution and comparison results.
**Feature Branch**: `037-superset-baseline-engine`
**Created**: 2026-07-07 | **Status**: Draft
## 1. User Persona & Context
* **Who is the user?**: QA engineer or dashboard owner validating reference metric values through Superset-native execution.
* **What is their goal?**: See whether a dashboard metric under specific filters matches approved baseline values without running direct SQL.
* **Context**: Agent workspace or later scenario UI displays inspection, execution, normalization, baseline, and comparison states.
## 2. Happy Path Narrative
The user asks the agent to validate a metric for a dashboard and filter set. The system inspects the dashboard query model, executes the related Superset chart or dataset query, normalizes the result, compares it with an approved baseline, and displays a source-aware comparison. The UI explicitly states that no direct SQL was executed.
## 3. Interface Mockups
### Query Model Summary
```text
┌──────────────────── Dashboard query model ──────────────────────────────────┐
│ Dashboard: FI-0080 | env=ss-dev │
│ Charts: 7 | Datasets: 3 | Native filters: 4 │
│ Capabilities: chart_data ✓ dataset_query ✓ xlsx_export ✓ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Metric Comparison
```text
┌──────────────────── Metric validation ───────────────────────────────────────┐
│ Metric: Просроченная ДЗ │
│ Chart: Итого просроченная ДЗ | Dataset: dataset_finance_debt │
│ Filters: Дата=2026-05-29, Контрагент=АСК │
│ │
│ Source Value Status │
│ Superset API 1 234 567.89 ✓ matches baseline │
│ Approved baseline 1 234 567.89 approved │
│ │
│ No direct SQL was executed. │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Baseline Candidate
```text
┌──────────────────── Baseline candidate ─────────────────────────────────────┐
│ No approved baseline found. │
│ Candidate from Superset execution: 1 234 567.89 │
│ Provenance: chart_id=128, dataset_id=77, filters_hash=sha256:... │
│ │
│ [Approve baseline] [Keep draft] [Discard] │
└─────────────────────────────────────────────────────────────────────────────┘
```
## 4. Error Experience
### Scenario A: Superset API Permission Denied
* **System Response**: Comparison card shows `403 permission denied`, target chart/dataset, and required action.
* **Recovery**: User can switch environment, request access, or skip that check.
### Scenario B: Stale Baseline
* **System Response**: Baseline status becomes warning-tone with fingerprint diff category.
* **Recovery**: User can run discovery and submit a new baseline candidate for approval.
### Scenario C: Inconclusive Normalization
* **System Response**: UI shows raw value and reason normalization failed.
* **Recovery**: User can mark as manual checkpoint or adjust metric mapping in a later planning phase.
## 5. Tone & Voice
* **Style**: Precise, source-aware, explicit about no direct SQL.
* **Terminology**: Use "Superset API", "query model", "baseline", "candidate", "stale", "fingerprint".
#endregion SupersetBaselineEngine.UxReference

25
specs/038-dashboard-scenario-model/checklists/requirements.md Normal file
View File

@@ -0,0 +1,25 @@
# Requirements Checklist: 038 Dashboard Scenario Model
**Purpose**: Validate ScenarioGraph specification quality.
**Created**: 2026-07-07
**Feature**: specs/038-dashboard-scenario-model/spec.md
## Spec Completeness
- [x] CHK001 User stories cover graph generation, validation, checklist mapping, and parameter/human checkpoint handling.
- [x] CHK002 Requirements reject direct script generation without validated scenario graph.
- [x] CHK003 No unresolved `[NEEDS CLARIFICATION]` markers remain.
- [x] CHK004 Edge cases include cycles, duplicate refs, stale baselines, unsupported XLSX, and unknown selectors.
## Constitution Coverage
- [x] CHK005 ADR relations include semantic protocol and upstream 036/037 dependencies.
- [x] CHK006 Decision memory records rejection of low-level user tool selection and raw script generation.
- [x] CHK007 Requirements support test-driven validation of C3+ model contracts.
- [x] CHK008 Artifact generation is downstream and not mixed into this model spec.
## Readiness for Plan
- [x] CHK009 Key entities define scenario, steps, refs, parameters, checklist cases, mappings, and validation result.
- [x] CHK010 Success criteria are measurable with deterministic fixtures and snapshots.
- [x] CHK011 Scope excludes UI implementation and Superset query engine internals.

111
specs/038-dashboard-scenario-model/spec.md Normal file
View File

@@ -0,0 +1,111 @@
#region DashboardScenarioModel.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,scenario,dashboard-testing]
@BRIEF Define a validated ScenarioGraph model for unique dashboard test flows generated by agents.
@RELATION DEPENDS_ON -> [ADR-0001]
@RELATION DEPENDS_ON -> [ADR-0002]
@RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
@RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec]
@RATIONALE Unique dashboard tests need a stable intermediate model between agent reasoning and generated artifacts; direct LLM-to-code generation is not reviewable or safely composable.
@REJECTED Asking users to choose low-level outputs such as Playwright vs SQL vs XLSX — rejected because each dashboard scenario is a goal-oriented flow whose steps select tools automatically.
@REJECTED Direct generation of executable scripts without a validated scenario graph — rejected because it hides missing selectors, baseline refs, and unsafe steps until runtime.
## Navigation (DSA Indexer keywords)
@SEMANTICS: spec, requirements, feature, scenario, graph, dashboard-testing, checklist, validation, artifacts
**Feature Branch**: `038-dashboard-scenario-model`
**Created**: 2026-07-07 | **Status**: Draft
**Input**: "Define the dashboard test scenario model used by agents to represent unique dashboard test flows as a validated ScenarioGraph. The model must express ordered and dependent steps across browser automation, Superset query execution, XLSX parsing, assertions, screenshots, reports, human checkpoints, baseline references, parameters, warnings, and missing context markers without exposing users to low-level tool selection."
## User Scenarios
### Story 1 — Build Scenario Graph From Dashboard Goal (P1)
**Why P1**: The agent must express a dashboard testing goal as a reviewable graph of steps before generating any executable artifacts.
**Independent Test**: Provide dashboard query model and checklist fixture input and verify a deterministic `DashboardTestScenario` graph is produced.
**Acceptance**:
1. **Given** a dashboard context, query model, and testing objective **When** the agent builds a scenario **Then** the output contains scenario id, dashboard context, parameters, steps, dependencies, expected outputs, warnings, and risk summary.
2. **Given** a scenario needs browser, Superset API, XLSX parsing, screenshots, assertions, and report steps **When** graph is rendered **Then** every step declares its tool and input/output refs.
3. **Given** required metadata is missing **When** graph is generated **Then** missing data is represented as `NEEDS_CONTEXT` or `NEEDS_SELECTOR`, not invented.
---
### Story 2 — Validate Scenario Graph Safety and Completeness (P1)
**Why P1**: Scenario artifacts must be generated only from a graph whose refs, baselines, tools, and risks are internally consistent.
**Independent Test**: Run valid and invalid scenario fixtures through the validator and verify precise errors for broken refs, missing baseline refs, and unsupported steps.
**Acceptance**:
1. **Given** a step consumes an output ref **When** validation runs **Then** the referenced producer must exist and precede or depend correctly.
2. **Given** an assertion compares a metric to expected truth **When** validation runs **Then** it must reference an approved baseline or a draft baseline candidate with explicit approval requirement.
3. **Given** a step uses an unsupported tool or unsafe action **When** validation runs **Then** the graph is rejected or marked blocked with a user-visible reason.
---
### Story 3 — Map Checklist Cases to Scenario Capabilities (P2)
**Why P2**: The PDF checklist should guide scenario coverage without forcing all dashboards into identical scripts.
**Independent Test**: Normalize the research checklist into capability-tagged cases and verify generated scenarios include applicable cases and mark non-applicable ones.
**Acceptance**:
1. **Given** the normalized checklist template contains basic, complex, and technical cases **When** a dashboard capability model is supplied **Then** applicable cases are mapped to scenario steps or human checkpoints.
2. **Given** a checklist case requires capabilities absent from the dashboard **When** mapping runs **Then** the case is marked unsupported or manual-only with rationale.
3. **Given** multiple tool choices could verify a case **When** mapping runs **Then** the scenario selects the tool chain that best matches the business goal and available capabilities.
---
### Story 4 — Represent Parameters and Human Checkpoints (P2)
**Why P2**: Unique dashboard scenarios require business inputs and sometimes cannot be fully automated.
**Independent Test**: Generate a scenario that requires test date, counterparty, and baseline selection; verify parameter prompts and human checkpoint steps are represented structurally.
**Acceptance**:
1. **Given** a scenario needs test data **When** graph is generated **Then** required parameters include name, type, validation rule, default/source, and affected steps.
2. **Given** an action cannot be automated reliably **When** graph is generated **Then** a human checkpoint step describes the manual action and expected evidence.
3. **Given** a user later supplies a parameter **When** graph is resolved **Then** dependent steps reference the resolved value without changing unrelated graph structure.
---
### Edge Cases
- Scenario has a cycle in dependencies → validator rejects with cycle path.
- Two steps produce the same output ref → validator rejects ambiguous ref.
- Baseline is stale → assertion step remains present but blocked or warning-gated.
- XLSX export is unavailable → XLSX-dependent checklist cases become unsupported or manual checkpoints.
- UI selector is unknown → browser step uses `NEEDS_SELECTOR` and blocks executable generation for that step.
## Requirements
### Functional
- **AGSCN-FR-001**: The system MUST define a `DashboardTestScenario` model with dashboard context, objective, parameters, steps, dependencies, outputs, artifacts, risks, and warnings.
- **AGSCN-FR-002**: Every scenario step MUST declare tool category, action, inputs, outputs, expected result, dependencies, and automation status.
- **AGSCN-FR-003**: Supported tool categories MUST include browser automation, Superset API execution, XLSX parsing, assertion, screenshot/evidence, report generation, artifact generation, and human checkpoint.
- **AGSCN-FR-004**: Assertions against reference values MUST use baseline references or baseline candidate references; raw expected numbers MUST NOT be embedded directly in executable steps.
- **AGSCN-FR-005**: Scenario validation MUST detect missing refs, cycles, duplicate outputs, missing baselines, stale baselines, unknown selectors, unsupported tools, and unresolved required parameters.
- **AGSCN-FR-006**: Checklist mapping MUST use normalized checklist cases derived from the research PDF and capability tags, not hardcoded one-size-fits-all scripts.
- **AGSCN-FR-007**: The model MUST allow manual/human checkpoint steps where automation is unsafe, unavailable, or underspecified.
- **AGSCN-FR-008**: Scenario output MUST be deterministic for the same dashboard query model, checklist template, baseline catalog, and user parameters.
- **AGSCN-FR-009**: The scenario model MUST remain implementation-neutral and must not require the user to choose low-level artifacts such as Playwright, XLSX, or API output upfront.
### Key Entities
- **DashboardTestScenario**: Reviewable graph representing a dashboard-specific testing objective and all steps required to validate it.
- **ScenarioStep**: One executable, generated, assertion, evidence, or human checkpoint node in the graph.
- **ScenarioParameter**: User- or context-provided input used by one or more scenario steps.
- **ScenarioRef**: Named output produced by one step and consumed by later steps.
- **ChecklistCase**: Normalized item from the research checklist with capability tags and expected verification semantics.
- **CapabilityMapping**: Decision record mapping dashboard capabilities to applicable checklist cases and step templates.
- **ScenarioValidationResult**: Structured validator output with errors, warnings, blockers, and graph coverage.
## Success Criteria
- **SC-001**: Scenario validator catches 100% of invalid fixture cases for missing refs, cycles, duplicate outputs, and missing baseline refs.
- **SC-002**: At least 80% of normalized PDF checklist cases are classifiable as automated, human checkpoint, unsupported, or needs-context for fixture dashboards.
- **SC-003**: Same inputs produce byte-stable scenario JSON/YAML in deterministic snapshot tests.
- **SC-004**: No generated scenario fixture embeds raw baseline numbers directly in executable steps.
- **SC-005**: Scenario graph preview can display phase order, tools per step, parameters, warnings, and blockers without reading generated code.
#endregion DashboardScenarioModel.Spec

80
specs/038-dashboard-scenario-model/ux_reference.md Normal file
View File

@@ -0,0 +1,80 @@
#region DashboardScenarioModel.UxReference [C:3] [TYPE ADR] [SEMANTICS ux,reference,scenario,dashboard-testing]
@BRIEF UX reference for reviewing a generated DashboardTestScenario graph as a goal-oriented test flow.
**Feature Branch**: `038-dashboard-scenario-model`
**Created**: 2026-07-07 | **Status**: Draft
## 1. User Persona & Context
* **Who is the user?**: QA engineer or dashboard owner reviewing the agent's proposed test scenario before execution or artifact generation.
* **What is their goal?**: Understand what will be tested, which tools each step uses, what parameters are needed, and what cannot be automated.
* **Context**: Agent workspace shows a scenario graph produced from dashboard metadata, checklist template, and baselines.
## 2. Happy Path Narrative
The agent proposes a scenario called "Проверка фильтров, метрик и XLSX выгрузки". The user sees phases, steps, dependencies, tool categories, expected outcomes, and missing parameters. The scenario is a business flow, not a menu of technologies, so the user approves the goal and parameters while the graph records tool selection internally.
## 3. Interface Mockups
### Scenario Summary
```text
┌──────────────────────── Proposed scenario ──────────────────────────────────┐
│ Goal: проверить фильтры, метрику, XLSX выгрузку и baseline │
│ Steps: 18 | Tools: browser, Superset API, XLSX, assertions, report │
│ Parameters required: test_date, counterparty │
│ Blockers: 0 | Warnings: 2 │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Scenario Graph
```text
[open_dashboard]
│
▼
[apply_filters] ───────► [execute_superset_metric]
│ │
▼ ▼
[download_xlsx] ───────► [parse_xlsx_metric]
│ │
└──────────────► [compare_to_baseline] ──► [generate_report]
```
### Step Table
```text
┌────┬────────────────────────────┬──────────────┬───────────────────────────┐
│ № │ Step │ Tool │ Expected result │
├────┼────────────────────────────┼──────────────┼───────────────────────────┤
│ 1 │ Открыть дашборд │ browser │ dashboard_loaded │
│ 2 │ Применить фильтры │ browser │ filter_state.normalized │
│ 3 │ Выполнить chart query │ Superset API │ metric value returned │
│ 4 │ Скачать XLSX │ browser │ xlsx.file │
│ 5 │ Сравнить с baseline │ assertion │ pass/fail/inconclusive │
└────┴────────────────────────────┴──────────────┴───────────────────────────┘
```
## 4. Error Experience
### Scenario A: Missing Selector
* **System Response**: Step is marked `NEEDS_SELECTOR` and executable generation for that step is blocked.
* **Recovery**: User can provide selector hint, convert to human checkpoint, or remove the step.
### Scenario B: Stale Baseline
* **System Response**: Assertion step shows warning and stale fingerprint category.
* **Recovery**: User can run baseline discovery through 037 or mark the check as pending.
### Scenario C: Unsupported Checklist Case
* **System Response**: Checklist case is listed under unsupported/manual-only with rationale.
* **Recovery**: User can accept partial coverage or add manual checkpoint instructions.
## 5. Tone & Voice
* **Style**: Goal-oriented, explicit about confidence and blockers.
* **Terminology**: Use "scenario", "step", "tool", "parameter", "baseline ref", "human checkpoint".
#endregion DashboardScenarioModel.UxReference

25
specs/039-dashboard-scenario-ui/checklists/requirements.md Normal file
View File

@@ -0,0 +1,25 @@
# Requirements Checklist: 039 Dashboard Scenario UI
**Purpose**: Validate user-facing dashboard scenario generation specification.
**Created**: 2026-07-07
**Feature**: specs/039-dashboard-scenario-ui/spec.md
## Spec Completeness
- [x] CHK001 User stories cover entry point, scenario preview, parameters, artifacts, save, and approval.
- [x] CHK002 Requirements avoid low-level tool dropdowns as the primary UX.
- [x] CHK003 No unresolved `[NEEDS CLARIFICATION]` markers remain.
- [x] CHK004 Edge cases include Superset errors, unresolved blockers, stale baselines, unavailable XLSX, and navigation recovery.
## Constitution Coverage
- [x] CHK005 ADR relations include frontend architecture, RBAC, and upstream specs 036-038.
- [x] CHK006 Decision memory rejects Playwright/SQL/XLSX primary dropdown UX.
- [x] CHK007 Svelte 5 runes/model-first requirement is explicit.
- [x] CHK008 HITL confirmation and RBAC are explicit for durable actions.
## Readiness for Plan
- [x] CHK009 UI entities are defined for entry action, workspace state, preview, parameters, baselines, artifacts, and confirmations.
- [x] CHK010 Success criteria are measurable through model, component, and browser tests.
- [x] CHK011 UX copy explicitly excludes direct SQL as a validation path.

127
specs/039-dashboard-scenario-ui/spec.md Normal file
View File

@@ -0,0 +1,127 @@
#region DashboardScenarioUi.Spec [C:3] [TYPE ADR] [SEMANTICS spec,requirements,ux,scenario,dashboard-testing]
@BRIEF User-facing dashboard test scenario generation experience driven by agent analysis, scenario preview, parameter collection, artifact preview, and HITL save/approval.
@RELATION DEPENDS_ON -> [ADR-0001]
@RELATION DEPENDS_ON -> [ADR-0005]
@RELATION DEPENDS_ON -> [ADR-0006]
@RELATION DEPENDS_ON -> [AgentTestStabilization.Spec]
@RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec]
@RELATION DEPENDS_ON -> [DashboardScenarioModel.Spec]
@RATIONALE Users should approve a business-level dashboard test scenario and required parameters; low-level tool chains are visible for trust but selected by the agent and scenario validator.
@REJECTED Dropdowns such as "Playwright UI tests" vs "SQL checks" vs "XLSX checks" — rejected because each dashboard requires a unique cross-tool scenario, and SQL is explicitly out of scope.
## Navigation (DSA Indexer keywords)
@SEMANTICS: spec, requirements, feature, ux, agent, scenario, dashboard-testing, artifacts, baseline
**Feature Branch**: `039-dashboard-scenario-ui`
**Created**: 2026-07-07 | **Status**: Draft
**Input**: "Provide the user-facing dashboard test scenario generation experience. From a dashboard page the user starts Create test scenario, the agent analyzes the dashboard, proposes a unique scenario graph, collects missing business parameters, previews generated artifacts and baseline impacts, and requires HITL confirmation before saving files or approving baselines."
## User Scenarios
### Story 1 — Start Scenario From Dashboard Page (P1)
**Why P1**: The feature must be discoverable from the dashboard the user wants to test and must carry the correct context into the agent.
**Independent Test**: On a dashboard page, click "Создать сценарий тестирования" and verify `/agent` opens with dashboard context and scenario intent.
**Acceptance**:
1. **Given** a user is viewing a dashboard **When** they click "Создать сценарий тестирования" **Then** `/agent` opens in Dashboard Test Scenario Agent mode with dashboard id, name, env, route, and intent visible.
2. **Given** environment context is missing **When** the user starts the flow **Then** the UI prompts for environment selection before scenario analysis.
3. **Given** the user lacks scenario-generation permission **When** they click the action **Then** a permission-denied recovery path appears and no agent run starts.
---
### Story 2 — Review Proposed Unique Scenario (P1)
**Why P1**: The user must see the proposed business flow and tool chain before artifact generation.
**Independent Test**: Feed fixture scenario graph data and verify the UI renders phases, steps, tools, outputs, warnings, blockers, and coverage.
**Acceptance**:
1. **Given** the agent analyzes a dashboard **When** scenario graph is ready **Then** the UI shows goal, phases, step table, dependency graph, tool categories, expected results, coverage, warnings, and blockers.
2. **Given** some checklist cases are not automatable **When** scenario preview renders **Then** they appear as human checkpoints, unsupported, or needs-context with rationale.
3. **Given** the scenario includes Superset API metric validation **When** preview renders **Then** the UI clearly states Superset-native execution is used and direct SQL is not used.
---
### Story 3 — Collect Business Parameters and Baseline Choices (P1)
**Why P1**: Unique dashboard tests need values such as test date, counterparty, and baseline selection before generation is meaningful.
**Independent Test**: Scenario preview requests parameters; user fills them; dependent step statuses update from unresolved to ready.
**Acceptance**:
1. **Given** a scenario requires parameters **When** preview renders **Then** each parameter has label, type, validation message, default/source if available, and affected steps.
2. **Given** an approved baseline exists for metric+filters **When** the user selects matching parameters **Then** baseline match is shown and assertion steps become ready.
3. **Given** no approved baseline exists **When** the user proceeds **Then** the UI offers discovery candidate flow and marks baseline approval as separate HITL action.
---
### Story 4 — Preview Generated Artifacts (P2)
**Why P2**: Users need to inspect generated files, report templates, and warnings before saving.
**Independent Test**: Generate draft artifacts from a fixture scenario and verify file tree, validation status, warnings, and content preview render.
**Acceptance**:
1. **Given** draft artifacts are generated **When** preview opens **Then** the UI lists scenario file, generated runner/check files, report template, baseline candidate files, and validation status.
2. **Given** an artifact has unresolved markers **When** preview renders **Then** save is warning-gated and unresolved markers are visible.
3. **Given** user downloads draft artifacts **When** download completes **Then** repository state remains unchanged.
---
### Story 5 — Save or Approve With HITL (P2)
**Why P2**: Saving test packs and approving baselines have durable effects and must require explicit user approval.
**Independent Test**: Trigger save and baseline approval actions and verify confirmation cards include target, risk, warnings, and approval reason where required.
**Acceptance**:
1. **Given** the user requests save **When** generated artifacts target repository paths **Then** confirmation shows target paths, files, warnings, and risk before saving.
2. **Given** baseline approval is requested **When** confirmation appears **Then** the user must provide a reason and can inspect provenance/diff.
3. **Given** the user denies either action **When** denial is submitted **Then** no save or approval occurs and the run records cancellation.
---
### Edge Cases
- Agent analysis fails due to Superset API error → UI shows retry, switch environment, or manual-only scenario recovery.
- Scenario has unresolved blockers → generation can continue only for safe preview; executable save is blocked until resolved or converted to human checkpoint.
- Baseline is stale → UI warns and offers discovery candidate flow, not silent update.
- XLSX export is unavailable → UI shows coverage gap and alternate Superset API/UI checks.
- User navigates away during generation → run id allows returning to latest draft state from 036.
## Requirements
### Functional
- **AGUI-FR-001**: Dashboard pages MUST expose a single business-level action labeled "Создать сценарий тестирования"; UI MUST NOT present low-level artifact/tool choices as the primary entry point.
- **AGUI-FR-002**: `/agent` MUST show Dashboard Test Scenario Agent mode when opened with scenario intent from 036.
- **AGUI-FR-003**: The UI MUST render structured progress stages from agent metadata: context, inspect, scenario, parameters, generate, validate, save.
- **AGUI-FR-004**: The scenario preview MUST show business goal, phases, step list, tool category per step, dependency graph, coverage, warnings, blockers, and expected results.
- **AGUI-FR-005**: Parameter collection MUST support typed required parameters and update dependent scenario readiness without restarting the full flow.
- **AGUI-FR-006**: Baseline UI MUST show approved baseline matches, stale warnings, missing baselines, and draft candidate approval paths.
- **AGUI-FR-007**: Generated artifact preview MUST show file tree, file content preview, validation status, unresolved markers, and warnings before save.
- **AGUI-FR-008**: Saving generated files and approving baselines MUST go through HITL confirmation from 036; baseline approval requires a reason.
- **AGUI-FR-009**: The UI MUST explicitly communicate that metric validation uses Superset-native execution and does not execute direct SQL.
- **AGUI-FR-010**: All new UI state MUST follow Svelte 5 runes/model-first conventions and remain accessible by keyboard for confirmations, parameter forms, and previews.
### Key Entities
- **ScenarioEntryAction**: Dashboard-page action that launches `/agent` with dashboard scenario intent.
- **ScenarioWorkspaceState**: Frontend state machine for context, progress, scenario preview, parameter collection, artifact preview, and confirmation states.
- **ScenarioPreviewCard**: UI representation of `DashboardTestScenario` from 038.
- **ParameterPanel**: Form for scenario business parameters and validation feedback.
- **BaselineImpactPanel**: UI section showing approved, stale, missing, and candidate baseline statuses.
- **ArtifactPreviewPanel**: UI file tree/content preview for draft generated artifacts.
- **ScenarioConfirmationCard**: HITL card for save and baseline approval actions.
## Success Criteria
- **SC-001**: Users can launch scenario generation from a dashboard page in one click and see correct context in `/agent` in frontend tests.
- **SC-002**: Scenario preview renders at least 15-step fixture graphs with phases, tools, warnings, and blockers without layout collapse at 1366px width.
- **SC-003**: Parameter updates resolve dependent scenario readiness within 200ms in model tests.
- **SC-004**: Artifact preview blocks save when unresolved markers are present in fixture data.
- **SC-005**: 100% of save and baseline approval flows require confirmation; baseline approval cannot submit without reason.
- **SC-006**: UX copy never presents direct SQL as a validation path for this feature.
#endregion DashboardScenarioUi.Spec

128
specs/039-dashboard-scenario-ui/ux_reference.md Normal file
View File

@@ -0,0 +1,128 @@
#region DashboardScenarioUi.UxReference [C:4] [TYPE ADR] [SEMANTICS ux,reference,scenario,dashboard-testing]
@BRIEF UX reference for the Dashboard Test Scenario Agent workspace and dashboard entry point.
@RELATION DEPENDS_ON -> [DashboardScenarioUi.Spec]
@RATIONALE The user experience is centered on approving a business scenario, not selecting implementation technologies.
@REJECTED Primary dropdown options for Playwright, SQL, and XLSX were rejected because they expose implementation details and conflict with unique cross-tool scenario generation.
**Feature Branch**: `039-dashboard-scenario-ui`
**Created**: 2026-07-07 | **Status**: Draft
## 1. User Persona & Context
* **Who is the user?**: QA engineer, dashboard owner, or analyst responsible for repeatable dashboard validation.
* **What is their goal?**: Generate a unique dashboard test scenario, provide business parameters, inspect generated artifacts, and approve durable changes.
* **Context**: Svelte UI in superset-tools; entry from dashboard page into `/agent` with scenario intent.
## 2. Happy Path Narrative
The user opens a dashboard and clicks "Создать сценарий тестирования". The agent analyzes the dashboard, presents a proposed scenario flow with browser, Superset API, XLSX, assertion, evidence, and report steps. The user fills required business parameters, previews generated artifacts and baseline impacts, then confirms save or keeps the artifacts as draft.
## 3. Interface Mockups
### Dashboard Entry
```text
┌─────────────────────────────────────────────────────────────────────────────┐
│ Dashboard: FI-0080 Env: ss-dev │
├─────────────────────────────────────────────────────────────────────────────┤
│ [Открыть в Superset] [Валидация] [Документация] │
│ │
│ [Создать сценарий тестирования] │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Agent Workspace
```text
┌─────────────────────────────────────────────────────────────────────────────┐
│ 🧪 Dashboard Test Scenario Agent ● connected │
├─────────────────────────────────────────────────────────────────────────────┤
│ Context: FI-0080 | dashboard_id=42 | env=ss-dev │
│ Progress: [Context ✓] → [Inspect ✓] → [Scenario …] → [Parameters] → [Save] │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Scenario Preview
```text
┌──────────────────────── Предложенный сценарий ──────────────────────────────┐
│ Цель: проверить фильтры, метрику и XLSX выгрузку │
│ Steps: 18 | Tools: browser, Superset API, XLSX, assertions, report │
│ │
│ № Step Tool Expected │
│ 1 Открыть дашборд browser dashboard_loaded │
│ 2 Применить фильтры browser filter_state.normalized │
│ 3 Выполнить chart query Superset API metric returned │
│ 4 Скачать XLSX browser xlsx.file │
│ 5 Сравнить с baseline assertion pass/fail/inconclusive │
│ │
│ [Открыть граф] [Заполнить параметры] [Сгенерировать draft] │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Parameter Panel
```text
┌──────────────────────── Параметры сценария ─────────────────────────────────┐
│ Дата тестирования [2026-05-29____________________] │
│ Контрагент [АСК___________________________] │
│ Baseline mode [использовать approved или создать candidate ▼] │
│ Проверять XLSX [✓] │
│ Делать screenshots [✓] │
│ │
│ [Применить] [Сбросить] │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Artifact Preview
```text
┌──────────────────────── Generated draft ────────────────────────────────────┐
│ dashboard_42_test_scenario/ │
│ ├── scenario.yaml ✓ valid │
│ ├── runner.plan.json ✓ valid │
│ ├── xlsx_assertions.py ⚠ needs review │
│ ├── report_template.md ✓ valid │
│ └── baseline_candidates.yaml ⚠ approval required │
│ │
│ [Preview file] [Download draft] [Save to repository] │
└─────────────────────────────────────────────────────────────────────────────┘
```
### Baseline Approval
```text
┌──────────────────────── Baseline approval ──────────────────────────────────┐
│ Metric: Просроченная ДЗ │
│ Filters: Дата=2026-05-29, Контрагент=АСК │
│ Candidate: 1 234 567.89 from Superset API │
│ Existing baseline: none │
│ Reason required: [Initial approved QA baseline________________________] │
│ │
│ [Confirm approval] [Keep draft] [Deny] │
└─────────────────────────────────────────────────────────────────────────────┘
```
## 4. Error Experience
### Scenario A: Superset Analysis Fails
* **System Response**: Error card identifies Superset API status and dashboard context.
* **Recovery**: Retry, switch environment, or continue with manual-only scenario shell.
### Scenario B: Unresolved Blockers
* **System Response**: Save action is disabled for executable artifacts; blockers are grouped by step.
* **Recovery**: Fill parameter, provide selector hint, convert to human checkpoint, or remove step.
### Scenario C: User Leaves Mid-Generation
* **System Response**: On return, `/agent` displays recoverable run id and latest draft state.
* **Recovery**: Resume preview, discard draft, or restart analysis.
## 5. Tone & Voice
* **Style**: Clear, operational, confidence-aware.
* **Terminology**: Use "scenario", "draft", "artifact", "baseline", "Superset API", "human checkpoint". Avoid "SQL check" for this feature.
#endregion DashboardScenarioUi.UxReference