busya
b529e8082a
Backend: - Add MetricItem, StatsCounts, ColumnDescriptionUpdate, MetricDescriptionUpdate DTOs - Add metric_count to DatasetItem and DatasetDetailResponse - Add server-side filtering via ?filter=unmapped|mapped|linked|all - Add PUT endpoints for column/metric description inline-edit - Add HTML stripping validation (max 2000 chars, plain text) - Add WebSocket dataset.updated event on task completion - RBAC: plugin:migration:WRITE for mutations, READ for reads - 14 pytest tests (stats, filter, metrics, inline-edit, validation, 502/503) Frontend: - Rewrite +page.svelte as split-view orchestrator (387 lines) - StatsBar.svelte — 4 metric tiles with aria-pressed, server-side filter dispatch - DatasetList.svelte — card grid with progress bars, checkboxes, search - DatasetPreview.svelte — presentational detail panel (container fetches) - ColumnsTable.svelte — inline-edit with type chips, bold/italic fallback - MetricsTable.svelte — inline-edit with expression hints - 52 vitest tests across 7 test files - i18n: 11 EN + 11 RU keys (with_mapping/с_маппингом) Spec: - 15 issues resolved (semantic label, filtering, endpoints, conflict, metric_count, RBAC, ID types, ports, i18n, validation, a11y, propagation, resize, perf, ownership) Status: 14/14 backend tests pass, 52/52 frontend tests pass, build succeeds Risk: Low — T051 browser validation pending (non-blocking)
15 KiB
Contracts: Dataset Lifecycle Workspace
Generated for speckit feature 030. All IDs scoped to this feature. Complexity per GRACE-Poly v2.6. Allowed predicates: DEPENDS_ON, CALLS, INHERITS, IMPLEMENTS, DISPATCHES, BINDS_TO.
Backend Contracts
#region MetricItem [C:1] [TYPE DataClass]
@BRIEF Pydantic DTO for a dataset metric — carries Superset metric metadata.
@RELATION DEPENDS_ON -> [DatasetDetailResponse]
File: backend/src/api/routes/datasets.py
class MetricItem(BaseModel):
id: int
metric_name: str
expression: str | None = None
verbose_name: str | None = None
description: str | None = None
metric_type: str | None = None
#region StatsCounts [C:1] [TYPE DataClass]
@BRIEF Aggregate statistics for the Stats Bar — computed from full dataset list.
File: backend/src/api/routes/datasets.py
class StatsCounts(BaseModel):
total: int
unmapped_count: int
mapped_count: int
linked_count: int
#region ColumnDescriptionUpdate [C:1] [TYPE DataClass]
@BRIEF Request DTO for inline-edit of a column description.
File: backend/src/api/routes/datasets.py
class ColumnDescriptionUpdate(BaseModel):
description: str
#region MetricDescriptionUpdate [C:1] [TYPE DataClass]
@BRIEF Request DTO for inline-edit of a metric description.
File: backend/src/api/routes/datasets.py
class MetricDescriptionUpdate(BaseModel):
description: str
#region DatasetDetailResponse [C:2] [TYPE DataClass] — MODIFIED
@BRIEF Detailed DTO for a dataset including columns, linked dashboards, and metrics.
@RELATION DEPENDS_ON -> [MetricItem]
File: backend/src/api/routes/datasets.py
Modification: Add metrics: list[MetricItem] and metric_count: int.
#region DatasetsResponse [C:2] [TYPE DataClass] — MODIFIED
@BRIEF Paginated response DTO for dataset listings with stats.
@RELATION DEPENDS_ON -> [StatsCounts]
File: backend/src/api/routes/datasets.py
Modification: Add stats: StatsCounts.
#region update_column_description [C:4] [TYPE Endpoint]
@BRIEF Save description for a single dataset column. Internal: GET full dataset from Superset, modify one column's description, PUT back.
@PRE: dataset_id and column_id must exist in the target environment.
@POST: Column description in Superset is updated. Response confirms success.
@SIDE_EFFECT: Mutates dataset metadata in upstream Superset instance via PUT.
@VALIDATION: description — string, max 2000 chars, plain text only (HTML stripped), may be empty string to clear description, no trimming applied (preserve leading/trailing whitespace). 404 if dataset or column not found. 502/503 if Superset upstream fails.
@ERROR: 400 — description exceeds max length or contains non-plaintext content. 404 — dataset_id or column_id not found. 502 — Superset GET failed. 503 — Superset PUT failed.
@RELATION CALLS -> [SupersetClient.get_dataset]
@RELATION CALLS -> [SupersetClient.update_dataset]
@RATIONALE Must perform GET→modify→PUT because Superset has no PATCH for individual columns — only full object PUT with override_columns=false.
@REJECTED Direct PUT from frontend — rejected because frontend would need to handle full Superset payload structure.
File: backend/src/api/routes/datasets.py
Route: PUT /api/datasets/{dataset_id}/columns/{column_id}/description
Method: PUT
Request: ColumnDescriptionUpdate
Response: {success: true, description: "new text"}
RBAC: has_permission("plugin:migration", "WRITE")
#region update_metric_description [C:4] [TYPE Endpoint]
@BRIEF Save description for a single dataset metric. Mirror of update_column_description for metrics.
@PRE: dataset_id and metric_id must exist in the target environment.
@POST: Metric description in Superset is updated.
@SIDE_EFFECT: Mutates dataset metadata in upstream Superset instance via PUT.
@VALIDATION: description — string, max 2000 chars, plain text only (HTML stripped), may be empty string to clear description, no trimming applied. 404 if dataset or metric not found. 502/503 if Superset upstream fails.
@ERROR: 400 — description exceeds max length or contains non-plaintext content. 404 — dataset_id or metric_id not found. 502 — Superset GET failed. 503 — Superset PUT failed.
@RELATION CALLS -> [SupersetClient.get_dataset]
@RELATION CALLS -> [SupersetClient.update_dataset]
File: backend/src/api/routes/datasets.py
Route: PUT /api/datasets/{dataset_id}/metrics/{metric_id}/description
Request: MetricDescriptionUpdate
Response: {success: true, description: "new text"}
RBAC: has_permission("plugin:migration", "WRITE")
#region get_datasets [C:4] [TYPE Endpoint] — MODIFIED
@BRIEF Fetch paginated dataset list with enhanced metadata, mapped fields, and stats counts.
@PRE: env_id must be valid. page >= 1, page_size in [1,100].
@POST: Returns paginated datasets plus StatsCounts for the Stats Bar.
@SIDE_EFFECT: Computes aggregate counts from full dataset list (in-memory, no DB write).
@RELATION CALLS -> [resource_service.get_datasets_with_status]
@RATIONALE Stats counts returned in the same response as datasets to avoid an extra API call for the Stats Bar (FR-022).
File: backend/src/api/routes/datasets.py
Modification: Compute stats dict from full dataset list before pagination slicing, add to DatasetsResponse.
#region get_dataset_detail [C:3] [TYPE Endpoint] — MODIFIED
@BRIEF Get detailed dataset information including columns, linked dashboards, and metrics.
@RELATION CALLS -> [SupersetClient.get_dataset_detail]
File: backend/src/api/routes/datasets.py
Modification: Extract dataset.get("metrics", []) from Superset response, map to MetricItem list.
#region SupersetClient.get_dataset_detail [C:3] [TYPE Function] — MODIFIED
@BRIEF Extract dataset detail from Superset API including metrics.
@RELATION CALLS -> [SupersetClient.get_dataset]
File: backend/src/core/superset_client/_datasets.py
Modification: Add metrics extraction loop after columns loop. Add metrics and metric_count to result dict.
Frontend Contracts
#region StatsBar [C:3] [TYPE Component] [SEMANTICS ui,dataset,stats,filter]
@BRIEF Displays 4 aggregate metric tiles (All, Without mapping, Mapped, Linked) and emits filter selection. Filtering is server-side via query param — parent container calls API on tile click.
@LAYER UI
@RELATION BINDS_TO -> [environmentContextStore]
@UX_STATE Loading -> 4 skeleton tiles pulsing.
@UX_STATE Loaded -> 4 tiles with numbers, active filter highlighted.
@UX_STATE Filtered -> Selected tile has accent border; parent triggers API reload with filter param.
@UX_STATE Empty -> All tiles show 0.
@UX_REACTIVITY Props -> stats: StatsCounts, activeFilter: string|null.
@UX_REACTIVITY Events -> onfilter(filterKey) dispatched on tile click; parent calls loadDatasets(filter=filterKey).
File: frontend/src/routes/datasets/StatsBar.svelte
#region DatasetList [C:4] [TYPE Component] [SEMANTICS ui,dataset,card,list,pagination]
@BRIEF Renders paginated dataset cards with mapping progress bars, checkboxes, and quick actions.
@LAYER UI
@RELATION BINDS_TO -> [environmentContextStore]
@RELATION BINDS_TO -> [taskDrawerStore]
@UX_STATE Loading -> Skeleton cards.
@UX_STATE Loaded -> Cards with progress bars.
@UX_STATE Selecting -> Checkboxes visible, checked cards highlighted.
@UX_STATE Empty -> "No datasets found" after search/filter.
@UX_FEEDBACK Click card → dispatches dataset selection for detail panel.
@UX_FEEDBACK Click checkbox → updates selection set.
@UX_FEEDBACK Click quick action → opens Map Columns or Generate Docs modal.
@UX_REACTIVITY Props -> datasets: DatasetCard[], selectedIds: Set, activeFilter: string|null.
@UX_REACTIVITY Events -> onselect(datasetId), onaction(dataset, action).
File: frontend/src/routes/datasets/DatasetList.svelte
#region DatasetPreview [C:3] [TYPE Component] [SEMANTICS ui,dataset,detail,columns,metrics]
@BRIEF Right panel showing dataset detail: header info, columns table, metrics table, linked dashboards. Presentational component — receives dataset data from parent container.
@LAYER UI
@RELATION DEPENDS_ON -> [ColumnsTable]
@RELATION DEPENDS_ON -> [MetricsTable]
@UX_STATE NoSelection -> "Select a dataset from the list" placeholder.
@UX_STATE Loading -> Skeleton for header + 2 tables.
@UX_STATE Loaded -> Full detail with columns and metrics rendered.
@UX_STATE Error -> Error banner with retry (retry triggers callback to parent).
@UX_REACTIVITY Props -> dataset: DatasetDetailResponse | null, isLoading: boolean, error: string | null.
@UX_REACTIVITY Events -> onretry() callback dispatched when retry button clicked.
@RATIONALE Container (DatasetHub) fetches data and passes it as props; DatasetPreview is a presentational component for cleaner testing and separation of concerns.
@REJECTED Self-fetching component — rejected because it couples data fetching with rendering, harder to test, and creates race conditions when switching datasets rapidly.
@UX_TEST: NoSelection -> {click: card, expected: Loading -> Loaded}.
File: frontend/src/routes/datasets/DatasetPreview.svelte
#region ColumnsTable [C:4] [TYPE Component] [SEMANTICS ui,dataset,columns,inline-edit]
@BRIEF Table of dataset columns with type chips, description, and inline-edit capability.
@LAYER UI
@RELATION CALLS -> [api_module]
@UX_STATE Display -> All rows in read mode.
@UX_STATE Editing -> One row in edit mode (textarea), others dimmed.
@UX_STATE Saving -> Spinner on save button.
@UX_STATE Error -> Red border on textarea + error message.
@UX_FEEDBACK verbose_name → shown bold; missing → column_name in italic as fallback.
@UX_FEEDBACK No description → "✏️ Add description" action prompt.
@UX_REACTIVITY Props -> columns: ColumnRow[], datasetId: number.
@UX_REACTIVITY LocalState -> isEditingRowId: number|null.
@RATIONALE Inline-edit uses optimised state per row to minimise re-renders across the table.
@REJECTED Modal-per-column editing — rejected as too many clicks for repetitive workflow.
File: frontend/src/routes/datasets/ColumnsTable.svelte
#region MetricsTable [C:4] [TYPE Component] [SEMANTICS ui,dataset,metrics,inline-edit]
@BRIEF Table of dataset metrics with expression, description, and inline-edit capability.
@LAYER UI
@RELATION CALLS -> [api_module]
@UX_STATE Display -> All rows in read mode with expression shown under name.
@UX_STATE Editing -> One row in edit mode.
@UX_STATE Saving -> Spinner on save button.
@UX_FEEDBACK verbose_name → shown bold; missing → metric_name in italic as fallback.
@UX_FEEDBACK No description → shows expression as hint + "✏️ Add description" prompt.
@UX_REACTIVITY Props -> metrics: MetricRow[], datasetId: number.
File: frontend/src/routes/datasets/MetricsTable.svelte
#region DatasetHub [C:5] [TYPE Page] — REWRITTEN
@BRIEF Dataset lifecycle workspace page — split-view with Stats Bar, card list, detail panel, and bulk actions.
@LAYER Page
@RELATION DEPENDS_ON -> [StatsBar]
@RELATION DEPENDS_ON -> [DatasetList]
@RELATION DEPENDS_ON -> [DatasetPreview]
@RELATION BINDS_TO -> [environmentContextStore]
@RELATION BINDS_TO -> [taskDrawerStore]
@PRE: Environment context is initialized before dataset loading.
@POST: Page renders split-view layout; dataset detail updates without full page reload.
@SIDE_EFFECT: Fetches datasets from API; subscribes to WebSocket dataset.updated events; opens modals for bulk actions.
@INVARIANT: Always shows datasets for the active environment from context store.
@DATA_CONTRACT: Receives DatasetsResponse (list + stats) and DatasetDetailResponse (detail panel). Sends ColumnDescriptionUpdate/MetricDescriptionUpdate on save.
@RATIONALE Split-view replaces separate list + detail pages to reduce navigation clicks and make the workspace feel integrated.
@REJECTED Multi-page navigation (list → detail separate routes) — rejected because it forces full page reload between selecting datasets, breaking flow.
File: frontend/src/routes/datasets/+page.svelte
Complexity Summary
| Contract | Type | C | File |
|---|---|---|---|
| MetricItem | DataClass | 1 | datasets.py |
| StatsCounts | DataClass | 1 | datasets.py |
| ColumnDescriptionUpdate | DataClass | 1 | datasets.py |
| MetricDescriptionUpdate | DataClass | 1 | datasets.py |
| DatasetDetailResponse (mod) | DataClass | 2 | datasets.py |
| DatasetsResponse (mod) | DataClass | 2 | datasets.py |
| get_dataset_detail (mod) | Endpoint | 3 | datasets.py |
| SupersetClient.get_dataset_detail (mod) | Function | 3 | _datasets.py |
| StatsBar | Component | 3 | StatsBar.svelte |
| DatasetPreview | Component | 3 | DatasetPreview.svelte |
| update_column_description | Endpoint | 4 | datasets.py |
| update_metric_description | Endpoint | 4 | datasets.py |
| get_datasets (mod) | Endpoint | 4 | datasets.py |
| DatasetList | Component | 4 | DatasetList.svelte |
| ColumnsTable | Component | 4 | ColumnsTable.svelte |
| MetricsTable | Component | 4 | MetricsTable.svelte |
| DatasetHub (rewrite) | Page | 5 | +page.svelte |
ADR Continuity
| ADR | Status | This Feature |
|---|---|---|
| ADR-0001 (module layout) | ACTIVE | ✅ Follows backend/src/api/routes/, frontend/src/routes/ placement |
| ADR-0002 (semantic protocol) | ACTIVE | ✅ All contracts use GRACE anchors with C1-C5 complexity |
| ADR-0003 (orchestrator pattern) | ACTIVE | ✅ Backend proxies Superset API, no direct frontend-to-Superset calls |
| ADR-0004 (plugin architecture) | ACTIVE | ✅ Mapping/docs tasks use existing plugin system (no new plugin) |
| ADR-0005 (auth RBAC) | ACTIVE | ✅ New mutation endpoints gated by plugin:migration:WRITE; read endpoints use plugin:migration:READ |
| ADR-0006 (frontend architecture) | ACTIVE | ✅ Svelte 5 runes, Tailwind, static SPA, vitest |
| ADR-0007 (rejected fromStore) | ACTIVE | ✅ Uses $state, avoids deprecated patterns |
Rejected Paths
| Path | Reason | Guardrail |
|---|---|---|
| PATCH endpoint on Superset directly | Superset v1 API has no PATCH for columns/metrics | override_columns=false with full-object PUT only |
| Direct frontend-to-Superset PUT | Violates orchestrator pattern (ADR-0003), exposes auth | All mutations go through ss-tools backend |
| Modal-per-row editing | Too many clicks for repetitive column description workflow | Inline edit via textarea toggle |
| Multi-page list→detail navigation | Full page reload between dataset selections breaks flow | Split-view within single SvelteKit route |
| Polling for Stats Bar refresh | Unnecessary network overhead, user chose WebSocket | WebSocket dataset.updated event |
| Client-side filtering for Stats Bar | Incompatible with server-side pagination — can only filter current page | Server-side ?filter= query parameter |
override_columns=true |
Risk of deleting unmapped columns on PUT | Always override_columns=false (FR-023) |