busya
ec6421de35
- Replace all occurrences of 'ss-tools' with 'superset-tools' in 104 files - Rename git bundle file ss-tools.bundle → superset-tools.bundle - Update .gitignore pattern accordingly - Preserve variable names (hasSsTools etc.) and code identifiers
265 lines
15 KiB
Markdown
265 lines
15 KiB
Markdown
# Contracts: Dataset Lifecycle Workspace
|
|
|
|
> Generated for speckit feature 030. All IDs scoped to this feature. Complexity per GRACE-Poly v2.6. Allowed predicates: DEPENDS_ON, CALLS, INHERITS, IMPLEMENTS, DISPATCHES, BINDS_TO.
|
|
|
|
---
|
|
|
|
## Backend Contracts
|
|
|
|
### #region MetricItem [C:1] [TYPE DataClass]
|
|
@BRIEF Pydantic DTO for a dataset metric — carries Superset metric metadata.
|
|
@RELATION DEPENDS_ON -> [DatasetDetailResponse]
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
|
|
```python
|
|
class MetricItem(BaseModel):
|
|
id: int
|
|
metric_name: str
|
|
expression: str | None = None
|
|
verbose_name: str | None = None
|
|
description: str | None = None
|
|
metric_type: str | None = None
|
|
```
|
|
|
|
### #region StatsCounts [C:1] [TYPE DataClass]
|
|
@BRIEF Aggregate statistics for the Stats Bar — computed from full dataset list.
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
|
|
```python
|
|
class StatsCounts(BaseModel):
|
|
total: int
|
|
unmapped_count: int
|
|
mapped_count: int
|
|
linked_count: int
|
|
```
|
|
|
|
### #region ColumnDescriptionUpdate [C:1] [TYPE DataClass]
|
|
@BRIEF Request DTO for inline-edit of a column description.
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
|
|
```python
|
|
class ColumnDescriptionUpdate(BaseModel):
|
|
description: str
|
|
```
|
|
|
|
### #region MetricDescriptionUpdate [C:1] [TYPE DataClass]
|
|
@BRIEF Request DTO for inline-edit of a metric description.
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
|
|
```python
|
|
class MetricDescriptionUpdate(BaseModel):
|
|
description: str
|
|
```
|
|
|
|
### #region DatasetDetailResponse [C:2] [TYPE DataClass] — MODIFIED
|
|
@BRIEF Detailed DTO for a dataset including columns, linked dashboards, and metrics.
|
|
@RELATION DEPENDS_ON -> [MetricItem]
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
**Modification:** Add `metrics: list[MetricItem]` and `metric_count: int`.
|
|
|
|
### #region DatasetsResponse [C:2] [TYPE DataClass] — MODIFIED
|
|
@BRIEF Paginated response DTO for dataset listings with stats.
|
|
@RELATION DEPENDS_ON -> [StatsCounts]
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
**Modification:** Add `stats: StatsCounts`.
|
|
|
|
### #region update_column_description [C:4] [TYPE Endpoint]
|
|
@BRIEF Save description for a single dataset column. Internal: GET full dataset from Superset, modify one column's description, PUT back.
|
|
@PRE: dataset_id and column_id must exist in the target environment.
|
|
@POST: Column description in Superset is updated. Response confirms success.
|
|
@SIDE_EFFECT: Mutates dataset metadata in upstream Superset instance via PUT.
|
|
@VALIDATION: description — string, max 2000 chars, plain text only (HTML stripped), may be empty string to clear description, no trimming applied (preserve leading/trailing whitespace). 404 if dataset or column not found. 502/503 if Superset upstream fails.
|
|
@ERROR: 400 — description exceeds max length or contains non-plaintext content. 404 — dataset_id or column_id not found. 502 — Superset GET failed. 503 — Superset PUT failed.
|
|
@RELATION CALLS -> [SupersetClient.get_dataset]
|
|
@RELATION CALLS -> [SupersetClient.update_dataset]
|
|
@RATIONALE Must perform GET→modify→PUT because Superset has no PATCH for individual columns — only full object PUT with `override_columns=false`.
|
|
@REJECTED Direct PUT from frontend — rejected because frontend would need to handle full Superset payload structure.
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
**Route:** `PUT /api/datasets/{dataset_id}/columns/{column_id}/description`
|
|
**Method:** `PUT`
|
|
**Request:** `ColumnDescriptionUpdate`
|
|
**Response:** `{success: true, description: "new text"}`
|
|
**RBAC:** `has_permission("plugin:migration", "WRITE")`
|
|
|
|
### #region update_metric_description [C:4] [TYPE Endpoint]
|
|
@BRIEF Save description for a single dataset metric. Mirror of update_column_description for metrics.
|
|
@PRE: dataset_id and metric_id must exist in the target environment.
|
|
@POST: Metric description in Superset is updated.
|
|
@SIDE_EFFECT: Mutates dataset metadata in upstream Superset instance via PUT.
|
|
@VALIDATION: description — string, max 2000 chars, plain text only (HTML stripped), may be empty string to clear description, no trimming applied. 404 if dataset or metric not found. 502/503 if Superset upstream fails.
|
|
@ERROR: 400 — description exceeds max length or contains non-plaintext content. 404 — dataset_id or metric_id not found. 502 — Superset GET failed. 503 — Superset PUT failed.
|
|
@RELATION CALLS -> [SupersetClient.get_dataset]
|
|
@RELATION CALLS -> [SupersetClient.update_dataset]
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
**Route:** `PUT /api/datasets/{dataset_id}/metrics/{metric_id}/description`
|
|
**Request:** `MetricDescriptionUpdate`
|
|
**Response:** `{success: true, description: "new text"}`
|
|
**RBAC:** `has_permission("plugin:migration", "WRITE")`
|
|
|
|
### #region get_datasets [C:4] [TYPE Endpoint] — MODIFIED
|
|
@BRIEF Fetch paginated dataset list with enhanced metadata, mapped fields, and stats counts.
|
|
@PRE: env_id must be valid. page >= 1, page_size in [1,100].
|
|
@POST: Returns paginated datasets plus StatsCounts for the Stats Bar.
|
|
@SIDE_EFFECT: Computes aggregate counts from full dataset list (in-memory, no DB write).
|
|
@RELATION CALLS -> [resource_service.get_datasets_with_status]
|
|
@RATIONALE Stats counts returned in the same response as datasets to avoid an extra API call for the Stats Bar (FR-022).
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
**Modification:** Compute `stats` dict from full dataset list before pagination slicing, add to DatasetsResponse.
|
|
|
|
### #region get_dataset_detail [C:3] [TYPE Endpoint] — MODIFIED
|
|
@BRIEF Get detailed dataset information including columns, linked dashboards, and metrics.
|
|
@RELATION CALLS -> [SupersetClient.get_dataset_detail]
|
|
**File:** `backend/src/api/routes/datasets.py`
|
|
**Modification:** Extract `dataset.get("metrics", [])` from Superset response, map to `MetricItem` list.
|
|
|
|
### #region SupersetClient.get_dataset_detail [C:3] [TYPE Function] — MODIFIED
|
|
@BRIEF Extract dataset detail from Superset API including metrics.
|
|
@RELATION CALLS -> [SupersetClient.get_dataset]
|
|
**File:** `backend/src/core/superset_client/_datasets.py`
|
|
**Modification:** Add metrics extraction loop after columns loop. Add `metrics` and `metric_count` to result dict.
|
|
|
|
---
|
|
|
|
## Frontend Contracts
|
|
|
|
### #region StatsBar [C:3] [TYPE Component] [SEMANTICS ui,dataset,stats,filter]
|
|
@BRIEF Displays 4 aggregate metric tiles (All, Without mapping, Mapped, Linked) and emits filter selection. Filtering is server-side via query param — parent container calls API on tile click.
|
|
@LAYER UI
|
|
@RELATION BINDS_TO -> [environmentContextStore]
|
|
@UX_STATE Loading -> 4 skeleton tiles pulsing.
|
|
@UX_STATE Loaded -> 4 tiles with numbers, active filter highlighted.
|
|
@UX_STATE Filtered -> Selected tile has accent border; parent triggers API reload with filter param.
|
|
@UX_STATE Empty -> All tiles show 0.
|
|
@UX_REACTIVITY Props -> `stats: StatsCounts`, `activeFilter: string|null`.
|
|
@UX_REACTIVITY Events -> `onfilter(filterKey)` dispatched on tile click; parent calls `loadDatasets(filter=filterKey)`.
|
|
**File:** `frontend/src/routes/datasets/StatsBar.svelte`
|
|
|
|
### #region DatasetList [C:4] [TYPE Component] [SEMANTICS ui,dataset,card,list,pagination]
|
|
@BRIEF Renders paginated dataset cards with mapping progress bars, checkboxes, and quick actions.
|
|
@LAYER UI
|
|
@RELATION BINDS_TO -> [environmentContextStore]
|
|
@RELATION BINDS_TO -> [taskDrawerStore]
|
|
@UX_STATE Loading -> Skeleton cards.
|
|
@UX_STATE Loaded -> Cards with progress bars.
|
|
@UX_STATE Selecting -> Checkboxes visible, checked cards highlighted.
|
|
@UX_STATE Empty -> "No datasets found" after search/filter.
|
|
@UX_FEEDBACK Click card → dispatches dataset selection for detail panel.
|
|
@UX_FEEDBACK Click checkbox → updates selection set.
|
|
@UX_FEEDBACK Click quick action → opens Map Columns or Generate Docs modal.
|
|
@UX_REACTIVITY Props -> `datasets: DatasetCard[]`, `selectedIds: Set`, `activeFilter: string|null`.
|
|
@UX_REACTIVITY Events -> `onselect(datasetId)`, `onaction(dataset, action)`.
|
|
**File:** `frontend/src/routes/datasets/DatasetList.svelte`
|
|
|
|
### #region DatasetPreview [C:3] [TYPE Component] [SEMANTICS ui,dataset,detail,columns,metrics]
|
|
@BRIEF Right panel showing dataset detail: header info, columns table, metrics table, linked dashboards. Presentational component — receives dataset data from parent container.
|
|
@LAYER UI
|
|
@RELATION DEPENDS_ON -> [ColumnsTable]
|
|
@RELATION DEPENDS_ON -> [MetricsTable]
|
|
@UX_STATE NoSelection -> "Select a dataset from the list" placeholder.
|
|
@UX_STATE Loading -> Skeleton for header + 2 tables.
|
|
@UX_STATE Loaded -> Full detail with columns and metrics rendered.
|
|
@UX_STATE Error -> Error banner with retry (retry triggers callback to parent).
|
|
@UX_REACTIVITY Props -> `dataset: DatasetDetailResponse | null`, `isLoading: boolean`, `error: string | null`.
|
|
@UX_REACTIVITY Events -> `onretry()` callback dispatched when retry button clicked.
|
|
@RATIONALE Container (DatasetHub) fetches data and passes it as props; DatasetPreview is a presentational component for cleaner testing and separation of concerns.
|
|
@REJECTED Self-fetching component — rejected because it couples data fetching with rendering, harder to test, and creates race conditions when switching datasets rapidly.
|
|
@UX_TEST: NoSelection -> {click: card, expected: Loading -> Loaded}.
|
|
**File:** `frontend/src/routes/datasets/DatasetPreview.svelte`
|
|
|
|
### #region ColumnsTable [C:4] [TYPE Component] [SEMANTICS ui,dataset,columns,inline-edit]
|
|
@BRIEF Table of dataset columns with type chips, description, and inline-edit capability.
|
|
@LAYER UI
|
|
@RELATION CALLS -> [api_module]
|
|
@UX_STATE Display -> All rows in read mode.
|
|
@UX_STATE Editing -> One row in edit mode (textarea), others dimmed.
|
|
@UX_STATE Saving -> Spinner on save button.
|
|
@UX_STATE Error -> Red border on textarea + error message.
|
|
@UX_FEEDBACK verbose_name → shown bold; missing → column_name in italic as fallback.
|
|
@UX_FEEDBACK No description → "✏️ Add description" action prompt.
|
|
@UX_REACTIVITY Props -> `columns: ColumnRow[]`, `datasetId: number`.
|
|
@UX_REACTIVITY LocalState -> `isEditingRowId: number|null`.
|
|
@RATIONALE Inline-edit uses optimised state per row to minimise re-renders across the table.
|
|
@REJECTED Modal-per-column editing — rejected as too many clicks for repetitive workflow.
|
|
**File:** `frontend/src/routes/datasets/ColumnsTable.svelte`
|
|
|
|
### #region MetricsTable [C:4] [TYPE Component] [SEMANTICS ui,dataset,metrics,inline-edit]
|
|
@BRIEF Table of dataset metrics with expression, description, and inline-edit capability.
|
|
@LAYER UI
|
|
@RELATION CALLS -> [api_module]
|
|
@UX_STATE Display -> All rows in read mode with expression shown under name.
|
|
@UX_STATE Editing -> One row in edit mode.
|
|
@UX_STATE Saving -> Spinner on save button.
|
|
@UX_FEEDBACK verbose_name → shown bold; missing → metric_name in italic as fallback.
|
|
@UX_FEEDBACK No description → shows expression as hint + "✏️ Add description" prompt.
|
|
@UX_REACTIVITY Props -> `metrics: MetricRow[]`, `datasetId: number`.
|
|
**File:** `frontend/src/routes/datasets/MetricsTable.svelte`
|
|
|
|
### #region DatasetHub [C:5] [TYPE Page] — REWRITTEN
|
|
@BRIEF Dataset lifecycle workspace page — split-view with Stats Bar, card list, detail panel, and bulk actions.
|
|
@LAYER Page
|
|
@RELATION DEPENDS_ON -> [StatsBar]
|
|
@RELATION DEPENDS_ON -> [DatasetList]
|
|
@RELATION DEPENDS_ON -> [DatasetPreview]
|
|
@RELATION BINDS_TO -> [environmentContextStore]
|
|
@RELATION BINDS_TO -> [taskDrawerStore]
|
|
@PRE: Environment context is initialized before dataset loading.
|
|
@POST: Page renders split-view layout; dataset detail updates without full page reload.
|
|
@SIDE_EFFECT: Fetches datasets from API; subscribes to WebSocket `dataset.updated` events; opens modals for bulk actions.
|
|
@INVARIANT: Always shows datasets for the active environment from context store.
|
|
@DATA_CONTRACT: Receives DatasetsResponse (list + stats) and DatasetDetailResponse (detail panel). Sends ColumnDescriptionUpdate/MetricDescriptionUpdate on save.
|
|
@RATIONALE Split-view replaces separate list + detail pages to reduce navigation clicks and make the workspace feel integrated.
|
|
@REJECTED Multi-page navigation (list → detail separate routes) — rejected because it forces full page reload between selecting datasets, breaking flow.
|
|
**File:** `frontend/src/routes/datasets/+page.svelte`
|
|
|
|
---
|
|
|
|
## Complexity Summary
|
|
|
|
| Contract | Type | C | File |
|
|
|----------|------|---|------|
|
|
| MetricItem | DataClass | 1 | datasets.py |
|
|
| StatsCounts | DataClass | 1 | datasets.py |
|
|
| ColumnDescriptionUpdate | DataClass | 1 | datasets.py |
|
|
| MetricDescriptionUpdate | DataClass | 1 | datasets.py |
|
|
| DatasetDetailResponse (mod) | DataClass | 2 | datasets.py |
|
|
| DatasetsResponse (mod) | DataClass | 2 | datasets.py |
|
|
| get_dataset_detail (mod) | Endpoint | 3 | datasets.py |
|
|
| SupersetClient.get_dataset_detail (mod) | Function | 3 | _datasets.py |
|
|
| StatsBar | Component | 3 | StatsBar.svelte |
|
|
| DatasetPreview | Component | 3 | DatasetPreview.svelte |
|
|
| update_column_description | Endpoint | 4 | datasets.py |
|
|
| update_metric_description | Endpoint | 4 | datasets.py |
|
|
| get_datasets (mod) | Endpoint | 4 | datasets.py |
|
|
| DatasetList | Component | 4 | DatasetList.svelte |
|
|
| ColumnsTable | Component | 4 | ColumnsTable.svelte |
|
|
| MetricsTable | Component | 4 | MetricsTable.svelte |
|
|
| DatasetHub (rewrite) | Page | 5 | +page.svelte |
|
|
|
|
---
|
|
|
|
## ADR Continuity
|
|
|
|
| ADR | Status | This Feature |
|
|
|-----|--------|-------------|
|
|
| ADR-0001 (module layout) | ACTIVE | ✅ Follows `backend/src/api/routes/`, `frontend/src/routes/` placement |
|
|
| ADR-0002 (semantic protocol) | ACTIVE | ✅ All contracts use GRACE anchors with C1-C5 complexity |
|
|
| ADR-0003 (orchestrator pattern) | ACTIVE | ✅ Backend proxies Superset API, no direct frontend-to-Superset calls |
|
|
| ADR-0004 (plugin architecture) | ACTIVE | ✅ Mapping/docs tasks use existing plugin system (no new plugin) |
|
|
| ADR-0005 (auth RBAC) | ACTIVE | ✅ New mutation endpoints gated by `plugin:migration:WRITE`; read endpoints use `plugin:migration:READ` |
|
|
| ADR-0006 (frontend architecture) | ACTIVE | ✅ Svelte 5 runes, Tailwind, static SPA, vitest |
|
|
| ADR-0007 (rejected fromStore) | ACTIVE | ✅ Uses `$state`, avoids deprecated patterns |
|
|
|
|
---
|
|
|
|
## Rejected Paths
|
|
|
|
| Path | Reason | Guardrail |
|
|
|------|--------|-----------|
|
|
| PATCH endpoint on Superset directly | Superset v1 API has no PATCH for columns/metrics | `override_columns=false` with full-object PUT only |
|
|
| Direct frontend-to-Superset PUT | Violates orchestrator pattern (ADR-0003), exposes auth | All mutations go through superset-tools backend |
|
|
| Modal-per-row editing | Too many clicks for repetitive column description workflow | Inline edit via textarea toggle |
|
|
| Multi-page list→detail navigation | Full page reload between dataset selections breaks flow | Split-view within single SvelteKit route |
|
|
| Polling for Stats Bar refresh | Unnecessary network overhead, user chose WebSocket | WebSocket `dataset.updated` event |
|
|
| Client-side filtering for Stats Bar | Incompatible with server-side pagination — can only filter current page | Server-side `?filter=` query parameter |
|
|
| `override_columns=true` | Risk of deleting unmapped columns on PUT | Always `override_columns=false` (FR-023) |
|