# Research: Dataset Lifecycle Workspace (030) ## R1 — Backend: Metrics in DatasetDetailResponse **Decision:** Add `metrics: list[MetricItem]` and `metric_count: int` to `DatasetDetailResponse`. Add `MetricItem` Pydantic DTO in `backend/src/api/routes/datasets.py`. In `SupersetClient.get_dataset_detail` (`backend/src/core/superset_client/_datasets.py`), extract `dataset.get("metrics", [])` from the Superset API response. **Rationale:** Superset's `GET /api/v1/dataset/{pk}` already returns `metrics` array with `metric_name`, `expression`, `verbose_name`, `description`, `metric_type`. Our backend currently ignores them. Extracting them is straightforward — no breaking API change, only additive fields. **Alternatives Considered:** - A separate `/datasets/{id}/metrics` endpoint → rejected, adds unnecessary API surface for data already present - Client-side fetch from Superset directly → rejected (bypasses auth, breaks SSR parity) **Impact:** `DatasetDetailResponse` gains 2 new fields. `MetricItem` DTO ~8 lines. `get_dataset_detail` gains ~15 lines. Zero DB changes. No migration. --- ## R2 — Backend: Inline-edit Endpoints **Decision:** Two new endpoints on ss-tools backend under existing `APIRouter(prefix="/api/datasets")`: - `PUT /api/datasets/{id}/columns/{col_id}/description` (body: `{"description": "text"}`) - `PUT /api/datasets/{id}/metrics/{metric_id}/description` (body: `{"description": "text"}`) Each performs: GET full dataset from Superset via `SupersetClient.get_dataset()` → modify target column/metric description → PUT back via `SupersetClient.update_dataset()` with `?override_columns=false`. **Rationale:** Superset has no PATCH for individual columns/metrics — only PUT on the full dataset with `override_columns=false`. A convenience endpoint on ss-tools handles the GET→modify→PUT dance server-side, keeping frontend payloads simple (`{description: "text"}`). **Alternatives Considered:** - Frontend does GET→modify→PUT directly → rejected: requires frontend to handle full Superset payload, increases attack surface - Single `PUT /api/datasets/{id}` with `{"columns":[{"id":...,"description":...}]}` → valid but more complex for frontend, reserved for future batch updates - Optimistic save with rollback → rejected: adds complexity without significant UX benefit for description editing **Impact:** +2 route handlers in `datasets.py` (~50 lines each). Reuses existing `SupersetClient.get_dataset()` and `SupersetClient.update_dataset()`. RBAC: `has_permission("plugin:migration","WRITE")` for mutations. --- ## R3 — Backend: Stats Counts + Server-Side Filtering in GET /api/datasets **Decision:** `GET /api/datasets` accepts optional query parameter `?filter=unmapped|mapped|linked|all`. When `filter` is present, the backend filters the full dataset list before pagination. Response always includes `stats` object with `total`, `unmapped_count`, `mapped_count`, `linked_count` — computed from the full (non-paginated, non-filtered) dataset list. **Rationale:** Server-side filtering ensures that all matching datasets are returned, not just those on the current page. Stats represent global state independently of the active filter. Previously considered client-side filtering was incompatible with server-side pagination (a client can only filter the current page, causing mismatch between Stats Bar counts and visible cards). **Alternatives Considered:** - Separate `/api/datasets/stats` endpoint → rejected, adds unnecessary request - Client-side counts (fetch all, count locally) → rejected, pagination would break counts **Impact:** `DatasetsResponse` DTO gains `stats: dict`. One additional aggregation loop in the route handler (~5 lines). No new DB queries. --- ## R4 — Backend: WebSocket Event `dataset.updated` **Decision:** After a mapping or documentation task completes, the task manager emits a WebSocket event `dataset.updated` with payload `{env_id, dataset_ids: []}`. The frontend subscribes to this event channel and automatically re-fetches `GET /api/datasets`. **Rationale:** Existing WebSocket infrastructure already streams task logs to the task drawer. Extending with a completion event avoids polling overhead. The `task_manager` already has an `on_task_complete` hook. **Alternatives Considered:** - Polling every N seconds → rejected (unnecessary network traffic, 33 datasets trivial but doesn't scale) - Manual refresh only → rejected by user in clarification Q5 **Impact:** One additional WebSocket message type in task completion handler (~5 lines). Frontend subscription in `+page.svelte` (~10 lines). No new infrastructure. --- ## R5 — Frontend: Component Architecture **Decision:** Replace current monolithic 974-line `+page.svelte` with composition of 5 leaf components: ``` +page.svelte (layout orchestrator, ~200 lines) ├── StatsBar.svelte (4 metric tiles, emits filter → parent calls API) ├── DatasetList.svelte (card list with pagination, checkbox selection) ├── DatasetPreview.svelte (presentational right panel: receives dataset/loading/error props; parent fetches detail API) │ ├── ColumnsTable.svelte (column rows with inline-edit ✏️) │ └── MetricsTable.svelte (metric rows with inline-edit ✏️) └── BulkActionPanel.svelte (reused from current implementation) ``` Each component follows ss-tools Svelte 5 conventions (runes `$state`, `$derived`, `$props`, Tailwind classes). **Rationale:** SS-tools fractal limit (INV_7): modules <400 lines, contract node cyclomatic complexity <10. The current 974-line page violates both. Decomposition into <400-line components is required by ADR-0002. **Alternatives Considered:** - Keep monolithic + extract inline-edit as helper → rejected (still violates INV_7) - Use SvelteKit layout groups for sub-panels → rejected (over-engineering; this is one page, not multi-route) **Impact:** ~5 new `.svelte` files in `frontend/src/routes/datasets/`. Existing `[id]/+page.svelte` remains as fallback but is largely superseded by `DatasetPreview.svelte`. --- ## R6 — Frontend: Inline Edit Interaction **Decision:** On ✏️ click, the column/metric row enters `isEditing` state. A `