# Research: Dataset Lifecycle Workspace (030)

## R1 — Backend: Metrics in DatasetDetailResponse

**Decision:** Add `metrics: list[MetricItem]` and `metric_count: int` to `DatasetDetailResponse`. Add `MetricItem` Pydantic DTO in `backend/src/api/routes/datasets.py`. In `SupersetClient.get_dataset_detail` (`backend/src/core/superset_client/_datasets.py`), extract `dataset.get("metrics", [])` from the Superset API response.

**Rationale:** Superset's `GET /api/v1/dataset/{pk}` already returns `metrics` array with `metric_name`, `expression`, `verbose_name`, `description`, `metric_type`. Our backend currently ignores them. Extracting them is straightforward — no breaking API change, only additive fields.

**Alternatives Considered:**
- A separate `/datasets/{id}/metrics` endpoint → rejected, adds unnecessary API surface for data already present
- Client-side fetch from Superset directly → rejected (bypasses auth, breaks SSR parity)

**Impact:** `DatasetDetailResponse` gains 2 new fields. `MetricItem` DTO ~8 lines. `get_dataset_detail` gains ~15 lines. Zero DB changes. No migration.

---

## R2 — Backend: Inline-edit Endpoints

**Decision:** Two new endpoints on ss-tools backend under existing `APIRouter(prefix="/api/datasets")`:
- `PUT /api/datasets/{id}/columns/{col_id}/description` (body: `{"description": "text"}`)
- `PUT /api/datasets/{id}/metrics/{metric_id}/description` (body: `{"description": "text"}`)

Each performs: GET full dataset from Superset via `SupersetClient.get_dataset()` → modify target column/metric description → PUT back via `SupersetClient.update_dataset()` with `?override_columns=false`.

**Rationale:** Superset has no PATCH for individual columns/metrics — only PUT on the full dataset with `override_columns=false`. A convenience endpoint on ss-tools handles the GET→modify→PUT dance server-side, keeping frontend payloads simple (`{description: "text"}`).

**Alternatives Considered:**
- Frontend does GET→modify→PUT directly → rejected: requires frontend to handle full Superset payload, increases attack surface
- Single `PUT /api/datasets/{id}` with `{"columns":[{"id":...,"description":...}]}` → valid but more complex for frontend, reserved for future batch updates
- Optimistic save with rollback → rejected: adds complexity without significant UX benefit for description editing

**Impact:** +2 route handlers in `datasets.py` (~50 lines each). Reuses existing `SupersetClient.get_dataset()` and `SupersetClient.update_dataset()`. RBAC: `has_permission("plugin:migration","WRITE")` for mutations.

---

## R3 — Backend: Stats Counts + Server-Side Filtering in GET /api/datasets

**Decision:** `GET /api/datasets` accepts optional query parameter `?filter=unmapped|mapped|linked|all`. When `filter` is present, the backend filters the full dataset list before pagination. Response always includes `stats` object with `total`, `unmapped_count`, `mapped_count`, `linked_count` — computed from the full (non-paginated, non-filtered) dataset list.

**Rationale:** Server-side filtering ensures that all matching datasets are returned, not just those on the current page. Stats represent global state independently of the active filter. Previously considered client-side filtering was incompatible with server-side pagination (a client can only filter the current page, causing mismatch between Stats Bar counts and visible cards).

**Alternatives Considered:**
- Separate `/api/datasets/stats` endpoint → rejected, adds unnecessary request
- Client-side counts (fetch all, count locally) → rejected, pagination would break counts

**Impact:** `DatasetsResponse` DTO gains `stats: dict`. One additional aggregation loop in the route handler (~5 lines). No new DB queries.

---

## R4 — Backend: WebSocket Event `dataset.updated`

**Decision:** After a mapping or documentation task completes, the task manager emits a WebSocket event `dataset.updated` with payload `{env_id, dataset_ids: []}`. The frontend subscribes to this event channel and automatically re-fetches `GET /api/datasets`.

**Rationale:** Existing WebSocket infrastructure already streams task logs to the task drawer. Extending with a completion event avoids polling overhead. The `task_manager` already has an `on_task_complete` hook.

**Alternatives Considered:**
- Polling every N seconds → rejected (unnecessary network traffic, 33 datasets trivial but doesn't scale)
- Manual refresh only → rejected by user in clarification Q5

**Impact:** One additional WebSocket message type in task completion handler (~5 lines). Frontend subscription in `+page.svelte` (~10 lines). No new infrastructure.

---

## R5 — Frontend: Component Architecture

**Decision:** Replace current monolithic 974-line `+page.svelte` with composition of 5 leaf components:

```
+page.svelte (layout orchestrator, ~200 lines)
├── StatsBar.svelte        (4 metric tiles, emits filter → parent calls API)
├── DatasetList.svelte     (card list with pagination, checkbox selection)
├── DatasetPreview.svelte  (presentational right panel: receives dataset/loading/error props; parent fetches detail API)
│   ├── ColumnsTable.svelte    (column rows with inline-edit ✏️)
│   └── MetricsTable.svelte    (metric rows with inline-edit ✏️)
└── BulkActionPanel.svelte (reused from current implementation)
```

Each component follows ss-tools Svelte 5 conventions (runes `$state`, `$derived`, `$props`, Tailwind classes).

**Rationale:** SS-tools fractal limit (INV_7): modules <400 lines, contract node cyclomatic complexity <10. The current 974-line page violates both. Decomposition into <400-line components is required by ADR-0002.

**Alternatives Considered:**
- Keep monolithic + extract inline-edit as helper → rejected (still violates INV_7)
- Use SvelteKit layout groups for sub-panels → rejected (over-engineering; this is one page, not multi-route)

**Impact:** ~5 new `.svelte` files in `frontend/src/routes/datasets/`. Existing `[id]/+page.svelte` remains as fallback but is largely superseded by `DatasetPreview.svelte`.

---

## R6 — Frontend: Inline Edit Interaction

**Decision:** On ✏️ click, the column/metric row enters `isEditing` state. A `<textarea>` replaces the description text. Save button calls `PUT /api/datasets/{id}/columns/{col_id}/description`. On success, row exits editing state and shows new description. On error, textarea stays open with error message. Cancel reverts to original value.

**Rationale:** Matches standard inline-edit UX pattern. Svelte 5 `$state` makes `isEditing` toggle trivial. Optimistic save rejected for simplicity — descriptions are short text, save latency is negligible.

**Alternatives Considered:**
- Modal with single textarea → rejected (too heavy for one field)
- Click-to-edit with blur-to-save → rejected (ambiguous "cancel" path)

**Impact:** `ColumnsTable.svelte` and `MetricsTable.svelte` each have `isEditingRowId` state + textarea logic. ~30 lines of template logic per table.

---

## R7 — Frontend: WebSocket Subscription

**Decision:** `+page.svelte` subscribes to `dataset.updated` WebSocket events in `onMount`. On receiving event with matching `env_id`, calls `loadDatasets()` to refresh the list and stats. No visual feedback needed — list updates in-place (Svelte reactivity).

**Rationale:** Pushes state updates without user action. Existing WebSocket connection is reused (no new connection). Svelte reactivity handles re-render.

**Alternatives Considered:**
- SSE / EventSource → rejected (WebSocket already connected for task logs)
- Separate WebSocket channel → rejected (overhead)

**Impact:** ~10 lines in `+page.svelte`. Reuses `wsStore` or equivalent from existing WebSocket infrastructure.

---

## R8 — i18n: New Keys

**Decision:** Add 11 new keys to `frontend/src/lib/i18n/locales/{en,ru}/datasets.json`: `all`, `without_mapping`, `with_mapping`, `linked_to_dashboards`, `metrics`, `metric_count`, `no_selection`, `save_description`, `add_description`, `columns`, `detail_empty`. No structural changes to i18n system.

**Rationale:** Existing i18n uses flat JSON key files loaded by locale. Adding keys is additive. No new dependencies.

**Impact:** +11 lines per locale file. Components reference via `$t.datasets.all`, etc.

---

## R9 — Backend: `override_columns=false` Guarantee

**Decision:** In line-edit endpoint handlers, when calling `SupersetClient.update_dataset()`, always append `?override_columns=false` to the PUT URL. This ensures Superset preserves all columns/metrics not mentioned in the payload.

**Rationale:** Superset's default is `override_columns=false` (safe), but explicit parameter ensures no accidental override if default changes. Risk: with `override_columns=true`, Superset would DELETE any column not present in the PUT payload — catastrophic.

**Alternatives Considered:**
- Rely on Superset default → rejected (fragile, no explicit guarantee)

**Impact:** One query parameter addition per PUT call.

---

## R10 — Module Placement & File Map

| Artifact | File Path | Operation |
|----------|-----------|-----------|
| MetricItem DTO | `backend/src/api/routes/datasets.py` | ADD class |
| Stats counts | `backend/src/api/routes/datasets.py` | MODIFY `DatasetsResponse`, `get_datasets()` |
| Column description endpoint | `backend/src/api/routes/datasets.py` | ADD route `PUT /columns/{col_id}/description` |
| Metric description endpoint | `backend/src/api/routes/datasets.py` | ADD route `PUT /metrics/{metric_id}/description` |
| Metrics extraction | `backend/src/core/superset_client/_datasets.py` | MODIFY `get_dataset_detail()` |
| WebSocket event | `backend/src/core/task_manager.py` (or equivalent) | MODIFY on_task_complete |
| StatsBar | `frontend/src/routes/datasets/StatsBar.svelte` | NEW |
| DatasetList | `frontend/src/routes/datasets/DatasetList.svelte` | NEW |
| DatasetPreview | `frontend/src/routes/datasets/DatasetPreview.svelte` | NEW |
| ColumnsTable | `frontend/src/routes/datasets/ColumnsTable.svelte` | NEW |
| MetricsTable | `frontend/src/routes/datasets/MetricsTable.svelte` | NEW |
| Page orchestrator | `frontend/src/routes/datasets/+page.svelte` | REWRITE |
| i18n EN | `frontend/src/lib/i18n/locales/en/datasets.json` | ADD keys |
| i18n RU | `frontend/src/lib/i18n/locales/ru/datasets.json` | ADD keys |
| Breadcrumb | `frontend/src/lib/components/layout/Breadcrumbs.svelte` | MODIFY |
| Sidebar nav | `frontend/src/lib/components/layout/sidebarNavigation.js` | VERIFY |

---

## Unresolved / Deferred

- **~~DatasetItem.metric_count in list view~~**: RESOLVED — `metric_count` added to `DatasetItem` DTO as required field. Backend must include metric count in list response (fetched from Superset dataset metadata during enrichment).
- **Accessibility (ARIA)**: Deferred to Polish phase. Components should use semantic HTML and `aria-busy`/`aria-live` where applicable. Minimum a11y requirements added to spec: Stats tiles as buttons with `aria-pressed`, slide-over traps focus and closes on Escape, inline edit errors announced via `aria-live`, skeleton loaders use `aria-busy`.
- **Large-scale (200+ datasets)**: Server-side filtering (`?filter=`) adopted for current scope. At 200+, pagination works normally — no client-side filtering needed.
- **Performance**: Detail panel target <500ms applies to render after API response. Loading state must appear within 100ms. Optional cache for previously opened dataset details deferred to implementation.
