ss-tools/specs/030-dataset-lifecycle-workspace/tasks.md

# Tasks: Dataset Lifecycle Workspace

**Feature**: 030-dataset-lifecycle-workspace
**Branch**: `030-dataset-lifecycle-workspace`
**Spec**: [spec.md](./spec.md) | **Plan**: [plan.md](./plan.md)

---

## Phase 1: Setup (i18n + Breadcrumb)

> No user stories yet — infrastructure that all stories depend on.

- [ ] T001 [P] Add 11 EN i18n keys to `frontend/src/lib/i18n/locales/en/datasets.json`: `all`, `without_mapping`, `with_mapping`, `linked_to_dashboards`, `metrics`, `metric_count`, `no_selection`, `save_description`, `add_description`, `columns`, `detail_empty`
- [ ] T002 [P] Add 11 RU i18n keys to `frontend/src/lib/i18n/locales/ru/datasets.json`: `все`, `без_маппинга`, `с_маппингом`, `связаны_с_дашбордами`, `метрики`, `метрик`, `выберите_датасет`, `сохранить_описание`, `добавить_описание`, `колонки`, `пустой_просмотр`
- [ ] T003 Verify breadcrumb for `/datasets` shows "Датасеты / Datasets" in `frontend/src/lib/components/layout/Breadcrumbs.svelte` (already fixed, confirm)
- [ ] T004 [P] Verify sidebar navigation entries for datasets in `frontend/src/lib/components/layout/sidebarNavigation.js` are correct

**Verification**: `cd frontend && npm run build` — проверка что билд не падает с новыми i18n ключами.

---

## Phase 2: Foundational — Backend Metrics API (US7)

> Blocks US4 (detail panel needs metrics). Must complete before any UI work with metrics data.

- [ ] T005 Add `MetricItem` Pydantic DTO (fields: `id`, `metric_name`, `expression`, `verbose_name`, `description`, `metric_type`) in `backend/src/api/routes/datasets.py` (RATIONALE: metrics exist in Superset API but are currently ignored; REJECTED: separate metrics endpoint adds unnecessary API surface)
- [ ] T006 Add `metrics: list[MetricItem]` and `metric_count: int` to `DatasetDetailResponse` in `backend/src/api/routes/datasets.py`
- [ ] T007 Extract `dataset.get("metrics", [])` from Superset API response in `SupersetClient.get_dataset_detail()` at `backend/src/core/superset_client/_datasets.py` (add ~15 lines after existing columns loop, map each metric dict to MetricItem-like dict)
- [ ] T007a Add `metric_count: int` to `DatasetItem` DTO in `backend/src/api/routes/datasets.py`. Enrich list items with metric count during `resource_service.get_datasets_with_status()` (extract `len(dataset.get("metrics", []))` from Superset dataset metadata). (RATIONALE: FR-025 — cards must display metric count; REJECTED: lazy-loading metric count per card — too many requests, UX jank)
- [ ] T008 Add `StatsCounts` Pydantic DTO (fields: `total`, `unmapped_count`, `mapped_count`, `linked_count`) in `backend/src/api/routes/datasets.py`
- [ ] T009 Add `stats: StatsCounts` to `DatasetsResponse` in `backend/src/api/routes/datasets.py`
- [ ] T010 Compute `stats` counts from full (non-paginated, non-filtered) dataset list in `get_datasets()` handler in `backend/src/api/routes/datasets.py` — counts: `total=len(datasets)`, `unmapped=sum(mapped==0)`, `mapped=sum(mapped>0)`, `linked=sum(linked>0)`. Add server-side filtering: accept `?filter=unmapped|mapped|linked|all`, filter full list before pagination slice. (RATIONALE: one API call for both list + stats per FR-022; server-side filtering ensures all matching datasets returned; REJECTED: separate /stats endpoint adds unnecessary request, client-side filtering incompatible with pagination)
- [ ] T011 Add inline-edit endpoints to `backend/src/api/routes/datasets.py`: `PUT /api/datasets/{dataset_id}/columns/{column_id}/description` (body: `ColumnDescriptionUpdate`, handler: GET full dataset → modify column → PUT to Superset with `?override_columns=false`) (RATIONALE: Superset lacks PATCH for individual columns; REJECTED: frontend direct PUT — bypasses orchestrator, exposes auth)
- [ ] T012 Add inline-edit endpoint to `backend/src/api/routes/datasets.py`: `PUT /api/datasets/{dataset_id}/metrics/{metric_id}/description` (body: `MetricDescriptionUpdate`, handler: mirror of T011 for metrics) (RATIONALE: same GET→modify→PUT pattern; REJECTED: merging columns+metrics into one endpoint — different Superset payload structures)
- [ ] T013 Add `dataset.updated` WebSocket event emission in task completion handler at `backend/src/core/task_manager.py` — when a mapping or documentation task completes, broadcast `{type: "dataset.updated", payload: {env_id, dataset_ids}}` via existing WebSocket manager (RATIONALE: real-time refresh without polling per FR-024; REJECTED: polling interval — unnecessary network overhead)
- [ ] T014 Write pytest tests for new endpoints in `backend/tests/__tests__/test_datasets.py`: test that `GET /api/datasets/{id}` returns `metrics` and `metric_count`; test that `PUT /columns/{col_id}/description` saves and returns updated description; test that `PUT /metrics/{metric_id}/description` saves correctly; test that `GET /api/datasets` returns `stats` object with correct counts

**Verification**: `cd backend && source .venv/bin/activate && python -m pytest -v backend/tests/__tests__/test_datasets.py`

---

## Phase 3: Stats Bar + Split-View Layout (US1 + US2 — P1)

> Core UI structure. All remaining stories depend on this layout being in place.

- [ ] T015 [US1] Create `StatsBar.svelte` component in `frontend/src/routes/datasets/StatsBar.svelte` — 4 metric tiles (Все, Без маппинга, С маппингом, С дашбордами), each showing count from `stats` prop; active tile has highlight border; dispatches `onfilter` event on click (parent calls API with `?filter=`); toggles filter on second click (dispatches `null` to clear); skeleton tiles when loading. Tiles use `aria-pressed` reflecting active state. (RATIONALE: quick diagnostic overview replaces empty space at top of current table page; REJECTED: dropdown filters — Stats Bar gives 1-click access)
- [ ] T016 [US2] Rewrite `frontend/src/routes/datasets/+page.svelte` as split-view orchestrator: replace monolithic 974-line table with layout shell using Tailwind grid `grid grid-cols-[40%_60%]` on desktop, `block` on mobile (<768px). Remove old table rendering, keep data loading logic, add `selectedDatasetId` state. Shell renders `<StatsBar>`, `<DatasetList>`, `<DatasetPreview>` components. (RATIONALE: decompose 974-line page into composable components per INV_7 fractal limit; REJECTED: keep monolithic page — violates semantic erosion rules at C5 complexity)
- [ ] T017 [US2] Add mobile responsive behavior in `+page.svelte`: at `max-width: 767px`, left panel takes full width; right panel opens as fixed slide-over overlay with backdrop and close/touch-outside-dismiss behavior (RATIONALE: FR-004 mobile requirement; REJECTED: separate mobile route — over-engineering)
- [ ] T018 [US2] Wire `environmentContextStore` into new `+page.svelte`: on environment switch from the context store, reset `selectedDatasetId`, reload datasets, update Stats Bar. Preserve existing `initializeEnvironmentContext` + `selectedEnvironmentStore` logic from current implementation.
- [ ] T019 [P] [US1] [US2] Write vitest component test for `StatsBar` in `frontend/src/routes/datasets/__tests__/StatsBar.test.js`: test 4 tiles render, counts display, click dispatches filter, skeleton shows when loading
- [ ] T020 [P] [US2] Write vitest component test for split-view layout in `frontend/src/routes/datasets/__tests__/split_view.test.js`: test grid layout on desktop, slide-over on mobile

**Verification**: 
- `cd frontend && npm run test`
- Browser: `http://localhost:8100/datasets` — Stats Bar visible, clicking tiles triggers API reload with `?filter=`, split-view layout renders

---

## Phase 4: Dataset Cards (US3 — P2)

> Cards replace old table rows. Depends on Phase 3 (split-view shell + Stats Bar).

- [ ] T021 [US3] Create `DatasetList.svelte` component in `frontend/src/routes/datasets/DatasetList.svelte` — card grid with pagination, search input, bulk selection checkboxes. Each card renders: `table_name` (linked), `schema`, `column_count`, `metric_count` (or hidden if 0), progress bar (mapped/total % with green/yellow/blue colors), quick-action buttons (🗺 Map Columns, 📄 Generate Docs). Clicking card body dispatches `onselect(datasetId)`. (RATIONALE: cards give richer visual info than table rows; REJECTED: keep table — doesn't show progress bars well)
- [ ] T022 [US3] Implement server-side filtering in `+page.svelte` orchestrator: when `activeFilter` changes to non-null value, call `loadDatasets(filter=activeFilter, page=1)`. Stats Bar tiles receive `activeFilter` prop and highlight accordingly. Backend filters full dataset list before pagination (FR-026).
- [ ] T023 [US3] Preserve existing pagination (server-side) in `DatasetList.svelte`: pagination controls call `loadDatasets(page=N)` via parent callback. Total count shown from `stats.total`. When filter is active, `stats.total` still shows global total; pagination shows total matching count from API response `total` field.
- [ ] T024 [US3] Migrate existing search input from old `+page.svelte` into `DatasetList.svelte` — debounced search calls `loadDatasets()` from parent via prop callback. Existing `debounce` util reused.
- [ ] T025 [US3] Wire quick-action buttons in `DatasetList.svelte`: "🗺 Map Columns" dispatches `onaction(dataset, 'map_columns')`; "📄 Generate Docs" dispatches `onaction(dataset, 'generate_docs')`. Parent `+page.svelte` opens existing modals (reused from current implementation).
- [ ] T026 [P] [US3] Write vitest component test for `DatasetList` in `frontend/src/routes/datasets/__tests__/DatasetList.test.js`: test cards render with all fields, progress bar color logic, filter removes non-matching cards, search triggers callback

**Verification**: `cd frontend && npm run test`

---

## Phase 5: Detail Panel + Metrics Component (US4 + US7-frontend — P2)

> Right panel of split-view. Depends on Phase 3 (split-view shell) and Phase 2 (US7 backend metrics).

- [ ] T027 [US4] Create `DatasetPreview.svelte` component in `frontend/src/routes/datasets/DatasetPreview.svelte` — presentational component receiving `dataset`, `isLoading`, `error` props. States: placeholder "Выберите датасет" (dataset=null, not loading), skeleton (isLoading=true), detail view (dataset data), error banner (error string + onretry callback). Parent `+page.svelte` (DatasetHub) fetches `GET /api/datasets/{id}` on card selection and passes result as props. (RATIONALE: container-fetches pattern for cleaner testing and separation of concerns; REJECTED: self-fetching component — couples data fetching with rendering)
- [ ] T028 [US4] Build dataset header section in `DatasetPreview.svelte`: `table_name` (bold), `schema` · `column_count` колонок · `metric_count` метрик, linked dashboards (clickable pills), created/changed dates. Linked dashboards navigate to `/dashboards/{slug}` on click. (FR-007)
- [ ] T029 [US4] Create `ColumnsTable.svelte` component in `frontend/src/routes/datasets/ColumnsTable.svelte` — table with columns: name (verbose_name bold, else column_name italic), type chip (color-coded: DATE=green, STRING=purple, INT=blue, FLOAT=orange), description (or "✏️ Добавить описание" action prompt), edit button. Rows from `dataset.columns`. (FR-008, FR-010, FR-011)
- [ ] T030 [US4] Create `MetricsTable.svelte` component in `frontend/src/routes/datasets/MetricsTable.svelte` — table with columns: metric_name (verbose_name bold, else metric_name italic), expression shown as gray hint under name when no description, description (or "✏️ Добавить описание" prompt), edit button. Rows from `dataset.metrics`. (FR-009, FR-010, FR-011, FR-012)
- [ ] T031 [US4] Wire `ColumnsTable` and `MetricsTable` into `DatasetPreview.svelte` as child components. Pass `dataset.columns` and `dataset.metrics` as props respectively.
- [ ] T032 [P] [US4] Write vitest component test for `DatasetPreview` in `frontend/src/routes/datasets/__tests__/DatasetPreview.test.js`: test no-selection placeholder, loading skeleton, loaded state with header+columns+metrics, error state
- [ ] T033 [P] [US4] Write vitest component test for `ColumnsTable` in `frontend/src/routes/datasets/__tests__/ColumnsTable.test.js`: test verbose_name bold/column_name italic fallback, type chip colors, mapped/unmapped badge, empty state

**Verification**: `cd frontend && npm run test`

---

## Phase 6: Inline Edit (US5 — P3)

> Click-to-edit descriptions. Depends on Phase 4 (columns/metrics visible) + Phase 2 (backend endpoints).

- [ ] T034 [US5] Implement inline-edit state in `ColumnsTable.svelte`: `$state(isEditingRowId)`. On ✏️ click: set `isEditingRowId`, render `<textarea>` with existing description (or empty), "Сохранить" and "Отмена" buttons. On Save: call `PUT /api/datasets/{id}/columns/{col_id}/description`, on success update local state and exit editing, on error show red border + error text. On Cancel: reset to original value, exit editing. (FR-013, FR-014)
- [ ] T035 [US5] Implement inline-edit state in `MetricsTable.svelte`: mirror of T034 but using `PUT /api/datasets/{id}/metrics/{metric_id}/description`. Same save/cancel/error behavior.
- [ ] T036 [US5] Handle API errors gracefully in both tables: 400 (bad request) — show validation message; 503 (Superset down) — show "Сервис временно недоступен"; network error — show "Нет соединения". Textarea stays open for retry.
- [ ] T037 [P] [US5] Write vitest component test for inline-edit in `frontend/src/routes/datasets/__tests__/inline_edit.test.js`: test click opens textarea with prefill, save calls API and updates description, cancel reverts, save error keeps textarea open

**Verification**: `cd frontend && npm run test`

---

## Phase 7: Bulk Actions (US6 — P3)

> Adapt existing bulk functionality to cards + split-view. Depends on Phase 3 (cards with checkboxes).

- [ ] T038 [US6] Integrate bulk selection from `DatasetList.svelte` into `+page.svelte` orchestrator: on checkbox change in DatasetList, update `selectedIds: Set` in parent. When `selectedIds.size >= 2`, show `BulkActionPanel` at bottom of page (fixed position, z-50). When `selectedIds.size === 0`, hide panel.
- [ ] T039 [US6] Re-implement `BulkActionPanel` in `+page.svelte` (can be inline or extracted component): show selected count "✓ N выбрано", action buttons "🗺 Маппинг колонок" and "📄 Генерация документации", "✕ Снять" to deselect all. Buttons open existing modals (Map Columns modal, Generate Docs modal) — preserve current modal code from old `+page.svelte`. (FR-015, FR-016)
- [ ] T040 [US6] Ensure bulk selection works across pages: when pagination changes page, selected IDs persist (already implemented in current `loadDatasets` logic — verify and preserve)
- [ ] T041 [P] [US6] Write vitest regression test for bulk actions in `frontend/src/routes/datasets/__tests__/bulk_actions.test.js`: test panel appears at 2+ selected, disappears at 0, map columns modal opens, generate docs modal opens, selected count displays correctly

**Verification**: `cd frontend && npm run test`

---

## Phase 8: Polish & Cross-Cutting Verification

> Final integration, WebSocket connection, edge cases, ADR compliance check.

- [ ] T042 Add WebSocket subscription in `+page.svelte`: on mount, subscribe to `dataset.updated` events via existing WebSocket store. On receiving event with matching `env_id`, call `loadDatasets()` to refresh list + Stats Bar. Unsubscribe on destroy. (FR-024)
- [ ] T043 Handle loading/empty/error states consistently across all components: StatsBar — skeleton tiles / error message; DatasetList — skeleton cards / "Датасеты не найдены" / error banner; DatasetPreview — placeholder / skeleton / error with retry. Use existing Tailwind `animate-pulse` skeleton pattern.
- [ ] T044 Handle edge case: 0 columns and 0 metrics — both tables show "Нет данных" instead of trying to render empty table
- [ ] T045 Handle edge case: API 503 (Superset unavailable) — show error banner at top of page with retry button, do not crash
- [ ] T046 Add belief runtime markers (`belief_scope`, `reason`, `reflect`, `explore`) in `datasets.py` for C4/C5 contracts: both inline-edit endpoints, modified `get_datasets()`, WebSocket event emission. Also in `_datasets.py` modified `get_dataset_detail()`. Follow `molecular-cot-logging` JSON-line format.
- [ ] T047 Full-stack verification: `cd backend && source .venv/bin/activate && python -m pytest -v` — all backend tests pass
- [ ] T048 Frontend verification: `cd frontend && npm run test` — all vitest tests pass; `npm run lint` — no lint errors; `npm run build` — production build succeeds
- [ ] T049 Semantic audit: run `axiom_semantic_validation audit_contracts` on `frontend/src/routes/datasets/` and `backend/src/api/routes/datasets.py` — no missing required metadata, no orphan C4/C5 contracts, no broken anchors
- [ ] T050 Rejected-path regression check: verify no task re-introduces `override_columns=true` (T011/T012 must use `false`); verify no direct Superset API call from frontend (all mutations go through ss-tools backend per ADR-0003)
- [ ] T051 Browser validation: navigate `http://localhost:8100/datasets`, verify Stats Bar loads with 33/28/5/12 counts, click filter tiles, click card → detail panel loads with columns+metrics, inline-edit a column description and verify save, select 2+ cards → bulk panel appears, click Map Columns → modal opens

**Verification**: `cd backend && source .venv/bin/activate && python -m pytest -v && cd frontend && npm run test && npm run build`

---

## Task Summary

| Phase | Tasks | Story | Parallel |
|-------|-------|-------|----------|
| 1. Setup | T001–T004 | — | T001∥T002, T003∥T004 |
| 2. Foundation (US7) | T005–T014 | US7 | T005–T010 partial; T011–T013 after T005–T009 |
| 3. Stats Bar + Split View (US1+US2) | T015–T020 | US1,US2 | T019∥T020 |
| 3. Stats Bar + Split View (US1+US2) | T015–T020 | US1,US2 | T019∥T020 |
| 4. Dataset Cards (US3) | T021–T026 | US3 | T026∥ after T021–T025 |
| 5. Detail Panel (US4+US7) | T027–T033 | US4 | T032∥T033 |
| 6. Inline Edit (US5) | T034–T037 | US5 | T037∥ after T034–T036 |
| 7. Bulk Actions (US6) | T038–T041 | US6 | T041∥ after T038–T040 |
| 8. Polish & Verification | T042–T051 | — | T046–T050 parallel after T042–T045 |

**Total**: 52 tasks across 8 phases.

### Story Independent Verification

| Story | How to verify independently |
|-------|---------------------------|
| US1 | Load `/datasets`, check 4 tiles visible, click → API called with `?filter=`, server returns filtered list |
| US2 | Desktop: 40/60 split grid visible; mobile: slide-over on card click |
| US3 | Card shows all fields + progress bar; filter/search/page work |
| US4 | Click card → right panel loads columns table + metrics table |
| US5 | Click ✏️ on column → textarea → save → updated description persists |
| US6 | Check 2+ cards → bottom panel appears → map/docs buttons work |
| US7 | `curl /api/datasets/32` → response has `metrics` and `metric_count` |

### ADR / Guardrail Coverage

| ADR | Guarded by tasks |
|-----|-----------------|
| ADR-0002 (semantic protocol) | T046 (belief markers), T049 (semantic audit) |
| ADR-0003 (orchestrator) | T011, T012, T050 (no direct frontend-to-Superset) |
| ADR-0006 (Svelte 5 runes) | All T015–T041 (components use runes only) |
| FR-023 (override_columns) | T011, T012, T050 (always `false`) |
| INV_7 (fractal limit) | T016 (decompose 974→<400 lines) |