ss-tools/specs/030-dataset-lifecycle-workspace/tasks.md

Tasks: Dataset Lifecycle Workspace

Feature: 030-dataset-lifecycle-workspace Branch: 030-dataset-lifecycle-workspace Spec: spec.md | Plan: plan.md

---

Phase 1: Setup (i18n + Breadcrumb)

No user stories yet — infrastructure that all stories depend on.

• T001 [P] Add 11 EN i18n keys to frontend/src/lib/i18n/locales/en/datasets.json: all, without_mapping, with_mapping, linked_to_dashboards, metrics, metric_count, no_selection, save_description, add_description, columns, detail_empty
• T002 [P] Add 11 RU i18n keys to frontend/src/lib/i18n/locales/ru/datasets.json: все, без_маппинга, с_маппингом, связаны_с_дашбордами, метрики, метрик, выберите_датасет, сохранить_описание, добавить_описание, колонки, пустой_просмотр
• T003 Verify breadcrumb for /datasets shows "Датасеты / Datasets" in frontend/src/lib/components/layout/Breadcrumbs.svelte (already fixed, confirm)
• T004 [P] Verify sidebar navigation entries for datasets in frontend/src/lib/components/layout/sidebarNavigation.js are correct

Verification: cd frontend && npm run build — проверка что билд не падает с новыми i18n ключами.

---

Phase 2: Foundational — Backend Metrics API (US7)

Blocks US4 (detail panel needs metrics). Must complete before any UI work with metrics data.

• T005 Add MetricItem Pydantic DTO (fields: id, metric_name, expression, verbose_name, description, metric_type) in backend/src/api/routes/datasets.py (RATIONALE: metrics exist in Superset API but are currently ignored; REJECTED: separate metrics endpoint adds unnecessary API surface)
• T006 Add metrics: list[MetricItem] and metric_count: int to DatasetDetailResponse in backend/src/api/routes/datasets.py
• T007 Extract dataset.get("metrics", []) from Superset API response in SupersetClient.get_dataset_detail() at backend/src/core/superset_client/_datasets.py (add ~15 lines after existing columns loop, map each metric dict to MetricItem-like dict)
• T007a Add metric_count: int to DatasetItem DTO in backend/src/api/routes/datasets.py. Enrich list items with metric count during resource_service.get_datasets_with_status() (extract len(dataset.get("metrics", [])) from Superset dataset metadata). (RATIONALE: FR-025 — cards must display metric count; REJECTED: lazy-loading metric count per card — too many requests, UX jank)
• T008 Add StatsCounts Pydantic DTO (fields: total, unmapped_count, mapped_count, linked_count) in backend/src/api/routes/datasets.py
• T009 Add stats: StatsCounts to DatasetsResponse in backend/src/api/routes/datasets.py
• T010 Compute stats counts from full (non-paginated, non-filtered) dataset list in get_datasets() handler in backend/src/api/routes/datasets.py — counts: total=len(datasets), unmapped=sum(mapped==0), mapped=sum(mapped>0), linked=sum(linked>0). Add server-side filtering: accept ?filter=unmapped|mapped|linked|all, filter full list before pagination slice. (RATIONALE: one API call for both list + stats per FR-022; server-side filtering ensures all matching datasets returned; REJECTED: separate /stats endpoint adds unnecessary request, client-side filtering incompatible with pagination)
• T011 Add inline-edit endpoints to backend/src/api/routes/datasets.py: PUT /api/datasets/{dataset_id}/columns/{column_id}/description (body: ColumnDescriptionUpdate, handler: GET full dataset → modify column → PUT to Superset with ?override_columns=false) (RATIONALE: Superset lacks PATCH for individual columns; REJECTED: frontend direct PUT — bypasses orchestrator, exposes auth)
• T012 Add inline-edit endpoint to backend/src/api/routes/datasets.py: PUT /api/datasets/{dataset_id}/metrics/{metric_id}/description (body: MetricDescriptionUpdate, handler: mirror of T011 for metrics) (RATIONALE: same GET→modify→PUT pattern; REJECTED: merging columns+metrics into one endpoint — different Superset payload structures)
• T013 Add dataset.updated WebSocket event emission in task completion handler at backend/src/core/task_manager.py — when a mapping or documentation task completes, broadcast {type: "dataset.updated", payload: {env_id, dataset_ids}} via existing WebSocket manager (RATIONALE: real-time refresh without polling per FR-024; REJECTED: polling interval — unnecessary network overhead)
• T014 Write pytest tests for new endpoints in backend/tests/__tests__/test_datasets.py: test that GET /api/datasets/{id} returns metrics and metric_count; test that PUT /columns/{col_id}/description saves and returns updated description; test that PUT /metrics/{metric_id}/description saves correctly; test that GET /api/datasets returns stats object with correct counts

Verification: cd backend && source .venv/bin/activate && python -m pytest -v backend/tests/__tests__/test_datasets.py

---

Phase 3: Stats Bar + Split-View Layout (US1 + US2 — P1)

Core UI structure. All remaining stories depend on this layout being in place.

• T015 [US1] Create StatsBar.svelte component in frontend/src/routes/datasets/StatsBar.svelte — 4 metric tiles (Все, Без маппинга, С маппингом, С дашбордами), each showing count from stats prop; active tile has highlight border; dispatches onfilter event on click (parent calls API with ?filter=); toggles filter on second click (dispatches null to clear); skeleton tiles when loading. Tiles use aria-pressed reflecting active state. (RATIONALE: quick diagnostic overview replaces empty space at top of current table page; REJECTED: dropdown filters — Stats Bar gives 1-click access)
• T016 [US2] Rewrite frontend/src/routes/datasets/+page.svelte as split-view orchestrator: replace monolithic 974-line table with layout shell using Tailwind grid grid grid-cols-[40%_60%] on desktop, block on mobile (<768px). Remove old table rendering, keep data loading logic, add selectedDatasetId state. Shell renders <StatsBar>, <DatasetList>, <DatasetPreview> components. (RATIONALE: decompose 974-line page into composable components per INV_7 fractal limit; REJECTED: keep monolithic page — violates semantic erosion rules at C5 complexity)
• T017 [US2] Add mobile responsive behavior in +page.svelte: at max-width: 767px, left panel takes full width; right panel opens as fixed slide-over overlay with backdrop and close/touch-outside-dismiss behavior (RATIONALE: FR-004 mobile requirement; REJECTED: separate mobile route — over-engineering)
• T018 [US2] Wire environmentContextStore into new +page.svelte: on environment switch from the context store, reset selectedDatasetId, reload datasets, update Stats Bar. Preserve existing initializeEnvironmentContext + selectedEnvironmentStore logic from current implementation.
• T019 [P] [US1] [US2] Write vitest component test for StatsBar in frontend/src/routes/datasets/__tests__/StatsBar.test.js: test 4 tiles render, counts display, click dispatches filter, skeleton shows when loading
• T020 [P] [US2] Write vitest component test for split-view layout in frontend/src/routes/datasets/__tests__/split_view.test.js: test grid layout on desktop, slide-over on mobile

Verification:

• cd frontend && npm run test
• Browser: http://localhost:8100/datasets — Stats Bar visible, clicking tiles triggers API reload with ?filter=, split-view layout renders

---

Phase 4: Dataset Cards (US3 — P2)

Cards replace old table rows. Depends on Phase 3 (split-view shell + Stats Bar).

• T021 [US3] Create DatasetList.svelte component in frontend/src/routes/datasets/DatasetList.svelte — card grid with pagination, search input, bulk selection checkboxes. Each card renders: table_name (linked), schema, column_count, metric_count (or hidden if 0), progress bar (mapped/total % with green/yellow/blue colors), quick-action buttons (🗺 Map Columns, 📄 Generate Docs). Clicking card body dispatches onselect(datasetId). (RATIONALE: cards give richer visual info than table rows; REJECTED: keep table — doesn't show progress bars well)
• T022 [US3] Implement server-side filtering in +page.svelte orchestrator: when activeFilter changes to non-null value, call loadDatasets(filter=activeFilter, page=1). Stats Bar tiles receive activeFilter prop and highlight accordingly. Backend filters full dataset list before pagination (FR-026).
• T023 [US3] Preserve existing pagination (server-side) in DatasetList.svelte: pagination controls call loadDatasets(page=N) via parent callback. Total count shown from stats.total. When filter is active, stats.total still shows global total; pagination shows total matching count from API response total field.
• T024 [US3] Migrate existing search input from old +page.svelte into DatasetList.svelte — debounced search calls loadDatasets() from parent via prop callback. Existing debounce util reused.
• T025 [US3] Wire quick-action buttons in DatasetList.svelte: "🗺 Map Columns" dispatches onaction(dataset, 'map_columns'); "📄 Generate Docs" dispatches onaction(dataset, 'generate_docs'). Parent +page.svelte opens existing modals (reused from current implementation).
• T026 [P] [US3] Write vitest component test for DatasetList in frontend/src/routes/datasets/__tests__/DatasetList.test.js: test cards render with all fields, progress bar color logic, filter removes non-matching cards, search triggers callback

Verification: cd frontend && npm run test

---

Phase 5: Detail Panel + Metrics Component (US4 + US7-frontend — P2)

Right panel of split-view. Depends on Phase 3 (split-view shell) and Phase 2 (US7 backend metrics).

• T027 [US4] Create DatasetPreview.svelte component in frontend/src/routes/datasets/DatasetPreview.svelte — presentational component receiving dataset, isLoading, error props. States: placeholder "Выберите датасет" (dataset=null, not loading), skeleton (isLoading=true), detail view (dataset data), error banner (error string + onretry callback). Parent +page.svelte (DatasetHub) fetches GET /api/datasets/{id} on card selection and passes result as props. (RATIONALE: container-fetches pattern for cleaner testing and separation of concerns; REJECTED: self-fetching component — couples data fetching with rendering)
• T028 [US4] Build dataset header section in DatasetPreview.svelte: table_name (bold), schema · column_count колонок · metric_count метрик, linked dashboards (clickable pills), created/changed dates. Linked dashboards navigate to /dashboards/{slug} on click. (FR-007)
• T029 [US4] Create ColumnsTable.svelte component in frontend/src/routes/datasets/ColumnsTable.svelte — table with columns: name (verbose_name bold, else column_name italic), type chip (color-coded: DATE=green, STRING=purple, INT=blue, FLOAT=orange), description (or "✏️ Добавить описание" action prompt), edit button. Rows from dataset.columns. (FR-008, FR-010, FR-011)
• T030 [US4] Create MetricsTable.svelte component in frontend/src/routes/datasets/MetricsTable.svelte — table with columns: metric_name (verbose_name bold, else metric_name italic), expression shown as gray hint under name when no description, description (or "✏️ Добавить описание" prompt), edit button. Rows from dataset.metrics. (FR-009, FR-010, FR-011, FR-012)
• T031 [US4] Wire ColumnsTable and MetricsTable into DatasetPreview.svelte as child components. Pass dataset.columns and dataset.metrics as props respectively.
• T032 [P] [US4] Write vitest component test for DatasetPreview in frontend/src/routes/datasets/__tests__/DatasetPreview.test.js: test no-selection placeholder, loading skeleton, loaded state with header+columns+metrics, error state
• T033 [P] [US4] Write vitest component test for ColumnsTable in frontend/src/routes/datasets/__tests__/ColumnsTable.test.js: test verbose_name bold/column_name italic fallback, type chip colors, mapped/unmapped badge, empty state

Verification: cd frontend && npm run test

---

Phase 6: Inline Edit (US5 — P3)

Click-to-edit descriptions. Depends on Phase 4 (columns/metrics visible) + Phase 2 (backend endpoints).

• T034 [US5] Implement inline-edit state in ColumnsTable.svelte: $state(isEditingRowId). On ✏️ click: set isEditingRowId, render <textarea> with existing description (or empty), "Сохранить" and "Отмена" buttons. On Save: call PUT /api/datasets/{id}/columns/{col_id}/description, on success update local state and exit editing, on error show red border + error text. On Cancel: reset to original value, exit editing. (FR-013, FR-014)
• T035 [US5] Implement inline-edit state in MetricsTable.svelte: mirror of T034 but using PUT /api/datasets/{id}/metrics/{metric_id}/description. Same save/cancel/error behavior.
• T036 [US5] Handle API errors gracefully in both tables: 400 (bad request) — show validation message; 503 (Superset down) — show "Сервис временно недоступен"; network error — show "Нет соединения". Textarea stays open for retry.
• T037 [P] [US5] Write vitest component test for inline-edit in frontend/src/routes/datasets/__tests__/inline_edit.test.js: test click opens textarea with prefill, save calls API and updates description, cancel reverts, save error keeps textarea open

Verification: cd frontend && npm run test

---

Phase 7: Bulk Actions (US6 — P3)

Adapt existing bulk functionality to cards + split-view. Depends on Phase 3 (cards with checkboxes).

• T038 [US6] Integrate bulk selection from DatasetList.svelte into +page.svelte orchestrator: on checkbox change in DatasetList, update selectedIds: Set in parent. When selectedIds.size >= 2, show BulkActionPanel at bottom of page (fixed position, z-50). When selectedIds.size === 0, hide panel.
• T039 [US6] Re-implement BulkActionPanel in +page.svelte (can be inline or extracted component): show selected count "✓ N выбрано", action buttons "🗺 Маппинг колонок" and "📄 Генерация документации", "✕ Снять" to deselect all. Buttons open existing modals (Map Columns modal, Generate Docs modal) — preserve current modal code from old +page.svelte. (FR-015, FR-016)
• T040 [US6] Ensure bulk selection works across pages: when pagination changes page, selected IDs persist (already implemented in current loadDatasets logic — verify and preserve)
• T041 [P] [US6] Write vitest regression test for bulk actions in frontend/src/routes/datasets/__tests__/bulk_actions.test.js: test panel appears at 2+ selected, disappears at 0, map columns modal opens, generate docs modal opens, selected count displays correctly

Verification: cd frontend && npm run test

---

Phase 8: Polish & Cross-Cutting Verification

Final integration, WebSocket connection, edge cases, ADR compliance check.

• T042 Add WebSocket subscription in +page.svelte: on mount, subscribe to dataset.updated events via existing WebSocket store. On receiving event with matching env_id, call loadDatasets() to refresh list + Stats Bar. Unsubscribe on destroy. (FR-024)
• T043 Handle loading/empty/error states consistently across all components: StatsBar — skeleton tiles / error message; DatasetList — skeleton cards / "Датасеты не найдены" / error banner; DatasetPreview — placeholder / skeleton / error with retry. Use existing Tailwind animate-pulse skeleton pattern.
• T044 Handle edge case: 0 columns and 0 metrics — both tables show "Нет данных" instead of trying to render empty table
• T045 Handle edge case: API 503 (Superset unavailable) — show error banner at top of page with retry button, do not crash
• T046 Add belief runtime markers (belief_scope, reason, reflect, explore) in datasets.py for C4/C5 contracts: both inline-edit endpoints, modified get_datasets(), WebSocket event emission. Also in _datasets.py modified get_dataset_detail(). Follow molecular-cot-logging JSON-line format.
• T047 Full-stack verification: cd backend && source .venv/bin/activate && python -m pytest -v — all backend tests pass
• T048 Frontend verification: cd frontend && npm run test — all vitest tests pass; npm run lint — no lint errors; npm run build — production build succeeds
• T049 Semantic audit: run axiom_semantic_validation audit_contracts on frontend/src/routes/datasets/ and backend/src/api/routes/datasets.py — no missing required metadata, no orphan C4/C5 contracts, no broken anchors
• T050 Rejected-path regression check: verify no task re-introduces override_columns=true (T011/T012 must use false); verify no direct Superset API call from frontend (all mutations go through ss-tools backend per ADR-0003)
• T051 Browser validation: navigate http://localhost:8100/datasets, verify Stats Bar loads with 33/28/5/12 counts, click filter tiles, click card → detail panel loads with columns+metrics, inline-edit a column description and verify save, select 2+ cards → bulk panel appears, click Map Columns → modal opens

Verification: cd backend && source .venv/bin/activate && python -m pytest -v && cd frontend && npm run test && npm run build

---

Task Summary

Phase	Tasks	Story	Parallel
1. Setup	T001–T004	—	T001∥T002, T003∥T004
2. Foundation (US7)	T005–T014	US7	T005–T010 partial; T011–T013 after T005–T009
3. Stats Bar + Split View (US1+US2)	T015–T020	US1,US2	T019∥T020
3. Stats Bar + Split View (US1+US2)	T015–T020	US1,US2	T019∥T020
4. Dataset Cards (US3)	T021–T026	US3	T026∥ after T021–T025
5. Detail Panel (US4+US7)	T027–T033	US4	T032∥T033
6. Inline Edit (US5)	T034–T037	US5	T037∥ after T034–T036
7. Bulk Actions (US6)	T038–T041	US6	T041∥ after T038–T040
8. Polish & Verification	T042–T051	—	T046–T050 parallel after T042–T045

Total: 52 tasks across 8 phases.

Story Independent Verification

Story	How to verify independently
US1	Load /datasets, check 4 tiles visible, click → API called with ?filter=, server returns filtered list
US2	Desktop: 40/60 split grid visible; mobile: slide-over on card click
US3	Card shows all fields + progress bar; filter/search/page work
US4	Click card → right panel loads columns table + metrics table
US5	Click ✏️ on column → textarea → save → updated description persists
US6	Check 2+ cards → bottom panel appears → map/docs buttons work
US7	curl /api/datasets/32 → response has metrics and metric_count

ADR / Guardrail Coverage

ADR	Guarded by tasks
ADR-0002 (semantic protocol)	T046 (belief markers), T049 (semantic audit)
ADR-0003 (orchestrator)	T011, T012, T050 (no direct frontend-to-Superset)
ADR-0006 (Svelte 5 runes)	All T015–T041 (components use runes only)
FR-023 (override_columns)	T011, T012, T050 (always false)
INV_7 (fractal limit)	T016 (decompose 974→<400 lines)