tasks ready

2025-12-27 06:37:03 +03:00
parent 65f61c1f80
commit dcbf0a7d7f
9 changed files with 424 additions and 3 deletions
--- a/.kilocode/rules/specify-rules.md
+++ b/.kilocode/rules/specify-rules.md
@@ -9,6 +9,8 @@ Auto-generated from all feature plans. Last updated: 2025-12-19
 - Python 3.9+, Node.js 18+ + FastAPI, SvelteKit, Tailwind CSS, Pydantic (005-fix-ui-ws-validation)
 - N/A (Configuration based) (005-fix-ui-ws-validation)
 - Filesystem (plugins, logs, backups), SQLite (optional, for job history if needed) (005-fix-ui-ws-validation)
+- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, Tailwind CSS (007-migration-dashboard-grid)
+- N/A (Superset API integration) (007-migration-dashboard-grid)

 - Python 3.9+ (Backend), Node.js 18+ (Frontend Build) (001-plugin-arch-svelte-ui)

@@ -29,10 +31,9 @@ cd src; pytest; ruff check .
 Python 3.9+ (Backend), Node.js 18+ (Frontend Build): Follow standard conventions

 ## Recent Changes
+- 007-migration-dashboard-grid: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, Tailwind CSS
+- 007-migration-dashboard-grid: Added [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
 - 006-configurable-belief-logs: Added Python 3.9+ + FastAPI (Backend), Pydantic (Config), Svelte (Frontend)
- 005-fix-ui-ws-validation: Added Python 3.9+ (Backend), Node.js 18+ (Frontend Build)
- 005-fix-ui-ws-validation: Added Python 3.9+, Node.js 18+ + FastAPI, SvelteKit, Tailwind CSS, Pydantic
- 005-fix-ui-ws-validation: Added Python 3.9+, Node.js 18+ + FastAPI, SvelteKit, Tailwind CSS, Pydantic


 <!-- MANUAL ADDITIONS START -->
--- a/specs/007-migration-dashboard-grid/checklists/requirements.md
+++ b/specs/007-migration-dashboard-grid/checklists/requirements.md
@@ -0,0 +1,36 @@
+# Specification Quality Checklist: Migration Plugin Dashboard Grid
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2025-12-27
+**Feature**: [specs/007-migration-dashboard-grid/spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- The specification clearly defines the UI requirements for the dashboard selection grid.
+- "Superset API" is mentioned as the source of truth, which is acceptable as it defines the data boundary.
+- Success criteria include specific performance metrics (<200ms filtering).
--- a/specs/007-migration-dashboard-grid/contracts/api.md
+++ b/specs/007-migration-dashboard-grid/contracts/api.md
@@ -0,0 +1,49 @@
+# API Contracts: Migration Dashboard Grid
+
+## Endpoints
+
+### GET /api/migration/dashboards
+
+Fetch a list of dashboards from the specified source environment.
+
+**Request:**
+- **Headers**:
+  - `x-source-env-id`: `string` (UUID of the source environment configuration)
+
+**Response:**
+- **Status**: `200 OK`
+- **Body**: `List[Dashboard]`
+
+```json
+[
+  {
+    "id": 101,
+    "title": "Sales Overview",
+    "changed_on": "2023-10-27T14:30:00Z",
+    "published": true,
+    "url": "/superset/dashboard/sales-overview/"
+  },
+  {
+    "id": 102,
+    "title": "Marketing Draft",
+    "changed_on": "2023-10-26T09:15:00Z",
+    "published": false,
+    "url": "/superset/dashboard/marketing-draft/"
+  }
+]
+```
+
+**Errors:**
+- `400 Bad Request`: Missing environment ID header.
+- `404 Not Found`: Environment configuration not found.
+- `502 Bad Gateway`: Error communicating with Superset.
+
+## Python Definitions
+
+```python
+# [DEF:backend.src.api.routes.migration.get_dashboards:Function]
+# @PURPOSE: Fetch dashboards from the specified source environment.
+# @PRE:     Header 'x-source-env-id' must be a valid environment UUID.
+# @POST:    Returns a list of Dashboard models with id, title, changed_on, and published status.
+# @PARAM:   source_env_id (str) - UUID of the source environment.
+# @RETURN:  List[DashboardModel]
--- a/specs/007-migration-dashboard-grid/data-model.md
+++ b/specs/007-migration-dashboard-grid/data-model.md
@@ -0,0 +1,38 @@
+# Data Model: Migration Dashboard Grid
+
+## Entities
+
+### Dashboard
+Represents a Superset dashboard with extended metadata for migration selection.
+
+| Field | Type | Description | Source |
+|-------|------|-------------|--------|
+| `id` | `int` | Unique identifier | Superset API |
+| `title` | `string` | Display name of the dashboard | Superset API (`dashboard_title`) |
+| `changed_on` | `datetime` | Last modification timestamp (UTC) | Superset API (`changed_on_utc`) |
+| `published` | `boolean` | Publication status (True=Published, False=Draft) | Superset API |
+| `url` | `string` | Link to the dashboard (optional, for preview) | Superset API (`url`) |
+
+### DashboardSelection
+State of the user's selection in the grid.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `selected_ids` | `Set<int>` | IDs of selected dashboards |
+| `is_all_selected` | `boolean` | Helper for "Select All" logic (optional) |
+
+## Component State (Frontend)
+
+### DashboardGrid
+State managed within the Svelte component.
+
+| State | Type | Description |
+|-------|------|-------------|
+| `dashboards` | `Array<Dashboard>` | Full list of fetched dashboards |
+| `filtered_dashboards` | `Array<Dashboard>` | List after applying text filter |
+| `paginated_dashboards` | `Array<Dashboard>` | Slice of filtered list for current view |
+| `filter_text` | `string` | Current search query |
+| `sort_by` | `string` | Column key to sort by (`title`, `changed_on`, `published`) |
+| `sort_asc` | `boolean` | Sort direction |
+| `page` | `int` | Current page number (0-indexed) |
+| `page_size` | `int` | Items per page (default 20) |
--- a/specs/007-migration-dashboard-grid/plan.md
+++ b/specs/007-migration-dashboard-grid/plan.md
@@ -0,0 +1,102 @@
+# Implementation Plan: [FEATURE]
+
+**Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link]
+**Input**: Feature specification from `/specs/[###-feature-name]/spec.md`
+
+**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
+
+## Summary
+
+[Extract from feature spec: primary requirement + technical approach from research]
+
+## Technical Context
+
+**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
+**Primary Dependencies**: FastAPI, SvelteKit, Tailwind CSS
+**Storage**: N/A (Superset API integration)
+**Testing**: pytest (Backend), vitest (Frontend)
+**Target Platform**: Web Browser, Linux Server
+**Project Type**: Web application
+**Performance Goals**: Filtering < 200ms
+**Constraints**: Pagination for large datasets
+**Scale/Scope**: < 1000 dashboards typical
+
+## Constitution Check
+
+*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
+
+- [x] **I. Causal Validity**: Contracts (`data-model.md`, `contracts/api.md`) defined before implementation.
+- [x] **II. Immutability**: No core architecture changes.
+- [x] **III. Semantic Format**: `[DEF]` syntax used in contracts.
+- [x] **IV. DbC**: Pre/Post conditions defined in contracts.
+- [x] **V. Belief State**: Logging standards acknowledged.
+
+## Project Structure
+
+### Documentation (this feature)
+
+```text
+specs/[###-feature]/
+├── plan.md              # This file (/speckit.plan command output)
+├── research.md          # Phase 0 output (/speckit.plan command)
+├── data-model.md        # Phase 1 output (/speckit.plan command)
+├── quickstart.md        # Phase 1 output (/speckit.plan command)
+├── contracts/           # Phase 1 output (/speckit.plan command)
+└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
+```
+
+### Source Code (repository root)
+<!--
+  ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
+  for this feature. Delete unused options and expand the chosen structure with
+  real paths (e.g., apps/admin, packages/something). The delivered plan must
+  not include Option labels.
+-->
+
+```text
+# [REMOVE IF UNUSED] Option 1: Single project (DEFAULT)
+src/
+├── models/
+├── services/
+├── cli/
+└── lib/
+
+tests/
+├── contract/
+├── integration/
+└── unit/
+
+# [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected)
+backend/
+├── src/
+│   ├── models/
+│   ├── services/
+│   └── api/
+└── tests/
+
+frontend/
+├── src/
+│   ├── components/
+│   ├── pages/
+│   └── services/
+└── tests/
+
+# [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected)
+api/
+└── [same as backend above]
+
+ios/ or android/
+└── [platform-specific structure: feature modules, UI flows, platform tests]
+```
+
+**Structure Decision**: [Document the selected structure and reference the real
+directories captured above]
+
+## Complexity Tracking
+
+> **Fill ONLY if Constitution Check has violations that must be justified**
+
+| Violation | Why Needed | Simpler Alternative Rejected Because |
+|-----------|------------|-------------------------------------|
+| [e.g., 4th project] | [current need] | [why 3 projects insufficient] |
+| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] |
--- a/specs/007-migration-dashboard-grid/quickstart.md
+++ b/specs/007-migration-dashboard-grid/quickstart.md
@@ -0,0 +1,23 @@
+# Quickstart: Migration Dashboard Grid
+
+## Overview
+The Migration Dashboard Grid allows users to select dashboards for migration using a rich interface with filtering, sorting, and status indicators.
+
+## Usage
+
+1.  **Navigate to Migration Page**: Go to the Migration section in the app.
+2.  **Select Source Environment**: Choose the Superset environment you want to migrate from.
+3.  **View Dashboards**: The grid will automatically load the list of available dashboards.
+    - **Filter**: Type in the "Search dashboards..." box to filter by name.
+    - **Sort**: Click on column headers (Name, Last Modified, Status) to sort.
+    - **Paginate**: Use the "Next" and "Previous" buttons to navigate through pages.
+4.  **Select Dashboards**:
+    - Click individual checkboxes to select specific dashboards.
+    - Click the "Select All" checkbox in the header to select **all** dashboards matching the current filter (even those on other pages).
+5.  **Proceed**: Once selection is complete, click "Next" to configure mappings.
+
+## Troubleshooting
+
+-   **"No dashboards found"**: Ensure the source environment is correctly configured and accessible.
+-   **Slow loading**: If the environment has thousands of dashboards, the initial load might take a few seconds.
+-   **Status "Unknown"**: If the API fails to return publication status, it will default to a neutral state.
--- a/specs/007-migration-dashboard-grid/research.md
+++ b/specs/007-migration-dashboard-grid/research.md
@@ -0,0 +1,66 @@
+# Research: Migration Dashboard Grid
+
+## 1. Superset API Capabilities
+
+**Objective**: Verify how to fetch dashboard metadata (title, last modified, status).
+
+- **Findings**:
+  - The `SupersetClient.get_dashboards` method in `superset_tool/client.py` already exists.
+  - It fetches the following columns by default: `["slug", "id", "changed_on_utc", "dashboard_title", "published"]`.
+  - This covers all requirements:
+    - Name -> `dashboard_title`
+    - Last Modified -> `changed_on_utc`
+    - Status -> `published`
+  - Pagination is handled internally by `_fetch_all_pages`, so the client fetches *all* dashboards.
+
+- **Decision**: Use the existing `SupersetClient.get_dashboards` method. No changes needed to the core client.
+
+## 2. Frontend Grid Implementation
+
+**Objective**: Choose a strategy for the grid component (filtering, sorting, pagination).
+
+- **Options**:
+  1.  **Server-side pagination**: Fetch page by page. Good for huge datasets (>10k).
+  2.  **Client-side pagination**: Fetch all, filter/sort/paginate in browser. Good for typical datasets (<1k) and simplifies "Select All" logic.
+
+- **Findings**:
+  - Spec explicitly requests "Client-side (Fetch all, filter locally)".
+  - Spec requires "Select All" to select *all filtered results*, which is trivial with client-side state but complex with server-side pagination (requires tracking exclusion list or query parameters).
+  - Existing `MappingTable.svelte` is too simple (no sorting/pagination).
+
+- **Decision**:
+  - Implement a new `DashboardGrid.svelte` component.
+  - **State Management**:
+    - `dashboards`: Array of all fetched dashboards.
+    - `filterText`: String for name filtering.
+    - `sortColumn`: 'title' | 'changed_on' | 'published'.
+    - `sortDirection`: 'asc' | 'desc'.
+    - `currentPage`: Integer.
+    - `pageSize`: Integer (default 20).
+    - `selectedIds`: Set/Array of selected dashboard IDs.
+  - **Logic**:
+    - `filteredDashboards`: Derived store/value based on `dashboards` + `filterText`.
+    - `sortedDashboards`: Derived from `filteredDashboards` + `sort` params.
+    - `paginatedDashboards`: Slice of `sortedDashboards` for current page.
+    - "Select All": Adds all IDs from `sortedDashboards` (not just `paginatedDashboards`) to `selectedIds`.
+
+## 3. API Contract
+
+**Objective**: Define the endpoint to serve dashboard data.
+
+- **Current State**: Need to check if an endpoint exists that returns this data.
+- **Requirement**: `GET /api/migration/dashboards` (or similar).
+- **Response Structure**:
+  ```json
+  [
+    {
+      "id": 123,
+      "title": "Sales Dashboard",
+      "changed_on": "2023-10-27T10:00:00Z",
+      "published": true
+    },
+    ...
+  ]
+  ```
+
+- **Decision**: Create or update a route in `backend/src/api/routes/migration.py` (or `mappings.py` if more appropriate, but `migration` seems better for source selection).
--- a/specs/007-migration-dashboard-grid/spec.md
+++ b/specs/007-migration-dashboard-grid/spec.md
@@ -0,0 +1,74 @@
+# Feature Specification: Migration Plugin Dashboard Grid
+
+**Feature Branch**: `007-migration-dashboard-grid`
+**Created**: 2025-12-27
+**Status**: Draft
+**Input**: User description: "Я хочу доработать плагин миграции. Выбор дашбордов должен осуществляться из списка-грида, с возможностью его фильтровать по наименованию. В гриде должны быть поля наименования дашборда, дата последнего изменения дашборда, плюс статус - опубликован или черновик"
+
+## Clarifications
+
+### Session 2025-12-27
+- Q: How should the grid handle data loading and filtering to ensure performance and usability? → A: **Client-side** (Fetch all, filter locally).
+- Q: Should the grid include a "Select All" checkbox in the header for bulk operations? → A: **Yes, include "Select All"**.
+- Q: How should the grid handle large lists of dashboards (e.g., >50)? → A: **Pagination** (e.g., 20 per page).
+- Q: Does the "Select All" checkbox select only the currently visible page of dashboards, or all dashboards that match the current filter? → A: **All matching filter** (Selects all filtered results, not just the visible page).
+- Q: What should happen if the user changes the filter while some items are already selected? → A: **Preserve selection** (Selected items remain selected even if hidden by new filter).
+
+## User Scenarios & Testing *(mandatory)*
+
+### User Story 1 - Advanced Dashboard Selection (Priority: P1)
+
+As a migration engineer, I want to select dashboards from a detailed grid view that includes status and modification dates, so that I can easily distinguish between draft/published versions and identify the most recent changes before migrating.
+
+**Why this priority**: Current selection mechanisms (likely simple dropdowns or lists) lack critical context (status, freshness), making it error-prone to select the right assets for migration.
+
+**Independent Test**: Can be tested by connecting to a Superset instance with known dashboards (some drafts, some published) and verifying the grid correctly displays their metadata and allows filtering/selection.
+
+**Acceptance Scenarios**:
+
+1. **Given** I have selected a source environment in the migration plugin, **When** the dashboard list loads, **Then** I see a grid view displaying "Dashboard Name", "Last Modified", and "Status" columns.
+2. **Given** the dashboard grid is displayed, **When** I type "Sales" into the filter input, **Then** the grid updates to show only dashboards containing "Sales" in their name.
+3. **Given** a dashboard is in "Draft" state in Superset, **When** it appears in the grid, **Then** the Status column clearly indicates "Draft" (vs "Published").
+4. **Given** I want to migrate multiple dashboards, **When** I check the boxes next to several rows, **Then** they are added to the selection for the migration job.
+5. **Given** the grid is populated, **When** I click the "Select All" checkbox in the header, **Then** all visible dashboards are selected.
+
+---
+
+### Edge Cases
+
+- **Empty Environment**: What happens if the source environment has no dashboards? System should display a "No dashboards found" message in the grid area.
+- **Missing Metadata**: What if the Superset API returns null for `changed_on` or `published`? System should display "N/A" or a default value (e.g., "Unknown") rather than crashing.
+- **Large Dataset**: How does the grid handle 1000+ dashboards? The grid MUST use pagination (default 20 items per page) to manage display density.
+
+## Requirements *(mandatory)*
+
+### Functional Requirements
+
+- **FR-001**: The system MUST fetch extended metadata for dashboards from the Superset API, specifically: Title, Last Modified Date (`changed_on`), and Published Status (`published`).
+- **FR-002**: The Migration Plugin UI MUST display a data grid component to list these dashboards.
+- **FR-003**: The grid MUST include sortable columns for:
+    - Name (Dashboard Title)
+    - Last Modified (Date/Time)
+    - Status (Published/Draft)
+- **FR-004**: The UI MUST provide a text filter input that filters the grid rows by Dashboard Name in real-time using client-side logic (fetching all dashboards once).
+- **FR-005**: The grid MUST support multi-row selection to allow migrating batches of dashboards.
+- **FR-006**: The selection state MUST be passed to the migration execution logic when the user initiates the migration.
+- **FR-007**: The grid header MUST include a "Select All" checkbox. When checked, it MUST select ALL dashboards matching the current filter criteria (spanning across all pages), not just the currently visible page.
+- **FR-008**: The grid MUST support pagination, displaying 20 rows per page by default, with navigation controls (Next/Prev/Page numbers).
+- **FR-009**: The selection state MUST be preserved across filter changes. Items selected before a filter change MUST remain selected even if they are hidden by the new filter.
+
+### Key Entities
+
+- **Dashboard Metadata**:
+    - `id`: Unique identifier from Superset.
+    - `title`: Display name.
+    - `changed_on`: Timestamp of last edit.
+    - `is_published`: Boolean status.
+
+## Success Criteria *(mandatory)*
+
+### Measurable Outcomes
+
+- **SC-001**: Users can identify the status (Draft/Published) of any dashboard in the list with 100% accuracy.
+- **SC-002**: Filtering a list of 100 dashboards takes less than 200ms to update the view.
+- **SC-003**: Users can successfully select and initiate migration for a mix of Draft and Published dashboards in a single operation.
--- a/specs/007-migration-dashboard-grid/tasks.md
+++ b/specs/007-migration-dashboard-grid/tasks.md
@@ -0,0 +1,32 @@
+# Tasks: Migration Plugin Dashboard Grid
+
+## Phase 1: Setup
+- [ ] T001 Create migration route module in backend/src/api/routes/migration.py
+- [ ] T002 Register migration router in backend/src/app.py
+- [ ] T003 Export migration router in backend/src/api/routes/__init__.py
+
+## Phase 2: Foundational
+- [ ] T004 Create Dashboard model in backend/src/models/dashboard.py
+
+## Phase 3: User Story 1 - Advanced Dashboard Selection
+- [ ] T005 [P] [US1] Extend SupersetClient to fetch dashboards with metadata (handling missing fields) in backend/src/core/superset_client.py
+- [ ] T006 [US1] Implement GET /api/migration/dashboards endpoint in backend/src/api/routes/migration.py
+- [ ] T007 [P] [US1] Create DashboardGrid component structure in frontend/src/components/DashboardGrid.svelte
+- [ ] T008 [US1] Implement data fetching and state management in frontend/src/components/DashboardGrid.svelte
+- [ ] T009 [US1] Implement client-side filtering logic in frontend/src/components/DashboardGrid.svelte
+- [ ] T010 [US1] Implement pagination logic in frontend/src/components/DashboardGrid.svelte
+- [ ] T011 [US1] Implement selection logic (single and Select All) in frontend/src/components/DashboardGrid.svelte
+- [ ] T012 [US1] Integrate DashboardGrid into migration page and connect selection to submission in frontend/src/routes/migration/+page.svelte
+- [ ] T013 [US1] Implement POST /api/migration/execute endpoint to accept selected dashboard IDs in backend/src/api/routes/migration.py
+
+## Phase 4: Polish
+- [ ] T014 Verify error handling and empty states in frontend/src/components/DashboardGrid.svelte
+- [ ] T015 Ensure consistent styling with Tailwind CSS in frontend/src/components/DashboardGrid.svelte
+
+## Dependencies
+- US1 depends on T004, T005, T006
+
+## Implementation Strategy
+- Implement backend support first (Models -> Client -> API)
+- Implement frontend component with mock data then connect API
+- Integrate into main page