diff --git a/.kilocode/rules/specify-rules.md b/.kilocode/rules/specify-rules.md index 7b8d44c..d3fc972 100644 --- a/.kilocode/rules/specify-rules.md +++ b/.kilocode/rules/specify-rules.md @@ -9,6 +9,8 @@ Auto-generated from all feature plans. Last updated: 2025-12-19 - Python 3.9+, Node.js 18+ + FastAPI, SvelteKit, Tailwind CSS, Pydantic (005-fix-ui-ws-validation) - N/A (Configuration based) (005-fix-ui-ws-validation) - Filesystem (plugins, logs, backups), SQLite (optional, for job history if needed) (005-fix-ui-ws-validation) +- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, Tailwind CSS (007-migration-dashboard-grid) +- N/A (Superset API integration) (007-migration-dashboard-grid) - Python 3.9+ (Backend), Node.js 18+ (Frontend Build) (001-plugin-arch-svelte-ui) @@ -29,10 +31,9 @@ cd src; pytest; ruff check . Python 3.9+ (Backend), Node.js 18+ (Frontend Build): Follow standard conventions ## Recent Changes +- 007-migration-dashboard-grid: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, Tailwind CSS +- 007-migration-dashboard-grid: Added [if applicable, e.g., PostgreSQL, CoreData, files or N/A] - 006-configurable-belief-logs: Added Python 3.9+ + FastAPI (Backend), Pydantic (Config), Svelte (Frontend) -- 005-fix-ui-ws-validation: Added Python 3.9+ (Backend), Node.js 18+ (Frontend Build) -- 005-fix-ui-ws-validation: Added Python 3.9+, Node.js 18+ + FastAPI, SvelteKit, Tailwind CSS, Pydantic -- 005-fix-ui-ws-validation: Added Python 3.9+, Node.js 18+ + FastAPI, SvelteKit, Tailwind CSS, Pydantic diff --git a/specs/007-migration-dashboard-grid/checklists/requirements.md b/specs/007-migration-dashboard-grid/checklists/requirements.md new file mode 100644 index 0000000..2db20b4 --- /dev/null +++ b/specs/007-migration-dashboard-grid/checklists/requirements.md @@ -0,0 +1,36 @@ +# Specification Quality Checklist: Migration Plugin Dashboard Grid + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: 2025-12-27 +**Feature**: [specs/007-migration-dashboard-grid/spec.md](../spec.md) + +## Content Quality + +- [x] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [x] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [x] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [x] No implementation details leak into specification + +## Notes + +- The specification clearly defines the UI requirements for the dashboard selection grid. +- "Superset API" is mentioned as the source of truth, which is acceptable as it defines the data boundary. +- Success criteria include specific performance metrics (<200ms filtering). \ No newline at end of file diff --git a/specs/007-migration-dashboard-grid/contracts/api.md b/specs/007-migration-dashboard-grid/contracts/api.md new file mode 100644 index 0000000..d7a2648 --- /dev/null +++ b/specs/007-migration-dashboard-grid/contracts/api.md @@ -0,0 +1,49 @@ +# API Contracts: Migration Dashboard Grid + +## Endpoints + +### GET /api/migration/dashboards + +Fetch a list of dashboards from the specified source environment. + +**Request:** +- **Headers**: + - `x-source-env-id`: `string` (UUID of the source environment configuration) + +**Response:** +- **Status**: `200 OK` +- **Body**: `List[Dashboard]` + +```json +[ + { + "id": 101, + "title": "Sales Overview", + "changed_on": "2023-10-27T14:30:00Z", + "published": true, + "url": "/superset/dashboard/sales-overview/" + }, + { + "id": 102, + "title": "Marketing Draft", + "changed_on": "2023-10-26T09:15:00Z", + "published": false, + "url": "/superset/dashboard/marketing-draft/" + } +] +``` + +**Errors:** +- `400 Bad Request`: Missing environment ID header. +- `404 Not Found`: Environment configuration not found. +- `502 Bad Gateway`: Error communicating with Superset. + +## Python Definitions + +```python +# [DEF:backend.src.api.routes.migration.get_dashboards:Function] +# @PURPOSE: Fetch dashboards from the specified source environment. +# @PRE: Header 'x-source-env-id' must be a valid environment UUID. +# @POST: Returns a list of Dashboard models with id, title, changed_on, and published status. +# @PARAM: source_env_id (str) - UUID of the source environment. +# @RETURN: List[DashboardModel] \ No newline at end of file diff --git a/specs/007-migration-dashboard-grid/data-model.md b/specs/007-migration-dashboard-grid/data-model.md new file mode 100644 index 0000000..5252f0b --- /dev/null +++ b/specs/007-migration-dashboard-grid/data-model.md @@ -0,0 +1,38 @@ +# Data Model: Migration Dashboard Grid + +## Entities + +### Dashboard +Represents a Superset dashboard with extended metadata for migration selection. + +| Field | Type | Description | Source | +|-------|------|-------------|--------| +| `id` | `int` | Unique identifier | Superset API | +| `title` | `string` | Display name of the dashboard | Superset API (`dashboard_title`) | +| `changed_on` | `datetime` | Last modification timestamp (UTC) | Superset API (`changed_on_utc`) | +| `published` | `boolean` | Publication status (True=Published, False=Draft) | Superset API | +| `url` | `string` | Link to the dashboard (optional, for preview) | Superset API (`url`) | + +### DashboardSelection +State of the user's selection in the grid. + +| Field | Type | Description | +|-------|------|-------------| +| `selected_ids` | `Set` | IDs of selected dashboards | +| `is_all_selected` | `boolean` | Helper for "Select All" logic (optional) | + +## Component State (Frontend) + +### DashboardGrid +State managed within the Svelte component. + +| State | Type | Description | +|-------|------|-------------| +| `dashboards` | `Array` | Full list of fetched dashboards | +| `filtered_dashboards` | `Array` | List after applying text filter | +| `paginated_dashboards` | `Array` | Slice of filtered list for current view | +| `filter_text` | `string` | Current search query | +| `sort_by` | `string` | Column key to sort by (`title`, `changed_on`, `published`) | +| `sort_asc` | `boolean` | Sort direction | +| `page` | `int` | Current page number (0-indexed) | +| `page_size` | `int` | Items per page (default 20) | \ No newline at end of file diff --git a/specs/007-migration-dashboard-grid/plan.md b/specs/007-migration-dashboard-grid/plan.md new file mode 100644 index 0000000..4bc65b7 --- /dev/null +++ b/specs/007-migration-dashboard-grid/plan.md @@ -0,0 +1,102 @@ +# Implementation Plan: [FEATURE] + +**Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link] +**Input**: Feature specification from `/specs/[###-feature-name]/spec.md` + +**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow. + +## Summary + +[Extract from feature spec: primary requirement + technical approach from research] + +## Technical Context + +**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend) +**Primary Dependencies**: FastAPI, SvelteKit, Tailwind CSS +**Storage**: N/A (Superset API integration) +**Testing**: pytest (Backend), vitest (Frontend) +**Target Platform**: Web Browser, Linux Server +**Project Type**: Web application +**Performance Goals**: Filtering < 200ms +**Constraints**: Pagination for large datasets +**Scale/Scope**: < 1000 dashboards typical + +## Constitution Check + +*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* + +- [x] **I. Causal Validity**: Contracts (`data-model.md`, `contracts/api.md`) defined before implementation. +- [x] **II. Immutability**: No core architecture changes. +- [x] **III. Semantic Format**: `[DEF]` syntax used in contracts. +- [x] **IV. DbC**: Pre/Post conditions defined in contracts. +- [x] **V. Belief State**: Logging standards acknowledged. + +## Project Structure + +### Documentation (this feature) + +```text +specs/[###-feature]/ +├── plan.md # This file (/speckit.plan command output) +├── research.md # Phase 0 output (/speckit.plan command) +├── data-model.md # Phase 1 output (/speckit.plan command) +├── quickstart.md # Phase 1 output (/speckit.plan command) +├── contracts/ # Phase 1 output (/speckit.plan command) +└── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan) +``` + +### Source Code (repository root) + + +```text +# [REMOVE IF UNUSED] Option 1: Single project (DEFAULT) +src/ +├── models/ +├── services/ +├── cli/ +└── lib/ + +tests/ +├── contract/ +├── integration/ +└── unit/ + +# [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected) +backend/ +├── src/ +│ ├── models/ +│ ├── services/ +│ └── api/ +└── tests/ + +frontend/ +├── src/ +│ ├── components/ +│ ├── pages/ +│ └── services/ +└── tests/ + +# [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected) +api/ +└── [same as backend above] + +ios/ or android/ +└── [platform-specific structure: feature modules, UI flows, platform tests] +``` + +**Structure Decision**: [Document the selected structure and reference the real +directories captured above] + +## Complexity Tracking + +> **Fill ONLY if Constitution Check has violations that must be justified** + +| Violation | Why Needed | Simpler Alternative Rejected Because | +|-----------|------------|-------------------------------------| +| [e.g., 4th project] | [current need] | [why 3 projects insufficient] | +| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] | diff --git a/specs/007-migration-dashboard-grid/quickstart.md b/specs/007-migration-dashboard-grid/quickstart.md new file mode 100644 index 0000000..e213654 --- /dev/null +++ b/specs/007-migration-dashboard-grid/quickstart.md @@ -0,0 +1,23 @@ +# Quickstart: Migration Dashboard Grid + +## Overview +The Migration Dashboard Grid allows users to select dashboards for migration using a rich interface with filtering, sorting, and status indicators. + +## Usage + +1. **Navigate to Migration Page**: Go to the Migration section in the app. +2. **Select Source Environment**: Choose the Superset environment you want to migrate from. +3. **View Dashboards**: The grid will automatically load the list of available dashboards. + - **Filter**: Type in the "Search dashboards..." box to filter by name. + - **Sort**: Click on column headers (Name, Last Modified, Status) to sort. + - **Paginate**: Use the "Next" and "Previous" buttons to navigate through pages. +4. **Select Dashboards**: + - Click individual checkboxes to select specific dashboards. + - Click the "Select All" checkbox in the header to select **all** dashboards matching the current filter (even those on other pages). +5. **Proceed**: Once selection is complete, click "Next" to configure mappings. + +## Troubleshooting + +- **"No dashboards found"**: Ensure the source environment is correctly configured and accessible. +- **Slow loading**: If the environment has thousands of dashboards, the initial load might take a few seconds. +- **Status "Unknown"**: If the API fails to return publication status, it will default to a neutral state. \ No newline at end of file diff --git a/specs/007-migration-dashboard-grid/research.md b/specs/007-migration-dashboard-grid/research.md new file mode 100644 index 0000000..83ff2c3 --- /dev/null +++ b/specs/007-migration-dashboard-grid/research.md @@ -0,0 +1,66 @@ +# Research: Migration Dashboard Grid + +## 1. Superset API Capabilities + +**Objective**: Verify how to fetch dashboard metadata (title, last modified, status). + +- **Findings**: + - The `SupersetClient.get_dashboards` method in `superset_tool/client.py` already exists. + - It fetches the following columns by default: `["slug", "id", "changed_on_utc", "dashboard_title", "published"]`. + - This covers all requirements: + - Name -> `dashboard_title` + - Last Modified -> `changed_on_utc` + - Status -> `published` + - Pagination is handled internally by `_fetch_all_pages`, so the client fetches *all* dashboards. + +- **Decision**: Use the existing `SupersetClient.get_dashboards` method. No changes needed to the core client. + +## 2. Frontend Grid Implementation + +**Objective**: Choose a strategy for the grid component (filtering, sorting, pagination). + +- **Options**: + 1. **Server-side pagination**: Fetch page by page. Good for huge datasets (>10k). + 2. **Client-side pagination**: Fetch all, filter/sort/paginate in browser. Good for typical datasets (<1k) and simplifies "Select All" logic. + +- **Findings**: + - Spec explicitly requests "Client-side (Fetch all, filter locally)". + - Spec requires "Select All" to select *all filtered results*, which is trivial with client-side state but complex with server-side pagination (requires tracking exclusion list or query parameters). + - Existing `MappingTable.svelte` is too simple (no sorting/pagination). + +- **Decision**: + - Implement a new `DashboardGrid.svelte` component. + - **State Management**: + - `dashboards`: Array of all fetched dashboards. + - `filterText`: String for name filtering. + - `sortColumn`: 'title' | 'changed_on' | 'published'. + - `sortDirection`: 'asc' | 'desc'. + - `currentPage`: Integer. + - `pageSize`: Integer (default 20). + - `selectedIds`: Set/Array of selected dashboard IDs. + - **Logic**: + - `filteredDashboards`: Derived store/value based on `dashboards` + `filterText`. + - `sortedDashboards`: Derived from `filteredDashboards` + `sort` params. + - `paginatedDashboards`: Slice of `sortedDashboards` for current page. + - "Select All": Adds all IDs from `sortedDashboards` (not just `paginatedDashboards`) to `selectedIds`. + +## 3. API Contract + +**Objective**: Define the endpoint to serve dashboard data. + +- **Current State**: Need to check if an endpoint exists that returns this data. +- **Requirement**: `GET /api/migration/dashboards` (or similar). +- **Response Structure**: + ```json + [ + { + "id": 123, + "title": "Sales Dashboard", + "changed_on": "2023-10-27T10:00:00Z", + "published": true + }, + ... + ] + ``` + +- **Decision**: Create or update a route in `backend/src/api/routes/migration.py` (or `mappings.py` if more appropriate, but `migration` seems better for source selection). diff --git a/specs/007-migration-dashboard-grid/spec.md b/specs/007-migration-dashboard-grid/spec.md new file mode 100644 index 0000000..e1d82a0 --- /dev/null +++ b/specs/007-migration-dashboard-grid/spec.md @@ -0,0 +1,74 @@ +# Feature Specification: Migration Plugin Dashboard Grid + +**Feature Branch**: `007-migration-dashboard-grid` +**Created**: 2025-12-27 +**Status**: Draft +**Input**: User description: "Я хочу доработать плагин миграции. Выбор дашбордов должен осуществляться из списка-грида, с возможностью его фильтровать по наименованию. В гриде должны быть поля наименования дашборда, дата последнего изменения дашборда, плюс статус - опубликован или черновик" + +## Clarifications + +### Session 2025-12-27 +- Q: How should the grid handle data loading and filtering to ensure performance and usability? → A: **Client-side** (Fetch all, filter locally). +- Q: Should the grid include a "Select All" checkbox in the header for bulk operations? → A: **Yes, include "Select All"**. +- Q: How should the grid handle large lists of dashboards (e.g., >50)? → A: **Pagination** (e.g., 20 per page). +- Q: Does the "Select All" checkbox select only the currently visible page of dashboards, or all dashboards that match the current filter? → A: **All matching filter** (Selects all filtered results, not just the visible page). +- Q: What should happen if the user changes the filter while some items are already selected? → A: **Preserve selection** (Selected items remain selected even if hidden by new filter). + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Advanced Dashboard Selection (Priority: P1) + +As a migration engineer, I want to select dashboards from a detailed grid view that includes status and modification dates, so that I can easily distinguish between draft/published versions and identify the most recent changes before migrating. + +**Why this priority**: Current selection mechanisms (likely simple dropdowns or lists) lack critical context (status, freshness), making it error-prone to select the right assets for migration. + +**Independent Test**: Can be tested by connecting to a Superset instance with known dashboards (some drafts, some published) and verifying the grid correctly displays their metadata and allows filtering/selection. + +**Acceptance Scenarios**: + +1. **Given** I have selected a source environment in the migration plugin, **When** the dashboard list loads, **Then** I see a grid view displaying "Dashboard Name", "Last Modified", and "Status" columns. +2. **Given** the dashboard grid is displayed, **When** I type "Sales" into the filter input, **Then** the grid updates to show only dashboards containing "Sales" in their name. +3. **Given** a dashboard is in "Draft" state in Superset, **When** it appears in the grid, **Then** the Status column clearly indicates "Draft" (vs "Published"). +4. **Given** I want to migrate multiple dashboards, **When** I check the boxes next to several rows, **Then** they are added to the selection for the migration job. +5. **Given** the grid is populated, **When** I click the "Select All" checkbox in the header, **Then** all visible dashboards are selected. + +--- + +### Edge Cases + +- **Empty Environment**: What happens if the source environment has no dashboards? System should display a "No dashboards found" message in the grid area. +- **Missing Metadata**: What if the Superset API returns null for `changed_on` or `published`? System should display "N/A" or a default value (e.g., "Unknown") rather than crashing. +- **Large Dataset**: How does the grid handle 1000+ dashboards? The grid MUST use pagination (default 20 items per page) to manage display density. + +## Requirements *(mandatory)* + +### Functional Requirements + +- **FR-001**: The system MUST fetch extended metadata for dashboards from the Superset API, specifically: Title, Last Modified Date (`changed_on`), and Published Status (`published`). +- **FR-002**: The Migration Plugin UI MUST display a data grid component to list these dashboards. +- **FR-003**: The grid MUST include sortable columns for: + - Name (Dashboard Title) + - Last Modified (Date/Time) + - Status (Published/Draft) +- **FR-004**: The UI MUST provide a text filter input that filters the grid rows by Dashboard Name in real-time using client-side logic (fetching all dashboards once). +- **FR-005**: The grid MUST support multi-row selection to allow migrating batches of dashboards. +- **FR-006**: The selection state MUST be passed to the migration execution logic when the user initiates the migration. +- **FR-007**: The grid header MUST include a "Select All" checkbox. When checked, it MUST select ALL dashboards matching the current filter criteria (spanning across all pages), not just the currently visible page. +- **FR-008**: The grid MUST support pagination, displaying 20 rows per page by default, with navigation controls (Next/Prev/Page numbers). +- **FR-009**: The selection state MUST be preserved across filter changes. Items selected before a filter change MUST remain selected even if they are hidden by the new filter. + +### Key Entities + +- **Dashboard Metadata**: + - `id`: Unique identifier from Superset. + - `title`: Display name. + - `changed_on`: Timestamp of last edit. + - `is_published`: Boolean status. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: Users can identify the status (Draft/Published) of any dashboard in the list with 100% accuracy. +- **SC-002**: Filtering a list of 100 dashboards takes less than 200ms to update the view. +- **SC-003**: Users can successfully select and initiate migration for a mix of Draft and Published dashboards in a single operation. diff --git a/specs/007-migration-dashboard-grid/tasks.md b/specs/007-migration-dashboard-grid/tasks.md new file mode 100644 index 0000000..884d49d --- /dev/null +++ b/specs/007-migration-dashboard-grid/tasks.md @@ -0,0 +1,32 @@ +# Tasks: Migration Plugin Dashboard Grid + +## Phase 1: Setup +- [ ] T001 Create migration route module in backend/src/api/routes/migration.py +- [ ] T002 Register migration router in backend/src/app.py +- [ ] T003 Export migration router in backend/src/api/routes/__init__.py + +## Phase 2: Foundational +- [ ] T004 Create Dashboard model in backend/src/models/dashboard.py + +## Phase 3: User Story 1 - Advanced Dashboard Selection +- [ ] T005 [P] [US1] Extend SupersetClient to fetch dashboards with metadata (handling missing fields) in backend/src/core/superset_client.py +- [ ] T006 [US1] Implement GET /api/migration/dashboards endpoint in backend/src/api/routes/migration.py +- [ ] T007 [P] [US1] Create DashboardGrid component structure in frontend/src/components/DashboardGrid.svelte +- [ ] T008 [US1] Implement data fetching and state management in frontend/src/components/DashboardGrid.svelte +- [ ] T009 [US1] Implement client-side filtering logic in frontend/src/components/DashboardGrid.svelte +- [ ] T010 [US1] Implement pagination logic in frontend/src/components/DashboardGrid.svelte +- [ ] T011 [US1] Implement selection logic (single and Select All) in frontend/src/components/DashboardGrid.svelte +- [ ] T012 [US1] Integrate DashboardGrid into migration page and connect selection to submission in frontend/src/routes/migration/+page.svelte +- [ ] T013 [US1] Implement POST /api/migration/execute endpoint to accept selected dashboard IDs in backend/src/api/routes/migration.py + +## Phase 4: Polish +- [ ] T014 Verify error handling and empty states in frontend/src/components/DashboardGrid.svelte +- [ ] T015 Ensure consistent styling with Tailwind CSS in frontend/src/components/DashboardGrid.svelte + +## Dependencies +- US1 depends on T004, T005, T006 + +## Implementation Strategy +- Implement backend support first (Models -> Client -> API) +- Implement frontend component with mock data then connect API +- Integrate into main page \ No newline at end of file