tech_lead / coder 2roles

This commit is contained in:
2025-12-27 08:02:59 +03:00
parent 07914c8728
commit 3d75a21127
17 changed files with 376 additions and 507 deletions

View File

@@ -2,48 +2,57 @@
## Endpoints
### GET /api/migration/dashboards
### 1. List Dashboards
**Method**: `GET`
**Path**: `/api/environments/{env_id}/dashboards`
**Purpose**: Fetch all dashboards from the specified environment for the grid.
Fetch a list of dashboards from the specified source environment.
**Request Parameters**:
- `env_id` (path): The ID of the environment to fetch from.
**Request:**
- **Headers**:
- `x-source-env-id`: `string` (UUID of the source environment configuration)
**Response**:
- **200 OK**:
```json
[
{
"id": 123,
"title": "Sales Dashboard",
"last_modified": "2023-10-27T10:00:00Z",
"status": "published"
},
{
"id": 124,
"title": "Draft Metrics",
"last_modified": "2023-10-26T15:30:00Z",
"status": "draft"
}
]
```
- **404 Not Found**: Environment not found.
- **500 Internal Server Error**: Superset API error.
**Response:**
- **Status**: `200 OK`
- **Body**: `List[Dashboard]`
## Components (Frontend)
```json
[
{
"id": 101,
"title": "Sales Overview",
"changed_on": "2023-10-27T14:30:00Z",
"published": true,
"url": "/superset/dashboard/sales-overview/"
},
{
"id": 102,
"title": "Marketing Draft",
"changed_on": "2023-10-26T09:15:00Z",
"published": false,
"url": "/superset/dashboard/marketing-draft/"
}
]
```
### DashboardGrid
**Props**:
- `dashboards`: `DashboardMetadata[]` - List of dashboards to display.
- `selectedIds`: `number[]` - IDs of currently selected dashboards.
**Errors:**
- `400 Bad Request`: Missing environment ID header.
- `404 Not Found`: Environment configuration not found.
- `502 Bad Gateway`: Error communicating with Superset.
**Events**:
- `selectionChanged`: Emitted when selection changes. Payload: `number[]` (new list of selected IDs).
## Python Definitions
**State**:
- `filterText`: string - Current filter text.
- `currentPage`: number - Current page index (0-based).
- `pageSize`: number - Items per page (default 20).
- `sortColumn`: string - 'title' | 'last_modified' | 'status'.
- `sortDirection`: 'asc' | 'desc'.
```python
# [DEF:backend.src.api.routes.migration.get_dashboards:Function]
# @PURPOSE: Fetch dashboards from the specified source environment.
# @PRE: Header 'x-source-env-id' must be a valid environment UUID.
# @POST: Returns a list of Dashboard models with id, title, changed_on, and published status.
# @PARAM: source_env_id (str) - UUID of the source environment.
# @RETURN: List[DashboardModel]
## Superset Client Extension
### `get_dashboards_summary`
**Signature**: `def get_dashboards_summary(self) -> List[Dict]`
**Purpose**: Fetches dashboard metadata optimized for the grid.
**Implementation Detail**:
- Calls `GET /api/v1/dashboard/` with query params `q=(columns:!(id,dashboard_title,changed_on_utc,published))`.
- Maps response fields to `DashboardMetadata` schema.

View File

@@ -2,37 +2,24 @@
## Entities
### Dashboard
Represents a Superset dashboard with extended metadata for migration selection.
### DashboardMetadata
**Source**: Superset API (`/api/v1/dashboard/`)
**Purpose**: Represents a dashboard available for migration.
| Field | Type | Description | Source |
|-------|------|-------------|--------|
| `id` | `int` | Unique identifier | Superset API |
| `title` | `string` | Display name of the dashboard | Superset API (`dashboard_title`) |
| `changed_on` | `datetime` | Last modification timestamp (UTC) | Superset API (`changed_on_utc`) |
| `published` | `boolean` | Publication status (True=Published, False=Draft) | Superset API |
| `url` | `string` | Link to the dashboard (optional, for preview) | Superset API (`url`) |
| Field | Type | Description | Source Mapping |
|-------|------|-------------|----------------|
| `id` | Integer | Unique identifier | `id` |
| `title` | String | Display name of the dashboard | `dashboard_title` |
| `last_modified` | String (ISO 8601) | Timestamp of last modification | `changed_on_utc` |
| `status` | Enum ('published', 'draft') | Publication status | `published` (boolean) -> 'published'/'draft' |
## Value Objects
### DashboardSelection
State of the user's selection in the grid.
**Purpose**: Represents the user's selection of dashboards to migrate.
| Field | Type | Description |
|-------|------|-------------|
| `selected_ids` | `Set<int>` | IDs of selected dashboards |
| `is_all_selected` | `boolean` | Helper for "Select All" logic (optional) |
## Component State (Frontend)
### DashboardGrid
State managed within the Svelte component.
| State | Type | Description |
|-------|------|-------------|
| `dashboards` | `Array<Dashboard>` | Full list of fetched dashboards |
| `filtered_dashboards` | `Array<Dashboard>` | List after applying text filter |
| `paginated_dashboards` | `Array<Dashboard>` | Slice of filtered list for current view |
| `filter_text` | `string` | Current search query |
| `sort_by` | `string` | Column key to sort by (`title`, `changed_on`, `published`) |
| `sort_asc` | `boolean` | Sort direction |
| `page` | `int` | Current page number (0-indexed) |
| `page_size` | `int` | Items per page (default 20) |
| `selected_ids` | List[Integer] | List of dashboard IDs selected for migration |
| `source_env_id` | String | ID of the source environment |
| `target_env_id` | String | ID of the target environment |

View File

@@ -12,24 +12,25 @@
## Technical Context
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
**Primary Dependencies**: FastAPI, SvelteKit, Tailwind CSS
**Storage**: N/A (Superset API integration)
**Testing**: pytest (Backend), vitest (Frontend)
**Target Platform**: Web Browser, Linux Server
**Project Type**: Web application
**Performance Goals**: Filtering < 200ms
**Constraints**: Pagination for large datasets
**Scale/Scope**: < 1000 dashboards typical
**Primary Dependencies**: FastAPI, SvelteKit, Tailwind CSS, Pydantic, Superset API
**Storage**: N/A (Superset API integration - read-only for metadata)
**Testing**: pytest (Backend), vitest (Frontend - inferred)
**Target Platform**: Linux server / Containerized
**Project Type**: web application (Backend + Frontend)
**Performance Goals**: Client-side filtering < 200ms for 100+ items
**Constraints**: Must handle large lists via pagination (Client-side). Spec says "Client-side (Fetch all, filter locally)" and "Pagination (e.g., 20 per page)". *RESOLVED: Fetch all, paginate locally.*
**Scale/Scope**: ~100s of dashboards per environment.
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
- [x] **I. Causal Validity**: Contracts (`data-model.md`, `contracts/api.md`) defined before implementation.
- [x] **II. Immutability**: No core architecture changes.
- [x] **III. Semantic Format**: `[DEF]` syntax used in contracts.
- [x] **IV. DbC**: Pre/Post conditions defined in contracts.
- [x] **V. Belief State**: Logging standards acknowledged.
- [x] **Causal Validity**: Contracts (API/Data Model) defined before implementation.
- [x] **Immutability**: Module headers (`[DEF]`) preserved/added.
- [x] **Semantic Format**: All new code uses `[DEF]` anchors and metadata.
- [x] **Fractal Complexity**: New components (Grid) kept modular; `SupersetClient` extensions are small methods.
**Status**: PASSED
## Project Structure
@@ -46,51 +47,33 @@ specs/[###-feature]/
```
### Source Code (repository root)
<!--
ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
for this feature. Delete unused options and expand the chosen structure with
real paths (e.g., apps/admin, packages/something). The delivered plan must
not include Option labels.
-->
```text
# [REMOVE IF UNUSED] Option 1: Single project (DEFAULT)
src/
├── models/
├── services/
├── cli/
└── lib/
tests/
├── contract/
├── integration/
└── unit/
# [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected)
backend/
├── src/
│ ├── models/
├── services/
└── api/
│ ├── api/
│ └── routes/
│ └── environments.py # Update to support dashboard fetching
│ ├── core/
│ │ └── superset_client.py # Update to fetch extended dashboard metadata
│ └── models/
│ └── dashboard.py # New model for Dashboard metadata
└── tests/
└── test_superset_client.py
frontend/
├── src/
│ ├── components/
│ ├── pages/
│ └── services/
└── tests/
# [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected)
api/
└── [same as backend above]
ios/ or android/
└── [platform-specific structure: feature modules, UI flows, platform tests]
│ ├── DashboardGrid.svelte # New component
│ └── Pagination.svelte # New component (if not exists)
│ ├── routes/
│ │ └── migration/
│ │ └── +page.svelte # Update to use DashboardGrid
│ └── types/
│ └── dashboard.ts # New type definitions
```
**Structure Decision**: [Document the selected structure and reference the real
directories captured above]
**Structure Decision**: Standard Web Application structure. Backend updates to `SupersetClient` and API routes to serve dashboard metadata. Frontend updates to include a new `DashboardGrid` component and integrate it into the migration flow.
## Complexity Tracking

View File

@@ -1,23 +1,31 @@
# Quickstart: Migration Dashboard Grid
## Overview
The Migration Dashboard Grid allows users to select dashboards for migration using a rich interface with filtering, sorting, and status indicators.
## Prerequisites
- Backend running (`uvicorn backend.src.app:app --reload`)
- Frontend running (`npm run dev`)
- Superset instance accessible and configured in `config.yaml`
## Usage
## Steps to Verify
1. **Navigate to Migration Page**: Go to the Migration section in the app.
2. **Select Source Environment**: Choose the Superset environment you want to migrate from.
3. **View Dashboards**: The grid will automatically load the list of available dashboards.
- **Filter**: Type in the "Search dashboards..." box to filter by name.
- **Sort**: Click on column headers (Name, Last Modified, Status) to sort.
- **Paginate**: Use the "Next" and "Previous" buttons to navigate through pages.
4. **Select Dashboards**:
- Click individual checkboxes to select specific dashboards.
- Click the "Select All" checkbox in the header to select **all** dashboards matching the current filter (even those on other pages).
5. **Proceed**: Once selection is complete, click "Next" to configure mappings.
1. **Navigate to Migration Page**:
- Open browser to `http://localhost:5173/migration`
- Select a Source Environment from the dropdown.
## Troubleshooting
2. **Verify Dashboard Grid**:
- The grid should appear below the environment selectors.
- It should list dashboards with columns: Title, Last Modified, Status.
- Status pills should be green (Published) or gray (Draft).
- **"No dashboards found"**: Ensure the source environment is correctly configured and accessible.
- **Slow loading**: If the environment has thousands of dashboards, the initial load might take a few seconds.
- **Status "Unknown"**: If the API fails to return publication status, it will default to a neutral state.
3. **Test Filtering**:
- Type in the "Search dashboards..." input.
- The list should filter instantly (client-side).
4. **Test Pagination**:
- If >20 dashboards, check pagination controls at the bottom.
- Navigate to next page.
5. **Test Selection**:
- Select a few dashboards.
- Change filter (hide selected).
- Clear filter -> Selection should persist.
- Click "Select All" -> Should select all matching current filter.

View File

@@ -1,66 +1,48 @@
# Research: Migration Dashboard Grid
## 1. Superset API Capabilities
## Unknowns & Clarifications
**Objective**: Verify how to fetch dashboard metadata (title, last modified, status).
### 1. Pagination vs Client-side Filtering
**Context**: The spec mentions "Client-side (Fetch all, filter locally)" (FR-004) but also "Pagination (e.g., 20 per page)" (FR-008).
**Resolution**:
- We will fetch ALL dashboard metadata from the Superset API in one go. The metadata (ID, Title, Status, Date) is lightweight. Even for 1000 dashboards, the payload is small (~100KB).
- **Client-side Pagination**: We will implement pagination purely on the frontend. This satisfies "Pagination" for UI performance/usability while keeping the "Fetch all" requirement for fast filtering.
- **Decision**: Fetch all, paginate locally.
- **Findings**:
- The `SupersetClient.get_dashboards` method in `superset_tool/client.py` already exists.
- It fetches the following columns by default: `["slug", "id", "changed_on_utc", "dashboard_title", "published"]`.
- This covers all requirements:
- Name -> `dashboard_title`
- Last Modified -> `changed_on_utc`
- Status -> `published`
- Pagination is handled internally by `_fetch_all_pages`, so the client fetches *all* dashboards.
### 2. Superset API for Dashboard Metadata
**Context**: Need to fetch `title`, `changed_on`, `published`.
**Research**:
- Superset API endpoint: `/api/v1/dashboard/`
- Standard response includes `result` array with `dashboard_title`, `changed_on_utc`, `published`.
- **Decision**: Use `GET /api/v1/dashboard/` with `q` parameter to select specific columns to minimize payload.
- Columns: `id`, `dashboard_title`, `changed_on_utc`, `published`.
- **Decision**: Use the existing `SupersetClient.get_dashboards` method. No changes needed to the core client.
### 3. Grid Component
**Context**: Need a grid with sorting, filtering, and selection.
**Options**:
- **Custom Svelte Table**: Lightweight, full control.
- **3rd Party Lib (e.g. svelte-headless-table)**: Powerful but maybe overkill.
- **Decision**: **Custom Svelte Component** (`DashboardGrid.svelte`).
- Why: Requirements are specific (Select All across pages, custom status pill, specific columns). A custom component using standard HTML table + Tailwind is simple and maintainable for this scope.
## 2. Frontend Grid Implementation
## Design Decisions
**Objective**: Choose a strategy for the grid component (filtering, sorting, pagination).
### Data Model
- **Dashboard**:
- `id`: string (or int, depends on Superset version, usually int for dashboards but we treat as ID)
- `title`: string
- `last_modified`: string (ISO date)
- `status`: 'published' | 'draft'
- **Options**:
1. **Server-side pagination**: Fetch page by page. Good for huge datasets (>10k).
2. **Client-side pagination**: Fetch all, filter/sort/paginate in browser. Good for typical datasets (<1k) and simplifies "Select All" logic.
### Architecture
- **Backend**:
- `SupersetClient.get_dashboards()`: Fetches list from Superset.
- `GET /api/environments/{id}/dashboards`: Proxy endpoint.
- **Frontend**:
- `DashboardGrid.svelte`: Handles display, sorting, pagination, and selection logic.
- `migration/+page.svelte`: Orchestrates fetching and passes data to Grid.
- **Findings**:
- Spec explicitly requests "Client-side (Fetch all, filter locally)".
- Spec requires "Select All" to select *all filtered results*, which is trivial with client-side state but complex with server-side pagination (requires tracking exclusion list or query parameters).
- Existing `MappingTable.svelte` is too simple (no sorting/pagination).
- **Decision**:
- Implement a new `DashboardGrid.svelte` component.
- **State Management**:
- `dashboards`: Array of all fetched dashboards.
- `filterText`: String for name filtering.
- `sortColumn`: 'title' | 'changed_on' | 'published'.
- `sortDirection`: 'asc' | 'desc'.
- `currentPage`: Integer.
- `pageSize`: Integer (default 20).
- `selectedIds`: Set/Array of selected dashboard IDs.
- **Logic**:
- `filteredDashboards`: Derived store/value based on `dashboards` + `filterText`.
- `sortedDashboards`: Derived from `filteredDashboards` + `sort` params.
- `paginatedDashboards`: Slice of `sortedDashboards` for current page.
- "Select All": Adds all IDs from `sortedDashboards` (not just `paginatedDashboards`) to `selectedIds`.
## 3. API Contract
**Objective**: Define the endpoint to serve dashboard data.
- **Current State**: Need to check if an endpoint exists that returns this data.
- **Requirement**: `GET /api/migration/dashboards` (or similar).
- **Response Structure**:
```json
[
{
"id": 123,
"title": "Sales Dashboard",
"changed_on": "2023-10-27T10:00:00Z",
"published": true
},
...
]
```
- **Decision**: Create or update a route in `backend/src/api/routes/migration.py` (or `mappings.py` if more appropriate, but `migration` seems better for source selection).
### UX/UI
- **Status Column**: Badge (Green for Published, Gray for Draft).
- **Selection**: Checkbox in first column.
- **Pagination**: Simple "Prev 1 of 5 Next" controls at bottom.

View File

@@ -13,6 +13,11 @@
- Q: How should the grid handle large lists of dashboards (e.g., >50)? → A: **Pagination** (e.g., 20 per page).
- Q: Does the "Select All" checkbox select only the currently visible page of dashboards, or all dashboards that match the current filter? → A: **All matching filter** (Selects all filtered results, not just the visible page).
- Q: What should happen if the user changes the filter while some items are already selected? → A: **Preserve selection** (Selected items remain selected even if hidden by new filter).
- Q: What should be the default sort order when the dashboard grid first loads? → A: **Last Modified Date (Newest first)**.
- Q: Should the grid include an "Owners" column to help distinguish dashboards with the same name? → A: **Yes, include Owners**.
- Q: How should the "Owners" column display multiple owners? → A: **Show first owner + count (e.g., "admin + 2") with tooltip**.
- Q: How should the "Status" (Draft/Published) be visually represented in the grid? → A: **Colored Badges/Chips**.
- Q: Should the grid include a "Preview" action (e.g., link to open the dashboard in Superset)? → A: **Yes, open in new tab**.
## User Scenarios & Testing *(mandatory)*
@@ -50,6 +55,7 @@ As a migration engineer, I want to select dashboards from a detailed grid view t
- Name (Dashboard Title)
- Last Modified (Date/Time)
- Status (Published/Draft)
- Owners (List of owner names)
- **FR-004**: The UI MUST provide a text filter input that filters the grid rows by Dashboard Name in real-time using client-side logic (fetching all dashboards once).
- **FR-005**: The grid MUST support multi-row selection to allow migrating batches of dashboards.
- **FR-006**: The selection state MUST be passed to the migration execution logic when the user initiates the migration.
@@ -64,6 +70,7 @@ As a migration engineer, I want to select dashboards from a detailed grid view t
- `title`: Display name.
- `changed_on`: Timestamp of last edit.
- `is_published`: Boolean status.
- `owners`: List of owner objects/names.
## Success Criteria *(mandatory)*

View File

@@ -0,0 +1,29 @@
---
description: "Architecture tasks for Migration Plugin Dashboard Grid"
---
# Architecture Tasks: Migration Plugin Dashboard Grid
**Role**: Architect Agent
**Goal**: Define the "What" and "Why" (Contracts, Scaffolding, Models) before implementation.
## Phase 1: Setup & Models
- [ ] A001 Define contracts/scaffolding for migration route in backend/src/api/routes/migration.py
- [ ] A002 Define contracts/scaffolding for Dashboard model in backend/src/models/dashboard.py
## Phase 2: User Story 1 - Advanced Dashboard Selection
- [ ] A003 [US1] Define contracts/scaffolding for SupersetClient extensions in backend/src/core/superset_client.py
- [ ] A004 [US1] Define contracts/scaffolding for GET /api/migration/dashboards endpoint in backend/src/api/routes/migration.py
- [ ] A005 [US1] Define contracts/scaffolding for DashboardGrid component in frontend/src/components/DashboardGrid.svelte
- [ ] A006 [US1] Define contracts/scaffolding for migration page integration in frontend/src/routes/migration/+page.svelte
- [ ] A007 [US1] Define contracts/scaffolding for POST /api/migration/execute endpoint in backend/src/api/routes/migration.py
## Handover Checklist
- [ ] All new files created with `[DEF]` anchors
- [ ] All functions/classes have `@PURPOSE`, `@PRE`, `@POST` tags
- [ ] No "naked code" (logic outside of anchors)
- [ ] `tasks-dev.md` is ready for the Developer Agent

View File

@@ -0,0 +1,34 @@
---
description: "Developer tasks for Migration Plugin Dashboard Grid"
---
# Developer Tasks: Migration Plugin Dashboard Grid
**Role**: Developer Agent
**Goal**: Implement the "How" (Logic, State, Error Handling) inside the defined contracts.
## Phase 1: Setup & Models
- [ ] D001 Implement logic for migration route in backend/src/api/routes/migration.py
- [ ] D002 Register migration router in backend/src/app.py
- [ ] D003 Export migration router in backend/src/api/routes/__init__.py
- [ ] D004 Implement logic for Dashboard model in backend/src/models/dashboard.py
## Phase 2: User Story 1 - Advanced Dashboard Selection
- [ ] D005 [P] [US1] Implement logic for SupersetClient extensions in backend/src/core/superset_client.py
- [ ] D006 [US1] Implement logic for GET /api/migration/dashboards endpoint in backend/src/api/routes/migration.py
- [ ] D007 [US1] Implement structure and styles for DashboardGrid component in frontend/src/components/DashboardGrid.svelte
- [ ] D008 [US1] Implement data fetching and state management in frontend/src/components/DashboardGrid.svelte
- [ ] D009 [US1] Implement client-side filtering logic in frontend/src/components/DashboardGrid.svelte
- [ ] D010 [US1] Implement pagination logic in frontend/src/components/DashboardGrid.svelte
- [ ] D011 [US1] Implement selection logic (single and Select All) in frontend/src/components/DashboardGrid.svelte
- [ ] D012 [US1] Integrate DashboardGrid and connect selection to submission in frontend/src/routes/migration/+page.svelte
- [ ] D013 [US1] Implement logic for POST /api/migration/execute endpoint in backend/src/api/routes/migration.py
- [ ] D014 [US1] Verify semantic compliance and belief state logging
## Polish & Quality Assurance
- [ ] D015 Verify error handling and empty states in frontend/src/components/DashboardGrid.svelte
- [ ] D016 Ensure consistent styling with Tailwind CSS in frontend/src/components/DashboardGrid.svelte

View File

@@ -1,32 +0,0 @@
# Tasks: Migration Plugin Dashboard Grid
## Phase 1: Setup
- [ ] T001 Create migration route module in backend/src/api/routes/migration.py
- [ ] T002 Register migration router in backend/src/app.py
- [ ] T003 Export migration router in backend/src/api/routes/__init__.py
## Phase 2: Foundational
- [ ] T004 Create Dashboard model in backend/src/models/dashboard.py
## Phase 3: User Story 1 - Advanced Dashboard Selection
- [ ] T005 [P] [US1] Extend SupersetClient to fetch dashboards with metadata (handling missing fields) in backend/src/core/superset_client.py
- [ ] T006 [US1] Implement GET /api/migration/dashboards endpoint in backend/src/api/routes/migration.py
- [ ] T007 [P] [US1] Create DashboardGrid component structure in frontend/src/components/DashboardGrid.svelte
- [ ] T008 [US1] Implement data fetching and state management in frontend/src/components/DashboardGrid.svelte
- [ ] T009 [US1] Implement client-side filtering logic in frontend/src/components/DashboardGrid.svelte
- [ ] T010 [US1] Implement pagination logic in frontend/src/components/DashboardGrid.svelte
- [ ] T011 [US1] Implement selection logic (single and Select All) in frontend/src/components/DashboardGrid.svelte
- [ ] T012 [US1] Integrate DashboardGrid into migration page and connect selection to submission in frontend/src/routes/migration/+page.svelte
- [ ] T013 [US1] Implement POST /api/migration/execute endpoint to accept selected dashboard IDs in backend/src/api/routes/migration.py
## Phase 4: Polish
- [ ] T014 Verify error handling and empty states in frontend/src/components/DashboardGrid.svelte
- [ ] T015 Ensure consistent styling with Tailwind CSS in frontend/src/components/DashboardGrid.svelte
## Dependencies
- US1 depends on T004, T005, T006
## Implementation Strategy
- Implement backend support first (Models -> Client -> API)
- Implement frontend component with mock data then connect API
- Integrate into main page