feat(migration): implement interactive mapping resolution workflow
- Add SQLite database integration for environments and mappings - Update TaskManager to support pausing tasks (AWAITING_MAPPING) - Modify MigrationPlugin to detect missing mappings and wait for resolution - Add frontend UI for handling missing mappings interactively - Create dedicated migration routes and API endpoints - Update .gitignore and project documentation
This commit is contained in:
186
specs/001-migration-ui-redesign/tasks.md
Normal file
186
specs/001-migration-ui-redesign/tasks.md
Normal file
@@ -0,0 +1,186 @@
|
||||
---
|
||||
|
||||
description: "Task list for Migration Process and UI Redesign implementation"
|
||||
---
|
||||
|
||||
# Tasks: Migration Process and UI Redesign
|
||||
|
||||
**Input**: Design documents from `specs/001-migration-ui-redesign/`
|
||||
**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, quickstart.md
|
||||
|
||||
**Tests**: Tests are NOT explicitly requested in the feature specification, so they are omitted from this task list.
|
||||
|
||||
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
|
||||
|
||||
## Format: `[ID] [P?] [Story] Description`
|
||||
|
||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
|
||||
- Include exact file paths in descriptions
|
||||
|
||||
## Path Conventions
|
||||
|
||||
- **Web app**: `backend/src/`, `frontend/src/`
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Setup (Shared Infrastructure)
|
||||
|
||||
**Purpose**: Project initialization and basic structure
|
||||
|
||||
- [ ] T001 Create project structure per implementation plan in `backend/src/` and `frontend/src/`
|
||||
- [ ] T002 [P] Install backend dependencies (rapidfuzz, sqlalchemy) in `backend/requirements.txt`
|
||||
- [ ] T003 [P] Install frontend dependencies (if any new) in `frontend/package.json`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Foundational (Blocking Prerequisites)
|
||||
|
||||
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
|
||||
|
||||
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
|
||||
|
||||
- [ ] T004 Setup SQLite database schema and SQLAlchemy models in `backend/src/models/mapping.py`
|
||||
- [ ] T005 [P] Implement fuzzy matching utility using RapidFuzz in `backend/src/core/utils/matching.py`
|
||||
- [ ] T006 [P] Extend SupersetClient to support database listing and metadata fetching in `backend/src/core/superset_client.py`
|
||||
- [ ] T007 Configure database mapping persistence layer in `backend/src/core/database.py`
|
||||
|
||||
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 1 - Environment-Based Migration Setup (Priority: P1) 🎯 MVP
|
||||
|
||||
**Goal**: Enable selection of source and target environments and toggle database replacement.
|
||||
|
||||
**Independent Test**: Open the migration page and verify that the "Source" and "Target" dropdowns are populated with configured environments and can be selected.
|
||||
|
||||
### Implementation for User Story 1
|
||||
|
||||
- [ ] T008 [P] [US1] Implement environment selection API endpoints in `backend/src/api/routes/environments.py`
|
||||
- [ ] T009 [P] [US1] Create `EnvSelector.svelte` component for source/target selection in `frontend/src/components/EnvSelector.svelte`
|
||||
- [ ] T010 [US1] Integrate `EnvSelector` and "Replace Database" toggle into migration dashboard in `frontend/src/routes/migration/+page.svelte`
|
||||
- [ ] T011 [US1] Add validation to ensure source and target environments are different in `frontend/src/routes/migration/+page.svelte`
|
||||
|
||||
**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 2 - Database Mapping Management (Priority: P1)
|
||||
|
||||
**Goal**: Fetch databases from environments, suggest mappings using fuzzy matching, and allow manual overrides/persistence.
|
||||
|
||||
**Independent Test**: Navigate to the "Database Mapping" tab, fetch databases, and verify that mappings can be created, saved, and edited.
|
||||
|
||||
### Implementation for User Story 2
|
||||
|
||||
- [ ] T012 [P] [US2] Implement database mapping CRUD API endpoints in `backend/src/api/routes/mappings.py`
|
||||
- [ ] T013 [US2] Implement mapping service with fuzzy matching logic in `backend/src/services/mapping_service.py`
|
||||
- [ ] T014 [P] [US2] Create `MappingTable.svelte` component for displaying and editing pairs in `frontend/src/components/MappingTable.svelte`
|
||||
- [ ] T015 [US2] Create database mapping management view in `frontend/src/routes/migration/mappings/+page.svelte`
|
||||
- [ ] T016 [US2] Implement "Fetch Databases" action and suggestion highlighting in `frontend/src/routes/migration/mappings/+page.svelte`
|
||||
|
||||
**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: User Story 3 - Migration with Automated DB Replacement (Priority: P2)
|
||||
|
||||
**Goal**: Intercept assets during migration, apply database mappings, and prompt for missing ones.
|
||||
|
||||
**Independent Test**: Run a migration with "Replace Database" enabled and verify that the resulting assets in the target environment point to the mapped databases.
|
||||
|
||||
### Implementation for User Story 3
|
||||
|
||||
- [ ] T017 [US3] Implement ZIP-based asset interception and YAML transformation logic in `backend/src/core/migration_engine.py`
|
||||
- [ ] T018 [US3] Integrate database mapping application into the migration job execution flow in `backend/src/core/task_manager.py`
|
||||
- [ ] T019 [P] [US3] Create `MissingMappingModal.svelte` for on-the-fly mapping prompts in `frontend/src/components/MissingMappingModal.svelte`
|
||||
- [ ] T020 [US3] Implement backend pause and frontend modal trigger for missing mappings in `backend/src/api/routes/tasks.py` and `frontend/src/components/TaskRunner.svelte`
|
||||
|
||||
**Checkpoint**: All user stories should now be independently functional.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Polish & Cross-Cutting Concerns
|
||||
|
||||
**Purpose**: Improvements that affect multiple user stories
|
||||
|
||||
- [ ] T021 [P] Update documentation in `docs/` to include database mapping instructions
|
||||
- [ ] T022 Code cleanup and refactoring of migration logic
|
||||
- [ ] T023 [P] Performance optimization for fuzzy matching and ZIP processing
|
||||
- [ ] T024 Run `quickstart.md` validation to ensure end-to-end flow works as documented
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
### Phase Dependencies
|
||||
|
||||
- **Setup (Phase 1)**: No dependencies - can start immediately
|
||||
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
|
||||
- **User Stories (Phase 3+)**: All depend on Foundational phase completion
|
||||
- User stories can then proceed in parallel (if staffed)
|
||||
- Or sequentially in priority order (P1 → P2 → P3)
|
||||
- **Polish (Final Phase)**: Depends on all desired user stories being complete
|
||||
|
||||
### User Story Dependencies
|
||||
|
||||
- **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
|
||||
- **User Story 2 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
|
||||
- **User Story 3 (P2)**: Can start after Foundational (Phase 2) - Depends on US1/US2 for mapping data and configuration
|
||||
|
||||
### Within Each User Story
|
||||
|
||||
- Models before services
|
||||
- Services before endpoints
|
||||
- Core implementation before integration
|
||||
- Story complete before moving to next priority
|
||||
|
||||
### Parallel Opportunities
|
||||
|
||||
- All Setup tasks marked [P] can run in parallel
|
||||
- All Foundational tasks marked [P] can run in parallel (within Phase 2)
|
||||
- Once Foundational phase completes, US1 and US2 can start in parallel
|
||||
- Models and UI components within a story marked [P] can run in parallel
|
||||
|
||||
---
|
||||
|
||||
## Parallel Example: User Story 2
|
||||
|
||||
```bash
|
||||
# Launch backend and frontend components for User Story 2 together:
|
||||
Task: "Implement database mapping CRUD API endpoints in backend/src/api/routes/mappings.py"
|
||||
Task: "Create MappingTable.svelte component for displaying and editing pairs in frontend/src/components/MappingTable.svelte"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### MVP First (User Story 1 & 2)
|
||||
|
||||
1. Complete Phase 1: Setup
|
||||
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
|
||||
3. Complete Phase 3: User Story 1
|
||||
4. Complete Phase 4: User Story 2
|
||||
5. **STOP and VALIDATE**: Test environment selection and mapping management independently
|
||||
6. Deploy/demo if ready
|
||||
|
||||
### Incremental Delivery
|
||||
|
||||
1. Complete Setup + Foundational → Foundation ready
|
||||
2. Add User Story 1 → Test independently → Deploy/Demo
|
||||
3. Add User Story 2 → Test independently → Deploy/Demo (MVP!)
|
||||
4. Add User Story 3 → Test independently → Deploy/Demo
|
||||
5. Each story adds value without breaking previous stories
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- [P] tasks = different files, no dependencies
|
||||
- [Story] label maps task to specific user story for traceability
|
||||
- Each user story should be independently completable and testable
|
||||
- Commit after each task or logical group
|
||||
- Stop at any checkpoint to validate story independently
|
||||
- Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence
|
||||
Reference in New Issue
Block a user