--- description: "Task list for Migration Process and UI Redesign implementation" --- # Tasks: Migration Process and UI Redesign **Input**: Design documents from `specs/001-migration-ui-redesign/` **Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, quickstart.md **Tests**: Tests are NOT explicitly requested in the feature specification, so they are omitted from this task list. **Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story. ## Format: `[ID] [P?] [Story] Description` - **[P]**: Can run in parallel (different files, no dependencies) - **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3) - Include exact file paths in descriptions ## Path Conventions - **Web app**: `backend/src/`, `frontend/src/` --- ## Phase 1: Setup (Shared Infrastructure) **Purpose**: Project initialization and basic structure - [ ] T001 Create project structure per implementation plan in `backend/src/` and `frontend/src/` - [ ] T002 [P] Install backend dependencies (rapidfuzz, sqlalchemy) in `backend/requirements.txt` - [ ] T003 [P] Install frontend dependencies (if any new) in `frontend/package.json` --- ## Phase 2: Foundational (Blocking Prerequisites) **Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented **⚠️ CRITICAL**: No user story work can begin until this phase is complete - [ ] T004 Setup SQLite database schema and SQLAlchemy models in `backend/src/models/mapping.py` - [ ] T005 [P] Implement fuzzy matching utility using RapidFuzz in `backend/src/core/utils/matching.py` - [ ] T006 [P] Extend SupersetClient to support database listing and metadata fetching in `backend/src/core/superset_client.py` - [ ] T007 Configure database mapping persistence layer in `backend/src/core/database.py` **Checkpoint**: Foundation ready - user story implementation can now begin in parallel --- ## Phase 3: User Story 1 - Environment-Based Migration Setup (Priority: P1) 🎯 MVP **Goal**: Enable selection of source and target environments and toggle database replacement. **Independent Test**: Open the migration page and verify that the "Source" and "Target" dropdowns are populated with configured environments and can be selected. ### Implementation for User Story 1 - [ ] T008 [P] [US1] Implement environment selection API endpoints in `backend/src/api/routes/environments.py` - [ ] T009 [P] [US1] Create `EnvSelector.svelte` component for source/target selection in `frontend/src/components/EnvSelector.svelte` - [ ] T010 [US1] Integrate `EnvSelector` and "Replace Database" toggle into migration dashboard in `frontend/src/routes/migration/+page.svelte` - [ ] T011 [US1] Add validation to ensure source and target environments are different in `frontend/src/routes/migration/+page.svelte` **Checkpoint**: At this point, User Story 1 should be fully functional and testable independently. --- ## Phase 4: User Story 2 - Database Mapping Management (Priority: P1) **Goal**: Fetch databases from environments, suggest mappings using fuzzy matching, and allow manual overrides/persistence. **Independent Test**: Navigate to the "Database Mapping" tab, fetch databases, and verify that mappings can be created, saved, and edited. ### Implementation for User Story 2 - [ ] T012 [P] [US2] Implement database mapping CRUD API endpoints in `backend/src/api/routes/mappings.py` - [ ] T013 [US2] Implement mapping service with fuzzy matching logic in `backend/src/services/mapping_service.py` - [ ] T014 [P] [US2] Create `MappingTable.svelte` component for displaying and editing pairs in `frontend/src/components/MappingTable.svelte` - [ ] T015 [US2] Create database mapping management view in `frontend/src/routes/migration/mappings/+page.svelte` - [ ] T016 [US2] Implement "Fetch Databases" action and suggestion highlighting in `frontend/src/routes/migration/mappings/+page.svelte` **Checkpoint**: At this point, User Stories 1 AND 2 should both work independently. --- ## Phase 5: User Story 3 - Migration with Automated DB Replacement (Priority: P2) **Goal**: Intercept assets during migration, apply database mappings, and prompt for missing ones. **Independent Test**: Run a migration with "Replace Database" enabled and verify that the resulting assets in the target environment point to the mapped databases. ### Implementation for User Story 3 - [ ] T017 [US3] Implement ZIP-based asset interception and YAML transformation logic in `backend/src/core/migration_engine.py` - [ ] T018 [US3] Integrate database mapping application into the migration job execution flow in `backend/src/core/task_manager.py` - [ ] T019 [P] [US3] Create `MissingMappingModal.svelte` for on-the-fly mapping prompts in `frontend/src/components/MissingMappingModal.svelte` - [ ] T020 [US3] Implement backend pause and frontend modal trigger for missing mappings in `backend/src/api/routes/tasks.py` and `frontend/src/components/TaskRunner.svelte` **Checkpoint**: All user stories should now be independently functional. --- ## Phase 6: Polish & Cross-Cutting Concerns **Purpose**: Improvements that affect multiple user stories - [ ] T021 [P] Update documentation in `docs/` to include database mapping instructions - [ ] T022 Code cleanup and refactoring of migration logic - [ ] T023 [P] Performance optimization for fuzzy matching and ZIP processing - [ ] T024 Run `quickstart.md` validation to ensure end-to-end flow works as documented --- ## Dependencies & Execution Order ### Phase Dependencies - **Setup (Phase 1)**: No dependencies - can start immediately - **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories - **User Stories (Phase 3+)**: All depend on Foundational phase completion - User stories can then proceed in parallel (if staffed) - Or sequentially in priority order (P1 → P2 → P3) - **Polish (Final Phase)**: Depends on all desired user stories being complete ### User Story Dependencies - **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories - **User Story 2 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories - **User Story 3 (P2)**: Can start after Foundational (Phase 2) - Depends on US1/US2 for mapping data and configuration ### Within Each User Story - Models before services - Services before endpoints - Core implementation before integration - Story complete before moving to next priority ### Parallel Opportunities - All Setup tasks marked [P] can run in parallel - All Foundational tasks marked [P] can run in parallel (within Phase 2) - Once Foundational phase completes, US1 and US2 can start in parallel - Models and UI components within a story marked [P] can run in parallel --- ## Parallel Example: User Story 2 ```bash # Launch backend and frontend components for User Story 2 together: Task: "Implement database mapping CRUD API endpoints in backend/src/api/routes/mappings.py" Task: "Create MappingTable.svelte component for displaying and editing pairs in frontend/src/components/MappingTable.svelte" ``` --- ## Implementation Strategy ### MVP First (User Story 1 & 2) 1. Complete Phase 1: Setup 2. Complete Phase 2: Foundational (CRITICAL - blocks all stories) 3. Complete Phase 3: User Story 1 4. Complete Phase 4: User Story 2 5. **STOP and VALIDATE**: Test environment selection and mapping management independently 6. Deploy/demo if ready ### Incremental Delivery 1. Complete Setup + Foundational → Foundation ready 2. Add User Story 1 → Test independently → Deploy/Demo 3. Add User Story 2 → Test independently → Deploy/Demo (MVP!) 4. Add User Story 3 → Test independently → Deploy/Demo 5. Each story adds value without breaking previous stories --- ## Notes - [P] tasks = different files, no dependencies - [Story] label maps task to specific user story for traceability - Each user story should be independently completable and testable - Commit after each task or logical group - Stop at any checkpoint to validate story independently - Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence