- Add SQLite database integration for environments and mappings - Update TaskManager to support pausing tasks (AWAITING_MAPPING) - Modify MigrationPlugin to detect missing mappings and wait for resolution - Add frontend UI for handling missing mappings interactively - Create dedicated migration routes and API endpoints - Update .gitignore and project documentation
8.0 KiB
description
| description |
|---|
| Task list for Migration Process and UI Redesign implementation |
Tasks: Migration Process and UI Redesign
Input: Design documents from specs/001-migration-ui-redesign/
Prerequisites: plan.md (required), spec.md (required for user stories), research.md, data-model.md, quickstart.md
Tests: Tests are NOT explicitly requested in the feature specification, so they are omitted from this task list.
Organization: Tasks are grouped by user story to enable independent implementation and testing of each story.
Format: [ID] [P?] [Story] Description
- [P]: Can run in parallel (different files, no dependencies)
- [Story]: Which user story this task belongs to (e.g., US1, US2, US3)
- Include exact file paths in descriptions
Path Conventions
- Web app:
backend/src/,frontend/src/
Phase 1: Setup (Shared Infrastructure)
Purpose: Project initialization and basic structure
- T001 Create project structure per implementation plan in
backend/src/andfrontend/src/ - T002 [P] Install backend dependencies (rapidfuzz, sqlalchemy) in
backend/requirements.txt - T003 [P] Install frontend dependencies (if any new) in
frontend/package.json
Phase 2: Foundational (Blocking Prerequisites)
Purpose: Core infrastructure that MUST be complete before ANY user story can be implemented
⚠️ CRITICAL: No user story work can begin until this phase is complete
- T004 Setup SQLite database schema and SQLAlchemy models in
backend/src/models/mapping.py - T005 [P] Implement fuzzy matching utility using RapidFuzz in
backend/src/core/utils/matching.py - T006 [P] Extend SupersetClient to support database listing and metadata fetching in
backend/src/core/superset_client.py - T007 Configure database mapping persistence layer in
backend/src/core/database.py
Checkpoint: Foundation ready - user story implementation can now begin in parallel
Phase 3: User Story 1 - Environment-Based Migration Setup (Priority: P1) 🎯 MVP
Goal: Enable selection of source and target environments and toggle database replacement.
Independent Test: Open the migration page and verify that the "Source" and "Target" dropdowns are populated with configured environments and can be selected.
Implementation for User Story 1
- T008 [P] [US1] Implement environment selection API endpoints in
backend/src/api/routes/environments.py - T009 [P] [US1] Create
EnvSelector.sveltecomponent for source/target selection infrontend/src/components/EnvSelector.svelte - T010 [US1] Integrate
EnvSelectorand "Replace Database" toggle into migration dashboard infrontend/src/routes/migration/+page.svelte - T011 [US1] Add validation to ensure source and target environments are different in
frontend/src/routes/migration/+page.svelte
Checkpoint: At this point, User Story 1 should be fully functional and testable independently.
Phase 4: User Story 2 - Database Mapping Management (Priority: P1)
Goal: Fetch databases from environments, suggest mappings using fuzzy matching, and allow manual overrides/persistence.
Independent Test: Navigate to the "Database Mapping" tab, fetch databases, and verify that mappings can be created, saved, and edited.
Implementation for User Story 2
- T012 [P] [US2] Implement database mapping CRUD API endpoints in
backend/src/api/routes/mappings.py - T013 [US2] Implement mapping service with fuzzy matching logic in
backend/src/services/mapping_service.py - T014 [P] [US2] Create
MappingTable.sveltecomponent for displaying and editing pairs infrontend/src/components/MappingTable.svelte - T015 [US2] Create database mapping management view in
frontend/src/routes/migration/mappings/+page.svelte - T016 [US2] Implement "Fetch Databases" action and suggestion highlighting in
frontend/src/routes/migration/mappings/+page.svelte
Checkpoint: At this point, User Stories 1 AND 2 should both work independently.
Phase 5: User Story 3 - Migration with Automated DB Replacement (Priority: P2)
Goal: Intercept assets during migration, apply database mappings, and prompt for missing ones.
Independent Test: Run a migration with "Replace Database" enabled and verify that the resulting assets in the target environment point to the mapped databases.
Implementation for User Story 3
- T017 [US3] Implement ZIP-based asset interception and YAML transformation logic in
backend/src/core/migration_engine.py - T018 [US3] Integrate database mapping application into the migration job execution flow in
backend/src/core/task_manager.py - T019 [P] [US3] Create
MissingMappingModal.sveltefor on-the-fly mapping prompts infrontend/src/components/MissingMappingModal.svelte - T020 [US3] Implement backend pause and frontend modal trigger for missing mappings in
backend/src/api/routes/tasks.pyandfrontend/src/components/TaskRunner.svelte
Checkpoint: All user stories should now be independently functional.
Phase 6: Polish & Cross-Cutting Concerns
Purpose: Improvements that affect multiple user stories
- T021 [P] Update documentation in
docs/to include database mapping instructions - T022 Code cleanup and refactoring of migration logic
- T023 [P] Performance optimization for fuzzy matching and ZIP processing
- T024 Run
quickstart.mdvalidation to ensure end-to-end flow works as documented
Dependencies & Execution Order
Phase Dependencies
- Setup (Phase 1): No dependencies - can start immediately
- Foundational (Phase 2): Depends on Setup completion - BLOCKS all user stories
- User Stories (Phase 3+): All depend on Foundational phase completion
- User stories can then proceed in parallel (if staffed)
- Or sequentially in priority order (P1 → P2 → P3)
- Polish (Final Phase): Depends on all desired user stories being complete
User Story Dependencies
- User Story 1 (P1): Can start after Foundational (Phase 2) - No dependencies on other stories
- User Story 2 (P1): Can start after Foundational (Phase 2) - No dependencies on other stories
- User Story 3 (P2): Can start after Foundational (Phase 2) - Depends on US1/US2 for mapping data and configuration
Within Each User Story
- Models before services
- Services before endpoints
- Core implementation before integration
- Story complete before moving to next priority
Parallel Opportunities
- All Setup tasks marked [P] can run in parallel
- All Foundational tasks marked [P] can run in parallel (within Phase 2)
- Once Foundational phase completes, US1 and US2 can start in parallel
- Models and UI components within a story marked [P] can run in parallel
Parallel Example: User Story 2
# Launch backend and frontend components for User Story 2 together:
Task: "Implement database mapping CRUD API endpoints in backend/src/api/routes/mappings.py"
Task: "Create MappingTable.svelte component for displaying and editing pairs in frontend/src/components/MappingTable.svelte"
Implementation Strategy
MVP First (User Story 1 & 2)
- Complete Phase 1: Setup
- Complete Phase 2: Foundational (CRITICAL - blocks all stories)
- Complete Phase 3: User Story 1
- Complete Phase 4: User Story 2
- STOP and VALIDATE: Test environment selection and mapping management independently
- Deploy/demo if ready
Incremental Delivery
- Complete Setup + Foundational → Foundation ready
- Add User Story 1 → Test independently → Deploy/Demo
- Add User Story 2 → Test independently → Deploy/Demo (MVP!)
- Add User Story 3 → Test independently → Deploy/Demo
- Each story adds value without breaking previous stories
Notes
- [P] tasks = different files, no dependencies
- [Story] label maps task to specific user story for traceability
- Each user story should be independently completable and testable
- Commit after each task or logical group
- Stop at any checkpoint to validate story independently
- Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence