523 lines
17 KiB
Markdown
523 lines
17 KiB
Markdown
# Tasks: Migration UI Improvements
|
|
|
|
**Feature**: Migration UI Improvements
|
|
**Branch**: `008-migration-ui-improvements`
|
|
**Generated**: 2025-12-27
|
|
**Status**: Ready for Implementation
|
|
|
|
## Overview
|
|
|
|
This document provides actionable, dependency-ordered tasks for implementing the Migration UI Improvements feature. All tasks follow the strict checklist format and are organized by user story for independent implementation and testing.
|
|
|
|
## Dependencies & Execution Order
|
|
|
|
### Phase 1: Setup (Project Initialization)
|
|
- **Goal**: Initialize project structure and verify prerequisites
|
|
- **Dependencies**: None
|
|
- **Test Criteria**: Project structure exists, all dependencies available
|
|
|
|
### Phase 2: Foundational (Blocking Prerequisites)
|
|
- **Goal**: Extend core components to support new functionality
|
|
- **Dependencies**: Phase 1 complete
|
|
- **Test Criteria**: Core extensions work independently
|
|
|
|
### Phase 3: User Story 1 - Task History and Status (P1)
|
|
- **Goal**: Display list of recent migration tasks with status
|
|
- **Dependencies**: Phase 2 complete
|
|
- **Test Criteria**: Start a migration, verify it appears in task list with correct status
|
|
|
|
### Phase 4: User Story 2 - Task Log Inspection (P2)
|
|
- **Goal**: View detailed logs for any migration task
|
|
- **Dependencies**: Phase 3 complete (uses task list)
|
|
- **Test Criteria**: Click "View Logs" button, see modal with task log entries
|
|
|
|
### Phase 5: User Story 3 - Interactive Password Resolution (P1)
|
|
- **Goal**: Handle missing database password errors interactively
|
|
- **Dependencies**: Phase 2 complete
|
|
- **Test Criteria**: Trigger migration with missing password, verify UI prompts instead of failing
|
|
|
|
### Phase 6: Polish & Cross-Cutting Concerns
|
|
- **Goal**: Finalize integration, add tests, documentation
|
|
- **Dependencies**: All previous phases complete
|
|
- **Test Criteria**: All user stories work together, tests pass
|
|
|
|
## Parallel Execution Opportunities
|
|
|
|
**Within Phase 3 (US1)**:
|
|
- Backend API endpoints can be implemented in parallel with frontend components
|
|
- Task list API and Task log API are independent
|
|
|
|
**Within Phase 5 (US3)**:
|
|
- Password prompt UI can be developed independently of backend resumption logic
|
|
- Error detection in MigrationPlugin can be tested separately
|
|
|
|
---
|
|
|
|
## Phase 1: Setup (Project Initialization)
|
|
|
|
- [ ] T001 Verify project structure and create missing directories
|
|
- Check backend/src/api/routes/ exists
|
|
- Check backend/src/models/ exists
|
|
- Check frontend/src/components/ exists
|
|
- Check frontend/src/services/ exists
|
|
- Create any missing directories per plan.md structure
|
|
|
|
- [ ] T002 Verify Python 3.9+ and Node.js 18+ dependencies
|
|
- Check Python version >= 3.9
|
|
- Check Node.js version >= 18
|
|
- Verify FastAPI, Pydantic, SQLAlchemy installed
|
|
- Verify SvelteKit, Tailwind CSS configured
|
|
|
|
- [ ] T003 Initialize task tracking for this implementation
|
|
- Create implementation log in backend/logs/implementation.log
|
|
- Document start time and initial state
|
|
|
|
---
|
|
|
|
## Phase 2: Foundational (Blocking Prerequisites)
|
|
|
|
### Backend Core Extensions
|
|
|
|
- [ ] T004 [P] Extend TaskStatus enum in backend/src/core/task_manager.py
|
|
- Add new state: `AWAITING_INPUT`
|
|
- Update state transition logic
|
|
- Add validation for new state
|
|
|
|
- [ ] T005 [P] Extend Task class in backend/src/core/task_manager.py
|
|
- Add `input_required: bool` field
|
|
- Add `input_request: Dict | None` field
|
|
- Add `logs: List[LogEntry]` field
|
|
- Update constructor and validation
|
|
|
|
- [ ] T006 [P] Implement task history retrieval in TaskManager
|
|
- Add `get_tasks(limit, offset, status)` method
|
|
- Add `get_task_logs(task_id)` method
|
|
- Add `persist_awaiting_input_tasks()` method
|
|
- Add `load_persisted_tasks()` method
|
|
|
|
- [ ] T007 [P] Create SQLite schema for persistent tasks
|
|
- Create migration script for `persistent_tasks` table
|
|
- Add indexes for status and created_at
|
|
- Test schema creation
|
|
|
|
- [ ] T008 [P] Extend MigrationPlugin error handling
|
|
- Add pattern matching for Superset password errors
|
|
- Detect "Must provide a password for the database" message
|
|
- Extract database name from error context
|
|
- Transition task to AWAITING_INPUT state
|
|
|
|
- [ ] T009 [P] Implement password injection mechanism
|
|
- Add method to resume task with credentials
|
|
- Validate password format
|
|
- Handle multiple database passwords
|
|
- Update task context with credentials
|
|
|
|
### Backend API Routes
|
|
|
|
- [ ] T010 [P] Create backend/src/api/routes/tasks.py
|
|
- Create file with basic route definitions
|
|
- Add route handlers with stubbed responses
|
|
- Register router in __init__.py (covered by T011)
|
|
- Add basic error handling structure
|
|
|
|
- [ ] T011 [P] Add task routes to backend/src/api/routes/__init__.py
|
|
- Import and register tasks router
|
|
- Verify route registration
|
|
|
|
### Frontend Services
|
|
|
|
- [ ] T012 [P] Create frontend/src/services/taskService.js
|
|
- Implement `getTasks(limit, offset, status)` function
|
|
- Implement `getTaskLogs(taskId)` function
|
|
- Implement `resumeTask(taskId, passwords)` function
|
|
- Add proper error handling
|
|
|
|
### Frontend Components
|
|
|
|
- [ ] T013 [P] Create frontend/src/components/TaskHistory.svelte
|
|
- Display task list with ID, status, start time
|
|
- Add "View Logs" action button
|
|
- Handle empty state
|
|
- Support real-time updates
|
|
|
|
- [ ] T014 [P] Create frontend/src/components/TaskLogViewer.svelte
|
|
- Display modal with log entries
|
|
- Show timestamp, level, message, context
|
|
- Auto-scroll to latest logs
|
|
- Close button functionality
|
|
|
|
- [ ] T015 [P] Create frontend/src/components/PasswordPrompt.svelte
|
|
- Display database name and error message
|
|
- Show password input fields (dynamic for multiple databases)
|
|
- Submit button with validation
|
|
- Error message display for invalid passwords
|
|
|
|
---
|
|
|
|
## Phase 3: User Story 1 - Task History and Status (P1)
|
|
|
|
**Goal**: As a user, I want to see a list of my recent and currently running migration tasks so that I can track progress and review past results.
|
|
|
|
**Independent Test**: Start a migration and verify it appears in a "Recent Tasks" list with its current status.
|
|
|
|
### Backend Implementation
|
|
|
|
- [ ] T016 [US1] Implement GET /api/tasks endpoint logic
|
|
- Query TaskManager for task list
|
|
- Apply limit/offset pagination
|
|
- Apply status filter if provided
|
|
- Return TaskListResponse format
|
|
|
|
- [ ] T017 [US1] Implement GET /api/tasks/{task_id}/logs endpoint logic
|
|
- Validate task_id exists
|
|
- Retrieve logs from TaskManager
|
|
- Return TaskLogResponse format
|
|
- Handle not found errors
|
|
|
|
- [ ] T018 [US1] Add task status update WebSocket support
|
|
- Extend existing WebSocket infrastructure
|
|
- Broadcast status changes to /ws/tasks/{task_id}/status
|
|
- Broadcast log updates to /ws/tasks/{task_id}/logs
|
|
|
|
### Frontend Implementation
|
|
|
|
- [ ] T019 [US1] Integrate TaskHistory component into migration page
|
|
- Add component to frontend/src/routes/migration/+page.svelte
|
|
- Fetch tasks on page load
|
|
- Handle loading state
|
|
- Display error messages
|
|
|
|
- [ ] T020 [US1] Implement real-time status updates
|
|
- Subscribe to WebSocket channel for task updates
|
|
- Update task list on status change
|
|
- Add visual indicators for running tasks
|
|
|
|
- [ ] T021 [US1] Add task list pagination
|
|
- Implement "Load More" button
|
|
- Handle offset updates
|
|
- Maintain current task list while loading more
|
|
|
|
### Testing
|
|
|
|
- [ ] T022 [US1] Test task list retrieval
|
|
- Create test migration tasks
|
|
- Verify API returns correct format
|
|
- Verify pagination works
|
|
- Verify status filtering works
|
|
|
|
- [ ] T023 [US1] Test real-time updates
|
|
- Start a migration
|
|
- Verify task appears in list
|
|
- Verify status updates in real-time
|
|
- Verify WebSocket messages are received
|
|
|
|
---
|
|
|
|
## Phase 4: User Story 2 - Task Log Inspection (P2)
|
|
|
|
**Goal**: As a user, I want to view the detailed logs of any migration task so that I can understand exactly what happened or why it failed.
|
|
|
|
**Independent Test**: Click a "View Logs" button for a task and see a modal or panel with the task's log entries.
|
|
|
|
### Backend Implementation
|
|
|
|
- [ ] T024 [P] [US2] Enhance log storage in TaskManager
|
|
- Ensure logs are retained for all task states
|
|
- Add log context preservation
|
|
- Implement log cleanup on task retention
|
|
|
|
### Frontend Implementation
|
|
|
|
- [ ] T025 [US2] Implement TaskLogViewer modal integration
|
|
- Add "View Logs" button to TaskHistory component
|
|
- Wire button to open TaskLogViewer modal
|
|
- Pass task_id to modal
|
|
- Fetch logs when modal opens
|
|
|
|
- [ ] T026 [US2] Implement log display formatting
|
|
- Color-code by log level (INFO=blue, WARNING=yellow, ERROR=red)
|
|
- Format timestamps nicely
|
|
- Display context as JSON or formatted text
|
|
- Auto-scroll to bottom on new logs
|
|
|
|
- [ ] T027 [US2] Add log refresh functionality
|
|
- Add refresh button in modal
|
|
- Poll for new logs every 5 seconds while modal open
|
|
- Show "new logs available" indicator
|
|
|
|
### Testing
|
|
|
|
- [ ] T028 [US2] Test log retrieval
|
|
- Create task with various log entries
|
|
- Verify logs are returned correctly
|
|
- Verify log context is preserved
|
|
- Test with large log files
|
|
|
|
- [ ] T029 [US2] Test log viewer UI
|
|
- Open logs for completed task
|
|
- Open logs for running task
|
|
- Verify formatting and readability
|
|
- Test refresh functionality
|
|
|
|
---
|
|
|
|
## Phase 5: User Story 3 - Interactive Password Resolution (P1)
|
|
|
|
**Goal**: As a user, I want the system to detect when a migration fails due to a missing database password and allow me to provide it interactively.
|
|
|
|
**Independent Test**: Trigger a migration that requires a DB password and verify the UI prompts for the password instead of just failing.
|
|
|
|
### Backend Implementation
|
|
|
|
- [ ] T030 [US3] Implement password error detection in MigrationPlugin
|
|
- Add error pattern matching for Superset 422 errors
|
|
- Extract database name from error message
|
|
- Create DatabasePasswordRequest object
|
|
- Transition task to AWAITING_INPUT state
|
|
|
|
- [ ] T031 [US3] Implement task resumption with passwords
|
|
- Add validation for password format
|
|
- Inject passwords into migration context
|
|
- Resume task execution from failure point
|
|
- Handle multiple database passwords
|
|
|
|
- [ ] T032 [US3] Add task persistence for AWAITING_INPUT state
|
|
- Persist task context and input_request
|
|
- Load persisted tasks on backend restart
|
|
- Clear persisted data on task completion
|
|
- Implement retention policy
|
|
|
|
### Frontend Implementation
|
|
|
|
- [ ] T033 [US3] Implement password prompt detection
|
|
- Monitor task status changes
|
|
- Detect AWAITING_INPUT state
|
|
- Show notification to user
|
|
- Update task list to show "Requires Input" indicator
|
|
|
|
- [ ] T034 [US3] Wire PasswordPrompt to task resumption
|
|
- Connect form submission to taskService.resumeTask()
|
|
- Handle success (close prompt, resume task)
|
|
- Handle failure (show error, keep prompt open)
|
|
- Support multiple database inputs
|
|
|
|
- [ ] T035 [US3] Add visual indicators for password-required tasks
|
|
- Highlight tasks needing input in task list
|
|
- Add badge or icon
|
|
- Show count of pending inputs
|
|
|
|
### Testing
|
|
|
|
- [ ] T036 [US3] Test password error detection
|
|
- Mock Superset password error
|
|
- Verify error is detected
|
|
- Verify task transitions to AWAITING_INPUT
|
|
- Verify DatabasePasswordRequest is created
|
|
|
|
- [ ] T037 [US3] Test password resumption
|
|
- Provide correct password
|
|
- Verify task resumes
|
|
- Verify task completes successfully
|
|
- Test with incorrect password (should prompt again)
|
|
|
|
- [ ] T038 [US3] Test persistence across restarts
|
|
- Create AWAITING_INPUT task
|
|
- Restart backend
|
|
- Verify task is loaded
|
|
- Verify password prompt still works
|
|
|
|
- [ ] T039 [US3] Test multiple database passwords
|
|
- Create migration requiring 2+ databases
|
|
- Verify all databases listed in prompt
|
|
- Verify all passwords submitted
|
|
- Verify task resumes with all credentials
|
|
|
|
---
|
|
|
|
## Phase 6: Polish & Cross-Cutting Concerns
|
|
|
|
### Integration & E2E
|
|
|
|
- [ ] T040 [P] Integrate all components on migration page
|
|
- Add TaskHistory to migration page
|
|
- Add password prompt handling
|
|
- Ensure WebSocket connections work
|
|
- Test complete user flow
|
|
|
|
- [ ] T041 [P] Add loading states and error boundaries
|
|
- Show loading spinners during API calls
|
|
- Handle API errors gracefully
|
|
- Show user-friendly error messages
|
|
- Add retry mechanisms
|
|
|
|
### Configuration & Security
|
|
|
|
- [ ] T042 [P] Add configuration options
|
|
- Task retention days (default: 30)
|
|
- Task retention limit (default: 100)
|
|
- Pagination limits (default: 10, max: 50)
|
|
- Password complexity requirements
|
|
|
|
- [ ] T043 [P] Security review
|
|
- Verify passwords are not logged
|
|
- Verify passwords are not stored permanently
|
|
- Verify input validation on all endpoints
|
|
- Check for SQL injection vulnerabilities
|
|
|
|
### Documentation
|
|
|
|
- [ ] T044 [P] Update API documentation
|
|
- Add new endpoints to OpenAPI spec
|
|
- Update API contract examples
|
|
- Document WebSocket channels
|
|
|
|
- [ ] T045 [P] Update quickstart guide
|
|
- Add task history section
|
|
- Add log viewing section
|
|
- Add password prompt section
|
|
- Add troubleshooting tips
|
|
|
|
### Testing & Quality
|
|
|
|
- [ ] T046 [P] Write unit tests for backend
|
|
- Test TaskManager extensions
|
|
- Test MigrationPlugin error detection
|
|
- Test API endpoints
|
|
- Test password validation
|
|
|
|
- [ ] T047 [P] Write unit tests for frontend
|
|
- Test taskService functions
|
|
- Test TaskHistory component
|
|
- Test TaskLogViewer component
|
|
- Test PasswordPrompt component
|
|
|
|
- [ ] T048 [P] Write integration tests
|
|
- Test complete password flow
|
|
- Test task persistence
|
|
- Test WebSocket updates
|
|
- Test error recovery
|
|
|
|
- [ ] T049 [P] Run full test suite
|
|
- Execute pytest for backend
|
|
- Execute frontend tests
|
|
- Fix any failing tests
|
|
- Verify all acceptance criteria met
|
|
|
|
- [ ] T050 [P] Final validation
|
|
- Verify all user stories work independently
|
|
- Verify all acceptance scenarios pass
|
|
- Check performance (pagination, real-time updates)
|
|
- Review code quality and security
|
|
|
|
---
|
|
|
|
## Implementation Strategy
|
|
|
|
### MVP Approach (Recommended)
|
|
|
|
**Focus on User Story 1 (P1) first**:
|
|
1. Complete Phase 1 (Setup)
|
|
2. Complete Phase 2 (Foundational) - Backend only
|
|
3. Complete Phase 3 (US1) - Task History
|
|
4. **MVP Complete**: Users can view task list and status
|
|
|
|
**Then add User Story 3 (P1)**:
|
|
5. Complete Phase 5 (US3) - Password Resolution
|
|
6. **Enhanced MVP**: Users can handle password errors
|
|
|
|
**Finally add User Story 2 (P2)**:
|
|
7. Complete Phase 4 (US2) - Log Inspection
|
|
8. **Full Feature**: Complete UI improvements
|
|
|
|
### Parallel Development
|
|
|
|
**Backend Team**:
|
|
- T004-T009 (Core extensions)
|
|
- T010-T011 (API routes)
|
|
- T016-T018 (US1 backend)
|
|
- T024 (US2 backend)
|
|
- T030-T032 (US3 backend)
|
|
|
|
**Frontend Team**:
|
|
- T012 (Services)
|
|
- T013-T015 (Components)
|
|
- T019-T021 (US1 frontend)
|
|
- T025-T027 (US2 frontend)
|
|
- T033-T035 (US3 frontend)
|
|
|
|
**Testing Team**:
|
|
- T022-T023 (US1 tests)
|
|
- T028-T029 (US2 tests)
|
|
- T036-T039 (US3 tests)
|
|
- T046-T050 (Integration & final)
|
|
|
|
### Risk Mitigation
|
|
|
|
1. **Superset API Changes**: Pattern matching may need updates if Superset changes error format
|
|
- Mitigation: Make pattern matching configurable
|
|
|
|
2. **WebSocket Scalability**: Real-time updates may impact performance with many users
|
|
- Mitigation: Use polling fallback, implement rate limiting
|
|
|
|
3. **Password Security**: Handling sensitive data requires careful implementation
|
|
- Mitigation: Never log passwords, clear from memory after use, use secure transport
|
|
|
|
4. **Task Persistence**: SQLite may not scale for high-volume task history
|
|
- Mitigation: Configurable retention, consider migration to full database later
|
|
|
|
---
|
|
|
|
## Success Criteria Validation
|
|
|
|
### User Story 1 - Task History and Status
|
|
- [ ] SC-001: Users can view status of last 10 migration tasks on migration page
|
|
- [ ] Real-time status updates work
|
|
- [ ] Task list shows ID, status, start time, actions
|
|
|
|
### User Story 2 - Task Log Inspection
|
|
- [ ] SC-002: Users can access full log in less than 2 clicks
|
|
- [ ] Log viewer shows timestamped messages
|
|
- [ ] Modal/panel works correctly
|
|
|
|
### User Story 3 - Interactive Password Resolution
|
|
- [ ] SC-003: 100% of "Missing Password" errors caught and presented as prompts
|
|
- [ ] SC-004: Users can resume blocked migration by providing password
|
|
- [ ] Task state transitions work correctly
|
|
- [ ] Multiple database passwords supported
|
|
|
|
### Cross-Cutting Requirements
|
|
- [ ] All API endpoints follow contract specifications
|
|
- [ ] All components follow existing patterns
|
|
- [ ] Security requirements met
|
|
- [ ] Performance acceptable (pagination, real-time updates)
|
|
- [ ] Configuration options available
|
|
- [ ] Documentation complete
|
|
|
|
---
|
|
|
|
## Task Summary
|
|
|
|
**Total Tasks**: 50
|
|
**Setup Phase**: 3 tasks
|
|
**Foundational Phase**: 12 tasks
|
|
**US1 Phase**: 8 tasks
|
|
**US2 Phase**: 6 tasks
|
|
**US3 Phase**: 10 tasks
|
|
**Polish Phase**: 11 tasks
|
|
|
|
**Parallel Opportunities**: 15+ tasks can be developed in parallel within their phases
|
|
|
|
**MVP Scope**: Tasks 1-23 (Setup, Foundational, US1) - ~23 tasks
|
|
**Full Feature**: All 50 tasks
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
1. **Review this tasks.md** with team leads
|
|
2. **Assign tasks** to backend/frontend developers
|
|
3. **Start Phase 1** immediately (Setup)
|
|
4. **Schedule daily standups** to track progress
|
|
5. **Use this document** as the single source of truth for implementation
|
|
|
|
**Ready for Implementation**: ✅ All design artifacts complete, tasks generated, dependencies mapped. |