# Tasks: Migration UI Improvements

**Feature**: Migration UI Improvements  
**Branch**: `008-migration-ui-improvements`  
**Generated**: 2025-12-27  
**Status**: Ready for Implementation

## Overview

This document provides actionable, dependency-ordered tasks for implementing the Migration UI Improvements feature. All tasks follow the strict checklist format and are organized by user story for independent implementation and testing.

## Dependencies & Execution Order

### Phase 1: Setup (Project Initialization)
- **Goal**: Initialize project structure and verify prerequisites
- **Dependencies**: None
- **Test Criteria**: Project structure exists, all dependencies available

### Phase 2: Foundational (Blocking Prerequisites)
- **Goal**: Extend core components to support new functionality
- **Dependencies**: Phase 1 complete
- **Test Criteria**: Core extensions work independently

### Phase 3: User Story 1 - Task History and Status (P1)
- **Goal**: Display list of recent migration tasks with status
- **Dependencies**: Phase 2 complete
- **Test Criteria**: Start a migration, verify it appears in task list with correct status

### Phase 4: User Story 2 - Task Log Inspection (P2)
- **Goal**: View detailed logs for any migration task
- **Dependencies**: Phase 3 complete (uses task list)
- **Test Criteria**: Click "View Logs" button, see modal with task log entries

### Phase 5: User Story 3 - Interactive Password Resolution (P1)
- **Goal**: Handle missing database password errors interactively
- **Dependencies**: Phase 2 complete
- **Test Criteria**: Trigger migration with missing password, verify UI prompts instead of failing

### Phase 6: Polish & Cross-Cutting Concerns
- **Goal**: Finalize integration, add tests, documentation
- **Dependencies**: All previous phases complete
- **Test Criteria**: All user stories work together, tests pass

## Parallel Execution Opportunities

**Within Phase 3 (US1)**:
- Backend API endpoints can be implemented in parallel with frontend components
- Task list API and Task log API are independent

**Within Phase 5 (US3)**:
- Password prompt UI can be developed independently of backend resumption logic
- Error detection in MigrationPlugin can be tested separately

---

## Phase 1: Setup (Project Initialization)

- [x] T001 Verify project structure and create missing directories
  - Check backend/src/api/routes/ exists
  - Check backend/src/models/ exists
  - Check frontend/src/components/ exists
  - Check frontend/src/services/ exists
  - Create any missing directories per plan.md structure

- [x] T002 Verify Python 3.9+ and Node.js 18+ dependencies
  - Check Python version >= 3.9
  - Check Node.js version >= 18
  - Verify FastAPI, Pydantic, SQLAlchemy installed
  - Verify SvelteKit, Tailwind CSS configured

- [x] T003 Initialize task tracking for this implementation
  - Create implementation log in backend/logs/implementation.log
  - Document start time and initial state

---

## Phase 2: Foundational (Blocking Prerequisites)

### Backend Core Extensions

- [x] T004 [P] Extend TaskStatus enum in backend/src/core/task_manager/models.py
  - Add new state: `AWAITING_INPUT`
  - Update state transition logic
  - Add validation for new state

- [x] T005 [P] Extend Task class in backend/src/core/task_manager/models.py
  - Add `input_required: bool` field
  - Add `input_request: Dict | None` field
  - Add `logs: List[LogEntry]` field
  - Update constructor and validation

- [x] T006 [P] Implement task history retrieval in TaskManager (backend/src/core/task_manager/manager.py)
  - Add `get_tasks(limit, offset, status)` method
  - Add `get_task_logs(task_id)` method
  - Add `persist_awaiting_input_tasks()` method
  - Add `load_persisted_tasks()` method

- [x] T007 [P] Verify/Update SQLite schema in backend/src/core/task_manager/persistence.py
  - Ensure `persistent_tasks` table exists with required fields
  - Add indexes for status and created_at if missing
  - Verify schema creation in `_ensure_db_exists`

- [x] T008 [P] Extend MigrationPlugin error handling
  - Add pattern matching for Superset password errors
  - Detect "Must provide a password for the database" message
  - Extract database name from error context
  - Transition task to AWAITING_INPUT state

- [x] T009 [P] Implement password injection mechanism
  - Add method to resume task with credentials
  - Validate password format
  - Handle multiple database passwords
  - Update task context with credentials

### Backend API Routes

- [x] T010 [P] Create backend/src/api/routes/tasks.py
   - Create file with basic route definitions
   - Add route handlers with stubbed responses
   - Register router in __init__.py (covered by T011)
   - Add basic error handling structure

- [x] T011 [P] Add task routes to backend/src/api/routes/__init__.py
  - Import and register tasks router
  - Verify route registration

### Frontend Services

- [x] T012 [P] Create frontend/src/services/taskService.js
  - Implement `getTasks(limit, offset, status)` function
  - Implement `getTaskLogs(taskId)` function
  - Implement `resumeTask(taskId, passwords)` function
  - Add proper error handling

### Frontend Components

- [x] T013 [P] Create frontend/src/components/TaskHistory.svelte
  - Display task list with ID, status, start time
  - Add "View Logs" action button
  - Handle empty state
  - Support real-time updates

- [x] T014 [P] Create frontend/src/components/TaskLogViewer.svelte
  - Display modal with log entries
  - Show timestamp, level, message, context
  - Auto-scroll to latest logs
  - Close button functionality

- [x] T015 [P] Create frontend/src/components/PasswordPrompt.svelte
  - Display database name and error message
  - Show password input fields (dynamic for multiple databases)
  - Submit button with validation
  - Error message display for invalid passwords

---

## Phase 3: User Story 1 - Task History and Status (P1)

**Goal**: As a user, I want to see a list of my recent and currently running migration tasks so that I can track progress and review past results.

**Independent Test**: Start a migration and verify it appears in a "Recent Tasks" list with its current status.

### Backend Implementation

- [x] T016 [US1] Implement GET /api/tasks endpoint logic
  - Query TaskManager for task list
  - Apply limit/offset pagination
  - Apply status filter if provided
  - Return TaskListResponse format

- [x] T017 [US1] Implement GET /api/tasks/{task_id}/logs endpoint logic
  - Validate task_id exists
  - Retrieve logs from TaskManager
  - Return TaskLogResponse format
  - Handle not found errors

- [x] T018 [US1] Add task status update WebSocket support
  - Extend existing WebSocket infrastructure
  - Broadcast status changes to /ws/tasks/{task_id}/status
  - Broadcast log updates to /ws/tasks/{task_id}/logs

### Frontend Implementation

- [x] T019 [US1] Integrate TaskHistory component into migration page
  - Add component to frontend/src/routes/migration/+page.svelte
  - Fetch tasks on page load
  - Handle loading state
  - Display error messages

- [x] T020 [US1] Implement real-time status updates
  - Subscribe to WebSocket channel for task updates
  - Update task list on status change
  - Add visual indicators for running tasks

- [x] T021 [US1] Add task list pagination
  - Implement "Load More" button
  - Handle offset updates
  - Maintain current task list while loading more

### Testing

- [x] T022 [US1] Test task list retrieval
  - Create test migration tasks
  - Verify API returns correct format
  - Verify pagination works
  - Verify status filtering works

- [x] T023 [US1] Test real-time updates
  - Start a migration
  - Verify task appears in list
  - Verify status updates in real-time
  - Verify WebSocket messages are received

---

## Phase 4: User Story 2 - Task Log Inspection (P2)

**Goal**: As a user, I want to view the detailed logs of any migration task so that I can understand exactly what happened or why it failed.

**Independent Test**: Click a "View Logs" button for a task and see a modal or panel with the task's log entries.

### Backend Implementation

- [x] T024 [P] [US2] Enhance log storage in TaskManager
  - Ensure logs are retained for all task states
  - Add log context preservation
  - Implement log cleanup on task retention

### Frontend Implementation

- [x] T025 [US2] Implement TaskLogViewer modal integration
  - Add "View Logs" button to TaskHistory component
  - Wire button to open TaskLogViewer modal
  - Pass task_id to modal
  - Fetch logs when modal opens

- [x] T026 [US2] Implement log display formatting
  - Color-code by log level (INFO=blue, WARNING=yellow, ERROR=red)
  - Format timestamps nicely
  - Display context as JSON or formatted text
  - Auto-scroll to bottom on new logs

- [x] T027 [US2] Add log refresh functionality
  - Add refresh button in modal
  - Poll for new logs every 5 seconds while modal open
  - Show "new logs available" indicator

### Testing

- [x] T028 [US2] Test log retrieval
  - Create task with various log entries
  - Verify logs are returned correctly
  - Verify log context is preserved
  - Test with large log files

- [x] T029 [US2] Test log viewer UI
  - Open logs for completed task
  - Open logs for running task
  - Verify formatting and readability
  - Test refresh functionality

---

## Phase 5: User Story 3 - Interactive Password Resolution (P1)

**Goal**: As a user, I want the system to detect when a migration fails due to a missing database password and allow me to provide it interactively.

**Independent Test**: Trigger a migration that requires a DB password and verify the UI prompts for the password instead of just failing.

### Backend Implementation

- [x] T030 [US3] Implement password error detection in MigrationPlugin
  - Add error pattern matching for Superset 422 errors
  - Extract database name from error message
  - Create DatabasePasswordRequest object
  - Transition task to AWAITING_INPUT state

- [x] T031 [US3] Implement task resumption with passwords
  - Add validation for password format
  - Inject passwords into migration context
  - Resume task execution from failure point
  - Handle multiple database passwords

- [x] T032 [US3] Add task persistence for AWAITING_INPUT state
  - Persist task context and input_request
  - Load persisted tasks on backend restart
  - Clear persisted data on task completion
  - Implement retention policy

### Frontend Implementation

- [x] T033 [US3] Implement password prompt detection
  - Monitor task status changes
  - Detect AWAITING_INPUT state
  - Show notification to user
  - Update task list to show "Requires Input" indicator

- [x] T034 [US3] Wire PasswordPrompt to task resumption
  - Connect form submission to taskService.resumeTask()
  - Handle success (close prompt, resume task)
  - Handle failure (show error, keep prompt open)
  - Support multiple database inputs

- [x] T035 [US3] Add visual indicators for password-required tasks
  - Highlight tasks needing input in task list
  - Add badge or icon
  - Show count of pending inputs

### Testing

- [x] T036 [US3] Test password error detection
  - Mock Superset password error
  - Verify error is detected
  - Verify task transitions to AWAITING_INPUT
  - Verify DatabasePasswordRequest is created

- [x] T037 [US3] Test password resumption
  - Provide correct password
  - Verify task resumes
  - Verify task completes successfully
  - Test with incorrect password (should prompt again)

- [x] T038 [US3] Test persistence across restarts
  - Create AWAITING_INPUT task
  - Restart backend
  - Verify task is loaded
  - Verify password prompt still works

- [x] T039 [US3] Test multiple database passwords
  - Create migration requiring 2+ databases
  - Verify all databases listed in prompt
  - Verify all passwords submitted
  - Verify task resumes with all credentials

---

## Phase 6: Polish & Cross-Cutting Concerns

### Integration & E2E

- [x] T040 [P] Integrate all components on migration page
  - Add TaskHistory to migration page
  - Add password prompt handling
  - Ensure WebSocket connections work
  - Test complete user flow

- [x] T041 [P] Add loading states and error boundaries
  - Show loading spinners during API calls
  - Handle API errors gracefully
  - Show user-friendly error messages
  - Add retry mechanisms

### Configuration & Security

- [x] T042 [P] Add configuration options to backend/src/core/config_models.py
  - Extend `GlobalSettings` with `task_retention` fields:
    - `retention_days` (default: 30)
    - `retention_limit` (default: 100)
  - Add `pagination_limit` to settings (default: 10)
  - Update `ConfigManager` to handle new fields

- [x] T043 [P] Security review
  - Verify passwords are not logged
  - Verify passwords are not stored permanently
  - Verify input validation on all endpoints
  - Check for SQL injection vulnerabilities

### Documentation

- [x] T044 [P] Update API documentation
  - Add new endpoints to OpenAPI spec
  - Update API contract examples
  - Document WebSocket channels

- [x] T045 [P] Update quickstart guide
  - Add task history section
  - Add log viewing section
  - Add password prompt section
  - Add troubleshooting tips

### Testing & Quality

- [x] T046 [P] Write unit tests for backend
  - Test TaskManager extensions
  - Test MigrationPlugin error detection
  - Test API endpoints
  - Test password validation

- [x] T047 [P] Write unit tests for frontend
  - Test taskService functions
  - Test TaskHistory component
  - Test TaskLogViewer component
  - Test PasswordPrompt component

- [x] T048 [P] Write integration tests
  - Test complete password flow
  - Test task persistence
  - Test WebSocket updates
  - Test error recovery

- [x] T049 [P] Run full test suite
  - Execute pytest for backend
  - Execute frontend tests
  - Fix any failing tests
  - Verify all acceptance criteria met

- [x] T050 [P] Final validation
  - Verify all user stories work independently
  - Verify all acceptance scenarios pass
  - Check performance (pagination, real-time updates)
  - Review code quality and security

---

## Implementation Strategy

### MVP Approach (Recommended)

**Focus on User Story 1 (P1) first**:
1. Complete Phase 1 (Setup)
2. Complete Phase 2 (Foundational) - Backend only
3. Complete Phase 3 (US1) - Task History
4. **MVP Complete**: Users can view task list and status

**Then add User Story 3 (P1)**:
5. Complete Phase 5 (US3) - Password Resolution
6. **Enhanced MVP**: Users can handle password errors

**Finally add User Story 2 (P2)**:
7. Complete Phase 4 (US2) - Log Inspection
8. **Full Feature**: Complete UI improvements

### Parallel Development

**Backend Team**:
- T004-T009 (Core extensions)
- T010-T011 (API routes)
- T016-T018 (US1 backend)
- T024 (US2 backend)
- T030-T032 (US3 backend)

**Frontend Team**:
- T012 (Services)
- T013-T015 (Components)
- T019-T021 (US1 frontend)
- T025-T027 (US2 frontend)
- T033-T035 (US3 frontend)

**Testing Team**:
- T022-T023 (US1 tests)
- T028-T029 (US2 tests)
- T036-T039 (US3 tests)
- T046-T050 (Integration & final)

### Risk Mitigation

1. **Superset API Changes**: Pattern matching may need updates if Superset changes error format
   - Mitigation: Make pattern matching configurable

2. **WebSocket Scalability**: Real-time updates may impact performance with many users
   - Mitigation: Use polling fallback, implement rate limiting

3. **Password Security**: Handling sensitive data requires careful implementation
   - Mitigation: Never log passwords, clear from memory after use, use secure transport

4. **Task Persistence**: SQLite may not scale for high-volume task history
   - Mitigation: Configurable retention, consider migration to full database later

---

## Success Criteria Validation

### User Story 1 - Task History and Status
- [ ] SC-001: Users can view status of last 10 migration tasks on migration page
- [ ] Real-time status updates work
- [ ] Task list shows ID, status, start time, actions

### User Story 2 - Task Log Inspection
- [ ] SC-002: Users can access full log in less than 2 clicks
- [ ] Log viewer shows timestamped messages
- [ ] Modal/panel works correctly

### User Story 3 - Interactive Password Resolution
- [ ] SC-003: 100% of "Missing Password" errors caught and presented as prompts
- [ ] SC-004: Users can resume blocked migration by providing password
- [ ] Task state transitions work correctly
- [ ] Multiple database passwords supported

### Cross-Cutting Requirements
- [ ] All API endpoints follow contract specifications
- [ ] All components follow existing patterns
- [ ] Security requirements met
- [ ] Performance acceptable (pagination, real-time updates)
- [ ] Configuration options available
- [ ] Documentation complete

---

## Task Summary

**Total Tasks**: 50  
**Setup Phase**: 3 tasks  
**Foundational Phase**: 12 tasks  
**US1 Phase**: 8 tasks  
**US2 Phase**: 6 tasks  
**US3 Phase**: 10 tasks  
**Polish Phase**: 11 tasks

**Parallel Opportunities**: 15+ tasks can be developed in parallel within their phases

**MVP Scope**: Tasks 1-23 (Setup, Foundational, US1) - ~23 tasks  
**Full Feature**: All 50 tasks

---

## Next Steps

1. **Review this tasks.md** with team leads
2. **Assign tasks** to backend/frontend developers
3. **Start Phase 1** immediately (Setup)
4. **Schedule daily standups** to track progress
5. **Use this document** as the single source of truth for implementation

**Ready for Implementation**: ✅ All design artifacts complete, tasks generated, dependencies mapped.