Password promt

This commit is contained in:
2025-12-30 17:21:12 +03:00
parent 4c9d554432
commit a032fe8457
20 changed files with 834 additions and 176 deletions

View File

@@ -1,6 +1,6 @@
# API Contracts: Migration UI Improvements
**Date**: 2025-12-27 | **Status**: Draft
**Date**: 2025-12-27 | **Status**: Implemented
## Overview
@@ -18,7 +18,7 @@ All endpoints require authentication using the existing session mechanism.
### 1. List Migration Tasks
**Endpoint**: `GET /tasks`
**Endpoint**: `GET /api/tasks`
**Purpose**: Retrieve a paginated list of migration tasks
@@ -26,7 +26,7 @@ All endpoints require authentication using the existing session mechanism.
```
limit: integer (query, optional) - Number of tasks to return (default: 10, max: 50)
offset: integer (query, optional) - Pagination offset (default: 0)
status: string (query, optional) - Filter by task status (PENDING, RUNNING, SUCCESS, FAILED, AWAITING_INPUT)
status: string (query, optional) - Filter by task status (PENDING, RUNNING, SUCCESS, FAILED, AWAITING_INPUT, AWAITING_MAPPING)
```
**Response**: `200 OK`

View File

@@ -55,20 +55,20 @@ This document provides actionable, dependency-ordered tasks for implementing the
## Phase 1: Setup (Project Initialization)
- [ ] T001 Verify project structure and create missing directories
- [x] T001 Verify project structure and create missing directories
- Check backend/src/api/routes/ exists
- Check backend/src/models/ exists
- Check frontend/src/components/ exists
- Check frontend/src/services/ exists
- Create any missing directories per plan.md structure
- [ ] T002 Verify Python 3.9+ and Node.js 18+ dependencies
- [x] T002 Verify Python 3.9+ and Node.js 18+ dependencies
- Check Python version >= 3.9
- Check Node.js version >= 18
- Verify FastAPI, Pydantic, SQLAlchemy installed
- Verify SvelteKit, Tailwind CSS configured
- [ ] T003 Initialize task tracking for this implementation
- [x] T003 Initialize task tracking for this implementation
- Create implementation log in backend/logs/implementation.log
- Document start time and initial state
@@ -78,35 +78,35 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Backend Core Extensions
- [ ] T004 [P] Extend TaskStatus enum in backend/src/core/task_manager.py
- [x] T004 [P] Extend TaskStatus enum in backend/src/core/task_manager/models.py
- Add new state: `AWAITING_INPUT`
- Update state transition logic
- Add validation for new state
- [ ] T005 [P] Extend Task class in backend/src/core/task_manager.py
- [x] T005 [P] Extend Task class in backend/src/core/task_manager/models.py
- Add `input_required: bool` field
- Add `input_request: Dict | None` field
- Add `logs: List[LogEntry]` field
- Update constructor and validation
- [ ] T006 [P] Implement task history retrieval in TaskManager
- [x] T006 [P] Implement task history retrieval in TaskManager (backend/src/core/task_manager/manager.py)
- Add `get_tasks(limit, offset, status)` method
- Add `get_task_logs(task_id)` method
- Add `persist_awaiting_input_tasks()` method
- Add `load_persisted_tasks()` method
- [ ] T007 [P] Create SQLite schema for persistent tasks
- Create migration script for `persistent_tasks` table
- Add indexes for status and created_at
- Test schema creation
- [x] T007 [P] Verify/Update SQLite schema in backend/src/core/task_manager/persistence.py
- Ensure `persistent_tasks` table exists with required fields
- Add indexes for status and created_at if missing
- Verify schema creation in `_ensure_db_exists`
- [ ] T008 [P] Extend MigrationPlugin error handling
- [x] T008 [P] Extend MigrationPlugin error handling
- Add pattern matching for Superset password errors
- Detect "Must provide a password for the database" message
- Extract database name from error context
- Transition task to AWAITING_INPUT state
- [ ] T009 [P] Implement password injection mechanism
- [x] T009 [P] Implement password injection mechanism
- Add method to resume task with credentials
- Validate password format
- Handle multiple database passwords
@@ -114,19 +114,19 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Backend API Routes
- [ ] T010 [P] Create backend/src/api/routes/tasks.py
- [x] T010 [P] Create backend/src/api/routes/tasks.py
- Create file with basic route definitions
- Add route handlers with stubbed responses
- Register router in __init__.py (covered by T011)
- Add basic error handling structure
- [ ] T011 [P] Add task routes to backend/src/api/routes/__init__.py
- [x] T011 [P] Add task routes to backend/src/api/routes/__init__.py
- Import and register tasks router
- Verify route registration
### Frontend Services
- [ ] T012 [P] Create frontend/src/services/taskService.js
- [x] T012 [P] Create frontend/src/services/taskService.js
- Implement `getTasks(limit, offset, status)` function
- Implement `getTaskLogs(taskId)` function
- Implement `resumeTask(taskId, passwords)` function
@@ -134,19 +134,19 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Frontend Components
- [ ] T013 [P] Create frontend/src/components/TaskHistory.svelte
- [x] T013 [P] Create frontend/src/components/TaskHistory.svelte
- Display task list with ID, status, start time
- Add "View Logs" action button
- Handle empty state
- Support real-time updates
- [ ] T014 [P] Create frontend/src/components/TaskLogViewer.svelte
- [x] T014 [P] Create frontend/src/components/TaskLogViewer.svelte
- Display modal with log entries
- Show timestamp, level, message, context
- Auto-scroll to latest logs
- Close button functionality
- [ ] T015 [P] Create frontend/src/components/PasswordPrompt.svelte
- [x] T015 [P] Create frontend/src/components/PasswordPrompt.svelte
- Display database name and error message
- Show password input fields (dynamic for multiple databases)
- Submit button with validation
@@ -162,50 +162,50 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Backend Implementation
- [ ] T016 [US1] Implement GET /api/tasks endpoint logic
- [x] T016 [US1] Implement GET /api/tasks endpoint logic
- Query TaskManager for task list
- Apply limit/offset pagination
- Apply status filter if provided
- Return TaskListResponse format
- [ ] T017 [US1] Implement GET /api/tasks/{task_id}/logs endpoint logic
- [x] T017 [US1] Implement GET /api/tasks/{task_id}/logs endpoint logic
- Validate task_id exists
- Retrieve logs from TaskManager
- Return TaskLogResponse format
- Handle not found errors
- [ ] T018 [US1] Add task status update WebSocket support
- [x] T018 [US1] Add task status update WebSocket support
- Extend existing WebSocket infrastructure
- Broadcast status changes to /ws/tasks/{task_id}/status
- Broadcast log updates to /ws/tasks/{task_id}/logs
### Frontend Implementation
- [ ] T019 [US1] Integrate TaskHistory component into migration page
- [x] T019 [US1] Integrate TaskHistory component into migration page
- Add component to frontend/src/routes/migration/+page.svelte
- Fetch tasks on page load
- Handle loading state
- Display error messages
- [ ] T020 [US1] Implement real-time status updates
- [x] T020 [US1] Implement real-time status updates
- Subscribe to WebSocket channel for task updates
- Update task list on status change
- Add visual indicators for running tasks
- [ ] T021 [US1] Add task list pagination
- [x] T021 [US1] Add task list pagination
- Implement "Load More" button
- Handle offset updates
- Maintain current task list while loading more
### Testing
- [ ] T022 [US1] Test task list retrieval
- [x] T022 [US1] Test task list retrieval
- Create test migration tasks
- Verify API returns correct format
- Verify pagination works
- Verify status filtering works
- [ ] T023 [US1] Test real-time updates
- [x] T023 [US1] Test real-time updates
- Start a migration
- Verify task appears in list
- Verify status updates in real-time
@@ -221,39 +221,39 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Backend Implementation
- [ ] T024 [P] [US2] Enhance log storage in TaskManager
- [x] T024 [P] [US2] Enhance log storage in TaskManager
- Ensure logs are retained for all task states
- Add log context preservation
- Implement log cleanup on task retention
### Frontend Implementation
- [ ] T025 [US2] Implement TaskLogViewer modal integration
- [x] T025 [US2] Implement TaskLogViewer modal integration
- Add "View Logs" button to TaskHistory component
- Wire button to open TaskLogViewer modal
- Pass task_id to modal
- Fetch logs when modal opens
- [ ] T026 [US2] Implement log display formatting
- [x] T026 [US2] Implement log display formatting
- Color-code by log level (INFO=blue, WARNING=yellow, ERROR=red)
- Format timestamps nicely
- Display context as JSON or formatted text
- Auto-scroll to bottom on new logs
- [ ] T027 [US2] Add log refresh functionality
- [x] T027 [US2] Add log refresh functionality
- Add refresh button in modal
- Poll for new logs every 5 seconds while modal open
- Show "new logs available" indicator
### Testing
- [ ] T028 [US2] Test log retrieval
- [x] T028 [US2] Test log retrieval
- Create task with various log entries
- Verify logs are returned correctly
- Verify log context is preserved
- Test with large log files
- [ ] T029 [US2] Test log viewer UI
- [x] T029 [US2] Test log viewer UI
- Open logs for completed task
- Open logs for running task
- Verify formatting and readability
@@ -269,19 +269,19 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Backend Implementation
- [ ] T030 [US3] Implement password error detection in MigrationPlugin
- [x] T030 [US3] Implement password error detection in MigrationPlugin
- Add error pattern matching for Superset 422 errors
- Extract database name from error message
- Create DatabasePasswordRequest object
- Transition task to AWAITING_INPUT state
- [ ] T031 [US3] Implement task resumption with passwords
- [x] T031 [US3] Implement task resumption with passwords
- Add validation for password format
- Inject passwords into migration context
- Resume task execution from failure point
- Handle multiple database passwords
- [ ] T032 [US3] Add task persistence for AWAITING_INPUT state
- [x] T032 [US3] Add task persistence for AWAITING_INPUT state
- Persist task context and input_request
- Load persisted tasks on backend restart
- Clear persisted data on task completion
@@ -289,44 +289,44 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Frontend Implementation
- [ ] T033 [US3] Implement password prompt detection
- [x] T033 [US3] Implement password prompt detection
- Monitor task status changes
- Detect AWAITING_INPUT state
- Show notification to user
- Update task list to show "Requires Input" indicator
- [ ] T034 [US3] Wire PasswordPrompt to task resumption
- [x] T034 [US3] Wire PasswordPrompt to task resumption
- Connect form submission to taskService.resumeTask()
- Handle success (close prompt, resume task)
- Handle failure (show error, keep prompt open)
- Support multiple database inputs
- [ ] T035 [US3] Add visual indicators for password-required tasks
- [x] T035 [US3] Add visual indicators for password-required tasks
- Highlight tasks needing input in task list
- Add badge or icon
- Show count of pending inputs
### Testing
- [ ] T036 [US3] Test password error detection
- [x] T036 [US3] Test password error detection
- Mock Superset password error
- Verify error is detected
- Verify task transitions to AWAITING_INPUT
- Verify DatabasePasswordRequest is created
- [ ] T037 [US3] Test password resumption
- [x] T037 [US3] Test password resumption
- Provide correct password
- Verify task resumes
- Verify task completes successfully
- Test with incorrect password (should prompt again)
- [ ] T038 [US3] Test persistence across restarts
- [x] T038 [US3] Test persistence across restarts
- Create AWAITING_INPUT task
- Restart backend
- Verify task is loaded
- Verify password prompt still works
- [ ] T039 [US3] Test multiple database passwords
- [x] T039 [US3] Test multiple database passwords
- Create migration requiring 2+ databases
- Verify all databases listed in prompt
- Verify all passwords submitted
@@ -338,13 +338,13 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Integration & E2E
- [ ] T040 [P] Integrate all components on migration page
- [x] T040 [P] Integrate all components on migration page
- Add TaskHistory to migration page
- Add password prompt handling
- Ensure WebSocket connections work
- Test complete user flow
- [ ] T041 [P] Add loading states and error boundaries
- [x] T041 [P] Add loading states and error boundaries
- Show loading spinners during API calls
- Handle API errors gracefully
- Show user-friendly error messages
@@ -352,13 +352,14 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Configuration & Security
- [ ] T042 [P] Add configuration options
- Task retention days (default: 30)
- Task retention limit (default: 100)
- Pagination limits (default: 10, max: 50)
- Password complexity requirements
- [x] T042 [P] Add configuration options to backend/src/core/config_models.py
- Extend `GlobalSettings` with `task_retention` fields:
- `retention_days` (default: 30)
- `retention_limit` (default: 100)
- Add `pagination_limit` to settings (default: 10)
- Update `ConfigManager` to handle new fields
- [ ] T043 [P] Security review
- [x] T043 [P] Security review
- Verify passwords are not logged
- Verify passwords are not stored permanently
- Verify input validation on all endpoints
@@ -366,12 +367,12 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Documentation
- [ ] T044 [P] Update API documentation
- [x] T044 [P] Update API documentation
- Add new endpoints to OpenAPI spec
- Update API contract examples
- Document WebSocket channels
- [ ] T045 [P] Update quickstart guide
- [x] T045 [P] Update quickstart guide
- Add task history section
- Add log viewing section
- Add password prompt section
@@ -379,31 +380,31 @@ This document provides actionable, dependency-ordered tasks for implementing the
### Testing & Quality
- [ ] T046 [P] Write unit tests for backend
- [x] T046 [P] Write unit tests for backend
- Test TaskManager extensions
- Test MigrationPlugin error detection
- Test API endpoints
- Test password validation
- [ ] T047 [P] Write unit tests for frontend
- [x] T047 [P] Write unit tests for frontend
- Test taskService functions
- Test TaskHistory component
- Test TaskLogViewer component
- Test PasswordPrompt component
- [ ] T048 [P] Write integration tests
- [x] T048 [P] Write integration tests
- Test complete password flow
- Test task persistence
- Test WebSocket updates
- Test error recovery
- [ ] T049 [P] Run full test suite
- [x] T049 [P] Run full test suite
- Execute pytest for backend
- Execute frontend tests
- Fix any failing tests
- Verify all acceptance criteria met
- [ ] T050 [P] Final validation
- [x] T050 [P] Final validation
- Verify all user stories work independently
- Verify all acceptance scenarios pass
- Check performance (pagination, real-time updates)