# Feature Specification: Migration Process and UI Redesign

**Feature Branch**: `001-migration-ui-redesign`  
**Created**: 2025-12-20  
**Status**: Draft  
**Input**: User description: "я хочу переработать процесс и интерфейс миграции. 1. Необходимо чтобы был выпадающий список enviroments (откуда и куда), а также просто галка замены БД 2. Процесс замены БД должен быть предустановленными парами , необходима отдельная вкладка которая бы считывала базы данных с источника и цели и позволяла их маппить, при этом первоначально эмпирически подставляя пары вида 'Dev Clickhouse' -> 'Prod Clickhouse'. Меппинг нужно сохранять и иметь возможность его редактировать"

## Clarifications

### Session 2025-12-20
- Q: Scope of Database Mapping → A: Map the full configuration object obtained from the Superset API.
- Q: Persistence of mappings → A: Use a SQLite database for storing mappings.
- Q: Handling of missing mappings during migration → A: Show a modal dialog during the migration process to prompt for missing mappings.
- Q: Empirical matching algorithm details → A: Use name-based fuzzy matching (ignoring common prefixes like Dev/Prod).
- Q: Scope of "Replace Database" toggle → A: Apply replacement to all assets (Dashboards, Datasets, Charts) included in the migration.
- Q: Backend exposure of Superset databases → A: Dedicated environment database endpoints (e.g., `/api/environments/{id}/databases`).
- Q: Superset API authentication → A: Use stored environment credentials from the backend.
- Q: Error handling for unreachable environments → A: Return structured error responses (502/503) with descriptive messages.
- Q: Database list filtering → A: Return all available databases with metadata (engine type, etc.).
- Q: Handling large database lists → A: Return full list (no pagination) for simplicity.

## User Scenarios & Testing *(mandatory)*

### User Story 1 - Environment-Based Migration Setup (Priority: P1)

As a migration operator, I want to easily select the source and target environments from a list so that I can quickly define the scope of my migration without manual URL entry.

**Why this priority**: This is the core interaction for starting any migration. Using predefined environments reduces errors and improves speed.

**Independent Test**: Can be tested by opening the migration page and verifying that the "Source" and "Target" dropdowns are populated with configured environments and can be selected.

**Acceptance Scenarios**:

1. **Given** multiple environments are configured in settings, **When** I open the migration page, **Then** I should see two dropdowns for "Source" and "Target" containing these environments.
2. **Given** a source and target are selected, **When** I toggle the "Replace Database" checkbox, **Then** the system should prepare to apply database mappings during the next migration step.

---

### User Story 2 - Database Mapping Management (Priority: P1)

As an administrator, I want to define how databases in my development environment map to databases in production so that my dashboards and datasets work correctly after migration.

**Why this priority**: Migrations often fail or require manual fixups because database references point to the wrong environment. Automated mapping is critical for reliable migrations.

**Independent Test**: Can be tested by navigating to the "Database Mapping" tab, fetching databases, and verifying that mappings can be created, saved, and edited.

**Acceptance Scenarios**:

1. **Given** a source and target environment are selected, **When** I open the "Database Mapping" tab, **Then** the system should fetch and display lists of databases from both environments.
2. **Given** the database lists are loaded, **When** the system identifies similar names (e.g., "Dev Clickhouse" and "Prod Clickhouse"), **Then** it should automatically suggest these as a mapping pair.
3. **Given** suggested or manual mappings, **When** I click "Save Mappings", **Then** these pairs should be persisted and associated with the selected environment pair.

---

### User Story 3 - Migration with Automated DB Replacement (Priority: P2)

As a user, I want the migration process to automatically update database references based on my saved mappings so that I don't have to manually edit exported files or post-migration settings.

**Why this priority**: This delivers the actual value of the mapping feature by automating a tedious and error-prone task.

**Independent Test**: Can be tested by running a migration with "Replace Database" enabled and verifying that the resulting assets in the target environment point to the mapped databases.

**Acceptance Scenarios**:

1. **Given** saved mappings exist for the selected environments, **When** I start a migration with "Replace Database" enabled, **Then** the system should replace all source database IDs/names with their corresponding target values during the transfer.
2. **Given** "Replace Database" is enabled but a source database has no mapping, **When** the migration runs, **Then** the system should pause and show a modal dialog prompting the user to provide a mapping on-the-fly for the missing database.

---

### Edge Cases

- **Environment Connectivity**: If the source or target environment is unreachable, the backend MUST return a structured error (502/503), and the frontend MUST display a clear connection error with a retry option.
- **Duplicate Mappings**: How does the system handle multiple source databases mapping to the same target database? (Assumption: This is allowed, as multiple dev DBs might consolidate into one prod DB).
- **Missing Target Database**: What if a mapped target database no longer exists in the target environment? (Assumption: Validation should occur before migration starts, highlighting broken mappings).

## Requirements *(mandatory)*

### Functional Requirements

- **FR-001**: System MUST provide dropdown menus for selecting "Source Environment" and "Target Environment" on the migration screen.
- **FR-002**: System MUST provide a "Replace Database" checkbox that, when enabled, triggers the database mapping logic for all assets (Dashboards, Datasets, Charts) during migration.
- **FR-003**: System MUST include a dedicated "Database Mapping" tab or view accessible from the migration interface.
- **FR-004**: System MUST fetch available databases from both source and target environments via their respective APIs when the mapping tab is opened.
- **FR-005**: System MUST implement a name-based fuzzy matching algorithm to suggest initial mappings, ignoring common environment prefixes (e.g., "Dev", "Prod").
- **FR-006**: System MUST allow users to manually override suggested mappings and create new ones via a drag-and-drop or dropdown-based interface.
- **FR-007**: System MUST persist database mappings in a local SQLite database, keyed by the source and target environment identifiers.
- **FR-008**: System MUST provide an "Edit" capability for existing mappings, allowing users to update or delete them.
- **FR-009**: During migration, if "Replace Database" is active, the system MUST intercept asset definitions (JSON/YAML) and replace database references according to the active mapping table.

### Key Entities *(include if feature involves data)*

- **Environment**: A configured Superset instance (Name, URL, Credentials).
- **Database Mapping**: A record linking a source database configuration (including metadata like engine type) to a target database configuration for a specific `source_env` -> `target_env` pair.
- **Migration Configuration**: The set of parameters for a migration job, including selected environments and the "Replace Database" toggle state.

## Success Criteria *(mandatory)*

### Measurable Outcomes

- **SC-001**: Users can complete a full database mapping for 5+ databases in under 60 seconds using the empirical suggestions.
- **SC-002**: 100% of assets migrated with "Replace Database" enabled correctly reference the target databases as defined in the mapping table.
- **SC-003**: Mapping persistence allows users to run subsequent migrations between the same environments without re-configuring database pairs in 100% of cases.
- **SC-004**: The system successfully identifies and suggests at least 90% of matching pairs when naming follows a "Prefix + Name" pattern (e.g., "Dev-Sales" -> "Prod-Sales").

## Assumptions

- **AS-001**: Environments are already configured in the application's global settings.
- **AS-002**: The backend has access to stored credentials for both source and target environments to perform API requests.
- **AS-003**: Database names or IDs are stable enough within an environment to be used as reliable mapping keys.