34 lines
2.1 KiB
Markdown
34 lines
2.1 KiB
Markdown
# Research: Migration Process and UI Redesign
|
|
|
|
## Decision: Fuzzy Matching Algorithm
|
|
- **Choice**: `RapidFuzz` library with `fuzz.token_sort_ratio`.
|
|
- **Rationale**: `RapidFuzz` is significantly faster than `FuzzyWuzzy` and provides robust string similarity metrics. `token_sort_ratio` is ideal for database names because it ignores word order and is less sensitive to prefixes like "Dev-" or "Prod-".
|
|
- **Alternatives considered**:
|
|
- `Levenshtein`: Too sensitive to string length and prefixes.
|
|
- `Jaro-Winkler`: Good for short strings but less effective for multi-word names with different orders.
|
|
|
|
## Decision: Asset Interception Strategy
|
|
- **Choice**: ZIP-based transformation during migration.
|
|
- **Rationale**: Superset's native export/import format is a ZIP archive containing YAML definitions. Intercepting this archive allows for precise modification of database references (UUIDs) before they reach the target environment.
|
|
- **Implementation**:
|
|
1. Export dashboard/dataset from source (ZIP).
|
|
2. Extract ZIP to a temporary directory.
|
|
3. Iterate through `datasets/*.yaml` files.
|
|
4. Replace `database_uuid` values based on the mapping table.
|
|
5. Re-package the ZIP.
|
|
6. Import to target.
|
|
|
|
## Decision: Database Mapping Persistence
|
|
- **Choice**: SQLite with SQLAlchemy/SQLModel.
|
|
- **Rationale**: SQLite is lightweight, requires no separate server, and is perfect for storing local configuration and mappings. It aligns with the project's existing stack.
|
|
- **Schema**:
|
|
- `Environment`: `id`, `name`, `url`, `credentials_id`.
|
|
- `DatabaseMapping`: `id`, `source_env_id`, `target_env_id`, `source_db_uuid`, `target_db_uuid`, `source_db_name`, `target_db_name`.
|
|
|
|
## Decision: Superset API Integration
|
|
- **Choice**: Extend existing `SupersetClient`.
|
|
- **Rationale**: `SupersetClient` already handles authentication, network requests, and basic CRUD for dashboards/datasets. Adding environment-specific fetching and database listing is a natural extension.
|
|
- **New Endpoints to use**:
|
|
- `GET /api/v1/database/`: List all databases.
|
|
- `GET /api/v1/database/{id}`: Get detailed database config.
|