ss-tools/specs/001-migration-ui-redesign/research.md

Research: Migration Process and UI Redesign

Decision: Fuzzy Matching Algorithm

• Choice: RapidFuzz library with fuzz.token_sort_ratio.
• Rationale: RapidFuzz is significantly faster than FuzzyWuzzy and provides robust string similarity metrics. token_sort_ratio is ideal for database names because it ignores word order and is less sensitive to prefixes like "Dev-" or "Prod-".
• Alternatives considered:
  • Levenshtein: Too sensitive to string length and prefixes.
  • Jaro-Winkler: Good for short strings but less effective for multi-word names with different orders.

Decision: Asset Interception Strategy

• Choice: ZIP-based transformation during migration.
• Rationale: Superset's native export/import format is a ZIP archive containing YAML definitions. Intercepting this archive allows for precise modification of database references (UUIDs) before they reach the target environment.
• Implementation:
  1. Export dashboard/dataset from source (ZIP).
  2. Extract ZIP to a temporary directory.
  3. Iterate through datasets/*.yaml files.
  4. Replace database_uuid values based on the mapping table.
  5. Re-package the ZIP.
  6. Import to target.

Decision: Database Mapping Persistence

• Choice: SQLite with SQLAlchemy/SQLModel.
• Rationale: SQLite is lightweight, requires no separate server, and is perfect for storing local configuration and mappings. It aligns with the project's existing stack.
• Schema:
  • Environment: id, name, url, credentials_id.
  • DatabaseMapping: id, source_env_id, target_env_id, source_db_uuid, target_db_uuid, source_db_name, target_db_name.

Decision: Superset API Integration

• Choice: Extend existing SupersetClient.
• Rationale: SupersetClient already handles authentication, network requests, and basic CRUD for dashboards/datasets. Adding environment-specific fetching and database listing is a natural extension.
• New Endpoints to use:
  • GET /api/v1/database/: List all databases.
  • GET /api/v1/database/{id}: Get detailed database config.