busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
4aa01b64706c4be424e17509b94464c17d11046c
ss-tools/specs/001-migration-ui-redesign/research.md
busya 2ffc3cc68f feat(migration): implement interactive mapping resolution workflow 
- Add SQLite database integration for environments and mappings
- Update TaskManager to support pausing tasks (AWAITING_MAPPING)
- Modify MigrationPlugin to detect missing mappings and wait for resolution
- Add frontend UI for handling missing mappings interactively
- Create dedicated migration routes and API endpoints
- Update .gitignore and project documentation
2025-12-25 22:27:29 +03:00

2.1 KiB
Raw Blame History

Research: Migration Process and UI Redesign

Decision: Fuzzy Matching Algorithm

  • Choice: RapidFuzz library with fuzz.token_sort_ratio.
  • Rationale: RapidFuzz is significantly faster than FuzzyWuzzy and provides robust string similarity metrics. token_sort_ratio is ideal for database names because it ignores word order and is less sensitive to prefixes like "Dev-" or "Prod-".
  • Alternatives considered:
    • Levenshtein: Too sensitive to string length and prefixes.
    • Jaro-Winkler: Good for short strings but less effective for multi-word names with different orders.

Decision: Asset Interception Strategy

  • Choice: ZIP-based transformation during migration.
  • Rationale: Superset's native export/import format is a ZIP archive containing YAML definitions. Intercepting this archive allows for precise modification of database references (UUIDs) before they reach the target environment.
  • Implementation:
    1. Export dashboard/dataset from source (ZIP).
    2. Extract ZIP to a temporary directory.
    3. Iterate through datasets/*.yaml files.
    4. Replace database_uuid values based on the mapping table.
    5. Re-package the ZIP.
    6. Import to target.

Decision: Database Mapping Persistence

  • Choice: SQLite with SQLAlchemy/SQLModel.
  • Rationale: SQLite is lightweight, requires no separate server, and is perfect for storing local configuration and mappings. It aligns with the project's existing stack.
  • Schema:
    • Environment: id, name, url, credentials_id.
    • DatabaseMapping: id, source_env_id, target_env_id, source_db_uuid, target_db_uuid, source_db_name, target_db_name.

Decision: Superset API Integration

  • Choice: Extend existing SupersetClient.
  • Rationale: SupersetClient already handles authentication, network requests, and basic CRUD for dashboards/datasets. Adding environment-specific fetching and database listing is a natural extension.
  • New Endpoints to use:
    • GET /api/v1/database/: List all databases.
    • GET /api/v1/database/{id}: Get detailed database config.