ss-tools/specs/project_map.md

Project Semantic Map

Compressed view for AI Context. Generated automatically.

• 📦 generate_semantic_map (Module)
  • 📝 Scans the codebase to generate a Semantic Map and Compliance Report based on the System Standard.
  • 🏗️ Layer: DevOps/Tooling
  • ƒ init (Function)
    • 📝 Mock init.
  • ƒ enter (Function)
    • 📝 Mock enter.
  • ƒ exit (Function)
    • 📝 Mock exit.
  • ℂ SemanticEntity (Class)
    • 📝 Represents a code entity (Module, Function, Component) found during parsing.
    • ƒ init (Function)
      • 📝 Initializes a new SemanticEntity instance.
    • ƒ to_dict (Function)
      • 📝 Serializes the entity to a dictionary for JSON output.
    • ƒ validate (Function)
      • 📝 Checks for semantic compliance (closure, mandatory tags, belief state).
    • ƒ get_score (Function)
      • 📝 Calculates a compliance score (0.0 to 1.0).
  • ƒ get_patterns (Function)
    • 📝 Returns regex patterns for a specific language.
  • ƒ parse_file (Function)
    • 📝 Parses a single file to extract semantic entities.
  • ℂ SemanticMapGenerator (Class)
    • 📝 Orchestrates the mapping process.
    • ƒ init (Function)
      • 📝 Initializes the generator with a root directory.
    • ƒ _load_gitignore (Function)
      • 📝 Loads patterns from .gitignore file.
    • ƒ _is_ignored (Function)
      • 📝 Checks if a path should be ignored based on .gitignore or hardcoded defaults.
    • ƒ run (Function)
      • 📝 Main execution flow.
      • 🔗 CALLS -> _walk_and_parse
      • 🔗 CALLS -> _generate_artifacts
    • ƒ _walk_and_parse (Function)
      • 📝 Recursively walks directories and triggers parsing.
    • ƒ _process_file_results (Function)
      • 📝 Validates entities and calculates file scores.
    • ƒ _generate_artifacts (Function)
      • 📝 Writes output files.
    • ƒ _generate_report (Function)
      • 📝 Generates the Markdown compliance report.
    • ƒ _collect_issues (Function)
      • 📝 Helper to collect issues for a specific file from the entity tree.
    • ƒ _generate_compressed_map (Function)
      • 📝 Generates the token-optimized project map.
    • ƒ _write_entity_md (Function)
      • 📝 Recursive helper to write entity tree to Markdown.
• 📦 main (Module)
  • 📝 Entry point for the Svelte application.
  • 🏗️ Layer: UI-Entry
  • 📦 app_instance (Data)
    • 📝 Initialized Svelte app instance.
• 🧩 App (Component)
  • 📝 The root component of the frontend application. Manages navigation and layout.
  • 🏗️ Layer: UI
  • ƒ handleFormSubmit (Function)
    • 📝 Handles form submission for task creation.
  • ƒ navigate (Function)
    • 📝 Changes the current page and resets state.
• 📦 stores_module (Module)
  • 📝 Global state management using Svelte stores.
  • 🏗️ Layer: UI-State
  • 📦 plugins (Data)
    • 📝 Store for the list of available plugins.
  • 📦 tasks (Data)
    • 📝 Store for the list of tasks.
  • 📦 selectedPlugin (Data)
    • 📝 Store for the currently selected plugin.
  • 📦 selectedTask (Data)
    • 📝 Store for the currently selected task.
  • 📦 currentPage (Data)
    • 📝 Store for the current page.
  • 📦 taskLogs (Data)
    • 📝 Store for the logs of the currently selected task.
  • ƒ fetchPlugins (Function)
    • 📝 Fetches plugins from the API and updates the plugins store.
  • ƒ fetchTasks (Function)
    • 📝 Fetches tasks from the API and updates the tasks store.
• 📦 toasts_module (Module)
  • 📝 Manages toast notifications using a Svelte writable store.
  • 🏗️ Layer: UI-State
  • 📦 toasts (Data)
    • 📝 Writable store containing the list of active toasts.
  • ƒ addToast (Function)
    • 📝 Adds a new toast message.
  • ƒ removeToast (Function)
    • 📝 Removes a toast message by ID.
• 📦 api_module (Module)
  • 📝 Handles all communication with the backend API.
  • 🏗️ Layer: Infra-API
  • ƒ getWsUrl (Function)
    • 📝 Returns the WebSocket URL for a specific task, with fallback logic.
  • ƒ fetchApi (Function)
    • 📝 Generic GET request wrapper.
  • ƒ postApi (Function)
    • 📝 Generic POST request wrapper.
  • ƒ requestApi (Function)
    • 📝 Generic request wrapper.
  • 📦 api (Data)
    • 📝 API client object with specific methods.
• ƒ selectPlugin (Function)
  • 📝 Handles plugin selection and navigation.
• ƒ handleFormSubmit (Function)
  • 📝 Handles task creation from dynamic form submission.
• ƒ load (Function)
  • 📝 Loads initial plugin data for the dashboard.
• 🧩 TaskManagementPage (Component)
  • 📝 Page for managing and monitoring tasks.
  • 🏗️ Layer: Page
  • ƒ loadInitialData (Function)
    • 📝 Loads tasks and environments on page initialization.
  • ƒ refreshTasks (Function)
    • 📝 Periodically refreshes the task list.
  • ƒ handleSelectTask (Function)
    • 📝 Updates the selected task ID when a task is clicked.
  • ƒ handleRunBackup (Function)
    • 📝 Triggers a manual backup task for the selected environment.
• 🧩 MigrationDashboard (Component)
  • 📝 Main dashboard for configuring and starting migrations.
  • 🏗️ Layer: Page
  • ƒ fetchEnvironments (Function)
    • 📝 Fetches the list of environments from the API.
  • ƒ fetchDashboards (Function)
    • 📝 Fetches dashboards for the selected source environment.
  • ƒ fetchDatabases (Function)
    • 📝 Fetches databases from both environments and gets suggestions.
  • ƒ handleMappingUpdate (Function)
    • 📝 Saves a mapping to the backend.
  • ƒ handleViewLogs (Function)
    • 📝 Opens the log viewer for a specific task.
  • ƒ handlePasswordPrompt (Function)
    • 📝 Reactive logic to show password prompt when a task is awaiting input.
  • ƒ handleResumeMigration (Function)
    • 📝 Resumes a migration task with provided passwords.
  • ƒ startMigration (Function)
    • 📝 Starts the migration process.
• 🧩 MappingManagement (Component)
  • 📝 Page for managing database mappings between environments.
  • 🏗️ Layer: Page
  • ƒ fetchEnvironments (Function)
    • 📝 Fetches the list of environments.
  • ƒ fetchDatabases (Function)
    • 📝 Fetches databases from both environments and gets suggestions.
  • ƒ handleUpdate (Function)
    • 📝 Saves a mapping to the backend.
• 🧩 SearchPage (Component)
  • 📝 Page for the dataset search tool.
  • 🏗️ Layer: UI
• 🧩 MapperPage (Component)
  • 📝 Page for the dataset column mapper tool.
  • 🏗️ Layer: UI
• 🧩 DebugPage (Component)
  • 📝 Page for system diagnostics and debugging.
  • 🏗️ Layer: UI
• ƒ handleSaveGlobal (Function)
  • 📝 Saves global application settings.
• ƒ handleAddOrUpdateEnv (Function)
  • 📝 Adds a new environment or updates an existing one.
• ƒ handleDeleteEnv (Function)
  • 📝 Deletes a Superset environment.
• ƒ handleTestEnv (Function)
  • 📝 Tests the connection to a Superset environment.
• ƒ editEnv (Function)
  • 📝 Populates the environment form for editing.
• ƒ resetEnvForm (Function)
  • 📝 Resets the environment creation/edit form to default state.
• ƒ load (Function)
  • 📝 Loads application settings and environment list.
• 🧩 ConnectionsSettingsPage (Component)
  • 📝 Page for managing database connection configurations.
  • 🏗️ Layer: UI
  • ƒ handleSuccess (Function)
    • 📝 Refreshes the connection list after a successful creation.
• 🧩 Dashboard (Component)
  • 📝 Displays the list of available plugins and allows selecting one.
  • 🏗️ Layer: UI
  • ƒ onMount (Function)
    • 📝 Fetch plugins when the component mounts.
  • ƒ selectPlugin (Function)
    • 📝 Selects a plugin to display its form.
• 🧩 Settings (Component)
  • 📝 The main settings page for the application, allowing management of environments and global settings.
  • 🏗️ Layer: UI
  • ƒ loadSettings (Function)
    • 📝 Loads settings from the backend.
  • ƒ handleSaveGlobal (Function)
    • 📝 Saves global settings to the backend.
  • ƒ handleAddOrUpdateEnv (Function)
    • 📝 Adds or updates an environment.
  • ƒ handleDeleteEnv (Function)
    • 📝 Deletes an environment.
  • ƒ handleTestEnv (Function)
    • 📝 Tests the connection to an environment.
  • ƒ editEnv (Function)
    • 📝 Sets the form to edit an existing environment.
  • ƒ resetEnvForm (Function)
    • 📝 Resets the environment form.
• ƒ getConnections (Function)
  • 📝 Fetch a list of saved connections.
• ƒ createConnection (Function)
  • 📝 Create a new connection configuration.
• ƒ deleteConnection (Function)
  • 📝 Delete a connection configuration.
• ƒ runTask (Function)
  • 📝 Start a new task for a given plugin.
• ƒ getTaskStatus (Function)
  • 📝 Fetch details for a specific task (to poll status or get result).
• ƒ getTasks (Function)
  • 📝 Fetch a list of tasks with pagination and optional status filter.
• ƒ getTask (Function)
  • 📝 Fetch details for a specific task.
• ƒ getTaskLogs (Function)
  • 📝 Fetch logs for a specific task.
• ƒ resumeTask (Function)
  • 📝 Resume a task that is awaiting input (e.g., passwords).
• ƒ resolveTask (Function)
  • 📝 Resolve a task that is awaiting mapping.
• ƒ clearTasks (Function)
  • 📝 Clear tasks based on status.
• 🧩 PasswordPrompt (Component)
  • 📝 A modal component to prompt the user for database passwords when a migration task is paused.
  • 🏗️ Layer: UI
  • ƒ handleSubmit (Function)
    • 📝 Validates and dispatches the passwords to resume the task.
  • ƒ handleCancel (Function)
    • 📝 Cancels the password prompt.
• 🧩 MappingTable (Component)
  • 📝 Displays and allows editing of database mappings.
  • 🏗️ Layer: Feature
  • ƒ updateMapping (Function)
    • 📝 Updates a mapping for a specific source database.
  • ƒ getSuggestion (Function)
    • 📝 Finds a suggestion for a source database.
• 🧩 TaskLogViewer (Component)
  • 📝 Displays detailed logs for a specific task in a modal or inline.
  • 🏗️ Layer: UI
  • ƒ fetchLogs (Function)
    • 📝 Fetches logs for the current task.
  • ƒ scrollToBottom (Function)
    • 📝 Scrolls the log container to the bottom.
  • ƒ handleScroll (Function)
    • 📝 Updates auto-scroll preference based on scroll position.
  • ƒ close (Function)
    • 📝 Closes the log viewer modal.
  • ƒ getLogLevelColor (Function)
    • 📝 Returns the CSS color class for a given log level.
  • ƒ onDestroy (Function)
    • 📝 Cleans up the polling interval.
• 🧩 Footer (Component)
  • 📝 Displays the application footer with copyright information.
  • 🏗️ Layer: UI
• 🧩 MissingMappingModal (Component)
  • 📝 Prompts the user to provide a database mapping when one is missing during migration.
  • 🏗️ Layer: Feature
  • ƒ resolve (Function)
    • 📝 Dispatches the resolution event with the selected mapping.
  • ƒ cancel (Function)
    • 📝 Cancels the mapping resolution modal.
• 🧩 DashboardGrid (Component)
  • 📝 Displays a grid of dashboards with selection and pagination.
  • 🏗️ Layer: Component
  • ƒ handleSort (Function)
    • 📝 Toggles sort direction or changes sort column.
  • ƒ handleSelectionChange (Function)
    • 📝 Handles individual checkbox changes.
  • ƒ handleSelectAll (Function)
    • 📝 Handles select all checkbox.
  • ƒ goToPage (Function)
    • 📝 Changes current page.
• 🧩 Navbar (Component)
  • 📝 Main navigation bar for the application.
  • 🏗️ Layer: UI
• 🧩 TaskHistory (Component)
  • 📝 Displays a list of recent tasks with their status and allows selecting them for viewing logs.
  • 🏗️ Layer: UI
  • ƒ fetchTasks (Function)
    • 📝 Fetches the list of recent tasks from the API.
  • ƒ clearTasks (Function)
    • 📝 Clears tasks from the history, optionally filtered by status.
  • ƒ selectTask (Function)
    • 📝 Selects a task and fetches its full details.
  • ƒ getStatusColor (Function)
    • 📝 Returns the CSS color class for a given task status.
  • ƒ onMount (Function)
    • 📝 Initializes the component by fetching tasks and starting polling.
  • ƒ onDestroy (Function)
    • 📝 Cleans up the polling interval when the component is destroyed.
• 🧩 Toast (Component)
  • 📝 Displays transient notifications (toasts) in the bottom-right corner.
  • 🏗️ Layer: UI
• 🧩 TaskRunner (Component)
  • 📝 Connects to a WebSocket to display real-time logs for a running task.
  • 🏗️ Layer: UI
  • ƒ connect (Function)
    • 📝 Establishes WebSocket connection with exponential backoff.
  • ƒ fetchTargetDatabases (Function)
    • 📝 Fetches the list of databases in the target environment.
  • ƒ handleMappingResolve (Function)
    • 📝 Handles the resolution of a missing database mapping.
  • ƒ handlePasswordResume (Function)
    • 📝 Handles the submission of database passwords to resume a task.
  • ƒ startDataTimeout (Function)
    • 📝 Starts a timeout to detect when the log stream has stalled.
  • ƒ resetDataTimeout (Function)
    • 📝 Resets the data stall timeout.
  • ƒ onMount (Function)
    • 📝 Initializes the component and subscribes to task selection changes.
  • ƒ onDestroy (Function)
    • 📝 Close WebSocket connection when the component is destroyed.
• 🧩 TaskList (Component)
  • 📝 Displays a list of tasks with their status and execution details.
  • 🏗️ Layer: Component
  • ƒ getStatusColor (Function)
    • 📝 Returns the CSS color class for a given task status.
  • ƒ formatTime (Function)
    • 📝 Formats a date string using date-fns.
  • ƒ handleTaskClick (Function)
    • 📝 Dispatches a select event when a task is clicked.
• 🧩 DynamicForm (Component)
  • 📝 Generates a form dynamically based on a JSON schema.
  • 🏗️ Layer: UI
  • ƒ handleSubmit (Function)
    • 📝 Dispatches the submit event with the form data.
  • ƒ initializeForm (Function)
    • 📝 Initialize form data with default values from the schema.
• 🧩 EnvSelector (Component)
  • 📝 Provides a UI component for selecting source and target environments.
  • 🏗️ Layer: Feature
  • ƒ handleSelect (Function)
    • 📝 Dispatches the selection change event.
• 🧩 ConnectionForm (Component)
  • 📝 UI component for creating a new database connection configuration.
  • 🏗️ Layer: UI
  • ƒ handleSubmit (Function)
    • 📝 Submits the connection form to the backend.
  • ƒ resetForm (Function)
    • 📝 Resets the connection form fields to their default values.
• 🧩 ConnectionList (Component)
  • 📝 UI component for listing and deleting saved database connection configurations.
  • 🏗️ Layer: UI
  • ƒ fetchConnections (Function)
    • 📝 Fetches the list of connections from the backend.
  • ƒ handleDelete (Function)
    • 📝 Deletes a connection configuration.
• 🧩 MapperTool (Component)
  • 📝 UI component for mapping dataset column verbose names using the MapperPlugin.
  • 🏗️ Layer: UI
  • ƒ fetchData (Function)
    • 📝 Fetches environments and saved connections.
  • ƒ handleRunMapper (Function)
    • 📝 Triggers the MapperPlugin task.
• 🧩 DebugTool (Component)
  • 📝 UI component for system diagnostics and debugging API responses.
  • 🏗️ Layer: UI
  • ƒ fetchEnvironments (Function)
    • 📝 Fetches available environments.
  • ƒ handleRunDebug (Function)
    • 📝 Triggers the debug task.
  • ƒ startPolling (Function)
    • 📝 Polls for task completion.
• 🧩 SearchTool (Component)
  • 📝 UI component for searching datasets using the SearchPlugin.
  • 🏗️ Layer: UI
  • ƒ fetchEnvironments (Function)
    • 📝 Fetches the list of available environments.
  • ƒ handleSearch (Function)
    • 📝 Triggers the SearchPlugin task.
  • ƒ startPolling (Function)
    • 📝 Polls for task completion and results.
• 📦 AppModule (Module)
  • 📝 The main entry point for the FastAPI application. It initializes the app, configures CORS, sets up dependencies, includes API routers, and defines the WebSocket endpoint for log streaming.
  • 🏗️ Layer: UI (API)
  • 📦 App (Global)
    • 📝 The global FastAPI application instance.
  • ƒ startup_event (Function)
    • 📝 Handles application startup tasks, such as starting the scheduler.
  • ƒ shutdown_event (Function)
    • 📝 Handles application shutdown tasks, such as stopping the scheduler.
  • ƒ log_requests (Function)
    • 📝 Middleware to log incoming HTTP requests and their response status.
  • ƒ websocket_endpoint (Function)
    • 📝 Provides a WebSocket endpoint for real-time log streaming of a task.
  • 📦 StaticFiles (Mount)
    • 📝 Mounts the frontend build directory to serve static assets.
    • ƒ serve_spa (Function)
      • 📝 Serves frontend static files or index.html for SPA routing.
    • ƒ read_root (Function)
      • 📝 A simple root endpoint to confirm that the API is running when frontend is missing.
• 📦 Dependencies (Module)
  • 📝 Manages the creation and provision of shared application dependencies, such as the PluginLoader and TaskManager, to avoid circular imports.
  • 🏗️ Layer: Core
  • ƒ get_config_manager (Function)
    • 📝 Dependency injector for the ConfigManager.
  • ƒ get_plugin_loader (Function)
    • 📝 Dependency injector for the PluginLoader.
  • ƒ get_task_manager (Function)
    • 📝 Dependency injector for the TaskManager.
  • ƒ get_scheduler_service (Function)
    • 📝 Dependency injector for the SchedulerService.
• 📦 backend.src.core.superset_client (Module)
  • 📝 Предоставляет высокоуровневый клиент для взаимодействия с Superset REST API, инкапсулируя логику запросов, обработку ошибок и пагинацию.
  • 🏗️ Layer: Core
  • ℂ SupersetClient (Class)
    • 📝 Класс-обёртка над Superset REST API, предоставляющий методы для работы с дашбордами и датасетами.
    • ƒ init (Function)
      • 📝 Инициализирует клиент, проверяет конфигурацию и создает сетевой клиент.
    • ƒ authenticate (Function)
      • 📝 Authenticates the client using the configured credentials.
    • ƒ headers (Function)
      • 📝 Возвращает базовые HTTP-заголовки, используемые сетевым клиентом.
    • ƒ get_dashboards (Function)
      • 📝 Получает полный список дашбордов, автоматически обрабатывая пагинацию.
    • ƒ get_dashboards_summary (Function)
      • 📝 Fetches dashboard metadata optimized for the grid.
    • ƒ export_dashboard (Function)
      • 📝 Экспортирует дашборд в виде ZIP-архива.
    • ƒ import_dashboard (Function)
      • 📝 Импортирует дашборд из ZIP-файла.
    • ƒ delete_dashboard (Function)
      • 📝 Удаляет дашборд по его ID или slug.
    • ƒ get_datasets (Function)
      • 📝 Получает полный список датасетов, автоматически обрабатывая пагинацию.
    • ƒ get_dataset (Function)
      • 📝 Получает информацию о конкретном датасете по его ID.
    • ƒ update_dataset (Function)
      • 📝 Обновляет данные датасета по его ID.
    • ƒ get_databases (Function)
      • 📝 Получает полный список баз данных.
    • ƒ get_database (Function)
      • 📝 Получает информацию о конкретной базе данных по её ID.
    • ƒ get_databases_summary (Function)
      • 📝 Fetch a summary of databases including uuid, name, and engine.
    • ƒ get_database_by_uuid (Function)
      • 📝 Find a database by its UUID.
    • ƒ _resolve_target_id_for_delete (Function)
    • ƒ _do_import (Function)
    • ƒ _validate_export_response (Function)
    • ƒ _resolve_export_filename (Function)
    • ƒ _validate_query_params (Function)
    • ƒ _fetch_total_object_count (Function)
    • ƒ _fetch_all_pages (Function)
    • ƒ _validate_import_file (Function)
• 📦 ConfigManagerModule (Module)
  • 📝 Manages application configuration, including loading/saving to JSON and CRUD for environments.
  • 🏗️ Layer: Core
  • 🔗 DEPENDS_ON -> ConfigModels
  • 🔗 CALLS -> logger
  • ℂ ConfigManager (Class)
    • 📝 A class to handle application configuration persistence and management.
    • ƒ init (Function)
      • 📝 Initializes the ConfigManager.
    • ƒ _load_config (Function)
      • 📝 Loads the configuration from disk or creates a default one.
    • ƒ _save_config_to_disk (Function)
      • 📝 Saves the provided configuration object to disk.
    • ƒ save (Function)
      • 📝 Saves the current configuration state to disk.
    • ƒ get_config (Function)
      • 📝 Returns the current configuration.
    • ƒ update_global_settings (Function)
      • 📝 Updates the global settings and persists the change.
    • ƒ validate_path (Function)
      • 📝 Validates if a path exists and is writable.
    • ƒ get_environments (Function)
      • 📝 Returns the list of configured environments.
    • ƒ has_environments (Function)
      • 📝 Checks if at least one environment is configured.
    • ƒ get_environment (Function)
      • 📝 Returns a single environment by ID.
    • ƒ add_environment (Function)
      • 📝 Adds a new environment to the configuration.
    • ƒ update_environment (Function)
      • 📝 Updates an existing environment.
    • ƒ delete_environment (Function)
      • 📝 Deletes an environment by ID.
• 📦 SchedulerModule (Module)
  • 📝 Manages scheduled tasks using APScheduler.
  • 🏗️ Layer: Core
  • ℂ SchedulerService (Class)
    • 📝 Provides a service to manage scheduled backup tasks.
    • ƒ init (Function)
      • 📝 Initializes the scheduler service with task and config managers.
    • ƒ start (Function)
      • 📝 Starts the background scheduler and loads initial schedules.
    • ƒ stop (Function)
      • 📝 Stops the background scheduler.
    • ƒ load_schedules (Function)
      • 📝 Loads backup schedules from configuration and registers them.
    • ƒ add_backup_job (Function)
      • 📝 Adds a scheduled backup job for an environment.
    • ƒ _trigger_backup (Function)
      • 📝 Triggered by the scheduler to start a backup task.
• 📦 ConfigModels (Module)
  • 📝 Defines the data models for application configuration using Pydantic.
  • 🏗️ Layer: Core
  • 📦 Schedule (DataClass)
    • 📝 Represents a backup schedule configuration.
  • 📦 Environment (DataClass)
    • 📝 Represents a Superset environment configuration.
  • 📦 LoggingConfig (DataClass)
    • 📝 Defines the configuration for the application's logging system.
  • 📦 GlobalSettings (DataClass)
    • 📝 Represents global application settings.
  • 📦 AppConfig (DataClass)
    • 📝 The root configuration model containing all application settings.
• 📦 backend.src.core.database (Module)
  • 📝 Configures the SQLite database connection and session management.
  • 🏗️ Layer: Core
  • 🔗 DEPENDS_ON -> sqlalchemy
  • 📦 DATABASE_URL (Constant)
  • 📦 TASKS_DATABASE_URL (Constant)
  • 📦 engine (Variable)
  • 📦 tasks_engine (Variable)
  • ℂ SessionLocal (Class)
    • 📝 A session factory for the main mappings database.
  • ℂ TasksSessionLocal (Class)
    • 📝 A session factory for the tasks execution database.
  • ƒ init_db (Function)
    • 📝 Initializes the database by creating all tables.
  • ƒ get_db (Function)
    • 📝 Dependency for getting a database session.
  • ƒ get_tasks_db (Function)
    • 📝 Dependency for getting a tasks database session.
• 📦 LoggerModule (Module)
  • 📝 Configures the application's logging system, including a custom handler for buffering logs and streaming them over WebSockets.
  • 🏗️ Layer: Core
  • ℂ BeliefFormatter (Class)
    • 📝 Custom logging formatter that adds belief state prefixes to log messages.
    • ƒ format (Function)
      • 📝 Formats the log record, adding belief state context if available.
  • ℂ LogEntry (Class)
    • 📝 A Pydantic model representing a single, structured log entry. This is a re-definition for consistency, as it's also defined in task_manager.py.
  • ƒ belief_scope (Function)
    • 📝 Context manager for structured Belief State logging.
  • ƒ configure_logger (Function)
    • 📝 Configures the logger with the provided logging settings.
  • ℂ WebSocketLogHandler (Class)
    • 📝 A custom logging handler that captures log records into a buffer. It is designed to be extended for real-time log streaming over WebSockets.
    • ƒ init (Function)
      • 📝 Initializes the handler with a fixed-capacity buffer.
    • ƒ emit (Function)
      • 📝 Captures a log record, formats it, and stores it in the buffer.
    • ƒ get_recent_logs (Function)
      • 📝 Returns a list of recent log entries from the buffer.
  • 📦 Logger (Global)
    • 📝 The global logger instance for the application, configured with both a console handler and the custom WebSocket handler.
    • ƒ believed (Function)
      • 📝 A decorator that wraps a function in a belief scope.
• ℂ PluginLoader (Class)
  • 📝 Scans a specified directory for Python modules, dynamically loads them, and registers any classes that are valid implementations of the PluginBase interface.
  • 🏗️ Layer: Core
  • ƒ init (Function)
    • 📝 Initializes the PluginLoader with a directory to scan.
  • ƒ _load_plugins (Function)
    • 📝 Scans the plugin directory and loads all valid plugins.
  • ƒ _load_module (Function)
    • 📝 Loads a single Python module and discovers PluginBase implementations.
  • ƒ _register_plugin (Function)
    • 📝 Registers a PluginBase instance and its configuration.
  • ƒ get_plugin (Function)
    • 📝 Retrieves a loaded plugin instance by its ID.
  • ƒ get_all_plugin_configs (Function)
    • 📝 Returns a list of all registered plugin configurations.
  • ƒ has_plugin (Function)
    • 📝 Checks if a plugin with the given ID is registered.
• 📦 backend.src.core.migration_engine (Module)
  • 📝 Handles the interception and transformation of Superset asset ZIP archives.
  • 🏗️ Layer: Core
  • 🔗 DEPENDS_ON -> PyYAML
  • ℂ MigrationEngine (Class)
    • 📝 Engine for transforming Superset export ZIPs.
    • ƒ transform_zip (Function)
      • 📝 Extracts ZIP, replaces database UUIDs in YAMLs, and re-packages.
    • ƒ _transform_yaml (Function)
      • 📝 Replaces database_uuid in a single YAML file.
• ℂ PluginBase (Class)
  • 📝 Defines the abstract base class that all plugins must implement to be recognized by the system. It enforces a common structure for plugin metadata and execution.
  • 🏗️ Layer: Core
  • ƒ id (Function)
    • 📝 Returns the unique identifier for the plugin.
  • ƒ name (Function)
    • 📝 Returns the human-readable name of the plugin.
  • ƒ description (Function)
    • 📝 Returns a brief description of the plugin.
  • ƒ version (Function)
    • 📝 Returns the version of the plugin.
  • ƒ get_schema (Function)
    • 📝 Returns the JSON schema for the plugin's input parameters.
  • ƒ execute (Function)
    • 📝 Executes the plugin's core logic.
• ℂ PluginConfig (Class)
  • 📝 A Pydantic model used to represent the validated configuration and metadata of a loaded plugin. This object is what gets exposed to the API layer.
  • 🏗️ Layer: Core
• 📦 backend.core.utils.fileio (Module)
  • 📝 Предоставляет набор утилит для управления файловыми операциями, включая работу с временными файлами, архивами ZIP, файлами YAML и очистку директорий.
  • 🏗️ Layer: Infra
  • 🔗 DEPENDS_ON -> backend.src.core.logger
  • 🔗 DEPENDS_ON -> pyyaml
  • ℂ InvalidZipFormatError (Class)
    • ƒ create_temp_file (Function)
      • 📝 Контекстный менеджер для создания временного файла или директории с гарантированным удалением.
    • ƒ remove_empty_directories (Function)
      • 📝 Рекурсивно удаляет все пустые поддиректории, начиная с указанного пути.
    • ƒ read_dashboard_from_disk (Function)
      • 📝 Читает бинарное содержимое файла с диска.
    • ƒ calculate_crc32 (Function)
      • 📝 Вычисляет контрольную сумму CRC32 для файла.
    • 📦 RetentionPolicy (DataClass)
      • 📝 Определяет политику хранения для архивов (ежедневные, еженедельные, ежемесячные).
    • ƒ archive_exports (Function)
      • 📝 Управляет архивом экспортированных файлов, применяя политику хранения и дедупликацию.
      • 🔗 CALLS -> apply_retention_policy
      • 🔗 CALLS -> calculate_crc32
    • ƒ apply_retention_policy (Function)
      • 📝 (Helper) Применяет политику хранения к списку файлов, возвращая те, что нужно сохранить.
    • ƒ save_and_unpack_dashboard (Function)
      • 📝 Сохраняет бинарное содержимое ZIP-архива на диск и опционально распаковывает его.
    • ƒ update_yamls (Function)
      • 📝 Обновляет конфигурации в YAML-файлах, заменяя значения или применяя regex.
      • 🔗 CALLS -> _update_yaml_file
    • ƒ _update_yaml_file (Function)
      • 📝 (Helper) Обновляет один YAML файл.
    • ƒ create_dashboard_export (Function)
      • 📝 Создает ZIP-архив из указанных исходных путей.
    • ƒ sanitize_filename (Function)
      • 📝 Очищает строку от символов, недопустимых в именах файлов.
    • ƒ get_filename_from_headers (Function)
      • 📝 Извлекает имя файла из HTTP заголовка 'Content-Disposition'.
    • ƒ consolidate_archive_folders (Function)
      • 📝 Консолидирует директории архивов на основе общего слага в имени.
• 📦 backend.core.utils.network (Module)
  • 📝 Инкапсулирует низкоуровневую HTTP-логику для взаимодействия с Superset API, включая аутентификацию, управление сессией, retry-логику и обработку ошибок.
  • 🏗️ Layer: Infra
  • 🔗 DEPENDS_ON -> backend.src.core.logger
  • 🔗 DEPENDS_ON -> requests
  • ℂ SupersetAPIError (Class)
    • ℂ AuthenticationError (Class)
• 📦 backend.src.core.utils.matching (Module)
  • 📝 Provides utility functions for fuzzy matching database names.
  • 🏗️ Layer: Core
  • 🔗 DEPENDS_ON -> rapidfuzz
  • ƒ suggest_mappings (Function)
    • 📝 Suggests mappings between source and target databases using fuzzy matching.
• 📦 backend.core.utils.dataset_mapper (Module)
  • 📝 Этот модуль отвечает за обновление метаданных (verbose_map) в датасетах Superset, извлекая их из PostgreSQL или XLSX-файлов.
  • 🏗️ Layer: Domain
  • 🔗 DEPENDS_ON -> backend.core.superset_client
  • 🔗 DEPENDS_ON -> pandas
  • 🔗 DEPENDS_ON -> psycopg2
  • ℂ DatasetMapper (Class)
    • 📝 Класс для меппинга и обновления verbose_map в датасетах Superset.
    • ƒ init (Function)
      • 📝 Initializes the mapper.
    • ƒ get_postgres_comments (Function)
      • 📝 Извлекает комментарии к колонкам из системного каталога PostgreSQL.
    • ƒ load_excel_mappings (Function)
      • 📝 Загружает меппинги 'column_name' -> 'column_comment' из XLSX файла.
    • ƒ run_mapping (Function)
      • 📝 Основная функция для выполнения меппинга и обновления verbose_map датасета в Superset.
      • 🔗 CALLS -> self.get_postgres_comments
      • 🔗 CALLS -> self.load_excel_mappings
      • 🔗 CALLS -> superset_client.get_dataset
      • 🔗 CALLS -> superset_client.update_dataset
• 📦 TaskPersistenceModule (Module)
  • 📝 Handles the persistence of tasks using SQLAlchemy and the tasks.db database.
  • 🏗️ Layer: Core
  • ℂ TaskPersistenceService (Class)
    • 📝 Provides methods to save and load tasks from the tasks.db database using SQLAlchemy.
    • ƒ init (Function)
      • 📝 Initializes the persistence service.
    • ƒ persist_task (Function)
      • 📝 Persists or updates a single task in the database.
    • ƒ persist_tasks (Function)
      • 📝 Persists multiple tasks.
    • ƒ load_tasks (Function)
      • 📝 Loads tasks from the database.
    • ƒ delete_tasks (Function)
      • 📝 Deletes specific tasks from the database.
• 📦 TaskManagerModule (Module)
  • 📝 Manages the lifecycle of tasks, including their creation, execution, and state tracking. It uses a thread pool to run plugins asynchronously.
  • 🏗️ Layer: Core
  • ℂ TaskManager (Class)
    • 📝 Manages the lifecycle of tasks, including their creation, execution, and state tracking.
    • ƒ init (Function)
      • 📝 Initialize the TaskManager with dependencies.
    • ƒ create_task (Function)
      • 📝 Creates and queues a new task for execution.
    • ƒ _run_task (Function)
      • 📝 Internal method to execute a task.
    • ƒ resolve_task (Function)
      • 📝 Resumes a task that is awaiting mapping.
    • ƒ wait_for_resolution (Function)
      • 📝 Pauses execution and waits for a resolution signal.
    • ƒ wait_for_input (Function)
      • 📝 Pauses execution and waits for user input.
    • ƒ get_task (Function)
      • 📝 Retrieves a task by its ID.
    • ƒ get_all_tasks (Function)
      • 📝 Retrieves all registered tasks.
    • ƒ get_tasks (Function)
      • 📝 Retrieves tasks with pagination and optional status filter.
    • ƒ get_task_logs (Function)
      • 📝 Retrieves logs for a specific task.
    • ƒ _add_log (Function)
      • 📝 Adds a log entry to a task and notifies subscribers.
    • ƒ subscribe_logs (Function)
      • 📝 Subscribes to real-time logs for a task.
    • ƒ unsubscribe_logs (Function)
      • 📝 Unsubscribes from real-time logs for a task.
    • ƒ load_persisted_tasks (Function)
      • 📝 Load persisted tasks using persistence service.
    • ƒ await_input (Function)
      • 📝 Transition a task to AWAITING_INPUT state with input request.
    • ƒ resume_task_with_password (Function)
      • 📝 Resume a task that is awaiting input with provided passwords.
    • ƒ clear_tasks (Function)
      • 📝 Clears tasks based on status filter.
• 📦 TaskManagerModels (Module)
  • 📝 Defines the data models and enumerations used by the Task Manager.
  • 🏗️ Layer: Core
  • 📦 TaskStatus (Enum)
    • 📝 Defines the possible states a task can be in during its lifecycle.
  • ℂ LogEntry (Class)
    • 📝 A Pydantic model representing a single, structured log entry associated with a task.
  • ℂ Task (Class)
    • 📝 A Pydantic model representing a single execution instance of a plugin, including its status, parameters, and logs.
    • ƒ init (Function)
      • 📝 Initializes the Task model and validates input_request for AWAITING_INPUT status.
• 📦 TaskCleanupModule (Module)
  • 📝 Implements task cleanup and retention policies.
  • 🏗️ Layer: Core
  • ℂ TaskCleanupService (Class)
    • 📝 Provides methods to clean up old task records.
    • ƒ init (Function)
      • 📝 Initializes the cleanup service with dependencies.
    • ƒ run_cleanup (Function)
      • 📝 Deletes tasks older than the configured retention period.
• 📦 TaskManagerPackage (Module)
  • 📝 Exports the public API of the task manager package.
  • 🏗️ Layer: Core
• 📦 AuthModule (Module)
  • 📝 Implements ADFS authentication using Authlib for FastAPI. It provides a dependency to protect endpoints.
  • 🏗️ Layer: UI (API)
  • ƒ get_current_user (Function)
    • 📝 Dependency to get the current user from the ADFS token.
• 📦 ConnectionsRouter (Module)
  • 📝 Defines the FastAPI router for managing external database connections.
  • 🏗️ Layer: UI (API)
  • ℂ ConnectionSchema (Class)
    • 📝 Pydantic model for connection response.
  • ℂ ConnectionCreate (Class)
    • 📝 Pydantic model for creating a connection.
  • ƒ list_connections (Function)
    • 📝 Lists all saved connections.
  • ƒ create_connection (Function)
    • 📝 Creates a new connection configuration.
  • ƒ delete_connection (Function)
    • 📝 Deletes a connection configuration.
• 📦 backend.src.api.routes.environments (Module)
  • 📝 API endpoints for listing environments and their databases.
  • 🏗️ Layer: API
  • 🔗 DEPENDS_ON -> backend.src.dependencies
  • 🔗 DEPENDS_ON -> backend.src.core.superset_client
  • 📦 ScheduleSchema (DataClass)
  • 📦 EnvironmentResponse (DataClass)
  • 📦 DatabaseResponse (DataClass)
  • ƒ get_environments (Function)
    • 📝 List all configured environments.
  • ƒ update_environment_schedule (Function)
    • 📝 Update backup schedule for an environment.
  • ƒ get_environment_databases (Function)
    • 📝 Fetch the list of databases from a specific environment.
• 📦 backend.src.api.routes.migration (Module)
  • 📝 API endpoints for migration operations.
  • 🏗️ Layer: API
  • 🔗 DEPENDS_ON -> backend.src.dependencies
  • 🔗 DEPENDS_ON -> backend.src.models.dashboard
  • ƒ get_dashboards (Function)
    • 📝 Fetch all dashboards from the specified environment for the grid.
  • ƒ execute_migration (Function)
    • 📝 Execute the migration of selected dashboards.
• 📦 PluginsRouter (Module)
  • 📝 Defines the FastAPI router for plugin-related endpoints, allowing clients to list available plugins.
  • 🏗️ Layer: UI (API)
  • ƒ list_plugins (Function)
    • 📝 Retrieve a list of all available plugins.
• 📦 backend.src.api.routes.mappings (Module)
  • 📝 API endpoints for managing database mappings and getting suggestions.
  • 🏗️ Layer: API
  • 🔗 DEPENDS_ON -> backend.src.dependencies
  • 🔗 DEPENDS_ON -> backend.src.core.database
  • 🔗 DEPENDS_ON -> backend.src.services.mapping_service
  • 📦 MappingCreate (DataClass)
  • 📦 MappingResponse (DataClass)
  • 📦 SuggestRequest (DataClass)
  • ƒ get_mappings (Function)
    • 📝 List all saved database mappings.
  • ƒ create_mapping (Function)
    • 📝 Create or update a database mapping.
  • ƒ suggest_mappings_api (Function)
    • 📝 Get suggested mappings based on fuzzy matching.
• 📦 SettingsRouter (Module)
  • 📝 Provides API endpoints for managing application settings and Superset environments.
  • 🏗️ Layer: UI (API)
  • 🔗 DEPENDS_ON -> ConfigManager
  • 🔗 DEPENDS_ON -> ConfigModels
  • ƒ get_settings (Function)
    • 📝 Retrieves all application settings.
  • ƒ update_global_settings (Function)
    • 📝 Updates global application settings.
  • ƒ get_environments (Function)
    • 📝 Lists all configured Superset environments.
  • ƒ add_environment (Function)
    • 📝 Adds a new Superset environment.
  • ƒ update_environment (Function)
    • 📝 Updates an existing Superset environment.
  • ƒ delete_environment (Function)
    • 📝 Deletes a Superset environment.
  • ƒ test_environment_connection (Function)
    • 📝 Tests the connection to a Superset environment.
  • ƒ validate_backup_path (Function)
    • 📝 Validates if a backup path exists and is writable.
• 📦 TasksRouter (Module)
  • 📝 Defines the FastAPI router for task-related endpoints, allowing clients to create, list, and get the status of tasks.
  • 🏗️ Layer: UI (API)
  • ƒ create_task (Function)
    • 📝 Create and start a new task for a given plugin.
  • ƒ list_tasks (Function)
    • 📝 Retrieve a list of tasks with pagination and optional status filter.
  • ƒ get_task (Function)
    • 📝 Retrieve the details of a specific task.
  • ƒ get_task_logs (Function)
    • 📝 Retrieve logs for a specific task.
  • ƒ resolve_task (Function)
    • 📝 Resolve a task that is awaiting mapping.
  • ƒ resume_task (Function)
    • 📝 Resume a task that is awaiting input (e.g., passwords).
  • ƒ clear_tasks (Function)
    • 📝 Clear tasks matching the status filter.
• 📦 backend.src.models.task (Module)
  • 📝 Defines the database schema for task execution records.
  • 🏗️ Layer: Domain
  • 🔗 DEPENDS_ON -> sqlalchemy
  • ℂ TaskRecord (Class)
    • 📝 Represents a persistent record of a task execution.
• 📦 backend.src.models.connection (Module)
  • 📝 Defines the database schema for external database connection configurations.
  • 🏗️ Layer: Domain
  • 🔗 DEPENDS_ON -> sqlalchemy
  • ℂ ConnectionConfig (Class)
    • 📝 Stores credentials for external databases used for column mapping.
• 📦 backend.src.models.mapping (Module)
  • 📝 Defines the database schema for environment metadata and database mappings using SQLAlchemy.
  • 🏗️ Layer: Domain
  • 🔗 DEPENDS_ON -> sqlalchemy
  • ℂ MigrationStatus (Class)
    • 📝 Enumeration of possible migration job statuses.
  • ℂ Environment (Class)
    • 📝 Represents a Superset instance environment.
  • ℂ DatabaseMapping (Class)
    • 📝 Represents a mapping between source and target databases.
  • ℂ MigrationJob (Class)
    • 📝 Represents a single migration execution job.
• 📦 backend.src.models.dashboard (Module)
  • 📝 Defines data models for dashboard metadata and selection.
  • 🏗️ Layer: Model
  • ℂ DashboardMetadata (Class)
    • 📝 Represents a dashboard available for migration.
  • ℂ DashboardSelection (Class)
    • 📝 Represents the user's selection of dashboards to migrate.
• 📦 backend.src.services.mapping_service (Module)
  • 📝 Orchestrates database fetching and fuzzy matching suggestions.
  • 🏗️ Layer: Service
  • 🔗 DEPENDS_ON -> backend.src.core.superset_client
  • 🔗 DEPENDS_ON -> backend.src.core.utils.matching
  • ℂ MappingService (Class)
    • 📝 Service for handling database mapping logic.
    • ƒ init (Function)
      • 📝 Initializes the mapping service with a config manager.
    • ƒ _get_client (Function)
      • 📝 Helper to get an initialized SupersetClient for an environment.
    • ƒ get_suggestions (Function)
      • 📝 Fetches databases from both environments and returns fuzzy matching suggestions.
• 📦 BackupPlugin (Module)
  • 📝 A plugin that provides functionality to back up Superset dashboards.
  • 🏗️ Layer: App
  • 🔗 DEPENDS_ON -> superset_tool.client
  • 🔗 DEPENDS_ON -> superset_tool.utils
  • ℂ BackupPlugin (Class)
    • 📝 Implementation of the backup plugin logic.
    • ƒ id (Function)
      • 📝 Returns the unique identifier for the backup plugin.
    • ƒ name (Function)
      • 📝 Returns the human-readable name of the backup plugin.
    • ƒ description (Function)
      • 📝 Returns a description of the backup plugin.
    • ƒ version (Function)
      • 📝 Returns the version of the backup plugin.
    • ƒ get_schema (Function)
      • 📝 Returns the JSON schema for backup plugin parameters.
    • ƒ execute (Function)
      • 📝 Executes the dashboard backup logic.
• 📦 DebugPluginModule (Module)
  • 📝 Implements a plugin for system diagnostics and debugging Superset API responses.
  • 🏗️ Layer: Plugins
  • ℂ DebugPlugin (Class)
    • 📝 Plugin for system diagnostics and debugging.
    • ƒ id (Function)
      • 📝 Returns the unique identifier for the debug plugin.
    • ƒ name (Function)
      • 📝 Returns the human-readable name of the debug plugin.
    • ƒ description (Function)
      • 📝 Returns a description of the debug plugin.
    • ƒ version (Function)
      • 📝 Returns the version of the debug plugin.
    • ƒ get_schema (Function)
      • 📝 Returns the JSON schema for the debug plugin parameters.
    • ƒ execute (Function)
      • 📝 Executes the debug logic.
    • ƒ _test_db_api (Function)
      • 📝 Tests database API connectivity for source and target environments.
    • ƒ _get_dataset_structure (Function)
      • 📝 Retrieves the structure of a dataset.
• 📦 SearchPluginModule (Module)
  • 📝 Implements a plugin for searching text patterns across all datasets in a specific Superset environment.
  • 🏗️ Layer: Plugins
  • ℂ SearchPlugin (Class)
    • 📝 Plugin for searching text patterns in Superset datasets.
    • ƒ id (Function)
      • 📝 Returns the unique identifier for the search plugin.
    • ƒ name (Function)
      • 📝 Returns the human-readable name of the search plugin.
    • ƒ description (Function)
      • 📝 Returns a description of the search plugin.
    • ƒ version (Function)
      • 📝 Returns the version of the search plugin.
    • ƒ get_schema (Function)
      • 📝 Returns the JSON schema for the search plugin parameters.
    • ƒ execute (Function)
      • 📝 Executes the dataset search logic.
    • ƒ _get_context (Function)
      • 📝 Extracts a small context around the match for display.
• 📦 MapperPluginModule (Module)
  • 📝 Implements a plugin for mapping dataset columns using external database connections or Excel files.
  • 🏗️ Layer: Plugins
  • ℂ MapperPlugin (Class)
    • 📝 Plugin for mapping dataset columns verbose names.
    • ƒ id (Function)
      • 📝 Returns the unique identifier for the mapper plugin.
    • ƒ name (Function)
      • 📝 Returns the human-readable name of the mapper plugin.
    • ƒ description (Function)
      • 📝 Returns a description of the mapper plugin.
    • ƒ version (Function)
      • 📝 Returns the version of the mapper plugin.
    • ƒ get_schema (Function)
      • 📝 Returns the JSON schema for the mapper plugin parameters.
    • ƒ execute (Function)
      • 📝 Executes the dataset mapping logic.
• 📦 MigrationPlugin (Module)
  • 📝 A plugin that provides functionality to migrate Superset dashboards between environments.
  • 🏗️ Layer: App
  • 🔗 DEPENDS_ON -> superset_tool.client
  • 🔗 DEPENDS_ON -> superset_tool.utils
  • ℂ MigrationPlugin (Class)
    • 📝 Implementation of the migration plugin logic.
    • ƒ id (Function)
      • 📝 Returns the unique identifier for the migration plugin.
    • ƒ name (Function)
      • 📝 Returns the human-readable name of the migration plugin.
    • ƒ description (Function)
      • 📝 Returns a description of the migration plugin.
    • ƒ version (Function)
      • 📝 Returns the version of the migration plugin.
    • ƒ get_schema (Function)
      • 📝 Returns the JSON schema for migration plugin parameters.
    • ƒ execute (Function)
      • 📝 Executes the dashboard migration logic.
• ƒ test_environment_model (Function)
  • 📝 Tests that Environment model correctly stores values.
• ƒ test_belief_scope_logs_entry_action_exit (Function)
  • 📝 Test that belief_scope generates [ID][Entry], [ID][Action], and [ID][Exit] logs.
• ƒ test_belief_scope_error_handling (Function)
  • 📝 Test that belief_scope logs Coherence:Failed on exception.
• ƒ test_belief_scope_success_coherence (Function)
  • 📝 Test that belief_scope logs Coherence:OK on success.