ss-tools/specs/010-refactor-cli-to-web/spec.md

[DEF:Spec:010-refactor-cli-to-web]

@TITLE: Refactor CLI Scripts to Web Application

@STATUS: DRAFT

@AUTHOR: Kilo Code

@CREATED: 2026-01-07

Clarifications

Session 2026-01-07

• Q: Кто должен иметь доступ к новым веб-инструментам (Поиск, Маппинг, Отладка)? → A: Все аутентифицированные пользователи.
• Q: Нужно ли сохранять конфигурации подключений для маппинга (источник, детали подключения) для повторного использования, или они вводятся каждый раз заново? → A: Сохранять для повторного использования (удобство пользователя).
• Q: Скрипт backup_script.py указан в списке на удаление, но функциональность резервного копирования не описана в требованиях. Нужно ли переносить функцию бэкапа в веб-интерфейс? → A: Функциональность бэкапа уже реализована в рамках задачи 009-backup-scheduler и доступна через TaskManager. Скрипт backup_script.py можно удалять, так как его заменяет новая система.

1. Overview

1.1. Problem Statement

The system currently relies on a set of command-line scripts for critical operations like searching datasets, mapping columns, and debugging. This requires users to have SSH access and knowledge of terminal commands, creating a disjointed user experience compared to the main Web Application. It also leads to maintenance overhead as core logic is duplicated between the CLI tools and the Web backend.

1.2. Goal

Integrate the functionality of the standalone CLI tools directly into the Web Application. This will provide a unified interface for all system operations, simplify maintenance by centralizing logic, and eliminate the need for direct terminal access.

1.3. Scope

• In Scope:
  • Integration of Dataset Search functionality into the Web UI.
  • Integration of Dataset Mapping functionality into the Web UI.
  • Integration of System Debugging tools into the Web UI.
  • Removal of legacy command-line scripts and their specific dependencies.
  • Verification that existing Backup functionality (from 009-backup-scheduler) fully covers the legacy backup_script.py capabilities before removal.
• Out of Scope:
  • Major redesign of the existing Web UI (functionality will be added using existing patterns).
  • Changes to the core business logic of the tools (porting existing logic only).

2. User Scenarios

2.1. Search Datasets

• Before: User logs in via SSH, runs a python script, edits the script to change the query, and reads text output in the terminal.
• After: User logs into the Web App, navigates to the "Search" tool, enters a query, and views results in a structured list within the browser.

2.2. Map Dataset Columns

• Before: User prepares a config or arguments and runs a python script from the terminal.
• After: User navigates to the "Dataset Mapper" tool in the Web App, fills out a form with source details, and executes the mapping. Progress is visible in the application's task list.

2.3. System Debugging

• Before: User runs various debug scripts manually to test connectivity or API structure.
• After: User navigates to a "Debug" section in the Web App, selects a diagnostic routine (e.g., "Test DB API"), and views the report in the application logs.

3. Functional Requirements

3.1. Web-Based Dataset Search

• The system must provide a user interface to search for text patterns across all Superset datasets.
• Users must be able to select the target environment.
• The system must display search results including the Dataset Name, Field Name, and the matching text context.

3.2. Web-Based Dataset Mapping

• The system must provide a user interface to map dataset column names/comments from external sources (e.g., Database, File).
• Users must be able to specify the source type and connection details via the UI.
• Connection configurations must be saved for reuse to improve user convenience.
• The system must provide feedback on the success or failure of the mapping operation.

3.3. Web-Based Diagnostics

• The system must provide a user interface to trigger system diagnostic routines.
• Supported diagnostics must include:
  • Retrieving dataset structure for debugging.
  • Testing Database API connectivity and response structure.
• Results must be viewable within the application.

3.4. Legacy Cleanup

• The system must function independently of the legacy CLI scripts.
• The legacy CLI scripts must be removed to prevent usage of deprecated tools.

3.5. Security & Access

• All authenticated users must have access to Search, Mapping, and Debugging tools.

4. Success Criteria

• Unified Experience: Users can perform Search, Mapping, and Debugging tasks entirely through the Web UI without using a terminal.
• Feature Parity: All capabilities previously available in the CLI scripts are available in the Web Application.
• Clean Codebase: The project no longer contains standalone CLI scripts (search_script.py, run_mapper.py, migration_script.py, backup_script.py, debug_db_api.py, get_dataset_structure.py).
• Dependency Reduction: The codebase no longer relies on CLI-specific libraries (e.g., whiptail).

5. Assumptions

• The existing Web Application plugin architecture supports the addition of these new tools.
• The existing logging and task management systems in the Web Application can handle the output from these tools.

[/DEF:Spec:010-refactor-cli-to-web]