Implementation Plan: Refactor CLI Scripts to Web Application

Branch: 010-refactor-cli-to-web | Date: 2026-01-07 | Spec: specs/010-refactor-cli-to-web/spec.md Input: Feature specification from /specs/010-refactor-cli-to-web/spec.md

Note: This template is filled in by the /speckit.plan command. See .specify/templates/commands/plan.md for the execution workflow.

Summary

This project aims to integrate the functionality of standalone CLI tools (search_script.py, run_mapper.py, debug_db_api.py, get_dataset_structure.py) into the existing Web Application (FastAPI + SvelteKit). This involves implementing new Plugins (SearchPlugin, MapperPlugin, DebugPlugin) to handle dataset searching, column mapping, and system diagnostics. The legacy CLI scripts will be removed after their functionality is successfully ported.

Technical Context

Language/Version: Python 3.9+ (Backend), Node.js 18+ (Frontend) Primary Dependencies: FastAPI, SvelteKit, Tailwind CSS, Pydantic, SQLAlchemy, superset_tool (internal lib) Storage: SQLite (for job history/results, connection configs), Filesystem (for temporary file uploads) Testing: pytest (Backend), vitest (Frontend - if applicable) Target Platform: Linux server (Dockerized environment) Project Type: Web Application (Backend + Frontend) Performance Goals: Search operations should handle large metadata sets without blocking the main thread (async/background tasks). Constraints: Must reuse existing superset_tool library. Must integrate with existing Authentication and TaskManager systems. Scale/Scope: 3 new tools (Search, Mapper, Debug), ~5-7 new API endpoints, ~3-4 new frontend pages.

Constitution Check

GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.

Causal Validity: Contracts (API/Data Models) will be defined before implementation.
Immutability: Architecture layers (superset_tool vs backend) will be respected.
Semantic Format: All new code will follow [DEF] syntax.
Fractal Complexity: New tools will be modularized as Plugins (SearchPlugin, MapperPlugin, DebugPlugin).

Project Structure

Documentation (this feature)

specs/010-refactor-cli-to-web/
├── plan.md              # This file (/speckit.plan command output)
├── research.md          # Phase 0 output (/speckit.plan command)
├── data-model.md        # Phase 1 output (/speckit.plan command)
├── quickstart.md        # Phase 1 output (/speckit.plan command)
├── contracts/           # Phase 1 output (/speckit.plan command)
└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)

Source Code (repository root)

backend/
├── src/
│   ├── api/
│   │   └── routes/
│   │       └── connections.py # New: Connection config endpoints
│   ├── models/
│   │   └── connection.py      # New: DB model for saved connections
│   └── plugins/
│       ├── search.py          # New: SearchPlugin
│       ├── mapper.py          # New: MapperPlugin
│       └── debug.py           # New: DebugPlugin
└── tests/

frontend/
├── src/
│   ├── components/
│   │   ├── tools/             # New: UI components for tools
│   │   │   ├── SearchTool.svelte
│   │   │   ├── MapperTool.svelte
│   │   │   └── DebugTool.svelte
│   ├── routes/
│   │   ├── tools/             # New: Pages
│   │   │   ├── search/
│   │   │   ├── mapper/
│   │   │   └── debug/
│   │   └── settings/
│   │       └── connections/   # New: Connection management
│   └── services/
│       └── toolsService.js    # New: API client

Structure Decision: We will extend the existing backend/ and frontend/ structure. The CLI logic will be refactored into plugins/ in the backend, executed via the generic Task API. The frontend will gain a new "Tools" section that interacts with these plugins.

Complexity Tracking

Fill ONLY if Constitution Check has violations that must be justified

Violation	Why Needed	Simpler Alternative Rejected Because
N/A

4.8 KiB Raw Blame History