docs: amend constitution to v1.6.0 (add 'Everything is a Plugin' principle) and refactor 010 plan
This commit is contained in:
@@ -1,14 +1,13 @@
|
||||
<!--
|
||||
SYNC IMPACT REPORT
|
||||
Version: 1.5.0 (Fractal Complexity Limit)
|
||||
Version: 1.6.0 (Plugin Architecture Mandate)
|
||||
Changes:
|
||||
- Added Section VI (Fractal Complexity Limit) to enforce strict module (~300 lines) and function (~30-50 lines) size limits.
|
||||
- Aims to maintain semantic coherence and avoid "Attention Sink".
|
||||
- Added Principle VII (Everything is a Plugin) to enforce modular plugin architecture for all features.
|
||||
- Updated Governance section.
|
||||
Templates Status:
|
||||
- .specify/templates/plan-template.md: ✅ Aligned.
|
||||
- .specify/templates/plan-template.md: ✅ Aligned (Dynamic check).
|
||||
- .specify/templates/spec-template.md: ✅ Aligned.
|
||||
- .specify/templates/tasks-arch-template.md: ✅ Aligned (New role-based split).
|
||||
- .specify/templates/tasks-dev-template.md: ✅ Aligned (New role-based split).
|
||||
- .specify/templates/tasks-template.md: ✅ Aligned.
|
||||
-->
|
||||
# Semantic Code Generation Constitution
|
||||
|
||||
@@ -47,6 +46,9 @@ To maintain semantic coherence and avoid "Attention Sink" issues:
|
||||
|
||||
This ensures every vector embedding remains sharp and focused.
|
||||
|
||||
### VII. Everything is a Plugin
|
||||
All functional extensions, tools, or major features must be implemented as modular Plugins inheriting from `PluginBase`. Logic should not reside in standalone services or scripts unless strictly necessary for core infrastructure. This ensures a unified execution model via the `TaskManager`, consistent logging, and modularity.
|
||||
|
||||
## File Structure Standards
|
||||
|
||||
### Python Modules
|
||||
@@ -96,4 +98,4 @@ This Constitution establishes the "Semantic Code Generation Protocol" as the sup
|
||||
- **Review**: Code reviews must verify that implementation matches the preceding contracts and that no "naked code" exists outside of semantic anchors.
|
||||
- **Compliance**: Failure to adhere to the `[DEF]` / `[/DEF]` structure (including matching closing tags) constitutes a build failure.
|
||||
|
||||
**Version**: 1.5.0 | **Ratified**: 2025-12-19 | **Last Amended**: 2025-12-27
|
||||
**Version**: 1.6.0 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-07
|
||||
|
||||
101
specs/010-refactor-cli-to-web/plan.md
Normal file
101
specs/010-refactor-cli-to-web/plan.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Implementation Plan: Refactor CLI Scripts to Web Application
|
||||
|
||||
**Branch**: `010-refactor-cli-to-web` | **Date**: 2026-01-07 | **Spec**: [specs/010-refactor-cli-to-web/spec.md](specs/010-refactor-cli-to-web/spec.md)
|
||||
**Input**: Feature specification from `/specs/010-refactor-cli-to-web/spec.md`
|
||||
|
||||
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
|
||||
|
||||
## Summary
|
||||
|
||||
This project aims to integrate the functionality of standalone CLI tools (`search_script.py`, `run_mapper.py`, `debug_db_api.py`, `get_dataset_structure.py`) into the existing Web Application (FastAPI + SvelteKit). This involves implementing new **Plugins** (`SearchPlugin`, `MapperPlugin`, `DebugPlugin`) to handle dataset searching, column mapping, and system diagnostics. The legacy CLI scripts will be removed after their functionality is successfully ported.
|
||||
|
||||
## Technical Context
|
||||
|
||||
<!--
|
||||
ACTION REQUIRED: Replace the content in this section with the technical details
|
||||
for the project. The structure here is presented in advisory capacity to guide
|
||||
the iteration process.
|
||||
-->
|
||||
|
||||
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
|
||||
**Primary Dependencies**: FastAPI, SvelteKit, Tailwind CSS, Pydantic, SQLAlchemy, `superset_tool` (internal lib)
|
||||
**Storage**: SQLite (for job history/results, connection configs), Filesystem (for temporary file uploads)
|
||||
**Testing**: pytest (Backend), vitest (Frontend - if applicable)
|
||||
**Target Platform**: Linux server (Dockerized environment)
|
||||
**Project Type**: Web Application (Backend + Frontend)
|
||||
**Performance Goals**: Search operations should handle large metadata sets without blocking the main thread (async/background tasks).
|
||||
**Constraints**: Must reuse existing `superset_tool` library. Must integrate with existing Authentication and TaskManager systems.
|
||||
**Scale/Scope**: 3 new tools (Search, Mapper, Debug), ~5-7 new API endpoints, ~3-4 new frontend pages.
|
||||
|
||||
## Constitution Check
|
||||
|
||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
||||
|
||||
1. **Causal Validity**: Contracts (API/Data Models) will be defined before implementation.
|
||||
2. **Immutability**: Architecture layers (`superset_tool` vs `backend`) will be respected.
|
||||
3. **Semantic Format**: All new code will follow `[DEF]` syntax.
|
||||
4. **Fractal Complexity**: New tools will be modularized as Plugins (`SearchPlugin`, `MapperPlugin`, `DebugPlugin`).
|
||||
|
||||
## Project Structure
|
||||
|
||||
### Documentation (this feature)
|
||||
|
||||
```text
|
||||
specs/010-refactor-cli-to-web/
|
||||
├── plan.md # This file (/speckit.plan command output)
|
||||
├── research.md # Phase 0 output (/speckit.plan command)
|
||||
├── data-model.md # Phase 1 output (/speckit.plan command)
|
||||
├── quickstart.md # Phase 1 output (/speckit.plan command)
|
||||
├── contracts/ # Phase 1 output (/speckit.plan command)
|
||||
└── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
|
||||
```
|
||||
|
||||
### Source Code (repository root)
|
||||
<!--
|
||||
ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
|
||||
for this feature. Delete unused options and expand the chosen structure with
|
||||
real paths (e.g., apps/admin, packages/something). The delivered plan must
|
||||
not include Option labels.
|
||||
-->
|
||||
|
||||
```text
|
||||
backend/
|
||||
├── src/
|
||||
│ ├── api/
|
||||
│ │ └── routes/
|
||||
│ │ └── connections.py # New: Connection config endpoints
|
||||
│ ├── models/
|
||||
│ │ └── connection.py # New: DB model for saved connections
|
||||
│ └── plugins/
|
||||
│ ├── search.py # New: SearchPlugin
|
||||
│ ├── mapper.py # New: MapperPlugin
|
||||
│ └── debug.py # New: DebugPlugin
|
||||
└── tests/
|
||||
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── components/
|
||||
│ │ ├── tools/ # New: UI components for tools
|
||||
│ │ │ ├── SearchTool.svelte
|
||||
│ │ │ ├── MapperTool.svelte
|
||||
│ │ │ └── DebugTool.svelte
|
||||
│ ├── routes/
|
||||
│ │ ├── tools/ # New: Pages
|
||||
│ │ │ ├── search/
|
||||
│ │ │ ├── mapper/
|
||||
│ │ │ └── debug/
|
||||
│ │ └── settings/
|
||||
│ │ └── connections/ # New: Connection management
|
||||
│ └── services/
|
||||
│ └── toolsService.js # New: API client
|
||||
```
|
||||
|
||||
**Structure Decision**: We will extend the existing `backend/` and `frontend/` structure. The CLI logic will be refactored into `plugins/` in the backend, executed via the generic Task API. The frontend will gain a new "Tools" section that interacts with these plugins.
|
||||
|
||||
## Complexity Tracking
|
||||
|
||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
||||
|
||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
||||
|-----------|------------|-------------------------------------|
|
||||
| N/A | | |
|
||||
68
specs/010-refactor-cli-to-web/tasks.md
Normal file
68
specs/010-refactor-cli-to-web/tasks.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Tasks: Refactor CLI Scripts to Web Application
|
||||
|
||||
**Feature**: Refactor CLI Scripts to Web Application
|
||||
**Status**: Pending
|
||||
**Spec**: [specs/010-refactor-cli-to-web/spec.md](specs/010-refactor-cli-to-web/spec.md)
|
||||
|
||||
## Phase 1: Core & Infrastructure
|
||||
*Goal: Enhance TaskManager to support returning results and set up Connection management.*
|
||||
|
||||
- [ ] T001 Create `backend/src/models/connection.py` with `ConnectionConfig` model (ensure secure storage of credentials)
|
||||
- [ ] T002 Update `backend/src/core/task_manager/models.py` to add `result: Optional[Dict[str, Any]]` field to `Task` model
|
||||
- [ ] T003 Update `backend/src/core/task_manager/manager.py` to capture `plugin.execute` return value and store it in `task.result`
|
||||
- [ ] T004 [P] Create `frontend/src/services/toolsService.js` for generic Task API communication (run task, poll status, get result)
|
||||
- [ ] T005 [P] Create `frontend/src/services/connectionService.js` for Connection API
|
||||
- [ ] T006 Create `frontend/src/components/tools/` directory structure
|
||||
|
||||
## Phase 2: User Story 1 - Search Datasets
|
||||
*Goal: Enable users to search datasets via the Web UI using a Plugin.*
|
||||
|
||||
- [ ] T007 [US1] Implement `SearchPlugin` in `backend/src/plugins/search.py` (port logic from `search_script.py`, return results)
|
||||
- [ ] T008 [US1] Create `frontend/src/components/tools/SearchTool.svelte` component (triggers SearchPlugin, displays results)
|
||||
- [ ] T009 [US1] Create `frontend/src/routes/tools/search/+page.svelte` page
|
||||
|
||||
## Phase 3: User Story 2 - Map Dataset Columns
|
||||
*Goal: Enable users to map columns using saved connections via a Plugin.*
|
||||
|
||||
- [ ] T010 [US2] Create API endpoints for Connections (`GET`, `POST`, `DELETE`) in `backend/src/api/routes/connections.py`
|
||||
- [ ] T011 [US2] Register `connections` router in `backend/src/api/routes/__init__.py`
|
||||
- [ ] T012 [US2] Implement `MapperPlugin` in `backend/src/plugins/mapper.py` (port logic from `run_mapper.py`, support `connection_id`)
|
||||
- [ ] T013 [US2] Create `frontend/src/components/tools/ConnectionForm.svelte` and `ConnectionList.svelte`
|
||||
- [ ] T014 [US2] Create `frontend/src/routes/settings/connections/+page.svelte` for managing connections
|
||||
- [ ] T015 [US2] Create `frontend/src/components/tools/MapperTool.svelte` component
|
||||
- [ ] T016 [US2] Create `frontend/src/routes/tools/mapper/+page.svelte` page
|
||||
|
||||
## Phase 4: User Story 3 - System Debugging
|
||||
*Goal: Enable users to run system diagnostics via a Plugin.*
|
||||
|
||||
- [ ] T017 [US3] Implement `DebugPlugin` in `backend/src/plugins/debug.py` (port logic from `debug_db_api.py`, `get_dataset_structure.py`)
|
||||
- [ ] T018 [US3] Create `frontend/src/components/tools/DebugTool.svelte` component
|
||||
- [ ] T019 [US3] Create `frontend/src/routes/tools/debug/+page.svelte` page
|
||||
|
||||
## Phase 5: Cleanup & Polish
|
||||
*Goal: Remove legacy scripts and ensure system stability.*
|
||||
|
||||
- [ ] T020 Verify all new tools are functional (Manual Test)
|
||||
- [ ] T021 Verify existing Backup functionality fully covers legacy `backup_script.py` capabilities
|
||||
- [ ] T022 Remove legacy script `search_script.py`
|
||||
- [ ] T023 Remove legacy script `run_mapper.py`
|
||||
- [ ] T024 Remove legacy script `debug_db_api.py`
|
||||
- [ ] T025 Remove legacy script `get_dataset_structure.py`
|
||||
- [ ] T026 Remove legacy script `backup_script.py`
|
||||
- [ ] T027 Update main navigation in `frontend/src/components/Navbar.svelte` to include "Tools" dropdown
|
||||
|
||||
## Dependencies
|
||||
|
||||
- Phase 2 (Search) depends on Phase 1 (Task Result support)
|
||||
- Phase 3 (Mapper) depends on Phase 1 (Connection Model)
|
||||
- Phase 4 (Debug) depends on Phase 1 (Task Result support)
|
||||
- Phase 5 (Cleanup) depends on Phase 2, 3, 4 completion
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
1. **Core**: Upgrade TaskManager first to support return values.
|
||||
2. **Search**: Implement SearchPlugin as the first "Result-returning" plugin.
|
||||
3. **Connections**: Build connection management.
|
||||
4. **Mapper**: Implement MapperPlugin using Connections.
|
||||
5. **Debug**: Implement DebugPlugin.
|
||||
6. **Cleanup**: Verify and delete.
|
||||
Reference in New Issue
Block a user