feat: implement plugin architecture and application settings with Svelte UI
- Added plugin base and loader for backend extensibility - Implemented application settings management with config persistence - Created Svelte-based frontend with Dashboard and Settings pages - Added API routes for plugins, tasks, and settings - Updated documentation and specifications - Improved project structure and developer tools
This commit is contained in:
34
specs/002-app-settings/checklists/requirements.md
Executable file
34
specs/002-app-settings/checklists/requirements.md
Executable file
@@ -0,0 +1,34 @@
|
||||
# Specification Quality Checklist: Add web application settings mechanism
|
||||
|
||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
||||
**Created**: 2025-12-20
|
||||
**Feature**: [specs/002-app-settings/spec.md](specs/002-app-settings/spec.md)
|
||||
|
||||
## Content Quality
|
||||
|
||||
- [x] No implementation details (languages, frameworks, APIs)
|
||||
- [x] Focused on user value and business needs
|
||||
- [x] Written for non-technical stakeholders
|
||||
- [x] All mandatory sections completed
|
||||
|
||||
## Requirement Completeness
|
||||
|
||||
- [x] No [NEEDS CLARIFICATION] markers remain
|
||||
- [x] Requirements are testable and unambiguous
|
||||
- [x] Success criteria are measurable
|
||||
- [x] Success criteria are technology-agnostic (no implementation details)
|
||||
- [x] All acceptance scenarios are defined
|
||||
- [x] Edge cases are identified
|
||||
- [x] Scope is clearly bounded
|
||||
- [x] Dependencies and assumptions identified
|
||||
|
||||
## Feature Readiness
|
||||
|
||||
- [x] All functional requirements have clear acceptance criteria
|
||||
- [x] User scenarios cover primary flows
|
||||
- [x] Feature meets measurable outcomes defined in Success Criteria
|
||||
- [x] No implementation details leak into specification
|
||||
|
||||
## Notes
|
||||
|
||||
- Initial specification covers all requested points with reasonable defaults for authentication and storage validation.
|
||||
102
specs/002-app-settings/plan.md
Executable file
102
specs/002-app-settings/plan.md
Executable file
@@ -0,0 +1,102 @@
|
||||
# Technical Plan: Web Application Settings Mechanism
|
||||
|
||||
This plan outlines the implementation of a settings management system for the Superset Tools application, allowing users to configure multiple Superset environments and global application settings (like backup storage) via the web UI.
|
||||
|
||||
## 1. Backend Architecture
|
||||
|
||||
### 1.1 Data Models (Pydantic)
|
||||
|
||||
We will define models in `backend/src/core/config_models.py`:
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel, Field
|
||||
from typing import List, Optional
|
||||
|
||||
class Environment(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
url: str
|
||||
username: str
|
||||
password: str # Will be masked in UI
|
||||
is_default: bool = False
|
||||
|
||||
class GlobalSettings(BaseModel):
|
||||
backup_path: str
|
||||
default_environment_id: Optional[str] = None
|
||||
|
||||
class AppConfig(BaseModel):
|
||||
environments: List[Environment] = []
|
||||
settings: GlobalSettings
|
||||
```
|
||||
|
||||
### 1.2 Configuration Manager
|
||||
|
||||
A new class `ConfigManager` in `backend/src/core/config_manager.py` will handle:
|
||||
- Loading/saving `AppConfig` to `config.json`.
|
||||
- CRUD operations for environments.
|
||||
- Updating global settings.
|
||||
- Validating backup paths and Superset URLs.
|
||||
|
||||
### 1.3 API Endpoints
|
||||
|
||||
New router `backend/src/api/routes/settings.py`:
|
||||
|
||||
- `GET /settings`: Retrieve all settings (masking passwords).
|
||||
- `PATCH /settings/global`: Update global settings (backup path, etc.).
|
||||
- `GET /settings/environments`: List all environments.
|
||||
- `POST /settings/environments`: Add a new environment.
|
||||
- `PUT /settings/environments/{id}`: Update an environment.
|
||||
- `DELETE /settings/environments/{id}`: Remove an environment.
|
||||
- `POST /settings/environments/{id}/test`: Test connection to a specific environment.
|
||||
|
||||
### 1.4 Integration
|
||||
|
||||
- Update `backend/src/dependencies.py` to provide a singleton `ConfigManager`.
|
||||
- Refactor `superset_tool/utils/init_clients.py` to fetch environment details from `ConfigManager` instead of hardcoded values.
|
||||
|
||||
## 2. Frontend Implementation
|
||||
|
||||
### 2.1 Settings Page
|
||||
|
||||
- Create `frontend/src/pages/Settings.svelte`.
|
||||
- Add a "Settings" link to the main navigation (likely in `App.svelte`).
|
||||
|
||||
### 2.2 Components
|
||||
|
||||
- **EnvironmentList**: Displays a table/list of configured environments with Edit/Delete buttons.
|
||||
- **EnvironmentForm**: A modal or inline form for adding/editing environments.
|
||||
- **GlobalSettingsForm**: Form for editing the backup storage path.
|
||||
|
||||
### 2.3 API Integration
|
||||
|
||||
- Add functions to `frontend/src/lib/api.js` for interacting with the new settings endpoints.
|
||||
|
||||
## 3. Workflow Diagram
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
UI[Web UI - Settings Page] --> API[FastAPI Settings Router]
|
||||
API --> CM[Config Manager]
|
||||
CM --> JSON[(config.json)]
|
||||
CM --> SS[Superset Instance] : Test Connection
|
||||
|
||||
Plugins[Plugins - Backup/Migration] --> CM : Get Env/Path
|
||||
```
|
||||
|
||||
## 4. Implementation Steps
|
||||
|
||||
1. **Backend Core**:
|
||||
- Create `config_models.py` and `config_manager.py`.
|
||||
- Implement file-based persistence.
|
||||
2. **Backend API**:
|
||||
- Implement `settings.py` router.
|
||||
- Register router in `app.py`.
|
||||
3. **Frontend UI**:
|
||||
- Create `Settings.svelte` and necessary components.
|
||||
- Implement API calls and state management.
|
||||
4. **Refactoring**:
|
||||
- Update `init_clients.py` to use the new configuration system.
|
||||
- Ensure existing plugins (Backup, Migration) use the configured settings.
|
||||
5. **Validation**:
|
||||
- Add path existence/write checks for backup storage.
|
||||
- Add URL/Connection checks for Superset environments.
|
||||
77
specs/002-app-settings/spec.md
Executable file
77
specs/002-app-settings/spec.md
Executable file
@@ -0,0 +1,77 @@
|
||||
# Feature Specification: Add web application settings mechanism
|
||||
|
||||
**Feature Branch**: `002-app-settings`
|
||||
**Created**: 2025-12-20
|
||||
**Status**: Draft
|
||||
**Input**: User description: "давай внесем полноценный механизм настройки веб приложения. Что нужно точно - 1. Интерфейс для добавления enviroments (разные сервера суперсета) 2. Интерфейс для настройки файлового хранилища бекапов"
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
|
||||
### User Story 1 - Manage Superset Environments (Priority: P1)
|
||||
|
||||
As an administrator, I want to add, edit, and remove Superset environment configurations (URL, credentials, name) so that the application can interact with multiple Superset instances.
|
||||
|
||||
**Why this priority**: This is the core functionality required for the tool to be useful across different stages (dev/prod) or different Superset clusters.
|
||||
|
||||
**Independent Test**: Can be fully tested by adding a new environment, verifying it appears in the list, and then deleting it.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** the settings page is open, **When** I enter valid Superset connection details and save, **Then** the new environment is added to the list of available targets.
|
||||
2. **Given** an existing environment, **When** I update its URL and save, **Then** the system uses the new URL for subsequent operations.
|
||||
3. **Given** an existing environment, **When** I delete it, **Then** it is no longer available for selection in other parts of the application.
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 - Configure Backup Storage (Priority: P1)
|
||||
|
||||
As an administrator, I want to configure the file path or storage location for backups so that I can control where system backups are stored.
|
||||
|
||||
**Why this priority**: Essential for the backup plugin to function correctly and for users to manage disk space/storage locations.
|
||||
|
||||
**Independent Test**: Can be tested by setting a backup path and verifying that the system validates the path's existence or accessibility.
|
||||
|
||||
**Acceptance Scenarios**:
|
||||
|
||||
1. **Given** the storage settings section, **When** I provide a valid local or network path, **Then** the system saves this as the default backup location.
|
||||
2. **Given** an invalid or inaccessible path, **When** I try to save, **Then** the system displays an error message and does not update the setting.
|
||||
|
||||
---
|
||||
|
||||
### Edge Cases
|
||||
|
||||
- **Duplicate Environments**: What happens when a user tries to add an environment with a name that already exists? (System should prevent duplicates).
|
||||
- **Invalid Credentials**: How does the system handle saving environments with incorrect credentials? (System should ideally validate connection on save).
|
||||
- **Path Permissions**: How does the system handle a backup path that is valid but the application lacks write permissions for? (System should check write permissions).
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST provide a dedicated settings interface in the web UI.
|
||||
- **FR-002**: System MUST allow users to create multiple named "Environments" for Superset.
|
||||
- **FR-003**: Each Environment MUST include: Name, Base URL, and Authentication details (e.g., Username/Password or API Key).
|
||||
- **FR-004**: System MUST allow setting a global "Backup Storage Path".
|
||||
- **FR-005**: System MUST persist these settings across application restarts.
|
||||
- **FR-006**: System MUST validate the Superset URL format before saving.
|
||||
- **FR-007**: System MUST verify that the Backup Storage Path is writable by the application.
|
||||
- **FR-008**: System MUST allow selecting a "Default" environment for operations.
|
||||
|
||||
### System Invariants (Constitution Check)
|
||||
|
||||
- **INV-001**: Sensitive credentials (passwords/keys) MUST NOT be displayed in plain text after being saved.
|
||||
- **INV-002**: At least one environment MUST be configured for the application to perform Superset-related tasks.
|
||||
|
||||
### Key Entities *(include if feature involves data)*
|
||||
|
||||
- **Environment**: Represents a Superset instance. Attributes: Unique ID, Name, URL, Credentials, IsDefault flag.
|
||||
- **AppConfiguration**: Singleton entity representing global settings. Attributes: BackupPath, DefaultEnvironmentID.
|
||||
|
||||
## Success Criteria *(mandatory)*
|
||||
|
||||
### Measurable Outcomes
|
||||
|
||||
- **SC-001**: Users can add a new Superset environment in under 30 seconds.
|
||||
- **SC-002**: 100% of saved environments are immediately available for use in backup/migration tasks.
|
||||
- **SC-003**: System prevents saving invalid backup paths 100% of the time.
|
||||
- **SC-004**: Configuration changes take effect without requiring a manual restart of the backend services.
|
||||
141
specs/002-app-settings/tasks.md
Normal file
141
specs/002-app-settings/tasks.md
Normal file
@@ -0,0 +1,141 @@
|
||||
---
|
||||
|
||||
description: "Task list for implementing the web application settings mechanism"
|
||||
---
|
||||
|
||||
# Tasks: Web Application Settings Mechanism
|
||||
|
||||
**Input**: Design documents from `specs/002-app-settings/`
|
||||
**Prerequisites**: plan.md (required), spec.md (required for user stories)
|
||||
|
||||
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
|
||||
|
||||
## Format: `[ID] [P?] [Story] Description`
|
||||
|
||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
||||
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
|
||||
- Include exact file paths in descriptions
|
||||
|
||||
## Phase 1: Setup (Shared Infrastructure)
|
||||
|
||||
**Purpose**: Project initialization and basic structure
|
||||
|
||||
- [x] T001 Create project structure for settings management in `backend/src/core/` and `backend/src/api/routes/`
|
||||
- [x] T002 [P] Initialize `frontend/src/pages/Settings.svelte` placeholder
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Foundational (Blocking Prerequisites)
|
||||
|
||||
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
|
||||
|
||||
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
|
||||
|
||||
- [x] T003 Implement configuration models in `backend/src/core/config_models.py`
|
||||
- [x] T004 Implement `ConfigManager` for JSON persistence in `backend/src/core/config_manager.py`
|
||||
- [x] T005 [P] Update `backend/src/dependencies.py` to provide `ConfigManager` singleton
|
||||
- [x] T006 [P] Setup API routing for settings in `backend/src/api/routes/settings.py` and register in `backend/src/app.py`
|
||||
|
||||
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 1 - Manage Superset Environments (Priority: P1) 🎯 MVP
|
||||
|
||||
**Goal**: Add, edit, and remove Superset environment configurations (URL, credentials, name) so that the application can interact with multiple Superset instances.
|
||||
|
||||
**Independent Test**: Add a new environment, verify it appears in the list, and then delete it.
|
||||
|
||||
### Implementation for User Story 1
|
||||
|
||||
- [x] T007 [P] [US1] Implement environment CRUD logic in `backend/src/core/config_manager.py`
|
||||
- [x] T008 [US1] Implement environment API endpoints in `backend/src/api/routes/settings.py`
|
||||
- [x] T009 [P] [US1] Add environment API methods to `frontend/src/lib/api.js`
|
||||
- [x] T010 [US1] Implement environment list and form UI in `frontend/src/pages/Settings.svelte`
|
||||
- [x] T011 [US1] Implement connection test logic in `backend/src/api/routes/settings.py`
|
||||
|
||||
**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 2 - Configure Backup Storage (Priority: P1)
|
||||
|
||||
**Goal**: Configure the file path or storage location for backups so that I can control where system backups are stored.
|
||||
|
||||
**Independent Test**: Set a backup path and verify that the system validates the path's existence or accessibility.
|
||||
|
||||
### Implementation for User Story 2
|
||||
|
||||
- [x] T012 [P] [US2] Implement global settings update logic in `backend/src/core/config_manager.py`
|
||||
- [x] T013 [US2] Implement global settings API endpoints in `backend/src/api/routes/settings.py`
|
||||
- [x] T014 [P] [US2] Add global settings API methods to `frontend/src/lib/api.js`
|
||||
- [x] T015 [US2] Implement backup storage configuration UI in `frontend/src/pages/Settings.svelte`
|
||||
- [x] T016 [US2] Add path validation and write permission checks in `backend/src/api/routes/settings.py`
|
||||
|
||||
**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Polish & Cross-Cutting Concerns
|
||||
|
||||
**Purpose**: Improvements that affect multiple user stories
|
||||
|
||||
- [x] T017 Refactor `superset_tool/utils/init_clients.py` to use `ConfigManager` for environment details
|
||||
- [x] T018 Update existing plugins (Backup, Migration) to fetch settings from `ConfigManager`
|
||||
- [x] T019 [P] Add password masking in `backend/src/api/routes/settings.py` and UI
|
||||
- [x] T020 [P] Add "Settings" link to navigation in `frontend/src/App.svelte`
|
||||
- [x] T021 [P] Documentation updates for settings mechanism in `docs/`
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
### Phase Dependencies
|
||||
|
||||
- **Setup (Phase 1)**: No dependencies - can start immediately
|
||||
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
|
||||
- **User Stories (Phase 3+)**: All depend on Foundational phase completion
|
||||
- User stories can then proceed in parallel (if staffed)
|
||||
- Or sequentially in priority order (P1 → P2 → P3)
|
||||
- **Polish (Final Phase)**: Depends on all desired user stories being complete
|
||||
|
||||
### User Story Dependencies
|
||||
|
||||
- **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
|
||||
- **User Story 2 (P1)**: Can start after Foundational (Phase 2) - Independent of US1
|
||||
|
||||
### Parallel Opportunities
|
||||
|
||||
- All Setup tasks marked [P] can run in parallel
|
||||
- All Foundational tasks marked [P] can run in parallel (within Phase 2)
|
||||
- Once Foundational phase completes, all user stories can start in parallel
|
||||
- Models and API methods within a story marked [P] can run in parallel
|
||||
|
||||
---
|
||||
|
||||
## Parallel Example: User Story 1
|
||||
|
||||
```bash
|
||||
# Launch backend and frontend tasks for User Story 1 together:
|
||||
Task: "Implement environment CRUD logic in backend/src/core/config_manager.py"
|
||||
Task: "Add environment API methods to frontend/src/lib/api.js"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Strategy
|
||||
|
||||
### MVP First (User Story 1 Only)
|
||||
|
||||
1. Complete Phase 1: Setup
|
||||
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
|
||||
3. Complete Phase 3: User Story 1
|
||||
4. **STOP and VALIDATE**: Test User Story 1 independently
|
||||
5. Deploy/demo if ready
|
||||
|
||||
### Incremental Delivery
|
||||
|
||||
1. Complete Setup + Foundational → Foundation ready
|
||||
2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
|
||||
3. Add User Story 2 → Test independently → Deploy/Demo
|
||||
4. Each story adds value without breaking previous stories
|
||||
Reference in New Issue
Block a user