docs(validation): update agent configs, skills, ADRs, specs
- qa-tester: add validation v2 testing checklist - speckit.plan: add component reuse scan for frontend - molecular-cot-logging: add Svelte logger + CLI reader - semantics-svelte: add RSM model-first protocol, frontend stack - templates: update plan/tasks/ux-reference templates - ADR-0004: add llm_dashboard_validation plugin registration - ADR-0008: add assistant tool registry for v2 validation - Specs: update 017 tasks from 51 to 99
This commit is contained in:
@@ -20,13 +20,15 @@ ss-tools Core
|
||||
├── core/plugin_executor.py # Subprocess execution, timeout, error boundary
|
||||
├── core/plugin_registry.py # Registered plugins, metadata, health
|
||||
└── plugins/ # Plugin packages (each = one directory)
|
||||
├── llm_analysis/ # LLM‑driven Superset data analysis
|
||||
├── llm_analysis/ # LLM‑driven Superset data analysis; v2 adds dual-path execution (Path A: Playwright screenshot + multimodal LLM; Path B: text-only API + dataset health checking)
|
||||
├── dataset_orchestration/ # LLM dataset operations
|
||||
└── git_integration/ # Git‑based version control for dashboards
|
||||
```
|
||||
|
||||
### Plugin Contract
|
||||
|
||||
> **v2 update:** Task-based validation was introduced via `ValidationTaskService`, creating persistent validation policies with `ValidationRun` aggregates for grouping per-dashboard results. Each run is tracked as a `ValidationRun` record with aggregate pass/fail/warn counts, replacing the previous single-shot task result pattern. See `ValidationTaskService` in `services/validation_service.py`.
|
||||
|
||||
Every plugin MUST provide:
|
||||
|
||||
1. **`plugin.toml`** — metadata manifest at the plugin root
|
||||
|
||||
@@ -69,6 +69,8 @@ Each existing tool handler is moved from `_dispatch.py:_dispatch_intent()` into
|
||||
|
||||
After migration, `_dispatch.py` is reduced to a thin `dispatch()` wrapper. The `_parse_command()` function is updated to also use the registry for keyword-to-operation mapping.
|
||||
|
||||
> **Step 8 update (v2):** `run_llm_validation` now creates a persistent validation task (policy) via `POST /api/validation-tasks` (i.e. `ValidationTaskService.create_task()`) and triggers an immediate run via `ValidationTaskService.trigger_run()`, replacing the previous ad-hoc `task_manager.create_task(plugin_id="llm_dashboard_validation")` dispatch. This makes LLM validation repeatable, schedulable, and auditable — each run is tracked as a `ValidationRun` with pass/fail/warn counts.
|
||||
|
||||
### 6. File layout after migration
|
||||
|
||||
```
|
||||
@@ -106,6 +108,7 @@ backend/src/api/routes/assistant/
|
||||
- **Testable** — `_tool_registry.py` can be unit-tested without DB or HTTP fixtures
|
||||
- **Discoverable** — `from _tool_registry import _tools` gives a complete inventory of all registered tools
|
||||
- **Safe-by-default** — a tool without `@assistant_tool` is not registered and cannot be invoked
|
||||
- **Task-based validation** — `run_llm_validation` now creates persistent validation tasks (policies) via `ValidationTaskService`, making each validation run repeatable, schedulable, and auditable through the `ValidationRun` aggregate model
|
||||
|
||||
### Negative
|
||||
- **One-time migration cost** — 10 existing handlers must be moved to new files (~200 LOC total)
|
||||
|
||||
Reference in New Issue
Block a user