ss-tools/specs/028-llm-datasource-supeset/plan.md

Implementation Plan: LLM Table Translation Service

Branch: 028-llm-datasource-supeset | Date: 2026-05-08 | Spec: spec.md Input: Feature specification from /specs/028-llm-datasource-supeset/spec.md (updated 2026-05-14 with multi-language & correction optimization)

Summary

Implement an LLM-powered table translation service as a new backend plugin (TranslationPlugin) with companion API routes, ORM models, Pydantic schemas, and Svelte frontend components. The service reads rows from a Superset datasource (translation column + context columns), auto-detects source language per row via LLM, sends batches to a configured LLM provider for ALL target languages simultaneously (multi-target in single structured JSON response), with optional per-batch filtered multilingual terminology dictionary context (language-pair-aware). Generates safe PostgreSQL INSERT/UPSERT SQL, submits to Superset via /api/v1/sqllab/execute/ with status polling. Supporting capabilities: multilingual terminology dictionaries (source_language + target_language per entry), configurable preview (1-100 rows), inline correction from any run result with dictionary submission, bulk find-and-replace, scheduled execution via APScheduler (new-key-only with baseline_expired fallback), structured event logging with per-language MetricSnapshot persistence, RBAC-gated access control, and 90-day retention with metric continuity.

Total implemented: 72 files, 29 128 LOC across backend (Python/FastAPI) and frontend (Svelte 5/SvelteKit).
Actual complexity distribution: 2× C5, 6× C4, 12× C3, 7× C2, 4× C1.
Note: Архитектура — service-based, не plugin-based. TranslatePlugin — только регистрационный скелет (C2). Бизнес-логика через независимые service-классы.

Technical Context

Language/Version: Python 3.13+ (backend), JavaScript/TypeScript (frontend Svelte 5)
Primary Dependencies: FastAPI 0.115+, SQLAlchemy 2.0+, APScheduler 3.x, Pydantic v2 (backend); SvelteKit 2.x, Svelte 5.43+, Vite 7.x, Tailwind CSS 3.x (frontend)
Storage: PostgreSQL 16 (shared with existing ss-tools schema)
Testing: pytest 8.x + pytest-asyncio (backend); vitest 4.x + @testing-library/svelte 5.x (frontend)
Target Platform: Linux server (Docker Compose: backend + frontend + PostgreSQL)
Project Type: web application — FastAPI REST + WebSocket backend, SvelteKit SPA frontend
Performance Goals: Preview of 10 rows within 30s (LLM-dependent); INSERT generation <5s for 1000 rows; schedule trigger precision ±60s; event recording <10s after occurrence
Constraints: RBAC enforcement via access-control matrix; dictionary per-batch filtering (case-insensitive, word-boundary-aware); 90-day detailed data retention with MetricSnapshot persistence; plugin lifecycle compatible with PluginBase; dialect-aware SQL generation (PostgreSQL/Greenplum, ClickHouse supported for MVP); snapshot isolation for in-progress runs; structured JSON LLM output required
Scale/Scope: Tens of thousands of rows per run; dictionaries of any size; 50 concurrent users; 5–10 active translation jobs per deployment

Constitution Check

GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.

Principle	Status	Evidence
I. Plugin Architecture	✅ PASS	Feature implemented as a new plugin inheriting from PluginBase (backend/src/plugins/translate/), consistent with existing llm_analysis, git, storage plugin pattern.
II. API-First Design	✅ PASS	All operations exposed via REST endpoints under /api/translate/ (jobs, dictionaries, runs, schedules, history) and WebSocket for run progress. Follows existing FastAPI route conventions.
III. Test-First	✅ PASS	Acceptance criteria per user story defined in spec; test strategy in research.md covers unit (pytest), integration (API + DB), and component (vitest + Svelte Testing Library) layers.
IV. RBAC & Security	✅ PASS	Granular permissions (translate.job.*, translate.dictionary.*, translate.schedule.manage, translate.history.view) via existing RBAC model. API keys encrypted via EncryptionManager. PII masking for LLM-facing context per existing patterns.
V. Observability & Retention	✅ PASS	Structured Translation Events (FR-046), per-job metrics (FR-047), notification integration (FR-048), 90-day retention with pruning (FR-049). C4+ flows instrumented with belief_scope/reason/reflect/explore markers.
Semantic Protocol Compliance	✅ PASS	All planned modules assigned @COMPLEXITY 1–5 with appropriate tag density per GRACE complexity scale (C1: anchor only, C2: +BRIEF, C3: +RELATION, C4: +PRE/POST/SIDE_EFFECT, C5: +DATA_CONTRACT/INVARIANT/RATIONALE/REJECTED). @RATIONALE/@REJECTED reserved for C5 contracts only. Canonical @RELATION PREDICATE -> TARGET_ID syntax. Anchor format: #region/#endregion for Python, <!-- #region -->/<!-- #endregion --> for Svelte markup.
Fractal Limit (INV_7)	✅ PASS	Planned modules kept under 400 lines each. No planned contract exceeds 150 lines or CC>10. Decomposition strategy documented in contracts/modules.md.

Gate Result: ✅ ALL PASS — no blocking constitutional or semantic conflicts.

Project Structure

Documentation (this feature)

specs/028-llm-datasource-supeset/
├── plan.md              # This file
├── research.md          # Phase 0 output
├── data-model.md        # Phase 1 output
├── quickstart.md        # Phase 1 output
├── contracts/           # Phase 1 output
│   └── modules.md       # Semantic contract design
├── spec.md              # Feature specification
├── ux_reference.md      # Interaction reference
└── checklists/
    └── requirements.md  # Quality validation

Source Code (repository root — ACTUAL as implemented)

backend/
├── src/
│   ├── api/routes/
│   │   └── translate/             # [ACTUAL] Пакет, а не один файл
│   │       ├── __init__.py        # Инициализация, re-export router
│   │       ├── _router.py         # APIRouter(prefix="/api/translate")
│   │       ├── _job_routes.py     # Job CRUD (249 строк)
│   │       ├── _run_routes.py     # Run execution, cancel, inline edit (554 строки)
│   │       ├── _run_list_routes.py # Run list, detail, CSV download (225 строк)
│   │       ├── _preview_routes.py  # Preview CRUD (194 строки)
│   │       ├── _dictionary_routes.py # Dictionary CRUD + import (391 строка)
│   │       ├── _correction_routes.py # Correction submission (100 строк)
│   │       ├── _schedule_routes.py   # Schedule CRUD (239 строк)
│   │       ├── _metrics_routes.py    # Metrics endpoints (65 строк)
│   │       └── _helpers.py           # Response helpers (71 строка)
│   ├── core/
│   │   ├── scheduler.py           # [REUSE] APScheduler
│   │   └── superset_client/       # [REUSE] Superset API client
│   ├── models/
│   │   └── translate.py           # [NEW] 15 ORM models (397 строк)
│   ├── plugins/
│   │   └── translate/             # [NEW] TranslationPlugin
│   │       ├── __init__.py        # Package (2 строки)
│   │       ├── plugin.py          # Plugin skeleton C2 (61 строка, регистрация)
│   │       ├── orchestrator.py    # C5 Run lifecycle (1 137 строк) ⭐
│   │       ├── executor.py        # C4 Batch LLM (1 974 строки) ⭐
│   │       ├── service.py         # C4 TranslateJobService + InlineCorrection + BulkReplace (1 052 строки) ⭐
│   │       ├── preview.py         # C4 Preview engine (1 303 строки) ⭐
│   │       ├── dictionary.py      # C4 Dictionary CRUD + filtering (1 007 строк) ⭐
│   │       ├── scheduler.py       # C4 Schedule management (436 строк)
│   │       ├── sql_generator.py   # C3 Dialect-aware SQL (378 строк)
│   │       ├── superset_executor.py # C3 Superset SQL Lab API (420 строк) [NEW в реализации]
│   │       ├── prompt_builder.py  # C3 Context-aware prompt (153 строки) [NEW]
│   │       ├── events.py          # C5 Event log + pruning (272 строки)
│   │       ├── metrics.py         # C3 Metrics aggregation (192 строки)
│   │       ├── _token_budget.py   # C2 Token estimation (341 строка) [NEW]
│   │       └── _utils.py          # C1 Normalization (33 строки) [NEW]
│   │       └── __tests__/         # Plugin-level unit tests (8 файлов, 5 645 строк)
│   ├── schemas/
│   │   └── translate.py           # [NEW] 35+ Pydantic schemas (688 строк)
│   └── services/
│       ├── llm_provider.py        # [REUSE] LLM provider management
│       └── llm_prompt_templates.py # [REUSE] Prompt rendering
└── tests/                         # Integration tests (7 файлов, 2 458 строк)

frontend/
├── src/
│   ├── routes/
│   │   └── translate/             # [NEW] SvelteKit routes
│   │       ├── +page.svelte       # Job list (279 строк)
│   │       ├── [id]/
│   │       │   └── +page.svelte   # Job config + preview + run (1 363 строки) ⭐
│   │       ├── dictionaries/
│   │       │   ├── +page.svelte   # Dictionary list (270 строк)
│   │       │   └── [id]/
│   │       │       └── +page.svelte # Dictionary editor (821 строка)
│   │       └── history/
│   │           └── +page.svelte   # Run history (523 строки)
│   └── lib/
│       ├── components/
│       │   └── translate/         # [NEW] Translation components (13 файлов)
│       │       ├── TranslationPreview.svelte       # C4 (561 строка)
│       │       ├── TranslationRunResult.svelte     # C4 (464 строки)
│       │       ├── BulkReplaceModal.svelte         # C3 (396 строк) [NEW]
│       │       ├── TranslationRunProgress.svelte   # C4 (310 строк)
│       │       ├── ScheduleConfig.svelte           # C3 (286 строк)
│       │       ├── BulkCorrectionSidebar.svelte    # C3 (285 строк)
│       │       ├── TermCorrectionPopup.svelte      # C3 (342 строки)
│       │       ├── CorrectionCell.svelte           # C3 (326 строк) [NEW]
│       │       ├── TranslationRunGlobalIndicator.svelte # C2 (198 строк) [NEW]
│       │       ├── TranslationMetricsDashboard.svelte   # C2 (184 строки) [NEW]
│       │       ├── DictionaryEditor.svelte         # C3 (на странице)
│       │       └── DictionaryList.svelte           # C3 (на странице)
│       ├── stores/
│       │   ├── translate.js       # [NEW] rune store (196 строк) [NEW в реализации]
│       │   └── translationRun.js   # [NEW] run progress store
│       └── api/
│           └── translate.js       # [NEW] API client (664 строки)
└── e2e/
    └── tests/
        └── translation.e2e.js     # [NEW] E2E тесты (142 строки)

Semantic Contract Guidance

• All planned contracts follow GRACE-Poly v2.6 protocol with #region/#endregion anchors (or [DEF:...]/[/DEF:...] for backward compatibility in docs).
• Python backend uses # #region/# #endregion comment-anchor syntax.
• Svelte components use <!-- #region -->/<!-- #endregion --> in markup and // #region/// #endregion in script blocks.
• Complexity assignments follow strict tag density per GRACE complexity scale (detailed in contracts/modules.md).
• C4+ Python modules instrumented with belief_scope() + reason()/reflect()/explore() markers.
• C5 contracts carry @RATIONALE and @REJECTED decision memory (C1-C4 MUST NOT have these tags).
• @RELATION targets reference existing contracts where applicable (e.g., [LLMProviderService:Module], [SchedulerService:Class], [SupersetClient:Module], [NotificationService:Module]).

Complexity Tracking

No constitutional violations detected. UPDATED 2026-05-17: Complexity скорректирована по факту реализации.

Contract	Plan (C?)	Actual (C?)	Файл	Примечание
TranslatePlugin:Module	C3	C2	plugin.py	Только регистрация; execute() — NotImplementedError
TranslationOrchestrator	C5	C5 ✅	orchestrator.py	Stateful lifecycle, PRE/POST, multi-step coordination
TranslationEventLog	C5	C5 ✅	events.py	Immutable event log + retention + MetricSnapshot
TranslationExecutor	C4	C4 ✅	executor.py	Batch LLM, caching, parsing, retry
TranslationPreview	C4	C4 ✅	preview.py	Quality gate, approve/edit/reject
DictionaryManager	C4	C4 ✅	dictionary.py	CRUD + import + per-batch filtering
TranslationScheduler	C4	C4 ✅	scheduler.py	APScheduler integration
TranslateJobService	—	C4 [NEW]	service.py	Job CRUD + InlineCorrection + BulkReplace
TranslateRoutes	C3	C4 🔴	__init__.py	Пакет с 11 подмодулями
SQLGenerator	C4	C3 🟢	sql_generator.py	Pure function, нет I/O
SupersetSqlLabExecutor	—	C3 [NEW]	superset_executor.py	Superset SQL Lab API клиент
ContextAwarePromptBuilder	—	C3 [NEW]	prompt_builder.py	Jaccard similarity, priority
InlineCorrectionService	C4	C3 🟢	service.py	Context capture, origin tracking
BulkFindReplaceService	C4	C3 🟢	service.py	Regex, preview, atomic apply
TranslationMetrics	C3	C3 ✅	metrics.py	Metrics aggregation
TranslationModels	C2	C2 ✅	models/translate.py	15 ORM models
TranslateSchemas	C2	C2 ✅	schemas/translate.py	35+ Pydantic schemas
TranslateApiClient	C2	C2 ✅	api/translate.js	API client
translateStore	C3	C3 ✅	translationRun.js	Svelte 5 rune store
TranslationLanguage	C1	C1 ✅	models/translate.py	DTO
TranslationRunLanguageStats	C1	C1 ✅	models/translate.py	DTO
CorrectionCell (Svelte)	—	C3 [NEW]	CorrectionCell.svelte	Inline edit + submit
BulkReplaceModal (Svelte)	—	C3 [NEW]	BulkReplaceModal.svelte	Find & replace modal
Other Svelte components	C3	C3 ✅	—	Все 10 компонентов