- effort-estimate-report.md: updated metrics (~182 files, ~40K LOC), added enhancement breakdown, updated comparative analysis
- spec.md: status → Core + Enhancement complete, added Enhancement Implementation Notes
- plan.md: all enhancement components marked ✅, metrics updated
- tasks.md: all 26 enhancement tasks (T135-T160) → [x], closure summary updated
- quickstart.md: added §10 Direct Database Insert flow
- data-model.md, research.md, contracts/modules.md, ux_reference.md, spec.ru.md, checklists/requirements.md: dates, statuses, metrics aligned
- all documents now reflect 2026-06-11 state: ~174-182 files, ~39-40K LOC, ~580 pytest, ~68 vitest
42 KiB
Semantic Contracts: LLM Table Translation Service
Feature Branch: 028-llm-datasource-supeset (актуальная: 032-translate-requests-httpx)
Date: 2026-06-11 (enhancement: Direct DB Insert + Connection Settings — IMPLEMENTED, ATTN-rules compliance)
Protocol: GRACE-Poly v2.6 — all contracts use ## @{ } doc-brace format with hierarchical IDs and SEMANTICS grouping
Backend Contracts (Python / FastAPI)
@{ Translate.Plugin [C:2] [TYPE Module] [SEMANTICS translate,plugin,registration]
@ingroup Translate @BRIEF Plugin entry point implementing PluginBase for the translation service. Registration artifact for plugin_loader auto-discovery. All business logic delegated to independent service classes. @RELATION INHERITS -> [PluginBase:Class] @RATIONALE Separate plugin avoids bloating LLMAnalysisPlugin beyond fractal limit (<400 lines). C2 because plugin.py is a registration skeleton; all execution logic lives in service classes. @REJECTED Extending LLMAnalysisPlugin would conflate two distinct feature domains (dashboard validation vs table translation). Executing business logic through PluginBase.execute() was rejected in favor of service-layer architecture for testability and route flexibility.
File: backend/src/plugins/translate/plugin.py
@} Translate.Plugin
@{ Translate.Orchestrator [C:5] [TYPE Class] [SEMANTICS translate,orchestration,lifecycle]
@ingroup Translate @BRIEF Central coordinator for translation run lifecycle: validates preconditions, manages preview quality gate, dispatches to executor, generates safe SQL, submits to Superset SQL Lab API OR DbExecutor (direct DB), manages retry, records events, enforces retention. @PRE Job configuration is saved and valid. Superset datasource is accessible. LLM provider reachable. Manual runs: preview accepted. Scheduled runs: prior successful manual run exists. NEW: when insert_method=direct_db, connection_id must reference a valid DatabaseConnection. @POST TranslationRun created with translation_status and insert_status (method recorded). INSERT SQL generated (safe dialect) and executed via SupersetSqlLabExecutor (sqllab) OR DbExecutor (direct_db). For direct_db: connection_snapshot stored, Superset query reference IS NULL. Structured events recorded for all lifecycle transitions. @SIDE_EFFECT Creates TranslationRun, TranslationBatch, TranslationRecord, TranslationLanguage, TranslationEvent, MetricSnapshot rows. Calls LLM provider API (token consumption). Calls Superset SQL Lab API OR native DB driver. @DATA_CONTRACT Input: TranslationJob (config snapshot, includes insert_method + connection_id) + datasource rows → Output: TranslationRun (result, includes insert_method + connection_snapshot) @INVARIANT A run must transition: pending → running → (completed|partial|failed|cancelled|skipped). insert_method ∈ {sqllab, direct_db}. When direct_db: connection_snapshot populated before INSERT, superset_query_id IS NULL. Snapshot isolation: in-progress runs use config snapshot. @RELATION CALLS -> [Translate.Preview] @RELATION CALLS -> [Translate.Executor] @RELATION CALLS -> [Translate.SQLGenerator] @RELATION CALLS -> [Core.SupersetSqlLabExecutor] @RELATION CALLS -> [Core.DbExecutor] @RELATION CALLS -> [Core.ConnectionService] @RELATION CALLS -> [Translate.EventLog] @RELATION CALLS -> [LLMProviderService:Module] @RELATION CALLS -> [SupersetClient:Module] @RATIONALE Centralized orchestrator is needed because preview gating, execution, SQL generation, Superset API / direct DB submission, event logging, and retry share state and must coordinate within a single transaction boundary. Direct DB insert is a PARALLEL path — not a replacement for Superset SQL Lab. Dispatches to correct executor based on job.insert_method. @REJECTED Distributed actor model was rejected — introduces eventual-consistency challenges. Replacing SupersetSqlLabExecutor entirely was rejected — some deployments rely on Superset audit trail. UPDATE statements are never generated — all modifications use INSERT/UPSERT per PostgreSQL dialect.
File: backend/src/plugins/translate/orchestrator.py
@} Translate.Orchestrator
@{ Translate.JobService [C:4] [TYPE Class] [SEMANTICS translate,jobs,service]
@ingroup Translate @BRIEF Central service layer for job CRUD, inline corrections with context capture, and bulk find-and-replace operations. @PRE Job configuration is saved and valid. For corrections: run record must exist with translated values. @POST Job CRUD reflected in database. Inline corrections create/update DictionaryEntry with context_data, usage_notes, origin tracking. Bulk find-and-replace updates TranslationLanguage.final_value. @SIDE_EFFECT Creates/updates DictionaryEntry rows. Updates TranslationLanguage.final_value. Logs correction events. @RELATION CALLS -> [Translate.DictionaryManager] @RELATION CALLS -> [Translate.EventLog] @RELATION CALLS -> [Translate.ContextAwarePrompt] @RELATION CALLS -> [Translate.BulkFindReplace] @RATIONALE C4: coordinates multiple domains (jobs, corrections, bulk operations) with stateful side effects and origin tracking.
File: backend/src/plugins/translate/service.py
@} Translate.JobService
@{ Translate.Preview [C:4] [TYPE Class] [SEMANTICS translate,preview,quality-gate]
@ingroup Translate @BRIEF Fetch configurable sample of source rows (1-100), send to LLM with context + filtered dictionary, return per-language translations for quality gate. Multi-target: single LLM call for ALL target languages. @PRE Job configuration saved with target_languages (list, min 1). Datasource accessible. Sample size configured (default 10, range 1-100). @POST PreviewSession created with config_hash and dict_snapshot_hash. PreviewRecord rows created with per-language TranslationPreviewLanguage entries. Accepting preview gates full execution. For sample_size >30, cost warning displayed. @SIDE_EFFECT Calls LLM provider API (token consumption). Creates PreviewSession, PreviewRecord, TranslationPreviewLanguage rows. @RELATION CALLS -> [LLMProviderService:Module] @RELATION CALLS -> [SupersetClient:Module] @RELATION CALLS -> [Translate.DictionaryManager]
File: backend/src/plugins/translate/preview.py
@} Translate.Preview
@{ Translate.Executor [C:4] [TYPE Class] [SEMANTICS translate,execution,batch-llm]
@ingroup Translate
@BRIEF Process source rows in auto-sized batches through LLM, request translations for ALL target languages in single structured JSON response, parse multi-language results, create per-language TranslationLanguage entries.
@PRE Run exists with translation_status running. Source rows fetched. Batch size configured. LLM provider configured.
@POST All processable rows have TranslationRecord + TranslationLanguage entries. Each batch has TranslationBatch record with statistics. Run statistics updated per language.
@SIDE_EFFECT Calls LLM provider API (token consumption). Creates TranslationBatch, TranslationRecord, TranslationLanguage rows.
@RELATION CALLS -> [LLMProviderService:Module]
@RELATION CALLS -> [Translate.DictionaryManager]
@RELATION CALLS -> [Translate.ContextAwarePrompt]
File: backend/src/plugins/translate/executor.py
@} Translate.Executor
@{ Translate.SQLGenerator [C:3] [TYPE Class] [SEMANTICS translate,sql,dialect]
@ingroup Translate @BRIEF Generate safe dialect-appropriate INSERT/UPSERT SQL from TranslationRecord/TranslationLanguage rows. Supports PostgreSQL/Greenplum (ON CONFLICT) and ClickHouse (plain INSERT). Pure function — no I/O, no mutation. @PRE TranslationRecord rows exist with translated/final values. Target table schema validated. @POST Returns a syntactically valid, injection-safe SQL INSERT/UPSERT statement for the detected dialect. @SIDE_EFFECT None — pure function. @RELATION CALLS -> [SupersetClient:Module] @RATIONALE C3 (not C4 as originally planned): SQLGenerator is a pure function with no state, no I/O, no external side effects.
File: backend/src/plugins/translate/sql_generator.py
@} Translate.SQLGenerator
@{ Core.SupersetSqlLabExecutor [C:3] [TYPE Class] [SEMANTICS superset,sqllab,insert]
@ingroup Core
@BRIEF Submit generated SQL to Superset SQL Lab API /api/v1/sqllab/execute/. Poll /api/v1/query/{id} for completion. Record Superset query reference, status, error details.
@NOTE This executor should NEVER connect directly to a DB. Direct DB is handled by Core.DbExecutor — a separate, parallel contract. See @see Core.DbExecutor.
@PRE TranslationRun exists with translation_status completed/partial. SQL generated and valid. Superset accessible.
@POST TranslationRun.insert_status updated (submitted → running → succeeded|failed). Superset query reference, error details, rows_affected stored.
@SIDE_EFFECT Calls Superset API (SQL execution, database write). Updates TranslationRun row.
@RELATION CALLS -> [SupersetClient:Module]
@RELATION CALLS -> [Translate.EventLog]
@REJECTED Direct database connection bypass — would skip Superset's SQL Lab audit and RBAC. THIS GUARDRAIL APPLIES TO THIS EXECUTOR ONLY. Direct DB execution is provided by Core.DbExecutor for deployments that explicitly opt out of Superset SQL Lab.
File: backend/src/plugins/translate/superset_executor.py
@} Core.SupersetSqlLabExecutor
@{ Translate.DictionaryManager [C:4] [TYPE Class] [SEMANTICS translate,dictionary,terminology]
@ingroup Translate @BRIEF CRUD for TerminologyDictionary and DictionaryEntry with language-pair-aware unique constraint. CSV/TSV import with conflict detection. Per-batch language-pair-filtered term extraction for LLM prompts. @PRE Dictionary model exists. User has required permissions. @POST CRUD operations reflected in database. Filtered term list returned for batch with language-pair matching. @SIDE_EFFECT Creates/updates/deletes DictionaryEntry rows. May read TranslationRun for origin tracking. @RELATION CALLS -> [DictionaryEntry:Class] @RELATION CALLS -> [TranslationJobDictionary:Class] @RELATION CALLS -> [Translate.ContextAwarePrompt]
File: backend/src/plugins/translate/dictionary.py
@} Translate.DictionaryManager
@{ Translate.ContextAwarePrompt [C:3] [TYPE Class] [SEMANTICS translate,prompt,context]
@ingroup Translate @BRIEF Construct LLM prompts with context-aware dictionary entries. Jaccard similarity for priority_context flagging (>50% overlap). Cap context rendering at ~500 tokens per entry. @RELATION DEPENDS_ON -> [Translate.DictionaryManager] @RELATION DEPENDS_ON -> [LLMProviderService:Module]
File: backend/src/plugins/translate/prompt_builder.py
@} Translate.ContextAwarePrompt
@{ Translate.Scheduler [C:4] [TYPE Class] [SEMANTICS translate,schedule,automation]
@ingroup Translate @BRIEF Manage translation job schedules with timezone and concurrency policy support. Register with APScheduler, handle trigger dispatch with skip/queue, enforce new-key-only strategy with baseline_expired fallback. @PRE SchedulerService is running. Job has configuration. Scheduled execution bypass: at least one prior successful manual run exists. @POST Schedule registered with APScheduler or removed. On trigger: new TranslationRun created with trigger_type=scheduled. New-key-only filter applied. On failure: NotificationService called. @SIDE_EFFECT Creates TranslationRun rows. Calls SchedulerService. Emits schedule_triggered/skipped/failed events. @RELATION CALLS -> [SchedulerService:Class] @RELATION CALLS -> [Translate.Orchestrator] @RELATION CALLS -> [Translate.EventLog] @RELATION CALLS -> [NotificationService:Module]
File: backend/src/plugins/translate/scheduler.py
@} Translate.Scheduler
@{ Translate.EventLog [C:5] [TYPE Class] [SEMANTICS translate,events,observability]
@ingroup Translate @BRIEF Structured event logging: write immutable events, enforce invariants (exactly one run_started + one terminal event per run), query for audit/dashboard, enforce 90-day retention pruning with MetricSnapshot persistence. @PRE TranslationEvent table exists. Event type recognized from VALID_EVENT_TYPES. @POST Event row created with type-specific payload. Pruning job persists MetricSnapshot, then removes events/records older than 90 days. @SIDE_EFFECT Creates TranslationEvent rows. Creates MetricSnapshot rows at pruning time. Deletes expired rows. @DATA_CONTRACT Input: (run_id?, job_id, event_type, payload: dict) → Output: TranslationEvent @INVARIANT Every created run MUST have exactly one run_started event and exactly one terminal event. Events are immutable after creation. Cumulative metrics survive pruning via MetricSnapshot. @RELATION CALLS -> [TranslationEvent:Class] @RELATION CALLS -> [MetricSnapshot:Class] @RATIONALE C5: event log is single source of truth for observability, metrics, and audit. Immutability, retention, and metric continuity invariants must be enforced. @REJECTED stdout-only logging lacks structured payload integrity. Event-sourced metrics without snapshots would lose cumulative data after pruning.
File: backend/src/plugins/translate/events.py
@} Translate.EventLog
@{ Translate.Metrics [C:3] [TYPE Class] [SEMANTICS translate,metrics,analytics]
@ingroup Translate @BRIEF Aggregate per-job metrics from live TranslationEvent log AND persistent MetricSnapshot table. Per-language breakdown from per_language_metrics JSON. @RELATION CALLS -> [Translate.EventLog] @RELATION CALLS -> [MetricSnapshot:Class]
File: backend/src/plugins/translate/metrics.py
@} Translate.Metrics
@{ Api.TranslateRoutes [C:4] [TYPE Package] [SEMANTICS api,routes,translate]
@ingroup Api @BRIEF FastAPI route handlers organized as package with 13 sub-modules: job CRUD, run execution, run edit, run list, run history, preview, dictionary, correction, schedule, metrics, helpers. All endpoints enforce RBAC per access-control matrix. @RELATION CALLS -> [Translate.Orchestrator] @RELATION CALLS -> [Translate.JobService] @RELATION CALLS -> [Translate.DictionaryManager] @RELATION CALLS -> [Translate.Scheduler] @RELATION CALLS -> [Translate.EventLog] @RELATION CALLS -> [Translate.Metrics] @RELATION BINDS_TO -> [PermissionChecker:Dependency] @RATIONALE C4: routes package spans 13 files, coordinates 6+ service dependencies, enforces RBAC across 14 permission strings. Split into sub-modules by domain to respect fractal limit. @REJECTED Single translate.py file rejected — would exceed fractal limit at 2 296 lines.
File: backend/src/api/routes/translate/ (package)
@} Api.TranslateRoutes
@{ Models.Translate [C:2] [TYPE Module] [SEMANTICS models,translate,orm]
@ingroup Models @BRIEF 15 SQLAlchemy ORM models: TranslationJob, TranslationRun, TranslationBatch, TranslationRecord, TranslationLanguage, TranslationEvent, TranslationPreviewSession, TranslationPreviewRecord, TranslationPreviewLanguage, TerminologyDictionary, DictionaryEntry, TranslationJobDictionary, TranslationSchedule, MetricSnapshot, TranslationRunLanguageStats. NEW: insert_method + connection_id on TranslationJob; insert_method + connection_snapshot on TranslationRun.
File: backend/src/models/translate.py
@} Models.Translate
@{ Schemas.Translate [C:2] [TYPE Module] [SEMANTICS schemas,translate,pydantic]
@ingroup Schemas @BRIEF 35+ Pydantic v2 request/response schemas for translation API endpoints including multi-language, context-aware, bulk operations, and new insert_method + connection_id fields.
File: backend/src/schemas/translate.py
@} Schemas.Translate
@{ Translate.BulkFindReplace [C:3] [TYPE Class] [SEMANTICS translate,correction,bulk]
@ingroup Translate @BRIEF Apply find-and-replace across translated values in a completed run. Supports regex or literal patterns, filtered by target language. Preview before atomic apply. Optionally submits to dictionary with context. @PRE Run must be completed. Pattern must be valid regex (if regex flag set). @POST All matching cells updated. Optional dictionary entries created with context_source="bulk". @SIDE_EFFECT Updates TranslationLanguage.final_value. Optionally creates DictionaryEntry rows. @RELATION DEPENDS_ON -> [Translate.JobService]
File: backend/src/plugins/translate/service.py (extended)
@} Translate.BulkFindReplace
Data Contracts (C1 DTOs)
@{ Translate.Language [C:1] [TYPE Class] [SEMANTICS translate,data,language]
@ingroup Translate @BRIEF Per-language translation storage for a TranslationRecord. One row per (record_id, language_code). @RELATION DEPENDS_ON -> [TranslationRecord:Class]
File: backend/src/models/translate.py
@} Translate.Language
@{ Translate.PreviewLanguage [C:1] [TYPE Class] [SEMANTICS translate,data,preview]
@ingroup Translate @BRIEF Per-language preview results. Mirrors TranslationLanguage structure for preview records. @RELATION DEPENDS_ON -> [TranslationPreviewRecord:Class]
File: backend/src/models/translate.py
@} Translate.PreviewLanguage
@{ Translate.RunLanguageStats [C:1] [TYPE Class] [SEMANTICS translate,data,statistics]
@ingroup Translate @BRIEF Per-language statistics for a translation run: total_rows, translated_rows, failed_rows, skipped_rows, token_count, estimated_cost per language_code. @RELATION DEPENDS_ON -> [TranslationRun:Class]
File: backend/src/models/translate.py
@} Translate.RunLanguageStats
Frontend Contracts (Svelte 5 / SvelteKit)
@{ Translate.JobList [C:3] [TYPE Component] [SEMANTICS translate,ui,job-list]
@ingroup Translate @BRIEF SvelteKit page listing all translation jobs with status/schedule indicators, target languages as badges, create/duplicate/delete actions. @UX_STATE idle, loading, empty, populated, error @UX_FEEDBACK loading skeleton during data fetch; toast on delete success; schedule status badges (active/paused) @UX_RECOVERY error state → retry button; empty → "Create your first job" CTA
File: frontend/src/routes/translate/+page.svelte
@} Translate.JobList
@{ Translate.JobConfig [C:4] [TYPE Component] [SEMANTICS translate,ui,job-config]
@ingroup Translate @BRIEF Tabbed page with Configuration, Preview, Run, and Schedule sections. Multi-language target selection, datasource column mapping, composite key support, LLM provider/dictionary attachment, sample size slider, inline correction, bulk find-replace, insert method selector. @UX_STATE idle, loading, configured, saving, validation_error, datasource_unavailable @UX_FEEDBACK column list reactivity on datasource change; cost warning at sample size >30; per-language statistics on run completion @UX_RECOVERY datasource_unavailable → "Select replacement datasource" flow; validation_error → highlight invalid fields with inline messages @UX_REACTIVITY column list $derived from datasource; dictionary list filtered by target_language
File: frontend/src/routes/translate/[id]/+page.svelte
@} Translate.JobConfig
@{ Translate.PreviewComponent [C:4] [TYPE Component] [SEMANTICS translate,ui,preview]
@ingroup Translate @BRIEF Side-by-side per-language preview of source rows with LLM translations, per-language approve/edit/reject, bulk approve, configurable sample size slider, cost estimate card, carry-forward edits to full run. @UX_STATE idle, loading, preview_loaded, preview_error, accepted, rejected, stale_config @UX_FEEDBACK spinner during LLM call; visual distinction for LLM-generated vs user-edited vs "und" warning badge @UX_RECOVERY retry preview button; re-fetch with updated config; individual row re-translate @UX_TEST idle → {click: "Preview", expected: loading → preview_loaded with 10 rows}
File: frontend/src/lib/components/translate/TranslationPreview.svelte
@} Translate.PreviewComponent
@{ Translate.RunProgress [C:4] [TYPE Component] [SEMANTICS translate,ui,progress]
@ingroup Translate @BRIEF Live progress display: progress bar, per-language batch counter, success/failure counts, cancel button. WebSocket-driven for real-time updates. @UX_STATE idle, running, cancelling, cancelled, completed, partial, failed, insert_pending, insert_running, insert_failed @UX_FEEDBACK per-language progress $derived; real-time batch/insert status; Superset/Direct-DB insert method badge @UX_RECOVERY retry failed batches; cancel run; download skipped rows as CSV @UX_TEST running → {click: "Cancel", expected: cancelling → cancelled}
File: frontend/src/lib/components/translate/TranslationRunProgress.svelte
@} Translate.RunProgress
@{ Translate.RunResult [C:4] [TYPE Component] [SEMANTICS translate,ui,result]
@ingroup Translate @BRIEF Completion summary with per-language statistics, insert method badge (SQL Lab / Direct DB), connection name (direct DB), generated SQL (audit/debug), inline feedback-loop correction controls. @UX_STATE completed, partial, failed, insert_failed @UX_FEEDBACK insert method badge; Superset execution status badge OR direct DB connection name + latency; per-language statistics section; SQL block for audit @UX_RECOVERY retry failed rows; retry insert; submit corrections; bulk find & replace @UX_TEST completed → {insert_method: "direct_db", expected: "Direct DB · Products DB · 0.8s" badge visible} @RELATION CALLS -> [Translate.TermCorrectionPopup] @RELATION CALLS -> [Translate.CorrectionCell]
File: frontend/src/lib/components/translate/TranslationRunResult.svelte
@} Translate.RunResult
@{ Translate.CorrectionCell [C:3] [TYPE Component] [SEMANTICS translate,ui,correction]
@ingroup Translate @BRIEF Inline-editable cell for translation results. Click to edit, shows "Submit to Dictionary" button after edit. Auto-fills language pair from row metadata. @UX_STATE view, editing, submitting, submitted, error @UX_FEEDBACK toast on submit success/error; "Dictionary ✓" badge on corrected rows @UX_RECOVERY retry button on submit failure; edit persists locally; error → show inline error message @UX_TEST view → {click: cell, expected: editing → type new value → {click: "Submit to Dictionary"} → submitted}
File: frontend/src/lib/components/translate/CorrectionCell.svelte
@RELATION DEPENDS_ON -> [Translate.ApiClient]
@RELATION BINDS_TO -> [Translate.RunStore]
@} Translate.CorrectionCell
@{ Translate.BulkReplaceModal [C:3] [TYPE Component] [SEMANTICS translate,ui,bulk-replace]
@ingroup Translate @BRIEF Modal for bulk find-and-replace across translated values. Pattern input (regex toggle), target language selection, preview table, apply with confirmation. @UX_STATE closed, configuring, previewing, applying, applied, error @UX_FEEDBACK preview table showing before/after; counter of affected rows; warning at >50 rows @UX_RECOVERY error state → retry with adjusted pattern; preview zero matches → "No matches found" with suggestion to adjust pattern @UX_REACTIVITY preview list $derived from pattern + run data @RELATION DEPENDS_ON -> [Translate.ApiClient]
File: frontend/src/lib/components/translate/BulkReplaceModal.svelte
@} Translate.BulkReplaceModal
@{ Translate.TermCorrectionPopup [C:3] [TYPE Component] [SEMANTICS translate,ui,correction-popup]
@ingroup Translate @BRIEF Popup for submitting corrections: source term (pre-filled), incorrect translation, corrected translation, source/target language (auto), dictionary selector, context_data preview (editable/removable), usage notes, conflict dialog. @UX_STATE closed, selecting, editing, submitting, conflict_detected, submitted @UX_FEEDBACK context auto-capture badge (🤖 Auto); prompt preview card; toast on submit @UX_RECOVERY conflict_detected → overwrite / keep existing / cancel dialog; error → retry with error message
File: frontend/src/lib/components/translate/TermCorrectionPopup.svelte
@} Translate.TermCorrectionPopup
@{ Translate.BulkCorrectionSidebar [C:3] [TYPE Component] [SEMANTICS translate,ui,bulk-correction]
@ingroup Translate @BRIEF Sidebar for bulk correction: collect multiple selected terms across rows, per-term correction inputs, submit all to dictionary atomically. @UX_STATE closed, collecting, reviewing, submitting, submitted @UX_FEEDBACK selected terms counter; per-term status indicator (pending/submitted/failed); toast on completion @UX_RECOVERY partial failure → show per-term errors with retry-per-term; empty selection → disable submit with hint @UX_REACTIVITY selected terms list $state
File: frontend/src/lib/components/translate/BulkCorrectionSidebar.svelte
@} Translate.BulkCorrectionSidebar
@{ Translate.ScheduleConfig [C:3] [TYPE Component] [SEMANTICS translate,ui,schedule]
@ingroup Translate @BRIEF Schedule panel: type selector (cron/interval/once), cron expression with validation, interval input, timezone selector, next-3-executions preview, concurrency policy, enable/disable toggle. @UX_STATE idle, editing, validating, enabled, disabled, no_prior_run_warning @UX_FEEDBACK next execution times $derived with timezone display; cron validation inline feedback; status indicator (active/paused) @UX_RECOVERY no_prior_run_warning → "Run manually first to enable schedule" CTA; validation error → highlight invalid field @UX_REACTIVITY next execution times $derived from schedule config @UX_TEST idle → {select: "Cron", enter: "0 6 * * 1", expected: validating → next 3 times displayed}
File: frontend/src/lib/components/translate/ScheduleConfig.svelte
@} Translate.ScheduleConfig
@{ Translate.DictionaryEditor [C:3] [TYPE Component] [SEMANTICS translate,ui,dictionary-editor]
@ingroup Translate @BRIEF Inline term editor with per-row language pair: add/edit/delete entries, CSV/TSV import with conflict preview, export, context_data + usage_notes display in expandable rows, context_source badges. @UX_STATE idle, loading, editing, importing, import_preview, import_conflict, saving @UX_FEEDBACK import preview with duplicate flags; inline save confirmation; context_source badges @UX_RECOVERY import_conflict → per-row resolve (overwrite / keep / skip); saving error → retry with toast
File: frontend/src/routes/translate/dictionaries/[id]/+page.svelte
@} Translate.DictionaryEditor
@{ Translate.DictionaryList [C:3] [TYPE Component] [SEMANTICS translate,ui,dictionary-list]
@ingroup Translate @BRIEF SvelteKit page listing dictionaries with name, term count, attached job count, create/delete actions. Blocks deletion of attached dictionaries. @UX_STATE idle, loading, empty, populated, delete_blocked @UX_FEEDBACK "used by N jobs" counter; delete blocked warning modal with job names @UX_RECOVERY delete_blocked → "Show affected jobs" link → detach and retry; empty → "Create first dictionary" CTA
File: frontend/src/routes/translate/dictionaries/+page.svelte
@} Translate.DictionaryList
@{ Translate.History [C:3] [TYPE Component] [SEMANTICS translate,ui,history]
@ingroup Translate @BRIEF Filterable run history: datasource, target table, per-language row counts, translation_status, insert_status, insert_method badge, date, user. Detail view with config snapshot, prompt, per-language translations with edit marks, INSERT SQL, execution reference. @UX_STATE idle, loading, empty, populated, detail_open, pruned @UX_FEEDBACK insert method badge (SQL Lab / Direct DB); "data unavailable" markers for pruned runs; filter pill counters @UX_RECOVERY empty → "No runs match filters" with clear filters action; pruned → "Run data expired (90+ days)" info
File: frontend/src/routes/translate/history/+page.svelte
@} Translate.History
@{ Translate.MetricsDashboard [C:2] [TYPE Component] [SEMANTICS translate,ui,metrics]
@ingroup Translate @BRIEF Admin dashboard displaying per-job metrics: run counts, success/failure ratio, cumulative tokens, cumulative cost, average latency, per-language breakdown. @UX_STATE idle, loading, populated, error @UX_FEEDBACK loading skeleton with pulse animation; per-language metric cards @UX_RECOVERY error → retry button with "Could not load metrics" message
File: frontend/src/lib/components/translate/TranslationMetricsDashboard.svelte
@} Translate.MetricsDashboard
@{ Translate.RunIndicator [C:2] [TYPE Component] [SEMANTICS translate,ui,indicator]
@ingroup Translate @BRIEF Global indicator showing active translation runs with progress. Polls or WebSocket-driven. @UX_STATE hidden, active, completed @UX_FEEDBACK pulse animation during active runs; per-language progress tooltip @UX_RECOVERY completed → auto-dismiss after 5s (configurable)
File: frontend/src/lib/components/translate/TranslationRunGlobalIndicator.svelte
@} Translate.RunIndicator
@{ Translate.ApiClient [C:2] [TYPE Module] [SEMANTICS translate,api,client]
@ingroup Translate @BRIEF API client wrapping requestApi/fetchApi for all translate endpoints: jobs, runs, preview, dictionaries, entries, corrections, schedules, metrics, bulk replace. NEW: fetchConnections, testConnection.
File: frontend/src/lib/api/translate/ (package)
@} Translate.ApiClient
@{ Translate.RunStore [C:3] [TYPE Store] [SEMANTICS translate,store,state]
@ingroup Translate @BRIEF Svelte 5 rune store for translation feature state: run progress, WebSocket connection status, per-language progress. @RELATION BINDS_TO -> [Translate.ApiClient] @RELATION BINDS_TO -> [TaskWebSocket:Module]
File: frontend/src/lib/stores/translationRun.svelte.ts
@} Translate.RunStore
Integration Contracts (Existing System — Reused)
| Contract ID | What It Provides | How Translation Uses It |
|---|---|---|
[LLMProviderService:Module] |
LLM API call with provider selection, key encryption, retry/fallback | Sends batches with constructed prompts; receives structured JSON translations |
[SupersetClient:Module] |
Superset API: datasource schema, SQL Lab execution, column metadata | Fetch column metadata, submit SQL to /api/v1/sqllab/execute/, poll status |
[SchedulerService:Class] |
APScheduler lifecycle, add_job/remove_job, cron/interval/date triggers | Translation schedules registered as APScheduler jobs |
[NotificationService:Module] |
Email/Telegram/Slack notification dispatch | Scheduled run failure notifications |
[PermissionChecker:Dependency] |
FastAPI dependency for RBAC enforcement | Route handlers annotated per 14-permission access-control matrix |
[TaskWebSocket:Module] |
WebSocket for real-time task progress streaming | Translation run progress events streamed to frontend |
[TaskContext:Class] |
Background task lifecycle context with structured logging | Orchestrator runs as async background task |
Enhancement Contracts — Direct DB Insert + Connection Settings (2026-06-10)
@{ Core.DatabaseConnection [C:1] [TYPE Model] [SEMANTICS settings,connections,database]
@ingroup Core
@BRIEF Pydantic model for a reusable database connection configuration stored in GlobalSettings.connections.
@RATIONALE Typed model prevents config corruption from ad-hoc dicts — field validation at the Pydantic boundary catches misconfiguration before any consumer reads it.
@REJECTED Keeping list[dict] was rejected — would require per-field validation in every consumer. A separate DB table was rejected — connections are configuration, not domain data; storing in app_configurations maintains the existing config persistence pattern.
File: backend/src/core/config_models.py
Fields: id: str (UUID), name: str (unique), host: str, port: int (default 5432), database: str, username: str, password: str (encrypted at rest via EncryptionManager, masked in API), dialect: Literal["postgresql","clickhouse","mysql"], extra_params: dict (optional JSON for sslmode etc.), pool_size: int (default 5), created_at: datetime, updated_at: datetime.
Replaces the existing connections: list[dict] stub in GlobalSettings. Backward-compatible: empty connections: [] → empty list[DatabaseConnection]. connection_id on TranslationJob references the UUID within this list — not a DB FK. Integrity validation is performed by ConnectionService on job save.
@TEST_EDGE Empty connections list migration (backward-compat) @TEST_EDGE Duplicate connection name rejection @TEST_EDGE Unsupported dialect rejection @TEST_EDGE Password masking in serialized output
@} Core.DatabaseConnection
@{ Core.ConnectionService [C:3] [TYPE Class] [SEMANTICS settings,connections,service]
@ingroup Core
@BRIEF CRUD service for DatabaseConnection configurations with encrypted password management and connectivity testing.
@NOTE DbExecutor and ConnectionService live in backend/src/core/ — they are core services, NOT subprocess plugins. ADR-0004's "No DB access for plugins" restriction applies to plugins executed via subprocess (plugin_executor.py), not to in-process core services. See ADR-0004 §"Isolation Guarantees".
@PRE ConfigManager is initialized. EncryptionManager is available.
@POST Connection CRUD reflected in GlobalSettings.connections. Test returns {success, latency_ms, error, db_version}.
@SIDE_EFFECT Reads/writes GlobalSettings via ConfigManager. Opens/closes temporary DB connections during test.
@RELATION CALLS -> [ConfigManager:Class]
@RELATION CALLS -> [EncryptionManager:Module]
@RATIONALE C3 because service has stateful side effects (encryption/decryption, temporary DB connections for testing) and must enforce referential integrity on deletion.
@REJECTED Embedding connection management in translate plugin was rejected — connections are cross-cutting infrastructure. Inline password encryption in API routes was rejected — would scatter encryption logic.
File: backend/src/core/connection_service.py
Operations:
create_connection(data)— encrypt password, append toGlobalSettings.connections, validate uniquenessupdate_connection(id, data)— empty password = keep existing; encrypt new passworddelete_connection(id)— blocked if anyTranslationJob.connection_idmatches (scans all jobs)get_connection(id)— returns with password DECRYPTED (internal use only)list_connections()— returns all with passwords MASKED (********)test_connection(id)— temp native driver connection,SELECT 1, returns latency + diagnostic errorvalidate_connection_refs()— scans all TranslationJobs, verifies referenced connection IDs exist; logs orphans
@TEST_EDGE Create with password encryption (verify stored ≠ plaintext) @TEST_EDGE Update with empty password (password unchanged) @TEST_EDGE Delete blocked by active job reference @TEST_EDGE Test connection success (latency measured) @TEST_EDGE Test connection auth failed (InvalidPasswordError) @TEST_EDGE List returns masked passwords @TEST_EDGE Duplicate name rejected on create @TEST_EDGE validate_connection_refs detects orphaned references
@} Core.ConnectionService
@{ Core.DbExecutor [C:3] [TYPE Class] [SEMANTICS database,execution,insert]
@ingroup Core
@BRIEF Execute SQL directly against a target database using native drivers (asyncpg for PostgreSQL, clickhouse-connect for ClickHouse). Alternative to Core.SupersetSqlLabExecutor — parallel path, NOT a replacement.
@NOTE This is a CORE SERVICE (backend/src/core/), not a subprocess plugin. ADR-0004 plugin DB restriction does NOT apply. The Translate.Orchestrator dispatches to this executor when job.insert_method = direct_db.
@PRE DatabaseConnection configuration is valid and saved. SQL is syntactically valid. Target database is network-reachable from the ss-tools backend.
@POST SQL executed in a transaction. Returns DbExecutionResult(success, rows_affected, execution_time_ms, error).
@SIDE_EFFECT Executes SQL against external database (writes data). Opens/closes native driver connections. Pool lifecycle per-connection.
@DATA_CONTRACT Input: (connection_id: str, sql: str, run_id: str) → Output: DbExecutionResult
@RELATION CALLS -> [Core.ConnectionService]
@RATIONALE Separate executor isolates DB-specific logic from Translate.Orchestrator (C5). Connection pooling, dialect routing, and driver error handling are DB concerns — not orchestration concerns.
@REJECTED Embedding direct DB logic into Translate.Orchestrator was rejected — would bloat C5 beyond fractal limit. SQLAlchemy Core was rejected — asyncpg/clickhouse-connect provide better async performance and dialect-specific features.
File: backend/src/core/db_executor.py
Execution flow: resolve connection → acquire pool → execute SQL in transaction → return result. Pool created lazily per connection_id, cached, recreated on config update.
@TEST_EDGE PostgreSQL INSERT via asyncpg (success) @TEST_EDGE ClickHouse INSERT via clickhouse-connect (success) @TEST_EDGE Connection timeout (asyncio.TimeoutError → error) @TEST_EDGE Auth failure (InvalidPasswordError → error) @TEST_EDGE SQL syntax error (driver error → error) @TEST_EDGE Connection pooling (same pool reused) @TEST_EDGE Transaction atomicity (partial INSERT rolled back)
@} Core.DbExecutor
@{ Settings.ConnectionsTab [C:2] [TYPE Component] [SEMANTICS settings,connections,ui]
@ingroup Settings @BRIEF Settings tab for managing database connections: list, add/edit form, test button, delete with dependency check. @UX_STATE idle, loading, empty, populated, adding, editing, testing, test_success, test_error, deleting, delete_blocked @UX_FEEDBACK spinner during test; toast on save/delete; dialect badge colors (PostgreSQL=blue, ClickHouse=orange, MySQL=teal); masked password with reveal toggle; test result inline with latency/error @UX_RECOVERY test_error → edit connection and retry test; delete_blocked → "Show affected jobs" link in warning modal; form validation error → highlight invalid fields @UX_TEST idle → {click: "Add Connection"} → adding → fill form → {click: "Test"} → testing → test_success → {click: "Save"} → populated @RELATION BINDS_TO -> [SettingsApiClient:Module]
File: frontend/src/routes/settings/ConnectionsTab.svelte
@} Settings.ConnectionsTab
@{ Translate.InsertMethodSelector [C:1] [TYPE Component] [SEMANTICS translate,insert,ui]
@ingroup Translate @BRIEF Radio group for choosing insert method in translation job config: "Superset SQL Lab API" (default) or "Direct Database Connection". @UX_STATE sqllab_selected, direct_db_selected, no_connections, connection_selected, testing_connection, test_success, test_error @UX_FEEDBACK dialect compatibility badge (green=compatible, gray=incompatible filtered out); connection status indicator; "No connections" empty state with CTA link @UX_RECOVERY no_connections → "Go to Settings → Connections" link; test_error → "Connection test failed" with error details @UX_TEST Idle → {click: "Direct DB"} → direct_db_selected → connection dropdown visible; {select: connection} → connection_selected → {click: "Test"} → testing_connection → test_success @RELATION BINDS_TO -> [Translate.ApiClient]
File: frontend/src/lib/components/translate/InsertMethodSelector.svelte
Props: insertMethod, connectionId, jobDialect, onUpdate. Connection dropdown filtered by jobDialect.
@} Translate.InsertMethodSelector
Test Fixtures (2026-06-10)
Backend: test_db_executor.py
| Fixture | Scope | Purpose |
|---|---|---|
test_db_connection |
function | DatabaseConnection model: host=localhost, port=5433, database=test_translate, username=test_user, password=test_pass, dialect=postgresql, pool_size=2 |
mock_asyncpg_pool |
function | AsyncMock for asyncpg.create_pool(). acquire() → mock conn with execute() returning "INSERT 0 5" |
mock_asyncpg_pool_auth_error |
function | Mock pool raising asyncpg.InvalidPasswordError on acquire() |
mock_asyncpg_pool_timeout |
function | Mock pool raising asyncio.TimeoutError on acquire() |
mock_asyncpg_pool_sql_error |
function | Mock pool → execute() raises asyncpg.SyntaxError |
patch_connection_service |
function | unittest.mock.patch for ConnectionService.get_connection() → test_db_connection |
sample_insert_sql_pg |
session | Real PostgreSQL UPSERT SQL with ON CONFLICT DO UPDATE |
sample_insert_sql_ch |
session | Real ClickHouse INSERT SQL |
Backend: test_settings_connections.py
| Fixture | Scope | Purpose |
|---|---|---|
config_manager_empty |
function | ConfigManager with GlobalSettings.connections = [] |
valid_payload_pg |
function | {name: "Test PG", host: "localhost", port: 5433, database: "test_db", username: "tester", password: "secret", dialect: "postgresql"} |
valid_payload_ch |
function | {name: "Test CH", host: "localhost", port: 9000, database: "analytics", username: "default", password: "", dialect: "clickhouse"} |
invalid_payload |
function | Missing database + username → expect 422 |
config_with_two |
function | Pre-populated ConfigManager with 1 PG + 1 CH connection |
mock_encryption |
function | Mock EncryptionManager: encrypt(x) → "enc_" + x, decrypt(x) → x.replace("enc_", "") |
mock_test_success |
function | Mock test_connection() → {success: True, latency_ms: 12, db_version: "PostgreSQL 16.3"} |
mock_test_failure |
function | Mock test_connection() → {success: False, error: "Connection timed out"} |
job_with_conn_ref |
function | TranslationJob with insert_method="direct_db", connection_id=<test_id> |
Backend: test_orchestrator.py (extended)
| Fixture | Scope | Purpose |
|---|---|---|
job_direct_db |
function | TranslationJob with insert_method="direct_db", connection_id=<valid_uuid> |
mock_db_exec_ok |
function | Mock DbExecutor.execute_sql() → DbExecutionResult(success=True, rows_affected=1241, execution_time_ms=850) |
mock_db_exec_fail |
function | Mock DbExecutor.execute_sql() → DbExecutionResult(success=False, error="Host unreachable") |
mock_conn_service |
function | Mock ConnectionService.get_connection() → test DatabaseConnection (postgresql) |
Frontend: InsertMethodSelector.test.ts
| Fixture | Purpose |
|---|---|
mockConnections |
[{id:"c1", name:"Products DB", dialect:"postgresql", host:"db.prod", port:5432}, {id:"c2", name:"CH", dialect:"clickhouse", host:"ch.anal", port:9000}] |
mockEmpty |
[] — tests empty state + Settings link |
mockFetch |
vi.fn() → mockConnections |
mockTest |
vi.fn() → {success: true, latency_ms: 12} |
renderSelector |
Mount with {insertMethod:"sqllab", connectionId:null, jobDialect:"postgresql", onUpdate:vi.fn()} |
Frontend: ConnectionsTab.test.ts
| Fixture | Purpose |
|---|---|
mockList |
3 connections with full fields |
mockCreate |
Valid creation payload |
mockApi |
{fetchConnections→mockList, createConnection→{id:"new"}, testConnection→{success:true, latency_ms:8}, deleteConnection→{success:true}} |
mockDeleteBlocked |
deleteConnection rejects {error:"blocked", blocking_jobs:["Products RU Translation"]} |
renderTab |
Mount with mocked API client via context |