ss-tools/specs/028-llm-datasource-supeset/tasks.md

Tasks: LLM Table Translation Service

Feature Branch: 028-llm-datasource-supeset (актуальная: 032-translate-requests-httpx) Input: Design documents from /specs/028-llm-datasource-supeset/ Prerequisites: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/modules.md

Tests: Test tasks are included for all C4/C5 backend contracts, new API endpoints, and Svelte components with @UX_STATE contracts. Test work traces to contract @PRE/@POST guarantees and spec acceptance scenarios.

Organization: Tasks are grouped by user story to enable independent implementation and testing of each story.

Format: [ID] [P?] [Story] Description

• [P]: Can run in parallel (different files, no dependencies)
• [Story]: Which user story this task belongs to (e.g., US1, US5)
• Include exact file paths in descriptions

---

Phase 1: Setup (Shared Infrastructure)

Purpose: Create plugin directory structure and register the new route module in the lazy-import registry.

• T001 Create translation plugin directory structure: backend/src/plugins/translate/__init__.py, backend/src/plugins/translate/plugin.py (empty skeleton), plus backend/src/plugins/translate/__tests__/__init__.py
• T002 Register translate route module in backend/src/api/routes/__init__.py — add "translate" to __all__ list inside [DEF:Route_Group_Contracts:Block]

---

Phase 2: Foundational (Blocking Prerequisites)

Purpose: ORM models, Pydantic schemas, plugin boilerplate, route skeleton, and database migration. ALL user stories depend on these artifacts.

⚠️ CRITICAL: No user story work can begin until this phase is complete.

ORM Models

• T003 [P] Create all SQLAlchemy ORM models in backend/src/models/translate.py: TranslationJob, TranslationRun, TranslationBatch, TranslationRecord, TranslationEvent, TranslationPreviewSession, TranslationPreviewRecord, TerminologyDictionary, DictionaryEntry, TranslationSchedule, TranslationJobDictionary, MetricSnapshot. Follow patterns from backend/src/models/llm.py (UUID PKs, generate_uuid, Base inheritance, JSON columns, UniqueConstraint, indexes, timezone-aware DateTime with callable defaults). Include source_term_normalized column on DictionaryEntry with unique constraint for case-insensitive matching.
• T004 [P] Create Pydantic v2 request/response schemas in backend/src/schemas/translate.py: TranslateJobCreate, TranslateJobUpdate, TranslateJobResponse, DictionaryCreate, DictionaryImport, DictionaryResponse, TermCorrectionSubmit, ScheduleConfig, TranslationRunResponse, TranslationPreviewResponse (with PreviewRow), MetricsResponse. Follow existing backend/src/schemas/ patterns (use BaseModel, Field with defaults/validation)

Plugin Skeleton

• T005 Create TranslatePlugin class in backend/src/plugins/translate/plugin.py inheriting from PluginBase. Implement id, name, description properties. Wire @RELATION INHERITS -> [PluginBase:Class] in contract header. (RATIONALE: separate plugin avoids bloating LLMAnalysisPlugin beyond fractal limit; REJECTED: extending LLMAnalysisPlugin would conflate domains)

Route Skeleton

• T006 Create backend/src/api/routes/translate.py with FastAPI APIRouter (prefix=/api/translate, tags=["translate"]). Define all endpoint stubs with pass bodies for now: CRUD jobs, CRUD dictionaries, preview trigger, run trigger, retry, schedule CRUD, run history, metrics, correction submission, dictionary import. Attach Depends(require_permission(...)) annotations. Register router in backend/src/app.py alongside existing routers.

Database Migration

• T007 Generate Alembic migration for all translate_* tables: translation_jobs, translation_runs, translation_records, translation_events, terminology_dictionaries, dictionary_entries, translation_schedules, translation_job_dictionaries. Run cd backend && alembic revision --autogenerate -m "add translation tables" and alembic upgrade head.

RBAC Registration

• T008 Register 13 permission strings in the RBAC seed/permission store: translate.job.view, translate.job.create, translate.job.edit, translate.job.delete, translate.job.execute, translate.dictionary.view, translate.dictionary.create, translate.dictionary.edit, translate.dictionary.delete, translate.schedule.view, translate.schedule.manage, translate.history.view, translate.metrics.view. Ensure admin role gets all; analyst role gets translate.job.view, translate.job.execute, translate.dictionary.view, translate.history.view. Update role seeding script if needed.

Checkpoint: Foundation ready — models, schemas, plugin, routes, migration, and RBAC all in place. User story implementation can now begin.

---

Phase 3: User Story 1 — Configure Translation Job (Priority: P1) 🎯 MVP

Goal: User can create, edit, delete, and list translation jobs with datasource selection, column mapping, key columns, target table configuration, LLM settings, and dictionary attachment.

Independent Test: Open Configuration form → select Superset datasource → pick translation/context/key columns → specify target table → save → verify job appears in list with correct settings.

Backend — Job CRUD

• T009 [P] [US1] Implement job CRUD service in backend/src/plugins/translate/plugin.py as methods on the TranslatePlugin class: create_job(), update_job(), delete_job(), get_job(), list_jobs(), duplicate_job(). Validate column existence via SupersetClient on create/update (FR-001, FR-002, FR-006). Enforce composite key support (FR-004). Detect virtual columns and warn (US1 acceptance scenario 5).
• T010 [US1] Implement /api/translate/jobs endpoints in backend/src/api/routes/translate.py: POST / (create), GET / (list), GET /{job_id} (get), PUT /{job_id} (update), DELETE /{job_id} (delete), POST /{job_id}/duplicate (duplicate — FR-021). Inject Depends(require_permission("translate.job.*")) per operation.
• T011 [US1] Implement /api/translate/datasources/{datasource_id}/columns endpoint that queries Superset for column metadata (name, type, is_physical flag) and the database dialect (backend/engine) from the connection configuration. Returns column list AND database_dialect field for the frontend. Cache dialect on TranslationJob.database_dialect at save time. Reject unsupported dialects at configuration time (FR-002, dialect detection).

Frontend — Job Config UI

• T012 [P] [US1] Create TranslateApiClient module in frontend/src/lib/api/translate.js: fetchJobs(), createJob(), updateJob(), deleteJob(), duplicateJob(), fetchDatasourceColumns(). Use existing requestApi/fetchApi wrapper pattern.
• T013 [US1] Create TranslationJobList SvelteKit page in frontend/src/routes/translate/+page.svelte: list all jobs with name, datasource, status/schedule indicators, create button, duplicate action. @UX_STATE: idle, loading, empty, populated, error.
• T014 [US1] Create TranslationJobConfig SvelteKit page in frontend/src/routes/translate/[id]/+page.svelte: datasource dropdown → column selectors (translation column, context columns, key columns with [+ Add key] for composite), target table/column inputs, LLM provider selector, target language, batch size, prompt template editor, dictionary attachment (multi-select with priority ordering). @UX_STATE: idle, loading, configured, saving, validation_error, datasource_unavailable. @UX_REACTIVITY: column list $derived from datasource selection.

Verification — US1

• T015 [US1] Write pytest integration tests for job CRUD API in backend/tests/test_translate_jobs.py: test create with valid config, create with missing translation column (expect 422), create with virtual key column (expect warning), update job, delete job, duplicate job. Mock SupersetClient for column metadata.
• T016 [US1] Verify US1 acceptance scenarios against specs/028-llm-datasource-supeset/spec.md User Story 1 (5 scenarios). Run cd backend && pytest backend/tests/test_translate_jobs.py -v.

Checkpoint: Job CRUD fully functional — user can create, edit, list, and duplicate translation jobs with validated column mappings.

---

Phase 4: User Story 5 — Terminology Dictionary Management (Priority: P2)

Goal: User can create, edit, delete dictionaries; add terms inline; import CSV/TSV with duplicate detection; attach dictionaries to jobs with priority ordering.

Independent Test: Create dictionary with 5 terms → import CSV with 50 terms → verify duplicates flagged → attach dictionary to job → verify dictionary appears in job config.

Backend — Dictionary CRUD + Import

• T017 [P] [US5] Implement DictionaryManager class in backend/src/plugins/translate/dictionary.py: create_dictionary(), update_dictionary(), delete_dictionary(), get_dictionary(), list_dictionaries(), add_entry(), edit_entry(), delete_entry(), clear_entries(). Enforce unique source_term per dictionary with conflict resolution (FR-026). Prevent deletion if attached to active/scheduled jobs (FR-030). @COMPLEXITY 4 — instrument with belief_scope/reason/reflect markers at mutation boundaries. (RATIONALE: C4 warranted because dictionary CRUD is stateful and must enforce referential integrity on deletion; REJECTED: pure C3 CRUD without state guards would allow orphaned job-dictionary links)
• T018 [US5] Implement CSV/TSV import in DictionaryManager: parse uploaded content, detect delimiter, create DictionaryEntry rows, preview with duplicate detection, return parse errors with line numbers for malformed rows (FR-025). Add DictionaryImport schema validation.
• T019 [US5] Implement /api/translate/dictionaries endpoints in backend/src/api/routes/translate.py: POST / (create), GET / (list), GET /{dict_id} (get with entries), PUT /{dict_id} (update), DELETE /{dict_id} (delete — blocked if attached), POST /{dict_id}/entries (add entry), PUT /{dict_id}/entries/{entry_id} (edit), DELETE /{dict_id}/entries/{entry_id} (delete), POST /{dict_id}/import (CSV/TSV import with preview).
• T020 [US5] Implement per-batch dictionary filtering logic in DictionaryManager.filter_for_batch(source_texts: list[str]) -> list[dict]: scan batch texts for substrings matching dictionary source_term values; return matched entries in priority order across all attached dictionaries (FR-044). This is consumed by US2 (preview) and US3 (executor).

Frontend — Dictionary UI

• T021 [P] [US5] Add dictionary API methods to frontend/src/lib/api/translate.js: fetchDictionaries(), createDictionary(), updateDictionary(), deleteDictionary(), fetchDictionaryEntries(), addEntry(), editEntry(), deleteEntry(), importDictionary().
• T022 [US5] Create DictionaryList SvelteKit page in frontend/src/routes/translate/dictionaries/+page.svelte: list dictionaries with name, language, term count, attached job count, create/delete actions. @UX_STATE: idle, loading, empty, populated, delete_blocked.
• T023 [US5] Create DictionaryEditor SvelteKit page in frontend/src/routes/translate/dictionaries/[id]/+page.svelte: inline term editor (source_term → target_translation), add/delete rows, CSV/TSV import with conflict preview, export. @UX_STATE: idle, loading, editing, importing, import_preview, import_conflict, saving. @UX_FEEDBACK: import preview with duplicate flags; toast on save.

Verification — US5

• T024 [US5] Write pytest tests for DictionaryManager in backend/src/plugins/translate/__tests__/test_dictionary.py: test create/update/delete, add entry with duplicate detection (expect conflict), import CSV with valid/invalid rows, delete dictionary blocked by active job, per-batch filtering returns matched terms.
• T025 [US5] Verify US5 acceptance scenarios against spec User Story 5 (6 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_dictionary.py -v.

Checkpoint: Dictionary management fully functional — CRUD, import, filtering, and job attachment all work.

---

Phase 5: User Story 2 — Preview Translated Output (Priority: P2)

Goal: User triggers preview on a saved job → system fetches sample rows → sends to LLM with context + dictionary → displays source/context/translation side-by-side → user approves/edits/rejects → preview state saved for execution gate.

Independent Test: Create job + dictionary → click Preview → verify 10 rows shown with LLM translations → approve 8, edit 1, reject 1 → verify state preserved.

Backend — Preview Engine

• T026 [US2] Implement TranslationPreview class in backend/src/plugins/translate/preview.py: preview_rows(job_id, sample_size). Fetch source rows from Superset via SupersetClient; construct LLM prompt using LLMProviderService + llm_prompt_templates.render_prompt() + DictionaryManager.filter_for_batch(); call LLM; return PreviewRow list. @COMPLEXITY 4 — instrument with belief_scope/reason/reflect at LLM call boundaries. (RATIONALE: C4 because preview is stateful (approve/edit/reject lifecycle) and calls external LLM API with side effects; REJECTED: making preview purely read-only without approval state would degrade UX by losing user decisions between preview and execution)
• T027 [US2] Implement token count and cost estimation in preview response: compute estimated tokens from sample → extrapolate to full dataset row count → apply provider pricing → return estimated_total_rows, estimated_tokens, estimated_cost in TranslationPreviewResponse (FR-014).
• T028 [US2] Implement preview quality gate: create persistent TranslationPreviewSession and TranslationPreviewRecord rows with config_hash and dict_snapshot_hash. Preview acceptance gates full execution; rejected preview sample rows are excluded from full run. Preview is a quality gate — unseen rows are processed normally in full run.
• T029 [US2] Implement /api/translate/jobs/{job_id}/preview endpoint: POST triggers preview, returns preview rows with status=pending. Add PUT /api/translate/jobs/{job_id}/preview/rows/{row_key} for approve/edit/reject actions. Add POST /api/translate/jobs/{job_id}/preview/approve-all for bulk approve.

Frontend — Preview UI

• T030 [P] [US2] Add preview API methods to frontend/src/lib/api/translate.js: fetchPreview(), approveRow(), editRow(), rejectRow(), approveAll().
• T031 [US2] Create TranslationPreview component in frontend/src/lib/components/translate/TranslationPreview.svelte: side-by-side table (source, context, LLM translation), approve/edit/reject buttons per row, bulk approve, cost estimate card before full run, row limit input. @UX_STATE: idle, loading, preview_loaded, preview_error, retrying. @UX_FEEDBACK: spinner during LLM call; visual distinction for LLM-generated vs user-edited values; cost estimate reactivity. @UX_RECOVERY: retry preview button; individual row re-translate.
• T032 [US2] Integrate TranslationPreview into TranslationJobConfig page (frontend/src/routes/translate/[id]/+page.svelte) as a tab or collapsible section that appears after job is saved.

Verification — US2

• T033 [US2] Write pytest tests for preview in backend/src/plugins/translate/__tests__/test_preview.py: test preview with valid job, preview with dictionary (verify glossary terms in prompt), preview row approve/edit/reject state transitions, cost estimation accuracy. Mock LLM provider responses.
• T034 [US2] Write vitest component test for TranslationPreview in frontend/src/lib/components/translate/__tests__/TranslationPreview.test.js: test rendering of preview rows, approve/reject/edit interactions, bulk approve behavior. Mock API client.
• T035 [US2] Verify US2 acceptance scenarios against spec User Story 2 (5 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_preview.py -v && cd frontend && npm run test -- --run.

Checkpoint: Preview flows complete — LLM translation with context + dictionary, approve/edit/reject lifecycle, cost estimation.

---

Phase 6: User Story 3 — Execute Translation & Insert Results (Priority: P3)

Goal: User triggers full batch execution → system processes rows in batches → generates INSERT SQL → user copies to SQL Lab or auto-executes → failed batches retryable.

Independent Test: Create job → preview + approve → execute → verify INSERT SQL generated with correct key columns → execute in SQL Lab → verify rows in target table.

Backend — Executor + SQL Generator + Orchestrator

• T036 [US3] Implement SQLGenerator class in backend/src/plugins/translate/sql_generator.py: generate_insert(records: list[TranslationRecord], job: TranslationJob) -> str. Detect dialect from job.database_dialect (cached from Superset connection at save time). Produce safe dialect-appropriate SQL: for PostgreSQL/Greenplum — INSERT INTO "target_table" ("key_cols"..., "target_col") VALUES (...) with quoted identifiers; support upsert_strategy: insert (plain INSERT), skip_existing (ON CONFLICT DO NOTHING), overwrite (ON CONFLICT DO UPDATE). For ClickHouse — INSERT INTO target_table (key_cols..., target_col) VALUES (...); skip_existing warns user (not natively supported); overwrite documented limitation. @COMPLEXITY 3. (RATIONALE: dialect-aware because Superset connections may use ClickHouse or PostgreSQL; REJECTED: PostgreSQL-only would break ClickHouse users; raw identifier interpolation rejected)
• T037 [US3] Implement TranslationExecutor class in backend/src/plugins/translate/executor.py: execute_run(run: TranslationRun, job: TranslationJob). Fetch all source rows from Superset; split into batches; for each batch: call DictionaryManager.filter_for_batch(), construct prompt via LLMProviderService, call LLM, create TranslationRecord rows with status translated/failed/skipped; handle batch-level retry on LLM failure (FR-015); skip NULL translation values (FR-016); reject NULL key values (FR-017); update run statistics. @COMPLEXITY 4 — instrument with belief_scope/reason/reflect at batch boundaries and error paths.
• T038 [US3] Implement TranslationOrchestrator class in backend/src/plugins/translate/orchestrator.py: start_run(job_id, trigger_type). Validate preconditions (job config valid, datasource accessible, LLM provider reachable); create TranslationRun with status running and config/dict snapshots (FR-019, FR-029); dispatch to TranslationExecutor; on completion call SQLGenerator; record TranslationEvent rows via TranslationEventLog (FR-046); enforce state transitions: pending → running → (completed | partial | failed) — no skipping. @COMPLEXITY 5 — full @PRE/@POST/@DATA_CONTRACT/@INVARIANT enforcement with @RATIONALE/@REJECTED. (RATIONALE: central coordinator is C5 because preview, execution, event logging, and retry share run state and must coordinate within a single transaction boundary; REJECTED: distributed actor model would introduce eventual-consistency challenges for status tracking at current scale)
• T039 [US3] Implement TranslationEventLog class in backend/src/plugins/translate/events.py: log_event(run_id, job_id, event_type, payload). Create immutable TranslationEvent row. query_events(job_id, filters) for audit/dashboard. prune_expired() for 90-day retention enforcement (FR-049) — scheduled via APScheduler cleanup job. @COMPLEXITY 5 — @INVARIANT: every run must have exactly one run_started and one terminal event. (RATIONALE: C5 warranted because event log is single source of truth for observability, metrics, and audit; REJECTED: stdout-only logging lacks structured payload integrity and cannot enforce terminal-event invariant)
• T040 [US3] Implement execution endpoints in backend/src/api/routes/translate.py: POST /api/translate/jobs/{job_id}/runs (trigger manual run — creates run, dispatches orchestrator which translates AND submits to Superset API), GET /api/translate/runs/{run_id} (status + statistics + insert_status + superset_query_id), GET /api/translate/runs/{run_id}/records (paginated TranslationRecord list), POST /api/translate/runs/{run_id}/retry (retry failed batches only — FR-015), POST /api/translate/runs/{run_id}/retry-insert (retry Superset insert only without re-translating). Inject Depends(require_permission("translate.job.execute")).

Frontend — Execution UI

• T041 [P] [US3] Add execution API methods to frontend/src/lib/api/translate.js: triggerRun(), fetchRunStatus(), fetchRunRecords(), retryFailedBatches().
• T042 [US3] Create TranslationRunProgress component in frontend/src/lib/components/translate/TranslationRunProgress.svelte: live progress bar (WebSocket-driven from TaskWebSocket), batch counter (N/M), success/failure/skip counts, cancel button. @UX_STATE: idle, running, pausing, cancelled, completed, partial, failed. @UX_FEEDBACK: progress percentage $derived from translated/total; real-time counts. @UX_RECOVERY: retry failed batches button; cancel run; download skipped rows.
• T043 [US3] Create TranslationRunResult component in frontend/src/lib/components/translate/TranslationRunResult.svelte: completion summary (rows translated/failed/skipped, token count, cost, insert_status), Superset execution reference with status badge, generated SQL block for audit/debugging (collapsed by default), retry-insert button. @UX_STATE: completed, partial, failed, insert_failed. @UX_FEEDBACK: Superset execution status badge; SQL block for audit.
• T044 [US3] Integrate TranslationRunProgress and TranslationRunResult into TranslationJobConfig page as the "Run" tab/section.

Verification — US3

• T045 [US3] Write pytest tests for SQLGenerator in backend/src/plugins/translate/__tests__/test_sql_generator.py: test INSERT with single key, composite key — for PostgreSQL dialect AND ClickHouse dialect. Test PostgreSQL UPSERT (ON CONFLICT DO NOTHING, ON CONFLICT DO UPDATE). Test ClickHouse plain INSERT and skip_existing warning. Test NULL key rejection, NULL translation value skipping, identifier quoting per dialect, injection safety. Validate SQL syntax correctness against each dialect.
• T046 [US3] Write pytest tests for executor + orchestrator in backend/src/plugins/translate/__tests__/test_orchestrator.py: test full run lifecycle (pending→running→completed), partial failure (one batch fails, rest succeed), batch retry, event log invariants, NULL handling. Mock LLM provider and SupersetClient.
• T047 [US3] Verify US3 acceptance scenarios against spec User Story 3 (5 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_orchestrator.py backend/src/plugins/translate/__tests__/test_sql_generator.py -v.

Checkpoint: Execution pipeline complete — batch processing, INSERT generation, retry, event logging. User can translate data and insert into target table.

---

Phase 7: User Story 6 — Feedback Loop (Correct → Dictionary) (Priority: P3)

Goal: In run results, user selects incorrect translation → submits correction to dictionary → dictionary updated with origin tracking → next run uses corrected term.

Independent Test: Complete a run → find incorrect translation → open correction popup → submit to dictionary → re-run preview → verify corrected term used.

Backend — Correction Submission

• T048 [US6] Implement correction submission endpoint in backend/src/api/routes/translate.py: POST /api/translate/corrections accepting TermCorrectionSubmit body. Validate target language match between dictionary and job (FR language validation edge case); detect existing entry conflict → return conflict response (FR-032); create DictionaryEntry with origin tracking (origin_run_id, origin_row_key, origin_user_id) per FR-033. Inject Depends(require_permission("translate.dictionary.edit")).
• T049 [US6] Implement bulk correction endpoint: POST /api/translate/corrections/bulk accepting array of TermCorrectionSubmit objects (FR-034). Process atomically — if any conflict is detected, return all conflicts for user resolution before partial apply.

Frontend — Correction UI

• T050 [P] [US6] Add correction API methods to frontend/src/lib/api/translate.js: submitCorrection(), submitBulkCorrections().
• T051 [US6] Create TermCorrectionPopup component in frontend/src/lib/components/translate/TermCorrectionPopup.svelte: text selection on source term and incorrect target translation → popup with source term (pre-filled from source column), incorrect target translation (pre-filled from selection), corrected target translation input, dictionary selector dropdown (filtered by target language), submit button, conflict dialog (overwrite/keep existing/cancel). @UX_STATE: closed, selecting, editing, submitting, conflict_detected, submitted. @UX_FEEDBACK: "Added to Dictionary" badge on corrected row.
• T052 [US6] Create BulkCorrectionSidebar component in frontend/src/lib/components/translate/BulkCorrectionSidebar.svelte: sidebar collecting selected terms across rows, per-term correction inputs, submit all to dictionary. @UX_STATE: closed, collecting, reviewing, submitting, submitted. @UX_REACTIVITY: selected terms list $state.
• T053 [US6] Integrate feedback-loop components into TranslationRunResult (T043) — add selection highlight behavior and correction triggers.

Verification — US6

• T054 [US6] Write pytest tests for correction endpoints in backend/tests/test_translate_corrections.py: test single correction, bulk correction, conflict detection (existing term), cross-language rejection, origin tracking fields populated. Verify corrected term appears in next preview's dictionary filter.
• T055 [US6] Verify US6 acceptance scenarios against spec User Story 6 (5 scenarios). Run cd backend && pytest backend/tests/test_translate_corrections.py -v.

Checkpoint: Feedback loop complete — corrections flow from results → dictionary → next run.

---

Phase 8: User Story 10 — Schedule Translation Jobs (Priority: P3)

Goal: User configures schedule → system triggers runs → new-key-only translation → optional auto-INSERT → failure notification → pause/resume.

Independent Test: Configure schedule (every 5 min for test) → wait for trigger → verify new TranslationRun created → verify only new keys translated → disable schedule → verify no more triggers.

Backend — Schedule Management + Trigger Dispatch

• T056 [US10] Implement TranslationScheduler class in backend/src/plugins/translate/scheduler.py: create_schedule(), update_schedule(), delete_schedule(), enable_schedule(), disable_schedule(), get_next_executions(schedule, n=3) (FR-036). Register schedule with existing SchedulerService via add_job() with cron/interval/date trigger. @COMPLEXITY 4 — instrument with belief_scope/reason/reflect. (RATIONALE: C4 because schedule management is stateful with APScheduler integration, concurrency policy enforcement, and trigger dispatch side effects)
• T057 [US10] Implement schedule trigger handler: _execute_scheduled_translation(job_id). Enforce concurrency policy: check if previous run for same job is still running → skip (log + event) or queue (start after previous completes) per FR-039. If proceeding: create new TranslationRun with trigger_type=scheduled; fetch source rows; apply new-key-only filter (FR-045) — compare current key values against previous successful run's key values; dispatch to TranslationOrchestrator. On failure, send notification via NotificationService (FR-041, FR-048). Schedule remains enabled for next trigger (US10 acceptance scenario 6).
• T058 [US10] Implement Superset SQL Lab API submission for all runs: create SupersetSqlLabExecutor class in backend/src/plugins/translate/superset_executor.py. Submit generated SQL to /api/v1/sqllab/execute/, poll execution status, update TranslationRun.insert_status, superset_query_id, rows_affected, error fields. For scheduled runs, this happens automatically; for manual runs, this happens on user trigger. Record insert_submitted/insert_succeeded/insert_failed events.
• T059 [US10] Implement schedule endpoints in backend/src/api/routes/translate.py: PUT /api/translate/jobs/{job_id}/schedule (create/update), DELETE /api/translate/jobs/{job_id}/schedule (remove), POST /api/translate/jobs/{job_id}/schedule/enable (FR-040), POST /api/translate/jobs/{job_id}/schedule/disable (FR-040). Inject Depends(require_permission("translate.schedule.manage")). Add schedule warning when editing job with active schedule (FR-042).
• T060 [US10] Extend SchedulerService.load_schedules() in backend/src/core/scheduler.py to discover and register active TranslationSchedule rows alongside existing backup schedules (R4).

Frontend — Schedule UI

• T061 [P] [US10] Add schedule API methods to frontend/src/lib/api/translate.js: updateSchedule(), deleteSchedule(), enableSchedule(), disableSchedule().
• T062 [US10] Create ScheduleConfig component in frontend/src/lib/components/translate/ScheduleConfig.svelte: type selector (cron/interval/once), cron expression input with validation, interval input, timezone selector, run-at datetime picker, next-3-executions preview (with timezone), concurrency policy selector (skip/queue), enable/disable toggle with status indicator. Warns if no prior successful manual run exists. @UX_STATE: idle, editing, validating, enabled, disabled, no_prior_run_warning. @UX_REACTIVITY: next execution times $derived from schedule config with timezone display.
• T063 [US10] Integrate ScheduleConfig into TranslationJobConfig page as the "Schedule" tab.

Verification — US10

• T064 [US10] Write pytest tests for scheduler in backend/src/plugins/translate/__tests__/test_scheduler.py: test schedule CRUD, cron expression validation, next-N-executions calculation, trigger dispatch with skip/queue concurrency, new-key-only filter (verify only unseen keys processed), auto-INSERT execution, failure notification, pause/resume, load on SchedulerService start.
• T065 [US10] Verify US7 acceptance scenarios against spec User Story 7 (8 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_scheduler.py -v.

Checkpoint: Scheduling complete — jobs can run automatically on schedule with new-key-only incremental translation and failure recovery.

---

Phase 9: User Story 4 — Translation History & Audit Trail (Priority: P4)

Goal: User views past runs with filterable list; inspects run details (config snapshot, prompt, translations, INSERT SQL); sees edit marks; duplicates job. Admin views metrics dashboard.

Independent Test: Run several translations → open history → filter by datasource → click run → verify config snapshot, prompt, translations with edit marks, INSERT SQL all shown.

Backend — History + Metrics Endpoints

• T066 [US4] Implement history endpoints in backend/src/api/routes/translate.py: GET /api/translate/runs (list with filters: job_id, datasource_id, target_table, status, date_from, date_to, pagination per FR-020), GET /api/translate/runs/{run_id} (detail with config_snapshot, prompt_used, records with llm_translation and user_edit fields visible — FR showing original vs user-edited).
• T067 [US4] Implement TranslationMetrics class in backend/src/plugins/translate/metrics.py: get_job_metrics(job_id) -> MetricsResponse. Aggregate from TranslationEvent table: total runs, success/failure counts, cumulative tokens, cumulative cost, average batch latency (FR-047). @COMPLEXITY 3.
• T068 [US4] Implement metrics endpoint: GET /api/translate/jobs/{job_id}/metrics. Inject Depends(require_permission("translate.history.view")).

Frontend — History + Metrics UI

• T069 [P] [US4] Add history API methods to frontend/src/lib/api/translate.js: fetchRunHistory(), fetchRunDetail(), fetchJobMetrics().
• T070 [US4] Create TranslationHistory SvelteKit page in frontend/src/routes/translate/history/+page.svelte: filterable table (datasource, target table, row count, status, date, user), click-to-expand detail with config snapshot, prompt, translation rows with edit marks, INSERT SQL. @UX_STATE: idle, loading, empty, populated, detail_open. @UX_REACTIVITY: filtered list $derived from filters.
• T071 [US4] Create admin metrics dashboard section (integrated into existing admin pages or standalone) displaying per-job metrics: run counts, success/failure ratio, cumulative tokens, cumulative cost, average latency. Use MetricsResponse schema.

Verification — US4

• T072 [US4] Write pytest tests for history + metrics in backend/tests/test_translate_history.py: test run list with filters, run detail with snapshots, metrics aggregation accuracy, TranslationEvent queryability.
• T073 [US4] Verify US4 acceptance scenarios against spec User Story 4 (4 scenarios). Run cd backend && pytest backend/tests/test_translate_history.py -v.

Checkpoint: History and audit complete — all runs traceable, metrics dashboard populated.

---

Phase 10: Polish & Cross-Cutting Concerns

Purpose: Retention enforcement, notification wiring, semantic audit, quickstart validation, and rejected-path regression protection.

• T074 [P] Implement 90-day retention pruning in TranslationEventLog.prune_expired(): run as APScheduler daily cleanup job. BEFORE pruning events/records: persist cumulative metrics as MetricSnapshot row (tokens, cost, run counts). Then prune TranslationRecord, TranslationPreviewRecord, TranslationEvent, and insert_sql/config_snapshot fields older than 90 days. Preserve TranslationRun metadata, MetricSnapshot rows, and superset_query_id. Verify metrics remain accurate post-prune (SC-014). (RATIONALE: metric snapshots prevent cumulative data loss from event pruning; REJECTED: indefinite retention would violate storage constraints)
• T075 [P] Wire scheduled-run failure notification: ensure TranslationScheduler trigger handler calls NotificationService.send() when a scheduled run fails (FR-041, FR-048). Test with mock notification provider.
• T076 [P] Instrument remaining C4/C5 Python flows with belief_scope/reason/reflect/explore markers where missing: TranslationOrchestrator.start_run() (entry/exit), TranslationExecutor.execute_run() (batch boundaries + error paths), DictionaryManager mutation boundaries, TranslationScheduler trigger dispatch. Verify via axiom_semantic_validation belief-runtime audit.
• T077 Run full semantic audit via axiom MCP tools:
  • axiom_semantic_validation audit_contracts --file_path backend/src/plugins/translate/ — verify all #region/#endregion anchors are properly closed, @RELATION targets resolve, no orphan contracts, C4+ contracts have required tag density per GRACE complexity scale
  • axiom_semantic_validation audit_belief_protocol --file_path backend/src/plugins/translate/ — verify @RATIONALE/@REJECTED present on all C5 contracts only (not on C1-C4)
  • axiom_semantic_validation audit_belief_runtime --file_path backend/src/plugins/translate/ — verify belief_scope/reason/reflect/explore markers exist in all C4+ module bodies
  • axiom_semantic_validation impact_analysis --contract_id TranslationOrchestrator:Class — verify no rejected path is accidentally re-enabled
• T078 Run quickstart validation: follow specs/028-llm-datasource-supeset/quickstart.md end-to-end — create dictionary → create job → preview → execute → verify INSERT SQL → submit correction → schedule → view history → verify metrics. Run cd backend && pytest -v, cd frontend && npm run test -- --run, cd backend && ruff check src/plugins/translate/ src/api/routes/translate.py src/models/translate.py src/schemas/translate.py.
• T079 Rejected-path regression guard: add a test case in backend/src/plugins/translate/__tests__/test_orchestrator.py verifying snapshot isolation — changing job config mid-run does NOT invalidate the running TranslationRun. Add a test case in backend/src/plugins/translate/__tests__/test_sql_generator.py verifying that UPDATE statements are never generated (only INSERT/UPSERT per PostgreSQL dialect). Add a test case in backend/src/plugins/translate/__tests__/test_dictionary.py verifying that duplicate source_term entries cannot coexist (UniqueConstraint enforced) and conflict resolution only offers overwrite/keep-existing. Add a test case in backend/src/plugins/translate/__tests__/test_retention.py verifying metric snapshots are persisted before event pruning and cumulative metrics remain accurate post-prune.
• T080 [P] Implement cancel run endpoint: POST /api/translate/runs/{run_id}/cancel in backend/src/api/routes/translate.py. Set translation_status=cancelled, mark in-progress batches as failed, do NOT submit INSERT SQL. Emit run_cancelled event. Inject Depends(require_permission("translate.job.execute")).
• T081 [P] Implement download skipped rows endpoint: GET /api/translate/runs/{run_id}/skipped.csv returning CSV of rows skipped due to NULL keys or translation failures. Use key_hash for efficient lookup.
• T082 [P] Compute key_hash for TranslationRecord and TranslationPreviewRecord: hash(canonical_json(key_values)) at creation time. Add config_hash for TranslationRun and TranslationPreviewSession: hash of effective config (columns, keys, target, prompt, dictionaries). Use for idempotency checks, new-key-only filtering, and stale preview detection.

---

Phase 11: Multi-Language & Correction Optimization ✅ COMPLETE (2026-05-14)

Purpose: Implement multi-language support (auto-detection, multi-target, per-language storage, multilingual dictionaries, improved correction workflow). All tasks below are NEW additions building on the existing single-target foundation.

Shared Infrastructure (Migration)

• T083 [P] Add TranslationLanguage, TranslationPreviewLanguage, TranslationRunLanguageStats models to backend/src/models/translate.py. Add Alembic migration. Existing TranslationRecord gains languages relationship; existing TranslationPreviewRecord gains languages relationship. Foreign keys and cascades.
• T084 [P] Add source_language, target_language columns to DictionaryEntry in backend/src/models/translate.py. Update unique constraint to (dictionary_id, source_term_normalized, source_language, target_language). Add migration.
• T085 [P] Migrate TranslationJob.target_language (str) → target_languages (JSON list) in backend/src/models/translate.py. Add migration script for existing jobs: target_languages = [old_target_language]. Keep source_language as optional fallback hint.
• T086 [P] Update Pydantic schemas in backend/src/schemas/translate.py: TranslateJobCreate gets target_languages: list[str] (required, min 1), source_language optional. Add TranslationLanguageResponse, TranslationPreviewLanguageResponse, BulkFindReplaceRequest, InlineCorrectionSubmit schemas.

US6 (NEW) — Auto-Detected Source Language

• T087 [P] [US6] Update LLM prompt construction in backend/src/plugins/translate/executor.py and backend/src/plugins/translate/preview.py: instruct LLM to return detected_source_language (BCP-47) alongside each row's translations. Parse and store per-row in TranslationLanguage.source_language_detected.
• T088 [P] [US6] Update TranslationPreview to display detected source language per row in preview results. Add detected_source_language field to preview response. Flag rows with "und" for manual review.
• T089 [P] [US6] Update TranslationExecutor to store detected source language per TranslationLanguage entry. Handle "und" (undetermined) rows — mark as flagged, allow manual override.
• T090 [US6] Write pytest tests for auto-detection: test with known source languages (en, fr, de, es), test with mixed-language content (expect "und"), test manual override of "und" rows.

US7 (NEW) — Multilingual Dictionaries

• T091 [P] [US7] Update DictionaryManager (backend/src/plugins/translate/dictionary.py): add source_language and target_language to all CRUD operations. Update unique constraint enforcement. Add language-pair filtering to filter_for_batch() method.
• T092 [P] [US7] Update dictionary import (frontend/src/routes/translate/dictionaries/[id]/+page.svelte): CSV/TSV import now accepts source_language, target_language columns (or defaults from UI). Add language pair columns to the entry table. Add filter bar by language pair.
• T093 [P] [US7] Update filter_for_batch() in backend/src/plugins/translate/dictionary.py: accept source_language and target_language parameters. Only return entries where source_language matches row's detected language AND target_language matches prompt's target language.
• T094 [P] [US7] Migrate existing single-language dictionaries: each entry gets source_language = job.source_language or "und", target_language = dictionary.target_language. Create Alembic migration data script.
• T095 [US7] Write pytest tests for multilingual dictionaries: test language-pair filtering, test migration of old dictionaries, test unique constraint with language pairs, test import with language columns.

US1 (UPDATED) — Multi-Language Job Configuration

• T096 [P] [US1] Update job configuration endpoints in backend/src/api/routes/translate/: POST/GET/PUT jobs now handle target_languages list. Validate min 1 language. Backward-compatible: single-language requests wrapped into list.
• T097 [US1] Update TranslationJobConfig component (frontend/src/routes/translate/[id]/+page.svelte): "Target Language" dropdown → multi-select with search. "Include source language as reference" checkbox. Source language field removed.
• T098 [US1] Update TranslationJobList component (frontend/src/routes/translate/+page.svelte): display target languages as badges/tags. Update job card layout.

US2 (UPDATED) — Multi-Language Preview

• T099 [P] [US2] Update TranslationPreview (backend/src/plugins/translate/preview.py): send source text to LLM once, request translations for ALL target languages + detected source language. Parse multi-language response. Create TranslationPreviewLanguage rows per language.
• T100 [P] [US2] Update preview endpoints in backend/src/api/routes/translate/: accept sample_size parameter (1-100, default 10). Return per-language preview data via TranslationPreviewLanguageResponse. Include estimated tokens/cost for multi-target.
• T101 [US2] Update TranslationPreview component (frontend/src/lib/components/translate/TranslationPreview.svelte): dynamic per-language columns. Sample size slider (1-100) with cost warning at >30. Per-language accept button. Added "und" warning badge for undetermined source language.
• T102 [P] [US2] Implement preview edit carry-forward: when preview is accepted with edited rows, store the edited values per-language. On full execution, check for existing edits by key_hash + language_code and use edited value instead of re-translating.
• T103 [US2] Write pytest tests for multi-language preview: test 2-language preview, test 5-language preview, test sample size boundary (1, 100), test cost estimation accuracy.

US3 (UPDATED) — Multi-Language Execution

• T104 [P] [US3] Update TranslationExecutor (backend/src/plugins/translate/executor.py): construct multi-language LLM prompt (one prompt requesting translations for ALL target languages). Parse multi-language JSON response. Create TranslationLanguage rows per language per record.
• T105 [P] [US3] Update TranslationOrchestrator (backend/src/plugins/translate/orchestrator.py): create TranslationRunLanguageStats rows per language on run start. Update statistics per language as batches complete. Track per-language token/cost.
• T106 [P] [US3] Update run status endpoints in backend/src/api/routes/translate/: return language_stats array in run detail response. Include per-language progress in progress events.
• T107 [US3] Update run result rendering in frontend/src/lib/components/translate/TranslationRunResult.svelte: show per-language statistics section with $derived. Display per-language columns in results table.
• T108 [US3] Write pytest tests for multi-target execution: test 3-language run, test partial failure (one language fails), test single-language backward compatibility, test per-language statistics accuracy.

US8 (NEW) — Improved Correction & Preview Workflow

• T109 [P] [US8] Implement inline correction endpoint in backend/src/api/routes/translate/: PUT /api/translate/runs/{run_id}/records/{record_id}/languages/{language_code} — update final_value, record user_edit. Optional submit_to_dictionary flag with dictionary_id parameter.
• T110 [P] [US8] Implement POST /api/translate/runs/{run_id}/bulk-replace endpoint: accepts BulkFindReplaceRequest (find_pattern, regex flag, replacement_text, target_language). Returns affected record list. On apply: updates all matching TranslationLanguage.final_value; optionally submits to dictionary.
• T111 [P] [US8] Add InlineCorrectionService in backend/src/plugins/translate/service.py: handle dictionary submission from correction — validate language pair, detect conflicts, create/update DictionaryEntry with origin tracking.
• T112 [P] [US8] Add BulkFindReplaceService in backend/src/plugins/translate/service.py: pattern matching across run records, preview generation, atomic apply with optional dictionary submission.
• T113 [US8] Create CorrectionCell component (frontend/src/lib/components/translate/CorrectionCell.svelte): inline editable cell with submit-to-dictionary action. Click → edit → save → "Submit to Dictionary" → popup. Language pair auto-filled.
• T114 [US8] Create BulkReplaceModal component (frontend/src/lib/components/translate/BulkReplaceModal.svelte): find/replace pattern inputs, regex toggle, preview table, apply button with confirmation.
• T115 [US8] Update preview component: add sample size slider (1-100) to preview trigger. Wire cost warning. No additional changes needed.
• T116 [US8] Add API methods to frontend/src/lib/api/translate.js: inlineEditCorrection(), submitCorrectionToDict(), bulkFindReplace(), bulkReplacePreview().
• T117 [US8] Write pytest tests for inline correction: test single correction → dictionary, test duplicate detection, test bulk replace preview accuracy, test bulk replace atomic apply.
• T118 [US8] Write vitest tests for CorrectionCell and BulkReplaceModal components: test edit flow, test submit-to-dictionary, test find/replace preview, test bulk apply.

US8b (NEW) — Context-Aware Correction & Dictionary [2026-05-14]

Purpose: Extend the correction workflow and dictionary with source-row context capture, context editing in popup, usage notes, and context-aware LLM prompt construction.

• T123 [P] [US8b] Add context fields to DictionaryEntry in backend/src/models/translate.py: context_data (JSON, nullable), usage_notes (Text, nullable), has_context (Boolean, default=False), context_source (String, nullable). Generate Alembic migration.
• T124 [P] [US8b] Update InlineCorrectionService (backend/src/plugins/translate/service.py): when a correction is submitted, auto-capture source row's context column values as context_data. Accept optional overrides from request. Accept usage_notes. Handle context_source tagging.
• T125 [P] [US8b] Create ContextAwarePromptBuilder in backend/src/plugins/translate/prompt_builder.py: render dictionary entries with context annotations for has_context=True. Compute Jaccard similarity between entry context_data and incoming row context. Flag >50% overlap as # PRIORITY. Cap context at ~500 tokens per entry.
• T126 [P] [US8b] Update dictionary import/export in API and frontend: CSV/TSV import accepts optional context_data, usage_notes columns. Export includes these fields.
• T127 [P] [US8b] Update BulkFindReplaceService: when submit_to_dictionary_with_context=true, auto-capture context from first matching row per unique term pair. context_source="bulk". Warn if context varies across matched rows for same term.
• T128 [US8b] Update CorrectionCell and TermCorrectionPopup: correction popup shows auto-captured context (with Edit/Remove), usage_notes textarea, context_source badge.
• T129 [US8b] Update BulkReplaceModal: add "Submit to dictionary with context" checkbox + "Usage notes (applied to all)" textarea.
• T130 [US8b] Update DictionaryEditor page: display context_data and usage_notes per entry in expandable row with inline editing. Add context_source badges.
• T131 [US8b] Update filter_for_batch() in backend/src/plugins/translate/dictionary.py: accept row_context parameter. Integrate ContextAwarePromptBuilder for priority matching.
• T132 [US8b] Write pytest tests for context-aware correction: test context capture (20 tests), context editing/removal, Jaccard similarity matching, priority flagging, 500-token truncation, context_source tagging.
• T133 [US8b] Write vitest tests for context UI: context display in popup, context editing, usage notes, badge display, context removal, submit (6 tests).
• T134 [P] [US8b] Update correction API endpoint schema: POST /api/translate/corrections now accepts optional context_data (JSON override), usage_notes (text), keep_context (boolean, default=true). Response includes new context fields. TermCorrectionSubmit now has keep_context: bool = True, endpoint passes it through.

---

Phase 12: User Story 12 — Database Connection Settings (Priority: P1) 🆕

Purpose: Create reusable DB connection configurations in common Settings. This is a prerequisite for direct DB insert (US11).

Independent Test: Navigate to Settings → Connections → Add PostgreSQL connection → Test → Save → Verify connection appears in list and is available in translation job insert method selector.

Backend — Connection Model + Service

• T135 [P] [US12] Create DatabaseConnection Pydantic model in backend/src/core/config_models.py: id: str, name: str (unique), host: str, port: int (default 5432), database: str, username: str, password: str (encrypted at rest), dialect: str (Literal["postgresql", "clickhouse", "mysql"]), extra_params: dict (optional JSON for sslmode, connect_timeout), pool_size: int (default 5), created_at: datetime, updated_at: datetime. Promote GlobalSettings.connections from list[dict] to list[DatabaseConnection]. Add backward-compat migration: existing connections: [] stays []. @RATIONALE: typed model prevents config corruption from ad-hoc dicts; @REJECTED: keeping list[dict] was rejected — would require per-field validation in every consumer.

• T136 [US12] Implement ConnectionService class in backend/src/core/connection_service.py: create_connection(data), update_connection(id, data), delete_connection(id) (blocked if referenced by active jobs — FR-072), get_connection(id), list_connections(), test_connection(id) (opens temp connection, runs SELECT 1, returns latency + success/error — FR-071). Password encrypted via EncryptionManager on save, decrypted on test, masked in API responses. @COMPLEXITY 3 — instrument with belief_scope/reason/reflect at mutation and test boundaries.

• T137 [US12] Implement connection CRUD endpoints in backend/src/api/routes/settings.py: GET /api/admin/settings/connections (list — passwords masked), POST /api/admin/settings/connections (create), PUT /api/admin/settings/connections/{id} (update — empty password = keep existing), DELETE /api/admin/settings/connections/{id} (delete — blocked by ref check), POST /api/admin/settings/connections/{id}/test (test connectivity). All endpoints protected by require_permission("settings.connections.manage"). Inject Depends(get_config_manager).

• T138 [P] [US12] Register settings.connections.manage permission in the RBAC permission store. Ensure admin role gets it. No other roles get it by default.

Frontend — Connections Tab

• T139 [P] [US12] Add connection API methods to frontend/src/lib/api.js (or a new connections.js): fetchConnections(), createConnection(), updateConnection(), deleteConnection(), testConnection(id). Use existing fetchApi/requestApi wrapper pattern.

• T140 [US12] Create ConnectionsTab.svelte component in frontend/src/routes/settings/ConnectionsTab.svelte: connection list table (name, host:port, database, dialect badge, used-by-N counter, created date), "Add Connection" button, inline edit/delete actions. Add/edit form in modal or expandable section: name, host, port, database, username, password (masked), dialect dropdown, extra params (optional textarea), pool size input. "Test Connection" button with result display (success + latency, or error). Delete with dependency warning. @UX_STATE: idle, loading, empty, populated, adding, editing, testing, test_success, test_error, deleting, delete_blocked. @UX_FEEDBACK: spinner during test; toast on save/delete; dialect badge colors; masked password.

• T141 [US12] Register "Connections" tab in frontend/src/routes/settings/settings-utils.ts — add "connections" to SETTINGS_TABS array. Update frontend/src/routes/settings/+page.svelte tab routing to render ConnectionsTab.

• T142 [US12] Wire connection data into consolidated settings API: ensure GET /api/admin/settings/consolidated returns connections (passwords masked) and PATCH /api/admin/settings/consolidated supports updating connections alongside other settings categories.

Verification — US12

• T143 [US12] Write pytest tests for ConnectionService in backend/tests/test_settings_connections.py: test create connection (password encryption verified — stored ≠ plaintext), update connection (password unchanged when empty), delete connection blocked by active job reference, test connection success/failure (mock DB driver), list connections (passwords masked in response), dialect validation (reject unsupported), duplicate name rejection.

• T144 [US12] Write vitest test for ConnectionsTab.svelte in frontend/src/lib/__tests__/ConnectionsTab.test.ts: test render connection list, add connection form, test button states, delete confirmation, masked password display, dialect badge rendering.

• T145 [US12] Verify US12 acceptance scenarios against spec User Story 12 (7 scenarios). Run cd backend && pytest backend/tests/test_settings_connections.py -v && cd frontend && npm run test -- --run.

Checkpoint: DB connections manageable in Settings — create, edit, test, delete with dependency safety. Ready for US11 to consume.

---

Phase 13: User Story 11 — Direct Database Insert (Priority: P2) 🆕

Purpose: Enable direct DB INSERT execution as an alternative to Superset SQL Lab API. Consumes connections from US12.

Independent Test: Configure a DB connection → create translation job with "Direct DB" insert method → execute run → verify rows in target table with no Superset API calls.

Backend — DB Executor

• T146 [US11] Create DbExecutor class in backend/src/core/db_executor.py: execute_sql(connection_id, sql, run_id) — resolves DatabaseConnection from config, opens native driver connection (asyncpg for PostgreSQL, clickhouse-connect for ClickHouse), executes SQL in a transaction, returns {success: bool, rows_affected: int, execution_time_ms: float, error: str|None}. Use connection pooling (per-connection pool, pool_size from config). Handle dialect-specific parameter binding (PostgreSQL $1/$2, ClickHouse %(key)s). Handle errors: timeout, auth failure, network, permission denied. @COMPLEXITY 3 — instrument with belief_scope/reason/reflect at connection acquisition, execution, and error boundaries. @RATIONALE: separate executor isolates DB-specific logic from translate orchestrator; @REJECTED: embedding DB drivers directly in orchestrator would bloat the C5 contract and mix concerns.

• T147 [P] [US11] Add asyncpg and clickhouse-connect to backend/requirements.txt. For development, asyncpg>=0.29.0 and clickhouse-connect>=0.7.0. Add to backend/.env.example notes about driver dependencies.

• T148 [US11] Add insert_method (Literal["sqllab", "direct_db"], default "sqllab") and connection_id (str, nullable — UUID referencing GlobalSettings.connections list, NOT a DB FK) fields to TranslationJob ORM model in backend/src/models/translate.py. Add Alembic migration. Add Pydantic model_validator in TranslateJobCreate/TranslateJobUpdate: if insert_method == "direct_db", connection_id MUST be non-null. Add ConnectionService.validate_connection_refs() — scans all jobs, verifies referenced connection IDs exist in GlobalSettings.connections, logs orphaned references as warnings. Call this on job save and on connection delete.

• T149 [US11] Add insert_method (Literal["sqllab", "direct_db"]) and connection_snapshot (JSON, nullable — stores {name, dialect, host, port, database}, NEVER password) fields to TranslationRun ORM model in backend/src/models/translate.py. Add Alembic migration. connection_snapshot stores non-sensitive connection metadata at run time for audit — survives connection deletion.

• T150 [US11] Update TranslationOrchestrator in backend/src/plugins/translate/orchestrator.py: after SQL generation, check job.insert_method. If sqllab — use existing SupersetSqlLabExecutor. If direct_db — resolve connection from ConnectionService, call DbExecutor.execute_sql(), record result (insert_status, rows_affected, insert_error_message). Before execution, snapshot connection metadata to run.connection_snapshot. @RATIONALE: orchestrator dispatch keeps insert-method logic in one place; @REJECTED: separate orchestrators per method would duplicate run lifecycle logic.

• T151 [US11] Update Pydantic schemas in backend/src/schemas/translate.py: TranslateJobCreate/TranslateJobUpdate accept insert_method (optional, default "sqllab") and connection_id (optional). TranslateJobResponse includes insert_method and connection_id. TranslationRunResponse includes insert_method, connection_snapshot (nullable), insert_status, insert_error_message.

• T152 [US11] Update /api/translate/jobs/{job_id}/runs endpoint: validate connection_id when insert_method = direct_db. Resolve connection dialect and verify it matches job's detected dialect. Include connection info in run response.

Frontend — Insert Method Selector

• T153 [P] [US11] Add connection API methods to frontend/src/lib/api/translate.js (or reuse from US12): fetchConnections() — already covered by T139. Update createJob()/updateJob() to include insert_method and connection_id.

• T154 [US11] Create InsertMethodSelector.svelte component in frontend/src/lib/components/translate/InsertMethodSelector.svelte: radio group: "Superset SQL Lab API" (default, shown with existing Superset env info), "Direct Database Connection" (shown with connection dropdown, filtered by compatible dialect). When "Direct DB" selected but no connections exist → inline message with link to Settings → Connections. When connection selected → optional inline test button. @UX_STATE: sqllab_selected, direct_db_selected, no_connections, connection_selected, testing_connection, test_success, test_error. @UX_FEEDBACK: dialect compatibility badge; connection status indicator; link to Settings when empty.

• T155 [US11] Integrate InsertMethodSelector into ConfigTabForm.svelte (frontend/src/lib/components/translate/ConfigTabForm.svelte) — add it after the target table/column section, before LLM settings. Wire insert_method and connection_id into the job config form state.

• T156 [US11] Update TranslationRunResult.svelte (frontend/src/lib/components/translate/TranslationRunResult.svelte): show insert method badge (SQL Lab / Direct DB). For direct DB runs: show connection name, execution time, rows affected. Hide Superset query reference field (empty for direct DB). For SQL Lab runs: keep existing Superset reference display.

Verification — US11

• T157 [US11] Write pytest tests for DbExecutor in backend/tests/test_db_executor.py: test PostgreSQL INSERT via asyncpg (mock or testcontainer), test ClickHouse INSERT via clickhouse-connect, test connection timeout, test auth failure, test SQL syntax error (record failure, don't crash), test connection pooling (reuse connections), test dialect-specific parameter binding.

• T158 [US11] Write pytest tests for orchestrator direct DB dispatch in backend/src/plugins/translate/__tests__/test_orchestrator.py: extend existing tests with insert_method=direct_db scenarios — test successful direct INSERT, test connection not found, test dialect mismatch, test connection_snapshot stored on run, test retry-insert on direct DB failure.

• T159 [US11] Write vitest test for InsertMethodSelector.svelte: test radio toggle, connection dropdown filtering, empty state with link to settings, connection test button states.

• T160 [US11] Verify US11 acceptance scenarios against spec User Story 11 (8 scenarios). Run cd backend && pytest backend/tests/test_db_executor.py backend/src/plugins/translate/__tests__/test_orchestrator.py -v && cd frontend && npm run test -- --run.

Checkpoint: Direct DB insert fully functional — user can choose insert method, execute directly, and audit results.

---

Dependencies & Execution Order

Phase Dependencies

• Setup (Phase 1): No dependencies — can start immediately
• Foundational (Phase 2): Depends on Setup — BLOCKS all user stories
• US1 (Phase 3): Depends on Foundational — no dependencies on other stories. Recommended start after Foundational.
• US5 (Phase 4): Depends on Foundational — can run in parallel with US1. Dictionary filtering will be consumed later.
• US2 (Phase 5): Depends on US1 (needs saved job) + US5 (needs dictionary filtering). Can start once US1 backend is stable.
• US3 (Phase 6): Depends on US1 + US2. Sequential after US2.
• US6/10 (Phase 7/8): Original US6+US10 (corrections, scheduling). Depends on US3.
• US4 (Phase 9): Depends on US3.
• Polish (Phase 10): Depends on all desired user stories.
• Phase 11 (Multi-Language) [NEW]:
  • Migration (T083-T086): Blocks ALL Phase 11 work — models, schemas, migration scripts MUST be done first
  • US6 auto-detection (T087-T090): Depends on migration. No dependencies on other Phase 11 stories.
  • US7 multilingual dict (T091-T095): Depends on migration. Can run in parallel with US6 auto-detection.
  • US1 updated (T096-T098): Depends on migration + US7 (for language-aware dict attachment). Can start after migration.
  • US2 preview (T099-T103): Depends on migration + US6 (auto-detection for preview display).
  • US3 execution (T104-T108): Depends on migration + US6 + US7. Sequential after preview changes.
  • US8 correction (T109-T118): Depends on migration + US3 (needs multi-language run results).
  • US8b context (T123-T134): Depends on US8 (inline correction exists) + US7 (dictionary filtering). Can start after T109-T114 are stable. Fits in US8's parallel slot.
  • US4 history (T119-T122): Depends on US3 (per-language run data). Can run in parallel with US8/US8b.
• Phase 12 (US12 — Connection Settings) 🆕: Depends on Foundational (Phase 2) only — can start immediately. BLOCKS US11.
• Phase 13 (US11 — Direct DB Insert) 🆕: Depends on US12 (needs connections) + US3 (needs orchestrator execution flow). Can start once US12 backend is stable.

Parallel Opportunities

Phase	Parallel Tasks	Notes
1	—	Sequential (only 2 tasks)
2	T003 ∥ T004	Models + Schemas in parallel
3 (US1)	T009 ∥ T012	Backend CRUD ∥ API client
4 (US5)	T017 ∥ T021	DictionaryManager ∥ API client
5 (US2)	T030 ∥ T031	API client ∥ Preview component
6 (US3)	T036 ∥ T041	SQLGenerator ∥ API client
7 (US6)	T050 ∥ T051 ∥ T052	API client ∥ Popup ∥ Sidebar
8 (US10)	T061 ∥ T062	API client ∥ ScheduleConfig
9 (US4)	T069 ∥ T070	API client ∥ History page
10	T074 ∥ T075 ∥ T076	Retention, notifications, belief instrumentation
11a (Migration)	T083 ∥ T084 ∥ T085 ∥ T086	All model/schema changes in parallel [NEW]
11b (US6 + US7)	T087 ∥ T088 ∥ T089 ∥ T091 ∥ T092 ∥ T093	Auto-detection + multilingual dict in parallel after migration [NEW]
11c (US8)	T109 ∥ T111 ∥ T112 ∥ T113 ∥ T114	Inline correction backend + frontend components [NEW]
11d (US8b)	T123 ∥ T124 ∥ T125 ∥ T126 ∥ T127	Context fields, builder, import/export, bulk [NEW]
12 (US12) 🆕	T135 ∥ T138 ∥ T139	Connection model, RBAC, API client
13 (US11) 🆕	T146 ∥ T147 ∥ T153 ∥ T154	DbExecutor, requirements, API client, InsertMethodSelector

Cross-Story Parallelism

After Migration (Phase 11a):

• US6 auto-detection and US7 multilingual dict can proceed in parallel
• After US3 original + Phase 11 US3: US8 correction and US4 history update can proceed in parallel
• US8b context can proceed in parallel with US8 frontend work once T123-T124 merge

---

Implementation Strategy

MVP First (US1 Only)

1. Phase 1 + Phase 2 → Foundation
2. Phase 3 (US1) → Job configuration CRUD
3. STOP and VALIDATE: User can create, list, edit, delete translation jobs
4. Deploy/demo — partial value (configuration ready, no translation yet)

Minimum Viable Feature (US1 + US5 + US2 + US3)

1. Foundation → US1 + US5 (parallel) → US2 → US3
2. STOP and VALIDATE: End-to-end translation flow works: configure → preview → execute → INSERT
3. This is the core feature — all remaining stories add automation (US10), quality improvement (US6), and visibility (US4)

Full Feature (All Stories)

1. MVP → US6 + US10 + US4 (parallel after US3) → Polish
2. Scheduled automation, feedback loop, and audit trail all functional

Multi-Language Extension (Phase 11)

1. Migration first → Models, schemas, Alembic, data migration
2. US6 + US7 in parallel → Auto-detection + Multilingual dictionaries
3. US1 – US3 sequentially → Multi-language job config → Preview → Execution
4. US8 + US4 in parallel → Inline correction + Per-language history/metrics
5. Quickstart re-validation → Update T078 to cover multi-language flow

Direct DB Insert Enhancement (Phases 12–13) 🆕

1. Phase 12 first → Database Connection Settings (US12): models, service, API, UI tab, tests. This is a standalone prerequisite.
2. Phase 13 after Phase 12 → Direct DB Insert (US11): DB executor, orchestrator dispatch, insert method selector UI, tests.
3. Quickstart updates → Add direct DB insert flow to quickstart.md.

---

Notes

• All file paths reference the actual repository structure (backend/src/, frontend/src/).
• @COMPLEXITY 4/5 backend contracts require belief_scope/reason/reflect markers — verified in T076.
• @RATIONALE/@REJECTED tags appear only in C5 contracts (TranslationOrchestrator, TranslationEventLog) per INV_7.
• Rejected paths are explicitly protected by regression tests in T079.
• [NEED_CONTEXT] markers: none — all contract targets resolve to existing or planned modules within this feature.
• The existing LLMProviderService, SupersetClient, SchedulerService, NotificationService, and TaskWebSocket contracts are reused without modification.
• Quickstart.md (T078) serves as the human-verifiable acceptance test for the full feature. After Phase 11, update quickstart to cover multi-language flow.
• Migration considerations:
  • TranslationJob.target_language → target_languages migration is one-time. Old API consumers sending single target_language are backward-compatible (wrapped into list server-side).
  • TerminologyDictionary.target_language removal: existing dictionaries get entries migrated with source_language = job.source_language or "und" and target_language = dictionary.target_language.
  • TranslationRecord → TranslationLanguage migration: existing single-language records get one TranslationLanguage entry per record.
• LLM prompt change: Multi-target prompts add non-trivial token overhead (~50% for 3 languages). The prompt structure is: "Translate the following text to languages: [ru, en, de]. Return JSON with detected_source_language and translations keyed by language code."
• Context-aware prompt format (US8b):
  • Entry without context: "Voiture" → "Car"
  • Entry with context: "Voiture" (context: Category=Automotive, Type=Engine) → "Car"
  • Entry with context + notes: "Voiture" (context: Category=Automotive) → "Car" # Usage: Only for passenger vehicles
  • Priority entry: "Voiture" (context: Category=Automotive) → "Car" # PRIORITY - context match: 75%
  • Prompt construction caps context at ~500 tokens/entry. Priority entries listed before non-priority entries within the same language pair block.
• Context similarity heuristic: Jaccard similarity on tokenized (lowercased, stopwords-removed) text of context column values. Threshold 0.5. Configurable via job settings. Stored per-run in config snapshot.

---

✅ Spec 028 Final Closure Summary (2026-06-11)

All 160 tasks complete (134 core + 26 enhancement). Enhancement Phase (US11–US12) ✅ COMPLETE.

Финальная статистика реализации

Метрика	Значение
Всего файлов	~180 (бэкенд: 78 source + 43 test, фронтенд: 30 source + 10 test, миграции: 7)
Всего строк кода	~40 000
Бэкенд Python	~30 000 строк
Фронтенд Svelte/JS/TS	~9 000 строк
Backend tests	~580 pytest functions
Frontend tests	~68 vitest functions
User stories	12 (US1–US12)
Enhancement tasks	26 (T135–T160) ✅
RBAC permissions	14 (+1: settings.connections.manage)

Enhancement: Direct DB Insert + Connection Settings (Phases 12–13)

Компонент	Новые/Изменённые файлы	Complexity
DatabaseConnection (Pydantic) ✅	config_models.py	C1
ConnectionService ✅	core/connection_service.py (NEW)	C3
DbExecutor ✅	core/db_executor.py (NEW)	C3
Connection CRUD endpoints ✅	api/routes/settings.py (+5 endpoints)	C3
TranslationJob (ORM update) ✅	models/translate.py (+insert_method, +connection_id)	C2
TranslationRun (ORM update) ✅	models/translate.py (+insert_method, +connection_snapshot)	C2
TranslationOrchestrator (update) ✅	plugins/translate/orchestrator.py (dispatch logic)	C5
ConnectionsTab.svelte ✅	routes/settings/ConnectionsTab.svelte (NEW)	C2
InsertMethodSelector.svelte ✅	components/translate/InsertMethodSelector.svelte (NEW)	C1

Verified (core) ✅

• Backend: ~580 pytest ✅
• Frontend: ~68 vitest ✅
• Enhancement verified: DbExecutor, ConnectionService, InsertMethodSelector ✅

Remaining (enhancement) ✅

None. All 26 enhancement tasks are [x].

Next Action

• ready_for_review — Spec 028 is complete, ready for merge to main

Структура файлов (полный инвентарь)

Backend plugin source (59 файлов, backend/src/plugins/translate/, 10 176 строк)

Группа	Файлов	Строк	C	Назначение
Orchestrator	14	1 631	C5	Run lifecycle coordinator (декомпозирован)
Executor / LLM	9	2 196	C4	Batch LLM + caching + async HTTP
Preview	11	1 291	C4	Preview engine (декомпозирован)
Dictionary	7	921	C4	CRUD + import + filter
Service	6	1 281	C4	Job CRUD + InlineCorrection + BulkReplace + target_schema
Standalone	12	2 854	C2-C4	Scheduler, SQL, SupersetExecutor, PromptBuilder, Events, Metrics, TokenBudget, Utils, LangDetect, TextCleaner

Backend routes (13 файлов, backend/src/api/routes/translate/, 2 296 строк)

Файл	Строк	Назначение
__init__.py	42	Package init + re-export
_router.py	13	APIRouter instance
_job_routes.py	285	Job CRUD
_run_routes.py	205	Run execution, cancel, inline edit
_run_list_routes.py	359	Run list, detail, CSV
_run_edit_routes.py	200	Run edit endpoints [NEW]
_run_history_routes.py	133	Run history [NEW]
_preview_routes.py	194	Preview
_dictionary_routes.py	391	Dictionary CRUD + import
_correction_routes.py	100	Correction submission
_schedule_routes.py	239	Schedule CRUD
_metrics_routes.py	65	Metrics
_helpers.py	70	Response helpers

Frontend компоненты (14 файлов)

Файл	Строк	C	Назначение
TranslationPreview.svelte	564	C4	Preview table
TranslationRunResult.svelte	618	C4	Run results
ConfigTabForm.svelte	539	C3	Configuration form [NEW]
BulkReplaceModal.svelte	438	C3	Find & replace
CorrectionCell.svelte	343	C3	Inline edit cell
RunTabContent.svelte	342	C3	Run tab content [NEW]
TermCorrectionPopup.svelte	345	C3	Correction popup
TargetSchemaHint.svelte	316	C2	Target schema hint [NEW]
ScheduleConfig.svelte	300	C3	Schedule editor
BulkCorrectionSidebar.svelte	288	C3	Bulk correction
TranslationRunProgress.svelte	277	C4	Live progress
TargetTabForm.svelte	224	C3	Target tab form [NEW]
TranslationRunGlobalIndicator.svelte	237	C2	Global run indicator
TranslationMetricsDashboard.svelte	186	C2	Metrics dashboard

Frontend страницы (5 файлов, 1 237 строк)

Файл	Строк	Назначение
translate/+page.svelte	288	Job list
translate/[id]/+page.svelte	269	Job config + run
translate/history/+page.svelte	231	Run history
translate/dictionaries/+page.svelte	247	Dictionary list
translate/dictionaries/[id]/+page.svelte	202	Dictionary editor

Phase 11 Completion Details

• Migration (T083-T086): Models, schemas, Alembic — done
• US6 Auto-detection (T087-T090): LLM returns BCP-47, tests added
• US7 Multilingual dict (T091-T095): Language-pair aware CRUD + filtering
• US1/2/3 Multi-language (T096-T108): Config → Preview → Execution
• US8 Inline correction (T109-T118): Endpoints, BulkReplaceModal, CorrectionCell
• US8b Context-aware (T123-T134): context_data, ContextAwarePromptBuilder, Jaccard
• US4 History (T119-T122): Per-language stats, MetricSnapshot

Расхождения с планом (документировано post-factum)

Аспект	План	Реальность
Архитектура	Plugin-based	Service-based (plugin.py — скелет C2)
Routes	Один translate.py	Пакет из 13 файлов (2 296 строк)
Plugin source	~15 файлов	59 файлов (полная декомпозиция: orchestrator 14, executor 9, preview 11, dictionary 7, service 6, standalone 12)
SQLGenerator	C4	C3 (pure function, нет I/O)
TranslateRoutes	C3	C4 (пакет 13 файлов, 6+ зависимостей)
Новые модули (post-05-17)	—	_lang_detect.py, _text_cleaner.py, _llm_async_http.py, service_target_schema.py
Новые компоненты (post-05-17)	—	ConfigTabForm, RunTabContent, TargetTabForm, TargetSchemaHint
Frontend pages	~3 256 строк	1 237 строк (рефакторинг)

Verified

• Backend: ~580 pytest ✅
• Frontend: ~68 vitest ✅
• Browser: validated — preview, correction, dictionary flows work
• Enhancement verified: DbExecutor, ConnectionService, InsertMethodSelector ✅
• Semantic audit: 0 warnings across all checks

Remaining

• None. All 160 tasks are [x].

Next Action

• ready_for_review — Spec 028 is complete, ready for merge to main