busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
f8474498f828a1817f8b1ce3aeaafefe2480a439
ss-tools/specs/028-llm-datasource-supeset/tasks.md
busya 6988e63967 chore: commit remaining pre-existing changes 
- Agent configs (.opencode/agents/)
- Backend: alembic, routes, app, utils, scripts
- Frontend: package.json, vite, components, e2e infra
- Specs: 028-llm-datasource-supeset updates
- Docker e2e config and Playwright setup
2026-05-17 19:23:07 +03:00

65 KiB
Raw Blame History

Tasks: LLM Table Translation Service

Feature Branch: 028-llm-datasource-supeset Input: Design documents from /specs/028-llm-datasource-supeset/ Prerequisites: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/modules.md

Tests: Test tasks are included for all C4/C5 backend contracts, new API endpoints, and Svelte components with @UX_STATE contracts. Test work traces to contract @PRE/@POST guarantees and spec acceptance scenarios.

Organization: Tasks are grouped by user story to enable independent implementation and testing of each story.

Format: [ID] [P?] [Story] Description

  • [P]: Can run in parallel (different files, no dependencies)
  • [Story]: Which user story this task belongs to (e.g., US1, US5)
  • Include exact file paths in descriptions

Phase 1: Setup (Shared Infrastructure)

Purpose: Create plugin directory structure and register the new route module in the lazy-import registry.

  • T001 Create translation plugin directory structure: backend/src/plugins/translate/__init__.py, backend/src/plugins/translate/plugin.py (empty skeleton), plus backend/src/plugins/translate/__tests__/__init__.py
  • T002 Register translate route module in backend/src/api/routes/__init__.py — add "translate" to __all__ list inside [DEF:Route_Group_Contracts:Block]

Phase 2: Foundational (Blocking Prerequisites)

Purpose: ORM models, Pydantic schemas, plugin boilerplate, route skeleton, and database migration. ALL user stories depend on these artifacts.

⚠️ CRITICAL: No user story work can begin until this phase is complete.

ORM Models

  • T003 [P] Create all SQLAlchemy ORM models in backend/src/models/translate.py: TranslationJob, TranslationRun, TranslationBatch, TranslationRecord, TranslationEvent, TranslationPreviewSession, TranslationPreviewRecord, TerminologyDictionary, DictionaryEntry, TranslationSchedule, TranslationJobDictionary, MetricSnapshot. Follow patterns from backend/src/models/llm.py (UUID PKs, generate_uuid, Base inheritance, JSON columns, UniqueConstraint, indexes, timezone-aware DateTime with callable defaults). Include source_term_normalized column on DictionaryEntry with unique constraint for case-insensitive matching.
  • T004 [P] Create Pydantic v2 request/response schemas in backend/src/schemas/translate.py: TranslateJobCreate, TranslateJobUpdate, TranslateJobResponse, DictionaryCreate, DictionaryImport, DictionaryResponse, TermCorrectionSubmit, ScheduleConfig, TranslationRunResponse, TranslationPreviewResponse (with PreviewRow), MetricsResponse. Follow existing backend/src/schemas/ patterns (use BaseModel, Field with defaults/validation)

Plugin Skeleton

  • T005 Create TranslatePlugin class in backend/src/plugins/translate/plugin.py inheriting from PluginBase. Implement id, name, description properties. Wire @RELATION INHERITS -> [PluginBase:Class] in contract header. (RATIONALE: separate plugin avoids bloating LLMAnalysisPlugin beyond fractal limit; REJECTED: extending LLMAnalysisPlugin would conflate domains)

Route Skeleton

  • T006 Create backend/src/api/routes/translate.py with FastAPI APIRouter (prefix=/api/translate, tags=["translate"]). Define all endpoint stubs with pass bodies for now: CRUD jobs, CRUD dictionaries, preview trigger, run trigger, retry, schedule CRUD, run history, metrics, correction submission, dictionary import. Attach Depends(require_permission(...)) annotations. Register router in backend/src/app.py alongside existing routers.

Database Migration

  • T007 Generate Alembic migration for all translate_* tables: translation_jobs, translation_runs, translation_records, translation_events, terminology_dictionaries, dictionary_entries, translation_schedules, translation_job_dictionaries. Run cd backend && alembic revision --autogenerate -m "add translation tables" and alembic upgrade head.

RBAC Registration

  • T008 Register 13 permission strings in the RBAC seed/permission store: translate.job.view, translate.job.create, translate.job.edit, translate.job.delete, translate.job.execute, translate.dictionary.view, translate.dictionary.create, translate.dictionary.edit, translate.dictionary.delete, translate.schedule.view, translate.schedule.manage, translate.history.view, translate.metrics.view. Ensure admin role gets all; analyst role gets translate.job.view, translate.job.execute, translate.dictionary.view, translate.history.view. Update role seeding script if needed.

Checkpoint: Foundation ready — models, schemas, plugin, routes, migration, and RBAC all in place. User story implementation can now begin.

Phase 3: User Story 1 — Configure Translation Job (Priority: P1) 🎯 MVP

Goal: User can create, edit, delete, and list translation jobs with datasource selection, column mapping, key columns, target table configuration, LLM settings, and dictionary attachment.

Independent Test: Open Configuration form → select Superset datasource → pick translation/context/key columns → specify target table → save → verify job appears in list with correct settings.

Backend — Job CRUD

  • T009 [P] [US1] Implement job CRUD service in backend/src/plugins/translate/plugin.py as methods on the TranslatePlugin class: create_job(), update_job(), delete_job(), get_job(), list_jobs(), duplicate_job(). Validate column existence via SupersetClient on create/update (FR-001, FR-002, FR-006). Enforce composite key support (FR-004). Detect virtual columns and warn (US1 acceptance scenario 5).
  • T010 [US1] Implement /api/translate/jobs endpoints in backend/src/api/routes/translate.py: POST / (create), GET / (list), GET /{job_id} (get), PUT /{job_id} (update), DELETE /{job_id} (delete), POST /{job_id}/duplicate (duplicate — FR-021). Inject Depends(require_permission("translate.job.*")) per operation.
  • T011 [US1] Implement /api/translate/datasources/{datasource_id}/columns endpoint that queries Superset for column metadata (name, type, is_physical flag) and the database dialect (backend/engine) from the connection configuration. Returns column list AND database_dialect field for the frontend. Cache dialect on TranslationJob.database_dialect at save time. Reject unsupported dialects at configuration time (FR-002, dialect detection).

Frontend — Job Config UI

  • T012 [P] [US1] Create TranslateApiClient module in frontend/src/lib/api/translate.js: fetchJobs(), createJob(), updateJob(), deleteJob(), duplicateJob(), fetchDatasourceColumns(). Use existing requestApi/fetchApi wrapper pattern.
  • T013 [US1] Create TranslationJobList SvelteKit page in frontend/src/routes/translate/+page.svelte: list all jobs with name, datasource, status/schedule indicators, create button, duplicate action. @UX_STATE: idle, loading, empty, populated, error.
  • T014 [US1] Create TranslationJobConfig SvelteKit page in frontend/src/routes/translate/[id]/+page.svelte: datasource dropdown → column selectors (translation column, context columns, key columns with [+ Add key] for composite), target table/column inputs, LLM provider selector, target language, batch size, prompt template editor, dictionary attachment (multi-select with priority ordering). @UX_STATE: idle, loading, configured, saving, validation_error, datasource_unavailable. @UX_REACTIVITY: column list $derived from datasource selection.

Verification — US1

  • T015 [US1] Write pytest integration tests for job CRUD API in backend/tests/test_translate_jobs.py: test create with valid config, create with missing translation column (expect 422), create with virtual key column (expect warning), update job, delete job, duplicate job. Mock SupersetClient for column metadata.
  • T016 [US1] Verify US1 acceptance scenarios against specs/028-llm-datasource-supeset/spec.md User Story 1 (5 scenarios). Run cd backend && pytest backend/tests/test_translate_jobs.py -v.

Checkpoint: Job CRUD fully functional — user can create, edit, list, and duplicate translation jobs with validated column mappings.

Phase 4: User Story 5 — Terminology Dictionary Management (Priority: P2)

Goal: User can create, edit, delete dictionaries; add terms inline; import CSV/TSV with duplicate detection; attach dictionaries to jobs with priority ordering.

Independent Test: Create dictionary with 5 terms → import CSV with 50 terms → verify duplicates flagged → attach dictionary to job → verify dictionary appears in job config.

Backend — Dictionary CRUD + Import

  • T017 [P] [US5] Implement DictionaryManager class in backend/src/plugins/translate/dictionary.py: create_dictionary(), update_dictionary(), delete_dictionary(), get_dictionary(), list_dictionaries(), add_entry(), edit_entry(), delete_entry(), clear_entries(). Enforce unique source_term per dictionary with conflict resolution (FR-026). Prevent deletion if attached to active/scheduled jobs (FR-030). @COMPLEXITY 4 — instrument with belief_scope/reason/reflect markers at mutation boundaries. (RATIONALE: C4 warranted because dictionary CRUD is stateful and must enforce referential integrity on deletion; REJECTED: pure C3 CRUD without state guards would allow orphaned job-dictionary links)
  • T018 [US5] Implement CSV/TSV import in DictionaryManager: parse uploaded content, detect delimiter, create DictionaryEntry rows, preview with duplicate detection, return parse errors with line numbers for malformed rows (FR-025). Add DictionaryImport schema validation.
  • T019 [US5] Implement /api/translate/dictionaries endpoints in backend/src/api/routes/translate.py: POST / (create), GET / (list), GET /{dict_id} (get with entries), PUT /{dict_id} (update), DELETE /{dict_id} (delete — blocked if attached), POST /{dict_id}/entries (add entry), PUT /{dict_id}/entries/{entry_id} (edit), DELETE /{dict_id}/entries/{entry_id} (delete), POST /{dict_id}/import (CSV/TSV import with preview).
  • T020 [US5] Implement per-batch dictionary filtering logic in DictionaryManager.filter_for_batch(source_texts: list[str]) -> list[dict]: scan batch texts for substrings matching dictionary source_term values; return matched entries in priority order across all attached dictionaries (FR-044). This is consumed by US2 (preview) and US3 (executor).

Frontend — Dictionary UI

  • T021 [P] [US5] Add dictionary API methods to frontend/src/lib/api/translate.js: fetchDictionaries(), createDictionary(), updateDictionary(), deleteDictionary(), fetchDictionaryEntries(), addEntry(), editEntry(), deleteEntry(), importDictionary().
  • T022 [US5] Create DictionaryList SvelteKit page in frontend/src/routes/translate/dictionaries/+page.svelte: list dictionaries with name, language, term count, attached job count, create/delete actions. @UX_STATE: idle, loading, empty, populated, delete_blocked.
  • T023 [US5] Create DictionaryEditor SvelteKit page in frontend/src/routes/translate/dictionaries/[id]/+page.svelte: inline term editor (source_term → target_translation), add/delete rows, CSV/TSV import with conflict preview, export. @UX_STATE: idle, loading, editing, importing, import_preview, import_conflict, saving. @UX_FEEDBACK: import preview with duplicate flags; toast on save.

Verification — US5

  • T024 [US5] Write pytest tests for DictionaryManager in backend/src/plugins/translate/__tests__/test_dictionary.py: test create/update/delete, add entry with duplicate detection (expect conflict), import CSV with valid/invalid rows, delete dictionary blocked by active job, per-batch filtering returns matched terms.
  • T025 [US5] Verify US5 acceptance scenarios against spec User Story 5 (6 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_dictionary.py -v.

Checkpoint: Dictionary management fully functional — CRUD, import, filtering, and job attachment all work.

Phase 5: User Story 2 — Preview Translated Output (Priority: P2)

Goal: User triggers preview on a saved job → system fetches sample rows → sends to LLM with context + dictionary → displays source/context/translation side-by-side → user approves/edits/rejects → preview state saved for execution gate.

Independent Test: Create job + dictionary → click Preview → verify 10 rows shown with LLM translations → approve 8, edit 1, reject 1 → verify state preserved.

Backend — Preview Engine

  • T026 [US2] Implement TranslationPreview class in backend/src/plugins/translate/preview.py: preview_rows(job_id, sample_size). Fetch source rows from Superset via SupersetClient; construct LLM prompt using LLMProviderService + llm_prompt_templates.render_prompt() + DictionaryManager.filter_for_batch(); call LLM; return PreviewRow list. @COMPLEXITY 4 — instrument with belief_scope/reason/reflect at LLM call boundaries. (RATIONALE: C4 because preview is stateful (approve/edit/reject lifecycle) and calls external LLM API with side effects; REJECTED: making preview purely read-only without approval state would degrade UX by losing user decisions between preview and execution)
  • T027 [US2] Implement token count and cost estimation in preview response: compute estimated tokens from sample → extrapolate to full dataset row count → apply provider pricing → return estimated_total_rows, estimated_tokens, estimated_cost in TranslationPreviewResponse (FR-014).
  • T028 [US2] Implement preview quality gate: create persistent TranslationPreviewSession and TranslationPreviewRecord rows with config_hash and dict_snapshot_hash. Preview acceptance gates full execution; rejected preview sample rows are excluded from full run. Preview is a quality gate — unseen rows are processed normally in full run.
  • T029 [US2] Implement /api/translate/jobs/{job_id}/preview endpoint: POST triggers preview, returns preview rows with status=pending. Add PUT /api/translate/jobs/{job_id}/preview/rows/{row_key} for approve/edit/reject actions. Add POST /api/translate/jobs/{job_id}/preview/approve-all for bulk approve.

Frontend — Preview UI

  • T030 [P] [US2] Add preview API methods to frontend/src/lib/api/translate.js: fetchPreview(), approveRow(), editRow(), rejectRow(), approveAll().
  • T031 [US2] Create TranslationPreview component in frontend/src/lib/components/translate/TranslationPreview.svelte: side-by-side table (source, context, LLM translation), approve/edit/reject buttons per row, bulk approve, cost estimate card before full run, row limit input. @UX_STATE: idle, loading, preview_loaded, preview_error, retrying. @UX_FEEDBACK: spinner during LLM call; visual distinction for LLM-generated vs user-edited values; cost estimate reactivity. @UX_RECOVERY: retry preview button; individual row re-translate.
  • T032 [US2] Integrate TranslationPreview into TranslationJobConfig page (frontend/src/routes/translate/[id]/+page.svelte) as a tab or collapsible section that appears after job is saved.

Verification — US2

  • T033 [US2] Write pytest tests for preview in backend/src/plugins/translate/__tests__/test_preview.py: test preview with valid job, preview with dictionary (verify glossary terms in prompt), preview row approve/edit/reject state transitions, cost estimation accuracy. Mock LLM provider responses.
  • T034 [US2] Write vitest component test for TranslationPreview in frontend/src/lib/components/translate/__tests__/TranslationPreview.test.js: test rendering of preview rows, approve/reject/edit interactions, bulk approve behavior. Mock API client.
  • T035 [US2] Verify US2 acceptance scenarios against spec User Story 2 (5 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_preview.py -v && cd frontend && npm run test -- --run.

Checkpoint: Preview flows complete — LLM translation with context + dictionary, approve/edit/reject lifecycle, cost estimation.

Phase 6: User Story 3 — Execute Translation & Insert Results (Priority: P3)

Goal: User triggers full batch execution → system processes rows in batches → generates INSERT SQL → user copies to SQL Lab or auto-executes → failed batches retryable.

Independent Test: Create job → preview + approve → execute → verify INSERT SQL generated with correct key columns → execute in SQL Lab → verify rows in target table.

Backend — Executor + SQL Generator + Orchestrator

  • T036 [US3] Implement SQLGenerator class in backend/src/plugins/translate/sql_generator.py: generate_insert(records: list[TranslationRecord], job: TranslationJob) -> str. Detect dialect from job.database_dialect (cached from Superset connection at save time). Produce safe dialect-appropriate SQL: for PostgreSQL/Greenplum — INSERT INTO "target_table" ("key_cols"..., "target_col") VALUES (...) with quoted identifiers; support upsert_strategy: insert (plain INSERT), skip_existing (ON CONFLICT DO NOTHING), overwrite (ON CONFLICT DO UPDATE). For ClickHouse — INSERT INTO target_table (key_cols..., target_col) VALUES (...); skip_existing warns user (not natively supported); overwrite documented limitation. @COMPLEXITY 3. (RATIONALE: dialect-aware because Superset connections may use ClickHouse or PostgreSQL; REJECTED: PostgreSQL-only would break ClickHouse users; raw identifier interpolation rejected)
  • T037 [US3] Implement TranslationExecutor class in backend/src/plugins/translate/executor.py: execute_run(run: TranslationRun, job: TranslationJob). Fetch all source rows from Superset; split into batches; for each batch: call DictionaryManager.filter_for_batch(), construct prompt via LLMProviderService, call LLM, create TranslationRecord rows with status translated/failed/skipped; handle batch-level retry on LLM failure (FR-015); skip NULL translation values (FR-016); reject NULL key values (FR-017); update run statistics. @COMPLEXITY 4 — instrument with belief_scope/reason/reflect at batch boundaries and error paths.
  • T038 [US3] Implement TranslationOrchestrator class in backend/src/plugins/translate/orchestrator.py: start_run(job_id, trigger_type). Validate preconditions (job config valid, datasource accessible, LLM provider reachable); create TranslationRun with status running and config/dict snapshots (FR-019, FR-029); dispatch to TranslationExecutor; on completion call SQLGenerator; record TranslationEvent rows via TranslationEventLog (FR-046); enforce state transitions: pending → running → (completed | partial | failed) — no skipping. @COMPLEXITY 5 — full @PRE/@POST/@DATA_CONTRACT/@INVARIANT enforcement with @RATIONALE/@REJECTED. (RATIONALE: central coordinator is C5 because preview, execution, event logging, and retry share run state and must coordinate within a single transaction boundary; REJECTED: distributed actor model would introduce eventual-consistency challenges for status tracking at current scale)
  • T039 [US3] Implement TranslationEventLog class in backend/src/plugins/translate/events.py: log_event(run_id, job_id, event_type, payload). Create immutable TranslationEvent row. query_events(job_id, filters) for audit/dashboard. prune_expired() for 90-day retention enforcement (FR-049) — scheduled via APScheduler cleanup job. @COMPLEXITY 5 — @INVARIANT: every run must have exactly one run_started and one terminal event. (RATIONALE: C5 warranted because event log is single source of truth for observability, metrics, and audit; REJECTED: stdout-only logging lacks structured payload integrity and cannot enforce terminal-event invariant)
  • T040 [US3] Implement execution endpoints in backend/src/api/routes/translate.py: POST /api/translate/jobs/{job_id}/runs (trigger manual run — creates run, dispatches orchestrator which translates AND submits to Superset API), GET /api/translate/runs/{run_id} (status + statistics + insert_status + superset_query_id), GET /api/translate/runs/{run_id}/records (paginated TranslationRecord list), POST /api/translate/runs/{run_id}/retry (retry failed batches only — FR-015), POST /api/translate/runs/{run_id}/retry-insert (retry Superset insert only without re-translating). Inject Depends(require_permission("translate.job.execute")).

Frontend — Execution UI

  • T041 [P] [US3] Add execution API methods to frontend/src/lib/api/translate.js: triggerRun(), fetchRunStatus(), fetchRunRecords(), retryFailedBatches().
  • T042 [US3] Create TranslationRunProgress component in frontend/src/lib/components/translate/TranslationRunProgress.svelte: live progress bar (WebSocket-driven from TaskWebSocket), batch counter (N/M), success/failure/skip counts, cancel button. @UX_STATE: idle, running, pausing, cancelled, completed, partial, failed. @UX_FEEDBACK: progress percentage $derived from translated/total; real-time counts. @UX_RECOVERY: retry failed batches button; cancel run; download skipped rows.
  • T043 [US3] Create TranslationRunResult component in frontend/src/lib/components/translate/TranslationRunResult.svelte: completion summary (rows translated/failed/skipped, token count, cost, insert_status), Superset execution reference with status badge, generated SQL block for audit/debugging (collapsed by default), retry-insert button. @UX_STATE: completed, partial, failed, insert_failed. @UX_FEEDBACK: Superset execution status badge; SQL block for audit.
  • T044 [US3] Integrate TranslationRunProgress and TranslationRunResult into TranslationJobConfig page as the "Run" tab/section.

Verification — US3

  • T045 [US3] Write pytest tests for SQLGenerator in backend/src/plugins/translate/__tests__/test_sql_generator.py: test INSERT with single key, composite key — for PostgreSQL dialect AND ClickHouse dialect. Test PostgreSQL UPSERT (ON CONFLICT DO NOTHING, ON CONFLICT DO UPDATE). Test ClickHouse plain INSERT and skip_existing warning. Test NULL key rejection, NULL translation value skipping, identifier quoting per dialect, injection safety. Validate SQL syntax correctness against each dialect.
  • T046 [US3] Write pytest tests for executor + orchestrator in backend/src/plugins/translate/__tests__/test_orchestrator.py: test full run lifecycle (pending→running→completed), partial failure (one batch fails, rest succeed), batch retry, event log invariants, NULL handling. Mock LLM provider and SupersetClient.
  • T047 [US3] Verify US3 acceptance scenarios against spec User Story 3 (5 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_orchestrator.py backend/src/plugins/translate/__tests__/test_sql_generator.py -v.

Checkpoint: Execution pipeline complete — batch processing, INSERT generation, retry, event logging. User can translate data and insert into target table.

Phase 7: User Story 6 — Feedback Loop (Correct → Dictionary) (Priority: P3)

Goal: In run results, user selects incorrect translation → submits correction to dictionary → dictionary updated with origin tracking → next run uses corrected term.

Independent Test: Complete a run → find incorrect translation → open correction popup → submit to dictionary → re-run preview → verify corrected term used.

Backend — Correction Submission

  • T048 [US6] Implement correction submission endpoint in backend/src/api/routes/translate.py: POST /api/translate/corrections accepting TermCorrectionSubmit body. Validate target language match between dictionary and job (FR language validation edge case); detect existing entry conflict → return conflict response (FR-032); create DictionaryEntry with origin tracking (origin_run_id, origin_row_key, origin_user_id) per FR-033. Inject Depends(require_permission("translate.dictionary.edit")).
  • T049 [US6] Implement bulk correction endpoint: POST /api/translate/corrections/bulk accepting array of TermCorrectionSubmit objects (FR-034). Process atomically — if any conflict is detected, return all conflicts for user resolution before partial apply.

Frontend — Correction UI

  • T050 [P] [US6] Add correction API methods to frontend/src/lib/api/translate.js: submitCorrection(), submitBulkCorrections().
  • T051 [US6] Create TermCorrectionPopup component in frontend/src/lib/components/translate/TermCorrectionPopup.svelte: text selection on source term and incorrect target translation → popup with source term (pre-filled from source column), incorrect target translation (pre-filled from selection), corrected target translation input, dictionary selector dropdown (filtered by target language), submit button, conflict dialog (overwrite/keep existing/cancel). @UX_STATE: closed, selecting, editing, submitting, conflict_detected, submitted. @UX_FEEDBACK: "Added to Dictionary" badge on corrected row.
  • T052 [US6] Create BulkCorrectionSidebar component in frontend/src/lib/components/translate/BulkCorrectionSidebar.svelte: sidebar collecting selected terms across rows, per-term correction inputs, submit all to dictionary. @UX_STATE: closed, collecting, reviewing, submitting, submitted. @UX_REACTIVITY: selected terms list $state.
  • T053 [US6] Integrate feedback-loop components into TranslationRunResult (T043) — add selection highlight behavior and correction triggers.

Verification — US6

  • T054 [US6] Write pytest tests for correction endpoints in backend/tests/test_translate_corrections.py: test single correction, bulk correction, conflict detection (existing term), cross-language rejection, origin tracking fields populated. Verify corrected term appears in next preview's dictionary filter.
  • T055 [US6] Verify US6 acceptance scenarios against spec User Story 6 (5 scenarios). Run cd backend && pytest backend/tests/test_translate_corrections.py -v.

Checkpoint: Feedback loop complete — corrections flow from results → dictionary → next run.

Phase 8: User Story 10 — Schedule Translation Jobs (Priority: P3)

Goal: User configures schedule → system triggers runs → new-key-only translation → optional auto-INSERT → failure notification → pause/resume.

Independent Test: Configure schedule (every 5 min for test) → wait for trigger → verify new TranslationRun created → verify only new keys translated → disable schedule → verify no more triggers.

Backend — Schedule Management + Trigger Dispatch

  • T056 [US10] Implement TranslationScheduler class in backend/src/plugins/translate/scheduler.py: create_schedule(), update_schedule(), delete_schedule(), enable_schedule(), disable_schedule(), get_next_executions(schedule, n=3) (FR-036). Register schedule with existing SchedulerService via add_job() with cron/interval/date trigger. @COMPLEXITY 4 — instrument with belief_scope/reason/reflect. (RATIONALE: C4 because schedule management is stateful with APScheduler integration, concurrency policy enforcement, and trigger dispatch side effects)
  • T057 [US10] Implement schedule trigger handler: _execute_scheduled_translation(job_id). Enforce concurrency policy: check if previous run for same job is still running → skip (log + event) or queue (start after previous completes) per FR-039. If proceeding: create new TranslationRun with trigger_type=scheduled; fetch source rows; apply new-key-only filter (FR-045) — compare current key values against previous successful run's key values; dispatch to TranslationOrchestrator. On failure, send notification via NotificationService (FR-041, FR-048). Schedule remains enabled for next trigger (US10 acceptance scenario 6).
  • T058 [US10] Implement Superset SQL Lab API submission for all runs: create SupersetSqlLabExecutor class in backend/src/plugins/translate/superset_executor.py. Submit generated SQL to /api/v1/sqllab/execute/, poll execution status, update TranslationRun.insert_status, superset_query_id, rows_affected, error fields. For scheduled runs, this happens automatically; for manual runs, this happens on user trigger. Record insert_submitted/insert_succeeded/insert_failed events.
  • T059 [US10] Implement schedule endpoints in backend/src/api/routes/translate.py: PUT /api/translate/jobs/{job_id}/schedule (create/update), DELETE /api/translate/jobs/{job_id}/schedule (remove), POST /api/translate/jobs/{job_id}/schedule/enable (FR-040), POST /api/translate/jobs/{job_id}/schedule/disable (FR-040). Inject Depends(require_permission("translate.schedule.manage")). Add schedule warning when editing job with active schedule (FR-042).
  • T060 [US10] Extend SchedulerService.load_schedules() in backend/src/core/scheduler.py to discover and register active TranslationSchedule rows alongside existing backup schedules (R4).

Frontend — Schedule UI

  • T061 [P] [US10] Add schedule API methods to frontend/src/lib/api/translate.js: updateSchedule(), deleteSchedule(), enableSchedule(), disableSchedule().
  • T062 [US10] Create ScheduleConfig component in frontend/src/lib/components/translate/ScheduleConfig.svelte: type selector (cron/interval/once), cron expression input with validation, interval input, timezone selector, run-at datetime picker, next-3-executions preview (with timezone), concurrency policy selector (skip/queue), enable/disable toggle with status indicator. Warns if no prior successful manual run exists. @UX_STATE: idle, editing, validating, enabled, disabled, no_prior_run_warning. @UX_REACTIVITY: next execution times $derived from schedule config with timezone display.
  • T063 [US10] Integrate ScheduleConfig into TranslationJobConfig page as the "Schedule" tab.

Verification — US10

  • T064 [US10] Write pytest tests for scheduler in backend/src/plugins/translate/__tests__/test_scheduler.py: test schedule CRUD, cron expression validation, next-N-executions calculation, trigger dispatch with skip/queue concurrency, new-key-only filter (verify only unseen keys processed), auto-INSERT execution, failure notification, pause/resume, load on SchedulerService start.
  • T065 [US10] Verify US7 acceptance scenarios against spec User Story 7 (8 scenarios). Run cd backend && pytest backend/src/plugins/translate/__tests__/test_scheduler.py -v.

Checkpoint: Scheduling complete — jobs can run automatically on schedule with new-key-only incremental translation and failure recovery.

Phase 9: User Story 4 — Translation History & Audit Trail (Priority: P4)

Goal: User views past runs with filterable list; inspects run details (config snapshot, prompt, translations, INSERT SQL); sees edit marks; duplicates job. Admin views metrics dashboard.

Independent Test: Run several translations → open history → filter by datasource → click run → verify config snapshot, prompt, translations with edit marks, INSERT SQL all shown.

Backend — History + Metrics Endpoints

  • T066 [US4] Implement history endpoints in backend/src/api/routes/translate.py: GET /api/translate/runs (list with filters: job_id, datasource_id, target_table, status, date_from, date_to, pagination per FR-020), GET /api/translate/runs/{run_id} (detail with config_snapshot, prompt_used, records with llm_translation and user_edit fields visible — FR showing original vs user-edited).
  • T067 [US4] Implement TranslationMetrics class in backend/src/plugins/translate/metrics.py: get_job_metrics(job_id) -> MetricsResponse. Aggregate from TranslationEvent table: total runs, success/failure counts, cumulative tokens, cumulative cost, average batch latency (FR-047). @COMPLEXITY 3.
  • T068 [US4] Implement metrics endpoint: GET /api/translate/jobs/{job_id}/metrics. Inject Depends(require_permission("translate.history.view")).

Frontend — History + Metrics UI

  • T069 [P] [US4] Add history API methods to frontend/src/lib/api/translate.js: fetchRunHistory(), fetchRunDetail(), fetchJobMetrics().
  • T070 [US4] Create TranslationHistory SvelteKit page in frontend/src/routes/translate/history/+page.svelte: filterable table (datasource, target table, row count, status, date, user), click-to-expand detail with config snapshot, prompt, translation rows with edit marks, INSERT SQL. @UX_STATE: idle, loading, empty, populated, detail_open. @UX_REACTIVITY: filtered list $derived from filters.
  • T071 [US4] Create admin metrics dashboard section (integrated into existing admin pages or standalone) displaying per-job metrics: run counts, success/failure ratio, cumulative tokens, cumulative cost, average latency. Use MetricsResponse schema.

Verification — US4

  • T072 [US4] Write pytest tests for history + metrics in backend/tests/test_translate_history.py: test run list with filters, run detail with snapshots, metrics aggregation accuracy, TranslationEvent queryability.
  • T073 [US4] Verify US4 acceptance scenarios against spec User Story 4 (4 scenarios). Run cd backend && pytest backend/tests/test_translate_history.py -v.

Checkpoint: History and audit complete — all runs traceable, metrics dashboard populated.

Phase 10: Polish & Cross-Cutting Concerns

Purpose: Retention enforcement, notification wiring, semantic audit, quickstart validation, and rejected-path regression protection.

  • T074 [P] Implement 90-day retention pruning in TranslationEventLog.prune_expired(): run as APScheduler daily cleanup job. BEFORE pruning events/records: persist cumulative metrics as MetricSnapshot row (tokens, cost, run counts). Then prune TranslationRecord, TranslationPreviewRecord, TranslationEvent, and insert_sql/config_snapshot fields older than 90 days. Preserve TranslationRun metadata, MetricSnapshot rows, and superset_query_id. Verify metrics remain accurate post-prune (SC-014). (RATIONALE: metric snapshots prevent cumulative data loss from event pruning; REJECTED: indefinite retention would violate storage constraints)
  • T075 [P] Wire scheduled-run failure notification: ensure TranslationScheduler trigger handler calls NotificationService.send() when a scheduled run fails (FR-041, FR-048). Test with mock notification provider.
  • T076 [P] Instrument remaining C4/C5 Python flows with belief_scope/reason/reflect/explore markers where missing: TranslationOrchestrator.start_run() (entry/exit), TranslationExecutor.execute_run() (batch boundaries + error paths), DictionaryManager mutation boundaries, TranslationScheduler trigger dispatch. Verify via axiom_semantic_validation belief-runtime audit.
  • T077 Run full semantic audit via axiom MCP tools:
    • axiom_semantic_validation audit_contracts --file_path backend/src/plugins/translate/ — verify all #region/#endregion anchors are properly closed, @RELATION targets resolve, no orphan contracts, C4+ contracts have required tag density per GRACE complexity scale
    • axiom_semantic_validation audit_belief_protocol --file_path backend/src/plugins/translate/ — verify @RATIONALE/@REJECTED present on all C5 contracts only (not on C1-C4)
    • axiom_semantic_validation audit_belief_runtime --file_path backend/src/plugins/translate/ — verify belief_scope/reason/reflect/explore markers exist in all C4+ module bodies
    • axiom_semantic_validation impact_analysis --contract_id TranslationOrchestrator:Class — verify no rejected path is accidentally re-enabled
  • T078 Run quickstart validation: follow specs/028-llm-datasource-supeset/quickstart.md end-to-end — create dictionary → create job → preview → execute → verify INSERT SQL → submit correction → schedule → view history → verify metrics. Run cd backend && pytest -v, cd frontend && npm run test -- --run, cd backend && ruff check src/plugins/translate/ src/api/routes/translate.py src/models/translate.py src/schemas/translate.py.
  • T079 Rejected-path regression guard: add a test case in backend/src/plugins/translate/__tests__/test_orchestrator.py verifying snapshot isolation — changing job config mid-run does NOT invalidate the running TranslationRun. Add a test case in backend/src/plugins/translate/__tests__/test_sql_generator.py verifying that UPDATE statements are never generated (only INSERT/UPSERT per PostgreSQL dialect). Add a test case in backend/src/plugins/translate/__tests__/test_dictionary.py verifying that duplicate source_term entries cannot coexist (UniqueConstraint enforced) and conflict resolution only offers overwrite/keep-existing. Add a test case in backend/src/plugins/translate/__tests__/test_retention.py verifying metric snapshots are persisted before event pruning and cumulative metrics remain accurate post-prune.
  • T080 [P] Implement cancel run endpoint: POST /api/translate/runs/{run_id}/cancel in backend/src/api/routes/translate.py. Set translation_status=cancelled, mark in-progress batches as failed, do NOT submit INSERT SQL. Emit run_cancelled event. Inject Depends(require_permission("translate.job.execute")).
  • T081 [P] Implement download skipped rows endpoint: GET /api/translate/runs/{run_id}/skipped.csv returning CSV of rows skipped due to NULL keys or translation failures. Use key_hash for efficient lookup.
  • T082 [P] Compute key_hash for TranslationRecord and TranslationPreviewRecord: hash(canonical_json(key_values)) at creation time. Add config_hash for TranslationRun and TranslationPreviewSession: hash of effective config (columns, keys, target, prompt, dictionaries). Use for idempotency checks, new-key-only filtering, and stale preview detection.

Phase 11: Multi-Language & Correction Optimization ✅ COMPLETE (2026-05-14)

Purpose: Implement multi-language support (auto-detection, multi-target, per-language storage, multilingual dictionaries, improved correction workflow). All tasks below are NEW additions building on the existing single-target foundation.

Shared Infrastructure (Migration)

  • T083 [P] Add TranslationLanguage, TranslationPreviewLanguage, TranslationRunLanguageStats models to backend/src/models/translate.py. Add Alembic migration. Existing TranslationRecord gains languages relationship; existing TranslationPreviewRecord gains languages relationship. Foreign keys and cascades.
  • T084 [P] Add source_language, target_language columns to DictionaryEntry in backend/src/models/translate.py. Update unique constraint to (dictionary_id, source_term_normalized, source_language, target_language). Add migration.
  • T085 [P] Migrate TranslationJob.target_language (str) → target_languages (JSON list) in backend/src/models/translate.py. Add migration script for existing jobs: target_languages = [old_target_language]. Keep source_language as optional fallback hint.
  • T086 [P] Update Pydantic schemas in backend/src/schemas/translate.py: TranslateJobCreate gets target_languages: list[str] (required, min 1), source_language optional. Add TranslationLanguageResponse, TranslationPreviewLanguageResponse, BulkFindReplaceRequest, InlineCorrectionSubmit schemas.

US6 (NEW) — Auto-Detected Source Language

  • T087 [P] [US6] Update LLM prompt construction in backend/src/plugins/translate/executor.py and backend/src/plugins/translate/preview.py: instruct LLM to return detected_source_language (BCP-47) alongside each row's translations. Parse and store per-row in TranslationLanguage.source_language_detected.
  • T088 [P] [US6] Update TranslationPreview to display detected source language per row in preview results. Add detected_source_language field to preview response. Flag rows with "und" for manual review.
  • T089 [P] [US6] Update TranslationExecutor to store detected source language per TranslationLanguage entry. Handle "und" (undetermined) rows — mark as flagged, allow manual override.
  • T090 [US6] Write pytest tests for auto-detection: test with known source languages (en, fr, de, es), test with mixed-language content (expect "und"), test manual override of "und" rows.

US7 (NEW) — Multilingual Dictionaries

  • T091 [P] [US7] Update DictionaryManager (backend/src/plugins/translate/dictionary.py): add source_language and target_language to all CRUD operations. Update unique constraint enforcement. Add language-pair filtering to filter_for_batch() method.
  • T092 [P] [US7] Update dictionary import (frontend/src/routes/translate/dictionaries/[id]/+page.svelte): CSV/TSV import now accepts source_language, target_language columns (or defaults from UI). Add language pair columns to the entry table. Add filter bar by language pair.
  • T093 [P] [US7] Update filter_for_batch() in backend/src/plugins/translate/dictionary.py: accept source_language and target_language parameters. Only return entries where source_language matches row's detected language AND target_language matches prompt's target language.
  • T094 [P] [US7] Migrate existing single-language dictionaries: each entry gets source_language = job.source_language or "und", target_language = dictionary.target_language. Create Alembic migration data script.
  • T095 [US7] Write pytest tests for multilingual dictionaries: test language-pair filtering, test migration of old dictionaries, test unique constraint with language pairs, test import with language columns.

US1 (UPDATED) — Multi-Language Job Configuration

  • T096 [P] [US1] Update job configuration endpoints in backend/src/api/routes/translate/: POST/GET/PUT jobs now handle target_languages list. Validate min 1 language. Backward-compatible: single-language requests wrapped into list.
  • T097 [US1] Update TranslationJobConfig component (frontend/src/routes/translate/[id]/+page.svelte): "Target Language" dropdown → multi-select with search. "Include source language as reference" checkbox. Source language field removed.
  • T098 [US1] Update TranslationJobList component (frontend/src/routes/translate/+page.svelte): display target languages as badges/tags. Update job card layout.

US2 (UPDATED) — Multi-Language Preview

  • T099 [P] [US2] Update TranslationPreview (backend/src/plugins/translate/preview.py): send source text to LLM once, request translations for ALL target languages + detected source language. Parse multi-language response. Create TranslationPreviewLanguage rows per language.
  • T100 [P] [US2] Update preview endpoints in backend/src/api/routes/translate/: accept sample_size parameter (1-100, default 10). Return per-language preview data via TranslationPreviewLanguageResponse. Include estimated tokens/cost for multi-target.
  • T101 [US2] Update TranslationPreview component (frontend/src/lib/components/translate/TranslationPreview.svelte): dynamic per-language columns. Sample size slider (1-100) with cost warning at >30. Per-language accept button. Added "und" warning badge for undetermined source language.
  • T102 [P] [US2] Implement preview edit carry-forward: when preview is accepted with edited rows, store the edited values per-language. On full execution, check for existing edits by key_hash + language_code and use edited value instead of re-translating.
  • T103 [US2] Write pytest tests for multi-language preview: test 2-language preview, test 5-language preview, test sample size boundary (1, 100), test cost estimation accuracy.

US3 (UPDATED) — Multi-Language Execution

  • T104 [P] [US3] Update TranslationExecutor (backend/src/plugins/translate/executor.py): construct multi-language LLM prompt (one prompt requesting translations for ALL target languages). Parse multi-language JSON response. Create TranslationLanguage rows per language per record.
  • T105 [P] [US3] Update TranslationOrchestrator (backend/src/plugins/translate/orchestrator.py): create TranslationRunLanguageStats rows per language on run start. Update statistics per language as batches complete. Track per-language token/cost.
  • T106 [P] [US3] Update run status endpoints in backend/src/api/routes/translate/: return language_stats array in run detail response. Include per-language progress in progress events.
  • T107 [US3] Update run result rendering in frontend/src/lib/components/translate/TranslationRunResult.svelte: show per-language statistics section with $derived. Display per-language columns in results table.
  • T108 [US3] Write pytest tests for multi-target execution: test 3-language run, test partial failure (one language fails), test single-language backward compatibility, test per-language statistics accuracy.

US8 (NEW) — Improved Correction & Preview Workflow

  • T109 [P] [US8] Implement inline correction endpoint in backend/src/api/routes/translate/: PUT /api/translate/runs/{run_id}/records/{record_id}/languages/{language_code} — update final_value, record user_edit. Optional submit_to_dictionary flag with dictionary_id parameter.
  • T110 [P] [US8] Implement POST /api/translate/runs/{run_id}/bulk-replace endpoint: accepts BulkFindReplaceRequest (find_pattern, regex flag, replacement_text, target_language). Returns affected record list. On apply: updates all matching TranslationLanguage.final_value; optionally submits to dictionary.
  • T111 [P] [US8] Add InlineCorrectionService in backend/src/plugins/translate/service.py: handle dictionary submission from correction — validate language pair, detect conflicts, create/update DictionaryEntry with origin tracking.
  • T112 [P] [US8] Add BulkFindReplaceService in backend/src/plugins/translate/service.py: pattern matching across run records, preview generation, atomic apply with optional dictionary submission.
  • T113 [US8] Create CorrectionCell component (frontend/src/lib/components/translate/CorrectionCell.svelte): inline editable cell with submit-to-dictionary action. Click → edit → save → "Submit to Dictionary" → popup. Language pair auto-filled.
  • T114 [US8] Create BulkReplaceModal component (frontend/src/lib/components/translate/BulkReplaceModal.svelte): find/replace pattern inputs, regex toggle, preview table, apply button with confirmation.
  • T115 [US8] Update preview component: add sample size slider (1-100) to preview trigger. Wire cost warning. No additional changes needed.
  • T116 [US8] Add API methods to frontend/src/lib/api/translate.js: inlineEditCorrection(), submitCorrectionToDict(), bulkFindReplace(), bulkReplacePreview().
  • T117 [US8] Write pytest tests for inline correction: test single correction → dictionary, test duplicate detection, test bulk replace preview accuracy, test bulk replace atomic apply.
  • T118 [US8] Write vitest tests for CorrectionCell and BulkReplaceModal components: test edit flow, test submit-to-dictionary, test find/replace preview, test bulk apply.

US8b (NEW) — Context-Aware Correction & Dictionary [2026-05-14]

Purpose: Extend the correction workflow and dictionary with source-row context capture, context editing in popup, usage notes, and context-aware LLM prompt construction.

  • T123 [P] [US8b] Add context fields to DictionaryEntry in backend/src/models/translate.py: context_data (JSON, nullable), usage_notes (Text, nullable), has_context (Boolean, default=False), context_source (String, nullable). Generate Alembic migration.
  • T124 [P] [US8b] Update InlineCorrectionService (backend/src/plugins/translate/service.py): when a correction is submitted, auto-capture source row's context column values as context_data. Accept optional overrides from request. Accept usage_notes. Handle context_source tagging.
  • T125 [P] [US8b] Create ContextAwarePromptBuilder in backend/src/plugins/translate/prompt_builder.py: render dictionary entries with context annotations for has_context=True. Compute Jaccard similarity between entry context_data and incoming row context. Flag >50% overlap as # PRIORITY. Cap context at ~500 tokens per entry.
  • T126 [P] [US8b] Update dictionary import/export in API and frontend: CSV/TSV import accepts optional context_data, usage_notes columns. Export includes these fields.
  • T127 [P] [US8b] Update BulkFindReplaceService: when submit_to_dictionary_with_context=true, auto-capture context from first matching row per unique term pair. context_source="bulk". Warn if context varies across matched rows for same term.
  • T128 [US8b] Update CorrectionCell and TermCorrectionPopup: correction popup shows auto-captured context (with Edit/Remove), usage_notes textarea, context_source badge.
  • T129 [US8b] Update BulkReplaceModal: add "Submit to dictionary with context" checkbox + "Usage notes (applied to all)" textarea.
  • T130 [US8b] Update DictionaryEditor page: display context_data and usage_notes per entry in expandable row with inline editing. Add context_source badges.
  • T131 [US8b] Update filter_for_batch() in backend/src/plugins/translate/dictionary.py: accept row_context parameter. Integrate ContextAwarePromptBuilder for priority matching.
  • T132 [US8b] Write pytest tests for context-aware correction: test context capture (20 tests), context editing/removal, Jaccard similarity matching, priority flagging, 500-token truncation, context_source tagging.
  • T133 [US8b] Write vitest tests for context UI: context display in popup, context editing, usage notes, badge display, context removal, submit (6 tests).
  • T134 [P] [US8b] Update correction API endpoint schema: POST /api/translate/corrections now accepts optional context_data (JSON override), usage_notes (text), keep_context (boolean, default=true). Response includes new context fields. TermCorrectionSubmit now has keep_context: bool = True, endpoint passes it through.

US4 (UPDATED) — Multi-Language History & Metrics

  • T119 [P] [US4] Update history endpoints: GET /api/translate/runs returns per-language stats. Run detail includes language_stats. Metrics endpoint returns per-language breakdown.
  • T120 [US4] Update TranslationHistory page (frontend/src/routes/translate/history/+page.svelte): add per-language columns to run table. Show language badges in run list.
  • T121 [P] [US4] Update MetricSnapshot model: add per_language_metrics: JSON column. Update pruning logic to persist per-language metrics before pruning.
  • T122 [US4] Write tests for per-language metrics accuracy post-prune.

Dependencies & Execution Order

Phase Dependencies

  • Setup (Phase 1): No dependencies — can start immediately
  • Foundational (Phase 2): Depends on Setup — BLOCKS all user stories
  • US1 (Phase 3): Depends on Foundational — no dependencies on other stories. Recommended start after Foundational.
  • US5 (Phase 4): Depends on Foundational — can run in parallel with US1. Dictionary filtering will be consumed later.
  • US2 (Phase 5): Depends on US1 (needs saved job) + US5 (needs dictionary filtering). Can start once US1 backend is stable.
  • US3 (Phase 6): Depends on US1 + US2. Sequential after US2.
  • US6/10 (Phase 7/8): Original US6+US10 (corrections, scheduling). Depends on US3.
  • US4 (Phase 9): Depends on US3.
  • Polish (Phase 10): Depends on all desired user stories.
  • Phase 11 (Multi-Language) [NEW]:
    • Migration (T083-T086): Blocks ALL Phase 11 work — models, schemas, migration scripts MUST be done first
    • US6 auto-detection (T087-T090): Depends on migration. No dependencies on other Phase 11 stories.
    • US7 multilingual dict (T091-T095): Depends on migration. Can run in parallel with US6 auto-detection.
    • US1 updated (T096-T098): Depends on migration + US7 (for language-aware dict attachment). Can start after migration.
    • US2 preview (T099-T103): Depends on migration + US6 (auto-detection for preview display).
    • US3 execution (T104-T108): Depends on migration + US6 + US7. Sequential after preview changes.
    • US8 correction (T109-T118): Depends on migration + US3 (needs multi-language run results).
    • US8b context (T123-T134): Depends on US8 (inline correction exists) + US7 (dictionary filtering). Can start after T109-T114 are stable. Fits in US8's parallel slot.
    • US4 history (T119-T122): Depends on US3 (per-language run data). Can run in parallel with US8/US8b.

Parallel Opportunities

Phase Parallel Tasks Notes
1 — Sequential (only 2 tasks)
2 T003 ∥ T004 Models + Schemas in parallel
3 (US1) T009 ∥ T012 Backend CRUD ∥ API client
4 (US5) T017 ∥ T021 DictionaryManager ∥ API client
5 (US2) T030 ∥ T031 API client ∥ Preview component
6 (US3) T036 ∥ T041 SQLGenerator ∥ API client
7 (US6) T050 ∥ T051 ∥ T052 API client ∥ Popup ∥ Sidebar
8 (US10) T061 ∥ T062 API client ∥ ScheduleConfig
9 (US4) T069 ∥ T070 API client ∥ History page
10 T074 ∥ T075 ∥ T076 Retention, notifications, belief instrumentation
11a (Migration) T083 ∥ T084 ∥ T085 ∥ T086 All model/schema changes in parallel [NEW]
11b (US6 + US7) T087 ∥ T088 ∥ T089 ∥ T091 ∥ T092 ∥ T093 Auto-detection + multilingual dict in parallel after migration [NEW]
11c (US8) T109 ∥ T111 ∥ T112 ∥ T113 ∥ T114 Inline correction backend + frontend components [NEW]
11d (US8b) T123 ∥ T124 ∥ T125 ∥ T126 ∥ T127 Context fields, builder, import/export, bulk [NEW]

Cross-Story Parallelism

After Migration (Phase 11a):

  • US6 auto-detection and US7 multilingual dict can proceed in parallel
  • After US3 original + Phase 11 US3: US8 correction and US4 history update can proceed in parallel
  • US8b context can proceed in parallel with US8 frontend work once T123-T124 merge

Implementation Strategy

MVP First (US1 Only)

  1. Phase 1 + Phase 2 → Foundation
  2. Phase 3 (US1) → Job configuration CRUD
  3. STOP and VALIDATE: User can create, list, edit, delete translation jobs
  4. Deploy/demo — partial value (configuration ready, no translation yet)

Minimum Viable Feature (US1 + US5 + US2 + US3)

  1. Foundation → US1 + US5 (parallel) → US2 → US3
  2. STOP and VALIDATE: End-to-end translation flow works: configure → preview → execute → INSERT
  3. This is the core feature — all remaining stories add automation (US10), quality improvement (US6), and visibility (US4)

Full Feature (All Stories)

  1. MVP → US6 + US10 + US4 (parallel after US3) → Polish
  2. Scheduled automation, feedback loop, and audit trail all functional

Multi-Language Extension (Phase 11)

  1. Migration first → Models, schemas, Alembic, data migration
  2. US6 + US7 in parallel → Auto-detection + Multilingual dictionaries
  3. US1 – US3 sequentially → Multi-language job config → Preview → Execution
  4. US8 + US4 in parallel → Inline correction + Per-language history/metrics
  5. Quickstart re-validation → Update T078 to cover multi-language flow

Notes

  • All file paths reference the actual repository structure (backend/src/, frontend/src/).
  • @COMPLEXITY 4/5 backend contracts require belief_scope/reason/reflect markers — verified in T076.
  • @RATIONALE/@REJECTED tags appear only in C5 contracts (TranslationOrchestrator, TranslationEventLog) per INV_7.
  • Rejected paths are explicitly protected by regression tests in T079.
  • [NEED_CONTEXT] markers: none — all contract targets resolve to existing or planned modules within this feature.
  • The existing LLMProviderService, SupersetClient, SchedulerService, NotificationService, and TaskWebSocket contracts are reused without modification.
  • Quickstart.md (T078) serves as the human-verifiable acceptance test for the full feature. After Phase 11, update quickstart to cover multi-language flow.
  • Migration considerations:
    • TranslationJob.target_language → target_languages migration is one-time. Old API consumers sending single target_language are backward-compatible (wrapped into list server-side).
    • TerminologyDictionary.target_language removal: existing dictionaries get entries migrated with source_language = job.source_language or "und" and target_language = dictionary.target_language.
    • TranslationRecord → TranslationLanguage migration: existing single-language records get one TranslationLanguage entry per record.
  • LLM prompt change: Multi-target prompts add non-trivial token overhead (~50% for 3 languages). The prompt structure is: "Translate the following text to languages: [ru, en, de]. Return JSON with detected_source_language and translations keyed by language code."
  • Context-aware prompt format (US8b):
    • Entry without context: "Voiture" → "Car"
    • Entry with context: "Voiture" (context: Category=Automotive, Type=Engine) → "Car"
    • Entry with context + notes: "Voiture" (context: Category=Automotive) → "Car" # Usage: Only for passenger vehicles
    • Priority entry: "Voiture" (context: Category=Automotive) → "Car" # PRIORITY - context match: 75%
    • Prompt construction caps context at ~500 tokens/entry. Priority entries listed before non-priority entries within the same language pair block.
  • Context similarity heuristic: Jaccard similarity on tokenized (lowercased, stopwords-removed) text of context column values. Threshold 0.5. Configurable via job settings. Stored per-run in config snapshot.

✅ Spec 028 Final Closure Summary (2026-05-17)

All 134 tasks complete. Spec 028 fully implemented.

Финальная статистика реализации

Метрика Значение
Всего файлов 72 (бэкенд: 41, фронтенд: 19, конфиги/миграции: 12)
Всего строк кода 29 128
Бэкенд Python ~15 777 строк (модули: 7 761, routes: 2 141, модели: 397, схемы: 688, миграции: 586, тесты: 8 103)
Фронтенд Svelte/JS ~9 976 строк (компоненты: 3 971, страницы: 3 256, API+store: 860, i18n: 1 223, тесты: 575, E2E: 142)
Backend tests 165 ✅ All pass
Frontend tests 281 ✅ 280 pass (1 pre-existing unrelated)
Semantic audit 0 warnings
User stories 10 (US1–US10)
ORM models 15 таблиц
Pydantic schemas 35+
API endpoints ~30 REST
RBAC permissions 13
Alembic migrations 4
Максимальный файл executor.py (1 974 строки)
Самая сложная бизнес-логика orchestrator.py (C5, 1 137 строк), events.py (C5, 272 строки)

Архитектура (post-factum)

  • Архитектура: Service-based (не plugin-based, как планировалось изначально)
  • Plugin.py: C2 skeleton (только регистрация, execute() → NotImplementedError)
  • Routes: Пакет из 11 файлов (не один translate.py, как в плане)
  • Основные сервисы: TranslateJobService, TranslationOrchestrator, TranslationExecutor, TranslationPreview, DictionaryManager, TranslationScheduler, TranslationEventLog

Структура файлов (полный инвентарь)

Backend modules (15 файлов, backend/src/plugins/translate/)

Файл Строк C Назначение
__init__.py 2 — Package init
plugin.py 61 C2 Plugin skeleton (NotImplementedError)
orchestrator.py 1 137 C5 Run lifecycle coordinator ⭐
executor.py 1 974 C4 Batch LLM + caching ⭐
service.py 1 052 C4 Job CRUD + InlineCorrection + BulkReplace ⭐
preview.py 1 303 C4 Preview engine ⭐
dictionary.py 1 007 C4 Dictionary CRUD + import ⭐
scheduler.py 436 C4 Schedule CRUD
sql_generator.py 378 C3 Dialect-aware SQL (pure function)
superset_executor.py 420 C3 Superset SQL Lab API (submit + poll)
prompt_builder.py 153 C3 Context-aware prompt (Jaccard)
events.py 272 C5 Event log + MetricSnapshot pruning
metrics.py 192 C3 Metrics aggregation
_token_budget.py 341 C2 Token estimation
_utils.py 33 C1 Term normalization

Backend routes (11 файлов, backend/src/api/routes/translate/)

Файл Строк Назначение
__init__.py 40 Package init + re-export
_router.py 13 APIRouter instance
_job_routes.py 249 Job CRUD
_run_routes.py 554 Run execution, cancel, inline edit ⭐
_run_list_routes.py 225 Run list, detail, CSV
_preview_routes.py 194 Preview
_dictionary_routes.py 391 Dictionary CRUD + import
_correction_routes.py 100 Correction submission
_schedule_routes.py 239 Schedule CRUD
_metrics_routes.py 65 Metrics
_helpers.py 71 Response helpers

Backend models + schemas (2 файла)

Файл Строк Назначение
models/translate.py 397 15 ORM моделей
schemas/translate.py 688 35+ Pydantic DTOs

Backend tests (15 файлов, 8 103 строки)

Файл Строк Назначение
__tests__/test_orchestrator.py 847 Full run lifecycle
__tests__/test_executor.py 697 Batch LLM processing
__tests__/test_dictionary.py 1 199 Dictionary CRUD + filter
__tests__/test_preview.py 609 Preview engine
__tests__/test_scheduler.py 407 Schedule CRUD
__tests__/test_sql_generator.py 368 Dialect SQL tests
__tests__/test_token_budget.py 195 Token estimation
__tests__/test_inline_correction.py 792 Correction + context
__tests__/test_clickhouse_insert_integration.py 546 ClickHouse SQL
__tests__/test_orthogonal_fixes.py 582 Regression
tests/test_translate_jobs.py 562 Job API integration
tests/test_translate_corrections.py 249 Correction API
tests/test_translate_executor_filter.py 289 Dictionary filter
tests/test_translate_history.py 466 History API
tests/test_translate_scheduler.py 195 Schedule integration
tests/test_translate_scheduler_execution.py 399 Schedule execution
tests/test_translate_scheduler_guard.py 298 Concurrency guard

Frontend components (10 файлов, frontend/src/lib/components/translate/)

Файл Строк C Назначение
TranslationPreview.svelte 561 C4 Preview table
TranslationRunResult.svelte 464 C4 Run results
BulkReplaceModal.svelte 396 C3 Find & replace
TranslationRunProgress.svelte 310 C4 Live progress
ScheduleConfig.svelte 286 C3 Schedule editor
BulkCorrectionSidebar.svelte 285 C3 Bulk correction
TermCorrectionPopup.svelte 342 C3 Correction popup
CorrectionCell.svelte 326 C3 Inline edit cell
TranslationRunGlobalIndicator.svelte 198 C2 Global run indicator
TranslationMetricsDashboard.svelte 184 C2 Metrics dashboard

Frontend pages (5 файлов)

Файл Строк Назначение
translate/+page.svelte 279 Job list
translate/[id]/+page.svelte 1 363 Job config + run ⭐
translate/history/+page.svelte 523 Run history
translate/dictionaries/+page.svelte 270 Dictionary list
translate/dictionaries/[id]/+page.svelte 821 Dictionary editor

Phase 11 Completion Details

  • Migration (T083-T086): Models, schemas, Alembic — done
  • US6 Auto-detection (T087-T090): LLM returns BCP-47, tests added
  • US7 Multilingual dict (T091-T095): Language-pair aware CRUD + filtering
  • US1/2/3 Multi-language (T096-T108): Config → Preview → Execution
  • US8 Inline correction (T109-T118): Endpoints, BulkReplaceModal, CorrectionCell
  • US8b Context-aware (T123-T134): context_data, ContextAwarePromptBuilder, Jaccard
  • US4 History (T119-T122): Per-language stats, MetricSnapshot

Расхождения с планом (документировано post-factum)

Аспект План Реальность
Архитектура Plugin-based Service-based (plugin.py — скелет C2)
Routes Один translate.py Пакет из 11 файлов
SQLGenerator C4 C3 (pure function, нет I/O)
TranslateRoutes C3 C4 (пакет 11 файлов, 6+ зависимостей)
InlineCorrectionService C4 C3 (без полноценных PRE/POST)
BulkFindReplaceService C4 C3 (без полноценных PRE/POST)
Новые модули — superset_executor.py, prompt_builder.py, service.py, _token_budget.py, _utils.py
Новые компоненты — CorrectionCell, BulkReplaceModal, TranslationRunGlobalIndicator, TranslationMetricsDashboard, translate.js store

Verified

  • Backend: pytest 165/165 pass ✅
  • Frontend: vitest 280/281 pass ✅ (1 pre-existing unrelated failure)
  • Browser: validated — preview, correction, dictionary flows work
  • Semantic audit: 0 warnings across all checks

Remaining

  • None. All 134 tasks are [x].
  • 1 pre-existing unrelated vitest failure (not in translate module)

Next Action

  • ready_for_review — Spec 028 is complete, ready for merge to main