ss-tools/specs/028-llm-datasource-supeset/quickstart.md

Quickstart: LLM Table Translation Service

Feature Branch: 028-llm-datasource-supeset (актуальная: 032-translate-requests-httpx)
Date: 2026-05-08 (updated 2026-06-11)

Prerequisites

• Running superset-tools instance (Docker Compose or local)
• Superset connection configured in superset-tools settings
• At least one LLM provider configured (Settings → LLM)
• Target insertable PostgreSQL physical table exists in Superset with compatible schema
• User has appropriate RBAC permissions (admin by default)
• Database connection configured in Settings → Connections (for direct DB insert)

1. Start the Application

# Docker (recommended)
cd /home/busya/dev/superset-tools
docker compose up --build

# Or local development
# Terminal 1 — Backend
cd backend
source .venv/bin/activate
python -m uvicorn src.app:app --reload --port 8001

# Terminal 2 — Frontend
cd frontend
npm run dev -- --port 5173

• Frontend: http://localhost:5173
• Backend API: http://localhost:8001
• API Docs: http://localhost:8001/docs

2. Create a Terminology Dictionary

Note (2026-05-17): Dictionaries are multilingual. Each entry carries its own source_language + target_language (BCP-47). Dictionary itself has no language field.

Via UI

1. Navigate to http://localhost:5173/translate/dictionaries
2. Click [+ New Dictionary]
3. Enter name: Product Terms
4. Add entries inline with language pairs or click [Import CSV]
5. Save

Via API

curl -X POST http://localhost:8001/api/translate/dictionaries \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "name": "Product Terms",
    "entries": [
      {"source_term": "invoice", "source_term_normalized": "invoice", "source_language": "en", "target_language": "ru", "target_term": "накладная"},
      {"source_term": "widget", "source_term_normalized": "widget", "source_language": "en", "target_language": "ru", "target_term": "виджет"},
      {"source_term": "backorder", "source_term_normalized": "backorder", "source_language": "en", "target_language": "ru", "target_term": "предзаказ"}
    ]
  }'

Expected: 201 Created with dictionary ID and entry count = 3.

3. Create a Translation Job

Via UI

1. Navigate to http://localhost:5173/translate
2. Click [+ New Translation Job]
3. Select Superset datasource → columns auto-populate
4. Set:
   • Translation column: product_name
   • Context columns: category_name, product_description
   • Key columns: product_id
   • Target table: products_i18n
   • Target column: translated_name
   • Target language: Russian
   • Attach dictionary: Product Terms
5. Click [Save & Preview]

Via API

curl -X POST http://localhost:8001/api/translate/jobs \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "name": "Products RU Translation",
    "datasource_id": "<datasource-uuid>",
    "source_table": "products",
    "translation_col": "product_name",
    "context_cols": ["category_name", "product_description"],
    "source_key_cols": ["product_id"],
    "target_key_cols": ["product_id"],
    "target_table": "products_i18n",
    "target_col": "translated_name",
    "target_languages": ["ru", "de"],       # [UPDATED] list, not single string
    "batch_size": 50,
    "dictionary_ids": ["<dictionary-uuid>"]
  }'

Expected: 201 Created with job ID. Validation passes (columns exist, target table accessible).

Error case: 422 if translation column is empty or target_languages is empty; 400 if target table not found.

4. Preview Translations

Via UI

1. Open the saved job → click [Preview]
2. System shows ~10 rows with source, context, and LLM translation
3. Approve good translations, edit or reject bad ones
4. Click [Approve All] or handle individually

Via API

curl -X POST http://localhost:8001/api/translate/jobs/<job-id>/preview \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"sample_size": 10}'

Expected: 200 with array of PreviewRow objects (source_text, context, llm_translation, status=pending).

Error case: 503 if LLM provider unreachable; error message includes provider name and reason.

5. Execute Full Translation Run

Via UI

1. After preview approval, click [Start Full Run]
2. Confirm cost estimate dialog
3. Watch live progress bar (WebSocket-driven)
4. On completion: view run summary with translation status, insert status, Superset query reference, and generated SQL (audit).

Via API

curl -X POST http://localhost:8001/api/translate/jobs/<job-id>/runs \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"upsert_strategy": "insert"}'

Expected: 202 Accepted with run ID. WebSocket messages stream progress. Final GET returns run with status=completed, translated_rows=N, insert_sql=<SQL>.

Partial failure: status=partial, failed_rows>0. [Retry Failed] available.

6. Execute INSERT through Superset SQL Lab API

Via UI

1. After translation completes, the system automatically submits SQL to Superset
2. Progress indicator shows: «📤 Submitting to Superset...»
3. On success: «✅ Insert succeeded · 1,241 rows affected · Query #a7f3b2c»
4. Click [View SQL] to audit the generated statement

Via API

# Trigger full run (backend handles Superset submission automatically)
curl -X POST http://localhost:8001/api/translate/jobs/<job-id>/runs \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{"upsert_strategy": "insert"}'

# Check run status (includes insert_status and superset_query_id)
curl http://localhost:8001/api/translate/runs/<run-id> \
  -H "Authorization: Bearer <token>"

Expected: Run response includes insert_status: "succeeded", superset_query_id, rows_affected.

Insert failure: insert_status: "failed", insert_error_message populated. [Retry Insert] re-submits without re-translating.

Verify in Target Table

-- Run directly in Superset SQL Lab to verify
SELECT * FROM products_i18n WHERE translated_name IS NOT NULL;

7. Feedback Loop — Correct a Translation

Via UI

1. Open run results → find a mistranslated word
2. Highlight the word → [Correct this term] popup
3. Enter correction → select dictionary → submit
4. Re-run preview to verify correction is used

Via API

curl -X POST http://localhost:8001/api/translate/corrections \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "record_id": "<record-uuid>",
    "source_term": "Monitor Stand",
    "source_term": "Monitor Stand",
    "incorrect_target_term": "Мониторная стойка",
    "corrected_target_term": "Подставка для монитора",
    "dictionary_id": "<dictionary-uuid>"
  }'

Expected: 201. Term pair added to dictionary. Conflict dialog if term already exists.

8. Configure Schedule

Via UI

1. Open job → Schedule tab
2. Set type: Cron → 0 6 * * 1 (every Monday 06:00)
3. Toggle auto-INSERT: ON
4. Verify next 3 execution times
5. Enable schedule

Via API

curl -X PUT http://localhost:8001/api/translate/jobs/<job-id>/schedule \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "schedule_type": "cron",
    "cron_expression": "0 6 * * 1",
    "timezone": "Europe/Moscow",
    "concurrency": "skip"
  }'

Expected: 200 with schedule config including next_run_at.

Verify: Check APScheduler jobs (backend log) or wait for next trigger and check run history.

9. View History and Metrics

Via UI

1. Navigate to http://localhost:5173/translate/history
2. Filter by datasource, target table, or date range
3. Click a run for details: config snapshot, prompt, translations, INSERT SQL

Via API

# List runs
curl http://localhost:8001/api/translate/runs?job_id=<job-id> \
  -H "Authorization: Bearer <token>"

# Get metrics
curl http://localhost:8001/api/translate/jobs/<job-id>/metrics \
  -H "Authorization: Bearer <token>"

Expected: Run list with status and row counts. Metrics with cumulative tokens and cost.

10. Direct Database Insert (NEW)

Prerequisites

• At least one database connection configured in Settings → Connections
• Connection must have a dialect matching the job's Superset datasource (PostgreSQL or ClickHouse)

Via UI

1. Navigate to Settings → Connections
2. Click [+ Add Connection]
3. Fill in: name, host, port, database, username, password, dialect (PostgreSQL/ClickHouse)
4. Click [Test Connection] → verify success (✅ Connected, )
5. Click [Save Connection]
6. Navigate to the translation job config
7. In the Insert Method section, select "Direct Database Connection"
8. Choose the saved connection from the dropdown (filtered by compatible dialect)
9. Save the job → execute run
10. Run result shows: "Insert: ✅ Direct DB · <connection_name> · "

Via API

# Create a database connection
curl -X POST http://localhost:8001/api/admin/settings/connections \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "name": "Products DB",
    "host": "db.internal.example.com",
    "port": 5432,
    "database": "products_i18n",
    "username": "translator",
    "password": "secret",
    "dialect": "postgresql",
    "pool_size": 5
  }'

# Test the connection
curl -X POST http://localhost:8001/api/admin/settings/connections/<conn-id>/test \
  -H "Authorization: Bearer <token>"

# Create job with direct DB insert
curl -X POST http://localhost:8001/api/translate/jobs \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <token>" \
  -d '{
    "name": "Direct DB Translation",
    "datasource_id": "<datasource-uuid>",
    "translation_col": "product_name",
    "target_languages": ["ru"],
    "insert_method": "direct_db",
    "connection_id": "<connection-uuid>",
    ...
  }'

Expected: 201 Created. Run uses DbExecutor with asyncpg/clickhouse-connect. Superset query reference is NULL. Connection name shown in run result.

Error case: 400 if connection_id missing when insert_method=direct_db; 404 if connection not found.

11. Verification Checklist

Backend Tests

cd backend
source .venv/bin/activate

# Unit tests for translation plugin (27 файлов в __tests__/)
pytest src/plugins/translate/__tests__/ -v

# Integration tests for translate API (10 файлов в tests/)
pytest tests/test_translate_jobs.py tests/test_translate_corrections.py tests/test_translate_history.py tests/test_translate_scheduler.py tests/test_translate_scheduler_execution.py tests/test_translate_scheduler_guard.py tests/test_translate_executor_filter.py tests/core/test_async_regression.py -v

# All backend tests
pytest -v

Frontend Tests

cd frontend
npm run test -- --run

Linting

# Python
cd backend && ruff check src/plugins/translate/ src/api/routes/translate/ src/models/translate.py src/schemas/translate.py src/core/connection_service.py src/core/db_executor.py

# Svelte
cd frontend && npm run build  # build includes type checking

Manual Smoke Test

1. Create dictionary with 3 terms → verify in list
2. Import CSV with 50 terms → verify no duplicates (check conflict dialog)
3. Create job → verify column list populates from datasource
4. Preview with empty dictionary → verify LLM still translates
5. Preview with attached dictionary → verify glossary terms used (check invoice → накладная)
6. Full run with 50 rows → verify INSERT SQL has 50 VALUES tuples
7. Scheduled run (set to every 5 min for test) → verify run appears in history
8. Feedback loop: correct 1 term → re-preview → verify correction reflected
9. Delete dictionary attached to active job → verify blocked
10. Check metrics dashboard → verify run counts and token totals
11. Configure direct DB insert: create connection → test → set job insert_method=direct_db → run → verify rows in target table with no Superset API calls