- Removes source_dialect and target_dialect from TerminologyDictionary model,
schemas, routes, helper serialization, and all tests
- Adds allowed_languages config field (BCP-47 language codes) to GlobalSettings
with validation, consolidated settings response, and API endpoint
- Adds alembic migration f1a2b3c4d5e6 to drop the two columns
1. dict_snapshot_hash now includes entry count + max(updated_at)
per dictionary. Previously only hashed dictionary IDs, meaning
edits to dictionary entries did NOT invalidate the translation
cache. Stale cached translations could be served after editing
dictionary entries. (HIGH severity)
2. _batch_insert.py now uses SQLGenerator.generate_batch() with
500-row chunking instead of a single massive INSERT statement.
Prevents potential SQL size limit issues with large batches or
many target languages. (LOW severity)
3. Fixed same bug in preview_response_parser.py —
compute_dict_snapshot_hash had identical ID-only hash flaw.
Tests: 69/69 translate tests pass.
Backend:
- TranslationRun model: add cache_hits Integer column (default 0)
- TranslationRunResponse schema: add cache_hits field
- _helpers.py/_run_list_routes.py/orchestrator_aggregator.py: include
cache_hits in all run API responses
- _batch_proc.py: count pre_rows served from translation cache per
batch, return cache_hits in batch result
- executor.py: accumulate cache_hits across batches, persist to run
- Alembic migration: dabc9709 — add cache_hits column to translation_runs
Frontend:
- translationRun.svelte.ts: add cacheHits to store state, WS handler,
polling handler
- TranslationRunProgress.svelte: 5-col stats grid with purple Cache card
- TranslationRunGlobalIndicator.svelte: 5-col stats with Cache
- TranslationRunResult.svelte: 5-col detail stats with Cache card
- History page: cache_hits shown in run list row + detail panel
Visual: cache hits shown in purple alongside green/yellow/red metrics
(total/success/failed/skipped). Visible during run + in history.
Tests: backend 69/69 translate tests ✅, frontend 698/698 tests ✅,
frontend build ✅
Bug 1 (secondary): _persist_pre() created TranslationLanguage with
empty final_value for non-source target languages when same_language
flag was set but no cache/approved value existed. This produced
is_original=0 rows with empty text in target table.
Fix: skip TranslationLanguage creation when fv is empty (continue
in the loop instead of falling through to else with empty string).
Bug 2 (diagnostic): No visibility into _classify() routing decisions.
Add logger.reason with pre/llm/total counts per batch.
Scenario trace: ru source + targets [ru,en] + no cache → llm_rows ✓
If same-language partial match hits cache for all targets → pre_rows ✓
If same-language partial match misses cache → llm_rows ✓
If same-language all-match (targets=[ru]) → short-circuit pre_rows ✓
Tests: 69/69 translate tests pass.
- DashboardHealthItem schema: add run_id, policy_id, execution_path,
issues_count, timings, token_usage, screenshot_paths, chunk_count,
dashboard_name fields for v2 LLM validation reporting.
- HealthService: populate v2 fields from validation run records.
- ValidationService: update to support v2 validation run fields.
- validation_tasks route: minor fix for v2 field handling.
- tailwind.config.js: refactor design tokens — add success/warning/info
palettes, surface/border/text hierarchy, semantic action colors.
- Agent configs: update svelte-coder and semantics-svelte for
.svelte.ts runes and Screen Model patterns.
Root cause: _classify() immediately skipped (continue) when detected
language matched ANY target language, preventing cache lookup and LLM
translation for remaining targets. E.g., ru source + targets [ru, en]
→ only ru row created, en never processed (8978 ru vs 18 en in DB).
Fix:
1. _classify(): Only short-circuit when ALL targets match detected
language. Partial match → mark _same_language but continue to
cache check and LLM for non-matching targets.
2. _persist_pre(): Unify two code paths into single loop over all
target languages. Same-language targets use source_text as-is;
cached targets use cache value; fallback to approved_translation.
3. SIM102: Merge nested 'if cl: if all(...)' → 'if cl and all(...)'
to keep cyclomatic complexity ≤ 10 (INV_7).
QA: All 69 translate tests pass. 7 scenario traces (A-G) verified.
No regressions in approved_translation, preview edits, or single-
target same-language flows.
When Kilo gateway + Nvidia NeMo :free model returns content: null
for image inputs, get_json_completion crashed with
'the JSON object must be str, bytes or bytearray, not NoneType'
and retried 5 times (unnecessary).
Fixes:
- _should_retry no longer retries RuntimeError with 'null content'
(retrying won't help — the model genuinely returned null)
- get_json_completion now raises RuntimeError explicitly when
content is None, with a clear message (instead of json.loads(None))
Two bugs preventing screenshot display in report UI:
1. Plugin's ValidationRecord constructor did NOT include
screenshot_paths (plural, JSON array). The column stayed NULL
even though paths were stored in raw_response. Records created
by the plugin now include screenshot_paths=webp_paths|jpeg_paths.
2. Backward compat for existing records: _record_to_dict now
falls back to extracting screenshot_paths from raw_response JSON
when the DB column is empty.
Also fixes: broken import statement in validation_service.py
(missing closing paren + missing imports from earlier edit).
The _supports_json_response_format check only looked for
'openrouter.ai' in base_url, missing the Kilo gateway
('api.kilo.ai'). Free-tier models routed through ANY gateway
(Nvidia NeMo :free via OpenRouter OR Kilo) reject json_object
response format, causing the LLM to return content: null.
Error: 'the JSON object must be str, bytes or bytearray, not NoneType'
Fix: check model name for ':free' and 'stepfun/' directly,
regardless of gateway. This covers OpenRouter, Kilo, and any
future gateway.
When max_images could not be detected (e.g. Kilo gateway doesn't
support OpenAI image format, probe returned 0), chunking was
disabled entirely and all screenshots sent in a single request.
Nvidia NeMo still enforces an 8-image limit regardless of the
gateway, causing 400 errors.
Fix: default to 8 images per chunk when max_images is 0 or None,
so chunking always applies for multi-tab dashboards.
Root cause of stuck 'running' runs: the plugin's execute() method
only processed dashboard_ids[0] and returned. The remaining N-1
dashboards were never processed, and _update_run_status was called
after each single dashboard without checking dashboard_count.
Fixes:
- execute() now loops over ALL dashboard_ids and processes each
via _execute_path_a or _execute_path_b
- _update_run_status is called once AFTER the loop finishes
- _update_run_status now checks that len(records) >= dashboard_count
before marking the run as finished (prevents premature completion)
- Per-dashboard _update_run_status calls removed from _execute_path_a
and _execute_path_b (the parent loop owns it now)
- Test updated for new batch return format {dashboards: [...], total: N}
When the backend restarts (deploy, crash, reload), the in-memory
TaskManager loses its queue. ValidationRun records left 'running'
in the DB would hang forever, blocking re-runs.
Fix: during lifespan startup, query all ValidationRun with
status='running' and force-stop them as FAIL with a clear summary.
This prevents the 'already has a running run' error after restarts.
- Extract _optimize_images() helper to eliminate duplicate
optimization code (was duplicated between initial pass and
quality-reduction fallback)
- Move quality-reduction estimate to only apply when NOT chunking
(each chunk fits the image limit by definition)
- Fix _merge_chunk_results to return 'chunk_count' instead of 'chunks'
for consistency with plugin.py
- Simplify plugin.py: analysis.get('chunk_count', 1) instead of
fallback chain
- Document that Kilo API gateway is incompatible with AsyncOpenAI
image format (probe returns 0)
Delete deprecated code that has been fully replaced by v2:
- backend: validation.py routes, validation_run_service.py
- frontend: validation.js API module, validation/ pages
Add binary-search probe endpoint POST /providers/{id}/probe-max-images
that discovers the per-request image limit by sending incrementally
more 1x1 JPEGs via the provider's own API. Result is cached in the
new max_images column on the provider config.
- LLMProviderConfig: add max_images: int | None
- LLMProvider (SQLAlchemy): add max_images column
- Migration ed28d34edde7: clean ADD COLUMN
- LLMProviderService: create/update/set_max_images
- POST /providers/{id}/probe-max-images: binary search + error parsing
- ProviderConfig.svelte: 'Detect' button in edit modal + HelpTooltip
- i18n (en/ru): 11 new keys for probe UI
1. Remove get_config_manager() call from translate_run_websocket
(@app.websocket /ws/translate/run/{run_id}) — was not imported
and unused. Also remove unused from sqlalchemy.orm import Session.
2. Fix prop name mismatch: showBulkReplace -> showPageBulkReplace
in RunTabContent. Use () for reactive two-way binding.
3. Add ('DRAFT') for status prop in RunTabContent.
Backend:
- Add /ws/translate/run/{run_id} WebSocket endpoint in app.py
- Streams structured run status every second (total_records,
successful_records, failed_records, progressPct, batch_count, etc.)
- Auto-detects terminal states (COMPLETED, FAILED, CANCELLED) and closes
- Authenticated via JWT/API key token query param
Frontend:
- Add getTranslateRunWsUrl() URL builder in api.js
- Update translationRunStore to use WebSocket instead of log stream
- WebSocket receives structured status JSON, updates store reactively
- Falls back to polling silently if WebSocket fails
- Keep polling as fallback for insert phase
Root cause: fetch_available_datasources did not return database_id,
so frontend could not auto-set target_database_id when user selected
a datasource. User then had to manually pick a database on Target Config
tab — and could accidentally select the wrong one.
Backend changes:
- Add 'database_id' field to datasources response (from db_info.get('id'))
- Add _database_name tracking in SupersetSqlLabExecutor with getter
- Add database_name + database_backend to TargetSchemaValidationResponse
- Enrich resolve_database_id logging with database_name and all_keys
Frontend changes:
- In selectDatasource(), auto-set targetDatabaseId from ds.database_id
when present, so the correct target DB is used for schema checks.
- Replace fragile substring check (any(kw in backend for kw in ('clickhouse', 'ch')))
with a proper _backend_normalize mapping + exact comparison.
Fixes misdetection of Greenplum (postgresql) and other backends.
- Add 'clickhousedb' to CLICKHOUSE_DIALECTS so SQL generator uses
backtick quoting and correct INSERT strategy for ClickHouse.
- Normalize _extract_dialect to map 'clickhousedb' -> 'clickhouse'
and 'greenplum' -> 'postgresql'.
OpenSSL 3.x ignores intermediate CA certs in -CAfile (verify code 20)
but correctly builds chains with -CApath (verify code 0).
All three clients now use capath:
- service.py: ssl.create_default_context(capath=...)
- _llm_http.py: verify='/etc/ssl/certs/'
- preview_llm_client.py: verify='/etc/ssl/certs/'
check_llm_certs.py rewritten for clarity.
- ADR-0009: comprehensive SSL certificate management strategy
- _get_ssl_verify() returns SSLContext with cafile= instead of bool True
(httpx deprecates verify=<str>, requests needs system CA, not certifi)
- _get_verify() in translate plugin returns system CA path
- All tests updated for SSLContext return type
- All QA findings C1-C3, H1-H4, M1 incorporated into ADR
preview_llm_client.py and _llm_http.py both use requests.post()
without verify=, causing SSLError when corporate CA not in trust store.
Added _get_verify() helper that reads LLM_SSL_VERIFY env var.
Introduce a centralized tool registry system for the assistant to manage
executable operations, permissions, and tool catalogs. This refactors
the assistant dispatch logic from a monolithic approach to a modular,
decorator-based registry.
Key changes:
- Implement `AssistantToolRegistry` to handle tool registration,
permission checks, and safe operation identification.
- Add a suite of new assistant tools: backup, commit, branch creation,
deployment, health summary, environment listing, LLM operations,
maintenance, migration, and dashboard searching.
- Refactor `_dispatch.py` and `_llm_planner.py` to utilize the new
registry for intent resolution and tool catalog building.
- Enhance the LLM planner to support more descriptive clarification
responses in Russian when intents are ambiguous.
- Update the frontend to support the new tool-based workflow, including
improved i18n labels for assistant operations and a new step-by-step
wizard for the migration page.
- Add ADR-0008 to document the architectural decision for the tool
registry.
- LLM_SSL_VERIFY env var to disable SSL verification for corporate proxies
- install_llm_ca_certs() entrypoint function — downloads PEM/DER certs
from LLM_CA_CERT_URLS, converts DER→PEM, installs to system CA store
- _format_connection_error() — detailed exception chain logging
- 7 new unit tests for _get_ssl_verify, _format_connection_error, verify= param
- LLM_CA_CERT_URLS and LLM_SSL_VERIFY in .env.enterprise-clean
- Removed duplicate @RELATION DEPENDS_ON -> [EXT:Library:tenacity]
After configure_logger() runs, logger.isEnabledFor(logging.INFO)
returns False despite level=10 (DEBUG). This is a CPython logging
framework anomaly that makes logger.info() silently drop messages.
Workaround: check isEnabledFor first; if False, write JSON directly
to sys.stderr bypassing the logging system entirely.
Per molecular CoT protocol: REASON and REFLECT are primary reasoning
bonds and must be visible at INFO level in production. DEBUG is
reserved for high-frequency loops.
- reason(): self.debug() → self.info()
- reflect(): self.debug() → self.info()
- explore(): stays at warning() (WARNING level)
- Also removed redundant logger.info() calls in lifespan (now reason()
itself provides visible startup logging)
- Added logger.info() calls before each startup step (encryption key,
Alembic, init_db, admin bootstrap, scheduler, app complete)
- Previously only logger.reason() was used which logs at DEBUG level
and is invisible in production/development at INFO log level
- This fixes 'no backend logs visible' when running via ./run.sh
- Added missing from fastapi import status (global_exception_handler crashed)
- Moved TraceContextMiddleware to outermost (trace_id now available in log_requests)
- Enhanced global_exception_handler with query_params, client host
- Added logger.explore() in WebSocket auth except blocks (was silent pass)
- Fixed middleware registration order
- Added @app.exception_handler(Exception) that logs full traceback via
logger.exception() into superset_tools_app logger (visible in docker logs)
- Previously FastAPI/Starlette wrote unhandled exceptions to uvicorn.error
logger, making 500 errors invisible in docker logs
- Added JSONResponse import for the handler
- Entrypoint: for legacy DBs, add is_admin via raw SQL then stamp head
(instead of running full migration chain which fails on existing tables)
- Fixed admin_api_keys.py Pydantic config (dict → ConfigDict)
- Added tests/test_alembic_migrations.py (PostgreSQL-only integration tests)
- Fixed infinite recursion in _create_table_if_not_exists (QA catch)
- Simplified init_db() to only call Base.metadata.create_all() on all
three engines — no more _ensure_*() inline additive migrations
- run_alembic_migrations() now raises on failure (no safety net to
fall back to)
- All schema changes (add/drop/rename columns, data migrations) must
go through Alembic. create_all() handles new tables only.
@REJECTED _ensure_*() inline migrations removed because they:
- Duplicated Alembic logic without versioning
- Created hidden schema drift (columns existed in DB but had no
corresponding Alembic migration)
- Made audits and fresh DB provisioning unreliable
- Added 4 missing columns to _ensure_translation_jobs_columns() inline
migration: target_language_column, target_source_column,
target_source_language_column, disable_reasoning — these were in the
SQLAlchemy model but had no Alembic migration or _ensure_*() coverage
- Added missing api_key, assistant, clean_release model imports to
alembic/env.py so autogenerate can see all tables
