The _supports_json_response_format check only looked for
'openrouter.ai' in base_url, missing the Kilo gateway
('api.kilo.ai'). Free-tier models routed through ANY gateway
(Nvidia NeMo :free via OpenRouter OR Kilo) reject json_object
response format, causing the LLM to return content: null.
Error: 'the JSON object must be str, bytes or bytearray, not NoneType'
Fix: check model name for ':free' and 'stepfun/' directly,
regardless of gateway. This covers OpenRouter, Kilo, and any
future gateway.
When max_images could not be detected (e.g. Kilo gateway doesn't
support OpenAI image format, probe returned 0), chunking was
disabled entirely and all screenshots sent in a single request.
Nvidia NeMo still enforces an 8-image limit regardless of the
gateway, causing 400 errors.
Fix: default to 8 images per chunk when max_images is 0 or None,
so chunking always applies for multi-tab dashboards.
Root cause of stuck 'running' runs: the plugin's execute() method
only processed dashboard_ids[0] and returned. The remaining N-1
dashboards were never processed, and _update_run_status was called
after each single dashboard without checking dashboard_count.
Fixes:
- execute() now loops over ALL dashboard_ids and processes each
via _execute_path_a or _execute_path_b
- _update_run_status is called once AFTER the loop finishes
- _update_run_status now checks that len(records) >= dashboard_count
before marking the run as finished (prevents premature completion)
- Per-dashboard _update_run_status calls removed from _execute_path_a
and _execute_path_b (the parent loop owns it now)
- Test updated for new batch return format {dashboards: [...], total: N}
When the backend restarts (deploy, crash, reload), the in-memory
TaskManager loses its queue. ValidationRun records left 'running'
in the DB would hang forever, blocking re-runs.
Fix: during lifespan startup, query all ValidationRun with
status='running' and force-stop them as FAIL with a clear summary.
This prevents the 'already has a running run' error after restarts.
- Extract _optimize_images() helper to eliminate duplicate
optimization code (was duplicated between initial pass and
quality-reduction fallback)
- Move quality-reduction estimate to only apply when NOT chunking
(each chunk fits the image limit by definition)
- Fix _merge_chunk_results to return 'chunk_count' instead of 'chunks'
for consistency with plugin.py
- Simplify plugin.py: analysis.get('chunk_count', 1) instead of
fallback chain
- Document that Kilo API gateway is incompatible with AsyncOpenAI
image format (probe returns 0)
Delete deprecated code that has been fully replaced by v2:
- backend: validation.py routes, validation_run_service.py
- frontend: validation.js API module, validation/ pages
Add binary-search probe endpoint POST /providers/{id}/probe-max-images
that discovers the per-request image limit by sending incrementally
more 1x1 JPEGs via the provider's own API. Result is cached in the
new max_images column on the provider config.
- LLMProviderConfig: add max_images: int | None
- LLMProvider (SQLAlchemy): add max_images column
- Migration ed28d34edde7: clean ADD COLUMN
- LLMProviderService: create/update/set_max_images
- POST /providers/{id}/probe-max-images: binary search + error parsing
- ProviderConfig.svelte: 'Detect' button in edit modal + HelpTooltip
- i18n (en/ru): 11 new keys for probe UI
1. Remove get_config_manager() call from translate_run_websocket
(@app.websocket /ws/translate/run/{run_id}) — was not imported
and unused. Also remove unused from sqlalchemy.orm import Session.
2. Fix prop name mismatch: showBulkReplace -> showPageBulkReplace
in RunTabContent. Use () for reactive two-way binding.
3. Add ('DRAFT') for status prop in RunTabContent.
Backend:
- Add /ws/translate/run/{run_id} WebSocket endpoint in app.py
- Streams structured run status every second (total_records,
successful_records, failed_records, progressPct, batch_count, etc.)
- Auto-detects terminal states (COMPLETED, FAILED, CANCELLED) and closes
- Authenticated via JWT/API key token query param
Frontend:
- Add getTranslateRunWsUrl() URL builder in api.js
- Update translationRunStore to use WebSocket instead of log stream
- WebSocket receives structured status JSON, updates store reactively
- Falls back to polling silently if WebSocket fails
- Keep polling as fallback for insert phase
Root cause: fetch_available_datasources did not return database_id,
so frontend could not auto-set target_database_id when user selected
a datasource. User then had to manually pick a database on Target Config
tab — and could accidentally select the wrong one.
Backend changes:
- Add 'database_id' field to datasources response (from db_info.get('id'))
- Add _database_name tracking in SupersetSqlLabExecutor with getter
- Add database_name + database_backend to TargetSchemaValidationResponse
- Enrich resolve_database_id logging with database_name and all_keys
Frontend changes:
- In selectDatasource(), auto-set targetDatabaseId from ds.database_id
when present, so the correct target DB is used for schema checks.
- Replace fragile substring check (any(kw in backend for kw in ('clickhouse', 'ch')))
with a proper _backend_normalize mapping + exact comparison.
Fixes misdetection of Greenplum (postgresql) and other backends.
- Add 'clickhousedb' to CLICKHOUSE_DIALECTS so SQL generator uses
backtick quoting and correct INSERT strategy for ClickHouse.
- Normalize _extract_dialect to map 'clickhousedb' -> 'clickhouse'
and 'greenplum' -> 'postgresql'.
OpenSSL 3.x ignores intermediate CA certs in -CAfile (verify code 20)
but correctly builds chains with -CApath (verify code 0).
All three clients now use capath:
- service.py: ssl.create_default_context(capath=...)
- _llm_http.py: verify='/etc/ssl/certs/'
- preview_llm_client.py: verify='/etc/ssl/certs/'
check_llm_certs.py rewritten for clarity.
- ADR-0009: comprehensive SSL certificate management strategy
- _get_ssl_verify() returns SSLContext with cafile= instead of bool True
(httpx deprecates verify=<str>, requests needs system CA, not certifi)
- _get_verify() in translate plugin returns system CA path
- All tests updated for SSLContext return type
- All QA findings C1-C3, H1-H4, M1 incorporated into ADR
preview_llm_client.py and _llm_http.py both use requests.post()
without verify=, causing SSLError when corporate CA not in trust store.
Added _get_verify() helper that reads LLM_SSL_VERIFY env var.
Introduce a centralized tool registry system for the assistant to manage
executable operations, permissions, and tool catalogs. This refactors
the assistant dispatch logic from a monolithic approach to a modular,
decorator-based registry.
Key changes:
- Implement `AssistantToolRegistry` to handle tool registration,
permission checks, and safe operation identification.
- Add a suite of new assistant tools: backup, commit, branch creation,
deployment, health summary, environment listing, LLM operations,
maintenance, migration, and dashboard searching.
- Refactor `_dispatch.py` and `_llm_planner.py` to utilize the new
registry for intent resolution and tool catalog building.
- Enhance the LLM planner to support more descriptive clarification
responses in Russian when intents are ambiguous.
- Update the frontend to support the new tool-based workflow, including
improved i18n labels for assistant operations and a new step-by-step
wizard for the migration page.
- Add ADR-0008 to document the architectural decision for the tool
registry.
- LLM_SSL_VERIFY env var to disable SSL verification for corporate proxies
- install_llm_ca_certs() entrypoint function — downloads PEM/DER certs
from LLM_CA_CERT_URLS, converts DER→PEM, installs to system CA store
- _format_connection_error() — detailed exception chain logging
- 7 new unit tests for _get_ssl_verify, _format_connection_error, verify= param
- LLM_CA_CERT_URLS and LLM_SSL_VERIFY in .env.enterprise-clean
- Removed duplicate @RELATION DEPENDS_ON -> [EXT:Library:tenacity]
After configure_logger() runs, logger.isEnabledFor(logging.INFO)
returns False despite level=10 (DEBUG). This is a CPython logging
framework anomaly that makes logger.info() silently drop messages.
Workaround: check isEnabledFor first; if False, write JSON directly
to sys.stderr bypassing the logging system entirely.
Per molecular CoT protocol: REASON and REFLECT are primary reasoning
bonds and must be visible at INFO level in production. DEBUG is
reserved for high-frequency loops.
- reason(): self.debug() → self.info()
- reflect(): self.debug() → self.info()
- explore(): stays at warning() (WARNING level)
- Also removed redundant logger.info() calls in lifespan (now reason()
itself provides visible startup logging)
- Added logger.info() calls before each startup step (encryption key,
Alembic, init_db, admin bootstrap, scheduler, app complete)
- Previously only logger.reason() was used which logs at DEBUG level
and is invisible in production/development at INFO log level
- This fixes 'no backend logs visible' when running via ./run.sh
- Added missing from fastapi import status (global_exception_handler crashed)
- Moved TraceContextMiddleware to outermost (trace_id now available in log_requests)
- Enhanced global_exception_handler with query_params, client host
- Added logger.explore() in WebSocket auth except blocks (was silent pass)
- Fixed middleware registration order
- Added @app.exception_handler(Exception) that logs full traceback via
logger.exception() into superset_tools_app logger (visible in docker logs)
- Previously FastAPI/Starlette wrote unhandled exceptions to uvicorn.error
logger, making 500 errors invisible in docker logs
- Added JSONResponse import for the handler
- Entrypoint: for legacy DBs, add is_admin via raw SQL then stamp head
(instead of running full migration chain which fails on existing tables)
- Fixed admin_api_keys.py Pydantic config (dict → ConfigDict)
- Added tests/test_alembic_migrations.py (PostgreSQL-only integration tests)
- Fixed infinite recursion in _create_table_if_not_exists (QA catch)
- Simplified init_db() to only call Base.metadata.create_all() on all
three engines — no more _ensure_*() inline additive migrations
- run_alembic_migrations() now raises on failure (no safety net to
fall back to)
- All schema changes (add/drop/rename columns, data migrations) must
go through Alembic. create_all() handles new tables only.
@REJECTED _ensure_*() inline migrations removed because they:
- Duplicated Alembic logic without versioning
- Created hidden schema drift (columns existed in DB but had no
corresponding Alembic migration)
- Made audits and fresh DB provisioning unreliable
- Added 4 missing columns to _ensure_translation_jobs_columns() inline
migration: target_language_column, target_source_column,
target_source_language_column, disable_reasoning — these were in the
SQLAlchemy model but had no Alembic migration or _ensure_*() coverage
- Added missing api_key, assistant, clean_release model imports to
alembic/env.py so autogenerate can see all tables
- Added alembic==1.18.4 to requirements.txt (was missing, transitive only)
- Added run_alembic_migrations() to startup_event in app.py — runs
before init_db() to apply all pending migrations
- Combined approach: Alembic for complex migrations first, then inline
_ensure_*() additive safety net in init_db()
- This ensures DB schema is always up to date on container startup
without manual intervention
The inline migration _ensure_roles_is_admin_column() was defined but
never called from init_db(). This caused 'column roles.is_admin does
not exist' errors when logging in against databases with an older
schema. Added the missing call to ensure the column is created during
startup.
Migration now sets is_admin=true for existing Admin roles.
has_permission falls back to role.name == 'Admin' check if
is_admin flag is missing (backward compat for pre-migration roles).
Alembic migration 86c7b1d6a710 adds is_admin BOOLEAN DEFAULT FALSE
to the roles table. Fixes login crash after M-3 change:
psycopg2.errors.UndefinedColumn: column roles.is_admin does not exist
Also adds _ensure_roles_is_admin_column() as an additive migration
safety net for environments that don't run alembic automatically.
To apply: cd backend && source .venv/bin/activate && alembic upgrade head
HSTS (Strict-Transport-Security) is now opt-in via FORCE_HTTPS=true.
Without it enabled, no HSTS header is sent — safe for dev environments
and enterprise deployments behind HTTP-only proxies without HTTPS certs.
When FORCE_HTTPS=true, sends 'max-age=31536000; includeSubDomains'.
Enterprise recommendation: set HSTS at nginx/ingress level instead.
.env.example documents the risk: enabling without certs breaks site access.
TrustedHostMiddleware blocked Vite proxied requests (Host: localhost:5173
vs allowed_hosts list). Replaced with simple HSTSMiddleware that adds
Strict-Transport-Security header without blocking any requests.
Template documents all required env vars:
- AUTH_SECRET_KEY and ENCRYPTION_KEY (secrets)
- DATABASE_URL, AUTH_DATABASE_URL, TASKS_DATABASE_URL (PostgreSQL)
- ALLOWED_ORIGINS, SESSION_SECRET_KEY (optional)
After [SEC:C-4] removed DEV_MODE fallback, the .env must have
AUTH_DATABASE_URL set explicitly or the app crashes at startup.
L-2: HSTS middleware via TrustedHostMiddleware — restricts allowed hosts
to ALLOWED_ORIGINS list, prevents Host header injection.
L-4: AD group name validation — ADGroupMappingCreate.ad_group validated
with regex: DOMAIN\groupname or CN=...,DC=... format. Empty or
invalid characters rejected.
L-5: INITIAL_ADMIN_PASSWORD warning — log warning that env-var password
is visible via /proc to other processes on the same host.
C-2 (CRITICAL): JWT decode_token now validates aud, iss, iat, jti claims.
Tokens without these claims are rejected. Audience 'ss-tools-api' and
issuer 'ss-tools' prevent cross-service token reuse.
H-2 (HIGH): Server-side JWT revocation via token_blacklist table.
- New TokenBlacklist model (SHA-256 hash only, never raw token)
- blacklist_token() on logout — revokes current session
- is_token_blacklisted() check in get_current_user
- Expired entries auto-pruned via _prune_blacklist()
H-3 (HIGH): Encryption key auto-generation removed — crash-early instead.
Previously, ensure_encryption_key() would auto-generate a Fernet key
on first run and write it to .env. This made encrypted data unrecoverable
when the key changed between deployments. Now raises RuntimeError with
a clear command to generate the key explicitly.
Also: update orthogonal security test for crash-early behavior.
