Stop preview/env fallbacks when a configured datasource is gone, skip the
duplicate final insert after streaming, and surface retry/scheduler/LLM
edge cases as FAILED instead of COMPLETED.
- search_dashboards: call /api/dashboards with page_context=other and
page_size=100 so the profile 'My Dashboards Only' filter can no longer
hide the whole catalog; parse available_total/effective_profile_filter
and report hidden-by-filter instead of a false 'no dashboards' answer
- prefetch_dashboards: same full-catalog context; fix dead code where
data=resp.json() sat after return '' inside the error branch, making
every 200 response raise NameError and the prefetch always return ''
- llm-status: ?force=1 bypasses the 30s health cache so the 'Retry now'
button performs a fresh probe instead of re-reading the stale status;
frontend keeps a single retry interval (previously stacked intervals
decayed the countdown faster than 1/s and fired duplicate probes)
- tests: agent tool/prefetch, backend route bypass + force param,
frontend retry/force coverage
- Replace the global env <select> in TopNavbar with an expandable
EnvironmentStatsWidget showing per-env total/mine/published/drafts
and health status (latency, unreachable), preserving env switching.
- Add GET /api/environments/stats: per-env counts (profile-actor matched)
+ lightweight health probe, gathered concurrently with an 8s probe
timeout and a process-local TTL cache (30s, coalescing) so the full
Superset dashboard catalog is not re-fetched on every dropdown open.
- Add available_total to GET /api/dashboards so grids can show how many
dashboards exist when the profile-default filter hides everything.
- Share ProfileFilterBanner across the dashboards hub and validation
task form: 'showing X of Y' reference info + explicit Show all /
Restore filter actions.
- Russian plural forms for dashboard counts (pluralRu helper) and
compact 'Опубл.' label; i18n keys en/ru.
- Ignore :memory:test_* SQLite test artifacts and drop them from the index.
- Tests: env stats endpoint (incl. caching), widget, model fallback,
plural helper, api client, integration.
- ENCRYPTION_KEY: smoke test generates a fresh key inline; templates use a placeholder
- drop RUSAL_ROOT.cer (client-specific public cert) + gitignore it
Remove dead schema left behind by removed features:
- dataset-review family (dataset_review_sessions, dataset_profiles, and
related children) from c3ad0afc — its non-cascading FKs broke environment
deletion with ForeignKeyViolation
- connection_configs from 74e64622
Both are unreachable from the app (no models register them).
Runtime migrations failed with 'Multiple head revisions are present' because
037 T081 (p2q3r4s5t6u7 -> verification_runs.dashboard_id) and a concurrent
session-activity change (a1b2c3d4e5f7) both branched from o1p2q3r4s5t6.
The failed 'upgrade head' left dashboard_id unapplied, causing
'column verification_runs.dashboard_id does not exist' on
GET /verification/history.
Add a no-op merge revision (015281bd7759) collapsing both into a single head
so 'upgrade head' applies the verification_runs.dashboard_id column.
Verified: ScriptDirectory.get_heads() == ['015281bd7759'].
QA review of the 036-041 closure range returned FAIL with 3 criticals, all
confirmed. Fixes:
C1 - breaker dead: on_result=persist_batch is now wired into RunnerPool
(breaker.record() fed per result); added test_breaker_abort_persists_partials
proving CIRCUIT_BREAKER_ABORT reachability + partial persistence.
C2 - index-based result mapping corrupted data under concurrency: results now
map by execution_id to their source item; test uses two distinct payloads
and asserts chart->digest pairing (previously masked by identical fixtures).
C3 - double-acquire of the shared client semaphore (deadlock invariant):
RunnerPool no longer manually acquires the client semaphore; capacity is
enforced by worker count, the client bounds total concurrency.
C4 - duplicated ScenarioGraph.Vlm.Analyze region: outer region renamed
ScenarioGraph.Vlm [TYPE Module].
H1 - _default_submit stub removed: analyze_screenshot requires submit=; no
silent empty-findings fallback.
M1 - test_capture_dispatch.py region closed.
M3 - capture.py raw_sha256 bypass removed: digest always derived from real
capture_bytes (no caller-supplied hash).
Verification: load_testing (77) + scenario (103) = 180 passed; ruff clean;
all region pairs balanced.
Close the 037 pipeline-automation and read-API gaps found in the audit:
deploy/release hooks did not create VerificationRun, and GET endpoints for
history/detail were absent even though 039 UI and client call them.
T080 - _release_routes.py: create_release now fires best-effort
_trigger_release_verification -> VerificationRun with trigger=release_create
(metric+structure); verification scheduling failures never roll back the
release transaction.
T081 - verification.py: add GET /verification/history (dashboard_id +
environment_id filters, newest-first) and GET /verification/{run_id}
(404 RUN_NOT_FOUND); reuse _record_to_response.
- verification_run.py + alembic migration p2q3r4s5t6u7: nullable indexed
dashboard_id populated from structure/visual/metric category_params.
- verification_service.py: _derive_dashboard_id helper.
Verification: release routes (32) + verification API (8) + persistence (21)
= 53 passed; ruff clean for changed code (pre-existing RUF012/UP017 on old
lines left untouched).
Close the 038 MVP runtime gaps found in the audit: VLM analysis previously
returned empty findings with no provider call, and capture registered a
synthetic sha256 derived from run/step ids instead of real image bytes.
T057 - vlm.py: replace _default_submit stub with real submit_screenshot that
resolves a multimodal provider via LLMProviderService (decrypted key,
multimodal-required gate) and calls Plugin.Service.LLMClient.get_json_completion
with the masked screenshot; analyze_screenshot is now async.
T058 - capture.py: dispatch_capture now REQUIRES real capture_bytes/masked_bytes
and computes sha256 from the actual image bytes (synthetic hashes forbidden);
scenario API accepts base64 capture/masked bytes.
T059 - test_scenario_vlm_e2e.py: capture -> VLM -> disposition end-to-end with
real bytes (masked bytes reach the provider; digest matches sha256 of bytes).
Verification: tests/services/dashboard_testing/scenario/ = 103 passed, ruff clean
(existing B008 on pre-existing draft-pack route lines untouched).
start_maintenance accepted any environment_id, creating a stuck PENDING
event that never transitioned for unknown environments. Add synchronous
404 guard (mirrors preview_dashboards), inject config_manager via Depends,
and cover with a regression test proving no event row is created.
Also fix mock_task_manager to await broadcast_maintenance_event (AsyncMock),
aligning the fixture with the production route's awaited call.
Add auto_end flag to maintenance_events: when set with an end_time, a
60s APScheduler scan dispatches the end task automatically. The scan
survives restarts and is de-duped by task_id; end_time alone stays
informational. Includes alembic migration, route/schema wiring, Svelte
checkbox with validation, examples, and backend + frontend tests.
The start endpoint declared 409 in OpenAPI responses but returned the
already_active idempotency hit as a plain 200. Now returns HTTP 409
Conflict with the declared MaintenanceAlreadyActiveResponse body
{maintenance_id, status: 'already_active'}.
Consumers updated to treat 409 already_active as idempotent success:
- bash example: 409 case in api_call
- python example: 409 branch in start_maintenance
- frontend form: info toast instead of error
New test: TestStartIdempotency verifies 409 + body + no new task
dispatched (naive datetimes to match SQLite tz-stripping).
Two defects fixed:
- ensure_banner_chart created an orphan 'Maintenance Banner' markdown chart
(polluted Charts menu and dashboard exports). The banner is now a native
MARKDOWN element in position_json; chart_id is a synthetic layout key.
- insert_banner_markdown_at_top blindly targeted GRID_ID; on ROOT->TABS
dashboards (FI-0085) GRID_ID is orphaned and the banner never rendered.
The layout is now normalized to ROOT->GRID_ID->[ROW-banner, ...] with
recursive parents update, matching the proven-working manual example.
Review-driven hardening:
- liveness check verifies reachability from ROOT (children graph), so
dashboards corrupted by the old bug self-heal on the next start.
- insert removes stale ROW-banner-*/MARKDOWN-banner-* keys (single banner).
- decision memory (@RATIONALE/@REJECTED) added to both modules.
- new ops script scripts/cleanup_maintenance_banner_charts.py (dry-run by
default) deletes already-created bogus banner charts in prod.
sqlparse raises SQLParseError above MAX_GROUPING_TOKENS=10000 tokens
(~25KB of typical SQL). The try/except fallback already handled it, but paid
~1s per oversized SQL for a parse doomed to fail. Add _SQLPARSE_SKIP_THRESHOLD
(30k chars) to bypass sqlparse for oversized text (~15x faster, 1.2s->0.08s for
a 212KB SQL) while keeping literal filtering for SQL under the threshold.
Tests: oversized-SQL skip-threshold behavior.
Discovery of virtual datasets now works, but a runtime blocker remained: any
virtual dataset whose SQL exceeds sqlparse's MAX_GROUPING_TOKENS (10000 tokens)
raised SQLParseError 'Maximum number of tokens exceeded (10000)' from
extract_tables_from_sql_span, which is called unguarded in the scan loop — one
oversized virtual dataset aborted the whole maintenance preview/start.
- extract_tables_from_sql_span now wraps sqlparse.parse + token walk in
try/except and falls back to regex-only extraction (keeping all schema.table
matches) instead of raising, so huge SQL no longer fails the scan.
- Tier-1 virtual filter uses value "" (not None) so the sql is_not_null filter
passes Superset's rison schema instead of always falling back to a full scan.
Tests: huge-SQL fallback (extractor) and huge-virtual-dataset scan resilience
(scanner). ADR-0020 updated with Decision 3.
Virtual (SQL) datasets were never matched, so maintenance discovery returned
0 affected dashboards. Two defects fixed:
- find_affected_dashboards filtered by is_sqllab_view, which is NOT a
filterable column in Superset's dataset list API (absent from search_columns),
so the query was rejected. Now discover virtual datasets via the filterable
sql column: primary server-side 'sql is_not_null' filter with a client-side
non-empty-sql scan as fallback (best-effort vs pagination cap), dedupe by id.
- AsyncAPIClient.request never called raise_for_status(), so rejected filters
(HTTP 400) were returned as bodies without a 'result' key and surfaced as
'Found 0 datasets', dead-coding the filtered->full-scan fallback. request()
now raises on non-2xx via the existing error mapper.
Tests cover both virtual-scan tiers, all fallback paths, the raise behavior,
and an end-to-end match with the real sql_table_extractor on production SQL.
Documented in ADR-0020.
Move hardcoded constants into GlobalSettings and surface them in the
Settings UI: task retention, auth rate limit, assistant history retention,
translate baseline expiry, default environment, and extended logging fields.
- consolidated settings API: new fields in GET/PATCH with re-validation
through GlobalSettings (422 on out-of-range instead of silent persist)
- rate limiter policy read live from settings with 60s cache + lock-free
fast path; cache invalidated centrally in ConfigManager on auth policy
change (covers PATCH /settings/global and /consolidated)
- shared settings_provider.get_global_settings() replaces three copies of
the fallback pattern; scheduler baseline fallback derives from model
default
- remove dead GlobalSettings fields (pagination_limit, ff_dataset_*,
LLM_*_RETENTION_DAYS, GLOBAL_VALIDATION_WORKER_LIMIT, AppAsyncRuntimeConfig)
- SystemSettings blocks save on out-of-range values; LoggingSettings gains
max_bytes/backup_count/agent_view/hide_routine_infra/log_level_for_agents;
EnvironmentsTab gains default environment selector
- tests: rate limiter settings-driven policy, consolidated PATCH 422 paths,
System tab save-blocking UX test
- RBAC: ensure_admin_role() guarantees the Admin role carries is_admin=True
(startup self-heal + create_admin promotion + role-is_admin UI checkbox in
admin/roles); update_role refuses to strip is_admin from the last admin role.
- WS: broadcast_maintenance_event is now awaited (3 sites) so maintenance
events actually reach clients (was an un-awaited coroutine RuntimeWarning).
- Pagination: MAX_PAGINATION_PAGES cap + clear error in fetch_paginated_data
to stop runaway loops on huge environments.
- Discovery: find_affected_dashboards and translate datasource picker filter
datasets/dashboards server-side (table_name/id filters, opr operator per
Superset OpenAPI) instead of full scans that hit the pagination token cap;
fallback to full scan when filters are rejected; virtual-dataset dedupe.
WebSocket endpoints now accept then close with real codes (4001 auth, 4003
permission) so clients detect auth failure via event.code instead of an opaque
403 handshake, ending the infinite reconnect storm. _authenticate_websocket
logs the actual JWT/API-key failure reason. Frontend WS consumers stop on
auth rejection and use capped exponential backoff for transient failures.
async_network.request() routes proxy 502/503/504 (HTML) responses to
NetworkError so migration/maintenance surface a clean 503 instead of a
500 JSON-parse traceback.
