Commit Graph

190 Commits

Author SHA1 Message Date
72fcf698af 033: fix preview_translation route + MultiSelect label regression
1. preview_translation route: missing await on async preview_rows()
   - preview_rows() is async def, called without await
   - returned coroutine object instead of result -> 'coroutine not iterable' error

2. MultiSelect.svelte: opt.label -> opt.name
   - option type is {code, name} but template used {opt.label}
   - rendered empty spans instead of language names
2026-06-05 11:38:35 +03:00
3fc09b086e 032: fix 2 critical QA issues — missing endregion + Tombstone dead helpers
C1: _llm_call.py — added missing #endregion _split_and_retry (violated INV_3)
C2: dashboards/_helpers.py — sync _find_dashboard_id_by_slug and
    _resolve_dashboard_id_from_ref typed Tombstone per INV_6 (dead code,
    callers use _async versions from git/_helpers.py)
2026-06-05 10:32:37 +03:00
4247e7a20e 032: fix final sync→async cascade (dataset_review, parsing, validation)
- _parsing.py: parse_superset_link + _recover_dataset_binding → async
    (await get_dashboard_detail, get_chart)
  - dataset_review/orchestrator.py: start_session + _build_recovery_bootstrap → async
    (await get_dataset_detail, parse_superset_link)
  - _routes.py (dataset_review): await orchestrator.start_session()
  - validation_tasks.py: await parse_superset_link + get_dashboard_detail
2026-06-05 09:55:34 +03:00
c7d8f4431e 032: fix remaining sync→async propagation (17 call sites)
Core fixes:
  - service_datasource.py: fetch_datasource_metadata() → async
  - service.py: create_job(), update_job() → async (callers await)
  - _job_routes.py: await create_job/update_job

Maintenance scanners:
  - _dashboard_scanner.py: 4 functions → async (find_affected, _get_linked,
    _apply_filters, _resolve_title)
  - _chart_manager.py: 3 functions → async
  - _banner_renderer.py: rebuild_banner → async
  - _orchestrators.py: 3 orchestrators → async
  - maintenance_banner.py: await async calls

Migration:
  - dry_run_orchestrator.py: run(), _build_target_signatures() → async
  - risk_assessor.py: build_risks() → async
  - migration.py: await service.run()
  - mapping_service.py: sync_environment() → async

Dead code:
  - _helpers.py: _find_dashboard_id_by_slug marked DEPRECATED
2026-06-05 08:56:37 +03:00
71000db785 032: fix run_translation route handler — sync def → async def (asyncio.create_task needs running loop)
run_translation was def (sync FastAPI handler runs in thread pool with no event
loop), but its body calls asyncio.create_task(_background_execute()) which
requires a running event loop. Changed to async def so FastAPI runs it on the
event loop directly.

Error: 'Run failed: no running event loop'
2026-06-05 08:40:46 +03:00
71156783cf 032: fix get_dashboards_page_async -> get_dashboards_page (method renamed during async migration)
get_dashboards_page_async() no longer exists — the sync/async split was
removed and the method is now simply get_dashboards_page() (already async).
Calls in dashboard slug resolution and git helpers were using the old name.
This caused AttributeError at runtime, making all slug-based dashboard
lookups fail with 'Dashboard not found'.
2026-06-05 08:39:05 +03:00
5eb116509b 032: dead code cleanup — remove sync APIClient, _llm_http, preview_llm_client, fix retry chains 2026-06-05 08:31:18 +03:00
ee9123bcf2 032: deep async propagation — orchestrator, insert, mapper, batch chains
Full async conversion for all sync callers of async SupersetClient methods:

orchestrator_sql.py: generate_and_insert_sql + _resolve_dialect → async
orchestrator_run_completion.py: complete_success → async (calls generate_and_insert_sql)
orchestrator_exec.py: execute_run → async (awaits complete_success)
orchestrator_runner.py: execute_run → async (delegates to engine)
orchestrator.py: execute_run + _generate_and_insert_sql → async
executor.py: _insert_batch_to_target → async (awaits batch insert)
_batch_proc.py: insert_batch_to_target → async
_batch_insert.py: insert_batch_to_target + _resolve_insert_backend + _execute_insert_sql → async
dataset_mapper.py: get_sqllab_mappings + run_mapping → async
  + await get_dataset, update_dataset, execute_and_poll
mapper.py: await on run_mapping + resolve_database_id calls
_run_routes.py: threading.Thread → asyncio.create_task (_background_execute async)
2026-06-05 00:13:01 +03:00
f416583a8c 032: fix async chain for target schema check + mapper + executor
resolve_database_id → async (was sync but called async SupersetClient methods)
  - await client.get_database(db_id)
  - await client.get_databases(...)

validate_target_table_schema → async (calls async resolve_database_id + execute_and_poll)
  - await executor.resolve_database_id(...)
  - await executor.execute_and_poll(...)

Route check_target_schema → await validate_target_table_schema(...)

MapperPlugin.execute → await executor.resolve_database_id(...)
  (execute() was already async def, just missing await)

SupersetSqlLabExecutor.execute_sql → await self.resolve_database_id()
  (was sync call to now-async method)
2026-06-05 00:04:54 +03:00
f6dc7c47e7 032: fix missing awaits in translate datasource endpoints (#5,6,7)
Three sync functions called async SupersetClient methods without await:
  - get_datasource_columns(): get_dataset_detail + get_database (async)
  - fetch_available_datasources(): get_datasets (async)
  - Route handlers: both calls missing await

Converted to async functions + added awaits. Both endpoints now return
proper data instead of coroutine errors.
2026-06-04 23:57:46 +03:00
380f8806fb 032: fix missing await on git_service.get_status() (async -> coroutine)
git_service.get_status() is async def, but was called without await,
returning a coroutine object instead of dict. The coroutine was then
used as a dict value in RepoStatusBatchResponse, causing Pydantic
ValidationError (dict type mismatch).
2026-06-04 23:54:42 +03:00
27bbf057ec 032: T045 — git services async (run_blocking for all blocking ops)
_merge.py partially done. Tests still pending.
2026-06-04 20:36:33 +03:00
846d2cb9a9 032: T019 — remaining route files migrated to async SupersetClient
All routes (assistant, migration, datasets, git) now use AsyncSupersetClient.
_helpers.py sync->async for dashboard ref resolution.
_detail_routes.py import fixed.

Known residual: MigrationDryRunService and IdMappingService still sync.
2026-06-04 20:02:33 +03:00
51afbee470 032: Phase 3 US1 — 13 mixins migrated to async + dashboard routes
T011-T017: All SupersetClient mixins now async.
T018: _detail_routes.py uses registry/AsyncSupersetClient.
T019: Partial — routes/environments, settings, profile, listing async.

RATIONALE: Big-bang merge of sync+AsyncSupersetClient.
REJECTED: dual-stack.

Remaining T019: assistant/*, migration, datasets, git helpers still use sync.
2026-06-04 19:55:43 +03:00
2436b2d320 feat(backend): add is_regex to dictionary API routes
Add is_regex field to list, add, and edit dictionary entry API responses,
and pass is_regex through to DictionaryEntryCRUD methods.
2026-06-04 16:17:36 +03:00
08103871f7 fix(backend): resolve test regressions
- Remove invalid sqlite=True parameter from composite index migration
  (sqlite=True is not a valid op.create_index parameter)
- Fix test_assistant_api assertions for updated response format
- Fix test_git_status_route edge case assertions
- Fix test_audit_service expected value after metric changes
- Fix test_session_repository assertion after store refactor
2026-06-04 16:16:18 +03:00
b823bc559b refactor(backend): split translate run routes into edit/history modules
Extract inline edit, bulk find-replace, and override language endpoints from
_run_routes.py into dedicated _run_edit_routes.py and _run_history_routes.py
modules to reduce module complexity below INV_7 limits.

Changes:
- _run_routes.py now only handles execution, retry, and cancel endpoints
- _run_edit_routes.py (new): inline edit, bulk find-replace, override language
- __init__.py registers the new route modules
- schemas/translate.py: add imports for extracted endpoints
2026-06-04 16:15:48 +03:00
15d47450a3 feat: add deduplicate + metrics to Run tab
Backend:
- deduplicate param in GET /runs/{id}/records — NOT EXISTS subquery
  with (created_at, id) tiebreaker for one row per source_hash
- source_data and source_hash added to records JSON response
- fix missing status import in _run_history_routes.py (NameError)
- fix tab/space mixup in orchestrator_aggregator.py
- remove duplicated @RELATION edges in metrics.py
- fix #region/#endregion style and @PURPOSE→@BRIEF in metrics.py

Frontend:
- RunTabContent: summary metrics bar (fetchJobMetrics) with HelpTooltips
- RunTabContent: trigger_type badge, duration, cache rate in run rows
- TranslationRunResult: deduplicate=true by default, paginated Load more
- TranslationRunResult: per-language token_count and estimated_cost
- TranslationRunResult: collapsible batch breakdown with timing
- TranslationRunResult: Bulk Replace button in header and records table
- TranslationRunResult: source_data key values shown under source text

i18n: all new keys in EN + RU (load_more_records, showing_records,
sum_*, help_sum_*, trigger_*, batch_*, duration_label, cost_label,
cache_rate, load_more_records)
2026-06-04 13:23:10 +03:00
814f2da139 perf(translate): fix slow translation startup — CJK estimation, output budget, provider token config
Root cause: batch sizing underestimated CJK token density (1.5→1.0 chars/token)
and ignored output budget as primary constraint, causing cascading finish_reason=length.

Changes:
- _token_budget.py: CJK_RATIO 1.5→1.0, OTHER_RATIO 2.2→1.8, safety factors 0.75/0.70
- _token_budget.py: new _compute_max_rows_by_output() — output budget is PRIMARY constraint
- _batch_sizer.py: resolve_provider_config() with DB-level context_window/max_output_tokens
- _batch_sizer.py: INPUT_SAFETY_FACTOR applied, max_rows_by_output used as row cap
- _llm_http.py: log actual usage.prompt_tokens/.completion_tokens from provider
- _llm_call.py: retry only missing rows after finish_reason=length (save partial result)
- models/llm.py + schema: provider-level context_window / max_output_tokens (nullable)
- services/llm_provider.py: get_provider_token_config() helper
- Alembic migration: add columns to llm_providers
- Svelte ProviderConfig: collapsible Advanced: Token Limits section
- 12 new tests (token budget, batch sizer, provider config)
- All 492 tests pass
2026-06-03 23:25:08 +03:00
1e6ec34c2b feat(backend): drop dictionary dialect columns, add allowed_languages
- Removes source_dialect and target_dialect from TerminologyDictionary model,
  schemas, routes, helper serialization, and all tests
- Adds allowed_languages config field (BCP-47 language codes) to GlobalSettings
  with validation, consolidated settings response, and API endpoint
- Adds alembic migration f1a2b3c4d5e6 to drop the two columns
2026-06-03 11:48:03 +03:00
f87092c4a7 feat(translate): add cache_hits counter — backend + frontend
Backend:
- TranslationRun model: add cache_hits Integer column (default 0)
- TranslationRunResponse schema: add cache_hits field
- _helpers.py/_run_list_routes.py/orchestrator_aggregator.py: include
  cache_hits in all run API responses
- _batch_proc.py: count pre_rows served from translation cache per
  batch, return cache_hits in batch result
- executor.py: accumulate cache_hits across batches, persist to run
- Alembic migration: dabc9709 — add cache_hits column to translation_runs

Frontend:
- translationRun.svelte.ts: add cacheHits to store state, WS handler,
  polling handler
- TranslationRunProgress.svelte: 5-col stats grid with purple Cache card
- TranslationRunGlobalIndicator.svelte: 5-col stats with Cache
- TranslationRunResult.svelte: 5-col detail stats with Cache card
- History page: cache_hits shown in run list row + detail panel

Visual: cache hits shown in purple alongside green/yellow/red metrics
(total/success/failed/skipped). Visible during run + in history.

Tests: backend 69/69 translate tests , frontend 698/698 tests ,
frontend build 
2026-06-02 11:59:57 +03:00
8219540ade feat(backend): add v2 LLM validation fields to health service
- DashboardHealthItem schema: add run_id, policy_id, execution_path,
  issues_count, timings, token_usage, screenshot_paths, chunk_count,
  dashboard_name fields for v2 LLM validation reporting.
- HealthService: populate v2 fields from validation run records.
- ValidationService: update to support v2 validation run fields.
- validation_tasks route: minor fix for v2 field handling.
- tailwind.config.js: refactor design tokens — add success/warning/info
  palettes, surface/border/text hierarchy, semantic action colors.
- Agent configs: update svelte-coder and semantics-svelte for
  .svelte.ts runes and Screen Model patterns.
2026-06-02 09:54:42 +03:00
28b28338c5 fix(translate): исправление INSERT + выгрузка CSV для неуспешных запусков
**Исправления ошибок:**
- superset_executor: убран несуществующий fallback endpoint /api/v1/sql_lab/execute/
  (правильный URL /api/v1/sqllab/execute/ подтверждён браузерным запросом Superset)
  Ошибка 500 (permission denied) больше не маскируется 404
- orchestrator_run_completion: run.status=FAILED при провале insert
  (раньше всегда ставился COMPLETED). timeout тоже treated as failure
- test_orchestrator: assertion обновлён COMPLETED→FAILED

**Новая функция — выгрузка CSV:**
- GET /api/translate/runs/{run_id}/failed.csv — возвращает CSV с успешно
  переведёнными, но не вставленными записями (status=SUCCESS)
- Колонки: record_id, source_*, target_sql, source_data (JSON), языки, error_message
- Метаданные запуска (run_status, insert_status, run_error) в конце файла
- Кнопки скачивания в 3 местах фронта:
  - История запусков (при status=FAILED)
  - TranslationRunProgress (insert_failed)
  - TranslationRunResult (failed/partial/insert_failed)
- downloadFailedCsv() через fetchApiBlob + Blob URL
2026-06-01 22:38:52 +03:00
7f7a85b2c5 refactor(validation): deduplicate image optimization, fix quality-reduction-before-chunking, unify chunk_count key
- Extract _optimize_images() helper to eliminate duplicate
  optimization code (was duplicated between initial pass and
  quality-reduction fallback)
- Move quality-reduction estimate to only apply when NOT chunking
  (each chunk fits the image limit by definition)
- Fix _merge_chunk_results to return 'chunk_count' instead of 'chunks'
  for consistency with plugin.py
- Simplify plugin.py: analysis.get('chunk_count', 1) instead of
  fallback chain
- Document that Kilo API gateway is incompatible with AsyncOpenAI
  image format (probe returns 0)
2026-05-31 22:57:58 +03:00
d2b53c2a79 feat(validation): v2 LLM dashboard validation — plugin, routes, services
Core implementation of the v2 LLM dashboard validation pipeline:
- LLM plugin with Path A (screenshots) and Path B (logs-only) execution
- Validation task management (CRUD, schedule, run)
- WebSocket task progress with Python 3.13 asyncio fix
- Cross-task runs listing (GET /validation-tasks/runs/all)
- RecordResponse schema for validation records
- JSON prompt helper, per-dashboard status aggregation
- Prompt templates with docs/git-commit/validation presets
- Migration: v2 validation models + description column
- Tests: plugin persistence, prompt templates, batch, payload, url
2026-05-31 22:32:20 +03:00
431330231f cleanup(validation): remove v1 validation routes and frontend pages
Delete deprecated code that has been fully replaced by v2:
- backend: validation.py routes, validation_run_service.py
- frontend: validation.js API module, validation/ pages
2026-05-31 22:32:14 +03:00
5e5f958eaa feat(llm): auto-detect max images per request for LLM providers
Add binary-search probe endpoint POST /providers/{id}/probe-max-images
that discovers the per-request image limit by sending incrementally
more 1x1 JPEGs via the provider's own API. Result is cached in the
new max_images column on the provider config.

- LLMProviderConfig: add max_images: int | None
- LLMProvider (SQLAlchemy): add max_images column
- Migration ed28d34edde7: clean ADD COLUMN
- LLMProviderService: create/update/set_max_images
- POST /providers/{id}/probe-max-images: binary search + error parsing
- ProviderConfig.svelte: 'Detect' button in edit modal + HelpTooltip
- i18n (en/ru): 11 new keys for probe UI
2026-05-31 22:31:36 +03:00
e17f4f64a9 websocket add 2026-05-31 17:17:30 +03:00
5deb01e182 fix 2026-05-29 16:15:41 +03:00
3619a2ae78 feat(assistant): implement tool registry and expand assistant capabilities
Introduce a centralized tool registry system for the assistant to manage
executable operations, permissions, and tool catalogs. This refactors
the assistant dispatch logic from a monolithic approach to a modular,
decorator-based registry.

Key changes:
- Implement `AssistantToolRegistry` to handle tool registration,
  permission checks, and safe operation identification.
- Add a suite of new assistant tools: backup, commit, branch creation,
  deployment, health summary, environment listing, LLM operations,
  maintenance, migration, and dashboard searching.
- Refactor `_dispatch.py` and `_llm_planner.py` to utilize the new
  registry for intent resolution and tool catalog building.
- Enhance the LLM planner to support more descriptive clarification
  responses in Russian when intents are ambiguous.
- Update the frontend to support the new tool-based workflow, including
  improved i18n labels for assistant operations and a new step-by-step
  wizard for the migration page.
- Add ADR-0008 to document the architectural decision for the tool
  registry.
2026-05-27 18:05:28 +03:00
ce49760e5a date fx 2026-05-27 16:05:04 +03:00
21e32ac4cc fix: legacy DB support + QA audit fixes
- Entrypoint: for legacy DBs, add is_admin via raw SQL then stamp head
  (instead of running full migration chain which fails on existing tables)
- Fixed admin_api_keys.py Pydantic config (dict → ConfigDict)
- Added tests/test_alembic_migrations.py (PostgreSQL-only integration tests)
- Fixed infinite recursion in _create_table_if_not_exists (QA catch)
2026-05-26 20:35:36 +03:00
4205618ee6 chore: eliminate all deprecation warnings from tests and linter
Warnings fixed:
- datetime.utcnow() → datetime.now(UTC) across 48+ files (src/ + tests/)
- datetime.utcnow (callback ref) → lambda: datetime.now(UTC) in model fields (18 files)
- Pydantic class Config → model_config = ConfigDict(...) (16 files)
- Pydantic .dict() → .model_dump() (8 files)
- ConfigDict(allow_population_by_field_name=True) → validate_by_name=True
- SQLAlchemy declarative_base() import path updated
- FastAPI on_event → lifespan context manager (app.py)
- Import sorting (ruff I001) auto-fixed across all files
- Fixed broken re-export chains that ruff F401 cleanup broke:
  _validate_bcp47: service.py now imports from dictionary_validation directly
  job_to_response: _job_routes.py and test imports from service_utils directly
  fetch_datasource_metadata: restored re-export in service.py
- Added missing TranslateJobService import in _job_routes.py (was deleted by F401)
- Added ConfigDict(protected_namespaces=()) for DashboardDatasetItem schema field
- pytest.ini: replaced deprecated importmode with asyncio_mode

All 440 tests pass with zero deprecation warnings.
2026-05-26 19:18:28 +03:00
ee08e2f3dd fix 2026-05-26 18:15:37 +03:00
a9a453109c security: critical auth fixes, test migration to SQLite+FK, 44 test fixes
CRITICAL (CVE-class):
- C-4: Remove DEV_MODE fallback with hardcoded postgres:postgres credentials
- C-3: WebSocket endpoints now require JWT or API key token (?token=) auth
- C-1: Superset environment passwords encrypted via Fernet at rest in DB
- H-4: SESSION_SECRET_KEY separated from JWT AUTH_SECRET_KEY
- H-1: Log injection via %s-formatting instead of f-strings
- H-5: X-Trace-ID header validated as UUID4 to prevent trace poisoning

INFRASTRUCTURE:
- New src/core/encryption.py — EncryptionManager extracted from llm_provider
- Test DB: per-module sqlite:///:memory: with PRAGMA foreign_keys=ON
- testcontainers PostgreSQL via TEST_DB=postgres env var
- conftest temp-file global engine (no 10GB shared-cache leak)

TEST FIXES (44 pre-existing → 0):
- test_smoke_plugins: module-level sys.modules mock isolated to per-test fixture
- test_migration_engine: EXT:Python:uuid → uuid syntax fix
- test_dashboards_api: mock env attributes + correct patch target
- test_constants_audit_fixes: sync expected constant names with actual
- test_defensive_guards: patch.object instead of module-level Repo mock
- test_clean_release_cli: removed empty config.json directory
- FK violations: TaskRecord parents in log_persistence, Environment in
  mapping_service, ReleaseCandidate in candidate_manifest_services

ORTHOGONAL TESTS (18 new):
- test_security_orthogonal.py: bcrypt 72-byte limit, unicode, API key
  format invariants, Fernet robustness, encryption key lifecycle,
  log injection protection, JWT edge cases

MODEL FIXES:
- MetricSnapshot.job_id: nullable with SET NULL (was broken FK for
  aggregate prune snapshots, hidden by SQLite)

CLEANUP:
- Removed stale :memory:test_main/test_auth/test_tasks file databases
- Removed duplicate #endregion in encryption_key.py
2026-05-26 14:58:49 +03:00
f49b7d909e feat: GRACE-Poly protocol optimization — ~3200→10 warnings (↓99.7%)
Full optimization cycle:

Protocol (15 files):
- 4-layer SSOT architecture for agent prompts & skills
- Anti-Corruption Protocol consolidated from 5 duplicates
- Tag-to-tier permissiveness matrix (all @tags allowed at all tiers)

Axiom config:
- complexity_rules: all 22+ tags available on C1-C5
- contract_type_overrides: removed (was narrowing per-type)
- 18 new tags added, LAYER enum expanded (Infra, Frontend, Atom, etc.)
- RELATION predicates expanded (USES, CONTAINS, BELONGS_TO, etc.)

Code fixes:
- 2216 @TAG: normalized to @TAG (colon→space)
- 518 [DEF] blocks migrated to #region/#endregion (37 files)
- VERIFIES→BINDS_TO, :Class/:Function suffixes removed, paths→IDs
- 1173-line _external_stubs.py deleted (EXT: handled natively)
- Batch EXT: reference audit (240 targets: 132 external, 99 internal, 9 fix)
- QA regression check: 0 regressions across all checks

Infrastructure:
- DuckDB rebuild stabilized (appender API, INSERT OR IGNORE)
- Anchor regex fix (parent-child BINDS_TO now resolves)
- EXT:*/DTO:/NEED_CONTEXT: regex fixed in validator
- 34MB Doxygen API portal (3194 contract pages)
2026-05-26 11:14:25 +03:00
9ffa8af1dc semantics 2026-05-26 09:30:41 +03:00
320f82ab95 feat(auth): implement API key authentication and management
Introduce a new API key authentication mechanism to support service-to-service
communication (e.g., Airflow, CI/CD) without browser-based JWT login.

- Add `APIKey` model and `APIKeyPrincipal` dependency for RBAC.
- Implement `require_api_key_or_jwt` dependency with environment scoping.
- Add admin CRUD endpoints for API key lifecycle management.
- Add API key management UI in System Settings.
- Update maintenance API to support explicit `environment_id` and API key auth.
- Add i18n support for API keys and connection settings.
- Include Python and Bash usage examples in the `examples/` directory.
- Update technical specifications and documentation.
2026-05-25 11:35:27 +03:00
31680a1bc9 fix(maintenance): auto-expiry + adaptive markdown height + tests
- Auto-expiry: expired events auto-end on GET /events via task manager
- Height: _estimate_markdown_height adapted to unit=8px scale with padding detection
- CRITICAL-1: removed dead import remove_chart_from_position (QA finding)
- CRITICAL-2: added chart alive check in ensure_banner_chart (QA finding)
- CRITICAL-3: fixed test assertions update_markdown_chart → update_banner_on_dashboard
- @POST contract: fixed return range [2,12] → [19,200]
- Tests: 8 new tests (auto-expiry + layout height)
2026-05-25 08:36:33 +03:00
0653a75ee7 fix(db): add ondelete cascade to all FK constraints, fix multimodal flag persistence
- Add ondelete=CASCADE/SET NULL to all environment and translate FK constraints
- Add is_multimodal to GET /api/settings/consolidated response
- Fix toggleActive to not overwrite is_multimodal with false
- New SearchableMultiSelect component for dashboard search with multi-select
- Fix validation task page: task data unwrapping, date formatting, dashboard multi-select
- Fix infinite effect loop on dashboards page via queueMicrotask guard
- 3 new Alembic migrations (merge f0e9d8c7b6a5, translate b0c1d2e3f4a5)
2026-05-22 09:21:24 +03:00
48cc02ea0b feat: LLM Validation section — 9 API endpoints + 3 Svelte pages + Playwright optimization
Backend:
- /api/validation/ CRUD for validation tasks (policies)
- Run history with 6 filters + pagination
- Alembic migrations (provider_id, policy_id)
- 28 pytest tests

Frontend:
- /validation — task list page with status filters + CRUD
- /validation/[id] — task config (Config + History tabs)
- /validation/history — run history with metrics
- Sidebar nav entry under Operations
- i18n en/ru

Playwright (ScreenshotService):
- Removed 25s hardcoded sleeps → conditional waits
- CDP fallback to full_page screenshot
- Total timeout enforcement (120s)
- Debug screenshots → temp dir with cleanup

QA fixes:
- Debug screenshot accumulation in production (tempdir + rmtree)
- Frontend API payload field name mismatch
- History issues field reference fix
- delete_runs scoped by policy_id

Belief protocol:
- @RATIONALE/@REJECTED on 27 C4+ contracts

Closes: VALIDATION-REWORK-2026
2026-05-21 20:29:36 +03:00
36643024cf refactor(connections): remove ConnectionConfig, migrate mapper to SQL Lab
Backend:
- Remove ConnectionConfig model, CRUD routes (connections.py), and tests
- Remove ensure_connection_configs_table from database.py
- Migrate MapperPlugin from psycopg2 direct PostgreSQL to SupersetSqlLabExecutor
- Add DatasetMapper.get_sqllab_mappings() with default information_schema query
- Update MapColumnsRequest: connection_id → database_id + sql_query
- Fix pydantic_settings v2 deprecation (Field(env=...) → validation_alias)
- Clean up app.py, routes/__init__.py, database.py imports

Frontend:
- Remove DatabaseConnectionsTab, ConnectionForm, ConnectionList components
- Remove /settings/connections route page and Navbar link
- Remove connectionService.js, connections i18n files
- Update MapperTool.svelte: postgres → sqllab source
- Update Map Columns modal: database selector + SQL query instead of connection_id
- Clean up settings page tabs, nav links, test mocks
- Clean up i18n keys (settings, nav, mapper)

Tests: 25/25 pass (14 030-feature + 11 route tests)
Build: succeeds
2026-05-21 18:47:41 +03:00
b529e8082a feat(030): Dataset Lifecycle Workspace — Stats Bar, split-view, inline-edit, bulk actions
Backend:
- Add MetricItem, StatsCounts, ColumnDescriptionUpdate, MetricDescriptionUpdate DTOs
- Add metric_count to DatasetItem and DatasetDetailResponse
- Add server-side filtering via ?filter=unmapped|mapped|linked|all
- Add PUT endpoints for column/metric description inline-edit
- Add HTML stripping validation (max 2000 chars, plain text)
- Add WebSocket dataset.updated event on task completion
- RBAC: plugin:migration:WRITE for mutations, READ for reads
- 14 pytest tests (stats, filter, metrics, inline-edit, validation, 502/503)

Frontend:
- Rewrite +page.svelte as split-view orchestrator (387 lines)
- StatsBar.svelte — 4 metric tiles with aria-pressed, server-side filter dispatch
- DatasetList.svelte — card grid with progress bars, checkboxes, search
- DatasetPreview.svelte — presentational detail panel (container fetches)
- ColumnsTable.svelte — inline-edit with type chips, bold/italic fallback
- MetricsTable.svelte — inline-edit with expression hints
- 52 vitest tests across 7 test files
- i18n: 11 EN + 11 RU keys (with_mapping/с_маппингом)

Spec:
- 15 issues resolved (semantic label, filtering, endpoints, conflict, metric_count, RBAC, ID types, ports, i18n, validation, a11y, propagation, resize, perf, ownership)

Status: 14/14 backend tests pass, 52/52 frontend tests pass, build succeeds
Risk: Low — T051 browser validation pending (non-blocking)
2026-05-21 13:58:47 +03:00
68740cd9ed semantic 2026-05-20 23:54:53 +03:00
084f782065 lang detect 2026-05-20 17:15:31 +03:00
09d12b3b68 semantics 2026-05-20 09:59:03 +03:00
b916ef94d5 semantics 2026-05-19 18:36:15 +03:00
6988e63967 chore: commit remaining pre-existing changes
- Agent configs (.opencode/agents/)
- Backend: alembic, routes, app, utils, scripts
- Frontend: package.json, vite, components, e2e infra
- Specs: 028-llm-datasource-supeset updates
- Docker e2e config and Playwright setup
2026-05-17 19:23:07 +03:00
cd868df261 fix(health): suppress 404 when health monitor disabled + fix audit test assertion
- Sidebar.svelte: defer healthStore.refresh() until feature flags confirm
  health_monitor is enabled; skip entirely when disabled
- health.js: remove throw error from catch block — log and return null
  instead, preventing 'Uncaught (in promise)' on expected 404
- test_report_audit_immutability.py: fix mock assertions —
  audit_service uses logger.reason/reflect/explore, not logger.info
- HealthStore and Sidebar now produce zero network noise when
  FEATURES__HEALTH_MONITOR=false
2026-05-17 14:18:02 +03:00
30ba70933d feat(llm): mask API keys in UI responses and prevent masked-key leakage in fetch/test/save payloads
- Add mask_api_key() and is_masked_or_placeholder() to llm_provider service
- Return masked keys in all provider CRUD endpoints
- Reject masked/placeholder keys in fetch_models and test_provider_config
- Show masked key with Change button in ProviderConfig.svelte edit form
- Exclude masked keys from fetch-models, test, and submit payloads on frontend
- Update semantics-core skill with clarified complexity tier rules
- Switch agent modes from subagent to all
2026-05-15 23:34:09 +03:00