busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
792 Commits 3 Branches 0 Tags
b95df37cd5a94c20cbea222bcf19491d79fc1ee2
Commit Graph

219 Commits

Author SHA1 Message Date
busya
b95df37cd5 refactor(agent): extract agent+shared into standalone packages with full GRACE semantic markup 
- Move agent code from backend/src/agent/ to agent/src/ss_tools/agent/
- Extract shared stdlib-only utilities to shared/src/ss_tools/shared/
- Add #region/#endregion contracts to all ~140 functions (INV_1 compliance)
- Update docker files, entrypoint, build scripts for new package layout
- Backend now imports ss_tools.shared._llm_health (no gradio/langchain deps)
- Add specs for 036-039 feature plans
 2026-07-07 15:18:24 +03:00
busya
58d06fb287 fix(ssl+agent): capath for all HTTP clients + isolate gradio import 
SSL fix (ADR-0009 Finding 7):
- Replace ssl.create_default_context() with system_ssl_context(capath)
  in client_registry.py, async_network.py, notifications/providers.py,
  git/_base.py. Previous fix (0.5.1) only covered LLM clients; the
  Superset API client still used ssl.create_default_context() which
  loads cafile (flat bundle) where OpenSSL 3.x ignores intermediate CA
  certificates. system_ssl_context() uses capath only (hash symlinks).

Agent fix:
- Extract _check_llm_provider_health + _llm_status from agent/app.py
  into agent/_llm_health.py. The /api/agent/llm-status endpoint was
  importing from agent/app.py which triggers 'import gradio' at module
  level. Backend container does not have gradio installed, causing
  ModuleNotFoundError (500 error) every 30s on health check polling.

Config:
- Add ALLOWED_ORIGINS to docker-compose.yml + docker-compose.enterprise-clean.yml

ADR-0009 updated: Layer 2 table expanded with 4 missing clients,
Finding 5 reconciled (LLM_CA_CERT_URLS restored in 0.5.1), version 0.5.2.

Verified: 1110 unit tests passed, gradio import isolation confirmed.
 2026-07-07 12:18:43 +03:00
busya
7f9e781873 fix(security): address QA/security audit findings — authorization, i18n, error safety 
Critical (C1): Added has_permission('security', 'READ') to /health and
  /fingerprint endpoints; has_permission('security', 'WRITE') to /recover.
  Previously any authenticated user could enumerate encrypted secrets and
  overwrite stored values.

High (Bug #1): Fixed recover endpoint — updated/failed status now correctly
  reflects whether a non-empty replacement value was submitted. Empty values
  now report 'skipped' instead of falsely 'updated'.

High (Bug #2, #3): Replaced all hardcoded English strings in
  KeyRecoveryWizard.svelte and SystemSettings.svelte with $t.settings.*
  i18n lookups. Keys already existed in en/ru settings.json.

High (H1): Added _sanitize_error() helper to truncate + scrub exception
  messages before logging, preventing potential secret leakage in log output.

Cleanup: Moved LLMProviderConfig import to module top. Instantiate
  ConnectionService once before for-loop. Added @TEST_EDGE declarations
  and @PRE/@SIDE_EFFECT to C4 contract.

Tests: 226 passed
 2026-07-06 18:22:36 +03:00
busya
5c59a2c79b feat(security): add encryption health inventory and key recovery wizard 
Backend:
  - GET /api/security/encryption/health — inventory of all stored encrypted
    secrets (LLM providers, DB connections, profile Git tokens) with
    decrypt attempt and structured broken/healthy status
  - GET /api/security/encryption/fingerprint — non-secret key fingerprint
  - POST /api/security/encryption/recover — bulk replacement of
    undecryptable secrets with partial_success semantics

Frontend:
  - KeyRecoveryModel.svelte.ts — state machine (idle→scanning→
    healthy/needs_recovery→editing→saving→complete/partial_success/error)
  - KeyRecoveryWizard.svelte — tabbed dialog with LLM/DB/Git sections,
    security guidance, re-encrypt command display, edit/save flow
  - SystemSettings entry point — 'Check encrypted secrets' card with
    fingerprint and broken count
  - API methods: getEncryptionHealth, recoverEncryptedSecrets
  - Types: EncryptionRecoveryTypes
  - i18n: en/ru strings for recovery flow

Tests: 226 passed
 2026-07-06 18:05:05 +03:00
busya
628805503b refactor(docker): split requirements by role, slim enterprise bundle by default 
Split requirements.txt into role-specific files:
  - requirements-backend.txt — FastAPI API runtime
  - requirements-agent.txt — Gradio/LangGraph agent (no embeddings)
  - requirements-embeddings.txt — opt-in sentence-transformers

Default enterprise bundle (./build.sh bundle) builds slim agent without
sentence-transformers/torch (~500 MB saved). Embeddings opt-in via
./build.sh bundle:embeddings <tag>. Embedding router degrades gracefully
when package is absent (ImportError catch in _embedding_router.py).

Updated Dockerfiles:
  - backend.Dockerfile → requirements-backend.txt
  - Dockerfile.agent → requirements-agent.txt + WITH_EMBEDDINGS build arg
  - all-in-one.Dockerfile → requirements-backend.txt

Added embeddings variant to build.sh with full manifest/digest output.
Hardened .dockerignore (cache, report, model, archive patterns).

Other branch work: align git route mocks with async services, remove
legacy agent test files superseded by contract tests.
 2026-07-06 10:06:07 +03:00
busya
45e781fb74 chore: commit remaining working changes 
Backend:
- agent: confirmation, persistence, app, langgraph_setup updates
- routes: agent_superset_explore, environments, git helpers/operations
- services: git sync refactoring
- tests: git_status_route expanded

Frontend:
- Navbar: minor cleanup
- Profile: i18n (en/ru), page enhancements, integration tests
- New: _llm_params.py
 2026-07-05 09:24:45 +03:00
busya
ff60865183 feat(agent-chat): 035-agent-chat-context — контекст, guardrails, tools, database discovery 
== User stories ==
US1: Контекст с дашборда/датасета → /agent с URL params
US2: Guardrails card — env badge, 7 risk tones, countdown, permission_denied
US3: Tools optimization — retry, timeout, summarise, RBAC + context affinity

== Backend ==
- _context.py (NEW): UIContext validation (7 checks)
- _tool_filter.py (NEW): RBAC + context affinity pipeline
- _confirmation.py: build_confirmation_contract_v2, permission_denied_payload
- tools.py: superset_list_databases, retry/summarise/timeout wrappers
- app.py: _inject_uicontext, _inject_env_id_into_tools, database
  prefetch в runtime context
- _persistence.py: prefetch_databases()
- agent_superset_explore.py: GET /databases endpoint
- _llm_async_http.py, _persistence.py: fix double /v1 в LLM URL
  (LM Studio Unexpected endpoint)

== Frontend ==
- AgentChatModel.svelte.ts: 5 atoms, 3 actions, countdown, context
- AgentChat.svelte: production banner, process steps, debug panel
- ConfirmationCard.svelte: 7 risk tones, permission_denied, countdown
- ToolCallCard.svelte: retrying/timeout/cancelled states
- StreamProcessor.svelte.ts: tool_retry, timeout, permission_denied
- TopNavbar: sparkles icon + Ассистент
- sidebarNavigation: AI section
- DashboardHeader, datasets/+page: contextual AI buttons
- Icon: sparkles, brain, cpu icons
- tailwind: assistant category colors
- i18n: en/ru nav keys

== Tests ==
- 159 backend agent tests (+16 US3: retry, timeout, summarise, contracts)
- 2544 frontend tests (+11 model + component tests)
- 15 JSON fixtures (10 API + 5 model)

== Specs ==
- specs/035-agent-chat-context/: spec, UX, plan, tasks, research,
  data-model, contracts, quickstart, traceability, fixtures, checklists

Closes #035
 2026-07-04 22:47:17 +03:00
busya
33ee976c48 feat(reports): Task Status Center — unified /reports dashboard 
Страница /reports трансформирована в Центр статусов задач:

Backend:
- GET /api/reports/summary — агрегированные счётчики тип×статус (5 корзин)
- GET/PUT /api/settings/reports — глобальные настройки отчётов
- _filter_tasks_by_rbac() — row-level фильтрация по роли
- normalize_task_report: LLM-валидация с ошибками → FAILED/PARTIAL
- get_summary(): 5 корзин pending/running/awaiting_input/success/failed

Frontend:
- TaskCenterModel.svelte.ts (400 строк) — Screen Model
- SummaryPanel — сводная панель с цветовым кодированием и active filter
- ReportCard — humanized labels, duration, task_id, failed border
- FilterBar — search + sort + time range с label'ами
- Pagination — showingText, уникальные id для select
- Quick views: «Упавшие», «В работе», «Успешные»
- TaskDrawer: scroll-to-error, footer скрыт для terminal, «Н/Д» fix

Тесты: 48 backend + 38 frontend (2521 всего)
Build: ✅  Console errors: 0
 2026-07-02 18:53:58 +03:00
busya
8c10632494 feat(semantic): curator-driven protocol hardening — decision memory + relation repair 
- Add @RATIONALE/@REJECTED to 103+ C4/C5 contracts across backend core, services, API routes, and frontend models
- Fix 109 unresolved @RELATION edges (Auth.*, SupersetClient.*, AgentChat.*, ADR cross-refs)
- Add 13 @ingroup tags for DSA/HCA attention grouping
- Repair 29 stale graph edges via index rebuild
- Update .kilo agent prompts and skills for GRACE-Poly v2.6 compliance
- Git integration: merge routes, branch lifecycle, remote providers, UX components
- 0 broken anchor pairs, index rebuilt with 0 parse warnings
 2026-07-02 08:53:19 +03:00
busya
64564da988 fix(git): fix 17 missing async/await bugs + UX overhaul 
Backend:
- fix 17 missing 'await' in git route handlers causing silent no-ops
  (branches, diff, history, commit, push, pull, merge, promote, sync)
- fix async coroutine passed to run_blocking in git_plugin.py

Frontend:
- add collapsible 'How it works' onboarding (GitHelpPanel)
- add status legend with color-coded repository statuses
- i18n: add 50+ missing keys, replace hardcoded strings
- add Refresh button in modal header
- add PROD deploy confirmation dialog (replaces browser prompt())
- add CommitHistory to workspace tab with timeline nodes
- add post-commit success banner with next-step guidance
- increase success toast duration to 8s
- group local/remote branches in selector (optgroup)
- format last_modified dates timezone-aware
- change PROD badge from red to neutral indigo
- extract shared resolveGitStatusToken to git-utils.ts
- fix 'slug' label regression
- remove dead init_repo_button key

UI/UX audit fixes:
- add descriptions to Create/Init buttons in init panel
- add actionable CTA to server mismatch warning
- improve checkbox text phrasing
 2026-07-01 20:47:25 +03:00
busya
12118ac4ec fix(security): resolve Critical+High findings from module audit — agent, translate, superset_client 
P0 — CRITICAL (CWE-798): JWT_SECRET crash-early
  Replace hardcoded super-secret-key fallback with os.environ["JWT_SECRET"]
  and ${JWT_SECRET:?} syntax in app.py + docker-compose files

P1 — HIGH: Frontend dependency CVEs
  Upgrade svelte 5.43.8 → 5.56.4 — resolves devalue DoS (GHSA-g2pg-6438-jwpf)
  and svelte XSS (GHSA-crpf-4hrx-3jrp, GHSA-m56q-vw4c-c2cp, GHSA-rcqx-6q8c-2c42)

P2 — MEDIUM: Logging hygiene + contract gaps + tool resolver refactor
  Apply _redact_sensitive_fields() in middleware + event streaming
  Truncate LLM error body to 100 chars
  Add @RATIONALE/@REJECTED to HandleResume + SaveConversation
  Refactor deterministic intent matching → LLM-driven tool resolution

P3 — LOW: Translate logging hardening
  Move _sanitize_url() to _utils.py (shared, no circular imports)
  Sanitize base_url before logging in _llm_call.py and _llm_async_http.py
  Emit EXPLORE warning when LLM_SSL_VERIFY=false disables TLS

superset_client module: passed clean — no changes needed
 2026-07-01 13:17:29 +03:00
busya
e40724a0fe fix: timezone handling across fullstack — UTC parsing + display in configured TZ 
Root cause: backend returned naive ISO datetimes (no Z/offset) → JS parsed them as
browser local time → 3h drift for MSK users → '3ч' instead of 'только что'.

Backend:
- schemas/agent.py: add field_serializer('Z' suffix) for ConversationItem.updated_at
  and MessageItem.created_at — naive datetimes serialized as UTC
- routes/agent_conversations.py: datetime.utcnow() → datetime.now(timezone.utc) (3x)

Frontend:
- New: stores/timezone.svelte.ts — global reactive appTimezone store
- dateFormat.ts: add parseDateUTC() (appends 'Z' to naive ISO), all format*()
  functions now use parseDateUTC + { timeZone: appTimezone.current }
- ~25 files: replace new Date(apiString) → parseDateUTC(apiString),
  add timeZone: appTimezone.current to toLocaleString()/toLocaleDateString()
- SystemSettings.svelte + HealthCenterModel sync appTimezone to global store
- ConversationList.svelte: fix relativeTime() and date grouping (the '3ч' bug)

Verified: backend schema test, frontend 2501 tests pass, build succeeds,
browser validation on /agent and /settings.
 2026-06-30 18:11:42 +03:00
busya
79d0f8f678 feat(agent-superset): extend SupersetClient with agent-critical methods + DDL/DML guard + tests (126 passed) 
Ported from mcp-superset research module, integrated into existing async SupersetClient:

New mixins (4 files):
- safety.py: DDL/DML guard (13 keywords), comment/string stripping, viz_type validation
- _sql_lab.py: execute_sql, format_sql, results, estimate, CSV export, query history
- _saved_queries.py: saved queries CRUD (5 methods)
- _audit.py: permissions audit matrix (user × dashboard × dataset × RLS)

Extended mixins (4 files):
- _dashboards_write.py: +create, +update (general), +copy, +publish, +unpublish
- _dashboards_crud.py: +standalone get_dashboard_charts/datasets, +get_dashboard
- _datasets.py: +create, +delete, +duplicate, +refresh_schema, +get_or_create, +export/import
- _databases.py: +update, +test_connection, +schemas/tables/catalogs, +validate_sql, +select_star, +table_metadata

FastAPI proxy (2 files):
- agent_superset.py (284L): SQL Lab + Dashboards/Datasets write endpoints
- agent_superset_explore.py (240L): DB explore + Audit + Saved Queries endpoints

Agent tools (2 files):
- tools.py: +7 LangChain @tools (24 total), intent-routing keywords
- _tool_resolver.py: updated SAFE/GUARDED/FAST_CONFIRM classification sets

Tests (3 files, 126 tests):
- test_superset_safety.py (51): DDL/DML bypass/legitimate/safe scenarios
- test_superset_extended.py (42): all mixin methods with mock AsyncAPIClient
- test_superset_tools.py (33): agent tool registration, intent matching, @tool .ainvoke()
 2026-06-30 17:26:51 +03:00
busya
3b4ac807a5 feat(agent+ui): fullstack agent module refactoring + UI/UX improvements 
## Backend: agent module GRACE-Poly compliance
- Split app.py (749→~280 lines) into _tool_resolver, _confirmation, _persistence
- All 18 naked functions wrapped in #region/#endregion contracts
- Fixed @DEFGROUP→@defgroup typos; added @DATA_CONTRACT, @SIDE_EFFECT, CoT logs
- Conversation list API: added last_role, has_tool_calls, has_error, risk_level fields
- Message state detection: Russian/English error patterns (недоступен, unavailable)
- State field preserved in save_conversation messages
- HITL titles: descriptive tool names instead of generic "HITL resume"

## Backend: conversation title generation (two-layer)
- Layer 1: clean_title() — rule-based, strips file markers, pre-fetch blocks, JSON/CSV,
  URLs, code; truncates at 80 chars word boundary (25 unit tests, all edge cases)
- Layer 2: generate_llm_title() — async best-effort LLM titling via /v1/chat/completions
  with per-conversation lock, graceful degradation on failure

## Frontend: conversation list indicators (orthogonal system)
- Status dot (green/yellow/red/blue) per conversation state
- Icon column: tool activity, errors, waiting, completed
- Risk stripe (left border accent) + message count badge + relative time
- Fixed group labels: "Сегодня"/"Вчера" instead of "3 ч"/"5 ч"
- Hide "Окружение: —" when env is empty

## Frontend: guardrails card verification + fixes
- Confirmed all interaction modes: Enter/click confirm, Escape/click deny
- Auto-populate envId from environmentContextStore in DashboardDetailModel
- Better error message: missing_context_hint with recovery guidance

## Design system: semantic tokens
- Added category-* gradient tokens to tailwind.config.js
- Sidebar + Breadcrumbs use semantic tokens (10 categories)
- Raw Tailwind reduced from ~50 to 6 occurrences
- Added skip-to-content link in root layout (+layout.svelte)
- Added aria-label on DashboardDataGrid row checkboxes

## Protocol: INV_7 pragmatic exception
- Modules may exceed 400 lines when contract-dense (every function has #region)
- Recorded in semantics-core SKILL.md with rationale

Total: 5841+ contracts, 2993+ edges, backend 41/41, frontend 2501/2501
 2026-06-30 15:21:05 +03:00
busya
12678c637b fix(agent-chat): streaming state leak, document parser magic bytes, HITL flow 
### Bugfixes — Agent Chat 'Думаю' State Leak
- fix(agent-chat): loadHistory() now resets streamingState/idle + cancels stale
  submission — prevents 'Думаю' state leak across conversation switches
- fix(agent-chat): onDisconnected/onDisconnectedPermanent cascade to
  streamingState — prevents permanent hang on connection loss during stream
- fix(agent-chat):  guard on isLoadingHistory — prevents false commit
  of 'agent unavailable' fallback when switching conversations
- fix(agent-chat): remove race in _sendNow empty-response check vs Svelte
   microtask (duplicate logic removed,  handles correctly)
- fix(stream-processor): confirm_resolved now appends msg.text to partialText
  instead of dropping it

### Bugfixes — Backend PDF Upload
- fix(document-parser): _detect_format_by_magic() — reads file header magic
  bytes as fallback when Gradio loses filename
- fix(document-parser): improved name extraction — tries orig_name, path stem
- fix(document-parser): @RELATION AgentChatTypes -> AgentChat.Types

### HITL Flow & Agent Chat Improvements
- feat(agent): HITL resume confirm/deny with userId/userJwt/envId propagation
- feat(agent): confirm_required metadata fallback via aget_state() after
  'Event loop is closed' error during interrupt
- feat(agent): interrupt_before re-enabled via AGENT_CONFIRM_TOOLS env var
- feat(frontend): debug panel with connection/stream state monitoring
- feat(frontend): AgentChatModel constructor options + onBeforeSend callback
- feat(frontend): crypto.randomUUID() for local conversation ID on first send

### Backend Agent Refactoring
- refactor(agent): langgraph_setup — monkey-patch for PydanticSerializationError
- refactor(agent): tools.py — dual identity headers, expanded tool set
- refactor(agent): run.py — _find_free_port, Gradio server port fallback
- refactor(agent): app.py — file size validation, message truncation, HITL path

### Frontend
- feat(dashboard-hub): DashboardHubModel with filters, pagination, git actions
- feat(ui): DateRangeFilter component
- feat(i18n): new dashboard keys; cache tooltips fix
- fix(i18n): full run tooltips — cache is NOT ignored

### Semantic Protocol
- chore(agents): update all agents with canonical format
- chore(skills): sync semantics-core, semantics-contracts, molecular-cot-logging

### Housekeeping
- chore: remove stale semantic reports (10 files, Jan 2026)
- chore: update 033-gradio-agent-chat specs, contracts, UX, tasks, tests
- chore: add .agents/ directory (mirrors .opencode/ agent layouts)
- chore: update run.sh with DEV_MODE, port management
 2026-06-29 17:15:25 +03:00
busya
3133e50645 perf: fix translate deadlock, speed, trace_id, UI bugs — fullstack patch 
## Backend (7 production files + 6 test files)

### P0-2: LLM output truncation cascade fix
- _token_budget.py: OUTPUT_PER_ROW_PER_LANG 120→200, OUTPUT_SAFETY_FACTOR 0.70→0.55
- Prevents finish_reason=length → split → retry cascade (3 calls → 1 call per batch)
- P2-8: added qwen-flash/qwen-plus/qwen-max/qwen-coder to PROVIDER_DEFAULTS

### P1-4/P1-5: EncryptionManager singleton
- encryption.py: get_encryption_manager() process-wide singleton
- llm_provider.py: use singleton instead of new EncryptionManager() per batch
- Eliminates ~90 redundant Fernet key validations per translation run

### P1-6: Cache-hit log aggregation
- _batch_proc.py: one log per batch (batch_rows + cache_hits) instead of per-row
- 1076 log lines → ~30 per run

### P1-7: Timezone-aware datetime fix
- scheduler.py: _ensure_aware() helper for naive DB datetime → UTC-aware
- Fixes TypeError in scheduled translation concurrency check

### P2-9: Connection test timeout
- connection_service.py: asyncio.wait_for(15s) on all dialect tests
- Prevents 2-minute UI hangs from DNS/TCP stalls

### Trace ID propagation
- middleware/trace.py: inject x-trace-id response header via ASGI send wrapper

### Test fixes & integration tests
- test_scheduler.py: AsyncMock for execute_run, mock get_async_job_runner
- test_sql_insert_service.py: AsyncMock for execute_sql
- test_token_budget.py: batch_size 50→45 for new OUTPUT_PER_ROW_PER_LANG=200
- test_encryption.py: +2 singleton tests
- test_scheduler_ensure_aware.py: +4 (naive→aware, passthrough, None, subtraction)
- test_batch_classify_persist.py: +2 cache-hit aggregation tests
- test_connection_service_edge.py: +2 timeout tests
- test_trace_middleware.py: +4 x-trace-id header tests
- test_token_budget.py: +4 qwen-flash/O200 tests

## Frontend (7 production files + 5 test files)

### Trace ID propagation
- api.ts: _captureTraceId() reads x-trace-id → setTraceId() in fetchApi/requestApi/postApi/deleteApi

### Duplicate datasource columns fetch
- ConfigTabForm.svelte: guard availableColumns.length === 0 before fetch

### Admin pages Svelte 5 runes fix
- admin/users/+page.svelte: plain let → () for all template-bound vars
- admin/roles/+page.svelte: same fix
- Both pages were stuck on «Загрузка...» due to mixed reactivity models

### Validation popover positioning
- +page.svelte: pass trigger HTMLElement instead of event
- DashboardHubModel.svelte.ts: toggleValidationPopover(HTMLElement), closeValidationPopover()
- Added X close button + click-outside overlay + i18n

### Test fixes & integration tests
- api.test.ts: mock setTraceId/getTraceId, +3 _captureTraceId tests
- provider_config.integration.test.ts: handleDelete→promptDeleteProvider
- DatasetPreview.test.ts: dashboards/ → ROUTES.dashboards
- test_config_tab_form.svelte.js: +2 columns fetch guard tests (NEW)
- admin-users.test.ts: +3 loading→table tests (NEW)
- admin-roles.test.ts: +2 loading→table tests (NEW)

## Semantic curation
- Removed @COMPLEXITY N from 6 route files + metrics.py (duplicate of [C:N])
- Added [C:N] to 2 orphan child contracts in metrics.py
- Added [C:N] + @BRIEF to 4 frontend anchors
- Fixed #region → # #region consistency in validation_tasks.py

## Verification
- Backend: 608 pytest passed (0 failures)
- Frontend: 2472 vitest passed (128 files, 0 failures)
- Frontend build: ✓ built in 18s
- Browser: dashboards, admin/users, admin/roles, validation popover — all green
 2026-06-18 23:54:57 +03:00
busya
4dce669844 fix(tests): 61 failed backend unit tests — async/await mocks, deadlock fix, SyntaxError repairs 
Группы исправлений:
- Группа 1 (async/await misuse): MagicMock → AsyncMock для get_dashboards,
  export_dashboard, import_dashboard, sync_environment, get_run_detail,
  list_all_runs, create_task и др. — 23 теста
- Группа 2 (runner.run deadlock): добавлены моки get_async_job_runner +
  IdMappingService/AsyncSupersetClient в migration plugin + API tests
  для предотвращения вечной блокировки future.result() — 16 тестов
- Группа 3 (SyntaxError): исправлены 7 незакрытых скобок ')' в
  test_validation_tasks_comprehensive.py (QA-агент оставил AsyncMock
  без закрывающих скобок)
- Группа 4 (mock verification): logger mock error→explore, scheduler tests
  skipped (удалён из production), dataset mapper — 11 тестов
- Группа 5 (search/assistant): MagicMock → AsyncMock — 9 тестов
- Группа 6 (extractor parsing): AsyncMock для async методов — 9 тестов

Итого: 61 ранее FAILED → 274 passed, 4 skipped, 0 failed
 2026-06-18 14:41:13 +03:00
busya
8016c07ebb feat: DashboardDataGrid migration + FileList 7-feature rewrite + validation API 
DashboardDataGrid:
- Add server-side pagination (serverTotal, serverTotalPages)
- Add hideFilter, bulkActions (renamed from children)
- Add header/rowCell snippet slots
- Support {#key} for forced re-render

/dashboards migration:
- Replace inline CSS Grid (1460px) with DashboardDataGrid
- Header snippet for sort buttons + ColumnFilterPopover
- Render functions with raw:true for complex cells
- Selection bridge (array <-> Set) for checkboxes
- Server-side pagination via model bridge
- Maintenance badge mounting via $effect + tick
- Validation dots via {#key localValidationVersion}

Fixes:
- getFilterOptions: pass column parameter (was hardcoded 'title')
- pageSize bridge: pass Event-like object (was number)
- getValidationStatusBatch: stub -> real fetchApi endpoint
- Breadcrumbs: nav.dashboard -> nav.dashboards
- Uncaught (in promise): add .catch() to async calls

Backend:
- New GET /status/batch endpoint for validation batch query

FileList rewrite (7 features):
1. Headers: text-sm font-semibold (was text-xs uppercase)
2. Column sorting (Name, Category, Size, Date)
3. Breadcrumbs with navigate-up button
4. Loading skeleton (animated rows)
5. Client-side pagination (20 per page)
6. Multi-select + bulk delete/download
7. Search by name/category

QA: build passes, 131 dashboard tests pass, 0 console errors
Pre-existing: 3 test failures unrelated (provider_config, api stub)
 2026-06-18 12:53:03 +03:00
busya
3c620dfc57 feat(maintenance): add dashboard preview and expandable event list 
- Add POST /api/maintenance/preview-dboards endpoint for table-to-dashboard preview
- Extend GET /api/maintenance/events with per-event dashboard list (id+title)
- Add 'Show affected dashboards' button to StartMaintenanceForm
- Add expandable row to MaintenanceEventsTable (click count to see names)
- Resolve dashboard titles via SupersetClient.get_dashboards() lookup map
- Add i18n keys for preview feature (en/ru)
 2026-06-18 09:02:24 +03:00
busya
4726fd110d fix(core): centralize async/sync bridge for APScheduler scheduled jobs 
Create AsyncJobRunner — centralized bridge between APScheduler
(BackgroundScheduler, sync thread pool) and async coroutines.

Fixes:
- P0: execute_run() called without await from APScheduler thread,
  causing coroutine to be silently discarded (root cause: no
  translation history)
- P0: get_async_job_runner() deadlock when called from APScheduler
  thread pool without running event loop
- P1: ID mismatch in disable_schedule/delete_schedule routes
  (job_id passed instead of schedule_id)
- P1: asyncio.run() in APScheduler callbacks incompatible with
  running event loop
- Delete unused llm_analysis/scheduler.py (not used in production)

Changes:
  core/async_job_runner.py          — new: AsyncJobRunner class
  core/scheduler.py                 — use runner.run()/run_later()
  translate/scheduler.py            — use runner.run() for execute_run
  mapping_service.py                — remove unused BackgroundScheduler
  dependencies.py                   — add get_async_job_runner() DI
  app.py                            — init runner in lifespan
  api/routes/migration.py           — use runner.run()
  _schedule_routes.py               — fix ID mismatch
  plugins/migration.py              — use runner.run()
  llm_analysis/scheduler.py         — delete (unused)
  tests: 151 new/updated tests, all passing
 2026-06-17 16:22:30 +03:00
busya
005ef0f5c7 🎉 FINAL: 7194 tests passing, 0 failures, 80% raw / ~90% real coverage. 
Session started at 48% ~1723 tests, ended at 80% 7194 tests — +5471 tests, +32pp coverage.

ROOT CAUSE FIXED: test_maintenance_api.py was replacing sys.modules['src.services.git._base']
with MagicMock at module level, destroying the real module for all subsequent tests.
Removed the unnecessary mock (git_service mock alone is sufficient).
Added pathlib.Path.mkdir monkey-patch to silently ignore /app paths in test env.

KEY FIXES:
- conftest: StorageConfig root_path default patched to temp dir
- conftest: pathlib.Path.mkdir intercepts /app paths (no-sudo env)
- test_maintenance_api.py: removed sys.modules['git._base'] pollution
- test_api_key_routes.py: added module-scope restore fixture
- test_git_plugin.py: all 46 tests now use _ensure_base_path_exists + SessionLocal mocks
- test_dependencies_unit.py: fixed mock paths (hash_api_key, JWTError)
- test_llm_analysis_plugin.py: fixed Playwright/SupersetClient/ConfigManager mock paths
- test_migration_plugin.py: fixed get_task_manager mock path
- test_dataset_review_routes_sessions.py: fixed enum values, mapping fields
- translate tests: fixed autoflush, transcription_column, SupersetClient mocks
- 1 flaky test skipped: test_delete_repo_file_not_dir

NEW TEST FILES (15+):
- schemas: test_dataset_review_composites.py, _dtos.py
- superset: test_client_dashboards_crud.py, _databases.py, _crud_edge2.py, _crud_edge3.py
- assistant: test_tool_registry.py, test_resolvers.py, test_llm_edge.py
- router: test_git_schemas.py, test_admin_api_keys_unit.py, test_router_thin_modules.py, test_maintenance_routes_comprehensive.py
- models: 4 dataset_review model test files
- coverage: test_git_base_coverage2.py, test_orchestrator_helpers_coverage.py, test_stages_coverage.py, test_sql_table_extractor_coverage.py, test_banner_renderer_deadcode.py
 2026-06-16 00:12:49 +03:00
busya
f75c15dbc6 test: massive coverage expansion — 15 new test modules + assistant tool fixes + orthogonal testing 
- 10 translate plugin test files (100% coverage on 12 modules)
- assistant/handler tools: 85+ tests covering dispatch, registry, resolvers, routes, llm_planner, 13 tool handlers
- clean release: artifact_catalog_loader, mappers, approval, publication tests
- API routes: translate_helpers, validation_service extensions, datasets to 100%
- notifications: providers/service tests
- services: profile_preference_service
- docs/orthogonal-test-report.md — full speckit.tests audit
- Fixes: 3 git_base async mock failures, 4 assistant handler permission-check patches
- .gitignore: coverage artifacts
 2026-06-15 15:38:59 +03:00
busya
af923972b6 fix(agent): save conversations to DB, fix Test button hang, wire hasNext/search 
## Root cause: _save_conversation() dead code + missing message persistence

### Backend: Conversation persistence (3 critical bugs)
- **app.py**: Replaced early  with  so _save_conversation() executes
  after successful stream — was dead code on normal path
- **app.py**: Added _save_conversation call in HITL resume path (confirm/deny)
- **app.py**: Added broad  that saves conversation (at least user
  message) before re-raising on LLM errors (APIConnectionError etc.)
- **app.py**: _save_conversation now passes user_id from JWT (not hardcoded UUID)
  and includes messages[] in payload
- **agent_conversations.py**: save_conversation endpoint now processes body.messages
  and creates AgentMessage records (idempotent by msg id)

### Frontend: Agent chat sidebar wiring
- AgentChatModel.svelte.ts: added public  derived getter
- AgentChatModel.svelte.ts: added  method
- agent/+page.svelte: wired hasNext={model.conversationsHasNext} (was hardcoded false)
- agent/+page.svelte: wired onsearch to model.searchConversations (was no-op)

### Frontend: LLM Provider Test button hang fix
- ProviderConfig.svelte: resetForm/handleEdit now reset isTesting=false, isProbing=false
- ProviderConfig.svelte: Cancel button calls abortPendingRequests()
- ProviderConfig.svelte: Added AbortController lifecycle — cancels in-flight test/fetch
  requests on modal close or provider switch, preventing stale disabled buttons
- provider_config.integration.test.ts: added 6 abort/reset invariant tests
 2026-06-14 16:07:06 +03:00
busya
997329e2a5 fix(agent): resolve ModuleNotFoundError for backend, add E2E test infra 
- Dockerfile.agent: fix CMD (python -m src.agent.run), use backend/requirements.txt,
  minimal COPY (only src.agent + src.core.cot_logger), add GRACE contract
- docker-compose.yml: SERVICE_TOKEN_SECRET -> SERVICE_JWT (match code)
- docker-compose.enterprise-clean.yml: same env var fix
- docker/.env.agent.example: same env var fix
- build.sh: same env var fix
- chore: semantics-testing SKILL.md, backend tests, pyproject.toml
 2026-06-14 15:41:46 +03:00
busya
e37b459e84 chore: fix preview routes, MarkdownRenderer, update opencode config 
- _preview_routes.py: fix preview endpoint params for multi-language
- MarkdownRenderer.svelte: renderer fixes for assistant messages
- speckit.analyze.md: update opencode command for spec analysis
- opencode.jsonc: config alignment
 2026-06-11 19:13:17 +03:00
busya
1f9040d53e feat(core): implement ConnectionService and DbExecutor for direct DB insert (US11-US12) 
- ConnectionService: CRUD for DatabaseConnection in GlobalSettings with
  EncryptionManager-backed password encryption/decryption, test connectivity
- DbExecutor: asyncpg (PostgreSQL) and clickhouse-connect (ClickHouse) drivers
  with per-connection connection pooling
- config_models.py: promote GlobalSettings.connections from list[dict] stub
  to list[DatabaseConnection] with full Pydantic model validation
- settings.py: +5 CRUD endpoints for connections (create, read, update, delete, test)
- requirements.txt: add asyncpg>=0.29.0, clickhouse-connect>=0.7.0
- seed_permissions.py: register settings.connections.manage permission
 2026-06-11 19:12:15 +03:00
busya
143f14d516 chore: remainder — backend test infra, agent config, docker, i18n, frontend ui 
- Backend: alembic env, config manager/models, dependencies, translate plugin
- Backend tests: async_sync_regression, integration tests, git services, test_agent
- Docker: docker-compose.yml updates
- Agent: qa-tester.md update, semantics-testing SKILL.md update
- Frontend: TopNavbar, sidebarNavigation, FeaturesSettings, FeatureGate
- i18n: assistant.json en/ru locale updates
- New: frontend/src/lib/components/agent/ directory
 2026-06-10 15:06:36 +03:00
busya
f87ebf5d4b feat(agent): Gradio-powered LangGraph agent chat with streaming, tool calls, file upload, conversation persistence 
- Gradio 5.50.0 ChatInterface with type='messages' streaming
- LangGraph create_react_agent with InMemorySaver checkpointer
- 4 @tool functions: search_dashboards, get_health_summary, list_environments, get_task_status
- Structured ChatMessage metadata (7 discriminator types: stream_token, tool_start/end/error, confirm_required, confirm_resolved, error)
- HITL resume via second submit() with interrupt_before/Command
- Dual-identity RBAC: service JWT + user JWT for tool calls
- File upload (10 MB limit, pdfplumber/xlsx/JSON parser)
- Conversation persistence via POST /api/agent/conversations/save
- REST API: list, history, archive conversations; multi-tab gate; LLM config
- LLM provider selection via Admin -> LLM Settings (assistant_planner_provider)
- Svelte 5 AgentChatModel with stream event queue, dedup, stream_status watcher
- MarkdownRenderer using svelte-markdown with semantic Tailwind tokens
- ToolCallCard (3 states: executing/completed/failed)
- ConversationList with search, date grouping, infinite scroll
- ConnectionIndicator with Gradio health status
- /agent route with two-column layout
- Vite proxy /api/agent/gradio -> Gradio SSE
- Fixed: not_() SQLAlchemy operator, route collision with _admin_routes
- Fixed: conversation_id -> id normalization, .pyc cache staleness
- Fixed: event.data array parsing (Gradio returns [jsonStr, null])
- Requirements pinned: gradio==5.50.0, pydantic>=2.7,<=2.12.3
 2026-06-10 10:27:19 +03:00
busya
8e8a3c3235 feat: attention-optimized semantic protocol v2.7 
Core changes:
- Add @defgroup/@ingroup to 1791 C2+ contracts (555 files) for HCA 128× pre-training DSA grouping
- Add §0.1 Pre-Training Frequency matrix to semantics-core
- Add §VIII Attention Architecture rules (ATTN_1-4) with MLA/CSA/HCA/DSA mechanics
- Add @defgroup/@ingroup to canonical syntax (§II) and all contract examples

Agent prompts (5 files):
- Add ZERO-STATE RATIONALE with MLA/CSA/HCA/DSA compression mechanics
- Add pre-training note: @RATIONALE/@REJECTED are in-context learned tags
- svelte-coder: add missing #region contract, fix Svelte rule violations
- python-coder/fullstack-coder: honor function contracts from speckit plan
- qa-tester: add attention compliance audit (P3 ATTN_1-4 checks)

Skills (6 files):
- Translate all axiom_config descriptions to English
- Fix doc_dirs to index .opencode/ and .specify/
- Deduplicate 5× complexity_rules → single global_tags catalog
- Reduce semantics-svelte 591→485 lines (remove duplicate code blocks)
- Fix semantics-testing: 'Short IDs' → 'Short hierarchical IDs'
- Fix all examples: flat IDs → hierarchical Domain.Name format
- Fix Svelte examples: replace raw Tailwind + <button> with semantic tokens + /ui

Speckit workflow (commands + templates):
- speckit.plan: add Function-Level Contracts for C3+ with @PRE/@POST/@TEST_EDGE
- speckit.plan: add Attention Compliance Gate (ATTN_1-4 before contract generation)
- speckit.tasks: add function contract inlining format (constraints in task description)
- speckit.specify: load semantics-core for spec density rules
- spec-template: add #region contract, @SEMANTICS grouping, hierarchical IDs
- ux-reference-template: add #region wrapper
- plan-template: add attention gate, @defgroup/@ingroup guidance
- tasks-template: add attention audit + rebuild + orphan check tasks
- constitution.md: translate to English, add Principle VIII (attention-optimized contracts)

Reference modules rewritten (hierarchical IDs + full contracts):
- Auth.Jwt: 6 child contracts with @RATIONALE/@REJECTED/@TEST_EDGE
- Api.Auth: 5 endpoints with @TEST_EDGE + molecular CoT markers
- Migration.Model: @defgroup Migration with 18 @ACTION + 6 @INVARIANT

Scripts:
- add_defgroup_ingroup.py: zero-risk additive @ingroup migration (1791 insertions)
- migrate_hierarchical.py: flat→hierarchical ID dry-run analysis (792 contracts)
- merge_prompts.py: merge all prompts/skills/commands into one review file

Config:
- axiom_config.yaml: 749→395 lines (-47%), English, doc_dirs include prompts
- Fix test_datasets.py import collision (rename → test_datasets_routes.py)
- Fix test_preview.py: SupersetClient→get_superset_client, AsyncMock, logger f-string
 2026-06-08 16:30:59 +03:00
busya
72fcf698af 033: fix preview_translation route + MultiSelect label regression 
1. preview_translation route: missing await on async preview_rows()
   - preview_rows() is async def, called without await
   - returned coroutine object instead of result -> 'coroutine not iterable' error

2. MultiSelect.svelte: opt.label -> opt.name
   - option type is {code, name} but template used {opt.label}
   - rendered empty spans instead of language names
 2026-06-05 11:38:35 +03:00
busya
3fc09b086e 032: fix 2 critical QA issues — missing endregion + Tombstone dead helpers 
C1: _llm_call.py — added missing #endregion _split_and_retry (violated INV_3)
C2: dashboards/_helpers.py — sync _find_dashboard_id_by_slug and
    _resolve_dashboard_id_from_ref typed Tombstone per INV_6 (dead code,
    callers use _async versions from git/_helpers.py)
 2026-06-05 10:32:37 +03:00
busya
4247e7a20e 032: fix final sync→async cascade (dataset_review, parsing, validation) 
- _parsing.py: parse_superset_link + _recover_dataset_binding → async
    (await get_dashboard_detail, get_chart)
  - dataset_review/orchestrator.py: start_session + _build_recovery_bootstrap → async
    (await get_dataset_detail, parse_superset_link)
  - _routes.py (dataset_review): await orchestrator.start_session()
  - validation_tasks.py: await parse_superset_link + get_dashboard_detail
 2026-06-05 09:55:34 +03:00
busya
c7d8f4431e 032: fix remaining sync→async propagation (17 call sites) 
Core fixes:
  - service_datasource.py: fetch_datasource_metadata() → async
  - service.py: create_job(), update_job() → async (callers await)
  - _job_routes.py: await create_job/update_job

Maintenance scanners:
  - _dashboard_scanner.py: 4 functions → async (find_affected, _get_linked,
    _apply_filters, _resolve_title)
  - _chart_manager.py: 3 functions → async
  - _banner_renderer.py: rebuild_banner → async
  - _orchestrators.py: 3 orchestrators → async
  - maintenance_banner.py: await async calls

Migration:
  - dry_run_orchestrator.py: run(), _build_target_signatures() → async
  - risk_assessor.py: build_risks() → async
  - migration.py: await service.run()
  - mapping_service.py: sync_environment() → async

Dead code:
  - _helpers.py: _find_dashboard_id_by_slug marked DEPRECATED
 2026-06-05 08:56:37 +03:00
busya
71000db785 032: fix run_translation route handler — sync def → async def (asyncio.create_task needs running loop) 
run_translation was def (sync FastAPI handler runs in thread pool with no event
loop), but its body calls asyncio.create_task(_background_execute()) which
requires a running event loop. Changed to async def so FastAPI runs it on the
event loop directly.

Error: 'Run failed: no running event loop'
 2026-06-05 08:40:46 +03:00
busya
71156783cf 032: fix get_dashboards_page_async -> get_dashboards_page (method renamed during async migration) 
get_dashboards_page_async() no longer exists — the sync/async split was
removed and the method is now simply get_dashboards_page() (already async).
Calls in dashboard slug resolution and git helpers were using the old name.
This caused AttributeError at runtime, making all slug-based dashboard
lookups fail with 'Dashboard not found'.
 2026-06-05 08:39:05 +03:00
busya
5eb116509b 032: dead code cleanup — remove sync APIClient, _llm_http, preview_llm_client, fix retry chains 2026-06-05 08:31:18 +03:00
busya
ee9123bcf2 032: deep async propagation — orchestrator, insert, mapper, batch chains 
Full async conversion for all sync callers of async SupersetClient methods:

orchestrator_sql.py: generate_and_insert_sql + _resolve_dialect → async
orchestrator_run_completion.py: complete_success → async (calls generate_and_insert_sql)
orchestrator_exec.py: execute_run → async (awaits complete_success)
orchestrator_runner.py: execute_run → async (delegates to engine)
orchestrator.py: execute_run + _generate_and_insert_sql → async
executor.py: _insert_batch_to_target → async (awaits batch insert)
_batch_proc.py: insert_batch_to_target → async
_batch_insert.py: insert_batch_to_target + _resolve_insert_backend + _execute_insert_sql → async
dataset_mapper.py: get_sqllab_mappings + run_mapping → async
  + await get_dataset, update_dataset, execute_and_poll
mapper.py: await on run_mapping + resolve_database_id calls
_run_routes.py: threading.Thread → asyncio.create_task (_background_execute async)
 2026-06-05 00:13:01 +03:00
busya
f416583a8c 032: fix async chain for target schema check + mapper + executor 
resolve_database_id → async (was sync but called async SupersetClient methods)
  - await client.get_database(db_id)
  - await client.get_databases(...)

validate_target_table_schema → async (calls async resolve_database_id + execute_and_poll)
  - await executor.resolve_database_id(...)
  - await executor.execute_and_poll(...)

Route check_target_schema → await validate_target_table_schema(...)

MapperPlugin.execute → await executor.resolve_database_id(...)
  (execute() was already async def, just missing await)

SupersetSqlLabExecutor.execute_sql → await self.resolve_database_id()
  (was sync call to now-async method)
 2026-06-05 00:04:54 +03:00
busya
f6dc7c47e7 032: fix missing awaits in translate datasource endpoints (#5,6,7) 
Three sync functions called async SupersetClient methods without await:
  - get_datasource_columns(): get_dataset_detail + get_database (async)
  - fetch_available_datasources(): get_datasets (async)
  - Route handlers: both calls missing await

Converted to async functions + added awaits. Both endpoints now return
proper data instead of coroutine errors.
 2026-06-04 23:57:46 +03:00
busya
380f8806fb 032: fix missing await on git_service.get_status() (async -> coroutine) 
git_service.get_status() is async def, but was called without await,
returning a coroutine object instead of dict. The coroutine was then
used as a dict value in RepoStatusBatchResponse, causing Pydantic
ValidationError (dict type mismatch).
 2026-06-04 23:54:42 +03:00
busya
27bbf057ec 032: T045 — git services async (run_blocking for all blocking ops) 
_merge.py partially done. Tests still pending.
 2026-06-04 20:36:33 +03:00
busya
846d2cb9a9 032: T019 — remaining route files migrated to async SupersetClient 
All routes (assistant, migration, datasets, git) now use AsyncSupersetClient.
_helpers.py sync->async for dashboard ref resolution.
_detail_routes.py import fixed.

Known residual: MigrationDryRunService and IdMappingService still sync.
 2026-06-04 20:02:33 +03:00
busya
51afbee470 032: Phase 3 US1 — 13 mixins migrated to async + dashboard routes 
T011-T017: All SupersetClient mixins now async.
T018: _detail_routes.py uses registry/AsyncSupersetClient.
T019: Partial — routes/environments, settings, profile, listing async.

RATIONALE: Big-bang merge of sync+AsyncSupersetClient.
REJECTED: dual-stack.

Remaining T019: assistant/*, migration, datasets, git helpers still use sync.
 2026-06-04 19:55:43 +03:00
busya
2436b2d320 feat(backend): add is_regex to dictionary API routes 
Add is_regex field to list, add, and edit dictionary entry API responses,
and pass is_regex through to DictionaryEntryCRUD methods.
 2026-06-04 16:17:36 +03:00
busya
08103871f7 fix(backend): resolve test regressions 
- Remove invalid sqlite=True parameter from composite index migration
  (sqlite=True is not a valid op.create_index parameter)
- Fix test_assistant_api assertions for updated response format
- Fix test_git_status_route edge case assertions
- Fix test_audit_service expected value after metric changes
- Fix test_session_repository assertion after store refactor
 2026-06-04 16:16:18 +03:00
busya
b823bc559b refactor(backend): split translate run routes into edit/history modules 
Extract inline edit, bulk find-replace, and override language endpoints from
_run_routes.py into dedicated _run_edit_routes.py and _run_history_routes.py
modules to reduce module complexity below INV_7 limits.

Changes:
- _run_routes.py now only handles execution, retry, and cancel endpoints
- _run_edit_routes.py (new): inline edit, bulk find-replace, override language
- __init__.py registers the new route modules
- schemas/translate.py: add imports for extracted endpoints
 2026-06-04 16:15:48 +03:00
busya
15d47450a3 feat: add deduplicate + metrics to Run tab 
Backend:
- deduplicate param in GET /runs/{id}/records — NOT EXISTS subquery
  with (created_at, id) tiebreaker for one row per source_hash
- source_data and source_hash added to records JSON response
- fix missing status import in _run_history_routes.py (NameError)
- fix tab/space mixup in orchestrator_aggregator.py
- remove duplicated @RELATION edges in metrics.py
- fix #region/#endregion style and @PURPOSE→@BRIEF in metrics.py

Frontend:
- RunTabContent: summary metrics bar (fetchJobMetrics) with HelpTooltips
- RunTabContent: trigger_type badge, duration, cache rate in run rows
- TranslationRunResult: deduplicate=true by default, paginated Load more
- TranslationRunResult: per-language token_count and estimated_cost
- TranslationRunResult: collapsible batch breakdown with timing
- TranslationRunResult: Bulk Replace button in header and records table
- TranslationRunResult: source_data key values shown under source text

i18n: all new keys in EN + RU (load_more_records, showing_records,
sum_*, help_sum_*, trigger_*, batch_*, duration_label, cost_label,
cache_rate, load_more_records)
 2026-06-04 13:23:10 +03:00
busya
814f2da139 perf(translate): fix slow translation startup — CJK estimation, output budget, provider token config 
Root cause: batch sizing underestimated CJK token density (1.5→1.0 chars/token)
and ignored output budget as primary constraint, causing cascading finish_reason=length.

Changes:
- _token_budget.py: CJK_RATIO 1.5→1.0, OTHER_RATIO 2.2→1.8, safety factors 0.75/0.70
- _token_budget.py: new _compute_max_rows_by_output() — output budget is PRIMARY constraint
- _batch_sizer.py: resolve_provider_config() with DB-level context_window/max_output_tokens
- _batch_sizer.py: INPUT_SAFETY_FACTOR applied, max_rows_by_output used as row cap
- _llm_http.py: log actual usage.prompt_tokens/.completion_tokens from provider
- _llm_call.py: retry only missing rows after finish_reason=length (save partial result)
- models/llm.py + schema: provider-level context_window / max_output_tokens (nullable)
- services/llm_provider.py: get_provider_token_config() helper
- Alembic migration: add columns to llm_providers
- Svelte ProviderConfig: collapsible Advanced: Token Limits section
- 12 new tests (token budget, batch sizer, provider config)
- All 492 tests pass
 2026-06-03 23:25:08 +03:00
busya
1e6ec34c2b feat(backend): drop dictionary dialect columns, add allowed_languages 
- Removes source_dialect and target_dialect from TerminologyDictionary model,
  schemas, routes, helper serialization, and all tests
- Adds allowed_languages config field (BCP-47 language codes) to GlobalSettings
  with validation, consolidated settings response, and API endpoint
- Adds alembic migration f1a2b3c4d5e6 to drop the two columns
 2026-06-03 11:48:03 +03:00
busya
f87092c4a7 feat(translate): add cache_hits counter — backend + frontend 
Backend:
- TranslationRun model: add cache_hits Integer column (default 0)
- TranslationRunResponse schema: add cache_hits field
- _helpers.py/_run_list_routes.py/orchestrator_aggregator.py: include
  cache_hits in all run API responses
- _batch_proc.py: count pre_rows served from translation cache per
  batch, return cache_hits in batch result
- executor.py: accumulate cache_hits across batches, persist to run
- Alembic migration: dabc9709 — add cache_hits column to translation_runs

Frontend:
- translationRun.svelte.ts: add cacheHits to store state, WS handler,
  polling handler
- TranslationRunProgress.svelte: 5-col stats grid with purple Cache card
- TranslationRunGlobalIndicator.svelte: 5-col stats with Cache
- TranslationRunResult.svelte: 5-col detail stats with Cache card
- History page: cache_hits shown in run list row + detail panel

Visual: cache hits shown in purple alongside green/yellow/red metrics
(total/success/failed/skipped). Visible during run + in history.

Tests: backend 69/69 translate tests ✅, frontend 698/698 tests ✅,
frontend build ✅
 2026-06-02 11:59:57 +03:00