3634df25a1
fix(translate): datasource change not persisted on save + preview column cleanup
...
Root cause: saveJob() used stale sourceDatasourceId (set once on load)
instead of live datasourceId (updated by ConfigTabForm via $bindable).
Since sourceDatasourceId was always truthy, the || fallback to datasourceId
never triggered — the old datasource ID was always sent to PUT.
Fixes:
- Removed dead sourceDatasourceId atom; saveJob() uses datasourceId directly
- Bound sourceTable via $bindable through ConfigTabForm; updated on select
- loadDatasourceColumns() syncs databaseDialect from Superset columns API
- saveJob() sends undefined for database_dialect="unknown" to force re-detect
- Backend: added direct_db+connection_id validation on update (mirroring create)
- Removed redundant "Язык ист." column from TranslationPreview table
- Removed unused getDetectedLang function
Tests:
- TranslationJobModel: datasourceId save mapping, dialect sync, unknown→undefined
- TranslateJobService: reject direct_db without connection_id, preserve existing
Verified: browser — translate_cross datasource persisted after save+reload,
dialect detected as "clickhouse", columns loaded (2), translation column preserved.
2026-07-07 23:50:08 +03:00
de023c7a31
fix(backend+frontend): migration deadlock, async pattern, timeout safety, fire-and-forget translations
...
Backend:
- MigrationPlugin.execute — remove AsyncJobRunner.run() deadlock on post-migration
ID sync (2 deadlocks total: plugins/migration.py + api/routes/migration.py
trigger_sync_now). Replace blocking runner.run with direct await.
- MigrationPlugin.execute — fix temp file leak (dry_run=True prevented cleanup).
- MigrationPlugin.execute — IdMappingService(SessionLocal()) now closed in finally.
- MigrationPlugin.execute — wire IdMappingService into MigrationEngine constructor
so cross-filter patching actually works instead of silently skipping.
- MigrationPlugin.execute — SupersetClient.aclose() in finally to prevent
httpx connection pool leak.
- TaskManager — _async_tasks dict leak (add_done_callback cleanup).
- JobLifecycle — CancelledError handler (tasks stuck in RUNNING after cancel).
- JobLifecycle — persist_task BEFORE _broadcast_task_status (crash consistency).
- JobLifecycle — wait_for_resolution/wait_for_input now have 3600s timeout.
- AsyncJobRunner.run — 300s timeout on future.result() (APScheduler thread safety).
Translate scheduler:
- execute_scheduled_translation — replace blocking runner.run(orch.execute_run(run))
with fire-and-forget TranslationOrchestrator.execute_background(). APScheduler
thread is freed in ms instead of blocking for the full translation duration.
Translation runs of 10k+ rows (200+ LLM batches, hours) no longer hit the
300s runner timeout.
- TranslationOrchestrator.execute_background — new static method: opens own DB
session, dispatches asyncio.create_task, handles errors + notification.
- Scheduler last_run_at updated at dispatch time (not after completion).
Frontend:
- MigrationModel.stepReady[3] now requires dryRunResult != null (was always
true, allowing UI to reach step 3 without dry-run).
- WizardModel.goToStep gate for step 3 uses stepReady[3].
- +page.svelte — dryRunResult no longer self-clears via reactive loop.
- Progress bar step 3 indicator gate fixed for new readiness logic.
Tests:
- 2 new regression tests for migration sync (deadlock-free, completes cleanly).
- test_migration_plugin.py — _make_mock_superset_client/_make_mock_mapping_service
helpers for proper async mock behavior (aclose, sync_environment AsyncMock).
- 10 scheduled-translation tests updated for fire-and-forget pattern.
- MigrationModel.test.ts — step 3 invariant + goToStep block test.
- All affected tests: 104 backend + 79 frontend = 183 passed.
2026-07-07 21:04:23 +03:00
0d09e24434
fix(agent): proxy gradio config and reduce translate blocking
2026-07-07 17:37:36 +03:00
b95df37cd5
refactor(agent): extract agent+shared into standalone packages with full GRACE semantic markup
...
- Move agent code from backend/src/agent/ to agent/src/ss_tools/agent/
- Extract shared stdlib-only utilities to shared/src/ss_tools/shared/
- Add #region/#endregion contracts to all ~140 functions (INV_1 compliance)
- Update docker files, entrypoint, build scripts for new package layout
- Backend now imports ss_tools.shared._llm_health (no gradio/langchain deps)
- Add specs for 036-039 feature plans
2026-07-07 15:18:24 +03:00
ce368429f7
fix(agent): rewrite _llm_health to use openai+httpx instead of langchain
...
Previous lazy-import fix still required langchain at function call time.
Root cause: langchain_openai/langchain_core are only in requirements-agent.txt,
not requirements-backend.txt. The /api/agent/llm-status endpoint runs in
the backend container which has 'openai' but not 'langchain'.
Rewrite _check_llm_provider_health() to use:
- AsyncOpenAI from 'openai' package (in both containers)
- httpx to call /api/agent/llm-config on localhost (same as agent does)
- system_ssl_context() for SSL (same as LLM test endpoint)
- openai exceptions (APIConnectionError, APITimeoutError, etc.)
No langchain dependency at all — works in both backend and agent containers.
Verified: 977 tests passed, import without langchain succeeds.
2026-07-07 13:34:07 +03:00
beed41d6c9
fix(agent): lazy imports in _llm_health — langchain_core not in backend container
...
_llm_health.py imported langchain_core, langchain_openai, and openai at
module level. These packages are only installed in the agent container
(requirements-agent.txt), not the backend container (requirements-backend.txt).
Moved all langchain/openai imports inside _check_llm_provider_health() with
ImportError handled gracefully — returns 'unavailable' status instead of
ModuleNotFoundError 500 error.
Root cause: the /api/agent/llm-status endpoint runs in the backend container,
which has httpx but not langchain. The agent container has all LLM deps.
Verified: import without langchain succeeds, health check returns 'unavailable'.
2026-07-07 13:24:45 +03:00
58d06fb287
fix(ssl+agent): capath for all HTTP clients + isolate gradio import
...
SSL fix (ADR-0009 Finding 7):
- Replace ssl.create_default_context() with system_ssl_context(capath)
in client_registry.py, async_network.py, notifications/providers.py,
git/_base.py. Previous fix (0.5.1) only covered LLM clients; the
Superset API client still used ssl.create_default_context() which
loads cafile (flat bundle) where OpenSSL 3.x ignores intermediate CA
certificates. system_ssl_context() uses capath only (hash symlinks).
Agent fix:
- Extract _check_llm_provider_health + _llm_status from agent/app.py
into agent/_llm_health.py. The /api/agent/llm-status endpoint was
importing from agent/app.py which triggers 'import gradio' at module
level. Backend container does not have gradio installed, causing
ModuleNotFoundError (500 error) every 30s on health check polling.
Config:
- Add ALLOWED_ORIGINS to docker-compose.yml + docker-compose.enterprise-clean.yml
ADR-0009 updated: Layer 2 table expanded with 4 missing clients,
Finding 5 reconciled (LLM_CA_CERT_URLS restored in 0.5.1), version 0.5.2.
Verified: 1110 unit tests passed, gradio import isolation confirmed.
2026-07-07 12:18:43 +03:00
18b9f2e94e
fix(certs): restore HTTP CA auto-download and test edge matrix
...
Restore LLM_CA_CERT_URLS support for corporate PKI HTTP cert delivery.
Downloaded certificates are installed into the llm/ CA category and share
the same system trust + NSS import path as CERTS_PATH-mounted certs.
Changes:
- certs.sh: add download_llm_ca_certs with HTTP-only curl policy,
PEM/DER handling, retries, llm/ storage, hash symlinks for llm+custom
- backend/agent entrypoints: run download before install_all_certs
- compose/env/docs: expose LLM_CA_CERT_URLS again and update ADR-0009
- integration tests: cover PEM/DER/CER, invalid certs, non-HTTP skips,
query-string filenames, private/server skips, empty inputs, combined
LLM_CA_CERT_URLS + CERTS_PATH channels, and NSS imports
Verified:
- 194 passed, 1 skipped integration tests
- full backend container smoke passed with testcontainers PostgreSQL
- docker bundle rebuilt for 0.5.0
2026-07-07 10:23:49 +03:00
1e61f098bd
fix(docker): add missing certs.sh COPY to backend/all-in-one Dockerfile
...
Backend entrypoint sources certs.sh for CA certificate installation,
but the file was never copied into the Docker image — causing
container crash loop on startup (regression in 8fd23f7e
2026-07-07 00:58:41 +03:00
98d91bb738
test(integration): add encrypted key Superset container test with SSL_KEY_PASSPHRASE
...
conftest.py: superset_container fixture supports {encrypted_key: True}
- Writes encrypted private key to container
- Decrypts with openssl rsa -passin env:SSL_KEY_PASSPHRASE
- Starts Flask/Werkzeug TLS server with decrypted key
- Falls back to unencrypted key for existing tests
test_superset_tls_custom_ca.py:
- NEW: TestEncryptedPrivateKeySuperset class
- test_superset_health_over_tls_with_encrypted_key
- test_openssl_capath_encrypted_key
- Uses parametrize({tls: True, encrypted_key: True})
Validates full production flow: encrypted key → SSL_KEY_PASSPHRASE
→ openssl decrypt → TLS handshake → client trust via capath.
2026-07-06 21:21:42 +03:00
8fd23f7ea1
refactor(ssl): centralize SSL trust management, remove LLM_SSL_VERIFY
...
Centralized SSL via one contract: CERTS_PATH=/opt/certs mounted into all containers.
Backend:
- NEW: backend/src/core/ssl.py — system_ssl_context(), httpx_verify(),
cert_dir_inventory()
- LLMClient._get_ssl_verify() → delegates to core.ssl
- _llm_async_http._get_verify() → delegates to core.ssl
- Removed LLM_SSL_VERIFY env reading from all runtime code
Docker:
- NEW: docker/certs.sh — shared cert installer (PEM/DER/cer to .crt conversion,
update-ca-certificates, hash symlinks, NSS import)
- NEW: docker/agent.entrypoint.sh — agent entrypoint with cert installation
- backend.entrypoint.sh → uses certs.sh instead of install_llm_ca_certs
- Dockerfile.agent → adds ca-certificates, openssl, entrypoint
Compose:
- Removed LLM_CA_CERT_URLS and LLM_SSL_VERIFY from all compose files
- Added CERTS_PATH volume mount to agent (dev + enterprise)
- Added certs volume mount to backend/agent in dev compose
Env examples:
- Removed LLM_SSL_VERIFY, LLM_CA_CERT_URLS from .env.example,
.env.enterprise-clean.example, .env.current.example, .env.master.example,
backend/.env.example
- Enhanced CERTS_PATH comments with accepted formats
Diagnostics:
- diag_container.py: removed LLM_* checks, added CERTS_PATH inventory,
uses core.ssl for context creation
Tests:
- Updated test_llm_analysis_service, test_llm_async_http,
test_client_headers to verify centralized ssl context (no env disable)
- 4/4 SSL tests pass
2026-07-06 21:00:28 +03:00
7f9e781873
fix(security): address QA/security audit findings — authorization, i18n, error safety
...
Critical (C1): Added has_permission('security', 'READ') to /health and
/fingerprint endpoints; has_permission('security', 'WRITE') to /recover.
Previously any authenticated user could enumerate encrypted secrets and
overwrite stored values.
High (Bug #1 ): Fixed recover endpoint — updated/failed status now correctly
reflects whether a non-empty replacement value was submitted. Empty values
now report 'skipped' instead of falsely 'updated'.
High (Bug #2 , #3 ): Replaced all hardcoded English strings in
KeyRecoveryWizard.svelte and SystemSettings.svelte with $t.settings.*
i18n lookups. Keys already existed in en/ru settings.json.
High (H1): Added _sanitize_error() helper to truncate + scrub exception
messages before logging, preventing potential secret leakage in log output.
Cleanup: Moved LLMProviderConfig import to module top. Instantiate
ConnectionService once before for-loop. Added @TEST_EDGE declarations
and @PRE/@SIDE_EFFECT to C4 contract.
Tests: 226 passed
2026-07-06 18:22:36 +03:00
5c59a2c79b
feat(security): add encryption health inventory and key recovery wizard
...
Backend:
- GET /api/security/encryption/health — inventory of all stored encrypted
secrets (LLM providers, DB connections, profile Git tokens) with
decrypt attempt and structured broken/healthy status
- GET /api/security/encryption/fingerprint — non-secret key fingerprint
- POST /api/security/encryption/recover — bulk replacement of
undecryptable secrets with partial_success semantics
Frontend:
- KeyRecoveryModel.svelte.ts — state machine (idle→scanning→
healthy/needs_recovery→editing→saving→complete/partial_success/error)
- KeyRecoveryWizard.svelte — tabbed dialog with LLM/DB/Git sections,
security guidance, re-encrypt command display, edit/save flow
- SystemSettings entry point — 'Check encrypted secrets' card with
fingerprint and broken count
- API methods: getEncryptionHealth, recoverEncryptedSecrets
- Types: EncryptionRecoveryTypes
- i18n: en/ru strings for recovery flow
Tests: 226 passed
2026-07-06 18:05:05 +03:00
556294aff5
fix(agent): pin gradio <6 to avoid breaking ChatInterface API changes
2026-07-06 16:18:55 +03:00
b6752e729e
fix(docker): add alembic to requirements-backend.txt, entrypoint needs it for migrations
2026-07-06 15:49:48 +03:00
48e3ff4503
refactor(env): unify Docker env vars — canonical AUTH_SECRET_KEY, remove JWT_SECRET fallback
...
Canonical variables:
- AUTH_SECRET_KEY — JWT signing key for backend + agent (was split across
AUTH_SECRET_KEY / JWT_SECRET)
- SERVICE_JWT — agent→backend service token
- No JWT_SECRET fallback: decoder fails with migration guidance if only
JWT_SECRET is set
Compose files unified:
- docker-compose.yml, docker-compose.enterprise-clean.yml,
docker-compose.e2e.yml — all use AUTH_SECRET_KEY
- build.sh generated compose now passes AUTH_SECRET_KEY + ENCRYPTION_KEY
to backend
Env examples unified and completed:
- .env.example — comprehensive template, all compose vars
- .env.enterprise-clean.example — production template
- backend/.env.example — backend-only run
- docker/.env.agent.example — agent-only run
- NEW: .env.current.example, .env.master.example,
.env.e2e.example, frontend/.env.example
Tests aligned:
- conftest sets AUTH_SECRET_KEY (canonical value matched across test files)
- test mocks use canonical name
- 1176 passed, 0 failed
2026-07-06 14:24:17 +03:00
a5b7adb61c
refactor(agent): lightweight JWT decoder — JWT_SECTRET first, AUTH_SECTRET_KEY fallback
...
Replaced src.core.auth.jwt.decode_token import with local _jwt_decoder.py:
- Tries JWT_SECTRET first, falls back to AUTH_SECTRET_KEY (avoids key mismatch
when both env vars are set in different test files)
- verify_aud disabled — backend tokens have audience; agent ignores
- No auth DB dependency — removed AUTH_DATABASE_URL requirement from agent
Cleanup:
- Removed src/core/auth/ from agent Dockerfile COPY
- Removed AUTH_SECTRET_KEY, AUTH_DATABASE_URL from agent compose env
- conftest sets AUTH_SECTRET_KEY for test consistency
- Updated test patches to new import path
Tests: 1176 passed, 0 failed
2026-07-06 13:45:22 +03:00
590b09f587
refactor(agent): use local JWT decoder instead of src.core.auth.jwt
...
Replace src.core.auth.jwt import with lightweight _jwt_decoder.py that
uses jose.jwt directly with JWT_SECRET env var. Avoids pulling AuthConfig
→ AUTH_DATABASE_URL/SECRET_KEY validators → SQLAlchemy models into agent.
Removed: AUTH_SECRET_KEY, AUTH_DATABASE_URL from agent compose env.
Removed: src/core/auth/ copies from Dockerfile.agent COPY section.
Added: src/agent/_jwt_decoder.py — stateless JWT validation.
2026-07-06 13:37:08 +03:00
468bdb9000
fix(agent): resolve missing imports — jose, auth config, AUTH_SECRET_KEY
...
- Added python-jose[cryptography] to requirements-agent.txt
- Made sqlalchemy.orm.Session and TokenBlacklist imports lazy in jwt.py
so decode_token works without pulling ORM models into agent image
- Added src/core/auth/__init__.py, config.py, jwt.py to agent Dockerfile COPY
- Added AUTH_SECRET_KEY env var to agent compose (reads from JWT_SECRET)
2026-07-06 12:13:26 +03:00
628805503b
refactor(docker): split requirements by role, slim enterprise bundle by default
...
Split requirements.txt into role-specific files:
- requirements-backend.txt — FastAPI API runtime
- requirements-agent.txt — Gradio/LangGraph agent (no embeddings)
- requirements-embeddings.txt — opt-in sentence-transformers
Default enterprise bundle (./build.sh bundle) builds slim agent without
sentence-transformers/torch (~500 MB saved). Embeddings opt-in via
./build.sh bundle:embeddings <tag>. Embedding router degrades gracefully
when package is absent (ImportError catch in _embedding_router.py).
Updated Dockerfiles:
- backend.Dockerfile → requirements-backend.txt
- Dockerfile.agent → requirements-agent.txt + WITH_EMBEDDINGS build arg
- all-in-one.Dockerfile → requirements-backend.txt
Added embeddings variant to build.sh with full manifest/digest output.
Hardened .dockerignore (cache, report, model, archive patterns).
Other branch work: align git route mocks with async services, remove
legacy agent test files superseded by contract tests.
2026-07-06 10:06:07 +03:00
57b7ee05e0
test(git): align route mocks with async services
2026-07-06 01:23:20 +03:00
1e56416c9f
test(backend): update legacy agent and auth expectations
2026-07-06 01:23:09 +03:00
28cd141e76
feat(reports): add task status settings and tests
2026-07-06 01:22:39 +03:00
082d6af3ab
test(agent-chat): audit guardrail and error handling
2026-07-06 01:20:12 +03:00
147d711657
feat(agent-chat): complete context guardrail event coverage
2026-07-05 14:14:42 +03:00
45e781fb74
chore: commit remaining working changes
...
Backend:
- agent: confirmation, persistence, app, langgraph_setup updates
- routes: agent_superset_explore, environments, git helpers/operations
- services: git sync refactoring
- tests: git_status_route expanded
Frontend:
- Navbar: minor cleanup
- Profile: i18n (en/ru), page enhancements, integration tests
- New: _llm_params.py
2026-07-05 09:24:45 +03:00
ff60865183
feat(agent-chat): 035-agent-chat-context — контекст, guardrails, tools, database discovery
...
== User stories ==
US1: Контекст с дашборда/датасета → /agent с URL params
US2: Guardrails card — env badge, 7 risk tones, countdown, permission_denied
US3: Tools optimization — retry, timeout, summarise, RBAC + context affinity
== Backend ==
- _context.py (NEW): UIContext validation (7 checks)
- _tool_filter.py (NEW): RBAC + context affinity pipeline
- _confirmation.py: build_confirmation_contract_v2, permission_denied_payload
- tools.py: superset_list_databases, retry/summarise/timeout wrappers
- app.py: _inject_uicontext, _inject_env_id_into_tools, database
prefetch в runtime context
- _persistence.py: prefetch_databases()
- agent_superset_explore.py: GET /databases endpoint
- _llm_async_http.py, _persistence.py: fix double /v1 в LLM URL
(LM Studio Unexpected endpoint)
== Frontend ==
- AgentChatModel.svelte.ts: 5 atoms, 3 actions, countdown, context
- AgentChat.svelte: production banner, process steps, debug panel
- ConfirmationCard.svelte: 7 risk tones, permission_denied, countdown
- ToolCallCard.svelte: retrying/timeout/cancelled states
- StreamProcessor.svelte.ts: tool_retry, timeout, permission_denied
- TopNavbar: sparkles icon + Ассистент
- sidebarNavigation: AI section
- DashboardHeader, datasets/+page: contextual AI buttons
- Icon: sparkles, brain, cpu icons
- tailwind: assistant category colors
- i18n: en/ru nav keys
== Tests ==
- 159 backend agent tests (+16 US3: retry, timeout, summarise, contracts)
- 2544 frontend tests (+11 model + component tests)
- 15 JSON fixtures (10 API + 5 model)
== Specs ==
- specs/035-agent-chat-context/: spec, UX, plan, tasks, research,
data-model, contracts, quickstart, traceability, fixtures, checklists
Closes #035
2026-07-04 22:47:17 +03:00
0237e2f7e8
chore: добавить пример Fernet-ключа в .env.example
...
- Сгенерированный пример ENCRYPTION_KEY с командой для генерации
- Пометка 'сгенерируйте свой для прода'
2026-07-04 15:01:23 +03:00
b78493aeb7
chore: добавить примеры значений в .env.example
...
- backend/.env.example — пример DATABASE_URL с комментарием
- root/.env.example — заполнены примеры для всех переменных
(AUTH_SECRET_KEY, DATABASE_URL, INITIAL_ADMIN_PASSWORD, POSTGRES_PASSWORD)
2026-07-04 14:59:39 +03:00
ec9bc76a37
cleanup: убрать мёртвые env-переменные, консолидировать чтение в agent/_config.py
...
Удалены из кода:
- JWT_SECRET — мёртвая (decode_token использует AUTH_SECRET_KEY)
- SESSION_SECRET_KEY — заменён на прямой AUTH_SECRET_KEY
- POSTGRES_URL — deprecated fallback, удалён из database.py и reencrypt.py
Консолидировано чтение env-переменных agent-модуля:
- Создан agent/_config.py — единый модуль для FASTAPI_URL,
SERVICE_JWT, GRADIO_*, STORAGE_ROOT, AGENT_* (9 констант)
- Все agent/*.py импортируют из _config вместо разрозненных os.getenv
Удалены or-дефолты (безопасность):
- agent/langgraph_setup.py — удалён hardcoded DB URL postgres:postgres
- agent/langgraph_setup.py — удалены fallback API URL и model name
- scripts/reencrypt.py — удалён hardcoded DB URL postgres:postgres
- plugins/llm_analysis/service.py — удалены or-дефолты URL/app name
.env.example — минимализация:
- backend/.env.example: только 4 обязательные переменные
- root/.env.example: обязательные + docker + SSO/админ
Обновлены тесты (139 passed)
2026-07-04 14:58:43 +03:00
a99c1d6d01
Improve agent UX and spec sync
2026-07-03 16:47:10 +03:00
500491c281
test(tls): add encrypted (passphrase-protected) private key integration tests
...
- ca_chain fixture: add server_key_encrypted (BestAvailableEncryption) +
server_key_passphrase to returned dict; existing fields unchanged
- TestEncryptedPrivateKey: 6 new tests covering
- ssl.SSLContext.load_cert_chain() with correct/wrong/missing passphrase
- asyncio SSL server end-to-end with encrypted key + full chain trust
- openssl rsa -check with wrong/missing passphrase (CLI validation)
- All 6 tests pass; 14/14 in test_superset_tls_custom_ca.py
2026-07-03 16:46:12 +03:00
33ee976c48
feat(reports): Task Status Center — unified /reports dashboard
...
Страница /reports трансформирована в Центр статусов задач:
Backend:
- GET /api/reports/summary — агрегированные счётчики тип×статус (5 корзин)
- GET/PUT /api/settings/reports — глобальные настройки отчётов
- _filter_tasks_by_rbac() — row-level фильтрация по роли
- normalize_task_report: LLM-валидация с ошибками → FAILED/PARTIAL
- get_summary(): 5 корзин pending/running/awaiting_input/success/failed
Frontend:
- TaskCenterModel.svelte.ts (400 строк) — Screen Model
- SummaryPanel — сводная панель с цветовым кодированием и active filter
- ReportCard — humanized labels, duration, task_id, failed border
- FilterBar — search + sort + time range с label'ами
- Pagination — showingText, уникальные id для select
- Quick views: «Упавшие», «В работе», «Успешные»
- TaskDrawer: scroll-to-error, footer скрыт для terminal, «Н/Д» fix
Тесты: 48 backend + 38 frontend (2521 всего)
Build: ✅ Console errors: 0
2026-07-02 18:53:58 +03:00
8c10632494
feat(semantic): curator-driven protocol hardening — decision memory + relation repair
...
- Add @RATIONALE/@REJECTED to 103+ C4/C5 contracts across backend core, services, API routes, and frontend models
- Fix 109 unresolved @RELATION edges (Auth.*, SupersetClient.*, AgentChat.*, ADR cross-refs)
- Add 13 @ingroup tags for DSA/HCA attention grouping
- Repair 29 stale graph edges via index rebuild
- Update .kilo agent prompts and skills for GRACE-Poly v2.6 compliance
- Git integration: merge routes, branch lifecycle, remote providers, UX components
- 0 broken anchor pairs, index rebuilt with 0 parse warnings
2026-07-02 08:53:19 +03:00
64564da988
fix(git): fix 17 missing async/await bugs + UX overhaul
...
Backend:
- fix 17 missing 'await' in git route handlers causing silent no-ops
(branches, diff, history, commit, push, pull, merge, promote, sync)
- fix async coroutine passed to run_blocking in git_plugin.py
Frontend:
- add collapsible 'How it works' onboarding (GitHelpPanel)
- add status legend with color-coded repository statuses
- i18n: add 50+ missing keys, replace hardcoded strings
- add Refresh button in modal header
- add PROD deploy confirmation dialog (replaces browser prompt())
- add CommitHistory to workspace tab with timeline nodes
- add post-commit success banner with next-step guidance
- increase success toast duration to 8s
- group local/remote branches in selector (optgroup)
- format last_modified dates timezone-aware
- change PROD badge from red to neutral indigo
- extract shared resolveGitStatusToken to git-utils.ts
- fix 'slug' label regression
- remove dead init_repo_button key
UI/UX audit fixes:
- add descriptions to Create/Init buttons in init panel
- add actionable CTA to server mismatch warning
- improve checkbox text phrasing
2026-07-01 20:47:25 +03:00
7613ad37ae
fix(agent): minimal safety net, zero-config router, LLM provider status UI
...
- Remove deterministic intent matching (keyword lists, infer_tool,
fast_confirmation, negation guard, classification sets)
- Embedding descriptions auto-generated from tool docstrings
- LLM provider health endpoint GET /api/agent/llm-status
- 3 error codes: LLM_PROVIDER_UNAVAILABLE, LLM_TIMEOUT, LLM_AUTH_ERROR
- Frontend banner with auto-retry 30s + input disable
- i18n for LLM status messages (assistant.json ru/en)
- 138 passing backend tests
2026-07-01 16:47:21 +03:00
12118ac4ec
fix(security): resolve Critical+High findings from module audit — agent, translate, superset_client
...
P0 — CRITICAL (CWE-798): JWT_SECRET crash-early
Replace hardcoded super-secret-key fallback with os.environ["JWT_SECRET"]
and ${JWT_SECRET:?} syntax in app.py + docker-compose files
P1 — HIGH: Frontend dependency CVEs
Upgrade svelte 5.43.8 → 5.56.4 — resolves devalue DoS (GHSA-g2pg-6438-jwpf)
and svelte XSS (GHSA-crpf-4hrx-3jrp, GHSA-m56q-vw4c-c2cp, GHSA-rcqx-6q8c-2c42)
P2 — MEDIUM: Logging hygiene + contract gaps + tool resolver refactor
Apply _redact_sensitive_fields() in middleware + event streaming
Truncate LLM error body to 100 chars
Add @RATIONALE/@REJECTED to HandleResume + SaveConversation
Refactor deterministic intent matching → LLM-driven tool resolution
P3 — LOW: Translate logging hardening
Move _sanitize_url() to _utils.py (shared, no circular imports)
Sanitize base_url before logging in _llm_call.py and _llm_async_http.py
Emit EXPLORE warning when LLM_SSL_VERIFY=false disables TLS
superset_client module: passed clean — no changes needed
2026-07-01 13:17:29 +03:00
e174c11d4a
tasks 033 updated
2026-06-30 19:05:17 +03:00
e40724a0fe
fix: timezone handling across fullstack — UTC parsing + display in configured TZ
...
Root cause: backend returned naive ISO datetimes (no Z/offset) → JS parsed them as
browser local time → 3h drift for MSK users → '3ч' instead of 'только что'.
Backend:
- schemas/agent.py: add field_serializer('Z' suffix) for ConversationItem.updated_at
and MessageItem.created_at — naive datetimes serialized as UTC
- routes/agent_conversations.py: datetime.utcnow() → datetime.now(timezone.utc) (3x)
Frontend:
- New: stores/timezone.svelte.ts — global reactive appTimezone store
- dateFormat.ts: add parseDateUTC() (appends 'Z' to naive ISO), all format*()
functions now use parseDateUTC + { timeZone: appTimezone.current }
- ~25 files: replace new Date(apiString) → parseDateUTC(apiString),
add timeZone: appTimezone.current to toLocaleString()/toLocaleDateString()
- SystemSettings.svelte + HealthCenterModel sync appTimezone to global store
- ConversationList.svelte: fix relativeTime() and date grouping (the '3ч' bug)
Verified: backend schema test, frontend 2501 tests pass, build succeeds,
browser validation on /agent and /settings.
2026-06-30 18:11:42 +03:00
60674c8639
fix(logging): defensive trace_id seed + falsy error/payload in CotJsonFormatter
...
log_requests middleware (BaseHTTPMiddleware) showed no-trace
because anyio.create_task_group() in Starlette 0.50.0 does not
always propagate ContextVar from raw ASGI TraceContextMiddleware.
Fix 1: defensive get_trace_id() check at log_requests entry —
if empty, seed_trace_id() to ensure every request has one.
Fix 2: CotJsonFormatter used 'if error:' and 'if payload:' which
silently drop empty strings (str(e)='') and empty dicts.
Changed to 'is not None' checks — preserves all data.
Root cause: 12 EXPLORE-without-error entries from belief_scope's
exception handler where Exception() has empty message.
2026-06-30 17:41:13 +03:00
2f238dee13
fix(agent): seed trace_id for agent process lifecycle
...
Agent logs had trace_id='no-trace' because the Gradio process
never called seed_trace_id(). The CotJsonFormatter reads trace_id
from ContextVar — without seeding, it defaults to empty string
displayed as 'no-trace'.
Fix:
- app.py: seed_trace_id() on every agent_handler invocation
- run.py: seed_trace_id() on agent startup (for LLM config fetch)
Each Gradio submit gets a fresh trace_id, making agent logs
correlatable with downstream FastAPI calls.
2026-06-30 17:33:21 +03:00
79d0f8f678
feat(agent-superset): extend SupersetClient with agent-critical methods + DDL/DML guard + tests (126 passed)
...
Ported from mcp-superset research module, integrated into existing async SupersetClient:
New mixins (4 files):
- safety.py: DDL/DML guard (13 keywords), comment/string stripping, viz_type validation
- _sql_lab.py: execute_sql, format_sql, results, estimate, CSV export, query history
- _saved_queries.py: saved queries CRUD (5 methods)
- _audit.py: permissions audit matrix (user × dashboard × dataset × RLS)
Extended mixins (4 files):
- _dashboards_write.py: +create, +update (general), +copy, +publish, +unpublish
- _dashboards_crud.py: +standalone get_dashboard_charts/datasets, +get_dashboard
- _datasets.py: +create, +delete, +duplicate, +refresh_schema, +get_or_create, +export/import
- _databases.py: +update, +test_connection, +schemas/tables/catalogs, +validate_sql, +select_star, +table_metadata
FastAPI proxy (2 files):
- agent_superset.py (284L): SQL Lab + Dashboards/Datasets write endpoints
- agent_superset_explore.py (240L): DB explore + Audit + Saved Queries endpoints
Agent tools (2 files):
- tools.py: +7 LangChain @tools (24 total), intent-routing keywords
- _tool_resolver.py: updated SAFE/GUARDED/FAST_CONFIRM classification sets
Tests (3 files, 126 tests):
- test_superset_safety.py (51): DDL/DML bypass/legitimate/safe scenarios
- test_superset_extended.py (42): all mixin methods with mock AsyncAPIClient
- test_superset_tools.py (33): agent tool registration, intent matching, @tool .ainvoke()
2026-06-30 17:26:51 +03:00
a0619ca049
fix(agent): unify logging API + add molecular-cot coverage to agent module
...
Phase 1 — API unification:
- Replace direct log() from cot_logger with canonical logger.reason/reflect/explore
across _confirmation.py, _persistence.py, _tool_resolver.py, app.py
Phase 2 — Gap filling:
- tools.py: add REASON/REFLECT/EXPLORE to 17 C3 tool functions (was 0 logs)
- app.py agent_handler (C4): add lifecycle REASON on entry, REFLECT on exit,
EXPLORE on OutputParserException + general exception
- langgraph_setup.py create_agent (C4): add REASON with model/config_source,
EXPLORE on env-var/InMemorySaver fallback, REFLECT on graph compilation
- _tool_resolver.py infer_tool_from_text (C3, 13 branches): REASON on inference
- _persistence.py: REFLECT on save_conversation success, EXPLORE on prefetch
Phase 3 — Plain log migration:
- middleware.py: plain logger.info() -> logger.reason()
- run.py: 7 plain logger.info/warning/error -> molecular REASON/EXPLORE
Phase 4 — Cleanup:
- cot_logger.py: deprecate MarkerLogger (@DEPRECATED + @REPLACED_BY)
- molecular-cot-logging SKILL.md: remove cot_span Section IV (never implemented),
renumber sections V-VII -> IV-VI, add cot_span rejection rationale
Verification: pytest 27/27, axiom rebuild 5844/2993/0 warnings
2026-06-30 15:48:46 +03:00
131c7cdfa4
chore: remaining pre-existing changes (storage, stream processor, tests, run.sh)
2026-06-30 15:21:15 +03:00
3b4ac807a5
feat(agent+ui): fullstack agent module refactoring + UI/UX improvements
...
## Backend: agent module GRACE-Poly compliance
- Split app.py (749→~280 lines) into _tool_resolver, _confirmation, _persistence
- All 18 naked functions wrapped in #region/#endregion contracts
- Fixed @DEFGROUP→@defgroup typos; added @DATA_CONTRACT, @SIDE_EFFECT, CoT logs
- Conversation list API: added last_role, has_tool_calls, has_error, risk_level fields
- Message state detection: Russian/English error patterns (недоступен, unavailable)
- State field preserved in save_conversation messages
- HITL titles: descriptive tool names instead of generic "HITL resume"
## Backend: conversation title generation (two-layer)
- Layer 1: clean_title() — rule-based, strips file markers, pre-fetch blocks, JSON/CSV,
URLs, code; truncates at 80 chars word boundary (25 unit tests, all edge cases)
- Layer 2: generate_llm_title() — async best-effort LLM titling via /v1/chat/completions
with per-conversation lock, graceful degradation on failure
## Frontend: conversation list indicators (orthogonal system)
- Status dot (green/yellow/red/blue) per conversation state
- Icon column: tool activity, errors, waiting, completed
- Risk stripe (left border accent) + message count badge + relative time
- Fixed group labels: "Сегодня"/"Вчера" instead of "3 ч"/"5 ч"
- Hide "Окружение: —" when env is empty
## Frontend: guardrails card verification + fixes
- Confirmed all interaction modes: Enter/click confirm, Escape/click deny
- Auto-populate envId from environmentContextStore in DashboardDetailModel
- Better error message: missing_context_hint with recovery guidance
## Design system: semantic tokens
- Added category-* gradient tokens to tailwind.config.js
- Sidebar + Breadcrumbs use semantic tokens (10 categories)
- Raw Tailwind reduced from ~50 to 6 occurrences
- Added skip-to-content link in root layout (+layout.svelte)
- Added aria-label on DashboardDataGrid row checkboxes
## Protocol: INV_7 pragmatic exception
- Modules may exceed 400 lines when contract-dense (every function has #region)
- Recorded in semantics-core SKILL.md with rationale
Total: 5841+ contracts, 2993+ edges, backend 41/41, frontend 2501/2501
2026-06-30 15:21:05 +03:00
e8d6d7d0db
fix(agent): critical agent chat bugs — backend startup & frontend streaming state
...
Backend (tools.py):
- Add Python docstrings to all 17 @tool functions (LangChain ValueError)
- Add @INVARIANT ADR: docstring requirement documented in module header
- Fix 2 f-string escaped-quote syntax errors (Python 3.13)
Frontend — compile errors (+page.svelte):
- Fix mismatched <button>/</Button> tags
- Fix missing Button import for mobile sidebar close
Frontend — streaming state loss on conversation switch (AgentChatModel):
- Add _commitStreamingPartial() helper — saves in-progress text before cancelling
- selectConversation() commits partial text to OLD conversation before switching
- createConversation() commits partial text before clearing state
- loadHistory() sets _userCancelled=true to suppress fallback messages
Frontend — ConnectionManager:
- Pass error reason through onDisconnectedPermanent callback → model.error
Frontend — null safety:
- Guard _client.submit() calls against null _client in _sendNow and resumeConfirm
2026-06-30 13:25:28 +03:00
12678c637b
fix(agent-chat): streaming state leak, document parser magic bytes, HITL flow
...
### Bugfixes — Agent Chat 'Думаю' State Leak
- fix(agent-chat): loadHistory() now resets streamingState/idle + cancels stale
submission — prevents 'Думаю' state leak across conversation switches
- fix(agent-chat): onDisconnected/onDisconnectedPermanent cascade to
streamingState — prevents permanent hang on connection loss during stream
- fix(agent-chat): guard on isLoadingHistory — prevents false commit
of 'agent unavailable' fallback when switching conversations
- fix(agent-chat): remove race in _sendNow empty-response check vs Svelte
microtask (duplicate logic removed, handles correctly)
- fix(stream-processor): confirm_resolved now appends msg.text to partialText
instead of dropping it
### Bugfixes — Backend PDF Upload
- fix(document-parser): _detect_format_by_magic() — reads file header magic
bytes as fallback when Gradio loses filename
- fix(document-parser): improved name extraction — tries orig_name, path stem
- fix(document-parser): @RELATION AgentChatTypes -> AgentChat.Types
### HITL Flow & Agent Chat Improvements
- feat(agent): HITL resume confirm/deny with userId/userJwt/envId propagation
- feat(agent): confirm_required metadata fallback via aget_state() after
'Event loop is closed' error during interrupt
- feat(agent): interrupt_before re-enabled via AGENT_CONFIRM_TOOLS env var
- feat(frontend): debug panel with connection/stream state monitoring
- feat(frontend): AgentChatModel constructor options + onBeforeSend callback
- feat(frontend): crypto.randomUUID() for local conversation ID on first send
### Backend Agent Refactoring
- refactor(agent): langgraph_setup — monkey-patch for PydanticSerializationError
- refactor(agent): tools.py — dual identity headers, expanded tool set
- refactor(agent): run.py — _find_free_port, Gradio server port fallback
- refactor(agent): app.py — file size validation, message truncation, HITL path
### Frontend
- feat(dashboard-hub): DashboardHubModel with filters, pagination, git actions
- feat(ui): DateRangeFilter component
- feat(i18n): new dashboard keys; cache tooltips fix
- fix(i18n): full run tooltips — cache is NOT ignored
### Semantic Protocol
- chore(agents): update all agents with canonical format
- chore(skills): sync semantics-core, semantics-contracts, molecular-cot-logging
### Housekeeping
- chore: remove stale semantic reports (10 files, Jan 2026)
- chore: update 033-gradio-agent-chat specs, contracts, UX, tasks, tests
- chore: add .agents/ directory (mirrors .opencode/ agent layouts)
- chore: update run.sh with DEV_MODE, port management
2026-06-29 17:15:25 +03:00
e07bf46b2b
test: integration + unit tests for cache fix and OUTPUT_SAFETY_FACTOR
...
## TestClassifyCacheSourceLang (8 tests) — regression for source-lang exclusion
- test_source_lang_in_tls_cache_all_translations: exact prod bug scenario
(detected_lang=ru, tls=[ru,en,fr,zh], cache={en,fr,zh} → pre_rows not LLM)
- test_source_lang_in_tls_cache_missing_one_translation: partial cache → LLM
- test_detected_und_tls_includes_und: und excluded correctly
- test_empty_detected_lang_no_exclusion: '' → no filtering
- test_source_lang_not_in_tls: no-op exclusion when lang not in tls
- test_multi_row_mixed_source_lang_cache: 4 rows, correct split
- test_all_same_lang_short_circuit: single target lang shortcut
- test_case_insensitive_detected_lang_matching: RU→ru
## TestBatchPipelineE2E (3 tests) — full pipeline integration
- test_warm_cache_full_pipeline_no_llm: 17 rows all cached → 0 LLM
- test_cold_cache_pipeline_all_llm: no cache → all LLM
- test_partial_cache_pipeline_split: 5 cached + 5 not → correct split
## TestOutputSafetyFactor (4 tests) — OUTPUT_SAFETY_FACTOR invariants
- test_output_safety_factor_not_above_070: ≤0.70 guard
- test_max_rows_by_output_qwen_flash_4langs: 14-24 range
- test_output_safety_factor_consistent_with_per_row: manual = actual
- test_single_lang_output_rows_above_20: ≥20 rows for 1 lang
## Semantic fixes
- Add missing #endregion TestClassifyCacheSourceLang
- Move floating @BRIEF tags inside their regions (REASONING_OVERHEAD, DICT_TOKENS_PER_ENTRY)
- Add @BRIEF to OUTPUT_PER_ROW_PER_LANG region
Verification: 705 tests pass (0 failures, 2 pre-existing warnings)
2026-06-19 14:59:05 +03:00
8809f9d5ce
fix(translate): deduplicate bulk replace buttons with distinct labels
...
- Rename page-level button to 'Массовая замена по всем запускам'
(bulk_replace_all key in i18n) to distinguish from run-level button
- Remove duplicate Bulk Replace button from records table header
in TranslationRunResult (kept only in result header)
- Fix i18n path for bulk_replace_all (was incorrectly in run namespace)
2026-06-19 14:45:43 +03:00
3133e50645
perf: fix translate deadlock, speed, trace_id, UI bugs — fullstack patch
...
## Backend (7 production files + 6 test files)
### P0-2: LLM output truncation cascade fix
- _token_budget.py: OUTPUT_PER_ROW_PER_LANG 120→200, OUTPUT_SAFETY_FACTOR 0.70→0.55
- Prevents finish_reason=length → split → retry cascade (3 calls → 1 call per batch)
- P2-8: added qwen-flash/qwen-plus/qwen-max/qwen-coder to PROVIDER_DEFAULTS
### P1-4/P1-5: EncryptionManager singleton
- encryption.py: get_encryption_manager() process-wide singleton
- llm_provider.py: use singleton instead of new EncryptionManager() per batch
- Eliminates ~90 redundant Fernet key validations per translation run
### P1-6: Cache-hit log aggregation
- _batch_proc.py: one log per batch (batch_rows + cache_hits) instead of per-row
- 1076 log lines → ~30 per run
### P1-7: Timezone-aware datetime fix
- scheduler.py: _ensure_aware() helper for naive DB datetime → UTC-aware
- Fixes TypeError in scheduled translation concurrency check
### P2-9: Connection test timeout
- connection_service.py: asyncio.wait_for(15s) on all dialect tests
- Prevents 2-minute UI hangs from DNS/TCP stalls
### Trace ID propagation
- middleware/trace.py: inject x-trace-id response header via ASGI send wrapper
### Test fixes & integration tests
- test_scheduler.py: AsyncMock for execute_run, mock get_async_job_runner
- test_sql_insert_service.py: AsyncMock for execute_sql
- test_token_budget.py: batch_size 50→45 for new OUTPUT_PER_ROW_PER_LANG=200
- test_encryption.py: +2 singleton tests
- test_scheduler_ensure_aware.py: +4 (naive→aware, passthrough, None, subtraction)
- test_batch_classify_persist.py: +2 cache-hit aggregation tests
- test_connection_service_edge.py: +2 timeout tests
- test_trace_middleware.py: +4 x-trace-id header tests
- test_token_budget.py: +4 qwen-flash/O200 tests
## Frontend (7 production files + 5 test files)
### Trace ID propagation
- api.ts: _captureTraceId() reads x-trace-id → setTraceId() in fetchApi/requestApi/postApi/deleteApi
### Duplicate datasource columns fetch
- ConfigTabForm.svelte: guard availableColumns.length === 0 before fetch
### Admin pages Svelte 5 runes fix
- admin/users/+page.svelte: plain let → () for all template-bound vars
- admin/roles/+page.svelte: same fix
- Both pages were stuck on «Загрузка...» due to mixed reactivity models
### Validation popover positioning
- +page.svelte: pass trigger HTMLElement instead of event
- DashboardHubModel.svelte.ts: toggleValidationPopover(HTMLElement), closeValidationPopover()
- Added X close button + click-outside overlay + i18n
### Test fixes & integration tests
- api.test.ts: mock setTraceId/getTraceId, +3 _captureTraceId tests
- provider_config.integration.test.ts: handleDelete→promptDeleteProvider
- DatasetPreview.test.ts: dashboards/ → ROUTES.dashboards
- test_config_tab_form.svelte.js: +2 columns fetch guard tests (NEW)
- admin-users.test.ts: +3 loading→table tests (NEW)
- admin-roles.test.ts: +2 loading→table tests (NEW)
## Semantic curation
- Removed @COMPLEXITY N from 6 route files + metrics.py (duplicate of [C:N])
- Added [C:N] to 2 orphan child contracts in metrics.py
- Added [C:N] + @BRIEF to 4 frontend anchors
- Fixed #region → # #region consistency in validation_tasks.py
## Verification
- Backend: 608 pytest passed (0 failures)
- Frontend: 2472 vitest passed (128 files, 0 failures)
- Frontend build: ✓ built in 18s
- Browser: dashboards, admin/users, admin/roles, validation popover — all green
2026-06-18 23:54:57 +03:00
