58d06fb287
fix(ssl+agent): capath for all HTTP clients + isolate gradio import
...
SSL fix (ADR-0009 Finding 7):
- Replace ssl.create_default_context() with system_ssl_context(capath)
in client_registry.py, async_network.py, notifications/providers.py,
git/_base.py. Previous fix (0.5.1) only covered LLM clients; the
Superset API client still used ssl.create_default_context() which
loads cafile (flat bundle) where OpenSSL 3.x ignores intermediate CA
certificates. system_ssl_context() uses capath only (hash symlinks).
Agent fix:
- Extract _check_llm_provider_health + _llm_status from agent/app.py
into agent/_llm_health.py. The /api/agent/llm-status endpoint was
importing from agent/app.py which triggers 'import gradio' at module
level. Backend container does not have gradio installed, causing
ModuleNotFoundError (500 error) every 30s on health check polling.
Config:
- Add ALLOWED_ORIGINS to docker-compose.yml + docker-compose.enterprise-clean.yml
ADR-0009 updated: Layer 2 table expanded with 4 missing clients,
Finding 5 reconciled (LLM_CA_CERT_URLS restored in 0.5.1), version 0.5.2.
Verified: 1110 unit tests passed, gradio import isolation confirmed.
2026-07-07 12:18:43 +03:00
1e61f098bd
fix(docker): add missing certs.sh COPY to backend/all-in-one Dockerfile
...
Backend entrypoint sources certs.sh for CA certificate installation,
but the file was never copied into the Docker image — causing
container crash loop on startup (regression in 8fd23f7e
2026-07-07 00:58:41 +03:00
8fd23f7ea1
refactor(ssl): centralize SSL trust management, remove LLM_SSL_VERIFY
...
Centralized SSL via one contract: CERTS_PATH=/opt/certs mounted into all containers.
Backend:
- NEW: backend/src/core/ssl.py — system_ssl_context(), httpx_verify(),
cert_dir_inventory()
- LLMClient._get_ssl_verify() → delegates to core.ssl
- _llm_async_http._get_verify() → delegates to core.ssl
- Removed LLM_SSL_VERIFY env reading from all runtime code
Docker:
- NEW: docker/certs.sh — shared cert installer (PEM/DER/cer to .crt conversion,
update-ca-certificates, hash symlinks, NSS import)
- NEW: docker/agent.entrypoint.sh — agent entrypoint with cert installation
- backend.entrypoint.sh → uses certs.sh instead of install_llm_ca_certs
- Dockerfile.agent → adds ca-certificates, openssl, entrypoint
Compose:
- Removed LLM_CA_CERT_URLS and LLM_SSL_VERIFY from all compose files
- Added CERTS_PATH volume mount to agent (dev + enterprise)
- Added certs volume mount to backend/agent in dev compose
Env examples:
- Removed LLM_SSL_VERIFY, LLM_CA_CERT_URLS from .env.example,
.env.enterprise-clean.example, .env.current.example, .env.master.example,
backend/.env.example
- Enhanced CERTS_PATH comments with accepted formats
Diagnostics:
- diag_container.py: removed LLM_* checks, added CERTS_PATH inventory,
uses core.ssl for context creation
Tests:
- Updated test_llm_analysis_service, test_llm_async_http,
test_client_headers to verify centralized ssl context (no env disable)
- 4/4 SSL tests pass
2026-07-06 21:00:28 +03:00
468bdb9000
fix(agent): resolve missing imports — jose, auth config, AUTH_SECRET_KEY
...
- Added python-jose[cryptography] to requirements-agent.txt
- Made sqlalchemy.orm.Session and TokenBlacklist imports lazy in jwt.py
so decode_token works without pulling ORM models into agent image
- Added src/core/auth/__init__.py, config.py, jwt.py to agent Dockerfile COPY
- Added AUTH_SECRET_KEY env var to agent compose (reads from JWT_SECRET)
2026-07-06 12:13:26 +03:00
ec9bc76a37
cleanup: убрать мёртвые env-переменные, консолидировать чтение в agent/_config.py
...
Удалены из кода:
- JWT_SECRET — мёртвая (decode_token использует AUTH_SECRET_KEY)
- SESSION_SECRET_KEY — заменён на прямой AUTH_SECRET_KEY
- POSTGRES_URL — deprecated fallback, удалён из database.py и reencrypt.py
Консолидировано чтение env-переменных agent-модуля:
- Создан agent/_config.py — единый модуль для FASTAPI_URL,
SERVICE_JWT, GRADIO_*, STORAGE_ROOT, AGENT_* (9 констант)
- Все agent/*.py импортируют из _config вместо разрозненных os.getenv
Удалены or-дефолты (безопасность):
- agent/langgraph_setup.py — удалён hardcoded DB URL postgres:postgres
- agent/langgraph_setup.py — удалены fallback API URL и model name
- scripts/reencrypt.py — удалён hardcoded DB URL postgres:postgres
- plugins/llm_analysis/service.py — удалены or-дефолты URL/app name
.env.example — минимализация:
- backend/.env.example: только 4 обязательные переменные
- root/.env.example: обязательные + docker + SSO/админ
Обновлены тесты (139 passed)
2026-07-04 14:58:43 +03:00
8c10632494
feat(semantic): curator-driven protocol hardening — decision memory + relation repair
...
- Add @RATIONALE/@REJECTED to 103+ C4/C5 contracts across backend core, services, API routes, and frontend models
- Fix 109 unresolved @RELATION edges (Auth.*, SupersetClient.*, AgentChat.*, ADR cross-refs)
- Add 13 @ingroup tags for DSA/HCA attention grouping
- Repair 29 stale graph edges via index rebuild
- Update .kilo agent prompts and skills for GRACE-Poly v2.6 compliance
- Git integration: merge routes, branch lifecycle, remote providers, UX components
- 0 broken anchor pairs, index rebuilt with 0 parse warnings
2026-07-02 08:53:19 +03:00
64564da988
fix(git): fix 17 missing async/await bugs + UX overhaul
...
Backend:
- fix 17 missing 'await' in git route handlers causing silent no-ops
(branches, diff, history, commit, push, pull, merge, promote, sync)
- fix async coroutine passed to run_blocking in git_plugin.py
Frontend:
- add collapsible 'How it works' onboarding (GitHelpPanel)
- add status legend with color-coded repository statuses
- i18n: add 50+ missing keys, replace hardcoded strings
- add Refresh button in modal header
- add PROD deploy confirmation dialog (replaces browser prompt())
- add CommitHistory to workspace tab with timeline nodes
- add post-commit success banner with next-step guidance
- increase success toast duration to 8s
- group local/remote branches in selector (optgroup)
- format last_modified dates timezone-aware
- change PROD badge from red to neutral indigo
- extract shared resolveGitStatusToken to git-utils.ts
- fix 'slug' label regression
- remove dead init_repo_button key
UI/UX audit fixes:
- add descriptions to Create/Init buttons in init panel
- add actionable CTA to server mismatch warning
- improve checkbox text phrasing
2026-07-01 20:47:25 +03:00
60674c8639
fix(logging): defensive trace_id seed + falsy error/payload in CotJsonFormatter
...
log_requests middleware (BaseHTTPMiddleware) showed no-trace
because anyio.create_task_group() in Starlette 0.50.0 does not
always propagate ContextVar from raw ASGI TraceContextMiddleware.
Fix 1: defensive get_trace_id() check at log_requests entry —
if empty, seed_trace_id() to ensure every request has one.
Fix 2: CotJsonFormatter used 'if error:' and 'if payload:' which
silently drop empty strings (str(e)='') and empty dicts.
Changed to 'is not None' checks — preserves all data.
Root cause: 12 EXPLORE-without-error entries from belief_scope's
exception handler where Exception() has empty message.
2026-06-30 17:41:13 +03:00
79d0f8f678
feat(agent-superset): extend SupersetClient with agent-critical methods + DDL/DML guard + tests (126 passed)
...
Ported from mcp-superset research module, integrated into existing async SupersetClient:
New mixins (4 files):
- safety.py: DDL/DML guard (13 keywords), comment/string stripping, viz_type validation
- _sql_lab.py: execute_sql, format_sql, results, estimate, CSV export, query history
- _saved_queries.py: saved queries CRUD (5 methods)
- _audit.py: permissions audit matrix (user × dashboard × dataset × RLS)
Extended mixins (4 files):
- _dashboards_write.py: +create, +update (general), +copy, +publish, +unpublish
- _dashboards_crud.py: +standalone get_dashboard_charts/datasets, +get_dashboard
- _datasets.py: +create, +delete, +duplicate, +refresh_schema, +get_or_create, +export/import
- _databases.py: +update, +test_connection, +schemas/tables/catalogs, +validate_sql, +select_star, +table_metadata
FastAPI proxy (2 files):
- agent_superset.py (284L): SQL Lab + Dashboards/Datasets write endpoints
- agent_superset_explore.py (240L): DB explore + Audit + Saved Queries endpoints
Agent tools (2 files):
- tools.py: +7 LangChain @tools (24 total), intent-routing keywords
- _tool_resolver.py: updated SAFE/GUARDED/FAST_CONFIRM classification sets
Tests (3 files, 126 tests):
- test_superset_safety.py (51): DDL/DML bypass/legitimate/safe scenarios
- test_superset_extended.py (42): all mixin methods with mock AsyncAPIClient
- test_superset_tools.py (33): agent tool registration, intent matching, @tool .ainvoke()
2026-06-30 17:26:51 +03:00
a0619ca049
fix(agent): unify logging API + add molecular-cot coverage to agent module
...
Phase 1 — API unification:
- Replace direct log() from cot_logger with canonical logger.reason/reflect/explore
across _confirmation.py, _persistence.py, _tool_resolver.py, app.py
Phase 2 — Gap filling:
- tools.py: add REASON/REFLECT/EXPLORE to 17 C3 tool functions (was 0 logs)
- app.py agent_handler (C4): add lifecycle REASON on entry, REFLECT on exit,
EXPLORE on OutputParserException + general exception
- langgraph_setup.py create_agent (C4): add REASON with model/config_source,
EXPLORE on env-var/InMemorySaver fallback, REFLECT on graph compilation
- _tool_resolver.py infer_tool_from_text (C3, 13 branches): REASON on inference
- _persistence.py: REFLECT on save_conversation success, EXPLORE on prefetch
Phase 3 — Plain log migration:
- middleware.py: plain logger.info() -> logger.reason()
- run.py: 7 plain logger.info/warning/error -> molecular REASON/EXPLORE
Phase 4 — Cleanup:
- cot_logger.py: deprecate MarkerLogger (@DEPRECATED + @REPLACED_BY)
- molecular-cot-logging SKILL.md: remove cot_span Section IV (never implemented),
renumber sections V-VII -> IV-VI, add cot_span rejection rationale
Verification: pytest 27/27, axiom rebuild 5844/2993/0 warnings
2026-06-30 15:48:46 +03:00
12678c637b
fix(agent-chat): streaming state leak, document parser magic bytes, HITL flow
...
### Bugfixes — Agent Chat 'Думаю' State Leak
- fix(agent-chat): loadHistory() now resets streamingState/idle + cancels stale
submission — prevents 'Думаю' state leak across conversation switches
- fix(agent-chat): onDisconnected/onDisconnectedPermanent cascade to
streamingState — prevents permanent hang on connection loss during stream
- fix(agent-chat): guard on isLoadingHistory — prevents false commit
of 'agent unavailable' fallback when switching conversations
- fix(agent-chat): remove race in _sendNow empty-response check vs Svelte
microtask (duplicate logic removed, handles correctly)
- fix(stream-processor): confirm_resolved now appends msg.text to partialText
instead of dropping it
### Bugfixes — Backend PDF Upload
- fix(document-parser): _detect_format_by_magic() — reads file header magic
bytes as fallback when Gradio loses filename
- fix(document-parser): improved name extraction — tries orig_name, path stem
- fix(document-parser): @RELATION AgentChatTypes -> AgentChat.Types
### HITL Flow & Agent Chat Improvements
- feat(agent): HITL resume confirm/deny with userId/userJwt/envId propagation
- feat(agent): confirm_required metadata fallback via aget_state() after
'Event loop is closed' error during interrupt
- feat(agent): interrupt_before re-enabled via AGENT_CONFIRM_TOOLS env var
- feat(frontend): debug panel with connection/stream state monitoring
- feat(frontend): AgentChatModel constructor options + onBeforeSend callback
- feat(frontend): crypto.randomUUID() for local conversation ID on first send
### Backend Agent Refactoring
- refactor(agent): langgraph_setup — monkey-patch for PydanticSerializationError
- refactor(agent): tools.py — dual identity headers, expanded tool set
- refactor(agent): run.py — _find_free_port, Gradio server port fallback
- refactor(agent): app.py — file size validation, message truncation, HITL path
### Frontend
- feat(dashboard-hub): DashboardHubModel with filters, pagination, git actions
- feat(ui): DateRangeFilter component
- feat(i18n): new dashboard keys; cache tooltips fix
- fix(i18n): full run tooltips — cache is NOT ignored
### Semantic Protocol
- chore(agents): update all agents with canonical format
- chore(skills): sync semantics-core, semantics-contracts, molecular-cot-logging
### Housekeeping
- chore: remove stale semantic reports (10 files, Jan 2026)
- chore: update 033-gradio-agent-chat specs, contracts, UX, tasks, tests
- chore: add .agents/ directory (mirrors .opencode/ agent layouts)
- chore: update run.sh with DEV_MODE, port management
2026-06-29 17:15:25 +03:00
3133e50645
perf: fix translate deadlock, speed, trace_id, UI bugs — fullstack patch
...
## Backend (7 production files + 6 test files)
### P0-2: LLM output truncation cascade fix
- _token_budget.py: OUTPUT_PER_ROW_PER_LANG 120→200, OUTPUT_SAFETY_FACTOR 0.70→0.55
- Prevents finish_reason=length → split → retry cascade (3 calls → 1 call per batch)
- P2-8: added qwen-flash/qwen-plus/qwen-max/qwen-coder to PROVIDER_DEFAULTS
### P1-4/P1-5: EncryptionManager singleton
- encryption.py: get_encryption_manager() process-wide singleton
- llm_provider.py: use singleton instead of new EncryptionManager() per batch
- Eliminates ~90 redundant Fernet key validations per translation run
### P1-6: Cache-hit log aggregation
- _batch_proc.py: one log per batch (batch_rows + cache_hits) instead of per-row
- 1076 log lines → ~30 per run
### P1-7: Timezone-aware datetime fix
- scheduler.py: _ensure_aware() helper for naive DB datetime → UTC-aware
- Fixes TypeError in scheduled translation concurrency check
### P2-9: Connection test timeout
- connection_service.py: asyncio.wait_for(15s) on all dialect tests
- Prevents 2-minute UI hangs from DNS/TCP stalls
### Trace ID propagation
- middleware/trace.py: inject x-trace-id response header via ASGI send wrapper
### Test fixes & integration tests
- test_scheduler.py: AsyncMock for execute_run, mock get_async_job_runner
- test_sql_insert_service.py: AsyncMock for execute_sql
- test_token_budget.py: batch_size 50→45 for new OUTPUT_PER_ROW_PER_LANG=200
- test_encryption.py: +2 singleton tests
- test_scheduler_ensure_aware.py: +4 (naive→aware, passthrough, None, subtraction)
- test_batch_classify_persist.py: +2 cache-hit aggregation tests
- test_connection_service_edge.py: +2 timeout tests
- test_trace_middleware.py: +4 x-trace-id header tests
- test_token_budget.py: +4 qwen-flash/O200 tests
## Frontend (7 production files + 5 test files)
### Trace ID propagation
- api.ts: _captureTraceId() reads x-trace-id → setTraceId() in fetchApi/requestApi/postApi/deleteApi
### Duplicate datasource columns fetch
- ConfigTabForm.svelte: guard availableColumns.length === 0 before fetch
### Admin pages Svelte 5 runes fix
- admin/users/+page.svelte: plain let → () for all template-bound vars
- admin/roles/+page.svelte: same fix
- Both pages were stuck on «Загрузка...» due to mixed reactivity models
### Validation popover positioning
- +page.svelte: pass trigger HTMLElement instead of event
- DashboardHubModel.svelte.ts: toggleValidationPopover(HTMLElement), closeValidationPopover()
- Added X close button + click-outside overlay + i18n
### Test fixes & integration tests
- api.test.ts: mock setTraceId/getTraceId, +3 _captureTraceId tests
- provider_config.integration.test.ts: handleDelete→promptDeleteProvider
- DatasetPreview.test.ts: dashboards/ → ROUTES.dashboards
- test_config_tab_form.svelte.js: +2 columns fetch guard tests (NEW)
- admin-users.test.ts: +3 loading→table tests (NEW)
- admin-roles.test.ts: +2 loading→table tests (NEW)
## Semantic curation
- Removed @COMPLEXITY N from 6 route files + metrics.py (duplicate of [C:N])
- Added [C:N] to 2 orphan child contracts in metrics.py
- Added [C:N] + @BRIEF to 4 frontend anchors
- Fixed #region → # #region consistency in validation_tasks.py
## Verification
- Backend: 608 pytest passed (0 failures)
- Frontend: 2472 vitest passed (128 files, 0 failures)
- Frontend build: ✓ built in 18s
- Browser: dashboards, admin/users, admin/roles, validation popover — all green
2026-06-18 23:54:57 +03:00
4dce669844
fix(tests): 61 failed backend unit tests — async/await mocks, deadlock fix, SyntaxError repairs
...
Группы исправлений:
- Группа 1 (async/await misuse): MagicMock → AsyncMock для get_dashboards,
export_dashboard, import_dashboard, sync_environment, get_run_detail,
list_all_runs, create_task и др. — 23 теста
- Группа 2 (runner.run deadlock): добавлены моки get_async_job_runner +
IdMappingService/AsyncSupersetClient в migration plugin + API tests
для предотвращения вечной блокировки future.result() — 16 тестов
- Группа 3 (SyntaxError): исправлены 7 незакрытых скобок ')' в
test_validation_tasks_comprehensive.py (QA-агент оставил AsyncMock
без закрывающих скобок)
- Группа 4 (mock verification): logger mock error→explore, scheduler tests
skipped (удалён из production), dataset mapper — 11 тестов
- Группа 5 (search/assistant): MagicMock → AsyncMock — 9 тестов
- Группа 6 (extractor parsing): AsyncMock для async методов — 9 тестов
Итого: 61 ранее FAILED → 274 passed, 4 skipped, 0 failed
2026-06-18 14:41:13 +03:00
4726fd110d
fix(core): centralize async/sync bridge for APScheduler scheduled jobs
...
Create AsyncJobRunner — centralized bridge between APScheduler
(BackgroundScheduler, sync thread pool) and async coroutines.
Fixes:
- P0: execute_run() called without await from APScheduler thread,
causing coroutine to be silently discarded (root cause: no
translation history)
- P0: get_async_job_runner() deadlock when called from APScheduler
thread pool without running event loop
- P1: ID mismatch in disable_schedule/delete_schedule routes
(job_id passed instead of schedule_id)
- P1: asyncio.run() in APScheduler callbacks incompatible with
running event loop
- Delete unused llm_analysis/scheduler.py (not used in production)
Changes:
core/async_job_runner.py — new: AsyncJobRunner class
core/scheduler.py — use runner.run()/run_later()
translate/scheduler.py — use runner.run() for execute_run
mapping_service.py — remove unused BackgroundScheduler
dependencies.py — add get_async_job_runner() DI
app.py — init runner in lifespan
api/routes/migration.py — use runner.run()
_schedule_routes.py — fix ID mismatch
plugins/migration.py — use runner.run()
llm_analysis/scheduler.py — delete (unused)
tests: 151 new/updated tests, all passing
2026-06-17 16:22:30 +03:00
ec6421de35
rename ss-tools to superset-tools across the entire project
...
- Replace all occurrences of 'ss-tools' with 'superset-tools' in 104 files
- Rename git bundle file ss-tools.bundle → superset-tools.bundle
- Update .gitignore pattern accordingly
- Preserve variable names (hasSsTools etc.) and code identifiers
2026-06-16 11:15:19 +03:00
fd27569488
test: final 6 agents — 172+ translate tests, llm_analysis 89%, git_plugin 100%, migration/deps/routes polished. 85-94% files pushed to 95%+. Bulk: backup/debug/maintenance plugins
2026-06-15 22:04:40 +03:00
f75c15dbc6
test: massive coverage expansion — 15 new test modules + assistant tool fixes + orthogonal testing
...
- 10 translate plugin test files (100% coverage on 12 modules)
- assistant/handler tools: 85+ tests covering dispatch, registry, resolvers, routes, llm_planner, 13 tool handlers
- clean release: artifact_catalog_loader, mappers, approval, publication tests
- API routes: translate_helpers, validation_service extensions, datasets to 100%
- notifications: providers/service tests
- services: profile_preference_service
- docs/orthogonal-test-report.md — full speckit.tests audit
- Fixes: 3 git_base async mock failures, 4 assistant handler permission-check patches
- .gitignore: coverage artifacts
2026-06-15 15:38:59 +03:00
27a20cbbeb
fix(ssl): replace verify=True with ssl.create_default_context() for corporate CA support
...
Core fix: all httpx.AsyncClient instances now use ssl.create_default_context()
instead of bool verify=True, which uses certifi and ignores system CA store.
This makes corporate CA certificates installed via update-ca-certificates
visible to Python HTTP clients.
Files:
- async_network.py: AsyncAPIClient.__init__ converts True→SSLContext
- client_registry.py: get_client converts bool→SSLContext before passing
- notifications/providers.py: _get_http_client uses ssl.create_default_context()
- services/git/_base.py: GitServiceBase uses ssl.create_default_context()
- translate/_llm_async_http.py: _get_verify returns SSLContext (not string)
Test: new integration test with 3-tier PKI (Root→Intermediate→Server),
TLS-protected Superset container, and custom CA installation via
update-ca-certificates or SSL_CERT_DIR fallback.
- conftest.py: added ca_chain, install_custom_ca, superset_tls_env fixtures;
parameterized superset_container for TLS mode
- test_superset_tls_custom_ca.py: 8 tests (openssl -CApath, -CAfile certifi,
httpx capath, httpx certifi, AsyncAPIClient, SupersetClient, fingerprint, verify=False)
2026-06-15 10:32:17 +03:00
55604498ae
feat(translate): add insert_method dispatch and model/schema updates for direct DB
2026-06-11 19:12:35 +03:00
1f9040d53e
feat(core): implement ConnectionService and DbExecutor for direct DB insert (US11-US12)
...
- ConnectionService: CRUD for DatabaseConnection in GlobalSettings with
EncryptionManager-backed password encryption/decryption, test connectivity
- DbExecutor: asyncpg (PostgreSQL) and clickhouse-connect (ClickHouse) drivers
with per-connection connection pooling
- config_models.py: promote GlobalSettings.connections from list[dict] stub
to list[DatabaseConnection] with full Pydantic model validation
- settings.py: +5 CRUD endpoints for connections (create, read, update, delete, test)
- requirements.txt: add asyncpg>=0.29.0, clickhouse-connect>=0.7.0
- seed_permissions.py: register settings.connections.manage permission
2026-06-11 19:12:15 +03:00
7760214170
test(agent): extend coverage — agent handler, confirmations, conversation API, tools
...
- test_agent_handler: additional edge cases for streaming, HITL, file upload
- test_confirmations: HITL confirm/deny lifecycle coverage
- test_conversation_api: conversation save/load persistence tests
- test_langchain_tools: tool registration, dual-auth header propagation
- ConversationList.test.ts (frontend): conversation list component tests
- conftest: shared fixtures for agent tests
- task_manager/manager: minor fixes from test coverage
- tasks.md/test-documentation.md: spec and test documentation updates
- speckit.test.md: speckit workflow documentation update
2026-06-10 16:38:06 +03:00
143f14d516
chore: remainder — backend test infra, agent config, docker, i18n, frontend ui
...
- Backend: alembic env, config manager/models, dependencies, translate plugin
- Backend tests: async_sync_regression, integration tests, git services, test_agent
- Docker: docker-compose.yml updates
- Agent: qa-tester.md update, semantics-testing SKILL.md update
- Frontend: TopNavbar, sidebarNavigation, FeaturesSettings, FeatureGate
- i18n: assistant.json en/ru locale updates
- New: frontend/src/lib/components/agent/ directory
2026-06-10 15:06:36 +03:00
8e8a3c3235
feat: attention-optimized semantic protocol v2.7
...
Core changes:
- Add @defgroup/@ingroup to 1791 C2+ contracts (555 files) for HCA 128× pre-training DSA grouping
- Add §0.1 Pre-Training Frequency matrix to semantics-core
- Add §VIII Attention Architecture rules (ATTN_1-4) with MLA/CSA/HCA/DSA mechanics
- Add @defgroup/@ingroup to canonical syntax (§II) and all contract examples
Agent prompts (5 files):
- Add ZERO-STATE RATIONALE with MLA/CSA/HCA/DSA compression mechanics
- Add pre-training note: @RATIONALE/@REJECTED are in-context learned tags
- svelte-coder: add missing #region contract, fix Svelte rule violations
- python-coder/fullstack-coder: honor function contracts from speckit plan
- qa-tester: add attention compliance audit (P3 ATTN_1-4 checks)
Skills (6 files):
- Translate all axiom_config descriptions to English
- Fix doc_dirs to index .opencode/ and .specify/
- Deduplicate 5× complexity_rules → single global_tags catalog
- Reduce semantics-svelte 591→485 lines (remove duplicate code blocks)
- Fix semantics-testing: 'Short IDs' → 'Short hierarchical IDs'
- Fix all examples: flat IDs → hierarchical Domain.Name format
- Fix Svelte examples: replace raw Tailwind + <button> with semantic tokens + /ui
Speckit workflow (commands + templates):
- speckit.plan: add Function-Level Contracts for C3+ with @PRE/@POST/@TEST_EDGE
- speckit.plan: add Attention Compliance Gate (ATTN_1-4 before contract generation)
- speckit.tasks: add function contract inlining format (constraints in task description)
- speckit.specify: load semantics-core for spec density rules
- spec-template: add #region contract, @SEMANTICS grouping, hierarchical IDs
- ux-reference-template: add #region wrapper
- plan-template: add attention gate, @defgroup/@ingroup guidance
- tasks-template: add attention audit + rebuild + orphan check tasks
- constitution.md: translate to English, add Principle VIII (attention-optimized contracts)
Reference modules rewritten (hierarchical IDs + full contracts):
- Auth.Jwt: 6 child contracts with @RATIONALE/@REJECTED/@TEST_EDGE
- Api.Auth: 5 endpoints with @TEST_EDGE + molecular CoT markers
- Migration.Model: @defgroup Migration with 18 @ACTION + 6 @INVARIANT
Scripts:
- add_defgroup_ingroup.py: zero-risk additive @ingroup migration (1791 insertions)
- migrate_hierarchical.py: flat→hierarchical ID dry-run analysis (792 contracts)
- merge_prompts.py: merge all prompts/skills/commands into one review file
Config:
- axiom_config.yaml: 749→395 lines (-47%), English, doc_dirs include prompts
- Fix test_datasets.py import collision (rename → test_datasets_routes.py)
- Fix test_preview.py: SupersetClient→get_superset_client, AsyncMock, logger f-string
2026-06-08 16:30:59 +03:00
f731a53c7d
038: add @RATIONALE/@REJECTED contracts to async C4/C5 modules
...
5 contracts updated:
- AsyncNetworkModule (C5): async migration rationale, per-client CSRF cookie rejection
- AsyncAPIClient (C4): auth lifecycle, cache-hit CSRF refresh
- AsyncAPIClient.request (C4): string vs dict handling, double-encoding root cause
- AsyncAPIClient.upload_file (C4): multipart async upload
- LLMAsyncHttpClient (C4): response.ok -> is_success, module-level httpx singleton
2026-06-05 17:02:30 +03:00
fbe0ba122c
037: fix 99 failing tests — missing await after async migration
...
Fixed async/sync boundary bugs across 14 test files. Root cause:
async def methods called without await in sync test functions.
Fixed files:
- test_translate_jobs.py (10): create_job/get_job/update_job/delete_job
- test_translate_scheduler.py (5): create_schedule/update/delete
- test_datasets.py (14): AsyncMock + corrected patch target
- test_mapping_service.py (11): sync_environment + MockSupersetClient
- test_defensive_guards.py (6): GitService/SupersetClient guards
- test_maintenance_service.py (29): all 6 maintenance services
- test_dry_run_orchestrator.py (1): run() without await
- test_dashboards_api.py (23): registry client via AsyncMock
- test_validation_tasks.py (4): trailing slash in POST URL
- test_superset_matrix.py (3): AsyncMock for compile_preview
- test_payload_reduction.py (6): LLMClient._optimize_image wrapper
- test_compliance_task_integration.py (2): event_bus ref
- test_smoke_plugins.py (1): flusher_stop_event fallback
- test_task_manager.py (1): _flusher_stop_event/thread fallback
Remaining 31 failures in test_task_manager.py (29) and
test_smoke_plugins.py (1) are pre-existing async migration gaps
(_flusher_stop_event moved to event_bus), not from this PR.
2026-06-05 15:43:35 +03:00
7f1937f10b
036: wire SupersetClientRegistry into translate + services flow
...
Replaces direct SupersetClient(env) calls with shared
get_superset_client(env) from client_registry.
Changed files (9):
- client_registry.py: added get_superset_client(), _build_env_id(),
accepts Environment models (not just dicts), fixed async_client attr
- superset_executor.py: _get_client() now async, uses shared client
- preview_executor.py, _run_source.py, service.py, service_datasource.py
- health_service.py, resource_service.py, debug.py
Impact per environment:
- 6 separate httpx.AsyncClient instances → 1 shared client
- 6 CSRF cookie fetches → 1 (on first access)
- 6 connection pools → 1 shared pool
- Shared semaphore for backpressure
- Shared cookie jar (fixes CSRF 'tokens do not match' on SQL Lab execute)
2026-06-05 15:14:44 +03:00
4cef6af041
fix
2026-06-05 15:01:34 +03:00
5d9d214bd6
034: fix double JSON serialization in AsyncAPIClient.request
...
Root cause: AsyncAPIClient.request() passes its parameter directly
to httpx.AsyncClient.request(json=data). When callers pass a pre-serialized
JSON string (data=json.dumps(dict)), httpx re-encodes it via json.dumps(),
resulting in a double-encoded JSON string body instead of a JSON object.
This caused ALL POST/PUT requests with string data to fail — Superset received
a JSON string instead of a JSON object, returning GENERIC_BACKEND_ERROR
('dictionary update sequence element #0 has length 1; 2 is required').
Fix: if data is a string, pass it via httpx parameter (raw body);
if it's a dict/list, pass via for automatic encoding.
Affected callers (6 files) now correctly send JSON objects:
- preview_executor.py: chart data requests
- superset_executor.py
- _run_source.py
- _datasets.py: update_dataset
- _datasets_preview.py: compile_dataset_preview
- _dashboards_write.py
Also simplified preview_executor.fetch_sample_rows back to single-strategy
(chart data API only) since the root cause is now fixed.
2026-06-05 12:07:55 +03:00
250b69db8c
032: fix coroutine never awaited in debug.py + task_logger.py
...
debug.py: _test_db_api and _get_dataset_structure were already async def
but called SupersetClient methods (authenticate, get_databases, get_dataset)
without await. Added await to 4 calls.
task_logger.py: _add_log callback is async def but _log() called it without
await, silently dropping all task log messages (RuntimeWarning: coroutine
never awaited). Changed to fire-and-forget via asyncio.ensure_future when
a running event loop is available, drops gracefully otherwise.
2026-06-05 11:23:53 +03:00
4247e7a20e
032: fix final sync→async cascade (dataset_review, parsing, validation)
...
- _parsing.py: parse_superset_link + _recover_dataset_binding → async
(await get_dashboard_detail, get_chart)
- dataset_review/orchestrator.py: start_session + _build_recovery_bootstrap → async
(await get_dataset_detail, parse_superset_link)
- _routes.py (dataset_review): await orchestrator.start_session()
- validation_tasks.py: await parse_superset_link + get_dashboard_detail
2026-06-05 09:55:34 +03:00
c7d8f4431e
032: fix remaining sync→async propagation (17 call sites)
...
Core fixes:
- service_datasource.py: fetch_datasource_metadata() → async
- service.py: create_job(), update_job() → async (callers await)
- _job_routes.py: await create_job/update_job
Maintenance scanners:
- _dashboard_scanner.py: 4 functions → async (find_affected, _get_linked,
_apply_filters, _resolve_title)
- _chart_manager.py: 3 functions → async
- _banner_renderer.py: rebuild_banner → async
- _orchestrators.py: 3 orchestrators → async
- maintenance_banner.py: await async calls
Migration:
- dry_run_orchestrator.py: run(), _build_target_signatures() → async
- risk_assessor.py: build_risks() → async
- migration.py: await service.run()
- mapping_service.py: sync_environment() → async
Dead code:
- _helpers.py: _find_dashboard_id_by_slug marked DEPRECATED
2026-06-05 08:56:37 +03:00
079b2dd37b
032: add aclose() to SupersetClient (was missing, used in dashboard detail routes)
...
SupersetClientBase was missing aclose() method. AsyncSupersetClient had it
via override, but code creating SupersetClient directly (e.g. dashboard tasks
history route) would fail with AttributeError on await client.aclose().
2026-06-05 08:33:49 +03:00
5eb116509b
032: dead code cleanup — remove sync APIClient, _llm_http, preview_llm_client, fix retry chains
2026-06-05 08:31:18 +03:00
ee9123bcf2
032: deep async propagation — orchestrator, insert, mapper, batch chains
...
Full async conversion for all sync callers of async SupersetClient methods:
orchestrator_sql.py: generate_and_insert_sql + _resolve_dialect → async
orchestrator_run_completion.py: complete_success → async (calls generate_and_insert_sql)
orchestrator_exec.py: execute_run → async (awaits complete_success)
orchestrator_runner.py: execute_run → async (delegates to engine)
orchestrator.py: execute_run + _generate_and_insert_sql → async
executor.py: _insert_batch_to_target → async (awaits batch insert)
_batch_proc.py: insert_batch_to_target → async
_batch_insert.py: insert_batch_to_target + _resolve_insert_backend + _execute_insert_sql → async
dataset_mapper.py: get_sqllab_mappings + run_mapping → async
+ await get_dataset, update_dataset, execute_and_poll
mapper.py: await on run_mapping + resolve_database_id calls
_run_routes.py: threading.Thread → asyncio.create_task (_background_execute async)
2026-06-05 00:13:01 +03:00
b190414747
032: final — tombstones, tests, cleanup
...
T055: APIClient tombstone in network.py
T056: AsyncSupersetClient @DEPRECATED marker
T057: _llm_http.py + preview_llm_client.py tombstone
T005-T006: AsyncAPIClient + semaphore tests
T020-T021: SupersetClient concurrency + rejected-path tests
network.py cleaned from 584 to 220 lines (orphan code removed)
All 20 async tests pass.
2026-06-04 20:57:20 +03:00
2e95971b1b
032: fix tests — AsyncMock for async SupersetClient methods
...
All 10 preview pipeline tests pass.
2026-06-04 20:42:31 +03:00
3bb5e6dde7
032: final — async_superset_client collapse, profile/superset_lookup async
2026-06-04 20:37:55 +03:00
5ca1131ba3
032: Phase 5 (T036-T039) + Phase 6 (T040-T044) completed
...
T036: superset_compilation_adapter fully async
T037: fileio.py async wrappers (aiofiles+run_blocking)
T038-T039: tests for plugins fileio concurrency
T040: providers async (aiosmtplib, httpx.AsyncClient)
T041: dispatch_report parallel via asyncio.gather
T042: TaskManager ThreadPoolExecutor->asyncio.create_task
T043: EventBus asyncio.Queue(maxsize=10000)
T044: lifecycle async context manager
Remaining: T045 git/_base.py, T046-T048 tests, T005-T020-T021 tests
2026-06-04 20:30:43 +03:00
51afbee470
032: Phase 3 US1 — 13 mixins migrated to async + dashboard routes
...
T011-T017: All SupersetClient mixins now async.
T018: _detail_routes.py uses registry/AsyncSupersetClient.
T019: Partial — routes/environments, settings, profile, listing async.
RATIONALE: Big-bang merge of sync+AsyncSupersetClient.
REJECTED: dual-stack.
Remaining T019: assistant/*, migration, datasets, git helpers still use sync.
2026-06-04 19:55:43 +03:00
496584e0da
032: Phase 1-2 — setup deps + AsyncAPIClient extend + client_registry + executors
...
Phase 1 (Setup):
- T001: requirements-dev.txt with pytest-httpx
- T002: aiofiles added to requirements.txt
- T003: aiosmtplib added to requirements.txt
- T004: EnvironmentConfig extended (connection_pool_size, etc.)
+ AppAsyncRuntimeConfig created (executor workers, shutdown)
Phase 2 (Foundational):
- T007: AsyncAPIClient extended — semaphore parameter, request() method
- T008a: SupersetClientRegistry — singleton per-env client/semaphore/lock
- T008b: run_blocking helper + bounded executors (db/file/git)
RATIONALE: httpx.AsyncClient replaces requests.Session; singleton
registry ensures global per-env semaphore; named executors prevent
thread pool exhaustion.
REJECTED: asyncio.to_thread (default executor, no backpressure);
per-request clients (lose pooling); dual-stack (rejected at clarify).
2026-06-04 19:45:57 +03:00
ee5ebf08de
fix(backend): migrate trace middleware to raw ASGI for contextvar isolation
...
BaseHTTPMiddleware (Starlette 0.50.0) uses anyio.create_task_group() internally,
creating separate asyncio tasks for dispatch vs call_next. ContextVars set in
dispatch() were not visible to outer middleware like log_requests.
Converting to raw ASGI middleware ensures trace_id is seeded in the root task
context, visible to ALL middleware layers.
Key changes:
- Replace BaseHTTPMiddleware with raw ASGI __call__(self, scope, receive, send)
- UUID v4 validation: check parsed.version == 4 explicitly instead of relying
on uuid.UUID(hex=..., version=4) which silently mutates non-v4 UUIDs
- Add @RATIONALE and @REJECTED tags per semantics-core protocol
- Update app.py comment to document the architectural decision
2026-06-04 16:15:40 +03:00
1e6ec34c2b
feat(backend): drop dictionary dialect columns, add allowed_languages
...
- Removes source_dialect and target_dialect from TerminologyDictionary model,
schemas, routes, helper serialization, and all tests
- Adds allowed_languages config field (BCP-47 language codes) to GlobalSettings
with validation, consolidated settings response, and API endpoint
- Adds alembic migration f1a2b3c4d5e6 to drop the two columns
2026-06-03 11:48:03 +03:00
4c5da7e4d9
alembic fix
2026-06-01 16:34:07 +03:00
d2b53c2a79
feat(validation): v2 LLM dashboard validation — plugin, routes, services
...
Core implementation of the v2 LLM dashboard validation pipeline:
- LLM plugin with Path A (screenshots) and Path B (logs-only) execution
- Validation task management (CRUD, schedule, run)
- WebSocket task progress with Python 3.13 asyncio fix
- Cross-task runs listing (GET /validation-tasks/runs/all)
- RecordResponse schema for validation records
- JSON prompt helper, per-dashboard status aggregation
- Prompt templates with docs/git-commit/validation presets
- Migration: v2 validation models + description column
- Tests: plugin persistence, prompt templates, batch, payload, url
2026-05-31 22:32:20 +03:00
e17f4f64a9
websocket add
2026-05-31 17:17:30 +03:00
55c7e323c4
fix: test assertions for molecular CoT markers, separate propagate=False to ConfigManager, fix ruff in env.py
2026-05-27 11:26:14 +03:00
143ab15744
fix: logging overhaul — Alembic fileConfig disable_existing_loggers=False, propagate=False on loggers, ADR docs, catch-up migration for missing columns, sqlparse dep, build archive naming
2026-05-27 11:19:12 +03:00
614bf9123a
log fix
2026-05-27 10:49:06 +03:00
073a7da9ec
fix: change reason() and reflect() from DEBUG to INFO level
...
Per molecular CoT protocol: REASON and REFLECT are primary reasoning
bonds and must be visible at INFO level in production. DEBUG is
reserved for high-frequency loops.
- reason(): self.debug() → self.info()
- reflect(): self.debug() → self.info()
- explore(): stays at warning() (WARNING level)
- Also removed redundant logger.info() calls in lifespan (now reason()
itself provides visible startup logging)
2026-05-27 10:09:12 +03:00
4205618ee6
chore: eliminate all deprecation warnings from tests and linter
...
Warnings fixed:
- datetime.utcnow() → datetime.now(UTC) across 48+ files (src/ + tests/)
- datetime.utcnow (callback ref) → lambda: datetime.now(UTC) in model fields (18 files)
- Pydantic class Config → model_config = ConfigDict(...) (16 files)
- Pydantic .dict() → .model_dump() (8 files)
- ConfigDict(allow_population_by_field_name=True) → validate_by_name=True
- SQLAlchemy declarative_base() import path updated
- FastAPI on_event → lifespan context manager (app.py)
- Import sorting (ruff I001) auto-fixed across all files
- Fixed broken re-export chains that ruff F401 cleanup broke:
_validate_bcp47: service.py now imports from dictionary_validation directly
job_to_response: _job_routes.py and test imports from service_utils directly
fetch_datasource_metadata: restored re-export in service.py
- Added missing TranslateJobService import in _job_routes.py (was deleted by F401)
- Added ConfigDict(protected_namespaces=()) for DashboardDatasetItem schema field
- pytest.ini: replaced deprecated importmode with asyncio_mode
All 440 tests pass with zero deprecation warnings.
2026-05-26 19:18:28 +03:00
