busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
660 Commits 3 Branches 0 Tags
4e6bfa8549a7bed46c48d5193de5336659672619
Commit Graph

371 Commits

Author SHA1 Message Date
busya
4e6bfa8549 test: 8 parallel agents — fix 150+ failures, add 30+ test files. schemas/models 99-100%, core/client_registry ~98%, maintenance 95-98%, git edges 97%, translate 98-100%, dashboard routes 95-96%, dataset_review 100% 2026-06-15 17:31:43 +03:00
busya
a18f19064f test: 6 agents — +52 test files across core, task_manager, translate routes, git/storage/migration routes, dataset_review deps/routes, settings. Fixed 4 failures 2026-06-15 17:08:08 +03:00
busya
c8e44a1b86 test: 6 parallel agents — +40 test files across core, agent, translate, services, API routes, dataset_review. core 100%, agent 100%, services 100%, translate plugin mostly done. Pending: ~10 minor failures to fix 2026-06-15 16:45:49 +03:00
busya
a20879fa37 test: +12 test modules — clean_release routes, gitea routes, dashboard detail, candidate_service, compliance_orchestrator, clarification_engine/orchestrator, dataset_review helpers. Fix 18 failures (assistant tools, maintence, dataset_review, approval, publication) 2026-06-15 16:26:42 +03:00
busya
fc7e6c3c15 test: add 7 more test modules — assistant cmd parser, history, dispatch, admin routes, dataset review, maintenance + semantic resolver 2026-06-15 16:06:18 +03:00
busya
f75c15dbc6 test: massive coverage expansion — 15 new test modules + assistant tool fixes + orthogonal testing 
- 10 translate plugin test files (100% coverage on 12 modules)
- assistant/handler tools: 85+ tests covering dispatch, registry, resolvers, routes, llm_planner, 13 tool handlers
- clean release: artifact_catalog_loader, mappers, approval, publication tests
- API routes: translate_helpers, validation_service extensions, datasets to 100%
- notifications: providers/service tests
- services: profile_preference_service
- docs/orthogonal-test-report.md — full speckit.tests audit
- Fixes: 3 git_base async mock failures, 4 assistant handler permission-check patches
- .gitignore: coverage artifacts
 2026-06-15 15:38:59 +03:00
busya
654f92e7ad fix(test): resolve 3 remaining git_base test interaction failures 
Root cause: unittest.mock.patch with new_callable=AsyncMock fails in
full-suite context due to asyncio event loop state from earlier tests.
Fix: use direct module dict patching (gb_mod.run_blocking = AsyncMock())
instead of patch() context manager. This bypasses the mock machinery
interaction with inherited event loop state.

Also add conftest.py in tests/services/git/ with event_loop fixture
for per-function isolation.

Result: 2602 passed, 0 failed, 3 skipped, 1 xpassed
 2026-06-15 15:05:26 +03:00
busya
ce0369ae5c test(backend): add 55+ test files to push coverage to 98% 
Subagents delivered tests across all uncovered backend modules:

Schemas (100%): agent, auth, health, profile, settings, validation
Services (98-100%): auth, profile, health, llm, mapping, resource,
  security, git, superset_lookup, sql_table_extractor, rbac
API routes (new): auth, admin, health, environments, plugins,
  dashboards (helpers, projection, actions, listing),
  git (config, deps, env, helpers)
Clean Release (100%): DTO, facade, policy_engine, stages,
  repos, preparation, source_isolation, compliance
Git services: base, remote_providers
Agent module: app, run, middleware, langgraph_setup
Core: trace, cleanup, ws_log_handler, timezone, auth (config/oauth/security), matching
Reports: normalizer, report_service, type_profiles
Notifications: service, providers

Also:
- .gitignore: add .coverage, *.cover, coverage-* dirs
- src/schemas/auth.py: fix AD group DN regex (comma in CN=...)
- Remove co-located src/services/__tests__/ (caused pytest module collision)
 2026-06-15 13:55:57 +03:00
busya
27a20cbbeb fix(ssl): replace verify=True with ssl.create_default_context() for corporate CA support 
Core fix: all httpx.AsyncClient instances now use ssl.create_default_context()
instead of bool verify=True, which uses certifi and ignores system CA store.
This makes corporate CA certificates installed via update-ca-certificates
visible to Python HTTP clients.

Files:
- async_network.py: AsyncAPIClient.__init__ converts True→SSLContext
- client_registry.py: get_client converts bool→SSLContext before passing
- notifications/providers.py: _get_http_client uses ssl.create_default_context()
- services/git/_base.py: GitServiceBase uses ssl.create_default_context()
- translate/_llm_async_http.py: _get_verify returns SSLContext (not string)

Test: new integration test with 3-tier PKI (Root→Intermediate→Server),
TLS-protected Superset container, and custom CA installation via
update-ca-certificates or SSL_CERT_DIR fallback.
- conftest.py: added ca_chain, install_custom_ca, superset_tls_env fixtures;
  parameterized superset_container for TLS mode
- test_superset_tls_custom_ca.py: 8 tests (openssl -CApath, -CAfile certifi,
  httpx capath, httpx certifi, AsyncAPIClient, SupersetClient, fingerprint, verify=False)
 2026-06-15 10:32:17 +03:00
busya
af923972b6 fix(agent): save conversations to DB, fix Test button hang, wire hasNext/search 
## Root cause: _save_conversation() dead code + missing message persistence

### Backend: Conversation persistence (3 critical bugs)
- **app.py**: Replaced early  with  so _save_conversation() executes
  after successful stream — was dead code on normal path
- **app.py**: Added _save_conversation call in HITL resume path (confirm/deny)
- **app.py**: Added broad  that saves conversation (at least user
  message) before re-raising on LLM errors (APIConnectionError etc.)
- **app.py**: _save_conversation now passes user_id from JWT (not hardcoded UUID)
  and includes messages[] in payload
- **agent_conversations.py**: save_conversation endpoint now processes body.messages
  and creates AgentMessage records (idempotent by msg id)

### Frontend: Agent chat sidebar wiring
- AgentChatModel.svelte.ts: added public  derived getter
- AgentChatModel.svelte.ts: added  method
- agent/+page.svelte: wired hasNext={model.conversationsHasNext} (was hardcoded false)
- agent/+page.svelte: wired onsearch to model.searchConversations (was no-op)

### Frontend: LLM Provider Test button hang fix
- ProviderConfig.svelte: resetForm/handleEdit now reset isTesting=false, isProbing=false
- ProviderConfig.svelte: Cancel button calls abortPendingRequests()
- ProviderConfig.svelte: Added AbortController lifecycle — cancels in-flight test/fetch
  requests on modal close or provider switch, preventing stale disabled buttons
- provider_config.integration.test.ts: added 6 abort/reset invariant tests
 2026-06-14 16:07:06 +03:00
busya
997329e2a5 fix(agent): resolve ModuleNotFoundError for backend, add E2E test infra 
- Dockerfile.agent: fix CMD (python -m src.agent.run), use backend/requirements.txt,
  minimal COPY (only src.agent + src.core.cot_logger), add GRACE contract
- docker-compose.yml: SERVICE_TOKEN_SECRET -> SERVICE_JWT (match code)
- docker-compose.enterprise-clean.yml: same env var fix
- docker/.env.agent.example: same env var fix
- build.sh: same env var fix
- chore: semantics-testing SKILL.md, backend tests, pyproject.toml
 2026-06-14 15:41:46 +03:00
busya
e37b459e84 chore: fix preview routes, MarkdownRenderer, update opencode config 
- _preview_routes.py: fix preview endpoint params for multi-language
- MarkdownRenderer.svelte: renderer fixes for assistant messages
- speckit.analyze.md: update opencode command for spec analysis
- opencode.jsonc: config alignment
 2026-06-11 19:13:17 +03:00
busya
61660b345f fix(migrations): add merge head and insert_method/connection migration 
- 6b8ca3b7405f: merge heads for translation+enhancement migration chain
- c0d1e2f3a4b5: add insert_method, connection_id to translation_jobs;
  add insert_method, connection_snapshot to translation_runs
- e1f2a3b4c5d6: removed (duplicate/replaced by consolidated migration)
 2026-06-11 19:12:49 +03:00
busya
50d6b9226e test(translate): add tests for ConnectionService, DbExecutor, orchestrator direct DB dispatch 
- 9 new enhancement test files: test_connection_service.py,
  test_db_executor.py, test_orchestrator_direct_db.py, test_batch_insert.py,
  test_lang_stats.py, test_response_field_coverage.py, test_retry.py,
  test_run_service.py, test_sql_insert_service.py
- 5 new integration tests: test_superset_sqllab_e2e.py,
  test_translate_clickhouse.py, test_translate_corrections.py,
  test_translate_schedules.py, test_translate_status_fk.py
- Updated existing tests for insert_method/connection_id fields
 2026-06-11 19:12:45 +03:00
busya
55604498ae feat(translate): add insert_method dispatch and model/schema updates for direct DB 2026-06-11 19:12:35 +03:00
busya
1f9040d53e feat(core): implement ConnectionService and DbExecutor for direct DB insert (US11-US12) 
- ConnectionService: CRUD for DatabaseConnection in GlobalSettings with
  EncryptionManager-backed password encryption/decryption, test connectivity
- DbExecutor: asyncpg (PostgreSQL) and clickhouse-connect (ClickHouse) drivers
  with per-connection connection pooling
- config_models.py: promote GlobalSettings.connections from list[dict] stub
  to list[DatabaseConnection] with full Pydantic model validation
- settings.py: +5 CRUD endpoints for connections (create, read, update, delete, test)
- requirements.txt: add asyncpg>=0.29.0, clickhouse-connect>=0.7.0
- seed_permissions.py: register settings.connections.manage permission
 2026-06-11 19:12:15 +03:00
busya
06ced7608d fix(migrations): add _table_exists guard to migrations touching create_all()-only tables 
- 2df63b7ce038: was checking _column_exists but not _table_exists.
  If llm_providers/validation_policies/llm_validation_results don't
  exist (fresh DB), _column_exists returns False and add_column crashes.
  Now checks _table_exists first.
- a1b2c3d4e5f6 (20260603_add_token_limits_to_llm_providers): no guard
  at all. llm_providers is created by create_all() at runtime. Guard
  matches pattern used by ed28d34edde7, 9f8e7d6c5b4a and others.
 2026-06-11 17:11:56 +03:00
busya
6855bb7010 fix(migrations): guard f0e9d8c7b6a5 per-table for create_all()-only tables 
dataset_review_sessions (and potentially other tables in FK_DEFS) are
created at runtime by init_db() → Base.metadata.create_all(), not by
Alembic migrations. On fresh databases, these tables don't exist when
migrations run, causing ALTER TABLE to crash.

Fix: check table existence before operating on each FK. If a table
doesn't exist, create_all() will create it with the correct FK
definition (the model already has ondelete='CASCADE').

This is the same pattern used by other migrations (9f8e7d6c5b4a,
ed28d34edde7, c9d8e7f6a5b4, 86c7b1d6a710) for create_all()-only tables
like llm_providers, roles, validation_policies, llm_validation_results.
 2026-06-11 17:04:37 +03:00
busya
35a403ac5d fix(migrations): create dataset_review_sessions table before f0e9d8c7b6a5 runs 
- Add e1f2a3b4c5d6 migration: create dataset_review_sessions with FK
  ondelete='CASCADE' (matching the model). Previously created at runtime
  by init_db() → create_all().
- Update f0e9d8c7b6a5: change down_revision from a5b6c7d8e9f0 to
  e1f2a3b4c5d6 so the table exists when this migration runs.
- All other create_all()-only tables (llm_providers, roles,
  validation_policies, llm_validation_results) already have guards
  (_table_exists()) in their respective migrations.
 2026-06-11 16:58:12 +03:00
busya
d94ddb063f fix(migrations): bring task_records into Alembic, remove guard workaround 
- Add d1e2f3a4b5c6 migration: create task_records table matching the
  TaskRecord model (previously created at runtime by init_db() via
  Base.metadata.create_all).
- Update a5b6c7d8e9f0: down_revision now points to d1e2f3a4b5c6, so
  task_records exists when this migration runs. Remove the
  inspector.has_table() guard (no longer needed).
- create_all() in init_db() becomes a no-op for task_records — the
  table is now fully managed by Alembic.
 2026-06-11 16:49:55 +03:00
busya
d24745a01e fix(superset): resolve JWT auth — install psycopg2 + superset_config.py 
Root cause: Superset 4.1.2 ignores SQLALCHEMY_DATABASE_URI env var —
always falls back to SQLite. With separate init+web containers, the
init container's SQLite DB is lost → web container has no admin user
→ JWT login returns 401.

Fix:
- Install psycopg2-binary at container start (python -m pip install)
- Create /tmp/superset_config.py via heredoc that reads SUPERSET_DB_URI
- Set SUPERSET_CONFIG_PATH=/tmp/superset_config.py
- Use Docker bridge IP (not localhost) for Postgres from inside container

Now all 10 integration tests pass, including:
- test_jwt_login_success (access_token + refresh_token)
- test_jwt_authenticated_api_call (Bearer token → /api/v1/dashboard/)
- 4 health checks + 2 form-auth + 2 client construct
 2026-06-11 15:23:12 +03:00
busya
1c9c2c298e test(superset): add SQL Lab API and executor integration tests 
- test_superset_sqllab_api_integration.py (8 tests): raw REST API with form-based
  auth — login via /login/, CSRF token, /api/v1/database/ listing,
  /api/v1/sqllab/execute/ (returns structured error without configured DB),
  400 on missing database_id, CSRF protection, 404 for non-existent DB,
  JWT failure documented per ADR-0012

- test_superset_sqllab_executor_integration.py (9 tests): httpx.AsyncClient
  with form-based auth — DB listing, SQL Lab, database by ID,
  SupersetSqlLabExecutor constructor + resolve_database_id,
  SupersetClient with container URL, batch_insert module import,
  raw API column listing

Total integration test suite: 25 tests across 3 files
(conftest: 6 fixtures — superset_container, superset_url, superset_admin_headers)
 2026-06-11 14:48:27 +03:00
busya
79f67e1952 test(superset): add Testcontainers setup for Apache Superset integration tests 
- Add 6 fixtures to integration conftest: superset_db_url, superset_secret_key,
  superset_admin_password, superset_container (init+web), superset_url,
  superset_admin_headers
- Two-container architecture: init (db upgrade → create-admin → init) +
  web (superset run -p 8088)
- Superset 4.1.2 pinned (6.x incompatible: no psycopg2, SUPERSET__ prefix required)
- 8 integration tests cover health, form-login auth, SupersetClient construction
- Document decision in ADR-0012 with architecture rationale, rejected alternatives,
  and migration path to Superset 6.x
- Update docs/architecture.md with testing infrastructure overview
 2026-06-11 14:45:31 +03:00
busya
7760214170 test(agent): extend coverage — agent handler, confirmations, conversation API, tools 
- test_agent_handler: additional edge cases for streaming, HITL, file upload
- test_confirmations: HITL confirm/deny lifecycle coverage
- test_conversation_api: conversation save/load persistence tests
- test_langchain_tools: tool registration, dual-auth header propagation
- ConversationList.test.ts (frontend): conversation list component tests
- conftest: shared fixtures for agent tests
- task_manager/manager: minor fixes from test coverage
- tasks.md/test-documentation.md: spec and test documentation updates
- speckit.test.md: speckit workflow documentation update
 2026-06-10 16:38:06 +03:00
busya
1fd5e55db7 fix(agent): auto-fallback to free port on GRADIO_SERVER_PORT conflict 
The Gradio agent (run.py) crashed with OSError when port 7860 was
already occupied by a previous instance. Added _find_free_port() that
scans up to 100 ports from the configured GRADIO_SERVER_PORT and picks
the first available one, logging a warning on fallback.

Contract updates:
- AgentChat.Run: [C:3] [TYPE Module] (was C2/Function), added
  @RATIONALE, @REJECTED, @SIDE_EFFECT for port-finding logic
- AgentChat.GradioApp: added @RATIONALE, @REJECTED
- AgentChat.LangGraph.Setup: added @REJECTED, deduplicated @RELATION
- AgentChat.Tools: added @RATIONALE
 2026-06-10 16:37:02 +03:00
busya
143f14d516 chore: remainder — backend test infra, agent config, docker, i18n, frontend ui 
- Backend: alembic env, config manager/models, dependencies, translate plugin
- Backend tests: async_sync_regression, integration tests, git services, test_agent
- Docker: docker-compose.yml updates
- Agent: qa-tester.md update, semantics-testing SKILL.md update
- Frontend: TopNavbar, sidebarNavigation, FeaturesSettings, FeatureGate
- i18n: assistant.json en/ru locale updates
- New: frontend/src/lib/components/agent/ directory
 2026-06-10 15:06:36 +03:00
busya
f87c873027 test(app): add split app.py tests and extend migration_engine coverage 
- app.py split from 1966-line test_app.py into 7 files (68 tests, 92% coverage):
  - test_app_lifespan: lifespan, ensure_initial_admin_user
  - test_app_handlers: exception handlers, HSTS
  - test_app_middleware: log_requests, middleware chain
  - test_app_ws_auth: _authenticate_websocket
  - test_app_ws_endpoint: WS main loop, 5 endpoint handlers
  - test_app_ws_events: task/maintenance/dataset/translate WS streams
  - test_app_spa: SPA serving, TestClient integration
- migration_engine: extended coverage with init, edge cases, error paths
 2026-06-10 14:57:18 +03:00
busya
442596a3dc test(services): add unit tests for profile_preference, resource, validation_service 
- profile_preference_service: 19 tests, 100% coverage (CRUD, validation, encryption,
  DTO conversion with mocked AuthRepository and EncryptionManager)
- resource_service: 50 tests, 100% coverage (dashboard/dataset enrichment with
  git/task status, pagination, activity summary, datetime normalization)
- validation_service: 63 tests across 3 files, 97% coverage (provider validation,
  environment validation, source resolution, run/record conversion, trigger_run,
  create/update/delete tasks, list/filter runs, get_run_detail)
 2026-06-10 14:57:11 +03:00
busya
c33792c8ee test(services): add unit tests for 7 service-layer modules 
- profile_utils: 40 tests, 100% coverage (sanitize, normalize, mask, validate payload)
- security_badge_service: 17 tests, 100% coverage (role/permission extraction, security summary)
- services_mapping: 4 tests, 100% coverage (client resolution, get_suggestions)
- superset_lookup_service: 10 tests, 100% coverage (resolve environment, lookup success/degraded)
- notification_providers: 16 tests, 95% coverage (SMTP, Telegram, Slack providers)
- llm_provider: 25 tests, 91% coverage (mask_api_key, CRUD with encrypted API keys)
- rbac_permission_catalog: 12 tests, 79% coverage (route scanning, sync to DB)
 2026-06-10 14:57:04 +03:00
busya
54a0317e98 test(core): add unit tests for 7 core utility modules 
- executors: 13 tests, 100% coverage (init/shutdown/run_blocking/run_cpu_blocking)
- fileio_utils: 33 tests, 45% coverage (sanitize_filename, get_filename_from_headers,
  calculate_crc32, create_temp_file, remove_empty_directories, create_dashboard_export,
  consolidate_archive_folders)
- client_registry: 14 tests, 92% coverage (get_client, get_superset_client,
  get_semaphore, get_auth_lock, shutdown)
- cot_logger: 24 tests, 100% coverage (seed/set/get trace_id, push/pop span,
  structured log with markers, MarkerLogger proxy)
- encryption: 12 tests, 100% coverage (key validation, encrypt/decrypt cycle)
- rate_limiter: 9 tests, 100% coverage (ban logic, window pruning, per-IP isolation)
- auth/logger: 10 tests, 100% coverage (_mask_details, log_security_event)
 2026-06-10 14:56:48 +03:00
busya
f87ebf5d4b feat(agent): Gradio-powered LangGraph agent chat with streaming, tool calls, file upload, conversation persistence 
- Gradio 5.50.0 ChatInterface with type='messages' streaming
- LangGraph create_react_agent with InMemorySaver checkpointer
- 4 @tool functions: search_dashboards, get_health_summary, list_environments, get_task_status
- Structured ChatMessage metadata (7 discriminator types: stream_token, tool_start/end/error, confirm_required, confirm_resolved, error)
- HITL resume via second submit() with interrupt_before/Command
- Dual-identity RBAC: service JWT + user JWT for tool calls
- File upload (10 MB limit, pdfplumber/xlsx/JSON parser)
- Conversation persistence via POST /api/agent/conversations/save
- REST API: list, history, archive conversations; multi-tab gate; LLM config
- LLM provider selection via Admin -> LLM Settings (assistant_planner_provider)
- Svelte 5 AgentChatModel with stream event queue, dedup, stream_status watcher
- MarkdownRenderer using svelte-markdown with semantic Tailwind tokens
- ToolCallCard (3 states: executing/completed/failed)
- ConversationList with search, date grouping, infinite scroll
- ConnectionIndicator with Gradio health status
- /agent route with two-column layout
- Vite proxy /api/agent/gradio -> Gradio SSE
- Fixed: not_() SQLAlchemy operator, route collision with _admin_routes
- Fixed: conversation_id -> id normalization, .pyc cache staleness
- Fixed: event.data array parsing (Gradio returns [jsonStr, null])
- Requirements pinned: gradio==5.50.0, pydantic>=2.7,<=2.12.3
 2026-06-10 10:27:19 +03:00
busya
2222261157 tasks read 2026-06-09 11:44:20 +03:00
busya
8e8a3c3235 feat: attention-optimized semantic protocol v2.7 
Core changes:
- Add @defgroup/@ingroup to 1791 C2+ contracts (555 files) for HCA 128× pre-training DSA grouping
- Add §0.1 Pre-Training Frequency matrix to semantics-core
- Add §VIII Attention Architecture rules (ATTN_1-4) with MLA/CSA/HCA/DSA mechanics
- Add @defgroup/@ingroup to canonical syntax (§II) and all contract examples

Agent prompts (5 files):
- Add ZERO-STATE RATIONALE with MLA/CSA/HCA/DSA compression mechanics
- Add pre-training note: @RATIONALE/@REJECTED are in-context learned tags
- svelte-coder: add missing #region contract, fix Svelte rule violations
- python-coder/fullstack-coder: honor function contracts from speckit plan
- qa-tester: add attention compliance audit (P3 ATTN_1-4 checks)

Skills (6 files):
- Translate all axiom_config descriptions to English
- Fix doc_dirs to index .opencode/ and .specify/
- Deduplicate 5× complexity_rules → single global_tags catalog
- Reduce semantics-svelte 591→485 lines (remove duplicate code blocks)
- Fix semantics-testing: 'Short IDs' → 'Short hierarchical IDs'
- Fix all examples: flat IDs → hierarchical Domain.Name format
- Fix Svelte examples: replace raw Tailwind + <button> with semantic tokens + /ui

Speckit workflow (commands + templates):
- speckit.plan: add Function-Level Contracts for C3+ with @PRE/@POST/@TEST_EDGE
- speckit.plan: add Attention Compliance Gate (ATTN_1-4 before contract generation)
- speckit.tasks: add function contract inlining format (constraints in task description)
- speckit.specify: load semantics-core for spec density rules
- spec-template: add #region contract, @SEMANTICS grouping, hierarchical IDs
- ux-reference-template: add #region wrapper
- plan-template: add attention gate, @defgroup/@ingroup guidance
- tasks-template: add attention audit + rebuild + orphan check tasks
- constitution.md: translate to English, add Principle VIII (attention-optimized contracts)

Reference modules rewritten (hierarchical IDs + full contracts):
- Auth.Jwt: 6 child contracts with @RATIONALE/@REJECTED/@TEST_EDGE
- Api.Auth: 5 endpoints with @TEST_EDGE + molecular CoT markers
- Migration.Model: @defgroup Migration with 18 @ACTION + 6 @INVARIANT

Scripts:
- add_defgroup_ingroup.py: zero-risk additive @ingroup migration (1791 insertions)
- migrate_hierarchical.py: flat→hierarchical ID dry-run analysis (792 contracts)
- merge_prompts.py: merge all prompts/skills/commands into one review file

Config:
- axiom_config.yaml: 749→395 lines (-47%), English, doc_dirs include prompts
- Fix test_datasets.py import collision (rename → test_datasets_routes.py)
- Fix test_preview.py: SupersetClient→get_superset_client, AsyncMock, logger f-string
 2026-06-08 16:30:59 +03:00
busya
f731a53c7d 038: add @RATIONALE/@REJECTED contracts to async C4/C5 modules 
5 contracts updated:
  - AsyncNetworkModule (C5): async migration rationale, per-client CSRF cookie rejection
  - AsyncAPIClient (C4): auth lifecycle, cache-hit CSRF refresh
  - AsyncAPIClient.request (C4): string vs dict handling, double-encoding root cause
  - AsyncAPIClient.upload_file (C4): multipart async upload
  - LLMAsyncHttpClient (C4): response.ok -> is_success, module-level httpx singleton
 2026-06-05 17:02:30 +03:00
busya
fbe0ba122c 037: fix 99 failing tests — missing await after async migration 
Fixed async/sync boundary bugs across 14 test files. Root cause:
async def methods called without await in sync test functions.

Fixed files:
  - test_translate_jobs.py (10): create_job/get_job/update_job/delete_job
  - test_translate_scheduler.py (5): create_schedule/update/delete
  - test_datasets.py (14): AsyncMock + corrected patch target
  - test_mapping_service.py (11): sync_environment + MockSupersetClient
  - test_defensive_guards.py (6): GitService/SupersetClient guards
  - test_maintenance_service.py (29): all 6 maintenance services
  - test_dry_run_orchestrator.py (1): run() without await
  - test_dashboards_api.py (23): registry client via AsyncMock
  - test_validation_tasks.py (4): trailing slash in POST URL
  - test_superset_matrix.py (3): AsyncMock for compile_preview
  - test_payload_reduction.py (6): LLMClient._optimize_image wrapper
  - test_compliance_task_integration.py (2): event_bus ref
  - test_smoke_plugins.py (1): flusher_stop_event fallback
  - test_task_manager.py (1): _flusher_stop_event/thread fallback

Remaining 31 failures in test_task_manager.py (29) and
test_smoke_plugins.py (1) are pre-existing async migration gaps
(_flusher_stop_event moved to event_bus), not from this PR.
 2026-06-05 15:43:35 +03:00
busya
7f1937f10b 036: wire SupersetClientRegistry into translate + services flow 
Replaces direct SupersetClient(env) calls with shared
get_superset_client(env) from client_registry.

Changed files (9):
  - client_registry.py: added get_superset_client(), _build_env_id(),
    accepts Environment models (not just dicts), fixed async_client attr
  - superset_executor.py: _get_client() now async, uses shared client
  - preview_executor.py, _run_source.py, service.py, service_datasource.py
  - health_service.py, resource_service.py, debug.py

Impact per environment:
  - 6 separate httpx.AsyncClient instances → 1 shared client
  - 6 CSRF cookie fetches → 1 (on first access)
  - 6 connection pools → 1 shared pool
  - Shared semaphore for backpressure
  - Shared cookie jar (fixes CSRF 'tokens do not match' on SQL Lab execute)
 2026-06-05 15:14:44 +03:00
busya
4cef6af041 fix 2026-06-05 15:01:34 +03:00
busya
50180aaa14 035: fix httpx.Response.ok -> httpx.Response.is_success in LLM client 
httpx.Response does not have .ok attribute (that's requests.Response).
Async migration missed this: _llm_async_http.py used response.ok in two
places, causing 502 errors when LLM API responded.

Fix: response.ok -> response.is_success
 2026-06-05 12:10:20 +03:00
busya
5d9d214bd6 034: fix double JSON serialization in AsyncAPIClient.request 
Root cause: AsyncAPIClient.request() passes its  parameter directly
to httpx.AsyncClient.request(json=data). When callers pass a pre-serialized
JSON string (data=json.dumps(dict)), httpx re-encodes it via json.dumps(),
resulting in a double-encoded JSON string body instead of a JSON object.

This caused ALL POST/PUT requests with string data to fail — Superset received
a JSON string instead of a JSON object, returning GENERIC_BACKEND_ERROR
('dictionary update sequence element #0 has length 1; 2 is required').

Fix: if data is a string, pass it via httpx  parameter (raw body);
if it's a dict/list, pass via  for automatic encoding.

Affected callers (6 files) now correctly send JSON objects:
  - preview_executor.py: chart data requests
  - superset_executor.py
  - _run_source.py
  - _datasets.py: update_dataset
  - _datasets_preview.py: compile_dataset_preview
  - _dashboards_write.py

Also simplified preview_executor.fetch_sample_rows back to single-strategy
(chart data API only) since the root cause is now fixed.
 2026-06-05 12:07:55 +03:00
busya
72fcf698af 033: fix preview_translation route + MultiSelect label regression 
1. preview_translation route: missing await on async preview_rows()
   - preview_rows() is async def, called without await
   - returned coroutine object instead of result -> 'coroutine not iterable' error

2. MultiSelect.svelte: opt.label -> opt.name
   - option type is {code, name} but template used {opt.label}
   - rendered empty spans instead of language names
 2026-06-05 11:38:35 +03:00
busya
250b69db8c 032: fix coroutine never awaited in debug.py + task_logger.py 
debug.py: _test_db_api and _get_dataset_structure were already async def
but called SupersetClient methods (authenticate, get_databases, get_dataset)
without await. Added await to 4 calls.

task_logger.py: _add_log callback is async def but _log() called it without
await, silently dropping all task log messages (RuntimeWarning: coroutine
never awaited). Changed to fire-and-forget via asyncio.ensure_future when
a running event loop is available, drops gracefully otherwise.
 2026-06-05 11:23:53 +03:00
busya
909b568784 032: add async regression tests (21 tests covering all fixed bug patterns) 2026-06-05 10:40:51 +03:00
busya
3fc09b086e 032: fix 2 critical QA issues — missing endregion + Tombstone dead helpers 
C1: _llm_call.py — added missing #endregion _split_and_retry (violated INV_3)
C2: dashboards/_helpers.py — sync _find_dashboard_id_by_slug and
    _resolve_dashboard_id_from_ref typed Tombstone per INV_6 (dead code,
    callers use _async versions from git/_helpers.py)
 2026-06-05 10:32:37 +03:00
busya
4247e7a20e 032: fix final sync→async cascade (dataset_review, parsing, validation) 
- _parsing.py: parse_superset_link + _recover_dataset_binding → async
    (await get_dashboard_detail, get_chart)
  - dataset_review/orchestrator.py: start_session + _build_recovery_bootstrap → async
    (await get_dataset_detail, parse_superset_link)
  - _routes.py (dataset_review): await orchestrator.start_session()
  - validation_tasks.py: await parse_superset_link + get_dashboard_detail
 2026-06-05 09:55:34 +03:00
busya
c7d8f4431e 032: fix remaining sync→async propagation (17 call sites) 
Core fixes:
  - service_datasource.py: fetch_datasource_metadata() → async
  - service.py: create_job(), update_job() → async (callers await)
  - _job_routes.py: await create_job/update_job

Maintenance scanners:
  - _dashboard_scanner.py: 4 functions → async (find_affected, _get_linked,
    _apply_filters, _resolve_title)
  - _chart_manager.py: 3 functions → async
  - _banner_renderer.py: rebuild_banner → async
  - _orchestrators.py: 3 orchestrators → async
  - maintenance_banner.py: await async calls

Migration:
  - dry_run_orchestrator.py: run(), _build_target_signatures() → async
  - risk_assessor.py: build_risks() → async
  - migration.py: await service.run()
  - mapping_service.py: sync_environment() → async

Dead code:
  - _helpers.py: _find_dashboard_id_by_slug marked DEPRECATED
 2026-06-05 08:56:37 +03:00
busya
a5cd06546f 032: fix missing await on TranslationExecutor.execute_run() in orchestrator_exec 
executor.execute_run() is async def but was called without await in
TranslationExecutionEngine.execute_run(). This returned a coroutine
instead of a TranslationRun, causing:
  'coroutine' object has no attribute 'status'
This made every translation run fail in the background execute path.
 2026-06-05 08:47:56 +03:00
busya
71000db785 032: fix run_translation route handler — sync def → async def (asyncio.create_task needs running loop) 
run_translation was def (sync FastAPI handler runs in thread pool with no event
loop), but its body calls asyncio.create_task(_background_execute()) which
requires a running event loop. Changed to async def so FastAPI runs it on the
event loop directly.

Error: 'Run failed: no running event loop'
 2026-06-05 08:40:46 +03:00
busya
71156783cf 032: fix get_dashboards_page_async -> get_dashboards_page (method renamed during async migration) 
get_dashboards_page_async() no longer exists — the sync/async split was
removed and the method is now simply get_dashboards_page() (already async).
Calls in dashboard slug resolution and git helpers were using the old name.
This caused AttributeError at runtime, making all slug-based dashboard
lookups fail with 'Dashboard not found'.
 2026-06-05 08:39:05 +03:00
busya
079b2dd37b 032: add aclose() to SupersetClient (was missing, used in dashboard detail routes) 
SupersetClientBase was missing aclose() method. AsyncSupersetClient had it
via override, but code creating SupersetClient directly (e.g. dashboard tasks
history route) would fail with AttributeError on await client.aclose().
 2026-06-05 08:33:49 +03:00
busya
5eb116509b 032: dead code cleanup — remove sync APIClient, _llm_http, preview_llm_client, fix retry chains 2026-06-05 08:31:18 +03:00