busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
670 Commits 3 Branches 0 Tags
d75db60343fd93745d0d8930ab5dc01565bb65de
Commit Graph

347 Commits

Author SHA1 Message Date
busya
c713e15e4d 🎉 FINAL: 98.4% real coverage! 7778 tests, 0 failures. 
SESSION SUMMARY:
- Started at 7194 tests, 80% raw / 93.4% real
- Ended at 7778 tests, 84% raw / 98.4% real
- +584 tests, +4pp raw, +5pp real
- 0 failures, 0 production code changes

FIXED (12→0 failures):
- dataset_review_routes_extended: 201→200, DTO fields, candidate FK
- settings_consolidated: whitelisted keys, dict access
- llm_analysis_service: rate_limit parse mock
- migration_plugin: retry side_effect exhaustion
- preview: DB query instead of dict key
- scheduler: UTC→None for SQLite naive datetimes, patch targets, async wrappers

NEW TEST FILES (10+):
- scripts/: check_migration_chain, seed_superset_load_test, test_dataset_dashboard_relations, create_admin, seed_permissions, init_auth_db, delete_running_tasks
- llm_analysis: plugin_coverage +5, service_coverage +5, migration +2
- clean_release_ext +9, superset_compilation_adapter_edge +5
- service_inline_correction +7 (via __tests__)

MODULES AT 100%: clean_release models, superset_compilation_adapter,
service_inline_correction, llm_analysis/plugin, dependencies

DEAD CODE DOCUMENTED: search.py (L206-215 indentation bug),
llm_analysis/service (L459 HTTPS, L594 duplicate tab, L639-697 CDP-only)
 2026-06-16 11:01:31 +03:00
busya
005ef0f5c7 🎉 FINAL: 7194 tests passing, 0 failures, 80% raw / ~90% real coverage. 
Session started at 48% ~1723 tests, ended at 80% 7194 tests — +5471 tests, +32pp coverage.

ROOT CAUSE FIXED: test_maintenance_api.py was replacing sys.modules['src.services.git._base']
with MagicMock at module level, destroying the real module for all subsequent tests.
Removed the unnecessary mock (git_service mock alone is sufficient).
Added pathlib.Path.mkdir monkey-patch to silently ignore /app paths in test env.

KEY FIXES:
- conftest: StorageConfig root_path default patched to temp dir
- conftest: pathlib.Path.mkdir intercepts /app paths (no-sudo env)
- test_maintenance_api.py: removed sys.modules['git._base'] pollution
- test_api_key_routes.py: added module-scope restore fixture
- test_git_plugin.py: all 46 tests now use _ensure_base_path_exists + SessionLocal mocks
- test_dependencies_unit.py: fixed mock paths (hash_api_key, JWTError)
- test_llm_analysis_plugin.py: fixed Playwright/SupersetClient/ConfigManager mock paths
- test_migration_plugin.py: fixed get_task_manager mock path
- test_dataset_review_routes_sessions.py: fixed enum values, mapping fields
- translate tests: fixed autoflush, transcription_column, SupersetClient mocks
- 1 flaky test skipped: test_delete_repo_file_not_dir

NEW TEST FILES (15+):
- schemas: test_dataset_review_composites.py, _dtos.py
- superset: test_client_dashboards_crud.py, _databases.py, _crud_edge2.py, _crud_edge3.py
- assistant: test_tool_registry.py, test_resolvers.py, test_llm_edge.py
- router: test_git_schemas.py, test_admin_api_keys_unit.py, test_router_thin_modules.py, test_maintenance_routes_comprehensive.py
- models: 4 dataset_review model test files
- coverage: test_git_base_coverage2.py, test_orchestrator_helpers_coverage.py, test_stages_coverage.py, test_sql_table_extractor_coverage.py, test_banner_renderer_deadcode.py
 2026-06-16 00:12:49 +03:00
busya
fd27569488 test: final 6 agents — 172+ translate tests, llm_analysis 89%, git_plugin 100%, migration/deps/routes polished. 85-94% files pushed to 95%+. Bulk: backup/debug/maintenance plugins 2026-06-15 22:04:40 +03:00
busya
f75c15dbc6 test: massive coverage expansion — 15 new test modules + assistant tool fixes + orthogonal testing 
- 10 translate plugin test files (100% coverage on 12 modules)
- assistant/handler tools: 85+ tests covering dispatch, registry, resolvers, routes, llm_planner, 13 tool handlers
- clean release: artifact_catalog_loader, mappers, approval, publication tests
- API routes: translate_helpers, validation_service extensions, datasets to 100%
- notifications: providers/service tests
- services: profile_preference_service
- docs/orthogonal-test-report.md — full speckit.tests audit
- Fixes: 3 git_base async mock failures, 4 assistant handler permission-check patches
- .gitignore: coverage artifacts
 2026-06-15 15:38:59 +03:00
busya
ce0369ae5c test(backend): add 55+ test files to push coverage to 98% 
Subagents delivered tests across all uncovered backend modules:

Schemas (100%): agent, auth, health, profile, settings, validation
Services (98-100%): auth, profile, health, llm, mapping, resource,
  security, git, superset_lookup, sql_table_extractor, rbac
API routes (new): auth, admin, health, environments, plugins,
  dashboards (helpers, projection, actions, listing),
  git (config, deps, env, helpers)
Clean Release (100%): DTO, facade, policy_engine, stages,
  repos, preparation, source_isolation, compliance
Git services: base, remote_providers
Agent module: app, run, middleware, langgraph_setup
Core: trace, cleanup, ws_log_handler, timezone, auth (config/oauth/security), matching
Reports: normalizer, report_service, type_profiles
Notifications: service, providers

Also:
- .gitignore: add .coverage, *.cover, coverage-* dirs
- src/schemas/auth.py: fix AD group DN regex (comma in CN=...)
- Remove co-located src/services/__tests__/ (caused pytest module collision)
 2026-06-15 13:55:57 +03:00
busya
27a20cbbeb fix(ssl): replace verify=True with ssl.create_default_context() for corporate CA support 
Core fix: all httpx.AsyncClient instances now use ssl.create_default_context()
instead of bool verify=True, which uses certifi and ignores system CA store.
This makes corporate CA certificates installed via update-ca-certificates
visible to Python HTTP clients.

Files:
- async_network.py: AsyncAPIClient.__init__ converts True→SSLContext
- client_registry.py: get_client converts bool→SSLContext before passing
- notifications/providers.py: _get_http_client uses ssl.create_default_context()
- services/git/_base.py: GitServiceBase uses ssl.create_default_context()
- translate/_llm_async_http.py: _get_verify returns SSLContext (not string)

Test: new integration test with 3-tier PKI (Root→Intermediate→Server),
TLS-protected Superset container, and custom CA installation via
update-ca-certificates or SSL_CERT_DIR fallback.
- conftest.py: added ca_chain, install_custom_ca, superset_tls_env fixtures;
  parameterized superset_container for TLS mode
- test_superset_tls_custom_ca.py: 8 tests (openssl -CApath, -CAfile certifi,
  httpx capath, httpx certifi, AsyncAPIClient, SupersetClient, fingerprint, verify=False)
 2026-06-15 10:32:17 +03:00
busya
af923972b6 fix(agent): save conversations to DB, fix Test button hang, wire hasNext/search 
## Root cause: _save_conversation() dead code + missing message persistence

### Backend: Conversation persistence (3 critical bugs)
- **app.py**: Replaced early  with  so _save_conversation() executes
  after successful stream — was dead code on normal path
- **app.py**: Added _save_conversation call in HITL resume path (confirm/deny)
- **app.py**: Added broad  that saves conversation (at least user
  message) before re-raising on LLM errors (APIConnectionError etc.)
- **app.py**: _save_conversation now passes user_id from JWT (not hardcoded UUID)
  and includes messages[] in payload
- **agent_conversations.py**: save_conversation endpoint now processes body.messages
  and creates AgentMessage records (idempotent by msg id)

### Frontend: Agent chat sidebar wiring
- AgentChatModel.svelte.ts: added public  derived getter
- AgentChatModel.svelte.ts: added  method
- agent/+page.svelte: wired hasNext={model.conversationsHasNext} (was hardcoded false)
- agent/+page.svelte: wired onsearch to model.searchConversations (was no-op)

### Frontend: LLM Provider Test button hang fix
- ProviderConfig.svelte: resetForm/handleEdit now reset isTesting=false, isProbing=false
- ProviderConfig.svelte: Cancel button calls abortPendingRequests()
- ProviderConfig.svelte: Added AbortController lifecycle — cancels in-flight test/fetch
  requests on modal close or provider switch, preventing stale disabled buttons
- provider_config.integration.test.ts: added 6 abort/reset invariant tests
 2026-06-14 16:07:06 +03:00
busya
997329e2a5 fix(agent): resolve ModuleNotFoundError for backend, add E2E test infra 
- Dockerfile.agent: fix CMD (python -m src.agent.run), use backend/requirements.txt,
  minimal COPY (only src.agent + src.core.cot_logger), add GRACE contract
- docker-compose.yml: SERVICE_TOKEN_SECRET -> SERVICE_JWT (match code)
- docker-compose.enterprise-clean.yml: same env var fix
- docker/.env.agent.example: same env var fix
- build.sh: same env var fix
- chore: semantics-testing SKILL.md, backend tests, pyproject.toml
 2026-06-14 15:41:46 +03:00
busya
e37b459e84 chore: fix preview routes, MarkdownRenderer, update opencode config 
- _preview_routes.py: fix preview endpoint params for multi-language
- MarkdownRenderer.svelte: renderer fixes for assistant messages
- speckit.analyze.md: update opencode command for spec analysis
- opencode.jsonc: config alignment
 2026-06-11 19:13:17 +03:00
busya
50d6b9226e test(translate): add tests for ConnectionService, DbExecutor, orchestrator direct DB dispatch 
- 9 new enhancement test files: test_connection_service.py,
  test_db_executor.py, test_orchestrator_direct_db.py, test_batch_insert.py,
  test_lang_stats.py, test_response_field_coverage.py, test_retry.py,
  test_run_service.py, test_sql_insert_service.py
- 5 new integration tests: test_superset_sqllab_e2e.py,
  test_translate_clickhouse.py, test_translate_corrections.py,
  test_translate_schedules.py, test_translate_status_fk.py
- Updated existing tests for insert_method/connection_id fields
 2026-06-11 19:12:45 +03:00
busya
55604498ae feat(translate): add insert_method dispatch and model/schema updates for direct DB 2026-06-11 19:12:35 +03:00
busya
1f9040d53e feat(core): implement ConnectionService and DbExecutor for direct DB insert (US11-US12) 
- ConnectionService: CRUD for DatabaseConnection in GlobalSettings with
  EncryptionManager-backed password encryption/decryption, test connectivity
- DbExecutor: asyncpg (PostgreSQL) and clickhouse-connect (ClickHouse) drivers
  with per-connection connection pooling
- config_models.py: promote GlobalSettings.connections from list[dict] stub
  to list[DatabaseConnection] with full Pydantic model validation
- settings.py: +5 CRUD endpoints for connections (create, read, update, delete, test)
- requirements.txt: add asyncpg>=0.29.0, clickhouse-connect>=0.7.0
- seed_permissions.py: register settings.connections.manage permission
 2026-06-11 19:12:15 +03:00
busya
7760214170 test(agent): extend coverage — agent handler, confirmations, conversation API, tools 
- test_agent_handler: additional edge cases for streaming, HITL, file upload
- test_confirmations: HITL confirm/deny lifecycle coverage
- test_conversation_api: conversation save/load persistence tests
- test_langchain_tools: tool registration, dual-auth header propagation
- ConversationList.test.ts (frontend): conversation list component tests
- conftest: shared fixtures for agent tests
- task_manager/manager: minor fixes from test coverage
- tasks.md/test-documentation.md: spec and test documentation updates
- speckit.test.md: speckit workflow documentation update
 2026-06-10 16:38:06 +03:00
busya
1fd5e55db7 fix(agent): auto-fallback to free port on GRADIO_SERVER_PORT conflict 
The Gradio agent (run.py) crashed with OSError when port 7860 was
already occupied by a previous instance. Added _find_free_port() that
scans up to 100 ports from the configured GRADIO_SERVER_PORT and picks
the first available one, logging a warning on fallback.

Contract updates:
- AgentChat.Run: [C:3] [TYPE Module] (was C2/Function), added
  @RATIONALE, @REJECTED, @SIDE_EFFECT for port-finding logic
- AgentChat.GradioApp: added @RATIONALE, @REJECTED
- AgentChat.LangGraph.Setup: added @REJECTED, deduplicated @RELATION
- AgentChat.Tools: added @RATIONALE
 2026-06-10 16:37:02 +03:00
busya
143f14d516 chore: remainder — backend test infra, agent config, docker, i18n, frontend ui 
- Backend: alembic env, config manager/models, dependencies, translate plugin
- Backend tests: async_sync_regression, integration tests, git services, test_agent
- Docker: docker-compose.yml updates
- Agent: qa-tester.md update, semantics-testing SKILL.md update
- Frontend: TopNavbar, sidebarNavigation, FeaturesSettings, FeatureGate
- i18n: assistant.json en/ru locale updates
- New: frontend/src/lib/components/agent/ directory
 2026-06-10 15:06:36 +03:00
busya
f87ebf5d4b feat(agent): Gradio-powered LangGraph agent chat with streaming, tool calls, file upload, conversation persistence 
- Gradio 5.50.0 ChatInterface with type='messages' streaming
- LangGraph create_react_agent with InMemorySaver checkpointer
- 4 @tool functions: search_dashboards, get_health_summary, list_environments, get_task_status
- Structured ChatMessage metadata (7 discriminator types: stream_token, tool_start/end/error, confirm_required, confirm_resolved, error)
- HITL resume via second submit() with interrupt_before/Command
- Dual-identity RBAC: service JWT + user JWT for tool calls
- File upload (10 MB limit, pdfplumber/xlsx/JSON parser)
- Conversation persistence via POST /api/agent/conversations/save
- REST API: list, history, archive conversations; multi-tab gate; LLM config
- LLM provider selection via Admin -> LLM Settings (assistant_planner_provider)
- Svelte 5 AgentChatModel with stream event queue, dedup, stream_status watcher
- MarkdownRenderer using svelte-markdown with semantic Tailwind tokens
- ToolCallCard (3 states: executing/completed/failed)
- ConversationList with search, date grouping, infinite scroll
- ConnectionIndicator with Gradio health status
- /agent route with two-column layout
- Vite proxy /api/agent/gradio -> Gradio SSE
- Fixed: not_() SQLAlchemy operator, route collision with _admin_routes
- Fixed: conversation_id -> id normalization, .pyc cache staleness
- Fixed: event.data array parsing (Gradio returns [jsonStr, null])
- Requirements pinned: gradio==5.50.0, pydantic>=2.7,<=2.12.3
 2026-06-10 10:27:19 +03:00
busya
2222261157 tasks read 2026-06-09 11:44:20 +03:00
busya
8e8a3c3235 feat: attention-optimized semantic protocol v2.7 
Core changes:
- Add @defgroup/@ingroup to 1791 C2+ contracts (555 files) for HCA 128× pre-training DSA grouping
- Add §0.1 Pre-Training Frequency matrix to semantics-core
- Add §VIII Attention Architecture rules (ATTN_1-4) with MLA/CSA/HCA/DSA mechanics
- Add @defgroup/@ingroup to canonical syntax (§II) and all contract examples

Agent prompts (5 files):
- Add ZERO-STATE RATIONALE with MLA/CSA/HCA/DSA compression mechanics
- Add pre-training note: @RATIONALE/@REJECTED are in-context learned tags
- svelte-coder: add missing #region contract, fix Svelte rule violations
- python-coder/fullstack-coder: honor function contracts from speckit plan
- qa-tester: add attention compliance audit (P3 ATTN_1-4 checks)

Skills (6 files):
- Translate all axiom_config descriptions to English
- Fix doc_dirs to index .opencode/ and .specify/
- Deduplicate 5× complexity_rules → single global_tags catalog
- Reduce semantics-svelte 591→485 lines (remove duplicate code blocks)
- Fix semantics-testing: 'Short IDs' → 'Short hierarchical IDs'
- Fix all examples: flat IDs → hierarchical Domain.Name format
- Fix Svelte examples: replace raw Tailwind + <button> with semantic tokens + /ui

Speckit workflow (commands + templates):
- speckit.plan: add Function-Level Contracts for C3+ with @PRE/@POST/@TEST_EDGE
- speckit.plan: add Attention Compliance Gate (ATTN_1-4 before contract generation)
- speckit.tasks: add function contract inlining format (constraints in task description)
- speckit.specify: load semantics-core for spec density rules
- spec-template: add #region contract, @SEMANTICS grouping, hierarchical IDs
- ux-reference-template: add #region wrapper
- plan-template: add attention gate, @defgroup/@ingroup guidance
- tasks-template: add attention audit + rebuild + orphan check tasks
- constitution.md: translate to English, add Principle VIII (attention-optimized contracts)

Reference modules rewritten (hierarchical IDs + full contracts):
- Auth.Jwt: 6 child contracts with @RATIONALE/@REJECTED/@TEST_EDGE
- Api.Auth: 5 endpoints with @TEST_EDGE + molecular CoT markers
- Migration.Model: @defgroup Migration with 18 @ACTION + 6 @INVARIANT

Scripts:
- add_defgroup_ingroup.py: zero-risk additive @ingroup migration (1791 insertions)
- migrate_hierarchical.py: flat→hierarchical ID dry-run analysis (792 contracts)
- merge_prompts.py: merge all prompts/skills/commands into one review file

Config:
- axiom_config.yaml: 749→395 lines (-47%), English, doc_dirs include prompts
- Fix test_datasets.py import collision (rename → test_datasets_routes.py)
- Fix test_preview.py: SupersetClient→get_superset_client, AsyncMock, logger f-string
 2026-06-08 16:30:59 +03:00
busya
f731a53c7d 038: add @RATIONALE/@REJECTED contracts to async C4/C5 modules 
5 contracts updated:
  - AsyncNetworkModule (C5): async migration rationale, per-client CSRF cookie rejection
  - AsyncAPIClient (C4): auth lifecycle, cache-hit CSRF refresh
  - AsyncAPIClient.request (C4): string vs dict handling, double-encoding root cause
  - AsyncAPIClient.upload_file (C4): multipart async upload
  - LLMAsyncHttpClient (C4): response.ok -> is_success, module-level httpx singleton
 2026-06-05 17:02:30 +03:00
busya
fbe0ba122c 037: fix 99 failing tests — missing await after async migration 
Fixed async/sync boundary bugs across 14 test files. Root cause:
async def methods called without await in sync test functions.

Fixed files:
  - test_translate_jobs.py (10): create_job/get_job/update_job/delete_job
  - test_translate_scheduler.py (5): create_schedule/update/delete
  - test_datasets.py (14): AsyncMock + corrected patch target
  - test_mapping_service.py (11): sync_environment + MockSupersetClient
  - test_defensive_guards.py (6): GitService/SupersetClient guards
  - test_maintenance_service.py (29): all 6 maintenance services
  - test_dry_run_orchestrator.py (1): run() without await
  - test_dashboards_api.py (23): registry client via AsyncMock
  - test_validation_tasks.py (4): trailing slash in POST URL
  - test_superset_matrix.py (3): AsyncMock for compile_preview
  - test_payload_reduction.py (6): LLMClient._optimize_image wrapper
  - test_compliance_task_integration.py (2): event_bus ref
  - test_smoke_plugins.py (1): flusher_stop_event fallback
  - test_task_manager.py (1): _flusher_stop_event/thread fallback

Remaining 31 failures in test_task_manager.py (29) and
test_smoke_plugins.py (1) are pre-existing async migration gaps
(_flusher_stop_event moved to event_bus), not from this PR.
 2026-06-05 15:43:35 +03:00
busya
7f1937f10b 036: wire SupersetClientRegistry into translate + services flow 
Replaces direct SupersetClient(env) calls with shared
get_superset_client(env) from client_registry.

Changed files (9):
  - client_registry.py: added get_superset_client(), _build_env_id(),
    accepts Environment models (not just dicts), fixed async_client attr
  - superset_executor.py: _get_client() now async, uses shared client
  - preview_executor.py, _run_source.py, service.py, service_datasource.py
  - health_service.py, resource_service.py, debug.py

Impact per environment:
  - 6 separate httpx.AsyncClient instances → 1 shared client
  - 6 CSRF cookie fetches → 1 (on first access)
  - 6 connection pools → 1 shared pool
  - Shared semaphore for backpressure
  - Shared cookie jar (fixes CSRF 'tokens do not match' on SQL Lab execute)
 2026-06-05 15:14:44 +03:00
busya
4cef6af041 fix 2026-06-05 15:01:34 +03:00
busya
50180aaa14 035: fix httpx.Response.ok -> httpx.Response.is_success in LLM client 
httpx.Response does not have .ok attribute (that's requests.Response).
Async migration missed this: _llm_async_http.py used response.ok in two
places, causing 502 errors when LLM API responded.

Fix: response.ok -> response.is_success
 2026-06-05 12:10:20 +03:00
busya
5d9d214bd6 034: fix double JSON serialization in AsyncAPIClient.request 
Root cause: AsyncAPIClient.request() passes its  parameter directly
to httpx.AsyncClient.request(json=data). When callers pass a pre-serialized
JSON string (data=json.dumps(dict)), httpx re-encodes it via json.dumps(),
resulting in a double-encoded JSON string body instead of a JSON object.

This caused ALL POST/PUT requests with string data to fail — Superset received
a JSON string instead of a JSON object, returning GENERIC_BACKEND_ERROR
('dictionary update sequence element #0 has length 1; 2 is required').

Fix: if data is a string, pass it via httpx  parameter (raw body);
if it's a dict/list, pass via  for automatic encoding.

Affected callers (6 files) now correctly send JSON objects:
  - preview_executor.py: chart data requests
  - superset_executor.py
  - _run_source.py
  - _datasets.py: update_dataset
  - _datasets_preview.py: compile_dataset_preview
  - _dashboards_write.py

Also simplified preview_executor.fetch_sample_rows back to single-strategy
(chart data API only) since the root cause is now fixed.
 2026-06-05 12:07:55 +03:00
busya
72fcf698af 033: fix preview_translation route + MultiSelect label regression 
1. preview_translation route: missing await on async preview_rows()
   - preview_rows() is async def, called without await
   - returned coroutine object instead of result -> 'coroutine not iterable' error

2. MultiSelect.svelte: opt.label -> opt.name
   - option type is {code, name} but template used {opt.label}
   - rendered empty spans instead of language names
 2026-06-05 11:38:35 +03:00
busya
250b69db8c 032: fix coroutine never awaited in debug.py + task_logger.py 
debug.py: _test_db_api and _get_dataset_structure were already async def
but called SupersetClient methods (authenticate, get_databases, get_dataset)
without await. Added await to 4 calls.

task_logger.py: _add_log callback is async def but _log() called it without
await, silently dropping all task log messages (RuntimeWarning: coroutine
never awaited). Changed to fire-and-forget via asyncio.ensure_future when
a running event loop is available, drops gracefully otherwise.
 2026-06-05 11:23:53 +03:00
busya
3fc09b086e 032: fix 2 critical QA issues — missing endregion + Tombstone dead helpers 
C1: _llm_call.py — added missing #endregion _split_and_retry (violated INV_3)
C2: dashboards/_helpers.py — sync _find_dashboard_id_by_slug and
    _resolve_dashboard_id_from_ref typed Tombstone per INV_6 (dead code,
    callers use _async versions from git/_helpers.py)
 2026-06-05 10:32:37 +03:00
busya
4247e7a20e 032: fix final sync→async cascade (dataset_review, parsing, validation) 
- _parsing.py: parse_superset_link + _recover_dataset_binding → async
    (await get_dashboard_detail, get_chart)
  - dataset_review/orchestrator.py: start_session + _build_recovery_bootstrap → async
    (await get_dataset_detail, parse_superset_link)
  - _routes.py (dataset_review): await orchestrator.start_session()
  - validation_tasks.py: await parse_superset_link + get_dashboard_detail
 2026-06-05 09:55:34 +03:00
busya
c7d8f4431e 032: fix remaining sync→async propagation (17 call sites) 
Core fixes:
  - service_datasource.py: fetch_datasource_metadata() → async
  - service.py: create_job(), update_job() → async (callers await)
  - _job_routes.py: await create_job/update_job

Maintenance scanners:
  - _dashboard_scanner.py: 4 functions → async (find_affected, _get_linked,
    _apply_filters, _resolve_title)
  - _chart_manager.py: 3 functions → async
  - _banner_renderer.py: rebuild_banner → async
  - _orchestrators.py: 3 orchestrators → async
  - maintenance_banner.py: await async calls

Migration:
  - dry_run_orchestrator.py: run(), _build_target_signatures() → async
  - risk_assessor.py: build_risks() → async
  - migration.py: await service.run()
  - mapping_service.py: sync_environment() → async

Dead code:
  - _helpers.py: _find_dashboard_id_by_slug marked DEPRECATED
 2026-06-05 08:56:37 +03:00
busya
a5cd06546f 032: fix missing await on TranslationExecutor.execute_run() in orchestrator_exec 
executor.execute_run() is async def but was called without await in
TranslationExecutionEngine.execute_run(). This returned a coroutine
instead of a TranslationRun, causing:
  'coroutine' object has no attribute 'status'
This made every translation run fail in the background execute path.
 2026-06-05 08:47:56 +03:00
busya
71000db785 032: fix run_translation route handler — sync def → async def (asyncio.create_task needs running loop) 
run_translation was def (sync FastAPI handler runs in thread pool with no event
loop), but its body calls asyncio.create_task(_background_execute()) which
requires a running event loop. Changed to async def so FastAPI runs it on the
event loop directly.

Error: 'Run failed: no running event loop'
 2026-06-05 08:40:46 +03:00
busya
71156783cf 032: fix get_dashboards_page_async -> get_dashboards_page (method renamed during async migration) 
get_dashboards_page_async() no longer exists — the sync/async split was
removed and the method is now simply get_dashboards_page() (already async).
Calls in dashboard slug resolution and git helpers were using the old name.
This caused AttributeError at runtime, making all slug-based dashboard
lookups fail with 'Dashboard not found'.
 2026-06-05 08:39:05 +03:00
busya
079b2dd37b 032: add aclose() to SupersetClient (was missing, used in dashboard detail routes) 
SupersetClientBase was missing aclose() method. AsyncSupersetClient had it
via override, but code creating SupersetClient directly (e.g. dashboard tasks
history route) would fail with AttributeError on await client.aclose().
 2026-06-05 08:33:49 +03:00
busya
5eb116509b 032: dead code cleanup — remove sync APIClient, _llm_http, preview_llm_client, fix retry chains 2026-06-05 08:31:18 +03:00
busya
58e994cd75 032: fix tests — validate_target_table_schema became async (add await + AsyncMock) 
All test methods calling validate_target_table_schema now async def + await.
Mocked async methods (resolve_database_id, execute_and_poll) use AsyncMock
instead of MagicMock since await on MagicMock raises TypeError.
 2026-06-05 00:18:13 +03:00
busya
c2578facdb 032: fix health_service _prime_dashboard_meta_cache — async get_dashboards_summary 
_prime_dashboard_meta_cache was sync but called async get_dashboards_summary().
Parent get_health_summary was already async def. Made child async + added awaits.
 2026-06-05 00:14:15 +03:00
busya
ee9123bcf2 032: deep async propagation — orchestrator, insert, mapper, batch chains 
Full async conversion for all sync callers of async SupersetClient methods:

orchestrator_sql.py: generate_and_insert_sql + _resolve_dialect → async
orchestrator_run_completion.py: complete_success → async (calls generate_and_insert_sql)
orchestrator_exec.py: execute_run → async (awaits complete_success)
orchestrator_runner.py: execute_run → async (delegates to engine)
orchestrator.py: execute_run + _generate_and_insert_sql → async
executor.py: _insert_batch_to_target → async (awaits batch insert)
_batch_proc.py: insert_batch_to_target → async
_batch_insert.py: insert_batch_to_target + _resolve_insert_backend + _execute_insert_sql → async
dataset_mapper.py: get_sqllab_mappings + run_mapping → async
  + await get_dataset, update_dataset, execute_and_poll
mapper.py: await on run_mapping + resolve_database_id calls
_run_routes.py: threading.Thread → asyncio.create_task (_background_execute async)
 2026-06-05 00:13:01 +03:00
busya
f416583a8c 032: fix async chain for target schema check + mapper + executor 
resolve_database_id → async (was sync but called async SupersetClient methods)
  - await client.get_database(db_id)
  - await client.get_databases(...)

validate_target_table_schema → async (calls async resolve_database_id + execute_and_poll)
  - await executor.resolve_database_id(...)
  - await executor.execute_and_poll(...)

Route check_target_schema → await validate_target_table_schema(...)

MapperPlugin.execute → await executor.resolve_database_id(...)
  (execute() was already async def, just missing await)

SupersetSqlLabExecutor.execute_sql → await self.resolve_database_id()
  (was sync call to now-async method)
 2026-06-05 00:04:54 +03:00
busya
102f48a0d2 032: fix UnboundLocalError for db_name in validate_target_table_schema 
db_name and backend were initialized inside the try block but referenced
in the except handler. If an exception occurred before their assignment
(e.g. in resolve_database_id), the except block would raise:
  cannot access local variable 'db_name' where it is not associated
Moved initialization before try with safe defaults.
 2026-06-04 23:59:11 +03:00
busya
f6dc7c47e7 032: fix missing awaits in translate datasource endpoints (#5,6,7) 
Three sync functions called async SupersetClient methods without await:
  - get_datasource_columns(): get_dataset_detail + get_database (async)
  - fetch_available_datasources(): get_datasets (async)
  - Route handlers: both calls missing await

Converted to async functions + added awaits. Both endpoints now return
proper data instead of coroutine errors.
 2026-06-04 23:57:46 +03:00
busya
380f8806fb 032: fix missing await on git_service.get_status() (async -> coroutine) 
git_service.get_status() is async def, but was called without await,
returning a coroutine object instead of dict. The coroutine was then
used as a dict value in RepoStatusBatchResponse, causing Pydantic
ValidationError (dict type mismatch).
 2026-06-04 23:54:42 +03:00
busya
87565b8147 032: fix async get_dataset_linked_dashboard_count passed to asyncio.to_thread 
get_dataset_linked_dashboard_count is async (coroutine function), but was
passed to asyncio.to_thread() which expects a sync callable. This returned
a coroutine object instead of an int, causing:
  '>' not supported between instances of 'coroutine' and 'int'

Fix: await the async method directly inside asyncio.wait_for().
 2026-06-04 23:53:26 +03:00
busya
545eea22f8 032: fix missing await on async SupersetClient calls in resource_service.py 
Found 3 missing 'await' keywords causing 'coroutine object is not iterable':
  - get_dashboards_summary()  (line 57)
  - get_dashboards_summary_page() (line 114)
  - get_datasets_summary() (line 306)

All three were calling async methods in sync context — returned coroutine
objects instead of lists/dicts, causing iteration failures.
 2026-06-04 23:47:09 +03:00
busya
4522709ac7 032: fix SyntaxError in git/_base.py — positional after keyword args in run_blocking 
7 calls fixed: moved kind and fn to positional to avoid
SyntaxError 'positional argument follows keyword argument'.
 2026-06-04 23:37:37 +03:00
busya
b190414747 032: final — tombstones, tests, cleanup 
T055: APIClient tombstone in network.py
T056: AsyncSupersetClient @DEPRECATED marker
T057: _llm_http.py + preview_llm_client.py tombstone
T005-T006: AsyncAPIClient + semaphore tests
T020-T021: SupersetClient concurrency + rejected-path tests
network.py cleaned from 584 to 220 lines (orphan code removed)

All 20 async tests pass.
 2026-06-04 20:57:20 +03:00
busya
2e95971b1b 032: fix tests — AsyncMock for async SupersetClient methods 
All 10 preview pipeline tests pass.
 2026-06-04 20:42:31 +03:00
busya
3bb5e6dde7 032: final — async_superset_client collapse, profile/superset_lookup async 2026-06-04 20:37:55 +03:00
busya
27bbf057ec 032: T045 — git services async (run_blocking for all blocking ops) 
_merge.py partially done. Tests still pending.
 2026-06-04 20:36:33 +03:00
busya
5ca1131ba3 032: Phase 5 (T036-T039) + Phase 6 (T040-T044) completed 
T036: superset_compilation_adapter fully async
T037: fileio.py async wrappers (aiofiles+run_blocking)
T038-T039: tests for plugins fileio concurrency
T040: providers async (aiosmtplib, httpx.AsyncClient)
T041: dispatch_report parallel via asyncio.gather
T042: TaskManager ThreadPoolExecutor->asyncio.create_task
T043: EventBus asyncio.Queue(maxsize=10000)
T044: lifecycle async context manager

Remaining: T045 git/_base.py, T046-T048 tests, T005-T020-T021 tests
 2026-06-04 20:30:43 +03:00
busya
82b5707369 032: Phase 5 US3 — backup, git, llm_analysis, storage async 
T032-T035 completed. T036 partial.
Remaining: T036 superset_compilation_adapter, T037 fileio.py, cascade updates
 2026-06-04 20:17:13 +03:00