busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
903 Commits 2 Branches 19 Tags
012f903a571634e173202bcfb8c6b0a9614a9118
Commit Graph

186 Commits

Author SHA1 Message Date
busya
012f903a57 feat(037): Phase 2 US1 — Inspect Dashboard Query Model (T006-T010) 
- T006: 4 deterministic inspection tests (basic, deterministic, inaccessible, missing)
- T007: 6 filter normalization tests (scope, hash, order, locale)
- T008: query_model.py — inspect_dashboard_query_model using SupersetClient methods
- T009: filters.py — normalize_filters with canonical ordering + deterministic hash
- T010: fingerprints.py — SHA-256 helpers for query model and filter hashing

10/10 tests pass. Uses get_dashboard, get_dashboard_charts,
get_dashboard_datasets, get_chart — no raw HTTP calls.
 2026-07-28 19:27:47 +03:00
busya
9d1e303ad9 feat(037): Phase 1 fixtures and DTOs (T001-T005) 
- T001: Superset dashboard fixture (FI-0080, 3 charts, 2 datasets, 2 filters)
- T002: Result fixtures (scalar, decimal, date, table, locale, malformed)
- T003: Baseline catalog fixtures (valid, invalid_no_release, stale, immutability)
- T004: Materialize fixtures into backend/tests/fixtures/dashboard_testing/
- T005: Pydantic DTOs — 30+ models covering query model, filters, execution,
  normalization, comparison, baseline catalog, candidates, structure diff,
  verification runs. All extra_forbid, typed, with invariants.
 2026-07-28 19:23:39 +03:00
busya
bf0ba897ac feat(036): fixtures T001-T003 — UIContext, events, snapshots 2026-07-28 19:12:52 +03:00
busya
9254299da0 feat(036): enforce mask_selectors in RegisterDraft + test 2026-07-28 19:05:55 +03:00
busya
f2d844cd91 test(036): evidence tests (4) + denial tests (5) — 80 backend, 14 frontend 2026-07-28 18:57:48 +03:00
busya
343d9e3917 test(036): artifact tests (12) + API tests (10) — 76/76 backend, fix RBAC Depends 2026-07-28 18:48:52 +03:00
busya
c9c6636ae8 test(036): event tests (12) + approval tests (11) — 54/54 backend 2026-07-28 11:42:54 +03:00
busya
6185e94d25 test(036): repository (11 tests) + agent context v2 (9 tests) + tool filter (6 tests) 2026-07-28 11:34:09 +03:00
busya
24fca2ec8a test(036): backend schemas (22 tests) + frontend AgentRunModel (13 tests) 2026-07-28 10:35:47 +03:00
root
3fd8525c4e fix: harden agent startup and websocket auth 2026-07-27 11:49:18 +03:00
root
6a0650b7a0 fix: service-to-service auth with SERVICE_JWT and robust Content-Disposition parsing 2026-07-24 17:49:39 +03:00
busya
c30cca78f3 smoke alembic 2026-07-23 18:53:45 +03:00
busya
7faa913767 fix: stabilize storage and test coverage 2026-07-23 16:45:15 +03:00
busya
7961ef51ba feat(migration): improve failure diagnostics 2026-07-23 16:44:27 +03:00
busya
63d82df53b test(coverage): add 200+ tests to push frontend + backend coverage above thresholds 
Backend (4 files, 73 tests):
- test_agent_superset_routes.py (27 tests, 35% -> 92%)
- test_agent_lifecycle_routes.py (11 tests, 50% -> 100%)
- test_agent_status_routes.py (6 tests, 57% -> 100%)
- test_git_release_routes.py (31 tests, 30% -> 99%)

Frontend (~15 files, ~120 tests):
- cron.ts: 0% -> 100%
- ReportsLogModel: 0% -> 99%
- parseCot.ts: 10% -> 100%
- sessionTimeout.ts: 64% -> 93%
- MappingsModel: 65% -> 100%
- TranslateHistoryModel: 65% -> 93%
- Migration.ExecutorModel: 70% -> 100%
- GitManagerModel: 78% -> 90%
- TranslationJobModel: 77% -> 80%
- ConfirmDialog: 59% -> 80%
- api.ts: 78% -> 80%

Coverage: frontend 0 violations, backend 7518 passed.
 2026-07-23 15:49:45 +03:00
busya
eeb3a05e42 fix(translate): preserve ClickHouse datetime keys 2026-07-23 12:42:56 +03:00
busya
cd4b91daa5 fix(rbac): close critical auth gaps — full RBAC audit remediation 
CRITICAL — unauthenticated endpoints (CWE-306):
- agent_superset.py: 10 SQL/dashboard/dataset proxy endpoints → plugin:superset_proxy:EXECUTE
- agent_superset_explore.py: 10 database explore endpoints → plugin:superset_proxy:READ
- clean_release.py / clean_release_v2.py: router-level deny-by-default → clean_release:MANAGE
- settings.py: PUT/DELETE/test environment → admin:settings:WRITE/READ
- tasks.py: log/stats/sources/export → tasks:READ

HIGH — authorization gaps (CWE-285, CWE-613):
- require_api_key_or_jwt: add token blacklist + is_active + is_admin flag checks
- get_current_user: add is_active check
- WebSocket: add _authorize_websocket() RBAC helper, permission-gate all 6 WS endpoints
- agent_conversations: add Depends(get_current_user) to save endpoint + router-level guard
- legacy validation redirect: add validation.task:VIEW guard

MEDIUM — consistency & architecture:
- admin.py: fix permission parsing split(':',1) → rsplit(':',1)
- app.py lifespan: sync RBAC permission catalog at startup
- schemas/auth.py: add is_admin to RoleSchema (with BeforeValidator), RoleCreate, RoleUpdate
- models/auth.py: add is_admin support in create_role/update_role handlers
- permissions.ts: expand KNOWN_ACTIONS (VIEW/CREATE/EDIT/MANAGE/APPROVE/PREVIEW/LAUNCH/LAUNCH_PROD)
- permissions.ts: isAdminUser checks is_admin flag from /auth/me
- Navbar.svelte: replace exact role name check with hasPermission()
- admin/+page.svelte, admin/settings/llm/+page.svelte: add ProtectedRoute guards

TESTS:
- test_dependencies_unit.py: fix 5 tests for new is_token_blacklisted + is_admin checks
- test_api_key_auth.py: fix test_jwt_precedence for is_token_blacklisted mock
- permissions.test.ts: update non-KNOWN_ACTION test to use unknown suffix 'xyz'

VERIFIED: 230 backend tests pass, 3257 frontend tests pass, index rebuilt (7373 contracts, 3832 edges)
 2026-07-23 12:38:19 +03:00
busya
fb8769c577 fix(tests): repair 39 broken tests after auth/authz hardening 
- Clean Release API (33 tests): added get_current_user dependency override
  with mock admin user in _make_client — router now requires has_permission

- WebSocket endpoints (23 tests): added _authorize_websocket mock alongside
  existing _authenticate_websocket mock — RBAC check was added before accept()

- Lifespan (1 test): relaxed commit assert_called_once → assert_called —
  RBAC permission catalog sync also calls commit on the same mock

All 7445 backend tests pass (0 failures).
 2026-07-23 12:32:50 +03:00
busya
b58275a234 fix(tests): use module-level _should_retry instead of LLMClient._should_retry 
_should_retry is a module-level function in service.py:885, not a class
method on LLMClient. Updated 6 tests in TestShouldRetryEdgeCases and
TestShouldRetryProviderFailures to import and call _should_retry directly.
 2026-07-23 11:59:44 +03:00
busya
d0213ff45b feat: session timeout management + LLM provider error hardening 
Backend:
- Typed LLM provider exception hierarchy (auth, config, transport, rate limit)
- Permanent provider errors (401/403) propagate as ProviderAuthenticationFailure
  instead of being swallowed as UNKNOWN — Task becomes FAILED, not SUCCESS
- Provider error normalization maps SDK exceptions to typed hierarchy
- _should_retry extracted to module level for cross-method reuse
- SessionActivity model (jti, user_id, issued_at, expires_at, last_activity_at)
- Backend-enforced idle + absolute session timeout in dependencies.py
- Session policy endpoint GET /api/auth/session
- GlobalSettings extended: session_idle_timeout, session_absolute_timeout,
  session_warning_minutes
- Consolidated settings API returns session policy fields
- Alembic migration 8e9f0a1b2c3d for session_activity table

Frontend:
- Global 401 session-expired handler in api.ts with dedup guard
- Health polling stops on 401 (isDisabled=true)
- SessionTimeoutGuard component (+layout.svelte) tracks idle/absolute deadlines
- SessionTimeoutDialog modal with countdown (Continue/Logout)
- BroadcastChannel cross-tab activity sync
- Login returnUrl support (validates, prevents open redirect)
- SystemSettings card for session timeout configuration
- ROUTES.login(returnUrl?) for all login redirects
- i18n (EN/RU) for session security UI
- 3256 frontend tests pass, build clean
 2026-07-23 11:27:17 +03:00
busya
b9c0fa4c28 feat(git): BI-first UI/UX rework of /git — wizard modal, smart grid, undo, pre-flight 
Grid (Phase 1):
- Smart row actions by sync status (Connect Git / Save version (N) / Manage / Diagnose)
- "What changed" column with compact stacked category badges
- Status filter chips, sticky bulk panel with live progress, inline row errors + retry
- Dedupe repo-status batch requests (grid feeds the Repositories tab)
- DashboardDataGrid: new actionsCell snippet

GitManager modal (Phases 2-4):
- Simple/Full mode toggle (localStorage), Simple = 3-step wizard
  (Changes → Verify → Publish) via new GitWizardStepper
- Undo center + undo toast: soft-undo unpublished commit
  (backend POST /repositories/{ref}/undo-commit, reset --soft HEAD~1,
  409 guards for pushed/detached/empty HEAD)
- Commit draft autosave per dashboard slug
- Keyboard: Ctrl+Enter commit, 1/2/3 step/tab navigation
- Contextual "You are here: step N" help on the /git page

Excellence (Phase 5):
- First-run GuidedTour (4 spotlight steps, restartable from help panel)
- Pre-flight checklist before create/publish release (ConfirmDialog children
  + confirmDisabled; red checks block the action)
- Rollback confirmation with revert-preview diff; guided conflict progress bar
- Preview link to PREPROD before publishing; human-readable version labels
- prefers-reduced-motion guard; page <title>; aria-live wizard announcements
- docs/design/git-ux-glossary.md — canonical action verbs, ru/en normalized

UX fixes (user feedback):
- Compact change chips (vertical stack, 10px) — no table horizontal scroll
- "Insert into version description" button next to AI key-changes summary
- Guided recovery for "binding belongs to another Git server" (CTA to settings)
- Instant rollback button reveal (CSS visibility via :global, no opacity repaint)

QA fixes:
- Glossary compliance: 0 "commit/коммит" in user-facing strings
- Contract coverage for rollback functions in CommitHistory
- Rollback label aligned to glossary ("Откат к версии" / "Revert to version")

Tests: backend 446 git passed + 5 new undo-commit edge cases;
frontend 3204 passed (8 pre-existing failures unrelated: pipeline locale,
ConfirmationCard, PasswordPrompt, assistant_chat, test_tasks);
new GitReleasePanel pre-flight tests 3/3 green; vite build green.
 2026-07-22 18:06:26 +03:00
root
632b730fff chore: migrate GRACE-Poly anchors to hierarchical dotted naming 
Systematic rename of all semantic anchors (#region, [DEF], @RELATION)
across 1400+ files — backend Python, frontend Svelte/TS, specs, docs:
- Flat anchors become Namespace.Module.Entity
- @RELATION references updated to match new anchor paths
- Zero business logic changes
 2026-07-22 11:48:15 +03:00
busya
456d531a13 feat: harden migration flows and integration coverage 2026-07-21 18:59:26 +03:00
busya
7bfc5553cf feat: live app log console, cross-filter, JSONL export 
Backend:
- /ws/app-logs — real-time app/cot log stream (raw JSONL)
- /api/logs/recent — REST snapshot of ring buffer
- GET /tasks/{id}/logs/export — streaming JSONL export with CoT parse + redaction
- Thread-safe ring buffer (seq-based polling) replaces unsafe asyncio.Queue
- GIL-friendly multi-row Core insert for log persistence
- task_id ContextVar propagates into CotJsonFormatter for CoT correlation
- Hot-apply logging level on settings update (FR-005)
- Buffer trim under DEBUG floods; drop DEBUG first, preserve ERROR/WARNING
- List projection (include_result=False) keeps reports list slim
- Security event consolidated to single REASON atom

Frontend:
- ReportsLogModel + ReportsLogPanel — full live JSONL console
- Cross-filter pinning: Tasks → Logs tab with task chip badges
- LogEntryRow — CoT-aware rendering (marker icons, expandable payload)
- TaskFilterChip, taskChipMeta — scannable type/id/env chips
- Global drawer push via CSS variable (lg+ padding, not overlay)
- i18n en/ru for all log console strings
- Ctrl+A in log panel selects only log lines (window-level handler)

QA fixes:
- Svelte 5 reactivity: SvelteDate/SvelteSet/SvelteURLSearchParams
- Fix seed_trace_id shadowing (F823) in lifecycle.py
- Remove dead code (selectedEnvironment, goToReportsPage)
- Add @BRIEF to C2 test functions, missing {#each} keys
- Remove unused import json as _json from app.py

All 3200+ frontend tests pass; backend lint clean.
 2026-07-20 21:41:34 +03:00
busya
49a566359a feat: translate module — runtime knobs, GRACE anchors, two-layer testing, QA fixes 
Implementation:
- Performance knobs: llm_batch_max_rows, llm_concurrency, insert_concurrency,
  multi_lang_mode, batch_aggressiveness, max_in_flight_batches
- Alembic migration f7a8b9c0d1e2 (idempotent, nullable, non-destructive)
- LLM provider capabilities: throughput_class, reasoning_control,
  supports_json_object, default/max_llm_concurrency
- TargetSchemaValidationRequest with conditional validator (sqllab/direct_db)
- Scheduler: background dispatch, local imports for lazy bootstrap

GRACE-Poly compliance:
- Semantic anchors on orchestrator_aggregator, orchestrator_sql, llm_provider
- Shared module _llm_http.py: _apply_reasoning_control extraction (INV_4)
- Alembic migration anchors per C3/C1 template
- Four renamed Svelte components: RunOutcomeSummary, DetectionQualityCard,
  LanguageStatList, SourceLanguageOverride

QA (this session):
- 11 backend test regressions fixed: spec'd MagicMock null fields for new
  columns, _check_translation_cache_bulk retarget, scheduler mock wiring,
  language_detection=auto assertions, token budget constant update
- 4 frontend eslint errors fixed: SvelteSet/SvelteDate imports,
  unused lang parameter, dead isTransientError
- Production bugfix: job_to_response() mapped 6 missing fields
- Pre-existing auth test failure documented (dependencies.py untouched)
- Axiom: 6440 contracts, 0 warnings
 2026-07-20 14:19:47 +03:00
busya
31b9a19a0c chore: commit remaining workspace updates 
Agent:
- lifecycle: run tracking, middleware hardening, langgraph setup
- tests: agent lifecycle + langgraph setup coverage

Backend:
- async_job_runner: resilience hardening, tests
- agent_conversations: run lifecycle integration
- translate: scheduler + orchestrator SQL adjustments
- schemas/services: agent_lifecycle model extensions

Frontend:
- TaskDrawer: UX improvements
- TaskLogPanel/Viewer: safety hardening, i18n (en/ru)
- FilterBar: report filters contract + tests
- Reports page: layout adjustments

Specs:
- 036-agent-test-stabilization: runs contract, modules, events
- 037-superset-baseline-engine: catalog schema, testing API, modules
- 038-dashboard-scenario-model: scenario schema, capture profile, modules
- 039-dashboard-scenario-ui: screen models, release verification UX, modules
- dashboard-verification-usecases: new cross-cutting spec
 2026-07-17 19:11:09 +03:00
busya
8f0d123ff8 fix migration resume and release workflow 2026-07-16 12:31:59 +03:00
busya
45ce585aba chore: commit remaining workspace updates 2026-07-16 07:53:41 +03:00
busya
20105f51c0 feat(translate): add run preflight and focus execution UX 2026-07-16 07:52:52 +03:00
busya
20071b8c7a security: fullstack hardening — task ownership, mapping validation, API-key scoping, test fixes 
Backend:
- Add validate_mapping_database_ownership() to verify source/target UUIDs
  belong to declared environments before persisting mappings (mappings.py)
- Add API-key environment scoping to get_mappings (filter) and
  suggest_mappings_api (enforce) (mappings.py)
- Add user_id Column to TaskRecord model + Alembic migration (task.py)
- Persist task.user_id on save, restore on load (persistence.py)
- Wire current_user.id into migrate_dashboards + backup_dashboards
  task creation (_action_routes.py)
- Fix test_migration_routes.py: module-level patch leak → autouse fixture,
  SupersetClient→AsyncSupersetClient, AsyncMock for sync_environment/run
- Fix 7 Pydantic serializer warnings: 'PENDING'→TaskStatus.PENDING
  in test_tasks.py + import TaskStatus

Frontend:
- Deepen isDryRunResult(): validate selection field, risk.items entries
  (all 5 fields), and diff object uuids individually (ExecutorModel.svelte.ts)

Prior work included: task password redaction, resume ownership checks,
canonical dry-run DTO alignment, migration UI callback fixes, credential
exposure reduction, assistant dry-run await fix.
 2026-07-15 23:02:23 +03:00
busya
30c8acf7ae fix(translate): handle invalid LLM JSON responses 2026-07-15 20:06:39 +03:00
busya
9922d7c87a fix(translate,scheduler): language detection, LLM error handling, and scheduler persistence 
Translation pipeline fixes (from production error log analysis):

Lang detect:
- Add "ru" to _COMMON_SOURCE_CODES for Cyrillic source detection
- Version detector cache keys (v2:) and include detector version in
  source hash — stale cache entries from old algorithm are invalidated
- Integrate _character_block_fallback into batch_detect() pipeline;
  only assign Cyrillic fallback when exactly one Cyrillic target exists
  (multiple ru/uk/be targets stay undetermined for LLM arbitration)

Batch processing:
- Case-insensitive cache language matching (cached_by_lang lookup)
- Propagate needs_review=True for undetected language rows in pre/cache path

LLM call (critical — fixes silent error hiding):
- Validate LLM row IDs against expected set; log unknown identifiers
- Retry only missing rows on incomplete response (bounded by recursion depth)
- Exhausted retries → FAILED (new _handle_incomplete_response method)
- Parse failures → FAILED instead of SKIPPED (_handle_parse_failure)
- NULL/Empty translations → FAILED via new _add_failed helper
- Source-language identity mapping preserved as TranslationLanguage entry
  (carries source_language_detected metadata that _build_insert_rows
  relies on for detected_src_lang derivation)
- finish_reason propagated to parser for truncation diagnostics

LLM parse:
- Structured incomplete-set logging with expected/received/missing counts

Target schema validation:
- Handle all non-success SQL Lab statuses (failed, timeout, error, stopped)
- Timeout gets explicit message prefix
- Route returns HTTP 502 with correct HTTPException passthrough

Scheduler persistence fix (production pickle error):
- Backup and validation APScheduler jobs now use module-level callbacks
  (execute_scheduled_backup, execute_scheduled_validation) instead of
  bound SchedulerService methods — prevents pickle failure from
  serializing TaskManager + dynamically loaded plugin classes
- Callback func identity verified via pickle round-trip smoke test

Tests:
- Update assertions: FAILED replaces SKIPPED for LLM error paths
- Restore source-language contract tests with identity-mapped values
- Add scheduler callback identity and args verification tests
- Update detector cache key tests for versioned format
- Update target schema error route test: 200→502

Orthogonal code review: MEDIUM finding (empty rec.languages when all
targets match detected source → _build_insert_rows fallback to "und")
fixed by preserving source-language TranslationLanguage entries.
 2026-07-15 13:07:41 +03:00
busya
8f4ee25415 fix(alembic): merge three migration heads + add smoke test for chain integrity 
- Created merge migration 7eaf84b7f6be joining heads:
  - 6b8ca3b7405f (previous merge of c0d1e2f3a4b5 + f2b3c4d5e6f7)
  - b4c5d6e7f8a9 (include_source_reference to translation_jobs)
  - f4a5b6c7d8e9 (preproduction validation to deployment records)

- Added smoke test (test_smoke_migration_chain.py) that:
  - Checks exactly 1 head (catches branch divergence)
  - Walks full chain verifying all down_revision links exist
  - Confirms all .py files are loaded as revisions
  - Runs WITHOUT a database (real ScriptDirectory, no mocks)
  - Catches what existing tests missed:
    * test_alembic_migrations.py skips on non-PostgreSQL
    * test_check_migration_chain.py uses mocks, not real files
 2026-07-14 17:33:16 +03:00
busya
9b3cc54646 feat: add backup integrity verification 2026-07-14 16:05:28 +03:00
busya
c3ad0afc17 refactor: remove rejected dataset review feature 2026-07-14 15:56:31 +03:00
busya
2a56ea5fc9 test: fix backend and frontend test contracts 2026-07-14 10:43:38 +03:00
busya
66497da72b feat(mapper): secure xlsx upload and dataset selection 2026-07-14 00:34:10 +03:00
busya
d630573402 fix(backup): orthogonal code review fixes + UI/UX overhaul 
## Critical fixes
- H1: Fix env_id/env/environment_id triple inconsistency in API route
  (_action_routes.py now passes 'environment_id', matches scheduler)
- H2: Decompose BackupPlugin.execute() — CC 13 → 5 methods, CC ≤ 5 each
- H3: Fix unhandled int() ValueError on non-numeric dashboard_ids
- H4: Add concurrent guard test with API-style params (env key fallback)
- H5: Make RetentionPolicy configurable via StorageConfig
  (retention_daily/weekly/monthly in config model + backend plugin)
- Fix: storage DELETE route missing 'await' on async delete_file (bug)

## UI/UX overhaul
- NEW: centralized cron utility (frontend/src/lib/utils/cron.ts)
  - validateCron(), calcNextCronRun(), formatNextRun()
  - Used by BackupManager, BackupDashboardModal
- BackupManager: replace custom BackupList with shared FileList
  - Adds: download, delete, bulk actions, search, sort, pagination
  - Adds: cron next-run preview below schedule input
  - Adds: auto-open TaskDrawer on backup task creation
- BackupDashboardModal: dead 'Cron Help' button removed
  - Adds: live cron validation + next-run preview
- StorageSettings: adds retention daily/weekly/monthly fields
- BackupCreateRequest type fixed to match actual API contract
- i18n: 7 new keys for en/ru (next_run, retention_*)
 2026-07-13 20:58:02 +03:00
busya
c2bd6cb441 feat(git): clarify dashboard release flow 2026-07-13 12:55:24 +03:00
busya
00d2619c86 fix: align DRAFT integration test with removed DRAFT check in validate_job_preconditions 
- DRAFT validation was removed from orchestrator_validation.py (moved to service layer)
- Integration test expected ValueError for DRAFT jobs — fixed to test actual behavior
- 194/194 integration tests pass, 0 failures
 2026-07-12 19:44:11 +03:00
busya
a39a76c87f feat(agent-centric-logging): consolidate CoT infra in shared, close REASON→REFLECT chains 
- shared/cot_logger.py is SSOT; backend/cot_logger.py deleted
- elapsed_ms timing in all REFLECT markers
- Frontend: REASON→REFLECT/EXPLORE in all fetch/post/delete/requestApi
- Dynamic src: route.GET.api.plugins instead of hardcoded api.request_handler
- trace_id generated immediately (no 'no-trace'), X-Trace-ID in both directions
- Global error handlers (window error + unhandledrejection + error.svelte)
- Fixed duplicate logging (shared/logger.py double StreamHandler)
- propagate=False in configure_logger (was in ConfigManager = duplicated startup logs)
- belief_scope: 'Coherence OK' → '{anchor}: completed' + elapsed_ms
- Fixed 28 pre-existing test failures (scheduler sig, DB columns, DRAFT validation, etc)
 2026-07-12 19:30:57 +03:00
busya
24d3b7d1f9 refactor(task-manager): implement task resilience and execution lifecycle improvements 
Enhance the reliability and observability of the task execution engine
by introducing retry mechanisms, idempotency, and structured progress
tracking.

- Implement centralized retry logic with exponential backoff support
  in `JobLifecycle`.
- Add `retry_task` API endpoint and `TaskManager` method for manual
  task restarts.
- Introduce task idempotency using `_idempotency_key` to prevent
  duplicate executions.
- Add `retry_count`, `max_retries`, `last_error`, and `progress` fields
  to the `Task` model and ensure persistence via `TaskPersistenceService`.
- Upgrade `SchedulerService` to use differential synchronization with
  the persistent `SQLAlchemyJobStore` for better job durability.
- Implement structured heartbeat logging to support real-time progress
  updates.
- Update project documentation and ADRs to reflect the new plugin
  runtime and task resilience patterns.
- Add comprehensive unit and integration tests for the new task
  lifecycle features.
 2026-07-12 15:31:56 +03:00
busya
0cb1f80cd6 feat(git): implement deployment tracking and enhance lifecycle UX 
Introduce a deployment recording system to track dashboard versions
across environments and improve the Git management user experience.

- Add `Deployment` model and Alembic migration to persist deployment
  history.
- Implement `GitDeploymentRecorder` and `GitFingerprint` plugins to
  automate deployment logging and content hashing.
- Add `get_deployment_status` API endpoint to retrieve real-time
  environment states.
- Refactor `GitLifecycleHeader` to prioritize Call-to-Action (CTA)
  buttons and improve visual hierarchy.
- Update `GitWorkspacePanel` to emphasize version saving and
  streamline commit workflows.
- Enhance `GitEnvironmentTimeline` with deployment status integration,
  collapsible UI, and improved theme consistency.
- Add auto-navigation logic in `GitManagerModel` to guide users to
  relevant tabs based on recommended actions.
- Clean up obsolete documentation and update i18n strings for
  git visualization features.
 2026-07-12 14:57:03 +03:00
busya
b39d9991b9 fix(logs): resolve old prod execution issues + full GRACE-Poly compliance 
- translate: pure-skip scheduled runs now COMPLETE (not FAILED); extract _compute_final_status
- scheduler: cron validation at update; pre-validate + skip legacy bad expr (/15*) in load
- git: robust get_branch_commits (existence check for 'prod' to avoid fatal)
- health: use explore for network/auth failures (reduces spam)
- Full semantic protocol updates across modules:
  - complete #region/#endregion + @BRIEF/@PRE/@POST/@INVARIANT/@RELATION/@RATIONALE
  - belief_scope + CoT markers (reason/explore/reflect)
  - semantics-testing compliance in tests
- Addresses exact symptoms from container_backend.log and backend/logs/app.*

Related to 039-dashboard-scenario-ui log analysis.
 2026-07-10 12:35:07 +03:00
busya
3b13c67c0d feat(translate): language detection, async HTTP LLM, history model, agent improvements 
- Add async HTTP-based LLM transport (_llm_async_http.py)
- Add orthogonal LLM call tests
- Improve language detection (_lang_detect.py) and batch insert
- Update translate schemas, service utils, preview constants/prompts
- Add TranslateHistoryModel with pagination and filtering
- Update agent confirmation, persistence, langgraph setup, run, tools
- Improve LLM health checking in shared module
- Update translate runs API, history route
 2026-07-08 19:35:49 +03:00
busya
d0facf4a3a fix(translate): datasource change not persisted on save + preview column cleanup 
Root cause: saveJob() used stale sourceDatasourceId (set once on load)
instead of live datasourceId (updated by ConfigTabForm via $bindable).
Since sourceDatasourceId was always truthy, the || fallback to datasourceId
never triggered — the old datasource ID was always sent to PUT.

Fixes:
- Removed dead sourceDatasourceId atom; saveJob() uses datasourceId directly
- Bound sourceTable via $bindable through ConfigTabForm; updated on select
- loadDatasourceColumns() syncs databaseDialect from Superset columns API
- saveJob() sends undefined for database_dialect="unknown" to force re-detect
- Backend: added direct_db+connection_id validation on update (mirroring create)
- Removed redundant "Язык ист." column from TranslationPreview table
- Removed unused getDetectedLang function

Tests:
- TranslationJobModel: datasourceId save mapping, dialect sync, unknown→undefined
- TranslateJobService: reject direct_db without connection_id, preserve existing

Verified: browser — translate_cross datasource persisted after save+reload,
dialect detected as "clickhouse", columns loaded (2), translation column preserved.
 2026-07-07 23:50:08 +03:00
busya
34aeeb92a2 fix(backend+frontend): migration deadlock, async pattern, timeout safety, fire-and-forget translations 
Backend:
- MigrationPlugin.execute — remove AsyncJobRunner.run() deadlock on post-migration
  ID sync (2 deadlocks total: plugins/migration.py + api/routes/migration.py
  trigger_sync_now). Replace blocking runner.run with direct await.
- MigrationPlugin.execute — fix temp file leak (dry_run=True prevented cleanup).
- MigrationPlugin.execute — IdMappingService(SessionLocal()) now closed in finally.
- MigrationPlugin.execute — wire IdMappingService into MigrationEngine constructor
  so cross-filter patching actually works instead of silently skipping.
- MigrationPlugin.execute — SupersetClient.aclose() in finally to prevent
  httpx connection pool leak.
- TaskManager — _async_tasks dict leak (add_done_callback cleanup).
- JobLifecycle — CancelledError handler (tasks stuck in RUNNING after cancel).
- JobLifecycle — persist_task BEFORE _broadcast_task_status (crash consistency).
- JobLifecycle — wait_for_resolution/wait_for_input now have 3600s timeout.
- AsyncJobRunner.run — 300s timeout on future.result() (APScheduler thread safety).

Translate scheduler:
- execute_scheduled_translation — replace blocking runner.run(orch.execute_run(run))
  with fire-and-forget TranslationOrchestrator.execute_background(). APScheduler
  thread is freed in ms instead of blocking for the full translation duration.
  Translation runs of 10k+ rows (200+ LLM batches, hours) no longer hit the
  300s runner timeout.
- TranslationOrchestrator.execute_background — new static method: opens own DB
  session, dispatches asyncio.create_task, handles errors + notification.
- Scheduler last_run_at updated at dispatch time (not after completion).

Frontend:
- MigrationModel.stepReady[3] now requires dryRunResult != null (was always
  true, allowing UI to reach step 3 without dry-run).
- WizardModel.goToStep gate for step 3 uses stepReady[3].
- +page.svelte  — dryRunResult no longer self-clears via reactive loop.
- Progress bar step 3 indicator gate fixed for new readiness logic.

Tests:
- 2 new regression tests for migration sync (deadlock-free, completes cleanly).
- test_migration_plugin.py — _make_mock_superset_client/_make_mock_mapping_service
  helpers for proper async mock behavior (aclose, sync_environment AsyncMock).
- 10 scheduled-translation tests updated for fire-and-forget pattern.
- MigrationModel.test.ts — step 3 invariant + goToStep block test.
- All affected tests: 104 backend + 79 frontend = 183 passed.
 2026-07-07 21:04:23 +03:00
busya
3f6d7222c3 fix(agent): proxy gradio config and reduce translate blocking 2026-07-07 17:37:36 +03:00
busya
cfb95fa3c8 refactor(agent): extract agent+shared into standalone packages with full GRACE semantic markup 
- Move agent code from backend/src/agent/ to agent/src/ss_tools/agent/
- Extract shared stdlib-only utilities to shared/src/ss_tools/shared/
- Add #region/#endregion contracts to all ~140 functions (INV_1 compliance)
- Update docker files, entrypoint, build scripts for new package layout
- Backend now imports ss_tools.shared._llm_health (no gradio/langchain deps)
- Add specs for 036-039 feature plans
 2026-07-07 15:18:24 +03:00
busya
1182b29fe0 fix(certs): restore HTTP CA auto-download and test edge matrix 
Restore LLM_CA_CERT_URLS support for corporate PKI HTTP cert delivery.
Downloaded certificates are installed into the llm/ CA category and share
the same system trust + NSS import path as CERTS_PATH-mounted certs.

Changes:
- certs.sh: add download_llm_ca_certs with HTTP-only curl policy,
  PEM/DER handling, retries, llm/ storage, hash symlinks for llm+custom
- backend/agent entrypoints: run download before install_all_certs
- compose/env/docs: expose LLM_CA_CERT_URLS again and update ADR-0009
- integration tests: cover PEM/DER/CER, invalid certs, non-HTTP skips,
  query-string filenames, private/server skips, empty inputs, combined
  LLM_CA_CERT_URLS + CERTS_PATH channels, and NSS imports

Verified:
- 194 passed, 1 skipped integration tests
- full backend container smoke passed with testcontainers PostgreSQL
- docker bundle rebuilt for 0.5.0
 2026-07-07 10:23:49 +03:00