busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
568 Commits 3 Branches 0 Tags
51afbee470d3e45ee93c1eec25ee2ba7ad2da95b
Commit Graph

43 Commits

Author SHA1 Message Date
busya
814f2da139 perf(translate): fix slow translation startup — CJK estimation, output budget, provider token config 
Root cause: batch sizing underestimated CJK token density (1.5→1.0 chars/token)
and ignored output budget as primary constraint, causing cascading finish_reason=length.

Changes:
- _token_budget.py: CJK_RATIO 1.5→1.0, OTHER_RATIO 2.2→1.8, safety factors 0.75/0.70
- _token_budget.py: new _compute_max_rows_by_output() — output budget is PRIMARY constraint
- _batch_sizer.py: resolve_provider_config() with DB-level context_window/max_output_tokens
- _batch_sizer.py: INPUT_SAFETY_FACTOR applied, max_rows_by_output used as row cap
- _llm_http.py: log actual usage.prompt_tokens/.completion_tokens from provider
- _llm_call.py: retry only missing rows after finish_reason=length (save partial result)
- models/llm.py + schema: provider-level context_window / max_output_tokens (nullable)
- services/llm_provider.py: get_provider_token_config() helper
- Alembic migration: add columns to llm_providers
- Svelte ProviderConfig: collapsible Advanced: Token Limits section
- 12 new tests (token budget, batch sizer, provider config)
- All 492 tests pass
 2026-06-03 23:25:08 +03:00
busya
ab90755fa1 fix(validation): process all dashboards in a run, not just the first one 
Root cause of stuck 'running' runs: the plugin's execute() method
only processed dashboard_ids[0] and returned. The remaining N-1
dashboards were never processed, and _update_run_status was called
after each single dashboard without checking dashboard_count.

Fixes:
- execute() now loops over ALL dashboard_ids and processes each
  via _execute_path_a or _execute_path_b
- _update_run_status is called once AFTER the loop finishes
- _update_run_status now checks that len(records) >= dashboard_count
  before marking the run as finished (prevents premature completion)
- Per-dashboard _update_run_status calls removed from _execute_path_a
  and _execute_path_b (the parent loop owns it now)
- Test updated for new batch return format {dashboards: [...], total: N}
 2026-05-31 23:03:46 +03:00
busya
2760fa09ea feat(validation): chunk screenshots by max_images limit + fix websocket crash 
- analyze_dashboard_multimodal now splits screenshots into chunks
  of max_images (from provider config) and sends them in parallel
- Results merged: worst status, deduped issues by (severity, msg, loc)
- New helper methods: _deduplicate_issues, _merge_chunk_results, _call_llm_for_images
- Plugin passes db_provider.max_images to the LLM client
- Report UI shows 'Chunked ×N' badge when analysis used multiple chunks
- i18n: added 'chunked' / 'По частям' key to validation.json
- Fix: isinstance(StopIteration) -> isinstance(_ws_exc, StopIteration)
  which crashed the websocket and broke task execution mid-flight
- Fix: update test mocks (_FakeLLMClient, _FakeScreenshotService)
 2026-05-31 22:43:06 +03:00
busya
d2b53c2a79 feat(validation): v2 LLM dashboard validation — plugin, routes, services 
Core implementation of the v2 LLM dashboard validation pipeline:
- LLM plugin with Path A (screenshots) and Path B (logs-only) execution
- Validation task management (CRUD, schedule, run)
- WebSocket task progress with Python 3.13 asyncio fix
- Cross-task runs listing (GET /validation-tasks/runs/all)
- RecordResponse schema for validation records
- JSON prompt helper, per-dashboard status aggregation
- Prompt templates with docs/git-commit/validation presets
- Migration: v2 validation models + description column
- Tests: plugin persistence, prompt templates, batch, payload, url
 2026-05-31 22:32:20 +03:00
busya
4205618ee6 chore: eliminate all deprecation warnings from tests and linter 
Warnings fixed:
- datetime.utcnow() → datetime.now(UTC) across 48+ files (src/ + tests/)
- datetime.utcnow (callback ref) → lambda: datetime.now(UTC) in model fields (18 files)
- Pydantic class Config → model_config = ConfigDict(...) (16 files)
- Pydantic .dict() → .model_dump() (8 files)
- ConfigDict(allow_population_by_field_name=True) → validate_by_name=True
- SQLAlchemy declarative_base() import path updated
- FastAPI on_event → lifespan context manager (app.py)
- Import sorting (ruff I001) auto-fixed across all files
- Fixed broken re-export chains that ruff F401 cleanup broke:
  _validate_bcp47: service.py now imports from dictionary_validation directly
  job_to_response: _job_routes.py and test imports from service_utils directly
  fetch_datasource_metadata: restored re-export in service.py
- Added missing TranslateJobService import in _job_routes.py (was deleted by F401)
- Added ConfigDict(protected_namespaces=()) for DashboardDatasetItem schema field
- pytest.ini: replaced deprecated importmode with asyncio_mode

All 440 tests pass with zero deprecation warnings.
 2026-05-26 19:18:28 +03:00
busya
f49b7d909e feat: GRACE-Poly protocol optimization — ~3200→10 warnings (↓99.7%) 
Full optimization cycle:

Protocol (15 files):
- 4-layer SSOT architecture for agent prompts & skills
- Anti-Corruption Protocol consolidated from 5 duplicates
- Tag-to-tier permissiveness matrix (all @tags allowed at all tiers)

Axiom config:
- complexity_rules: all 22+ tags available on C1-C5
- contract_type_overrides: removed (was narrowing per-type)
- 18 new tags added, LAYER enum expanded (Infra, Frontend, Atom, etc.)
- RELATION predicates expanded (USES, CONTAINS, BELONGS_TO, etc.)

Code fixes:
- 2216 @TAG: normalized to @TAG (colon→space)
- 518 [DEF] blocks migrated to #region/#endregion (37 files)
- VERIFIES→BINDS_TO, :Class/:Function suffixes removed, paths→IDs
- 1173-line _external_stubs.py deleted (EXT: handled natively)
- Batch EXT: reference audit (240 targets: 132 external, 99 internal, 9 fix)
- QA regression check: 0 regressions across all checks

Infrastructure:
- DuckDB rebuild stabilized (appender API, INSERT OR IGNORE)
- Anchor regex fix (parent-child BINDS_TO now resolves)
- EXT:*/DTO:/NEED_CONTEXT: regex fixed in validator
- 34MB Doxygen API portal (3194 contract pages)
 2026-05-26 11:14:25 +03:00
busya
9ffa8af1dc semantics 2026-05-26 09:30:41 +03:00
busya
68740cd9ed semantic 2026-05-20 23:54:53 +03:00
busya
084f782065 lang detect 2026-05-20 17:15:31 +03:00
busya
30ba70933d feat(llm): mask API keys in UI responses and prevent masked-key leakage in fetch/test/save payloads 
- Add mask_api_key() and is_masked_or_placeholder() to llm_provider service
- Return masked keys in all provider CRUD endpoints
- Reject masked/placeholder keys in fetch_models and test_provider_config
- Show masked key with Change button in ProviderConfig.svelte edit form
- Exclude masked keys from fetch-models, test, and submit payloads on frontend
- Update semantics-core skill with clarified complexity tier rules
- Switch agent modes from subagent to all
 2026-05-15 23:34:09 +03:00
busya
e754efc7bd feat(llm): add LiteLLM provider — enum, client, tests, QA follow-ups 
- Add LITELLM = "litellm" to LLMProviderType enum (already in HEAD from prev commit)
- Add comment in LLMClient.__init__ explaining standard Bearer auth for LiteLLM
- Add regression test test_provider_type_enum_values — guard against value drift
- Add test_litellm_client_uses_default_bearer_auth — verify no extra headers
- Clean up duplicate @RELATION lines in models.py
- All 20 backend + 4 frontend tests pass
 2026-05-15 22:11:55 +03:00
busya
c6189876b3 chore(lint): apply ruff --fix (4443 auto-fixes) 
Auto-fixed categories:
- F401: unused imports removed
- I001: import blocks sorted
- W293: trailing whitespace stripped
- UP035: deprecated typing imports replaced
- SIM: simplify suggestions applied
- ARG: unused args prefixed with underscore
- T201: print statements removed
- F841: unused variables removed
- RUF059: unpacked variables prefixed

Remaining ~1500 unfixable errors (C901, B904, N806, E402) require manual work.

Backend smoke tests: 13/13 passed.
 2026-05-14 11:20:17 +03:00
busya
a3c7c402b7 fix(llm): add fetch-models endpoint, fix SQL Lab INSERT (client_id truncation, sync mode, target_column, timestamp normalization) 
- Add POST /api/llm/providers/fetch-models route with LLMClient.fetch_models()
- Add target_column to TranslationJob model/schema/service/orchestrator
- Fix SQL Lab execute: truncate client_id to 11 chars (varchar(11))
- Switch SQL Lab to sync mode (runAsync: false) — no Celery workers
- Fix polling: unwrap nested result from Superset query API
- Fix ClickHouse timestamp: normalize float timestamps to YYYY-MM-DD
 2026-05-13 20:06:15 +03:00
busya
39ab647851 semantics 2026-05-13 14:15:33 +03:00
busya
306c5ae742 semantics: complete DEF-to-region migration, fix regressions 
- Convert legacy [DEF🆔Type] anchors to #region/#endregion across 329 files
- Reinstate _normalize_timestamp_value in sql_generator.py
- Fix MarkerLogger→logger migration in events.py (molecular CoT markers)
- Fix dataset_review orchestrator dependencies (_build_execution_snapshot)
- Fix config_manager stale-record deletion (moved to save path only)
- Add 77 missing [/DEF:] closers in 5 unbalanced test files
- Update assistant_chat.integration.test.js for #region format
- Apply molecular-cot-logging markers (REASON/REFLECT/EXPLORE) via logger.* methods
 2026-05-12 23:54:55 +03:00
busya
fe8978f716 semantics 2026-05-12 20:06:16 +03:00
busya
277ce7b3b3 skills + agents 2026-04-19 20:05:45 +03:00
busya
f5c90af287 semantics 2026-04-01 21:57:51 +03:00
busya
10c15d0e93 feat(ui): surface review readiness and blockers 
Add readiness hints, blocker summaries, and progress cues across
the dataset review workspace to clarify partial recovery and
launch gating states.

Highlight read-only preview snapshots, pending semantic review,
clarification queue status, and unresolved validation blockers to
guide the next recommended action.

Document known limitations in backend test doubles so permission,
task dispatch, and uncovered persistence branches are explicit.
 2026-03-21 15:48:28 +03:00
busya
e86204df6c test semantics 2026-03-21 15:23:06 +03:00
busya
2da548fd71 fix: finalize semantic repair and test updates 2026-03-21 15:07:06 +03:00
busya
81406bc2e2 fix: commit semantic repair changes 2026-03-21 11:22:25 +03:00
busya
cf601a717e fix: commit verified semantic repair changes 2026-03-21 11:05:20 +03:00
busya
67867f8220 refactor(semantics): migrate legacy @TIER to @COMPLEXITY annotations 
- Replaced @TIER: TRIVIAL with @COMPLEXITY: 1
- Replaced @TIER: STANDARD with @COMPLEXITY: 3
- Replaced @TIER: CRITICAL with @COMPLEXITY: 5
- Manually elevated specific critical/complex components to levels 2 and 4
- Ignored legacy, specs, and node_modules directories
- Updated generated semantic map
 2026-03-16 10:06:44 +03:00
busya
bcce55fbf9 feat add connections management and health summary improvements 2026-03-15 16:40:43 +03:00
busya
e047efb8cc fix dashboard validation fallback and semantic relation parsing 2026-03-15 16:32:39 +03:00
busya
4efce79df5 Fix LLM validation and dashboard health hot paths 2026-03-15 13:18:51 +03:00
busya
5e6fe2fe61 security: rotate bootstrap and clean workspace 2026-03-13 12:14:37 +03:00
busya
80102661db semantic clean up 2026-03-10 19:38:10 +03:00
busya
b4b17d89b4 код написан 2026-03-10 12:00:18 +03:00
busya
145a8642ad fix(dashboards): normalize naive/aware datetimes in resource task ordering 2026-03-10 09:29:40 +03:00
busya
52195f3f01 feat(rbac): auto-sync permission catalog from declared route/plugin guards 2026-03-06 11:30:58 +03:00
busya
0c68fbcef5 test: remediate and stabilize auxiliary backend and frontend tests 
- Standardized task log, LLM provider, and report profile tests.
- Relocated auxiliary tests into __tests__ directories for consistency.
- Updated git_service and defensive guards with minor stability fixes discovered during testing.
- Added UX integration tests for the reports list component.
 2026-03-04 13:54:06 +03:00
busya
4aba3df734 причесываем лог 2026-02-28 10:47:19 +03:00
busya
7f96a983a9 [ 
{
    "file": "backend/src/api/routes/__tests__/test_dashboards.py",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "All 9 previous findings remediated. @TEST_FIXTURE data aligned, all @TEST_EDGE scenarios covered, all @PRE negative tests present, all @SIDE_EFFECT assertions added. Full contract compliance."
  },
  {
    "file": "backend/src/api/routes/__tests__/test_datasets.py",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "All 6 previous findings remediated. Full @PRE boundary coverage including page_size>100, empty IDs, missing env. @SIDE_EFFECT assertions added. 503 error path tested."
  },
  {
    "file": "backend/src/core/auth/__tests__/test_auth.py",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "All 4 previous findings remediated. @SIDE_EFFECT last_login verified. Inactive user @PRE negative test added. Empty hash edge case covered. provision_adfs_user tested for both new and existing user paths."
  },
  {
    "file": "backend/src/services/__tests__/test_resource_service.py",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "Both prior recommendations implemented. Full edge case coverage for _get_last_task_for_resource. No anti-patterns detected."
  },
  {
    "file": "backend/tests/test_resource_hubs.py",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "Pagination boundary tests added. All @TEST_EDGE scenarios now covered. No anti-patterns detected."
  },
  {
    "file": "frontend/src/lib/components/assistant/__tests__/assistant_chat.integration.test.js",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "No changes since previous audit. Contract scanning remains sound."
  },
  {
    "file": "frontend/src/lib/components/assistant/__tests__/assistant_confirmation.integration.test.js",
    "verdict": "APPROVED",
    "rejection_reason": "NONE",
    "audit_details": {
      "target_invoked": true,
      "pre_conditions_tested": true,
      "post_conditions_tested": true,
      "test_data_used": true
    },
    "feedback": "No changes since previous audit. Confirmation flow testing remains sound."
  }
]
 2026-02-27 09:59:57 +03:00
busya
eb512a63a0 test now STANDARD tier 2026-02-26 18:38:26 +03:00
busya
f850d5d28d { "verdict": "APPROVED", "rejection_reason": "NONE", "audit_details": { "target_invoked": true, "pre_conditions_tested": true, "post_conditions_tested": true, "test_data_used": true }, "feedback": "Both test files have successfully passed the audit. The 'task_log_viewer.test.js' suite now correctly imports and mounts the real Svelte component using Test Library, fully eliminating the logic mirror/tautology issue. The 'test_logger.py' suite now properly implements negative tests for the @PRE constraint in 'belief_scope' and fully verifies all @POST effects triggered by 'configure_logger'." } 2026-02-24 21:55:13 +03:00
busya
fffcc73fb9 fix(validation): respect settings-bound provider and correct multimodal heuristic 2026-02-24 16:04:14 +03:00
busya
4bc023c2fa fix(llm-validation): accept stepfun multimodal models and return 422 on capability mismatch 2026-02-24 16:00:23 +03:00
busya
e97e19fc31 feat(assistant): add multi-dialog UX, task-aware llm settings, and i18n cleanup 2026-02-23 23:45:01 +03:00
busya
6bded3adac chat worked 2026-02-23 20:20:25 +03:00
busya
f1f46f4e11 semantic update 2026-02-23 13:15:48 +03:00
busya
d89c92ad35 tests ready 2026-02-19 13:33:20 +03:00