Commit Graph

27 Commits

Author SHA1 Message Date
7f7a85b2c5 refactor(validation): deduplicate image optimization, fix quality-reduction-before-chunking, unify chunk_count key
- Extract _optimize_images() helper to eliminate duplicate
  optimization code (was duplicated between initial pass and
  quality-reduction fallback)
- Move quality-reduction estimate to only apply when NOT chunking
  (each chunk fits the image limit by definition)
- Fix _merge_chunk_results to return 'chunk_count' instead of 'chunks'
  for consistency with plugin.py
- Simplify plugin.py: analysis.get('chunk_count', 1) instead of
  fallback chain
- Document that Kilo API gateway is incompatible with AsyncOpenAI
  image format (probe returns 0)
2026-05-31 22:57:58 +03:00
2760fa09ea feat(validation): chunk screenshots by max_images limit + fix websocket crash
- analyze_dashboard_multimodal now splits screenshots into chunks
  of max_images (from provider config) and sends them in parallel
- Results merged: worst status, deduped issues by (severity, msg, loc)
- New helper methods: _deduplicate_issues, _merge_chunk_results, _call_llm_for_images
- Plugin passes db_provider.max_images to the LLM client
- Report UI shows 'Chunked ×N' badge when analysis used multiple chunks
- i18n: added 'chunked' / 'По частям' key to validation.json
- Fix: isinstance(StopIteration) -> isinstance(_ws_exc, StopIteration)
  which crashed the websocket and broke task execution mid-flight
- Fix: update test mocks (_FakeLLMClient, _FakeScreenshotService)
2026-05-31 22:43:06 +03:00
d2b53c2a79 feat(validation): v2 LLM dashboard validation — plugin, routes, services
Core implementation of the v2 LLM dashboard validation pipeline:
- LLM plugin with Path A (screenshots) and Path B (logs-only) execution
- Validation task management (CRUD, schedule, run)
- WebSocket task progress with Python 3.13 asyncio fix
- Cross-task runs listing (GET /validation-tasks/runs/all)
- RecordResponse schema for validation records
- JSON prompt helper, per-dashboard status aggregation
- Prompt templates with docs/git-commit/validation presets
- Migration: v2 validation models + description column
- Tests: plugin persistence, prompt templates, batch, payload, url
2026-05-31 22:32:20 +03:00
8ecea09222 fix: capath instead of cafile for all LLM clients
OpenSSL 3.x ignores intermediate CA certs in -CAfile (verify code 20)
but correctly builds chains with -CApath (verify code 0).
All three clients now use capath:
  - service.py: ssl.create_default_context(capath=...)
  - _llm_http.py: verify='/etc/ssl/certs/'
  - preview_llm_client.py: verify='/etc/ssl/certs/'
check_llm_certs.py rewritten for clarity.
2026-05-29 11:02:48 +03:00
19dd447345 feat: ADR-0009 SSL cert management, fix certifi vs system CA
- ADR-0009: comprehensive SSL certificate management strategy
- _get_ssl_verify() returns SSLContext with cafile= instead of bool True
  (httpx deprecates verify=<str>, requests needs system CA, not certifi)
- _get_verify() in translate plugin returns system CA path
- All tests updated for SSLContext return type
- All QA findings C1-C3, H1-H4, M1 incorporated into ADR
2026-05-28 12:06:42 +03:00
ce49760e5a date fx 2026-05-27 16:05:04 +03:00
7ffddff2a6 feat: add LLM SSL verify control and auto CA cert download
- LLM_SSL_VERIFY env var to disable SSL verification for corporate proxies
- install_llm_ca_certs() entrypoint function — downloads PEM/DER certs
  from LLM_CA_CERT_URLS, converts DER→PEM, installs to system CA store
- _format_connection_error() — detailed exception chain logging
- 7 new unit tests for _get_ssl_verify, _format_connection_error, verify= param
- LLM_CA_CERT_URLS and LLM_SSL_VERIFY in .env.enterprise-clean
- Removed duplicate @RELATION DEPENDS_ON -> [EXT:Library:tenacity]
2026-05-27 14:44:46 +03:00
4205618ee6 chore: eliminate all deprecation warnings from tests and linter
Warnings fixed:
- datetime.utcnow() → datetime.now(UTC) across 48+ files (src/ + tests/)
- datetime.utcnow (callback ref) → lambda: datetime.now(UTC) in model fields (18 files)
- Pydantic class Config → model_config = ConfigDict(...) (16 files)
- Pydantic .dict() → .model_dump() (8 files)
- ConfigDict(allow_population_by_field_name=True) → validate_by_name=True
- SQLAlchemy declarative_base() import path updated
- FastAPI on_event → lifespan context manager (app.py)
- Import sorting (ruff I001) auto-fixed across all files
- Fixed broken re-export chains that ruff F401 cleanup broke:
  _validate_bcp47: service.py now imports from dictionary_validation directly
  job_to_response: _job_routes.py and test imports from service_utils directly
  fetch_datasource_metadata: restored re-export in service.py
- Added missing TranslateJobService import in _job_routes.py (was deleted by F401)
- Added ConfigDict(protected_namespaces=()) for DashboardDatasetItem schema field
- pytest.ini: replaced deprecated importmode with asyncio_mode

All 440 tests pass with zero deprecation warnings.
2026-05-26 19:18:28 +03:00
9ffa8af1dc semantics 2026-05-26 09:30:41 +03:00
31680a1bc9 fix(maintenance): auto-expiry + adaptive markdown height + tests
- Auto-expiry: expired events auto-end on GET /events via task manager
- Height: _estimate_markdown_height adapted to unit=8px scale with padding detection
- CRITICAL-1: removed dead import remove_chart_from_position (QA finding)
- CRITICAL-2: added chart alive check in ensure_banner_chart (QA finding)
- CRITICAL-3: fixed test assertions update_markdown_chart → update_banner_on_dashboard
- @POST contract: fixed return range [2,12] → [19,200]
- Tests: 8 new tests (auto-expiry + layout height)
2026-05-25 08:36:33 +03:00
48cc02ea0b feat: LLM Validation section — 9 API endpoints + 3 Svelte pages + Playwright optimization
Backend:
- /api/validation/ CRUD for validation tasks (policies)
- Run history with 6 filters + pagination
- Alembic migrations (provider_id, policy_id)
- 28 pytest tests

Frontend:
- /validation — task list page with status filters + CRUD
- /validation/[id] — task config (Config + History tabs)
- /validation/history — run history with metrics
- Sidebar nav entry under Operations
- i18n en/ru

Playwright (ScreenshotService):
- Removed 25s hardcoded sleeps → conditional waits
- CDP fallback to full_page screenshot
- Total timeout enforcement (120s)
- Debug screenshots → temp dir with cleanup

QA fixes:
- Debug screenshot accumulation in production (tempdir + rmtree)
- Frontend API payload field name mismatch
- History issues field reference fix
- delete_runs scoped by policy_id

Belief protocol:
- @RATIONALE/@REJECTED on 27 C4+ contracts

Closes: VALIDATION-REWORK-2026
2026-05-21 20:29:36 +03:00
68740cd9ed semantic 2026-05-20 23:54:53 +03:00
b916ef94d5 semantics 2026-05-19 18:36:15 +03:00
e754efc7bd feat(llm): add LiteLLM provider — enum, client, tests, QA follow-ups
- Add LITELLM = "litellm" to LLMProviderType enum (already in HEAD from prev commit)
- Add comment in LLMClient.__init__ explaining standard Bearer auth for LiteLLM
- Add regression test test_provider_type_enum_values — guard against value drift
- Add test_litellm_client_uses_default_bearer_auth — verify no extra headers
- Clean up duplicate @RELATION lines in models.py
- All 20 backend + 4 frontend tests pass
2026-05-15 22:11:55 +03:00
c6189876b3 chore(lint): apply ruff --fix (4443 auto-fixes)
Auto-fixed categories:
- F401: unused imports removed
- I001: import blocks sorted
- W293: trailing whitespace stripped
- UP035: deprecated typing imports replaced
- SIM: simplify suggestions applied
- ARG: unused args prefixed with underscore
- T201: print statements removed
- F841: unused variables removed
- RUF059: unpacked variables prefixed

Remaining ~1500 unfixable errors (C901, B904, N806, E402) require manual work.

Backend smoke tests: 13/13 passed.
2026-05-14 11:20:17 +03:00
a3c7c402b7 fix(llm): add fetch-models endpoint, fix SQL Lab INSERT (client_id truncation, sync mode, target_column, timestamp normalization)
- Add POST /api/llm/providers/fetch-models route with LLMClient.fetch_models()
- Add target_column to TranslationJob model/schema/service/orchestrator
- Fix SQL Lab execute: truncate client_id to 11 chars (varchar(11))
- Switch SQL Lab to sync mode (runAsync: false) — no Celery workers
- Fix polling: unwrap nested result from Superset query API
- Fix ClickHouse timestamp: normalize float timestamps to YYYY-MM-DD
2026-05-13 20:06:15 +03:00
39ab647851 semantics 2026-05-13 14:15:33 +03:00
306c5ae742 semantics: complete DEF-to-region migration, fix regressions
- Convert legacy [DEF🆔Type] anchors to #region/#endregion across 329 files
- Reinstate _normalize_timestamp_value in sql_generator.py
- Fix MarkerLogger→logger migration in events.py (molecular CoT markers)
- Fix dataset_review orchestrator dependencies (_build_execution_snapshot)
- Fix config_manager stale-record deletion (moved to save path only)
- Add 77 missing [/DEF:] closers in 5 unbalanced test files
- Update assistant_chat.integration.test.js for #region format
- Apply molecular-cot-logging markers (REASON/REFLECT/EXPLORE) via logger.* methods
2026-05-12 23:54:55 +03:00
fe8978f716 semantics 2026-05-12 20:06:16 +03:00
67867f8220 refactor(semantics): migrate legacy @TIER to @COMPLEXITY annotations
- Replaced @TIER: TRIVIAL with @COMPLEXITY: 1
- Replaced @TIER: STANDARD with @COMPLEXITY: 3
- Replaced @TIER: CRITICAL with @COMPLEXITY: 5
- Manually elevated specific critical/complex components to levels 2 and 4
- Ignored legacy, specs, and node_modules directories
- Updated generated semantic map
2026-03-16 10:06:44 +03:00
4efce79df5 Fix LLM validation and dashboard health hot paths 2026-03-15 13:18:51 +03:00
6ab2ea6f1c +ai update 2026-02-26 17:54:23 +03:00
8699ded112 fix(llm): skip unsupported json_object mode for openrouter stepfun models 2026-02-24 14:22:08 +03:00
6bded3adac chat worked 2026-02-23 20:20:25 +03:00
5ab1549cdd linter + новые таски 2026-02-10 12:53:01 +03:00
2d09ff4dc8 Похоже работает 2026-02-07 11:26:06 +03:00
161d70648c Вроде работает 2026-01-30 11:10:16 +03:00