busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
1,180 Commits 3 Branches 19 Tags
master
Commit Graph

36 Commits

Author SHA1 Message Date
busya
3c99e3ca77 feat(dashboard-testing): add chart health diagnostics and per-tab evidence 
Focused matrix and cleanup checks pass. Native Superset error/recovery acceptance and final semantic freeze remain open; this is not release GO.
 2026-10-02 16:23:22 +03:00
busya
54bdbcc403 refactor(semantics): close dashboard testing protocol audit 2026-10-02 12:47:12 +03:00
busya
bcc69f4bbe feat(dashboard-testing): add sampled traversal and ClickHouse test lab 
Add versioned metric graph authority, owned browser evidence, paginated and all-tab traversal, deterministic sampling policies, and analyst-facing run inspection.

Provision the DEV/PREPROD/PROD Superset, Gitea and million-row ClickHouse lab; retain reproducible lifecycle evidence and explicit incomplete-traversal limits.
 2026-10-02 10:54:42 +03:00
busya
4d5ef6bed0 feat: dashboard-testing UX remediation + admin MCP governance + i18n sweep 
- UX audit Fix 1-5: actionable MCP HandoffSurface (endpoint/discovery/onboarding), reachable
  approval loop (sidebar section + waiting-for-me badge), honest MCP entry labels, ROUTES SSOT
  for dashboard-testing/load-testing (+ link-integrity coverage)
- scenario registry hub (/dashboard-testing/scenarios) built on ScenarioRegistryModel
- admin MCP governance: read-only GET /api/admin/mcp/catalog (catalog x roles + DCR clients) +
  /admin/mcp page/model/api + ROUTES.admin.mcp + sidebar entry
- full i18n sweep of dashboard-testing routes and scenario-* component trees (ru/en)
- accumulated workspace: MCP OAuth/DCR, automation idempotency migration, docker/nginx, specs
 2026-09-05 20:16:26 +03:00
busya
65121cac6b feat(logging): self-diagnosing EXPLORE + shared/ absorption + belief analytics (ADR-0021/0022) 
T0: absorb shared/ into backend — cot_logger→src/core, CotJsonFormatter→src/core/cot_formatter.py, _llm_http/_llm_health/ssl→src/core/utils; imports rewritten (26 prod + tests, patch targets); run.sh/backend.Dockerfile/requirements/.axiom source_dirs/semantic_health/AGENTS/INSTALL cleaned; ADR-0022 supersedes ADR-0015; fixed latent CI defects (ss_tools ImportError, record.message in logger tests, same-name test-module collision).

ADR-0021 wire enrichment (additive): contract_id/claim/error_code/loc fields; _contract_id ContextVar + resolve_contract_id (explicit > belief_scope > declared-src mirror, derived src never mirrors); EXPLORE auto-loc via single frame walk; facade error auto-fill; 2KB payload cap with payload_truncated/payload_bytes markers; migrated 85 error="CODE" sites to error_code= (12 files); pilot editor/load.py; superset preview payload-bomb inlined bodies removed.

Analytics SSOT src/core/log_stats.py (bond transition matrix, orphan-EXPLORE ratio, REFLECT pairing, intent families, coverage, insufficient-sample flag); pretty_cot.py --stats/--digest/--trajectory/--story over one engine; log_gap_service three-tier ground-truth triangulation (FAILED w/o EXPLORE etc.) + GET /api/reports/log-stats|task-log-gaps (polling-suppressed); scripts/cot_audit.py CLI; enriched fields persisted into task_logs.payload for tier queries.

Frontend: ReportsAnalyticsModel + AnalyticsStatsPanel (Logs tab) + TaskGapPanel and per-row T1/T2/T3 gap badges (Tasks tab); cot-logger.ts ADR-0021 opts; i18n en/ru. Scheduler console spam fixed: apscheduler logger demoted to WARNING via LoggingConfig.scheduler_log_level. .axiom belief patterns -> $OBJ.* (alias undercount). molecular-cot-logging skill updated (fields, decision rules, tie-break, CLI) and synced.

Reviewed orthogonally: F1 cot_span contract pollution, F2 cap boundary accounting, F3 digest over-dedup, F4 trace-state bound, F5 tier metadata — fixed with regression tests. Validation: backend 11287 passed + ruff + compileall; frontend 3446 passed + lint + build; CLI smoke on live app.log.
 2026-09-04 20:56:41 +03:00
busya
731aaaa8df feat(mcp): 050 unified MCP interface — parity tools, durable gates, authoring E2E, assistant decommission 
044 provider runtime completion (pre-staged workstream): capacity/operator
stores, provider ops/protocol/reconciler/dispatch revalidation, exploration
sandbox runtime, alembic 0014-0016, live canary evidence (browser provider
6/6 against the live stand).

050 Phases 0-2: RBAC FastMCP server with a 45-tool explicit catalog,
OAuth/DCR transport guards with bounded bodies, per-call provenance
(McpToolInvocationRecord), durable ActionApprovalGate + CAS decide +
leased/fenced poller, authoring workspace ops, bounded-response discipline,
hidden-vs-gated matrices.

Parity domains (T012-T014): gated git/deploy/migration/backup/llm tools with
reviewed dispatch adapters in explicit poll chains; Superset reads/writes with
a dedicated plugin:superset_sql risk class (terminal PROD denial via the
canonical execution-policy criterion, hardened danger-SQL guard covering
INTO/CALL/SET/REFRESH/file primitives/multi-statement); baseline 037 tools
over the shared REST-surface services.

Evidence (T016/T023/T028): REST-vs-MCP field parity on shared 037 fixtures;
vertical E2E from tools/list through registry revision activation with real
scenario:EDIT RBAC; sandbox-to-revision promotion E2E with unsafe-payload and
caller-digest rejection; dispatcher soak (three poll cycles, exactly-once).

Orthogonal QA+security audit hardening: enforced response_limit fail-closed
envelope, poisoned-exploration fail-closed (EXPLORATION_TARGET_UNRESOLVED),
sha256 exploration evidence digests, actor-UUID task ownership, is_active
guard on baseline consume, 038 resolver description=None selector fix.

Phase 3/4 decommission: HandoffSurface behind the MCP_DECOMMISSION flag,
then unconditional removal — agent/ service tree, chat components/models/
stores/types, gradio proxies (vite + nginx), agent service in run.sh,
docker-compose profiles, build.sh bundles; /agent renders the handoff only.
Docs: AGENTS.md/INSTALL.md two-service rewrite; 036-047 drift amendments
marked done; WORKSTATE checkpoints with all evidence.

Suites: backend 11199 passed / 240 skipped / 1 xpassed; frontend 3454 passed
(197 files), lint 0 errors, build OK; browser E2E login+handoff 6/6 twice on
the isolated compose stack (no 7860); ruff/compileall clean.

Misc: gitignore hardening (tmp/, tool model cache); E2E selector repairs
(nav strict-mode, invalid-credentials passthrough detail).
 2026-09-03 07:37:14 +03:00
busya
341d54399a fix(semantics): repair GRACE-Poly anchors, relations, and metadata 
Repair broken/unclosed semantic-protocol markup across the codebase:

- INV_3 region pairing: restore 87 bare '# #endregion' (6 test_structure_*
  files) and 321 bare '// #endregion' (13 frontend test files) with their
  matching IDs; hand-fix 26 Python + 12 frontend + 2 E2E files with
  mismatched/missing/premature/duplicate closes; fully migrate
  test_orthogonal_fixes.py from single- to double-hash anchors.
  Workspace-wide stack-pair scan now passes 6569 code files with 0 problems.
- Relations: normalize legacy predicates USES/CONTAINS/BELONGS_TO to
  canonical DEPENDS_ON/BINDS_TO (28 edits).
- Metadata: normalize @LAYER and [TYPE] enum values to the canonical
  vocabulary (App->Application, lib->Infra, Service/Route/Checklist/
  Declaration/Decorator/Functions -> Module/Function/Block, etc.).
- Config: extend tags.LAYER.enum; fix tools.py prose '#region' phantom that
  broke anchor nesting.
- Remove dead scratch scripts backend/_convert_defs.py, _batch_convert_defs.py.

Index refresh: contract count 9576 -> 10821; schema_unclosed_anchor
1179 -> 729 (remaining 729 are markdown legacy [DEF:]/brace anchors, a
separate migration phase).
 2026-09-01 12:24:42 +03:00
busya
0415a2ed7d chore: accumulate uncommitted workspace changes 2026-08-26 17:03:22 +03:00
busya
80dad15458 fix(maintenance): render banner as native MARKDOWN element, not a chart 
Two defects fixed:
- ensure_banner_chart created an orphan 'Maintenance Banner' markdown chart
  (polluted Charts menu and dashboard exports). The banner is now a native
  MARKDOWN element in position_json; chart_id is a synthetic layout key.
- insert_banner_markdown_at_top blindly targeted GRID_ID; on ROOT->TABS
  dashboards (FI-0085) GRID_ID is orphaned and the banner never rendered.
  The layout is now normalized to ROOT->GRID_ID->[ROW-banner, ...] with
  recursive parents update, matching the proven-working manual example.

Review-driven hardening:
- liveness check verifies reachability from ROOT (children graph), so
  dashboards corrupted by the old bug self-heal on the next start.
- insert removes stale ROW-banner-*/MARKDOWN-banner-* keys (single banner).
- decision memory (@RATIONALE/@REJECTED) added to both modules.
- new ops script scripts/cleanup_maintenance_banner_charts.py (dry-run by
  default) deletes already-created bogus banner charts in prod.
 2026-08-04 08:46:22 +07:00
root
632b730fff chore: migrate GRACE-Poly anchors to hierarchical dotted naming 
Systematic rename of all semantic anchors (#region, [DEF], @RELATION)
across 1400+ files — backend Python, frontend Svelte/TS, specs, docs:
- Flat anchors become Namespace.Module.Entity
- @RELATION references updated to match new anchor paths
- Zero business logic changes
 2026-07-22 11:48:15 +03:00
busya
d1d2a0f92e feat: tiered test infrastructure with Makefile + smart selector + OpenCode commands 
Root Makefile with timeout-protected test targets:
  - Tier 1 (<30s):   make test, make test-unit, make test-frontend
  - Tier 2 (smart):  make test-related F=file.py (via @RELATION BINDS_TO)
  - Tier 3 (<5min):  make test-integration (Docker, --run-integration)
  - Coverage:        make coverage (backend + frontend)
  - Lint:            make lint (ruff + eslint)

Smart test selector (scripts/find-related-tests.py):
  - Extracts module names from #region anchors, class/function defs
  - Searches 400 BINDS_TO entries across all test files
  - Confidence scoring: exact > case-insensitive > substring > heuristic
  - Fallback: filename-based fuzzy matching

OpenCode commands:
  - /test.all      — full suite + coverage
  - /test.unit     — fast unit tests (<30s)
  - /test.related  — smart selection by file
  - /test.coverage — coverage reports with thresholds

Speckit workflow updated:
  - speckit.test.md:   raw pytest → make targets with timeout safety
  - speckit.plan.md:   quickstart uses make targets
  - speckit.tasks.md:  verification uses make targets
  - speckit.implement.md: default stack uses make targets

Frontend: added 'coverage' script to package.json
 2026-07-21 18:24:53 +03:00
busya
a39a76c87f feat(agent-centric-logging): consolidate CoT infra in shared, close REASON→REFLECT chains 
- shared/cot_logger.py is SSOT; backend/cot_logger.py deleted
- elapsed_ms timing in all REFLECT markers
- Frontend: REASON→REFLECT/EXPLORE in all fetch/post/delete/requestApi
- Dynamic src: route.GET.api.plugins instead of hardcoded api.request_handler
- trace_id generated immediately (no 'no-trace'), X-Trace-ID in both directions
- Global error handlers (window error + unhandledrejection + error.svelte)
- Fixed duplicate logging (shared/logger.py double StreamHandler)
- propagate=False in configure_logger (was in ConfigManager = duplicated startup logs)
- belief_scope: 'Coherence OK' → '{anchor}: completed' + elapsed_ms
- Fixed 28 pre-existing test failures (scheduler sig, DB columns, DRAFT validation, etc)
 2026-07-12 19:30:57 +03:00
busya
702ee25ae4 fix(docker): add missing certs.sh COPY to backend/all-in-one Dockerfile 
Backend entrypoint sources certs.sh for CA certificate installation,
but the file was never copied into the Docker image — causing
container crash loop on startup (regression in 7f932610).

Changes:
- docker/backend.Dockerfile, docker/all-in-one.Dockerfile: COPY certs.sh
- docker/backend.entrypoint.sh: remove dead install_llm_ca_certs,
  install_certificates (replaced by docker/certs.sh); update @INVARIANT
- backend/src/core/ssl.py: fix describe_context() to report actual
  system cert count (SSLContext.capath attr does not exist in Python 3)
- __tests__: rewrite stale tests asserting LLM_SSL_VERIFY=false →
  verify=False; behaviour is permanently removed — always CERT_REQUIRED
- backend/tests/integration/test_backend_container.py [new]: 5 tests
  verifying certs.sh presence, sourceability, and full-stack smoke
  (testcontainers PG → entrypoint → migrations → health)
- conftest.py: restore health-wait loop and fixtures in superset_container
- ADR-0009, README.md, scripts, .env: align docs with centralized SSL
 2026-07-07 00:58:41 +03:00
busya
7f93261060 refactor(ssl): centralize SSL trust management, remove LLM_SSL_VERIFY 
Centralized SSL via one contract: CERTS_PATH=/opt/certs mounted into all containers.

Backend:
  - NEW: backend/src/core/ssl.py — system_ssl_context(), httpx_verify(),
    cert_dir_inventory()
  - LLMClient._get_ssl_verify() → delegates to core.ssl
  - _llm_async_http._get_verify() → delegates to core.ssl
  - Removed LLM_SSL_VERIFY env reading from all runtime code

Docker:
  - NEW: docker/certs.sh — shared cert installer (PEM/DER/cer to .crt conversion,
    update-ca-certificates, hash symlinks, NSS import)
  - NEW: docker/agent.entrypoint.sh — agent entrypoint with cert installation
  - backend.entrypoint.sh → uses certs.sh instead of install_llm_ca_certs
  - Dockerfile.agent → adds ca-certificates, openssl, entrypoint

Compose:
  - Removed LLM_CA_CERT_URLS and LLM_SSL_VERIFY from all compose files
  - Added CERTS_PATH volume mount to agent (dev + enterprise)
  - Added certs volume mount to backend/agent in dev compose

Env examples:
  - Removed LLM_SSL_VERIFY, LLM_CA_CERT_URLS from .env.example,
    .env.enterprise-clean.example, .env.current.example, .env.master.example,
    backend/.env.example
  - Enhanced CERTS_PATH comments with accepted formats

Diagnostics:
  - diag_container.py: removed LLM_* checks, added CERTS_PATH inventory,
    uses core.ssl for context creation

Tests:
  - Updated test_llm_analysis_service, test_llm_async_http,
    test_client_headers to verify centralized ssl context (no env disable)
  - 4/4 SSL tests pass
 2026-07-06 21:00:28 +03:00
busya
4590e1d3cd fix(scripts): fix --target arg parsing in diag_container.py 2026-07-06 20:37:22 +03:00
busya
912a3f0f70 feat(scripts): add container diagnostic script for SSL cert + encryption health 
Checks 6 categories:
  1. Environment — ENCRYPTION_KEY, LLM_SSL_VERIFY, LLM_CA_CERT_URLS
  2. System CA store — ca-certificates.crt, custom/LLM certs, hash symlinks
  3. OpenSSL connectivity — cafile vs capath (per ADR-0009 Finding 7)
  4. Python SSLContext(capath) — what _get_ssl_verify() uses
  5. httpx(capath) — integration test
  6. Encryption key health — decrypt test for all LLM provider api_keys

Usage:
  docker cp scripts/diag_container.py superset-tools-backend-1:/tmp/
  docker compose exec backend python3 /tmp/diag_container.py --target lite.ai.rusal.com:443
 2026-07-06 20:27:03 +03:00
busya
2f1916706a rename ss-tools to superset-tools across the entire project 
- Replace all occurrences of 'ss-tools' with 'superset-tools' in 104 files
- Rename git bundle file ss-tools.bundle → superset-tools.bundle
- Update .gitignore pattern accordingly
- Preserve variable names (hasSsTools etc.) and code identifiers
 2026-06-16 11:15:19 +03:00
busya
1e85e57cbd feat(coverage): add coverage-summary script, README section, ADR-0013 
- scripts/coverage-summary.sh — unified coverage summary generator
  Runs pytest+coverage (unit/integration) and vitest+coverage,
  parses results, generates single HTML report with both stacks.
  Supports --unit, --backend-only, --frontend-only, --output-dir.
- README.md — add 'Покрытие кода' sub-section under Тестирование
- docs/adr/ADR-0013-coverage-reporting.md — architectural decision record
 2026-06-15 13:40:22 +03:00
busya
cb1a5a4f13 fix: capath instead of cafile for all LLM clients 
OpenSSL 3.x ignores intermediate CA certs in -CAfile (verify code 20)
but correctly builds chains with -CApath (verify code 0).
All three clients now use capath:
  - service.py: ssl.create_default_context(capath=...)
  - _llm_http.py: verify='/etc/ssl/certs/'
  - preview_llm_client.py: verify='/etc/ssl/certs/'
check_llm_certs.py rewritten for clarity.
 2026-05-29 11:02:48 +03:00
busya
9186b2abe0 fix: check_llm_certs.py — fixes from user review 
- openssl s_client: stdin=EOF to prevent hang (input='')
- argparse for --target CLI argument
- in_bundle check: awk+openssl per-cert fingerprint comparison
- NamedTemporaryFile: mode='wb' for binary writes
- Added httpx_capath and requests_cafile_capath tests
- Graceful handling of empty ca-certificates.crt
 2026-05-28 23:22:53 +03:00
busya
48aa0b27b8 fix: check_llm_certs.py — handle DER files, openssl timeout 20s, binary PEM reads 2026-05-28 23:10:18 +03:00
busya
21f3b29699 fix: check_llm_certs.py — clean main() structure, safe imports 2026-05-28 22:58:37 +03:00
busya
730ee1b165 feat: rewrite SSL diag in Python — tests all real libraries 
Tests all 4 SSL methods with actual project libraries:
  - httpx (verify=True/False/SSLContext cafile/cafile+capath)
  - requests (verify=True/False/cafile/capath)
  - openssl s_client (default/CAfile/CApath/chain bundle)
  - certutil (NSS/Chromium)
  - certifi vs system CA comparison
Outputs JSON summary with AI diagnosis hint.
 2026-05-28 22:57:08 +03:00
busya
2552cb7546 fix: rewrite check_llm_certs.sh — tests all methods AI needs 
Tests 4 variants: openssl s_client (CAfile, CApath, default),
Python httpx (cafile, cafile+capath), requests (cafile, capath),
NSS certutil. Output format is machine-parseable for AI diagnosis.
 2026-05-28 22:56:19 +03:00
busya
3f6b06b4b8 feat: add SSL certificate diag script (check_llm_certs.sh) 
Runs from inside Docker container to verify all 4 SSL layers:
1. System CA store (openssl s_client)
2. NSS database (certutil) for Chromium
3. Python httpx (SSLContext with cafile)
4. Python requests (verify with system CA path)
Reads LLM_SSL_VERIFY and CA paths from .env.enterprise-clean
 2026-05-28 12:13:44 +03:00
busya
d5cf5f0cf7 semantics 2026-05-26 09:30:41 +03:00
busya
8212f8d8c6 fix(translate): fix batch sizing — use real available input budget, respect job.batch_size, lazy playwright in docker 
- _batch_sizer.py: compute per_batch_budget from actual available_input_budget
  (context_window - max_output_tokens) instead of sum of first N rows
- _batch_sizer.py: cap max_rows_hard_cap with job.batch_size (user config)
- _token_budget.py: add available_input_budget and max_output_tokens to return
- preview.py: validate required config fields before preview
- requirements-docker.txt: add playwright pip package (~5 MB)
- backend.entrypoint.sh: lazy playwright install chromium on first start
- build.sh: switch tar to tar.xz (-T0 -9), 5.5x smaller bundles
- README.md: add offline bundle deployment instructions
- playwright.config.js: add testMatch for *.e2e.js files
- frontend: preview tab config validation, i18n keys, translationRun store
 2026-05-17 23:32:00 +03:00
busya
ecb3b11f22 fix(translate): Kilo API response_format, reasoning_effort skip, structured_outputs fallback, refusal handling 
- Enable response_format (json_object) for all providers including Kilo/OpenRouter
- Skip reasoning_effort for Kilo/OpenRouter (returns 400 — mandatory reasoning)
- Add structured_outputs fallback: retry once without response_format if upstream
  provider (e.g. StepFun) rejects it with 'structured_outputs is not supported'
- Handle model refusal field (refusal) instead of treating as empty content
- Fix None-handling guards for message/finish_reason/content fields
- Apply same fixes to both preview.py and executor.py _call_openai_compatible
- Connect orphaned TermCorrectionPopup, BulkReplaceModal, BulkCorrectionSidebar
 2026-05-15 18:12:29 +03:00
busya
99006e5e07 fix(translate): add EXPLORE logging for LLM JSON parse failures 
Preview + executor now log the raw LLM response (first 1000 chars)
when JSON parsing fails, enabling debugging of malformed LLM output.

Previously ValueErrors from JSON parse failures were silently returned
as 400 without any diagnostic data in logs.
 2026-05-15 09:15:36 +03:00
busya
ffcfec9acf semantics 2026-05-13 14:15:33 +03:00
busya
07c064bb86 fix: finalize semantic repair and test updates 2026-03-21 15:07:06 +03:00
busya
978fe7b34d fix: commit semantic repair changes 2026-03-21 11:22:25 +03:00
busya
7ecadcb83d security: rotate bootstrap and clean workspace 2026-03-13 12:14:37 +03:00
busya
9bd2a7001b Add docker admin bootstrap for clean release 2026-03-13 11:41:44 +03:00
busya
6f8da29986 chore: include docker image metadata in offline bundle manifest 2026-03-11 12:40:54 +03:00
busya
683bd8c370 feat: add offline docker bundle for enterprise clean releases 2026-03-11 12:35:01 +03:00