busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
1,158 Commits 3 Branches 19 Tags
9acd9badabd2ca299d7e843e593df5c90a3cc798
Commit Graph

30 Commits

Author SHA1 Message Date
busya
5758ae4a83 fix(database): object-level reset and optional DB_SCHEMA isolation 
Safe one-shot reset, fail-fast on unknown revisions, object-level drops for non-owner corporate PG, rollback before advisory unlock, configurable DB_SCHEMA via search_path, e2e migration matrix tests.
 2026-09-11 16:17:11 +03:00
busya
4d5ef6bed0 feat: dashboard-testing UX remediation + admin MCP governance + i18n sweep 
- UX audit Fix 1-5: actionable MCP HandoffSurface (endpoint/discovery/onboarding), reachable
  approval loop (sidebar section + waiting-for-me badge), honest MCP entry labels, ROUTES SSOT
  for dashboard-testing/load-testing (+ link-integrity coverage)
- scenario registry hub (/dashboard-testing/scenarios) built on ScenarioRegistryModel
- admin MCP governance: read-only GET /api/admin/mcp/catalog (catalog x roles + DCR clients) +
  /admin/mcp page/model/api + ROUTES.admin.mcp + sidebar entry
- full i18n sweep of dashboard-testing routes and scenario-* component trees (ru/en)
- accumulated workspace: MCP OAuth/DCR, automation idempotency migration, docker/nginx, specs
 2026-09-05 20:16:26 +03:00
busya
731aaaa8df feat(mcp): 050 unified MCP interface — parity tools, durable gates, authoring E2E, assistant decommission 
044 provider runtime completion (pre-staged workstream): capacity/operator
stores, provider ops/protocol/reconciler/dispatch revalidation, exploration
sandbox runtime, alembic 0014-0016, live canary evidence (browser provider
6/6 against the live stand).

050 Phases 0-2: RBAC FastMCP server with a 45-tool explicit catalog,
OAuth/DCR transport guards with bounded bodies, per-call provenance
(McpToolInvocationRecord), durable ActionApprovalGate + CAS decide +
leased/fenced poller, authoring workspace ops, bounded-response discipline,
hidden-vs-gated matrices.

Parity domains (T012-T014): gated git/deploy/migration/backup/llm tools with
reviewed dispatch adapters in explicit poll chains; Superset reads/writes with
a dedicated plugin:superset_sql risk class (terminal PROD denial via the
canonical execution-policy criterion, hardened danger-SQL guard covering
INTO/CALL/SET/REFRESH/file primitives/multi-statement); baseline 037 tools
over the shared REST-surface services.

Evidence (T016/T023/T028): REST-vs-MCP field parity on shared 037 fixtures;
vertical E2E from tools/list through registry revision activation with real
scenario:EDIT RBAC; sandbox-to-revision promotion E2E with unsafe-payload and
caller-digest rejection; dispatcher soak (three poll cycles, exactly-once).

Orthogonal QA+security audit hardening: enforced response_limit fail-closed
envelope, poisoned-exploration fail-closed (EXPLORATION_TARGET_UNRESOLVED),
sha256 exploration evidence digests, actor-UUID task ownership, is_active
guard on baseline consume, 038 resolver description=None selector fix.

Phase 3/4 decommission: HandoffSurface behind the MCP_DECOMMISSION flag,
then unconditional removal — agent/ service tree, chat components/models/
stores/types, gradio proxies (vite + nginx), agent service in run.sh,
docker-compose profiles, build.sh bundles; /agent renders the handoff only.
Docs: AGENTS.md/INSTALL.md two-service rewrite; 036-047 drift amendments
marked done; WORKSTATE checkpoints with all evidence.

Suites: backend 11199 passed / 240 skipped / 1 xpassed; frontend 3454 passed
(197 files), lint 0 errors, build OK; browser E2E login+handoff 6/6 twice on
the isolated compose stack (no 7860); ruff/compileall clean.

Misc: gitignore hardening (tmp/, tool model cache); E2E selector repairs
(nav strict-mode, invalid-credentials passthrough detail).
 2026-09-03 07:37:14 +03:00
busya
4b9619e229 refactor(deploy): standardize enterprise environment file 2026-08-25 15:30:10 +03:00
busya
a2ede492b7 fix(database): reset any legacy Alembic revision 2026-08-25 15:21:42 +03:00
busya
5bc1b62bd0 refactor: unify initialization and reset migration baseline 2026-08-25 11:42:31 +03:00
busya
9cb5717a78 fix(agent): auto-start scenario chat, robust HITL resume, strict service auth 
- frontend: fix auto-start on dashboards->/agent navigation (undefined params
  ReferenceError), route initial connect through ConnectionManager with auto-retry,
  reset runModel on objectId change and failed recovery
- agent: fix closure-over-loop-variable bug in _inject_env_id_into_tools (env now
  resolved from request-local ContextVar; idempotent wrapping), make
  execute_dashboard_result.result_key optional, resilient checkpoint resume with
  ToolMessage repair + direct-tool fallback, remove dead fast-path, consolidate
  tool_call parsing in _tool_resolver, context-safe ContextVar resets
- backend: llm-config gated by strict service-only auth (no user-JWT fallback),
  tighten idempotent run reuse (dashboard/env/intent match + 6h staleness),
  terminal event transitions run.status to COMPLETED/FAILED/CANCELLED,
  null-safe metric parsing in dashboard query model
- run.sh/docker-compose: require SERVICE_JWT (random per-run secret) instead of
  public default
 2026-08-06 18:26:50 +07:00
busya
c30cca78f3 smoke alembic 2026-07-23 18:53:45 +03:00
busya
d8bbe4baa8 text 2026-07-23 16:51:36 +03:00
busya
3f6d7222c3 fix(agent): proxy gradio config and reduce translate blocking 2026-07-07 17:37:36 +03:00
busya
6f02990347 fix(ssl+agent): capath for all HTTP clients + isolate gradio import 
SSL fix (ADR-0009 Finding 7):
- Replace ssl.create_default_context() with system_ssl_context(capath)
  in client_registry.py, async_network.py, notifications/providers.py,
  git/_base.py. Previous fix (0.5.1) only covered LLM clients; the
  Superset API client still used ssl.create_default_context() which
  loads cafile (flat bundle) where OpenSSL 3.x ignores intermediate CA
  certificates. system_ssl_context() uses capath only (hash symlinks).

Agent fix:
- Extract _check_llm_provider_health + _llm_status from agent/app.py
  into agent/_llm_health.py. The /api/agent/llm-status endpoint was
  importing from agent/app.py which triggers 'import gradio' at module
  level. Backend container does not have gradio installed, causing
  ModuleNotFoundError (500 error) every 30s on health check polling.

Config:
- Add ALLOWED_ORIGINS to docker-compose.yml + docker-compose.enterprise-clean.yml

ADR-0009 updated: Layer 2 table expanded with 4 missing clients,
Finding 5 reconciled (LLM_CA_CERT_URLS restored in 0.5.1), version 0.5.2.

Verified: 1110 unit tests passed, gradio import isolation confirmed.
 2026-07-07 12:18:43 +03:00
busya
1182b29fe0 fix(certs): restore HTTP CA auto-download and test edge matrix 
Restore LLM_CA_CERT_URLS support for corporate PKI HTTP cert delivery.
Downloaded certificates are installed into the llm/ CA category and share
the same system trust + NSS import path as CERTS_PATH-mounted certs.

Changes:
- certs.sh: add download_llm_ca_certs with HTTP-only curl policy,
  PEM/DER handling, retries, llm/ storage, hash symlinks for llm+custom
- backend/agent entrypoints: run download before install_all_certs
- compose/env/docs: expose LLM_CA_CERT_URLS again and update ADR-0009
- integration tests: cover PEM/DER/CER, invalid certs, non-HTTP skips,
  query-string filenames, private/server skips, empty inputs, combined
  LLM_CA_CERT_URLS + CERTS_PATH channels, and NSS imports

Verified:
- 194 passed, 1 skipped integration tests
- full backend container smoke passed with testcontainers PostgreSQL
- docker bundle rebuilt for 0.5.0
 2026-07-07 10:23:49 +03:00
busya
7f93261060 refactor(ssl): centralize SSL trust management, remove LLM_SSL_VERIFY 
Centralized SSL via one contract: CERTS_PATH=/opt/certs mounted into all containers.

Backend:
  - NEW: backend/src/core/ssl.py — system_ssl_context(), httpx_verify(),
    cert_dir_inventory()
  - LLMClient._get_ssl_verify() → delegates to core.ssl
  - _llm_async_http._get_verify() → delegates to core.ssl
  - Removed LLM_SSL_VERIFY env reading from all runtime code

Docker:
  - NEW: docker/certs.sh — shared cert installer (PEM/DER/cer to .crt conversion,
    update-ca-certificates, hash symlinks, NSS import)
  - NEW: docker/agent.entrypoint.sh — agent entrypoint with cert installation
  - backend.entrypoint.sh → uses certs.sh instead of install_llm_ca_certs
  - Dockerfile.agent → adds ca-certificates, openssl, entrypoint

Compose:
  - Removed LLM_CA_CERT_URLS and LLM_SSL_VERIFY from all compose files
  - Added CERTS_PATH volume mount to agent (dev + enterprise)
  - Added certs volume mount to backend/agent in dev compose

Env examples:
  - Removed LLM_SSL_VERIFY, LLM_CA_CERT_URLS from .env.example,
    .env.enterprise-clean.example, .env.current.example, .env.master.example,
    backend/.env.example
  - Enhanced CERTS_PATH comments with accepted formats

Diagnostics:
  - diag_container.py: removed LLM_* checks, added CERTS_PATH inventory,
    uses core.ssl for context creation

Tests:
  - Updated test_llm_analysis_service, test_llm_async_http,
    test_client_headers to verify centralized ssl context (no env disable)
  - 4/4 SSL tests pass
 2026-07-06 21:00:28 +03:00
busya
2716f15804 refactor(env): unify Docker env vars — canonical AUTH_SECRET_KEY, remove JWT_SECRET fallback 
Canonical variables:
  - AUTH_SECRET_KEY — JWT signing key for backend + agent (was split across
    AUTH_SECRET_KEY / JWT_SECRET)
  - SERVICE_JWT — agent→backend service token
  - No JWT_SECRET fallback: decoder fails with migration guidance if only
    JWT_SECRET is set

Compose files unified:
  - docker-compose.yml, docker-compose.enterprise-clean.yml,
    docker-compose.e2e.yml — all use AUTH_SECRET_KEY
  - build.sh generated compose now passes AUTH_SECRET_KEY + ENCRYPTION_KEY
    to backend

Env examples unified and completed:
  - .env.example — comprehensive template, all compose vars
  - .env.enterprise-clean.example — production template
  - backend/.env.example — backend-only run
  - docker/.env.agent.example — agent-only run
  - NEW: .env.current.example, .env.master.example,
    .env.e2e.example, frontend/.env.example

Tests aligned:
  - conftest sets AUTH_SECRET_KEY (canonical value matched across test files)
  - test mocks use canonical name
  - 1176 passed, 0 failed
 2026-07-06 14:24:17 +03:00
busya
fcfea2ddc8 refactor(agent): use local JWT decoder instead of src.core.auth.jwt 
Replace src.core.auth.jwt import with lightweight _jwt_decoder.py that
uses jose.jwt directly with JWT_SECRET env var. Avoids pulling AuthConfig
→ AUTH_DATABASE_URL/SECRET_KEY validators → SQLAlchemy models into agent.

Removed: AUTH_SECRET_KEY, AUTH_DATABASE_URL from agent compose env.
Removed: src/core/auth/ copies from Dockerfile.agent COPY section.
Added:   src/agent/_jwt_decoder.py — stateless JWT validation.
 2026-07-06 13:37:08 +03:00
busya
bdb20e7132 fix(agent): pass AUTH_DATABASE_URL to agent container, auth config requires it 2026-07-06 12:19:14 +03:00
busya
752dc1cbaa fix(agent): resolve missing imports — jose, auth config, AUTH_SECRET_KEY 
- Added python-jose[cryptography] to requirements-agent.txt
- Made sqlalchemy.orm.Session and TokenBlacklist imports lazy in jwt.py
  so decode_token works without pulling ORM models into agent image
- Added src/core/auth/__init__.py, config.py, jwt.py to agent Dockerfile COPY
- Added AUTH_SECRET_KEY env var to agent compose (reads from JWT_SECRET)
 2026-07-06 12:13:26 +03:00
busya
75c19eed0b feat(ssl): add passphrase-protected key and PKCS#12 support in nginx entrypoint 
- docker/frontend.entrypoint.sh: add 3 new functions:
  - extract_p12_if_needed — openssl pkcs12 extraction to .crt+.key
  - resolve_ssl_passphrase — read SSL_KEY_PASSPHRASE env var
  - decrypt_key_if_needed — openssl rsa decryption before nginx start
- Pipeline: install CA -> extract p12 -> get passphrase -> select config -> decrypt key -> start nginx
- Crash-early: encrypted key without SSL_KEY_PASSPHRASE exits entrypoint
- docker-compose.enterprise-clean.yml: add SSL_KEY_PASSPHRASE to frontend env
- .env.enterprise-clean.example: document SSL_KEY_PASSPHRASE usage
- build.sh: add SSL_KEY_PASSPHRASE to generated deploy compose
 2026-07-04 17:05:51 +03:00
busya
ce20f541b6 fix(security): resolve Critical+High findings from module audit — agent, translate, superset_client 
P0 — CRITICAL (CWE-798): JWT_SECRET crash-early
  Replace hardcoded super-secret-key fallback with os.environ["JWT_SECRET"]
  and ${JWT_SECRET:?} syntax in app.py + docker-compose files

P1 — HIGH: Frontend dependency CVEs
  Upgrade svelte 5.43.8 → 5.56.4 — resolves devalue DoS (GHSA-g2pg-6438-jwpf)
  and svelte XSS (GHSA-crpf-4hrx-3jrp, GHSA-m56q-vw4c-c2cp, GHSA-rcqx-6q8c-2c42)

P2 — MEDIUM: Logging hygiene + contract gaps + tool resolver refactor
  Apply _redact_sensitive_fields() in middleware + event streaming
  Truncate LLM error body to 100 chars
  Add @RATIONALE/@REJECTED to HandleResume + SaveConversation
  Refactor deterministic intent matching → LLM-driven tool resolution

P3 — LOW: Translate logging hardening
  Move _sanitize_url() to _utils.py (shared, no circular imports)
  Sanitize base_url before logging in _llm_call.py and _llm_async_http.py
  Emit EXPLORE warning when LLM_SSL_VERIFY=false disables TLS

superset_client module: passed clean — no changes needed
 2026-07-01 13:17:29 +03:00
busya
a7a72cf885 fix(agent): resolve ModuleNotFoundError for backend, add E2E test infra 
- Dockerfile.agent: fix CMD (python -m src.agent.run), use backend/requirements.txt,
  minimal COPY (only src.agent + src.core.cot_logger), add GRACE contract
- docker-compose.yml: SERVICE_TOKEN_SECRET -> SERVICE_JWT (match code)
- docker-compose.enterprise-clean.yml: same env var fix
- docker/.env.agent.example: same env var fix
- build.sh: same env var fix
- chore: semantics-testing SKILL.md, backend tests, pyproject.toml
 2026-06-14 15:41:46 +03:00
busya
62a52f4600 feat(bundle): add agent image to build.sh bundle + enterprise-clean compose 
- build.sh bundle_release() now builds ss-tools-agent:{tag} from
  docker/Dockerfile.agent alongside backend and frontend
- Generated docker-compose.enterprise-clean.yml includes agent service
  (image: + pull_policy: never, port 7860, depends_on: backend)
- Manifest .txt and .json include agent image fields
- sha256sums and load instructions updated for three images
- Static docker-compose.enterprise-clean.yml adds agent service (build
  from source) for local 'up enterprise-clean' profile
 2026-06-11 16:13:02 +03:00
busya
b6d18654d0 alembic fix 2026-06-01 16:34:07 +03:00
busya
3cb4f8b60b fix: pass LLM_SSL_VERIFY env var through docker-compose to container 2026-05-27 15:26:38 +03:00
busya
bb3b12a5bb feat: add LLM SSL verify control and auto CA cert download 
- LLM_SSL_VERIFY env var to disable SSL verification for corporate proxies
- install_llm_ca_certs() entrypoint function — downloads PEM/DER certs
  from LLM_CA_CERT_URLS, converts DER→PEM, installs to system CA store
- _format_connection_error() — detailed exception chain logging
- 7 new unit tests for _get_ssl_verify, _format_connection_error, verify= param
- LLM_CA_CERT_URLS and LLM_SSL_VERIFY in .env.enterprise-clean
- Removed duplicate @RELATION DEPENDS_ON -> [EXT:Library:tenacity]
 2026-05-27 14:44:46 +03:00
busya
dfc74b5c65 fix(maintenance): auto-expiry + adaptive markdown height + tests 
- Auto-expiry: expired events auto-end on GET /events via task manager
- Height: _estimate_markdown_height adapted to unit=8px scale with padding detection
- CRITICAL-1: removed dead import remove_chart_from_position (QA finding)
- CRITICAL-2: added chart alive check in ensure_banner_chart (QA finding)
- CRITICAL-3: fixed test assertions update_markdown_chart → update_banner_on_dashboard
- @POST contract: fixed return range [2,12] → [19,200]
- Tests: 8 new tests (auto-expiry + layout height)
 2026-05-25 08:36:33 +03:00
busya
d48e6c0d83 semantics 2026-05-20 09:59:03 +03:00
busya
13fcd544d2 semantic cleanup 2026-05-08 10:07:05 +03:00
busya
0bc6f59a7a Bootstrap initial admin via env and add compose profiles 2026-03-17 19:16:25 +03:00
busya
9bd2a7001b Add docker admin bootstrap for clean release 2026-03-13 11:41:44 +03:00
busya
683bd8c370 feat: add offline docker bundle for enterprise clean releases 2026-03-11 12:35:01 +03:00