Commit Graph

38 Commits

Author SHA1 Message Date
5758ae4a83 fix(database): object-level reset and optional DB_SCHEMA isolation
Safe one-shot reset, fail-fast on unknown revisions, object-level drops for non-owner corporate PG, rollback before advisory unlock, configurable DB_SCHEMA via search_path, e2e migration matrix tests.
2026-09-11 16:17:11 +03:00
4d5ef6bed0 feat: dashboard-testing UX remediation + admin MCP governance + i18n sweep
- UX audit Fix 1-5: actionable MCP HandoffSurface (endpoint/discovery/onboarding), reachable
  approval loop (sidebar section + waiting-for-me badge), honest MCP entry labels, ROUTES SSOT
  for dashboard-testing/load-testing (+ link-integrity coverage)
- scenario registry hub (/dashboard-testing/scenarios) built on ScenarioRegistryModel
- admin MCP governance: read-only GET /api/admin/mcp/catalog (catalog x roles + DCR clients) +
  /admin/mcp page/model/api + ROUTES.admin.mcp + sidebar entry
- full i18n sweep of dashboard-testing routes and scenario-* component trees (ru/en)
- accumulated workspace: MCP OAuth/DCR, automation idempotency migration, docker/nginx, specs
2026-09-05 20:16:26 +03:00
731aaaa8df feat(mcp): 050 unified MCP interface — parity tools, durable gates, authoring E2E, assistant decommission
044 provider runtime completion (pre-staged workstream): capacity/operator
stores, provider ops/protocol/reconciler/dispatch revalidation, exploration
sandbox runtime, alembic 0014-0016, live canary evidence (browser provider
6/6 against the live stand).

050 Phases 0-2: RBAC FastMCP server with a 45-tool explicit catalog,
OAuth/DCR transport guards with bounded bodies, per-call provenance
(McpToolInvocationRecord), durable ActionApprovalGate + CAS decide +
leased/fenced poller, authoring workspace ops, bounded-response discipline,
hidden-vs-gated matrices.

Parity domains (T012-T014): gated git/deploy/migration/backup/llm tools with
reviewed dispatch adapters in explicit poll chains; Superset reads/writes with
a dedicated plugin:superset_sql risk class (terminal PROD denial via the
canonical execution-policy criterion, hardened danger-SQL guard covering
INTO/CALL/SET/REFRESH/file primitives/multi-statement); baseline 037 tools
over the shared REST-surface services.

Evidence (T016/T023/T028): REST-vs-MCP field parity on shared 037 fixtures;
vertical E2E from tools/list through registry revision activation with real
scenario:EDIT RBAC; sandbox-to-revision promotion E2E with unsafe-payload and
caller-digest rejection; dispatcher soak (three poll cycles, exactly-once).

Orthogonal QA+security audit hardening: enforced response_limit fail-closed
envelope, poisoned-exploration fail-closed (EXPLORATION_TARGET_UNRESOLVED),
sha256 exploration evidence digests, actor-UUID task ownership, is_active
guard on baseline consume, 038 resolver description=None selector fix.

Phase 3/4 decommission: HandoffSurface behind the MCP_DECOMMISSION flag,
then unconditional removal — agent/ service tree, chat components/models/
stores/types, gradio proxies (vite + nginx), agent service in run.sh,
docker-compose profiles, build.sh bundles; /agent renders the handoff only.
Docs: AGENTS.md/INSTALL.md two-service rewrite; 036-047 drift amendments
marked done; WORKSTATE checkpoints with all evidence.

Suites: backend 11199 passed / 240 skipped / 1 xpassed; frontend 3454 passed
(197 files), lint 0 errors, build OK; browser E2E login+handoff 6/6 twice on
the isolated compose stack (no 7860); ruff/compileall clean.

Misc: gitignore hardening (tmp/, tool model cache); E2E selector repairs
(nav strict-mode, invalid-credentials passthrough detail).
2026-09-03 07:37:14 +03:00
4b9619e229 refactor(deploy): standardize enterprise environment file 2026-08-25 15:30:10 +03:00
a2ede492b7 fix(database): reset any legacy Alembic revision 2026-08-25 15:21:42 +03:00
e80c954b37 fix(build): defer bundle secrets to deployment 2026-08-25 12:24:51 +03:00
5bc1b62bd0 refactor: unify initialization and reset migration baseline 2026-08-25 11:42:31 +03:00
2a0f334717 chore: remove hardcoded Fernet key and client cert
- ENCRYPTION_KEY: smoke test generates a fresh key inline; templates use a placeholder
- drop RUSAL_ROOT.cer (client-specific public cert) + gitignore it
2026-08-18 12:32:06 +03:00
1e0dacaf1b build: add dependency caching for bundle builds
Cache pip/npm dependency downloads via BuildKit cache mounts and skip
the postgres pull when the image is already present locally, so a
repeat ./build.sh bundle run does not download dependencies twice.
2026-08-03 17:37:57 +07:00
c30cca78f3 smoke alembic 2026-07-23 18:53:45 +03:00
d8bbe4baa8 text 2026-07-23 16:51:36 +03:00
root
632b730fff chore: migrate GRACE-Poly anchors to hierarchical dotted naming
Systematic rename of all semantic anchors (#region, [DEF], @RELATION)
across 1400+ files — backend Python, frontend Svelte/TS, specs, docs:
- Flat anchors become Namespace.Module.Entity
- @RELATION references updated to match new anchor paths
- Zero business logic changes
2026-07-22 11:48:15 +03:00
2d0fe96a63 chore: add GRACE semantic contracts across agent + backend + build scripts
- Add @RELATION/@RATIONALE/@REJECTED headers to agent modules
- Add GRACE contracts to backend services, models, schemas
- Update build.sh with single-image build commands (build:backend|frontend|agent)
- Update docker entrypoint: openssl rsa → openssl pkey (alg-agnostic)
- Add @RATIONALE to backend core modules (cot_logger, config_manager, ssl)
2026-07-08 11:10:59 +03:00
3f6d7222c3 fix(agent): proxy gradio config and reduce translate blocking 2026-07-07 17:37:36 +03:00
89cd123dec fix(frontend): dynamic year + git commit hash in APP_VERSION
- Footer.svelte: replace hardcoded '2025' with new Date().getFullYear()
- build.sh: append git short hash to APP_VERSION for all bundle commands
  (bundle, bundle:embeddings, bundle:light). Format: '0.5.2+6f029903'.
  Previously only the tag was passed (e.g. '0.5.2'), and .git was not
  available in Docker context so vite.config.js fell back to '0.0.0'.

Verified: frontend assets contain '0.5.2+6f029903', getFullYear() replaces
hardcoded 2025.
2026-07-07 12:29:46 +03:00
680da3baa3 xz -3 2026-07-06 18:26:31 +03:00
2716f15804 refactor(env): unify Docker env vars — canonical AUTH_SECRET_KEY, remove JWT_SECRET fallback
Canonical variables:
  - AUTH_SECRET_KEY — JWT signing key for backend + agent (was split across
    AUTH_SECRET_KEY / JWT_SECRET)
  - SERVICE_JWT — agent→backend service token
  - No JWT_SECRET fallback: decoder fails with migration guidance if only
    JWT_SECRET is set

Compose files unified:
  - docker-compose.yml, docker-compose.enterprise-clean.yml,
    docker-compose.e2e.yml — all use AUTH_SECRET_KEY
  - build.sh generated compose now passes AUTH_SECRET_KEY + ENCRYPTION_KEY
    to backend

Env examples unified and completed:
  - .env.example — comprehensive template, all compose vars
  - .env.enterprise-clean.example — production template
  - backend/.env.example — backend-only run
  - docker/.env.agent.example — agent-only run
  - NEW: .env.current.example, .env.master.example,
    .env.e2e.example, frontend/.env.example

Tests aligned:
  - conftest sets AUTH_SECRET_KEY (canonical value matched across test files)
  - test mocks use canonical name
  - 1176 passed, 0 failed
2026-07-06 14:24:17 +03:00
fcfea2ddc8 refactor(agent): use local JWT decoder instead of src.core.auth.jwt
Replace src.core.auth.jwt import with lightweight _jwt_decoder.py that
uses jose.jwt directly with JWT_SECRET env var. Avoids pulling AuthConfig
→ AUTH_DATABASE_URL/SECRET_KEY validators → SQLAlchemy models into agent.

Removed: AUTH_SECRET_KEY, AUTH_DATABASE_URL from agent compose env.
Removed: src/core/auth/ copies from Dockerfile.agent COPY section.
Added:   src/agent/_jwt_decoder.py — stateless JWT validation.
2026-07-06 13:37:08 +03:00
bdb20e7132 fix(agent): pass AUTH_DATABASE_URL to agent container, auth config requires it 2026-07-06 12:19:14 +03:00
752dc1cbaa fix(agent): resolve missing imports — jose, auth config, AUTH_SECRET_KEY
- Added python-jose[cryptography] to requirements-agent.txt
- Made sqlalchemy.orm.Session and TokenBlacklist imports lazy in jwt.py
  so decode_token works without pulling ORM models into agent image
- Added src/core/auth/__init__.py, config.py, jwt.py to agent Dockerfile COPY
- Added AUTH_SECRET_KEY env var to agent compose (reads from JWT_SECRET)
2026-07-06 12:13:26 +03:00
dcee86c260 refactor(docker): split requirements by role, slim enterprise bundle by default
Split requirements.txt into role-specific files:
  - requirements-backend.txt — FastAPI API runtime
  - requirements-agent.txt — Gradio/LangGraph agent (no embeddings)
  - requirements-embeddings.txt — opt-in sentence-transformers

Default enterprise bundle (./build.sh bundle) builds slim agent without
sentence-transformers/torch (~500 MB saved). Embeddings opt-in via
./build.sh bundle:embeddings <tag>. Embedding router degrades gracefully
when package is absent (ImportError catch in _embedding_router.py).

Updated Dockerfiles:
  - backend.Dockerfile → requirements-backend.txt
  - Dockerfile.agent → requirements-agent.txt + WITH_EMBEDDINGS build arg
  - all-in-one.Dockerfile → requirements-backend.txt

Added embeddings variant to build.sh with full manifest/digest output.
Hardened .dockerignore (cache, report, model, archive patterns).

Other branch work: align git route mocks with async services, remove
legacy agent test files superseded by contract tests.
2026-07-06 10:06:07 +03:00
75c19eed0b feat(ssl): add passphrase-protected key and PKCS#12 support in nginx entrypoint
- docker/frontend.entrypoint.sh: add 3 new functions:
  - extract_p12_if_needed — openssl pkcs12 extraction to .crt+.key
  - resolve_ssl_passphrase — read SSL_KEY_PASSPHRASE env var
  - decrypt_key_if_needed — openssl rsa decryption before nginx start
- Pipeline: install CA -> extract p12 -> get passphrase -> select config -> decrypt key -> start nginx
- Crash-early: encrypted key without SSL_KEY_PASSPHRASE exits entrypoint
- docker-compose.enterprise-clean.yml: add SSL_KEY_PASSPHRASE to frontend env
- .env.enterprise-clean.example: document SSL_KEY_PASSPHRASE usage
- build.sh: add SSL_KEY_PASSPHRASE to generated deploy compose
2026-07-04 17:05:51 +03:00
bcab488e83 fix(version): inject APP_VERSION via Docker build-arg instead of git describe
In Docker builds, .git directory is not available in the container,
so git describe --tags fails and fallback is '0.0.0'. Now:
- build.sh passes --build-arg APP_VERSION=${tag} to frontend build
- Dockerfile accepts ARG APP_VERSION and sets ENV
- vite.config.js checks process.env.APP_VERSION first, then git describe
2026-06-18 18:37:11 +03:00
2f1916706a rename ss-tools to superset-tools across the entire project
- Replace all occurrences of 'ss-tools' with 'superset-tools' in 104 files
- Rename git bundle file ss-tools.bundle → superset-tools.bundle
- Update .gitignore pattern accordingly
- Preserve variable names (hasSsTools etc.) and code identifiers
2026-06-16 11:15:19 +03:00
a7a72cf885 fix(agent): resolve ModuleNotFoundError for backend, add E2E test infra
- Dockerfile.agent: fix CMD (python -m src.agent.run), use backend/requirements.txt,
  minimal COPY (only src.agent + src.core.cot_logger), add GRACE contract
- docker-compose.yml: SERVICE_TOKEN_SECRET -> SERVICE_JWT (match code)
- docker-compose.enterprise-clean.yml: same env var fix
- docker/.env.agent.example: same env var fix
- build.sh: same env var fix
- chore: semantics-testing SKILL.md, backend tests, pyproject.toml
2026-06-14 15:41:46 +03:00
62a52f4600 feat(bundle): add agent image to build.sh bundle + enterprise-clean compose
- build.sh bundle_release() now builds ss-tools-agent:{tag} from
  docker/Dockerfile.agent alongside backend and frontend
- Generated docker-compose.enterprise-clean.yml includes agent service
  (image: + pull_policy: never, port 7860, depends_on: backend)
- Manifest .txt and .json include agent image fields
- sha256sums and load instructions updated for three images
- Static docker-compose.enterprise-clean.yml adds agent service (build
  from source) for local 'up enterprise-clean' profile
2026-06-11 16:13:02 +03:00
7600ce4e3f fix: test assertions for molecular CoT markers, separate propagate=False to ConfigManager, fix ruff in env.py 2026-05-27 11:26:14 +03:00
029f9031bc fix: logging overhaul — Alembic fileConfig disable_existing_loggers=False, propagate=False on loggers, ADR docs, catch-up migration for missing columns, sqlparse dep, build archive naming 2026-05-27 11:19:12 +03:00
d5cf5f0cf7 semantics 2026-05-26 09:30:41 +03:00
dfc74b5c65 fix(maintenance): auto-expiry + adaptive markdown height + tests
- Auto-expiry: expired events auto-end on GET /events via task manager
- Height: _estimate_markdown_height adapted to unit=8px scale with padding detection
- CRITICAL-1: removed dead import remove_chart_from_position (QA finding)
- CRITICAL-2: added chart alive check in ensure_banner_chart (QA finding)
- CRITICAL-3: fixed test assertions update_markdown_chart → update_banner_on_dashboard
- @POST contract: fixed return range [2,12] → [19,200]
- Tests: 8 new tests (auto-expiry + layout height)
2026-05-25 08:36:33 +03:00
d48e6c0d83 semantics 2026-05-20 09:59:03 +03:00
8212f8d8c6 fix(translate): fix batch sizing — use real available input budget, respect job.batch_size, lazy playwright in docker
- _batch_sizer.py: compute per_batch_budget from actual available_input_budget
  (context_window - max_output_tokens) instead of sum of first N rows
- _batch_sizer.py: cap max_rows_hard_cap with job.batch_size (user config)
- _token_budget.py: add available_input_budget and max_output_tokens to return
- preview.py: validate required config fields before preview
- requirements-docker.txt: add playwright pip package (~5 MB)
- backend.entrypoint.sh: lazy playwright install chromium on first start
- build.sh: switch tar to tar.xz (-T0 -9), 5.5x smaller bundles
- README.md: add offline bundle deployment instructions
- playwright.config.js: add testMatch for *.e2e.js files
- frontend: preview tab config validation, i18n keys, translationRun store
2026-05-17 23:32:00 +03:00
ecb3b11f22 fix(translate): Kilo API response_format, reasoning_effort skip, structured_outputs fallback, refusal handling
- Enable response_format (json_object) for all providers including Kilo/OpenRouter
- Skip reasoning_effort for Kilo/OpenRouter (returns 400 — mandatory reasoning)
- Add structured_outputs fallback: retry once without response_format if upstream
  provider (e.g. StepFun) rejects it with 'structured_outputs is not supported'
- Handle model refusal field (refusal) instead of treating as empty content
- Fix None-handling guards for message/finish_reason/content fields
- Apply same fixes to both preview.py and executor.py _call_openai_compatible
- Connect orphaned TermCorrectionPopup, BulkReplaceModal, BulkCorrectionSidebar
2026-05-15 18:12:29 +03:00
d17f75a65d feat(ui): add chat-driven dataset review flow
Move dataset review clarification into the assistant workspace and
rework the review page into a chat-centric layout with execution rails.

Add session-scoped assistant actions for mappings, semantic fields,
and SQL preview generation. Introduce optimistic locking for dataset
review mutations, propagate session versions through API responses,
and mask imported filter values before assistant exposure.

Refresh tests, i18n, and spec artifacts to match the new workflow.

BREAKING CHANGE: dataset review mutation endpoints now require the
X-Session-Version header, and clarification is no longer handled
through ClarificationDialog-based flows
2026-03-26 13:33:12 +03:00
978fe7b34d fix: commit semantic repair changes 2026-03-21 11:22:25 +03:00
0bc6f59a7a Bootstrap initial admin via env and add compose profiles 2026-03-17 19:16:25 +03:00
aa685e357a feat: Implement LLM provider deletion and refactor ConfigManager to preserve unknown payload sections. 2026-03-14 09:19:08 +03:00
c7b73acc6a semantic update 2026-02-23 13:15:48 +03:00