Commit Graph

650 Commits

Author SHA1 Message Date
354c6684d8 feat(dashboard-testing): pin published baselines in schedules 2026-09-25 11:44:44 +03:00
e7bac04e98 feat(dashboard-testing): add reference URL baseline workflow and analyst UX
Discover dashboard filters, datasets and metrics from a parsed reference URL; capture and review baseline candidates with source provenance. Improve scenario DAG and run result views, add isolated browser coverage, and align contracts and ADRs.
2026-09-25 10:36:37 +03:00
6d58bb97d0 fix(dashboard-testing): authenticate browser SSE run stream via query token 2026-09-24 09:39:49 +03:00
233edfd135 fix(dashboard-testing): require run ACL principal and share VIEW predicate 2026-09-24 09:34:51 +03:00
d8b0bd58ad fix(dashboard-testing): enforce run/case object ACLs and recurrence guards 2026-09-24 09:32:58 +03:00
a122c328a1 feat(backend): harden automation retention and execution 2026-09-24 00:32:05 +03:00
fc3697f35f feat(dashboard-testing): complete analyst workflows 2026-09-24 00:31:22 +03:00
300c2a2954 feat(maintenance): add dashboard-targeted banners and template library 2026-09-22 21:18:15 +03:00
f0d7b79523 fix(rbac): single admin authority, gate privileged surfaces, enforce session on dual-auth
- is_admin flag is the sole admin authority; remove role-name fallbacks (dependencies, reports, tasks, agent lifecycle, security badge, frontend)
- normalize (resource, ACTION) across REST/MCP/catalog; strict admin role/permission parsing (400 on unknown)
- gate LLM provider CRUD/status/test on admin:settings; require auth on agent llm-status; quarantine listing requires scenario:automation MANAGE
- enforce logical-session revocation/idle in require_api_key_or_jwt
- last-admin lockout guards for user/role CRUD; block non-admin from granting is_admin
- frontend: flag-only admin, route permission map from the nav registry, write-control gating, 403 dedupe
- tests: orthogonal edge coverage (flag-only admin, lockout boundaries, session states, normalization)
2026-09-22 21:14:35 +03:00
5a5ac4dd7c feat(dashboard-testing): wave 4 offline P0 — revision-log wiring, period rollover gate, injection-safe prompt
G-BSL-CATALOG: consume/materialization appends the server-owned CatalogRevision under the catalog lock and rolls it back with the YAML on any failure (no phantom revision); the publish settle mirrors the outcome onto the digest-matched head without demoting a landed commit.

G-BSL-DERIVED: a closed period that is not the launch-declared reporting_period blocks the start typed (BaselinePeriodStale = BASELINE_STALE) with a facts-only rebaseline proposal, no automatic rebaseline, and REPORTING_PERIOD_INVALID for a malformed declaration.

G-LLM-SECURITY: the evaluation prompt declares evidence-as-data, coerces the verdict to the closed enum, drops findings for undeclared criteria and refuses provider-owned identity; 7 injection vectors run through the submit seam with no live LLM.

INV_7: publication/baseline/start/evaluation modules split under 400 lines and CC<=10; moved contracts tombstoned with @REPLACED_BY. Matrix, 037/044 traceability and WORKSTATE accounting.
2026-09-22 21:13:52 +03:00
65684d6c65 test(mcp): REST/MCP error-shape parity matrix with a shared start-error classifier (050 T051)
- start_run.py: classify_start_error — one canonical mapper for ENVIRONMENT_NOT_CONFIGURED,
  every BASELINE_* code, IDEMPOTENCY_KEY_REUSED and existing UPPER_SNAKE typed codes; unknown
  ValueErrors collapse to RUN_START_CONFLICT. REST and MCP now share it, so the same failure
  can no longer present a transport-dependent error vocabulary (the divergence the matrix caught:
  MCP returned raw exception text while REST collapsed to RUN_START_CONFLICT).
- Parity matrix (12+ vectors): env/baseline/conflict/disabled-action/disabled-automation/
  investigation-RBAC/unauthenticated, plus service-principal human-gate denial.
- Base-drift adaptations during integration: the investigation-case ACL vector now asserts the
  real no-bypass invariant (foreign case collapses to not_found via MCP, no content leak) since
  the investigation MCP tools exist on this release; terminal-transition atomicity expectations
  now include the queue_created receipt and pin replay idempotency (2 receipts, no growth).
- 17 parity tests; union wave-3 verification: 1845 passed.
2026-09-22 12:45:28 +03:00
223296bc57 feat(analytics): atomic investigation triage and exact-context comparability (INV-ATOMIC-001)
- locking.py: transaction-scoped pg_advisory_xact_lock per fingerprint (no-op on SQLite);
  serializes queue/case/episode writers without new migrations (D11).
- investigation.py: queue_signal/open_case lock then SELECT FOR UPDATE + populate_existing;
  set_disposition validates case CAS then performs case status/version, queue closure,
  AgentAction and episode resolution inside one savepoint (zero partial projection), and
  emits the idempotent case_dispositioned receipt after the savepoint commits.
- recurring.py: record_occurrence locked + FOR UPDATE (one open episode).
- context.py: AnalyticsContextKey and assess_comparability — same environment with a differing
  baseline family/release/RLS/principal is typed ineligible/mismatch; eligible-sequence
  classification separates regression/flaky from model variance.
- comparison.py: compatibility carries comparable = same_scenario AND exact-context + context ids.
- Tests: 22 unit (13 new) + 7 PostgreSQL Testcontainers concurrency vectors (--run-integration).
- Merged with the notification wiring on analytics/investigation.py (both changes kept).
2026-09-22 12:38:00 +03:00
def2bf4038 feat(automation): poisoned-run quarantine policy with durable failure signatures (SCHED-SOAK offline)
- poisoned_store.py: durable JSONL identical-failure counter (fcntl.flock + intra-process lock
  + thread-local reentrancy, append+fsync, torn-line tolerant, CAS quarantine epoch); no
  Alembic/ORM changes (046 migrations frozen).
- poisoned.py: N=3 identical infra failures -> typed POISONED_RUN_QUARANTINE, matching
  schedules disabled, exactly one blocked notification carrying the DLQ payload; success
  resets the pair; operator unquarantine (CAS) re-enables schedules and resets counters.
- REST: GET /scenario-automation/quarantine and POST /quarantine/{scenario_id}/release
  (scenario:automation MANAGE; 404 NOT_QUARANTINED / 409 STALE_QUARANTINE_VERSION).
- Tests: 13 policy + 2 API quarantine vectors; merged with the master ReadAcl/RetentionReceipts
  classes (conflict resolved keeping both). Focused 31 passed; 046 T023 offline slice.
2026-09-22 12:35:39 +03:00
93e97228e9 feat(automation): notification wiring for terminal, queue and case lifecycle events (NOTIFY-001)
- notify.py: fail-closed emit vs durable domains (durable rejects 'human', zero-human policy),
  channel/sla_seconds on the receipt payload, optional idempotency_key with receipt dedupe.
- analytics/investigation.py: idempotent queue_created / case_opened / case_dispositioned
  receipts keyed on the immutable fingerprint and case id + prior decision_version.
- execution/terminal_effects.py: idempotent terminal receipts; BASELINE_STALE runs emit a
  dedicated baseline_stale receipt instead of a generic blocked one.
- test_notifications_wiring.py: 6 gate vectors + fail-closed + emit/persist domain split (8 tests).
- 047 T026 / 046 T022 wiring; focused 8 passed, tests/services/dashboard_testing 1313 passed, ruff clean.
2026-09-22 12:33:27 +03:00
a4db469077 fix(translate): typed temporal transfer, language arbitration, MERGE key
- sql_generator: normalize ISO/epoch values only against a resolved target type
  (Nullable/LowCardinality unwrapped); untyped epoch-looking strings are preserved and
  unparseable temporal values raise instead of coercing. Removes the heuristic that
  wrote sentinel dates (2106-02-07 = UInt32 max) into business date columns.
- sql_generator: ClickHouse string-literal escaping — backslash, quote and control
  characters escaped; ANSI '' doubling alone silently rewrote paths and tabs on insert.
- service_target_schema: one fetch_target_column_types resolver for both direct_db and
  SQL Lab insert paths (300s cache) plus a single column-catalog SQL builder; EXPLORE
  trace when the catalog is unreachable/empty instead of a silent untyped fallback.
- _lang_detect: script-conflict downgrade no longer depends on the target language list;
  letterless text becomes und; short/number-heavy Cyrillic cannot survive as a Latin
  language; sanitize_detected_language validates the LLM arbitration answer by script.
- _batch_proc/_utils: und rows actually reach LLM arbitration (same-language, cache and
  preview short-circuits blocked) and trust a cache entry only when its stored source
  language is confirmed; detector version bumped to invalidate the v2 cache.
- _key_mapping: single source->target key mapping; MERGE with target_language_column
  absent from target_key_cols rejected in preflight before any fetch/LLM; lang_code is a
  target-only discriminator (no fictitious source key). Removes 5 divergent copies.
- frontend: global run indicator elapsed timer (1s tick, server started_at, cleanup)
  reusing formatDuration; MERGE-key inline validation with en/ru i18n keys; UX contract
  and decision-memory updates.
- tests: real ClickHouse testcontainers coverage on dm.debt_comment_translations
  (Date32/DateTime transfer, epoch/millis/sentinel/boundaries, NULL semantics, text
  escaping round-trip, MERGE language discriminator, chunked batch, production
  projection); unit coverage for typed normalization, escaping, key mapping and the
  arbitration cache. Existing integration module had a stale untyped-epoch expectation.
2026-09-22 11:36:44 +03:00
fc0c426537 feat(baseline): correctness truth chain — catalog CAS log, transform provenance, selection proposal, walker evaluation binding (AGBASE-FR-014..018, T082-T088, T051/T068/T069)
- catalog_revision_log.py: append-only JSONL revision log with If-Match CAS, duplicate
  baseline/coordinate rejection, thread-race single-head proof, retire/invalidate
  history preservation, publish_failed reconcile on the same revision (T082-T084 core).
- scenario_transform_provenance.py: server-owned 044 ScenarioArtifact chain for
  scenario_transform candidates — run/owner/activeness/digest/passed-step/value equality,
  forged refs typed-rejected (T086, AGBASE-FR-017).
- baseline_selection.py + schemas: facts-only BaselineSelectionProposal (cross_check ->
  checklist -> big-number/closed-period -> filter_sentinel -> lineage_repr), hard 50-entry
  budget with recorded exclusions, default+business filter contexts, CAS review with
  accept/drop/adjust (T087, AGBASE-FR-016); observatory_entries tier in the catalog
  revision schema + resolver smuggle-guard (T088, AGBASE-FR-015).
- transform_dsl.py: bounded structured op-tree DSL (sum_column/difference/ratio/scale,
  depth<=4, 10k rows, canonical decimals, byte-stable) wired as derive_value into
  bounded_transform (T068, AGSCN-FR-018).
- evaluation_aggregation.py: verdict policy single_shot/best_of_n with strict-majority
  quorum, tie/all-error/quorum-miss/below-threshold typed non-pass (T069, AGSCN-FR-022).
- evaluation_binding.py + walker: dependency-ordered binding of the persisted covering
  AgentEvaluation into the comparison decision (T051/T046); deferral until the covering
  evaluation persists with a deadlock guard (_step_feeds_evaluation); semantic_comparison
  scope in the policy mapper — required-mode evidence producers pass rows 1-8, the
  semantic overlay binds to comparison steps only (production-chain ordering);
  capacity_block.py extracted (INV_7 walker 398 LOC).
2026-09-22 09:39:43 +03:00
c01fcffdbe feat(execution): P0 runtime safety wave — provider shutdown drain, mutation readback, filter canary (SCEX-FR-037/038, BSC-FILT-001)
- ProviderEventLoop.stop(): close_all_sessions fan-out on the live loop before detach +
  bounded pending-task drain before loop.close() (zero 'Task was destroyed' live-proven);
  close_all_sessions weak-registry shutdown fan-out in browser_session_managers.
- SCEX-FR-038 independent mutation readback: browser_readback.py (same-session fresh
  client_id SELECT, cleanup-aware expectation, canonical 6dp float normalization);
  transport wiring adds readback_verified checkpoint; BrowserTransportReadbackMismatch ->
  reconciliation_required/effect unknown/retry blocked; INV_7 kept (browser.py 395 LOC
  via factory-helpers extraction).
- Live canaries: actions/mutation (readonly-canary-20260918T174659/174740/184343Z) and
  native filters (filters-canary-20260918T195946Z: values/clear/replay/chart PASS; date
  vector absent on stand — external prerequisite).
- Robustness fixes from live flake: pre-apply Escape (stale overlay in reused sessions)
  and bounded antd dropdown settle in browser_native_filter.
- Root-cause of the 82.75/82.74 divergence: mid-step post-SELECT vs post-restore
  lifecycle skew, not a cache defect.
2026-09-22 09:39:17 +03:00
c9e6e642e3 fix(analytics): implement case index API for investigation findability (SCAN-FR-020) 2026-09-18 14:54:09 +03:00
67c7b1d89e feat(baseline): scenario_transform candidate kind with run/expression provenance (AGBASE-FR-017) 2026-09-18 13:36:59 +03:00
87a90f6f64 feat(scenario): set_step_evaluation op — E-class spec pinning with code-token guard (AGSCN-FR-021/034) 2026-09-18 13:24:46 +03:00
a59f5e3cbb feat(scenario): zero-human capability mapping — human_checkpoint emission replaced with unsupported (AGSCN-FR-019/020) 2026-09-18 12:58:52 +03:00
4274e40395 feat(scenario): wave-2 observe drivers — assert_dom, inspect_filter_options, navigate_tabs, wait_for_selector (AGSCN-FR-024) 2026-09-18 12:47:01 +03:00
5d86da54a4 Merge feat/038-registry-filters: registry 038.5.0 disabled-action discipline, native filter search/clear/date modes, click URL outcome 2026-09-18 12:04:50 +03:00
683347bee8 feat(scenario): native filter search/clear/date modes and click URL outcome (AGSCN-FR-024) 2026-09-18 12:04:37 +03:00
d29959f6c5 feat(scenario): registry 038.5.0 with disabled-action discipline (AGSCN-FR-025) 2026-09-18 11:39:37 +03:00
c022448671 Merge feat/047-investigation-mcp: investigation MCP tools, queue/case projections, catalog 2.4.0 (SCAN-FR-016..019, MCPX-FR-031) 2026-09-18 10:50:39 +03:00
f123210bb2 fix(scenario): keep ScenarioStep.inputs as canonical ref list; pin action args in action_inputs 2026-09-18 09:51:04 +03:00
5eea72bc46 fix(mcp): align investigation catalog contract 2026-09-18 09:17:06 +03:00
23414f9af6 feat(mcp): add investigation case tools 2026-09-17 22:11:28 +03:00
534d488f73 feat(scenario): add typed step inputs 2026-09-17 21:19:15 +03:00
ec917ac677 feat(scenario): 044 provider INV_7 decomposition + T044/T045 live canaries; evidence-backed spec status
The 044 provider lifecycle is decomposed to satisfy INV_7 and closed with live canaries;
the remaining acceptance row (T046 graph-level terminal PASS) is diagnosed with executable proof.

- split browser_readonly_actions (534->160), browser_session (501->58) and browser.py (407->398)
  into sibling modules with frozen contract IDs and frozen import surface (facades re-export every
  moved public name); monkeypatch seams follow the owning modules
- gates per phase: scoped 044 suite 1590 passed and full backend 11269 passed / 0 failed, no delta
- T042b: six PREPROD browser canaries GREEN under the Wave-C run-scoped session architecture;
  harness gains SS_CANARY_STORAGE_ROOT and fixes the dead-loop scheduler drain hang
- T044: new artifact-content live HTTP canary 11/11 (real app, PostgreSQL, JWTs, live-captured bytes)
- T045: new fault-injection canary 16/16 (bounded loop refusals, deadline cancellation, shutdown
  drain, capacity quarantine + TTL reconcile, receipt CAS, late response, cancel drain)
- T046: walker-level evaluation->comparison binding diagnosed (compare-only + mandatory ->
  EVALUATION_UNAVAILABLE; bound evaluation -> BASELINE_AND_SEMANTIC_PASS); T043/T046 stay open
- refresh 044 spec/traceability/checklist/quickstart/SESSION_STATE/contracts/plan/data-model and the
  037/050 rows to evidence-backed states; split three malformed multi-target @RELATION lines
- fix skill drift in semantics-python/semantics-svelte (logger facade, notify(), i18n proxy)
2026-09-17 18:05:45 +03:00
4746af2f3e fix(health): address consolidation review findings 2026-09-16 16:11:02 +03:00
3de0756d4f refactor(validation): remove legacy LLM dashboard validation mechanism
Scenario-based dashboard testing (042/044) and scenario automation (046)
replace the legacy policy/run based validation surface.

- drop backend validation routes, service, schemas and DashboardValidationPlugin
- add migration 0025_drop_legacy_validation removing legacy tables
- redirect legacy settings/automation page to scenario automation surface
- remove frontend validation models, routes, components and i18n bundles
- repair dangling semantic relations and dead plugin-id fixtures
2026-09-16 11:13:50 +03:00
8522a2ee3c feat(scenario): complete dashboard testing UX flow 2026-09-15 10:12:17 +03:00
b4148cfe5f fix(automation): scheduled runs execute as schedule-owner principal; MCP upsert signature parity — live scheduled happy path proven (046 T018 CLOSED) 2026-09-11 20:22:47 +03:00
d0466a2c1e fix(automation): deterministic scheduled-run idempotency key + typed scheduled-reject observability (046 happy path) 2026-09-11 20:01:53 +03:00
578a6110e7 fix(scenario): T045 review hardening — RUN_PROD REST gate, fail-marked typed errors, marker-bound adoption, unique idempotency, path guard 2026-09-11 19:05:20 +03:00
72018383c4 test(mcp): catalog list includes publish_baseline_catalog (050 T045) 2026-09-11 18:14:30 +03:00
6dde7e15c3 feat(scenario): 050 T045 — gated MCP publish_baseline_catalog + 037 publication worker (CAS, receipts, reconcile), REST parity 2026-09-11 18:13:09 +03:00
6da072b9d4 test(database): cover DB_SCHEMA isolation and prepare_database reset
- env_settings schema resolution/search_path args incl. invalid-name rejection
- dedicated schema pins engine search_path
- e2e: no-public-privilege role, missing-schema bootstrap/denied, object-kind
  reset (sequences/views/domains)
- enterprise template asserts DB_SCHEMA
2026-09-11 17:27:45 +03:00
7cb02f4fe0 test(scenario): DEF-01 promotion graph validation regressions
Structure-first validation: prose double-hyphen passes, SQL in title rejected,
needs_baseline is not a promotion gate, legacy snapshot stays free-text scan.
2026-09-11 17:27:45 +03:00
7291169343 feat(scenario): durable 037 baseline catalog publication
- PublicationOperation model + 0023 migration with idempotent inspector guard
- publication_worker: reviewed-gate binding, branch-head CAS, commit/receipt
  reconcile on retry (never duplicate/force-push)
- REST /api/catalog-publications (RUN_PROD) sharing the worker with the gated
  publish_baseline_catalog MCP tool (050 T045 / MCPX-FR-030 parity)
- MCP op input schema, catalog registration and approved-dispatch adapter
- tests: worker, REST surface, MCP gate/dispatch parity
2026-09-11 17:27:43 +03:00
71135c0822 fix(maintenance): single-banner multi-event render and per-env end-all
- multi-event builds exactly one banner (dedup messages <br>, min-start/max-end
  envelope, open-ended keeps 'уточняется') instead of nested invalid HTML
- event.environment_id is authoritative for states/banner rows (settings target
  only legacy fallback)
- end_all resolves a SupersetClient per event environment via memoized factory
- structured reason/explore logging for fan-out target resolution
- suppress toast for unconfigured /maintenance/settings 404
2026-09-11 17:27:39 +03:00
5758ae4a83 fix(database): object-level reset and optional DB_SCHEMA isolation
Safe one-shot reset, fail-fast on unknown revisions, object-level drops for non-owner corporate PG, rollback before advisory unlock, configurable DB_SCHEMA via search_path, e2e migration matrix tests.
2026-09-11 16:17:11 +03:00
e334863771 build(backend): declare python-dotenv explicitly (review nit — direct import) 2026-09-11 15:10:24 +03:00
de5e098a1b fix(scenario): review hardening — run.sh key reuse, resolver-canonical publisher envelope, strict pin asserts, honest T045/T046 statuses 2026-09-11 15:09:16 +03:00
7d3fb8a770 feat(scenario): live baseline pin via published Gitea catalog — publisher POST/PUT contract, run.sh key-wipe fix, T045/T046 closed 2026-09-11 14:07:20 +03:00
1411c03f9b fix(scenario): orthogonal review hardening — binding fingerprint guard, strict image digest, typed publisher errors, T029m human-loop closure 2026-09-11 10:10:01 +03:00
d04927bda7 feat(scenario): T029m live MCP replay — binding identity fixes, Gitea catalog publisher, 050 statuses 2026-09-11 08:40:37 +03:00
792bb1251d feat(scenario): live eval trust — MIME sniffing, multimodal evidence, baseline pin stamping, binding admin surface 2026-09-11 08:40:19 +03:00