ss-tools/specs/PRODUCTION-ACCEPTANCE-MATRIX-043-050.md

#region ProductionAcceptance.Matrix043050 [C:5] [TYPE ADR] [SEMANTICS production,acceptance,gates,scenario,baseline,execution,monitor,automation,analytics,mcp] @BRIEF Unified conjunctive production acceptance matrix for the current 037/038/044/045/046/047/050 scenario-testing chain after the 2026-09-17/18 amendments and live canaries. @RELATION DEPENDS_ON -> [SupersetBaselineEngine.Spec] @RELATION DEPENDS_ON -> [BaselineEngine.ReferenceUrl] @RELATION DEPENDS_ON -> [DashboardScenarioModel.Spec] @RELATION DEPENDS_ON -> [ScenarioExecution.Spec] @RELATION DEPENDS_ON -> [ScenarioRunMonitor.Spec] @RELATION DEPENDS_ON -> [ScenarioAutomation.Spec] @RELATION DEPENDS_ON -> [ScenarioAnalytics.Spec] @RELATION DEPENDS_ON -> [McpInterface.Spec] @RATIONALE Prior acceptance rows were fragmented across seven feature packages and WORKSTATE; a single conjunctive table is required so a local unit PASS cannot be mistaken for production GO while a live, safety or operations gate remains OPEN. @REJECTED Taking the newest historical checklist status per package was rejected — many older [x] rows predate drift amendments, zero-human policy, evidence tiering and current live findings. @INVARIANT Production GO requires every P0 row CLOSED and every P1 row either CLOSED or explicitly accepted by a signed release decision; CLOSED requires retained executable evidence, never prose or schema shape alone.

1. Status vocabulary

State	Meaning
CLOSED	Falsifiable verifier executed against the current contract; retained command/output or live evidence is linked.
PARTIAL	Implementation/evidence exists but at least one named acceptance vector remains open.
OPEN	Missing implementation or missing executable proof.
BLOCKED	Cannot execute until an external prerequisite/decision exists.
DEFERRED	Explicit post-GA item; it cannot hide a P0 requirement.

GO rule: GO ⇔ all P0=CLOSED ∧ all accepted P1 have signed disposition ∧ full suites/migrations/semantic audit green on the release commit ∧ no unresolved critical security or reconciliation finding.

2. Unified matrix

Gate	Priority	Owner spec/task	Falsifiable acceptance	Current evidence	State
G-BSL-CATALOG	P0	037 T082–T084	Catalog CAS, one head under concurrent approval, server-owned capture bytes, rebaseline/supersede/retire/publish failure reconcile without duplicate revision.	2026-09-21 core CLOSED: catalog_revision_log.py append-only JSONL CAS log — duplicate baseline/coordinate rejection, stale If-Match typed conflict, thread-race single-head proof, retire/invalidate append-only (history byte-identical), publish_failed→reconcile on the same revision; 8/8 test_catalog_revision_log.py. Wiring CLOSED 2026-09-22: catalog_revision_wiring.py builds the server-owned projection and approvals.consume_approval appends through materialization._commit_with_catalog_compensation under the same catalog lock (failed DB commit restores catalog AND log bytes — no phantom revision); publication_worker mirrors the settle onto the digest-matched head (PUBLISH_REVISION_LOG_MISMATCH on a wrong bound log, PUBLISH_REVISION_LOG_ABSENT traced when the deployment binds none); 10 test_catalog_revision_wiring.py + 3 test_publication_worker.py.	CLOSED
G-BSL-REFERENCE-URL	P0	037 T091; 050 T052	New metric/visual/scenario-derived baseline starts from a same-dashboard URL; active native filters are parsed and frozen; candidate→publication→run pin retains reference identity; scheduled replay never re-fetches short-link state; different dashboard and unverified legacy entries fail closed.	2026-09-24 uncommitted: URL preview parses permalink and native filter key, runs bounded chart-data queries and maps metric coordinates to charts; both supplied ss-dev dashboard 10 URLs returned 10/10 metric coordinates, one native filter and no coordinate errors against the live stand. A different-dashboard permalink is rejected. REST/MCP preview and capture surfaces exist; URL source is stored with metric candidate/revision/pin and unverified legacy entries fail closed. 2026-09-25: selected metric batch defers candidate commits to the route transaction; second-coordinate failure rolls back earlier drafts (2 focused atomicity tests). Generic screenshot metadata is caller-writable, so visual approval stays fail-closed until server-owned URL-bound visual capture exists (2 trust-boundary tests). The exact native_filters_key=A4_rwhfQta0 link was rechecked live via configured ss-dev client: dashboard 10, platform=3DS, 10/10 metrics executed and 6 distinct formulas suggested (including SUM(Global_Sales) on Games and count on Publishers With Most Titles). A transient systemd-resolved feature-state failure was repaired with resolvectl reset-server-features; the backend repeated 10/10 without DNS override. The ordinary local database has no dashboard 10 scenario/release. A separate isolated Compose fixture with a test-only synthetic release proved browser URL entry → active filter/chart-attributed preview → six draft captures; Chromium 2/2 also proved a different-dashboard field error. Isolated DB inspection verified one AgentRun, six candidates, six capture executions, exact URL SHA-256 and normalized platform IN ["3DS"]. The project was removed after the run; screenshots/traces are under /tmp/reference-url-e2e-*.png and /tmp/reference-url-playwright-results/. A subsequent isolated Chromium 2/2 pass joined URL capture to human source-check/reasoned approval: one chart 96/dataset 20 metric entered the local catalog, and its revision retained verified URL bytes/hash plus platform=3DS. The revision publication state is materialized with no Git receipt. The run exposed and fixed synthetic fixture SemVer and catalog/revision schema oneOf versus both authoritative IDs. Still open: real release and Git publication receipt, visual/scenario-transform URL capture, run pin/scheduled replay, participant UX acceptance and release-commit verification.	PARTIAL
G-BSL-DERIVED	P0	037 T085–T090; 038 T068/T070	Bounded transform output creates scenario_transform candidate from server artifact; curated ≤50 gating coordinates; operator approval; observatory entry cannot reach PASS; period stale→blocked→candidate.	2026-09-21 all offline axes CLOSED: T086 scenario_transform_provenance.py (8/8); T087 baseline_selection.py facts-only ≤50 proposal + CAS review (8/8); T088 observatory tier excluded from pin + smuggle-guard (11/11); T068 transform_dsl.py bounded DSL derivation over declared refs — byte-stable canonical decimals, depth/row bounds, no SQL/Python/network (12/12). 2026-09-22 wave 4 — period lifecycle offline CLOSED: baseline_staleness.py detects a closed period that is not the launch-declared reporting_period, resolve_baseline_pin blocks typed (BaselinePeriodStale=BASELINE_STALE) pre-I/O, facts-only proposal with automatic_rebaseline=false (8/8 test_baseline_stale_lifecycle.py). 2026-09-23 launch-receipt CLOSED (wave 5 Phase B): BaselineEngine.BaselineStaleness.LaunchReceipt emits the durable baseline_stale receipt on a separate committed session after the caller rollback — wired once per transport (REST scenario_runs.py 422 + MCP tools_scenario.py blocked, 050 T044 parity kept; answers byte-identical to the pre-receipt contract); idempotency key collapses REST+MCP double calls to one row; emission failure never masks the block (6 vectors test_start_run_period_receipt.py, parity matrix 17 green). 2026-09-23 offline transform capture wired: bounded_transform stores canonical JSON output bytes in DraftStorage and returns ref/digest/MIME/length; walker registers the active ScenarioArtifact bound to run, logical step and attempt, and T086 provenance accepts that artifact while rejecting forged hash/value/ref. Focused suite 50/50 plus independent executor→bytes→register→candidate bridge 1/1 passed. Remaining: live run/candidate canary; ./run.sh --skip-install is blocked during database preparation because host TCP connections to the healthy Docker PostgreSQL on port 5432 close unexpectedly, so /api/ready never starts.	PARTIAL (offline transform artifact wiring closed; live run open)
G-SCENARIO-AUTHOR	P0	038 T063–T070	Typed action/evaluation ops; no embedded metric truth; no new HumanStep; disabled actions reject before I/O; TransformSpec/DecisionPolicy schemas deterministic.	534d488f,f123210b,d29959f6,a59f5e3c,87a90f6f; unit suites green. Row corrected 2026-09-22: the earlier "Transform/best-of-N open" wording was stale — T068 transform_dsl.py (12/12) and T069 evaluation_aggregation.py (best-of-N strict majority, no quiet PASS) closed them per WORKSTATE 2026-09-21. Remaining: live scheduled zero-human run (tracked under G-LLM-SECURITY/G-E2E-EVALUATION).	PARTIAL (offline schema/op axes closed)
G-BROWSER-CORE	P0	044 T047/T048	Live Superset browser actions: open/wait/refresh + assert_dom/inspect_filter_options/navigate_tabs/wait_for_selector; owned durable evidence; no fabricated PASS.	readonly-canary-20260918T125144Z* 3/3; 125451Z* 4/4; unit 811.	CLOSED
G-BROWSER-FILTERS	P0	044 T048 / CHK038	Live search_text, set, clear, date-picker; inspect state before/after; chart-data response; clear checkpoint replay.	2026-09-25 live URL-bound canary: filters-canary-20260925T191720Z.json against ss-dev dashboard 11 with native_filters_key=CIcbBivwWiM: inspect → Country=Canada → 5 charts settled → inspect selected state → clear → 6 charts settled → inspect cleared state; checkpoint replay retained; shutdown leftover_sessions=0, task_destroyed_warnings=[]. Browser transport fixed global Sec-Fetch-* headers and Ant Design label→control resolution. Date vector remains BLOCKED: no date filter on this stand; search is structurally N/A for the selected control.	PARTIAL (date prerequisite external)
G-BROWSER-CROSS-DASH	P1	044 T055	navigate_dashboard driver, filter reset, new checkpoint segment, old checkpoint rejected, cross-dashboard fixture.	Registry disabled: pending driver.	OPEN
G-E2E-EVALUATION	P0	038 T069; 044 T043/T046/T051	Browser→artifact→baseline comparison→bound immutable AgentEvaluation→pinned DecisionPolicy→terminal PASS/FAIL/inconclusive; scheduled zero-human real-LLM run; no evidence-free PASS.	2026-09-21 live chain proven: canary v5 live-canary-v5-binding-20260921T164034Z.json — browser open → 8 durable artifacts → real-LLM evaluation persisted (ce4685e7…) → walker binds it into the compare decision (evaluation_input + agent_evaluation_ids) → policy resolves honestly. 2026-09-25 multimodal capability re-proven: configured Omniroute deepseek-flash accepted a live canary PNG in an OpenAI image_url message and returned IMAGE_CANARY_OK with HTTP 200 (gateway route reported gpt-5.6-sol); the former blanket “text-only fallback / vision route unavailable” note is stale. Remaining: full v5 rerun with this provider, terminal policy PASS proof, and scheduled zero-human real-LLM/injection proof.	PARTIAL (chain proven; full terminal/scheduled proof open)
G-LLM-SECURITY	P0	038 AGSCN-FR-022; 044 SCEX-FR-035	Evidence-as-data; zero tools; strict output; prompt/evidence injection cannot force PASS; malformed/low-confidence/disagreement→non-pass.	Code-token op guard 87a90f6f. Offline CLOSED 2026-09-22 (wave 4): prompt carries evidence_handling.evidence_is_data=true (injected text stays in the data section, never in instructions); strict verdict-enum + confidence-range coercion; findings for an undeclared criterion dropped → inconclusive + EVALUATION_FINDING_UNSUPPORTED; provider text cannot establish baseline_pin/spec/template identity; compromised-double canary cannot resolve to a policy passed; submit seam exposes no tool channel; 7 test_llm_injection_offline.py. Residual (live): LLM-INJ-001 live scheduled zero-human real-LLM proof.	PARTIAL (offline closed; live proof open)
G-PROVIDER-SHUTDOWN	P0	044 T052 / SCEX-FR-037	After canary and soak: zero pending Playwright Connection.run, page/context/browser tasks; shutdown bounded.	2026-09-18: close_all_sessions+stop() bounded drain; unit leak-negative 3/3; live canaries readonly-canary-20260918T{174659,184343}Z: 0 destroy warnings, thread dead, 0 leftover sessions.	CLOSED
G-MUTATION-RECON	P0	044 T053 / SCEX-FR-038	Provider post_rows equals independent pinned SQL readback; mismatch→reconciliation_required/blocked; restore independently verified.	2026-09-18: same-session fresh-client_id readback in transport PASS path + receipt proof; mismatch→reconciliation_required (unit); live mutation canary readonly-canary-20260918T184343Z: readback_ok=true ×2, failures=[]; 82.75/82.74 root-caused as lifecycle skew (mid-step post-SELECT vs post-restore), both values correct.	CLOSED
G-EVIDENCE-TIER	P1	044 T054; 046 T025	hot_runs=2, holds, TranscodeReceipt source→target digest, verify-before-delete, lossless/lossy marker, lossy pixel refusal, idempotent sweep.	Contract only; backend absent (TranscodeReceipt/hot_runs not in backend/src). UI tier badge (tier_hot/tier_cold/untiered) is contract-declared but not implemented — it would bind an API field the backend does not emit.	OPEN (blocked on backend)
G-ARTIFACT-CONTENT	P0	044 T044	Auth/ACL/status/headers/digest/MIME/length/expired/corrupt/traversal/range.	2026-09-17 live artifact canary 11/11.	CLOSED
G-CAPACITY-FAULTS	P0	044 T045	Startup/readiness/deadline/cancel/drain/quarantine/reconcile/late-response fault matrix.	2026-09-17 live fault canary 16/16.	CLOSED
G-MONITOR	P1	045 RUNMON-FR-015..017	Launch gates usable; legacy checkpoint read; new runs no waiting-human; tiered evidence markers; reload/reconnect.	UI unit/build proven (frontend 3425 vitest + build green on 2026-09-23). 2026-09-23 reload/reconnect offline CLOSED: L1 vectors — a fresh model recovers waiting_human from the server with exactly one checkpoint step (a reload never appends a second), and rebinding closes the previous EventSource while named listeners never stack. Tiering absent (blocked on G-EVIDENCE-TIER).	PARTIAL (reload/reconnect closed; tiering absent)
G-SCHEDULER-CORE	P0	046 T019/T020	REST/MCP automation parity, ownership, disabled/legacy HumanStep rejection, idempotent crash/replay/concurrent due events.	T019/T020 closed with existing tests.	CLOSED
G-NOTIFICATIONS	P0	046 T022; 047 T026	Failed/inconclusive run, queue created, case opened/disposition, stale baseline each produce durable notification receipt within declared SLA/channel.	2026-09-22: notify.py fail-closed emit vs durable domains (durable rejects human), channel/sla payload, idempotent receipts; 047 wiring for queue_created/case_opened/case_dispositioned; terminal receipts + dedicated baseline_stale. 8 focused tests (6 gate vectors), tests/services/dashboard_testing 1313 green.	CLOSED
G-DLQ-SOAK	P0	046 T023/T024/T026	N identical infra failures→quarantine; recovery; queue-depth/capacity SLO; 72h soak + 3 restarts + 5/15/50-tab canaries, zero duplicates.	2026-09-22 offline CLOSED: poisoned_store.py durable JSONL identical-failure counter (flock, append+fsync, CAS epoch; no migration) + poisoned.py N=3 quarantine/no-retry, one blocked DLQ notification, operator CAS unquarantine; 13 policy + 2 API tests. 2026-09-23 operator recovery UI CLOSED: AutomationQuarantineModel + QuarantinePanel on the automation route — quarantine list (error/identical count/CAS version), confirm-release echoing the displayed version, 409 STALE_QUARANTINE_VERSION → reload+notice, 404 NOT_QUARANTINED → reload+notice; 11 frontend tests + updated page mock. 2026-09-23 retry-dispatcher wiring CLOSED (wave 5 Phase A): automation/retry_policy.py closed allowlist of retryable infra codes (fail-closed, prefix matching rejected) + MAX_ATTEMPTS=3 bound to the quarantine threshold; execution/retry_dispatcher.py failed→queued CAS re-queue (durable identical-failure counter is the attempt authority — ScenarioRun has no attempt column, D11 migrations frozen) skipping quarantined pairs and the ceiling; terminal accounting in terminal_effects.py — one terminal failure = one counter record (automation sources + allowlisted codes only; passed automation runs reset the streak, operator CAS still exclusively clears quarantine); quarantine gate in dispatch_runs.py skips stale queued rows of quarantined pairs until operator release; scheduler tick composes retry→dispatch; 10 test vectors (CAS exactly-once, N=3 quarantine, distinct-code reset, content/manual exclusion, gate+release, ceiling, notification epoch). Residual (live): 72h soak with 3 restart storms, 5/15/50-tab canaries.	PARTIAL (offline + recovery UI + retry wiring closed; soak live)
G-INVESTIGATION-MCP	P0	047 T022/T023	MCP queue/get/note/proposal bounded, RBAC/owner, service writes denied, proposal never closes case.	23414f9a,5eea72bc; 73 green.	CLOSED
G-INVESTIGATION-UI	P1	047 T024/T025/T027	queued+in_case findability, case index, linked runs/evidence/i18n, browser queue→case→disposition.	e7925d0e,c9e6e642,9acd9bad presentation slice: grouped queue, business summary, evidence cards, collapsed technical refs; 13 focused tests. 2026-09-24 uncommitted follow-up: deterministic isolated seed + Chromium queue→correct case→linked run/evidence→resolved→stale CAS, 1 passed/0 skipped; trace/screenshot retained in frontend/test-results/; UI POST fix with 10 focused tests, lint/build green.	CLOSED for scoped UI/E2E implementation; release-commit regression remains G-FULL-REGRESSION
G-INVESTIGATION-ATOMIC	P0	047 T019–T021/T028	PostgreSQL concurrency: one active episode/case; stale CAS rollback; exact-context comparability; immutable evidence.	2026-09-22: analytics/locking.py (pg_advisory_xact_lock per fingerprint) + set_disposition savepoint (zero partial projection); analytics/context.py exact-context comparability with typed mismatch/ineligible; 22 unit + 7 Testcontainers PG16 race vectors (tests/integration/test_scenario_investigation_concurrency.py).	CLOSED
G-MCP-PARITY	P0	050 T044/T047/T048/T051	REST/MCP error/auth/lifecycle parity, catalog/visibility mirrors, service cannot decide human gates; all prerequisites externally reachable.	2026-09-22: shared classify_start_error consumed by REST and MCP removes the transport-dependent error vocabulary; tests/test_mcp_rest_error_parity.py 17-vector matrix (env/baseline codes/idempotency/conflict/disabled action/disabled automation/investigation RBAC/unauthenticated + service-principal human-gate denial + MCP no-bypass on case ACL). Live external chain proven earlier.	CLOSED
G-MCP-TEST-PACK-PROFILE	P1	050 T029a; 038 ServerOwnedPipeline	External MCP profile preview derives server-owned case coverage/metric coordinates/filter applicability; digest-CAS selector and coordinate choices re-inspect context; unresolved context/baseline never mint save handles or become runnable.	2026-09-29: runtime catalog 2.6.0 registers both tools as authenticated-human read previews; 34 focused tests and Ruff/compileall pass. Coordinate choice advances to needs_baseline without expected values. Domain context/published-baseline resolution, eligible-handle minting and fresh external profile→bootstrap E2E remain open.	PARTIAL / NO-GO
G-MCP-BASELINE-SELECTION	P1	050 T049	Agent proposes bounded curated baseline scope; operator review handle; server capture; observatory non-gating.	Server-side facts-only selection exists (services/dashboard_testing/baseline_selection.py + T087 tests) but no MCP tool; the operator review UI is contract-declared and not implemented (the service has no HTTP/MCP surface to bind).	OPEN (blocked on the MCP/HTTP surface)
G-MCP-EXPLORATION	P1	050 T050	Read-only exploration_step in isolated workspace with CAS/receipts; reuses 044 session; no raw code or second Playwright stack.	Fixed one-shot exploration exists; interactive operation absent.	OPEN
G-STORAGE-BUDGET	P1	044/046 tiering	Measured hot/cold artifact volume and quota alarms meet the approved retention budget for 20-table dashboards.	Estimates only; no telemetry-based acceptance.	OPEN
G-SECURITY-OPS	P0	044/046/050	Local provider perimeter, artifact ACL, secret-redaction, key expiry/rotation, PII retention policy, no credentials in evidence.	Artifact ACL/provenance proven. 2026-09-23 offline axes CLOSED (wave 5 Phase C): C.1 key rotation contract proven executable (test_rotation_reencrypt_roundtrip: old-key ciphertext unreadable under new key, readable after re-encrypt, legacy values reported not converted) + Core.EncryptionKey.KeyAge fail-closed age verdict from the .env mtime (unknown age → warning age_unknown, exceeded interval → age_exceeded, EXPLORE-traced; no key material in verdicts); C.2 evidence redaction at the single write boundary — ScenarioExecution.EvaluationAdapter.Raw redacts every string leaf via the plugin RedactionService (per-leaf, not serialized-string regex which corrupts JSON; findings/structure preserved — test_persist_raw_response_redacts_credentials_and_pii); C.3 scheduled retention now classifies registered raw VLM responses at 7d, screenshots/artifacts at 30d, discovers expired ScenarioArtifact rows, commits durable deletion receipts before the existing hold-aware delete/verify-absent sweep, and holds open investigation cases. scenario_evidence_retention_days controls only explicitly classified scenario_evidence rows (default 90d; no current producer emits this class). scenario_evidence_archive_target is declared but unused; no live archive is claimed. G-STORAGE-BUDGET remains a separate P1 tiering/budget gate. Residual: live perimeter exercise (rotation drill on a stand, PII sweep against live evidence) and live retention exercise.	PARTIAL (offline retention wired; live drill open)
G-BA-UX-READINESS	P0	039 T085–T088; 045 T023–T026; 047 T029–T033; 046 T027/T028	Five business analysts complete seven scripted workflows: scenario setup/coverage, baseline review, launch, result/evidence interpretation, investigation/disposition, schedule operations, history/filter behavior. Every task scores ≥4/5; ≥80% task steps need no moderator; no false approval/PASS/case closure; keyboard-only plus named screen-reader controls; evidence tied to release commit.	OPEN, interim expert review 2026-09-24: pre-redesign scores: prepare test 2/5, baseline 1/5, launch 2/5, result/evidence 2/5, investigation/disposition 1/5, schedules 3/5, history/filter 2/5; mean 1.9/5. Uncommitted queue/case slice has deterministic isolated Chromium proof. The uncommitted baseline-review slice has owner-scoped candidate/approved overview, verified source, reasoned gate and publication, current/stale comparison and recovery; seeded isolated Chromium 1/1 passed. Scoped provisional expert baseline score 4/5, not a participant score; period-change/curated-selection UX remains. No five-analyst scores, keyboard/screen-reader acceptance, no-help evidence or tested release commit exist.	OPEN
G-DR-ROLLBACK	P1	cross-cutting	Restore DB + baseline Git + evidence store; migration rollback/roll-forward runbook; recovery RTO/RPO exercise.	No unified exercise.	OPEN
G-FULL-REGRESSION	P0	release	Full backend pytest, frontend vitest/lint/build, ruff, compileall, PostgreSQL Alembic heads/upgrade/check, isolated browser E2E on release commit.	Scoped runs green; no current full-suite release gate after 2026-09-18 changes.	OPEN
G-SEMANTIC-HEALTH	P0	release	Axiom verify/rebuild: 0 parse warnings; no new naked critical blocks/unresolved critical relations; docs/nav rebuilt.	Pending after current spec/code amendments.	OPEN
G-RELEASE-SIGNOFF	P0	release owner	All P0 CLOSED, P1 dispositions signed, unified matrix frozen with evidence refs and commit; NO-GO→GO recorded.	Product remains NO-GO.	OPEN

3. Release sequence

1. Close P0 runtime safety first: G-PROVIDER-SHUTDOWN, G-MUTATION-RECON, G-BROWSER-FILTERS.
2. Close correctness truth chain: TransformSpec + curated/observatory baseline + E2E evaluation (G-BSL-DERIVED, G-E2E-EVALUATION, G-LLM-SECURITY).
3. Close operations: notifications, DLQ/soak, atomic investigation, MCP parity.
4. Close G-BA-UX-READINESS: every critical analyst task ≥4/5 with retained browser and participant evidence.
5. Run full regression, migration and semantic health on one release commit.
6. Freeze this matrix; obtain release-owner signoff.

4. Matrix maintenance rule

Any future spec amendment that adds a production invariant MUST add/update one row here in the same commit. A row state may move to CLOSED only when its owner package traceability links retained executable evidence; WORKSTATE narrative alone is insufficient.

#endregion ProductionAcceptance.Matrix043050