ss-tools/specs/050-mcp-interface/traceability.md

Traceability — 050-mcp-interface

Production acceptance traceability — 2026-09-29

Historical rows above identify prior tests/code only; removed agent UI paths are retired. The following audited gates remain partial/open independent of local suite totals.

Requirement	Domain contract / DTO	Task	Falsifiable acceptance	State
MCPX-FR-030	Contract-complete public parity; data model	T044	REST/MCP lifecycle/read/auth errors and disabled automation validation are identical; service principal cannot decide human gate.	CLOSED 2026-09-22: shared classify_start_error (REST+MCP) + tests/test_mcp_rest_error_parity.py 17-vector matrix incl. service-principal human-gate denial.
MCPX-FR-030	Contract-complete public parity; data model	T045	consume/publish failures return typed errors/pending state with no legacy fallback; every prerequisite is externally MCP-reachable.	CLOSED (gated MCP publish_baseline_catalog + 037 publication worker, REST parity; live gate→CAS→commit→receipt chain 2026-09-11)
MCPX-FR-030	Contract-complete public parity; data model	T046	Fresh external-client chain preserves authoritative context and complete baseline pin without raw ORM/REST repair; no frontend agent controls/routes/requests.	CLOSED (MCP --baseline chain carries the plan pin without raw ORM; REST canary proves the record stamping)
MCPX-FR-030; external-MCP-only UI	manual editor/review; read-only evidence	production tasks	No frontend agent prompt/chat/assistant editing/proposal generation/workspace/start/handoff routes or requests; human approval remains usable.	OPEN
MCPX-FR-031	Investigation tools	T047	queue/case read, human-only note/proposal, bounded refs, catalog/RBAC parity.	CLOSED 2026-09-18 — 23414f9a/5eea72bc, 73 focused green.
MCPX-FR-032	Authoring input/evaluation ops	T048	set_step_inputs/set_step_evaluation pass external proposal/diff validation.	CLOSED — 534d488f/f123210b/87a90f6f.
MCPX-FR-033	Exploration step	T050	sandbox read-only interactive step with CAS/receipts and no second Playwright stack.	OPEN.
MCPX-FR-034	Prompt dangerous-content profile	T051	REST/MCP validation/error parity plus injection negative tests.	PARTIAL 2026-09-22: parity matrix CLOSED (test_mcp_rest_error_parity.py); LLM evidence-injection negative gate (LLM-INJ-001) still OPEN.
MCPX-FR-035	Reference URL invariant, tool schema	T052	Agent can request same-dashboard URL; MCP and REST resolve identical filter state and coordinate; neither accepts caller values as authority; published pin survives scheduled replay without resolving key again.	PARTIAL 2026-09-25; 2026-09-30 attempted external HTTP/SSE human OAuth flow in an isolated Compose project. Backend and fixture seed were healthy, but environment registration stopped before MCP calls because ss-dev returned HTTP 502 from Caddy (DNS resolved). The project and volume were removed; no candidate was written. The local human OAuth→MCP HTTP/SSE regression stalled before result and was interrupted after ~20s. An earlier unauthenticated ss-prod URL redirected to login; later user-supplied credentials enabled read-only dashboard 11 URL/filter resolution, with no MCP capture. In-process MCP/RBAC tests and REST browser evidence remain. Isolated Git receipt, run pin and scheduled replay wiring are proven; live MCP invocation, an agent asking for the URL and real dashboard release remain open. Details: live MCP attempt.
MCPX-FR-036 / T029a	038 ServerOwnedPipeline; typed MCP profile inputs	T029a	Fresh dashboard inspection → complete selected-case profile/metric coordinates → digest-CAS typed selector/coordinate choice → eligible registered handle → atomic bootstrap; forged/stale values create zero side effects.	PARTIAL 2026-09-29: accepted commits 2f094915/ce3dc56b/1b6edaf7/1a3d3958/3406433b add durable owner-scoped CAS/idempotency state, server-owned eligibility receipt binding, receipt-bound bootstrap admission, and a linear idempotent Alembic chain. Latest executed behavioral evidence remains profile-focused 9 green and selected MCP/profile 9 passed, 2 failed (test_external_client_creates_and_activates_scenario_revision_end_to_end, test_propose_test_pack_profile_returns_unresolved_preview_only). Unverified working tree: fixes sequential CAS and receipt forwarding, scopes metric blockers, adds owner-session registration without caller graph, advertises catalog 2.7.0, and aligns unbound runtime parameters with 038/044 as warnings rather than authoring blockers. Ruff/compileall/diff-check/AXIOM verification pass. On 2026-09-30 the two named regressions passed, and the combined targeted MCP/registry selection reached 217 passed with one stalled HTTP initialize test deselected. A synthetic isolated in-process MCP fixture proves selector-only profile CAS, eligible graph-free registration, current bootstrap and stale-fingerprint atomic rejection; inspection is mocked. Typed step-level authoring context, exact published-entry/filter binding, runnable baseline_ref semantics and fresh external wire-client DB E2E remain open.

MCPX-FR-036 current status (2026-10-01): the row above is a dated pre-implementation checkpoint. A fresh external MCP client now completed profile/CAS → exact published M01 binding → register/bootstrap/current revision in isolated Docker, followed by manual PASS → mutated FAIL → restored PASS and independent scheduled PASS. The owned table_text_v1 evaluator repeated the same four verdicts with immutable evaluation and owned evidence. See Stage 6 status. This closes only the supported M01 slice; full-catalog, external dashboard 11 and production GO remain OPEN. MCPX-FR-035 still requires its separate external URL-capture parity gate. Live parser follow-up (2026-09-30): ss-prod dashboard 11 emitted filter_timegrain with an active time_grain_sqla state. The 037 parser now canonicalizes this Superset alias to TIME_GRAIN; live resolve_reference_filter_context passed on permalink Ez3n5bWxa9d (dashboard 11, 10 charts, 1 dataset, scoped active filter). This resolves that filter spelling at context resolution only; the MCPX-FR-035 gate remains PARTIAL pending the remaining live capture/publication/scheduled vectors.

T029a selector-only checkpoint (2026-09-30): a fail-closed 037 published-entry selector was implemented and semantically reviewed before profile/MCP wiring. It loads through the durable receipt-bound Git path, recomputes per-entry digests and requires expected release ID/version/commit. The server-owned effective filter hash was absent from the durable profile at that selector-only checkpoint; metric_name → result_key existed in 037/038 without a durable profile binding. The later CAS selection closes this selection-only seam, while needs_baseline remains preview_only. No runnable baseline comparison is claimed for this slice. See baseline binding slice. The shared 037 pin resolver also now returns typed BASELINE_EVIDENCE_UNAVAILABLE for malformed filter/provenance payloads before field access; semantic and static checks pass, with no runtime proof claimed. The 2026-09-30 graph audit additionally found no typed coordinate identity on execute_metric: a unique comparison/direct-producer pair cannot prove the selected coordinate. Keep selection snapshots inert and baseline.default unchanged until a server-issued coordinate→producer binding and validator are present in canonical graph and receipt contracts.

Later 2026-09-30 checkpoint: the locator-only MCP needs_baseline resolution now persists a server-derived exact-entry/release/filter selection under CAS, with the raw URL transient. This supersedes the earlier selector-only statement above; it does not clear needs_baseline or make a graph runnable. The 038 v1 metric producer has no typed coordinate, so T029a still needs the v2 producer coordinate, graph/receipt binding, admission checks and runtime comparison. The requested /superset/dashboard/sales/ release is also pending: the slug is dashboard 11; the local ledger has no deployment/release and Git release configuration has no PREPROD stage. Superset credentials enabled read-only lookup, while ss-tools release endpoints require a human application session. MCPX-FR-035 and MCPX-FR-036 remain PARTIAL. See the live release checkpoint.

Sources: production gap, coverage gap, baseline gap. Spec schema/static checks prove contract structure only; live canary/runtime closure and optional approved performance baseline are not claimed.

T029m live external replay — evidence 2026-09-11

Gate row E2E-EXT-002 CLOSED. Full external MCP chain on the live stand, zero non-MCP seeding:

• Client: specs/044-dashboard-scenario-execution/prototype/live_mcp_replay.py (committed; transport-only adaptation of tests/test_mcp_client_flow_http.py OAuth/PKCE + tests/test_mcp_initial_scenario_e2e.py tool chain).
• Run: backend/.venv/bin/python specs/044-dashboard-scenario-execution/prototype/live_mcp_replay.py against http://127.0.0.1:8000 → run 110a6517-e433-4860-9d77-231521acba19, scenario e2687f03-f6d3-45f7-b114-d27b023ae085, revision fd126da0-eb22-4da6-be73-387589762689, agent_run a6168c7d-aa93-439e-8c2e-161f585b211f.
• Outcome: register_draft_pack context_authority=verified; PROD start → one durable gate → approved; live capture_screenshot passed (8 durable refs); apply_native_filter typed BROWSER_ACTION_NOT_SUPPORTED; terminal inconclusive (honest fail-closed).
• Full trace + defects fixed: docs/2026-09-11-sales-prod-mcp-replay.md. The baseline-pinned variant is now proven: live_mcp_replay.py --baseline resolves a full runner_plan.baseline_pin from the published catalog through the MCP start (no raw ORM); T045 remains OPEN only for the MCP publish tool.

T045/T046 published-catalog baseline pin — evidence 2026-09-11

T046 baseline-pin CLOSED; T045 OPEN (MCP publish tool missing):

• Publisher: backend/src/scripts/publish_catalog.py (create=POST / update=PUT-with-sha; validates the resolver-canonical envelope; typed auth/conflict/unavailable; offline tests/scripts/test_publish_catalog.py).
• Published: catalogs/ss-prod-visual/v1.json @ busya/ss-tools — commits e41d6ad2 (create), 3bb2c63e (update-sha proof), 4d5b7e4c/3f782574 (resolver-canonical envelope).
• Publish-failure canary: bogus token → PUBLISH_AUTH_FAILED (HTTP 401), exit 1, no partial state.
• REST canary v4 (ace916a0…, re-run adeabe63…): selector → pin from published bytes → strict runner_plan.baseline_pin == AgentEvaluation.baseline_pin (walker stamping).
• MCP --baseline chain (live_mcp_replay.py --baseline, no raw ORM): compile baseline capability → start with published selector → full runner_plan.baseline_pin (set/version/digest).
• Trace: docs/reports/agentic-runtime-live-canary-v4-baseline-pin-2026-09-11.md. T045 residual (curated MCP publish tool) resolved the same day — see the next section.

T045 gated MCP publish — evidence 2026-09-11

Gate row T045 CLOSED. The publish operation is externally MCP-reachable with the full 037 publication-worker contract:

• Worker: ScenarioExecution.PublicationWorker + migration 0023_publication_operations (durable state machine: publish_pending → committed → published / publish_failed with retained operation+commit).
• MCP: gated tool publish_baseline_catalog (human-only, scenario RUN_PROD, requires_approval) → approval_required{gate_id} → decide_approval → scheduler dispatch through the reviewed adapter McpOpsDispatch.BaselinePublishAdapter.
• REST parity: POST/GET /api/catalog-publications (RUN_PROD) on the same worker/idempotency/CAS contract.
• LIVE: gate 4522ae96… → approved → operation b8797bbb… published with receipt (head 3f782574… → abce63db…); moved-HEAD canary 6fe450f1… → typed PUBLISH_HEAD_MOVED, durable publish_failed.
• Trace: docs/reports/agentic-runtime-live-mcp-publish-t045-2026-09-11.md.