From 99bbdb43982516459ac4fa0ec4bb7c139935f935 Mon Sep 17 00:00:00 2001 From: busya Date: Mon, 6 Jul 2026 01:22:07 +0300 Subject: [PATCH] docs(agent-chat): update 035 verification traceability --- specs/035-agent-chat-context/plan.md | 22 +++++++++- specs/035-agent-chat-context/traceability.md | 42 ++++++++++---------- 2 files changed, 43 insertions(+), 21 deletions(-) diff --git a/specs/035-agent-chat-context/plan.md b/specs/035-agent-chat-context/plan.md index c8c246d5..8567309a 100644 --- a/specs/035-agent-chat-context/plan.md +++ b/specs/035-agent-chat-context/plan.md @@ -10,6 +10,20 @@ Refinement of the agent chat system across three vectors: 2. **Guardrails refinement** (P2): Risk classification extended from tool-name prefixes to include environment context and tool danger level. ConfirmationCard gains environment badges, destructive-operation countdown (10s), permission-denied state with alternatives, and visual risk toning. 3. **Tool optimisation** (P3): Backend: RBAC tool filtering, auto-retry on transient HTTP errors, semantic response summarisation, configurable tool timeout, context-aware tool affinity. Frontend: ToolCallCard extended with retry/timeout/cancelled states. +## Implementation Status — 2026-07-05 Fact Check + +Read-only audit against source/tests found production code and dedicated US1/US2 tests implemented. Live E2E/quickstart validation remains the primary open gate. + +| Area | Status | Evidence / Gap | +|------|:------:|----------------| +| Fixtures and foundational types | ✅ Implemented | API/model fixtures materialized; `UIContext`, `ConfirmMetadataV2`, `PermissionDeniedMetadata`, and extended `ToolCall` types are present. | +| Context-aware `/agent` flow | ✅ Implemented | `/agent` reads URL params, model builds UIContext/pill, backend validates/injects context, and tool filtering pipeline is present. | +| Guardrails production code | ✅ Implemented | Confirmation v2, permission_denied, invocation guard, countdown, deny/cancel flow, and distinct write_dev/write_staging/write_prod visual states are present. | +| Tool optimisation | ✅ Implemented | Retry, summarise, timeout helpers and ToolCallCard retry/timeout/cancelled states are present with dedicated tests. | +| Debug panel pipeline | ✅ Implemented | Backend `pipeline_result` SSE feeds `effectiveToolPipeline`, rendered in the debug panel. | +| Dedicated US1/US2 tests | ✅ Implemented | `test_agent_context.py`, `test_agent_tool_filter.py`, `test_agent_confirmation_v2.py`, `AgentChatModel.context.test.ts`, and `ConfirmationCard.ux.test.ts` are present. | +| Verification gates | ⚠️ Partially verified | Scoped backend/frontend tests and build pass; live E2E/quickstart remains pending. | + ## Technical Context **Language/Version**: Python 3.9+ (backend), TypeScript (frontend Svelte 5 runes-only) @@ -38,6 +52,12 @@ Refinement of the agent chat system across three vectors: | **VII. Test-Driven for C3+ Contracts** | ✅ PASS | Fixtures generated for C3+ contracts. `@TEST_EDGE` declarations enable TDD. | | **VIII. Attention-Optimized Contracts** | ✅ PASS | All contract IDs hierarchical: `AgentChat.Context`, `AgentChat.Confirmation`, `AgentChat.ToolFilter`. Shared `[SEMANTICS agent-chat,...]` keyword. | -**No blocking violations.** Re-check after Phase 1: verify no contract exceeds 150 lines, no module exceeds 400 lines post-extension. +**No blocking violations at planning time.** 2026-07-05 verification notes production code and scoped tests are present; live E2E/quickstart remains open. + +## Current Next Steps + +1. Execute live E2E scenarios against a running backend + Gradio + frontend stack. +2. Run quickstart validation once LLM quota is available. +3. Track full-repo lint debt separately from the 035 feature gate. #endregion — plan metadata section diff --git a/specs/035-agent-chat-context/traceability.md b/specs/035-agent-chat-context/traceability.md index 31d35c80..ae245b3d 100644 --- a/specs/035-agent-chat-context/traceability.md +++ b/specs/035-agent-chat-context/traceability.md @@ -3,31 +3,33 @@ ## Traceability Matrix -| Story | Requirement | Model/Contract | API/SSE | Backend Task | Frontend Task | Test | -|-------|------------|----------------|---------|-------------|--------------|------| -| US1 | AGCTX-FR-001, FR-003 | AgentChat.Model | — | — | T010, T011, T012 | T007 | -| US1 | AGCTX-FR-002, FR-004 | AgentChat.Context.Inject, Validate | Gradio submit (uicontext at pos 6) | T005, T013, T014 | — | T008 | -| US1 | AGCTX-FR-005 | AgentChat.ToolFilter.Pipeline | — | T006 (build_tool_pipeline) | — | T009 | -| US1 | prod warning | AgentChat.Model | — | — | T015, T016 | T007 | -| US2 | AGGRD-FR-001, FR-002 | AgentChat.Confirmation.GuardV2 | confirm_required (extended) | T022 | T025, T026, T027 | T020, T021 | -| US2 | AGGRD-FR-003 | AgentChat.Confirmation.PermissionDenied | permission_denied (NEW) | T023, T024 | T028 | T020 | -| US2 | AGGRD-FR-004 | AgentChat.StreamProcessor | confirm_resolved (denied) | — | T029 | T021 | -| US3 | AGTL-FR-001, FR-005 | AgentChat.ToolFilter.InvocationGuard | permission_denied | T006, T024 | — | T009 | -| US3 | AGTL-FR-002 | AgentChat.Tools.Retry | tool_retry (NEW) | T037 | T040, T041 | T033, T036 | -| US3 | AGTL-FR-003 | AgentChat.Tools.Summarise | — | T038 | — | T034 | -| US3 | AGTL-FR-004 | AgentChat.Tools.Timeout | tool_timeout (extended) | T039 | T040, T041 | T035, T036 | -| US3 | AGTL-FR-006 (SHOULD) | AgentChat.EmbeddingRouter | — | post-MVP | — | — | -| SC-006 | AGCTX-FR-004 (validation) | AgentChat.Context.Validate | — | T005 | — | T008 | +**2026-07-05 verification update**: production code and dedicated US1/US2 tests are present. Remaining open items are live E2E / quickstart validation requiring a running stack and available LLM quota. + +| Story | Requirement | Model/Contract | API/SSE | Backend Task | Frontend Task | Test | 2026-07-05 Status | +|-------|------------|----------------|---------|-------------|--------------|------|-------------------| +| US1 | AGCTX-FR-001, FR-003 | AgentChat.Model | — | — | T010, T011, T012 | T007 | ✅ Implemented + `AgentChatModel.context.test.ts` | +| US1 | AGCTX-FR-002, FR-004 | AgentChat.Context.Inject, Validate | Gradio submit (uicontext appended) | T005, T013, T014 | — | T008 | ✅ Implemented + `test_agent_context.py` | +| US1 | AGCTX-FR-005 | AgentChat.ToolFilter.Pipeline | `pipeline_result` debug SSE | T006 | T042 | T009 | ✅ Implemented + `test_agent_tool_filter.py` | +| US1 | prod warning | AgentChat.Model | — | — | T015, T016 | T007 | ✅ Implemented + model context test | +| US2 | AGGRD-FR-001, FR-002 | AgentChat.Confirmation.GuardV2 | confirm_required (extended) | T022 | T025, T026, T027 | T020, T021 | ✅ Implemented + `test_agent_confirmation_v2.py` + UX tests | +| US2 | AGGRD-FR-003 | AgentChat.Confirmation.PermissionDenied | permission_denied | T023, T024 | T028 | T020 | ✅ Implemented + contract tests | +| US2 | AGGRD-FR-004 | AgentChat.StreamProcessor | confirm_resolved (denied) | — | T029 | T021 | ✅ Implemented + UX tests | +| US3 | AGTL-FR-001, FR-005 | AgentChat.ToolFilter.InvocationGuard | permission_denied | T006, T024 | — | T009 | ✅ Implemented + pipeline/RBAC tests | +| US3 | AGTL-FR-002 | AgentChat.Tools.Retry | tool_retry | T037 | T040, T041 | T033, T036 | ✅ Implemented and tested | +| US3 | AGTL-FR-003 | AgentChat.Tools.Summarise | — | T038 | — | T034 | ✅ Implemented and tested | +| US3 | AGTL-FR-004 | AgentChat.Tools.Timeout | tool_timeout | T039 | T040, T041 | T035, T036 | ✅ Implemented and tested | +| US3 | AGTL-FR-006 (SHOULD) | AgentChat.EmbeddingRouter | — | post-MVP | — | — | Deferred post-MVP | +| SC-006 | AGCTX-FR-004 (validation) | AgentChat.Context.Validate | — | T005 | — | T008 | ✅ Validator + dedicated context tests | ## Impact Analysis | If you change... | Contracts verify | Tests verify | Screens depend | |-----------------|-----------------|-------------|----------------| -| agent_handler uicontext pos | AgentChat.LangGraph.Handler | T008 | AgentChat panel | -| _CONTEXT_TOOL_AFFINITY | AgentChat.ToolFilter.Pipeline | T009 | AgentChat panel | -| _TOOL_PERMISSIONS | AgentChat.ToolFilter.InvocationGuard | T020 | AgentChat panel | -| build_confirmation_contract_v2 | AgentChat.Confirmation.GuardV2 | T020, T021 | ConfirmationCard | -| yield_permission_denied | AgentChat.Confirmation.PermissionDenied | T020 | ConfirmationCard | +| agent_handler uicontext pos | AgentChat.LangGraph.Handler | T008 / `test_agent_context.py` | AgentChat panel | +| _CONTEXT_TOOL_AFFINITY | AgentChat.ToolFilter.Pipeline | T009 / `test_agent_tool_filter.py` | AgentChat panel | +| _TOOL_PERMISSIONS | AgentChat.ToolFilter.InvocationGuard | T009/T020 | AgentChat panel | +| build_confirmation_contract_v2 | AgentChat.Confirmation.GuardV2 | T020/T021 / `test_agent_confirmation_v2.py` | ConfirmationCard | +| yield_permission_denied | AgentChat.Confirmation.PermissionDenied | T020 + UX tests | ConfirmationCard | | _retry_read_tool | AgentChat.Tools.Retry | T033 | ToolCallCard | | _execute_with_timeout | AgentChat.Tools.Timeout | T035 | ToolCallCard |