docs(agent-chat): update 035 verification traceability
This commit is contained in:
@@ -10,6 +10,20 @@ Refinement of the agent chat system across three vectors:
|
||||
2. **Guardrails refinement** (P2): Risk classification extended from tool-name prefixes to include environment context and tool danger level. ConfirmationCard gains environment badges, destructive-operation countdown (10s), permission-denied state with alternatives, and visual risk toning.
|
||||
3. **Tool optimisation** (P3): Backend: RBAC tool filtering, auto-retry on transient HTTP errors, semantic response summarisation, configurable tool timeout, context-aware tool affinity. Frontend: ToolCallCard extended with retry/timeout/cancelled states.
|
||||
|
||||
## Implementation Status — 2026-07-05 Fact Check
|
||||
|
||||
Read-only audit against source/tests found production code and dedicated US1/US2 tests implemented. Live E2E/quickstart validation remains the primary open gate.
|
||||
|
||||
| Area | Status | Evidence / Gap |
|
||||
|------|:------:|----------------|
|
||||
| Fixtures and foundational types | ✅ Implemented | API/model fixtures materialized; `UIContext`, `ConfirmMetadataV2`, `PermissionDeniedMetadata`, and extended `ToolCall` types are present. |
|
||||
| Context-aware `/agent` flow | ✅ Implemented | `/agent` reads URL params, model builds UIContext/pill, backend validates/injects context, and tool filtering pipeline is present. |
|
||||
| Guardrails production code | ✅ Implemented | Confirmation v2, permission_denied, invocation guard, countdown, deny/cancel flow, and distinct write_dev/write_staging/write_prod visual states are present. |
|
||||
| Tool optimisation | ✅ Implemented | Retry, summarise, timeout helpers and ToolCallCard retry/timeout/cancelled states are present with dedicated tests. |
|
||||
| Debug panel pipeline | ✅ Implemented | Backend `pipeline_result` SSE feeds `effectiveToolPipeline`, rendered in the debug panel. |
|
||||
| Dedicated US1/US2 tests | ✅ Implemented | `test_agent_context.py`, `test_agent_tool_filter.py`, `test_agent_confirmation_v2.py`, `AgentChatModel.context.test.ts`, and `ConfirmationCard.ux.test.ts` are present. |
|
||||
| Verification gates | ⚠️ Partially verified | Scoped backend/frontend tests and build pass; live E2E/quickstart remains pending. |
|
||||
|
||||
## Technical Context
|
||||
|
||||
**Language/Version**: Python 3.9+ (backend), TypeScript (frontend Svelte 5 runes-only)
|
||||
@@ -38,6 +52,12 @@ Refinement of the agent chat system across three vectors:
|
||||
| **VII. Test-Driven for C3+ Contracts** | ✅ PASS | Fixtures generated for C3+ contracts. `@TEST_EDGE` declarations enable TDD. |
|
||||
| **VIII. Attention-Optimized Contracts** | ✅ PASS | All contract IDs hierarchical: `AgentChat.Context`, `AgentChat.Confirmation`, `AgentChat.ToolFilter`. Shared `[SEMANTICS agent-chat,...]` keyword. |
|
||||
|
||||
**No blocking violations.** Re-check after Phase 1: verify no contract exceeds 150 lines, no module exceeds 400 lines post-extension.
|
||||
**No blocking violations at planning time.** 2026-07-05 verification notes production code and scoped tests are present; live E2E/quickstart remains open.
|
||||
|
||||
## Current Next Steps
|
||||
|
||||
1. Execute live E2E scenarios against a running backend + Gradio + frontend stack.
|
||||
2. Run quickstart validation once LLM quota is available.
|
||||
3. Track full-repo lint debt separately from the 035 feature gate.
|
||||
|
||||
#endregion — plan metadata section
|
||||
|
||||
@@ -3,31 +3,33 @@
|
||||
|
||||
## Traceability Matrix
|
||||
|
||||
| Story | Requirement | Model/Contract | API/SSE | Backend Task | Frontend Task | Test |
|
||||
|-------|------------|----------------|---------|-------------|--------------|------|
|
||||
| US1 | AGCTX-FR-001, FR-003 | AgentChat.Model | — | — | T010, T011, T012 | T007 |
|
||||
| US1 | AGCTX-FR-002, FR-004 | AgentChat.Context.Inject, Validate | Gradio submit (uicontext at pos 6) | T005, T013, T014 | — | T008 |
|
||||
| US1 | AGCTX-FR-005 | AgentChat.ToolFilter.Pipeline | — | T006 (build_tool_pipeline) | — | T009 |
|
||||
| US1 | prod warning | AgentChat.Model | — | — | T015, T016 | T007 |
|
||||
| US2 | AGGRD-FR-001, FR-002 | AgentChat.Confirmation.GuardV2 | confirm_required (extended) | T022 | T025, T026, T027 | T020, T021 |
|
||||
| US2 | AGGRD-FR-003 | AgentChat.Confirmation.PermissionDenied | permission_denied (NEW) | T023, T024 | T028 | T020 |
|
||||
| US2 | AGGRD-FR-004 | AgentChat.StreamProcessor | confirm_resolved (denied) | — | T029 | T021 |
|
||||
| US3 | AGTL-FR-001, FR-005 | AgentChat.ToolFilter.InvocationGuard | permission_denied | T006, T024 | — | T009 |
|
||||
| US3 | AGTL-FR-002 | AgentChat.Tools.Retry | tool_retry (NEW) | T037 | T040, T041 | T033, T036 |
|
||||
| US3 | AGTL-FR-003 | AgentChat.Tools.Summarise | — | T038 | — | T034 |
|
||||
| US3 | AGTL-FR-004 | AgentChat.Tools.Timeout | tool_timeout (extended) | T039 | T040, T041 | T035, T036 |
|
||||
| US3 | AGTL-FR-006 (SHOULD) | AgentChat.EmbeddingRouter | — | post-MVP | — | — |
|
||||
| SC-006 | AGCTX-FR-004 (validation) | AgentChat.Context.Validate | — | T005 | — | T008 |
|
||||
**2026-07-05 verification update**: production code and dedicated US1/US2 tests are present. Remaining open items are live E2E / quickstart validation requiring a running stack and available LLM quota.
|
||||
|
||||
| Story | Requirement | Model/Contract | API/SSE | Backend Task | Frontend Task | Test | 2026-07-05 Status |
|
||||
|-------|------------|----------------|---------|-------------|--------------|------|-------------------|
|
||||
| US1 | AGCTX-FR-001, FR-003 | AgentChat.Model | — | — | T010, T011, T012 | T007 | ✅ Implemented + `AgentChatModel.context.test.ts` |
|
||||
| US1 | AGCTX-FR-002, FR-004 | AgentChat.Context.Inject, Validate | Gradio submit (uicontext appended) | T005, T013, T014 | — | T008 | ✅ Implemented + `test_agent_context.py` |
|
||||
| US1 | AGCTX-FR-005 | AgentChat.ToolFilter.Pipeline | `pipeline_result` debug SSE | T006 | T042 | T009 | ✅ Implemented + `test_agent_tool_filter.py` |
|
||||
| US1 | prod warning | AgentChat.Model | — | — | T015, T016 | T007 | ✅ Implemented + model context test |
|
||||
| US2 | AGGRD-FR-001, FR-002 | AgentChat.Confirmation.GuardV2 | confirm_required (extended) | T022 | T025, T026, T027 | T020, T021 | ✅ Implemented + `test_agent_confirmation_v2.py` + UX tests |
|
||||
| US2 | AGGRD-FR-003 | AgentChat.Confirmation.PermissionDenied | permission_denied | T023, T024 | T028 | T020 | ✅ Implemented + contract tests |
|
||||
| US2 | AGGRD-FR-004 | AgentChat.StreamProcessor | confirm_resolved (denied) | — | T029 | T021 | ✅ Implemented + UX tests |
|
||||
| US3 | AGTL-FR-001, FR-005 | AgentChat.ToolFilter.InvocationGuard | permission_denied | T006, T024 | — | T009 | ✅ Implemented + pipeline/RBAC tests |
|
||||
| US3 | AGTL-FR-002 | AgentChat.Tools.Retry | tool_retry | T037 | T040, T041 | T033, T036 | ✅ Implemented and tested |
|
||||
| US3 | AGTL-FR-003 | AgentChat.Tools.Summarise | — | T038 | — | T034 | ✅ Implemented and tested |
|
||||
| US3 | AGTL-FR-004 | AgentChat.Tools.Timeout | tool_timeout | T039 | T040, T041 | T035, T036 | ✅ Implemented and tested |
|
||||
| US3 | AGTL-FR-006 (SHOULD) | AgentChat.EmbeddingRouter | — | post-MVP | — | — | Deferred post-MVP |
|
||||
| SC-006 | AGCTX-FR-004 (validation) | AgentChat.Context.Validate | — | T005 | — | T008 | ✅ Validator + dedicated context tests |
|
||||
|
||||
## Impact Analysis
|
||||
|
||||
| If you change... | Contracts verify | Tests verify | Screens depend |
|
||||
|-----------------|-----------------|-------------|----------------|
|
||||
| agent_handler uicontext pos | AgentChat.LangGraph.Handler | T008 | AgentChat panel |
|
||||
| _CONTEXT_TOOL_AFFINITY | AgentChat.ToolFilter.Pipeline | T009 | AgentChat panel |
|
||||
| _TOOL_PERMISSIONS | AgentChat.ToolFilter.InvocationGuard | T020 | AgentChat panel |
|
||||
| build_confirmation_contract_v2 | AgentChat.Confirmation.GuardV2 | T020, T021 | ConfirmationCard |
|
||||
| yield_permission_denied | AgentChat.Confirmation.PermissionDenied | T020 | ConfirmationCard |
|
||||
| agent_handler uicontext pos | AgentChat.LangGraph.Handler | T008 / `test_agent_context.py` | AgentChat panel |
|
||||
| _CONTEXT_TOOL_AFFINITY | AgentChat.ToolFilter.Pipeline | T009 / `test_agent_tool_filter.py` | AgentChat panel |
|
||||
| _TOOL_PERMISSIONS | AgentChat.ToolFilter.InvocationGuard | T009/T020 | AgentChat panel |
|
||||
| build_confirmation_contract_v2 | AgentChat.Confirmation.GuardV2 | T020/T021 / `test_agent_confirmation_v2.py` | ConfirmationCard |
|
||||
| yield_permission_denied | AgentChat.Confirmation.PermissionDenied | T020 + UX tests | ConfirmationCard |
|
||||
| _retry_read_tool | AgentChat.Tools.Retry | T033 | ToolCallCard |
|
||||
| _execute_with_timeout | AgentChat.Tools.Timeout | T035 | ToolCallCard |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user