ss-tools/specs/033-gradio-agent-chat/tests/test-documentation.md

#region AgentChat.TestDocs [C:2] [TYPE ADR] [SEMANTICS test,documentation,agent-chat] @BRIEF Test documentation for Gradio Agent Chat (LangGraph) — coverage summary, semantic audit, commands.

Test Documentation: Gradio Agent Chat (LangGraph)

Feature: specs/033-gradio-agent-chat/spec.md Created: 2026-06-10 Updated: 2026-06-10 Tester: QA Agent (Verification Loop)

---

Overview

Gradio agent backend + Svelte frontend via @gradio/client submit(). LangGraph create_react_agent() + static interrupt_before + PostgresSaver. Resume uses the checkpoint thread and disables interrupt_before for the resumed run to avoid repeated guardrails. Structured JSON metadata streaming. No REST confirmations, no custom WebSocket.

Test Strategy:

• Unit Tests (co-located in __tests__/ directories)
• Integration Tests (backend API test client)
• Contract Tests (for API endpoints and semantic contract boundaries)
• L1 Model Invariant Tests (no DOM render, ~10ms)
• L2 UX Contract Tests (with @testing-library/svelte, ~500ms)
• Semantic Contract Verification (@PRE, @POST, @SIDE_EFFECT, @TEST_*)
• UX Contract Verification (@UX_STATE, @UX_FEEDBACK, @UX_RECOVERY)
• Fixture Materialization Tests (JSON fixture structure validation)
• E2E Tests (require running Gradio + FastAPI + PostgreSQL)

---

Test Coverage Matrix

Module / Contract	Type	File	Tests	Status
AgentChat.GradioApp.Handler	C4	backend/src/agent/app.py	test_agent_handler.py (8)	✅ 8/8
AgentChat.LangGraph.Setup	C4	backend/src/agent/langgraph_setup.py	Indirect via handler tests	⚠️ No direct tests
AgentChat.Tools	C4	backend/src/agent/tools.py	test_langchain_tools.py + handler/langgraph setup tests	✅ native 17-tool catalog + compact subset
AgentChat.Context	C3	backend/src/agent/context.py	Indirect via tool tests	⚠️ No direct tests
AgentChat.Middleware	C3	backend/src/agent/middleware.py	—	❌ No tests
AgentChat.Document.Parser	C3	backend/src/agent/document_parser.py	test_document_parser.py (9)	✅ 9/9
AgentChat.Api.Conversations	C3	backend/src/api/routes/agent_conversations.py	test_conversation_api.py (11)	✅ 11/11 (NEW)
AgentChat.Model	C4	frontend/src/lib/models/AgentChatModel.svelte.ts	AgentChatModel.test.ts + .2.test.ts	✅ 100+ L1 tests
AgentChat.Panel	C4	frontend/src/lib/components/assistant/AssistantChatPanel.svelte	Integration tests + L2	✅
AgentChat.ConversationList	C3	frontend/src/lib/components/assistant/ConversationList.svelte	ConversationList.test.ts (12)	✅ 12/12 (NEW)
AgentChat.ToolCallCard	C2	frontend/src/lib/components/assistant/ToolCallCard.svelte	ToolCallCard.test.ts (5)	✅ 5/5
Fixture Manifest	C2	fixtures/manifest.md (19 fixtures)	test_model_fixtures.py + test_api_fixtures.py	✅ 14/14

---

Test Cases

Backend — Agent Handler (test_agent_handler.py)

Target File: backend/src/agent/app.py

ID	Test Case	Type	Expected	Status
H001	Empty message returns immediately	Unit	0 chunks	✅
H002	Missing auth continues gracefully	Unit	No UNAUTHORIZED error	✅
H003	Invalid JWT continues gracefully	Unit	No UNAUTHORIZED error	✅
H004	Streaming yields stream_token chunks	Unit	stream_token metadata	✅
H005	Resume confirm yields confirm_resolved	Unit	confirm_resolved metadata	✅
H006	Resume deny yields confirm_resolved (denied)	Unit	confirm_resolved: denied	✅

Backend — LangChain Tools (test_langchain_tools.py)

ID	Test Case	Type	Expected	Status
T001	Dual-identity JWT headers sent	Unit	Auth + X-User-JWT headers	✅
T002	Env fallback when ContextVar empty	Unit	SERVICE_JWT env var used	✅
T003	HTTP exception propagates	Unit	Exception raised	✅
T004	Native tool catalog and compact subset selection	Unit	17-tool catalog; get_tools_for_query() returns bounded subsets	✅
T005	search_dashboards has args_schema	Unit	Pydantic schema with query	✅
T006	Tool calls correct URL	Unit	GET /api/dashboards	✅
T007	Tool handles env_id param	Unit	Correct URL params	✅

Backend — Conversation API (test_conversation_api.py)

ID	Test Case	Type	Expected	Status
C001	Save creates new conversation	Integration	200, saved=true	✅
C002	Save updates existing conversation	Integration	200, title updated	✅
C003	Save defaults user_id to admin	Integration	200	✅
C004	Active gate returns {active: false}	Integration	200	✅
C005	LLM config endpoint reachable	Integration	200	✅
C006	LLM config shape on 200	Integration	configured field	✅
C007	Legacy history empty for nonexistent	Integration	200, items=[]	✅
C008	Legacy conversations list	Integration	200, items	✅
C009	Legacy pagination params	Integration	200	✅

Backend — Confirmations (test_confirmations.py)

ID	Test Case	Type	Expected	Status
CF001	Concurrent send lock	Unit	CONCURRENT_SEND error	✅
CF002	Unknown action as normal message	Unit	stream_token, not confirm	✅
CF003	Graph creation failure handled	Unit	Graceful handling	✅

Backend — Document Parser (test_document_parser.py)

ID	Test Case	Type	Expected	Status
D001	Valid PDF extracts text	Unit	Text content	✅
D002	Empty PDF raises ParseError	Unit	ParseError	✅
D003	Non-existent PDF raises error	Unit	Error	✅
D004	Valid XLSX extracts sheets	Unit	Sheet data	✅
D005	Empty XLSX sheet	Unit	Sheet name only	✅
D006	Non-XLSX raises ParseError	Unit	ParseError	✅
D007	TXT upload parsed	Unit	Text content	✅
D008	JSON upload parsed	Unit	Text content	✅
D009	Unsupported format raises error	Unit	ParseError	✅

Frontend — ConversationList (ConversationList.test.ts)

ID	Test Case	Type	Expected	Status
L001	Renders conversation titles	L2 UX	All 3 titles visible	✅
L002	Highlights active conversation	L2 UX	bg-surface-muted class	✅
L003	Shows message count	L2 UX	"5 msgs" text	✅
L004	Skeleton on loading	L2 UX	animate-pulse visible	✅
L005	No skeleton when loaded	L2 UX	No pulse	✅
L006	Empty placeholder	L2 UX	"No conversations"	✅
L007	Search fires with debounce	L2 UX	300ms delay	✅
L008	Delete calls ondelete after confirm	L2 UX	ondelete called	✅
L009	Delete no-op on cancel	L2 UX	ondelete not called	✅
L010	onselect on click	L2 UX	Called with id	✅
L011	Load more button	L2 UX	Visible when hasNext	✅
L012	No load more when !hasNext	L2 UX	Hidden	✅

Backend — Fixtures (test_model_fixtures.py, test_api_fixtures.py)

ID	Test Case	Type	Expected	Status
F001	SendMessage fixture structure	Fixture	Correct fields	✅
F002	CancelGeneration fixture	Fixture	Correct fields	✅
F003	ResumeConfirm fixture	Fixture	Correct fields	✅
F004	ResumeDeny fixture	Fixture	Correct fields	✅
F005	Rejected WebSocket fixture	Fixture	No WebSocket imports	✅
F006	API fixtures load (9 files)	Fixture	Valid JSON	✅
F007	Conversations list valid	Fixture	200 + items	✅
F008	Conversations list empty	Fixture	200 + []	✅
F009	Conversations missing auth	Fixture	401	✅
F010	History valid	Fixture	200 + messages	✅
F011	History not found	Fixture	404	✅
F012	Service token valid	Fixture	200	✅
F013	Service token invalid	Fixture	401	✅
F014	Conversations list invalid page	Fixture	422	✅

---

Test Execution Reports

Report 2026-06-10 (Final)

Executed by: QA Agent Duration: ~5 min Result: Pass (with pre-existing unrelated failures)

Summary:

Suite	Total	Passed	Failed	Status
Backend agent tests	47	47	0	✅
Backend full suite*	1229	1212	17	⚠️ (15 pre-existing)
Frontend full suite	2431	2431	0	✅
Frontend build	1	1	0	✅
Backend ruff lint (agent)	20	12 fixed	8 remaining	⚠️ (pre-existing)
Frontend lint	—	—	—	—

* 15 of 17 failures are pre-existing (test_task_manager: 14, test_smoke_app: 1). 2 LLM config tests fail in full suite context due to environmental contamination (pass in isolation).

---

Semantic Audit Verdict: PASS

Projection	Status	Notes
P1 Contract Completeness	✅ PASS	All source files have proper anchors; minor: AgentChat.Context missing @BRIEF, AgentChat.Run missing @POST
P2 Decision-Memory	✅ PASS	All @REJECTED paths documented; regression test for WebSocket rejection exists
P3 Attention Resilience	✅ PASS	All anchors pass density rules, hierarchical IDs, <150 line contracts
P4 Coverage & Traceability	✅ PASS (with gaps)	4 new test files added; gaps documented below
P5 Architecture Realism	✅ PASS	Tests in correct directories
P6 Protocol Alignment	✅ PASS	Canonical syntax used
P7 Non-Functional	✅ PASS	Molecular CoT logging in all C4+ code

Coverage Gaps Remaining

Contract	Missing Coverage	Priority	Notes
AgentChat.LangGraph.Setup	No direct unit tests	Medium	create_agent(), configure_from_api()
AgentChat.Context	No direct unit tests	Low	Thin ContextVar wrapper
AgentChat.Middleware	No tests for audit logging	Medium	Async write to assistant_audit is TODO
AgentChat.Panel	No L2 component tests	Low	Integration tests exist
AgentChat.Page	No tests for two-column layout	Low	Simple routing component
AgentChat.ConfirmationCard	Component not yet implemented	Low	Confirmation handled inline in Panel

---

Anti-Patterns & Rules

✅ DO

1. Hardcoded fixtures for expected values (no logic mirrors)
2. External dependency mock at [EXT:...] boundaries
3. L1 model tests without DOM render for invariants
4. L2 UX tests with @testing-library/svelte for visual states

❌ DON'T

1. No logic mirrors — fixtures use INLINE_JSON or file paths
2. No WebSocket imports in frontend model (regression-tested)
3. No REST confirmation endpoints in new agent code (T096 verified)
4. No @COMPLEXITY N or @C N standalone tags

---

Rejected-Path Regression Coverage

Rejected Path	Test	Status
Custom WebSocket protocol	test_fixture_rejected_websocket — scans source for WebSocket imports	✅
@assistant_tool registry	Agent runtime uses backend/src/agent/tools.py native @tool; _tool_registry.py is legacy/deprecated only	✅ current implementation fact
ConfirmationRiskMiddleware	Implicit — middleware.py has @REJECTED tag	⚠️ No explicit regression test
REST confirmation endpoints	T096 grep audit — no confirm/deny routes in agent code	✅
Non-streaming	Implicit — all tests verify streaming	✅

---

Related Documents

• spec.md
• plan.md
• tasks.md
• contracts/modules.md
• contracts/ux/agent-chat-ux.md
• contracts/ux/decisions.md
• fixtures/manifest.md
• traceability.md

#endregion AgentChat.TestDocs