Requirements Quality Checklist: Gradio Agent Chat — Full Sweep

Purpose: Validate ALL requirements artifacts (spec, plan, tasks, contracts, research, data-model) for completeness, clarity, consistency, traceability, and decision-memory readiness after the final LangGraph/structured-metadata architecture pivot. Created: 2026-06-08 Feature: specs/033-gradio-agent-chat/spec.md

1. Requirement Completeness

CHK001 Are all 25 functional requirements (FR-001 through FR-025) individually testable via acceptance criteria, not just stated as desired behavior? [Completeness, Spec §Requirements]
CHK002 Do Stories 3 through 6 have explicit, written acceptance criteria matching the level of detail in Stories 1 and 2? [Gap, Spec §User Stories — Stories 3-6 lack explicit AC]
CHK003 Are rollback/recovery requirements defined for every state mutation operation (delete conversation, tool execution, confirmation timeout)? [Coverage, Gap, Spec §Edge Cases]
CHK004 Are requirements for the additional_inputs payload shape (which fields, order, types) documented unambiguously across all three layers (Gradio Python, Gradio JS, Svelte model)? [Completeness, Spec §FR-001/FR-009]
CHK005 Does the spec define how Gradio JS Client.submit() payload maps to Gradio Python handler parameters for the __resume__ case? [Gap, Spec §FR-009]
CHK006 Are file upload rejection reasons (size, format, encryption) specified as part of the requirements rather than deferred to implementation? [Completeness, Spec §FR-021]

2. Requirement Clarity

CHK007 Is "structured JSON metadata in ChatMessage.metadata" (FR-016) specified with an exhaustive list of all metadata.type values and their payload shapes? [Clarity, Spec §FR-016, data-model.md §1]
CHK008 Are the 7 StreamMetadata type discriminator values (stream_token, tool_start, tool_end, tool_error, confirm_required, confirm_resolved, error) documented in both spec and data-model without drift? [Consistency, Spec §FR-016 vs data-model.md §1]
CHK009 Is "concurrency_limit=1" specified as per-handler or per-user, and are the consequences of this choice documented? [Clarity, Spec §FR-015 — risk of global limit vs per-user limit]
CHK010 Is the "stateless JWT validation" (FR-020) explicitly defined in terms of algorithm, required claims, and secret management? [Clarity, Spec §FR-020]
CHK011 Does "RunnableWithMessageHistory auto-loads context" (FR-013) define what "context" means — last N messages, summary, or full thread? [Clarity, Spec §FR-013]
CHK012 Is "soft-delete / Archive" (FR-011) consistently described across all artifacts (spec says Archive, tasks say delete, fixtures say is_archived=true)? [Consistency, Spec §FR-011 vs tasks.md T055 vs fixtures]

3. Requirement Consistency

CHK013 Do all contracts in contracts/modules.md reference the correct LangGraph API (create_react_agent, PostgresSaver, interrupt(), Command(resume=...)) without leftover HumanInTheLoopMiddleware or create_agent references? [Consistency, contracts/modules.md — check AgentChat.LangChain.Setup for stale middleware ref]
CHK014 Do the research decisions (§1-8) use the same terminology as the spec FRs? (Research §3 says "HumanInTheLoopMiddleware" but spec FR-008 says "interrupt()" — drift.) [Consistency, research.md §3 vs spec.md FR-008]
CHK015 Is "dual identity" defined identically in spec (FR-007/FR-019), api-ux.md, data-model.md, and tasks.md — same header names, same validation sequence? [Consistency across artifacts]
CHK016 Do the 19 fixture files in fixtures/ align with the C3+ contracts they claim to verify — correct endpoint paths, correct error codes, correct response shapes? [Traceability, fixtures/manifest.md vs contracts/modules.md]
CHK017 Are task numbers in tasks.md unique and non-overlapping? (T026 appears in both Phase 3 and Phase 4.) [Consistency, tasks.md — numbering overlap]
CHK018 Does plan.md's summary match the final LangGraph/structured-metadata architecture, or does it still reference LangChain v1/AgentExecutor patterns? [Consistency, plan.md — after multiple overwrites from setup-plan.sh]

4. Acceptance Criteria Quality

CHK019 Can SC-001 (tool selection ≥85% accuracy) be measured with the acceptance test suite defined in tasks, or is a separate benchmark methodology required? [Measurability, Spec §Success Criteria]
CHK020 Is "first token <1.5s" (SC-002) testable end-to-end (browser → nginx → Gradio → LangGraph → LLM → back) or only in isolation? [Measurability, Spec §SC-002]
CHK021 Are the P3 Stories (Story 5 — conversation persistence, Story 6 — file upload) verifiable independently of the P1 streaming chat, per the "story independence" claim? [Measurability, tasks.md §Story Independence]
CHK022 Does "UX reference validation — verify all 15 FSM states" (T090) define acceptance criteria for each state, or just a blanket requirement to verify? [Clarity, tasks.md T090]

5. Scenario Coverage

CHK023 Are requirements specified for the "Gradio handler receives confirm/deny action in additional_inputs" scenario — what happens if action is neither "confirm", "deny", nor null? [Coverage, Exception Flow, Spec §FR-009]
CHK024 Are requirements specified for "second submit arrives before first stream ended" (race condition between HITL confirm and primary stream cleanup)? [Coverage, Exception Flow, Gap]
CHK025 Are requirements specified for PostgresSaver initialization — auto-create tables, migration script, or manual setup? [Coverage, Gap — data-model.md mentions checkpointer but not schema setup]
CHK026 Are requirements specified for the "conversation_id mismatch" scenario — what happens when additional_inputs conversation_id doesn't match the primary message context? [Coverage, Gap]
CHK027 Are requirements specified for Gradio container startup ordering — does the agent container wait for FastAPI and PostgreSQL to be ready before starting? [Coverage, Dependency]

6. Edge Case Coverage

CHK028 Is the edge case "LLM returns malformed tool-call JSON → retry" (listed in spec) traceable to a specific FR or task? (Not found in tasks T082 — retry task was removed.) [Gap, Spec §Edge Cases vs tasks.md]
CHK029 Is "message too long → truncate" quantified with a specific token limit in an FR? (No FR defines the limit.) [Gap, Spec §Edge Cases]
CHK030 Is the edge case "Gradio container restarts mid-conversation" addressed for BOTH in-memory state (lost) AND checkpoint state (preserved via PostgresSaver)? [Coverage, Spec §Edge Cases vs FR-012]
CHK031 Is the "concurrent message submit from second tab" rejection UX specified — toast, error state, or silent drop? [Gap, Spec §Edge Cases]

7. Non-Functional Requirements

CHK032 Are performance requirements specified for all critical paths — not just first-token latency (SC-002) but also tool-call execution, confirmation resume, and file upload parsing? [Completeness, Non-Functional]
CHK033 Are time-to-live (TTL) or garbage-collection requirements specified for: pending interrupts, archived conversations, expired service JWTs, stale checkpoints? [Completeness, Non-Functional]
CHK034 Are concurrency requirements specified for the Gradio container — max concurrent users, max conversations per user, max messages per conversation? [Gap, Non-Functional, Scale]
CHK035 Are accessibility requirements (ARIA roles, keyboard navigation, screen reader flows) specified for the new chat components? [Gap, Non-Functional, A11y]
CHK036 Are monitoring/observability requirements specified beyond audit logging — Gradio container healthcheck, LangGraph execution metrics, streaming latency tracking? [Gap, Non-Functional, Observability]

8. Dependencies & Assumptions

CHK037 Is the assumption that langgraph-checkpoint-postgres auto-creates required tables validated, or is an explicit migration setup required? [Assumption, Spec §FR-012, Gap in T016]
CHK038 Is the version constraint langgraph>=0.2 validated against actual PyPI availability, or is this a best-guess version? [Assumption, Dependencies]
CHK039 Is the dependency multimodal=True (FR-001) documented with its side effects — that message becomes {text, files} dict and the handler signature changes? [Clarity, Spec §FR-021 vs FR-001]
CHK040 Are external service dependencies (LLM provider, PostgreSQL) documented with failure-mode requirements — what happens on LLM timeout vs PostgreSQL down vs nginx misconfig? [Completeness, Dependencies]

9. Decision Memory & ADRs

CHK041 Are all 5 @REJECTED patterns from spec.md traceable to verification tasks that explicitly prevent their resurrection? (T092 covers 3, T094 covers 1 — is @REJECTED "REST confirmation endpoints" covered?) [Gap, Decision Memory, Spec @REJECTED vs tasks]
CHK042 Is the decision to reject REST confirmation endpoints (research §3) guarded by a specific task or invariant, or is it only stated in research.md? [Traceability, research.md §3 → tasks]
CHK043 Does the "dual identity" RBAC decision have explicit @RATIONALE explaining WHY both service JWT and user JWT are needed rather than just one? [Decision Memory, Spec §FR-007]
CHK044 Are the "8 rejected alternatives" from research.md traceable to specific guardrails in tasks.md — or are they informational only, with no enforcement mechanism? [Traceability, research.md vs tasks.md]
CHK045 Are ADR artifacts explicitly referenced from the plan where the feature depends on, extends, or overrides an existing ADR? (FR-022 references ADR-0002, FR-005 references ADR-0005/0008 — are ADR numbers correct?) [Traceability, Spec §FR-005/FR-022 vs docs/adr/]
CHK046 Can a coder determine from the planning artifacts alone which implementation shortcuts are forbidden without re-reading research or spec? (e.g., "don't use WebSocket, don't add REST confirmation endpoints, don't reuse @assistant_tool") [Clarity, Decision Memory]

10. Ambiguities & Conflicts

CHK047 Does the term "concurrency_limit=1" in FR-015 create a conflict with the requirement to handle multiple users simultaneously? (Gradio concurrency_limit is global, not per-user.) [Conflict, Spec §FR-015 vs multi-user requirement]
CHK048 Is there a conflict between "Gradio does NOT pass history — only conversation_id" (FR-013) and the fact that Gradio ChatInterface inherently passes history as the second handler parameter? [Conflict, Spec §FR-013 vs Gradio API contract]
CHK049 Does the AgentChat.LangChain.Setup contract in contracts/modules.md still reference HumanInTheLoopMiddleware, conflicting with spec FR-008 (LangGraph interrupt)? [Conflict, contracts/modules.md vs spec.md FR-008]
CHK050 Does the ResumeConfirm/ConfirmationCard contract reference __resume__ while api-ux.md uses additional_inputs[1]="confirm"/"deny"? [Conflict, contracts/modules.md vs api-ux.md]
CHK051 Does "backend/src/agent/app.py" plan to run a separate Gradio entrypoint, and does this file path conflict with the existing backend src structure (which uses __init__.py based packages)? [Ambiguity, plan.md project structure vs existing backend architecture]

11 KiB Raw Blame History