busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
754 Commits 3 Branches 0 Tags
147d7116574d64887828a5413ab92c8e909771f9
Commit Graph

10 Commits

Author SHA1 Message Date
busya
147d711657 feat(agent-chat): complete context guardrail event coverage 2026-07-05 14:14:42 +03:00
busya
ff60865183 feat(agent-chat): 035-agent-chat-context — контекст, guardrails, tools, database discovery 
== User stories ==
US1: Контекст с дашборда/датасета → /agent с URL params
US2: Guardrails card — env badge, 7 risk tones, countdown, permission_denied
US3: Tools optimization — retry, timeout, summarise, RBAC + context affinity

== Backend ==
- _context.py (NEW): UIContext validation (7 checks)
- _tool_filter.py (NEW): RBAC + context affinity pipeline
- _confirmation.py: build_confirmation_contract_v2, permission_denied_payload
- tools.py: superset_list_databases, retry/summarise/timeout wrappers
- app.py: _inject_uicontext, _inject_env_id_into_tools, database
  prefetch в runtime context
- _persistence.py: prefetch_databases()
- agent_superset_explore.py: GET /databases endpoint
- _llm_async_http.py, _persistence.py: fix double /v1 в LLM URL
  (LM Studio Unexpected endpoint)

== Frontend ==
- AgentChatModel.svelte.ts: 5 atoms, 3 actions, countdown, context
- AgentChat.svelte: production banner, process steps, debug panel
- ConfirmationCard.svelte: 7 risk tones, permission_denied, countdown
- ToolCallCard.svelte: retrying/timeout/cancelled states
- StreamProcessor.svelte.ts: tool_retry, timeout, permission_denied
- TopNavbar: sparkles icon + Ассистент
- sidebarNavigation: AI section
- DashboardHeader, datasets/+page: contextual AI buttons
- Icon: sparkles, brain, cpu icons
- tailwind: assistant category colors
- i18n: en/ru nav keys

== Tests ==
- 159 backend agent tests (+16 US3: retry, timeout, summarise, contracts)
- 2544 frontend tests (+11 model + component tests)
- 15 JSON fixtures (10 API + 5 model)

== Specs ==
- specs/035-agent-chat-context/: spec, UX, plan, tasks, research,
  data-model, contracts, quickstart, traceability, fixtures, checklists

Closes #035
 2026-07-04 22:47:17 +03:00
busya
ec9bc76a37 cleanup: убрать мёртвые env-переменные, консолидировать чтение в agent/_config.py 
Удалены из кода:
- JWT_SECRET — мёртвая (decode_token использует AUTH_SECRET_KEY)
- SESSION_SECRET_KEY — заменён на прямой AUTH_SECRET_KEY
- POSTGRES_URL — deprecated fallback, удалён из database.py и reencrypt.py

Консолидировано чтение env-переменных agent-модуля:
- Создан agent/_config.py — единый модуль для FASTAPI_URL,
  SERVICE_JWT, GRADIO_*, STORAGE_ROOT, AGENT_* (9 констант)
- Все agent/*.py импортируют из _config вместо разрозненных os.getenv

Удалены or-дефолты (безопасность):
- agent/langgraph_setup.py — удалён hardcoded DB URL postgres:postgres
- agent/langgraph_setup.py — удалены fallback API URL и model name
- scripts/reencrypt.py — удалён hardcoded DB URL postgres:postgres
- plugins/llm_analysis/service.py — удалены or-дефолты URL/app name

.env.example — минимализация:
- backend/.env.example: только 4 обязательные переменные
- root/.env.example: обязательные + docker + SSO/админ

Обновлены тесты (139 passed)
 2026-07-04 14:58:43 +03:00
busya
12118ac4ec fix(security): resolve Critical+High findings from module audit — agent, translate, superset_client 
P0 — CRITICAL (CWE-798): JWT_SECRET crash-early
  Replace hardcoded super-secret-key fallback with os.environ["JWT_SECRET"]
  and ${JWT_SECRET:?} syntax in app.py + docker-compose files

P1 — HIGH: Frontend dependency CVEs
  Upgrade svelte 5.43.8 → 5.56.4 — resolves devalue DoS (GHSA-g2pg-6438-jwpf)
  and svelte XSS (GHSA-crpf-4hrx-3jrp, GHSA-m56q-vw4c-c2cp, GHSA-rcqx-6q8c-2c42)

P2 — MEDIUM: Logging hygiene + contract gaps + tool resolver refactor
  Apply _redact_sensitive_fields() in middleware + event streaming
  Truncate LLM error body to 100 chars
  Add @RATIONALE/@REJECTED to HandleResume + SaveConversation
  Refactor deterministic intent matching → LLM-driven tool resolution

P3 — LOW: Translate logging hardening
  Move _sanitize_url() to _utils.py (shared, no circular imports)
  Sanitize base_url before logging in _llm_call.py and _llm_async_http.py
  Emit EXPLORE warning when LLM_SSL_VERIFY=false disables TLS

superset_client module: passed clean — no changes needed
 2026-07-01 13:17:29 +03:00
busya
79d0f8f678 feat(agent-superset): extend SupersetClient with agent-critical methods + DDL/DML guard + tests (126 passed) 
Ported from mcp-superset research module, integrated into existing async SupersetClient:

New mixins (4 files):
- safety.py: DDL/DML guard (13 keywords), comment/string stripping, viz_type validation
- _sql_lab.py: execute_sql, format_sql, results, estimate, CSV export, query history
- _saved_queries.py: saved queries CRUD (5 methods)
- _audit.py: permissions audit matrix (user × dashboard × dataset × RLS)

Extended mixins (4 files):
- _dashboards_write.py: +create, +update (general), +copy, +publish, +unpublish
- _dashboards_crud.py: +standalone get_dashboard_charts/datasets, +get_dashboard
- _datasets.py: +create, +delete, +duplicate, +refresh_schema, +get_or_create, +export/import
- _databases.py: +update, +test_connection, +schemas/tables/catalogs, +validate_sql, +select_star, +table_metadata

FastAPI proxy (2 files):
- agent_superset.py (284L): SQL Lab + Dashboards/Datasets write endpoints
- agent_superset_explore.py (240L): DB explore + Audit + Saved Queries endpoints

Agent tools (2 files):
- tools.py: +7 LangChain @tools (24 total), intent-routing keywords
- _tool_resolver.py: updated SAFE/GUARDED/FAST_CONFIRM classification sets

Tests (3 files, 126 tests):
- test_superset_safety.py (51): DDL/DML bypass/legitimate/safe scenarios
- test_superset_extended.py (42): all mixin methods with mock AsyncAPIClient
- test_superset_tools.py (33): agent tool registration, intent matching, @tool .ainvoke()
 2026-06-30 17:26:51 +03:00
busya
a0619ca049 fix(agent): unify logging API + add molecular-cot coverage to agent module 
Phase 1 — API unification:
- Replace direct log() from cot_logger with canonical logger.reason/reflect/explore
  across _confirmation.py, _persistence.py, _tool_resolver.py, app.py

Phase 2 — Gap filling:
- tools.py: add REASON/REFLECT/EXPLORE to 17 C3 tool functions (was 0 logs)
- app.py agent_handler (C4): add lifecycle REASON on entry, REFLECT on exit,
  EXPLORE on OutputParserException + general exception
- langgraph_setup.py create_agent (C4): add REASON with model/config_source,
  EXPLORE on env-var/InMemorySaver fallback, REFLECT on graph compilation
- _tool_resolver.py infer_tool_from_text (C3, 13 branches): REASON on inference
- _persistence.py: REFLECT on save_conversation success, EXPLORE on prefetch

Phase 3 — Plain log migration:
- middleware.py: plain logger.info() -> logger.reason()
- run.py: 7 plain logger.info/warning/error -> molecular REASON/EXPLORE

Phase 4 — Cleanup:
- cot_logger.py: deprecate MarkerLogger (@DEPRECATED + @REPLACED_BY)
- molecular-cot-logging SKILL.md: remove cot_span Section IV (never implemented),
  renumber sections V-VII -> IV-VI, add cot_span rejection rationale

Verification: pytest 27/27, axiom rebuild 5844/2993/0 warnings
 2026-06-30 15:48:46 +03:00
busya
e8d6d7d0db fix(agent): critical agent chat bugs — backend startup & frontend streaming state 
Backend (tools.py):
- Add Python docstrings to all 17 @tool functions (LangChain ValueError)
- Add @INVARIANT ADR: docstring requirement documented in module header
- Fix 2 f-string escaped-quote syntax errors (Python 3.13)

Frontend — compile errors (+page.svelte):
- Fix mismatched <button>/</Button> tags
- Fix missing Button import for mobile sidebar close

Frontend — streaming state loss on conversation switch (AgentChatModel):
- Add _commitStreamingPartial() helper — saves in-progress text before cancelling
- selectConversation() commits partial text to OLD conversation before switching
- createConversation() commits partial text before clearing state
- loadHistory() sets _userCancelled=true to suppress fallback messages

Frontend — ConnectionManager:
- Pass error reason through onDisconnectedPermanent callback → model.error

Frontend — null safety:
- Guard _client.submit() calls against null _client in _sendNow and resumeConfirm
 2026-06-30 13:25:28 +03:00
busya
12678c637b fix(agent-chat): streaming state leak, document parser magic bytes, HITL flow 
### Bugfixes — Agent Chat 'Думаю' State Leak
- fix(agent-chat): loadHistory() now resets streamingState/idle + cancels stale
  submission — prevents 'Думаю' state leak across conversation switches
- fix(agent-chat): onDisconnected/onDisconnectedPermanent cascade to
  streamingState — prevents permanent hang on connection loss during stream
- fix(agent-chat):  guard on isLoadingHistory — prevents false commit
  of 'agent unavailable' fallback when switching conversations
- fix(agent-chat): remove race in _sendNow empty-response check vs Svelte
   microtask (duplicate logic removed,  handles correctly)
- fix(stream-processor): confirm_resolved now appends msg.text to partialText
  instead of dropping it

### Bugfixes — Backend PDF Upload
- fix(document-parser): _detect_format_by_magic() — reads file header magic
  bytes as fallback when Gradio loses filename
- fix(document-parser): improved name extraction — tries orig_name, path stem
- fix(document-parser): @RELATION AgentChatTypes -> AgentChat.Types

### HITL Flow & Agent Chat Improvements
- feat(agent): HITL resume confirm/deny with userId/userJwt/envId propagation
- feat(agent): confirm_required metadata fallback via aget_state() after
  'Event loop is closed' error during interrupt
- feat(agent): interrupt_before re-enabled via AGENT_CONFIRM_TOOLS env var
- feat(frontend): debug panel with connection/stream state monitoring
- feat(frontend): AgentChatModel constructor options + onBeforeSend callback
- feat(frontend): crypto.randomUUID() for local conversation ID on first send

### Backend Agent Refactoring
- refactor(agent): langgraph_setup — monkey-patch for PydanticSerializationError
- refactor(agent): tools.py — dual identity headers, expanded tool set
- refactor(agent): run.py — _find_free_port, Gradio server port fallback
- refactor(agent): app.py — file size validation, message truncation, HITL path

### Frontend
- feat(dashboard-hub): DashboardHubModel with filters, pagination, git actions
- feat(ui): DateRangeFilter component
- feat(i18n): new dashboard keys; cache tooltips fix
- fix(i18n): full run tooltips — cache is NOT ignored

### Semantic Protocol
- chore(agents): update all agents with canonical format
- chore(skills): sync semantics-core, semantics-contracts, molecular-cot-logging

### Housekeeping
- chore: remove stale semantic reports (10 files, Jan 2026)
- chore: update 033-gradio-agent-chat specs, contracts, UX, tasks, tests
- chore: add .agents/ directory (mirrors .opencode/ agent layouts)
- chore: update run.sh with DEV_MODE, port management
 2026-06-29 17:15:25 +03:00
busya
1fd5e55db7 fix(agent): auto-fallback to free port on GRADIO_SERVER_PORT conflict 
The Gradio agent (run.py) crashed with OSError when port 7860 was
already occupied by a previous instance. Added _find_free_port() that
scans up to 100 ports from the configured GRADIO_SERVER_PORT and picks
the first available one, logging a warning on fallback.

Contract updates:
- AgentChat.Run: [C:3] [TYPE Module] (was C2/Function), added
  @RATIONALE, @REJECTED, @SIDE_EFFECT for port-finding logic
- AgentChat.GradioApp: added @RATIONALE, @REJECTED
- AgentChat.LangGraph.Setup: added @REJECTED, deduplicated @RELATION
- AgentChat.Tools: added @RATIONALE
 2026-06-10 16:37:02 +03:00
busya
f87ebf5d4b feat(agent): Gradio-powered LangGraph agent chat with streaming, tool calls, file upload, conversation persistence 
- Gradio 5.50.0 ChatInterface with type='messages' streaming
- LangGraph create_react_agent with InMemorySaver checkpointer
- 4 @tool functions: search_dashboards, get_health_summary, list_environments, get_task_status
- Structured ChatMessage metadata (7 discriminator types: stream_token, tool_start/end/error, confirm_required, confirm_resolved, error)
- HITL resume via second submit() with interrupt_before/Command
- Dual-identity RBAC: service JWT + user JWT for tool calls
- File upload (10 MB limit, pdfplumber/xlsx/JSON parser)
- Conversation persistence via POST /api/agent/conversations/save
- REST API: list, history, archive conversations; multi-tab gate; LLM config
- LLM provider selection via Admin -> LLM Settings (assistant_planner_provider)
- Svelte 5 AgentChatModel with stream event queue, dedup, stream_status watcher
- MarkdownRenderer using svelte-markdown with semantic Tailwind tokens
- ToolCallCard (3 states: executing/completed/failed)
- ConversationList with search, date grouping, infinite scroll
- ConnectionIndicator with Gradio health status
- /agent route with two-column layout
- Vite proxy /api/agent/gradio -> Gradio SSE
- Fixed: not_() SQLAlchemy operator, route collision with _admin_routes
- Fixed: conversation_id -> id normalization, .pyc cache staleness
- Fixed: event.data array parsing (Gradio returns [jsonStr, null])
- Requirements pinned: gradio==5.50.0, pydantic>=2.7,<=2.12.3
 2026-06-10 10:27:19 +03:00