feat(security): add encryption health inventory and key recovery wizard
Backend:
- GET /api/security/encryption/health — inventory of all stored encrypted
secrets (LLM providers, DB connections, profile Git tokens) with
decrypt attempt and structured broken/healthy status
- GET /api/security/encryption/fingerprint — non-secret key fingerprint
- POST /api/security/encryption/recover — bulk replacement of
undecryptable secrets with partial_success semantics
Frontend:
- KeyRecoveryModel.svelte.ts — state machine (idle→scanning→
healthy/needs_recovery→editing→saving→complete/partial_success/error)
- KeyRecoveryWizard.svelte — tabbed dialog with LLM/DB/Git sections,
security guidance, re-encrypt command display, edit/save flow
- SystemSettings entry point — 'Check encrypted secrets' card with
fingerprint and broken count
- API methods: getEncryptionHealth, recoverEncryptedSecrets
- Types: EncryptionRecoveryTypes
- i18n: en/ru strings for recovery flow
Tests: 226 passed
This commit is contained in:
@@ -1,245 +1,621 @@
|
||||
# Plan: unify Docker/.env variables and examples
|
||||
# Plan: key-change recovery UX for encrypted secrets
|
||||
|
||||
## Current problem
|
||||
## Problem
|
||||
|
||||
В проекте сейчас есть drift между compose-файлами и env-примерами:
|
||||
When operators change keys, two different user-facing failures can occur:
|
||||
|
||||
- `backend` использует `AUTH_SECRET_KEY` как канонический JWT signing secret (`src/core/auth/config.py`).
|
||||
- `agent` исторически использовал `JWT_SECRET`, а затем получил временный fallback на `AUTH_SECRET_KEY` в `src/agent/_jwt_decoder.py`; этот fallback нужно удалить.
|
||||
- `.env.example`, `.env.enterprise-clean.example`, `backend/.env.example`, `docker/.env.agent.example`, `docker-compose.yml`, `docker-compose.enterprise-clean.yml` и generated compose внутри `build.sh` описывают разные наборы переменных.
|
||||
- `docker/.env.agent.example` устарел: там `AUTH_SECRET_KEY`, но compose раньше прокидывал `JWT_SECRET`; также не перечислены agent tuning переменные.
|
||||
- `docker-compose.enterprise-clean.yml` и `build.sh` generated compose должны быть синхронны, иначе локальная сборка и bundle дают разное runtime-поведение.
|
||||
- Локальные `.env.current` / `.env.master` содержат только порты/admin/features и не показывают auth/LLM/certs/security переменные.
|
||||
1. `AUTH_SECRET_KEY` changed
|
||||
- Existing JWT tokens become invalid.
|
||||
- Recovery is re-login / clear stale frontend auth token.
|
||||
- No encrypted business data is lost.
|
||||
|
||||
## Canonical naming decision
|
||||
2. `ENCRYPTION_KEY` changed
|
||||
- Persisted encrypted secrets cannot be decrypted with the new key.
|
||||
- Affected secrets include:
|
||||
- LLM provider API keys (`backend/src/services/llm_provider.py`)
|
||||
- database/environment connection passwords (`backend/src/core/connection_service.py`)
|
||||
- Git personal access tokens in profile preferences (`backend/src/services/profile_preference_service.py`, git route helpers)
|
||||
- Recovery requires either:
|
||||
- restoring old key;
|
||||
- running `src.scripts.reencrypt` with old + new keys;
|
||||
- or re-entering all secrets manually if old key is unavailable.
|
||||
|
||||
Use `AUTH_SECRET_KEY` as the canonical token-signing secret across backend and agent.
|
||||
The requested frontend dialog should cover the manual recovery path: show exactly which secrets are invalid after an `ENCRYPTION_KEY` change and guide the admin to re-enter fresh values.
|
||||
|
||||
Rationale:
|
||||
## Existing code observations
|
||||
|
||||
- Backend already requires `AUTH_SECRET_KEY` through `AuthConfig`.
|
||||
- Root/backend examples already document `AUTH_SECRET_KEY`.
|
||||
- `JWT_SECRET` is ambiguous: it can mean backend auth token signing, service token signing, or agent token validation.
|
||||
- No fallback: `JWT_SECRET` must be removed from Docker/runtime examples and from agent JWT decoding logic. Deployments must define `AUTH_SECRET_KEY` only.
|
||||
- LLM provider UI exists at `frontend/src/routes/admin/settings/llm/+page.svelte` and uses `frontend/src/lib/components/llm/ProviderConfig.svelte`.
|
||||
- DB connection UI exists in `frontend/src/routes/settings/ConnectionsTab.svelte`.
|
||||
- Existing migration password modal is `frontend/src/lib/components/migration/PasswordPrompt.svelte`; it can inspire password-entry UX but should not be reused directly because key recovery spans multiple secret domains.
|
||||
- API wrapper `frontend/src/lib/api.ts` already normalizes errors with `status`, `detail`, `error_code`.
|
||||
- Backend already surfaces LLM decrypt failure in `backend/src/api/routes/llm.py` with message: provider may have been encrypted with a different encryption key and must be updated with a new API key.
|
||||
- Backend connection decryption raises clear `RuntimeError` messages in `ConnectionService._decrypt_password`, but there is no central endpoint that inventories all broken secrets.
|
||||
|
||||
Canonical variables:
|
||||
## LLM provider key storage audit
|
||||
|
||||
- `AUTH_SECRET_KEY`: canonical shared user JWT signing key, backend + agent.
|
||||
- `JWT_SECRET`: removed/deprecated hard error — do not use in compose, examples, or agent runtime.
|
||||
- `SERVICE_JWT`: service-to-backend token for agent calls; separate from user JWT signing key.
|
||||
- `ENCRYPTION_KEY`: Fernet key for encrypted connection credentials/API keys.
|
||||
Current flow for all supported providers (`openai`, `openrouter`, `kilo`, `litellm`) is provider-type agnostic: every provider stores exactly one encrypted `api_key` in `llm_providers.api_key`.
|
||||
|
||||
## Files to update
|
||||
Backend storage:
|
||||
|
||||
### 1. `docker-compose.yml`
|
||||
- Model: `backend/src/models/llm.py`
|
||||
- `LLMProvider.provider_type`: string (`openai`, `openrouter`, `kilo`, `litellm`).
|
||||
- `LLMProvider.api_key`: `String(nullable=False)`, intended to contain encrypted ciphertext.
|
||||
- Pydantic input: `backend/src/plugins/llm_analysis/models.py`
|
||||
- `LLMProviderConfig.api_key: str | None = None`.
|
||||
- Create path: `backend/src/services/llm_provider.py`
|
||||
- `create_provider()` always does `self.encryption.encrypt(config.api_key)`.
|
||||
- Since `api_key` can be `None` in schema but `EncryptionManager.encrypt()` expects `str`, create requires a real key in practice, but this is not explicitly validated before encryption.
|
||||
- Update path: `backend/src/services/llm_provider.py`
|
||||
- `update_provider()` only re-encrypts when `api_key` is not masked/placeholder.
|
||||
- `is_masked_or_placeholder()` treats `None`, empty string, `********`, and strings containing `...` as display-only, so existing encrypted value is preserved.
|
||||
- List/read path: `backend/src/api/routes/llm.py`
|
||||
- `GET /api/llm/providers` decrypts each provider key and returns `mask_api_key(plaintext)`.
|
||||
- If decryption fails, `get_decrypted_api_key()` returns `None`, and UI receives `api_key: ""`; this loses the distinction between “no key” and “key encrypted with old ENCRYPTION_KEY”.
|
||||
- Test path: `POST /api/llm/providers/{id}/test`
|
||||
- Decrypts stored key.
|
||||
- If decrypt returns `None`, returns HTTP 500 with text detail asking user to update the provider with a new API key.
|
||||
- It does not currently emit machine-readable `error_code=ENCRYPTION_KEY_MISMATCH`.
|
||||
- Fetch models path: `POST /api/llm/providers/fetch-models`
|
||||
- If `api_key` is not provided and `provider_id` is present, attempts to decrypt stored key.
|
||||
- If decrypt fails, it currently continues with `api_key or "sk-placeholder"`, which can produce a confusing provider/API error instead of a clear encryption-key mismatch.
|
||||
|
||||
Unify local dev compose with canonical variable names:
|
||||
Frontend behavior:
|
||||
|
||||
- DB service:
|
||||
- Use `${POSTGRES_DB:-ss_tools}`, `${POSTGRES_USER:-postgres}`, `${POSTGRES_PASSWORD:-postgres}` instead of hardcoded values.
|
||||
- Healthcheck should use the same values.
|
||||
- Backend service:
|
||||
- Construct `DATABASE_URL`, `TASKS_DATABASE_URL`, `AUTH_DATABASE_URL` from the same POSTGRES variables.
|
||||
- Pass `AUTH_SECRET_KEY`, `ENCRYPTION_KEY`, `JWT_AUDIENCE`, `JWT_ISSUER`, `ALLOWED_ORIGINS`, `FORCE_HTTPS`, `APP_TIMEZONE` where relevant.
|
||||
- Pass `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `LLM_CA_CERT_URLS`, `LLM_SSL_VERIFY` consistently.
|
||||
- Agent service:
|
||||
- Pass `AUTH_SECRET_KEY` only.
|
||||
- Remove `JWT_SECRET` entirely; do not include compatibility aliases.
|
||||
- Keep `SERVICE_JWT`, `DATABASE_URL`, `FASTAPI_URL`, `LLM_*`, `GRADIO_*`, `AGENT_*` variables.
|
||||
- Frontend service:
|
||||
- Add `SSL_KEY_PASSPHRASE` if local frontend can terminate SSL similarly to enterprise.
|
||||
- `ProviderConfig.svelte` supports `openai`, `openrouter`, `kilo`, `litellm` from one form.
|
||||
- On edit, backend returns a masked key such as `sk-...abcd`; UI shows it in a non-input code block with a `Change` button.
|
||||
- On submit, if editing and `api_key` is empty or masked, the frontend deletes `api_key` from payload so backend keeps the existing encrypted key.
|
||||
- If user clicks `Change` and enters a new key, frontend sends the plaintext key and backend encrypts it with current `ENCRYPTION_KEY`.
|
||||
- Current gap: if backend cannot decrypt due to changed `ENCRYPTION_KEY`, the list endpoint returns an empty `api_key`, so UI may show a blank password input rather than an explicit “stored key cannot be decrypted” recovery state.
|
||||
|
||||
### 2. `docker-compose.enterprise-clean.yml`
|
||||
Implications for recovery wizard:
|
||||
|
||||
Make enterprise compose match the canonical env contract:
|
||||
- Health endpoint must inspect all `LLMProvider` rows regardless of `provider_type`.
|
||||
- For each provider with non-empty `api_key`, try decrypt.
|
||||
- Return `type=llm_provider`, `id`, `label=name`, `metadata={provider_type, base_url, default_model, is_active}`.
|
||||
- If decrypt fails, classify as `reason=decrypt_failed` and `requires=["api_key"]`.
|
||||
- Recover endpoint can call the same update/provider service path with a new plaintext `api_key` and preserve all other provider fields.
|
||||
- LLM provider list endpoint should not silently collapse decrypt failure to empty key; it should include a safe metadata flag in future, e.g. `api_key_status: "masked" | "missing" | "decrypt_failed"`, or rely on the separate health endpoint.
|
||||
|
||||
- Backend:
|
||||
- Keep external Postgres URL composition from POSTGRES variables.
|
||||
- Add/pass `AUTH_SECRET_KEY` explicitly.
|
||||
- Keep required `ENCRYPTION_KEY`.
|
||||
- Add optional documented variables that backend reads:
|
||||
- `JWT_AUDIENCE`, `JWT_ISSUER`
|
||||
- `ALLOWED_ORIGINS`, `FORCE_HTTPS`
|
||||
- `APP_TIMEZONE`
|
||||
- `LLM_SSL_VERIFY`, `LLM_CA_CERT_URLS`
|
||||
- `OPENROUTER_SITE_URL`, `OPENROUTER_APP_NAME`
|
||||
- Agent:
|
||||
- Use `AUTH_SECRET_KEY: ${AUTH_SECRET_KEY:?AUTH_SECRET_KEY must be set}`.
|
||||
- Remove `JWT_SECRET` entirely; no fallback/alias support.
|
||||
- Keep `DATABASE_URL` for LangGraph checkpoint persistence.
|
||||
- Include all agent optional knobs in comments/examples, but compose can omit defaults if code defaults are sufficient.
|
||||
- Frontend:
|
||||
- Keep `SSL_KEY_PASSPHRASE`, `CERTS_PATH` mounts, host ports.
|
||||
## UX goal
|
||||
|
||||
### 3. `docker-compose.e2e.yml`
|
||||
Add an admin-facing recovery dialog / wizard that appears when encrypted secrets are no longer decryptable after `ENCRYPTION_KEY` changed.
|
||||
|
||||
Keep test isolation but align naming:
|
||||
It should answer:
|
||||
|
||||
- Use `AUTH_SECRET_KEY`, `ENCRYPTION_KEY`, `DATABASE_URL`, `AUTH_DATABASE_URL`, `TASKS_DATABASE_URL` consistently.
|
||||
- Add `SERVICE_JWT` if e2e agent/service paths are enabled later.
|
||||
- Add a matching `.env.e2e.example` so CI/local users do not infer variables from comments.
|
||||
- What happened?
|
||||
- Which secrets need re-entry?
|
||||
- What is safe to do now?
|
||||
- Which secrets were already fixed?
|
||||
- Which actions require old `ENCRYPTION_KEY` vs manual re-entry?
|
||||
|
||||
### 4. `build.sh`
|
||||
## Proposed UX: `Key Recovery Wizard`
|
||||
|
||||
Generated compose files must match source compose:
|
||||
### Entry points
|
||||
|
||||
- Release `bundle` generated `docker-compose.enterprise-clean.yml`:
|
||||
- Replace `JWT_SECRET` with canonical `AUTH_SECRET_KEY` for agent and backend.
|
||||
- Add/pass backend `AUTH_SECRET_KEY` if missing.
|
||||
- Ensure generated env variable names match `.env.enterprise-clean.example` exactly.
|
||||
- `bundle:embeddings` generated compose:
|
||||
- Apply the same env contract as default bundle.
|
||||
- `bundle:light` generated compose/env:
|
||||
- Ensure backend all-in-one receives `AUTH_SECRET_KEY`, `ENCRYPTION_KEY`, DB URLs, admin vars, LLM vars.
|
||||
- Help text:
|
||||
- State that `AUTH_SECRET_KEY` is required and `JWT_SECRET` is unsupported.
|
||||
1. Automatic trigger
|
||||
- Any API error with `error_code = ENCRYPTION_KEY_MISMATCH` or compatible message opens a global recovery banner/dialog for admins.
|
||||
- Non-admin users see a simpler message: “Credentials need administrator attention.”
|
||||
|
||||
### 5. `.env.example`
|
||||
2. Manual entry
|
||||
- Add a button/card in Settings → System or Admin → Settings:
|
||||
- “Check encrypted secrets”
|
||||
- “Recover after ENCRYPTION_KEY change”
|
||||
|
||||
Turn this into the canonical local/dev template with all Docker-relevant variables grouped consistently:
|
||||
3. Contextual entry
|
||||
- LLM provider test failure offers “Update API key now”.
|
||||
- Connection test failure offers “Update password now”.
|
||||
- Git operation token decrypt failure offers “Update Git token in profile”.
|
||||
|
||||
Sections:
|
||||
### Wizard states
|
||||
|
||||
1. Project/compose
|
||||
- `COMPOSE_PROJECT_NAME`
|
||||
2. Postgres
|
||||
- `POSTGRES_IMAGE`, `POSTGRES_HOST`, `POSTGRES_PORT`, `POSTGRES_HOST_PORT`, `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`
|
||||
3. Database URLs
|
||||
- `DATABASE_URL`, `TASKS_DATABASE_URL`, `AUTH_DATABASE_URL`
|
||||
- Note: compose may derive URLs, direct backend local run uses URLs.
|
||||
4. Security
|
||||
- `AUTH_SECRET_KEY`, `ENCRYPTION_KEY`, `JWT_AUDIENCE`, `JWT_ISSUER`.
|
||||
- Add an explicit comment: `JWT_SECRET` is unsupported; migrate to `AUTH_SECRET_KEY`.
|
||||
5. Ports
|
||||
- `BACKEND_HOST_PORT`, `FRONTEND_HOST_PORT`, `FRONTEND_SSL_PORT`, `AGENT_HOST_PORT`
|
||||
6. Admin bootstrap
|
||||
- `INITIAL_ADMIN_CREATE`, `INITIAL_ADMIN_USERNAME`, `INITIAL_ADMIN_PASSWORD`, `INITIAL_ADMIN_EMAIL`
|
||||
7. LLM/provider
|
||||
- `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL`, `LLM_SSL_VERIFY`, `LLM_CA_CERT_URLS`
|
||||
8. Agent
|
||||
- `SERVICE_JWT`, `GRADIO_SERVER_PORT`, `GRADIO_ALLOW_PORT_FALLBACK`, `AGENT_ENABLE_LLM_TITLES`, `AGENT_TITLE_GENERATION_TIMEOUT_S`, `AGENT_PREFETCH_DASHBOARD_LIMIT`, `AGENT_CONFIRM_TOOLS`, `AGENT_INTERRUPT_BEFORE`, `EMBEDDING_*`
|
||||
9. Certificates/frontend
|
||||
- `CERTS_PATH`, `SSL_KEY_PASSPHRASE`
|
||||
10. Runtime behavior
|
||||
- `ENABLE_BELIEF_STATE_LOGGING`, `TASK_LOG_LEVEL`, `APP_TIMEZONE`, `ALLOWED_ORIGINS`, `FORCE_HTTPS`
|
||||
11. Features
|
||||
- `FEATURES__DATASET_REVIEW`, `FEATURES__HEALTH_MONITOR`
|
||||
12. Optional integrations
|
||||
- `ADFS_*`, `OPENROUTER_*`, `GITEA_TOKEN` only where relevant.
|
||||
Use a model-first Svelte 5 state machine in `frontend/src/lib/models/KeyRecoveryModel.svelte.ts`.
|
||||
|
||||
### 6. `.env.enterprise-clean.example`
|
||||
States:
|
||||
|
||||
Mirror `.env.example` but with production/external-Postgres defaults and enterprise comments:
|
||||
- `idle` — not loaded.
|
||||
- `scanning` — calls backend inventory endpoint.
|
||||
- `healthy` — no broken encrypted secrets found.
|
||||
- `needs_recovery` — one or more broken secrets found.
|
||||
- `editing` — admin is entering replacement values.
|
||||
- `saving` — replacement secrets are being submitted.
|
||||
- `partial_success` — some secrets fixed, some still failed.
|
||||
- `complete` — all known broken secrets fixed.
|
||||
- `error` — inventory or save failed.
|
||||
|
||||
- Include every variable used by `docker-compose.enterprise-clean.yml` and generated bundle compose.
|
||||
- Use placeholder secrets, not real values.
|
||||
- Add explicit generation commands for `AUTH_SECRET_KEY`, `ENCRYPTION_KEY`, and `SERVICE_JWT`.
|
||||
- Include `AGENT_HOST_PORT` (currently missing from example but used by compose).
|
||||
- Include `LLM_SSL_VERIFY` (currently used by compose, missing from example).
|
||||
- Include `AUTH_SECRET_KEY` (currently required by backend/agent, missing from enterprise example).
|
||||
- Include `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL` (compose/build uses them, example incomplete).
|
||||
### Dialog layout
|
||||
|
||||
### 7. `backend/.env.example`
|
||||
Component: `frontend/src/lib/components/security/KeyRecoveryWizard.svelte`
|
||||
|
||||
Keep backend-focused, but synchronize with canonical names:
|
||||
Use `$lib/ui` atoms: `Card`, `Button`, `Input`, `Badge`, `EmptyState`, `ConfirmDialog`.
|
||||
|
||||
- Include backend-only/source-run variables:
|
||||
- `DATABASE_URL`, `TASKS_DATABASE_URL`, `AUTH_DATABASE_URL`
|
||||
- `AUTH_SECRET_KEY`, `ENCRYPTION_KEY`, `JWT_AUDIENCE`, `JWT_ISSUER`
|
||||
- `INITIAL_ADMIN_*`
|
||||
- `LLM_SSL_VERIFY`, `LLM_CA_CERT_URLS`
|
||||
- `ALLOWED_ORIGINS`, `FORCE_HTTPS`, `APP_TIMEZONE`
|
||||
- `ADFS_*`, `OPENROUTER_*`
|
||||
- Do not include frontend/agent-only host port variables except where helpful comments point users to root `.env.example`.
|
||||
Header:
|
||||
|
||||
### 8. `docker/.env.agent.example`
|
||||
- Title: “Encrypted credentials need re-entry”
|
||||
- Severity: warning/destructive depending on failure count.
|
||||
- Explanation:
|
||||
- “The server can no longer decrypt some saved secrets with the current ENCRYPTION_KEY. This usually happens after changing ENCRYPTION_KEY without running key rotation.”
|
||||
|
||||
Make it agent-only and current:
|
||||
Recovery options panel:
|
||||
|
||||
- Replace `AUTH_SECRET_KEY=<shared-with-backend-secret>` with canonical `AUTH_SECRET_KEY=...` and note it must match backend.
|
||||
- Remove stale implication that `AUTH_DATABASE_URL` is needed by agent.
|
||||
- Include:
|
||||
- `AUTH_SECRET_KEY`
|
||||
- `SERVICE_JWT`
|
||||
- `FASTAPI_URL`
|
||||
- `DATABASE_URL`
|
||||
- `LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL`
|
||||
- `GRADIO_SERVER_NAME`, `GRADIO_SERVER_PORT`, `GRADIO_ALLOW_PORT_FALLBACK`
|
||||
- `AGENT_ENABLE_LLM_TITLES`, `AGENT_TITLE_GENERATION_TIMEOUT_S`, `AGENT_PREFETCH_DASHBOARD_LIMIT`, `AGENT_CONFIRM_TOOLS`, `AGENT_INTERRUPT_BEFORE`
|
||||
- `EMBEDDING_SIMILARITY_THRESHOLD`, `EMBEDDING_TOP_K`
|
||||
- Do not include `JWT_SECRET`; add a one-line warning comment that old `JWT_SECRET` deployments must rename it to `AUTH_SECRET_KEY`.
|
||||
1. Recommended if old key exists:
|
||||
- “Restore old key or run re-encryption tool”
|
||||
- Shows command snippet:
|
||||
- `OLD_ENCRYPTION_KEY=<old> NEW_ENCRYPTION_KEY=<new> python -m src.scripts.reencrypt --dry-run`
|
||||
- Not executable from frontend; informational only.
|
||||
|
||||
### 9. New example files
|
||||
2. Manual re-entry:
|
||||
- “I do not have old key — re-enter affected secrets now”
|
||||
- Opens forms grouped by secret type.
|
||||
|
||||
Add missing templates so every operational env file has an example:
|
||||
Inventory sections:
|
||||
|
||||
- `.env.current.example`
|
||||
- local current-branch port offsets and admin demo values.
|
||||
- `.env.master.example`
|
||||
- local master-branch port offsets.
|
||||
- `.env.e2e.example`
|
||||
- e2e stack ports and test credentials.
|
||||
- `frontend/.env.example`
|
||||
- frontend/e2e visible variables if any are used; otherwise a short file saying frontend runtime config is generated/proxied via nginx and API calls.
|
||||
- LLM Providers
|
||||
- rows: provider name, base URL, model, status, API key input.
|
||||
- action: save key, test provider.
|
||||
- Database Connections / Environments
|
||||
- rows: connection name or environment/database name, host, database, username, password input.
|
||||
- action: save password, test connection.
|
||||
- Git tokens
|
||||
- rows: user/profile identity, token status, token input.
|
||||
- action: save token / go to profile.
|
||||
- Other encrypted app configurations
|
||||
- fallback list for any backend-reported unknown secret type.
|
||||
|
||||
Do not modify real secret-bearing files:
|
||||
Footer:
|
||||
|
||||
- `.env.enterprise-clean`
|
||||
- `backend/.env`
|
||||
- `frontend/.env`
|
||||
- `frontend/.env.bak`
|
||||
- `frontend/e2e/.env.e2e`
|
||||
- “Rescan”
|
||||
- “Save entered secrets”
|
||||
- “Close”
|
||||
- “Open deployment runbook” if docs route exists.
|
||||
|
||||
### 10. Source code canonicalization
|
||||
### UI/UX pseudographics
|
||||
|
||||
Update `backend/src/agent/_jwt_decoder.py` to enforce the final canonical contract:
|
||||
#### 1. Global banner after decrypt failure
|
||||
|
||||
- `AUTH_SECRET_KEY` is canonical.
|
||||
- `JWT_SECRET` is not read and not accepted.
|
||||
- If `AUTH_SECRET_KEY` is missing, fail fast with a clear `JWTError` / startup-visible error.
|
||||
- If `JWT_SECRET` is present but `AUTH_SECRET_KEY` is absent, do not silently accept it; error message should instruct operator to rename `JWT_SECRET` to `AUTH_SECRET_KEY`.
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ ⚠ Saved credentials cannot be decrypted with current ENCRYPTION_KEY │
|
||||
│ LLM keys: 1 broken DB passwords: 2 broken Git tokens: 1 needs users │
|
||||
│ │
|
||||
│ [Open recovery wizard] [Read rotation guide] [Dismiss for now] │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Behavior change required: remove the current dual-key fallback (`JWT_SECRET` first, then `AUTH_SECRET_KEY`). Decode backend-issued tokens using only `AUTH_SECRET_KEY`.
|
||||
Behavior:
|
||||
|
||||
- Admin sees action buttons.
|
||||
- Non-admin sees read-only version: “Contact administrator.”
|
||||
- Banner appears from any `ENCRYPTION_KEY_MISMATCH` API error or manual health scan.
|
||||
|
||||
#### 2. Settings card entry point
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ Encryption key recovery │
|
||||
│ Current key fingerprint: sha256:9f14a2c8 │
|
||||
│ Status: ⚠ 4 saved credentials need attention │
|
||||
│ │
|
||||
│ Changing AUTH_SECRET_KEY logs users out. │
|
||||
│ Changing ENCRYPTION_KEY requires re-entering │
|
||||
│ stored secrets or running re-encryption. │
|
||||
│ │
|
||||
│ [Check encrypted secrets] [Open recovery] │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 3. Wizard step 1 — explain and choose path
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Encrypted credentials need re-entry [×] │
|
||||
├──────────────────────────────────────────────────────────────────────────────┤
|
||||
│ The server cannot decrypt some saved secrets with the current ENCRYPTION_KEY. │
|
||||
│ This usually happens when ENCRYPTION_KEY was changed without re-encryption. │
|
||||
│ │
|
||||
│ Important distinction: │
|
||||
│ • AUTH_SECRET_KEY change → users log in again; no stored secrets are lost. │
|
||||
│ • ENCRYPTION_KEY change → stored passwords/API keys must be re-encrypted or │
|
||||
│ re-entered manually. │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Recommended if old key is available │ │
|
||||
│ │ Run the backend re-encryption tool before manual edits. │ │
|
||||
│ │ │ │
|
||||
│ │ OLD_ENCRYPTION_KEY=<old> NEW_ENCRYPTION_KEY=<new> \ │ │
|
||||
│ │ python -m src.scripts.reencrypt --dry-run │ │
|
||||
│ │ │ │
|
||||
│ │ [Copy command] [Open runbook] │ │
|
||||
│ └──────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Old key is unavailable │ │
|
||||
│ │ Re-enter the affected secrets below. Old encrypted values cannot be │ │
|
||||
│ │ recovered cryptographically. │ │
|
||||
│ │ │ │
|
||||
│ │ [Continue to manual recovery] │ │
|
||||
│ └──────────────────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 4. Wizard step 2 — inventory overview
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Recovery checklist [×] │
|
||||
├──────────────────────────────────────────────────────────────────────────────┤
|
||||
│ Key fingerprint: sha256:9f14a2c8 [Rescan] │
|
||||
│ │
|
||||
│ Summary │
|
||||
│ ┌───────────────┬──────────┬────────────┬────────────┐ │
|
||||
│ │ Secret type │ Total │ Broken │ Fixed now │ │
|
||||
│ ├───────────────┼──────────┼────────────┼────────────┤ │
|
||||
│ │ LLM providers │ 2 │ 1 │ 0 │ │
|
||||
│ │ DB passwords │ 4 │ 2 │ 0 │ │
|
||||
│ │ Git tokens │ 3 │ 1 │ User action│ │
|
||||
│ └───────────────┴──────────┴────────────┴────────────┘ │
|
||||
│ │
|
||||
│ [LLM Providers: 1] [Database Connections: 2] [Git tokens: 1] │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 5. Wizard step 3 — LLM providers
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ LLM provider API keys │
|
||||
├──────────────────────────────────────────────────────────────────────────────┤
|
||||
│ OpenAI production ⚠ cannot decrypt │
|
||||
│ Base URL: https://api.openai.com/v1 │
|
||||
│ Model: gpt-4o │
|
||||
│ │
|
||||
│ New API key │
|
||||
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ sk-•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• │ │
|
||||
│ └──────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ [Test] [Save this key] │
|
||||
│ │
|
||||
│ Kilo gateway ✓ healthy │
|
||||
│ No action required. │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 6. Wizard step 4 — database passwords
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Database / environment passwords │
|
||||
├──────────────────────────────────────────────────────────────────────────────┤
|
||||
│ DWH prod ⚠ cannot decrypt │
|
||||
│ host: rgm-s-dwhpg01.hq.root.ad db: ss_tools user: ss_tools_user │
|
||||
│ │
|
||||
│ New password │
|
||||
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ ••••••••••••••••••••••••••• │ │
|
||||
│ └──────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ [Test connection] [Save this password] │
|
||||
│ │
|
||||
│ ClickHouse analytics ⚠ cannot decrypt │
|
||||
│ host: ch-prod.company.local db: analytics user: readonly │
|
||||
│ │
|
||||
│ New password │
|
||||
│ ┌──────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ │ │
|
||||
│ └──────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ [Test connection] [Save this password] │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 7. Wizard step 5 — Git token policy
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Git personal access tokens │
|
||||
├──────────────────────────────────────────────────────────────────────────────┤
|
||||
│ 1 user token cannot be decrypted. │
|
||||
│ │
|
||||
│ For privacy and audit safety, admins do not overwrite personal Git tokens. │
|
||||
│ Each affected user must open Profile → Git token and enter a new PAT. │
|
||||
│ │
|
||||
│ Affected users │
|
||||
│ ┌──────────────────────────────┬─────────────────────────────┬────────────┐ │
|
||||
│ │ User │ Status │ Action │ │
|
||||
│ ├──────────────────────────────┼─────────────────────────────┼────────────┤ │
|
||||
│ │ ivan.petrov │ ⚠ token needs re-entry │ Notify │ │
|
||||
│ └──────────────────────────────┴─────────────────────────────┴────────────┘ │
|
||||
│ │
|
||||
│ [Copy user instruction] │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 8. Save confirmation
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ Overwrite saved secrets? [×]│
|
||||
├──────────────────────────────────────────────┤
|
||||
│ This will replace encrypted values for: │
|
||||
│ • 1 LLM provider API key │
|
||||
│ • 2 database passwords │
|
||||
│ │
|
||||
│ Old encrypted values will not be shown again. │
|
||||
│ │
|
||||
│ [Cancel] [Save replacements] │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 9. Partial success result
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Recovery result │
|
||||
├──────────────────────────────────────────────────────────────────────────────┤
|
||||
│ ✓ OpenAI production API key saved and tested │
|
||||
│ ✓ DWH prod password saved and tested │
|
||||
│ ⚠ ClickHouse analytics password saved, but connection test failed │
|
||||
│ Reason: authentication failed for user readonly │
|
||||
│ │
|
||||
│ Remaining actions │
|
||||
│ • Re-enter ClickHouse analytics password │
|
||||
│ • Ask affected users to refresh Git PAT in Profile │
|
||||
│ │
|
||||
│ [Back to edit] [Rescan] [Close] │
|
||||
└──────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
#### 10. Healthy state
|
||||
|
||||
```text
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ ✓ Encrypted credentials are healthy │
|
||||
├──────────────────────────────────────────────┤
|
||||
│ All known saved secrets can be decrypted with │
|
||||
│ the current ENCRYPTION_KEY. │
|
||||
│ │
|
||||
│ Key fingerprint: sha256:9f14a2c8 │
|
||||
│ Last scan: 2026-07-06 17:48 │
|
||||
│ │
|
||||
│ [Close] [Scan again] │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Copy / messaging
|
||||
|
||||
For admins:
|
||||
|
||||
- “Changing `AUTH_SECRET_KEY` only logs users out. Changing `ENCRYPTION_KEY` affects stored passwords and API keys.”
|
||||
- “If you still have the old ENCRYPTION_KEY, do not re-enter secrets manually yet; use the re-encryption tool to preserve all values.”
|
||||
- “If old key is unavailable, the app cannot recover old encrypted values. Re-enter the affected secrets below.”
|
||||
|
||||
For non-admins:
|
||||
|
||||
- “Saved credentials are temporarily unavailable. Contact an administrator to update encrypted secrets.”
|
||||
|
||||
## Backend support needed
|
||||
|
||||
Add explicit machine-readable encryption-health endpoints. Without this, frontend must infer from scattered 500 messages, which is fragile.
|
||||
|
||||
### Endpoint 1: inventory
|
||||
|
||||
`GET /api/security/encryption/health`
|
||||
|
||||
Response shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "healthy" | "needs_recovery",
|
||||
"key_fingerprint": "sha256:abcd1234",
|
||||
"summary": {
|
||||
"llm_providers_total": 2,
|
||||
"llm_providers_broken": 1,
|
||||
"connections_total": 4,
|
||||
"connections_broken": 2,
|
||||
"profile_tokens_total": 3,
|
||||
"profile_tokens_broken": 1
|
||||
},
|
||||
"items": [
|
||||
{
|
||||
"id": "provider-1",
|
||||
"type": "llm_provider",
|
||||
"label": "OpenAI production",
|
||||
"location": "Admin / LLM Providers",
|
||||
"status": "broken",
|
||||
"reason": "decrypt_failed",
|
||||
"requires": ["api_key"],
|
||||
"metadata": { "base_url": "https://api.openai.com/v1", "default_model": "gpt-4o" }
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Must not return decrypted values.
|
||||
- Must not log secret values.
|
||||
- Should use safe decrypt attempts and classify `decrypt_failed` vs `missing_secret`.
|
||||
- Should require admin permission.
|
||||
|
||||
### Endpoint 2: bulk replacement
|
||||
|
||||
`POST /api/security/encryption/recover`
|
||||
|
||||
Request shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{ "id": "provider-1", "type": "llm_provider", "values": { "api_key": "sk-..." } },
|
||||
{ "id": "conn-1", "type": "database_connection", "values": { "password": "..." } },
|
||||
{ "id": "profile:user-1", "type": "profile_git_token", "values": { "token": "..." } }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "partial_success" | "complete",
|
||||
"updated": ["provider-1"],
|
||||
"failed": [
|
||||
{ "id": "conn-1", "type": "database_connection", "reason": "validation_failed" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Validate inputs by type.
|
||||
- Encrypt with current `ENCRYPTION_KEY`.
|
||||
- Preserve unrelated fields.
|
||||
- Commit per item or transactional batch; prefer per item with explicit partial result so one bad secret does not block all recovery.
|
||||
- Require admin permission.
|
||||
|
||||
### Endpoint 3: optional key fingerprint
|
||||
|
||||
`GET /api/security/encryption/fingerprint`
|
||||
|
||||
Returns non-secret fingerprint for operator correlation:
|
||||
|
||||
```json
|
||||
{ "fingerprint": "sha256:abcd1234" }
|
||||
```
|
||||
|
||||
Could be merged into health endpoint.
|
||||
|
||||
## Backend changes by secret type
|
||||
|
||||
### LLM providers
|
||||
|
||||
- Add inventory logic over `LLMProvider` rows with non-empty `api_key`.
|
||||
- Attempt decrypt via existing `LLMProviderService.get_decrypted_api_key` or lower-level decrypt helper that returns structured error instead of only `None`.
|
||||
- Add replacement path that calls existing provider update flow with new `api_key`.
|
||||
|
||||
### Database connections / environments
|
||||
|
||||
- Inventory `ConfigManager.config.settings.connections` and any environment password fields.
|
||||
- Detect broken Fernet-looking values using `is_fernet_token` and decrypt attempt.
|
||||
- Replacement path updates password fields using existing `ConnectionService.update_connection` so encryption remains centralized.
|
||||
|
||||
### Git profile tokens
|
||||
|
||||
- Inventory preferences with `git_personal_access_token_encrypted`.
|
||||
- Admin recovery wizard should NOT let admins overwrite personal Git PATs for other users.
|
||||
- Admin inventory may show aggregate/count and impacted user identity only if existing permissions already allow it, but action must route users to Profile.
|
||||
- Per-current-user profile recovery should reuse `ProfilePreferenceService` to encrypt and persist the new token.
|
||||
|
||||
## Frontend implementation plan
|
||||
|
||||
1. Add types
|
||||
- `frontend/src/types/encryptionRecovery.ts`
|
||||
- Types: `EncryptedSecretType`, `EncryptionHealthResponse`, `RecoveryItem`, `RecoverySubmitPayload`, `RecoveryResult`.
|
||||
|
||||
2. Add API methods in `frontend/src/lib/api.ts`
|
||||
- `getEncryptionHealth()`
|
||||
- `recoverEncryptedSecrets(payload)`
|
||||
|
||||
3. Add model
|
||||
- `frontend/src/lib/models/KeyRecoveryModel.svelte.ts`
|
||||
- Owns state machine, form values, validation, submit/rescan.
|
||||
|
||||
4. Add components
|
||||
- `frontend/src/lib/components/security/KeyRecoveryWizard.svelte`
|
||||
- `frontend/src/lib/components/security/KeyRecoveryBanner.svelte`
|
||||
- Optional subcomponents:
|
||||
- `RecoveryItemRow.svelte`
|
||||
- `RecoverySecretInput.svelte`
|
||||
|
||||
5. Add settings entry point
|
||||
- Preferred: `frontend/src/routes/settings/SystemSettings.svelte` or `frontend/src/routes/admin/settings/+page.svelte` if admin-only.
|
||||
- Add card: “Encryption key recovery”.
|
||||
- Show status from health endpoint.
|
||||
|
||||
6. Add global detection
|
||||
- In `frontend/src/lib/api.ts`, if `error.error_code === "ENCRYPTION_KEY_MISMATCH"`, dispatch a global event/store signal.
|
||||
- Add top-level banner in layout/nav shell, only for admins.
|
||||
- Keep toast but add actionable “Open recovery wizard”.
|
||||
|
||||
7. Add i18n strings
|
||||
- Russian and English labels/messages for recovery flow.
|
||||
|
||||
8. Add tests
|
||||
- Model tests for state transitions: healthy, needs recovery, partial success, complete, error.
|
||||
- Component tests for rendering grouped items and submitting only filled secrets.
|
||||
- API error normalization test for `ENCRYPTION_KEY_MISMATCH`.
|
||||
|
||||
## Error contract changes
|
||||
|
||||
Normalize backend errors so frontend can reliably open the dialog:
|
||||
|
||||
```json
|
||||
{
|
||||
"detail": {
|
||||
"message": "Stored secret cannot be decrypted with current ENCRYPTION_KEY",
|
||||
"error_code": "ENCRYPTION_KEY_MISMATCH",
|
||||
"secret_type": "llm_provider",
|
||||
"secret_id": "provider-1",
|
||||
"recovery_action": "reenter_secret_or_run_reencrypt"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Apply this to:
|
||||
|
||||
- LLM provider test/decrypt endpoints.
|
||||
- Connection test/update/decrypt endpoints.
|
||||
- Git PAT decrypt paths where user action can recover.
|
||||
|
||||
## UX safeguards
|
||||
|
||||
- Never show old encrypted values.
|
||||
- Never echo typed secrets back after save.
|
||||
- Use password inputs with reveal toggles where existing patterns allow.
|
||||
- Add confirmation before bulk save: “This overwrites stored secrets with newly entered values.”
|
||||
- Add “old key available?” guidance before manual forms to reduce accidental data loss.
|
||||
- Do not let non-admins bulk update global secrets.
|
||||
|
||||
## Validation plan
|
||||
|
||||
1. Static env inventory
|
||||
- Search `${VAR}` in all compose files and generated compose sections.
|
||||
- Search `os.getenv`, `validation_alias`, and frontend `process.env`/`import.meta.env` usage.
|
||||
- Verify every deploy-relevant variable appears in at least one matching example file.
|
||||
Backend:
|
||||
|
||||
2. Compose config validation
|
||||
- `docker compose --env-file .env.example -f docker-compose.yml config`
|
||||
- `docker compose --env-file .env.enterprise-clean.example -f docker-compose.enterprise-clean.yml config`
|
||||
- `docker compose --env-file .env.e2e.example -f docker-compose.e2e.yml config`
|
||||
- Unit test health endpoint with:
|
||||
- all secrets decryptable;
|
||||
- one LLM provider broken;
|
||||
- one DB connection broken;
|
||||
- mixed broken/healthy.
|
||||
- Unit test recover endpoint encrypts new secrets and health returns healthy after recovery.
|
||||
- Verify no secret values appear in logs/responses.
|
||||
|
||||
3. Build script generated compose validation
|
||||
- Run `bash -n build.sh`.
|
||||
- Dry/read review generated heredoc sections for `bundle`, `bundle:embeddings`, and `bundle:light` to confirm same env contract.
|
||||
- Optionally build a lightweight tag and inspect generated `dist/docker/docker-compose*.yml`.
|
||||
Frontend:
|
||||
|
||||
4. Agent smoke validation
|
||||
- Run agent image import smoke test with only `AUTH_SECRET_KEY`, not `JWT_SECRET`.
|
||||
- Run a negative smoke test with only `JWT_SECRET` and confirm startup/import path fails with migration guidance.
|
||||
- Confirm no `AUTH_DATABASE_URL` is required by agent startup.
|
||||
- Confirm JWT decoder accepts backend-issued tokens.
|
||||
- Vitest model tests for `KeyRecoveryModel`.
|
||||
- Component tests for grouped recovery UI.
|
||||
- Manual browser validation:
|
||||
- mock broken LLM provider -> dialog shows LLM section.
|
||||
- enter API key -> save -> status moves to fixed.
|
||||
- broken connection -> password prompt shown.
|
||||
- non-admin sees read-only notice.
|
||||
|
||||
5. Backend validation
|
||||
- Confirm backend starts with canonical `.env.example`/enterprise env contract.
|
||||
- Run targeted tests:
|
||||
- `pytest backend/tests/test_agent/`
|
||||
- `pytest backend/tests/test_core/test_auth_config.py backend/tests/test_core/test_jwt.py`
|
||||
End-to-end smoke:
|
||||
|
||||
6. Docs sanity
|
||||
- Ensure no examples tell operators to add `AUTH_DATABASE_URL` to the agent.
|
||||
- Ensure no compose/example/generated bundle file contains `JWT_SECRET` as a usable variable.
|
||||
- Ensure generated bundle instructions mention `AUTH_SECRET_KEY` as required.
|
||||
1. Start stack with `ENCRYPTION_KEY=old`, create LLM provider + DB connection.
|
||||
2. Restart backend with `ENCRYPTION_KEY=new` without re-encryption.
|
||||
3. Open Settings/Admin.
|
||||
4. Verify recovery wizard lists broken provider/connection.
|
||||
5. Re-enter secrets.
|
||||
6. Verify LLM provider test and DB connection test pass.
|
||||
|
||||
## Expected outcome
|
||||
## Implementation order
|
||||
|
||||
- One canonical security secret name: `AUTH_SECRET_KEY`.
|
||||
- No `JWT_SECRET` fallback remains in agent runtime or Docker examples.
|
||||
- All compose files and generated bundle compose use the same names.
|
||||
- Every operational env file has a corresponding example.
|
||||
- Enterprise operators no longer need to guess why `.env.enterprise-clean.example`, root `.env.example`, backend `.env.example`, and agent `.env.agent.example` differ.
|
||||
- Agent remains slim and independent from backend auth DB config.
|
||||
1. Backend error contract + health inventory endpoint.
|
||||
2. Backend recover endpoint for LLM providers and DB connections.
|
||||
3. Frontend API types/methods + model.
|
||||
4. Wizard component + settings/admin entry point.
|
||||
5. Global banner/error trigger.
|
||||
6. Current-user Profile Git PAT recovery as second phase; admin wizard links users to Profile instead of collecting their tokens.
|
||||
7. Tests and browser validation.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Profile Git PATs: each user updates their own token in Profile; admins do not bulk overwrite personal PATs.
|
||||
- Bulk recovery save behavior: per-item partial success. One invalid password/API key must not block other valid replacements.
|
||||
- Re-encryption command: show command snippet + link to runbook; do not execute shell commands from UI.
|
||||
|
||||
317
backend/src/api/routes/encryption_health.py
Normal file
317
backend/src/api/routes/encryption_health.py
Normal file
@@ -0,0 +1,317 @@
|
||||
# #region EncryptionHealthRoutes [C:4] [TYPE Module] [SEMANTICS api,security,encryption,health,recovery]
|
||||
# @BRIEF API endpoints for encryption health inventory and key-change recovery.
|
||||
# @LAYER API
|
||||
# @RELATION DEPENDS_ON -> [EncryptionManager]
|
||||
# @RELATION DEPENDS_ON -> [LLMProviderService]
|
||||
# @RELATION DEPENDS_ON -> [ConnectionService]
|
||||
# @RATIONALE Centralizes secret inventory and recovery after ENCRYPTION_KEY change.
|
||||
# Without this, operators must manually trace decrypt failures across
|
||||
# scattered API responses.
|
||||
|
||||
import hashlib
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from ...core.config_manager import ConfigManager
|
||||
from ...core.connection_service import ConnectionService
|
||||
from ...core.database import get_db
|
||||
from ...core.encryption import get_encryption_manager, is_fernet_token
|
||||
from ...core.logger import belief_scope, logger
|
||||
from ...dependencies import get_config_manager, get_current_user
|
||||
from ...models.llm import LLMProvider
|
||||
from ...schemas.auth import User
|
||||
from ...services.llm_provider import LLMProviderService
|
||||
|
||||
router = APIRouter(prefix="/api/security/encryption", tags=["Security"])
|
||||
|
||||
# ── Pydantic models ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
class EncryptionHealthItem(BaseModel):
|
||||
id: str
|
||||
type: str # llm_provider | database_connection | profile_git_token
|
||||
label: str
|
||||
status: str # healthy | broken | missing_key
|
||||
reason: str | None = None
|
||||
requires: list[str] = []
|
||||
metadata: dict = {}
|
||||
|
||||
|
||||
class EncryptionHealthResponse(BaseModel):
|
||||
status: str # healthy | needs_recovery
|
||||
key_fingerprint: str
|
||||
summary: dict
|
||||
items: list[EncryptionHealthItem]
|
||||
|
||||
|
||||
class RecoveryItem(BaseModel):
|
||||
id: str
|
||||
type: str
|
||||
values: dict = {}
|
||||
|
||||
|
||||
class RecoveryPayload(BaseModel):
|
||||
items: list[RecoveryItem]
|
||||
|
||||
|
||||
class RecoveryResultItem(BaseModel):
|
||||
id: str
|
||||
type: str
|
||||
status: str # updated | failed | skipped
|
||||
|
||||
|
||||
class RecoveryResponse(BaseModel):
|
||||
status: str # complete | partial_success | failed
|
||||
updated: list[RecoveryResultItem] = []
|
||||
failed: list[RecoveryResultItem] = []
|
||||
|
||||
|
||||
# ── Helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _key_fingerprint() -> str:
|
||||
try:
|
||||
encryption = get_encryption_manager()
|
||||
raw = encryption.key
|
||||
return "sha256:" + hashlib.sha256(raw).hexdigest()[:8]
|
||||
except Exception:
|
||||
return "unavailable"
|
||||
|
||||
|
||||
def _try_decrypt(value: str) -> tuple[bool, str | None]:
|
||||
if not is_fernet_token(value):
|
||||
return False, "not_fernet_token"
|
||||
try:
|
||||
get_encryption_manager().decrypt(value)
|
||||
return True, None
|
||||
except Exception as e:
|
||||
return False, str(e)[:200]
|
||||
|
||||
|
||||
# ── Inventory ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _inventory_llm_providers(db: Session) -> list[dict]:
|
||||
items = []
|
||||
service = LLMProviderService(db)
|
||||
providers = service.get_all_providers()
|
||||
for p in providers:
|
||||
if not p.api_key:
|
||||
items.append(
|
||||
{
|
||||
"id": p.id,
|
||||
"type": "llm_provider",
|
||||
"label": p.name,
|
||||
"status": "missing_key",
|
||||
"reason": "no_key_stored",
|
||||
"requires": ["api_key"],
|
||||
"metadata": {
|
||||
"provider_type": p.provider_type,
|
||||
"base_url": p.base_url,
|
||||
"default_model": p.default_model,
|
||||
"is_active": bool(p.is_active),
|
||||
},
|
||||
}
|
||||
)
|
||||
continue
|
||||
ok, err = _try_decrypt(p.api_key)
|
||||
items.append(
|
||||
{
|
||||
"id": p.id,
|
||||
"type": "llm_provider",
|
||||
"label": p.name,
|
||||
"status": "healthy" if ok else "broken",
|
||||
"reason": None if ok else (err or "decrypt_failed"),
|
||||
"requires": [] if ok else ["api_key"],
|
||||
"metadata": {
|
||||
"provider_type": p.provider_type,
|
||||
"base_url": p.base_url,
|
||||
"default_model": p.default_model,
|
||||
"is_active": bool(p.is_active),
|
||||
},
|
||||
}
|
||||
)
|
||||
return items
|
||||
|
||||
|
||||
def _inventory_connections(config_manager: ConfigManager) -> list[dict]:
|
||||
items = []
|
||||
for conn in config_manager.config.settings.connections:
|
||||
pwd = conn.password
|
||||
if not pwd:
|
||||
items.append(
|
||||
{
|
||||
"id": conn.id,
|
||||
"type": "database_connection",
|
||||
"label": conn.name,
|
||||
"status": "missing_key",
|
||||
"reason": "no_password_stored",
|
||||
"requires": ["password"],
|
||||
"metadata": {"host": conn.host, "database": conn.database, "username": conn.username},
|
||||
}
|
||||
)
|
||||
continue
|
||||
ok, err = _try_decrypt(pwd)
|
||||
items.append(
|
||||
{
|
||||
"id": conn.id,
|
||||
"type": "database_connection",
|
||||
"label": conn.name,
|
||||
"status": "healthy" if ok else "broken",
|
||||
"reason": None if ok else (err or "decrypt_failed"),
|
||||
"requires": [] if ok else ["password"],
|
||||
"metadata": {"host": conn.host, "database": conn.database, "username": conn.username},
|
||||
}
|
||||
)
|
||||
return items
|
||||
|
||||
|
||||
def _inventory_profile_tokens() -> list[dict]:
|
||||
items = []
|
||||
try:
|
||||
from ...core.database import AuthSessionLocal
|
||||
from ...models.profile import UserDashboardPreference
|
||||
|
||||
db_auth = AuthSessionLocal()
|
||||
try:
|
||||
prefs = (
|
||||
db_auth.query(UserDashboardPreference)
|
||||
.filter(
|
||||
UserDashboardPreference.git_personal_access_token_encrypted.isnot(None),
|
||||
UserDashboardPreference.git_personal_access_token_encrypted != "",
|
||||
)
|
||||
.all()
|
||||
)
|
||||
total = len(prefs)
|
||||
broken = 0
|
||||
for pref in prefs:
|
||||
ok, _ = _try_decrypt(pref.git_personal_access_token_encrypted)
|
||||
if not ok:
|
||||
broken += 1
|
||||
if total > 0:
|
||||
items.append(
|
||||
{
|
||||
"id": "profile:aggregate",
|
||||
"type": "profile_git_token",
|
||||
"label": f"{broken}/{total} git token(s) need attention",
|
||||
"status": "broken" if broken > 0 else "healthy",
|
||||
"reason": f"{broken} of {total} cannot decrypt" if broken > 0 else None,
|
||||
"requires": [] if broken == 0 else ["user_action_profile"],
|
||||
"metadata": {"total": total, "broken": broken},
|
||||
}
|
||||
)
|
||||
finally:
|
||||
db_auth.close()
|
||||
except Exception as e:
|
||||
logger.warning("Failed to inventory profile tokens", extra={"error": str(e)})
|
||||
return items
|
||||
|
||||
|
||||
# ── GET /api/security/encryption/health ──────────────────────────────
|
||||
|
||||
|
||||
@router.get("/health", response_model=EncryptionHealthResponse)
|
||||
async def encryption_health(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
config_manager: ConfigManager = Depends(get_config_manager),
|
||||
):
|
||||
with belief_scope("encryption_health"):
|
||||
llm_items = _inventory_llm_providers(db)
|
||||
conn_items = _inventory_connections(config_manager)
|
||||
token_items = _inventory_profile_tokens()
|
||||
|
||||
all_items = llm_items + conn_items + token_items
|
||||
broken = [it for it in all_items if it["status"] == "broken"]
|
||||
|
||||
summary = {
|
||||
"llm_providers_total": len(llm_items),
|
||||
"llm_providers_broken": len([it for it in llm_items if it["status"] == "broken"]),
|
||||
"connections_total": len(conn_items),
|
||||
"connections_broken": len([it for it in conn_items if it["status"] == "broken"]),
|
||||
"profile_tokens_broken": len([it for it in token_items if it["status"] == "broken"]),
|
||||
}
|
||||
|
||||
return EncryptionHealthResponse(
|
||||
status="needs_recovery" if broken else "healthy",
|
||||
key_fingerprint=_key_fingerprint(),
|
||||
summary=summary,
|
||||
items=[EncryptionHealthItem(**it) for it in all_items],
|
||||
)
|
||||
|
||||
|
||||
# ── GET /api/security/encryption/fingerprint ─────────────────────────
|
||||
|
||||
|
||||
@router.get("/fingerprint")
|
||||
async def encryption_fingerprint(
|
||||
current_user: User = Depends(get_current_user),
|
||||
):
|
||||
return {"fingerprint": _key_fingerprint()}
|
||||
|
||||
|
||||
# ── POST /api/security/encryption/recover ────────────────────────────
|
||||
|
||||
|
||||
@router.post("/recover", response_model=RecoveryResponse)
|
||||
async def encryption_recover(
|
||||
payload: RecoveryPayload,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: Session = Depends(get_db),
|
||||
config_manager: ConfigManager = Depends(get_config_manager),
|
||||
):
|
||||
with belief_scope("encryption_recover"):
|
||||
updated: list[RecoveryResultItem] = []
|
||||
failed: list[RecoveryResultItem] = []
|
||||
|
||||
for item in payload.items:
|
||||
try:
|
||||
if item.type == "llm_provider":
|
||||
service = LLMProviderService(db)
|
||||
provider = service.get_provider(item.id)
|
||||
if not provider:
|
||||
failed.append(RecoveryResultItem(id=item.id, type=item.type, status="failed"))
|
||||
continue
|
||||
new_key = item.values.get("api_key", "")
|
||||
if new_key:
|
||||
from ...plugins.llm_analysis.models import LLMProviderConfig, LLMProviderType
|
||||
|
||||
config = LLMProviderConfig(
|
||||
provider_type=LLMProviderType(provider.provider_type),
|
||||
name=provider.name,
|
||||
base_url=provider.base_url,
|
||||
api_key=new_key,
|
||||
default_model=provider.default_model,
|
||||
is_active=bool(provider.is_active),
|
||||
is_multimodal=bool(provider.is_multimodal) if provider.is_multimodal is not None else False,
|
||||
max_images=provider.max_images,
|
||||
context_window=provider.context_window,
|
||||
max_output_tokens=provider.max_output_tokens,
|
||||
)
|
||||
service.update_provider(item.id, config)
|
||||
updated.append(RecoveryResultItem(id=item.id, type=item.type, status="updated"))
|
||||
|
||||
elif item.type == "database_connection":
|
||||
conn_service = ConnectionService(config_manager)
|
||||
new_pwd = item.values.get("password", "")
|
||||
if new_pwd:
|
||||
conn_service.update_connection(item.id, {"password": new_pwd})
|
||||
updated.append(RecoveryResultItem(id=item.id, type=item.type, status="updated"))
|
||||
|
||||
else:
|
||||
failed.append(RecoveryResultItem(id=item.id, type=item.type, status="skipped"))
|
||||
|
||||
except Exception as e:
|
||||
logger.warning("Recovery failed", extra={"id": item.id, "type": item.type, "error": str(e)})
|
||||
failed.append(RecoveryResultItem(id=item.id, type=item.type, status="failed"))
|
||||
|
||||
if not updated and failed:
|
||||
return RecoveryResponse(status="failed", updated=[], failed=failed)
|
||||
if failed:
|
||||
return RecoveryResponse(status="partial_success", updated=updated, failed=failed)
|
||||
return RecoveryResponse(status="complete", updated=updated, failed=[])
|
||||
|
||||
|
||||
# #endregion EncryptionHealthRoutes
|
||||
@@ -47,6 +47,7 @@ from .api.routes import (
|
||||
dashboards,
|
||||
dataset_review,
|
||||
datasets,
|
||||
encryption_health,
|
||||
environments,
|
||||
git,
|
||||
health,
|
||||
@@ -102,6 +103,7 @@ async def lifespan(app: FastAPI):
|
||||
from src.core.database import SessionLocal as _Db
|
||||
from src.models.llm import ValidationRun as _VR
|
||||
from datetime import datetime, timezone
|
||||
|
||||
_s: _Ses = _Db()
|
||||
_stuck = _s.query(_VR).filter(_VR.status == "running").all()
|
||||
for _r in _stuck:
|
||||
@@ -125,6 +127,8 @@ async def lifespan(app: FastAPI):
|
||||
yield
|
||||
# Shutdown
|
||||
scheduler.stop()
|
||||
|
||||
|
||||
# #endregion lifespan
|
||||
# #region FastAPI_App [C:3] [TYPE Global] [SEMANTICS app, fastapi, instance, route-registry]
|
||||
# @ingroup Module
|
||||
@@ -145,7 +149,10 @@ app = FastAPI(
|
||||
# the root task context, making trace_id visible to ALL middleware layers.
|
||||
# See trace.py @RATIONALE for details.
|
||||
from .core.middleware.trace import TraceContextMiddleware # noqa: E402
|
||||
|
||||
app.add_middleware(TraceContextMiddleware)
|
||||
|
||||
|
||||
# #endregion FastAPI_App
|
||||
# #region ensure_initial_admin_user [C:3] [TYPE Function]
|
||||
# @ingroup Module
|
||||
@@ -158,9 +165,7 @@ def ensure_initial_admin_user() -> None:
|
||||
username = os.getenv("INITIAL_ADMIN_USERNAME", "").strip()
|
||||
password = os.getenv("INITIAL_ADMIN_PASSWORD", "").strip()
|
||||
if not username or not password:
|
||||
logger.explore(
|
||||
"INITIAL_ADMIN_CREATE enabled but credentials missing; skipping bootstrap"
|
||||
)
|
||||
logger.explore("INITIAL_ADMIN_CREATE enabled but credentials missing; skipping bootstrap")
|
||||
return
|
||||
# Warn about env-var password — visible via /proc to other processes
|
||||
logger.explore(
|
||||
@@ -202,6 +207,8 @@ def ensure_initial_admin_user() -> None:
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
# #endregion ensure_initial_admin_user
|
||||
# #region run_alembic_migrations [C:2] [TYPE Function]
|
||||
# @ingroup Module
|
||||
@@ -233,7 +240,6 @@ def run_alembic_migrations() -> None:
|
||||
# #endregion run_alembic_migrations
|
||||
|
||||
|
||||
|
||||
# #region app_middleware [TYPE Block]
|
||||
# @ingroup Module
|
||||
# @BRIEF Configure application-wide middleware (Session, CORS).
|
||||
@@ -267,6 +273,7 @@ app.add_middleware(
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
|
||||
# HSTS — Strict-Transport-Security header (see [SEC:L-2])
|
||||
# Only active when FORCE_HTTPS=true (enterprise deployments with proper certs).
|
||||
# In dev or behind HTTP-only proxies, HSTS is disabled to avoid lockout.
|
||||
@@ -277,6 +284,7 @@ class HSTSMiddleware(BaseHTTPMiddleware):
|
||||
(add_header Strict-Transport-Security ...). This middleware is a
|
||||
fallback for environments where nginx is not configured to do so.
|
||||
"""
|
||||
|
||||
def __init__(self, app):
|
||||
super().__init__(app)
|
||||
self._enabled = os.getenv("FORCE_HTTPS", "").strip().lower() in {"1", "true", "yes"}
|
||||
@@ -287,7 +295,10 @@ class HSTSMiddleware(BaseHTTPMiddleware):
|
||||
response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains"
|
||||
return response
|
||||
|
||||
|
||||
app.add_middleware(HSTSMiddleware)
|
||||
|
||||
|
||||
# #endregion app_middleware
|
||||
# #region global_exception_handler [C:2] [TYPE Function]
|
||||
# @ingroup Module
|
||||
@@ -314,6 +325,8 @@ async def global_exception_handler(request: Request, exc: Exception):
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
content={"detail": "Internal server error", "path": request.url.path},
|
||||
)
|
||||
|
||||
|
||||
# #endregion global_exception_handler
|
||||
|
||||
|
||||
@@ -329,6 +342,8 @@ async def network_error_handler(request: Request, exc: NetworkError):
|
||||
status_code=503,
|
||||
detail="Environment unavailable. Please check if the Superset instance is running.",
|
||||
)
|
||||
|
||||
|
||||
# #endregion network_error_handler
|
||||
# #region log_requests [C:3] [TYPE Function]
|
||||
# @ingroup Module
|
||||
@@ -349,13 +364,11 @@ async def log_requests(request: Request, call_next):
|
||||
is_polling = request.url.path.endswith("/api/tasks") and request.method == "GET"
|
||||
if not is_polling:
|
||||
import json as _json, logging as _lg
|
||||
|
||||
if logger.isEnabledFor(_lg.INFO):
|
||||
logger.reason("Incoming request", payload={"method": request.method, "path": request.url.path})
|
||||
else:
|
||||
_json.dump({"ts": __import__('datetime').datetime.now(__import__('datetime').UTC).isoformat(),
|
||||
"level": "INFO", "src": "log_requests", "marker": "REASON",
|
||||
"intent": f"Incoming request: {request.method} {request.url.path}"},
|
||||
sys.stderr, ensure_ascii=False)
|
||||
_json.dump({"ts": __import__("datetime").datetime.now(__import__("datetime").UTC).isoformat(), "level": "INFO", "src": "log_requests", "marker": "REASON", "intent": f"Incoming request: {request.method} {request.url.path}"}, sys.stderr, ensure_ascii=False)
|
||||
sys.stderr.write("\n")
|
||||
sys.stderr.flush()
|
||||
try:
|
||||
@@ -364,10 +377,7 @@ async def log_requests(request: Request, call_next):
|
||||
if logger.isEnabledFor(_lg.INFO):
|
||||
logger.reflect("Response", payload={"status": response.status_code, "path": request.url.path})
|
||||
else:
|
||||
_json.dump({"ts": __import__('datetime').datetime.now(__import__('datetime').UTC).isoformat(),
|
||||
"level": "INFO", "src": "log_requests", "marker": "REFLECT",
|
||||
"intent": f"Response: {response.status_code} for {request.url.path}"},
|
||||
sys.stderr, ensure_ascii=False)
|
||||
_json.dump({"ts": __import__("datetime").datetime.now(__import__("datetime").UTC).isoformat(), "level": "INFO", "src": "log_requests", "marker": "REFLECT", "intent": f"Response: {response.status_code} for {request.url.path}"}, sys.stderr, ensure_ascii=False)
|
||||
sys.stderr.write("\n")
|
||||
sys.stderr.flush()
|
||||
return response
|
||||
@@ -377,6 +387,8 @@ async def log_requests(request: Request, call_next):
|
||||
status_code=503,
|
||||
detail="Environment unavailable. Please check if the Superset instance is running.",
|
||||
)
|
||||
|
||||
|
||||
# #endregion log_requests
|
||||
# #region API_Routes [C:3] [TYPE Block]
|
||||
# @ingroup Module
|
||||
@@ -419,9 +431,12 @@ app.include_router(clean_release_v2.router)
|
||||
app.include_router(profile.router)
|
||||
app.include_router(dataset_review.router)
|
||||
app.include_router(health.router)
|
||||
app.include_router(encryption_health.router)
|
||||
app.include_router(translate.router)
|
||||
app.include_router(validation_tasks, prefix="/api/validation-tasks", tags=["Validation Tasks"])
|
||||
app.include_router(maintenance.maintenance_router)
|
||||
|
||||
|
||||
# #endregion API_Routes
|
||||
# #region api.include_routers [C:1] [TYPE Action] [SEMANTICS routes, registration, api]
|
||||
# @BRIEF Registers all API routers with the FastAPI application.
|
||||
@@ -491,8 +506,11 @@ async def _authenticate_websocket(websocket: WebSocket, endpoint_name: str) -> b
|
||||
error="Invalid token",
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
# #endregion _authenticate_websocket
|
||||
|
||||
|
||||
# #region websocket_endpoint [C:5] [TYPE Function]
|
||||
# @ingroup Module
|
||||
# @BRIEF Provides a WebSocket endpoint for real-time log streaming of a task with server-side filtering.
|
||||
@@ -531,9 +549,7 @@ async def _authenticate_websocket(websocket: WebSocket, endpoint_name: str) -> b
|
||||
# @TEST_EDGE ws_auth_missing_token -> connection rejected with 4001
|
||||
# @TEST_EDGE ws_auth_invalid_token -> connection rejected with 4001
|
||||
@app.websocket("/ws/logs/{task_id}")
|
||||
async def websocket_endpoint(
|
||||
websocket: WebSocket, task_id: str, source: str = None, level: str = None
|
||||
):
|
||||
async def websocket_endpoint(websocket: WebSocket, task_id: str, source: str = None, level: str = None):
|
||||
"""
|
||||
WebSocket endpoint for real-time log streaming AND task status updates.
|
||||
Sends two message types:
|
||||
@@ -546,7 +562,6 @@ async def websocket_endpoint(
|
||||
"""
|
||||
seed_trace_id()
|
||||
with belief_scope("websocket_endpoint", f"task_id={task_id}"):
|
||||
|
||||
# ── WebSocket authentication (see [SEC:C-3]) ──
|
||||
if not await _authenticate_websocket(websocket, "ws/logs"):
|
||||
await websocket.close(code=4001, reason="Authentication required")
|
||||
@@ -598,11 +613,13 @@ async def websocket_endpoint(
|
||||
"input_required": task.input_required,
|
||||
"input_request": task.input_request,
|
||||
}
|
||||
await websocket.send_json({
|
||||
"type": "task_status",
|
||||
"task_id": task.id,
|
||||
"task": status_dict,
|
||||
})
|
||||
await websocket.send_json(
|
||||
{
|
||||
"type": "task_status",
|
||||
"task_id": task.id,
|
||||
"task": status_dict,
|
||||
}
|
||||
)
|
||||
|
||||
try:
|
||||
# ── Send initial task status ──
|
||||
@@ -640,9 +657,7 @@ async def websocket_endpoint(
|
||||
task = task_manager.get_task(task_id)
|
||||
if task and task.status == "AWAITING_INPUT" and task.input_request:
|
||||
synthetic_log = {
|
||||
"timestamp": task.logs[-1].timestamp.isoformat()
|
||||
if task.logs
|
||||
else "2024-01-01T00:00:00",
|
||||
"timestamp": task.logs[-1].timestamp.isoformat() if task.logs else "2024-01-01T00:00:00",
|
||||
"level": "INFO",
|
||||
"message": "Task paused for user input (Connection Re-established)",
|
||||
"context": {"input_request": task.input_request},
|
||||
@@ -690,10 +705,7 @@ async def websocket_endpoint(
|
||||
"level": log_dict.get("level"),
|
||||
},
|
||||
)
|
||||
if (
|
||||
"Task completed successfully" in result.message
|
||||
or "Task failed" in result.message
|
||||
):
|
||||
if "Task completed successfully" in result.message or "Task failed" in result.message:
|
||||
logger.reason(
|
||||
"Observed terminal task log entry; delaying to preserve client visibility",
|
||||
payload={"task_id": task_id, "message": result.message},
|
||||
@@ -724,8 +736,11 @@ async def websocket_endpoint(
|
||||
"Released WebSocket log and status queue subscriptions",
|
||||
payload={"task_id": task_id},
|
||||
)
|
||||
|
||||
|
||||
# #endregion websocket_endpoint
|
||||
|
||||
|
||||
# #region task_events_websocket [C:4] [TYPE Function]
|
||||
# @ingroup Module
|
||||
# @BRIEF WebSocket endpoint for global task events (status changes for ALL tasks).
|
||||
@@ -779,8 +794,11 @@ async def task_events_websocket(websocket: WebSocket):
|
||||
finally:
|
||||
task_manager.unsubscribe_task_events(event_queue)
|
||||
logger.reflect("Released global task events subscription")
|
||||
|
||||
|
||||
# #endregion task_events_websocket
|
||||
|
||||
|
||||
# #region maintenance_events_websocket [C:4] [TYPE Function]
|
||||
# @ingroup Module
|
||||
# @BRIEF WebSocket endpoint for maintenance events (created/ended/banner changes).
|
||||
@@ -833,8 +851,11 @@ async def maintenance_events_websocket(websocket: WebSocket):
|
||||
finally:
|
||||
task_manager.unsubscribe_maintenance_events(event_queue)
|
||||
logger.reflect("Released maintenance events subscription")
|
||||
|
||||
|
||||
# #endregion maintenance_events_websocket
|
||||
|
||||
|
||||
# #region dataset_websocket_endpoint [C:4] [TYPE Function]
|
||||
# @ingroup Module
|
||||
# @BRIEF WebSocket endpoint for dataset.updated events — auto-refresh on task completion.
|
||||
@@ -846,7 +867,6 @@ async def maintenance_events_websocket(websocket: WebSocket):
|
||||
async def dataset_websocket_endpoint(websocket: WebSocket, env_id: str):
|
||||
seed_trace_id()
|
||||
with belief_scope("dataset_websocket_endpoint", f"env_id={env_id}"):
|
||||
|
||||
# ── WebSocket authentication (see [SEC:C-3]) ──
|
||||
if not await _authenticate_websocket(websocket, "ws/datasets"):
|
||||
await websocket.close(code=4001, reason="Authentication required")
|
||||
@@ -868,6 +888,8 @@ async def dataset_websocket_endpoint(websocket: WebSocket, env_id: str):
|
||||
finally:
|
||||
task_manager.unsubscribe_dataset_events(env_id, queue)
|
||||
logger.reflect("Released dataset event subscription", payload={"env_id": env_id})
|
||||
|
||||
|
||||
# #endregion dataset_websocket_endpoint
|
||||
# #region translate_run_websocket [C:3] [TYPE Function]
|
||||
# @ingroup Module
|
||||
@@ -891,6 +913,7 @@ async def translate_run_websocket(websocket: WebSocket, run_id: str):
|
||||
from .core.database import SessionLocal
|
||||
from .plugins.translate.orchestrator_aggregator import TranslationResultAggregator
|
||||
from .plugins.translate.events import TranslationEventLog
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
event_log = TranslationEventLog(db)
|
||||
@@ -916,14 +939,15 @@ async def translate_run_websocket(websocket: WebSocket, run_id: str):
|
||||
except Exception as exc:
|
||||
logger.explore("Translate run WS error", payload={"run_id": run_id}, error=str(exc))
|
||||
logger.reflect("Translate run WS closed", payload={"run_id": run_id})
|
||||
|
||||
|
||||
# #endregion translate_run_websocket
|
||||
# #region StaticFiles [C:1] [TYPE Mount] [SEMANTICS static, frontend, spa]
|
||||
# @BRIEF Mounts the frontend build directory to serve static assets.
|
||||
frontend_path = project_root / "frontend" / "build"
|
||||
if frontend_path.exists():
|
||||
app.mount(
|
||||
"/_app", StaticFiles(directory=str(frontend_path / "_app")), name="static"
|
||||
)
|
||||
app.mount("/_app", StaticFiles(directory=str(frontend_path / "_app")), name="static")
|
||||
|
||||
# #region serve_spa [TYPE Function] [C:1]
|
||||
# @PURPOSE Serves the SPA frontend for any path not matched by API routes.
|
||||
# @PRE frontend_path exists.
|
||||
@@ -933,20 +957,15 @@ if frontend_path.exists():
|
||||
with belief_scope("serve_spa"):
|
||||
# Only serve SPA for non-API paths
|
||||
# API routes are registered separately and should be matched by FastAPI first
|
||||
if file_path and (
|
||||
file_path.startswith("api/")
|
||||
or file_path.startswith("/api/")
|
||||
or file_path == "api"
|
||||
):
|
||||
if file_path and (file_path.startswith("api/") or file_path.startswith("/api/") or file_path == "api"):
|
||||
# This should not happen if API routers are properly registered
|
||||
# Return 404 instead of serving HTML
|
||||
raise HTTPException(
|
||||
status_code=404, detail=f"API endpoint not found: {file_path}"
|
||||
)
|
||||
raise HTTPException(status_code=404, detail=f"API endpoint not found: {file_path}")
|
||||
full_path = frontend_path / file_path
|
||||
if file_path and full_path.is_file():
|
||||
return FileResponse(str(full_path))
|
||||
return FileResponse(str(frontend_path / "index.html"))
|
||||
|
||||
# #endregion serve_spa
|
||||
else:
|
||||
# #region read_root [TYPE Function] [C:1]
|
||||
@@ -956,9 +975,8 @@ else:
|
||||
@app.get("/")
|
||||
async def read_root():
|
||||
with belief_scope("read_root"):
|
||||
return {
|
||||
"message": "Superset Tools API is running (Frontend build not found)"
|
||||
}
|
||||
return {"message": "Superset Tools API is running (Frontend build not found)"}
|
||||
|
||||
# #endregion read_root
|
||||
# #endregion StaticFiles
|
||||
# #endregion AppModule
|
||||
|
||||
@@ -1081,6 +1081,16 @@ export const api = {
|
||||
// @RELATION DEPENDS_ON -> [requestApi]
|
||||
revokeApiKey: <T = unknown>(keyId: string) => requestApi<T>(`/admin/api-keys/${keyId}`, 'DELETE'),
|
||||
// #endregion revokeApiKey
|
||||
|
||||
// #region getEncryptionHealth [C:2] [TYPE Function] [SEMANTICS security,encryption,health]
|
||||
// @BRIEF Inventory all stored encrypted secrets and report broken ones.
|
||||
getEncryptionHealth: <T = unknown>() => fetchApi<T>('/security/encryption/health'),
|
||||
// #endregion getEncryptionHealth
|
||||
|
||||
// #region recoverEncryptedSecrets [C:2] [TYPE Function] [SEMANTICS security,encryption,recover]
|
||||
// @BRIEF Submit replacement secrets for items that could not be decrypted.
|
||||
recoverEncryptedSecrets: <T = unknown>(payload: unknown) => postApi<T>('/security/encryption/recover', payload),
|
||||
// #endregion recoverEncryptedSecrets
|
||||
};
|
||||
// #endregion ApiRegistry
|
||||
// #endregion ApiModule
|
||||
@@ -1130,3 +1140,5 @@ export const createConnection = api.createConnection;
|
||||
export const updateConnection = api.updateConnection;
|
||||
export const deleteConnection = api.deleteConnection;
|
||||
export const testConnection = api.testConnection;
|
||||
export const getEncryptionHealth = api.getEncryptionHealth;
|
||||
export const recoverEncryptedSecrets = api.recoverEncryptedSecrets;
|
||||
|
||||
273
frontend/src/lib/components/security/KeyRecoveryWizard.svelte
Normal file
273
frontend/src/lib/components/security/KeyRecoveryWizard.svelte
Normal file
@@ -0,0 +1,273 @@
|
||||
<!-- #region KeyRecoveryWizard [C:3] [TYPE Component] [SEMANTICS security,encryption,recovery,wizard] -->
|
||||
<!-- @BRIEF Key recovery wizard — guides admin through fixing encrypted secrets after ENCRYPTION_KEY change. -->
|
||||
<!-- @RELATION BINDS_TO -> [KeyRecoveryModel] -->
|
||||
<!-- @UX_STATE: idle -> hidden, model.state === 'idle' -> no render -->
|
||||
<!-- @UX_STATE: scanning -> spinner, key fingerprint, elapsed -->
|
||||
<!-- @UX_STATE: healthy -> green checkmark, "All credentials healthy" -->
|
||||
<!-- @UX_STATE: needs_recovery -> list by tab (LLM/DB/Git), action buttons -->
|
||||
<!-- @UX_STATE: editing -> input forms for each broken secret -->
|
||||
<!-- @UX_STATE: saving -> spinner, saving indicator -->
|
||||
<!-- @UX_STATE: partial_success -> updated/failed breakdown -->
|
||||
<!-- @UX_STATE: complete -> green checkmark, close -->
|
||||
<!-- @UX_STATE: error -> error banner, retry -->
|
||||
<!-- @UX_FEEDBACK green badge for saved items, red for failed -->
|
||||
<!-- @UX_RECOVERY Rescan button, Re-enter values -->
|
||||
<script lang="ts">
|
||||
import { KeyRecoveryModel } from "$lib/models/KeyRecoveryModel.svelte";
|
||||
import type { EncryptionHealthItem, RecoveryTab } from "../../../types/encryptionRecovery";
|
||||
|
||||
let {
|
||||
show = false,
|
||||
onclose = () => {},
|
||||
}: { show?: boolean; onclose?: () => void } = $props();
|
||||
|
||||
const model = new KeyRecoveryModel();
|
||||
|
||||
let selectedTab: RecoveryTab = $state("llm_provider");
|
||||
|
||||
$effect(() => {
|
||||
if (show && model.state === "idle") {
|
||||
model.scan();
|
||||
}
|
||||
});
|
||||
|
||||
function handleClose() {
|
||||
model.dismiss();
|
||||
onclose();
|
||||
}
|
||||
|
||||
function itemTypeLabel(type: string): string {
|
||||
switch (type) {
|
||||
case "llm_provider": return "LLM Provider";
|
||||
case "database_connection": return "DB Connection";
|
||||
case "profile_git_token": return "Git Token";
|
||||
default: return type;
|
||||
}
|
||||
}
|
||||
|
||||
function itemLocation(item: EncryptionHealthItem): string {
|
||||
switch (item.type) {
|
||||
case "llm_provider":
|
||||
return `${item.metadata?.provider_type ?? "?"} — ${item.metadata?.base_url ?? "?"}`;
|
||||
case "database_connection":
|
||||
return `${item.metadata?.host ?? "?"}/${item.metadata?.database ?? "?"} (${item.metadata?.username ?? "?"})`;
|
||||
default:
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
function itemIdType(item: EncryptionHealthItem): string {
|
||||
return `${item.type}#${item.id}`;
|
||||
}
|
||||
|
||||
function brokenByType(type: string): EncryptionHealthItem[] {
|
||||
return model.brokenItems.filter(i => i.type === type);
|
||||
}
|
||||
|
||||
const tabFilters: { key: RecoveryTab; label: string; brokenCount: number }[] = $derived([
|
||||
{ key: "llm_provider" as RecoveryTab, label: "LLM Providers", brokenCount: brokenByType("llm_provider").length },
|
||||
{ key: "database_connection" as RecoveryTab, label: "DB Connections", brokenCount: brokenByType("database_connection").length },
|
||||
{ key: "profile_git_token" as RecoveryTab, label: "Git Tokens", brokenCount: brokenByType("profile_git_token").length },
|
||||
]);
|
||||
|
||||
const brokenFiltered: EncryptionHealthItem[] = $derived(
|
||||
brokenByType(selectedTab)
|
||||
);
|
||||
|
||||
const recoverableTypes: Set<string> = new Set(["llm_provider", "database_connection"]);
|
||||
</script>
|
||||
|
||||
{#if show && model.state !== "idle"}
|
||||
<div
|
||||
class="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center z-50"
|
||||
role="dialog"
|
||||
aria-modal="true"
|
||||
>
|
||||
<div class="bg-surface-card rounded-lg shadow-xl w-full max-w-2xl max-h-[85vh] flex flex-col overflow-hidden">
|
||||
<!-- Header -->
|
||||
<div class="flex items-center justify-between px-6 py-4 border-b border-border">
|
||||
<h2 class="text-lg font-semibold text-text">
|
||||
{model.state === "scanning"
|
||||
? "Scanning encrypted secrets..."
|
||||
: model.state === "healthy"
|
||||
? "Encrypted credentials are healthy"
|
||||
: model.state === "complete"
|
||||
? "Recovery complete"
|
||||
: "Encrypted credentials need re-entry"}
|
||||
</h2>
|
||||
<button
|
||||
class="text-text-muted hover:text-text text-xl leading-none"
|
||||
onclick={handleClose}
|
||||
aria-label="Close"
|
||||
>×</button>
|
||||
</div>
|
||||
|
||||
<!-- Body -->
|
||||
<div class="flex-1 overflow-y-auto px-6 py-4 space-y-4">
|
||||
|
||||
<!-- Scanning -->
|
||||
{#if model.state === "scanning"}
|
||||
<div class="flex items-center gap-3 py-8 justify-center">
|
||||
<span class="inline-block w-5 h-5 border-2 border-border-strong border-t-transparent rounded-full animate-spin"></span>
|
||||
<span class="text-text-muted">Checking all stored secrets...</span>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<!-- Error -->
|
||||
{#if model.state === "error"}
|
||||
<div class="bg-destructive-light border border-destructive-ring text-destructive px-4 py-3 rounded">
|
||||
{model.error || "Scan failed"}
|
||||
</div>
|
||||
<button
|
||||
class="rounded bg-primary px-4 py-2 text-sm text-white"
|
||||
onclick={() => model.scan()}
|
||||
>Retry scan</button>
|
||||
{/if}
|
||||
|
||||
<!-- Healthy -->
|
||||
{#if model.state === "healthy"}
|
||||
<div class="flex items-center gap-3 py-4">
|
||||
<span class="text-success text-2xl">✓</span>
|
||||
<div>
|
||||
<p class="text-text font-medium">All known encrypted secrets can be decrypted.</p>
|
||||
<p class="text-text-muted text-sm">
|
||||
Key fingerprint: <code class="font-mono">{model.health?.key_fingerprint ?? "?"}</code>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
<!-- Needs Recovery / Editing / Partial / Saving / Complete -->
|
||||
{#if ["needs_recovery", "editing", "partial_success", "saving", "complete"].includes(model.state)}
|
||||
<!-- Summary -->
|
||||
<div class="text-sm text-text-muted space-y-1">
|
||||
<p>Key fingerprint: <code class="font-mono text-xs">{model.health?.key_fingerprint ?? "?"}</code></p>
|
||||
<p>
|
||||
Changing <strong>AUTH_SECRET_KEY</strong> logs users out.
|
||||
Changing <strong>ENCRYPTION_KEY</strong> requires re-entering stored secrets or running re-encryption.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<!-- Tabs -->
|
||||
<div class="flex gap-1 border-b border-border">
|
||||
{#each tabFilters as tab}
|
||||
<button
|
||||
class="px-3 py-2 text-sm font-medium border-b-2 transition-colors
|
||||
{selectedTab === tab.key ? 'border-primary text-primary' : 'border-transparent text-text-muted hover:text-text'}"
|
||||
onclick={() => { selectedTab = tab.key; }}
|
||||
>
|
||||
{tab.label}
|
||||
{#if tab.brokenCount > 0}
|
||||
<span class="ml-1 px-1.5 py-0.5 text-xs rounded-full bg-destructive-light text-destructive">{tab.brokenCount}</span>
|
||||
{/if}
|
||||
</button>
|
||||
{/each}
|
||||
</div>
|
||||
|
||||
<!-- Item list per tab -->
|
||||
<div class="space-y-3">
|
||||
{#each brokenFiltered as item (itemIdType(item))}
|
||||
<div class="rounded-lg border border-border bg-surface-muted p-3" class:border-destructive-ring={model.failedIds.has(item.id)}>
|
||||
<div class="flex items-center justify-between mb-1">
|
||||
<div>
|
||||
<span class="font-medium text-text">{item.label}</span>
|
||||
<span class="ml-2 text-xs text-text-muted">{itemTypeLabel(item.type)}</span>
|
||||
</div>
|
||||
<span class="text-xs px-2 py-0.5 rounded-full bg-destructive-light text-destructive">
|
||||
⚠ cannot decrypt
|
||||
</span>
|
||||
</div>
|
||||
<div class="text-xs text-text-muted mb-2">{itemLocation(item)}</div>
|
||||
|
||||
{#if model.state === "editing" && recoverableTypes.has(item.type)}
|
||||
<div class="flex gap-2 mt-2">
|
||||
<input
|
||||
type="password"
|
||||
class="flex-1 rounded border border-border-strong px-2 py-1 text-sm"
|
||||
placeholder={item.requires[0] === "api_key" ? "Enter new API key..." : "Enter new password..."}
|
||||
value={model.secretValues[item.id] || ""}
|
||||
oninput={(e: Event) => {
|
||||
const t = e.target as HTMLInputElement;
|
||||
model.setSecretValue(item.id, t.value);
|
||||
}}
|
||||
/>
|
||||
{#if model.savingIds.has(item.id)}
|
||||
<span class="text-sm text-text-muted self-center">Saving...</span>
|
||||
{:else if model.savedIds.has(item.id)}
|
||||
<span class="text-sm text-success self-center">✓ Saved</span>
|
||||
{:else if model.failedIds.has(item.id)}
|
||||
<span class="text-sm text-destructive self-center">✗ Failed</span>
|
||||
{/if}
|
||||
</div>
|
||||
{:else if item.type === "profile_git_token"}
|
||||
<p class="text-xs text-text-subtle mt-1">
|
||||
Git tokens are personal. Each user must open <strong>Profile → Git token</strong> and enter a new PAT.
|
||||
</p>
|
||||
{/if}
|
||||
</div>
|
||||
{/each}
|
||||
</div>
|
||||
|
||||
{#if brokenFiltered.length === 0}
|
||||
<p class="text-text-muted text-sm">No broken items in this category.</p>
|
||||
{/if}
|
||||
{/if}
|
||||
|
||||
<!-- Recovery info panel -->
|
||||
{#if model.state === "needs_recovery"}
|
||||
<div class="rounded-lg bg-surface-muted border border-border p-3 text-sm">
|
||||
<p class="font-medium text-text mb-1">Recommended if old key is available:</p>
|
||||
<code class="block text-xs font-mono text-text-muted bg-surface-card p-2 rounded">
|
||||
OLD_ENCRYPTION_KEY=<old> NEW_ENCRYPTION_KEY=<new> \
|
||||
python -m src.scripts.reencrypt --dry-run
|
||||
</code>
|
||||
</div>
|
||||
{/if}
|
||||
|
||||
{#if model.state === "partial_success"}
|
||||
<div class="rounded-lg bg-warning-light border border-warning-DEFAULT p-3 text-sm text-warning">
|
||||
Some secrets saved, some failed. Check failed items above and re-enter.
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<!-- Footer -->
|
||||
<div class="flex items-center justify-between px-6 py-3 border-t border-border bg-surface-muted">
|
||||
<div class="flex gap-2">
|
||||
<button
|
||||
class="rounded px-3 py-1.5 text-sm border border-border hover:bg-surface-card transition"
|
||||
onclick={() => model.rescan()}
|
||||
disabled={model.state === "scanning" || model.state === "saving"}
|
||||
>↻ Rescan</button>
|
||||
</div>
|
||||
<div class="flex gap-2">
|
||||
{#if model.state === "needs_recovery"}
|
||||
<button
|
||||
class="rounded bg-primary px-4 py-1.5 text-sm text-white hover:bg-primary-hover transition"
|
||||
onclick={() => model.startEditing()}
|
||||
>Enter replacement values</button>
|
||||
{:else if model.state === "editing"}
|
||||
<button
|
||||
class="rounded px-3 py-1.5 text-sm border border-border hover:bg-surface-card transition"
|
||||
onclick={() => model.cancelEditing()}
|
||||
>Cancel</button>
|
||||
<button
|
||||
class="rounded bg-primary px-4 py-1.5 text-sm text-white hover:bg-primary-hover transition disabled:opacity-50"
|
||||
onclick={() => model.saveSecrets()}
|
||||
disabled={Object.values(model.secretValues).every(v => !v?.trim())}
|
||||
>Save entered secrets</button>
|
||||
{:else if model.state === "saving"}
|
||||
<button disabled class="rounded bg-primary px-4 py-1.5 text-sm text-white opacity-50">
|
||||
Saving...
|
||||
</button>
|
||||
{/if}
|
||||
<button
|
||||
class="rounded px-3 py-1.5 text-sm border border-border hover:bg-surface-card transition"
|
||||
onclick={handleClose}
|
||||
>Close</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{/if}
|
||||
<!-- #endregion KeyRecoveryWizard -->
|
||||
@@ -247,5 +247,33 @@
|
||||
"connections_test_success": "Connected — {latency}ms",
|
||||
"connections_test_error": "Connection test failed",
|
||||
"connections_load_failed": "Failed to load connections",
|
||||
"connections_dismiss": "Dismiss"
|
||||
"connections_dismiss": "Dismiss",
|
||||
|
||||
"encryption_recovery_title": "Encryption Key Recovery",
|
||||
"encryption_recovery_desc": "Check whether all stored encrypted credentials can be decrypted with the current ENCRYPTION_KEY. If credentials were encrypted with a different key, they must be re-entered or re-encrypted.",
|
||||
"encryption_recovery_scan": "Check encrypted secrets",
|
||||
"encryption_recovery_healthy": "All credentials healthy",
|
||||
"encryption_recovery_broken": "{count} saved credential(s) need attention",
|
||||
"encryption_recovery_scanning": "Checking all stored secrets...",
|
||||
"encryption_recovery_llm_tab": "LLM Providers",
|
||||
"encryption_recovery_db_tab": "DB Connections",
|
||||
"encryption_recovery_git_tab": "Git Tokens",
|
||||
"encryption_recovery_cannot_decrypt": "Cannot decrypt",
|
||||
"encryption_recovery_reenter_key": "Enter new API key...",
|
||||
"encryption_recovery_reenter_password": "Enter new password...",
|
||||
"encryption_recovery_save_secrets": "Save entered secrets",
|
||||
"encryption_recovery_cancel": "Cancel",
|
||||
"encryption_recovery_close": "Close",
|
||||
"encryption_recovery_rescan": "Rescan",
|
||||
"encryption_recovery_retry": "Retry scan",
|
||||
"encryption_recovery_complete": "Recovery complete",
|
||||
"encryption_recovery_key_fingerprint": "Key fingerprint",
|
||||
"encryption_recovery_auth_vs_encryption": "Changing AUTH_SECRET_KEY logs users out. Changing ENCRYPTION_KEY requires re-entering stored secrets or running re-encryption.",
|
||||
"encryption_recovery_recommended": "Recommended if old key is available:",
|
||||
"encryption_recovery_reencrypt_cmd": "OLD_ENCRYPTION_KEY=<old> NEW_ENCRYPTION_KEY=<new> \\\n python -m src.scripts.reencrypt --dry-run",
|
||||
"encryption_recovery_enter_values": "Enter replacement values",
|
||||
"encryption_recovery_git_token_hint": "Git tokens are personal. Each user must open Profile → Git token and enter a new PAT.",
|
||||
"encryption_recovery_partial_success": "Some secrets saved, some failed. Check failed items above and re-enter.",
|
||||
"encryption_recovery_saving": "Saving..."
|
||||
}
|
||||
}
|
||||
|
||||
@@ -247,5 +247,33 @@
|
||||
"connections_test_success": "Подключено — {latency}ms",
|
||||
"connections_test_error": "Не удалось проверить подключение",
|
||||
"connections_load_failed": "Не удалось загрузить подключения",
|
||||
"connections_dismiss": "Закрыть"
|
||||
"connections_dismiss": "Закрыть",
|
||||
|
||||
"encryption_recovery_title": "Восстановление ключа шифрования",
|
||||
"encryption_recovery_desc": "Проверить, расшифровываются ли все сохранённые секреты текущим ENCRYPTION_KEY. Если учётные данные были зашифрованы другим ключом, их нужно ввести заново или выполнить перешифрование.",
|
||||
"encryption_recovery_scan": "Проверить зашифрованные секреты",
|
||||
"encryption_recovery_healthy": "Все учётные данные в порядке",
|
||||
"encryption_recovery_broken": "{count} сохранённых секретов требуют внимания",
|
||||
"encryption_recovery_scanning": "Проверяю сохранённые секреты...",
|
||||
"encryption_recovery_llm_tab": "LLM провайдеры",
|
||||
"encryption_recovery_db_tab": "Подключения к БД",
|
||||
"encryption_recovery_git_tab": "Git токены",
|
||||
"encryption_recovery_cannot_decrypt": "Не расшифровывается",
|
||||
"encryption_recovery_reenter_key": "Введите новый API ключ...",
|
||||
"encryption_recovery_reenter_password": "Введите новый пароль...",
|
||||
"encryption_recovery_save_secrets": "Сохранить введённые секреты",
|
||||
"encryption_recovery_cancel": "Отмена",
|
||||
"encryption_recovery_close": "Закрыть",
|
||||
"encryption_recovery_rescan": "Обновить",
|
||||
"encryption_recovery_retry": "Повторить проверку",
|
||||
"encryption_recovery_complete": "Восстановление завершено",
|
||||
"encryption_recovery_key_fingerprint": "Отпечаток ключа",
|
||||
"encryption_recovery_auth_vs_encryption": "Смена AUTH_SECRET_KEY разлогинивает пользователей. Смена ENCRYPTION_KEY требует повторного ввода сохранённых секретов или перешифрования.",
|
||||
"encryption_recovery_recommended": "Рекомендуется, если доступен старый ключ:",
|
||||
"encryption_recovery_reencrypt_cmd": "OLD_ENCRYPTION_KEY=<старый> NEW_ENCRYPTION_KEY=<новый> \\\n python -m src.scripts.reencrypt --dry-run",
|
||||
"encryption_recovery_enter_values": "Ввести новые значения",
|
||||
"encryption_recovery_git_token_hint": "Git токены персональные. Каждый пользователь должен открыть Профиль → Git токен и ввести новый PAT.",
|
||||
"encryption_recovery_partial_success": "Часть секретов сохранена, часть не удалась. Проверьте ошибки выше и введите значения заново.",
|
||||
"encryption_recovery_saving": "Сохранение..."
|
||||
}
|
||||
}
|
||||
|
||||
159
frontend/src/lib/models/KeyRecoveryModel.svelte.ts
Normal file
159
frontend/src/lib/models/KeyRecoveryModel.svelte.ts
Normal file
@@ -0,0 +1,159 @@
|
||||
// #region KeyRecoveryModel [C:3] [TYPE Model] [SEMANTICS security,encryption,recovery,model]
|
||||
// @BRIEF State model for key-change recovery wizard — inventory, editing, save.
|
||||
// @INVARIANT Editing changes are local until saved; cancel discards all edits.
|
||||
// @STATE idle — Not yet loaded.
|
||||
// @STATE scanning — Fetching health from backend.
|
||||
// @STATE healthy — All secrets decryptable.
|
||||
// @STATE needs_recovery — One or more broken secrets found.
|
||||
// @STATE editing — User entering replacement values.
|
||||
// @STATE saving — Replacement secrets being submitted.
|
||||
// @STATE partial_success — Some fixed, some failed.
|
||||
// @STATE complete — All fixed.
|
||||
// @STATE error — Scan or save failed.
|
||||
// @ACTION scan() — Fetch health from backend.
|
||||
// @ACTION startEditing(secretId) — Start editing a specific secret.
|
||||
// @ACTION setSecretValue(id, key, value) — Set a replacement value.
|
||||
// @ACTION saveSecrets() — Submit all entered replacements.
|
||||
// @ACTION cancelEditing() — Discard all edits and return to needs_recovery.
|
||||
// @ACTION dismiss() — Close wizard, return to idle.
|
||||
|
||||
import { getEncryptionHealth, recoverEncryptedSecrets } from "$lib/api";
|
||||
import type {
|
||||
EncryptionHealthResponse,
|
||||
EncryptionHealthItem,
|
||||
RecoveryResponse,
|
||||
RecoveryWizardState,
|
||||
RecoveryTab,
|
||||
SecretStatus,
|
||||
} from "../../types/encryptionRecovery";
|
||||
|
||||
interface SecretEntry {
|
||||
item: EncryptionHealthItem;
|
||||
editValue: string; // replacement value entered by user
|
||||
}
|
||||
|
||||
export class KeyRecoveryModel {
|
||||
// ── Atoms ──────────────────────────────────────────────────────
|
||||
state: RecoveryWizardState = $state("idle");
|
||||
health: EncryptionHealthResponse | null = $state(null);
|
||||
error: string | null = $state(null);
|
||||
activeTab: RecoveryTab = $state("llm_provider");
|
||||
|
||||
// Form values: Map<itemId, replacementString>
|
||||
secretValues: Record<string, string> = $state({});
|
||||
savingIds: Set<string> = $state(new Set());
|
||||
savedIds: Set<string> = $state(new Set());
|
||||
failedIds: Set<string> = $state(new Set());
|
||||
|
||||
// ── Derived ────────────────────────────────────────────────────
|
||||
brokenItems: EncryptionHealthItem[] = $derived(
|
||||
this.health?.items?.filter(i => i.status === "broken") ?? []
|
||||
);
|
||||
|
||||
brokenByType = $derived.by((type: string) =>
|
||||
this.brokenItems.filter(i => i.type === type)
|
||||
);
|
||||
|
||||
allHealthy: boolean = $derived(
|
||||
this.health?.status === "healthy"
|
||||
);
|
||||
|
||||
// Count of broken items that still need attention
|
||||
remainingBroken: number = $derived(
|
||||
this.brokenItems.length - this.savedIds.size
|
||||
);
|
||||
|
||||
// ── Actions ────────────────────────────────────────────────────
|
||||
async scan(): Promise<void> {
|
||||
this.state = "scanning";
|
||||
this.error = null;
|
||||
try {
|
||||
const data = await getEncryptionHealth<EncryptionHealthResponse>();
|
||||
this.health = data;
|
||||
this.secretValues = {};
|
||||
this.savingIds = new Set();
|
||||
this.savedIds = new Set();
|
||||
this.failedIds = new Set();
|
||||
this.state = data.status === "healthy" ? "healthy" : "needs_recovery";
|
||||
} catch (e: unknown) {
|
||||
this.error = e instanceof Error ? e.message : "Scan failed";
|
||||
this.state = "error";
|
||||
}
|
||||
}
|
||||
|
||||
startEditing(): void {
|
||||
this.state = "editing";
|
||||
}
|
||||
|
||||
setSecretValue(id: string, value: string): void {
|
||||
this.secretValues = { ...this.secretValues, [id]: value };
|
||||
}
|
||||
|
||||
cancelEditing(): void {
|
||||
if (this.health) {
|
||||
this.state = this.health.status === "healthy" ? "healthy" : "needs_recovery";
|
||||
} else {
|
||||
this.state = "idle";
|
||||
}
|
||||
}
|
||||
|
||||
async saveSecrets(): Promise<void> {
|
||||
this.state = "saving";
|
||||
this.error = null;
|
||||
const items = [];
|
||||
for (const item of this.brokenItems) {
|
||||
const val = this.secretValues[item.id];
|
||||
if (val && val.trim()) {
|
||||
const key = item.requires[0] || "value";
|
||||
items.push({ id: item.id, type: item.type, values: { [key]: val } });
|
||||
this.savingIds = new Set([...this.savingIds, item.id]);
|
||||
}
|
||||
}
|
||||
|
||||
if (items.length === 0) {
|
||||
this.state = "needs_recovery";
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const resp = await recoverEncryptedSecrets<RecoveryResponse>({ items });
|
||||
const newSaved = new Set(this.savedIds);
|
||||
const newFailed = new Set(this.failedIds);
|
||||
for (const r of resp.updated) {
|
||||
newSaved.add(r.id);
|
||||
this.savingIds.delete(r.id);
|
||||
}
|
||||
for (const r of resp.failed) {
|
||||
newFailed.add(r.id);
|
||||
this.savingIds.delete(r.id);
|
||||
}
|
||||
this.savedIds = newSaved;
|
||||
this.failedIds = newFailed;
|
||||
|
||||
if (resp.status === "complete") {
|
||||
this.state = "complete";
|
||||
await this.scan();
|
||||
} else if (resp.status === "partial_success") {
|
||||
this.state = "partial_success";
|
||||
await this.scan();
|
||||
} else {
|
||||
this.state = "error";
|
||||
this.error = "All recoveries failed";
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
this.error = e instanceof Error ? e.message : "Save failed";
|
||||
this.state = "error";
|
||||
}
|
||||
}
|
||||
|
||||
dismiss(): void {
|
||||
this.state = "idle";
|
||||
this.health = null;
|
||||
this.error = null;
|
||||
}
|
||||
|
||||
rescan(): void {
|
||||
this.scan();
|
||||
}
|
||||
}
|
||||
// #endregion KeyRecoveryModel
|
||||
@@ -6,10 +6,27 @@
|
||||
<script lang="ts">
|
||||
import { t, locale } from "$lib/i18n/index.svelte.js";
|
||||
import ApiKeysTab from "$lib/components/settings/ApiKeysTab.svelte";
|
||||
import KeyRecoveryWizard from "$lib/components/security/KeyRecoveryWizard.svelte";
|
||||
import { appTimezone } from "$lib/stores/timezone.svelte.js";
|
||||
import { getEncryptionHealth } from "$lib/api";
|
||||
import type { EncryptionHealthResponse } from "../../types/encryptionRecovery";
|
||||
|
||||
let { settings = $bindable(), onSave } = $props();
|
||||
|
||||
let showRecoveryWizard = $state(false);
|
||||
let encHealthSummary: { fingerprint: string; broken: number } | null = $state(null);
|
||||
|
||||
async function loadEncHealth() {
|
||||
try {
|
||||
const data = await getEncryptionHealth<EncryptionHealthResponse>();
|
||||
const broken = data.items.filter(i => i.status === "broken").length;
|
||||
encHealthSummary = { fingerprint: data.key_fingerprint, broken };
|
||||
} catch {
|
||||
encHealthSummary = null;
|
||||
}
|
||||
}
|
||||
$effect(() => { loadEncHealth(); });
|
||||
|
||||
async function handleSave() {
|
||||
// Sync app timezone to global store before persisting
|
||||
if (settings.app_timezone) {
|
||||
@@ -99,5 +116,39 @@
|
||||
<div class="border-t border-border pt-8">
|
||||
<ApiKeysTab />
|
||||
</div>
|
||||
|
||||
<!-- Encryption Key Recovery -->
|
||||
<div class="border-t border-border pt-8 mt-8">
|
||||
<h2 class="text-xl font-bold mb-4">Encryption Key Recovery</h2>
|
||||
<p class="text-text-muted mb-4">
|
||||
Check whether all stored encrypted credentials can be decrypted with the current ENCRYPTION_KEY.
|
||||
If credentials were encrypted with a different key, they must be re-entered or re-encrypted.
|
||||
</p>
|
||||
|
||||
<div class="bg-surface-muted rounded-lg border border-border p-4">
|
||||
<div class="flex items-center justify-between">
|
||||
<div>
|
||||
<div class="text-sm font-medium text-text">
|
||||
Key fingerprint: <code class="font-mono text-xs">{encHealthSummary?.fingerprint ?? "loading..."}</code>
|
||||
</div>
|
||||
<div class="text-sm mt-1">
|
||||
{#if encHealthSummary === null}
|
||||
<span class="text-text-muted">Checking...</span>
|
||||
{:else if encHealthSummary.broken === 0}
|
||||
<span class="text-success">✓ All credentials healthy</span>
|
||||
{:else}
|
||||
<span class="text-destructive">⚠ {encHealthSummary.broken} saved credential{encHealthSummary.broken !== 1 ? "s" : ""} need attention</span>
|
||||
{/if}
|
||||
</div>
|
||||
</div>
|
||||
<button
|
||||
class="rounded bg-primary px-4 py-2 text-sm text-white hover:bg-primary-hover transition"
|
||||
onclick={async () => { await loadEncHealth(); showRecoveryWizard = true; }}
|
||||
>Check encrypted secrets</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<KeyRecoveryWizard show={showRecoveryWizard} onclose={() => { showRecoveryWizard = false; }} />
|
||||
</div>
|
||||
<!-- #endregion SystemSettings -->
|
||||
|
||||
73
frontend/src/types/encryptionRecovery.ts
Normal file
73
frontend/src/types/encryptionRecovery.ts
Normal file
@@ -0,0 +1,73 @@
|
||||
// #region EncryptionRecoveryTypes [C:2] [TYPE Module] [SEMANTICS types,encryption,recovery,health]
|
||||
// @BRIEF TypeScript DTOs for encryption health inventory and key-change recovery.
|
||||
// @RELATION DEPENDS_ON -> [EncryptionHealthRoutes:Backend]
|
||||
|
||||
export type EncryptedSecretType = "llm_provider" | "database_connection" | "profile_git_token";
|
||||
|
||||
export type SecretStatus = "healthy" | "broken" | "missing_key";
|
||||
|
||||
export interface EncryptionHealthItem {
|
||||
id: string;
|
||||
type: EncryptedSecretType;
|
||||
label: string;
|
||||
status: SecretStatus;
|
||||
reason: string | null;
|
||||
requires: string[];
|
||||
metadata: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface EncryptionSummary {
|
||||
llm_providers_total: number;
|
||||
llm_providers_broken: number;
|
||||
connections_total: number;
|
||||
connections_broken: number;
|
||||
profile_tokens_broken: number;
|
||||
}
|
||||
|
||||
export interface EncryptionHealthResponse {
|
||||
status: "healthy" | "needs_recovery";
|
||||
key_fingerprint: string;
|
||||
summary: EncryptionSummary;
|
||||
items: EncryptionHealthItem[];
|
||||
}
|
||||
|
||||
export interface FingerprintResponse {
|
||||
fingerprint: string;
|
||||
}
|
||||
|
||||
export interface RecoveryItem {
|
||||
id: string;
|
||||
type: EncryptedSecretType;
|
||||
values: Record<string, string>;
|
||||
}
|
||||
|
||||
export interface RecoveryPayload {
|
||||
items: RecoveryItem[];
|
||||
}
|
||||
|
||||
export interface RecoveryResultItem {
|
||||
id: string;
|
||||
type: EncryptedSecretType | string;
|
||||
status: "updated" | "failed" | "skipped";
|
||||
}
|
||||
|
||||
export interface RecoveryResponse {
|
||||
status: "complete" | "partial_success" | "failed";
|
||||
updated: RecoveryResultItem[];
|
||||
failed: RecoveryResultItem[];
|
||||
}
|
||||
|
||||
export type RecoveryWizardState =
|
||||
| "idle"
|
||||
| "scanning"
|
||||
| "healthy"
|
||||
| "needs_recovery"
|
||||
| "editing"
|
||||
| "saving"
|
||||
| "partial_success"
|
||||
| "complete"
|
||||
| "error";
|
||||
|
||||
export type RecoveryTab = "llm_provider" | "database_connection" | "profile_git_token";
|
||||
|
||||
// #endregion EncryptionRecoveryTypes
|
||||
Reference in New Issue
Block a user