fix(certs): restore HTTP CA auto-download and test edge matrix

Restore LLM_CA_CERT_URLS support for corporate PKI HTTP cert delivery.
Downloaded certificates are installed into the llm/ CA category and share
the same system trust + NSS import path as CERTS_PATH-mounted certs.

Changes:
- certs.sh: add download_llm_ca_certs with HTTP-only curl policy,
  PEM/DER handling, retries, llm/ storage, hash symlinks for llm+custom
- backend/agent entrypoints: run download before install_all_certs
- compose/env/docs: expose LLM_CA_CERT_URLS again and update ADR-0009
- integration tests: cover PEM/DER/CER, invalid certs, non-HTTP skips,
  query-string filenames, private/server skips, empty inputs, combined
  LLM_CA_CERT_URLS + CERTS_PATH channels, and NSS imports

Verified:
- 194 passed, 1 skipped integration tests
- full backend container smoke passed with testcontainers PostgreSQL
- docker bundle rebuilt for 0.5.0

docs/adr/ADR-0009-ssl-certificate-management.md

@@ -1,6 +1,6 @@
 # [DEF:ADR-0009:ADR]
 # @STATUS ACTIVE
-# @PURPOSE Define the strategy for corporate SSL certificate management across all HTTP clients in superset-tools (httpx, requests, openai), covering system CA store, NSS database for Chromium/Playwright. Centralized CERTS_PATH contract replaces per-client SSL toggles. LLM_SSL_VERIFY and LLM_CA_CERT_URLS removed in 0.2.x.
+# @PURPOSE Define the strategy for corporate SSL certificate management across all HTTP clients in superset-tools (httpx, requests, openai), covering system CA store, NSS database for Chromium/Playwright. Two input channels: (1) HTTP download from corporate PKI via LLM_CA_CERT_URLS (primary), (2) volume mount via CERTS_PATH (additional). LLM_SSL_VERIFY removed in 0.2.x.
 # @RELATION DEPENDS_ON -> [ADR-0001:ADR]
 # @RELATION DEPENDS_ON -> [ADR-0004:ADR]
 # @RELATION CALLS -> [docker/backend.entrypoint.sh]
@@ -97,24 +97,32 @@ intermediate CA in OpenSSL 3.x. `verify=<capath_dir>` works correctly.
 - Trust attributes: `"C,,"` (trusted CA for TLS server certs)
 - Fixes `ERR_CERT_AUTHORITY_INVALID` in Playwright dashboard screenshots
 
-### Layer 4: Centralized CERTS_PATH Trust Contract (0.2.x)
+### Layer 4: LLM_CA_CERT_URLS HTTP Download (0.5.1)
 
-- **REMOVED**: `LLM_SSL_VERIFY` — TLS verification is always enabled; no env escape hatch exists.
-- **REMOVED**: `LLM_CA_CERT_URLS` — certificate download by URL is no longer supported.
-- **Replaced by**: `CERTS_PATH` volume mount (`./certs:/opt/certs:ro`) as the single certificate input.
-- All trust anchors come from mounted `CERTS_PATH` directory (`.crt`, `.pem`, `.cer`, `.der` files).
+- Env var `LLM_CA_CERT_URLS` — space-separated list of HTTP URLs to CA certificates
+- Downloaded on container startup by `certs.sh:download_llm_ca_certs()`
+- URLs use **plain HTTP only** — no TLS chicken-and-egg problem
+- Supports PEM and DER formats (auto-detected, DER auto-converted to PEM)
+- Retry logic: 3 attempts with 2-second delay
+- Downloaded certs saved to `/usr/local/share/ca-certificates/llm/`
+- `update-ca-certificates --fresh` includes them in the system bundle
+- Example: `LLM_CA_CERT_URLS="http://pki.rusal.com/CertData/UC_RUSAL_Policy_CA.crt http://pki.rusal.com/CertData/UC_RUSAL_RGM_Issuing_CA.crt"`
+
+### Layer 5: CERTS_PATH Volume Mount
+
+- Env var `CERTS_PATH` — directory with `.crt`/`.pem`/`.cer`/`.der` files
+- Mounted as read-only volume: `${CERTS_PATH:-./certs}:/opt/certs:ro`
+- Installed by `certs.sh:install_all_certs()` **after** `download_llm_ca_certs()`
+- Server/private files (server.crt, *.key, *.p12, *.pfx) are excluded from CA import
 - Shared `docker/certs.sh` handles installation across all containers (backend, frontend, agent).
-- Centralized Python helper `backend/src/core/ssl.py` provides `system_ssl_context()` / `httpx_verify()`.
-- No code path disables TLS verification — the `verify=False` escape hatch is permanently removed.
-- **Migration**: Place corporate CA files in `./certs/`, remove `LLM_SSL_VERIFY`/`LLM_CA_CERT_URLS` from `.env`.
 
 ## Key Files
 
 | File | Role |
 |------|------|
 | `docker/backend.Dockerfile` | Installs `libnss3-tools` (certutil), Playwright Chromium |
-| `docker/certs.sh` | Shared cert installer: `install_all_certs`, `install_ca_to_nss`, `normalize_cert` |
-| `docker/backend.entrypoint.sh` | Sources `certs.sh`, calls `install_all_certs` + `install_ca_to_nss` |
+| `docker/certs.sh` | Shared cert installer: `download_llm_ca_certs`, `install_all_certs`, `install_ca_to_nss`, `normalize_cert` |
+| `docker/backend.entrypoint.sh` | Sources `certs.sh`, calls `download_llm_ca_certs` → `install_all_certs` → `install_ca_to_nss` |
 | `docker/frontend.entrypoint.sh` | Installs CA certs from `CERTS_PATH` into Alpine store |
 | `docker/agent.entrypoint.sh` | Sources `certs.sh`, installs certs for agent container |
 | `backend/src/core/ssl.py` | Centralized SSL helper: `system_ssl_context()`, `httpx_verify()`, `describe_context()` |
@@ -250,5 +258,6 @@ docker compose -f docker-compose.enterprise-clean.yml \
 | 0.1.6 | QA fixes: fingerprint dedup, hash symlink collision, NSS collision, DER→PEM error handling, chicken-and-egg TLS, nullglob, translate plugin SSL |
 | 0.1.7 | **capath instead of cafile**: OpenSSL 3.x cafile limitation discovered and fixed. `.crt` extension for downloaded certs. Diagnostic script rewritten. |
 | 0.2.x | **Centralized SSL**: `LLM_SSL_VERIFY` and `LLM_CA_CERT_URLS` removed. Shared `docker/certs.sh` for all containers. `backend/src/core/ssl.py` central helper. `CERTS_PATH` volume mount is the only certificate input. Verify=False escape hatch permanently removed. |
+| 0.5.1 | **Restored**: `LLM_CA_CERT_URLS` HTTP download as primary cert input channel. `download_llm_ca_certs()` added to `certs.sh`. Downloaded certs saved to `llm/` subdirectory, `install_all_certs` extended to create hash symlinks for both `custom/` and `llm/` dirs. `CERTS_PATH` volume mount kept as secondary channel. Integration test added for HTTP download flow. |
 
 # [/DEF:ADR-0009:ADR]