docs(specs): session-2 handoff + evidence-based 044/050 task statuses and session-state update
This commit is contained in:
@@ -187,6 +187,20 @@ def test_resolution_adopts_exact_enabled_match():
|
||||
assert resolved is not None and resolved.binding_ref == "live-binding-server-001"
|
||||
|
||||
|
||||
def test_resolution_adopts_model_shaped_record():
|
||||
"""The parsed config carries Pydantic ScenarioLiveExecutionBindingConfig instances, not dicts."""
|
||||
snapshot = _binding_snapshot()
|
||||
manager = _settings_config_manager(
|
||||
[SimpleNamespace(enabled=True, binding_snapshot=snapshot)]
|
||||
)
|
||||
resolved = _resolve_configured_live_binding(
|
||||
environment_id=_BINDING_ENV, plan=_plan(),
|
||||
principal_fingerprint=hashlib.sha256(_BINDING_ACTOR.encode()).hexdigest(),
|
||||
config_manager=manager,
|
||||
)
|
||||
assert resolved is not None and resolved.binding_ref == "live-binding-server-001"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"record_override, environment_id, actor",
|
||||
[
|
||||
|
||||
130
docs/reports/agentic-runtime-handoff-2026-09-10-evening.md
Normal file
130
docs/reports/agentic-runtime-handoff-2026-09-10-evening.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# Промежуточный отчёт: agentic runtime — handoff (session 2, Front 6 Track A + Track B)
|
||||
|
||||
**Дата:** 2026-09-10 (вечер)
|
||||
**Предыдущий отчёт:** `docs/reports/agentic-runtime-implementation-handoff-2026-09-10.md` (утро — Fronts 1–5 offline, uncommitted)
|
||||
**Статус:** Front 6 Track A **завершён и закоммичен**; Track B (live chain) **проведён на живом стенде и закоммичен**, кроме пунктов, заблокированных внешними факторами (Gitea PAT, мультимодальные изображения).
|
||||
**Верификация:** полная backend-суита **11527 passed / 0 failed / 244 skipped / 1 xpassed** (2026-09-10, после всех коммитов и ревью-фиксов).
|
||||
|
||||
---
|
||||
|
||||
## 1. Коммиты этой сессии (все — `scenario`-workstream, кроме двух migration от пользователя)
|
||||
|
||||
```
|
||||
edc9a98a fix(scenario): code-review hardening — drop unknown criterion findings, fail closed on unsupported verdict, remove dead branch
|
||||
bf8abff7 docs(specs): live canary v2 trace + restore operation_id in browser evidence details
|
||||
3458343d fix(scenario): live agent-evaluation chain — manifest byte-lengths, response normalization, prompt contract
|
||||
bbbd4ccf feat(scenario): published-catalog source fallback + server-side live binding resolution at start
|
||||
043a1455 docs(specs): record live provider canary evidence in quickstart statuses
|
||||
fe11a636 docs(scenario): live provider canary trace (T7/T8 passed) + two production defects closed
|
||||
ba2f1f45 fix(scenario): browser preflight probe deadlocked lifespan loop thread (300s startup hang)
|
||||
3e5cc942 refactor(scenario): wrap naked helpers in GRACE regions (INV_1)
|
||||
1b795975 test(mcp): adapt initial-scenario e2e to fail-closed baseline resolver (Front 5-R)
|
||||
a21481ea refactor(scenario): INV_7 split runner into walker/start/dispatch/crash-recovery facade
|
||||
8f057f30 refactor(scenario): INV_7 split executors + lifecycle, repair orphan relations, fix serializer fixture
|
||||
b024e608 fix(scenario): commit missing Slice G router + eligibility service
|
||||
83727aa7 feat(scenario): complete offline agentic runtime chain
|
||||
```
|
||||
Пользовательские коммиты, попавшие в диапазон и **не относящиеся** к этому workstream: `befa03e3`, `25327024` (migration).
|
||||
|
||||
---
|
||||
|
||||
## 2. Что сделано
|
||||
|
||||
### Front 6 Track A — offline структурный долг (закрыт)
|
||||
- **INV_7 полностью вычищен** для execution-пакета: `runner.py` 1207→**121** (фасад; 6 новых модулей `terminal_effects/walker/registry_builder/start_run/crash_recovery/dispatch_runs`), `lifecycle.py` 611→**299** (`lifecycle_helpers/cancel_lifecycle/timeout_lifecycle`), `executors.py` 508→**257** (`executor_helpers/offline_executors`). Все модули <400 (max `walker.py` 392).
|
||||
- **Frozen region-ID** `ScenarioExecution.Runner.*` / `Executors.*` сохранены дословно; Python-импорты `execution.runner.*` работают через фасад.
|
||||
- **Orphan relations** 409→406: `handles.py`, `context_authority.py`, `baseline_catalog_locking.py` (направление связи исправлено на `CALLED_BY`).
|
||||
- **Pydantic serializer warning** в `test_validator_branches` — корневой фикс (валидный `Expected(kind="structural")`).
|
||||
- **INV_1**: naked-функции моих модулей обёрнуты в регионы (36→32 по execution/; остаток — pre-existing чужих модулей).
|
||||
- **`test_mcp_initial_scenario_e2e`** адаптирован к fail-closed BaselineResolver (published snapshot + явный селектор).
|
||||
|
||||
### Front 6 Track B — live chain (проведён, см. §5)
|
||||
- **D1 published-catalog source** (`execution/published_catalog_source.py`): caller-side Gitea-загрузчик, fail-closed матрица (9 тестов), вайринг в `start_run` единой точкой (только при явном селекторе). Резолвер остался [EXT]-seam (D11, без ORM).
|
||||
- **D2 server-side live-binding resolution**: `start_run._resolve_configured_live_binding` — резолв из `settings.scenario_live_execution_bindings` по `(environment, dashboard, principal=sha256(actor))`; поддержаны dict и Pydantic-формы записи; клиент binding не передаёт. 12 тестов.
|
||||
- **T7/T8 live-провайдеры + канарейки**: binding `ss-prod-d11-live-001` (live-inspect дашборда 11, fingerprint `sha256:16838a47…`); `/api/ready` → browser/screenshot/evidence_storage/bindings = ready. Два живых прогона:
|
||||
- canary v1 (прямой вызов production-функций): run `597274d3` — browser+8 screenshots, durable evidence.
|
||||
- canary v2 (REST-only): run `4eebfab3` — server-side binding → PROD gate → scheduler dispatch → browser → screenshots → **реальный LLM qwen3.8-flash** → **`AgentEvaluation` запись сохранена** → DecisionPolicy row 11.
|
||||
- **Три production-дефекта найдены и закрыты живыми прогонами**: preflight-deadlock (`ba2f1f45`), отсутствие `artifact_byte_lengths` (пустой evaluation-манифест), невалидируемые provider-ответы (нормализация + промпт-контракт, `3458343d`).
|
||||
- **Кодревью (`edc9a98a`)**: отброшены findings с неизвестным criterion_id; pass/fail без допустимого finding фейлится в inconclusive; удалена мёртвая ветка; no-op `setdefault` исправлен; гейт каталога ужесточён.
|
||||
|
||||
---
|
||||
|
||||
## 3. Состояние живого стенда (операционные детали для следующего агента)
|
||||
|
||||
| Что | Значение |
|
||||
|---|---|
|
||||
| Процесс стенда | managed background `bgp_08bc71ab5001FP1c1s1Pt9FH2h` (`./run.sh --skip-install`, persistent) |
|
||||
| Порты | backend 8000, frontend 5173 |
|
||||
| App-логин | `admin` / `admin` (пароль сброшен в dev-БД через штатный хэшер) |
|
||||
| PostgreSQL | `localhost:5432/ss_tools` (postgres/postgres) |
|
||||
| `backend/.env` | добавлен `STORAGE_ROOT_PATH=/home/busya/dev/ss-tools-storage` (env-authority по `Core.EnvSettings`) |
|
||||
| Storage | `/home/busya/dev/ss-tools-storage/drafts/<run_id>/` (evidence-байты) |
|
||||
| Binding | `ss-prod-d11-live-001` в `app_configurations.settings.scenario_live_execution_bindings` (enabled; principal = sha256 admin **user id** `5d9c4bed-…`) |
|
||||
| Superset | `https://ss-prod.bebesh.ru` (admin/admin), dashboard 11 «Sales Dashboard» |
|
||||
| LLM | `llm_providers` id `a13fcc27-03d3-42a7-afb4-b4d77e31e805`, openai/qwen3.8-flash, base `https://www.sophnet.com/api/open-apis/v1`, multimodal=true |
|
||||
| Gitea | `https://git.bebesh.ru` (user `busya`) — **токен отклонён 401**, см. §4 |
|
||||
|
||||
Артефакты (не коммитить, deployment state): `backend/.env`, строка binding в БД, сброшенный пароль.
|
||||
|
||||
---
|
||||
|
||||
## 4. Остаток работы (приоритезировано, с точными шагами)
|
||||
|
||||
### Блокер 1 — Gitea PAT (T6 live publication)
|
||||
Предоставленный токен `69b42c79…` отвергается Gitea 1.24.3 всеми способами (`Authorization: token`, `Bearer`, basic `busya:<token>`, `PRIVATE-TOKEN`): ответы «user does not exist [uid: 0]» / «password is invalid» — пользователь `busya` существует, токен невалиден (отозван/ошибка копирования). **Нужен новый PAT.**
|
||||
После получения: положить в `backend/.env` (`PUBLISHED_CATALOG_GITEA_URL/TOKEN/REPO/REF`, опц. `PUBLISHED_CATALOG_PATH_TEMPLATE`), опубликовать bootstrap-генерацию (`catalogs/ss-prod-visual/v1.json` из `specs/044-.../fixtures/production-contract-refresh.json`) и снять live-trace pin-резолюции (baseline-backed run).
|
||||
|
||||
### Блокер 2 — мультимодальное прикрепление изображений (evaluation verdicts)
|
||||
Промпт `evaluation_adapter` текстовый (`get_json_completion([{role,content}])`) — модель **не видит** скриншоты, поэтому вердикты честно inconclusive/low-confidence. Нужно: загрузка evidence-байтов по manifest refs + image-content-parts в `LLMClient` (bounded задача, после неё визуальные PASS станут доверяемыми). Не входит в текущий пакет.
|
||||
|
||||
### Прочие открытые пункты
|
||||
- **`baseline_pin` в `AgentEvaluation`-записи всегда `{}`** — решить, должен ли walker штамповать пиннутый пин в запись (baseline-backed планы).
|
||||
- **MIME-хардкод**: screenshot→`image/jpeg`, browser→`image/png`; при WebP-архивации будет `EVALUATION_EVIDENCE_METADATA_MISMATCH`. Выводить MIME из сигнатуры байтов.
|
||||
- **REST/MCP не экспонируют `live_execution_binding`** — D2 резолвит его server-side, но доверенной admin-поверхности для регистрации binding'ов (кроме прямой записи в конфиг) нет.
|
||||
- **T029m** (050): live-stand replay sales-сценария целиком через MCP — OPEN.
|
||||
- **050 T044–T046** (REST/MCP parity) — OPEN.
|
||||
- **`Expected.threshold`** (038 schema + models) — отложено (нормативный schema).
|
||||
- **Optional perf/quality baseline** — out of scope.
|
||||
|
||||
---
|
||||
|
||||
## 5. Отчёты-артефакты (durable decision memory)
|
||||
|
||||
| Документ | Содержание |
|
||||
|---|---|
|
||||
| `docs/reports/agentic-runtime-orchestration-plan-2026-09-10.md` | общий план + execution record (§7/§8) |
|
||||
| `docs/reports/agentic-runtime-implementation-handoff-2026-09-10.md` | утренний handoff (Fronts 1–5) |
|
||||
| `docs/reports/agentic-runtime-live-canary-2026-09-10.md` | canary v1 (T7/T8) + 2 дефекта |
|
||||
| `docs/reports/agentic-runtime-live-canary-v2-2026-09-10.md` | canary v2 (REST + evaluation) + дефекты + design-наблюдения |
|
||||
| `.kilo/plans/1789036353814-agentic-runtime-front6-plan.md` | план Front 6 |
|
||||
| `.kilo/plans/agentic-runtime-t6-live-chain-worker-plan.md` | пакет этого воркера |
|
||||
| `specs/044-.../prototype/live_canary_v2.py` | воспроизводимая live-канарейка (REST-only) |
|
||||
|
||||
---
|
||||
|
||||
## 6. Верификация (актуальные команды)
|
||||
|
||||
```bash
|
||||
cd backend && source .venv/bin/activate
|
||||
python -m pytest -q # 11527 passed / 0 failed / 244 skipped (integration)
|
||||
python -m ruff check src/services/dashboard_testing src/api/routes/dashboard_testing
|
||||
python -m compileall -q src
|
||||
python -m alembic heads # один head: 0022_agent_evaluation
|
||||
cd .. && backend/.venv/bin/python specs/044-dashboard-scenario-execution/prototype/validate_contract_refresh.py \
|
||||
specs/044-dashboard-scenario-execution/fixtures/production-contract-refresh.json # 11/5/49 PASS
|
||||
|
||||
# live (стенд поднят): серверный резолв binding + полная цепь
|
||||
backend/.venv/bin/python specs/044-dashboard-scenario-execution/prototype/live_canary_v2.py
|
||||
curl -s http://127.0.0.1:8000/api/ready | python3 -m json.tool
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Ограничения и инварианты
|
||||
|
||||
- **GRACE-Poly:** `#region/#endregion` на новых функциях; `@`-теги одной строкой; реальные решения — `@RATIONALE`/`@REJECTED`; INV_7 <400; INV_9 без синтетики; после правок анкеров — `axiom_search rebuild` (0 parse warnings).
|
||||
- **Без новых зависимостей** (httpx уже используется); **без новых Alembic-миграций** (D11); нормативные `*.schema.json` не менять.
|
||||
- **Не трогать** незакоммиченную работу пользователя: maintenance-сервисы, migration (`core/migration/*`, `dry_run_orchestrator.py`), `frontend/src/lib/api.ts` + его тесты, `specs/017-*`, `.kilo/agents/*`, `.kilo/agent-manager.json`.
|
||||
- **Секреты только в `backend/.env`** (gitignored): Gitea PAT/токены в git не коммитить.
|
||||
- **tasks.md/specs не отмечать «complete» без executable trace** (правило session-state).
|
||||
- Дизайн-наблюдение (не баг): при `evaluation_mode=required` каждый policy-normative шаг без собственного `evaluation_input` уходит в row 9 `EVALUATION_UNAVAILABLE` — пиннутая семантика 038 (см. canary v2 отчёт).
|
||||
@@ -85,20 +85,20 @@ Closed in this pass (docs only; no `backend/` or `frontend/` edits; `tasks.md` c
|
||||
- `specs/050-mcp-interface/quickstart.md` created (was missing). MCP-only agent interaction; catalog `2.0.0` / `PINNED_CATALOG_MAJOR=2`; no frontend agent workspace.
|
||||
- Refresh report created: `docs/reports/ss-prod-agentic-e2e-spec-refresh-report-2026-09-08.md`.
|
||||
|
||||
Runtime A+B/C/F plus Fronts 1–3 (production adapter, walker rows 9–14, `ACTION_REGISTRY_VERSION` `038.3.0`) landed 2026-09-10 in the working tree, independently verified, still **uncommitted**. Production GO remains OPEN.
|
||||
Runtime A+B/C/F plus Fronts 1–5 (production adapter, walker rows 9–14, `ACTION_REGISTRY_VERSION` `038.4.0`) landed 2026-09-10 and are **committed** (`83727aa7`…`edc9a98a`), independently verified. Front 6 Track A (INV_7 splits) and Track B (live chain) are done. Production GO remains OPEN only on the external blockers in "Remaining OPEN" below.
|
||||
|
||||
## Remaining OPEN (do not mark Closed)
|
||||
|
||||
- Slice E live Playwright — driver/reconciler canary; compiler visual-chain offline landed 2026-09-10 (`038.4.0`).
|
||||
- Slice G live storage canary — GET/HEAD runtime landed 2026-09-10 offline.
|
||||
- BaselineResolver live Git publication — pin-from-injected-snapshot landed 2026-09-10; no CatalogRevision ORM.
|
||||
- Live providers — browser/screenshot registration, LLM Settings, provider-loop canary on a live stand.
|
||||
- 050 T029m live-stand replay; T044–T046 REST/MCP consume/publish parity; T030 negative UI acceptance.
|
||||
- Slice E live Playwright — driver/reconciler canary; compiler visual-chain offline landed 2026-09-10 (`038.4.0`). **Update 2026-09-10 (evening):** browser read-only live canary PASSED (`open_dashboard`, checkpoint + page URL + PNG digest); driver/reconciler + forced timeout canary still OPEN.
|
||||
- Slice G live storage canary — GET/HEAD runtime landed 2026-09-10 offline. **Update 2026-09-10 (evening):** durable live storage proven (8 screenshots, digests, bytes on disk); live GET/HEAD ACL/status canary still OPEN.
|
||||
- BaselineResolver live Git publication — pin-from-injected-snapshot landed 2026-09-10; no CatalogRevision ORM. **Update 2026-09-10 (evening):** caller-side published-catalog source landed (`bbbd4ccf`, fail-closed); Gitea PAT rejected 401 → live pin BLOCKED.
|
||||
- Live providers — browser/screenshot registration, LLM Settings, provider-loop canary on a live stand. **Update 2026-09-10 (evening):** DONE live — binding `ss-prod-d11-live-001`, `/api/ready` browser/screenshot/bindings ready, live evaluation through qwen3.8-flash, `AgentEvaluation` persisted (canary v2 `4eebfab3`). OPEN: multimodal image attachment (prompts are text-only).
|
||||
- 050 T029m live-stand replay; T044–T046 REST/MCP consume/publish parity; T030 negative UI acceptance. **Update 2026-09-10 (evening):** REST-only chain proven; MCP-external replay still OPEN.
|
||||
- Optional approved performance/quality baseline (out of scope).
|
||||
|
||||
## Next action
|
||||
|
||||
Fronts 1–5 offline of `docs/reports/agentic-runtime-orchestration-plan-2026-09-10.md` are closed in the working tree (independently verified). Next is live deploy (browser/screenshot/LLM registration, provider-loop canary) and Front 6 INV_7 splits — not this pass.
|
||||
Fronts 1–5 offline and Front 6 Track A (INV_7 splits, orphan relations, INV_1) are **committed** (through `edc9a98a`); full backend suite `11527 passed / 0 failed`. Front 6 Track B live chain is proven end-to-end through REST (canary v2 `4eebfab3`). Next: (1) supply a valid Gitea PAT to close T6 live publication/pin; (2) attach evidence images to the evaluation prompt (multimodal) so visual verdicts become trustworthy; (3) run the full MCP-external replay (T029m). See `docs/reports/agentic-runtime-handoff-2026-09-10-evening.md` for the session-2 handoff (stand state, commit map, exact next steps).
|
||||
|
||||
Offline contract gate (re-run anytime; does not require runtime edits):
|
||||
|
||||
|
||||
@@ -26,6 +26,8 @@
|
||||
| Visual chain | Compiler emits browser→capture→comparison→evaluation→policy (Slice E) | One-action templates; live Playwright canary PASSED 2026-09-10 (browser `open_dashboard` + screenshot capture vs ss-prod dashboard 11; comparison/evaluation steps not in the canary graph) |
|
||||
| Live providers | Registered browser/screenshot + multimodal LLM | Binding `ss-prod-d11-live-001` resolved **server-side at REST start** (no binding in the request body); `/api/ready`: browser/screenshot ready, bindings 1; canary v2 live trace: `docs/reports/agentic-runtime-live-canary-v2-2026-09-10.md` |
|
||||
|
||||
**Open live gaps (2026-09-10, for the next agent):** multimodal image attachment (prompts are text-only → visual verdicts stay inconclusive); `baseline_pin` in the persisted `AgentEvaluation` record is `{}` for baseline-backed plans; screenshot/browser evidence MIME is hardcoded (`image/jpeg`/`image/png`) and would mismatch a WebP archive; live baseline pin is blocked on a valid Gitea PAT.
|
||||
|
||||
The former agent-driven product-UI flow (dashboard → `/agent` workspace → agent-generated scenario) is SUPERSEDED. 044 is a headless execution engine; agent interaction is external MCP (050). Product UI (045) is read-only evidence plus human approvals.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -133,17 +133,31 @@
|
||||
`backend/src/services/dashboard_testing/execution/providers/xlsx.py`.
|
||||
- [x] T038 [P] [US2] Implement deterministic ReportProvider and ArtifactProvider durable registration
|
||||
with manifest/digest ownership in `backend/src/services/dashboard_testing/execution/providers/artifacts.py`.
|
||||
- [ ] T039 [US2] Implement AgentEvaluationProvider and deterministic DecisionPolicy integration in
|
||||
`backend/src/services/dashboard_testing/execution/providers/agent_evaluation.py`.
|
||||
@INVARIANT: model verdict never directly sets ScenarioResult or consumes HumanCheckpoint.
|
||||
- [ ] T040 [US2] Add startup deployment registration and readiness preflight for all provider capabilities
|
||||
in `backend/src/services/dashboard_testing/execution/provider_bootstrap.py`.
|
||||
- [x] T039 [US2] Implement AgentEvaluationProvider and deterministic DecisionPolicy integration.
|
||||
**Status (2026-09-10): done.** Implementation lives at `execution/agent_evaluation.py`
|
||||
(store/parser/`submit_evaluation`) + `execution/evaluation_adapter.py` (production adapter) +
|
||||
`executors.py`/`live_composition.py` wiring — the task's `providers/agent_evaluation.py` path is stale.
|
||||
@INVARIANT: model verdict never directly sets ScenarioResult or consumes HumanCheckpoint.
|
||||
Evidence: live canary v2 run `4eebfab3` — real qwen3.8-flash evaluation, `AgentEvaluation` row
|
||||
persisted, DecisionPolicy row 11 `LOW_CONFIDENCE` (rows 9–14 reachable); trace
|
||||
`docs/reports/agentic-runtime-live-canary-v2-2026-09-10.md`.
|
||||
- [~] T040 [US2] Add startup deployment registration and readiness preflight for all provider capabilities.
|
||||
**Status (2026-09-10): implemented and live-verified; deployment-evidence rule partially met.**
|
||||
Actual modules: `execution/live_composition.py` + `execution/providers/preflight.py` + `app.py`
|
||||
lifespan (the `provider_bootstrap.py` path is stale). Live `/api/ready`: provider_loop/evidence_storage/
|
||||
browser/screenshot = ready, bindings registered = 1; the browser-probe startup deadlock is fixed
|
||||
(`ba2f1f45`). Remaining per the deployment-evidence rule: provider/version + capability fingerprint +
|
||||
redacted dependency diagnostics are not yet part of the readiness payload.
|
||||
- [x] T041 [US2] Add common provider contract tests for unavailable/dependency failure/capacity
|
||||
exhaustion/timeout/cancel/duplicate/late response/ownership mismatch/malformed result/cleanup and
|
||||
reconciliation in `backend/tests/services/dashboard_testing/registry/test_provider_contract.py`.
|
||||
- [ ] T042 [US2] Add provider-specific contract tests in
|
||||
`backend/tests/services/dashboard_testing/registry/test_provider_*.py` and real deployment health
|
||||
checks for Superset, Browser and Screenshot bindings.
|
||||
- [~] T042 [US2] Add provider-specific contract tests in `test_provider_*.py` and real deployment health
|
||||
checks for Superset, Browser and Screenshot bindings.
|
||||
**Status (2026-09-10): partial.** Browser/Screenshot/preflight/contract tests present and green
|
||||
(`test_provider_browser.py`, `test_provider_screenshot.py`, `test_provider_preflight.py`,
|
||||
`test_provider_contract.py`); live health proven for browser/screenshot plus Superset-provider
|
||||
composition (canary v1 `597274d3`, canary v2 `4eebfab3`). Missing: a Superset-specific contract test
|
||||
and one live Superset query through the binding (canary graphs used browser+screenshot+evaluation).
|
||||
- [x] T042c [US2] Add dispatcher policy/binding revalidation tests in
|
||||
`backend/tests/services/dashboard_testing/registry/test_provider_contract.py`: reclassification to PROD
|
||||
or a changed provider/security fingerprint after approval prevents all provider I/O and returns typed
|
||||
@@ -154,6 +168,10 @@
|
||||
`GO` requires all BrowserProvider acceptance vectors to pass and one owned evidence receipt.
|
||||
Contract target before runtime enablement: **90/100**; runtime score remains below target until
|
||||
T034, T040-T042 and this canary task are complete.
|
||||
**Status (2026-09-10):** PROD live traces exist (canary v1 `597274d3`, canary v2 `4eebfab3`:
|
||||
browser `open_dashboard` passed with checkpoint + real page URL + PNG digest; readiness payload
|
||||
captured in the trace reports). Still OPEN: PREPROD canaries, the forced timeout/cleanup canary,
|
||||
and the owned receipt under `evidence/browser-provider/`.
|
||||
|
||||
## Requirement evidence rules
|
||||
|
||||
@@ -225,10 +243,10 @@ Historical [x] rows above retain only their dated local/transport evidence; they
|
||||
|
||||
Contract: [Production baseline-backed evaluation](contracts/production-chain.md).
|
||||
|
||||
- [ ] T043 [P0/P1/P2] Baseline set/version/catalog/release/commit/IDs/digests affect idempotency; moving catalog after admission cannot change plan/result; legacy unpinned result is ineligible. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T044 [P0/P1/P2] GET/HEAD prove same ACL/status/headers; MIME/digest/length checked before bytes, cross-owner hidden, expired410, corrupt409, traversal/range/oversize rejected. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T045 [P0/P1/P2] Startup/readiness/start-loop and shutdown/drain/cancel/reconcile survive fault injection; unknown effect quarantines capacity; late response cannot win. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T046 [P0/P1/P2] End-to-end real browser→capture→durable artifact→deterministic comparison→optional immutable evaluation→policy→result; all required evidence present before PASS. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T043 [P0/P1/P2] Baseline set/version/catalog/release/commit/IDs/digests affect idempotency; moving catalog after admission cannot change plan/result; legacy unpinned result is ineligible. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** request-hash pinning + fail-closed resolver landed offline (commits `83727aa7`/`bbbd4ccf`, `test_start_run_server_authority.py`); the caller-side published-catalog source is wired. Live pin-from-Gitea trace BLOCKED on a valid Gitea PAT (provided token rejected 401).
|
||||
- [ ] T044 [P0/P1/P2] GET/HEAD prove same ACL/status/headers; MIME/digest/length checked before bytes, cross-owner hidden, expired410, corrupt409, traversal/range/oversize rejected. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** Slice G GET/HEAD runtime landed and committed (`83727aa7`, `scenario_artifact_content.py`); live storage canary proved durable bytes with digests (canary v1 `597274d3`). Live ACL/status/header canary not yet exercised.
|
||||
- [ ] T045 [P0/P1/P2] Startup/readiness/start-loop and shutdown/drain/cancel/reconcile survive fault injection; unknown effect quarantines capacity; late response cannot win. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** startup/readiness live-verified (provider loop ready, readiness preflight; browser-probe deadlock fixed `ba2f1f45`); capacity leases claimed/released across the live canaries. Fault-injection drain/cancel/reconcile canary not yet exercised.
|
||||
- [ ] T046 [P0/P1/P2] End-to-end real browser→capture→durable artifact→deterministic comparison→optional immutable evaluation→policy→result; all required evidence present before PASS. Implement at the existing 044 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** the chain real browser→capture→durable artifact→immutable evaluation→policy→result is PROVEN live (canary v2 `4eebfab3`: `AgentEvaluation` persisted, DecisionPolicy row 11). Still missing: the deterministic comparison step in a live graph and a live baseline pin (Gitea PAT). Note: the canary's typed inconclusive outcome is the honest verdict while prompts remain text-only (images not attached).
|
||||
|
||||
Frontend boundary for this package: manual CRUD/editor, human review/approval, monitoring and read-only evidence/evaluation only; all agent interaction is external MCP. No agent chat/prompt/assistant editing/proposal generation/workspace/start/handoff controls. Runtime removal is OPEN, not performed by this spec refresh. Optional approved performance baseline is outside scope.
|
||||
|
||||
|
||||
@@ -70,7 +70,7 @@
|
||||
- [x] T029j Test-honesty remediation (ADR-0024 §3, binding rule): vertical/E2E tests obtain every prerequisite through a boundary the principal under test can reach; raw-ORM seeding of a chain prerequisite in a test claiming external reachability is forbidden. Executed: `test_mcp_initial_scenario_e2e.py::_pack()` replaced the raw `AgentRun(...)` insert with the production `create_agent_run` service boundary with honest test metadata; new `tests/test_mcp_agent_run_reachability.py` pins (a) the external-reachability REQUIREMENT as `strict=True` xfail (flips the suite red the moment `create_agent_run` lands without spec follow-through) and (b) the current typed zero-side-effect denial (`DRAFT_PACK_ACCESS_DENIED`, no handle rows). Evidence: see `TEST-001` row in spec.md release gates (targeted pytest green 2026-09-07). *(Update 2026-09-07, T029i closure: строгий xfail-пин сработал как спроектирован — конвертирован в обычный requirement-тест (unmarked, green) вместе с конвертацией вертикали в fully external chain; denial-pin сохранён.)*
|
||||
- [x] T029k Context-authority-derived capability map (MCPX-FR-028; 038 amendment 2026-09-07): server-owned derivation of `capabilities`/`has_dataset_fields` from the authoritative `DashboardQueryModel` (the same live inspection `context_authority` binds), environment policy and provider readiness — exposed through `inspect_dashboard_context` (derived-capability section) and consumed by the persisted compile boundaries (MCP `scenario_compile` + REST `api_compile_scenario`); caller-declared capabilities remain accepted only as an explicit subset narrowing, never as an authority that downgrades verifiable facts into `human_checkpoint`; unresolved facts stay `needs_context`/`needs_selector`/`needs_baseline`; genuinely unsafe PROD mutation contexts keep `human_checkpoint`; HumanStep revisions remain automation-ineligible (SCEX-FR-004a stands). Evidence: `CAP-001` row + capability-derivation unit/E2E tests. Доказательство (2026-09-07): новый `src/services/dashboard_testing/scenario/capability_authority.py` (236 LOC, `ScenarioGraph.CapabilityAuthority`): `derive_capabilities` — только верифицируемые факты (native_filters; text_filter по STRING-фильтру; time_rollover по DATE/TIME/TIME_GRAIN; table_filter+pagination по executable table-viz; xlsx_export из capabilities-флага; dataset_field_read+has_dataset_fields по accessible dataset с колонками; browser — только при readiness-факте из T040-снимка composition-root), `NEVER_DERIVED` = row_edit/bulk_edit/persistence_refresh/safe_test_data/safe_clock_fixture/cross_dashboard (unsafe-mutation автоматизация метаданными невозможна — module @INVARIANT); `merge_capabilities` — derived-wins в ОБЕ стороны (declared-false не понижает проверяемую правду; declared-true не фабрикует) + overrides-аудит; legacy/unparseable payload → дословный caller-declared passthrough (register-time context_authority остаётся жёстким гейтом). Wiring: MCP `inspect_scenario` (+additive `capability_authority` секция), `inspect_dashboard_context` (+`derived_capabilities`), REST `api_compile_scenario` (parity через общий `build_capability_authority` choke point + additive секция). Тесты: фикстура `query_model_sales.json` (форма sales-стенда); unit truth-table `tests/services/dashboard_testing/scenario/test_capability_authority.py` (7, включая **CAP-001 classification-fix через `map_all`**: полевого shape декларации → B01–B04/T01–T03 `automated`, C04–C06 `unsupported`, B05–B09/C01–C03/C02/C07 легитимно `human_checkpoint`); tool-level `tests/test_mcp_capability_authority.py` (3: derived-wins overrides `["text_filter","xlsx_export"]`, legacy caller_declared, derived_capabilities в inspect-ответе); REST-parity pin `test_scenario_routes.py::test_capability_authority_derived_wins`. Браузер-readiness seam запинен monkeypatch (детерминизм против глобального composition-root состояния в full-suite). Gate: 5-файловый срез `67 passed`; полный suite `11357 passed`.
|
||||
- [x] T029l Disposition vocabulary clarity (MCPX-FR-029; 044/045 amendment 2026-09-07): the 044 lifecycle mapping (`confirm`→`passed`, `false_positive`/`inconclusive`→`inconclusive`, aliases `pass`/`fail`) is immutable and stays; human-facing wording aligns to the persisted outcome — RU labels «Подтвердить соответствие» / «Проблема не подтверждена» / «Недостаточно данных» (+ EN parity), confirm-button restyled from `bg-destructive` to a positive token, `decide_checkpoint`/`decide_approval` MCP tool descriptions and 045 monitor docs carry the outcome table, vitest pins updated (`WaitingForMeView.test.ts`, `HumanCheckpointPanel` tests). Evidence: `DISP-001` row. Доказательство (2026-09-07): `waiting_disposition_confirm/false_positive` в ru/en `dashboard-testing.json` именуют персистентный исход (confirm→passed: «Подтвердить соответствие»/"Confirm conformance"; false_positive: «Проблема не подтверждена»/"Issue not confirmed"; inconclusive без изменений); confirm-кнопки `HumanCheckpointPanel.svelte` и `WaitingForMeView.svelte` переведены `bg-destructive` → `bg-primary` (прецедент approve-кнопки ApprovalDecisionPanel) + @RATIONALE в контрактах компонентов; API-вокабуляр не переименован (continuity аудита/CAS); MCP `decide_checkpoint` docstring (видим в tools/list) несёт immutable outcome-mapping и правило «confirm = проверка пройдена, никогда не подтверждение дефекта»; vitest-пины обновлены (`RunMonitorViews.test.ts` — новый DISP-001 pin лейбла+стиля+dispatch "confirm", `WaitingForMeView.test.ts`, `run.ux.test.ts`). Lifecycle-маппинг не менялся — backend-тесты decision-пути зелёные без правок. Gate: frontend `3507 passed` (206 файлов), lint `0 errors / 364 warnings` (baseline), `npm run build` OK.
|
||||
- [ ] T029m Live-stand replay of the field run: after T029i/T029k/T029l, re-run the 2026-09-07 sales scenario externally against the live stand — `inspect_dashboard_context` → derived-capability compile/validate → `create_agent_run` → `register_draft_pack` → `bootstrap_authoring_scenario` → `start_scenario_run` (PROD gate observed, no unexpected high-risk approvals) → `list_checkpoints`/`decide_checkpoint` human loop with aligned labels; retain the run report beside `docs/2026-09-07-sales-prod-mcp-run.md`. Evidence: `E2E-EXT-002` row.
|
||||
- [ ] T029m Live-stand replay of the field run: after T029i/T029k/T029l, re-run the 2026-09-07 sales scenario externally against the live stand — `inspect_dashboard_context` → derived-capability compile/validate → `create_agent_run` → `register_draft_pack` → `bootstrap_authoring_scenario` → `start_scenario_run` (PROD gate observed, no unexpected high-risk approvals) → `list_checkpoints`/`decide_checkpoint` human loop with aligned labels; retain the run report beside `docs/2026-09-07-sales-prod-mcp-run.md`. Evidence: `E2E-EXT-002` row. **Status (2026-09-10): OPEN.** The live stand is up with registered browser/screenshot providers and a server-resolved live binding, and the REST-only live chain is proven (canary v2 `4eebfab3`; see `docs/reports/agentic-runtime-live-canary-v2-2026-09-10.md`). Remaining for this task: the full MCP-external replay (`inspect_dashboard_context` → compile/validate → `create_agent_run` → `register_draft_pack` → `bootstrap_authoring_scenario` → `start_scenario_run` → checkpoint loop), plus the baseline pin once the Gitea PAT is valid.
|
||||
|
||||
## Phase 3 — Frontend decommission (flag-driven)
|
||||
|
||||
@@ -92,8 +92,8 @@ Historical [x] rows above retain only their dated local/transport evidence; they
|
||||
|
||||
Contract: [Contract-complete public parity](contracts/modules.md).
|
||||
|
||||
- [ ] T044 [P0/P1/P2] REST/MCP lifecycle/read/auth errors and disabled automation validation are identical; service principal cannot decide human gate. Implement at the existing 050 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T045 [P0/P1/P2] consume/publish failures return typed errors/pending state with no legacy fallback; every prerequisite is externally MCP-reachable. Implement at the existing 050 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T046 [P0/P1/P2] Fresh external-client chain preserves authoritative context and complete baseline pin without raw ORM/REST repair; no frontend agent controls/routes/requests. Implement at the existing 050 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md.
|
||||
- [ ] T044 [P0/P1/P2] REST/MCP lifecycle/read/auth errors and disabled automation validation are identical; service principal cannot decide human gate. Implement at the existing 050 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** offline parity tests green; live REST-only canary v2 exercised start→gate→approve→dispatch (`4eebfab3`). MCP-side live replay (T029m) still OPEN.
|
||||
- [ ] T045 [P0/P1/P2] consume/publish failures return typed errors/pending state with no legacy fallback; every prerequisite is externally MCP-reachable. Implement at the existing 050 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** the caller-side published-catalog source returns typed `None`/fail-closed (commit `bbbd4ccf`); Gitea publication path BLOCKED on a valid PAT. No live publish failure canary yet.
|
||||
- [ ] T046 [P0/P1/P2] Fresh external-client chain preserves authoritative context and complete baseline pin without raw ORM/REST repair; no frontend agent controls/routes/requests. Implement at the existing 050 domain boundary; verify with independent hardcoded fixtures and retain command/evidence references in traceability.md. **Status (2026-09-10):** a fresh REST client chain (login→start→gate→approve→dispatch) preserves the server-resolved binding and authoritative context live (canary v2 `4eebfab3`); the complete baseline pin is blocked on the Gitea PAT, and the MCP-external replay (T029m) is OPEN.
|
||||
|
||||
Frontend boundary for this package: manual CRUD/editor, human review/approval, monitoring and read-only evidence/evaluation only; all agent interaction is external MCP. No agent chat/prompt/assistant editing/proposal generation/workspace/start/handoff controls. Runtime removal is OPEN, not performed by this spec refresh. Optional approved performance baseline is outside scope.
|
||||
|
||||
Reference in New Issue
Block a user