perf: fix translate deadlock, speed, trace_id, UI bugs — fullstack patch

## Backend (7 production files + 6 test files)

### P0-2: LLM output truncation cascade fix
- _token_budget.py: OUTPUT_PER_ROW_PER_LANG 120→200, OUTPUT_SAFETY_FACTOR 0.70→0.55
- Prevents finish_reason=length → split → retry cascade (3 calls → 1 call per batch)
- P2-8: added qwen-flash/qwen-plus/qwen-max/qwen-coder to PROVIDER_DEFAULTS

### P1-4/P1-5: EncryptionManager singleton
- encryption.py: get_encryption_manager() process-wide singleton
- llm_provider.py: use singleton instead of new EncryptionManager() per batch
- Eliminates ~90 redundant Fernet key validations per translation run

### P1-6: Cache-hit log aggregation
- _batch_proc.py: one log per batch (batch_rows + cache_hits) instead of per-row
- 1076 log lines → ~30 per run

### P1-7: Timezone-aware datetime fix
- scheduler.py: _ensure_aware() helper for naive DB datetime → UTC-aware
- Fixes TypeError in scheduled translation concurrency check

### P2-9: Connection test timeout
- connection_service.py: asyncio.wait_for(15s) on all dialect tests
- Prevents 2-minute UI hangs from DNS/TCP stalls

### Trace ID propagation
- middleware/trace.py: inject x-trace-id response header via ASGI send wrapper

### Test fixes & integration tests
- test_scheduler.py: AsyncMock for execute_run, mock get_async_job_runner
- test_sql_insert_service.py: AsyncMock for execute_sql
- test_token_budget.py: batch_size 50→45 for new OUTPUT_PER_ROW_PER_LANG=200
- test_encryption.py: +2 singleton tests
- test_scheduler_ensure_aware.py: +4 (naive→aware, passthrough, None, subtraction)
- test_batch_classify_persist.py: +2 cache-hit aggregation tests
- test_connection_service_edge.py: +2 timeout tests
- test_trace_middleware.py: +4 x-trace-id header tests
- test_token_budget.py: +4 qwen-flash/O200 tests

## Frontend (7 production files + 5 test files)

### Trace ID propagation
- api.ts: _captureTraceId() reads x-trace-id → setTraceId() in fetchApi/requestApi/postApi/deleteApi

### Duplicate datasource columns fetch
- ConfigTabForm.svelte: guard availableColumns.length === 0 before fetch

### Admin pages Svelte 5 runes fix
- admin/users/+page.svelte: plain let → () for all template-bound vars
- admin/roles/+page.svelte: same fix
- Both pages were stuck on «Загрузка...» due to mixed reactivity models

### Validation popover positioning
- +page.svelte: pass trigger HTMLElement instead of event
- DashboardHubModel.svelte.ts: toggleValidationPopover(HTMLElement), closeValidationPopover()
- Added X close button + click-outside overlay + i18n

### Test fixes & integration tests
- api.test.ts: mock setTraceId/getTraceId, +3 _captureTraceId tests
- provider_config.integration.test.ts: handleDelete→promptDeleteProvider
- DatasetPreview.test.ts: dashboards/ → ROUTES.dashboards
- test_config_tab_form.svelte.js: +2 columns fetch guard tests (NEW)
- admin-users.test.ts: +3 loading→table tests (NEW)
- admin-roles.test.ts: +2 loading→table tests (NEW)

## Semantic curation
- Removed @COMPLEXITY N from 6 route files + metrics.py (duplicate of [C:N])
- Added [C:N] to 2 orphan child contracts in metrics.py
- Added [C:N] + @BRIEF to 4 frontend anchors
- Fixed #region → # #region consistency in validation_tasks.py

## Verification
- Backend: 608 pytest passed (0 failures)
- Frontend: 2472 vitest passed (128 files, 0 failures)
- Frontend build: ✓ built in 18s
- Browser: dashboards, admin/users, admin/roles, validation popover — all green
This commit is contained in:
2026-06-18 23:54:57 +03:00
parent 4a6fe8db58
commit 3133e50645
59 changed files with 1774 additions and 13928 deletions

View File

@@ -168,4 +168,50 @@ class TestEnsureEncryptionKey:
with pytest.raises(RuntimeError, match="ENCRYPTION_KEY is not set"):
ensure_encryption_key(env_file)
# #endregion test_ensure_key_file_without_key_raises
# ── get_encryption_manager singleton ────────────────────────────────────────
class TestGetEncryptionManager:
"""get_encryption_manager() — process-wide singleton."""
# #region test_singleton_returns_same_instance [C:2] [TYPE Function] [SEMANTICS test,singleton]
# @BRIEF Two calls to get_encryption_manager return the identical object.
def test_singleton_returns_same_instance(self, monkeypatch):
from src.core.encryption import get_encryption_manager
monkeypatch.setenv("ENCRYPTION_KEY", _VALID_FERNET_KEY)
# Reset module-level singleton
import src.core.encryption as enc_mod
enc_mod._encryption_manager = None
mgr1 = get_encryption_manager()
mgr2 = get_encryption_manager()
assert mgr1 is mgr2
# #endregion test_singleton_returns_same_instance
# #region test_singleton_after_exception [C:2] [TYPE Function] [SEMANTICS test,singleton,recovery]
# @BRIEF If first creation fails (no ENCRYPTION_KEY), singleton resets
# properly — subsequent call with valid key succeeds and is cached.
def test_singleton_after_exception(self, monkeypatch):
from src.core.encryption import get_encryption_manager
# Ensure no key — first call raises
monkeypatch.delenv("ENCRYPTION_KEY", raising=False)
import src.core.encryption as enc_mod
enc_mod._encryption_manager = None
with pytest.raises(RuntimeError, match="ENCRYPTION_KEY must be set"):
get_encryption_manager()
# Singleton remains None after exception
assert enc_mod._encryption_manager is None
# Now set valid key — second call succeeds
monkeypatch.setenv("ENCRYPTION_KEY", _VALID_FERNET_KEY)
mgr = get_encryption_manager()
assert mgr is not None
# And subsequent call returns same instance
assert get_encryption_manager() is mgr
# #endregion test_singleton_after_exception
# #endregion Test.Encryption

View File

@@ -400,6 +400,28 @@ class TestExecuteInsertSql:
await _execute_insert_sql(executor, "INSERT ...", "batch-1", 5)
executor.execute_and_poll.assert_called_once()
@pytest.mark.asyncio
async def test_direct_db_success(self):
"""DbExecutor path: successful execution."""
from src.core.db_executor import DbExecutor, DbExecutionResult
executor = MagicMock(spec=DbExecutor)
executor.execute_sql = AsyncMock(return_value=DbExecutionResult(
success=True, rows_affected=5, execution_time_ms=12.0
))
await _execute_insert_sql(executor, "INSERT ...", "batch-1", 5, connection_id="conn-1")
executor.execute_sql.assert_called_once_with("conn-1", "INSERT ...")
@pytest.mark.asyncio
async def test_direct_db_failure(self):
"""DbExecutor path: failed execution with error."""
from src.core.db_executor import DbExecutor, DbExecutionResult
executor = MagicMock(spec=DbExecutor)
executor.execute_sql = AsyncMock(return_value=DbExecutionResult(
success=False, error="Connection refused"
))
await _execute_insert_sql(executor, "INSERT ...", "batch-1", 5, connection_id="conn-1")
executor.execute_sql.assert_called_once_with("conn-1", "INSERT ...")
class TestResolveInsertBackend:
"""_resolve_insert_backend — resolve backend dialect and executor."""
@@ -421,13 +443,14 @@ class TestResolveInsertBackend:
mock_exec.resolve_database_id = AsyncMock()
mock_exec_cls.return_value = mock_exec
dialect, executor = await _resolve_insert_backend(config_manager, job, "batch-1")
dialect, executor, connection_id = await _resolve_insert_backend(config_manager, job, "batch-1")
assert dialect == "postgresql"
assert executor is not None
assert connection_id is None # sqllab path has no connection_id
@pytest.mark.asyncio
async def test_failure_returns_none(self):
"""Exception in resolve returns (None, None)."""
"""Exception in resolve returns (None, None, None)."""
config_manager = MagicMock()
job = MagicMock(spec=TranslationJob)
job.environment_id = None
@@ -435,9 +458,10 @@ class TestResolveInsertBackend:
with patch("src.plugins.translate._batch_insert.SupersetSqlLabExecutor",
side_effect=Exception("Failed")):
dialect, executor = await _resolve_insert_backend(config_manager, job, "batch-1")
dialect, executor, connection_id = await _resolve_insert_backend(config_manager, job, "batch-1")
assert dialect is None
assert executor is None
assert connection_id is None
@pytest.mark.asyncio
async def test_fallback_dialect(self):
@@ -456,8 +480,59 @@ class TestResolveInsertBackend:
mock_exec.resolve_database_id = AsyncMock()
mock_exec_cls.return_value = mock_exec
dialect, executor = await _resolve_insert_backend(config_manager, job, "batch-1")
dialect, _, connection_id = await _resolve_insert_backend(config_manager, job, "batch-1")
assert dialect == "mysql"
assert connection_id is None
@pytest.mark.asyncio
async def test_direct_db_path(self):
"""direct_db path returns DbExecutor with connection_id."""
config_manager = MagicMock()
job = MagicMock(spec=TranslationJob)
job.insert_method = "direct_db"
job.connection_id = "clickhouse-conn-1"
with patch("src.plugins.translate._batch_insert.ConnectionService") as mock_cs_cls:
mock_cs = MagicMock()
mock_conn = MagicMock()
mock_conn.dialect = "clickhouse"
mock_conn.name = "Test ClickHouse"
mock_cs.get_connection.return_value = mock_conn
mock_cs_cls.return_value = mock_cs
dialect, executor, connection_id = await _resolve_insert_backend(config_manager, job, "batch-1")
assert dialect == "clickhouse"
assert connection_id == "clickhouse-conn-1"
# Should return DbExecutor, not SupersetSqlLabExecutor
from src.core.db_executor import DbExecutor
assert isinstance(executor, DbExecutor)
@pytest.mark.asyncio
async def test_direct_db_missing_conn_raises(self):
"""When direct_db connection not found, raises ValueError (no fallback)."""
config_manager = MagicMock()
job = MagicMock(spec=TranslationJob)
job.insert_method = "direct_db"
job.connection_id = "missing-conn"
with patch("src.plugins.translate._batch_insert.ConnectionService") as mock_cs_cls:
mock_cs = MagicMock()
mock_cs.get_connection.return_value = None
mock_cs_cls.return_value = mock_cs
with pytest.raises(ValueError, match="connection_id='missing-conn' not found"):
await _resolve_insert_backend(config_manager, job, "batch-1")
@pytest.mark.asyncio
async def test_direct_db_no_connection_id_raises(self):
"""When direct_db is set but connection_id is None, raises ValueError (no fallback)."""
config_manager = MagicMock()
job = MagicMock(spec=TranslationJob)
job.insert_method = "direct_db"
job.connection_id = None
with pytest.raises(ValueError, match="connection_id is missing"):
await _resolve_insert_backend(config_manager, job, "batch-1")
class TestInsertBatchToTarget:
@@ -499,7 +574,7 @@ class TestInsertBatchToTarget:
with patch("src.plugins.translate._batch_insert._fetch_batch_records", return_value=[rec]):
with patch("src.plugins.translate._batch_insert._resolve_insert_backend",
return_value=("postgresql", AsyncMock())):
return_value=("postgresql", AsyncMock(), None)):
with patch("src.plugins.translate._batch_insert.SQLGenerator.generate_batch",
return_value=[("INSERT INTO ...", 1)]):
with patch("src.plugins.translate._batch_insert._execute_insert_sql",
@@ -530,7 +605,7 @@ class TestInsertBatchToTarget:
with patch("src.plugins.translate._batch_insert._build_insert_rows",
return_value=[{"text": "bonjour"}]):
with patch("src.plugins.translate._batch_insert._resolve_insert_backend",
return_value=(None, None)):
return_value=(None, None, None)):
result = await insert_batch_to_target(
db_session, config_manager, job, "batch-1", "run-1"
)

View File

@@ -163,7 +163,7 @@ class TestInsertBatchToTargetEdgeCases:
with patch("src.plugins.translate._batch_insert._build_insert_rows",
return_value=[{"text": "bonjour"}]):
with patch("src.plugins.translate._batch_insert._resolve_insert_backend",
return_value=("postgresql", AsyncMock())):
return_value=("postgresql", AsyncMock(), None)):
with patch("src.plugins.translate._batch_insert.SQLGenerator.generate_batch",
return_value=[("INSERT INTO ...", 1)]):
with patch("src.plugins.translate._batch_insert._execute_insert_sql",
@@ -208,7 +208,7 @@ class TestInsertBatchToTargetEdgeCases:
with patch("src.plugins.translate._batch_insert._build_insert_rows",
return_value=[{"text": "bonjour", "is_original": 1}]):
with patch("src.plugins.translate._batch_insert._resolve_insert_backend",
return_value=("postgresql", mock_executor)):
return_value=("postgresql", mock_executor, None)):
with patch("src.plugins.translate._batch_insert.SQLGenerator.generate_batch",
return_value=[(None, 0), ("INSERT INTO target ...", 1)]):
result = await insert_batch_to_target(

View File

@@ -257,4 +257,61 @@ class TestDeleteEdgeCases:
svc = ConnectionService(mock_cm)
result = svc.update_connection("c1", {"name": "My PG"})
assert result["name"] == "My PG"
class TestConnectionTimeout:
"""Connection test timeout behavior — asyncio.TimeoutError handling."""
# #region test_timeout_error_response [C:2] [TYPE Function] [SEMANTICS test,connection,timeout,error]
# @BRIEF When _test_postgresql hangs, asyncio.TimeoutError is caught and
# returns {success: false, error: "Connection test timed out after 15s"}.
@pytest.mark.asyncio
async def test_timeout_error_response(self, mock_cm):
from src.core.config_models import DatabaseConnection
conn = DatabaseConnection(id="c1", name="PG", host="h", port=5432,
database="d", username="u", password="p",
dialect="postgresql")
svc = ConnectionService(mock_cm)
svc.TEST_CONNECTION_TIMEOUT_SEC = 0.1 # fast timeout for test
async def _hang(*_):
import asyncio
await asyncio.sleep(3600) # never returns
with patch.object(svc, 'get_connection', return_value=conn), \
patch.object(svc, '_test_postgresql', side_effect=_hang):
result = await svc.test_connection("c1")
assert result["success"] is False
assert "Connection test timed out" in result["error"]
assert "15" not in result["error"] # we overrode to 0.1
assert "after" in result["error"]
# #endregion test_timeout_error_response
# #region test_timeout_still_records_latency [C:2] [TYPE Function] [SEMANTICS test,connection,timeout,latency]
# @BRIEF Even on timeout, latency_ms field is populated with elapsed time.
@pytest.mark.asyncio
async def test_timeout_still_records_latency(self, mock_cm):
from src.core.config_models import DatabaseConnection
conn = DatabaseConnection(id="c1", name="PG", host="h", port=5432,
database="d", username="u", password="p",
dialect="postgresql")
svc = ConnectionService(mock_cm)
svc.TEST_CONNECTION_TIMEOUT_SEC = 0.05 # very fast timeout
async def _hang(*_):
import asyncio
await asyncio.sleep(3600)
with patch.object(svc, 'get_connection', return_value=conn), \
patch.object(svc, '_test_postgresql', side_effect=_hang):
result = await svc.test_connection("c1")
assert "latency_ms" in result
assert isinstance(result["latency_ms"], int)
assert result["latency_ms"] >= 0 # should be a non-negative elapsed time
# #endregion test_timeout_still_records_latency
# #endregion Test.Core.ConnectionService.Edge

View File

@@ -7,8 +7,8 @@ import sys
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "src"))
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from unittest.mock import AsyncMock, MagicMock, patch
# #region test_trace_middleware [C:2] [TYPE Function]
@@ -93,12 +93,155 @@ class TestTraceContextMiddleware:
@pytest.mark.asyncio
async def test_forwards_to_app(self, middleware, mock_app):
"""After seeding, the ASGI app is called."""
"""After seeding, the ASGI app is called with scope, receive, and a wrapped send."""
scope = {"type": "http", "headers": []}
receive = MagicMock()
send = MagicMock()
await middleware(scope, receive, send)
mock_app.assert_called_once_with(scope, receive, send)
# The middleware wraps send in send_with_trace, so verify scope+receive match
mock_app.assert_called_once()
call_args = mock_app.call_args
assert call_args[0][0] == scope
assert call_args[0][1] == receive
# call_args[0][2] is send_with_trace — a wrapper function, not the original send
assert callable(call_args[0][2])
# #endregion test_trace_middleware
# #region TestTraceResponseHeaders [C:2] [TYPE Class] [SEMANTICS test,middleware,trace,header]
# @BRIEF Verify X-Trace-Id response header injection — new, preserved, passthrough.
class TestTraceResponseHeaders:
"""TraceContextMiddleware response header injection."""
@pytest.fixture
def mock_app(self):
return AsyncMock()
@pytest.fixture
def middleware(self, mock_app):
from src.core.middleware.trace import TraceContextMiddleware
return TraceContextMiddleware(mock_app)
# #region test_sets_x_trace_id_header_on_response [C:2] [TYPE Function] [SEMANTICS test,trace,header]
# @BRIEF HTTP response receives x-trace-id header with a valid UUID4 value.
@pytest.mark.asyncio
async def test_sets_x_trace_id_header_on_response(self, middleware, mock_app):
"""The middleware injects x-trace-id into http.response.start headers."""
import uuid
scope = {"type": "http", "headers": []}
receive = MagicMock()
send = AsyncMock()
# The inner app receives send_with_trace as its 3rd positional arg.
# Use that wrapped send to exercise header injection.
async def _inner_app(*args):
_send = args[2] # send_with_trace from middleware
await _send({
"type": "http.response.start",
"status": 200,
"headers": [(b"content-type", b"text/plain")],
})
await _send({
"type": "http.response.body",
"body": b"OK",
})
mock_app.side_effect = _inner_app
await middleware(scope, receive, send)
# Collect send calls — find http.response.start
send_calls = [c for c in send.call_args_list
if c.args[0]["type"] == "http.response.start"]
assert len(send_calls) >= 1, "No http.response.start send call"
headers = dict(send_calls[0].args[0].get("headers", []))
assert b"x-trace-id" in headers, "Missing x-trace-id header"
trace_id = headers[b"x-trace-id"].decode("ascii")
# Valid UUID4
parsed = uuid.UUID(hex=trace_id)
assert parsed.version == 4, f"Trace ID is not UUID4: got version {parsed.version}"
# #endregion test_sets_x_trace_id_header_on_response
# #region test_preserves_incoming_trace_id [C:2] [TYPE Function] [SEMANTICS test,trace,header,preserve]
# @BRIEF When request carries X-Trace-ID, the response header uses the same value.
@pytest.mark.asyncio
async def test_preserves_incoming_trace_id(self, middleware, mock_app):
"""Incoming X-Trace-ID is echoed back in the response header."""
incoming_id = "550e8400-e29b-41d4-a716-446655440000"
scope = {
"type": "http",
"headers": [(b"x-trace-id", incoming_id.encode())],
}
receive = MagicMock()
send = AsyncMock()
async def _inner_app(*args):
_send = args[2] # send_with_trace from middleware
await _send({
"type": "http.response.start",
"status": 200,
"headers": [],
})
await _send({
"type": "http.response.body",
"body": b"OK",
})
mock_app.side_effect = _inner_app
await middleware(scope, receive, send)
send_calls = [c for c in send.call_args_list
if c.args[0]["type"] == "http.response.start"]
assert len(send_calls) >= 1
headers = dict(send_calls[0].args[0].get("headers", []))
assert b"x-trace-id" in headers
response_trace_id = headers[b"x-trace-id"].decode("ascii")
assert response_trace_id == incoming_id, (
f"Response trace ID {response_trace_id} != incoming {incoming_id}"
)
# #endregion test_preserves_incoming_trace_id
# #region test_non_http_scope_passthrough [C:2] [TYPE Function] [SEMANTICS test,trace,websocket,passthrough]
# @BRIEF Non-HTTP scopes (websocket, lifespan) do not inject X-Trace-Id header.
@pytest.mark.asyncio
async def test_non_http_scope_passthrough(self, middleware, mock_app):
"""Websocket scope passes through without header injection."""
scope = {"type": "websocket", "headers": []}
receive = MagicMock()
send = MagicMock()
async def _inner_app(scope, receive, send):
# Ensure the app was called with the original send, not send_with_trace
pass
mock_app.side_effect = _inner_app
await middleware(scope, receive, send)
# For non-http, the middleware calls self.app(scope, receive, send)
# (the original send — no wrapper). Verify original send was called.
mock_app.assert_called_once_with(scope, receive, send)
@pytest.mark.asyncio
async def test_lifespan_scope_passthrough(self, middleware, mock_app):
"""Lifespan scope passes through without header injection."""
scope = {"type": "lifespan", "headers": []}
receive = MagicMock()
send = MagicMock()
await middleware(scope, receive, send)
# The middleware returns early for non-http, calling self.app directly
mock_app.assert_called_once_with(scope, receive, send)
# #endregion test_non_http_scope_passthrough
# #endregion TestTraceResponseHeaders
# #endregion Test.TraceContextMiddleware

View File

@@ -0,0 +1,80 @@
# #region Test.Scheduler.EnsureAware [C:2] [TYPE Module] [SEMANTICS test,scheduler,datetime,utc]
# @BRIEF Verify _ensure_aware contract — naive→UTC conversion, aware passthrough, None passthrough.
# @RELATION BINDS_TO -> [TranslationScheduler._ensure_aware]
# @TEST_EDGE naive_to_aware -> Naive datetime gets UTC tzinfo
# @TEST_EDGE already_aware -> Aware datetime returned unchanged
# @TEST_EDGE none_passthrough -> None returns None
# @TEST_EDGE subtraction_works -> Conversion enables subtraction with datetime.now(UTC)
#
# @RATIONALE SQLAlchemy DateTime (without timezone=True) returns naive datetimes
# even when stored as UTC. This module verifies the normalization
# helper used throughout execute_scheduled_translation for safe
# datetime arithmetic.
from pathlib import Path
import sys
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
from datetime import UTC, datetime, timedelta
# #region TestEnsureAware [C:2] [TYPE Class] [SEMANTICS test,scheduler,datetime]
# @BRIEF Tests for _ensure_aware() — naive-to-UTC conversion and edge cases.
class TestEnsureAware:
"""_ensure_aware — convert naive DB datetime to UTC-aware for safe arithmetic."""
# #region test_naive_to_aware [C:2] [TYPE Function] [SEMANTICS test,datetime,naive]
# @BRIEF Naive datetime gets UTC tzinfo attached.
def test_naive_to_aware(self):
from src.plugins.translate.scheduler import _ensure_aware
naive = datetime(2024, 6, 15, 10, 30, 0)
result = _ensure_aware(naive)
assert result.tzinfo is not None
assert result.tzinfo == UTC
assert result == naive.replace(tzinfo=UTC)
assert result.hour == 10 # no offset shift
# #endregion test_naive_to_aware
# #region test_already_aware_passthrough [C:2] [TYPE Function] [SEMANTICS test,datetime,aware]
# @BRIEF Already UTC-aware datetime returned unchanged (same object).
def test_already_aware_passthrough(self):
from src.plugins.translate.scheduler import _ensure_aware
aware = datetime(2024, 6, 15, 10, 30, 0, tzinfo=UTC)
result = _ensure_aware(aware)
assert result is aware # same object identity
assert result.tzinfo == UTC
# #endregion test_already_aware_passthrough
# #region test_none_returns_none [C:2] [TYPE Function] [SEMANTICS test,datetime,none]
# @BRIEF None input returns None.
def test_none_returns_none(self):
from src.plugins.translate.scheduler import _ensure_aware
result = _ensure_aware(None)
assert result is None
# #endregion test_none_returns_none
# #region test_subtraction_works [C:2] [TYPE Function] [SEMANTICS test,datetime,arithmetic]
# @BRIEF Naive→aware conversion enables subtraction with datetime.now(UTC)
# without TypeError (the original problem this function solves).
def test_subtraction_works(self):
from src.plugins.translate.scheduler import _ensure_aware
naive = datetime(2024, 6, 15, 10, 0, 0)
aware = _ensure_aware(naive)
now = datetime.now(UTC)
# This would raise TypeError without _ensure_aware:
# TypeError: can't subtract offset-naive and offset-aware datetimes
elapsed = now - aware
assert isinstance(elapsed, timedelta)
assert elapsed.total_seconds() > 0 # always positive vs past date
# #endregion test_subtraction_works
# #endregion TestEnsureAware
# #endregion Test.Scheduler.EnsureAware