feat(translate,scheduler,trace): new-key-only mode, stale PENDING protection, trace_id propagation

Translations:
- FR-045: new-key-only execution mode — translate only rows with unseen keys
- Compare source row keys against TranslationRecord.source_data from last succeeded run
- Baseline expired (>90 days) fallback to full mode with baseline_expired event
- run_noop early return when zero new keys (skip LLM + SQL)

Scheduler reliability:
- Stale PENDING protection: runs older than 1h auto-marked FAILED, no longer block schedule
- load_schedules() reloads active translation schedules from DB on restart
- add_translation_job/remove_translation_job register/unregister with APScheduler
- execution_mode column on translation_schedules with additive DB migration

Automation view:
- GET /settings/automation/translation-schedules endpoint
- Translation schedules displayed on Automation page below validation policies

Trace propagation:
- seed_trace_id() in all background entry points:
  TaskManager._run_task/_flusher_loop, SchedulerService._trigger_backup,
  websocket_endpoint, IdMappingService, all standalone scripts

Migrations:
- dictionary_entries: origin_run_id, origin_row_key, origin_user_id
- translation_schedules: execution_mode

Tests:
- 3 new test modules (22 tests): core_scheduler, executor_filter, scheduler_execution+guard
- Fix pre-existing translate test isolation (conftest.py)
- Fix test_list_runs_filter_status parameter
This commit is contained in:
2026-05-14 10:13:56 +03:00
parent d1695fe536
commit 8d0bc6fb93
26 changed files with 1525 additions and 19 deletions

View File

@@ -579,6 +579,96 @@ def _ensure_dataset_review_session_columns(bind_engine):
# #endregion _ensure_dataset_review_session_columns
# #region _ensure_translation_schedules_columns [C:3] [TYPE Function]
# @BRIEF Applies additive schema upgrades for translation_schedules table.
# @PRE: bind_engine points to application database.
# @POST: Missing columns are added without data loss.
# @RELATION DEPENDS_ON -> [engine]
def _ensure_translation_schedules_columns(bind_engine):
with belief_scope("_ensure_translation_schedules_columns"):
table_name = "translation_schedules"
inspector = inspect(bind_engine)
if table_name not in inspector.get_table_names():
return
existing_columns = {
str(column.get("name") or "").strip()
for column in inspector.get_columns(table_name)
}
alter_statements = []
if "execution_mode" not in existing_columns:
alter_statements.append(
"ALTER TABLE translation_schedules "
"ADD COLUMN execution_mode VARCHAR NOT NULL DEFAULT 'full'"
)
if not alter_statements:
return
try:
with bind_engine.begin() as connection:
for statement in alter_statements:
connection.execute(text(statement))
except Exception as migration_error:
logger.explore(
"TranslationSchedule additive migration failed",
extra={"src": "database", "error": str(migration_error)},
)
# #endregion _ensure_translation_schedules_columns
# #region _ensure_dictionary_entries_columns [C:3] [TYPE Function]
# @BRIEF Additive migration for dictionary_entries origin tracking columns.
# @RELATION DEPENDS_ON -> [engine]
def _ensure_dictionary_entries_columns(bind_engine):
with belief_scope("_ensure_dictionary_entries_columns"):
table_name = "dictionary_entries"
inspector = inspect(bind_engine)
if table_name not in inspector.get_table_names():
return
existing_columns = {
str(column.get("name") or "").strip()
for column in inspector.get_columns(table_name)
}
alter_statements = []
if "origin_run_id" not in existing_columns:
alter_statements.append(
"ALTER TABLE dictionary_entries "
"ADD COLUMN origin_run_id VARCHAR"
)
if "origin_row_key" not in existing_columns:
alter_statements.append(
"ALTER TABLE dictionary_entries "
"ADD COLUMN origin_row_key VARCHAR"
)
if "origin_user_id" not in existing_columns:
alter_statements.append(
"ALTER TABLE dictionary_entries "
"ADD COLUMN origin_user_id VARCHAR"
)
if not alter_statements:
return
try:
with bind_engine.begin() as connection:
for statement in alter_statements:
connection.execute(text(statement))
except Exception as migration_error:
logger.explore(
"DictionaryEntry additive migration failed",
extra={"src": "database", "error": str(migration_error)},
)
# #endregion _ensure_dictionary_entries_columns
# #region init_db [C:3] [TYPE Function]
# @BRIEF Initializes the database by creating all tables.
# @PRE: engine, tasks_engine and auth_engine are initialized.
@@ -587,6 +677,7 @@ def _ensure_dataset_review_session_columns(bind_engine):
# @RELATION CALLS -> [ensure_connection_configs_table]
# @RELATION CALLS -> [_ensure_filter_source_enum_values]
# @RELATION CALLS -> [_ensure_dataset_review_session_columns]
# @RELATION CALLS -> [_ensure_dictionary_entries_columns]
def init_db():
with belief_scope("init_db"):
Base.metadata.create_all(bind=engine)
@@ -601,6 +692,8 @@ def init_db():
_ensure_filter_source_enum_values(engine)
_ensure_dataset_review_session_columns(engine)
_ensure_translation_jobs_columns(engine)
_ensure_translation_schedules_columns(engine)
_ensure_dictionary_entries_columns(engine)
# #endregion init_db