busya
9922d7c87a
fix(translate,scheduler): language detection, LLM error handling, and scheduler persistence
Translation pipeline fixes (from production error log analysis):
Lang detect:
- Add "ru" to _COMMON_SOURCE_CODES for Cyrillic source detection
- Version detector cache keys (v2:) and include detector version in
source hash — stale cache entries from old algorithm are invalidated
- Integrate _character_block_fallback into batch_detect() pipeline;
only assign Cyrillic fallback when exactly one Cyrillic target exists
(multiple ru/uk/be targets stay undetermined for LLM arbitration)
Batch processing:
- Case-insensitive cache language matching (cached_by_lang lookup)
- Propagate needs_review=True for undetected language rows in pre/cache path
LLM call (critical — fixes silent error hiding):
- Validate LLM row IDs against expected set; log unknown identifiers
- Retry only missing rows on incomplete response (bounded by recursion depth)
- Exhausted retries → FAILED (new _handle_incomplete_response method)
- Parse failures → FAILED instead of SKIPPED (_handle_parse_failure)
- NULL/Empty translations → FAILED via new _add_failed helper
- Source-language identity mapping preserved as TranslationLanguage entry
(carries source_language_detected metadata that _build_insert_rows
relies on for detected_src_lang derivation)
- finish_reason propagated to parser for truncation diagnostics
LLM parse:
- Structured incomplete-set logging with expected/received/missing counts
Target schema validation:
- Handle all non-success SQL Lab statuses (failed, timeout, error, stopped)
- Timeout gets explicit message prefix
- Route returns HTTP 502 with correct HTTPException passthrough
Scheduler persistence fix (production pickle error):
- Backup and validation APScheduler jobs now use module-level callbacks
(execute_scheduled_backup, execute_scheduled_validation) instead of
bound SchedulerService methods — prevents pickle failure from
serializing TaskManager + dynamically loaded plugin classes
- Callback func identity verified via pickle round-trip smoke test
Tests:
- Update assertions: FAILED replaces SKIPPED for LLM error paths
- Restore source-language contract tests with identity-mapped values
- Add scheduler callback identity and args verification tests
- Update detector cache key tests for versioned format
- Update target schema error route test: 200→502
Orthogonal code review: MEDIUM finding (empty rec.languages when all
targets match detected source → _build_insert_rows fallback to "und")
fixed by preserving source-language TranslationLanguage entries.
2026-07-15 13:07:41 +03:00
..
2026-07-15 13:07:41 +03:00
2026-07-14 16:05:28 +03:00
2026-07-04 22:47:17 +03:00
2026-07-12 19:44:11 +03:00
2026-07-14 15:56:31 +03:00
2026-07-15 13:07:41 +03:00
2026-07-14 15:56:31 +03:00
2026-06-16 11:01:31 +03:00
2026-07-14 15:56:31 +03:00
2026-07-07 15:18:24 +03:00
2026-07-14 15:56:31 +03:00
2026-07-14 10:43:38 +03:00
2026-06-01 16:34:07 +03:00
2026-05-26 19:18:28 +03:00
2026-05-26 19:18:28 +03:00
2026-06-16 00:12:49 +03:00
2026-06-10 14:57:18 +03:00
2026-06-10 14:57:18 +03:00
2026-06-15 22:04:40 +03:00
2026-06-16 11:01:31 +03:00
2026-06-10 14:57:18 +03:00
2026-06-16 11:01:31 +03:00
2026-06-16 11:01:31 +03:00
2026-07-06 01:23:09 +03:00
2026-06-15 17:31:43 +03:00
2026-06-16 12:11:49 +03:00
2026-05-26 19:18:28 +03:00
2026-07-15 13:07:41 +03:00
2026-06-16 12:01:03 +03:00
2026-06-05 15:43:35 +03:00
2026-06-16 12:11:49 +03:00
2026-06-16 12:11:49 +03:00
2026-07-06 01:23:09 +03:00
2026-05-26 19:18:28 +03:00
2026-07-12 19:30:57 +03:00
2026-05-26 19:18:28 +03:00
2026-06-15 19:31:56 +03:00
2026-07-12 19:30:57 +03:00
2026-05-26 19:18:28 +03:00
2026-06-16 00:12:49 +03:00
2026-06-05 15:43:35 +03:00
2026-06-15 17:31:43 +03:00
2026-06-16 11:01:31 +03:00
2026-06-15 17:31:43 +03:00
2026-06-15 16:45:49 +03:00
2026-06-15 17:31:43 +03:00
2026-07-02 08:53:19 +03:00
2026-06-15 17:31:43 +03:00
2026-06-16 12:01:03 +03:00
2026-06-16 12:11:49 +03:00
2026-05-26 19:18:28 +03:00
2026-06-18 23:54:57 +03:00
2026-06-16 12:01:03 +03:00
2026-05-26 19:18:28 +03:00
2026-07-06 01:23:09 +03:00
2026-07-12 19:30:57 +03:00
2026-07-14 17:33:16 +03:00
2026-06-16 12:01:03 +03:00
2026-06-16 00:12:49 +03:00
2026-06-16 00:12:49 +03:00
2026-05-26 19:18:28 +03:00
2026-07-14 15:56:31 +03:00
2026-07-12 15:31:56 +03:00
2026-06-15 19:31:56 +03:00
2026-06-11 19:12:45 +03:00
2026-05-26 19:18:28 +03:00
2026-06-11 19:12:45 +03:00
2026-06-11 19:12:45 +03:00
2026-07-12 19:30:57 +03:00
2026-07-12 19:30:57 +03:00
2026-07-12 19:30:57 +03:00
2026-06-15 22:04:40 +03:00