feat(translate): replace LLM-based language detection with local lingua-detector
- Add _lang_detect.py module wrapping lingua-language-detector (pure Python, 0.076ms/call) - Add batch_detect() with detector caching for performance - Pre-filter rows where detected source language matches target (same-language skip) - Remove detected_source_language from LLM prompt templates (token savings) - Use local detection for cached rows (was always 'und') - Preserve backward compat: LLM detected_source_language as fallback when lingua returns 'und' - Add 24 unit tests for detection, edge cases, caching, and mapping
This commit is contained in:
@@ -58,3 +58,4 @@ Pillow
|
||||
# скачивается лениво при первом запуске контейнера через entrypoint
|
||||
playwright
|
||||
ruff>=0.11.0
|
||||
lingua-language-detector==2.2.0
|
||||
|
||||
Reference in New Issue
Block a user