test: integration + unit tests for cache fix and OUTPUT_SAFETY_FACTOR
## TestClassifyCacheSourceLang (8 tests) — regression for source-lang exclusion
- test_source_lang_in_tls_cache_all_translations: exact prod bug scenario
(detected_lang=ru, tls=[ru,en,fr,zh], cache={en,fr,zh} → pre_rows not LLM)
- test_source_lang_in_tls_cache_missing_one_translation: partial cache → LLM
- test_detected_und_tls_includes_und: und excluded correctly
- test_empty_detected_lang_no_exclusion: '' → no filtering
- test_source_lang_not_in_tls: no-op exclusion when lang not in tls
- test_multi_row_mixed_source_lang_cache: 4 rows, correct split
- test_all_same_lang_short_circuit: single target lang shortcut
- test_case_insensitive_detected_lang_matching: RU→ru
## TestBatchPipelineE2E (3 tests) — full pipeline integration
- test_warm_cache_full_pipeline_no_llm: 17 rows all cached → 0 LLM
- test_cold_cache_pipeline_all_llm: no cache → all LLM
- test_partial_cache_pipeline_split: 5 cached + 5 not → correct split
## TestOutputSafetyFactor (4 tests) — OUTPUT_SAFETY_FACTOR invariants
- test_output_safety_factor_not_above_070: ≤0.70 guard
- test_max_rows_by_output_qwen_flash_4langs: 14-24 range
- test_output_safety_factor_consistent_with_per_row: manual = actual
- test_single_lang_output_rows_above_20: ≥20 rows for 1 lang
## Semantic fixes
- Add missing #endregion TestClassifyCacheSourceLang
- Move floating @BRIEF tags inside their regions (REASONING_OVERHEAD, DICT_TOKENS_PER_ENTRY)
- Add @BRIEF to OUTPUT_PER_ROW_PER_LANG region
Verification: 705 tests pass (0 failures, 2 pre-existing warnings)
This commit is contained in:
@@ -17,7 +17,6 @@ include = ["src*"]
|
|||||||
[tool.pytest.ini_options]
|
[tool.pytest.ini_options]
|
||||||
pythonpath = ["."]
|
pythonpath = ["."]
|
||||||
asyncio_mode = "auto"
|
asyncio_mode = "auto"
|
||||||
testpaths = ["tests", "src/plugins"]
|
|
||||||
markers = [
|
markers = [
|
||||||
"integration: Integration tests requiring external services (Docker, Testcontainers, Superset). Use --run-integration to enable.",
|
"integration: Integration tests requiring external services (Docker, Testcontainers, Superset). Use --run-integration to enable.",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -394,6 +394,9 @@ class TestClassifyCacheSourceLang:
|
|||||||
# #endregion test_case_insensitive_detected_lang_matching
|
# #endregion test_case_insensitive_detected_lang_matching
|
||||||
|
|
||||||
|
|
||||||
|
# #endregion TestClassifyCacheSourceLang
|
||||||
|
|
||||||
|
|
||||||
# ── _persist_pre() Tests ───────────────────────────────────────────
|
# ── _persist_pre() Tests ───────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -24,9 +24,9 @@ DEFAULT_CONTEXT_WINDOW = 64000
|
|||||||
|
|
||||||
DEFAULT_MAX_OUTPUT_TOKENS = 16384
|
DEFAULT_MAX_OUTPUT_TOKENS = 16384
|
||||||
# #endregion DEFAULT_MAX_OUTPUT_TOKENS
|
# #endregion DEFAULT_MAX_OUTPUT_TOKENS
|
||||||
# @BRIEF CoT reasoning overhead tokens for DeepSeek models (~2000 tokens for chain-of-thought).
|
|
||||||
# #region REASONING_OVERHEAD [TYPE Constant]
|
# #region REASONING_OVERHEAD [TYPE Constant]
|
||||||
# @ingroup Translate
|
# @ingroup Translate
|
||||||
|
# @BRIEF CoT reasoning overhead tokens for DeepSeek models (~2000 tokens for chain-of-thought).
|
||||||
|
|
||||||
REASONING_OVERHEAD = 2000
|
REASONING_OVERHEAD = 2000
|
||||||
# #endregion REASONING_OVERHEAD
|
# #endregion REASONING_OVERHEAD
|
||||||
@@ -53,6 +53,7 @@ PROVIDER_DEFAULTS: dict[str, dict[str, int]] = {
|
|||||||
# #endregion PROVIDER_DEFAULTS
|
# #endregion PROVIDER_DEFAULTS
|
||||||
# #region OUTPUT_PER_ROW_PER_LANG [TYPE Constant]
|
# #region OUTPUT_PER_ROW_PER_LANG [TYPE Constant]
|
||||||
# @ingroup Translate
|
# @ingroup Translate
|
||||||
|
# @BRIEF Output tokens budgeted per row per target language for batch sizing.
|
||||||
|
|
||||||
# Increased from 120 to 200 because production logs show finish_reason=length
|
# Increased from 120 to 200 because production logs show finish_reason=length
|
||||||
# even with 120 tok/row/lang — long source texts produce long translations.
|
# even with 120 tok/row/lang — long source texts produce long translations.
|
||||||
@@ -71,9 +72,9 @@ JSON_OVERHEAD_PER_ROW = 50
|
|||||||
# Increased from 300 to 600 to account for longer template, system msg, and dict preamble.
|
# Increased from 300 to 600 to account for longer template, system msg, and dict preamble.
|
||||||
PROMPT_BASE_TOKENS = 600
|
PROMPT_BASE_TOKENS = 600
|
||||||
# #endregion PROMPT_BASE_TOKENS
|
# #endregion PROMPT_BASE_TOKENS
|
||||||
# @BRIEF Cap for dictionary tokens in estimation to avoid overestimating.
|
|
||||||
# #region DICT_TOKENS_PER_ENTRY [TYPE Constant]
|
# #region DICT_TOKENS_PER_ENTRY [TYPE Constant]
|
||||||
# @ingroup Translate
|
# @ingroup Translate
|
||||||
|
# @BRIEF Cap for dictionary tokens in estimation to avoid overestimating.
|
||||||
|
|
||||||
DICT_TOKENS_PER_ENTRY = 20
|
DICT_TOKENS_PER_ENTRY = 20
|
||||||
# #endregion DICT_TOKENS_PER_ENTRY
|
# #endregion DICT_TOKENS_PER_ENTRY
|
||||||
|
|||||||
Reference in New Issue
Block a user