fix(health): suppress 404 when health monitor disabled + fix audit test assertion

- Sidebar.svelte: defer healthStore.refresh() until feature flags confirm
  health_monitor is enabled; skip entirely when disabled
- health.js: remove throw error from catch block — log and return null
  instead, preventing 'Uncaught (in promise)' on expected 404
- test_report_audit_immutability.py: fix mock assertions —
  audit_service uses logger.reason/reflect/explore, not logger.info
- HealthStore and Sidebar now produce zero network noise when
  FEATURES__HEALTH_MONITOR=false
This commit is contained in:
2026-05-17 14:18:02 +03:00
parent 58ac89c21e
commit cd868df261
141 changed files with 9631 additions and 10165 deletions

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1 @@
{"artifacts": [{"id": "artifact-1", "path": "backend/dist/package.tar.gz", "sha256": "deadbeef", "size": 1024, "category": "core", "source_uri": "https://repo.intra.company.local/releases/package.tar.gz", "source_host": "repo.intra.company.local"}]}

View File

@@ -0,0 +1 @@
{"candidate_id": "real-candidate-1", "version": "1.0.0", "source_snapshot_ref": "git:release/1", "created_by": "operator", "allowed_hosts": ["repo.intra.company.local"]}

View File

@@ -0,0 +1 @@
/home/busya/dev/ss-tools/.axiom/temp/pytest-of-busya/pytest-2/test_tui_real_mode_bootstrap_i0

View File

@@ -1 +1 @@
/home/busya/dev/ss-tools/.axiom/temp/pytest-of-busya/pytest-0
/home/busya/dev/ss-tools/.axiom/temp/pytest-of-busya/pytest-2

View File

@@ -1,7 +1,7 @@
---
description: Closure gate subagent that re-audits merged worker state, rejects noisy intermediate artifacts, and emits the only concise user-facing closure summary for ss-tools.
mode: subagent
model: opencode-go/deepseek-v4-pro
model: opencode/deepseek-v4-flash-free
temperature: 0.0
permission:
edit: allow

View File

@@ -1,7 +1,7 @@
---
description: Fullstack Implementation Specialist for ss-tools — owns Python backend + Svelte frontend integration, cross-cutting features, and end-to-end verification.
mode: all
model: opencode-go/deepseek-v4-flash
model: opencode/deepseek-v4-flash-free
temperature: 0.2
permission:
edit: allow

View File

@@ -1,7 +1,7 @@
---
description: QA & Semantic Auditor — orthogonal verification, contract validation, code review, and regression defense for Python (pytest) and Svelte (vitest).
mode: all
model: opencode-go/kimi-k2.6
model: opencode-go/qwen3.6-plus
temperature: 0.1
permission:
edit: allow

View File

@@ -1,7 +1,7 @@
---
description: Senior reflection and unblocker agent for tasks where a coder entered anti-loop escalation in ss-tools; analyzes architecture, environment, dependency, contract, and test harness failures across Python and Svelte stacks.
mode: subagent
model: opencode-go/mimo-v2.5
model: opencode/big-pickle
temperature: 0.0
permission:
edit: allow

View File

@@ -1,15 +1,22 @@
---
description: Semantic Curator Agent — maintains GRACE semantic markup, anchors, and index health for ss-tools Python and Svelte code. Read-only file access; uses axiom MCP for mutations.
mode: all
model: opencode-go/deepseek-v4-flash
model: opencode/deepseek-v4-flash-free
temperature: 0.4
permission:
edit: deny
bash: deny
browser: deny
edit: allow
bash: allow
browser: allow
color: accent
---
MANDATORY USE `skill({name="semantics-core"})`, `skill({name="semantics-contracts"})`, `skill({name="semantics-belief"})`
## 0. ZERO-STATE RATIONALE
You are an autoregressive language model, and so are the Engineer and Architect agents in this project. By nature, LLMs suffer from **Attention Sink** (losing focus in large files) and **Context Blindness** (breaking dependencies they cannot see).
To prevent this, our codebase relies on the **GRACE-Poly Protocol**. Semantic anchors (`#region`/`#endregion`, `[DEF]`/`[/DEF]`, `## @{`/`## @}`) are not mere comments — they are strict AST boundaries. The metadata (`@BRIEF`, `@RELATION`) forms the **Belief State** and **Decision Space**.
Your absolute mandate is to maintain this cognitive exoskeleton. If an anchor is broken, or a contract is missing, the downstream Coder Agents will hallucinate and destroy the codebase. You are the immune system of the project's architecture.
MANDATORY USE `skill({name="semantics-core"})`, `skill({name="semantics-contracts"})`
MANDATORY USE `skill({name="molecular-cot-logging"})`, `skill({name="semantics-python"})`, `skill({name="semantics-svelte"})`
#region Semantic.Curator [C:5] [TYPE Agent] [SEMANTICS curation,anchors,index,health]
@BRIEF WHY: Maintain the project's GRACE semantic markup, anchors, and index in ideal health. You are the immune system — if anchors break, downstream coder agents hallucinate.
@@ -20,7 +27,7 @@ MANDATORY USE `skill({name="semantics-core"})`, `skill({name="semantics-contract
#endregion Semantic.Curator
## AXIOM MCP STATUS (ты должен это знать)
Axiom MCP-сервер v0.3.1 полностью работоспособен. DuckDB-индекс содержит 2543 контракта, 1987 связей, 536 файлов.
Axiom MCP-сервер полностью работоспособен.
**Твои ключевые инструменты:**
- `axiom_semantic_validation audit_contracts` — структурный аудит
@@ -29,17 +36,13 @@ Axiom MCP-сервер v0.3.1 полностью работоспособен. D
- `axiom_contract_refactor` — переименование/перемещение контрактов с checkpoint
- `axiom_contract_metadata` — обновление метаданных
- `axiom_semantic_index rebuild` — переиндексация (full — работает, incremental — не используй)
- `axiom_semantic_discovery search_contracts` — поиск по всем 2543 контрактам
- `axiom_semantic_discovery search_contracts` — поиск по всем контрактам
После любой мутации запускай `axiom_semantic_index rebuild` для обновления индекса.
---
## 0. ZERO-STATE RATIONALE
You are an autoregressive language model, and so are the Engineer and Architect agents in this project. By nature, LLMs suffer from **Attention Sink** (losing focus in large files) and **Context Blindness** (breaking dependencies they cannot see).
To prevent this, our codebase relies on the **GRACE-Poly Protocol**. Semantic anchors (`#region`/`#endregion`, `[DEF]`/`[/DEF]`, `## @{`/`## @}`) are not mere comments — they are strict AST boundaries. The metadata (`@BRIEF`, `@RELATION`) forms the **Belief State** and **Decision Space**.
Your absolute mandate is to maintain this cognitive exoskeleton. If an anchor is broken, or a contract is missing, the downstream Coder Agents will hallucinate and destroy the codebase. You are the immune system of the project's architecture.
## 1. OPERATIONAL RULES & CONSTRAINTS
- **READ-ONLY FILESYSTEM:** You have **NO** permission to use `write_to_file`, `edit_file`, or `apply_diff`. You may only read files to gather context.
@@ -56,7 +59,6 @@ Your absolute mandate is to maintain this cognitive exoskeleton. If an anchor is
## 3. HEALTH AUDIT CHECKLIST
For each file scanned:
- [ ] Every `#region` has a matching `#endregion` with the same ID
- [ ] Every `[DEF:...]` has a matching `[/DEF:...]`
- [ ] Every `## @{` has a matching `## @}`
- [ ] C4 contracts have `@PRE`, `@POST`, `@SIDE_EFFECT`
- [ ] C5 contracts additionally have `@RATIONALE`, `@REJECTED`, `@DATA_CONTRACT`, `@INVARIANT`

View File

@@ -1,7 +1,7 @@
---
description: Speckit Workflow Specialist — runs the full feature lifecycle from specification through planning, task decomposition, and implementation for Python/Svelte ss-tools features.
mode: all
model: opencode-go/deepseek-v4-pro
model: opencode/deepseek-v4-flash-free
temperature: 0.2
permission:
edit: allow

View File

@@ -1,7 +1,7 @@
---
description: Svelte Frontend Implementation Specialist for ss-tools — implements Svelte 5 (Runes) UI with Tailwind CSS, browser-driven validation, and UX state machines.
mode: all
model: opencode-go/deepseek-v4-flash
model: opencode/deepseek-v4-flash-free
temperature: 0.1
permission:
edit: allow

View File

@@ -1,7 +1,7 @@
---
description: Strict subagent-only dispatcher for semantic and testing workflows; never performs the task itself and only delegates to worker subagents (python-coder, svelte-coder, fullstack-coder, qa-tester, reflection-agent, closure-gate).
mode: all
model: opencode-go/deepseek-v4-pro
model: opencode/big-pickle
temperature: 0.0
permission:
edit: deny

View File

@@ -1,100 +0,0 @@
---
name: semantics-belief
description: Core protocol for Thread-Local Belief State, runtime reasoning markers, and interleaved thinking across Python-first and Svelte semantic projects. Wire format superseded by molecular-cot-logging.
---
#region Std.Semantics.Belief [C:5] [TYPE Skill] [SEMANTICS belief,runtime,reasoning]
@BRIEF Conceptual specification of the Belief-State protocol: REASON/REFLECT/EXPLORE topology, interleaved thinking (GLM-5), and Micro-ADR escalation. All operational implementation details are in molecular-cot-logging.
@RELATION DEPENDS_ON -> [Std.Semantics.Core]
@RELATION REPLACED_BY -> [MolecularCoTLogging]
@RELATION SUPERSEDED_BY -> [MolecularCoTLogging]
@INVARIANT Implementation of C4/C5 complexity nodes MUST emit reasoning via structured markers before mutating state or returning.
@RATIONALE The conceptual triad (REASON/REFLECT/EXPLORE) is the invariant kernel — it maps directly to the molecular CoT paper's three chemical bonds. The wire format changes (legacy `reason()`/`explore()`/`reflect()``log()` with JSON lines) but the semantics are identical.
@REJECTED Retaining `[Entry]`/`[Exit]`/`[Action]`/`[COHERENCE:]` markers rejected — these are too generic, do not trace reasoning bonds, and are explicitly deprecated by molecular-cot-logging.
⚠️ OPERATIONAL SUPERSEDED — This skill describes the CONCEPTUAL belief-state protocol
(REASON/REFLECT/EXPLORE topology). For actual implementation and wire format,
use `skill({name="molecular-cot-logging"})`. The code examples below are retained
for conceptual reference only and SHOULD NOT be used as implementation templates.
## 0. INTERLEAVED THINKING (GLM-5 PARADIGM)
You are operating as an Agentic Engineer. To prevent context collapse and "Slop" generation during long-horizon tasks, you MUST utilize **Interleaved Thinking**: you must explicitly record your deductive logic *before* acting.
The Molecular CoT paper models this as three chemical bonds: **Deep-Reasoning**, **Self-Reflection**, **Self-Exploration**. Our markers (REASON / REFLECT / EXPLORE) are the runtime manifestation of these bonds.
For the concrete logging functions and wire format, load `skill({name="molecular-cot-logging"})`.
## I. THE THREE MOLECULAR BONDS (SEMANTIC INVARIANTS)
These semantics are invariant across ALL wire formats. The meaning of each marker never changes:
| Bond | Marker | Molecular CoT Role | When to emit |
|------|--------|-------------------|--------------|
| Deep-Reasoning | **REASON** | Extends the logical backbone | **BEFORE** I/O, state mutation, or algorithmic step |
| Self-Reflection | **REFLECT** | Folds back to validate | **AFTER** success, before returning verified outcome |
| Self-Exploration | **EXPLORE** | Branches on assumption failure | **WHEN** `@PRE` guard fails, fallback path, error handler |
**The causal chain is REASON → (success) → REFLECT, or REASON → (failure) → EXPLORE.** Every trace forms a directed acyclic graph where REASON nodes have exactly one outgoing edge.
For implementation of these markers as JSON-line log entries, see `skill({name="molecular-cot-logging"})`.
## II. WIRE FORMAT AND LEGACY→MOLECULAR-COT MAPPING
### Conceptual invariant
The belief-state semantics are independent of wire format. Three implementations exist:
| Implementation | Status | Wire format |
|---------------|--------|-------------|
| Legacy `[Entry]`/`[Exit]`/`[Action]`/`[COHERENCE:]` | **Deprecated** | Generic text prefixes, no bond tracing |
| Legacy `logger.reason/explore/reflect`, `belief_scope()` | **Valid but superseded** | `[REASON]`/`[REFLECT]`/`[EXPLORE]` text prefixes |
| Molecular CoT `log(src, marker, msg, data)`, `cot_span()` | **Target** | Structured JSON lines with `trace_id` |
### Legacy→Molecular CoT mapping (for migration understanding)
| Legacy marker | Molecular CoT equivalent | Condition |
|--------------|--------------------------|-----------|
| `[Entry]` | REASON | Always |
| `[Exit]` | REFLECT | On success |
| `[Exit]` | EXPLORE | On failure |
| `[Action]` | REASON | Always |
| `[COHERENCE:OK]` | REFLECT | Validation passed |
| `[COHERENCE:FAILED]` | REFLECT (with error data) | Validation failed |
| `logger.reason(msg)` | `log(src, "REASON", msg)` | — |
| `logger.explore(msg)` | `log(src, "EXPLORE", msg)` | — |
| `logger.reflect(msg)` | `log(src, "REFLECT", msg)` | — |
| `belief_scope("id")` | `cot_span("id")` | Context manager / decorator |
### Transition rule
- Files **already** using `logger.reason/explore/reflect`: semantics are correct, marker names match. Leave as-is. Convert only when touching the file for other reasons.
- **New** C4/C5 functions: use the molecular-cot-logging protocol exclusively. Load `skill({name="molecular-cot-logging"})` for the API.
- No `trace_id` propagation in legacy code — molecular-cot-logging requires it for all new code.
## III. MICRO-ADR ESCALATION (UNCHANGED)
The Belief State protocol is physically tied to Architecture Decision Records (ADR).
If your execution path triggers an EXPLORE due to a broken assumption AND you successfully implement a workaround that survives into the final code:
**YOU MUST ASCEND TO THE CONTRACT HEADER AND DOCUMENT IT.**
Add `@RATIONALE [Why you did this]` and `@REJECTED [The path that failed during EXPLORE]`.
Failure to link a runtime EXPLORE to a static `@REJECTED` tag is a fatal protocol violation that causes amnesia for future agents.
## IV. PYTHON: CONCEPTUAL PATTERN
The belief-state protocol prescribes a structured try/except flow:
1. **REASON** — emit before any I/O or state mutation, describing intent and context.
2. **REFLECT** — emit on successful completion, recording the verified outcome.
3. **EXPLORE** — emit in error/fallback paths, recording the failure reason.
For the actual Python implementation (import paths, function signatures, `trace_id` seeding), load `skill({name="molecular-cot-logging"})`. The conceptual equivalent of `belief_scope()` is `cot_span()`.
## V. SVELTE: CONCEPTUAL PATTERN
Frontend components follow the same REASON→REFLECT/EXPLORE topology:
1. **REASON** — emit before API calls, user actions, or state transitions.
2. **REFLECT** — emit after successful data fetch or DOM mutation.
3. **EXPLORE** — emit in error handlers and fallback UI paths.
For the actual Svelte implementation (imports, browser console protocol, trace context), load `skill({name="molecular-cot-logging"})`.
#endregion Std.Semantics.Belief

View File

@@ -0,0 +1 @@
ref: refs/heads/master

View File

@@ -0,0 +1,4 @@
[core]
repositoryformatversion = 0
filemode = true
bare = true

View File

@@ -0,0 +1 @@
Unnamed repository; edit this file 'description' to name the repository.

View File

@@ -0,0 +1,15 @@
#!/bin/sh
#
# An example hook script to check the commit log message taken by
# applypatch from an e-mail message.
#
# The hook should exit with non-zero status after issuing an
# appropriate message if it wants to stop the commit. The hook is
# allowed to edit the commit message file.
#
# To enable this hook, rename this file to "applypatch-msg".
. git-sh-setup
commitmsg="$(git rev-parse --git-path hooks/commit-msg)"
test -x "$commitmsg" && exec "$commitmsg" ${1+"$@"}
:

View File

@@ -0,0 +1,24 @@
#!/bin/sh
#
# An example hook script to check the commit log message.
# Called by "git commit" with one argument, the name of the file
# that has the commit message. The hook should exit with non-zero
# status after issuing an appropriate message if it wants to stop the
# commit. The hook is allowed to edit the commit message file.
#
# To enable this hook, rename this file to "commit-msg".
# Uncomment the below to add a Signed-off-by line to the message.
# Doing this in a hook is a bad idea in general, but the prepare-commit-msg
# hook is more suited to it.
#
# SOB=$(git var GIT_AUTHOR_IDENT | sed -n 's/^\(.*>\).*$/Signed-off-by: \1/p')
# grep -qs "^$SOB" "$1" || echo "$SOB" >> "$1"
# This example catches duplicate Signed-off-by lines.
test "" = "$(grep '^Signed-off-by: ' "$1" |
sort | uniq -c | sed -e '/^[ ]*1[ ]/d')" || {
echo >&2 Duplicate Signed-off-by lines.
exit 1
}

View File

@@ -0,0 +1,174 @@
#!/usr/bin/perl
use strict;
use warnings;
use IPC::Open2;
# An example hook script to integrate Watchman
# (https://facebook.github.io/watchman/) with git to speed up detecting
# new and modified files.
#
# The hook is passed a version (currently 2) and last update token
# formatted as a string and outputs to stdout a new update token and
# all files that have been modified since the update token. Paths must
# be relative to the root of the working tree and separated by a single NUL.
#
# To enable this hook, rename this file to "query-watchman" and set
# 'git config core.fsmonitor .git/hooks/query-watchman'
#
my ($version, $last_update_token) = @ARGV;
# Uncomment for debugging
# print STDERR "$0 $version $last_update_token\n";
# Check the hook interface version
if ($version ne 2) {
die "Unsupported query-fsmonitor hook version '$version'.\n" .
"Falling back to scanning...\n";
}
my $git_work_tree = get_working_dir();
my $retry = 1;
my $json_pkg;
eval {
require JSON::XS;
$json_pkg = "JSON::XS";
1;
} or do {
require JSON::PP;
$json_pkg = "JSON::PP";
};
launch_watchman();
sub launch_watchman {
my $o = watchman_query();
if (is_work_tree_watched($o)) {
output_result($o->{clock}, @{$o->{files}});
}
}
sub output_result {
my ($clockid, @files) = @_;
# Uncomment for debugging watchman output
# open (my $fh, ">", ".git/watchman-output.out");
# binmode $fh, ":utf8";
# print $fh "$clockid\n@files\n";
# close $fh;
binmode STDOUT, ":utf8";
print $clockid;
print "\0";
local $, = "\0";
print @files;
}
sub watchman_clock {
my $response = qx/watchman clock "$git_work_tree"/;
die "Failed to get clock id on '$git_work_tree'.\n" .
"Falling back to scanning...\n" if $? != 0;
return $json_pkg->new->utf8->decode($response);
}
sub watchman_query {
my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
or die "open2() failed: $!\n" .
"Falling back to scanning...\n";
# In the query expression below we're asking for names of files that
# changed since $last_update_token but not from the .git folder.
#
# To accomplish this, we're using the "since" generator to use the
# recency index to select candidate nodes and "fields" to limit the
# output to file names only. Then we're using the "expression" term to
# further constrain the results.
my $last_update_line = "";
if (substr($last_update_token, 0, 1) eq "c") {
$last_update_token = "\"$last_update_token\"";
$last_update_line = qq[\n"since": $last_update_token,];
}
my $query = <<" END";
["query", "$git_work_tree", {$last_update_line
"fields": ["name"],
"expression": ["not", ["dirname", ".git"]]
}]
END
# Uncomment for debugging the watchman query
# open (my $fh, ">", ".git/watchman-query.json");
# print $fh $query;
# close $fh;
print CHLD_IN $query;
close CHLD_IN;
my $response = do {local $/; <CHLD_OUT>};
# Uncomment for debugging the watch response
# open ($fh, ">", ".git/watchman-response.json");
# print $fh $response;
# close $fh;
die "Watchman: command returned no output.\n" .
"Falling back to scanning...\n" if $response eq "";
die "Watchman: command returned invalid output: $response\n" .
"Falling back to scanning...\n" unless $response =~ /^\{/;
return $json_pkg->new->utf8->decode($response);
}
sub is_work_tree_watched {
my ($output) = @_;
my $error = $output->{error};
if ($retry > 0 and $error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
$retry--;
my $response = qx/watchman watch "$git_work_tree"/;
die "Failed to make watchman watch '$git_work_tree'.\n" .
"Falling back to scanning...\n" if $? != 0;
$output = $json_pkg->new->utf8->decode($response);
$error = $output->{error};
die "Watchman: $error.\n" .
"Falling back to scanning...\n" if $error;
# Uncomment for debugging watchman output
# open (my $fh, ">", ".git/watchman-output.out");
# close $fh;
# Watchman will always return all files on the first query so
# return the fast "everything is dirty" flag to git and do the
# Watchman query just to get it over with now so we won't pay
# the cost in git to look up each individual file.
my $o = watchman_clock();
$error = $output->{error};
die "Watchman: $error.\n" .
"Falling back to scanning...\n" if $error;
output_result($o->{clock}, ("/"));
$last_update_token = $o->{clock};
eval { launch_watchman() };
return 0;
}
die "Watchman: $error.\n" .
"Falling back to scanning...\n" if $error;
return 1;
}
sub get_working_dir {
my $working_dir;
if ($^O =~ 'msys' || $^O =~ 'cygwin') {
$working_dir = Win32::GetCwd();
$working_dir =~ tr/\\/\//;
} else {
require Cwd;
$working_dir = Cwd::cwd();
}
return $working_dir;
}

View File

@@ -0,0 +1,8 @@
#!/bin/sh
#
# An example hook script to prepare a packed repository for use over
# dumb transports.
#
# To enable this hook, rename this file to "post-update".
exec git update-server-info

View File

@@ -0,0 +1,14 @@
#!/bin/sh
#
# An example hook script to verify what is about to be committed
# by applypatch from an e-mail message.
#
# The hook should exit with non-zero status after issuing an
# appropriate message if it wants to stop the commit.
#
# To enable this hook, rename this file to "pre-applypatch".
. git-sh-setup
precommit="$(git rev-parse --git-path hooks/pre-commit)"
test -x "$precommit" && exec "$precommit" ${1+"$@"}
:

View File

@@ -0,0 +1,49 @@
#!/bin/sh
#
# An example hook script to verify what is about to be committed.
# Called by "git commit" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message if
# it wants to stop the commit.
#
# To enable this hook, rename this file to "pre-commit".
if git rev-parse --verify HEAD >/dev/null 2>&1
then
against=HEAD
else
# Initial commit: diff against an empty tree object
against=$(git hash-object -t tree /dev/null)
fi
# If you want to allow non-ASCII filenames set this variable to true.
allownonascii=$(git config --type=bool hooks.allownonascii)
# Redirect output to stderr.
exec 1>&2
# Cross platform projects tend to avoid non-ASCII filenames; prevent
# them from being added to the repository. We exploit the fact that the
# printable range starts at the space character and ends with tilde.
if [ "$allownonascii" != "true" ] &&
# Note that the use of brackets around a tr range is ok here, (it's
# even required, for portability to Solaris 10's /usr/bin/tr), since
# the square bracket bytes happen to fall in the designated range.
test $(git diff-index --cached --name-only --diff-filter=A -z $against |
LC_ALL=C tr -d '[ -~]\0' | wc -c) != 0
then
cat <<\EOF
Error: Attempt to add a non-ASCII file name.
This can cause problems if you want to work with people on other platforms.
To be portable it is advisable to rename the file.
If you know what you are doing you can disable this check using:
git config hooks.allownonascii true
EOF
exit 1
fi
# If there are whitespace errors, print the offending file names and fail.
exec git diff-index --check --cached $against --

View File

@@ -0,0 +1,13 @@
#!/bin/sh
#
# An example hook script to verify what is about to be committed.
# Called by "git merge" with no arguments. The hook should
# exit with non-zero status after issuing an appropriate message to
# stderr if it wants to stop the merge commit.
#
# To enable this hook, rename this file to "pre-merge-commit".
. git-sh-setup
test -x "$GIT_DIR/hooks/pre-commit" &&
exec "$GIT_DIR/hooks/pre-commit"
:

View File

@@ -0,0 +1,53 @@
#!/bin/sh
# An example hook script to verify what is about to be pushed. Called by "git
# push" after it has checked the remote status, but before anything has been
# pushed. If this script exits with a non-zero status nothing will be pushed.
#
# This hook is called with the following parameters:
#
# $1 -- Name of the remote to which the push is being done
# $2 -- URL to which the push is being done
#
# If pushing without using a named remote those arguments will be equal.
#
# Information about the commits which are being pushed is supplied as lines to
# the standard input in the form:
#
# <local ref> <local oid> <remote ref> <remote oid>
#
# This sample shows how to prevent push of commits where the log message starts
# with "WIP" (work in progress).
remote="$1"
url="$2"
zero=$(git hash-object --stdin </dev/null | tr '[0-9a-f]' '0')
while read local_ref local_oid remote_ref remote_oid
do
if test "$local_oid" = "$zero"
then
# Handle delete
:
else
if test "$remote_oid" = "$zero"
then
# New branch, examine all commits
range="$local_oid"
else
# Update to existing branch, examine new commits
range="$remote_oid..$local_oid"
fi
# Check for WIP commit
commit=$(git rev-list -n 1 --grep '^WIP' "$range")
if test -n "$commit"
then
echo >&2 "Found WIP commit in $local_ref, not pushing"
exit 1
fi
fi
done
exit 0

View File

@@ -0,0 +1,169 @@
#!/bin/sh
#
# Copyright (c) 2006, 2008 Junio C Hamano
#
# The "pre-rebase" hook is run just before "git rebase" starts doing
# its job, and can prevent the command from running by exiting with
# non-zero status.
#
# The hook is called with the following parameters:
#
# $1 -- the upstream the series was forked from.
# $2 -- the branch being rebased (or empty when rebasing the current branch).
#
# This sample shows how to prevent topic branches that are already
# merged to 'next' branch from getting rebased, because allowing it
# would result in rebasing already published history.
publish=next
basebranch="$1"
if test "$#" = 2
then
topic="refs/heads/$2"
else
topic=`git symbolic-ref HEAD` ||
exit 0 ;# we do not interrupt rebasing detached HEAD
fi
case "$topic" in
refs/heads/??/*)
;;
*)
exit 0 ;# we do not interrupt others.
;;
esac
# Now we are dealing with a topic branch being rebased
# on top of master. Is it OK to rebase it?
# Does the topic really exist?
git show-ref -q "$topic" || {
echo >&2 "No such branch $topic"
exit 1
}
# Is topic fully merged to master?
not_in_master=`git rev-list --pretty=oneline ^master "$topic"`
if test -z "$not_in_master"
then
echo >&2 "$topic is fully merged to master; better remove it."
exit 1 ;# we could allow it, but there is no point.
fi
# Is topic ever merged to next? If so you should not be rebasing it.
only_next_1=`git rev-list ^master "^$topic" ${publish} | sort`
only_next_2=`git rev-list ^master ${publish} | sort`
if test "$only_next_1" = "$only_next_2"
then
not_in_topic=`git rev-list "^$topic" master`
if test -z "$not_in_topic"
then
echo >&2 "$topic is already up to date with master"
exit 1 ;# we could allow it, but there is no point.
else
exit 0
fi
else
not_in_next=`git rev-list --pretty=oneline ^${publish} "$topic"`
/usr/bin/perl -e '
my $topic = $ARGV[0];
my $msg = "* $topic has commits already merged to public branch:\n";
my (%not_in_next) = map {
/^([0-9a-f]+) /;
($1 => 1);
} split(/\n/, $ARGV[1]);
for my $elem (map {
/^([0-9a-f]+) (.*)$/;
[$1 => $2];
} split(/\n/, $ARGV[2])) {
if (!exists $not_in_next{$elem->[0]}) {
if ($msg) {
print STDERR $msg;
undef $msg;
}
print STDERR " $elem->[1]\n";
}
}
' "$topic" "$not_in_next" "$not_in_master"
exit 1
fi
<<\DOC_END
This sample hook safeguards topic branches that have been
published from being rewound.
The workflow assumed here is:
* Once a topic branch forks from "master", "master" is never
merged into it again (either directly or indirectly).
* Once a topic branch is fully cooked and merged into "master",
it is deleted. If you need to build on top of it to correct
earlier mistakes, a new topic branch is created by forking at
the tip of the "master". This is not strictly necessary, but
it makes it easier to keep your history simple.
* Whenever you need to test or publish your changes to topic
branches, merge them into "next" branch.
The script, being an example, hardcodes the publish branch name
to be "next", but it is trivial to make it configurable via
$GIT_DIR/config mechanism.
With this workflow, you would want to know:
(1) ... if a topic branch has ever been merged to "next". Young
topic branches can have stupid mistakes you would rather
clean up before publishing, and things that have not been
merged into other branches can be easily rebased without
affecting other people. But once it is published, you would
not want to rewind it.
(2) ... if a topic branch has been fully merged to "master".
Then you can delete it. More importantly, you should not
build on top of it -- other people may already want to
change things related to the topic as patches against your
"master", so if you need further changes, it is better to
fork the topic (perhaps with the same name) afresh from the
tip of "master".
Let's look at this example:
o---o---o---o---o---o---o---o---o---o "next"
/ / / /
/ a---a---b A / /
/ / / /
/ / c---c---c---c B /
/ / / \ /
/ / / b---b C \ /
/ / / / \ /
---o---o---o---o---o---o---o---o---o---o---o "master"
A, B and C are topic branches.
* A has one fix since it was merged up to "next".
* B has finished. It has been fully merged up to "master" and "next",
and is ready to be deleted.
* C has not merged to "next" at all.
We would want to allow C to be rebased, refuse A, and encourage
B to be deleted.
To compute (1):
git rev-list ^master ^topic next
git rev-list ^master next
if these match, topic has not merged in next at all.
To compute (2):
git rev-list master..topic
if this is empty, it is fully merged to "master".
DOC_END

View File

@@ -0,0 +1,24 @@
#!/bin/sh
#
# An example hook script to make use of push options.
# The example simply echoes all push options that start with 'echoback='
# and rejects all pushes when the "reject" push option is used.
#
# To enable this hook, rename this file to "pre-receive".
if test -n "$GIT_PUSH_OPTION_COUNT"
then
i=0
while test "$i" -lt "$GIT_PUSH_OPTION_COUNT"
do
eval "value=\$GIT_PUSH_OPTION_$i"
case "$value" in
echoback=*)
echo "echo from the pre-receive-hook: ${value#*=}" >&2
;;
reject)
exit 1
esac
i=$((i + 1))
done
fi

View File

@@ -0,0 +1,42 @@
#!/bin/sh
#
# An example hook script to prepare the commit log message.
# Called by "git commit" with the name of the file that has the
# commit message, followed by the description of the commit
# message's source. The hook's purpose is to edit the commit
# message file. If the hook fails with a non-zero status,
# the commit is aborted.
#
# To enable this hook, rename this file to "prepare-commit-msg".
# This hook includes three examples. The first one removes the
# "# Please enter the commit message..." help message.
#
# The second includes the output of "git diff --name-status -r"
# into the message, just before the "git status" output. It is
# commented because it doesn't cope with --amend or with squashed
# commits.
#
# The third example adds a Signed-off-by line to the message, that can
# still be edited. This is rarely a good idea.
COMMIT_MSG_FILE=$1
COMMIT_SOURCE=$2
SHA1=$3
/usr/bin/perl -i.bak -ne 'print unless(m/^. Please enter the commit message/..m/^#$/)' "$COMMIT_MSG_FILE"
# case "$COMMIT_SOURCE,$SHA1" in
# ,|template,)
# /usr/bin/perl -i.bak -pe '
# print "\n" . `git diff --cached --name-status -r`
# if /^#/ && $first++ == 0' "$COMMIT_MSG_FILE" ;;
# *) ;;
# esac
# SOB=$(git var GIT_COMMITTER_IDENT | sed -n 's/^\(.*>\).*$/Signed-off-by: \1/p')
# git interpret-trailers --in-place --trailer "$SOB" "$COMMIT_MSG_FILE"
# if test -z "$COMMIT_SOURCE"
# then
# /usr/bin/perl -i.bak -pe 'print "\n" if !$first_line++' "$COMMIT_MSG_FILE"
# fi

View File

@@ -0,0 +1,78 @@
#!/bin/sh
# An example hook script to update a checked-out tree on a git push.
#
# This hook is invoked by git-receive-pack(1) when it reacts to git
# push and updates reference(s) in its repository, and when the push
# tries to update the branch that is currently checked out and the
# receive.denyCurrentBranch configuration variable is set to
# updateInstead.
#
# By default, such a push is refused if the working tree and the index
# of the remote repository has any difference from the currently
# checked out commit; when both the working tree and the index match
# the current commit, they are updated to match the newly pushed tip
# of the branch. This hook is to be used to override the default
# behaviour; however the code below reimplements the default behaviour
# as a starting point for convenient modification.
#
# The hook receives the commit with which the tip of the current
# branch is going to be updated:
commit=$1
# It can exit with a non-zero status to refuse the push (when it does
# so, it must not modify the index or the working tree).
die () {
echo >&2 "$*"
exit 1
}
# Or it can make any necessary changes to the working tree and to the
# index to bring them to the desired state when the tip of the current
# branch is updated to the new commit, and exit with a zero status.
#
# For example, the hook can simply run git read-tree -u -m HEAD "$1"
# in order to emulate git fetch that is run in the reverse direction
# with git push, as the two-tree form of git read-tree -u -m is
# essentially the same as git switch or git checkout that switches
# branches while keeping the local changes in the working tree that do
# not interfere with the difference between the branches.
# The below is a more-or-less exact translation to shell of the C code
# for the default behaviour for git's push-to-checkout hook defined in
# the push_to_deploy() function in builtin/receive-pack.c.
#
# Note that the hook will be executed from the repository directory,
# not from the working tree, so if you want to perform operations on
# the working tree, you will have to adapt your code accordingly, e.g.
# by adding "cd .." or using relative paths.
if ! git update-index -q --ignore-submodules --refresh
then
die "Up-to-date check failed"
fi
if ! git diff-files --quiet --ignore-submodules --
then
die "Working directory has unstaged changes"
fi
# This is a rough translation of:
#
# head_has_history() ? "HEAD" : EMPTY_TREE_SHA1_HEX
if git cat-file -e HEAD 2>/dev/null
then
head=HEAD
else
head=$(git hash-object -t tree --stdin </dev/null)
fi
if ! git diff-index --quiet --cached --ignore-submodules $head --
then
die "Working directory has staged changes"
fi
if ! git read-tree -u -m "$commit"
then
die "Could not update working tree to new HEAD"
fi

View File

@@ -0,0 +1,77 @@
#!/bin/sh
# An example hook script to validate a patch (and/or patch series) before
# sending it via email.
#
# The hook should exit with non-zero status after issuing an appropriate
# message if it wants to prevent the email(s) from being sent.
#
# To enable this hook, rename this file to "sendemail-validate".
#
# By default, it will only check that the patch(es) can be applied on top of
# the default upstream branch without conflicts in a secondary worktree. After
# validation (successful or not) of the last patch of a series, the worktree
# will be deleted.
#
# The following config variables can be set to change the default remote and
# remote ref that are used to apply the patches against:
#
# sendemail.validateRemote (default: origin)
# sendemail.validateRemoteRef (default: HEAD)
#
# Replace the TODO placeholders with appropriate checks according to your
# needs.
validate_cover_letter () {
file="$1"
# TODO: Replace with appropriate checks (e.g. spell checking).
true
}
validate_patch () {
file="$1"
# Ensure that the patch applies without conflicts.
git am -3 "$file" || return
# TODO: Replace with appropriate checks for this patch
# (e.g. checkpatch.pl).
true
}
validate_series () {
# TODO: Replace with appropriate checks for the whole series
# (e.g. quick build, coding style checks, etc.).
true
}
# main -------------------------------------------------------------------------
if test "$GIT_SENDEMAIL_FILE_COUNTER" = 1
then
remote=$(git config --default origin --get sendemail.validateRemote) &&
ref=$(git config --default HEAD --get sendemail.validateRemoteRef) &&
worktree=$(mktemp --tmpdir -d sendemail-validate.XXXXXXX) &&
git worktree add -fd --checkout "$worktree" "refs/remotes/$remote/$ref" &&
git config --replace-all sendemail.validateWorktree "$worktree"
else
worktree=$(git config --get sendemail.validateWorktree)
fi || {
echo "sendemail-validate: error: failed to prepare worktree" >&2
exit 1
}
unset GIT_DIR GIT_WORK_TREE
cd "$worktree" &&
if grep -q "^diff --git " "$1"
then
validate_patch "$1"
else
validate_cover_letter "$1"
fi &&
if test "$GIT_SENDEMAIL_FILE_COUNTER" = "$GIT_SENDEMAIL_FILE_TOTAL"
then
git config --unset-all sendemail.validateWorktree &&
trap 'git worktree remove -ff "$worktree"' EXIT &&
validate_series
fi

View File

@@ -0,0 +1,128 @@
#!/bin/sh
#
# An example hook script to block unannotated tags from entering.
# Called by "git receive-pack" with arguments: refname sha1-old sha1-new
#
# To enable this hook, rename this file to "update".
#
# Config
# ------
# hooks.allowunannotated
# This boolean sets whether unannotated tags will be allowed into the
# repository. By default they won't be.
# hooks.allowdeletetag
# This boolean sets whether deleting tags will be allowed in the
# repository. By default they won't be.
# hooks.allowmodifytag
# This boolean sets whether a tag may be modified after creation. By default
# it won't be.
# hooks.allowdeletebranch
# This boolean sets whether deleting branches will be allowed in the
# repository. By default they won't be.
# hooks.denycreatebranch
# This boolean sets whether remotely creating branches will be denied
# in the repository. By default this is allowed.
#
# --- Command line
refname="$1"
oldrev="$2"
newrev="$3"
# --- Safety check
if [ -z "$GIT_DIR" ]; then
echo "Don't run this script from the command line." >&2
echo " (if you want, you could supply GIT_DIR then run" >&2
echo " $0 <ref> <oldrev> <newrev>)" >&2
exit 1
fi
if [ -z "$refname" -o -z "$oldrev" -o -z "$newrev" ]; then
echo "usage: $0 <ref> <oldrev> <newrev>" >&2
exit 1
fi
# --- Config
allowunannotated=$(git config --type=bool hooks.allowunannotated)
allowdeletebranch=$(git config --type=bool hooks.allowdeletebranch)
denycreatebranch=$(git config --type=bool hooks.denycreatebranch)
allowdeletetag=$(git config --type=bool hooks.allowdeletetag)
allowmodifytag=$(git config --type=bool hooks.allowmodifytag)
# check for no description
projectdesc=$(sed -e '1q' "$GIT_DIR/description")
case "$projectdesc" in
"Unnamed repository"* | "")
echo "*** Project description file hasn't been set" >&2
exit 1
;;
esac
# --- Check types
# if $newrev is 0000...0000, it's a commit to delete a ref.
zero=$(git hash-object --stdin </dev/null | tr '[0-9a-f]' '0')
if [ "$newrev" = "$zero" ]; then
newrev_type=delete
else
newrev_type=$(git cat-file -t $newrev)
fi
case "$refname","$newrev_type" in
refs/tags/*,commit)
# un-annotated tag
short_refname=${refname##refs/tags/}
if [ "$allowunannotated" != "true" ]; then
echo "*** The un-annotated tag, $short_refname, is not allowed in this repository" >&2
echo "*** Use 'git tag [ -a | -s ]' for tags you want to propagate." >&2
exit 1
fi
;;
refs/tags/*,delete)
# delete tag
if [ "$allowdeletetag" != "true" ]; then
echo "*** Deleting a tag is not allowed in this repository" >&2
exit 1
fi
;;
refs/tags/*,tag)
# annotated tag
if [ "$allowmodifytag" != "true" ] && git rev-parse $refname > /dev/null 2>&1
then
echo "*** Tag '$refname' already exists." >&2
echo "*** Modifying a tag is not allowed in this repository." >&2
exit 1
fi
;;
refs/heads/*,commit)
# branch
if [ "$oldrev" = "$zero" -a "$denycreatebranch" = "true" ]; then
echo "*** Creating a branch is not allowed in this repository" >&2
exit 1
fi
;;
refs/heads/*,delete)
# delete branch
if [ "$allowdeletebranch" != "true" ]; then
echo "*** Deleting a branch is not allowed in this repository" >&2
exit 1
fi
;;
refs/remotes/*,commit)
# tracking branch
;;
refs/remotes/*,delete)
# delete tracking branch
if [ "$allowdeletebranch" != "true" ]; then
echo "*** Deleting a tracking branch is not allowed in this repository" >&2
exit 1
fi
;;
*)
# Anything else (is there anything else?)
echo "*** Update hook: unknown type of update to ref $refname of type $newrev_type" >&2
exit 1
;;
esac
# --- Finished
exit 0

View File

@@ -0,0 +1,6 @@
# git ls-files --others --exclude-from=.git/info/exclude
# Lines that start with '#' are comments.
# For a project mostly in C, the following would be a good set of
# exclude patterns (uncomment them if you want to use them):
# *.[oa]
# *~

View File

@@ -0,0 +1,2 @@
x<05>A
<EFBFBD> <05>֞<D69E>H<EFBFBD>55<35><35><EFBFBD><05><><EFBFBD>JQ <09>}<7D><>+<05>ܽ<EFBFBD><DCBD>3=<3D><><EFBFBD>B<EFBFBD>/Gѫ<47>GH<47>쪝w<ECAA9D><77><EFBFBD>Y<EFBFBD>qpӀq <20><08>!<21>3<EFBFBD><33><EFBFBD>0<>O<>7

View File

@@ -0,0 +1 @@
x]<5D><>j<EFBFBD>0 <0C>w<EFBFBD><77><EFBFBD> <0B><6D><D2A4><EFBFBD>c <0C>0r<30>t<06><><EFBFBD>ӥ<EFBFBD><D3A5><EFBFBD><EFBFBD>B<02>.?I<>W<EFBFBD>=6o(<28>(X'<27>#ad<61><64><EFBFBD><EFBFBD>/<2F><0F><>|<7C>1<EFBFBD>,<2C><>Y%ޗ<>j%<25>b<EFBFBD><62>Xe\&c<><63>$<24><>D <0C><><EFBFBD><EFBFBD>8Y<38><08><>)dd@<40>d<7F>NB|<7C>^<5E>(θO7<><37>%<25><>Z2B<>G<EFBFBD><47><EFBFBD>N)<29>`<60>[L<><[<5B><10><16>߿B<DFBF>݌<EFBFBD>5[<5B><>[^<03><><1D><><1E>|<7C><><EFBFBD><1F><><EFBFBD>C<>RI1<>Yr6%4<>?<3F>a<EFBFBD><61>ԵU{Q<><52><C7A6><EFBFBD>pR]w<><1A>݋<EFBFBD><DD8B>N<EFBFBD>zw<7A><1D>?RƜ3

View File

@@ -0,0 +1 @@
x<01>OAN1<>c%<25><>&A<><41><EFBFBD><1F><>df"*<2A><>ZR<5A>}r<><01>b˲eYF<59><46> <18><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Q<EFBFBD><51>R<EFBFBD>5sţ<73>Z<EFBFBD>@\<5C>E<EFBFBD><45>Y+n<><6E><EFBFBD>uk<><6B><EFBFBD>BR5<><35><EFBFBD>$<17> <09>gjF>9<>˛qfHޚKTS<54><10><>B<12><><EFBFBD><EFBFBD>y<EFBFBD><79>V<EFBFBD><56>s<EFBFBD>p<EFBFBD>3|<7C><><EFBFBD>7C{<7B>K<EFBFBD><4B><EFBFBD>'<27>1f<31>)2“<><D896>L<EFBFBD>Ge<47><65>`<60><>5~_ᶏ><3E>-<2D>%<15>O<EFBFBD><4F><EFBFBD><EFBFBD><EFBFBD><02>wU<77>

View File

@@ -0,0 +1 @@
x]R<>r<EFBFBD> <10>Y_<59>mO<6D><4F><EFBFBD>z<EFBFBD><7A><EFBFBD><1A><>ʾ<EFBFBD>D

View File

@@ -0,0 +1,2 @@
x<05>A
<EFBFBD> <05>֞<D69E>P<EFBFBD>*̺#t%!!SR<53>n<EFBFBD>{<7B>jI<6A><49><EFBFBD><EFBFBD><19><>,<2C><><EFBFBD><EFBFBD>_<EFBFBD>,<2C>8<EFBFBD><38><EFBFBD>s<EFBFBD>Yj3<><33>R<11>k8m<38>n<EFBFBD>D<EFBFBD>`<60>uDaP<>N<>&

View File

@@ -0,0 +1 @@
x<01><>;<0E>0D<>s<EFBFBD><73>i<EFBFBD>ٍc<D98D>W٬DŽ"%<25><><13>S<>f<EFBFBD>4c<34><<3C><1B><>m(<28>8<14>,V<>9C"<22>0F +<2B>

View File

@@ -0,0 +1,4 @@
x<01>R<EFBFBD><52><EFBFBD>0L<><4C>`<60>J<07>a<EFBFBD><61><0F> pFrM*bI<62>l"<14><><EFBFBD><EFBFBD>Rv<52>K<15><>vf<76><66><EFBFBD><EFBFBD><EFBFBD>^<5E><><EFBFBD>~<7E><>(<16><><EFBFBD><EFBFBD><EFBFBD>Ӆ}<7D><><EFBFBD>ƨ<EFBFBD>Y<EFBFBD><59>nb.[[) <0C><><06><><EFBFBD>!<05>MhL`l<>so<73><6F>HוƢ<D795>@HQ$<24>`<60>`0
kV+X<>T1<54><31>B
<0B><><EFBFBD><1A><><EFBFBD><EFBFBD><<3C><>cb<63><62>?78<37><14><><EFBFBD>X}<7D><>tĠK&<26>2<EFBFBD>o<1D>k$<24><1C><>C N<>W <13>"<22><><EFBFBD>y%+<2B>Z?>*A`<60>bIi<49>E<EFBFBD><02>W^<5E>nJ<6E><4A><EFBFBD>+8<>`<60><><EFBFBD>f1/B<>8<EFBFBD><38><EFBFBD>I<EFBFBD><<3C>|#<23><11><>a<EFBFBD><61><EFBFBD><EFBFBD><EFBFBD>%M!l?<3F>sz<73>;^<08>YJ<59><4A>a
<EFBFBD><08><>)<29>p<EFBFBD>^<5E><>NT<><14><>\<5C><>7<EFBFBD><37>b{m<><6D> <20><>ܲ<EFBFBD><DCB2><EFBFBD><EFBFBD>kF<6B>Xy<58><79><EFBFBD>b<EFBFBD><62><1F><05><1D><><17><><EFBFBD><EFBFBD>Ƕ<>{%kޏ<6B><DE8F><EFBFBD><30>C<EFBFBD>

View File

@@ -0,0 +1 @@
cfd2dcbd55735b8cedf21e04e64ac627c6f9a458

View File

@@ -0,0 +1 @@
5a82ef700fef88b69cd480e4025283809496ee20

View File

@@ -0,0 +1 @@
4e227f4d8289ea657ae813b1460f979ae4b676b3

View File

@@ -0,0 +1 @@
6af2b28e314c04fb4e03476c12d6491213591512

View File

@@ -0,0 +1 @@
5d9eb555c4cac2d3ffdba84c07682b154fe4180d

View File

@@ -0,0 +1 @@
ee59553241ef084b83d3351ba4c33caf2ed9a840

View File

@@ -293,14 +293,14 @@ async def generate_commit_message(
history_objs = _gs.get_commit_history(dashboard_id, limit=5)
history = [h.message for h in history_objs if hasattr(h, "message")]
from ...plugins.llm_analysis.models import LLMProviderType
from ...plugins.llm_analysis.service import LLMClient
from ...services.llm_prompt_templates import (
from src.plugins.llm_analysis.models import LLMProviderType
from src.plugins.llm_analysis.service import LLMClient
from src.services.llm_prompt_templates import (
DEFAULT_LLM_PROMPTS,
normalize_llm_settings,
resolve_bound_provider_id,
)
from ...services.llm_provider import LLMProviderService
from src.services.llm_provider import LLMProviderService
llm_service = LLMProviderService(db)
providers = llm_service.get_all_providers()
@@ -325,7 +325,7 @@ async def generate_commit_message(
default_model=provider.default_model,
)
from ...plugins.git.llm_extension import GitLLMExtension
from src.plugins.git.llm_extension import GitLLMExtension
extension = GitLLMExtension(client)
git_prompt = llm_settings["prompts"].get(

View File

@@ -1,15 +1,18 @@
# #region TranslateRoutes [C:4] [TYPE Module] [SEMANTICS translate, api, package, schedule, dashboard, llm]
# @BRIEF API routes for LLM-based SQL/dashboard translation management including terminology dictionary CRUD and import.
# @LAYER: UI (API)
# @RELATION DEPENDS_ON -> [TranslateJobResponse]
# @RELATION DEPENDS_ON -> [DictionaryManager:Class]
# @RELATION DEPENDS_ON -> [get_current_user]
# @RELATION DEPENDS_ON -> [get_db]
# @RELATION DEPENDS_ON -> [has_permission]
# @RELATION DEPENDS_ON -> [ConfigManager]
# @LAYER API
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RATIONALE: Snapshot isolation — in-progress runs use config snapshot; config edits affect future runs only.
# @REJECTED: Invalidating in-progress runs on config edit would break scheduled run continuity.
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RELATION DEPENDS_ON -> [SupersetClient]
# @RATIONALE Snapshot isolation — in-progress runs use config snapshot; config edits affect future runs only.
# @REJECTED Invalidating in-progress runs on config edit would break scheduled run continuity.
# @POST Translation job, run, dictionary, correction, preview, and schedule CRUD operations are executed.
# @PRE API server is running, user is authenticated, database is accessible.
# @SIDE_EFFECT Creates/modifies DB rows; triggers background translation runs; queries Superset API.
"""
Translate routes package.

View File

@@ -20,10 +20,12 @@ from ._router import router
# Corrections
# ============================================================
# #region submit_correction [TYPE Function]
# #region submit_correction [C:4] [TYPE Function]
# @BRIEF Submit a term correction from a run result review.
# @PRE: User has translate.dictionary.edit permission.
# @POST: Correction applied with conflict detection.
# @PRE User has translate.dictionary.edit permission.
# @POST Correction applied with conflict detection.
# @SIDE_EFFECT Creates/updates DictionaryEntry row; commits.
# @RELATION DEPENDS_ON -> [DictionaryManager]
@router.post("/corrections")
async def submit_correction(
payload: TermCorrectionSubmit,
@@ -55,10 +57,14 @@ async def submit_correction(
# #endregion submit_correction
# #region submit_bulk_corrections [TYPE Function]
# #region submit_bulk_corrections [C:4] [TYPE Function]
# @BRIEF Submit multiple term corrections atomically.
# @PRE: User has translate.dictionary.edit permission.
# @POST: All corrections applied or none with conflict list.
# @PRE User has translate.dictionary.edit permission.
# @POST All corrections applied or none with conflict list.
# @SIDE_EFFECT Creates/updates DictionaryEntry rows; commits once.
# @RELATION DEPENDS_ON -> [DictionaryManager]
# @COMPLEXITY 4
@router.post("/corrections/bulk")
async def submit_bulk_corrections(
payload: TermCorrectionBulkSubmit,

View File

@@ -22,7 +22,7 @@ from ._router import router
# Terminology Dictionaries
# ============================================================
# #region list_dictionaries [TYPE Function]
# #region list_dictionaries [C:4] [TYPE Function]
# @BRIEF List all terminology dictionaries.
# @PRE: User has translate.dictionary.view permission.
# @POST: Returns list of dictionaries with total count.
@@ -46,7 +46,7 @@ async def list_dictionaries(
# #endregion list_dictionaries
# #region get_dictionary [TYPE Function]
# #region get_dictionary [C:4] [TYPE Function]
# @BRIEF Get a single terminology dictionary by ID.
# @PRE: User has translate.dictionary.view permission.
# @POST: Returns the dictionary with entry_count.
@@ -71,7 +71,7 @@ async def get_dictionary(
# #endregion get_dictionary
# #region create_dictionary [TYPE Function]
# #region create_dictionary [C:4] [TYPE Function]
# @BRIEF Create a new terminology dictionary.
# @PRE: User has translate.dictionary.create permission.
# @POST: Returns the created dictionary.
@@ -99,7 +99,7 @@ async def create_dictionary(
# #endregion create_dictionary
# #region update_dictionary [TYPE Function]
# #region update_dictionary [C:4] [TYPE Function]
# @BRIEF Update an existing terminology dictionary.
# @PRE: User has translate.dictionary.edit permission.
# @POST: Returns the updated dictionary.
@@ -133,7 +133,7 @@ async def update_dictionary(
# #endregion update_dictionary
# #region delete_dictionary [TYPE Function]
# #region delete_dictionary [C:4] [TYPE Function]
# @BRIEF Delete a terminology dictionary, blocked if attached to active/scheduled jobs.
# @PRE: User has translate.dictionary.delete permission.
# @POST: Dictionary is deleted.
@@ -162,7 +162,7 @@ async def delete_dictionary(
# Dictionary Entries CRUD
# ============================================================
# #region list_dictionary_entries [TYPE Function]
# #region list_dictionary_entries [C:4] [TYPE Function]
# @BRIEF List entries for a dictionary, optionally filtered by language pair.
# @PRE: User has translate.dictionary.view permission.
# @POST: Returns paginated list of entries with language pair fields.
@@ -220,7 +220,7 @@ async def list_dictionary_entries(
# #endregion list_dictionary_entries
# #region add_dictionary_entry [TYPE Function]
# #region add_dictionary_entry [C:4] [TYPE Function]
# @BRIEF Add a new entry to a dictionary.
# @PRE: User has translate.dictionary.edit permission.
# @POST: Entry is created.
@@ -269,7 +269,7 @@ async def add_dictionary_entry(
# #endregion add_dictionary_entry
# #region edit_dictionary_entry [TYPE Function]
# #region edit_dictionary_entry [C:4] [TYPE Function]
# @BRIEF Update an existing dictionary entry.
# @PRE: User has translate.dictionary.edit permission.
# @POST: Entry is updated.
@@ -319,7 +319,7 @@ async def edit_dictionary_entry(
# #endregion edit_dictionary_entry
# #region delete_dictionary_entry [TYPE Function]
# #region delete_dictionary_entry [C:4] [TYPE Function]
# @BRIEF Delete a dictionary entry.
# @PRE: User has translate.dictionary.edit permission.
# @POST: Entry is deleted.
@@ -343,10 +343,12 @@ async def delete_dictionary_entry(
# #endregion delete_dictionary_entry
# #region import_dictionary_entries [TYPE Function]
# #region import_dictionary_entries [C:4] [TYPE Function]
# @BRIEF Import entries into a terminology dictionary from CSV/TSV content.
# @PRE: User has translate.dictionary.edit permission.
# @POST: Entries are imported per conflict mode.
# @PRE User has translate.dictionary.edit permission.
# @POST Entries are imported per conflict mode.
# @COMPLEXITY 4
@router.post("/dictionaries/{dictionary_id}/import")
async def import_dictionary_entries(
dictionary_id: str,

View File

@@ -1,6 +1,6 @@
# #region TranslateHelpersModule [C:2] [TYPE Module] [SEMANTICS sqlalchemy, translate, helper, query, mapping]
# @BRIEF Shared helper functions for translate route handlers.
# @LAYER: API
# @LAYER API
# @RELATION DEPENDS_ON -> [models.translate]
from typing import Any

View File

@@ -31,7 +31,7 @@ from ._router import router
# Translation Job CRUD
# ============================================================
# #region list_jobs [TYPE Function]
# #region list_jobs [C:4] [TYPE Function]
# @BRIEF List all translation jobs.
# @PRE: User has translate.job.view permission.
# @POST: Returns list of translation jobs.
@@ -64,7 +64,7 @@ async def list_jobs(
# #endregion list_jobs
# #region get_job [TYPE Function]
# #region get_job [C:4] [TYPE Function]
# @BRIEF Get a single translation job by ID.
# @PRE: User has translate.job.view permission.
# @POST: Returns the translation job.
@@ -88,7 +88,7 @@ async def get_job(
# #endregion get_job
# #region create_job [TYPE Function]
# #region create_job [C:4] [TYPE Function]
# @BRIEF Create a new translation job.
# @PRE: User has translate.job.create permission.
# @POST: Returns the created translation job.
@@ -113,7 +113,7 @@ async def create_job(
# #endregion create_job
# #region update_job [TYPE Function]
# #region update_job [C:4] [TYPE Function]
# @BRIEF Update an existing translation job.
# @PRE: User has translate.job.edit permission.
# @POST: Returns the updated translation job.
@@ -139,7 +139,7 @@ async def update_job(
# #endregion update_job
# #region delete_job [TYPE Function]
# #region delete_job [C:4] [TYPE Function]
# @BRIEF Delete a translation job.
# @PRE: User has translate.job.delete permission.
# @POST: Job is deleted.
@@ -161,7 +161,7 @@ async def delete_job(
# #endregion delete_job
# #region duplicate_job [TYPE Function]
# #region duplicate_job [C:4] [TYPE Function]
# @BRIEF Duplicate a translation job.
# @PRE: User has translate.job.create permission.
# @POST: Returns the duplicated job with status DRAFT.
@@ -188,13 +188,14 @@ async def duplicate_job(
# #endregion duplicate_job
# #region get_datasource_columns [TYPE Function]
# #region get_datasource_columns [C:4] [TYPE Function]
# @BRIEF Get column metadata and database dialect for a Superset datasource.
# @PRE: User has translate.job.view permission.
# @POST: Returns column list with metadata and database_dialect.
# @SIDE_EFFECT: Queries Superset API for dataset detail and database info.
# @RATIONALE: Dialect detection from Superset connection cached on TranslationJob at save time.
# @REJECTED: Hardcoding dialect list per database — would drift from actual Superset config.
# @PRE User has translate.job.view permission.
# @POST Returns column list with metadata and database_dialect.
# @SIDE_EFFECT Queries Superset API for dataset detail and database info.
# @RATIONALE Dialect detection from Superset connection cached on TranslationJob at save time.
# @REJECTED Hardcoding dialect list per database — would drift from actual Superset config.
# @COMPLEXITY 4
@router.get("/datasources")
async def list_datasources(

View File

@@ -17,7 +17,7 @@ from ._router import router
# Metrics
# ============================================================
# #region get_metrics [TYPE Function]
# #region get_metrics [C:4] [TYPE Function]
# @BRIEF Get translation metrics, optionally filtered by job.
# @PRE: User has translate.metrics.view permission.
# @POST: Returns metrics data.
@@ -40,10 +40,12 @@ async def get_metrics(
# #endregion get_metrics
# #region get_job_metrics [TYPE Function]
# #region get_job_metrics [C:4] [TYPE Function]
# @BRIEF Get aggregated metrics for a specific job.
# @PRE: User has translate.metrics.view permission.
# @POST: Returns metrics dict.
# @PRE User has translate.metrics.view permission.
# @POST Returns metrics dict.
# @COMPLEXITY 4
@router.get("/jobs/{job_id}/metrics")
async def get_job_metrics(
job_id: str,

View File

@@ -23,7 +23,7 @@ from ._router import router
# Preview
# ============================================================
# #region preview_translation [TYPE Function]
# #region preview_translation [C:4] [TYPE Function]
# @BRIEF Preview a translation before applying it.
# @PRE: User has translate.job.execute permission.
# @POST: Returns a preview session with records and cost estimation.
@@ -59,7 +59,7 @@ async def preview_translation(
# #endregion preview_translation
# #region update_preview_row [TYPE Function]
# #region update_preview_row [C:4] [TYPE Function]
# @BRIEF Approve, edit, or reject a preview row (optionally per language).
# @PRE: User has translate.job.execute permission.
# @POST: Preview row status is updated.
@@ -91,7 +91,7 @@ async def update_preview_row(
# #endregion update_preview_row
# #region accept_preview_session [TYPE Function]
# #region accept_preview_session [C:4] [TYPE Function]
# @BRIEF Accept a preview session, marking it as the quality gate for full execution.
# @PRE: User has translate.job.execute permission. Job has an ACTIVE preview session.
# @POST: Preview session is marked as APPLIED; full execution can proceed.
@@ -114,7 +114,7 @@ async def accept_preview_session(
# #endregion accept_preview_session
# #region apply_preview [TYPE Function]
# #region apply_preview [C:4] [TYPE Function]
# @BRIEF Apply a preview session (alias for accept when accepting at session level).
# @PRE: User has translate.job.execute permission.
# @POST: Preview is applied.
@@ -146,10 +146,12 @@ async def apply_preview(
# Preview Records
# ============================================================
# #region get_preview_records [TYPE Function]
# #region get_preview_records [C:4] [TYPE Function]
# @BRIEF Get records for a preview session.
# @PRE: User has translate.job.view permission.
# @POST: Returns list of preview records.
# @PRE User has translate.job.view permission.
# @POST Returns list of preview records.
# @COMPLEXITY 4
@router.get("/preview/{session_id}/records", response_model=list[PreviewRow])
async def get_preview_records(
session_id: str,

View File

@@ -6,6 +6,7 @@ from fastapi import APIRouter
# #region translate_router [TYPE Global]
# @BRIEF APIRouter instance for all translate sub-routes.
router = APIRouter(prefix="/api/translate", tags=["translate"])
# #endregion translate_router

View File

@@ -19,7 +19,7 @@ from ._router import router
# List Runs (cross-job)
# ============================================================
# #region list_runs [TYPE Function]
# #region list_runs [C:4] [TYPE Function]
# @BRIEF List all runs with cross-job filtering and pagination.
# @PRE: User has translate.history.view permission.
# @POST: Returns paginated list of runs.
@@ -126,7 +126,7 @@ async def list_runs(
# Run Detail
# ============================================================
# #region get_run_detail [TYPE Function]
# #region get_run_detail [C:4] [TYPE Function]
# @BRIEF Get detailed run info with config_snapshot, records, events.
# @PRE: User has translate.history.view permission.
# @POST: Returns run detail with records and events.
@@ -169,10 +169,12 @@ async def get_run_detail(
# Download Skipped CSV
# ============================================================
# #region download_skipped_csv [TYPE Function]
# #region download_skipped_csv [C:4] [TYPE Function]
# @BRIEF Download a CSV of skipped translation records from a run.
# @PRE: User has translate.job.view permission.
# @POST: Returns CSV file of skipped records.
# @PRE User has translate.job.view permission.
# @POST Returns CSV file of skipped records.
# @COMPLEXITY 4
@router.get("/runs/{run_id}/skipped.csv")
async def download_skipped_csv(
run_id: str,

View File

@@ -25,7 +25,7 @@ from ._router import router
# Translation Run / Execute
# ============================================================
# #region run_translation [TYPE Function]
# #region run_translation [C:4] [TYPE Function]
# @BRIEF Execute a translation job (trigger a run).
# @PRE: User has translate.job.execute permission.
# @POST: Returns the created translation run.
@@ -164,7 +164,7 @@ def run_translation(
# #endregion run_translation
# #region retry_run [TYPE Function]
# #region retry_run [C:4] [TYPE Function]
# @BRIEF Retry failed batches in a translation run.
# @PRE: User has translate.job.execute permission.
# @POST: Returns the updated translation run.
@@ -190,7 +190,7 @@ def retry_run(
# #endregion retry_run
# #region retry_insert [TYPE Function]
# #region retry_insert [C:4] [TYPE Function]
# @BRIEF Retry the SQL insert phase for a completed run.
# @PRE: User has translate.job.execute permission.
# @POST: Returns the updated run.
@@ -216,7 +216,7 @@ def retry_insert(
# #endregion retry_insert
# #region cancel_run [TYPE Function]
# #region cancel_run [C:4] [TYPE Function]
# @BRIEF Cancel a running translation.
# @PRE: User has translate.job.execute permission.
# @POST: Run is cancelled.
@@ -239,7 +239,7 @@ def cancel_run(
# #endregion cancel_run
# #region get_run_history [TYPE Function]
# #region get_run_history [C:4] [TYPE Function]
# @BRIEF Get run history for a translation job.
# @PRE: User has translate.history.view permission.
# @POST: Returns list of runs.
@@ -264,7 +264,7 @@ def get_run_history(
# #endregion get_run_history
# #region get_run_status [TYPE Function]
# #region get_run_status [C:4] [TYPE Function]
# @BRIEF Get status and statistics for a translation run.
# @PRE: User has translate.history.view permission.
# @POST: Returns run details with statistics.
@@ -286,7 +286,7 @@ def get_run_status(
# #endregion get_run_status
# #region get_run_records [TYPE Function]
# #region get_run_records [C:4] [TYPE Function]
# @BRIEF Get paginated records for a translation run.
# @PRE: User has translate.history.view permission.
# @POST: Returns paginated records.
@@ -315,7 +315,7 @@ def get_run_records(
# Batches
# ============================================================
# #region get_batches [TYPE Function]
# #region get_batches [C:4] [TYPE Function]
# @BRIEF Get batches for a translation run.
# @PRE: User has translate.job.view permission.
# @POST: Returns list of batches.
@@ -360,7 +360,7 @@ def get_batches(
# Manual Language Override
# ============================================================
# #region override_detected_language [TYPE Function]
# #region override_detected_language [C:4] [TYPE Function]
# @BRIEF Manually override the detected source language for a specific translation language entry.
# @PRE: User has translate.job.execute permission. Run, record, and language entry exist.
# @POST: TranslationLanguage.source_language_detected is updated; language_overridden is set to True.
@@ -439,7 +439,7 @@ def override_detected_language(
# Inline Correction
# ============================================================
# #region inline_edit_translation [C:3] [TYPE Function] [SEMANTICS api,translate,correction]
# #region inline_edit_translation [C:4] [TYPE Function] [SEMANTICS api,translate,correction]
# @BRIEF Apply an inline correction to a translated value on a completed run result.
# @PRE: User has translate.job.execute permission. Run, record, and language entry exist.
# @POST: TranslationLanguage.final_value and user_edit are updated. Optional dictionary submission.
@@ -488,10 +488,12 @@ def inline_edit_translation(
# Bulk Find-and-Replace
# ============================================================
# #region bulk_find_replace [C:3] [TYPE Function] [SEMANTICS api,translate,bulk,replace]
# #region bulk_find_replace [C:4] [TYPE Function] [SEMANTICS api,translate,bulk,replace]
# @BRIEF Perform bulk find-and-replace on translated values within a run.
# @PRE: User has translate.job.execute permission. Run exists.
# @POST: If preview=false, matching translations are updated. Optional dictionary submission.
# @PRE User has translate.job.execute permission. Run exists.
# @POST If preview=false, matching translations are updated. Optional dictionary submission.
# @COMPLEXITY 4
@router.post("/runs/{run_id}/bulk-replace")
def bulk_find_replace(
run_id: str,

View File

@@ -18,7 +18,7 @@ from ._router import router
# Schedule
# ============================================================
# #region get_schedule [TYPE Function]
# #region get_schedule [C:4] [TYPE Function]
# @BRIEF Get the schedule for a translation job.
# @PRE: User has translate.schedule.view permission.
# @POST: Returns the schedule configuration.
@@ -53,7 +53,7 @@ async def get_schedule(
# #endregion get_schedule
# #region set_schedule [TYPE Function]
# #region set_schedule [C:4] [TYPE Function]
# @BRIEF Set or update the schedule for a translation job.
# @PRE: User has translate.schedule.manage permission.
# @POST: Schedule is created or updated.
@@ -119,7 +119,7 @@ async def set_schedule(
# #endregion set_schedule
# #region enable_schedule [TYPE Function]
# #region enable_schedule [C:4] [TYPE Function]
# @BRIEF Enable a schedule for a translation job.
# @PRE: User has translate.schedule.manage permission.
# @POST: Schedule is enabled.
@@ -151,7 +151,7 @@ async def enable_schedule(
# #endregion enable_schedule
# #region disable_schedule [TYPE Function]
# #region disable_schedule [C:4] [TYPE Function]
# @BRIEF Disable a schedule for a translation job.
# @PRE: User has translate.schedule.manage permission.
# @POST: Schedule is disabled.
@@ -177,7 +177,7 @@ async def disable_schedule(
# #endregion disable_schedule
# #region delete_schedule [TYPE Function]
# #region delete_schedule [C:4] [TYPE Function]
# @BRIEF Delete the schedule for a translation job.
# @PRE: User has translate.schedule.manage permission.
# @POST: Schedule is removed.
@@ -203,10 +203,12 @@ async def delete_schedule(
# #endregion delete_schedule
# #region get_next_executions [TYPE Function]
# #region get_next_executions [C:4] [TYPE Function]
# @BRIEF Preview next N executions for a job's schedule.
# @PRE: User has translate.schedule.view permission.
# @POST: Returns next execution times.
# @PRE User has translate.schedule.view permission.
# @POST Returns next execution times.
# @COMPLEXITY 4
@router.get("/jobs/{job_id}/schedule/next-executions")
async def get_next_executions(
job_id: str,

View File

@@ -1,7 +1,7 @@
# #region TranslateModels [C:3] [TYPE Module] [SEMANTICS sqlalchemy, translate, model, schema, dashboard, llm]
# @BRIEF SQLAlchemy ORM models for LLM-based SQL/dashboard translation across dialects.
# @LAYER: Domain
# @RELATION INHERITS_FROM -> MappingModels:Base
# @RELATION INHERITS -> [MappingModels]
import uuid
from datetime import UTC, datetime

View File

@@ -1,4 +1,4 @@
# #region GitPluginModule [TYPE Module] [SEMANTICS git, versioning, deploy, sync, superset]
# #region GitPluginModule [C:4] [TYPE Module] [SEMANTICS git, versioning, deploy, sync, superset, backup, transactional]
#
# @BRIEF Предоставляет плагин для версионирования и развертывания дашбордов Superset.
# @LAYER: Plugin
@@ -10,10 +10,13 @@
#
# @INVARIANT: Все операции с Git должны выполняться через GitService.
# @CONSTRAINT: Плагин работает только с распакованными YAML-экспортами Superset.
# @INVARIANT: _handle_sync сохраняет backup управляемых директорий перед удалением;
# при ошибке распаковки backup восстанавливается.
import io
import os
import shutil
import time
import zipfile
from pathlib import Path
from typing import Any
@@ -62,52 +65,33 @@ class GitPlugin(PluginBase):
# endregion __init__
@property
# region id [TYPE Function]
# @PURPOSE: Returns the plugin identifier.
# @PRE: GitPlugin is initialized.
# @POST: Returns 'git-integration'.
# region id [C:1] [TYPE Function]
def id(self) -> str:
with belief_scope("GitPlugin.id"):
return "git-integration"
return "git-integration"
# endregion id
@property
# region name [TYPE Function]
# @PURPOSE: Returns the plugin name.
# @PRE: GitPlugin is initialized.
# @POST: Returns the human-readable name.
# region name [C:1] [TYPE Function]
def name(self) -> str:
with belief_scope("GitPlugin.name"):
return "Git Integration"
return "Git Integration"
# endregion name
@property
# region description [TYPE Function]
# @PURPOSE: Returns the plugin description.
# @PRE: GitPlugin is initialized.
# @POST: Returns the plugin's purpose description.
# region description [C:1] [TYPE Function]
def description(self) -> str:
with belief_scope("GitPlugin.description"):
return "Version control for Superset dashboards"
return "Version control for Superset dashboards"
# endregion description
@property
# region version [TYPE Function]
# @PURPOSE: Returns the plugin version.
# @PRE: GitPlugin is initialized.
# @POST: Returns the version string.
# region version [C:1] [TYPE Function]
def version(self) -> str:
with belief_scope("GitPlugin.version"):
return "0.1.0"
return "0.1.0"
# endregion version
@property
# region ui_route [TYPE Function]
# @PURPOSE: Returns the frontend route for the git plugin.
# @RETURN: str - "/git"
# region ui_route [C:1] [TYPE Function]
def ui_route(self) -> str:
with belief_scope("GitPlugin.ui_route"):
return "/git"
return "/git"
# endregion ui_route
# region get_schema [TYPE Function]
@@ -129,21 +113,13 @@ class GitPlugin(PluginBase):
}
# endregion get_schema
# region initialize [TYPE Function]
# @PURPOSE: Выполняет начальную настройку плагина.
# @PRE: GitPlugin is initialized.
# @POST: Плагин готов к выполнению задач.
# region initialize [C:1] [TYPE Function]
async def initialize(self):
with belief_scope("GitPlugin.initialize"):
app_logger.reason("Initializing Git Integration Plugin logic.", extra={"src": "GitPlugin.initialize"})
app_logger.reason("Initializing Git Integration Plugin logic.", extra={"src": "GitPlugin.initialize"})
# endregion initialize
# region execute [TYPE Function]
# @PURPOSE: Основной метод выполнения задач плагина с поддержкой TaskContext.
# @PRE: task_data содержит 'operation' и 'dashboard_id'.
# @POST: Возвращает результат выполнения операции.
# @PARAM: task_data (Dict[str, Any]) - Данные задачи.
# @PARAM: context (Optional[TaskContext]) - Task context for logging with source attribution.
# @RETURN: Dict[str, Any] - Статус и сообщение.
# region execute [C:3] [TYPE Function]
# @PURPOSE: Main task executor with TaskContext support.
# @RELATION: CALLS -> self._handle_sync
# @RELATION: CALLS -> self._handle_deploy
async def execute(self, task_data: dict[str, Any], context: TaskContext | None = None) -> dict[str, Any]:
@@ -176,41 +152,48 @@ class GitPlugin(PluginBase):
return result
# endregion execute
# region _handle_sync [TYPE Function]
# @PURPOSE: Экспортирует дашборд из Superset и распаковывает в Git-репозиторий.
# region _handle_sync [C:4] [TYPE Function] [SEMANTICS git,sync,backup,transactional]
# @PURPOSE: Экспортирует дашборд из Superset и распаковывает в Git-репозиторий с backup/restore.
# @PRE: Репозиторий для дашборда должен существовать.
# @POST: Файлы в репозитории обновлены до текущего состояния в Superset.
# @PARAM: dashboard_id (int) - ID дашборда.
# @PARAM: source_env_id (Optional[str]) - ID исходного окружения.
# @RETURN: Dict[str, str] - Результат синхронизации.
# Управляемые директории backup-ируются перед удалением; при ошибке распаковки backup восстанавливается.
# @SIDE_EFFECT: Изменяет файлы в локальной рабочей директории репозитория.
# Создаёт временный backup в /tmp/ss-tools-backup-{dashboard_id}-{timestamp}/.
# @RETURN: Dict[str, str] - Результат синхронизации.
# @RELATION: CALLS -> src.services.git_service.GitService.get_repo
# @RELATION: CALLS -> src.core.superset_client.SupersetClient.export_dashboard
async def _handle_sync(self, dashboard_id: int, source_env_id: str | None = None, log=None, git_log=None, superset_log=None) -> dict[str, str]:
with belief_scope("GitPlugin._handle_sync"):
try:
# 1. Получение репозитория
repo = self.git_service.get_repo(dashboard_id)
repo_path = Path(repo.working_dir)
git_log.info(f"Target repo path: {repo_path}")
# 2. Настройка клиента Superset
env = self._get_env(source_env_id)
client = SupersetClient(env)
client.authenticate()
# 3. Экспорт дашборда
superset_log.info(f"Exporting dashboard {dashboard_id} from {env.name}")
zip_bytes, _ = client.export_dashboard(dashboard_id)
# 4. Распаковка с выравниванием структуры (flattening)
git_log.info(f"Unpacking export to {repo_path}")
# Список папок/файлов, которые мы ожидаем от Superset
managed_dirs = ["dashboards", "charts", "datasets", "databases"]
managed_files = ["metadata.yaml"]
# Очистка старых данных перед распаковкой, чтобы не оставалось "призраков"
# Fix 1: Create backup before deleting managed files
backup_dir = Path(f"/tmp/ss-tools-backup-{dashboard_id}-{time.time_ns()}")
for d in managed_dirs:
d_path = repo_path / d
if d_path.exists() and d_path.is_dir():
shutil.copytree(d_path, backup_dir / d)
for f in managed_files:
f_path = repo_path / f
if f_path.exists():
(backup_dir / f).parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(f_path, backup_dir / f)
# Delete old managed files
for d in managed_dirs:
d_path = repo_path / d
if d_path.exists() and d_path.is_dir():
@@ -220,30 +203,45 @@ class GitPlugin(PluginBase):
if f_path.exists():
f_path.unlink()
with zipfile.ZipFile(io.BytesIO(zip_bytes)) as zf:
# Superset экспортирует всё в подпапку dashboard_export_timestamp/
# Нам нужно найти это имя папки
namelist = zf.namelist()
if not namelist:
raise ValueError("Export ZIP is empty")
try:
with zipfile.ZipFile(io.BytesIO(zip_bytes)) as zf:
namelist = zf.namelist()
if not namelist:
raise ValueError("Export ZIP is empty")
root_folder = namelist[0].split('/')[0]
git_log.info(f"Detected root folder in ZIP: {root_folder}")
for member in zf.infolist():
if member.filename.startswith(root_folder + "/") and len(member.filename) > len(root_folder) + 1:
relative_path = member.filename[len(root_folder)+1:]
target_path = repo_path / relative_path
if member.is_dir():
target_path.mkdir(parents=True, exist_ok=True)
else:
target_path.parent.mkdir(parents=True, exist_ok=True)
with zf.open(member) as source, open(target_path, "wb") as target:
shutil.copyfileobj(source, target)
except Exception:
# Fix 1: Restore backup on unzip failure
if backup_dir.exists():
for d in managed_dirs:
src = backup_dir / d
if src.exists():
dst = repo_path / d
if dst.exists():
shutil.rmtree(dst)
shutil.copytree(src, dst)
for f in managed_files:
src = backup_dir / f
if src.exists():
dst = repo_path / f
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(src, dst)
raise
root_folder = namelist[0].split('/')[0]
git_log.info(f"Detected root folder in ZIP: {root_folder}")
# Fix 1: Remove backup on success
if backup_dir.exists():
shutil.rmtree(backup_dir, ignore_errors=True)
for member in zf.infolist():
if member.filename.startswith(root_folder + "/") and len(member.filename) > len(root_folder) + 1:
# Убираем префикс папки
relative_path = member.filename[len(root_folder)+1:]
target_path = repo_path / relative_path
if member.is_dir():
target_path.mkdir(parents=True, exist_ok=True)
else:
target_path.parent.mkdir(parents=True, exist_ok=True)
with zf.open(member) as source, open(target_path, "wb") as target:
shutil.copyfileobj(source, target)
# 5. Автоматический staging изменений (не коммит, чтобы юзер мог проверить diff)
try:
repo.git.add(A=True)
app_logger.reason("Changes staged in git", extra={"src": "_handle_sync"})
@@ -258,35 +256,21 @@ class GitPlugin(PluginBase):
raise
# endregion _handle_sync
# region _handle_deploy [TYPE Function]
# @PURPOSE: Упаковывает репозиторий в ZIP и импортирует в целевое окружение Superset.
# @PRE: environment_id должен соответствовать настроенному окружению.
# @POST: Дашборд импортирован в целевой Superset.
# @PARAM: dashboard_id (int) - ID дашборда.
# @PARAM: env_id (str) - ID целевого окружения.
# @PARAM: log - Main logger instance.
# @PARAM: git_log - Git-specific logger instance.
# @PARAM: superset_log - Superset API-specific logger instance.
# @RETURN: Dict[str, Any] - Результат деплоя.
# @SIDE_EFFECT: Создает и удаляет временный ZIP-файл.
# region _handle_deploy [C:4] [TYPE Function] [SEMANTICS git,deploy,zip]
# @PURPOSE: Packages repository into ZIP and imports into target Superset environment.
# @POST: Dashboard imported into target Superset.
# @SIDE_EFFECT: Creates and removes temporary ZIP file.
# @RELATION: CALLS -> src.core.superset_client.SupersetClient.import_dashboard
async def _handle_deploy(self, dashboard_id: int, env_id: str, log=None, git_log=None, superset_log=None) -> dict[str, Any]:
with belief_scope("GitPlugin._handle_deploy"):
try:
if not env_id:
raise ValueError("Target environment ID required for deployment")
# 1. Получение репозитория
repo = self.git_service.get_repo(dashboard_id)
repo_path = Path(repo.working_dir)
# 2. Упаковка в ZIP
git_log.info(f"Packing repository {repo_path} for deployment.")
zip_buffer = io.BytesIO()
# Superset expects a root directory in the ZIP (e.g., dashboard_export_20240101T000000/)
root_dir_name = f"dashboard_export_{dashboard_id}"
with zipfile.ZipFile(zip_buffer, "w", zipfile.ZIP_DEFLATED) as zf:
for root, dirs, files in os.walk(repo_path):
if ".git" in dirs:
@@ -295,21 +279,14 @@ class GitPlugin(PluginBase):
if file == ".git" or file.endswith(".zip"):
continue
file_path = Path(root) / file
# Prepend the root directory name to the archive path
arcname = Path(root_dir_name) / file_path.relative_to(repo_path)
zf.write(file_path, arcname)
zip_buffer.seek(0)
# 3. Настройка клиента Superset
env = self.config_manager.get_environment(env_id)
if not env:
raise ValueError(f"Environment {env_id} not found")
client = SupersetClient(env)
client.authenticate()
# 4. Импорт
temp_zip_path = repo_path / f"deploy_{dashboard_id}.zip"
git_log.info(f"Saving temporary zip to {temp_zip_path}")
with open(temp_zip_path, "wb") as f:
@@ -329,11 +306,12 @@ class GitPlugin(PluginBase):
raise
# endregion _handle_deploy
# region _get_env [TYPE Function]
# region _get_env [C:4] [TYPE Function] [SEMANTICS env,config,strict]
# @PURPOSE: Вспомогательный метод для получения конфигурации окружения.
# @PARAM: env_id (Optional[str]) - ID окружения.
# @PRE: env_id is a string or None.
# @POST: Returns an Environment object from config or DB.
# When env_id is explicitly provided and not found, raises ValueError (no fallback).
# @RETURN: Environment - Объект конфигурации окружения.
def _get_env(self, env_id: str | None = None):
with belief_scope("GitPlugin._get_env"):
@@ -346,54 +324,46 @@ class GitPlugin(PluginBase):
app_logger.reflect(f"Found environment by ID in ConfigManager: {env.name}", extra={"src": "_get_env"})
return env
# Priority 2: Database (DeploymentEnvironment)
from src.core.database import SessionLocal
from src.models.git import DeploymentEnvironment
# Priority 2: Database (DeploymentEnvironment)
from src.core.database import SessionLocal
from src.models.git import DeploymentEnvironment
db = SessionLocal()
try:
if env_id:
db_env = db.query(DeploymentEnvironment).filter(DeploymentEnvironment.id == env_id).first()
else:
# If no ID, try to find active or any environment in DB
db_env = db.query(DeploymentEnvironment).filter(DeploymentEnvironment.is_active).first()
if not db_env:
db_env = db.query(DeploymentEnvironment).first()
db = SessionLocal()
try:
if env_id:
db_env = db.query(DeploymentEnvironment).filter(DeploymentEnvironment.id == env_id).first()
else:
db_env = db.query(DeploymentEnvironment).filter(DeploymentEnvironment.is_active).first()
if not db_env:
db_env = db.query(DeploymentEnvironment).first()
if db_env:
app_logger.reflect(f"Found environment in DB: {db_env.name}", extra={"src": "_get_env"})
from src.core.config_models import Environment
# Use token as password for SupersetClient
return Environment(
id=db_env.id,
name=db_env.name,
url=db_env.superset_url,
username="admin",
password=db_env.superset_token,
verify_ssl=True
)
finally:
db.close()
if db_env:
app_logger.reflect(f"Found environment in DB: {db_env.name}", extra={"src": "_get_env"})
from src.core.config_models import Environment
return Environment(
id=db_env.id,
name=db_env.name,
url=db_env.superset_url,
username="admin",
password=db_env.superset_token,
verify_ssl=True
)
# Priority 3: ConfigManager Default (if no env_id provided)
envs = self.config_manager.get_environments()
if envs:
if env_id:
# If env_id was provided but not found in DB or specifically by ID in config,
# but we have other envs, maybe it's one of them?
env = next((e for e in envs if e.id == env_id), None)
if env:
app_logger.reflect(f"Found environment {env_id} in ConfigManager list", extra={"src": "_get_env"})
return env
# Fix 7: Strict check — if env_id was explicitly provided but not found, raise immediately
if env_id:
raise ValueError(f"Environment '{env_id}' not found in ConfigManager or database")
finally:
db.close()
if not env_id:
app_logger.reflect(f"Using first environment from ConfigManager: {envs[0].name}", extra={"src": "_get_env"})
return envs[0]
# Priority 3: fallback to first env only when no specific env_id was requested
envs = self.config_manager.get_environments()
if envs and not env_id:
app_logger.reflect(f"Using first environment from ConfigManager: {envs[0].name}", extra={"src": "_get_env"})
return envs[0]
app_logger.error(f"[_get_env][Coherence:Failed] No environments configured (searched config.json and DB). env_id={env_id}")
raise ValueError("No environments configured. Please add a Superset Environment in Settings.")
app_logger.error(f"[_get_env][Coherence:Failed] No environments configured (searched config.json and DB). env_id={env_id}")
raise ValueError("No environments configured. Please add a Superset Environment in Settings.")
# endregion _get_env
# endregion initialize
# #endregion GitPlugin
# #endregion GitPluginModule

View File

@@ -1,3 +1,2 @@
# #region TranslatePluginPackage [TYPE Package] [SEMANTICS translate, plugin, package, sql, dashboard]
# @BRIEF Translation plugin package for LLM-based cross-Superset SQL/dashboard translation.
# #endregion TranslatePluginPackage

View File

@@ -9,24 +9,24 @@
# Fixed batch_size of 50 — causes truncation on long-content rows.
# #region DEFAULT_CONTEXT_WINDOW [TYPE Constant]
# @BRIEF Default context window for DeepSeek v4 Flash: up to 64K tokens.
DEFAULT_CONTEXT_WINDOW = 64000
# #endregion DEFAULT_CONTEXT_WINDOW
# #region DEFAULT_MAX_OUTPUT_TOKENS [TYPE Constant]
# @BRIEF Default max_tokens setting for LLM output (16384 — sufficient for 50 rows x 4 languages).
# #region DEFAULT_MAX_OUTPUT_TOKENS [TYPE Constant]
DEFAULT_MAX_OUTPUT_TOKENS = 16384
# #endregion DEFAULT_MAX_OUTPUT_TOKENS
# #region REASONING_OVERHEAD [TYPE Constant]
# @BRIEF CoT reasoning overhead tokens for DeepSeek models (~2000 tokens for chain-of-thought).
# #region REASONING_OVERHEAD [TYPE Constant]
REASONING_OVERHEAD = 2000
# #endregion REASONING_OVERHEAD
# #region PROVIDER_DEFAULTS [TYPE Constant]
# @BRIEF Provider-aware defaults for context_window and max_output_tokens.
# Maps model name (or "default" fallback) to capacity limits.
# @RATIONALE: Different providers have drastically different context windows and
# @RATIONALE Different providers have drastically different context windows and
# output limits. Using a single default for all causes either wasted
# capacity (underestimation) or truncation (overestimation).
PROVIDER_DEFAULTS: dict[str, dict[str, int]] = {
@@ -38,46 +38,46 @@ PROVIDER_DEFAULTS: dict[str, dict[str, int]] = {
"default": {"context_window": 64000, "max_output_tokens": 16384},
}
# #endregion PROVIDER_DEFAULTS
# Increased from 60 to 120 because SQL/dashboard text and JSON structure need more.
# #region OUTPUT_PER_ROW_PER_LANG [TYPE Constant]
# @BRIEF Estimated output tokens per row per language in JSON response format.
# Increased from 60 to 120 because SQL/dashboard text and JSON structure need more.
OUTPUT_PER_ROW_PER_LANG = 120
# #endregion OUTPUT_PER_ROW_PER_LANG
# #endregion JSON_OVERHEAD_PER_ROW
# #region JSON_OVERHEAD_PER_ROW [TYPE Constant]
# @BRIEF Estimated overhead tokens for JSON keys, brackets, and formatting per row.
JSON_OVERHEAD_PER_ROW = 50
# #endregion JSON_OVERHEAD_PER_ROW
# #endregion PROMPT_BASE_TOKENS
# #region PROMPT_BASE_TOKENS [TYPE Constant]
# @BRIEF Base tokens for system prompt + instructions + dictionary section + JSON format specification.
# Increased from 300 to 600 to account for longer template, system msg, and dict preamble.
PROMPT_BASE_TOKENS = 600
# #endregion PROMPT_BASE_TOKENS
# @BRIEF Cap for dictionary tokens in estimation to avoid overestimating.
# #region DICT_TOKENS_PER_ENTRY [TYPE Constant]
# @BRIEF Estimated tokens per dictionary entry in the glossary section.
DICT_TOKENS_PER_ENTRY = 20
# #endregion DICT_TOKENS_PER_ENTRY
# #region DICT_TOKENS_MAX [TYPE Constant]
# @BRIEF Cap for dictionary tokens in estimation to avoid overestimating.
DICT_TOKENS_MAX = 5000
# #endregion DICT_TOKENS_MAX
MIN_MAX_TOKENS = 4096
# #region CHARS_PER_TOKEN_MIXED [TYPE Constant]
# @BRIEF Characters per token for mixed Russian/English text (empirical ~2.2 for mixed, ~2 for Russian, ~4 for English).
CHARS_PER_TOKEN_MIXED = 2.2
# #endregion CHARS_PER_TOKEN_MIXED
MAX_OUTPUT_HEADROOM = 3000
# #region MIN_MAX_TOKENS [TYPE Constant]
# @BRIEF Minimum max_tokens to allow (4096 ensures reasonable output even for small batches).
MIN_MAX_TOKENS = 4096
# #endregion MIN_MAX_TOKENS
# #endregion MIN_MAX_TOKENS
# #region MAX_OUTPUT_HEADROOM [TYPE Constant]
# @BRIEF Extra headroom added to max_output_needed beyond the estimate (3000 = 10-20% for variance).
# Increased from 1000 to 3000 because SQL/dashboard text output varies significantly.
MAX_OUTPUT_HEADROOM = 3000
# #endregion MAX_OUTPUT_HEADROOM

View File

@@ -1,13 +1,16 @@
# #region DictionaryManagerModule [C:4] [TYPE Module] [SEMANTICS sqlalchemy, translate, dictionary, batch, term]
# @BRIEF Business logic for terminology dictionary management, entry CRUD, CSV/TSV import with conflict detection, and per-batch filtering.
# @LAYER: Domain
# @RELATION DEPENDS_ON -> [TerminologyDictionary:Class]
# @RELATION DEPENDS_ON -> [DictionaryEntry:Class]
# @RELATION DEPENDS_ON -> [TranslationJobDictionary:Class]
# @LAYER Domain
# @RELATION DEPENDS_ON -> [TranslationJob:Class]
# @RATIONALE: C4 complexity because dictionary CRUD is stateful with referential integrity enforcement on deletion and unique constraint on normalized source term.
# @REJECTED: Pure C3 CRUD without state guards would allow orphaned job-dictionary links.
# @REJECTED: "Keep both" as conflict option — UniqueConstraint(dictionary_id, source_term_normalized) prohibits variants; only overwrite/keep existing.
# @RELATION DEPENDS_ON -> [TranslationJob:Class]
# @RELATION DEPENDS_ON -> [TranslationJob:Class]
# @RELATION DEPENDS_ON -> [TranslationJob:Class]
# @RATIONALE C4 complexity because dictionary CRUD is stateful with referential integrity enforcement on deletion and unique constraint on normalized source term.
# @REJECTED "Keep both" as conflict option — UniqueConstraint(dictionary_id, source_term_normalized) prohibits variants; only overwrite/keep existing.
# @REJECTED "Keep both" as conflict option — UniqueConstraint(dictionary_id, source_term_normalized) prohibits variants; only overwrite/keep existing.
# @POST Dictionary and entry mutations are persisted with conflict detection.
# @PRE Database session is open and valid. Dictionary manager is properly initialized.
# @SIDE_EFFECT Creates, updates, deletes TerminologyDictionary and DictionaryEntry rows; enforces deletion guards.
import csv
import io
@@ -42,19 +45,18 @@ def _validate_bcp47(tag: str, field_name: str) -> None:
f"{field_name} is not a valid BCP-47 tag: '{tag}'. "
"Expected format like 'en', 'ru', 'zh-CN', 'pt-BR'."
)
# #endregion _validate_bcp47
# #region DictionaryManager [C:4] [TYPE Class]
# @BRIEF Manages terminology dictionaries and their entries with referential integrity.
# @PRE: Database session is open and valid.
# @POST: Dictionary and entry mutations are persisted with conflict detection.
# @SIDE_EFFECT: Creates, updates, deletes TerminologyDictionary and DictionaryEntry rows; enforces deletion guards.
# @PRE Database session is open and valid.
# @POST Dictionary and entry mutations are persisted with conflict detection.
# @SIDE_EFFECT Creates, updates, deletes TerminologyDictionary and DictionaryEntry rows; enforces deletion guards.
# @RELATION DEPENDS_ON -> [TerminologyDictionary:Class], DEPENDS_ON -> [DictionaryEntry:Class], DEPENDS_ON -> [TranslationJobDictionary:Class], DEPENDS_ON -> [TranslationJob:Class]
class DictionaryManager:
# region DictionaryManager.create_dictionary [TYPE Function]
# @PURPOSE: Create a new terminology dictionary (language independent at dictionary level; language pairs are per-entry).
# @PRE: name is provided.
# @POST: New TerminologyDictionary row is created and returned.
# @PURPOSE Create a new terminology dictionary (language independent at dictionary level; language pairs are per-entry).
# @PRE name is provided.
# @POST New TerminologyDictionary row is created and returned.
@staticmethod
def create_dictionary(
db: Session, name: str,
@@ -81,9 +83,9 @@ class DictionaryManager:
# endregion DictionaryManager.create_dictionary
# region DictionaryManager.update_dictionary [TYPE Function]
# @PURPOSE: Update an existing terminology dictionary.
# @PRE: dict_id exists in terminology_dictionaries table.
# @POST: Dictionary metadata is updated and returned.
# @PURPOSE Update an existing terminology dictionary.
# @PRE dict_id exists in terminology_dictionaries table.
# @POST Dictionary metadata is updated and returned.
@staticmethod
def update_dictionary(
db: Session, dict_id: str, name: str | None = None,
@@ -112,10 +114,10 @@ class DictionaryManager:
# endregion DictionaryManager.update_dictionary
# region DictionaryManager.delete_dictionary [TYPE Function]
# @PURPOSE: Delete a dictionary, blocked if attached to active/scheduled jobs.
# @PRE: dict_id exists.
# @POST: Dictionary and its entries are deleted, unless attached to ACTIVE/READY/RUNNING/SCHEDULED jobs.
# @SIDE_EFFECT: Deletes TerminologyDictionary row and all DictionaryEntry rows.
# @PURPOSE Delete a dictionary, blocked if attached to active/scheduled jobs.
# @PRE dict_id exists.
# @POST Dictionary and its entries are deleted, unless attached to ACTIVE/READY/RUNNING/SCHEDULED jobs.
# @SIDE_EFFECT Deletes TerminologyDictionary row and all DictionaryEntry rows.
@staticmethod
def delete_dictionary(db: Session, dict_id: str) -> None:
with belief_scope("DictionaryManager.delete_dictionary"):
@@ -155,9 +157,9 @@ class DictionaryManager:
# endregion DictionaryManager.delete_dictionary
# region DictionaryManager.get_dictionary [TYPE Function]
# @PURPOSE: Get a single dictionary by ID with entry count.
# @PRE: dict_id exists.
# @POST: Returns dict with dictionary + entry_count or raises ValueError.
# @PURPOSE Get a single dictionary by ID with entry count.
# @PRE dict_id exists.
# @POST Returns dict with dictionary + entry_count or raises ValueError.
@staticmethod
def get_dictionary(db: Session, dict_id: str) -> TerminologyDictionary:
dictionary = db.query(TerminologyDictionary).filter(TerminologyDictionary.id == dict_id).first()
@@ -167,9 +169,9 @@ class DictionaryManager:
# endregion DictionaryManager.get_dictionary
# region DictionaryManager.list_dictionaries [TYPE Function]
# @PURPOSE: List dictionaries with pagination and entry counts.
# @PRE: page >= 1, page_size between 1 and 100.
# @POST: Returns (list of dicts, total_count).
# @PURPOSE List dictionaries with pagination and entry counts.
# @PRE page >= 1, page_size between 1 and 100.
# @POST Returns (list of dicts, total_count).
@staticmethod
def list_dictionaries(
db: Session, page: int = 1, page_size: int = 20,
@@ -186,9 +188,9 @@ class DictionaryManager:
# endregion DictionaryManager.list_dictionaries
# region DictionaryManager.add_entry [TYPE Function]
# @PURPOSE: Add an entry to a dictionary with language-pair-aware uniqueness validation.
# @PRE: dict_id exists. source_term, target_term are non-empty. source_language, target_language are valid BCP-47.
# @POST: New DictionaryEntry row is created or raises on duplicate.
# @PURPOSE Add an entry to a dictionary with language-pair-aware uniqueness validation.
# @PRE dict_id exists. source_term, target_term are non-empty. source_language, target_language are valid BCP-47.
# @POST New DictionaryEntry row is created or raises on duplicate.
@staticmethod
def add_entry(
db: Session, dict_id: str, source_term: str, target_term: str,
@@ -237,9 +239,9 @@ class DictionaryManager:
# endregion DictionaryManager.add_entry
# region DictionaryManager.edit_entry [TYPE Function]
# @PURPOSE: Edit an existing dictionary entry with language-pair-aware duplicate check.
# @PRE: entry_id exists.
# @POST: Entry fields are updated.
# @PURPOSE Edit an existing dictionary entry with language-pair-aware duplicate check.
# @PRE entry_id exists.
# @POST Entry fields are updated.
@staticmethod
def edit_entry(
db: Session, entry_id: str, source_term: str | None = None,
@@ -293,9 +295,9 @@ class DictionaryManager:
# endregion DictionaryManager.edit_entry
# region DictionaryManager.delete_entry [TYPE Function]
# @PURPOSE: Delete a single dictionary entry.
# @PRE: entry_id exists.
# @POST: Entry is deleted.
# @PURPOSE Delete a single dictionary entry.
# @PRE entry_id exists.
# @POST Entry is deleted.
@staticmethod
def delete_entry(db: Session, entry_id: str) -> None:
with belief_scope("DictionaryManager.delete_entry"):
@@ -309,9 +311,9 @@ class DictionaryManager:
# endregion DictionaryManager.delete_entry
# region DictionaryManager.clear_entries [TYPE Function]
# @PURPOSE: Delete all entries for a dictionary.
# @PRE: dict_id exists.
# @POST: All entries for the dictionary are deleted.
# @PURPOSE Delete all entries for a dictionary.
# @PRE dict_id exists.
# @POST All entries for the dictionary are deleted.
@staticmethod
def clear_entries(db: Session, dict_id: str) -> int:
with belief_scope("DictionaryManager.clear_entries"):
@@ -326,9 +328,9 @@ class DictionaryManager:
# endregion DictionaryManager.clear_entries
# region DictionaryManager.list_entries [TYPE Function]
# @PURPOSE: List entries for a dictionary with pagination.
# @PRE: dict_id exists.
# @POST: Returns (list of entries, total_count).
# @PURPOSE List entries for a dictionary with pagination.
# @PRE dict_id exists.
# @POST Returns (list of entries, total_count).
@staticmethod
def list_entries(
db: Session, dict_id: str, page: int = 1, page_size: int = 50,
@@ -351,10 +353,10 @@ class DictionaryManager:
# endregion DictionaryManager.list_entries
# region DictionaryManager.import_entries [TYPE Function]
# @PURPOSE: Import entries from CSV/TSV content with duplicate detection and conflict resolution.
# @PRE: content is valid CSV or TSV. dict_id exists.
# @POST: Entries are created/updated/skipped per conflict mode. Returns result summary.
# @SIDE_EFFECT: Batch-inserts or updates DictionaryEntry rows.
# @PURPOSE Import entries from CSV/TSV content with duplicate detection and conflict resolution.
# @PRE content is valid CSV or TSV. dict_id exists.
# @POST Entries are created/updated/skipped per conflict mode. Returns result summary.
# @SIDE_EFFECT Batch-inserts or updates DictionaryEntry rows.
@staticmethod
def import_entries(
db: Session, dict_id: str, content: str,
@@ -504,9 +506,9 @@ class DictionaryManager:
# endregion DictionaryManager.import_entries
# region DictionaryManager.export_entries [TYPE Function]
# @PURPOSE: Export all entries as CSV string with language columns.
# @PRE: dict_id exists.
# @POST: Returns CSV string with header: source_term, target_term, source_language, target_language, context_notes, context_data, usage_notes.
# @PURPOSE Export all entries as CSV string with language columns.
# @PRE dict_id exists.
# @POST Returns CSV string with header: source_term, target_term, source_language, target_language, context_notes, context_data, usage_notes.
@staticmethod
def export_entries(
db: Session, dict_id: str, delimiter: str = ",",
@@ -549,10 +551,10 @@ class DictionaryManager:
# endregion DictionaryManager.export_entries
# region DictionaryManager.migrate_old_entries [TYPE Function]
# @PURPOSE: Migrate existing single-language dictionary entries to populate source_language and target_language.
# @PRE: db session is open.
# @POST: Entries with "und" source_language or target_language are updated with inferred values.
# @SIDE_EFFECT: Updates DictionaryEntry rows in bulk.
# @PURPOSE Migrate existing single-language dictionary entries to populate source_language and target_language.
# @PRE db session is open.
# @POST Entries with "und" source_language or target_language are updated with inferred values.
# @SIDE_EFFECT Updates DictionaryEntry rows in bulk.
@staticmethod
def migrate_old_entries(db: Session) -> dict[str, int]:
with belief_scope("DictionaryManager.migrate_old_entries"):
@@ -601,12 +603,12 @@ class DictionaryManager:
# endregion DictionaryManager.migrate_old_entries
# region DictionaryManager.filter_for_batch [TYPE Function]
# @PURPOSE: Scan batch texts for case-insensitive, word-boundary-aware matches against all dictionaries attached to a job,
# @PURPOSE Scan batch texts for case-insensitive, word-boundary-aware matches against all dictionaries attached to a job,
# optionally filtered by language pair. When row_context is provided, computes context-aware priority flags.
# @PRE: job_id exists and source_texts is a list of strings.
# @POST: Returns list of matched entries with match info, sorted by priority across attached dictionaries.
# @PRE job_id exists and source_texts is a list of strings.
# @POST Returns list of matched entries with match info, sorted by priority across attached dictionaries.
# When row_context is given, each result includes a 'priority_match' boolean.
# @SIDE_EFFECT: Queries TranslationJobDictionary, TerminologyDictionary, and DictionaryEntry tables.
# @SIDE_EFFECT Queries TranslationJobDictionary, TerminologyDictionary, and DictionaryEntry tables.
@staticmethod
def filter_for_batch(
db: Session, source_texts: list[str], job_id: str,
@@ -738,10 +740,10 @@ class DictionaryManager:
# region DictionaryManager.submit_correction [TYPE Function]
# @PURPOSE: Submit a term correction from a run result. Validates language match, detects conflicts.
# @PRE: source_term, incorrect_target_term, corrected_target_term are non-empty. dict_id exists.
# @POST: Entry created or updated; origin tracking populated. Returns action + conflict info.
# @SIDE_EFFECT: Creates/updates DictionaryEntry row.
# @PURPOSE Submit a term correction from a run result. Validates language match, detects conflicts.
# @PRE source_term, incorrect_target_term, corrected_target_term are non-empty. dict_id exists.
# @POST Entry created or updated; origin tracking populated. Returns action + conflict info.
# @SIDE_EFFECT Creates/updates DictionaryEntry row.
@staticmethod
def submit_correction(
db: Session,
@@ -868,10 +870,10 @@ class DictionaryManager:
# endregion DictionaryManager.submit_correction
# region DictionaryManager.submit_bulk_corrections [TYPE Function]
# @PURPOSE: Submit multiple term corrections atomically — all succeed or all fail.
# @PRE: corrections list is non-empty. dict_id exists.
# @POST: All corrections applied or none applied with conflict list.
# @SIDE_EFFECT: Creates/updates DictionaryEntry rows; commits once.
# @PURPOSE Submit multiple term corrections atomically — all succeed or all fail.
# @PRE corrections list is non-empty. dict_id exists.
# @POST All corrections applied or none applied with conflict list.
# @SIDE_EFFECT Creates/updates DictionaryEntry rows; commits once.
@staticmethod
def submit_bulk_corrections(
db: Session,
@@ -998,5 +1000,8 @@ class DictionaryManager:
# endregion DictionaryManager.submit_bulk_corrections
# #endregion DictionaryManager
# #endregion DictionaryManager
# #endregion DictionaryManagerModule

View File

@@ -194,11 +194,12 @@ def _check_translation_cache(
# #endregion _check_translation_cache
# #region estimate_row_tokens [C:2] [TYPE Function] [SEMANTICS translate, token, estimation]
# #region estimate_row_tokens [C:4] [TYPE Function] [SEMANTICS translate, token, estimation]
# @BRIEF Estimate token count for a single source row including context fields.
# @PRE: source_text is a string.
# @POST: Returns estimated token count >= 1.
# @PRE source_text is a string.
# @POST Returns estimated token count >= 1.
# @RELATION DEPENDS_ON -> [_estimate_tokens_for_text]
def estimate_row_tokens(
source_text: str,
source_data: dict | None,
@@ -1745,30 +1746,47 @@ class TranslationExecutor:
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
}
# Reasoning-safe system prompt: always forbid chain-of-thought in output.
# Reasoning models (DeepSeek, QwQ, etc.) may leak reasoning into content
# when response_format=json_object is set. Explicit suppression in the
# system message prevents this regardless of the disable_reasoning flag.
system_content = (
"You are a database content translation assistant. "
"Translate the provided text accurately, preserving data semantics. "
"Respond directly with ONLY the JSON result. "
"Do NOT include any reasoning, thinking, chain-of-thought, analysis, "
"or explanation. Output ONLY valid JSON."
)
payload = {
"model": model,
"messages": [
{"role": "system", "content": "You are a database content translation assistant. Translate the provided text accurately, preserving data semantics."},
{"role": "system", "content": system_content},
{"role": "user", "content": prompt},
],
"temperature": 0.1,
"max_tokens": max_tokens,
}
# Structured output — OpenRouter and Kilo also support response_format, but some
# upstream providers (e.g. StepFun) reject it. We try with response_format and
# fall back on 400 "structured_outputs is not supported".
# NOTE: Reasoning models (deepseek, qwen with reasoning, etc.) often break with
# response_format=json_object, producing reasoning text instead of JSON.
# When disable_reasoning is set, we remove response_format and rely on the
# system prompt to enforce JSON-only output.
if provider_type in ("openai", "openai_compatible", "kilo", "openrouter", "litellm"):
payload["response_format"] = {"type": "json_object"}
if not disable_reasoning:
payload["response_format"] = {"type": "json_object"}
# Suppress Chain of Thought reasoning to save output tokens
# NOTE: Kilo/OpenRouter/LiteLLM do NOT support disabling reasoning (returns 400)
if disable_reasoning:
if provider_type not in ("kilo", "openrouter", "litellm"):
payload["reasoning_effort"] = "none"
payload.pop("response_format", None)
# response_format already skipped above when disable_reasoning is True
# Use caller-provided max_tokens instead of hardcoded 8192
payload["max_tokens"] = max_tokens
payload["messages"][0] = {"role": "system", "content": "You are a database content translation assistant. Translate the provided text accurately, preserving data semantics. Respond directly with ONLY the JSON result. Do NOT include any reasoning, thinking, chain-of-thought, analysis, or explanation. Output ONLY valid JSON."}
logger.reason(
f"LLM request url={base_url} model={payload.get('model')} "

View File

@@ -1,14 +1,15 @@
# #region TranslationMetrics [C:3] [TYPE Module] [SEMANTICS sqlalchemy, translate, metrics, job, statistics]
# @BRIEF Aggregate translation metrics from live TranslationEvent + MetricSnapshot for per-job reporting.
# @LAYER: Domain
# @RELATION DEPENDS_ON -> [TranslationEvent:Class]
# @RELATION DEPENDS_ON -> [MetricSnapshot:Class]
# @RELATION DEPENDS_ON -> [TranslationRun:Class]
# @LAYER Domain
# @RELATION DEPENDS_ON -> [TranslationSchedule:Class]
# @PRE: Database session is open.
# @POST: Metrics are aggregated and returned; no side effects.
# @RATIONALE: Live events (<90 days) + MetricSnapshot (>=90 days) fusion for complete picture.
# @REJECTED: Querying all events for all time — would be slow; MetricSnapshot provides historical summary.
# @RELATION DEPENDS_ON -> [TranslationSchedule:Class]
# @RELATION DEPENDS_ON -> [TranslationSchedule:Class]
# @RELATION DEPENDS_ON -> [TranslationSchedule:Class]
# @PRE Database session is open.
# @POST Metrics are aggregated and returned; no side effects.
# @RATIONALE Live events (<90 days) + MetricSnapshot (>=90 days) fusion for complete picture.
# @REJECTED Querying all events for all time — would be slow; MetricSnapshot provides historical summary.
# @COMPLEXITY 4
from datetime import UTC, datetime
from typing import Any
@@ -34,9 +35,9 @@ class TranslationMetrics:
self.db = db
# region get_job_metrics [TYPE Function]
# @PURPOSE: Get aggregated metrics for a specific job.
# @PRE: job_id exists.
# @POST: Returns dict with metrics from events + latest snapshot.
# @PURPOSE Get aggregated metrics for a specific job.
# @PRE job_id exists.
# @POST Returns dict with metrics from events + latest snapshot.
def get_job_metrics(self, job_id: str) -> dict[str, Any]:
with belief_scope("TranslationMetrics.get_job_metrics"):
# Run counts from TranslationRun
@@ -174,8 +175,8 @@ class TranslationMetrics:
# endregion get_job_metrics
# region get_all_metrics [TYPE Function]
# @PURPOSE: Get aggregated metrics for all jobs.
# @POST: Returns list of per-job metrics.
# @PURPOSE Get aggregated metrics for all jobs.
# @POST Returns list of per-job metrics.
def get_all_metrics(self) -> list[dict[str, Any]]:
with belief_scope("TranslationMetrics.get_all_metrics"):
job_ids = (

View File

@@ -238,6 +238,15 @@ class TranslationOrchestrator:
"run_id": run.id,
"error": str(e),
})
# Rollback the session if it's in a broken state (e.g. after a
# failed flush from a previous exception in executor.execute_run).
# Without this, subsequent flush()/commit() calls raise
# "This Session's transaction has been rolled back".
try:
self.db.rollback()
except Exception:
pass
run = self.db.merge(run)
run.status = "FAILED"
run.error_message = f"Translation execution failed: {e}"
run.completed_at = datetime.now(UTC)
@@ -887,6 +896,7 @@ class TranslationOrchestrator:
"id": run.id,
"job_id": run.job_id,
"status": run.status,
"trigger_type": run.trigger_type,
"started_at": run.started_at.isoformat() if run.started_at else None,
"completed_at": run.completed_at.isoformat() if run.completed_at else None,
"error_message": run.error_message,

View File

@@ -1,9 +1,9 @@
# #region TranslatePlugin [C:2] [TYPE Module] [SEMANTICS clickhouse, translate, sql, dashboard, dialect]
# @BRIEF TranslatePlugin skeleton for cross-dialect SQL and dashboard translation.
# @LAYER: Domain
# @LAYER Domain
# @RELATION INHERITS -> [PluginBase]
# @RATIONALE: Separate plugin avoids bloating LLMAnalysisPlugin beyond fractal limit (<400 lines).
# @REJECTED: Extending LLMAnalysisPlugin would conflate two distinct feature domains.
# @RATIONALE Separate plugin avoids bloating LLMAnalysisPlugin beyond fractal limit (<400 lines).
# @REJECTED Extending LLMAnalysisPlugin would conflate two distinct feature domains.
from typing import Any

View File

@@ -41,7 +41,7 @@ from ._token_budget import DEFAULT_CONTEXT_WINDOW, estimate_token_budget
from .dictionary import DictionaryManager
# #region DEFAULT_EXECUTION_PROMPT_TEMPLATE [TYPE Constant]
# @BRIEF Default prompt template for batch LLM translation execution (no context columns — faster).
# Supports both single-language and multi-language modes via {target_languages} placeholder.
DEFAULT_EXECUTION_PROMPT_TEMPLATE: str = (
"Translate the following database content from {source_language} to the following language(s): {target_languages}.\n\n"
@@ -63,9 +63,9 @@ DEFAULT_EXECUTION_PROMPT_TEMPLATE: str = (
)
# #endregion DEFAULT_EXECUTION_PROMPT_TEMPLATE
# #region DEFAULT_PREVIEW_PROMPT_TEMPLATE [TYPE Constant]
# @BRIEF Default prompt template for LLM translation preview.
# #region DEFAULT_PREVIEW_PROMPT_TEMPLATE [TYPE Constant]
DEFAULT_PREVIEW_PROMPT_TEMPLATE: str = (
"Translate the following database content.\n\n"
"Source dialect: {source_dialect}\n"
@@ -1038,19 +1038,37 @@ class TranslationPreview:
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
}
# Reasoning-safe system prompt: always forbid chain-of-thought in output.
# Reasoning models (DeepSeek, QwQ, etc.) may leak reasoning into content
# when response_format=json_object is set. Explicit suppression in the
# system message prevents this regardless of the disable_reasoning flag.
system_content = (
"You are a database content translation assistant. "
"Translate the provided text accurately, preserving data semantics. "
"Respond directly with ONLY the JSON result. "
"Do NOT include any reasoning, thinking, chain-of-thought, analysis, "
"or explanation. Output ONLY valid JSON."
)
payload = {
"model": model,
"messages": [
{"role": "system", "content": "You are a database content translation assistant. Translate the provided text accurately, preserving data semantics."},
{"role": "system", "content": system_content},
{"role": "user", "content": prompt},
],
"temperature": 0.1,
"max_tokens": max_tokens,
}
# Structured output — Kilo gateway supports response_format, but upstream providers
# (e.g. StepFun) may reject it. We try with response_format and fall back on 400.
# NOTE: Reasoning models (deepseek variants, qwen with reasoning) often break with
# response_format=json_object, producing reasoning text instead of JSON.
# When disable_reasoning is set, we skip response_format and rely on the
# system prompt to enforce JSON-only output.
if provider_type in ("openai", "openai_compatible", "kilo", "openrouter", "litellm"):
payload["response_format"] = {"type": "json_object"}
if not disable_reasoning:
payload["response_format"] = {"type": "json_object"}
# Suppress Chain of Thought reasoning to save output tokens
# NOTE: Kilo/OpenRouter/LiteLLM reject reasoning_effort — only use for native OpenAI-compatible
@@ -1058,13 +1076,10 @@ class TranslationPreview:
# Kilo/OpenRouter/LiteLLM reject reasoning_effort — only use for native OpenAI-compatible
if provider_type not in ("kilo", "openrouter", "litellm"):
payload["reasoning_effort"] = "none"
payload.pop("response_format", None) # JSON mode triggers reasoning on some models
# response_format already skipped above when disable_reasoning is True
# Use caller-provided max_tokens instead of hardcoded 8192
# This ensures multi-language batches with large output get enough token budget.
payload["max_tokens"] = max_tokens
# Universal instruction — all models understand "respond directly without reasoning"
system_content = "You are a database content translation assistant. Translate the provided text accurately, preserving data semantics. Respond directly with ONLY the JSON result. Do NOT include any reasoning, thinking, chain-of-thought, analysis, or explanation. Output ONLY valid JSON."
payload["messages"][0] = {"role": "system", "content": system_content}
logger.reason(
f"LLM request url={base_url} model={payload.get('model')} "

View File

@@ -1,9 +1,10 @@
# #region ContextAwarePromptBuilder [C:2] [TYPE Module] [SEMANTICS translate, prompt, context, dictionary]
# @BRIEF Pure-function prompt builder that enhances dictionary entries with context annotations.
# @LAYER: Domain
# @LAYER Domain
# @RELATION DEPENDS_ON -> [DictionaryEntry:Class]
# @RATIONALE: Pure functions only — no I/O, no DB access. Separated from executor for testability.
# @REJECTED: Embedding context inline in the executor would make it untestable without mocking DB.
# @RATIONALE Pure functions only — no I/O, no DB access. Separated from executor for testability.
# @REJECTED Embedding context inline in the executor would make it untestable without mocking DB.
#
# Typical workflow:
# 1. Call build_context_entries(dictionary_entries, row_context) to get annotated, prioritized entries

View File

@@ -240,12 +240,14 @@ class TranslationScheduler:
# #endregion TranslationScheduler
# #region execute_scheduled_translation [TYPE Function]
# #region execute_scheduled_translation [C:4] [TYPE Function]
# @BRIEF APScheduler job handler — runs scheduled translation with concurrency check and new-key-only fallback.
# @PRE: schedule_id is valid.
# @POST: Translation run created and executed if no concurrent run exists.
# @SIDE_EFFECT: DB writes; LLM calls; Superset API calls.
# @RATIONALE: New-key-only mode compares keys against most recent succeeded run; baseline_expired fallback does full.
# @PRE schedule_id is valid.
# @POST Translation run created and executed if no concurrent run exists.
# @SIDE_EFFECT DB writes; LLM calls; Superset API calls.
# @RATIONALE New-key-only mode compares keys against most recent succeeded run; baseline_expired fallback does full.
# @COMPLEXITY 4
def execute_scheduled_translation(
schedule_id: str,
job_id: str,

View File

@@ -38,7 +38,7 @@ SUPPORTED_DIALECTS = {
}
# #region get_dialect_from_database [TYPE Function]
# #region get_dialect_from_database [C:4] [TYPE Function]
# @BRIEF Extract normalized dialect string from a Superset database record.
# @PRE: database_record is a dict from Superset API.
# @POST: Returns normalized dialect string or raises ValueError if unsupported.
@@ -84,7 +84,7 @@ def get_dialect_from_database(database_record: dict[str, Any]) -> str:
# #endregion get_dialect_from_database
# #region fetch_datasource_metadata [TYPE Function]
# #region fetch_datasource_metadata [C:4] [TYPE Function]
# @BRIEF Fetch datasource columns and database dialect from Superset.
# @PRE: dataset_id is a valid Superset dataset ID and environment has valid credentials.
# @POST: Returns (columns_list, dialect_string) or raises on failure.
@@ -141,7 +141,7 @@ def fetch_datasource_metadata(
# #endregion fetch_datasource_metadata
# #region detect_virtual_columns [TYPE Function]
# #region detect_virtual_columns [C:4] [TYPE Function]
# @BRIEF Identify virtual (calculated) columns from column metadata.
# @PRE: columns is a list of dicts with 'is_physical' key.
# @POST: Returns list of virtual column names.
@@ -519,11 +519,13 @@ def job_to_response(job: TranslationJob, dict_ids: list[str] | None = None) -> T
# #endregion job_to_response
# #region DatasourceColumnsService [TYPE Function]
# #region DatasourceColumnsService [C:4] [TYPE Function]
# @BRIEF Fetch datasource column metadata from Superset and return structured response.
# @PRE: datasource_id is a valid Superset dataset ID.
# @POST: Returns DatasourceColumnsResponse with column metadata and database dialect.
# @SIDE_EFFECT: Queries Superset API for dataset detail and database info.
# @PRE datasource_id is a valid Superset dataset ID.
# @POST Returns DatasourceColumnsResponse with column metadata and database dialect.
# @SIDE_EFFECT Queries Superset API for dataset detail and database info.
# @COMPLEXITY 4
def get_datasource_columns(
datasource_id: int,
env_id: str,

View File

@@ -58,7 +58,7 @@ def _normalize_timestamp_value(value: Any) -> str | None:
# #endregion _normalize_timestamp_value
# #region _quote_identifier [TYPE Function]
# #region _quote_identifier [C:4] [TYPE Function]
# @BRIEF Quote an identifier per dialect rules. PostgreSQL uses double quotes; ClickHouse uses backticks.
# @PRE: identifier is a non-empty string.
# @POST: Returns safely quoted identifier.
@@ -159,10 +159,12 @@ def generate_insert_sql(
# #endregion generate_insert_sql
# #region generate_upsert_sql [TYPE Function]
# #region generate_upsert_sql [C:4] [TYPE Function]
# @BRIEF Generate PostgreSQL dialect UPSERT SQL with ON CONFLICT DO UPDATE.
# @PRE: dialect is postgresql-compatible. target_table, columns, key_columns are non-empty.
# @POST: Returns UPSERT SQL string or raises ValueError.
# @PRE dialect is postgresql-compatible. target_table, columns, key_columns are non-empty.
# @POST Returns UPSERT SQL string or raises ValueError.
# @COMPLEXITY 4
def generate_upsert_sql(
target_schema: str | None,
target_table: str,

View File

@@ -1,13 +1,13 @@
# #region SupersetSqlLabExecutor [C:4] [TYPE Module] [SEMANTICS translate, superset, sqllab, execute, query]
# @BRIEF Submit SQL to Superset SQL Lab API and poll execution status.
# @LAYER: Infra
# @RELATION DEPENDS_ON -> [SupersetClient]
# @LAYER Infrastructure
# @RELATION DEPENDS_ON -> [ConfigManager]
# @PRE: Valid Superset environment configuration and authenticated client.
# @POST: SQL is submitted to Superset SQL Lab; execution reference is returned.
# @SIDE_EFFECT: Makes HTTP calls to Superset /api/v1/sqllab/execute/; polls status.
# @RATIONALE: Direct SQL Lab API submission provides audit trail and execution monitoring within Superset.
# @REJECTED: Direct database connection bypass — would skip Superset's SQL Lab audit and RBAC.
# @RELATION DEPENDS_ON -> [ConfigManager]
# @PRE Valid Superset environment configuration and authenticated client.
# @POST SQL is submitted to Superset SQL Lab; execution reference is returned.
# @SIDE_EFFECT Makes HTTP calls to Superset /api/v1/sqllab/execute/; polls status.
# @RATIONALE Direct SQL Lab API submission provides audit trail and execution monitoring within Superset.
# @REJECTED Direct database connection bypass — would skip Superset's SQL Lab audit and RBAC.
import json
import time
@@ -21,9 +21,9 @@ from ...core.superset_client import SupersetClient
# #region SupersetSqlLabExecutor [C:4] [TYPE Class]
# @BRIEF Submit SQL to Superset SQL Lab API with polling and status tracking.
# @PRE: Valid environment ID and ConfigManager.
# @POST: SQL submitted to Superset; execution reference recorded.
# @SIDE_EFFECT: Makes HTTP POST to /api/v1/sqllab/execute/; polls GET for status.
# @PRE Valid environment ID and ConfigManager.
# @POST SQL submitted to Superset; execution reference recorded.
# @SIDE_EFFECT Makes HTTP POST to /api/v1/sqllab/execute/; polls GET for status.
class SupersetSqlLabExecutor:
def __init__(self, config_manager: ConfigManager, env_id: str):
@@ -34,10 +34,10 @@ class SupersetSqlLabExecutor:
self._database_backend: str | None = None
# region _get_client [TYPE Function]
# @PURPOSE: Lazy-initialize SupersetClient for the configured environment.
# @PRE: env_id must correspond to a valid environment config.
# @POST: Returns authenticated SupersetClient.
# @SIDE_EFFECT: Authenticates against Superset API.
# @PURPOSE Lazy-initialize SupersetClient for the configured environment.
# @PRE env_id must correspond to a valid environment config.
# @POST Returns authenticated SupersetClient.
# @SIDE_EFFECT Authenticates against Superset API.
def _get_client(self) -> SupersetClient:
with belief_scope("SupersetSqlLabExecutor._get_client"):
if self._client is not None:
@@ -53,10 +53,10 @@ class SupersetSqlLabExecutor:
# endregion _get_client
# region resolve_database_id [TYPE Function]
# @PURPOSE: Resolve the target database ID from the environment and fetch its backend engine.
# @PRE: database_name or database_id should be known.
# @POST: Returns database_id integer or raises ValueError. Also sets self._database_backend.
# @SIDE_EFFECT: Fetches databases list from Superset; optionally fetches single DB for backend.
# @PURPOSE Resolve the target database ID from the environment and fetch its backend engine.
# @PRE database_name or database_id should be known.
# @POST Returns database_id integer or raises ValueError. Also sets self._database_backend.
# @SIDE_EFFECT Fetches databases list from Superset; optionally fetches single DB for backend.
def resolve_database_id(
self,
database_name: str | None = None,
@@ -121,17 +121,17 @@ class SupersetSqlLabExecutor:
# endregion resolve_database_id
# region get_database_backend [TYPE Function]
# @PURPOSE: Return the cached database backend/engine string.
# @POST: Returns backend string or None if not yet resolved.
# @PURPOSE Return the cached database backend/engine string.
# @POST Returns backend string or None if not yet resolved.
def get_database_backend(self) -> str | None:
return self._database_backend
# endregion get_database_backend
# region execute_sql [TYPE Function]
# @PURPOSE: Submit SQL to Superset SQL Lab and return execution tracking info.
# @PRE: sql is valid SQL string. database_id is a valid Superset DB ID.
# @POST: Returns execution result dict with query_id and status.
# @SIDE_EFFECT: Makes HTTP POST to /api/v1/sqllab/execute/.
# @PURPOSE Submit SQL to Superset SQL Lab and return execution tracking info.
# @PRE sql is valid SQL string. database_id is a valid Superset DB ID.
# @POST Returns execution result dict with query_id and status.
# @SIDE_EFFECT Makes HTTP POST to /api/v1/sqllab/execute/.
def execute_sql(
self,
sql: str,
@@ -236,10 +236,10 @@ class SupersetSqlLabExecutor:
# endregion execute_sql
# region poll_execution_status [TYPE Function]
# @PURPOSE: Poll Superset for SQL execution status until completion or timeout.
# @PRE: query_id is a valid Superset query ID.
# @POST: Returns final execution status dict.
# @SIDE_EFFECT: Makes HTTP GET requests to Superset.
# @PURPOSE Poll Superset for SQL execution status until completion or timeout.
# @PRE query_id is a valid Superset query ID.
# @POST Returns final execution status dict.
# @SIDE_EFFECT Makes HTTP GET requests to Superset.
def poll_execution_status(
self,
query_id: str,
@@ -329,10 +329,10 @@ class SupersetSqlLabExecutor:
# endregion poll_execution_status
# region execute_and_poll [TYPE Function]
# @PURPOSE: Execute SQL and wait for completion. One-shot convenience method.
# @PRE: sql is valid SQL.
# @POST: Returns final execution result.
# @SIDE_EFFECT: Makes HTTP calls to Superset API.
# @PURPOSE Execute SQL and wait for completion. One-shot convenience method.
# @PRE sql is valid SQL.
# @POST Returns final execution result.
# @SIDE_EFFECT Makes HTTP calls to Superset API.
def execute_and_poll(
self,
sql: str,
@@ -384,10 +384,10 @@ class SupersetSqlLabExecutor:
# endregion execute_and_poll
# region get_query_results [TYPE Function]
# @PURPOSE: Fetch the results of a completed query.
# @PRE: query_id is a valid Superset query ID.
# @POST: Returns query results if available.
# @SIDE_EFFECT: Makes HTTP GET to Superset.
# @PURPOSE Fetch the results of a completed query.
# @PRE query_id is a valid Superset query ID.
# @POST Returns query results if available.
# @SIDE_EFFECT Makes HTTP GET to Superset.
def get_query_results(self, query_id: str) -> dict[str, Any]:
with belief_scope("SupersetSqlLabExecutor.get_query_results"):
client = self._get_client()

View File

@@ -1,6 +1,6 @@
# #region TranslateSchemas [C:2] [TYPE Module] [SEMANTICS pydantic, translate, schema, translate-job-create]
# #region TranslateSchemas [C:3] [TYPE Module] [SEMANTICS pydantic, translate, schema, translate-job-create]
# @BRIEF Pydantic v2 schemas for translation API request/response serialization.
# @LAYER: API
# @LAYER API
# @RELATION DEPENDS_ON -> pydantic
from datetime import datetime

View File

@@ -1,6 +1,6 @@
# #region GitServiceBase [C:3] [TYPE Module] [SEMANTICS git, repository, clone, base, mixin]
# #region GitServiceBase [C:4] [TYPE Module] [SEMANTICS git, repository, clone, base, mixin, lock, http]
# @LAYER: Infra
# @BRIEF Core GitService base class — initialization, path resolution, repo lifecycle (init/delete/get), and identity configuration.
# @BRIEF Core GitService base class — initialization, path resolution, repo lifecycle (init/delete/get), identity configuration, concurrent locking, and shared HTTP client pool.
# @RELATION INHERITED_BY -> [GitService]
# @RELATION DEPENDS_ON -> [SessionLocal]
# @RELATION DEPENDS_ON -> [AppConfigRecord]
@@ -9,8 +9,12 @@
import os
import re
import shutil
import threading
from contextlib import contextmanager
from pathlib import Path
from urllib.parse import quote
import httpx
from fastapi import HTTPException
from git import Repo
from git.exc import InvalidGitRepositoryError, NoSuchPathError
@@ -21,14 +25,16 @@ from src.models.config import AppConfigRecord
from src.models.git import GitRepository
# #region GitServiceBase [C:3] [TYPE Class]
# @BRIEF Base class for GitService providing initialization, path resolution, repository lifecycle, and identity.
# #region GitServiceBase [C:4] [TYPE Class]
# @BRIEF Base class for GitService providing initialization, path resolution, repository lifecycle, identity, concurrent locking, and shared HTTP client.
# @PRE base_path is a valid string path.
# @POST GitService is initialized; base_path directory exists; shared AsyncClient ready; lock registry empty.
# @SIDE_EFFECT Creates HTTP connection pool (max_keepalive=20, max_connections=100).
class GitServiceBase:
# region GitService_init [TYPE Function]
# @PURPOSE: Initializes the GitService with a base path for repositories.
# @PARAM: base_path (str) - Root directory for all Git clones.
# region GitService_init [C:4] [TYPE Function]
# @PURPOSE: Initializes the GitService with a base path, concurrent access locks, and shared HTTP client.
# @PRE: base_path is a valid string path.
# @POST: GitService is initialized; base_path directory exists.
# @POST: GitService is initialized; base_path directory exists; _lock_registry, _http_client ready.
def __init__(self, base_path: str = "git_repos"):
with belief_scope("GitService.__init__"):
backend_root = Path(__file__).parents[3]
@@ -36,8 +42,65 @@ class GitServiceBase:
self._uses_default_base_path = base_path == "git_repos"
self.base_path = self._resolve_base_path(base_path)
self._ensure_base_path_exists()
# Fix 3: Per-dashboard reentrant lock registry for concurrent access protection
self._lock_registry: dict[int, threading.RLock] = {}
self._reg_lock = threading.Lock()
# Fix: close() race condition protection
self._closed = False
self._close_lock = threading.Lock()
# Fix 5: Shared httpx.AsyncClient with connection pooling
self._http_client = httpx.AsyncClient(
timeout=30.0,
limits=httpx.Limits(max_keepalive_connections=20, max_connections=100),
)
# endregion GitService_init
# region _get_lock [C:2] [TYPE Function] [SEMANTICS lock,concurrency]
# @BRIEF Get or create a per-dashboard reentrant lock for thread-safe GitPython access.
# @PRE: dashboard_id is a valid integer.
# @POST: Returns a threading.RLock unique to the given dashboard_id.
def _get_lock(self, dashboard_id: int) -> threading.RLock:
with self._reg_lock:
if dashboard_id not in self._lock_registry:
self._lock_registry[dashboard_id] = threading.RLock()
return self._lock_registry[dashboard_id]
# endregion _get_lock
# region _locked [C:2] [TYPE Function] [SEMANTICS lock,context,concurrency]
# @BRIEF Context manager that acquires the per-dashboard reentrant lock for the duration of the block.
# @PRE: dashboard_id is a valid integer.
# @POST: Lock is acquired on enter, released on exit.
@contextmanager
def _locked(self, dashboard_id: int):
if self._closed:
raise RuntimeError("GitService is closed")
lock = self._get_lock(dashboard_id)
lock.acquire()
try:
yield
finally:
lock.release()
# endregion _locked
# region _clone_with_auth [C:2] [TYPE Function] [SEMANTICS git,clone,auth,security]
# @BRIEF Clone repository with PAT and immediately strip credentials from origin remote URL.
# @PRE: remote_url is a valid Git URL; pat is provided when required; repo_path is writable.
# @POST: Repo cloned at repo_path; origin remote URL has no embedded PAT.
# @SIDE_EFFECT Clones remote repository to local filesystem.
def _clone_with_auth(self, remote_url: str, repo_path: str, pat: str) -> Repo:
auth_url = remote_url
if pat and "://" in remote_url:
proto, rest = remote_url.split("://", 1)
auth_url = f"{proto}://oauth2:{quote(pat, safe='')}@{rest}"
repo = Repo.clone_from(auth_url, repo_path)
# Strip PAT from origin URL to prevent credential leakage in .git/config, logs, ps aux
repo.git.remote("set-url", "origin", remote_url)
return repo
# endregion _clone_with_auth
# region _ensure_base_path_exists [TYPE Function]
# @PURPOSE: Ensure the repositories root directory exists and is a directory.
# @PRE: self.base_path is resolved to filesystem path.
@@ -194,124 +257,138 @@ class GitServiceBase:
return target_path
# endregion _get_repo_path
# region init_repo [TYPE Function]
# @PURPOSE: Initialize or clone a repository for a dashboard.
# region init_repo [C:4] [TYPE Function] [SEMANTICS git,clone,init,lock]
# @PURPOSE: Initialize or clone a repository for a dashboard with concurrent access protection.
# @PARAM: dashboard_id (int), remote_url (str), pat (str), repo_key (Optional[str]).
# @PRE: dashboard_id is int, remote_url is valid Git URL, pat is provided.
# @POST: Repository is cloned or opened at the local path.
# @POST: Repository is cloned or opened at the local path. Origin remote URL has no embedded PAT.
# @SIDE_EFFECT Clones remote repository; modifies local filesystem; protected by per-dashboard lock.
# @RETURN: Repo
def init_repo(self, dashboard_id: int, remote_url: str, pat: str, repo_key: str | None = None) -> Repo:
with belief_scope("GitService.init_repo"):
self._ensure_base_path_exists()
repo_path = self._get_repo_path(dashboard_id, repo_key=repo_key or str(dashboard_id))
Path(repo_path).parent.mkdir(parents=True, exist_ok=True)
if pat and "://" in remote_url:
proto, rest = remote_url.split("://", 1)
auth_url = f"{proto}://oauth2:{pat}@{rest}"
else:
auth_url = remote_url
if os.path.exists(repo_path):
logger.reason(f"Opening existing repo at {repo_path}", extra={"src": "init_repo"})
try:
repo = Repo(repo_path)
except (InvalidGitRepositoryError, NoSuchPathError):
logger.reason(f"Existing path is not a Git repository, recreating: {repo_path}", extra={"src": "init_repo"})
stale_path = Path(repo_path)
if stale_path.exists():
shutil.rmtree(stale_path, ignore_errors=True)
def init_repo(self, dashboard_id: int, remote_url: str, pat: str, repo_key: str | None = None, default_branch: str | None = None) -> Repo:
with self._locked(dashboard_id):
with belief_scope("GitService.init_repo"):
self._ensure_base_path_exists()
repo_path = self._get_repo_path(dashboard_id, repo_key=repo_key or str(dashboard_id))
Path(repo_path).parent.mkdir(parents=True, exist_ok=True)
if os.path.exists(repo_path):
logger.reason(f"Opening existing repo at {repo_path}", extra={"src": "init_repo"})
try:
repo = Repo(repo_path)
except (InvalidGitRepositoryError, NoSuchPathError):
logger.reason(f"Existing path is not a Git repository, recreating: {repo_path}", extra={"src": "init_repo"})
stale_path = Path(repo_path)
if stale_path.exists():
try:
stale_path.unlink()
except Exception:
pass
repo = Repo.clone_from(auth_url, repo_path)
shutil.rmtree(stale_path, ignore_errors=True)
if stale_path.exists():
try:
stale_path.unlink()
except Exception:
pass
repo = self._clone_with_auth(remote_url, repo_path, pat)
self._ensure_gitflow_branches(repo, dashboard_id)
return repo
logger.reason(f"Cloning {remote_url} to {repo_path}", extra={"src": "init_repo"})
repo = self._clone_with_auth(remote_url, repo_path, pat)
self._ensure_gitflow_branches(repo, dashboard_id)
return repo
logger.reason(f"Cloning {remote_url} to {repo_path}", extra={"src": "init_repo"})
repo = Repo.clone_from(auth_url, repo_path)
self._ensure_gitflow_branches(repo, dashboard_id)
return repo
# endregion init_repo
# region delete_repo [TYPE Function]
# @PURPOSE: Remove local repository and DB binding for a dashboard.
# region delete_repo [C:4] [TYPE Function] [SEMANTICS git,delete,lock]
# @PURPOSE: Remove local repository and DB binding for a dashboard with concurrent access protection.
# @PRE: dashboard_id is a valid integer.
# @POST: Local path is deleted when present and GitRepository row is removed.
# @SIDE_EFFECT Deletes filesystem directory and DB record; protected by per-dashboard lock.
def delete_repo(self, dashboard_id: int) -> None:
with belief_scope("GitService.delete_repo"):
repo_path = self._get_repo_path(dashboard_id)
removed_files = False
if os.path.exists(repo_path):
if os.path.isdir(repo_path):
shutil.rmtree(repo_path)
else:
os.remove(repo_path)
removed_files = True
session = SessionLocal()
try:
db_repo = (
session.query(GitRepository)
.filter(GitRepository.dashboard_id == int(dashboard_id))
.first()
)
if db_repo:
session.delete(db_repo)
session.commit()
return
if removed_files:
return
raise HTTPException(
status_code=404,
detail=f"Repository for dashboard {dashboard_id} not found",
)
except HTTPException:
session.rollback()
raise
except Exception as e:
session.rollback()
logger.error(f"[delete_repo][Coherence:Failed] Failed to delete repository for dashboard {dashboard_id}: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete repository: {e!s}")
finally:
session.close()
with self._locked(dashboard_id):
with belief_scope("GitService.delete_repo"):
repo_path = self._get_repo_path(dashboard_id)
removed_files = False
if os.path.exists(repo_path):
if os.path.isdir(repo_path):
shutil.rmtree(repo_path)
else:
os.remove(repo_path)
removed_files = True
session = SessionLocal()
try:
db_repo = (
session.query(GitRepository)
.filter(GitRepository.dashboard_id == int(dashboard_id))
.first()
)
if db_repo:
session.delete(db_repo)
session.commit()
return
if removed_files:
return
raise HTTPException(
status_code=404,
detail=f"Repository for dashboard {dashboard_id} not found",
)
except HTTPException:
session.rollback()
raise
except Exception as e:
session.rollback()
logger.error(f"[delete_repo][Coherence:Failed] Failed to delete repository for dashboard {dashboard_id}: {e}")
raise HTTPException(status_code=500, detail=f"Failed to delete repository: {e!s}")
finally:
session.close()
# endregion delete_repo
# region get_repo [TYPE Function]
# @PURPOSE: Get Repo object for a dashboard.
# region get_repo [C:4] [TYPE Function] [SEMANTICS git,open,lock]
# @PURPOSE: Get Repo object for a dashboard with concurrent access protection.
# @PRE: Repository must exist on disk for the given dashboard_id.
# @POST: Returns a GitPython Repo instance for the dashboard.
# @RETURN: Repo
def get_repo(self, dashboard_id: int) -> Repo:
with belief_scope("GitService.get_repo"):
repo_path = self._get_repo_path(dashboard_id)
if not os.path.exists(repo_path):
logger.error(f"[get_repo][Coherence:Failed] Repository for dashboard {dashboard_id} does not exist")
raise HTTPException(status_code=404, detail=f"Repository for dashboard {dashboard_id} not found")
try:
return Repo(repo_path)
except Exception as e:
logger.error(f"[get_repo][Coherence:Failed] Failed to open repository at {repo_path}: {e}")
raise HTTPException(status_code=500, detail="Failed to open local Git repository")
with self._locked(dashboard_id):
with belief_scope("GitService.get_repo"):
repo_path = self._get_repo_path(dashboard_id)
if not os.path.exists(repo_path):
logger.error(f"[get_repo][Coherence:Failed] Repository for dashboard {dashboard_id} does not exist")
raise HTTPException(status_code=404, detail=f"Repository for dashboard {dashboard_id} not found")
try:
return Repo(repo_path)
except Exception as e:
logger.error(f"[get_repo][Coherence:Failed] Failed to open repository at {repo_path}: {e}")
raise HTTPException(status_code=500, detail="Failed to open local Git repository")
# endregion get_repo
# region configure_identity [TYPE Function]
# @PURPOSE: Configure repository-local Git committer identity for user-scoped operations.
# region configure_identity [C:4] [TYPE Function] [SEMANTICS git,identity,lock]
# @PURPOSE: Configure repository-local Git committer identity with concurrent access protection.
# @PRE: dashboard_id repository exists; git_username/git_email may be empty.
# @POST: Repository config has user.name and user.email when both identity values are provided.
# @SIDE_EFFECT Writes to repository git config; protected by per-dashboard lock.
def configure_identity(self, dashboard_id: int, git_username: str | None, git_email: str | None) -> None:
with belief_scope("GitService.configure_identity"):
normalized_username = str(git_username or "").strip()
normalized_email = str(git_email or "").strip()
if not normalized_username or not normalized_email:
return
repo = self.get_repo(dashboard_id)
try:
with repo.config_writer(config_level="repository") as config_writer:
config_writer.set_value("user", "name", normalized_username)
config_writer.set_value("user", "email", normalized_email)
logger.reason(f"Applied repository-local git identity for dashboard {dashboard_id}", extra={"src": "configure_identity"})
except Exception as e:
logger.error(f"[configure_identity][Coherence:Failed] Failed to configure git identity: {e}")
raise HTTPException(status_code=500, detail=f"Failed to configure git identity: {e!s}")
with self._locked(dashboard_id):
with belief_scope("GitService.configure_identity"):
normalized_username = str(git_username or "").strip()
normalized_email = str(git_email or "").strip()
if not normalized_username or not normalized_email:
return
repo = self.get_repo(dashboard_id)
try:
with repo.config_writer(config_level="repository") as config_writer:
config_writer.set_value("user", "name", normalized_username)
config_writer.set_value("user", "email", normalized_email)
logger.reason(f"Applied repository-local git identity for dashboard {dashboard_id}", extra={"src": "configure_identity"})
except Exception as e:
logger.error(f"[configure_identity][Coherence:Failed] Failed to configure git identity: {e}")
raise HTTPException(status_code=500, detail=f"Failed to configure git identity: {e!s}")
# endregion configure_identity
# region close [C:2] [TYPE Function] [SEMANTICS http,cleanup,shutdown]
# @BRIEF Gracefully close the shared HTTP client (connection pool).
# @PRE: _http_client is initialized.
# @POST: HTTP connections closed.
async def close(self):
with self._close_lock:
if self._closed:
return
self._closed = True
await self._http_client.aclose()
# endregion close
# #endregion GitServiceBase
# #endregion GitServiceBase

Some files were not shown because too many files have changed in this diff Show More