Commit Graph

166 Commits

Author SHA1 Message Date
316d97ae25 refactor: migrate translate engine to GRACE-Poly v2.6 semantic protocol
- Convert all 84 contracts from legacy [DEF:] to #region/#endregion syntax
- Fix complexity tiers: 14 modules re-tiered (6 C4 route modules, 7 C4→C5 plugin services)
- Remove forbidden tags: @RATIONALE/@REJECTED stripped from C1–C4 contracts
- Add required tags: @PRE/@POST/@SIDE_EFFECT on C4, @RELATION on C3, @DATA_CONTRACT/@INVARIANT on C5
- Add belief runtime markers (reason/reflect/explore) to 7 service.py functions
- Fix @LAYER: route files → UI, plugins → Domain, superset_executor → Infra
- Fix pre-existing test mock_service fixture in test_orchestrator.py
- 196/196 translation tests pass, zero regressions
2026-05-12 14:32:28 +03:00
c2bff20937 fix: persist environment selection, support Kilo AI provider, resolve Superset virtual columns for translation preview
- Add environment_id to TranslationJob model/schema/API + DB migration
- Pass environmentId from page to TranslationPreview and fetchPreview
- Fix _fetch_sample_rows: use result_type='samples' to include virtual cols
- Add 'kilo' and 'openrouter' to supported LLM provider types
- Make response_format conditional (skip for non-OpenAI upstream providers)
- Remove source_table field from form (redundant with datasource)
- Restore datasourceSearch display from saved job on page load
- Add request/response logging for LLM API calls
2026-05-09 23:10:15 +03:00
85121191d4 feat: implement LLM table translation service with searchable datasource dropdown, i18n, sidebar, and RBAC
- Add backend plugin modules: preview, executor, orchestrator, events, sql_generator, superset_executor, dictionary, scheduler, metrics
- Add API routes for translate jobs, runs, dictionaries, preview, schedule, metrics, corrections
- Add ORM models (12 tables) and Pydantic schemas (15 DTOs)
- Register translate router in app.py with RBAC permission guards
- Add frontend pages: job list, job config, dictionary list/editor, history
- Add 7 reusable Svelte 5 components: Preview, RunProgress, RunResult, TermCorrection, BulkCorrection, ScheduleConfig, MetricsDashboard
- Add searchable datasource dropdown with Superset API integration
- Add i18n (en/ru) with ~300 keys across common, config, jobs, dictionaries, preview namespaces
- Add Translation sidebar category with Jobs/Dictionaries/History sub-items
- Hide health monitor error toast via suppressToast API option
- Add 69 backend tests and 44 frontend test files
- Fix: SupersetClient env resolution (string -> Environment object)
- Fix: Dataset detail API returns proper database dict
- Fix: Database dialect extraction fallback when metadata incomplete
2026-05-09 19:34:25 +03:00
c899a369f3 refactor: decompose 6 monolithic modules (>1000 LOC) into domain packages
Decompose 6 large modules into packages with <400-line modules to satisfy INV_7:

- superset_client.py (2145->0) -> superset_client/ package (12 modules)
- assistant.py (2683->0) -> assistant/ package (12 modules)
- git_service.py (2101->11 shim) -> services/git/ package (9 modules)
- git.py (1757->0) -> routes/git/ package (11 modules)
- dashboards.py (1619->0) -> routes/dashboards/ package (8 modules)
- translate.py (1587->0) -> routes/translate/ package (11 modules)
- superset_context_extractor.py (1397->0) -> utils/superset_context_extractor/ (7 modules)

All modules <400 lines (INV_7). All 80 tests pass.
All external imports preserved via __init__.py re-exports or shim.
Semantic [DEF] contracts with @LAYER/@SEMANTICS added to all Module types.
2026-05-09 10:13:07 +03:00
13fcd544d2 semantic cleanup 2026-05-08 10:07:05 +03:00
48366d237a fix(git): replace git diff --cached with git status --porcelain in get_status
repo.is_dirty() and repo.index.diff("HEAD") internally call git diff --cached,
which fails with exit 129 (unknown option) in some environments.

Introduce _parse_status_porcelain() using git status --porcelain,
a self-contained command that avoids --cached entirely.

All 25 git route tests continue to pass.
2026-05-08 09:55:10 +03:00
a69446473b refactor 2026-04-24 17:10:02 +03:00
6a2741182b skills + agents 2026-04-19 20:05:45 +03:00
721caee611 semantics 2026-04-02 12:12:23 +03:00
e9fe80784f prune contracts 2026-04-01 22:31:10 +03:00
f1a809e431 + 2026-04-01 22:04:19 +03:00
78d6995e13 semantics 2026-04-01 21:57:51 +03:00
69e86a97e7 semantics update 2026-04-01 15:30:13 +03:00
3dbac3ce69 semantic 2026-04-01 14:51:45 +03:00
e83879cf06 semantics 2026-03-27 21:27:31 +03:00
d17f75a65d feat(ui): add chat-driven dataset review flow
Move dataset review clarification into the assistant workspace and
rework the review page into a chat-centric layout with execution rails.

Add session-scoped assistant actions for mappings, semantic fields,
and SQL preview generation. Introduce optimistic locking for dataset
review mutations, propagate session versions through API responses,
and mask imported filter values before assistant exposure.

Refresh tests, i18n, and spec artifacts to match the new workflow.

BREAKING CHANGE: dataset review mutation endpoints now require the
X-Session-Version header, and clarification is no longer handled
through ClarificationDialog-based flows
2026-03-26 13:33:12 +03:00
5a25324d75 feat(ui): surface review readiness and blockers
Add readiness hints, blocker summaries, and progress cues across
the dataset review workspace to clarify partial recovery and
launch gating states.

Highlight read-only preview snapshots, pending semantic review,
clarification queue status, and unresolved validation blockers to
guide the next recommended action.

Document known limitations in backend test doubles so permission,
task dispatch, and uncovered persistence branches are explicit.
2026-03-21 15:48:28 +03:00
eba124423a test semantics 2026-03-21 15:23:06 +03:00
07c064bb86 fix: finalize semantic repair and test updates 2026-03-21 15:07:06 +03:00
1a79589632 fix: resolve missing ENCRYPTION_KEY in assistant route tests 2026-03-21 12:19:47 +03:00
978fe7b34d fix: commit semantic repair changes 2026-03-21 11:22:25 +03:00
959be4f4b2 fix: commit verified semantic repair changes 2026-03-21 11:05:20 +03:00
a1c6596554 semantics 2026-03-20 20:01:58 +03:00
f6171d72e4 subagents 2026-03-20 17:20:24 +03:00
47aadfc178 semantic 2026-03-18 08:45:15 +03:00
0bc6f59a7a Bootstrap initial admin via env and add compose profiles 2026-03-17 19:16:25 +03:00
2095d96b16 fix(final-phase): finalize dataset review audit blockers 2026-03-17 18:23:02 +03:00
2dbef02f81 fix(us3): align dataset review contracts and acceptance gates 2026-03-17 18:20:36 +03:00
c34a982bf8 fix(tests): Add model imports to fix SQLAlchemy registration in matrix tests 2026-03-17 14:33:15 +03:00
f526a978c3 feat(027): Final Phase T038-T043 implementation
- T038: SessionEvent logger and persistence logic
  - Added SessionEventLogger service with explicit audit event persistence
  - Added SessionEvent model with events relationship on DatasetReviewSession
  - Integrated event logging into orchestrator flows and API mutation endpoints

- T039: Semantic source version propagation
  - Added source_version column to SemanticFieldEntry
  - Added propagate_source_version_update() to SemanticResolver
  - Preserves locked/manual field invariants during propagation

- T040: Batch approval API and UI actions
  - Added batch semantic approval endpoint (/fields/semantic/approve-batch)
  - Added batch mapping approval endpoint (/mappings/approve-batch)
  - Added batch approval actions to SemanticLayerReview and ExecutionMappingReview components
  - Aligned batch semantics with single-item approval contracts

- T041: Superset compatibility matrix tests
  - Added test_superset_matrix.py with preview and SQL Lab fallback coverage
  - Tests verify client method preference and matrix fallback behavior

- T042: RBAC audit sweep on session-mutation endpoints
  - Added _require_owner_mutation_scope() helper
  - Applied owner guards to update_session, delete_session, and all mutation endpoints
  - Ensured no bypass of existing permission checks

- T043: i18n coverage for dataset-review UI
  - Added workspace state labels (empty/importing/review) to en.json and ru.json
  - Added batch action labels for semantics and mappings
  - Fixed workspace state comparison to lowercase strings
  - Removed hardcoded workspace state display strings

Signed-off-by: Implementation Specialist <impl@ss-tools>
2026-03-17 14:29:33 +03:00
2c3fd28bf3 fix(027): stabilize shared acceptance gates and compatibility collateral 2026-03-17 11:07:49 +03:00
d56084e5c8 feat(us1): add dataset review orchestration automatic review slice 2026-03-17 10:57:49 +03:00
2c2bd1a7dd fix: repository collaborator access and stale findings persistence issues 2026-03-16 23:43:37 +03:00
578020cdb2 feat: initial dataset review orchestration flow implementation 2026-03-16 23:43:03 +03:00
4c6e881347 fix 2026-03-16 21:27:33 +03:00
30323d8273 chore: update semantic contracts and git merge handling 2026-03-16 20:34:28 +03:00
51a0973dc1 docs(semantics): simplify test markup protocol (Section VIII) and sync workflows 2026-03-16 18:18:57 +03:00
0096500903 semantic 2026-03-16 16:45:08 +03:00
96d8712b6f refactor(semantics): migrate legacy @TIER to @COMPLEXITY annotations
- Replaced @TIER: TRIVIAL with @COMPLEXITY: 1
- Replaced @TIER: STANDARD with @COMPLEXITY: 3
- Replaced @TIER: CRITICAL with @COMPLEXITY: 5
- Manually elevated specific critical/complex components to levels 2 and 4
- Ignored legacy, specs, and node_modules directories
- Updated generated semantic map
2026-03-16 10:06:44 +03:00
df69b2af49 chore(semantics): checkpoint orphan-reduction hub normalization batch 2026-03-15 22:14:05 +03:00
a054aaa5d6 chore(semantic): remediate backend core contracts 2026-03-15 21:23:44 +03:00
613a923c36 chore(semantic): checkpoint remediation progress 2026-03-15 21:08:00 +03:00
d61987bd11 mcp 2026-03-15 20:29:11 +03:00
12b522f852 feat add connections management and health summary improvements 2026-03-15 16:40:43 +03:00
2d26f0de63 fix dashboard validation fallback and semantic relation parsing 2026-03-15 16:32:39 +03:00
6838ac86df Fix LLM validation and dashboard health hot paths 2026-03-15 13:18:51 +03:00
aa685e357a feat: Implement LLM provider deletion and refactor ConfigManager to preserve unknown payload sections. 2026-03-14 09:19:08 +03:00
7ecadcb83d security: rotate bootstrap and clean workspace 2026-03-13 12:14:37 +03:00
4c7ab70c42 Commit remaining workspace changes 2026-03-13 11:45:06 +03:00
9bd2a7001b Add docker admin bootstrap for clean release 2026-03-13 11:41:44 +03:00