busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
497 Commits 3 Branches 0 Tags
03557d38cc95a0c38d06daaedf7a081d88b5f88a
Commit Graph

39 Commits

Author SHA1 Message Date
busya
4205618ee6 chore: eliminate all deprecation warnings from tests and linter 
Warnings fixed:
- datetime.utcnow() → datetime.now(UTC) across 48+ files (src/ + tests/)
- datetime.utcnow (callback ref) → lambda: datetime.now(UTC) in model fields (18 files)
- Pydantic class Config → model_config = ConfigDict(...) (16 files)
- Pydantic .dict() → .model_dump() (8 files)
- ConfigDict(allow_population_by_field_name=True) → validate_by_name=True
- SQLAlchemy declarative_base() import path updated
- FastAPI on_event → lifespan context manager (app.py)
- Import sorting (ruff I001) auto-fixed across all files
- Fixed broken re-export chains that ruff F401 cleanup broke:
  _validate_bcp47: service.py now imports from dictionary_validation directly
  job_to_response: _job_routes.py and test imports from service_utils directly
  fetch_datasource_metadata: restored re-export in service.py
- Added missing TranslateJobService import in _job_routes.py (was deleted by F401)
- Added ConfigDict(protected_namespaces=()) for DashboardDatasetItem schema field
- pytest.ini: replaced deprecated importmode with asyncio_mode

All 440 tests pass with zero deprecation warnings.
 2026-05-26 19:18:28 +03:00
busya
9ffa8af1dc semantics 2026-05-26 09:30:41 +03:00
busya
36643024cf refactor(connections): remove ConnectionConfig, migrate mapper to SQL Lab 
Backend:
- Remove ConnectionConfig model, CRUD routes (connections.py), and tests
- Remove ensure_connection_configs_table from database.py
- Migrate MapperPlugin from psycopg2 direct PostgreSQL to SupersetSqlLabExecutor
- Add DatasetMapper.get_sqllab_mappings() with default information_schema query
- Update MapColumnsRequest: connection_id → database_id + sql_query
- Fix pydantic_settings v2 deprecation (Field(env=...) → validation_alias)
- Clean up app.py, routes/__init__.py, database.py imports

Frontend:
- Remove DatabaseConnectionsTab, ConnectionForm, ConnectionList components
- Remove /settings/connections route page and Navbar link
- Remove connectionService.js, connections i18n files
- Update MapperTool.svelte: postgres → sqllab source
- Update Map Columns modal: database selector + SQL query instead of connection_id
- Clean up settings page tabs, nav links, test mocks
- Clean up i18n keys (settings, nav, mapper)

Tests: 25/25 pass (14 030-feature + 11 route tests)
Build: succeeds
 2026-05-21 18:47:41 +03:00
busya
68740cd9ed semantic 2026-05-20 23:54:53 +03:00
busya
6988e63967 chore: commit remaining pre-existing changes 
- Agent configs (.opencode/agents/)
- Backend: alembic, routes, app, utils, scripts
- Frontend: package.json, vite, components, e2e infra
- Specs: 028-llm-datasource-supeset updates
- Docker e2e config and Playwright setup
 2026-05-17 19:23:07 +03:00
busya
c6189876b3 chore(lint): apply ruff --fix (4443 auto-fixes) 
Auto-fixed categories:
- F401: unused imports removed
- I001: import blocks sorted
- W293: trailing whitespace stripped
- UP035: deprecated typing imports replaced
- SIM: simplify suggestions applied
- ARG: unused args prefixed with underscore
- T201: print statements removed
- F841: unused variables removed
- RUF059: unpacked variables prefixed

Remaining ~1500 unfixable errors (C901, B904, N806, E402) require manual work.

Backend smoke tests: 13/13 passed.
 2026-05-14 11:20:17 +03:00
busya
a3c7c402b7 fix(llm): add fetch-models endpoint, fix SQL Lab INSERT (client_id truncation, sync mode, target_column, timestamp normalization) 
- Add POST /api/llm/providers/fetch-models route with LLMClient.fetch_models()
- Add target_column to TranslationJob model/schema/service/orchestrator
- Fix SQL Lab execute: truncate client_id to 11 chars (varchar(11))
- Switch SQL Lab to sync mode (runAsync: false) — no Celery workers
- Fix polling: unwrap nested result from Superset query API
- Fix ClickHouse timestamp: normalize float timestamps to YYYY-MM-DD
 2026-05-13 20:06:15 +03:00
busya
39ab647851 semantics 2026-05-13 14:15:33 +03:00
busya
83b8f4d999 logs: migrate to molecular CoT JSON protocol, fix regressions 
- Replace BeliefFormatter with CotJsonFormatter (single-line JSON with
  ts, level, trace_id, src, marker, intent, payload, error, span_id)
- Migrate belief_scope to use cot_log() for structured JSON output
- Update monkey-patched reason/reflect/explore to pass extra= dict
- Strip 200+ inline [REASON]/[REFLECT]/[EXPLORE] markers from 50+ files
- Remove belief_scope from hot-path utilities (_parse_datetime,
  _json_load_if_needed) to eliminate startup log noise
- Register TraceContextMiddleware in app.py (was implemented but unused)
- Seed trace_id in startup_event/shutdown_event for background tasks
- Fix broken relative imports in translate routes (3 dots → 4 dots)
- Migrate 43 translate_routes log calls to logger.reason/explore
- Fix SyntaxError (positional arg after extra=) in _repo_operations_routes
- Fix IndentationError in rbac_permission_catalog except-block
- Add smoke test tests/test_smoke_app.py (catches import errors before run)
 2026-05-13 09:44:50 +03:00
busya
b6f46fe9f5 logger: migrate belief_scope from legacy markers to molecular CoT protocol 
Replace [Entry]/[Exit]/[Action]/[COHERENCE:OK] with [REASON]/[REFLECT]/[EXPLORE]
in belief_scope context manager, BeliefFormatter, and 34 source files.
Per molecular-cot-logging skill invariants.
 2026-05-13 08:46:24 +03:00
busya
306c5ae742 semantics: complete DEF-to-region migration, fix regressions 
- Convert legacy [DEF🆔Type] anchors to #region/#endregion across 329 files
- Reinstate _normalize_timestamp_value in sql_generator.py
- Fix MarkerLogger→logger migration in events.py (molecular CoT markers)
- Fix dataset_review orchestrator dependencies (_build_execution_snapshot)
- Fix config_manager stale-record deletion (moved to save path only)
- Add 77 missing [/DEF:] closers in 5 unbalanced test files
- Update assistant_chat.integration.test.js for #region format
- Apply molecular-cot-logging markers (REASON/REFLECT/EXPLORE) via logger.* methods
 2026-05-12 23:54:55 +03:00
busya
fe8978f716 semantics 2026-05-12 20:06:16 +03:00
busya
b17b5333c7 log refactor 2026-05-12 19:30:15 +03:00
busya
1d59df2233 refactor 2026-05-12 14:52:27 +03:00
busya
201886eeb0 refactor: decompose 6 monolithic modules (>1000 LOC) into domain packages 
Decompose 6 large modules into packages with <400-line modules to satisfy INV_7:

- superset_client.py (2145->0) -> superset_client/ package (12 modules)
- assistant.py (2683->0) -> assistant/ package (12 modules)
- git_service.py (2101->11 shim) -> services/git/ package (9 modules)
- git.py (1757->0) -> routes/git/ package (11 modules)
- dashboards.py (1619->0) -> routes/dashboards/ package (8 modules)
- translate.py (1587->0) -> routes/translate/ package (11 modules)
- superset_context_extractor.py (1397->0) -> utils/superset_context_extractor/ (7 modules)

All modules <400 lines (INV_7). All 80 tests pass.
All external imports preserved via __init__.py re-exports or shim.
Semantic [DEF] contracts with @LAYER/@SEMANTICS added to all Module types.
 2026-05-09 10:13:07 +03:00
busya
2228a00035 semantics 2026-04-02 12:12:23 +03:00
busya
f5c90af287 semantics 2026-04-01 21:57:51 +03:00
busya
202823aba5 feat(ui): add chat-driven dataset review flow 
Move dataset review clarification into the assistant workspace and
rework the review page into a chat-centric layout with execution rails.

Add session-scoped assistant actions for mappings, semantic fields,
and SQL preview generation. Introduce optimistic locking for dataset
review mutations, propagate session versions through API responses,
and mask imported filter values before assistant exposure.

Refresh tests, i18n, and spec artifacts to match the new workflow.

BREAKING CHANGE: dataset review mutation endpoints now require the
X-Session-Version header, and clarification is no longer handled
through ClarificationDialog-based flows
 2026-03-26 13:33:12 +03:00
busya
81406bc2e2 fix: commit semantic repair changes 2026-03-21 11:22:25 +03:00
busya
4d4f204f1c semantics 2026-03-20 20:01:58 +03:00
busya
62d7c660ef subagents 2026-03-20 17:20:24 +03:00
busya
7cff6bdcf8 semantic 2026-03-18 08:45:15 +03:00
busya
b51a68168c feat(027): Final Phase T038-T043 implementation 
- T038: SessionEvent logger and persistence logic
  - Added SessionEventLogger service with explicit audit event persistence
  - Added SessionEvent model with events relationship on DatasetReviewSession
  - Integrated event logging into orchestrator flows and API mutation endpoints

- T039: Semantic source version propagation
  - Added source_version column to SemanticFieldEntry
  - Added propagate_source_version_update() to SemanticResolver
  - Preserves locked/manual field invariants during propagation

- T040: Batch approval API and UI actions
  - Added batch semantic approval endpoint (/fields/semantic/approve-batch)
  - Added batch mapping approval endpoint (/mappings/approve-batch)
  - Added batch approval actions to SemanticLayerReview and ExecutionMappingReview components
  - Aligned batch semantics with single-item approval contracts

- T041: Superset compatibility matrix tests
  - Added test_superset_matrix.py with preview and SQL Lab fallback coverage
  - Tests verify client method preference and matrix fallback behavior

- T042: RBAC audit sweep on session-mutation endpoints
  - Added _require_owner_mutation_scope() helper
  - Applied owner guards to update_session, delete_session, and all mutation endpoints
  - Ensured no bypass of existing permission checks

- T043: i18n coverage for dataset-review UI
  - Added workspace state labels (empty/importing/review) to en.json and ru.json
  - Added batch action labels for semantics and mappings
  - Fixed workspace state comparison to lowercase strings
  - Removed hardcoded workspace state display strings

Signed-off-by: Implementation Specialist <impl@ss-tools>
 2026-03-17 14:29:33 +03:00
busya
268ac66a38 feat(us1): add dataset review orchestration automatic review slice 2026-03-17 10:57:49 +03:00
busya
54a7bc3242 semantic 2026-03-16 16:45:08 +03:00
busya
67867f8220 refactor(semantics): migrate legacy @TIER to @COMPLEXITY annotations 
- Replaced @TIER: TRIVIAL with @COMPLEXITY: 1
- Replaced @TIER: STANDARD with @COMPLEXITY: 3
- Replaced @TIER: CRITICAL with @COMPLEXITY: 5
- Manually elevated specific critical/complex components to levels 2 and 4
- Ignored legacy, specs, and node_modules directories
- Updated generated semantic map
 2026-03-16 10:06:44 +03:00
busya
1ff034a73f chore(semantic): remediate backend core contracts 2026-03-15 21:23:44 +03:00
busya
a95109da4a feat: add slug-only dashboard profile filter and unify backend imports 2026-03-11 12:20:34 +03:00
busya
6775c7e3db feat(clean-release): complete compliance redesign phases and polish tasks T047-T052 2026-03-10 09:11:26 +03:00
busya
4f273f33ea chore: commit remaining workspace changes 2026-03-03 19:51:17 +03:00
busya
f01e35f36f fix(logging): suppress per-request belief scope spam in API client 2026-02-25 20:52:12 +03:00
busya
c73be2f647 task panel 2026-02-19 09:43:01 +03:00
busya
e242b61c5b fix 2026-02-15 11:11:30 +03:00
busya
5ab1549cdd linter + новые таски 2026-02-10 12:53:01 +03:00
busya
2d09ff4dc8 Похоже работает 2026-02-07 11:26:06 +03:00
busya
2288abfc5a semantic cleanup 2026-01-23 21:58:32 +03:00
busya
943dc06dfd refactor complete 2026-01-22 17:37:17 +03:00
busya
54a9ff81c3 project map script | semantic parcer 2026-01-01 16:58:21 +03:00
busya
26e9865e5a 001-migration-ui-redesign (#3) 
Reviewed-on: #3
 2025-12-26 18:17:58 +03:00