busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
642 Commits 3 Branches 0 Tags
61660b345fa21bae01fa46f688d4844512b0626f
Commit Graph

166 Commits

Author SHA1 Message Date
busya
55604498ae feat(translate): add insert_method dispatch and model/schema updates for direct DB 2026-06-11 19:12:35 +03:00
busya
1f9040d53e feat(core): implement ConnectionService and DbExecutor for direct DB insert (US11-US12) 
- ConnectionService: CRUD for DatabaseConnection in GlobalSettings with
  EncryptionManager-backed password encryption/decryption, test connectivity
- DbExecutor: asyncpg (PostgreSQL) and clickhouse-connect (ClickHouse) drivers
  with per-connection connection pooling
- config_models.py: promote GlobalSettings.connections from list[dict] stub
  to list[DatabaseConnection] with full Pydantic model validation
- settings.py: +5 CRUD endpoints for connections (create, read, update, delete, test)
- requirements.txt: add asyncpg>=0.29.0, clickhouse-connect>=0.7.0
- seed_permissions.py: register settings.connections.manage permission
 2026-06-11 19:12:15 +03:00
busya
7760214170 test(agent): extend coverage — agent handler, confirmations, conversation API, tools 
- test_agent_handler: additional edge cases for streaming, HITL, file upload
- test_confirmations: HITL confirm/deny lifecycle coverage
- test_conversation_api: conversation save/load persistence tests
- test_langchain_tools: tool registration, dual-auth header propagation
- ConversationList.test.ts (frontend): conversation list component tests
- conftest: shared fixtures for agent tests
- task_manager/manager: minor fixes from test coverage
- tasks.md/test-documentation.md: spec and test documentation updates
- speckit.test.md: speckit workflow documentation update
 2026-06-10 16:38:06 +03:00
busya
143f14d516 chore: remainder — backend test infra, agent config, docker, i18n, frontend ui 
- Backend: alembic env, config manager/models, dependencies, translate plugin
- Backend tests: async_sync_regression, integration tests, git services, test_agent
- Docker: docker-compose.yml updates
- Agent: qa-tester.md update, semantics-testing SKILL.md update
- Frontend: TopNavbar, sidebarNavigation, FeaturesSettings, FeatureGate
- i18n: assistant.json en/ru locale updates
- New: frontend/src/lib/components/agent/ directory
 2026-06-10 15:06:36 +03:00
busya
8e8a3c3235 feat: attention-optimized semantic protocol v2.7 
Core changes:
- Add @defgroup/@ingroup to 1791 C2+ contracts (555 files) for HCA 128× pre-training DSA grouping
- Add §0.1 Pre-Training Frequency matrix to semantics-core
- Add §VIII Attention Architecture rules (ATTN_1-4) with MLA/CSA/HCA/DSA mechanics
- Add @defgroup/@ingroup to canonical syntax (§II) and all contract examples

Agent prompts (5 files):
- Add ZERO-STATE RATIONALE with MLA/CSA/HCA/DSA compression mechanics
- Add pre-training note: @RATIONALE/@REJECTED are in-context learned tags
- svelte-coder: add missing #region contract, fix Svelte rule violations
- python-coder/fullstack-coder: honor function contracts from speckit plan
- qa-tester: add attention compliance audit (P3 ATTN_1-4 checks)

Skills (6 files):
- Translate all axiom_config descriptions to English
- Fix doc_dirs to index .opencode/ and .specify/
- Deduplicate 5× complexity_rules → single global_tags catalog
- Reduce semantics-svelte 591→485 lines (remove duplicate code blocks)
- Fix semantics-testing: 'Short IDs' → 'Short hierarchical IDs'
- Fix all examples: flat IDs → hierarchical Domain.Name format
- Fix Svelte examples: replace raw Tailwind + <button> with semantic tokens + /ui

Speckit workflow (commands + templates):
- speckit.plan: add Function-Level Contracts for C3+ with @PRE/@POST/@TEST_EDGE
- speckit.plan: add Attention Compliance Gate (ATTN_1-4 before contract generation)
- speckit.tasks: add function contract inlining format (constraints in task description)
- speckit.specify: load semantics-core for spec density rules
- spec-template: add #region contract, @SEMANTICS grouping, hierarchical IDs
- ux-reference-template: add #region wrapper
- plan-template: add attention gate, @defgroup/@ingroup guidance
- tasks-template: add attention audit + rebuild + orphan check tasks
- constitution.md: translate to English, add Principle VIII (attention-optimized contracts)

Reference modules rewritten (hierarchical IDs + full contracts):
- Auth.Jwt: 6 child contracts with @RATIONALE/@REJECTED/@TEST_EDGE
- Api.Auth: 5 endpoints with @TEST_EDGE + molecular CoT markers
- Migration.Model: @defgroup Migration with 18 @ACTION + 6 @INVARIANT

Scripts:
- add_defgroup_ingroup.py: zero-risk additive @ingroup migration (1791 insertions)
- migrate_hierarchical.py: flat→hierarchical ID dry-run analysis (792 contracts)
- merge_prompts.py: merge all prompts/skills/commands into one review file

Config:
- axiom_config.yaml: 749→395 lines (-47%), English, doc_dirs include prompts
- Fix test_datasets.py import collision (rename → test_datasets_routes.py)
- Fix test_preview.py: SupersetClient→get_superset_client, AsyncMock, logger f-string
 2026-06-08 16:30:59 +03:00
busya
f731a53c7d 038: add @RATIONALE/@REJECTED contracts to async C4/C5 modules 
5 contracts updated:
  - AsyncNetworkModule (C5): async migration rationale, per-client CSRF cookie rejection
  - AsyncAPIClient (C4): auth lifecycle, cache-hit CSRF refresh
  - AsyncAPIClient.request (C4): string vs dict handling, double-encoding root cause
  - AsyncAPIClient.upload_file (C4): multipart async upload
  - LLMAsyncHttpClient (C4): response.ok -> is_success, module-level httpx singleton
 2026-06-05 17:02:30 +03:00
busya
fbe0ba122c 037: fix 99 failing tests — missing await after async migration 
Fixed async/sync boundary bugs across 14 test files. Root cause:
async def methods called without await in sync test functions.

Fixed files:
  - test_translate_jobs.py (10): create_job/get_job/update_job/delete_job
  - test_translate_scheduler.py (5): create_schedule/update/delete
  - test_datasets.py (14): AsyncMock + corrected patch target
  - test_mapping_service.py (11): sync_environment + MockSupersetClient
  - test_defensive_guards.py (6): GitService/SupersetClient guards
  - test_maintenance_service.py (29): all 6 maintenance services
  - test_dry_run_orchestrator.py (1): run() without await
  - test_dashboards_api.py (23): registry client via AsyncMock
  - test_validation_tasks.py (4): trailing slash in POST URL
  - test_superset_matrix.py (3): AsyncMock for compile_preview
  - test_payload_reduction.py (6): LLMClient._optimize_image wrapper
  - test_compliance_task_integration.py (2): event_bus ref
  - test_smoke_plugins.py (1): flusher_stop_event fallback
  - test_task_manager.py (1): _flusher_stop_event/thread fallback

Remaining 31 failures in test_task_manager.py (29) and
test_smoke_plugins.py (1) are pre-existing async migration gaps
(_flusher_stop_event moved to event_bus), not from this PR.
 2026-06-05 15:43:35 +03:00
busya
7f1937f10b 036: wire SupersetClientRegistry into translate + services flow 
Replaces direct SupersetClient(env) calls with shared
get_superset_client(env) from client_registry.

Changed files (9):
  - client_registry.py: added get_superset_client(), _build_env_id(),
    accepts Environment models (not just dicts), fixed async_client attr
  - superset_executor.py: _get_client() now async, uses shared client
  - preview_executor.py, _run_source.py, service.py, service_datasource.py
  - health_service.py, resource_service.py, debug.py

Impact per environment:
  - 6 separate httpx.AsyncClient instances → 1 shared client
  - 6 CSRF cookie fetches → 1 (on first access)
  - 6 connection pools → 1 shared pool
  - Shared semaphore for backpressure
  - Shared cookie jar (fixes CSRF 'tokens do not match' on SQL Lab execute)
 2026-06-05 15:14:44 +03:00
busya
4cef6af041 fix 2026-06-05 15:01:34 +03:00
busya
5d9d214bd6 034: fix double JSON serialization in AsyncAPIClient.request 
Root cause: AsyncAPIClient.request() passes its  parameter directly
to httpx.AsyncClient.request(json=data). When callers pass a pre-serialized
JSON string (data=json.dumps(dict)), httpx re-encodes it via json.dumps(),
resulting in a double-encoded JSON string body instead of a JSON object.

This caused ALL POST/PUT requests with string data to fail — Superset received
a JSON string instead of a JSON object, returning GENERIC_BACKEND_ERROR
('dictionary update sequence element #0 has length 1; 2 is required').

Fix: if data is a string, pass it via httpx  parameter (raw body);
if it's a dict/list, pass via  for automatic encoding.

Affected callers (6 files) now correctly send JSON objects:
  - preview_executor.py: chart data requests
  - superset_executor.py
  - _run_source.py
  - _datasets.py: update_dataset
  - _datasets_preview.py: compile_dataset_preview
  - _dashboards_write.py

Also simplified preview_executor.fetch_sample_rows back to single-strategy
(chart data API only) since the root cause is now fixed.
 2026-06-05 12:07:55 +03:00
busya
250b69db8c 032: fix coroutine never awaited in debug.py + task_logger.py 
debug.py: _test_db_api and _get_dataset_structure were already async def
but called SupersetClient methods (authenticate, get_databases, get_dataset)
without await. Added await to 4 calls.

task_logger.py: _add_log callback is async def but _log() called it without
await, silently dropping all task log messages (RuntimeWarning: coroutine
never awaited). Changed to fire-and-forget via asyncio.ensure_future when
a running event loop is available, drops gracefully otherwise.
 2026-06-05 11:23:53 +03:00
busya
4247e7a20e 032: fix final sync→async cascade (dataset_review, parsing, validation) 
- _parsing.py: parse_superset_link + _recover_dataset_binding → async
    (await get_dashboard_detail, get_chart)
  - dataset_review/orchestrator.py: start_session + _build_recovery_bootstrap → async
    (await get_dataset_detail, parse_superset_link)
  - _routes.py (dataset_review): await orchestrator.start_session()
  - validation_tasks.py: await parse_superset_link + get_dashboard_detail
 2026-06-05 09:55:34 +03:00
busya
c7d8f4431e 032: fix remaining sync→async propagation (17 call sites) 
Core fixes:
  - service_datasource.py: fetch_datasource_metadata() → async
  - service.py: create_job(), update_job() → async (callers await)
  - _job_routes.py: await create_job/update_job

Maintenance scanners:
  - _dashboard_scanner.py: 4 functions → async (find_affected, _get_linked,
    _apply_filters, _resolve_title)
  - _chart_manager.py: 3 functions → async
  - _banner_renderer.py: rebuild_banner → async
  - _orchestrators.py: 3 orchestrators → async
  - maintenance_banner.py: await async calls

Migration:
  - dry_run_orchestrator.py: run(), _build_target_signatures() → async
  - risk_assessor.py: build_risks() → async
  - migration.py: await service.run()
  - mapping_service.py: sync_environment() → async

Dead code:
  - _helpers.py: _find_dashboard_id_by_slug marked DEPRECATED
 2026-06-05 08:56:37 +03:00
busya
079b2dd37b 032: add aclose() to SupersetClient (was missing, used in dashboard detail routes) 
SupersetClientBase was missing aclose() method. AsyncSupersetClient had it
via override, but code creating SupersetClient directly (e.g. dashboard tasks
history route) would fail with AttributeError on await client.aclose().
 2026-06-05 08:33:49 +03:00
busya
5eb116509b 032: dead code cleanup — remove sync APIClient, _llm_http, preview_llm_client, fix retry chains 2026-06-05 08:31:18 +03:00
busya
ee9123bcf2 032: deep async propagation — orchestrator, insert, mapper, batch chains 
Full async conversion for all sync callers of async SupersetClient methods:

orchestrator_sql.py: generate_and_insert_sql + _resolve_dialect → async
orchestrator_run_completion.py: complete_success → async (calls generate_and_insert_sql)
orchestrator_exec.py: execute_run → async (awaits complete_success)
orchestrator_runner.py: execute_run → async (delegates to engine)
orchestrator.py: execute_run + _generate_and_insert_sql → async
executor.py: _insert_batch_to_target → async (awaits batch insert)
_batch_proc.py: insert_batch_to_target → async
_batch_insert.py: insert_batch_to_target + _resolve_insert_backend + _execute_insert_sql → async
dataset_mapper.py: get_sqllab_mappings + run_mapping → async
  + await get_dataset, update_dataset, execute_and_poll
mapper.py: await on run_mapping + resolve_database_id calls
_run_routes.py: threading.Thread → asyncio.create_task (_background_execute async)
 2026-06-05 00:13:01 +03:00
busya
b190414747 032: final — tombstones, tests, cleanup 
T055: APIClient tombstone in network.py
T056: AsyncSupersetClient @DEPRECATED marker
T057: _llm_http.py + preview_llm_client.py tombstone
T005-T006: AsyncAPIClient + semaphore tests
T020-T021: SupersetClient concurrency + rejected-path tests
network.py cleaned from 584 to 220 lines (orphan code removed)

All 20 async tests pass.
 2026-06-04 20:57:20 +03:00
busya
2e95971b1b 032: fix tests — AsyncMock for async SupersetClient methods 
All 10 preview pipeline tests pass.
 2026-06-04 20:42:31 +03:00
busya
3bb5e6dde7 032: final — async_superset_client collapse, profile/superset_lookup async 2026-06-04 20:37:55 +03:00
busya
5ca1131ba3 032: Phase 5 (T036-T039) + Phase 6 (T040-T044) completed 
T036: superset_compilation_adapter fully async
T037: fileio.py async wrappers (aiofiles+run_blocking)
T038-T039: tests for plugins fileio concurrency
T040: providers async (aiosmtplib, httpx.AsyncClient)
T041: dispatch_report parallel via asyncio.gather
T042: TaskManager ThreadPoolExecutor->asyncio.create_task
T043: EventBus asyncio.Queue(maxsize=10000)
T044: lifecycle async context manager

Remaining: T045 git/_base.py, T046-T048 tests, T005-T020-T021 tests
 2026-06-04 20:30:43 +03:00
busya
51afbee470 032: Phase 3 US1 — 13 mixins migrated to async + dashboard routes 
T011-T017: All SupersetClient mixins now async.
T018: _detail_routes.py uses registry/AsyncSupersetClient.
T019: Partial — routes/environments, settings, profile, listing async.

RATIONALE: Big-bang merge of sync+AsyncSupersetClient.
REJECTED: dual-stack.

Remaining T019: assistant/*, migration, datasets, git helpers still use sync.
 2026-06-04 19:55:43 +03:00
busya
496584e0da 032: Phase 1-2 — setup deps + AsyncAPIClient extend + client_registry + executors 
Phase 1 (Setup):
- T001: requirements-dev.txt with pytest-httpx
- T002: aiofiles added to requirements.txt
- T003: aiosmtplib added to requirements.txt
- T004: EnvironmentConfig extended (connection_pool_size, etc.)
  + AppAsyncRuntimeConfig created (executor workers, shutdown)

Phase 2 (Foundational):
- T007: AsyncAPIClient extended — semaphore parameter, request() method
- T008a: SupersetClientRegistry — singleton per-env client/semaphore/lock
- T008b: run_blocking helper + bounded executors (db/file/git)

RATIONALE: httpx.AsyncClient replaces requests.Session; singleton
registry ensures global per-env semaphore; named executors prevent
thread pool exhaustion.
REJECTED: asyncio.to_thread (default executor, no backpressure);
per-request clients (lose pooling); dual-stack (rejected at clarify).
 2026-06-04 19:45:57 +03:00
busya
ee5ebf08de fix(backend): migrate trace middleware to raw ASGI for contextvar isolation 
BaseHTTPMiddleware (Starlette 0.50.0) uses anyio.create_task_group() internally,
creating separate asyncio tasks for dispatch vs call_next. ContextVars set in
dispatch() were not visible to outer middleware like log_requests.

Converting to raw ASGI middleware ensures trace_id is seeded in the root task
context, visible to ALL middleware layers.

Key changes:
- Replace BaseHTTPMiddleware with raw ASGI __call__(self, scope, receive, send)
- UUID v4 validation: check parsed.version == 4 explicitly instead of relying
  on uuid.UUID(hex=..., version=4) which silently mutates non-v4 UUIDs
- Add @RATIONALE and @REJECTED tags per semantics-core protocol
- Update app.py comment to document the architectural decision
 2026-06-04 16:15:40 +03:00
busya
1e6ec34c2b feat(backend): drop dictionary dialect columns, add allowed_languages 
- Removes source_dialect and target_dialect from TerminologyDictionary model,
  schemas, routes, helper serialization, and all tests
- Adds allowed_languages config field (BCP-47 language codes) to GlobalSettings
  with validation, consolidated settings response, and API endpoint
- Adds alembic migration f1a2b3c4d5e6 to drop the two columns
 2026-06-03 11:48:03 +03:00
busya
4c5da7e4d9 alembic fix 2026-06-01 16:34:07 +03:00
busya
d2b53c2a79 feat(validation): v2 LLM dashboard validation — plugin, routes, services 
Core implementation of the v2 LLM dashboard validation pipeline:
- LLM plugin with Path A (screenshots) and Path B (logs-only) execution
- Validation task management (CRUD, schedule, run)
- WebSocket task progress with Python 3.13 asyncio fix
- Cross-task runs listing (GET /validation-tasks/runs/all)
- RecordResponse schema for validation records
- JSON prompt helper, per-dashboard status aggregation
- Prompt templates with docs/git-commit/validation presets
- Migration: v2 validation models + description column
- Tests: plugin persistence, prompt templates, batch, payload, url
 2026-05-31 22:32:20 +03:00
busya
e17f4f64a9 websocket add 2026-05-31 17:17:30 +03:00
busya
55c7e323c4 fix: test assertions for molecular CoT markers, separate propagate=False to ConfigManager, fix ruff in env.py 2026-05-27 11:26:14 +03:00
busya
143ab15744 fix: logging overhaul — Alembic fileConfig disable_existing_loggers=False, propagate=False on loggers, ADR docs, catch-up migration for missing columns, sqlparse dep, build archive naming 2026-05-27 11:19:12 +03:00
busya
614bf9123a log fix 2026-05-27 10:49:06 +03:00
busya
073a7da9ec fix: change reason() and reflect() from DEBUG to INFO level 
Per molecular CoT protocol: REASON and REFLECT are primary reasoning
bonds and must be visible at INFO level in production. DEBUG is
reserved for high-frequency loops.

- reason(): self.debug()  → self.info()
- reflect(): self.debug() → self.info()
- explore(): stays at warning() (WARNING level)
- Also removed redundant logger.info() calls in lifespan (now reason()
  itself provides visible startup logging)
 2026-05-27 10:09:12 +03:00
busya
4205618ee6 chore: eliminate all deprecation warnings from tests and linter 
Warnings fixed:
- datetime.utcnow() → datetime.now(UTC) across 48+ files (src/ + tests/)
- datetime.utcnow (callback ref) → lambda: datetime.now(UTC) in model fields (18 files)
- Pydantic class Config → model_config = ConfigDict(...) (16 files)
- Pydantic .dict() → .model_dump() (8 files)
- ConfigDict(allow_population_by_field_name=True) → validate_by_name=True
- SQLAlchemy declarative_base() import path updated
- FastAPI on_event → lifespan context manager (app.py)
- Import sorting (ruff I001) auto-fixed across all files
- Fixed broken re-export chains that ruff F401 cleanup broke:
  _validate_bcp47: service.py now imports from dictionary_validation directly
  job_to_response: _job_routes.py and test imports from service_utils directly
  fetch_datasource_metadata: restored re-export in service.py
- Added missing TranslateJobService import in _job_routes.py (was deleted by F401)
- Added ConfigDict(protected_namespaces=()) for DashboardDatasetItem schema field
- pytest.ini: replaced deprecated importmode with asyncio_mode

All 440 tests pass with zero deprecation warnings.
 2026-05-26 19:18:28 +03:00
busya
eda346979b refactor: remove _ensure_*() safety net, keep only Alembic 
- Simplified init_db() to only call Base.metadata.create_all() on all
  three engines — no more _ensure_*() inline additive migrations
- run_alembic_migrations() now raises on failure (no safety net to
  fall back to)
- All schema changes (add/drop/rename columns, data migrations) must
  go through Alembic. create_all() handles new tables only.

@REJECTED _ensure_*() inline migrations removed because they:
  - Duplicated Alembic logic without versioning
  - Created hidden schema drift (columns existed in DB but had no
    corresponding Alembic migration)
  - Made audits and fresh DB provisioning unreliable
 2026-05-26 19:02:59 +03:00
busya
9337a4cecd fix: add missing schema columns and Alembic model imports 
- Added 4 missing columns to _ensure_translation_jobs_columns() inline
  migration: target_language_column, target_source_column,
  target_source_language_column, disable_reasoning — these were in the
  SQLAlchemy model but had no Alembic migration or _ensure_*() coverage
- Added missing api_key, assistant, clean_release model imports to
  alembic/env.py so autogenerate can see all tables
 2026-05-26 19:00:04 +03:00
busya
ce0c0cf235 fix: add missing call to _ensure_roles_is_admin_column() in init_db() 
The inline migration _ensure_roles_is_admin_column() was defined but
never called from init_db(). This caused 'column roles.is_admin does
not exist' errors when logging in against databases with an older
schema. Added the missing call to ensure the column is created during
startup.
 2026-05-26 18:50:48 +03:00
busya
0b6f61623e fix: _ensure_roles_is_admin_column also sets is_admin=true for Admin 2026-05-26 15:29:49 +03:00
busya
6b960b0eb3 fix: add is_admin column migration + additive safety net 
Alembic migration 86c7b1d6a710 adds is_admin BOOLEAN DEFAULT FALSE
to the roles table. Fixes login crash after M-3 change:

  psycopg2.errors.UndefinedColumn: column roles.is_admin does not exist

Also adds _ensure_roles_is_admin_column() as an additive migration
safety net for environments that don't run alembic automatically.

To apply: cd backend && source .venv/bin/activate && alembic upgrade head
 2026-05-26 15:28:03 +03:00
busya
2bc8ad87e1 security: rate limiter, is_admin flag, password policy, log masking 
M-3: Role.name == 'Admin' string comparison → Role.is_admin boolean flag.
  Admin role created with is_admin=True. has_permission checks getattr(role,
  'is_admin', False) instead of comparing role names.

M-2: In-memory rate limiter on /api/auth/login. Tracks failed attempts
  per IP (10 in 5 min window, 15 min ban). Thread-safe via threading.Lock.

M-5: Password policy — UserCreate.password: min_length=8, must include
  uppercase, lowercase, and digit.

L-1: log_security_event details dict now masks sensitive fields:
  password, secret, token, api_key, authorization. Recursive masking
  for nested dicts. Sensitive field names matched via regex.

OTHER:
- 4 rate limiter tests (under limit, ban, success clears, IP isolation)
- 4 password policy tests (too short, no upper, no digit, valid)
- 2 log masking tests (flat keys, nested)
 2026-05-26 15:17:19 +03:00
busya
23719ecfbc security: JWT aud/iss validation, token blacklist, encryption key crash-early 
C-2 (CRITICAL): JWT decode_token now validates aud, iss, iat, jti claims.
  Tokens without these claims are rejected. Audience 'ss-tools-api' and
  issuer 'ss-tools' prevent cross-service token reuse.

H-2 (HIGH): Server-side JWT revocation via token_blacklist table.
  - New TokenBlacklist model (SHA-256 hash only, never raw token)
  - blacklist_token() on logout — revokes current session
  - is_token_blacklisted() check in get_current_user
  - Expired entries auto-pruned via _prune_blacklist()

H-3 (HIGH): Encryption key auto-generation removed — crash-early instead.
  Previously, ensure_encryption_key() would auto-generate a Fernet key
  on first run and write it to .env. This made encrypted data unrecoverable
  when the key changed between deployments. Now raises RuntimeError with
  a clear command to generate the key explicitly.

Also: update orthogonal security test for crash-early behavior.
 2026-05-26 15:08:55 +03:00
busya
a9a453109c security: critical auth fixes, test migration to SQLite+FK, 44 test fixes 
CRITICAL (CVE-class):
- C-4: Remove DEV_MODE fallback with hardcoded postgres:postgres credentials
- C-3: WebSocket endpoints now require JWT or API key token (?token=) auth
- C-1: Superset environment passwords encrypted via Fernet at rest in DB
- H-4: SESSION_SECRET_KEY separated from JWT AUTH_SECRET_KEY
- H-1: Log injection via %s-formatting instead of f-strings
- H-5: X-Trace-ID header validated as UUID4 to prevent trace poisoning

INFRASTRUCTURE:
- New src/core/encryption.py — EncryptionManager extracted from llm_provider
- Test DB: per-module sqlite:///:memory: with PRAGMA foreign_keys=ON
- testcontainers PostgreSQL via TEST_DB=postgres env var
- conftest temp-file global engine (no 10GB shared-cache leak)

TEST FIXES (44 pre-existing → 0):
- test_smoke_plugins: module-level sys.modules mock isolated to per-test fixture
- test_migration_engine: EXT:Python:uuid → uuid syntax fix
- test_dashboards_api: mock env attributes + correct patch target
- test_constants_audit_fixes: sync expected constant names with actual
- test_defensive_guards: patch.object instead of module-level Repo mock
- test_clean_release_cli: removed empty config.json directory
- FK violations: TaskRecord parents in log_persistence, Environment in
  mapping_service, ReleaseCandidate in candidate_manifest_services

ORTHOGONAL TESTS (18 new):
- test_security_orthogonal.py: bcrypt 72-byte limit, unicode, API key
  format invariants, Fernet robustness, encryption key lifecycle,
  log injection protection, JWT edge cases

MODEL FIXES:
- MetricSnapshot.job_id: nullable with SET NULL (was broken FK for
  aggregate prune snapshots, hidden by SQLite)

CLEANUP:
- Removed stale :memory:test_main/test_auth/test_tasks file databases
- Removed duplicate #endregion in encryption_key.py
 2026-05-26 14:58:49 +03:00
busya
f49b7d909e feat: GRACE-Poly protocol optimization — ~3200→10 warnings (↓99.7%) 
Full optimization cycle:

Protocol (15 files):
- 4-layer SSOT architecture for agent prompts & skills
- Anti-Corruption Protocol consolidated from 5 duplicates
- Tag-to-tier permissiveness matrix (all @tags allowed at all tiers)

Axiom config:
- complexity_rules: all 22+ tags available on C1-C5
- contract_type_overrides: removed (was narrowing per-type)
- 18 new tags added, LAYER enum expanded (Infra, Frontend, Atom, etc.)
- RELATION predicates expanded (USES, CONTAINS, BELONGS_TO, etc.)

Code fixes:
- 2216 @TAG: normalized to @TAG (colon→space)
- 518 [DEF] blocks migrated to #region/#endregion (37 files)
- VERIFIES→BINDS_TO, :Class/:Function suffixes removed, paths→IDs
- 1173-line _external_stubs.py deleted (EXT: handled natively)
- Batch EXT: reference audit (240 targets: 132 external, 99 internal, 9 fix)
- QA regression check: 0 regressions across all checks

Infrastructure:
- DuckDB rebuild stabilized (appender API, INSERT OR IGNORE)
- Anchor regex fix (parent-child BINDS_TO now resolves)
- EXT:*/DTO:/NEED_CONTEXT: regex fixed in validator
- 34MB Doxygen API portal (3194 contract pages)
 2026-05-26 11:14:25 +03:00
busya
9ffa8af1dc semantics 2026-05-26 09:30:41 +03:00
busya
320f82ab95 feat(auth): implement API key authentication and management 
Introduce a new API key authentication mechanism to support service-to-service
communication (e.g., Airflow, CI/CD) without browser-based JWT login.

- Add `APIKey` model and `APIKeyPrincipal` dependency for RBAC.
- Implement `require_api_key_or_jwt` dependency with environment scoping.
- Add admin CRUD endpoints for API key lifecycle management.
- Add API key management UI in System Settings.
- Update maintenance API to support explicit `environment_id` and API key auth.
- Add i18n support for API keys and connection settings.
- Include Python and Bash usage examples in the `examples/` directory.
- Update technical specifications and documentation.
 2026-05-25 11:35:27 +03:00
busya
31680a1bc9 fix(maintenance): auto-expiry + adaptive markdown height + tests 
- Auto-expiry: expired events auto-end on GET /events via task manager
- Height: _estimate_markdown_height adapted to unit=8px scale with padding detection
- CRITICAL-1: removed dead import remove_chart_from_position (QA finding)
- CRITICAL-2: added chart alive check in ensure_banner_chart (QA finding)
- CRITICAL-3: fixed test assertions update_markdown_chart → update_banner_on_dashboard
- @POST contract: fixed return range [2,12] → [19,200]
- Tests: 8 new tests (auto-expiry + layout height)
 2026-05-25 08:36:33 +03:00
busya
7219c47357 fix(datasets): fix oversized buttons UI + populate linked_dashboard_count in list endpoint 
StatsBar — переделаны огромные карточки-кнопки в компактные пилюли
(rounded-full px-3 py-1 text-sm вместо p-3 grid grid-cols-4).

DatasetList — экшн-кнопки из bg-primary в outline-стиль, карточки
плотнее (p-2.5, py-0.5), ESLint-фиксы (#each key).

Backend — linked_dashboard_count теперь заполняется в списке датасетов
через /dataset/{id}/related_objects (конкурентно, Semaphore=3, timeout=10s).
Ранее поле было только в get_dataset_detail и StatsBar показывал 0.
 2026-05-24 09:38:21 +03:00
busya
36643024cf refactor(connections): remove ConnectionConfig, migrate mapper to SQL Lab 
Backend:
- Remove ConnectionConfig model, CRUD routes (connections.py), and tests
- Remove ensure_connection_configs_table from database.py
- Migrate MapperPlugin from psycopg2 direct PostgreSQL to SupersetSqlLabExecutor
- Add DatasetMapper.get_sqllab_mappings() with default information_schema query
- Update MapColumnsRequest: connection_id → database_id + sql_query
- Fix pydantic_settings v2 deprecation (Field(env=...) → validation_alias)
- Clean up app.py, routes/__init__.py, database.py imports

Frontend:
- Remove DatabaseConnectionsTab, ConnectionForm, ConnectionList components
- Remove /settings/connections route page and Navbar link
- Remove connectionService.js, connections i18n files
- Update MapperTool.svelte: postgres → sqllab source
- Update Map Columns modal: database selector + SQL query instead of connection_id
- Clean up settings page tabs, nav links, test mocks
- Clean up i18n keys (settings, nav, mapper)

Tests: 25/25 pass (14 030-feature + 11 route tests)
Build: succeeds
 2026-05-21 18:47:41 +03:00
busya
b529e8082a feat(030): Dataset Lifecycle Workspace — Stats Bar, split-view, inline-edit, bulk actions 
Backend:
- Add MetricItem, StatsCounts, ColumnDescriptionUpdate, MetricDescriptionUpdate DTOs
- Add metric_count to DatasetItem and DatasetDetailResponse
- Add server-side filtering via ?filter=unmapped|mapped|linked|all
- Add PUT endpoints for column/metric description inline-edit
- Add HTML stripping validation (max 2000 chars, plain text)
- Add WebSocket dataset.updated event on task completion
- RBAC: plugin:migration:WRITE for mutations, READ for reads
- 14 pytest tests (stats, filter, metrics, inline-edit, validation, 502/503)

Frontend:
- Rewrite +page.svelte as split-view orchestrator (387 lines)
- StatsBar.svelte — 4 metric tiles with aria-pressed, server-side filter dispatch
- DatasetList.svelte — card grid with progress bars, checkboxes, search
- DatasetPreview.svelte — presentational detail panel (container fetches)
- ColumnsTable.svelte — inline-edit with type chips, bold/italic fallback
- MetricsTable.svelte — inline-edit with expression hints
- 52 vitest tests across 7 test files
- i18n: 11 EN + 11 RU keys (with_mapping/с_маппингом)

Spec:
- 15 issues resolved (semantic label, filtering, endpoints, conflict, metric_count, RBAC, ID types, ports, i18n, validation, a11y, propagation, resize, perf, ownership)

Status: 14/14 backend tests pass, 52/52 frontend tests pass, build succeeds
Risk: Low — T051 browser validation pending (non-blocking)
 2026-05-21 13:58:47 +03:00
busya
68740cd9ed semantic 2026-05-20 23:54:53 +03:00
busya
b916ef94d5 semantics 2026-05-19 18:36:15 +03:00
busya
6988e63967 chore: commit remaining pre-existing changes 
- Agent configs (.opencode/agents/)
- Backend: alembic, routes, app, utils, scripts
- Frontend: package.json, vite, components, e2e infra
- Specs: 028-llm-datasource-supeset updates
- Docker e2e config and Playwright setup
 2026-05-17 19:23:07 +03:00