feat(031): Maintenance Banner for Dashboards — full spec→plan→tasks package
Spec: 5 user stories (P1-P3), 17 FRs, 7 SCs, 12 edge cases, RBAC matrix Plan: 7 research decisions, 15 semantic contracts (C1-C4) Data: 4 SQLAlchemy models (Banner entity enforces one-chart-per-dashboard) UX: async-only API, banner template with optional variables, admin settings UI Tasks: 64 tasks across 8 phases, 12 reused frontend components Workflow: component reuse scan mandated in speckit.plan/tasks + semantics-svelte Constitution: filled with 7 architecture principles from ADRs
This commit is contained in:
@@ -13,17 +13,27 @@
|
||||
@FILE backend/src/models/maintenance.py
|
||||
@RELATION DEPENDS_ON -> [Base:Model]
|
||||
|
||||
Columns: id (str PK), task_id (str, FK→tasks.id), tables (JSON), start_time (datetime tz), end_time (datetime tz), message (text?), status (enum: active/completed/failed), created_at, updated_at.
|
||||
Columns: id (str PK), task_id (str, loose reference to TaskManager), environment_id (str — snapshot of target env at creation), tables (JSON), start_time (datetime tz, required), end_time (datetime tz, nullable), message (text?), status (enum: pending/applying/active/partial/ending/completed/failed), created_at, updated_at.
|
||||
|
||||
### [DEF:MaintenanceDashboardBanner:Model]
|
||||
@COMPLEXITY 1
|
||||
@PURPOSE Canonical "one chart per dashboard" entity — enforces invariant exactly one banner chart per dashboard regardless of active event count. Unique partial index on (environment_id, dashboard_id) WHERE status='active'.
|
||||
@LAYER models
|
||||
@FILE backend/src/models/maintenance.py
|
||||
@RELATION DEPENDS_ON -> [Base:Model]
|
||||
|
||||
Columns: id (str PK), environment_id (str), dashboard_id (int), chart_id (int? — Superset markdown chart ID), banner_text (text — current aggregated markdown), status (enum: active/removed), created_at, updated_at.
|
||||
|
||||
### [DEF:MaintenanceDashboardState:Model]
|
||||
@COMPLEXITY 1
|
||||
@PURPOSE Junction model linking maintenance events to dashboards with chart tracking.
|
||||
@PURPOSE Junction linking events to dashboards. References shared MaintenanceDashboardBanner — does not own chart_id directly.
|
||||
@LAYER models
|
||||
@FILE backend/src/models/maintenance.py
|
||||
@RELATION DEPENDS_ON -> [Base:Model]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceEvent:Model]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceDashboardBanner:Model]
|
||||
|
||||
Columns: id (str PK), event_id (str FK→maintenance_events.id), dashboard_id (int), environment_id (str), chart_id (int?), status (enum: active/removed/removal_failed), created_at, removed_at. Index: (dashboard_id, status).
|
||||
Columns: id (str PK), event_id (str FK→maintenance_events.id), dashboard_id (int), banner_id (str FK→maintenance_dashboard_banners.id), status (enum: pending_apply/active/removing/removed/apply_failed/removal_failed), created_at, updated_at. Index: (dashboard_id, status).
|
||||
|
||||
### [DEF:MaintenanceSettings:Model]
|
||||
@COMPLEXITY 1
|
||||
@@ -32,7 +42,7 @@ Columns: id (str PK), event_id (str FK→maintenance_events.id), dashboard_id (i
|
||||
@FILE backend/src/models/maintenance.py
|
||||
@RELATION DEPENDS_ON -> [Base:Model]
|
||||
|
||||
Columns: id (str PK, "default"), target_environment_id (str), dashboard_scope (enum: published_only/draft_only/all), excluded_dashboard_ids (JSON), forced_dashboard_ids (JSON), updated_at.
|
||||
Columns: id (str PK, "default"), target_environment_id (str), display_timezone (str, default "UTC"), banner_template (text — markdown with optional `{start_time}`, `{end_time}`, `{message}`; default includes all three), dashboard_scope (enum: published_only/draft_only/all), excluded_dashboard_ids (JSON), forced_dashboard_ids (JSON), updated_at.
|
||||
|
||||
---
|
||||
|
||||
@@ -43,48 +53,31 @@ Columns: id (str PK, "default"), target_environment_id (str), dashboard_scope (e
|
||||
@FILE backend/src/api/routes/maintenance/_schemas.py
|
||||
@RELATION DEPENDS_ON -> [pydantic.BaseModel]
|
||||
|
||||
Schemas: MaintenanceStartRequest, MaintenanceSettingsUpdate, MaintenanceStartResponse, MaintenanceTaskResult, MaintenanceEndResponse, MaintenanceEndTaskResult, MaintenanceEndAllTaskResult, MaintenanceSettingsResponse, MaintenanceEventItem, MaintenanceEventListResponse, MaintenanceDashboardBannerState.
|
||||
|
||||
---
|
||||
|
||||
### [DEF:SupersetDashboardsWriteMixin:Class]
|
||||
@COMPLEXITY 4
|
||||
@PURPOSE Extends SupersetClient with write operations: create markdown chart, update dashboard layout, delete chart.
|
||||
@PURPOSE Extends SupersetClient with dashboard write operations.
|
||||
@LAYER core
|
||||
@FILE backend/src/core/superset_client/_dashboards_write.py
|
||||
@RELATION DEPENDS_ON -> [SupersetClientBase:Class]
|
||||
@RELATION DEPENDS_ON -> [NetworkClient:Class]
|
||||
@PRE SupersetClient is initialized with valid bearer token and environment base URL.
|
||||
@POST Chart created in Superset; chart_id returned. Dashboard position_json updated.
|
||||
@SIDE_EFFECT Modifies Superset dashboards via REST API (POST /chart/, PUT /dashboard/{id}, DELETE /chart/{id}).
|
||||
@RATIONALE Direct chart API is faster than export-import cycle for single-chart operations (see research.md R1).
|
||||
@POST Chart created/updated/deleted in Superset. Dashboard position_json updated.
|
||||
@SIDE_EFFECT Modifies Superset dashboards via REST API.
|
||||
@RATIONALE Direct chart API faster than export-import cycle for single-chart ops.
|
||||
|
||||
Methods:
|
||||
- `create_markdown_chart(dashboard_id: int, markdown_text: str) -> int` — Creates viz_type="markdown" chart, returns chart_id.
|
||||
- `update_dashboard_layout(dashboard_id: int, chart_id: int, position: str) -> dict` — Patches position_json to insert chart at given position.
|
||||
- `delete_chart(chart_id: int) -> bool` — Deletes a chart from Superset.
|
||||
- `get_dashboard_layout(dashboard_id: int) -> dict` — Fetches current position_json for reading existing layout.
|
||||
|
||||
---
|
||||
Methods: `create_markdown_chart`, `update_markdown_chart`, `update_dashboard_layout` (insert at (0,0), 12 cols, shift existing down), `remove_chart_from_layout`, `delete_chart`, `get_dashboard_layout`.
|
||||
|
||||
### [DEF:SqlTableExtractor:Module]
|
||||
@COMPLEXITY 2
|
||||
@PURPOSE Extract fully-qualified table names (schema.table) from Superset virtual dataset SQL+Jinja text. Uses two-phase approach: (1) regex global extraction of all schema.table candidates from raw text, (2) sqlparse filtering to reject candidates inside SQL string literals. Jinja is NOT pre-stripped — table names inside {% set %} blocks are preserved.
|
||||
@PURPOSE Extract schema.table from virtual dataset SQL+Jinja. Two-phase: Jinja span detection, regex global extraction, sqlparse filtering only on SQL spans (string literals).
|
||||
@LAYER services
|
||||
@FILE backend/src/services/sql_table_extractor.py
|
||||
@RELATION DEPENDS_ON -> [sqlparse]
|
||||
@RATIONALE Table names inside Jinja {% set %} blocks (common in Superset virtual datasets) are invisible to SQL parsers after Jinja stripping. Global extraction first, then false-positive filtering, ensures full coverage. See research.md R3.
|
||||
|
||||
Functions:
|
||||
- `extract_tables_from_sql(raw_sql: str) -> set[str]` — Orchestrator: calls extract_candidates then filter_string_literals. Returns deduplicated set of "schema.table" names (case-insensitive matching per clarification Q3).
|
||||
- `extract_table_candidates(raw_sql: str) -> list[str]` — Regex finds all `[a-zA-Z][\w]*\.[a-zA-Z][\w]*` patterns across raw text (SQL + Jinja).
|
||||
- `filter_string_literals(raw_sql: str, candidates: list[str]) -> set[str]` — Uses sqlparse tokenizer to reject candidates whose byte position falls inside a SQL string literal or comment.
|
||||
|
||||
---
|
||||
@RATIONALE Table names in {% set %} blocks must survive; stripping Jinja first loses them.
|
||||
|
||||
### [DEF:MaintenanceService:Class]
|
||||
@COMPLEXITY 4
|
||||
@PURPOSE Core business logic: find affected dashboards, apply banners, remove banners, rebuild aggregated banner text.
|
||||
@PURPOSE Core business logic: discovery, banner placement/removal, text rebuild.
|
||||
@LAYER services
|
||||
@FILE backend/src/services/maintenance_service.py
|
||||
@RELATION DEPENDS_ON -> [SupersetClient:Class]
|
||||
@@ -92,26 +85,19 @@ Functions:
|
||||
@RELATION DEPENDS_ON -> [SqlTableExtractor:Module]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceEvent:Model]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceDashboardState:Model]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceDashboardBanner:Model]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceSettings:Model]
|
||||
@RELATION DEPENDS_ON -> [TaskManager:Class]
|
||||
@PRE Active database session available. Superset client initialized for target environment.
|
||||
@POST MaintenanceEvent created/updated. MaintenanceDashboardState records created/updated. Chart added/removed from Superset dashboards.
|
||||
@SIDE_EFFECT Creates/deletes charts in Superset; writes to maintenance_events and maintenance_dashboard_states tables; dispatches WebSocket task progress events.
|
||||
@RATIONALE Single service orchestrates the full maintenance lifecycle, ensuring consistency between local state and Superset state (see research.md R2).
|
||||
@PRE Active DB session. Superset client initialized.
|
||||
@POST Event/state/banner records updated. Charts created/updated/deleted.
|
||||
@SIDE_EFFECT Superset mutations; DB writes; WebSocket progress events.
|
||||
@RATIONALE Single orchestrator ensures consistency between local state and Superset.
|
||||
|
||||
Methods:
|
||||
- `start_maintenance(tables, start_time, end_time, message, db_session) -> MaintenanceEvent` — C4: Discovery + banner placement.
|
||||
- `end_maintenance(event_id, db_session) -> None` — C4: Banner removal for one event.
|
||||
- `end_all_maintenance(db_session) -> None` — C4: Bulk banner removal.
|
||||
- `find_affected_dashboards(tables, superset_client) -> list[dict]` — C3: Cross-references tables against datasets.
|
||||
- `build_banner_text(events: list[MaintenanceEvent]) -> str` — C2: Aggregates text from multiple active events.
|
||||
- `rebuild_banner(dashboard_id, superset_client, db_session) -> None` — C3: Updates chart markdown text in Superset.
|
||||
|
||||
---
|
||||
Methods: `start_maintenance` (idempotency by (tables, start, end)), `end_maintenance`, `end_all_maintenance`, `find_affected_dashboards`, `ensure_banner_chart` (shared chart per dashboard), `build_banner_text` (template substitution), `rebuild_banner` (update_markdown_chart).
|
||||
|
||||
### [DEF:MaintenanceRoutes:Module]
|
||||
@COMPLEXITY 3
|
||||
@PURPOSE FastAPI route handlers for maintenance API endpoints.
|
||||
@PURPOSE FastAPI route handlers for maintenance endpoints.
|
||||
@LAYER api
|
||||
@FILE backend/src/api/routes/maintenance/_routes.py
|
||||
@RELATION DEPENDS_ON -> [MaintenanceService:Class]
|
||||
@@ -119,117 +105,92 @@ Methods:
|
||||
@RELATION DEPENDS_ON -> [TaskManager:Class]
|
||||
@RELATION DEPENDS_ON -> [AuthMiddleware:Class]
|
||||
|
||||
Endpoints:
|
||||
- `POST /api/maintenance/start` → MaintenanceStartResponse
|
||||
- `POST /api/maintenance/{maintenance_id}/end` → MaintenanceEndResponse
|
||||
- `POST /api/maintenance/end-all` → MaintenanceEndResponse
|
||||
- `GET /api/maintenance/events` → MaintenanceEventListResponse
|
||||
- `GET /api/maintenance/settings` → MaintenanceSettingsResponse
|
||||
- `PUT /api/maintenance/settings` → MaintenanceSettingsResponse
|
||||
- `GET /api/maintenance/dashboard-banners` → list[MaintenanceDashboardBannerState]
|
||||
|
||||
---
|
||||
7 endpoints: start (always 202), end (202), end-all (202), events (GET), settings (GET/PUT), dashboard-banners (GET). RBAC per FR-015 matrix.
|
||||
|
||||
### [DEF:MaintenanceRouter:Module]
|
||||
@COMPLEXITY 2
|
||||
@PURPOSE APIRouter assembly with prefix and tag registration.
|
||||
@PURPOSE APIRouter assembly with prefix and tag.
|
||||
@LAYER api
|
||||
@FILE backend/src/api/routes/maintenance/_router.py
|
||||
@RELATION DEPENDS_ON -> [MaintenanceRoutes:Module]
|
||||
|
||||
Creates `APIRouter(prefix="/api/maintenance", tags=["maintenance"])` and includes all route handlers.
|
||||
---
|
||||
|
||||
## Frontend Contracts
|
||||
|
||||
### [DEF:MaintenanceStore:Module]
|
||||
@COMPLEXITY 3
|
||||
@PURPOSE Svelte 5 rune store for maintenance event state, shared across components.
|
||||
@PURPOSE Svelte 5 rune store for maintenance state. Shared by management page and Dashboard Hub badge.
|
||||
@LAYER store
|
||||
@FILE frontend/src/lib/stores/maintenance.svelte.js
|
||||
@RELATION DEPENDS_ON -> [requestApi:Function]
|
||||
@RATIONALE Shared store enables maintenance badge in Dashboard Hub and management page to share state without prop drilling (ADR-0006).
|
||||
@RATIONALE Shared store avoids prop drilling across routes. Uses `$effect(subscribe)` per ADR-0007.
|
||||
|
||||
State:
|
||||
- `events`: $state — active and completed event lists.
|
||||
- `settings`: $state — MaintenanceSettings object.
|
||||
- `dashboardBanners`: $state — Map<dashboardId, bannerState> for Dashboard Hub indicator.
|
||||
- `isLoading`: $state — loading flag.
|
||||
- `error`: $state — error message.
|
||||
State: `events` ($state), `settings` ($state), `dashboardBanners` ($state Map), `isLoading`, `error`. Auto-polls events every 30s; WebSocket covers task progress.
|
||||
|
||||
### [DEF:MaintenanceApiClient:Module]
|
||||
@COMPLEXITY 2
|
||||
@PURPOSE Typed fetch wrappers for all maintenance API endpoints.
|
||||
@PURPOSE Typed fetch wrappers for maintenance API.
|
||||
@LAYER api
|
||||
@FILE frontend/src/lib/api/maintenance.js
|
||||
@RELATION DEPENDS_ON -> [requestApi:Function]
|
||||
|
||||
Functions:
|
||||
- `startMaintenance(payload)` → `{task_id, maintenance_id}`
|
||||
- `endMaintenance(maintenanceId)` → `{task_id}`
|
||||
- `endAllMaintenance()` → `{task_id}`
|
||||
- `getEvents()` → `{active, completed}`
|
||||
- `getSettings()` → `MaintenanceSettingsResponse`
|
||||
- `updateSettings(payload)` → `MaintenanceSettingsResponse`
|
||||
- `getDashboardBanners()` → `Map<id, state>`
|
||||
|
||||
---
|
||||
7 functions mirroring the 7 API endpoints.
|
||||
|
||||
### [DEF:MaintenanceBannerPage:Component]
|
||||
@COMPLEXITY 3
|
||||
@PURPOSE Management page: settings panel + active/completed events tables + bulk actions.
|
||||
@PURPOSE Management page at `/maintenance`. Compose settings panel + events table from store.
|
||||
@LAYER route
|
||||
@FILE frontend/src/routes/maintenance/+page.svelte
|
||||
@RELATION DEPENDS_ON -> [MaintenanceStore:Module]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceSettingsPanel:Component]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceEventsTable:Component]
|
||||
@UX_STATE idle -> Page loaded with settings and events tables.
|
||||
@UX_STATE loading -> Skeleton placeholders for both panels.
|
||||
@UX_STATE action_in_progress -> Remove button shows spinner; other actions disabled.
|
||||
@UX_STATE error -> Error toast with retry button.
|
||||
@UX_STATE empty -> "No active maintenance events" placeholder.
|
||||
@UX_FEEDBACK Toast on success (green), error (red), warning (yellow partial).
|
||||
@UX_RECOVERY Retry button on error. Auto-polling resumes after recovery.
|
||||
@UX_REACTIVITY $effect polls MaintenanceStore every 30s.
|
||||
|
||||
---
|
||||
@UX_STATE idle/loading/action_in_progress/error/empty
|
||||
@UX_FEEDBACK Toast via `addToast()` from `$lib/toasts.js`
|
||||
|
||||
### [DEF:MaintenanceSettingsPanel:Component]
|
||||
@COMPLEXITY 3
|
||||
@PURPOSE Collapsible settings form: dashboard scope radio, excluded/forced dashboard multi-select.
|
||||
@PURPOSE Collapsible settings form using existing UI primitives.
|
||||
@LAYER component
|
||||
@FILE frontend/src/lib/components/MaintenanceSettingsPanel.svelte
|
||||
@RELATION DEPENDS_ON -> [MaintenanceStore:Module]
|
||||
@UX_STATE idle -> Form with current values.
|
||||
@UX_STATE saving -> Save button disabled, spinner.
|
||||
@UX_STATE saved -> Brief green flash on form border.
|
||||
@UX_STATE error -> Red border, error message below form.
|
||||
@UX_FEEDBACK Toast on save success/failure.
|
||||
@UX_RECOVERY Retry save on error.
|
||||
@RELATION DEPENDS_ON -> [Select:Component] (from `$lib/ui/Select.svelte`)
|
||||
@RELATION DEPENDS_ON -> [Input:Component] (from `$lib/ui/Input.svelte`)
|
||||
@RELATION DEPENDS_ON -> [SearchableMultiSelect:Component] (from `$lib/components/ui/SearchableMultiSelect.svelte`)
|
||||
@RATIONALE Reuses existing form atoms (Select, Input) and dashboard picker (SearchableMultiSelect) — zero new UI primitives needed.
|
||||
|
||||
---
|
||||
Form fields:
|
||||
- Environment: `<Select>` with env options
|
||||
- Timezone: `<Input>` text
|
||||
- Scope: native `<input type="radio" bind:group>` (matches ScheduleConfig pattern)
|
||||
- Excluded/Forced: `<SearchableMultiSelect>` with dashboards as `{id, name}`
|
||||
- Banner template: `<textarea>` with live preview div
|
||||
- Save: `<Button isLoading={isSaving}>` from `$lib/ui/Button.svelte`
|
||||
|
||||
Collapsible via `<details><summary>` (matches migration page pattern).
|
||||
|
||||
@UX_STATE idle/saving/saved/error
|
||||
|
||||
### [DEF:MaintenanceEventsTable:Component]
|
||||
@COMPLEXITY 3
|
||||
@PURPOSE Table of active/completed maintenance events with per-event and bulk removal actions.
|
||||
@PURPOSE Active/completed events table with removal actions.
|
||||
@LAYER component
|
||||
@FILE frontend/src/lib/components/MaintenanceEventsTable.svelte
|
||||
@RELATION DEPENDS_ON -> [MaintenanceStore:Module]
|
||||
@RELATION DEPENDS_ON -> [MaintenanceApiClient:Module]
|
||||
@UX_STATE idle -> Rows with action buttons.
|
||||
@UX_STATE removing -> Row action button shows spinner, Confirm dialog for "Remove All".
|
||||
@UX_STATE error -> Failed rows highlighted red with retry button.
|
||||
@UX_FEEDBACK Toast on removal success/failure. Confirmation dialog for "Remove All".
|
||||
@UX_RECOVERY Per-row retry on removal failure.
|
||||
@RELATION DEPENDS_ON -> [Button:Component] (from `$lib/ui/Button.svelte`)
|
||||
|
||||
---
|
||||
Uses inline table pattern from health page (`min-w-full divide-y`), skeleton rows (`animate-pulse`), empty state (`border-dashed bg-gray-50`), badges (inline Tailwind `rounded-full`), confirmation via native `confirm()`. Spinner via `Button(isLoading={true})`.
|
||||
|
||||
@UX_STATE idle/removing/error
|
||||
|
||||
### [DEF:DashboardGridBannerIndicator:Component]
|
||||
@COMPLEXITY 2
|
||||
@PURPOSE Maintenance badge cell in Dashboard Hub table — shows orange badge when dashboard has active banner.
|
||||
@PURPOSE Orange badge in Dashboard Hub row when maintenance active.
|
||||
@LAYER component
|
||||
@FILE frontend/src/lib/components/DashboardMaintenanceBadge.svelte
|
||||
@RELATION DEPENDS_ON -> [MaintenanceStore:Module]
|
||||
@UX_STATE no_maintenance -> No badge rendered (empty cell).
|
||||
@UX_STATE maintenance_active -> Orange badge "⚠️ Maintenance" with tooltip.
|
||||
@UX_STATE loading -> Grey skeleton badge.
|
||||
@UX_REACTIVITY $derived from MaintenanceStore.dashboardBanners.
|
||||
|
||||
Badge: `rounded-full px-2.5 py-0.5 text-xs font-medium bg-orange-100 text-orange-700` (matches existing badge convention). Tooltip via native `title` attribute (matches codebase-wide pattern). No badge when inactive. Skeleton: `animate-pulse bg-gray-200 h-5 w-24 rounded-full`.
|
||||
|
||||
@UX_STATE no_maintenance/maintenance_active/loading
|
||||
|
||||
@@ -12,11 +12,13 @@
|
||||
│ (single row) │ │ │
|
||||
│ ────────────────── │ │ id: str (PK) │
|
||||
│ target_environment │◄──────│ task_id: str (FK→Task) │
|
||||
│ dashboard_scope │ │ tables: JSON (str[]) │
|
||||
│ excluded_ids: JSON │ │ start_time: datetime │
|
||||
│ forced_ids: JSON │ │ end_time: datetime │
|
||||
│ updated_at │ │ message: str? │
|
||||
└──────────────────────┘ │ status: enum(active|comp|fail)│
|
||||
│ display_timezone │ │ tables: JSON (str[]) │
|
||||
│ dashboard_scope │ │ start_time: datetime │
|
||||
│ excluded_ids: JSON │ │ end_time: datetime │
|
||||
│ forced_ids: JSON │ │ message: str? │
|
||||
│ updated_at │ │ status: enum(pending|applying │
|
||||
└──────────────────────┘ │ |active|partial|ending │
|
||||
│ |completed|failed) │
|
||||
│ created_at / updated_at │
|
||||
└──────────┬────────────────────┘
|
||||
│ 1:N
|
||||
@@ -27,11 +29,28 @@
|
||||
│ id: str (PK) │
|
||||
│ event_id: str (FK→Event) │
|
||||
│ dashboard_id: int │
|
||||
│ banner_id: str (FK→Banner) │
|
||||
│ status: enum(pending_apply │
|
||||
│ |active|removing │
|
||||
│ |removed|apply_failed │
|
||||
│ |removal_failed) │
|
||||
│ created_at / updated_at │
|
||||
└──────────┬──────────────────┘
|
||||
│ N:1
|
||||
▼
|
||||
┌─────────────────────────────┐
|
||||
│ MaintenanceDashboardBanner │
|
||||
│ (ONE chart per dashboard) │
|
||||
│ │
|
||||
│ id: str (PK) │
|
||||
│ environment_id: str │
|
||||
│ dashboard_id: int │
|
||||
│ chart_id: int? (Superset) │
|
||||
│ status: enum(active|removed │
|
||||
│ |removal_failed) │
|
||||
│ created_at / removed_at │
|
||||
│ banner_text: text │
|
||||
│ status: enum(active|removed)│
|
||||
│ UNIQUE (env_id, dash_id, │
|
||||
│ status=active) │
|
||||
│ created_at / updated_at │
|
||||
└─────────────────────────────┘
|
||||
```
|
||||
|
||||
@@ -48,29 +67,33 @@ from src.models.base import Base
|
||||
import enum
|
||||
|
||||
class MaintenanceEventStatus(str, enum.Enum):
|
||||
PENDING = "pending"
|
||||
APPLYING = "applying"
|
||||
ACTIVE = "active"
|
||||
PARTIAL = "partial" # some dashboards succeeded, some failed
|
||||
ENDING = "ending"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
|
||||
class MaintenanceEvent(Base):
|
||||
__tablename__ = "maintenance_events"
|
||||
|
||||
id = Column(String, primary_key=True, index=True) # "m-ev-20260521-abc123"
|
||||
task_id = Column(String, nullable=True, index=True) # FK to tasks.id (loose coupling)
|
||||
tables = Column(JSON, nullable=False) # ["raw.sales", "raw.inventory"]
|
||||
id = Column(String, primary_key=True, index=True)
|
||||
task_id = Column(String, nullable=True, index=True) # Loose reference — no FK to avoid circular dependency
|
||||
environment_id = Column(String, nullable=False) # Target env at event creation (snapshot)
|
||||
tables = Column(JSON, nullable=False)
|
||||
start_time = Column(DateTime(timezone=True), nullable=False)
|
||||
end_time = Column(DateTime(timezone=True), nullable=False)
|
||||
end_time = Column(DateTime(timezone=True), nullable=True) # nullable — "Окончание: уточняется" when absent
|
||||
message = Column(Text, nullable=True)
|
||||
status = Column(
|
||||
Enum(MaintenanceEventStatus),
|
||||
nullable=False,
|
||||
default=MaintenanceEventStatus.ACTIVE,
|
||||
default=MaintenanceEventStatus.PENDING,
|
||||
index=True
|
||||
)
|
||||
created_at = Column(DateTime(timezone=True), server_default="now()")
|
||||
updated_at = Column(DateTime(timezone=True), server_default="now()", onupdate="now()")
|
||||
|
||||
# Relationships
|
||||
dashboard_states = relationship(
|
||||
"MaintenanceDashboardState",
|
||||
back_populates="maintenance_event",
|
||||
@@ -78,34 +101,66 @@ class MaintenanceEvent(Base):
|
||||
)
|
||||
```
|
||||
|
||||
### MaintenanceDashboardBanner
|
||||
|
||||
```python
|
||||
class MaintenanceDashboardBannerStatus(str, enum.Enum):
|
||||
ACTIVE = "active"
|
||||
REMOVED = "removed"
|
||||
|
||||
class MaintenanceDashboardBanner(Base):
|
||||
__tablename__ = "maintenance_dashboard_banners"
|
||||
|
||||
id = Column(String, primary_key=True, index=True)
|
||||
environment_id = Column(String, nullable=False)
|
||||
dashboard_id = Column(Integer, nullable=False, index=True)
|
||||
chart_id = Column(Integer, nullable=True) # Superset markdown chart ID
|
||||
banner_text = Column(Text, nullable=True) # Current aggregated markdown
|
||||
status = Column(
|
||||
Enum(MaintenanceDashboardBannerStatus),
|
||||
nullable=False,
|
||||
default=MaintenanceDashboardBannerStatus.ACTIVE
|
||||
)
|
||||
created_at = Column(DateTime(timezone=True), server_default="now()")
|
||||
updated_at = Column(DateTime(timezone=True), server_default="now()", onupdate="now()")
|
||||
|
||||
dashboard_states = relationship("MaintenanceDashboardState", back_populates="banner")
|
||||
|
||||
__table_args__ = (
|
||||
Index("ix_banner_unique_active", "environment_id", "dashboard_id",
|
||||
unique=True, postgresql_where=(status == "active")),
|
||||
)
|
||||
```
|
||||
|
||||
### MaintenanceDashboardState
|
||||
|
||||
```python
|
||||
class MaintenanceDashboardStateStatus(str, enum.Enum):
|
||||
PENDING_APPLY = "pending_apply"
|
||||
ACTIVE = "active"
|
||||
REMOVING = "removing"
|
||||
REMOVED = "removed"
|
||||
APPLY_FAILED = "apply_failed"
|
||||
REMOVAL_FAILED = "removal_failed"
|
||||
|
||||
class MaintenanceDashboardState(Base):
|
||||
__tablename__ = "maintenance_dashboard_states"
|
||||
|
||||
id = Column(String, primary_key=True, index=True) # UUID
|
||||
id = Column(String, primary_key=True, index=True)
|
||||
event_id = Column(String, ForeignKey("maintenance_events.id"), nullable=False, index=True)
|
||||
dashboard_id = Column(Integer, nullable=False, index=True) # Superset dashboard ID
|
||||
environment_id = Column(String, nullable=False) # Target environment ID
|
||||
chart_id = Column(Integer, nullable=True) # Superset markdown chart ID (null until created)
|
||||
dashboard_id = Column(Integer, nullable=False, index=True)
|
||||
banner_id = Column(String, ForeignKey("maintenance_dashboard_banners.id"), nullable=True)
|
||||
status = Column(
|
||||
Enum(MaintenanceDashboardStateStatus),
|
||||
nullable=False,
|
||||
default=MaintenanceDashboardStateStatus.ACTIVE
|
||||
default=MaintenanceDashboardStateStatus.PENDING_APPLY
|
||||
)
|
||||
created_at = Column(DateTime(timezone=True), server_default="now()")
|
||||
removed_at = Column(DateTime(timezone=True), nullable=True)
|
||||
updated_at = Column(DateTime(timezone=True), server_default="now()", onupdate="now()")
|
||||
|
||||
# Relationships
|
||||
maintenance_event = relationship("MaintenanceEvent", back_populates="dashboard_states")
|
||||
banner = relationship("MaintenanceDashboardBanner", back_populates="dashboard_states")
|
||||
|
||||
# Composite index for dashboard hub queries
|
||||
__table_args__ = (
|
||||
Index("ix_mds_dashboard_status", "dashboard_id", "status"),
|
||||
)
|
||||
@@ -122,8 +177,18 @@ class DashboardScope(str, enum.Enum):
|
||||
class MaintenanceSettings(Base):
|
||||
__tablename__ = "maintenance_settings"
|
||||
|
||||
id = Column(String, primary_key=True, default="default") # Single-row: "default"
|
||||
target_environment_id = Column(String, nullable=False) # Production env ID
|
||||
id = Column(String, primary_key=True, default="default")
|
||||
target_environment_id = Column(String, nullable=False)
|
||||
display_timezone = Column(String, nullable=False, default="UTC")
|
||||
banner_template = Column(Text, nullable=False, default=(
|
||||
'<div style="background:#FFF3E0;padding:16px;border-left:4px solid #FF9800;border-radius:4px">\n\n'
|
||||
'## ⚠️ Технические работы\n\n'
|
||||
'{message}\n\n'
|
||||
'**Начало:** {start_time}\n'
|
||||
'**Конец:** {end_time}\n\n'
|
||||
'*Данные могут быть неполными или временно недоступны.*\n\n'
|
||||
'</div>'
|
||||
))
|
||||
dashboard_scope = Column(
|
||||
Enum(DashboardScope),
|
||||
nullable=False,
|
||||
@@ -132,6 +197,10 @@ class MaintenanceSettings(Base):
|
||||
excluded_dashboard_ids = Column(JSON, nullable=False, default=list)
|
||||
forced_dashboard_ids = Column(JSON, nullable=False, default=list)
|
||||
updated_at = Column(DateTime(timezone=True), server_default="now()", onupdate="now()")
|
||||
|
||||
__table_args__ = (
|
||||
CheckConstraint("id = 'default'", name="ck_settings_singleton"),
|
||||
)
|
||||
```
|
||||
|
||||
## Pydantic Schemas
|
||||
@@ -145,14 +214,16 @@ from pydantic import BaseModel, Field
|
||||
from datetime import datetime
|
||||
|
||||
class MaintenanceStartRequest(BaseModel):
|
||||
tables: list[str] = Field(..., min_length=1, description="Fully-qualified table names (schema.table)")
|
||||
start_time: datetime
|
||||
end_time: datetime
|
||||
tables: list[str] = Field(..., min_length=1)
|
||||
start_time: datetime # required
|
||||
end_time: datetime | None = None # optional — absent → «уточняется»
|
||||
message: str | None = Field(None, max_length=500)
|
||||
|
||||
class MaintenanceSettingsUpdate(BaseModel):
|
||||
target_environment_id: str | None = None
|
||||
dashboard_scope: str | None = None # "published_only" | "draft_only" | "all"
|
||||
display_timezone: str | None = None
|
||||
banner_template: str | None = None
|
||||
dashboard_scope: str | None = None
|
||||
excluded_dashboard_ids: list[int] | None = None
|
||||
forced_dashboard_ids: list[int] | None = None
|
||||
```
|
||||
@@ -199,6 +270,8 @@ class MaintenanceEndAllTaskResult(BaseModel):
|
||||
|
||||
class MaintenanceSettingsResponse(BaseModel):
|
||||
target_environment_id: str
|
||||
display_timezone: str
|
||||
banner_template: str
|
||||
dashboard_scope: str
|
||||
excluded_dashboard_ids: list[int]
|
||||
forced_dashboard_ids: list[int]
|
||||
@@ -240,20 +313,31 @@ class MaintenanceDashboardBannerState(BaseModel):
|
||||
|
||||
### MaintenanceEvent
|
||||
```
|
||||
[created] → ACTIVE ──→ COMPLETED
|
||||
└──→ FAILED
|
||||
[created] → PENDING → APPLYING → ACTIVE (all banners placed)
|
||||
└──→ PARTIAL (some banners failed)
|
||||
└──→ FAILED (all banners failed)
|
||||
ACTIVE ──→ ENDING → COMPLETED
|
||||
PARTIAL ──→ ENDING → COMPLETED
|
||||
```
|
||||
|
||||
### MaintenanceDashboardBanner
|
||||
```
|
||||
[created] → ACTIVE ──→ REMOVED
|
||||
```
|
||||
|
||||
### MaintenanceDashboardState
|
||||
```
|
||||
[created] → ACTIVE ──→ REMOVED
|
||||
└──→ REMOVAL_FAILED
|
||||
[created] → PENDING_APPLY → ACTIVE
|
||||
└──→ APPLY_FAILED
|
||||
ACTIVE ──→ REMOVING → REMOVED
|
||||
└──→ REMOVAL_FAILED
|
||||
```
|
||||
|
||||
### Event Conflict Matrix
|
||||
| Dashboard has | New /start | Behavior |
|
||||
|--------------|-----------|----------|
|
||||
| No banner | tables match | Create MaintenanceDashboardState (ACTIVE) + Create chart |
|
||||
| Banner from M1 (ACTIVE) | tables differ | Create new MaintenanceDashboardState (ACTIVE) + Rebuild banner text from ALL active events |
|
||||
| Banner from M1 (ACTIVE) | same tables (idempotent) | Return existing maintenance_id, no state change |
|
||||
| Banner from M1 (REMOVED) | any tables | M1 already completed, treat as no banner |
|
||||
| Dashboard has | New /start | Idempotency Key | Behavior |
|
||||
|--------------|-----------|-----------------|----------|
|
||||
| No banner | tables match | — | Create `MaintenanceDashboardBanner` + `MaintenanceDashboardState(PENDING_APPLY→ACTIVE)` |
|
||||
| Banner ACTIVE from M1 | tables + (start, end) differ | New event | Create `MaintenanceDashboardState(PENDING_APPLY)` → link to existing Banner → rebuild text |
|
||||
| Banner ACTIVE from M1 | same (tables, start, end) | Duplicate | Return existing `maintenance_id`, `status:"already_active"` |
|
||||
| Banner ACTIVE from M1 | same tables+start, end was None, now set | New event | Different `end_time` value → new event. Old event remains active until explicitly ended. |
|
||||
| Banner REMOVED | any tables | — | M1 already completed, treat as no banner |
|
||||
|
||||
@@ -102,8 +102,8 @@ frontend/
|
||||
|
||||
## Semantic Contract Guidance
|
||||
|
||||
All 14 contracts defined in `contracts/modules.md`:
|
||||
- **Complexity 1** (3): MaintenanceEvent, MaintenanceDashboardState, MaintenanceSettings models, Pydantic schemas
|
||||
All 15 contracts defined in `contracts/modules.md`:
|
||||
- **Complexity 1** (4): MaintenanceEvent, MaintenanceDashboardBanner, MaintenanceDashboardState, MaintenanceSettings models, Pydantic schemas
|
||||
- **Complexity 2** (3): SqlTableExtractor, MaintenanceRouter, MaintenanceApiClient, DashboardMaintenanceBadge
|
||||
- **Complexity 3** (5): MaintenanceRoutes, MaintenanceStore, MaintenanceBannerPage, MaintenanceSettingsPanel, MaintenanceEventsTable
|
||||
- **Complexity 4** (3): SupersetDashboardsWriteMixin, MaintenanceService (start/end/end_all methods)
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.9+ with virtualenv (`backend/.venv/`)
|
||||
- Python 3.13+ with virtualenv (`backend/.venv/`)
|
||||
- Node.js 18+ with npm (`frontend/node_modules/`)
|
||||
- PostgreSQL 16 running (local or Docker)
|
||||
- Superset instance configured in ss-tools environments
|
||||
|
||||
@@ -63,16 +63,16 @@ Follow ADR-0001 canonical layout with these specific placements:
|
||||
## R3: SQL Table Name Extraction for Virtual Datasets
|
||||
|
||||
### Decision
|
||||
**Extract `schema.table` patterns from raw SQL+Jinja text (no pre-stripping), then filter false positives via sqlparse tokenizer.** This is the reverse of the initially planned approach — search globally first, verify locally, rather than strip-then-parse.
|
||||
**Extract `schema.table` patterns from raw SQL+Jinja text using three-phase approach: (1) detect Jinja block spans (`{% %}` and `{{ }}`), (2) in Jinja spans — extract table names from string literals (`"schema.table"`), (3) in SQL spans — regex global extraction + sqlparse filtering to reject string literal false positives.** Jinja is NOT pre-stripped — table names inside `{% set %}` blocks are preserved.
|
||||
|
||||
### Rationale
|
||||
- Real-world Superset virtual datasets embed table names inside Jinja `{% set %}` blocks (e.g., a `selected_dataset` dictionary mapping dataset types to fully-qualified table names). If Jinja is stripped before parsing, these table references are lost.
|
||||
- Example from a production dataset: `{% set selected_dataset = {'Быстро': "dm_view.sales_final", 'Медленно': "dm_view.sales_raw"} %}` — both table names live inside a Jinja block, invisible to any SQL parser after stripping.
|
||||
- **New two-phase approach:**
|
||||
1. **Global extraction**: Regex `[a-zA-Z][\w]*\.[a-zA-Z][\w]*` on raw text captures ALL `schema.table` candidates (both in plain SQL and inside Jinja blocks).
|
||||
2. **False-positive filtering**: sqlparse tokenizes the raw text; each candidate is rejected if its position falls inside a SQL string literal (`'...'`) or comment.
|
||||
- This captures table names from ALL contexts — FROM, JOIN, Jinja `{% set %}`, Jinja string values — while filtering out `'2025.12.31'` (date in string literal) and similar false matches.
|
||||
- `sqlparse` remains the right tool for phase 2 because it classifies tokens as Identifier vs String vs Comment — the only classification needed for filtering.
|
||||
- Real-world Superset virtual datasets embed table names inside Jinja `{% set %}` blocks (e.g., a `selected_dataset` dictionary). If Jinja is stripped before parsing, these are lost.
|
||||
- sqlparse cannot correctly tokenize Jinja code — treating `{% %}` as unknown tokens and potentially misclassifying positions. Therefore Jinja spans are detected first and processed separately.
|
||||
- **Three-phase approach:**
|
||||
1. **Span detection**: Split raw text into Jinja block spans and SQL spans using regex for `{%...%}` and `{{...}}` boundaries.
|
||||
2. **Jinja spans**: Inside Jinja blocks, extract `"schema.table"` from string literal values — these are NOT SQL strings and must not be filtered by sqlparse.
|
||||
3. **SQL spans**: Regex `[a-zA-Z][\w]*\.[a-zA-Z][\w]*` finds candidates; sqlparse tokenizer rejects any whose byte position falls inside a SQL string literal.
|
||||
- This captures tables from ALL contexts — FROM, JOIN, Jinja `{% set %}` values — while filtering out `'2026.04.30'` (date in SQL string literal).
|
||||
|
||||
### Alternatives Considered
|
||||
- **Strip Jinja → parse SQL (original plan)**: Rejected — loses tables embedded in Jinja `{% set %}` blocks (empirically proven with production dataset example).
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
- Q: Which Superset environment should /api/maintenance/start target (ss-tools manages multiple)? → A: System always targets the production environment (pre-configured in ss-tools settings). External tool does NOT pass environment_id.
|
||||
- Q: Should /api/maintenance/start be synchronous (blocking) or asynchronous (task-based)? → A: Async — returns `{task_id, status: "pending"}` immediately; operator polls `GET /api/tasks/{task_id}` or uses WebSocket for progress (matches existing TaskManager pattern).
|
||||
- Q: How should table names from the API be matched against dataset references? → A: Exact `schema.table` match only (e.g., `"raw.sales"` matches only `"raw.sales"`, not unqualified `"sales"`). Case-insensitive.
|
||||
- Q: Should duplicate /start calls (same tables, same times) create a new MaintenanceEvent or be idempotent? → A: Idempotent — detect active event with same tables and return existing maintenance_id with status "already_active".
|
||||
- Q: Should duplicate /start calls (same tables, same times) create a new MaintenanceEvent or be idempotent? → A: Idempotent by `(tables, start_time, end_time)` key. Same tuple = `already_active`. Changed `end_time` (or adding it when previously omitted) = new event. Omitted `end_time` treated as `None` in the key. The `message` field is NOT part of the idempotency key — duplicate calls with a different message return `already_active` and the original message is preserved.
|
||||
- Q: When /end is called on event M1 but dashboard D is also affected by active event M2, should the banner be removed? → A: No — banner stays on D but is rebuilt with updated text (only M2's info). Removal occurs only when the LAST active event affecting D is ended.
|
||||
|
||||
## User Scenarios & Testing *(mandatory)*
|
||||
@@ -39,10 +39,12 @@ ss-tools finds all dashboards in Superset whose datasets reference the given tab
|
||||
|
||||
3. **Given** dashboard D already has a banner from a previous maintenance event, **When** a new `POST /api/maintenance/start` request arrives referencing a different table that also affects D, **Then** the banner is updated (time and text replaced) rather than duplicated.
|
||||
|
||||
4. **Given** table `raw.archived` is not used in any dashboard, **When** external tool passes `{"tables": ["raw.archived"]}`, **Then** API returns `{"affected_dashboards": 0, "message": "No dashboards found referencing tables: raw.archived"}` and no dashboard is modified.
|
||||
4. **Given** table `raw.archived` is not used in any dashboard, **When** external tool passes `{"tables": ["raw.archived"]}`, **Then** API returns `{task_id, maintenance_id, status: "pending"}` immediately. Upon task completion, the task result contains `{status: "no_match", affected_dashboards: 0, unmatched_tables: ["raw.archived"]}`. No dashboard is modified.
|
||||
|
||||
5. **Given** an active MaintenanceEvent E already exists with tables `["raw.sales"]`, **When** external tool calls `/start` again with the same tables, **Then** API returns `{maintenance_id: E.id, status: "already_active"}` without creating a duplicate event.
|
||||
|
||||
6. **Given** external tool does not know the end time, **When** it calls `/start` with `{"tables": ["raw.sales"], "start_time": "2026-05-21T22:00:00Z"}` (no end_time), **Then** API returns `202 {task_id}`, and the banner shows «Окончание: уточняется».
|
||||
|
||||
---
|
||||
|
||||
### User Story 2 — External tool removes maintenance banner upon completion (Priority: P1)
|
||||
@@ -138,15 +140,19 @@ The Dashboard Hub table shows an indicator when a dashboard has an active mainte
|
||||
|
||||
- **Dashboard present in both excluded and forced lists**: Forced takes priority (explicit inclusion overrides exclusion).
|
||||
|
||||
- **Duplicate /start call with identical parameters**: If an active MaintenanceEvent already exists with the same table list, the system returns the existing `maintenance_id` with status `"already_active"` instead of creating a duplicate event. This prevents unnecessary banner churn and Superset API calls.
|
||||
- **Duplicate /start call with identical parameters**: Idempotency key is `(tables, start_time, end_time)`. Same tuple → `"already_active"`. Same tables + start_time but different end_time (or adding end_time when previously omitted) → new event. Omitted `end_time` is `None` in the key.
|
||||
|
||||
- **Message HTML escaping**: The `message` field in the API request is plain text only. All HTML special characters (`<`, `>`, `&`, `"`, `'`) are escaped before rendering in the Superset markdown chart. Raw HTML/CSS injection via the message field is rejected at the API boundary (400).
|
||||
|
||||
- **Time zones**: Maintenance start and end times must be correctly displayed in the banner text respecting timezone (accept ISO 8601 with timezone, display in local timezone or UTC as configured).
|
||||
|
||||
- **Tabbed dashboards**: The banner is added to the root dashboard layout only. If a Superset dashboard uses tabs, the banner may not be visible when viewing individual tabs — this is a known limitation. Future enhancement may add per-tab banner placement.
|
||||
|
||||
## Requirements *(mandatory)*
|
||||
|
||||
### Functional Requirements
|
||||
|
||||
- **FR-001**: System MUST provide API endpoint `POST /api/maintenance/start` accepting: list of table names (`tables: str[]`), start time (`start_time: datetime`), end time (`end_time: datetime`), optional message text (`message: str | None`). Returns immediately with `{task_id, status: "pending"}`; actual banner placement runs asynchronously via TaskManager. The system always targets the production Superset environment (configured in MaintenanceSettings).
|
||||
- **FR-001**: System MUST provide API endpoint `POST /api/maintenance/start` accepting: list of table names (`tables: str[]`, max 100 entries, sorted set for idempotency), start time (`start_time: datetime` — required, in the future ±1h grace period), end time (`end_time: datetime | None` — optional, must be > start_time if provided), optional message text (`message: str | None`). **Always** returns `202 {task_id, maintenance_id, status: "pending"}` immediately after validation and event creation. Sync short-circuits only for `already_active` (idempotency) and validation errors (400). When `end_time` is omitted, the banner displays «Окончание: уточняется».
|
||||
|
||||
- **FR-002**: System MUST, upon receiving a maintenance start request, find all dashboards in the configured production Superset environment whose datasets reference the given tables.
|
||||
|
||||
@@ -156,7 +162,7 @@ The Dashboard Hub table shows an indicator when a dashboard has an active mainte
|
||||
|
||||
- **FR-004**: System MUST add a markdown chart (Superset `viz_type: "markdown"`) at the top of each affected dashboard with maintenance information.
|
||||
|
||||
- **FR-005**: The banner text MUST contain: a warning indicator (⚠️), the fact of ongoing maintenance, start time, planned end time, and (optionally) an additional message.
|
||||
- **FR-005**: The banner text is rendered from `banner_template` (configured in MaintenanceSettings). For a single active event, variables `{start_time}`, `{end_time}`, `{message}` are directly substituted. For multiple active events, aggregated text is built by rendering each event's info as a sub-block (e.g., "Event N: {start} – {end}"), and the combined text is substituted as `{message}`. Variables not present in the template are replaced with an empty string. The `message` field from the API request is plain text — HTML special characters are escaped before substitution.
|
||||
|
||||
- **FR-006**: System MUST provide API endpoint `POST /api/maintenance/{maintenance_id}/end` for removing banners from dashboards affected by a specific maintenance event. Returns `{task_id}` immediately; removal runs asynchronously.
|
||||
|
||||
@@ -166,9 +172,13 @@ The Dashboard Hub table shows an indicator when a dashboard has an active mainte
|
||||
|
||||
- **FR-009**: System MUST support maintenance mode settings via `GET/PUT /api/maintenance/settings`:
|
||||
- `target_environment_id`: `str` — the production Superset environment to scan/modify
|
||||
- `display_timezone`: `str` — timezone for banner text display (default: `"UTC"`)
|
||||
- `banner_template`: `str` — markdown template with optional variables `{start_time}`, `{end_time}`, `{message}`. Variables are substituted at render time; missing variables are omitted. Default template includes all three with ⚠️ header and styling (see UX reference).
|
||||
- `dashboard_scope`: `"published_only"` | `"draft_only"` | `"all"` (default: `"published_only"`)
|
||||
- `excluded_dashboard_ids`: `int[]` — dashboards NEVER receiving banners
|
||||
- `forced_dashboard_ids`: `int[]` — dashboards ALWAYS receiving banners (overrides excluded)
|
||||
- `forced_dashboard_ids`: `int[]` — dashboards ALWAYS receiving banners (overrides excluded per FR-009-b)
|
||||
|
||||
- **FR-009-b**: When a dashboard is present in BOTH `forced_dashboard_ids` and `excluded_dashboard_ids`, forced inclusion takes priority — the banner IS placed.
|
||||
|
||||
- **FR-010**: System MUST provide a UI management page for maintenance mode with:
|
||||
- Table of active and completed maintenance events
|
||||
@@ -183,13 +193,32 @@ The Dashboard Hub table shows an indicator when a dashboard has an active mainte
|
||||
|
||||
- **FR-014**: All dashboard-modifying operations via Superset API MUST require the `operator` or `admin` role (per RBAC ADR-0005).
|
||||
|
||||
- **FR-015**: System MUST enforce RBAC per this matrix:
|
||||
|
||||
| Endpoint / UI | viewer | operator | admin |
|
||||
|---|---|---|---|
|
||||
| `GET /api/maintenance/dashboard-banners` | ✅ | ✅ | ✅ |
|
||||
| `GET /api/maintenance/events` | ❌ | ✅ | ✅ |
|
||||
| `POST /api/maintenance/start` | ❌ | ✅ | ✅ |
|
||||
| `POST /api/maintenance/{id}/end` | ❌ | ✅ | ✅ |
|
||||
| `POST /api/maintenance/end-all` | ❌ | ✅ | ✅ |
|
||||
| `GET /api/maintenance/settings` | ❌ | ✅ | ✅ |
|
||||
| `PUT /api/maintenance/settings` | ❌ | ❌ | ✅ |
|
||||
| `/maintenance` UI page | ❌ | ✅ (read-only) | ✅ |
|
||||
|
||||
- **FR-016**: System MUST provide `GET /api/maintenance/events` returning active and completed event lists with affected dashboard counts.
|
||||
|
||||
- **FR-017**: System MUST provide `GET /api/maintenance/dashboard-banners` returning per-dashboard banner state for the Dashboard Hub indicator.
|
||||
|
||||
### Key Entities
|
||||
|
||||
- **MaintenanceEvent**: A record of a maintenance event. Contains: id, task_id (links to TaskManager task), table list, start time, end time, optional message, affected dashboard IDs, status (active/completed/failed), created_at, updated_at.
|
||||
|
||||
- **MaintenanceDashboardState**: The banner state for a specific dashboard. Contains: maintenance_event_id, dashboard_id, environment_id, chart_id (Superset markdown chart ID), status (active/removed/removal_failed), created_at, removed_at.
|
||||
- **MaintenanceDashboardBanner** (NEW): The canonical "one chart per dashboard" entity. Unique constraint on `(environment_id, dashboard_id, status='active')`. Contains: id, environment_id, dashboard_id, chart_id (Superset markdown chart ID), banner_text (current aggregated markdown), updated_at. Only ONE active banner chart exists per dashboard regardless of how many MaintenanceEvents affect it.
|
||||
|
||||
- **MaintenanceSettings**: Maintenance mode configuration. Contains: target_environment_id (str — the production Superset environment), dashboard_scope (enum), excluded_dashboard_ids (int[]), forced_dashboard_ids (int[]), updated_at.
|
||||
- **MaintenanceDashboardState**: Links a maintenance event to a dashboard. Contains: id, maintenance_event_id, dashboard_id, banner_id (FK → MaintenanceDashboardBanner), status (pending_apply/active/removing/removed/apply_failed/removal_failed), created_at, updated_at. Does NOT own chart_id directly — references the shared MaintenanceDashboardBanner.
|
||||
|
||||
- **MaintenanceSettings**: Maintenance mode configuration. Contains: id, target_environment_id (str), display_timezone (str, default "UTC"), banner_template (str — markdown with optional `{start_time}`, `{end_time}`, `{message}` variables; default has all three), dashboard_scope (enum), excluded_dashboard_ids (int[]), forced_dashboard_ids (int[]), updated_at.
|
||||
|
||||
- **Dashboard** (existing Superset entity): Used for the dashboard → datasets → tables relationship. Has `published`/`draft` status.
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@
|
||||
|
||||
- [ ] T001 Create `backend/src/api/routes/maintenance/` package directory structure (`__init__.py`)
|
||||
- [ ] T002 [P] Create `backend/tests/` test file stubs for maintenance feature: `test_maintenance_service.py`, `test_sql_table_extractor.py`, `test_maintenance_api.py`
|
||||
- [ ] T003 [P] Create frontend directory structure: `frontend/src/routes/maintenance/`, `frontend/src/lib/components/MaintenanceSettingsPanel.svelte`, `frontend/src/lib/components/MaintenanceEventsTable.svelte`, `frontend/src/lib/components/DashboardMaintenanceBadge.svelte` (empty placeholder files)
|
||||
- [ ] T003 [P] Create frontend directory structure: `frontend/src/routes/maintenance/`, `frontend/src/lib/stores/maintenance.svelte.js`, `frontend/src/lib/api/maintenance.js` (empty placeholder files). Note: UI components reuse existing primitives — no new component scaffolds needed.
|
||||
- [ ] T004 Generate Alembic migration for new tables (`maintenance_events`, `maintenance_dashboard_states`, `maintenance_settings`) via `alembic revision --autogenerate -m "add maintenance banner tables"` after models are defined in Phase 2
|
||||
|
||||
---
|
||||
@@ -34,9 +34,10 @@
|
||||
|
||||
### Database Models
|
||||
|
||||
- [ ] T005 [P] [--] Implement `MaintenanceEvent` SQLAlchemy model in `backend/src/models/maintenance.py` with columns (id, task_id, tables JSON, start_time, end_time, message, status enum, timestamps) and relationship to MaintenanceDashboardState. Contract: `[DEF:MaintenanceEvent:Model]` C1.
|
||||
- [ ] T006 [P] [--] Implement `MaintenanceDashboardState` SQLAlchemy model in `backend/src/models/maintenance.py` with columns (id, event_id FK, dashboard_id, environment_id, chart_id, status enum, created_at, removed_at) and composite index on (dashboard_id, status). Contract: `[DEF:MaintenanceDashboardState:Model]` C1.
|
||||
- [ ] T007 [P] [--] Implement `MaintenanceSettings` SQLAlchemy model in `backend/src/models/maintenance.py` — single-row table (id="default") with columns (target_environment_id, dashboard_scope enum, excluded_dashboard_ids JSON, forced_dashboard_ids JSON, updated_at). Contract: `[DEF:MaintenanceSettings:Model]` C1.
|
||||
- [ ] T005 [P] [--] Implement `MaintenanceEvent` SQLAlchemy model in `backend/src/models/maintenance.py` with columns (id, task_id — loose reference, environment_id — snapshot of target env, tables JSON, start_time, end_time nullable, message, status enum: pending/applying/active/partial/ending/completed/failed, timestamps) and relationship to MaintenanceDashboardState. Contract: `[DEF:MaintenanceEvent:Model]` C1.
|
||||
- [ ] T005a [P] [--] Implement `MaintenanceDashboardBanner` SQLAlchemy model in `backend/src/models/maintenance.py` — canonical "one chart per dashboard" entity with columns (id, environment_id, dashboard_id, chart_id, banner_text, status enum: active/removed, timestamps) and unique partial index on (environment_id, dashboard_id) WHERE status='active'. Contract: `[DEF:MaintenanceDashboardBanner:Model]` C1.
|
||||
- [ ] T006 [P] [--] Implement `MaintenanceDashboardState` SQLAlchemy model in `backend/src/models/maintenance.py` with columns (id, event_id FK, dashboard_id, banner_id FK→maintenance_dashboard_banners, status enum: pending_apply/active/removing/removed/apply_failed/removal_failed, timestamps) and composite index on (dashboard_id, status). Contract: `[DEF:MaintenanceDashboardState:Model]` C1.
|
||||
- [ ] T007 [P] [--] Implement `MaintenanceSettings` SQLAlchemy model in `backend/src/models/maintenance.py` — single-row table (id="default") with CheckConstraint(id='default'), columns (target_environment_id, display_timezone default "UTC", banner_template text, dashboard_scope enum, excluded_dashboard_ids JSON, forced_dashboard_ids JSON, updated_at). Contract: `[DEF:MaintenanceSettings:Model]` C1.
|
||||
|
||||
### Pydantic Schemas
|
||||
|
||||
@@ -44,12 +45,12 @@
|
||||
|
||||
### SupersetClient Extension (C4 — REQUIRES belief runtime)
|
||||
|
||||
- [ ] T009 [--] Implement `SupersetDashboardsWriteMixin` in `backend/src/core/superset_client/_dashboards_write.py` with methods: `create_markdown_chart(dashboard_id, markdown_text) → chart_id` (POST /api/v1/chart/ with viz_type="markdown"), `update_dashboard_layout(dashboard_id, chart_id, position)` (PUT /api/v1/dashboard/{id} with patched position_json), `delete_chart(chart_id)` (DELETE /api/v1/chart/{id}), `get_dashboard_layout(dashboard_id)` (GET /api/v1/dashboard/{id} for position_json). Contract: `[DEF:SupersetDashboardsWriteMixin:Class]` C4. (RATIONALE: direct chart API faster than export-import for single-chart ops per research.md R1; REJECTED: export→modify YAML→re-import cycle — too slow for 50+ dashboards)
|
||||
- [ ] T009 [--] Implement `SupersetDashboardsWriteMixin` in `backend/src/core/superset_client/_dashboards_write.py` with methods: `create_markdown_chart(dashboard_id, markdown_text) → chart_id` (POST /api/v1/chart/ with viz_type="markdown"), `update_markdown_chart(chart_id, markdown_text)` (PUT /api/v1/chart/{id} to update markdown params), `update_dashboard_layout(dashboard_id, chart_id)` (insert at position (0,0), full width 12 cols, shift existing components down), `remove_chart_from_layout(dashboard_id, chart_id)` (remove from position_json), `delete_chart(chart_id)` (DELETE /api/v1/chart/{id}), `get_dashboard_layout(dashboard_id)` (GET /api/v1/dashboard/{id} for position_json). Contract: `[DEF:SupersetDashboardsWriteMixin:Class]` C4. (RATIONALE: chart_id tracked in MaintenanceDashboardBanner DB table — no need to scan position_json)
|
||||
- [ ] T010 [--] Instrument `SupersetDashboardsWriteMixin` with belief runtime markers: `belief_scope` context manager wrapping each Superset API call, `log(REASON)` before POST/PUT/DELETE, `log(REFLECT)` on success, `log(EXPLORE)` on HTTP errors.
|
||||
|
||||
### SQL Table Extractor (C2)
|
||||
|
||||
- [ ] T011 [--] Implement `SqlTableExtractor` module in `backend/src/services/sql_table_extractor.py` with two-phase extraction: `extract_table_candidates(raw_sql)` using regex `[a-zA-Z][\w]*\.[a-zA-Z][\w]*` on raw SQL+Jinja text, and `filter_string_literals(raw_sql, candidates)` using sqlparse tokenizer to reject candidates inside SQL string literals/comments. Orchestrator function: `extract_tables_from_sql(raw_sql) → set[str]`. Contract: `[DEF:SqlTableExtractor:Module]` C2. (RATIONALE: table names in Jinja {% set %} blocks survive global extraction; REJECTED: strip-Jinja-first approach loses 60% of table references per production dataset analysis)
|
||||
- [ ] T011 [--] Implement `SqlTableExtractor` module in `backend/src/services/sql_table_extractor.py` with two-phase extraction: (1) determine Jinja block spans vs SQL spans, (2) in Jinja spans — extract `"schema.table"` from string values, (3) in SQL spans — regex `[a-zA-Z][\w]*\.[a-zA-Z][\w]*` + sqlparse filtering to reject SQL string literal/comment positions. Orchestrator: `extract_tables_from_sql(raw_sql) → set[str]`. Contract: `[DEF:SqlTableExtractor:Module]` C2. (RATIONALE: Jinja string values in {% set %} blocks must NOT be filtered as SQL strings; REJECTED: sqlparse-only approach loses Jinja-embedded tables)
|
||||
|
||||
### Router & App Registration
|
||||
|
||||
@@ -73,17 +74,18 @@
|
||||
|
||||
### Tests for US1 (Write FIRST, ensure they FAIL)
|
||||
|
||||
- [ ] T016 [P] [US1] Write contract test for `POST /api/maintenance/start` in `backend/tests/test_maintenance_api.py` — test valid request returns `{task_id, status: "pending"}`, test duplicate request returns `{status: "already_active"}`, test no-match returns `{status: "no_match"}`, test RBAC returns 403 for viewer role.
|
||||
- [ ] T016 [P] [US1] Write contract test for `POST /api/maintenance/start` in `backend/tests/test_maintenance_api.py` — test valid request returns `202`, test missing `start_time` returns 422, test `end_time < start_time` returns 422, test past `start_time` >1h returns 422, test `tables` >100 entries returns 422, test optional `end_time` (omitted → task result shows «уточняется»), test duplicate request returns `200 {status:"already_active"}`, test no-match in task result, test RBAC returns 403, test HTML injection in message returns 400.
|
||||
- [ ] T017 [P] [US1] Write unit test for `SqlTableExtractor.extract_tables_from_sql()` in `backend/tests/test_sql_table_extractor.py` — test table-based matching, virtual-SQL extraction with Jinja `{% set %}`, string literal false-positive rejection, case-insensitive matching.
|
||||
- [ ] T018 [P] [US1] Write integration test for `MaintenanceService.start_maintenance()` in `backend/tests/test_maintenance_service.py` — test dashboard discovery, markdown chart creation, MaintenanceEvent persistence, MaintenanceDashboardState creation, idempotency check.
|
||||
|
||||
### Implementation for US1
|
||||
|
||||
- [ ] T019 [US1] Implement `find_affected_dashboards()` method in `backend/src/services/maintenance_service.py` — fetches all datasets from Superset, matches table-based datasets via (schema, table_name), matches virtual datasets via SqlTableExtractor, applies scope/excluded/forced filtering from MaintenanceSettings, returns deduplicated dashboard list. Contract C3 within `[DEF:MaintenanceService:Class]`.
|
||||
- [ ] T020 [US1] Implement `build_banner_text()` method in `backend/src/services/maintenance_service.py` — aggregates multiple active events into a single markdown string with ⚠️ header, start/end times per event, and optional message. Handles single-event (simple text) and multi-event (aggregated text) cases. Contract C2 within `[DEF:MaintenanceService:Class]`.
|
||||
- [ ] T021 [US1] Implement `start_maintenance()` method in `backend/src/services/maintenance_service.py` — orchestrates full flow: (1) idempotency check (reuse active event with same tables), (2) `find_affected_dashboards()`, (3) create MaintenanceEvent record, (4) for each dashboard: create markdown chart via SupersetDashboardsWriteMixin, create MaintenanceDashboardState, patch dashboard layout, handle partial failures with 207 response, (5) if dashboard has existing banner from other events → rebuild aggregated text. Contract C4 within `[DEF:MaintenanceService:Class]`. (RATIONALE: single orchestrator ensures consistency between local DB state and Superset chart state)
|
||||
- [ ] T019a [US1] Implement `ensure_banner_chart()` method in `backend/src/services/maintenance_service.py` — idempotently gets or creates the shared `MaintenanceDashboardBanner` for a dashboard. If banner exists (ACTIVE in DB) → return it. Otherwise → `create_markdown_chart()` + `update_dashboard_layout()`, store chart_id in banner row. Contract C3. (RATIONALE: DB is source of truth for chart existence — no position_json scanning needed; REJECTED: per-event chart creation)
|
||||
- [ ] T020 [US1] Implement `build_banner_text()` method in `backend/src/services/maintenance_service.py` — reads `banner_template` and `display_timezone` from MaintenanceSettings, substitutes `{start_time}`, `{end_time}`, `{message}` variables. Missing variables in template are simply omitted. Single-event: direct substitution. Multi-event: aggregates per-event start/end/message, then substitutes into template. Handles absent `end_time` → «Окончание: уточняется». Contract C2.
|
||||
- [ ] T021 [US1] Implement `start_maintenance()` method in `backend/src/services/maintenance_service.py` — orchestrates full flow: (1) idempotency check by `(tables, start_time, end_time)` key using row-level lock (`SELECT ... FOR UPDATE` on maintenance_events to prevent race condition from concurrent identical /start calls), (2) create MaintenanceEvent (status=PENDING→APPLYING), (3) `find_affected_dashboards()`, (4) for each dashboard: `ensure_banner_chart()` → create MaintenanceDashboardState (PENDING_APPLY→ACTIVE or APPLY_FAILED), rebuild aggregated banner text via `build_banner_text()` + `update_markdown_chart()`, (5) transition event to ACTIVE (all success) or PARTIAL (some failed). Contract C4.
|
||||
- [ ] T022 [US1] Instrument `start_maintenance()` with belief runtime: `belief_scope("start_maintenance")`, `log(REASON)` at entry/checkpoint, `log(REFLECT)` on completion, `log(EXPLORE)` on Superset errors.
|
||||
- [ ] T023 [US1] Implement `POST /api/maintenance/start` route handler in `backend/src/api/routes/maintenance/_routes.py` — validates MaintenanceStartRequest, dispatches task via TaskManager.create_task() with type="maintenance_banner_apply", returns MaintenanceStartResponse with task_id immediately. RBAC: `Depends(require_role("operator"))`. Handles no-match fast-path (returns synchronously without task).
|
||||
- [ ] T023 [US1] Implement `POST /api/maintenance/start` route handler in `backend/src/api/routes/maintenance/_routes.py` — validates MaintenanceStartRequest: `start_time` required, `end_time` optional (absent → banner text shows «Окончание: уточняется»), message field sanitized (reject raw HTML, escape special chars). Idempotency check by `(tables, start_time, end_time)` key. Otherwise: create MaintenanceEvent, dispatch task via TaskManager, return `202 {task_id, maintenance_id, status:"pending"}`. RBAC: `Depends(require_role("operator"))`.
|
||||
- [ ] T024 [US1] Run backend verification: `cd backend && source .venv/bin/activate && python -m pytest backend/tests/test_maintenance_api.py backend/tests/test_maintenance_service.py backend/tests/test_sql_table_extractor.py -v`
|
||||
|
||||
**Checkpoint**: `POST /api/maintenance/start` fully functional. Banners placed on affected dashboards. Idempotency working.
|
||||
@@ -104,8 +106,8 @@
|
||||
|
||||
### Implementation for US2
|
||||
|
||||
- [ ] T027 [US2] Implement `rebuild_banner()` method in `backend/src/services/maintenance_service.py` — for a given dashboard, queries all active MaintenanceDashboardState records, calls `build_banner_text()` for aggregated text, updates the existing markdown chart in Superset via `update_markdown_chart()`. Contract C3 within `[DEF:MaintenanceService:Class]`.
|
||||
- [ ] T028 [US2] Implement `end_maintenance()` method in `backend/src/services/maintenance_service.py` — for each dashboard in the event: (1) check if OTHER active events still affect this dashboard; (2) if yes → call `rebuild_banner()` (remove this event's info from text); (3) if no → delete chart via SupersetDashboardsWriteMixin.delete_chart(), update MaintenanceDashboardState to REMOVED; (4) transition MaintenanceEvent to COMPLETED. Contract C4. (RATIONALE: conditional removal prevents banner flicker when multiple events overlap; REJECTED: unconditional removal that briefly deletes then re-creates the banner)
|
||||
- [ ] T027 [US2] Implement `rebuild_banner()` method in `backend/src/services/maintenance_service.py` — for a given `MaintenanceDashboardBanner`, queries all active `MaintenanceDashboardState` records linked to it, calls `build_banner_text()` for aggregated text, updates the shared markdown chart in Superset via `SupersetDashboardsWriteMixin.update_markdown_chart()`. Contract C3 within `[DEF:MaintenanceService:Class]`.
|
||||
- [ ] T028 [US2] Implement `end_maintenance()` method in `backend/src/services/maintenance_service.py` — set event status to ENDING, then for each active `MaintenanceDashboardState` in the event: (1) check if OTHER active event-states still reference the same banner; (2) if yes → call `rebuild_banner()` (remove this event's info from aggregated text), transition state to REMOVED; (3) if no → transition banner to REMOVED, call `remove_chart_from_layout()` then `delete_chart()`, transition state to REMOVING→REMOVED (or REMOVAL_FAILED on error); (4) transition event to COMPLETED. Contract C4. (RATIONALE: conditional removal prevents banner flicker; REJECTED: unconditional removal)
|
||||
- [ ] T029 [US2] Instrument `end_maintenance()` with belief runtime markers.
|
||||
- [ ] T030 [US2] Implement `end_all_maintenance()` method in `backend/src/services/maintenance_service.py` — iterates all active events, calls `end_maintenance()` for each. Contract C4.
|
||||
- [ ] T031 [US2] Implement `POST /api/maintenance/{maintenance_id}/end` and `POST /api/maintenance/end-all` route handlers in `backend/src/api/routes/maintenance/_routes.py` — dispatch tasks via TaskManager, return task_id immediately. RBAC: `Depends(require_role("operator"))`.
|
||||
@@ -127,9 +129,9 @@
|
||||
|
||||
### Implementation for US4
|
||||
|
||||
- [ ] T034 [US4] Implement `GET /api/maintenance/settings` and `PUT /api/maintenance/settings` route handlers in `backend/src/api/routes/maintenance/_routes.py`. GET: read MaintenanceSettings row. PUT: validate and update, return updated settings. RBAC: GET = authenticated, PUT = `Depends(require_role("admin"))`.
|
||||
- [ ] T034 [US4] Implement `GET /api/maintenance/settings` and `PUT /api/maintenance/settings` route handlers in `backend/src/api/routes/maintenance/_routes.py`. GET: read MaintenanceSettings row. PUT: validate and update fields (target_environment_id, display_timezone, banner_template, dashboard_scope, excluded, forced). Return updated settings. RBAC: GET = `operator+`, PUT = `admin`.
|
||||
- [ ] T035 [P] [US4] Implement `MaintenanceApiClient` module in `frontend/src/lib/api/maintenance.js` — typed fetch wrappers: `getSettings()`, `updateSettings(payload)`, `getEvents()`, `startMaintenance(payload)`, `endMaintenance(id)`, `endAllMaintenance()`, `getDashboardBanners()`. Uses `requestApi` from existing API layer. Contract: `[DEF:MaintenanceApiClient:Module]` C2.
|
||||
- [ ] T036 [US4] Implement `MaintenanceSettingsPanel` component in `frontend/src/lib/components/MaintenanceSettingsPanel.svelte` — collapsible form with: target environment dropdown (from existing environments config), dashboard scope radio group (Published/Drafts/All), excluded dashboards multi-select (search + chip badges), forced dashboards multi-select. Save button with loading state. UX states: idle/saving/saved/error. Toast feedback on save. Contract: `[DEF:MaintenanceSettingsPanel:Component]` C3.
|
||||
- [ ] T036 [US4] Implement `MaintenanceSettingsPanel` component in `frontend/src/lib/components/MaintenanceSettingsPanel.svelte` — collapsible via `<details><summary>`. Form uses: `<Select>` from `$lib/ui/Select.svelte` for target environment, `<Input>` from `$lib/ui/Input.svelte` for timezone, native `<input type="radio" bind:group>` for scope (matches ScheduleConfig pattern), `<SearchableMultiSelect>` from `$lib/components/ui/SearchableMultiSelect.svelte` for excluded/forced dashboards (options: `{id, name}` from dashboard list), `<textarea>` for banner_template with live preview, `<Button isLoading={isSaving}>` from `$lib/ui/Button.svelte` for save. Toast via `addToast()` from `$lib/toasts.js`. UX states: idle/saving/saved/error. Contract C3.
|
||||
- [ ] T037 [US4] Run verification: `cd backend && source .venv/bin/activate && python -m pytest backend/tests/test_maintenance_api.py -v -k "settings"` + `cd frontend && npx vitest run tests/maintenance.test.ts -v -k "settings"`
|
||||
|
||||
**Checkpoint**: Settings API operational. Settings panel renders in UI.
|
||||
@@ -151,9 +153,9 @@
|
||||
|
||||
- [ ] T040 [US3] Implement `GET /api/maintenance/events` route handler in `backend/src/api/routes/maintenance/_routes.py` — returns MaintenanceEventListResponse with active and completed events, each including affected dashboard count. RBAC: authenticated.
|
||||
- [ ] T041 [US3] Implement `MaintenanceStore` in `frontend/src/lib/stores/maintenance.svelte.js` — Svelte 5 rune store with `$state` for events (active + completed), settings, dashboardBanners, isLoading, error. Uses `$effect(() => subscribe(...))` pattern per ADR-0007 (REJECTED: fromStore + $derived). Auto-polls `getEvents()` every 30s via `$effect` (WebSocket covers task progress; polling ensures event-list consistency after task completion). Contract: `[DEF:MaintenanceStore:Module]` C3.
|
||||
- [ ] T042 [US3] Implement `MaintenanceEventsTable` component in `frontend/src/lib/components/MaintenanceEventsTable.svelte` — renders table with columns: Event ID, Tables (truncated with tooltip), Affected Dashboards, Start/End Times, Status badge, Actions. "Remove Banners" button per active event. "Remove All Banners" button above table with confirmation dialog. Completed events in collapsible section. UX states: idle/removing/error. Toast feedback. Contract: `[DEF:MaintenanceEventsTable:Component]` C3.
|
||||
- [ ] T043 [US3] Implement `MaintenanceBannerPage` in `frontend/src/routes/maintenance/+page.svelte` — composes MaintenanceSettingsPanel + MaintenanceEventsTable, orchestrates loading states, passes store data to children. UX states: idle/loading/action_in_progress/error/empty. Contract: `[DEF:MaintenanceBannerPage:Component]` C3.
|
||||
- [ ] T044 [US3] Add navigation entry for `/maintenance` in sidebar — add category to "Operations" section in `frontend/src/lib/components/layout/sidebarNavigation.js` with id="maintenance", path="/maintenance", requiredPermission="plugin:migration", requiredAction="WRITE". Add translation keys to `frontend/src/lib/i18n/locales/en/nav.json` and `frontend/src/lib/i18n/locales/ru/nav.json`.
|
||||
- [ ] T042 [US3] Implement `MaintenanceEventsTable` component in `frontend/src/lib/components/MaintenanceEventsTable.svelte` — inline `<table class="min-w-full divide-y divide-gray-200">` (matches health page pattern). Columns: Event ID, Tables (truncated with native `title` tooltip), Affected Dashboards, Start/End Times, Status (inline badge: `rounded-full px-2.5 py-0.5 text-xs font-medium`), Actions (`<Button>` from `$lib/ui/Button.svelte`). Per-event "Remove" button with `isLoading`. "Remove All" with `confirm()` (matches health page). Skeleton: `animate-pulse bg-gray-200 h-4` rows. Empty: `border-dashed border-gray-300 bg-gray-50 p-8 text-center`. Completed events collapsible via `<details><summary>`. Toast via `addToast()`. Contract C3.
|
||||
- [ ] T043 [US3] Implement `MaintenanceBannerPage` in `frontend/src/routes/maintenance/+page.svelte` — composes `MaintenanceSettingsPanel` + `MaintenanceEventsTable`, passes store data. Uses `addToast()` from `$lib/toasts.js` for feedback (Toast.svelte already mounted in root layout). UX states: idle/loading/action_in_progress/error/empty. Contract C3.
|
||||
- [ ] T044 [US3] Add navigation entry for `/maintenance` in sidebar — add category to "Operations" section in `frontend/src/lib/components/layout/sidebarNavigation.js` with id="maintenance", path="/maintenance", requiredPermission="maintenance:write", requiredAction="READ" (operator+ see page, admin has full access per RBAC matrix FR-015). Add translation keys to `frontend/src/lib/i18n/locales/en/nav.json` and `frontend/src/lib/i18n/locales/ru/nav.json`.
|
||||
- [ ] T045 [US3] Run full frontend verification: `cd frontend && npx vitest run tests/maintenance.test.ts tests/maintenance-store.test.ts -v` + `cd frontend && npm run build`
|
||||
|
||||
**Checkpoint**: Admin management page fully functional. Settings + events tables operational.
|
||||
@@ -174,7 +176,7 @@
|
||||
|
||||
- [ ] T047 [US5] Implement `GET /api/maintenance/dashboard-banners` route handler in `backend/src/api/routes/maintenance/_routes.py` — returns list of MaintenanceDashboardBannerState: {dashboard_id, active: bool, events: [...], start_time, end_time}. Queries MaintenanceDashboardState where status=ACTIVE. RBAC: authenticated.
|
||||
- [ ] T048 [US5] Add `getDashboardBanners()` to MaintenanceStore — fetches banner states and populates `dashboardBanners` $state map.
|
||||
- [ ] T049 [US5] Implement `DashboardMaintenanceBadge` component in `frontend/src/lib/components/DashboardMaintenanceBadge.svelte` — orange badge "⚠️ Maintenance" with tooltip showing start/end times. Renders nothing when no active banner. Grey skeleton when loading. Contract: `[DEF:DashboardGridBannerIndicator:Component]` C2.
|
||||
- [ ] T049 [US5] Implement `DashboardMaintenanceBadge` component in `frontend/src/lib/components/DashboardMaintenanceBadge.svelte` — orange badge `rounded-full px-2.5 py-0.5 text-xs font-medium bg-orange-100 text-orange-700 border border-orange-200` (matches existing DashboardHub badge convention). Native `title` attribute for tooltip (codebase-wide pattern). Renders nothing when no active banner. Loading: `animate-pulse bg-gray-200 h-5 w-24 rounded-full`. Contract C2.
|
||||
- [ ] T050 [US5] Modify `frontend/src/routes/dashboards/+page.svelte` — add `DashboardMaintenanceBadge` import, insert badge column/cell in dashboard table rows, wire `$derived` from MaintenanceStore.dashboardBanners. Ensure compliance with ADR-0007 (no fromStore + $derived pattern). Export visual verification: open Dashboard Hub page after US1 and confirm badge appears via `chrome-devtools` snapshot.
|
||||
- [ ] T051 [US5] Run verification: `cd frontend && npx vitest run tests/maintenance.test.ts -v -k "badge"` + `cd frontend && npm run build`
|
||||
|
||||
@@ -192,7 +194,7 @@
|
||||
- [ ] T055 [P] Run frontend full test suite: `cd frontend && npm run test`
|
||||
- [ ] T056 [P] Run frontend build check: `cd frontend && npm run build`
|
||||
- [ ] T057 Run semantic contract audit: `axiom_semantic_validation audit_contracts` on `backend/src/api/routes/maintenance/`, `backend/src/services/maintenance_service.py`, `backend/src/services/sql_table_extractor.py`, `backend/src/core/superset_client/_dashboards_write.py`, `backend/src/models/maintenance.py` — verify no missing C4 tags, no orphan relations, all `@RATIONALE`/`@REJECTED` present.
|
||||
- [ ] T058 Run ADR rejected-path regression check: verify that `fromStore + $derived` is NOT used in any new frontend file (ADR-0007), verify all Svelte components use runes syntax (ADR-0006), verify no export-import cycle is used for chart placement (research.md R1 rejected).
|
||||
- [ ] T058 Run ADR rejected-path regression check: verify `fromStore + $derived` NOT used (ADR-0007), Svelte 5 runes syntax only (ADR-0006), no export-import cycle for chart placement (research.md R1 rejected), no per-event chart creation (one-chart-per-dashboard invariant via MaintenanceDashboardBanner unique index), no `plugin:migration` permission in maintenance code (FR-015 RBAC matrix).
|
||||
- [ ] T059 Run belief runtime audit: `axiom_semantic_validation audit_belief_runtime` on C4 contracts — verify `belief_scope`, `log(REASON)`, `log(REFLECT)`, `log(EXPLORE)` instrumentation present in `start_maintenance()`, `end_maintenance()`, and `SupersetDashboardsWriteMixin`.
|
||||
- [ ] T060 Run quickstart.md validation: follow all steps in `specs/031-maintenance-banner/quickstart.md` (backend/frontend/Docker) and confirm no failures.
|
||||
- [ ] T061 [P] Update `frontend/src/lib/i18n/` with new translation keys for maintenance banner UI strings (English + Russian placeholders): `maintenance.title`, `maintenance.active`, `maintenance.completed`, `maintenance.removeBanners`, `maintenance.removeAll`, `maintenance.settings`, `maintenance.scope`, `maintenance.excluded`, `maintenance.forced`, `maintenance.save`, `maintenance.noActiveEvents`, `maintenance.tooltip`.
|
||||
@@ -250,19 +252,20 @@ Each story is an independently shippable increment. P1 stories (US1+US2) form th
|
||||
|----------------|---------------|-----------|
|
||||
| ADR-0001 (Module Layout) | T001, T005-T007, T019-T023 | All files in canonical paths |
|
||||
| ADR-0003 (Orchestrator) | T009 | Superset REST API only, no DB manipulation |
|
||||
| ADR-0005 (RBAC) | T013, T023, T031, T034 | `require_role("operator"/"admin")` on all endpoints |
|
||||
| ADR-0005 (RBAC) | T013, T023, T031, T034, T044, FR-015 | RBAC matrix enforced; dedicated `maintenance:write` permission, not `plugin:migration` |
|
||||
| ADR-0006 (Svelte 5 Runes) | T041, T049 | `$state`, `$derived`, `$effect` only; no legacy `$:` or `writable` |
|
||||
| ADR-0007 (fromStore REJECTED) | T041 | Store uses `$effect(() => subscribe(...))`; task T058 verifies |
|
||||
| Research R1 (Direct API) | T009 | `POST /chart/` not export-import; task T058 verifies |
|
||||
| Research R3 (Global Extraction) | T011 | No Jinja pre-stripping; two-phase extraction |
|
||||
| Research R1 (Direct API) | T009 | `POST /chart/` + `update_markdown_chart` not export-import; T058 verifies |
|
||||
| Research R3 (Global + Jinja spans) | T011 | Jinja block spans preserved, not stripped; sqlparse only for SQL spans |
|
||||
| Research R4 (TaskManager) | T023, T031 | `TaskManager.create_task()` for async execution |
|
||||
| Research R5 (Dedicated Tables) | T005-T007 | Not AppConfigRecord JSON |
|
||||
| Research R5 (Dedicated Tables) | T005-T007, T005a | Not AppConfigRecord JSON; one-chart-per-dashboard via unique partial index |
|
||||
| FR-015 (RBAC Matrix) | T023, T031, T034, T044 | Endpoint-by-endpoint role requirements |
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- Task IDs T001-T061 are sequential. 62 total tasks.
|
||||
- Task IDs T001-T061 are sequential, plus T005a, T019a, T026a. 64 total tasks.
|
||||
- Tasks marked [P] can run in parallel within their phase.
|
||||
- Backend tests use `pytest -v`; frontend tests use `vitest run -v`.
|
||||
- All C4 contracts (T009-T010, T021-T022, T028-T029) MUST include belief runtime instrumentation.
|
||||
|
||||
@@ -24,7 +24,7 @@
|
||||
## 2. The "Happy Path" Narrative
|
||||
|
||||
**Maintenance start (Operator → ss-tools → Superset)**
|
||||
An Airflow DAG finishes clearing staging tables and triggers: `curl -X POST https://ss-tools/api/maintenance/start -d '{"tables":["raw.sales","raw.inventory"],"start_time":"2026-05-21T22:00:00+03:00","end_time":"2026-05-22T02:00:00+03:00"}'`. The API immediately returns `{"task_id":"task-abc","maintenance_id":"m-abc123","status":"pending"}`. The operator polls `GET /api/tasks/task-abc` (or receives a WebSocket notification). Within 60 seconds, the task completes, and 12 dashboards in Superset display an orange banner at the very top: "⚠️ Maintenance in progress: 2026-05-21 22:00 — 2026-05-22 02:00 MSK. Data may be incomplete." Dashboard viewers see the warning immediately upon opening the page.
|
||||
An Airflow DAG finishes clearing staging tables and triggers: `curl -X POST https://ss-tools/api/maintenance/start -d '{"tables":["raw.sales","raw.inventory"],"start_time":"2026-05-21T22:00:00+03:00","end_time":"2026-05-22T02:00:00+03:00"}'`. The API immediately returns `{"task_id":"task-abc","maintenance_id":"m-abc123","status":"pending"}`. The operator polls `GET /api/tasks/task-abc`. Within 60 seconds, the task completes, and 12 dashboards in Superset display an orange banner at the very top. If the operator doesn't know the end time, they can omit `end_time` — the banner will show «End: TBD» instead of a specific time. Dashboard viewers see the warning immediately upon opening the page.
|
||||
|
||||
**Maintenance end (Operator → ss-tools → Superset)**
|
||||
The ETL pipeline completes successfully. The Airflow DAG calls `curl -X POST https://ss-tools/api/maintenance/m-abc123/end`. The API returns `{"task_id":"task-end-abc","status":"pending"}`. Upon task completion, banners disappear from all dashboards. In the ss-tools UI, the administrator sees the event marked as "Completed".
|
||||
@@ -38,6 +38,7 @@ An administrator notices the ETL has stalled and banners remain past the schedul
|
||||
|
||||
```bash
|
||||
# ===== Start maintenance =====
|
||||
# With end_time:
|
||||
$ curl -X POST https://ss-tools/api/maintenance/start \
|
||||
-H "Authorization: Bearer $JWT" \
|
||||
-H "Content-Type: application/json" \
|
||||
@@ -48,6 +49,15 @@ $ curl -X POST https://ss-tools/api/maintenance/start \
|
||||
"message": "Scheduled data mart refresh"
|
||||
}'
|
||||
|
||||
# Without end_time (banner shows «Окончание: уточняется»):
|
||||
$ curl -X POST https://ss-tools/api/maintenance/start \
|
||||
-H "Authorization: Bearer $JWT" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"tables": ["raw.sales"],
|
||||
"start_time": "2026-05-21T22:00:00+03:00"
|
||||
}'
|
||||
|
||||
# Success response — async (HTTP 202):
|
||||
{
|
||||
"task_id": "task-maint-abc123",
|
||||
@@ -95,13 +105,23 @@ $ curl https://ss-tools/api/tasks/task-maint-abc123 \
|
||||
}
|
||||
}
|
||||
|
||||
# No dashboards found — sync fast-path (HTTP 200, no task needed):
|
||||
# No dashboards found — always async (HTTP 202, result in task):
|
||||
# API response:
|
||||
{
|
||||
"maintenance_id": null,
|
||||
"status": "no_match",
|
||||
"affected_dashboards": 0,
|
||||
"message": "No dashboards found referencing tables: raw.archived",
|
||||
"unmatched_tables": ["raw.archived"]
|
||||
"task_id": "task-maint-noop",
|
||||
"maintenance_id": "m-ev-20260521-noop",
|
||||
"status": "pending"
|
||||
}
|
||||
# Task result on completion:
|
||||
{
|
||||
"task_id": "task-maint-noop",
|
||||
"status": "completed",
|
||||
"result": {
|
||||
"maintenance_id": "m-ev-20260521-noop",
|
||||
"status": "no_match",
|
||||
"affected_dashboards": 0,
|
||||
"unmatched_tables": ["raw.archived"]
|
||||
}
|
||||
}
|
||||
|
||||
# ===== End maintenance =====
|
||||
@@ -158,6 +178,8 @@ $ curl https://ss-tools/api/maintenance/settings \
|
||||
# Response:
|
||||
{
|
||||
"target_environment_id": "prod",
|
||||
"display_timezone": "Europe/Moscow",
|
||||
"banner_template": "<div style=\"background:#FFF3E0;...\">## ⚠️ Технические работы\n\n{message}\n\n**Начало:** {start_time}\n**Конец:** {end_time}\n\n*Данные могут быть неполными.*\n</div>",
|
||||
"dashboard_scope": "published_only",
|
||||
"excluded_dashboard_ids": [99, 101],
|
||||
"forced_dashboard_ids": [10, 42],
|
||||
@@ -218,7 +240,13 @@ $ curl -X PUT https://ss-tools/api/maintenance/settings \
|
||||
|
||||
Data may be incomplete or temporarily unavailable.
|
||||
```
|
||||
* **Style**: Markdown with orange-tinted background (CSS via markdown HTML or Superset markdown CSS class). Bold header.
|
||||
* **Стиль**: Markdown с настраиваемым шаблоном (`banner_template` в настройках). Переменные `{start_time}`, `{end_time}`, `{message}` опциональны — шаблон работает с любым набором. По умолчанию: оранжевый фон с рамкой.
|
||||
|
||||
* **Примеры шаблонов**:
|
||||
- **По умолчанию**: `<div style="background:#FFF3E0;...">## ⚠️ Технические работы\n\n{message}\n\n**Начало:** {start_time}\n**Конец:** {end_time}\n\n*Данные могут быть неполными.*</div>`
|
||||
- **Минимальный**: `> ⚠️ Ведутся технические работы. Данные могут быть неполными.` — без переменных, просто статичный текст
|
||||
- **Тёмная тема**: `<div style="background:#FFEBEE;color:#B71C1C;padding:12px">🚧 **{start_time} – {end_time}**{message}</div>`
|
||||
- **Только время**: `⚠️ Работы: {start_time} — {end_time}` — без message
|
||||
|
||||
## 4. The "Error" Experience
|
||||
|
||||
@@ -271,5 +299,6 @@ $ curl -X PUT https://ss-tools/api/maintenance/settings \
|
||||
End: {end_time_formatted}
|
||||
Data may be incomplete or temporarily unavailable.
|
||||
```
|
||||
If `end_time` is not provided, «End: TBD» (or localized equivalent) is shown.
|
||||
If `message` is not provided, the first body line is omitted.
|
||||
* **Dashboard Hub tooltip**: "Maintenance active: {start} — {end}"
|
||||
|
||||
Reference in New Issue
Block a user