ss-tools/specs/031-maintenance-banner/spec.md

Feature Specification: Maintenance Banner for Dashboards

Feature Branch: 031-maintenance-banner
Created: 2026-05-21
Status: Draft
Input: User description: "Service for adding a maintenance banner to dashboards during ETL table updates. External tool passes names of tables being updated to an API method in ss-tools; ss-tools finds dashboards with datasets referencing those tables (both table-based and virtual SQL Jinja datasets) and adds a static text-chart at the top of the dashboard with maintenance information. Support passing start and end times. Banner removal — centrally from ss-tools or externally via API."

Clarifications

Session 2026-05-21

• Q: Which Superset environment should /api/maintenance/start target (ss-tools manages multiple)? → A: System always targets the production environment (pre-configured in ss-tools settings). External tool does NOT pass environment_id.
• Q: Should /api/maintenance/start be synchronous (blocking) or asynchronous (task-based)? → A: Async — returns {task_id, status: "pending"} immediately; operator polls GET /api/tasks/{task_id} or uses WebSocket for progress (matches existing TaskManager pattern).
• Q: How should table names from the API be matched against dataset references? → A: Exact schema.table match only (e.g., "raw.sales" matches only "raw.sales", not unqualified "sales"). Case-insensitive.
• Q: Should duplicate /start calls (same tables, same times) create a new MaintenanceEvent or be idempotent? → A: Idempotent by (tables, start_time, end_time) key. Same tuple = already_active. Changed end_time (or adding it when previously omitted) = new event. Omitted end_time treated as None in the key. The message field is NOT part of the idempotency key — duplicate calls with a different message return already_active and the original message is preserved.
• Q: When /end is called on event M1 but dashboard D is also affected by active event M2, should the banner be removed? → A: No — banner stays on D but is rebuilt with updated text (only M2's info). Removal occurs only when the LAST active event affecting D is ended.

User Scenarios & Testing (mandatory)

User Story 1 — External tool initiates maintenance banner display (Priority: P1)

An ETL operator triggers a table refresh. The external tool (e.g., Airflow DAG) calls an ss-tools API endpoint, passing:

• List of table names being updated (e.g., ["raw.sales", "raw.inventory"])
• Maintenance start time
• Planned end time
• (optional) custom message text

ss-tools finds all dashboards in Superset whose datasets reference the given tables (both table-based datasets and virtual/SQL Jinja datasets whose SQL query references the tables). For each affected dashboard, ss-tools adds a markdown chart at the very top with the maintenance warning.

Why this priority: This is the core use case — without it, the feature has no value. It enables automatic notification of dashboard users about potentially stale data during ETL.

Independent Test: Can be tested by calling the API with a table name known to be used in a test dashboard, then verifying a markdown chart appears at the top of that dashboard in Superset.

Acceptance Scenarios:

1. Given dashboard D in the production environment uses a dataset referencing table raw.sales, When external tool calls POST /api/maintenance/start with {"tables": ["raw.sales"], "start_time": "2026-05-21T22:00:00Z", "end_time": "2026-05-22T02:00:00Z"}, Then API returns {task_id: "t-...", status: "pending"} immediately, and upon task completion dashboard D in Superset shows a markdown chart at the top: "⚠️ Maintenance in progress: 2026-05-21 22:00 — 2026-05-22 02:00 UTC. Data may be incomplete."

2. Given dashboard D uses a virtual SQL dataset with query SELECT * FROM raw.sales JOIN raw.inventory ON ..., When external tool passes table raw.inventory, Then dashboard D also receives the banner (since the table is found in the dataset SQL text).

3. Given dashboard D already has a banner from a previous maintenance event, When a new POST /api/maintenance/start request arrives referencing a different table that also affects D, Then the banner is updated (time and text replaced) rather than duplicated.

4. Given table raw.archived is not used in any dashboard, When external tool passes {"tables": ["raw.archived"]}, Then API returns {task_id, maintenance_id, status: "pending"} immediately. Upon task completion, the task result contains {status: "no_match", affected_dashboards: 0, unmatched_tables: ["raw.archived"]}. No dashboard is modified.

5. Given an active MaintenanceEvent E already exists with tables ["raw.sales"], When external tool calls /start again with the same tables, Then API returns {maintenance_id: E.id, status: "already_active"} without creating a duplicate event.

6. Given external tool does not know the end time, When it calls /start with {"tables": ["raw.sales"], "start_time": "2026-05-21T22:00:00Z"} (no end_time), Then API returns 202 {task_id}, and the banner shows «Окончание: уточняется».

---

User Story 2 — External tool removes maintenance banner upon completion (Priority: P1)

The ETL process completes. The external tool calls an ss-tools API endpoint to remove banners. ss-tools removes markdown charts from all dashboards affected by the given maintenance event.

Why this priority: Critical for restoring normal dashboard appearance after maintenance. Users must not see stale warnings.

Independent Test: After US1, call POST /api/maintenance/{id}/end and verify charts disappear from dashboards.

Acceptance Scenarios:

1. Given maintenance event M added banners to dashboards D1, D2 (no other active events affect D1 or D2), When external tool calls POST /api/maintenance/{maintenance_id}/end, Then banners are removed from D1 and D2, and maintenance event M is marked as completed.

2. Given maintenance event M is already completed, When external tool calls POST /api/maintenance/{maintenance_id}/end again, Then API returns {"status": "already_completed", "message": "Maintenance M was already completed at ..."} (idempotent).

3. Given dashboard D is affected by active events M1 and M2, When external tool calls /end on M1, Then the banner on D is NOT removed — instead, the banner text is rebuilt to show only M2's information. M1 is marked as completed. The banner is fully removed only when M2 also ends.

---

User Story 3 — Administrator manages banners via ss-tools UI (Priority: P2)

An administrator opens the ss-tools web interface, sees a list of active maintenance events and affected dashboards. They can selectively or bulk-remove banners without waiting for the external tool.

Why this priority: Administrators need manual override capability in case of ETL tool failure or emergency banner removal.

Independent Test: Navigate to maintenance management page, view active events, click "Remove All" and verify banners disappear.

Acceptance Scenarios:

1. Given active maintenance events M1 (affects D1, D2) and M2 (affects D3), When administrator opens the "Maintenance Banners" page, Then they see a table with M1, M2, affected dashboard counts, start/end times, and action buttons.

2. Given administrator is on the management page, When they select M1 and click "Remove Banners", Then markdown charts are removed from D1, D2, and M1 status changes to "Completed".

3. Given administrator is on the management page, When they click "Remove All", Then banners are removed from all dashboards across all active maintenance events.

---

User Story 4 — Maintenance mode scope configuration (Priority: P3)

Administrator configures which dashboards are affected by automatic banner placement:

• Published only, drafts only, or both
• List of dashboards excluded from maintenance mode (hard ban — banner is never added)
• List of dashboards forcibly included (banner always added, even if no table match is found)

Why this priority: Configuration flexibility allows adaptation to specific processes, but the system works with sensible defaults.

Independent Test: Configure exclusion of dashboard D1, then trigger maintenance on a table used in D1 — banner must NOT appear on D1, but must appear on D2.

Acceptance Scenarios:

1. Given default settings (scope = published_only, excluded = [], forced = []), When maintenance affects draft dashboard D, Then banner is NOT added to D.

2. Given administrator added dashboard D to the excluded list, When maintenance affects a table used in D, Then banner is NOT added to D, and logs record the reason: "Dashboard D is in exclusion list".

3. Given administrator added dashboard D to the forced list, When maintenance does NOT affect any table in D, Then banner IS STILL added to D (forced inclusion).

4. Given scope = all (published + draft), When maintenance affects tables in D1 (published) and D2 (draft), Then banner is added to BOTH dashboards.

---

User Story 5 — Maintenance status indicator in dashboard grid (Priority: P3)

The Dashboard Hub table shows an indicator when a dashboard has an active maintenance banner. Users can see this information without opening Superset.

Why this priority: Improves operator UX but does not block core functionality.

Independent Test: After US1, the dashboard grid shows a maintenance badge in the status column.

Acceptance Scenarios:

1. Given dashboard D has an active maintenance banner, When a user opens the Dashboard Hub, Then dashboard D's row shows an orange badge "⚠️ Maintenance" with a tooltip showing start and end times.

2. Given the maintenance banner is removed from dashboard D, When the user refreshes the Dashboard Hub page, Then the badge disappears.

---

Edge Cases

• Multiple maintenance events conflict: If two ETL processes simultaneously update different tables affecting the same dashboard — the banner must aggregate information from all active events. When one event ends, the banner text is rebuilt from remaining active events rather than removed. Full banner removal occurs only when ALL events affecting that dashboard are completed.

• Dashboard with no datasets (empty dashboard): Must not receive a banner, even if tables are listed in the maintenance event.

• Table referenced in multiple datasets within one dashboard: Banner is added only once per dashboard, regardless of how many dataset references the table has within it.

• Network error contacting Superset API: If some dashboards are processed successfully and others fail due to Superset errors, the API returns a partial success response listing the problematic dashboards.

• Very large table list (100+): API must process Superset queries in batches and return progress or a warning about extended processing time.

• Dashboard deleted from Superset between discovery and banner placement: Handle 404 from Superset API, log the event, and continue processing the rest.

• Virtual dataset with Jinja and SQL: Table names may appear both in plain SQL (FROM/JOIN) and inside Jinja {% set %} blocks (e.g., dictionary mapping dataset types to table names). Extraction uses global pattern matching on raw text (not Jinja-stripping first) to capture ALL schema.table occurrences, then sqlparse filters out false positives inside SQL string literals. Unqualified table references are not matched.

• Dashboard present in both excluded and forced lists: Forced takes priority (explicit inclusion overrides exclusion).

• Duplicate /start call with identical parameters: Idempotency key is (tables, start_time, end_time). Same tuple → "already_active". Same tables + start_time but different end_time (or adding end_time when previously omitted) → new event. Omitted end_time is None in the key.

• Message HTML escaping: The message field in the API request is plain text only. All HTML special characters (<, >, &, ", ') are escaped before rendering in the Superset markdown chart. Raw HTML/CSS injection via the message field is rejected at the API boundary (400).

• Time zones: Maintenance start and end times must be correctly displayed in the banner text respecting timezone (accept ISO 8601 with timezone, display in local timezone or UTC as configured).

• Tabbed dashboards: The banner is added to the root dashboard layout only. If a Superset dashboard uses tabs, the banner may not be visible when viewing individual tabs — this is a known limitation. Future enhancement may add per-tab banner placement.

Requirements (mandatory)

Functional Requirements

• FR-001: System MUST provide API endpoint POST /api/maintenance/start accepting: list of table names (tables: str[], max 100 entries, sorted set for idempotency), start time (start_time: datetime — required, in the future ±1h grace period), end time (end_time: datetime | None — optional, must be > start_time if provided), optional message text (message: str | None). Always returns 202 {task_id, maintenance_id, status: "pending"} immediately after validation and event creation. Sync short-circuits only for already_active (idempotency) and validation errors (400). When end_time is omitted, the banner displays «Окончание: уточняется».

• FR-002: System MUST, upon receiving a maintenance start request, find all dashboards in the configured production Superset environment whose datasets reference the given tables.

• FR-003: System MUST support finding tables in two dataset types using case-insensitive exact schema.table matching:

  • Table-based (physical tables): match on {schema}.{table_name} format (e.g., "raw.sales" matches dataset with schema=raw, table_name=sales).
  • Virtual/SQL (is_sqllab_view = true): extract ALL fully-qualified schema.table references from the raw SQL+Jinja text using global pattern matching (regex), then filter false positives via sqlparse tokenizer. This captures table names in FROM/JOIN clauses, Jinja {% set %} blocks, CTE bodies, and subqueries. Jinja is NOT pre-stripped — table names inside {% set %} blocks are preserved. Unqualified references are NOT matched.

• FR-004: System MUST add a markdown chart (Superset viz_type: "markdown") at the top of each affected dashboard with maintenance information.

• FR-005: The banner text is rendered from banner_template (configured in MaintenanceSettings). For a single active event, variables {start_time}, {end_time}, {message} are directly substituted. For multiple active events, aggregated text is built by rendering each event's info as a sub-block (e.g., "Event N: {start} – {end}"), and the combined text is substituted as {message}. Variables not present in the template are replaced with an empty string. The message field from the API request is plain text — HTML special characters are escaped before substitution.

• FR-006: System MUST provide API endpoint POST /api/maintenance/{maintenance_id}/end for removing banners from dashboards affected by a specific maintenance event. Returns {task_id} immediately; removal runs asynchronously.

• FR-007: System MUST provide API endpoint POST /api/maintenance/end-all for bulk removal of all active banners from all dashboards. Returns {task_id} immediately; removal runs asynchronously.

• FR-008: System MUST persist a maintenance event history in the local database with fields: id, task_id, table list, start time, end time, optional message, status (active/completed/failed), timestamps. Affected dashboard tracking uses the normalized MaintenanceDashboardState relationship, not an inline dashboard ID list.

• FR-009: System MUST support maintenance mode settings via GET/PUT /api/maintenance/settings:

  • target_environment_id: str — the production Superset environment to scan/modify
  • display_timezone: str — timezone for banner text display (default: "UTC")
  • banner_template: str — markdown template with optional variables {start_time}, {end_time}, {message}. Variables are substituted at render time; missing variables are omitted. Default template includes all three with ⚠️ header and styling (see UX reference).
  • dashboard_scope: "published_only" | "draft_only" | "all" (default: "published_only")
  • excluded_dashboard_ids: int[] — dashboards NEVER receiving banners
  • forced_dashboard_ids: int[] — dashboards ALWAYS receiving banners (overrides excluded per FR-009-b)

• FR-009-b: When a dashboard is present in BOTH forced_dashboard_ids and excluded_dashboard_ids, forced inclusion takes priority — the banner IS placed.

• FR-010: System MUST provide a UI management page for maintenance mode with:

  • Table of active and completed maintenance events
  • Selective and bulk banner removal
  • Settings configuration form (scope, excluded, forced)

• FR-011: System MUST display an active banner indicator in the Dashboard Hub table as a badge with a tooltip.

• FR-012: System MUST handle multiple maintenance events on the same dashboard: aggregate information and remove the banner only after ALL affecting events are completed.

• FR-013: System MUST, on partial success, return a detailed response listing successfully processed and problematic dashboards.

• FR-014: All dashboard-modifying operations via Superset API MUST require the operator or admin role (per RBAC ADR-0005).

• FR-015: System MUST enforce RBAC per this matrix:

Endpoint / UI	viewer	operator	admin
GET /api/maintenance/dashboard-banners	✅	✅	✅
GET /api/maintenance/events	❌	✅	✅
POST /api/maintenance/start	❌	✅	✅
POST /api/maintenance/{id}/end	❌	✅	✅
POST /api/maintenance/end-all	❌	✅	✅
GET /api/maintenance/settings	❌	✅	✅
PUT /api/maintenance/settings	❌	❌	✅
/maintenance UI page	❌	✅ (read-only)	✅

• FR-016: System MUST provide GET /api/maintenance/events returning active and completed event lists with affected dashboard counts.

• FR-017: System MUST provide GET /api/maintenance/dashboard-banners returning per-dashboard banner state for the Dashboard Hub indicator.

Key Entities

• MaintenanceEvent: A record of a maintenance event. Contains: id, task_id (links to TaskManager task), table list, start time, end time, optional message, affected dashboard IDs, status (active/completed/failed), created_at, updated_at.

• MaintenanceDashboardBanner (NEW): The canonical "one chart per dashboard" entity. Unique constraint on (environment_id, dashboard_id, status='active'). Contains: id, environment_id, dashboard_id, chart_id (Superset markdown chart ID), banner_text (current aggregated markdown), updated_at. Only ONE active banner chart exists per dashboard regardless of how many MaintenanceEvents affect it.

• MaintenanceDashboardState: Links a maintenance event to a dashboard. Contains: id, maintenance_event_id, dashboard_id, banner_id (FK → MaintenanceDashboardBanner), status (pending_apply/active/removing/removed/apply_failed/removal_failed), created_at, updated_at. Does NOT own chart_id directly — references the shared MaintenanceDashboardBanner.

• MaintenanceSettings: Maintenance mode configuration. Contains: id, target_environment_id (str), display_timezone (str, default "UTC"), banner_template (str — markdown with optional {start_time}, {end_time}, {message} variables; default has all three), dashboard_scope (enum), excluded_dashboard_ids (int[]), forced_dashboard_ids (int[]), updated_at.

• Dashboard (existing Superset entity): Used for the dashboard → datasets → tables relationship. Has published/draft status.

• Dataset (existing Superset entity): Two types: table-based (direct table reference) and virtual/SQL (is_sqllab_view = true, contains SQL query).

Success Criteria (mandatory)

Measurable Outcomes

• SC-001: POST /api/maintenance/start returns {task_id} within 1 second (immediate async dispatch). The background task finds all affected dashboards (up to 50) and places banners within 60 seconds.

• SC-002: After task completion, the banner appears on each dashboard in Superset within the same 60-second window (including markdown chart creation via Superset API).

• SC-003: POST /api/maintenance/{id}/end returns {task_id} within 1 second; the background task removes banners from all affected dashboards (up to 50) within 30 seconds.

• SC-004: SQL table name extraction from virtual datasets correctly identifies all fully-qualified schema.table references embedded in Jinja {% set %} blocks, plain SQL FROM/JOIN, CTE bodies, and subqueries, while filtering out string-literal false positives (date literals like '2026.04.30') — with ≥ 95% accuracy on a test corpus of production virtual dataset SQL+Jinja samples.

• SC-005: Administrator can remove banners from all dashboards via UI in fewer than 3 clicks.

• SC-006: Conflicting maintenance events on the same dashboard do not produce duplicate banners — exactly one markdown chart per dashboard at all times.

• SC-007: On Superset API network errors, the system correctly returns a partial result without losing information about already-processed dashboards.

UX Reference

See ux_reference.md for detailed interaction flows, UI mockups, and error recovery paths.