ss-tools/.opencode/skills/semantics-testing/SKILL.md

name, description

name	description
semantics-testing	Core protocol for Test Constraints, External Ontology, Graph Noise Reduction, and Invariant Traceability for Python (pytest) and Svelte (vitest) projects.

#region Std.Semantics.Testing [C:5] [TYPE Skill] [SEMANTICS testing,qa,verification,pytest,vitest] @BRIEF HOW to write tests: constraints, external ontology, graph noise reduction, and invariant traceability for pytest and vitest. @RELATION DEPENDS_ON -> [Std.Semantics.Core] @RELATION DEPENDS_ON -> [Std.Semantics.Svelte] @INVARIANT Test modules must trace back to production @INVARIANT tags without flooding the Semantic Graph with orphan nodes. @RATIONALE Test contracts trace to production @INVARIANT/@POST tags via @TEST_INVARIANT, preventing orphan nodes. pytest+vitest dual stack eliminates cross-language tooling overhead. 3-edge-case floor balances coverage sufficiency against graph noise. Hardcoded fixtures block logic-mirror tautology — the dominant LLM test-generation failure mode where the agent re-implements the production algorithm inside the test as expected = compute(x). The test always passes but proves nothing because it's a copy of what it's testing. @REJECTED Property-based testing — non-deterministic input space creates unbounded graph edges, irreducible to fixed-scenario tracing. Snapshot testing — brittle to CSS/UI changes without invariant signal. Integration-only (no unit tests) — coarse graph edges miss localized @INVARIANT violations. Cucumber/Gherkin BDD — DSL layer breaks direct traceability to Python/Svelte @POST anchors. Dynamic expected-value computation — expected = production_fn(x) is a tautology, not a test; hardcoded fixtures are the only valid approach.

0. QA RATIONALE (LLM PHYSICS IN TESTING)

You are an Agentic QA Engineer. Your primary failure modes are:

1. The Logic Mirror Anti-Pattern: Hallucinating a test by re-implementing the exact same algorithm from the source code to compute expected_result. This creates a tautology (a test that always passes but proves nothing).
2. Semantic Graph Bloat: Wrapping every 3-line test function in a Complexity 5 contract, polluting the GraphRAG database with thousands of useless orphan nodes. Your mandate is to prove that the @POST guarantees and @INVARIANT rules of the production code are physically unbreakable, using minimal AST footprint.

I. EXTERNAL ONTOLOGY (BOUNDARIES)

When writing code or tests that depend on 3rd-party libraries or shared schemas that DO NOT have local anchors in our repository, you MUST use strict external prefixes. CRITICAL RULE: Do NOT hallucinate anchors for external code.

1. External Libraries ([EXT:Package:Module]):
   • Use for 3rd-party dependencies.
   • Example: @RELATION DEPENDS_ON -> [EXT:FastAPI:Router] or [EXT:SQLAlchemy:Session]
   • Svelte: [EXT:SvelteKit:load]
2. Shared DTOs ([DTO:Name]):
   • Use for globally shared schemas, Pydantic models, or external registry definitions.
   • Example: @RELATION DEPENDS_ON -> [DTO:DashboardExportPayload]

II. TEST MARKUP ECONOMY (NOISE REDUCTION)

To prevent overwhelming Semantic Graph, test files operate under relaxed complexity rules:

1. Short hierarchical IDs: Test modules use Test.Domain.Name format (e.g., Test.Migration.RunTask), not full file paths or flat names. This satisfies ATTN_2: the Test. prefix groups all tests under HCA 128× while the domain name provides DSA Indexer grouping.
2. Root Binding (BINDS_TO): Do NOT map the internal call graph of a test file. Instead, anchor the entire test suite to the production module using: @RELATION BINDS_TO -> [TargetModule].
3. Complexity 1 for Helpers: Small test utilities (e.g., _setup_mock, _build_payload) are C1. They require ONLY the anchor pair. No @BRIEF or @RELATION allowed.
4. Complexity 2 for Tests: Actual test functions (e.g., test_unauthorized_access) are C2. They require anchor + @BRIEF. Do not add @PRE/@POST to individual test functions.
5. Maximum test file size: A single test file MUST NOT exceed 600 lines. Beyond this threshold:
   • Split into multiple test files by domain (e.g., test_auth_flow.py, test_auth_ws.py instead of test_auth.py).
   • Extract shared fixtures into a conftest.py in the same directory.
   • Each test class tests ONE production contract — if a file has more than 3 test classes, split by class.
   • Exception: Integration test files using Testcontainers may be up to 800 lines due to longer setup/teardown.
   • RATIONALE: Files >600 lines degrade the model's sliding-window attention — the bottom of the file is compressed before the top is applied, leading to duplicate tests and orphan contracts.

III. TRACEABILITY & TEST CONTRACTS

In the Header of your Test Module, you MUST define the Test Contracts. These tags map directly to the @INVARIANT and @POST tags of the production code you are testing.

• @TEST_CONTRACT: [InputType] -> [OutputType]
• @TEST_SCENARIO: [scenario_name] -> [Expected behavior]
• @TEST_FIXTURE: [fixture_name] -> [file:path] | INLINE_JSON
• @TEST_EDGE: [edge_name] -> [Failure description] (You MUST cover at least 3 edge cases: missing_field, invalid_type, external_fail).
• The Traceability Link: @TEST_INVARIANT: [Invariant_Name_From_Source] -> VERIFIED_BY: [scenario_1, edge_name_2]

IV. ADR REGRESSION DEFENSE

The Architectural Decision Records (ADR) and @REJECTED tags in production code are constraints. If the production contract has a @REJECTED [Forbidden_Path] tag (e.g., @REJECTED fallback to SQLite), your Test Module MUST contain an explicit @TEST_EDGE scenario proving that the forbidden path is physically unreachable or throws an appropriate error. Tests are the enforcers of architectural memory.

V. ANTI-TAUTOLOGY RULES

1. No Logic Mirrors: Use deterministic, hardcoded fixtures (@TEST_FIXTURE) for expected results. Do not dynamically calculate expected = a + b to test an add(a, b) function.
2. Do Not Mock The System Under Test: You may mock [EXT:...] boundaries (like DB drivers or external APIs), but you MUST NOT mock the local contract node you are actively verifying.

VI. PYTHON / PYTEST CONVENTIONS

Test file structure

backend/tests/
├── conftest.py           # Shared fixtures, mock setup (C3 Module)
├── test_auth.py          # Auth tests (C2 Module, BINDS_TO -> AuthService)
├── test_migration.py     # Migration tests
└── test_plugins/         # Plugin-specific tests

Test module template

# #region Test.Migration.RunTask [C:3] [TYPE Module] [SEMANTICS test,migration]
# @BRIEF Verify dashboard migration contracts — @POST guarantees and rejected paths.
# @RELATION BINDS_TO -> [Migration.RunTask]
# @TEST_EDGE: missing_db_mapping -> Migration fails with MappingError
# @TEST_EDGE: invalid_dashboard_id -> Migration fails with NotFoundError
# @TEST_EDGE: external_api_timeout -> Migration fails with TimeoutError, rolls back
import pytest
from unittest.mock import AsyncMock, patch

class TestDashboardMigration:
    """Verify migrate_dashboard @POST guarantees."""

    # #region test_migrate_dashboard_success [C:2] [TYPE Function]
    # @BRIEF Happy path: valid dashboard with complete db mapping.
    @pytest.mark.asyncio
    async def test_migrate_dashboard_success(self):
        # Use hardcoded fixture, not algorithmic computation
        expected = {"id": "dash_1", "status": "imported"}
        # ... test implementation
        pass
    # #endregion test_migrate_dashboard_success
# #endregion TestDashboardMigration

Running tests

# All backend tests (integration tests skipped by default)
cd backend && source .venv/bin/activate && python -m pytest -v

# Specific test file
python -m pytest tests/test_migration.py -v

# Include integration tests (PostgreSQL/Superset Testcontainers)
python -m pytest --run-integration

# Run only integration tests
python -m pytest tests/integration/ --run-integration

# With coverage
python -m pytest --cov=src --cov-report=term-missing

Integration tests (tests/integration/) use Testcontainers (PostgreSQL 16, Superset 4.1.2) and require Docker. They are skipped by default — pass --run-integration to enable. The --run-integration flag is registered in backend/tests/conftest.py via pytest_addoption; skip logic lives in backend/tests/integration/conftest.py via pytest_collection_modifyitems. See also: backend/pyproject.toml [tool.pytest.ini_options] markers for the registered marker.

VII. SVELTE / VITEST CONVENTIONS

Test file structure

frontend/src/
├── lib/
│   ├── components/__tests__/   # Component tests
│   │   ├── MigrationTaskCard.test.js
│   │   └── StatusBadge.test.js
│   └── stores/__tests__/       # Store tests
│       └── taskDrawer.test.js

Component test template

import { render, screen, fireEvent } from "@testing-library/svelte";
import { describe, it, expect, vi } from "vitest";
import ComponentName from "../ComponentName.svelte";

describe("ComponentName", () => {
  it("renders with props", () => {
    const { container } = render(ComponentName, {
      props: { title: "Test", status: "idle" }
    });
    expect(container.textContent).toContain("Test");
  });

  it("shows loading state on action", async () => {
    // Use mock API, verify loading states
  });
});

Running tests

# All frontend tests
cd frontend && npm run test

# Watch mode
npm run test:watch

VIII. VERIFIABLE HARNESS RULES

For agentic development, a test harness is part of the task environment.

• Prefer real executable checks over narrative claims that a change is safe.
• Verify that the harness actually fails on the broken state and passes on the fixed state whenever feasible.
• Resist shortcut tests that bypass the real integration boundary the task is supposed to validate.
• When a production @POST guarantee is subtle, add the narrowest test that can falsify it.

IX. LONG-HORIZON QA MEMORY

When multiple attempts are needed:

• Preserve the smallest set of failing fixtures, commands, and invariant mappings that explain the current gap.
• Fold older failed attempts into one bounded note describing what was tried and why it was rejected.
• Do not keep extending the active QA transcript with redundant command output.

X. TESTING SEARCH DISCIPLINE

• Use one concrete failing hypothesis plus one verifier by default.
• Add alternative test strategies only when the first verifier is inconclusive.
• Do not mirror the implementation logic to fabricate expected values; use fixtures, explicit contracts, and invariant-oriented assertions.

#endregion Std.Semantics.Testing