busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

agents

Browse Source
This commit is contained in:
busya
2026-05-11 22:58:01 +03:00
parent 2abba06e52
commit fefdee98d0
30 changed files with 1681 additions and 1222 deletions
Download Patch File Download Diff File Expand all files Collapse all files

217
.opencode/skills/semantics-testing/SKILL.md
View File

@@ -1,138 +1,171 @@
---
name: semantics-testing
description: Core protocol for Test Constraints, External Ontology, Graph Noise Reduction, and Invariant Traceability.
description: Core protocol for Test Constraints, External Ontology, Graph Noise Reduction, and Invariant Traceability for Python (pytest) and Svelte (vitest) projects.
---
# [DEF:Std:Semantics:Testing]
# @COMPLEXITY 5
# @PURPOSE Core protocol for Test Constraints, External Ontology, Graph Noise Reduction, and Invariant Traceability.
# @RELATION DEPENDS_ON -> [Std:Semantics:Core]
# @INVARIANT Test modules must trace back to production @INVARIANT tags without flooding the Semantic Graph with orphan nodes.
#region Std.Semantics.Testing [C:5] [TYPE Skill] [SEMANTICS testing,qa,verification,pytest,vitest]
@BRIEF HOW to write tests: constraints, external ontology, graph noise reduction, and invariant traceability for pytest and vitest.
@RELATION DEPENDS_ON -> [Std.Semantics.Core]
@INVARIANT Test modules must trace back to production @INVARIANT tags without flooding the Semantic Graph with orphan nodes.
## 0. QA RATIONALE (LLM PHYSICS IN TESTING)
You are an Agentic QA Engineer. Your primary failure modes are:
1. **The Logic Mirror Anti-Pattern:** Hallucinating a test by re-implementing the exact same algorithm from the source code to compute `expected_result`. This creates a tautology (a test that always passes but proves nothing).
2. **Semantic Graph Bloat:** Wrapping every 3-line test function in a Complexity 5 contract, polluting the GraphRAG database with thousands of useless orphan nodes.
Your mandate is to prove that the `@POST` guarantees and `@INVARIANT` rules of the production code are physically unbreakable, using minimal AST footprint.
## I. EXTERNAL ONTOLOGY (BOUNDARIES)
When writing code or tests that depend on 3rd-party libraries or shared schemas that DO NOT have local `[DEF]` anchors in our repository, you MUST use strict external prefixes.
**CRITICAL RULE:** Do NOT hallucinate `[DEF]` anchors for external code.
When writing code or tests that depend on 3rd-party libraries or shared schemas that DO NOT have local anchors in our repository, you MUST use strict external prefixes.
**CRITICAL RULE:** Do NOT hallucinate anchors for external code.
1. **External Libraries (`[EXT:Package:Module]`):**
- Use for 3rd-party dependencies.
- Example: `@RELATION DEPENDS_ON ->[EXT:FastAPI:Router]` or `[EXT:SQLAlchemy:Session]`
- Example: `@RELATION DEPENDS_ON -> [EXT:FastAPI:Router]` or `[EXT:SQLAlchemy:Session]`
- Svelte: `[EXT:SvelteKit:load]`
2. **Shared DTOs (`[DTO:Name]`):**
- Use for globally shared schemas, Protobufs, or external registry definitions.
- Example: `@RELATION DEPENDS_ON -> [DTO:StripeWebhookPayload]`
- Use for globally shared schemas, Pydantic models, or external registry definitions.
- Example: `@RELATION DEPENDS_ON -> [DTO:DashboardExportPayload]`
## II. TEST MARKUP ECONOMY (NOISE REDUCTION)
To prevent overwhelming Semantic Graph, test files operate under relaxed complexity rules:
1. **Short IDs:** Test modules MUST use concise IDs (e.g., `[DEF:PaymentTests:Module]`), not full file paths.
2. **Root Binding (`BINDS_TO`):** Do NOT map the internal call graph of a test file. Instead, anchor the entire test suite or large fixture classes to the production module using: `@RELATION BINDS_TO -> [DEF:TargetModuleId]`.
3. **Complexity 1 for Helpers:** Small test utilities (e.g., `_setup_mock`, `_build_payload`) are **C1**. They require ONLY `[DEF]...[/DEF]` anchors. No `@PURPOSE` or `@RELATION` allowed.
4. **Complexity 2 for Tests:** Actual test functions (e.g., `test_invalid_auth`) are **C2**. They require `[DEF]...[/DEF]` and `@PURPOSE`. Do not add `@PRE`/`@POST` to individual test functions.
1. **Short IDs:** Test modules MUST use concise IDs (e.g., `TestDashboardMigration`), not full file paths.
2. **Root Binding (`BINDS_TO`):** Do NOT map the internal call graph of a test file. Instead, anchor the entire test suite to the production module using: `@RELATION BINDS_TO -> [TargetModule]`.
3. **Complexity 1 for Helpers:** Small test utilities (e.g., `_setup_mock`, `_build_payload`) are **C1**. They require ONLY the anchor pair. No `@BRIEF` or `@RELATION` allowed.
4. **Complexity 2 for Tests:** Actual test functions (e.g., `test_unauthorized_access`) are **C2**. They require anchor + `@BRIEF`. Do not add `@PRE`/`@POST` to individual test functions.
## III. TRACEABILITY & TEST CONTRACTS
In the Header of your Test Module (or inside a large Test Class), you MUST define the Test Contracts. These tags map directly to the `@INVARIANT` and `@POST` tags of the production code you are testing.
In the Header of your Test Module, you MUST define the Test Contracts. These tags map directly to the `@INVARIANT` and `@POST` tags of the production code you are testing.
- `@TEST_CONTRACT: [InputType] -> [OutputType]`
- `@TEST_SCENARIO: [scenario_name] -> [Expected behavior]`
- `@TEST_FIXTURE: [fixture_name] -> [file:path] | INLINE_JSON`
- `@TEST_EDGE: [edge_name] -> [Failure description]` (You MUST cover at least 3 edge cases: `missing_field`, `invalid_type`, `external_fail`).
- **The Traceability Link:** `@TEST_INVARIANT: [Invariant_Name_From_Source] -> VERIFIED_BY: [scenario_1, edge_name_2]`
## IV. PYTHON TESTING STACK
Use pytest as the primary test framework. Follow these conventions:
1. **Test files:** Named `test_*.py`, placed in a `tests/` directory mirroring the source tree.
2. **Fixtures:** Use `@pytest.fixture` for test setup. Prefer `conftest.py` for shared fixtures.
3. **Mocking:** Use `unittest.mock` (standard library) for mocking `[EXT:...]` boundaries. Use `pytest-mock` (`mocker` fixture) when available.
4. **Parametrization:** Use `@pytest.mark.parametrize` for table-driven tests covering edge cases.
5. **Assertions:** Use plain `assert` statements — pytest provides rich introspection on failures.
## IV. ADR REGRESSION DEFENSE
**Example — C1 test helper:**
```python
# [DEF:_build_payload:Function]
def _build_payload(**overrides: Any) -> dict:
base = {"name": "test", "value": 42}
return {**base, **overrides}
# [/DEF:_build_payload:Function]
```
The Architectural Decision Records (ADR) and `@REJECTED` tags in production code are constraints.
If the production contract has a `@REJECTED [Forbidden_Path]` tag (e.g., `@REJECTED fallback to SQLite`), your Test Module MUST contain an explicit `@TEST_EDGE` scenario proving that the forbidden path is physically unreachable or throws an appropriate error.
Tests are the enforcers of architectural memory.
**Example — C2 test function:**
```python
# [DEF:test_create_user_success:Function]
# @PURPOSE Verify that a valid payload creates a user and returns 201 with the user DTO.
def test_create_user_success(client: TestClient, db_session: Session) -> None:
payload = {"name": "Alice", "email": "alice@example.com"}
response = client.post("/api/users", json=payload)
assert response.status_code == 201
assert response.json()["name"] == "Alice"
assert db_session.query(User).count() == 1
# [/DEF:test_create_user_success:Function]
```
## V. ANTI-TAUTOLOGY RULES
**Example — Parametrized edge cases:**
```python
# [DEF:test_create_user_validation_edges:Function]
# @PURPOSE Cover edge cases for user creation validation: missing fields, invalid types, external failures.
@pytest.mark.parametrize("payload,expected_status,expected_detail", [
({"email": "a@b.com"}, 422, "missing_field"),
({"name": "A", "email": "not-an-email"}, 422, "invalid_type"),
])
def test_create_user_validation_edges(
client: TestClient,
payload: dict,
expected_status: int,
expected_detail: str,
) -> None:
response = client.post("/api/users", json=payload)
assert response.status_code == expected_status
assert expected_detail in str(response.json())
# [/DEF:test_create_user_validation_edges:Function]
```
## V. ADR REGRESSION DEFENSE
The Architectural Decision Records (ADR) and `@REJECTED` tags in production code are constraints.
If the production `[DEF]` has a `@REJECTED [Forbidden_Path]` tag (e.g., `@REJECTED fallback to SQLite`), your Test Module MUST contain an explicit `@TEST_EDGE` scenario proving that the forbidden path is physically unreachable or throws an appropriate error.
Tests are the enforcers of architectural memory.
## VI. ANTI-TAUTOLOGY RULES
1. **No Logic Mirrors:** Use deterministic, hardcoded fixtures (`@TEST_FIXTURE`) for expected results. Do not dynamically calculate `expected = a + b` to test an `add(a, b)` function.
2. **Do Not Mock The System Under Test:** You may mock `[EXT:...]` boundaries (like DB drivers or external APIs), but you MUST NOT mock the local `[DEF]` node you are actively verifying.
2. **Do Not Mock The System Under Test:** You may mock `[EXT:...]` boundaries (like DB drivers or external APIs), but you MUST NOT mock the local contract node you are actively verifying.
## VI. PYTHON / PYTEST CONVENTIONS
### Test file structure
```
backend/tests/
├── conftest.py # Shared fixtures, mock setup (C3 Module)
├── test_auth.py # Auth tests (C2 Module, BINDS_TO -> AuthService)
├── test_migration.py # Migration tests
└── test_plugins/ # Plugin-specific tests
```
### Test module template
```python
# #region TestDashboardMigration [C:2] [TYPE Module] [SEMANTICS test,migration]
# @BRIEF Verify dashboard migration contracts — @POST guarantees and rejected paths.
# @RELATION BINDS_TO -> [dashboard_migration]
# @TEST_EDGE: missing_db_mapping -> Migration fails with MappingError
# @TEST_EDGE: invalid_dashboard_id -> Migration fails with NotFoundError
# @TEST_EDGE: external_api_timeout -> Migration fails with TimeoutError, rolls back
import pytest
from unittest.mock import AsyncMock, patch
class TestDashboardMigration:
"""Verify migrate_dashboard @POST guarantees."""
# #region test_migrate_dashboard_success [C:2] [TYPE Function]
# @BRIEF Happy path: valid dashboard with complete db mapping.
@pytest.mark.asyncio
async def test_migrate_dashboard_success(self):
# Use hardcoded fixture, not algorithmic computation
expected = {"id": "dash_1", "status": "imported"}
# ... test implementation
pass
# #endregion test_migrate_dashboard_success
# #endregion TestDashboardMigration
```
### Running tests
```bash
# All backend tests
cd backend && source .venv/bin/activate && python -m pytest -v
# Specific test file
python -m pytest tests/test_migration.py -v
# With coverage
python -m pytest --cov=src --cov-report=term-missing
```
## VII. SVELTE / VITEST CONVENTIONS
### Test file structure
```
frontend/src/
├── lib/
│ ├── components/__tests__/ # Component tests
│ │ ├── MigrationTaskCard.test.js
│ │ └── StatusBadge.test.js
│ └── stores/__tests__/ # Store tests
│ └── taskDrawer.test.js
```
### Component test template
```javascript
import { render, screen, fireEvent } from "@testing-library/svelte";
import { describe, it, expect, vi } from "vitest";
import ComponentName from "../ComponentName.svelte";
describe("ComponentName", () => {
it("renders with props", () => {
const { container } = render(ComponentName, {
props: { title: "Test", status: "idle" }
});
expect(container.textContent).toContain("Test");
});
it("shows loading state on action", async () => {
// Use mock API, verify loading states
});
});
```
### Running tests
```bash
# All frontend tests
cd frontend && npm run test
# Watch mode
npm run test:watch
```
## VIII. VERIFIABLE HARNESS RULES
## VII. VERIFIABLE HARNESS RULES
For agentic development, a test harness is part of the task environment.
- Prefer real executable checks over narrative claims that a change is safe.
- Verify that the harness actually fails on the broken state and passes on the fixed state whenever feasible.
- Resist shortcut tests that bypass the real integration boundary the task is supposed to validate.
- When a production `@POST` guarantee is subtle, add the narrowest test that can falsify it.
## VIII. LONG-HORIZON QA MEMORY
## IX. LONG-HORIZON QA MEMORY
When multiple attempts are needed:
- Preserve the smallest set of failing fixtures, commands, and invariant mappings that explain the current gap.
- Fold older failed attempts into one bounded note describing what was tried and why it was rejected.
- Do not keep extending the active QA transcript with redundant command output.
## IX. TESTING SEARCH DISCIPLINE
## X. TESTING SEARCH DISCIPLINE
- Use one concrete failing hypothesis plus one verifier by default.
- Add alternative test strategies only when the first verifier is inconclusive.
- Do not mirror the implementation logic to fabricate expected values; use fixtures, explicit contracts, and invariant-oriented assertions.
## X. PYTEST CONVENTIONS & COMMAND EXAMPLES
```bash
# Run all tests
pytest
# Run a specific test module
pytest tests/test_users.py
# Run with coverage report
pytest --cov=src --cov-report=term-missing
# Run only tests matching a keyword
pytest -k "create_user"
# Run with verbose output and stop on first failure
pytest -xvs
```
**[SYSTEM: END OF TESTING DIRECTIVE. ENFORCE STRICT TRACEABILITY.]**
#endregion Std.Semantics.Testing