@probelabs/visor 0.1.174-ee → 0.1.175-ee

package/README.md

@@ -34,6 +34,7 @@ Visor is an open-source workflow engine that lets you define multi-step AI pipel
 | **Chat assistant / Bot** | [Bot Integrations](docs/bot-integrations.md) | [teams-assistant.yaml](examples/teams-assistant.yaml) |
 | **Run shell commands + AI** | [Command Provider](docs/command-provider.md) | [ai-with-bash.yaml](examples/ai-with-bash.yaml) |
 | **Connect MCP tools** | [MCP Provider](docs/mcp-provider.md) | [mcp-provider-example.yaml](examples/mcp-provider-example.yaml) |
+| **Add API integrations (TDD)** | [Guide: TDD Assistant Workflows](docs/guides/tdd-assistant-workflows.md) | [workable.tests.yaml](https://github.com/TykTechnologies/REFINE/blob/main/Oel/tests/workable.tests.yaml) |
 
 > **First time?** Run `npx visor init` to scaffold a working config, then `npx visor` to run it.
 
@@ -774,7 +775,7 @@ Learn more: [docs/enterprise-policy.md](docs/enterprise-policy.md)
 [Configuration](docs/configuration.md) · [AI config](docs/ai-configuration.md) · [CLI commands](docs/commands.md) · [GitHub Auth](docs/github-auth.md) · [CI/CLI mode](docs/ci-cli-mode.md) · [GitHub Action reference](docs/action-reference.md) · [Migration](docs/migration.md) · [FAQ](docs/faq.md) · [Glossary](docs/glossary.md)
 
 **Guides:**
-[Tools & Toolkits](docs/tools-and-toolkits.md) · [Assistant workflows](docs/assistant-workflows.md) · [Workflow creation](docs/workflow-creation-guide.md) · [Workflow style guide](docs/guides/workflow-style-guide.md) · [Dependencies](docs/dependencies.md) · [forEach propagation](docs/foreach-dependency-propagation.md) · [Failure routing](docs/failure-routing.md) · [Router patterns](docs/router-patterns.md) · [Lifecycle hooks](docs/lifecycle-hooks.md) · [Liquid templates](docs/liquid-templates.md) · [Schema-template system](docs/schema-templates.md) · [Fail conditions](docs/fail-if.md) · [Failure conditions schema](docs/failure-conditions-schema.md) · [Failure conditions impl](docs/failure-conditions-implementation.md) · [Timeouts](docs/timeouts.md) · [Execution limits](docs/limits.md) · [Event triggers](docs/event-triggers.md) · [Output formats](docs/output-formats.md) · [Output formatting](docs/output-formatting.md) · [Default output schema](docs/default-output-schema.md) · [Output history](docs/output-history.md) · [Reusable workflows](docs/workflows.md) · [Criticality modes](docs/guides/criticality-modes.md) · [Fault management](docs/guides/fault-management-and-contracts.md)
+[Tools & Toolkits](docs/tools-and-toolkits.md) · [Assistant workflows](docs/assistant-workflows.md) · [TDD for assistant workflows](docs/guides/tdd-assistant-workflows.md) · [Workflow creation](docs/workflow-creation-guide.md) · [Workflow style guide](docs/guides/workflow-style-guide.md) · [Dependencies](docs/dependencies.md) · [forEach propagation](docs/foreach-dependency-propagation.md) · [Failure routing](docs/failure-routing.md) · [Router patterns](docs/router-patterns.md) · [Lifecycle hooks](docs/lifecycle-hooks.md) · [Liquid templates](docs/liquid-templates.md) · [Schema-template system](docs/schema-templates.md) · [Fail conditions](docs/fail-if.md) · [Failure conditions schema](docs/failure-conditions-schema.md) · [Failure conditions impl](docs/failure-conditions-implementation.md) · [Timeouts](docs/timeouts.md) · [Execution limits](docs/limits.md) · [Event triggers](docs/event-triggers.md) · [Output formats](docs/output-formats.md) · [Output formatting](docs/output-formatting.md) · [Default output schema](docs/default-output-schema.md) · [Output history](docs/output-history.md) · [Reusable workflows](docs/workflows.md) · [Criticality modes](docs/guides/criticality-modes.md) · [Fault management](docs/guides/fault-management-and-contracts.md)
 
 **Providers:**
 [A2A](docs/a2a-provider.md) · [Command](docs/command-provider.md) · [Script](docs/script.md) · [MCP](docs/mcp-provider.md) · [MCP tools for AI](docs/mcp.md) · [Claude Code](docs/claude-code.md) · [AI custom tools](docs/ai-custom-tools.md) · [AI custom tools usage](docs/ai-custom-tools-usage.md) · [Custom tools](docs/custom-tools.md) · [GitHub ops](docs/github-ops.md) · [Git checkout](docs/providers/git-checkout.md) · [HTTP integration](docs/http.md) · [Memory](docs/memory.md) · [Human input](docs/human-input-provider.md) · [Custom providers](docs/pluggable.md)
@@ -783,7 +784,7 @@ Learn more: [docs/enterprise-policy.md](docs/enterprise-policy.md)
 [Security](docs/security.md) · [Performance](docs/performance.md) · [Observability](docs/observability.md) · [Debugging](docs/debugging.md) · [Debug visualizer](docs/debug-visualizer.md) · [Telemetry setup](docs/telemetry-setup.md) · [Dashboards](docs/dashboards/README.md) · [Troubleshooting](docs/troubleshooting.md) · [Suppressions](docs/suppressions.md) · [GitHub checks](docs/GITHUB_CHECKS.md) · [Bot integrations](docs/bot-integrations.md) · [Slack](docs/slack-integration.md) · [Telegram](docs/telegram-integration.md) · [Email](docs/email-integration.md) · [WhatsApp](docs/whatsapp-integration.md) · [Teams](docs/teams-integration.md) · [Scheduler](docs/scheduler.md) · [Sandbox engines](docs/sandbox-engines.md)
 
 **Testing:**
-[Getting started](docs/testing/getting-started.md) · [DSL reference](docs/testing/dsl-reference.md) · [Flows](docs/testing/flows.md) · [Fixtures & mocks](docs/testing/fixtures-and-mocks.md) · [Assertions](docs/testing/assertions.md) · [Cookbook](docs/testing/cookbook.md) · [CLI & reporters](docs/testing/cli.md) · [CI integration](docs/testing/ci.md) · [Troubleshooting](docs/testing/troubleshooting.md)
+[Getting started](docs/testing/getting-started.md) · [DSL reference](docs/testing/dsl-reference.md) · [Flows](docs/testing/flows.md) · [Fixtures & mocks](docs/testing/fixtures-and-mocks.md) · [Assertions](docs/testing/assertions.md) · [Cookbook](docs/testing/cookbook.md) · [TDD for assistants](docs/guides/tdd-assistant-workflows.md) · [CLI & reporters](docs/testing/cli.md) · [CI integration](docs/testing/ci.md) · [Troubleshooting](docs/testing/troubleshooting.md)
 
 **Enterprise:**
 [Licensing](docs/licensing.md) · [Enterprise policy](docs/enterprise-policy.md) · [Scheduler storage](docs/scheduler-storage.md) · [Database operations](docs/database-operations.md) · [Capacity planning](docs/capacity-planning.md) · [Production deployment](docs/production-deployment.md) · [Deployment](docs/DEPLOYMENT.md)

package/dist/cli-main.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"","sourceRoot":"","sources":["file:///home/runner/work/visor/visor/src/cli-main.ts"],"names":[],"mappings":"AAirCA;;GAEG;AACH,wBAAsB,IAAI,IAAI,OAAO,CAAC,IAAI,CAAC,CA4gE1C"}
+{"version":3,"file":"","sourceRoot":"","sources":["file:///home/runner/work/visor/visor/src/cli-main.ts"],"names":[],"mappings":"AAkrCA;;GAEG;AACH,wBAAsB,IAAI,IAAI,OAAO,CAAC,IAAI,CAAC,CA4gE1C"}

package/dist/docs/guides/tdd-assistant-workflows.md

@@ -0,0 +1,519 @@
+# Test-Driven Development for Assistant Workflows
+
+Build and iterate on AI assistant workflows using visor's test framework. This guide covers the full cycle: define your workflow, write tests with mocks, then run against real AI to iterate on prompt quality and assertions.
+
+## The TDD Cycle
+
+1. **Define the workflow** — skills, tools, knowledge, intents
+2. **Write tests with mocks** — expected conversations and assertions
+3. **Run with mocks** — validate structure, routing, and assertion logic
+4. **Run with `--no-mocks`** — real AI + real tools, iterate on quality
+5. **Refine** — fix prompts, relax over-strict assertions, improve knowledge
+
+## Setting Up
+
+### Project Structure
+
+```
+my-assistant/
+├── assistant.yaml           # main workflow config
+├── config/
+│   └── skills.yaml          # skill definitions
+├── docs/
+│   └── api-reference.md     # knowledge docs for skills
+├── tests/
+│   └── skills.tests.yaml    # test file
+└── .env                     # API tokens (not committed)
+```
+
+### Test File Basics
+
+Test files extend your main config:
+
+```yaml
+# tests/skills.tests.yaml
+version: "1.0"
+extends: "../assistant.yaml"
+
+tests:
+  defaults:
+    strict: false    # required for --no-mocks (internal steps run without expectations)
+
+  cases:
+    - name: basic-question
+      conversation:
+        routing: { max_loops: 2 }
+        turns:
+          - role: user
+            text: "What services do we run?"
+            mocks:
+              chat:
+                text: "We run 3 services: API gateway, dashboard, and pump."
+                intent: chat
+                skills: []
+            expect:
+              outputs:
+                - step: chat
+                  path: text
+                  matches: "(?i)gateway|dashboard|pump"
+```
+
+Key fields:
+- **`extends`** — imports the full workflow so the test runner knows all steps, skills, and routing
+- **`strict: false`** — prevents failures from internal steps (routing, config building) that don't have assertions
+- **`conversation`** — sugar syntax that auto-expands turns into flow stages with message history
+
+## Writing Conversation Tests
+
+### Single-Turn Test
+
+The simplest test: one user message, assert on the response.
+
+```yaml
+- name: greeting
+  conversation:
+    routing: { max_loops: 2 }
+    turns:
+      - role: user
+        text: "Hello, who are you?"
+        mocks:
+          chat:
+            text: "I'm your engineering assistant. I can help with code, deployments, and more."
+            intent: chat
+            skills: []
+        expect:
+          calls:
+            - step: chat
+              exactly: 1
+          outputs:
+            - step: chat
+              path: text
+              matches: "(?i)assistant|help"
+```
+
+### Multi-Turn Conversation
+
+Each turn's history automatically includes previous turns. Mock response text becomes assistant messages in subsequent turns.
+
+```yaml
+- name: code-explore-then-explain
+  conversation:
+    routing: { max_loops: 2 }
+    turns:
+      - role: user
+        text: "Find the authentication middleware in the backend service"
+        mocks:
+          chat:
+            text: "Found `auth.go` in `internal/middleware/`. It checks JWT tokens."
+            intent: chat
+            skills: [code-explorer]
+        expect:
+          outputs:
+            - step: chat
+              path: text
+              matches: "(?i)auth|middleware"
+
+      - role: user
+        text: "Explain how the JWT validation works in that auth middleware you found"
+        mocks:
+          chat:
+            text: "The middleware extracts the Bearer token, validates the signature..."
+            intent: chat
+            skills: [code-explorer]
+        expect:
+          llm_judge:
+            - step: chat
+              turn: current
+              path: text
+              prompt: "Does the response explain JWT validation with technical details?"
+              schema:
+                properties:
+                  explains_jwt:
+                    type: boolean
+                required: [explains_jwt]
+              assert:
+                explains_jwt: true
+```
+
+### Mock Response Format
+
+Mocks simulate what the `chat` step returns:
+
+```yaml
+mocks:
+  chat:
+    text: "The response text..."
+    intent: chat              # which intent was classified
+    skills: [code-explorer]   # which skills were activated
+```
+
+Use **fictional data** in mocks. The mock text becomes the assistant message in subsequent turn history.
+
+## Assertion Types
+
+### Regex Matching (`outputs`)
+
+Pattern-match on response fields:
+
+```yaml
+expect:
+  outputs:
+    - step: chat
+      path: text
+      matches: "(?i)kubernetes|k8s"
+    - step: chat
+      turn: current        # only check this turn's output
+      path: text
+      matches: "(?i)deploy"
+```
+
+### Call Counting (`calls`)
+
+Assert how many times a step ran:
+
+```yaml
+expect:
+  calls:
+    - step: chat
+      exactly: 1
+```
+
+### LLM Judge (`llm_judge`)
+
+Semantic evaluation — assert on meaning, not exact text. This is the most powerful assertion for AI responses:
+
+```yaml
+expect:
+  llm_judge:
+    - step: chat
+      turn: current
+      path: text
+      prompt: |
+        Does the response provide a clear architectural overview
+        with component names and their responsibilities?
+      schema:
+        properties:
+          names_components:
+            type: boolean
+            description: "Names specific components or services?"
+          explains_responsibilities:
+            type: boolean
+            description: "Explains what each component does?"
+        required: [names_components, explains_responsibilities]
+      assert:
+        names_components: true
+        explains_responsibilities: true
+```
+
+The judge returns a JSON object matching your schema. `assert` checks specific fields. You can include fields in the schema for observability without asserting them.
+
+### Cross-Turn Assertions
+
+Reference previous turns using `turn: N` (1-based):
+
+```yaml
+# In turn 2, verify turn 1 was good too
+expect:
+  llm_judge:
+    - step: chat
+      turn: current
+      path: text
+      prompt: "Does turn 2 build on the context from turn 1?"
+      ...
+    - step: chat
+      turn: 1
+      path: text
+      prompt: "Did turn 1 provide a good foundation?"
+      ...
+```
+
+## Running Tests
+
+```bash
+# Validate structure (no AI calls)
+visor test --config tests/skills.tests.yaml
+
+# Run a single case
+visor test --config tests/skills.tests.yaml --case basic-question
+
+# Run with real AI and real tools
+visor test --config tests/skills.tests.yaml --no-mocks
+
+# Real AI, single case
+visor test --config tests/skills.tests.yaml --case basic-question --no-mocks
+
+# Debug mode
+visor test --config tests/skills.tests.yaml --case basic-question --no-mocks --debug
+```
+
+### Mock vs No-Mock Mode
+
+| | Mock mode | `--no-mocks` |
+|--|-----------|-------------|
+| AI calls | Mocked responses | Real AI provider |
+| Tools | Not called | Real tool execution |
+| `routing.max_loops` | Use `0` | Use `2+` (AI needs iterations for tool calls) |
+| `strict` | Can be `true` | Must be `false` (internal steps fire) |
+| Speed | Fast (seconds) | Slow (AI latency + tool calls) |
+| Use for | Structure validation, CI | Prompt quality iteration |
+
+## Iterating with `--no-mocks`
+
+This is where the real work happens. Common issues and fixes:
+
+### AI ignores tool guidance
+
+**Symptom:** AI calls wrong endpoint, uses wrong arguments, or skips required steps.
+
+**Fix:** Improve the knowledge doc with explicit instructions:
+
+```yaml
+# In your skill knowledge:
+knowledge: |
+  ### Important: Always search by project ID
+  When looking up items, **always** use `/projects/{id}/items`
+  — never the global `/items` endpoint which returns all projects.
+```
+
+Also make user prompts more specific:
+
+```yaml
+# Before (too vague):
+text: "List the items in review"
+
+# After (explicit):
+text: "List the items in review stage for the Backend project"
+```
+
+### Assertion too strict for real responses
+
+**Symptom:** Mock includes specific details but real AI response formats them differently.
+
+**Fix:** Keep the field in the schema for observability but remove from `assert`:
+
+```yaml
+schema:
+  properties:
+    lists_items:
+      type: boolean
+    includes_links:
+      type: boolean
+      description: "Includes profile links?"
+  required: [lists_items, includes_links]
+assert:
+  lists_items: true
+  # includes_links intentionally not asserted — real API
+  # doesn't always return this without extra calls
+```
+
+### Turn 2 loses context from Turn 1
+
+**Symptom:** AI asks "which items?" instead of referencing turn 1 results.
+
+**Fix:** Make follow-up prompts self-contained:
+
+```yaml
+# Before:
+text: "Now compare these candidates"
+
+# After:
+text: "Now compare the 3 candidates you just evaluated for the SRE role"
+```
+
+### `strict: false` not working
+
+**Symptom:** Test fails with "Step executed without expect: chat.route-intent".
+
+**Fix:** `strict: false` must be at the **test case level** or in `tests.defaults`, not inside `conversation:`:
+
+```yaml
+# Wrong — ignored by conversation sugar:
+conversation:
+  strict: false
+  turns: ...
+
+# Correct — applied to expanded flow stages:
+- name: my-test
+  strict: false
+  conversation:
+    turns: ...
+
+# Or set globally:
+tests:
+  defaults:
+    strict: false
+```
+
+## Example: Adding an API Integration Skill
+
+Here's a complete example adding an external REST API as a skill.
+
+### 1. Define the Skill
+
+```yaml
+# config/skills.yaml
+- id: hr-system
+  description: |
+    request relates to recruiting, hiring pipeline, candidates,
+    job postings, or HR pipeline management.
+    Examples: "list candidates", "show open positions", "evaluate candidate"
+  tools:
+    hr-api:
+      type: http_client
+      base_url: "https://api.hr-system.com/v3"
+      auth:
+        type: bearer
+        token: "${HR_API_TOKEN}"
+      headers:
+        Content-Type: "application/json"
+  knowledge: |
+    {% readfile "docs/hr-api-reference.md" %}
+```
+
+### 2. Write the Knowledge Doc
+
+```markdown
+## HR API Reference
+
+Call the `hr-api` tool with these arguments:
+
+| Argument | Required | Description |
+|----------|----------|-------------|
+| `path`   | yes      | API path (e.g. `/jobs`, `/candidates/{id}`) |
+| `method` | no       | HTTP method (default: `GET`) |
+| `query`  | no       | Query parameters |
+| `body`   | no       | Request body for POST/PUT |
+
+### Endpoints
+
+| Operation | Method | Path |
+|-----------|--------|------|
+| List jobs | GET | `/jobs` |
+| List candidates | GET | `/jobs/{id}/candidates` |
+| Get candidate | GET | `/jobs/{id}/candidates/{cid}` |
+
+### Important
+Always use job-specific endpoints (`/jobs/{id}/candidates`)
+— never the global `/candidates` endpoint.
+```
+
+### 3. Write Tests
+
+```yaml
+# tests/hr.tests.yaml
+version: "1.0"
+extends: "../assistant.yaml"
+
+tests:
+  defaults:
+    strict: false
+
+  cases:
+    - name: hr-pipeline-stats
+      description: "Show candidate counts per pipeline stage"
+      conversation:
+        routing: { max_loops: 2 }
+        turns:
+          - role: user
+            text: "Show me candidate statistics per stage for the SRE role"
+            mocks:
+              chat:
+                text: |
+                  Pipeline for **Site Reliability Engineer**:
+                  | Stage | Count |
+                  |-------|-------|
+                  | Sourced | 12 |
+                  | Applied | 8 |
+                  | Screening | 5 |
+                intent: chat
+                skills: [hr-system]
+            expect:
+              outputs:
+                - step: chat
+                  path: text
+                  matches: "(?i)sourced|applied|screen"
+              llm_judge:
+                - step: chat
+                  turn: current
+                  path: text
+                  prompt: |
+                    Does the response show candidates broken down
+                    by pipeline stage with numbers?
+                  schema:
+                    properties:
+                      has_stage_breakdown:
+                        type: boolean
+                      covers_multiple_stages:
+                        type: boolean
+                  assert:
+                    has_stage_breakdown: true
+                    covers_multiple_stages: true
+```
+
+### 4. Iterate
+
+```bash
+# First: validate test structure
+visor test --config tests/hr.tests.yaml
+
+# Then: run against real AI + API
+visor test --config tests/hr.tests.yaml --no-mocks
+
+# Iterate on a specific failing case
+visor test --config tests/hr.tests.yaml --case hr-pipeline-stats --no-mocks --debug
+```
+
+Each iteration typically involves:
+1. Run `--no-mocks` and read the failure
+2. Fix the knowledge doc (wrong endpoint? missing guidance?) or the user prompt (too vague?)
+3. Relax assertions that are too strict for real responses
+4. Re-run until green
+
+## Example: MCP Command Tool Skill
+
+Skills can also use external MCP servers:
+
+```yaml
+# config/skills.yaml
+- id: jira
+  description: |
+    request relates to Jira issues, tickets, sprints, or project tracking.
+  tools:
+    atlassian:
+      command: uvx
+      args: [mcp-atlassian]
+      env:
+        JIRA_URL: "${JIRA_URL}"
+        JIRA_API_TOKEN: "${JIRA_API_TOKEN}"
+      allowedMethods: [jira_get_issue, jira_search, jira_create_issue]
+  knowledge: |
+    You have access to Jira via the atlassian MCP tool.
+    Use `jira_search` with JQL queries to find issues.
+    Use `jira_get_issue` with an issue key like `PROJ-123`.
+```
+
+Test the same way — conversation sugar works identically regardless of tool type.
+
+## Checklist
+
+When adding a new skill with tests:
+
+- [ ] Add skill to `config/skills.yaml` with description, tools, and knowledge
+- [ ] Write knowledge doc in `docs/` (be explicit about which endpoints/methods to use)
+- [ ] Add secrets to `.env`
+- [ ] Create `tests/<skill>.tests.yaml` with `extends`
+- [ ] Write first test case with mocks — run `visor test` to validate
+- [ ] Run with `--no-mocks` — iterate on knowledge doc and prompts
+- [ ] Add multi-turn tests for complex flows
+- [ ] Relax assertions that are too strict for real responses
+
+## Related Documentation
+
+- [Getting Started](../testing/getting-started.md) — test framework basics
+- [DSL Reference](../testing/dsl-reference.md) — complete test YAML schema
+- [Assertions](../testing/assertions.md) — all assertion types including LLM judge
+- [Flows](../testing/flows.md) — multi-stage tests and conversation sugar
+- [Fixtures and Mocks](../testing/fixtures-and-mocks.md) — mock format reference
+- [Cookbook](../testing/cookbook.md) — copy-pasteable test recipes
+- [Assistant Workflows](../assistant-workflows.md) — skills, intents, and tool types
+- [Tools & Toolkits](../tools-and-toolkits.md) — tool definition reference

package/dist/docs/testing/dsl-reference.md

@@ -29,6 +29,16 @@ tests:
     tags: "local,fast"         # or [local, fast]
     exclude_tags: "experimental,slow"  # or [experimental, slow]
 
+  hooks:                        # (optional) lifecycle hooks
+    before_all:
+      exec: <shell-command>     # runs once before all cases
+    after_all:
+      exec: <shell-command>     # runs once after all cases (always)
+    before_each:
+      exec: <shell-command>     # runs before each case
+    after_each:
+      exec: <shell-command>     # runs after each case (always)
+
   fixtures: []                  # (optional) suite-level custom fixtures
 
   cases:
@@ -37,6 +47,13 @@ tests:
       skip: false|true
       ai_include_code_context: false  # per-case override
 
+      hooks:                       # (optional) per-case lifecycle hooks
+        before:
+          exec: <shell-command>  # runs before this case
+        after:
+          exec: <shell-command>  # runs after this case (always)
+          timeout: 10000         # optional timeout in ms
+
       # Single-event case
       event: pr_opened | pr_updated | pr_closed | issue_opened | issue_comment | manual
       fixture: <builtin|{ builtin, overrides }>
@@ -85,6 +102,82 @@ tests:
             error_code: 500
 ```
 
+## Lifecycle Hooks
+
+Hooks let you run shell commands at key points in the test lifecycle — useful for seeding databases, starting servers, or cleaning up test data.
+
+### Suite-level hooks
+
+Defined under `tests.hooks`:
+
+```yaml
+tests:
+  hooks:
+    before_all:
+      exec: npx tsx test-data/seed-db.ts
+    after_all:
+      exec: npx tsx test-data/clean-db.ts
+    before_each:
+      exec: npx tsx test-data/reset-state.ts
+    after_each:
+      exec: npx tsx test-data/cleanup-case.ts
+  cases: [...]
+```
+
+| Hook | When | Runs |
+|------|------|------|
+| `before_all` | Once before any case | If it fails, all cases are skipped |
+| `after_all` | Once after all cases | Always runs (like `finally`) |
+| `before_each` | Before every case | If it fails, that case is skipped |
+| `after_each` | After every case | Always runs (like `finally`) |
+
+### Case-level hooks
+
+Defined under `case.hooks`:
+
+```yaml
+cases:
+  - name: update-settlement
+    hooks:
+      before:
+        exec: npx tsx test-data/seed-db.ts --case update-settlement
+      after:
+        exec: npx tsx test-data/seed-db.ts --clean
+        timeout: 10000   # optional, default 30000ms
+    event: manual
+    mocks: { ... }
+```
+
+| Hook | When | Runs |
+|------|------|------|
+| `before` | Before this specific case (after `before_each`) | If it fails, the case is skipped |
+| `after` | After this specific case (before `after_each`) | Always runs (like `finally`) |
+
+### Hook properties
+
+| Property | Type | Required | Description |
+|----------|------|----------|-------------|
+| `exec` | string | yes | Shell command to run |
+| `timeout` | number | no | Timeout in ms (default: 30000) |
+
+### Execution order
+
+For each case, hooks run in this order:
+
+1. `before_each` (suite)
+2. `before` (case)
+3. *test execution*
+4. `after` (case)
+5. `after_each` (suite)
+
+Hooks inherit all environment variables from the parent process, so seed scripts can use the same `DB_PATH`, API keys, etc. that your checks use.
+
+### Error handling
+
+- If `before_all` fails → all cases are skipped and reported as failed
+- If `before_each` or `before` fails → that case is skipped and reported as failed
+- `after`, `after_each`, and `after_all` always run, even if the test or a prior hook failed
+
 ## Fixtures
 
 - Built-in GitHub fixtures: `gh.pr_open.minimal`, `gh.pr_sync.minimal`, `gh.pr_closed.minimal`, `gh.issue_open.minimal`, `gh.issue_comment.standard`, `gh.issue_comment.visor_help`, `gh.issue_comment.visor_regenerate`.