@probelabs/visor 0.1.124 → 0.1.126
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/config.d.ts.map +1 -1
- package/dist/docs/DEPLOYMENT.md +117 -11
- package/dist/docs/GITHUB_CHECKS.md +18 -4
- package/dist/docs/NPM_USAGE.md +112 -39
- package/dist/docs/action-reference.md +63 -9
- package/dist/docs/advanced-ai.md +58 -51
- package/dist/docs/ai-configuration.md +99 -11
- package/dist/docs/ai-custom-tools-usage.md +70 -33
- package/dist/docs/ai-custom-tools.md +50 -27
- package/dist/docs/architecture.md +1232 -0
- package/dist/docs/bot-transports-rfc.md +13 -3
- package/dist/docs/ci-cli-mode.md +116 -8
- package/dist/docs/claude-code.md +111 -41
- package/dist/docs/command-provider.md +37 -15
- package/dist/docs/commands.md +252 -6
- package/dist/docs/configuration.md +138 -4
- package/dist/docs/contributing.md +737 -0
- package/dist/docs/custom-tools.md +39 -8
- package/dist/docs/dashboards/README.md +33 -19
- package/dist/docs/debug-visualizer-progress.md +14 -13
- package/dist/docs/debug-visualizer-rfc.md +14 -13
- package/dist/docs/debug-visualizer.md +30 -5
- package/dist/docs/debugging.md +73 -8
- package/dist/docs/default-output-schema.md +24 -20
- package/dist/docs/dependencies.md +75 -21
- package/dist/docs/dev-playbook.md +85 -9
- package/dist/docs/engine-pause-resume-rfc.md +11 -11
- package/dist/docs/engine-state-machine-plan.md +10 -3
- package/dist/docs/event-driven-github-integration-rfc.md +20 -11
- package/dist/docs/event-triggers.md +95 -6
- package/dist/docs/execution-statistics-rfc.md +16 -4
- package/dist/docs/fact-validator-gap-analysis.md +12 -1
- package/dist/docs/fact-validator-implementation-plan.md +19 -11
- package/dist/docs/fail-if.md +116 -11
- package/dist/docs/failure-conditions-implementation.md +40 -6
- package/dist/docs/failure-conditions-schema.md +243 -87
- package/dist/docs/failure-routing-rfc.md +43 -18
- package/dist/docs/failure-routing.md +80 -23
- package/dist/docs/faq.md +836 -0
- package/dist/docs/foreach-dependency-propagation.md +32 -15
- package/dist/docs/github-ops.md +6 -5
- package/dist/docs/glossary.md +322 -0
- package/dist/docs/goto-forward-run-plan.md +23 -10
- package/dist/docs/guides/criticality-modes.md +15 -13
- package/dist/docs/guides/fault-management-and-contracts.md +8 -5
- package/dist/docs/guides/workflow-style-guide.md +17 -8
- package/dist/docs/http.md +102 -3
- package/dist/docs/human-input-provider.md +20 -36
- package/dist/docs/index.md +206 -0
- package/dist/docs/lifecycle-hooks.md +322 -2
- package/dist/docs/limits.md +20 -5
- package/dist/docs/liquid-templates.md +86 -14
- package/dist/docs/loop-routing-refactor.md +4 -2
- package/dist/docs/mcp-provider.md +53 -19
- package/dist/docs/mcp.md +27 -1
- package/dist/docs/memory.md +7 -2
- package/dist/docs/migration.md +596 -0
- package/dist/docs/observability.md +227 -6
- package/dist/docs/output-formats.md +388 -9
- package/dist/docs/output-history.md +36 -6
- package/dist/docs/performance.md +510 -4
- package/dist/docs/pluggable.md +95 -4
- package/dist/docs/proposals/snapshot-scope-execution.md +6 -5
- package/dist/docs/providers/git-checkout.md +16 -14
- package/dist/docs/providers/noop.md +696 -0
- package/dist/docs/recipes.md +8 -9
- package/dist/docs/rfc/git-checkout-step.md +3 -1
- package/dist/docs/rfc/on_init-hook.md +18 -5
- package/dist/docs/rfc/workspace-isolation.md +16 -0
- package/dist/docs/roadmap/criticality-implementation-tasks.md +27 -27
- package/dist/docs/router-patterns.md +155 -43
- package/dist/docs/schema-templates.md +51 -15
- package/dist/docs/script.md +162 -13
- package/dist/docs/sdk.md +46 -12
- package/dist/docs/security.md +464 -5
- package/dist/docs/slack-integration.md +481 -0
- package/dist/docs/tag-filtering.md +60 -20
- package/dist/docs/telemetry-setup.md +157 -46
- package/dist/docs/test-framework-rfc.md +37 -36
- package/dist/docs/testing/assertions.md +92 -4
- package/dist/docs/testing/ci.md +56 -7
- package/dist/docs/testing/cli.md +57 -15
- package/dist/docs/testing/cookbook.md +53 -20
- package/dist/docs/testing/dsl-reference.md +110 -9
- package/dist/docs/testing/fixtures-and-mocks.md +28 -3
- package/dist/docs/testing/flows.md +59 -4
- package/dist/docs/testing/getting-started.md +14 -13
- package/dist/docs/testing/troubleshooting.md +39 -2
- package/dist/docs/timeouts.md +174 -18
- package/dist/docs/troubleshooting.md +176 -6
- package/dist/docs/workflow-creation-guide.md +101 -3
- package/dist/docs/workflows.md +138 -41
- package/dist/examples/README.md +169 -4
- package/dist/examples/ai-custom-tools-simple.yaml +2 -3
- package/dist/examples/cron-webhook-config.yaml +15 -0
- package/dist/examples/forEach-example.yaml +6 -0
- package/dist/examples/git-checkout-basic.yaml +4 -0
- package/dist/examples/git-checkout-compare.yaml +6 -0
- package/dist/examples/git-checkout-cross-repo.yaml +7 -0
- package/dist/examples/http-integration-config.yaml +30 -0
- package/dist/examples/https-server-config.yaml +15 -0
- package/dist/examples/mcp-provider-example.yaml +10 -10
- package/dist/examples/transform-example.yaml +3 -0
- package/dist/examples/webhook-pipeline-config.yaml +18 -0
- package/dist/examples/workflows/workflow-composition-example.yaml +4 -0
- package/dist/frontends/slack-frontend.d.ts +2 -0
- package/dist/frontends/slack-frontend.d.ts.map +1 -1
- package/dist/generated/config-schema.d.ts +11 -7
- package/dist/generated/config-schema.d.ts.map +1 -1
- package/dist/generated/config-schema.json +11 -7
- package/dist/index.js +3127 -974
- package/dist/output/traces/{run-2026-01-28T16-15-24-569Z.ndjson → run-2026-01-31T16-37-22-321Z.ndjson} +84 -84
- package/dist/output/traces/{run-2026-01-28T16-16-09-757Z.ndjson → run-2026-01-31T16-38-06-031Z.ndjson} +1013 -1013
- package/dist/providers/ai-check-provider.d.ts +9 -2
- package/dist/providers/ai-check-provider.d.ts.map +1 -1
- package/dist/providers/command-check-provider.d.ts.map +1 -1
- package/dist/providers/mcp-custom-sse-server.d.ts +17 -1
- package/dist/providers/mcp-custom-sse-server.d.ts.map +1 -1
- package/dist/providers/workflow-check-provider.d.ts.map +1 -1
- package/dist/providers/workflow-tool-executor.d.ts +68 -0
- package/dist/providers/workflow-tool-executor.d.ts.map +1 -0
- package/dist/sdk/{check-provider-registry-AQ3JETBG.mjs → check-provider-registry-3KI5RKXT.mjs} +6 -5
- package/dist/sdk/check-provider-registry-IYILYY35.mjs +28 -0
- package/dist/sdk/chunk-2CPMMNIX.mjs +1459 -0
- package/dist/sdk/chunk-2CPMMNIX.mjs.map +1 -0
- package/dist/sdk/chunk-5LI6T4O3.mjs +3600 -0
- package/dist/sdk/chunk-5LI6T4O3.mjs.map +1 -0
- package/dist/sdk/{chunk-YLQ4UN62.mjs → chunk-A4PGHURG.mjs} +6838 -6257
- package/dist/sdk/chunk-A4PGHURG.mjs.map +1 -0
- package/dist/sdk/chunk-EXFGO4FX.mjs +147 -0
- package/dist/sdk/chunk-EXFGO4FX.mjs.map +1 -0
- package/dist/sdk/chunk-PJ7K5UFC.mjs +17732 -0
- package/dist/sdk/chunk-PJ7K5UFC.mjs.map +1 -0
- package/dist/sdk/{chunk-BHZ4CKUS.mjs → chunk-PXFIALUH.mjs} +77 -8
- package/dist/sdk/chunk-PXFIALUH.mjs.map +1 -0
- package/dist/sdk/{chunk-PVITVJ6J.mjs → chunk-RTKJXNZS.mjs} +32 -9
- package/dist/sdk/chunk-RTKJXNZS.mjs.map +1 -0
- package/dist/sdk/chunk-VW2GBXQT.mjs +606 -0
- package/dist/sdk/chunk-VW2GBXQT.mjs.map +1 -0
- package/dist/sdk/{config-RQQPMLRD.mjs → config-5AUYQFHE.mjs} +2 -2
- package/dist/sdk/config-6CUVEH7H.mjs +16 -0
- package/dist/sdk/config-6CUVEH7H.mjs.map +1 -0
- package/dist/sdk/{github-frontend-6Q4BISZX.mjs → github-frontend-BZ4N3BFZ.mjs} +7 -3
- package/dist/sdk/github-frontend-BZ4N3BFZ.mjs.map +1 -0
- package/dist/sdk/host-4MT3EW2I.mjs +52 -0
- package/dist/sdk/{host-P5NQICP7.mjs → host-NYWXLIFC.mjs} +2 -2
- package/dist/sdk/host-NYWXLIFC.mjs.map +1 -0
- package/dist/sdk/{routing-DEY2AIXM.mjs → routing-6R42GXUO.mjs} +2 -2
- package/dist/sdk/routing-6R42GXUO.mjs.map +1 -0
- package/dist/sdk/routing-7FXPULTO.mjs +24 -0
- package/dist/sdk/routing-7FXPULTO.mjs.map +1 -0
- package/dist/sdk/sdk.d.mts +3 -1
- package/dist/sdk/sdk.d.ts +3 -1
- package/dist/sdk/sdk.js +12163 -11204
- package/dist/sdk/sdk.js.map +1 -1
- package/dist/sdk/sdk.mjs +14 -10
- package/dist/sdk/sdk.mjs.map +1 -1
- package/dist/sdk/slack-frontend-JUT3TYVC.mjs +821 -0
- package/dist/sdk/slack-frontend-JUT3TYVC.mjs.map +1 -0
- package/dist/sdk/workflow-check-provider-H3CUOLUD.mjs +28 -0
- package/dist/sdk/workflow-check-provider-H3CUOLUD.mjs.map +1 -0
- package/dist/sdk/workflow-check-provider-YUNNF4KC.mjs +28 -0
- package/dist/sdk/workflow-check-provider-YUNNF4KC.mjs.map +1 -0
- package/dist/sdk/workflow-registry-KFWSDSLM.mjs +12 -0
- package/dist/sdk/workflow-registry-KFWSDSLM.mjs.map +1 -0
- package/dist/slack/socket-runner.d.ts +2 -0
- package/dist/slack/socket-runner.d.ts.map +1 -1
- package/dist/state-machine/context/workflow-inputs.d.ts +20 -0
- package/dist/state-machine/context/workflow-inputs.d.ts.map +1 -0
- package/dist/state-machine/dispatch/execution-invoker.d.ts.map +1 -1
- package/dist/state-machine/dispatch/foreach-processor.d.ts.map +1 -1
- package/dist/state-machine/dispatch/stats-manager.d.ts.map +1 -1
- package/dist/state-machine/states/level-dispatch.d.ts.map +1 -1
- package/dist/state-machine/states/routing.d.ts +2 -1
- package/dist/state-machine/states/routing.d.ts.map +1 -1
- package/dist/traces/{run-2026-01-28T16-15-24-569Z.ndjson → run-2026-01-31T16-37-22-321Z.ndjson} +84 -84
- package/dist/traces/{run-2026-01-28T16-16-09-757Z.ndjson → run-2026-01-31T16-38-06-031Z.ndjson} +1013 -1013
- package/dist/types/config.d.ts +3 -1
- package/dist/types/config.d.ts.map +1 -1
- package/dist/utils/human-id.d.ts +12 -0
- package/dist/utils/human-id.d.ts.map +1 -0
- package/dist/utils/worktree-manager.d.ts +3 -0
- package/dist/utils/worktree-manager.d.ts.map +1 -1
- package/dist/workflow-executor.d.ts.map +1 -1
- package/dist/workflow-registry.d.ts +1 -0
- package/dist/workflow-registry.d.ts.map +1 -1
- package/package.json +2 -2
- package/dist/sdk/chunk-BHZ4CKUS.mjs.map +0 -1
- package/dist/sdk/chunk-PVITVJ6J.mjs.map +0 -1
- package/dist/sdk/chunk-YLQ4UN62.mjs.map +0 -1
- package/dist/sdk/github-frontend-6Q4BISZX.mjs.map +0 -1
- /package/dist/sdk/{check-provider-registry-AQ3JETBG.mjs.map → check-provider-registry-3KI5RKXT.mjs.map} +0 -0
- /package/dist/sdk/{config-RQQPMLRD.mjs.map → check-provider-registry-IYILYY35.mjs.map} +0 -0
- /package/dist/sdk/{routing-DEY2AIXM.mjs.map → config-5AUYQFHE.mjs.map} +0 -0
- /package/dist/sdk/{host-P5NQICP7.mjs.map → host-4MT3EW2I.mjs.map} +0 -0
package/dist/docs/testing/cli.md
CHANGED
|
@@ -5,37 +5,79 @@ Run integration tests for your Visor config using the built-in `test` subcommand
|
|
|
5
5
|
## Commands
|
|
6
6
|
|
|
7
7
|
- Discover tests file and list cases
|
|
8
|
-
- `visor test --list [--config defaults
|
|
8
|
+
- `visor test --list [--config defaults/visor.tests.yaml]`
|
|
9
9
|
- Run cases
|
|
10
|
-
- `visor test [--config defaults
|
|
10
|
+
- `visor test [--config defaults/visor.tests.yaml] [--only <substring>] [--bail]`
|
|
11
11
|
- Validate tests YAML without running
|
|
12
|
-
- `visor test --validate [--config defaults
|
|
12
|
+
- `visor test --validate [--config defaults/visor.tests.yaml]`
|
|
13
|
+
|
|
14
|
+
## Auto-Discovery
|
|
15
|
+
|
|
16
|
+
When no `--config` is provided, the test runner searches for test files in the following order:
|
|
17
|
+
|
|
18
|
+
1. `defaults/visor.tests.yaml` or `defaults/visor.tests.yml`
|
|
19
|
+
2. `.visor.tests.yaml` or `.visor.tests.yml` in the project root
|
|
20
|
+
|
|
21
|
+
You can also pass a directory or glob pattern as a positional argument to discover multiple test suites:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
visor test defaults/ # Run all suites in defaults/
|
|
25
|
+
visor test "**/*.tests.yaml" # Run all matching suites
|
|
26
|
+
```
|
|
13
27
|
|
|
14
28
|
## Flags
|
|
15
29
|
|
|
16
|
-
|
|
30
|
+
### Core Flags
|
|
31
|
+
|
|
32
|
+
- `--config <path>`: Path to `.visor.tests.yaml` (auto-discovers if not specified).
|
|
17
33
|
- `--only <filter>`: Run cases whose `name` contains the substring (case-insensitive).
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
- Examples: `--only pr-review-e2e-flow#facts-invalid`, `--only pr-review-e2e-flow#3` (1‑based index)
|
|
34
|
+
- Stage filter: append `#<stage>` to run only a flow stage.
|
|
35
|
+
- Examples: `--only pr-review-e2e-flow#facts-invalid`, `--only pr-review-e2e-flow#3` (1-based index)
|
|
21
36
|
- `--bail`: Stop on first failure.
|
|
22
|
-
- `--
|
|
23
|
-
- `--
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
37
|
+
- `--list`: List discovered test cases without running them.
|
|
38
|
+
- `--validate`: Validate tests YAML syntax without running.
|
|
39
|
+
|
|
40
|
+
### Parallelism
|
|
41
|
+
|
|
42
|
+
- `--max-parallel <N>`: Run up to N test cases concurrently within a suite (default: 1).
|
|
43
|
+
- `--max-suites <N>`: Run up to N test suites concurrently when discovering multiple files (default: number of CPUs).
|
|
44
|
+
|
|
45
|
+
### Output & Reporting
|
|
46
|
+
|
|
47
|
+
- `--json <path|->`: Write a minimal JSON summary (`-` for stdout).
|
|
48
|
+
- `--report junit:<path>`: Write a JUnit XML report.
|
|
49
|
+
- `--summary md:<path>`: Write a Markdown summary.
|
|
50
|
+
- `--progress compact|detailed`: Progress verbosity (default: compact).
|
|
27
51
|
- `--prompt-max-chars <N>`: Truncate captured prompt text to N characters.
|
|
28
52
|
|
|
53
|
+
### Debugging
|
|
54
|
+
|
|
55
|
+
- `--debug`: Enable debug mode for verbose output (equivalent to `VISOR_DEBUG=true`).
|
|
56
|
+
- `--no-mocks`: Run tests without mock injection. Real providers execute and outputs are printed as suggested mocks.
|
|
57
|
+
|
|
29
58
|
## Output
|
|
30
59
|
|
|
31
60
|
- Per-case PASS/FAIL lines
|
|
32
61
|
- Coverage table (expected vs actual step runs)
|
|
33
|
-
- Summary totals
|
|
62
|
+
- Summary totals (Jest-style format)
|
|
34
63
|
|
|
35
64
|
## Tips
|
|
36
65
|
|
|
37
66
|
- Use `--validate` when iterating on tests to catch typos early.
|
|
38
67
|
- Keep `strict: true` in `tests.defaults` to surface missing `expect` quickly.
|
|
39
68
|
- For large suites, increase `--max-parallel` to improve throughput.
|
|
40
|
-
-
|
|
41
|
-
|
|
69
|
+
- Use `--no-mocks` to capture real provider outputs, then copy the suggested mocks into your test case.
|
|
70
|
+
- Enable debug logs with `--debug` or `VISOR_DEBUG=true`:
|
|
71
|
+
```bash
|
|
72
|
+
visor test --debug --config defaults/visor.tests.yaml --only pr-review-e2e-flow#facts-invalid
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Related Documentation
|
|
76
|
+
|
|
77
|
+
- [Getting Started](./getting-started.md) - Introduction to the test framework
|
|
78
|
+
- [DSL Reference](./dsl-reference.md) - Complete test YAML schema
|
|
79
|
+
- [Assertions](./assertions.md) - Available assertion types
|
|
80
|
+
- [Fixtures and Mocks](./fixtures-and-mocks.md) - Managing test data
|
|
81
|
+
- [Flows](./flows.md) - Multi-stage test flows
|
|
82
|
+
- [CI Integration](./ci.md) - Running tests in CI pipelines
|
|
83
|
+
- [Troubleshooting](./troubleshooting.md) - Common issues and solutions
|
|
@@ -86,70 +86,93 @@ Copy‑pasteable recipes for common scenarios.
|
|
|
86
86
|
fixture: gh.issue_comment.visor_help
|
|
87
87
|
env: { ENABLE_FACT_VALIDATION: "true" }
|
|
88
88
|
mocks:
|
|
89
|
-
comment-assistant: { text: "We rely on defaults
|
|
89
|
+
comment-assistant: { text: "We rely on defaults/visor.yaml line 11 for max_parallelism=4.", intent: comment_reply }
|
|
90
90
|
extract-facts:
|
|
91
91
|
- { id: f1, category: Configuration, claim: "max_parallelism defaults to 4", verifiable: true }
|
|
92
92
|
validate-fact[]:
|
|
93
|
-
- { fact_id: f1, is_valid: true, confidence: high, evidence: "defaults
|
|
93
|
+
- { fact_id: f1, claim: "max_parallelism defaults to 4", is_valid: true, confidence: high, evidence: "defaults/visor.yaml:11" }
|
|
94
94
|
expect:
|
|
95
95
|
calls:
|
|
96
|
+
- step: comment-assistant
|
|
97
|
+
exactly: 1
|
|
96
98
|
- step: extract-facts
|
|
97
99
|
exactly: 1
|
|
98
100
|
- step: validate-fact
|
|
99
101
|
at_least: 1
|
|
100
|
-
- step: aggregate-validations
|
|
101
|
-
exactly: 1
|
|
102
102
|
```
|
|
103
103
|
|
|
104
104
|
## 6) Facts invalid (correction reply)
|
|
105
105
|
|
|
106
|
+
When a fact is invalid, the correction flow triggers a re-run. Due to goto forward-running dependents, `extract-facts` and `validate-fact` also run again.
|
|
107
|
+
|
|
106
108
|
```yaml
|
|
107
109
|
- name: facts-invalid
|
|
108
110
|
event: issue_comment
|
|
109
111
|
fixture: gh.issue_comment.visor_help
|
|
110
112
|
env: { ENABLE_FACT_VALIDATION: "true" }
|
|
113
|
+
routing:
|
|
114
|
+
max_loops: 1
|
|
111
115
|
mocks:
|
|
112
|
-
comment-assistant: { text: "We rely on defaults
|
|
116
|
+
comment-assistant: { text: "We rely on defaults/visor.yaml line 11 for max_parallelism=4.", intent: comment_reply }
|
|
113
117
|
extract-facts:
|
|
114
118
|
- { id: f1, category: Configuration, claim: "max_parallelism defaults to 4", verifiable: true }
|
|
115
119
|
validate-fact[]:
|
|
116
|
-
- { fact_id: f1, is_valid: false, confidence: high, evidence: "defaults
|
|
120
|
+
- { fact_id: f1, claim: "max_parallelism defaults to 4", is_valid: false, confidence: high, evidence: "defaults/visor.yaml:11 does not set 4", correction: "max_parallelism defaults to 3" }
|
|
117
121
|
expect:
|
|
118
122
|
calls:
|
|
119
123
|
- step: comment-assistant
|
|
120
124
|
exactly: 2
|
|
121
|
-
- step:
|
|
125
|
+
- step: extract-facts
|
|
126
|
+
exactly: 2
|
|
127
|
+
- step: validate-fact
|
|
128
|
+
exactly: 2
|
|
129
|
+
- step: aggregate
|
|
122
130
|
exactly: 1
|
|
123
|
-
|
|
124
|
-
- step:
|
|
125
|
-
|
|
126
|
-
|
|
131
|
+
outputs:
|
|
132
|
+
- step: validate-fact
|
|
133
|
+
where: { path: fact_id, equals: f1 }
|
|
134
|
+
path: correction
|
|
135
|
+
equals: "max_parallelism defaults to 3"
|
|
127
136
|
```
|
|
128
137
|
|
|
129
138
|
## 7) Two facts (one invalid)
|
|
130
139
|
|
|
140
|
+
With two facts extracted where only one is invalid, the correction pass runs for the invalid fact. Due to goto forward-running dependents, `extract-facts` and `validate-fact` run again on retry.
|
|
141
|
+
|
|
131
142
|
```yaml
|
|
132
143
|
- name: facts-two-items
|
|
133
144
|
event: issue_comment
|
|
134
145
|
fixture: gh.issue_comment.visor_help
|
|
135
146
|
env: { ENABLE_FACT_VALIDATION: "true" }
|
|
147
|
+
routing:
|
|
148
|
+
max_loops: 1
|
|
136
149
|
mocks:
|
|
137
|
-
comment-assistant: { text: "We rely on defaults
|
|
150
|
+
comment-assistant: { text: "We rely on defaults/visor.yaml for concurrency defaults.", intent: comment_reply }
|
|
138
151
|
extract-facts:
|
|
139
152
|
- { id: f1, category: Configuration, claim: "max_parallelism defaults to 4", verifiable: true }
|
|
140
153
|
- { id: f2, category: Feature, claim: "Fast mode is enabled by default", verifiable: true }
|
|
141
154
|
validate-fact[]:
|
|
142
|
-
- { fact_id: f1, is_valid: false, confidence: high, evidence: "defaults
|
|
143
|
-
- { fact_id: f2, is_valid: true, confidence: high, evidence: "src/config.ts:FAST_MODE=true" }
|
|
155
|
+
- { fact_id: f1, claim: "max_parallelism defaults to 4", is_valid: false, confidence: high, evidence: "defaults/visor.yaml:11", correction: "max_parallelism defaults to 3" }
|
|
156
|
+
- { fact_id: f2, claim: "Fast mode is enabled by default", is_valid: true, confidence: high, evidence: "src/config.ts:FAST_MODE=true" }
|
|
144
157
|
expect:
|
|
145
158
|
calls:
|
|
146
|
-
- step: validate-fact
|
|
147
|
-
exactly: 2
|
|
148
|
-
prompts:
|
|
149
159
|
- step: comment-assistant
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
160
|
+
exactly: 2
|
|
161
|
+
- step: extract-facts
|
|
162
|
+
exactly: 2
|
|
163
|
+
- step: validate-fact
|
|
164
|
+
exactly: 4
|
|
165
|
+
- step: aggregate
|
|
166
|
+
exactly: 1
|
|
167
|
+
outputs:
|
|
168
|
+
- step: validate-fact
|
|
169
|
+
where: { path: fact_id, equals: f1 }
|
|
170
|
+
path: is_valid
|
|
171
|
+
equals: false
|
|
172
|
+
- step: validate-fact
|
|
173
|
+
where: { path: fact_id, equals: f2 }
|
|
174
|
+
path: is_valid
|
|
175
|
+
equals: true
|
|
153
176
|
```
|
|
154
177
|
|
|
155
178
|
## 8) GitHub negative mode
|
|
@@ -170,3 +193,13 @@ Copy‑pasteable recipes for common scenarios.
|
|
|
170
193
|
message_contains: "github/op_failed"
|
|
171
194
|
```
|
|
172
195
|
|
|
196
|
+
## Related Documentation
|
|
197
|
+
|
|
198
|
+
- [Getting Started](./getting-started.md) - Introduction to the test framework
|
|
199
|
+
- [DSL Reference](./dsl-reference.md) - Complete test YAML schema
|
|
200
|
+
- [Assertions](./assertions.md) - Available assertion types
|
|
201
|
+
- [Fixtures and Mocks](./fixtures-and-mocks.md) - Managing test data
|
|
202
|
+
- [Flows](./flows.md) - Multi-stage test flows
|
|
203
|
+
- [CLI](./cli.md) - Test runner command line options
|
|
204
|
+
- [CI Integration](./ci.md) - Running tests in CI pipelines
|
|
205
|
+
- [Troubleshooting](./troubleshooting.md) - Common issues and solutions
|
|
@@ -11,9 +11,17 @@ tests:
|
|
|
11
11
|
strict: true # default strict mode
|
|
12
12
|
ai_provider: mock # force AI provider to mock
|
|
13
13
|
prompt_max_chars: 16000 # truncate captured prompts (optional)
|
|
14
|
+
ai_include_code_context: false # include PR diff/context in AI prompts (default: false)
|
|
15
|
+
fail_on_unexpected_calls: false # fail if unexpected provider calls occur
|
|
16
|
+
frontends: ["github"] # enable specific frontends during tests
|
|
14
17
|
github_recorder: # optional negative modes
|
|
15
18
|
error_code: 0 # e.g., 429
|
|
16
19
|
timeout_ms: 0 # e.g., 1000
|
|
20
|
+
macros: # reusable expect blocks (see Reusable Macros)
|
|
21
|
+
basic-check:
|
|
22
|
+
calls:
|
|
23
|
+
- step: overview
|
|
24
|
+
at_least: 1
|
|
17
25
|
# Optional: include/exclude checks by tags (same semantics as main CLI)
|
|
18
26
|
tags: "local,fast" # or [local, fast]
|
|
19
27
|
exclude_tags: "experimental,slow" # or [experimental, slow]
|
|
@@ -24,16 +32,20 @@ tests:
|
|
|
24
32
|
- name: <string>
|
|
25
33
|
description: <markdown>
|
|
26
34
|
skip: false|true
|
|
35
|
+
ai_include_code_context: false # per-case override
|
|
27
36
|
|
|
28
37
|
# Single-event case
|
|
29
38
|
event: pr_opened | pr_updated | pr_closed | issue_opened | issue_comment | manual
|
|
30
39
|
fixture: <builtin|{ builtin, overrides }>
|
|
31
40
|
env: { <KEY>: <VALUE>, ... }
|
|
32
41
|
mocks: { <step>: <value>, <step>[]: [<value>...] }
|
|
42
|
+
workflow_input: { <key>: <value>, ... } # inputs for workflow testing
|
|
33
43
|
expect: <expect-block>
|
|
34
44
|
strict: true|false # overrides defaults.strict
|
|
35
45
|
tags: "security,fast" # optional per-case include filter
|
|
36
46
|
exclude_tags: "slow" # optional per-case exclude filter
|
|
47
|
+
github_recorder: # per-case recorder overrides
|
|
48
|
+
error_code: 429
|
|
37
49
|
|
|
38
50
|
# OR flow case
|
|
39
51
|
flow:
|
|
@@ -42,23 +54,31 @@ tests:
|
|
|
42
54
|
fixture: ...
|
|
43
55
|
env: ...
|
|
44
56
|
mocks: ... # merged with flow-level mocks
|
|
57
|
+
routing: # per-stage routing overrides
|
|
58
|
+
max_loops: 10
|
|
45
59
|
expect: <expect-block>
|
|
46
60
|
strict: true|false # per-stage fallback to case/defaults
|
|
47
61
|
tags: "security" # optional per-stage include filter
|
|
48
62
|
exclude_tags: "slow" # optional per-stage exclude filter
|
|
63
|
+
github_recorder: # per-stage recorder overrides
|
|
64
|
+
error_code: 500
|
|
49
65
|
```
|
|
50
66
|
|
|
51
67
|
## Fixtures
|
|
52
68
|
|
|
53
|
-
- Built-in GitHub fixtures: `gh.pr_open.minimal`, `gh.pr_sync.minimal`, `gh.issue_open.minimal`, `gh.issue_comment.standard`, `gh.issue_comment.visor_help`, `gh.issue_comment.visor_regenerate`.
|
|
69
|
+
- Built-in GitHub fixtures: `gh.pr_open.minimal`, `gh.pr_sync.minimal`, `gh.pr_closed.minimal`, `gh.issue_open.minimal`, `gh.issue_comment.standard`, `gh.issue_comment.visor_help`, `gh.issue_comment.visor_regenerate`.
|
|
54
70
|
- Use `overrides` to tweak titles, numbers, payload slices.
|
|
55
71
|
|
|
72
|
+
See [Fixtures and Mocks](./fixtures-and-mocks.md) for details.
|
|
73
|
+
|
|
56
74
|
## Mocks
|
|
57
75
|
|
|
58
76
|
- Keys are step names; for forEach children use `step[]` (e.g., `validate-fact[]`).
|
|
59
77
|
- AI mocks may be structured JSON if a schema is configured for the step; otherwise use `text` and optional fields used by templates.
|
|
60
78
|
- Command/HTTP mocks emulate provider shape (`stdout`, `exit_code`, or HTTP body/status headers) and bypass real execution.
|
|
61
79
|
|
|
80
|
+
See [Fixtures and Mocks](./fixtures-and-mocks.md) for detailed mock examples.
|
|
81
|
+
|
|
62
82
|
Inline example (AI with schema + list mocks):
|
|
63
83
|
|
|
64
84
|
```yaml
|
|
@@ -78,13 +98,20 @@ mocks:
|
|
|
78
98
|
|
|
79
99
|
```yaml
|
|
80
100
|
expect:
|
|
101
|
+
use: [macro-name] # reference macros from tests.defaults.macros
|
|
102
|
+
|
|
81
103
|
calls:
|
|
82
|
-
- step: <name>
|
|
104
|
+
- step: <name>
|
|
105
|
+
exactly|at_least|at_most: <number>
|
|
106
|
+
- provider: github|slack # provider-level calls
|
|
107
|
+
op: <rest.op> # e.g., labels.add, chat.postMessage
|
|
83
108
|
exactly|at_least|at_most: <number>
|
|
84
|
-
args: { contains: [..]
|
|
109
|
+
args: { contains: [..] } # provider args matching
|
|
85
110
|
|
|
86
111
|
no_calls:
|
|
87
|
-
- step: <name>
|
|
112
|
+
- step: <name>
|
|
113
|
+
- provider: github|slack
|
|
114
|
+
op: <rest.op>
|
|
88
115
|
|
|
89
116
|
prompts:
|
|
90
117
|
- step: <name>
|
|
@@ -97,7 +124,7 @@ expect:
|
|
|
97
124
|
|
|
98
125
|
outputs:
|
|
99
126
|
- step: <name>
|
|
100
|
-
index: first|last|<N>
|
|
127
|
+
index: first|last|<N>
|
|
101
128
|
where: { path: <expr>, equals|matches: <v> }
|
|
102
129
|
path: <expr> # dot/bracket, e.g. tags['review-effort']
|
|
103
130
|
equals: <primitive>
|
|
@@ -105,12 +132,29 @@ expect:
|
|
|
105
132
|
matches: <regex>
|
|
106
133
|
contains_unordered: [..]
|
|
107
134
|
|
|
135
|
+
workflow_output: # assert on workflow-level outputs (for workflow testing)
|
|
136
|
+
- path: <output-name> # path into workflow outputs object
|
|
137
|
+
equals: <primitive>
|
|
138
|
+
equalsDeep: <object>
|
|
139
|
+
matches: <regex>
|
|
140
|
+
contains: <string|[..]> # substring check
|
|
141
|
+
not_contains: <string|[..]>
|
|
142
|
+
contains_unordered: [..]
|
|
143
|
+
where: { path: <expr>, equals|matches: <v> }
|
|
144
|
+
|
|
108
145
|
fail:
|
|
109
146
|
message_contains: <string> # assert overall case failure message
|
|
110
147
|
|
|
111
148
|
strict_violation: # assert strict failure for a missing expect on a step
|
|
112
149
|
for_step: <name>
|
|
113
150
|
message_contains: <string>
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
**Supported providers for `calls` and `no_calls`:**
|
|
154
|
+
- `github`: GitHub API operations (`labels.add`, `issues.createComment`, `pulls.createReview`, `checks.create`, `checks.update`)
|
|
155
|
+
- `slack`: Slack API operations (`chat.postMessage`)
|
|
156
|
+
|
|
157
|
+
See [Assertions](./assertions.md) for detailed assertion syntax and examples.
|
|
114
158
|
|
|
115
159
|
Inline example (calls + prompts + outputs):
|
|
116
160
|
|
|
@@ -131,7 +175,6 @@ expect:
|
|
|
131
175
|
path: "tags['review-effort']"
|
|
132
176
|
equals: 2
|
|
133
177
|
```
|
|
134
|
-
```
|
|
135
178
|
|
|
136
179
|
Note on dependencies: test execution honors your base config routing, including `depends_on`. You can express ANY‑OF groups using pipe syntax in the base config (e.g., `depends_on: ["issue-assistant|comment-assistant"]`). The runner mixes these with normal ALL‑OF deps.
|
|
137
180
|
|
|
@@ -152,6 +195,63 @@ Note on dependencies: test execution honors your base config routing, including
|
|
|
152
195
|
- Run one case: `visor test --only label-flow`
|
|
153
196
|
- Run one stage: `visor test --only pr-review-e2e-flow#facts-invalid`
|
|
154
197
|
- JSON/JUnit/Markdown reporters: `--json`, `--report junit:<path>`, `--summary md:<path>`
|
|
198
|
+
|
|
199
|
+
See [CLI Reference](./cli.md) for all available options.
|
|
200
|
+
|
|
201
|
+
## Reusable Macros
|
|
202
|
+
|
|
203
|
+
Define reusable assertion blocks in `tests.defaults.macros` and reference them with `use`:
|
|
204
|
+
|
|
205
|
+
```yaml
|
|
206
|
+
tests:
|
|
207
|
+
defaults:
|
|
208
|
+
macros:
|
|
209
|
+
basic-github-check:
|
|
210
|
+
calls:
|
|
211
|
+
- provider: github
|
|
212
|
+
op: checks.create
|
|
213
|
+
at_least: 1
|
|
214
|
+
overview-ran:
|
|
215
|
+
calls:
|
|
216
|
+
- step: overview
|
|
217
|
+
exactly: 1
|
|
218
|
+
|
|
219
|
+
cases:
|
|
220
|
+
- name: my-test
|
|
221
|
+
event: pr_opened
|
|
222
|
+
expect:
|
|
223
|
+
use: [basic-github-check, overview-ran]
|
|
224
|
+
calls:
|
|
225
|
+
- step: extra-step
|
|
226
|
+
exactly: 1
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Macros are merged with inline expectations, allowing you to compose reusable assertion patterns.
|
|
230
|
+
|
|
231
|
+
## Workflow Testing
|
|
232
|
+
|
|
233
|
+
Test standalone workflows by providing `workflow_input` and asserting on `workflow_output`:
|
|
234
|
+
|
|
235
|
+
```yaml
|
|
236
|
+
tests:
|
|
237
|
+
cases:
|
|
238
|
+
- name: test-workflow
|
|
239
|
+
event: manual
|
|
240
|
+
workflow_input:
|
|
241
|
+
repo_url: "https://github.com/example/repo"
|
|
242
|
+
branch: "main"
|
|
243
|
+
mocks:
|
|
244
|
+
fetch-data:
|
|
245
|
+
status: 200
|
|
246
|
+
data: { items: [1, 2, 3] }
|
|
247
|
+
expect:
|
|
248
|
+
workflow_output:
|
|
249
|
+
- path: summary
|
|
250
|
+
contains: "completed"
|
|
251
|
+
- path: items_count
|
|
252
|
+
equals: 3
|
|
253
|
+
```
|
|
254
|
+
|
|
155
255
|
## JavaScript in Tests and Routing (run_js, goto_js, value_js, transform_js)
|
|
156
256
|
|
|
157
257
|
### Tags default semantics in tests
|
|
@@ -184,8 +284,9 @@ Tips
|
|
|
184
284
|
- Use `Array.prototype.at(-1)` to read the last item. Example: `const last = (outputs_history['validate-fact'] || []).at(-1) || [];`.
|
|
185
285
|
- For reshaping small maps, `Object.entries` + `Object.fromEntries` is concise and readable.
|
|
186
286
|
|
|
187
|
-
Example: wave
|
|
188
|
-
|
|
287
|
+
Example: wave-scoped correction gate
|
|
288
|
+
|
|
289
|
+
```yaml
|
|
189
290
|
run_js: |
|
|
190
291
|
const facts = (outputs_history['extract-facts'] || []).at(-1) || [];
|
|
191
292
|
const ids = facts.map(f => String(f.id || '')).filter(Boolean);
|
|
@@ -196,4 +297,4 @@ run_js: |
|
|
|
196
297
|
return (event && event.name) === 'issue_opened' ? ['issue-assistant'] : ['comment-assistant'];
|
|
197
298
|
```
|
|
198
299
|
|
|
199
|
-
This evaluates the last `extract-facts` wave, finds the corresponding `validate-fact` results, and schedules a single correction pass when any item is invalid or low
|
|
300
|
+
This evaluates the last `extract-facts` wave, finds the corresponding `validate-fact` results, and schedules a single correction pass when any item is invalid or low-confidence.
|
|
@@ -13,6 +13,7 @@ Use via `fixture: gh.<name>` or `fixture: { builtin: gh.<name>, overrides: {...}
|
|
|
13
13
|
- `gh.issue_comment.standard` — normal human comment on a PR/issue.
|
|
14
14
|
- `gh.issue_comment.visor_help` — comment containing `/visor help`.
|
|
15
15
|
- `gh.issue_comment.visor_regenerate` — `/visor Regenerate reviews`.
|
|
16
|
+
- `gh.issue_comment.edited` — issue_comment edited action.
|
|
16
17
|
|
|
17
18
|
Overrides allow tailored inputs:
|
|
18
19
|
|
|
@@ -57,7 +58,7 @@ mocks:
|
|
|
57
58
|
|
|
58
59
|
# AI plain text schema
|
|
59
60
|
comment-assistant:
|
|
60
|
-
text: "Sure, here
|
|
61
|
+
text: "Sure, here's how I can help."
|
|
61
62
|
intent: comment_reply
|
|
62
63
|
|
|
63
64
|
# Array outputs (e.g., extract-facts)
|
|
@@ -68,8 +69,9 @@ mocks:
|
|
|
68
69
|
unit-tests:
|
|
69
70
|
stdout: '{"passed": 128, "failed": 0}'
|
|
70
71
|
exit_code: 0
|
|
72
|
+
```
|
|
71
73
|
|
|
72
|
-
### Per
|
|
74
|
+
### Per-call list mocks (for forEach children)
|
|
73
75
|
|
|
74
76
|
When a step runs once per item (e.g., `validate-fact` depends on `extract-facts`), provide a list under the `[]` suffix:
|
|
75
77
|
|
|
@@ -83,9 +85,32 @@ mocks:
|
|
|
83
85
|
- { fact_id: f2, is_valid: true, confidence: high, evidence: "src/config.ts:FAST_MODE=true" }
|
|
84
86
|
```
|
|
85
87
|
|
|
86
|
-
The runner distributes items in order; if the list is shorter than invocations, the last entry is reused. This fits any forEach
|
|
88
|
+
The runner distributes items in order; if the list is shorter than invocations, the last entry is reused. This fits any forEach-style step you define (naming is up to you).
|
|
89
|
+
|
|
90
|
+
### HTTP client mocks
|
|
91
|
+
|
|
92
|
+
For `http-client` provider steps, mocks can specify HTTP response fields:
|
|
93
|
+
|
|
94
|
+
```yaml
|
|
95
|
+
mocks:
|
|
96
|
+
fetch-api-data:
|
|
97
|
+
status: 200
|
|
98
|
+
body: '{"items": [1, 2, 3]}'
|
|
99
|
+
headers:
|
|
100
|
+
content-type: application/json
|
|
87
101
|
```
|
|
88
102
|
|
|
89
103
|
Notes:
|
|
90
104
|
- No `returns:` key; provide values directly.
|
|
91
105
|
- For HTTP/Command providers, mocks bypass real execution and are recorded for assertions.
|
|
106
|
+
|
|
107
|
+
## Related Documentation
|
|
108
|
+
|
|
109
|
+
- [Getting Started](./getting-started.md) - Introduction to the test framework
|
|
110
|
+
- [DSL Reference](./dsl-reference.md) - Complete test YAML schema
|
|
111
|
+
- [Assertions](./assertions.md) - Available assertion types
|
|
112
|
+
- [Flows](./flows.md) - Multi-stage test flows
|
|
113
|
+
- [Cookbook](./cookbook.md) - Copy-pasteable test recipes
|
|
114
|
+
- [CLI](./cli.md) - Test runner command line options
|
|
115
|
+
- [CI Integration](./ci.md) - Running tests in CI pipelines
|
|
116
|
+
- [Troubleshooting](./troubleshooting.md) - Common issues and solutions
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
> Model realistic user journeys across multiple external events in one case.
|
|
4
4
|
|
|
5
|
-
A flow case defines a `flow:` array of stages. Each stage has its own `event`, `fixture`, optional `env`
|
|
5
|
+
A flow case defines a `flow:` array of stages. Each stage has its own `event`, `fixture`, and optional settings like `env`, `mocks`, `routing`, `tags`, `github_recorder`, plus `expect`.
|
|
6
6
|
|
|
7
7
|
```yaml
|
|
8
8
|
- name: pr-review-e2e-flow
|
|
@@ -35,8 +35,9 @@ A flow case defines a `flow:` array of stages. Each stage has its own `event`, `
|
|
|
35
35
|
|
|
36
36
|
## Stage selection and deltas
|
|
37
37
|
|
|
38
|
-
- Run a single stage: `--only case#stage` (name substring) or `--only case#N` (1
|
|
39
|
-
-
|
|
38
|
+
- Run a single stage: `--only case#stage` (name substring match, case-insensitive) or `--only case#N` (1-based index).
|
|
39
|
+
- Examples: `--only pr-review-e2e-flow#facts-invalid`, `--only pr-review-e2e-flow#3`
|
|
40
|
+
- Coverage, prompts, outputs, and provider calls are computed per-stage as deltas from the previous stage.
|
|
40
41
|
- The same engine instance is reused across stages, so memory and output history carry over.
|
|
41
42
|
|
|
42
43
|
## Ordering and `on_finish`
|
|
@@ -81,12 +82,66 @@ flow:
|
|
|
81
82
|
contains: ["<previous_response>", "Correction:"]
|
|
82
83
|
```
|
|
83
84
|
|
|
84
|
-
## Stage-local
|
|
85
|
+
## Stage-local configuration
|
|
86
|
+
|
|
87
|
+
### Mocks and env
|
|
85
88
|
|
|
86
89
|
- Stage mocks override flow-level defaults: the runner merges `{...flow.mocks, ...stage.mocks}`.
|
|
87
90
|
- `env:` applies only for the stage and is restored afterward.
|
|
88
91
|
|
|
92
|
+
### Routing overrides
|
|
93
|
+
|
|
94
|
+
Per-stage routing settings override the base config for that stage only:
|
|
95
|
+
|
|
96
|
+
```yaml
|
|
97
|
+
flow:
|
|
98
|
+
- name: correction-loop
|
|
99
|
+
event: issue_comment
|
|
100
|
+
routing:
|
|
101
|
+
max_loops: 10 # allow more iterations for this stage
|
|
102
|
+
# ...
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### Tag filtering
|
|
106
|
+
|
|
107
|
+
Tags can be specified at flow-level and/or per-stage. They are merged with suite defaults:
|
|
108
|
+
|
|
109
|
+
```yaml
|
|
110
|
+
- name: my-flow
|
|
111
|
+
tags: "github" # flow-level include filter
|
|
112
|
+
exclude_tags: "slow" # flow-level exclude filter
|
|
113
|
+
flow:
|
|
114
|
+
- name: stage-one
|
|
115
|
+
tags: "security" # additional per-stage filter
|
|
116
|
+
# ...
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
### GitHub recorder overrides
|
|
120
|
+
|
|
121
|
+
Simulate GitHub API errors or timeouts per-stage:
|
|
122
|
+
|
|
123
|
+
```yaml
|
|
124
|
+
flow:
|
|
125
|
+
- name: api-error-stage
|
|
126
|
+
event: pr_opened
|
|
127
|
+
github_recorder:
|
|
128
|
+
error_code: 429 # simulate rate limit
|
|
129
|
+
# ...
|
|
130
|
+
```
|
|
131
|
+
|
|
89
132
|
## Debugging flows
|
|
90
133
|
|
|
91
134
|
- Set `VISOR_DEBUG=true` to print stage headers, selected checks, and internal debug lines from the engine.
|
|
92
135
|
- To reduce noise, limit the run to a stage: `VISOR_DEBUG=true visor test --only pr-review-e2e-flow#facts-invalid`.
|
|
136
|
+
- Use the CLI `--debug` flag as a shorthand: `visor test --debug --only case#stage`.
|
|
137
|
+
|
|
138
|
+
## Related Documentation
|
|
139
|
+
|
|
140
|
+
- [Getting Started](./getting-started.md) - Introduction to the test framework
|
|
141
|
+
- [DSL Reference](./dsl-reference.md) - Complete test YAML schema
|
|
142
|
+
- [Assertions](./assertions.md) - Available assertion types
|
|
143
|
+
- [Fixtures and Mocks](./fixtures-and-mocks.md) - Managing test data
|
|
144
|
+
- [Cookbook](./cookbook.md) - Copy-pasteable test recipes
|
|
145
|
+
- [CLI](./cli.md) - Test runner command line options
|
|
146
|
+
- [CI Integration](./ci.md) - Running tests in CI pipelines
|
|
147
|
+
- [Troubleshooting](./troubleshooting.md) - Common issues and solutions
|
|
@@ -4,10 +4,10 @@ This is the developer-facing guide for writing and running integration tests for
|
|
|
4
4
|
|
|
5
5
|
## TL;DR
|
|
6
6
|
|
|
7
|
-
- Put your tests in `defaults
|
|
7
|
+
- Put your tests in `defaults/visor.tests.yaml`.
|
|
8
8
|
- Reference your base config with `extends: ".visor.yaml"`.
|
|
9
9
|
- Use built-in GitHub fixtures like `gh.pr_open.minimal`.
|
|
10
|
-
- Run with `visor test --config defaults
|
|
10
|
+
- Run with `visor test --config defaults/visor.tests.yaml`.
|
|
11
11
|
- Validate only with `visor test --validate`.
|
|
12
12
|
|
|
13
13
|
```yaml
|
|
@@ -78,16 +78,17 @@ Run `visor test --validate` to get precise YAML-path errors and suggestions:
|
|
|
78
78
|
```
|
|
79
79
|
❌ Tests file has 2 error(s):
|
|
80
80
|
• tests.cases[0].expext: must NOT have additional properties (Did you mean "expect"?)
|
|
81
|
-
• tests.cases[3].event: must be equal to one of the allowed values (allowed: manual, pr_opened, pr_updated, pr_closed, issue_opened, issue_comment)
|
|
81
|
+
• tests.cases[3].event: must be equal to one of the allowed values (allowed: manual, pr_opened, pr_updated, pr_closed, issue_opened, issue_comment, schedule, webhook_received)
|
|
82
82
|
```
|
|
83
83
|
|
|
84
|
-
Next
|
|
85
|
-
|
|
86
|
-
-
|
|
87
|
-
-
|
|
88
|
-
-
|
|
89
|
-
-
|
|
90
|
-
-
|
|
91
|
-
-
|
|
92
|
-
-
|
|
93
|
-
-
|
|
84
|
+
## Next Steps
|
|
85
|
+
|
|
86
|
+
- [DSL Reference](./dsl-reference.md) - Complete test YAML schema
|
|
87
|
+
- [Flows](./flows.md) - Multi-stage test flows
|
|
88
|
+
- [Fixtures and Mocks](./fixtures-and-mocks.md) - Managing test data
|
|
89
|
+
- [Assertions](./assertions.md) - Available assertion types
|
|
90
|
+
- [Cookbook](./cookbook.md) - Common patterns and recipes
|
|
91
|
+
- [CLI and Reporters](./cli.md) - Command-line options and output formats
|
|
92
|
+
- [CI Integration](./ci.md) - Running tests in CI pipelines
|
|
93
|
+
- [Troubleshooting](./troubleshooting.md) - Common issues and solutions
|
|
94
|
+
- Browse `defaults/visor.tests.yaml` for full examples.
|