@jmoak/vibrissa 0.1.0 → 2.0.0-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE CHANGED
@@ -1,21 +1,21 @@
1
- MIT License
2
-
3
- Copyright (c) 2025 Jordan Moak
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining a copy
6
- of this software and associated documentation files (the "Software"), to deal
7
- in the Software without restriction, including without limitation the rights
8
- to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
- copies of the Software, and to permit persons to whom the Software is
10
- furnished to do so, subject to the following conditions:
11
-
12
- The above copyright notice and this permission notice shall be included in all
13
- copies or substantial portions of the Software.
14
-
15
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
- IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
- FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
- AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
- LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
- OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
- SOFTWARE.
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Jordan Moak
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,315 +1,329 @@
1
- ## vibrissa (whisker) — JSON‑Driven MCP Integration Test Runner
1
+ # vibrissa
2
2
 
3
- End‑to‑end test your MCP server over stdio using declarative JSON cases. Ship stable, contract‑focused integration suites that humans can review and LLMs can author.
3
+ > *vibrissa* (n.): a whisker; a sensory organ for probing the world.
4
4
 
5
- ### Why Vibrissa?
6
- - Integration‑first: validates real I/O over MCP stdio
7
- - JSON cases: simple, diff‑friendly, easy to generate
8
- - Powerful matching: exact, partial, regex, wildcard, unordered arrays
9
- - Flexible: CLI (`vib-test`) or library API
5
+ **Contract testing for MCP servers.** Record real responses as JSON, replay them in CI,
6
+ lock your tool manifest, fuzz your schemas. No test code: the whole suite is data.
10
7
 
11
- ### Requirements
12
- - Node.js >= 20
13
- - ESM-only package (use `import`, not `require`)
8
+ [![npm](https://img.shields.io/npm/v/%40jmoak%2Fvibrissa?color=cb3837&label=npm)](https://www.npmjs.com/package/@jmoak/vibrissa)
9
+ [![CI](https://github.com/JMoak/vibrissa/actions/workflows/ci.yml/badge.svg)](https://github.com/JMoak/vibrissa/actions/workflows/ci.yml)
10
+ [![node >= 20](https://img.shields.io/badge/node-%E2%89%A5%2020-3c873a)](https://nodejs.org)
11
+ [![MIT](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
14
12
 
15
- ### Install
16
- ```bash
17
- npm i -D @jmoak/vibrissa
18
- ```
13
+ Vibrissa drives your server over stdio or Streamable HTTP, speaks both MCP 2026-07-28
14
+ and the 2025 revisions, and diffs every tool, resource, and prompt result against
15
+ committed JSON. It also snapshots the server's whole surface into a lock file, so a
16
+ changed schema or a quietly rewritten tool description fails the PR instead of reaching
17
+ a client.
19
18
 
20
- ### Quickstart
21
19
  ```bash
22
20
  npm i -D @jmoak/vibrissa
23
- vib init
24
- # edit vibrissa.json server block if needed
25
- vib tools
26
- vib call echo --arg text=hello --arg uppercase=true
27
- vib record echo --arg text=hello --arg uppercase=true --name "echo uppercase"
28
- vib run
21
+ npx vib init # vibrissa.json + a real sample case from your server
22
+ npx vib discover # protocol era, capabilities, identity
23
+ npx vib tools # what the server exposes
24
+ npx vib call echo --arg text=hello
25
+ npx vib record echo --arg text=hello --name "echo hello"
26
+ npx vib manifest record # lock the surface
27
+ npx vib run # replay every case: the CI gate
28
+ npx vib check # spec sanity + description lint
29
+ npx vib fuzz echo --runs 200 # schema-driven fuzzing
29
30
  ```
30
31
 
31
- Or scaffold by hand:
32
+ Releases ship through npm [trusted publishing](https://docs.npmjs.com/trusted-publishers/)
33
+ with provenance: each version links to the commit and workflow that built it.
32
34
 
33
- 1) Create `vibrissa.json` in your MCP server repo (root):
34
- ```json
35
- {
36
- "$schema": "./node_modules/@jmoak/vibrissa/schema/vibrissa.schema.json",
37
- "server": { "cmd": "node", "args": ["dist/index.js"] },
38
- "globs": ["tests/integration/**/*.json"]
39
- }
40
- ```
35
+ ---
36
+
37
+ ## Why
38
+
39
+ MCP testing has sorted itself into layers:
40
+
41
+ | Layer | Question | Tools |
42
+ |---|---|---|
43
+ | 0: Conformance | Does the server speak the spec? | official conformance suite; `vib check` covers the deterministic subset |
44
+ | **1: Contract** | **Given these args, does this server return this?** | **Vibrissa** |
45
+ | 2: Robustness | Fuzzing, poisoning, rug pulls | `vib fuzz`, `vib check`, manifest lock (deterministic subset) |
46
+ | 3: Model evals | Does an LLM pick the right tool? | eval platforms; out of core by design |
47
+
48
+ Vibrissa owns Layer 1 and absorbs the deterministic halves of 0 and 2. No LLM in the
49
+ critical path, no accounts, no hosted anything: CI gates are reproducible and the loop is
50
+ sub-second.
51
+
52
+ The 2026-07-28 revision rewrote the core. Each change lands on what a contract tester
53
+ must do:
54
+
55
+ | 2026-07-28 change | Vibrissa |
56
+ |---|---|
57
+ | `initialize` replaced by `server/discover` | `protocol: auto` negotiates either era |
58
+ | `resultType` + `_meta` serverInfo on every result | normalized away; old goldens stay green |
59
+ | Multi Round-Trip Requests (`input_required`) | `respond` scripts the answers |
60
+ | No protocol sessions; state in server-minted handles | flows with `capture` and `{{name}}` |
61
+ | Deterministic `tools/list` with cache hints | manifest lock; `vib check` |
62
+ | JSON Schema 2020-12 `outputSchema` | enforced on every call |
63
+ | Tasks extension with `tasks/get` polling | `await`, `expectTask` |
64
+ | Renumbered error codes | stable code names on both eras |
65
+
66
+ ## The loop
67
+
68
+ | Step | Command | What it does |
69
+ |---|---|---|
70
+ | explore | `vib discover`, `vib tools`, `vib resources`, `vib prompts` | see the surface |
71
+ | probe | `vib call <tool>`, `vib read <uri>`, `vib get-prompt <name>` | one live request, printed |
72
+ | capture | `vib record ...`, `vib tap -- <server>` | real responses become golden cases |
73
+ | gate | `vib run` | replay everything, diff the manifest; non-zero on any mismatch |
74
+ | harden | `vib check`, `vib fuzz <tool>` | spec sanity, poisoning lint, generated edge cases |
75
+
76
+ Every command except `tap` takes `--json`. [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector)
77
+ is still the place to click around: `vib inspect` opens it with your config.
78
+
79
+ ## Anatomy of a case
80
+
81
+ The v1 shape is still the core shape. `tests/integration/echo.basic.json`:
41
82
 
42
- 2) Create a case: `tests/integration/echo.basic.json`
43
83
  ```json
44
84
  {
45
85
  "name": "echo basic",
46
86
  "tool": "echo",
47
- "args": {
48
- "text": "hello",
49
- "uppercase": true
50
- },
51
- "expect": {
52
- "content": [
53
- {
54
- "type": "text",
55
- "text": "HELLO"
56
- }
57
- ]
58
- }
87
+ "args": { "text": "hello" },
88
+ "expect": { "content": [{ "type": "text", "text": "hello" }] }
59
89
  }
60
90
  ```
61
91
 
62
- 3) Run
63
- ```bash
64
- vib-test run
65
- ```
92
+ `expect` is exact deep equality over the normalized result unless a matcher says
93
+ otherwise. `resource`, `resourceTemplate`, and `prompt` work the same way as `tool`.
66
94
 
67
- Optional: add an NPM script
68
- ```json
69
- {
70
- "scripts": {
71
- "test:integration": "vib-test run"
72
- }
73
- }
74
- ```
95
+ **A flow** chains steps on one session and passes values forward:
75
96
 
76
- ### Test Case Semantics
77
- - Default is exact deep equality.
78
- - Partial
79
- ```json
80
- {
81
- "expect": {
82
- "$partial": true,
83
- "content": [
84
- {
85
- "type": "text"
86
- }
87
- ]
88
- }
89
- }
90
- ```
91
- - Regex
92
- ```json
93
- {
94
- "expect": {
95
- "content": [
96
- {
97
- "type": "text",
98
- "text": {
99
- "$regex": "^hel+o$",
100
- "flags": "i"
101
- }
102
- }
103
- ]
104
- }
105
- }
106
- ```
107
- - Wildcard (glob‑like)
108
- ```json
109
- {
110
- "expect": {
111
- "content": [
112
- {
113
- "type": "text",
114
- "text": {
115
- "$wildcard": "hel*o"
116
- }
117
- }
118
- ]
119
- }
120
- }
121
- ```
122
- - Unordered arrays
123
97
  ```json
124
98
  {
125
- "expect": {
126
- "content": {
127
- "$unordered": true,
128
- "value": [
129
- {
130
- "type": "text"
131
- }
132
- ]
99
+ "name": "sum flows into echo",
100
+ "steps": [
101
+ { "tool": "typed", "args": { "a": 2, "b": 3 }, "capture": { "sum": "structuredContent.sum" } },
102
+ {
103
+ "tool": "echo",
104
+ "args": { "text": "sum is {{sum}}" },
105
+ "expect": { "content": [{ "type": "text", "text": "sum is 5" }] }
133
106
  }
134
- }
107
+ ]
135
108
  }
136
109
  ```
137
- - Error expectations
110
+
111
+ **An MRTR case** answers the server's question and asserts it was asked:
112
+
138
113
  ```json
139
114
  {
140
- "expectError": {
141
- "code": "InvalidParams",
142
- "message": {
143
- "$regex": "Invalid.*echo"
144
- }
145
- }
115
+ "name": "confirm deploy",
116
+ "tool": "confirm",
117
+ "args": { "action": "deploy" },
118
+ "respond": { "elicitation/create": { "action": "accept", "content": { "confirm": true } } },
119
+ "expectInputRequired": { "$partial": true, "elicitation/create": { "message": "Proceed with deploy?" } },
120
+ "expect": { "content": [{ "type": "text", "text": "confirmed: deploy" }] }
146
121
  }
147
122
  ```
148
123
 
149
- Note: `expect` is matched against the **entire** `tools/call` result object. If your server
150
- returns fields beyond `content` (e.g. `structuredContent`, `_meta`), an exact match will fail
151
- with an "unexpected key" mismatch — either include those fields in `expect` or set
152
- `"$partial": true` to assert only the fields you care about. `vib record` captures the full
153
- result, so recorded cases are always exact-match ready.
124
+ Also: `setup` / `teardown`, `await` for long-running tasks, `budgetMs`, `tags`, `skip`,
125
+ `flaky`, per-case `server`, `$ref` fragments. Every key: [docs/CASES.md](./docs/CASES.md).
126
+
127
+ ## Matchers
128
+
129
+ Matchers nest anywhere in an expectation. Several in one object must all pass.
130
+
131
+ | Matcher | Example | Asserts |
132
+ |---|---|---|
133
+ | `$partial` | `{"$partial": true, "isError": false}` | only the fields you name |
134
+ | `$regex` | `{"$regex": "^hel+o$", "flags": "i"}` | pattern match |
135
+ | `$wildcard` | `{"$wildcard": "req_*"}` | glob match |
136
+ | `$unordered` | `{"$unordered": true, "value": [1, 2]}` | array in any order |
137
+ | `$any` / `$ignore` | `{"$any": true}` | present / don't care |
138
+ | `$type` | `{"$type": ["string", "null"]}` | JSON type |
139
+ | `$schema` | `{"$schema": {"type": "string", "format": "date-time"}}` | inline JSON Schema |
140
+ | `$oneOf` | `{"$oneOf": ["queued", "running"]}` | any alternative |
141
+ | `$contains` | `{"$contains": {"type": "image"}, "count": 1}` | some element matches |
142
+ | `$length` / `$number` | `{"$length": {"min": 1}}` | sizes and ranges |
143
+ | `$ref` | `{"$ref": "./shared/user.json"}` | reuse a fragment |
144
+ | `$literal` | `{"$literal": {"$ref": "#/a"}}` | data that looks like a matcher |
145
+
146
+ `expectError` asserts that a call fails correctly:
147
+ `{"code": "InvalidParams", "message": {"$regex": "text"}}`. Full reference:
148
+ [docs/MATCHERS.md](./docs/MATCHERS.md).
149
+
150
+ ## Manifest lock
151
+
152
+ `vib manifest record` snapshots `server/discover`, `tools/list`, `resources/list`,
153
+ `resources/templates/list`, and `prompts/list` into
154
+ `tests/integration/__manifest__.json`. From then on `vib run` diffs the live surface
155
+ against it and fails on added or removed tools, changed input or output schemas, changed
156
+ annotations, changed capabilities, and **changed descriptions**.
157
+
158
+ It is `package-lock.json` for your MCP surface. It also catches rug pulls: a tool
159
+ description that changes after review, the move behind tool-poisoning attacks, shows up
160
+ as a reviewed diff in git instead of a silent change. Each change is marked breaking or
161
+ not (a removed property or a newly required one is; a reworded description is drift, but
162
+ not breaking), with a word diff for prose. Accept intended changes with
163
+ `vib run --update-manifest` and commit the lock with the PR that caused them. One lock
164
+ serves both protocol eras; `requireManifest: true` makes a missing lock fail CI.
165
+
166
+ ## Checks and fuzz
167
+
168
+ **`vib check`** runs offline checks over the manifest: `spec/*` sanity (known protocol
169
+ version, deterministic `tools/list`, cache hints, valid 2020-12 schemas, resolvable
170
+ `$ref`s) and `desc/*` lint for hidden instructions, invisible Unicode, tool shadowing,
171
+ credential paths, and exfiltration URLs in model-visible text. It is a smoke detector,
172
+ not a scanner. `vib run --check` folds it into CI. Ids and allow-listing:
173
+ [docs/CHECKS.md](./docs/CHECKS.md).
174
+
175
+ **`vib fuzz <tool>`** generates arguments from the tool's `inputSchema`: valid,
176
+ boundary, invalid, and (with `--inject`) prompt-injection payloads. It checks invariants
177
+ that need no golden: valid args never cause a protocol error, invalid args get
178
+ `InvalidParams`, the server never crashes. Seeded and reproducible;
179
+ `--record-failures` writes each counterexample as a case. See [docs/FUZZ.md](./docs/FUZZ.md).
180
+
181
+ ## Reporters and CI
182
+
183
+ Reporters: `console` (default), `json` (`reports/results.json`), `junit`
184
+ (`reports/junit.xml`), `github` (annotations + job summary), `tap`. Use several at once.
185
+
186
+ ```yaml
187
+ # .github/workflows/vibrissa.yml (vib init --ci writes this)
188
+ name: vibrissa
189
+ on:
190
+ push:
191
+ branches: [main]
192
+ pull_request:
193
+ jobs:
194
+ contract:
195
+ runs-on: ubuntu-latest
196
+ steps:
197
+ - uses: actions/checkout@v4
198
+ - uses: actions/setup-node@v4
199
+ with: { node-version: 22, cache: npm }
200
+ - run: npm ci
201
+ - run: npm run build --if-present
202
+ - run: npx vib run --profile ci
203
+ - uses: actions/upload-artifact@v4
204
+ if: failure()
205
+ with: { name: vibrissa-reports, path: reports/ }
206
+ ```
207
+
208
+ **Delta gating:** `vib run --baseline reports/last-green.json` fails only on regressions
209
+ against a previous green run, so a known-broken case does not block unrelated PRs.
210
+ `--repeat 5` measures flake rates, `--shard 2/4` splits the suite across jobs, and every
211
+ failing case gets a JSON-RPC transcript under `reports/traces/`. Details:
212
+ [docs/CI.md](./docs/CI.md).
154
213
 
155
- ### CLI — vib / vib-test
214
+ ## Agents write great cases
156
215
 
157
- Subcommands (default is `run` when omitted):
216
+ JSON cases are the most generatable test format there is, and `record` keeps agents
217
+ honest. Start the agent with **`vib help cases`**: the whole case format, every matcher,
218
+ flows, MRTR, and tasks on one screen. Then the same loop you use:
158
219
 
159
- ```bash
160
- vib init
161
- vib tools
162
- vib call echo --arg text=hello --arg uppercase=true
163
- vib record echo --arg text=hello --arg uppercase=true --name "echo uppercase"
164
- vib run
165
- vib inspect # opens MCP Inspector with the same server config
166
- ```
220
+ 1. `vib tools --json`: the agent reads the contract, schemas included.
221
+ 2. `vib call <tool> --args '{...}' --json`: it probes behavior and edge cases.
222
+ 3. `vib record ...`: real responses become goldens. No hallucinated envelopes, and
223
+ timestamps, UUIDs, and values that change between calls are loosened to matchers
224
+ automatically. (The stable check calls the tool twice; pass `--no-stable-check` for
225
+ tools with side effects. Tools annotated destructive are called once.)
226
+ 4. `vib tap -- node dist/index.js`: or record real client sessions and pick calls with
227
+ `vib record --from-tap`.
228
+ 5. `vib run --json --brief`: the agent verifies its own suite from one JSON report on
229
+ stdout, listing only what failed.
167
230
 
168
- | Command | Purpose |
169
- |--------|---------|
170
- | `run` | Execute JSON cases (CI gate) |
171
- | `tools` | List tools + argument shapes |
172
- | `call` | One-shot `tools/call`, print result |
173
- | `record` | One-shot call → write a golden case file |
174
- | `init` | Scaffold `vibrissa.json` + sample case + npm script |
175
- | `inspect` | Launch [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector) against this server |
176
-
177
- **Division of labor:** use Inspector for visual exploration and debugging; use Vibrissa `tools` → `call` → `record` → `run` for the authoring/regression loop. `inspect` reuses your `vibrissa.json` server block so you do not re-type command/args/env.
178
-
179
- Shared options:
180
- - **--config**: path to your config file (otherwise auto-discovered from the current directory)
181
- - **--server**: override server command (quoted argv ok), e.g. `--server "node dist/index.js"`
182
- - **--server-cwd**: working directory for starting the server (overrides `server.cwd`; relative paths resolve from the shell cwd)
183
- - **--env KEY=VALUE**: extra environment (repeatable)
184
- - **--timeout**: per-call / spawn timeout in ms
185
- - **--json**: machine-readable output for `tools` / `call` / `record`
186
-
187
- `run` options:
188
- - **--cases**: glob of JSON cases (relative to the shell cwd when passed on the CLI)
189
- - **--allow-empty**: exit 0 when no cases match (by default, zero matches is a failure)
190
- - **--fail-fast**, **--pretty** / **--no-pretty**
191
-
192
- `record` options: `--out`, `--name`, `--partial`, `--force`
193
-
194
- Notes:
195
- - Globs in `vibrissa.json` resolve relative to the config file's directory.
196
- - Relative `server.cwd` values in config also resolve from that directory.
197
- - Matching zero case files exits `1` unless `--allow-empty` / `allowEmpty: true` is set.
198
- - Recorded cases include `$schema` pointing at `schema/case.schema.json` for editor validation.
199
-
200
- ### Case Layout (recommended)
201
- ```
202
- tests/
203
- integration/
204
- echo.basic.json
205
- tools/
206
- invalid-args.json
207
- flow.multistep.json
208
- ```
231
+ Everything is built for that loop: `--json` always prints one object with `ok` (errors
232
+ too, as `{ "ok": false, "error": { "type", "code", "message", "hint" }, "next" }`), unknown
233
+ flags, tools, keys, matchers, and error codes all fail with a did-you-mean, and every
234
+ error message says what to do next.
235
+
236
+ The worst an agent can do is record a case you don't want, and that is a one-file JSON
237
+ review. `vib explain` paraphrases what each case asserts and flags weak assertions (a
238
+ bare `$partial`, `expectError: {}`, no expectation), for faster PR review.
239
+
240
+ Cases can also be emitted from a [`tdd-dsl`](https://github.com/Rosavera-I/tdd-dsl)
241
+ contract: `tdd-dsl emit --target vibrissa --out-dir tests/integration contract.tdd`.
209
242
 
210
- ### Configuration (vibrissa.json)
211
- Create a `vibrissa.json` in your project root to avoid repeating flags and enable IDE IntelliSense via `$schema`.
243
+ ## Configuration
244
+
245
+ `vibrissa.json` at the repo root (`vib init` creates it):
212
246
 
213
247
  ```json
214
248
  {
215
249
  "$schema": "./node_modules/@jmoak/vibrissa/schema/vibrissa.schema.json",
216
- "server": {
217
- "cmd": "node",
218
- "args": ["dist/index.js"],
219
- "cwd": ".",
220
- "env": {
221
- "LOG_LEVEL": "info"
250
+ "servers": {
251
+ "default": { "cmd": "node", "args": ["dist/index.js"] },
252
+ "staging": {
253
+ "url": "https://mcp.example.com/mcp",
254
+ "headers": { "Authorization": "Bearer ${MCP_TOKEN}" }
222
255
  }
223
256
  },
224
- "globs": [
225
- "tests/integration/**/*.json"
226
- ],
257
+ "defaultServer": "default",
258
+ "globs": ["tests/integration/**/*.json"],
227
259
  "timeoutMs": 15000,
228
- "failFast": false
260
+ "profiles": {
261
+ "ci": { "reporters": ["console", "github", "junit", "json"], "check": true, "requireManifest": true },
262
+ "legacy": { "servers": { "default": { "protocol": "legacy" } } }
263
+ }
229
264
  }
230
265
  ```
231
266
 
232
- Then run:
233
- ```bash
234
- vib-test
235
- ```
267
+ Precedence is defaults, then the file, then `--profile`, then CLI flags. Paths resolve
268
+ from the config file's directory. Zero matching cases fails the run unless `allowEmpty`.
269
+ Every key: [docs/CONFIG.md](./docs/CONFIG.md).
270
+
271
+ ## Compatibility
272
+
273
+ v1 case files and v1 `vibrissa.json` (with `server`) load unchanged, and goldens
274
+ recorded against 2025 servers keep passing against 2026 servers because normalization
275
+ strips the new envelope fields. `vib migrate` rewrites old error-code names and the v1
276
+ `$schema` paths. See [docs/MIGRATION.md](./docs/MIGRATION.md).
236
277
 
237
- Notes:
238
- - `$schema` gives editor validation/autocomplete. Using the local path works immediately after install and offline.
239
- - CLI flags always override config values.
278
+ ## Library API
240
279
 
241
- ### Library API
242
280
  ```ts
243
- import { runCases } from "vibrissa";
244
-
245
- await runCases({
246
- server: { cmd: process.execPath, args: ["dist/index.js"], env: {} },
247
- globs: ["tests/integration/**/*.json"],
248
- failFast: true,
249
- timeoutMs: 15000
250
- });
251
- ```
281
+ import { loadConfig, runCases } from '@jmoak/vibrissa'
252
282
 
253
- ### CI (GitHub Actions)
254
- ```yaml
255
- name: integration
256
- on: [push, pull_request]
257
- jobs:
258
- test:
259
- runs-on: ubuntu-latest
260
- steps:
261
- - uses: actions/checkout@v4
262
- - uses: actions/setup-node@v4
263
- with:
264
- node-version: 20
265
- cache: npm
266
- - run: npm ci
267
- - run: npm run build
268
- - run: npx vib-test --config vibrissa.json
269
- - uses: actions/upload-artifact@v4
270
- if: always()
271
- with:
272
- name: junit
273
- path: reports/junit/junit.xml
283
+ const { options } = loadConfig(process.cwd(), { profile: 'ci' })
284
+ process.exitCode = await runCases(options) // 0 ok, 1 failures, 2 config error
274
285
  ```
275
286
 
276
- ### What to Test
277
- - Tools contract: `listTools`, `tools/call`
278
- - Arg validation: codes/messages for bad inputs
279
- - Multi‑step flows: sequences across tools
280
- - Streaming/long ops: use partial/regex for stability
281
- - Optional performance thresholds per case
287
+ The v1 call shape (`runCases({ server: { cmd, args }, globs })`) still works.
288
+ `new Runner(options).execute()` returns the full result instead of an exit code.
289
+
290
+ ### Vitest bridge
282
291
 
283
- ### Troubleshooting
284
- - Hanging cases: increase `--timeout` and inspect server logs
285
- - Flaky text: prefer `$partial` or `$regex`
286
- - Globs not matching: run with quotes and verify working directory
292
+ Teams with an existing Vitest suite can run the contract suite inside it, gated on the same
293
+ things as `vib run`:
287
294
 
288
- ### License
289
- MIT
295
+ ```ts
296
+ // tests/mcp.contract.test.ts
297
+ import { describeCases } from '@jmoak/vibrissa/vitest'
290
298
 
291
- ### Authoring from tdd-dsl
299
+ await describeCases({ profile: 'ci', checks: true })
300
+ ```
292
301
 
293
- You can author Vibrissa cases from a shared `.tdd` contract and emit JSON with [tdd-dsl](https://github.com/JMoak/tdd-dsl):
302
+ It registers a manifest-lock test per server, a checks test (with `checks`), and one `it`
303
+ per case. A string argument is a case glob. Options (`config`, `profile`, `globs`,
304
+ `checks`, `lockManifest`, `title`, `api`, runner options): [docs/CI.md](./docs/CI.md#inside-a-vitest-suite).
294
305
 
295
- ```text
296
- suite "Echo MCP contract"
297
- target vibrissa "echo-server"
306
+ Also exported: `McpSession`, `loadCases`, `executeCase`, `matchValue`,
307
+ `normalizeResult`, `diffManifest`, `createReporters`, `registerReporter`, `runChecks`,
308
+ `generateArgs`.
298
309
 
299
- case "echo basic":
300
- given input:
301
- {"text": "hello"}
302
- when call "echo"
303
- then equals:
304
- {"content": [{"type": "text", "text": "hello"}]}
305
- ```
310
+ ## Docs
306
311
 
307
- ```bash
308
- tdd-dsl emit --target vibrissa --out-dir tests/integration contract.tdd
309
- vib run
310
- ```
312
+ | Doc | Covers |
313
+ |---|---|
314
+ | [CASES.md](./docs/CASES.md) | every case-file key, flows, MRTR, tasks, recipes |
315
+ | [MATCHERS.md](./docs/MATCHERS.md) | the assertion language |
316
+ | [CONFIG.md](./docs/CONFIG.md) | `vibrissa.json`, servers, profiles |
317
+ | [CLI.md](./docs/CLI.md) | every command and flag |
318
+ | [CI.md](./docs/CI.md) | GitHub Actions, JUnit, shards, baselines |
319
+ | [CHECKS.md](./docs/CHECKS.md) | `vib check` ids |
320
+ | [FUZZ.md](./docs/FUZZ.md) | `vib fuzz` generation and invariants |
321
+ | [MIGRATION.md](./docs/MIGRATION.md) | v1 to v2 |
322
+ | [FUTURE.md](./docs/FUTURE.md) | after 2.0 |
311
323
 
312
- Mapping: `when call` → `tool`, `given input` → `args`, `then equals` → `expect`. Prefer recording once with `vib record` so `expect` matches the real MCP `tools/call` envelope.
324
+ Contributing (humans and agents): [AGENTS.md](./AGENTS.md). Releasing:
325
+ [RELEASING.md](./RELEASING.md).
313
326
 
314
- **Layering:** keep Vitest/Jest for unit tests of Vibrissa itself and of MCP handler logic. Use Vibrissa JSON for protocol e2e. Do **not** pull Python / tdd-dsl into Vibrissa CI — emit cases in the contract repo (or regenerate locally) and commit the JSON that `vib run` executes.
327
+ ## License
315
328
 
329
+ [MIT](./LICENSE) © Jordan Moak
package/bin/vib-test.js CHANGED
@@ -1,2 +1,2 @@
1
1
  #!/usr/bin/env node
2
- import('../dist/cli.js')
2
+ import('../dist/cli.js').then((cli) => cli.runCli(process.argv.slice(2)))