mcp-integration-harness 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +62 -16
- package/dist/coverage.d.ts +56 -0
- package/dist/coverage.js +85 -0
- package/dist/coverage.js.map +1 -1
- package/dist/harness.d.ts +39 -4
- package/dist/harness.js +92 -10
- package/dist/harness.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -44,6 +44,15 @@ Four things are additionally untouched by any in-process test: `src/index.ts`,
|
|
|
44
44
|
elicitation across a **process boundary**. This spawns the built artifact, so
|
|
45
45
|
all four are on the path.
|
|
46
46
|
|
|
47
|
+
On framing, be precise about what that buys. Anything the transport reports out
|
|
48
|
+
of band — a line that parses as JSON but is not a JSON-RPC message, an era
|
|
49
|
+
mismatch, an unknown message id — fails the run and names the tool it happened
|
|
50
|
+
on. A line that is not JSON **at all** is swallowed inside the SDK's read
|
|
51
|
+
buffer, below any hook a client can install; if it also lacks a trailing
|
|
52
|
+
newline it corrupts the next real message, which surfaces as a request timeout
|
|
53
|
+
with a hint pointing at stdout. Owning stdout outright would need a transport of
|
|
54
|
+
our own, and that is not what this is yet.
|
|
55
|
+
|
|
47
56
|
## `expectEveryToolExercised`
|
|
48
57
|
|
|
49
58
|
The part worth copying even if you write the rest yourself.
|
|
@@ -66,6 +75,34 @@ It fails in three directions, not one:
|
|
|
66
75
|
All three are reported together, so a fourteen-repository rollout is one
|
|
67
76
|
afternoon rather than fourteen CI rounds.
|
|
68
77
|
|
|
78
|
+
## `expectEveryToolDeclaresOutputSchema`
|
|
79
|
+
|
|
80
|
+
The same shape of check, one level up: not "was this tool called" but "does it
|
|
81
|
+
say what it returns".
|
|
82
|
+
|
|
83
|
+
```ts
|
|
84
|
+
const { tools } = await harness.client.listTools();
|
|
85
|
+
expectEveryToolDeclaresOutputSchema(tools, {
|
|
86
|
+
call_tool: "forwards a child server's result; the shape is the child's",
|
|
87
|
+
});
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
It checks **presence**, and deliberately not conformance. A server that declares
|
|
91
|
+
an `outputSchema` and then answers with something else never gets that answer
|
|
92
|
+
onto the wire — the SDK validates `structuredContent` against the advertised
|
|
93
|
+
schema server-side and turns a mismatch into a failed call. So every ordinary
|
|
94
|
+
assertion in your suite is already a schema-against-reality check, and a
|
|
95
|
+
validator in here would only re-examine data that could not have arrived if it
|
|
96
|
+
were wrong.
|
|
97
|
+
|
|
98
|
+
The one thing it does check about the schema itself is that its root is an
|
|
99
|
+
object. SEP-2106 lets an output schema describe an array or a scalar, but a
|
|
100
|
+
2025-era client is served that same tool with the schema rewritten to
|
|
101
|
+
`{result: …}` — so a tool with a non-object root answers in two different shapes
|
|
102
|
+
depending on who asked. A list is `{ items: [...] }`.
|
|
103
|
+
|
|
104
|
+
Exemptions are a `Record<tool, reason>` and rot in the same three directions.
|
|
105
|
+
|
|
69
106
|
## `assertLoopback`
|
|
70
107
|
|
|
71
108
|
The guard that matters more than the tests it protects.
|
|
@@ -82,25 +119,34 @@ production". Hosts are compared numerically via
|
|
|
82
119
|
`[::ffff:127.0.0.1]` and `localhost.` count and `127.example.com` — a hostname
|
|
83
120
|
anybody can register — does not.
|
|
84
121
|
|
|
85
|
-
`startServer` is the same idea as a property: the child gets `PATH
|
|
86
|
-
|
|
87
|
-
accident.
|
|
122
|
+
`startServer` is the same idea as a property: the child gets `PATH`, a `HOME`
|
|
123
|
+
pointing at a temporary directory, and the variables you passed. Nothing else is
|
|
124
|
+
inherited, so nothing else can be inherited by accident.
|
|
125
|
+
|
|
126
|
+
That needs enforcing rather than merely not asking for it. `StdioClientTransport`
|
|
127
|
+
merges `getDefaultEnvironment()` _underneath_ whatever it is handed, which
|
|
128
|
+
carries `HOME`, `LOGNAME`, `SHELL`, `TERM`, `USER` — and on Windows the
|
|
129
|
+
`APPDATA` family — through from the parent. A server, or any dependency of one,
|
|
130
|
+
that reads `~/.netrc`, `~/.npmrc` or a credential file under `os.homedir()`
|
|
131
|
+
would otherwise run your suite as you. Those names are blanked explicitly.
|
|
88
132
|
|
|
89
133
|
## API
|
|
90
134
|
|
|
91
|
-
| Export
|
|
92
|
-
|
|
|
93
|
-
| `startServer(options)`
|
|
94
|
-
| `harness.call(name, args)`
|
|
95
|
-
| `harness.raw(name, args)`
|
|
96
|
-
| `harness.confirmed(…)`
|
|
97
|
-
| `harness.prompts`
|
|
98
|
-
| `harness.stderr()`
|
|
99
|
-
| `expectEveryToolExercised`
|
|
100
|
-
| `toolCoverage`
|
|
101
|
-
| `
|
|
102
|
-
| `
|
|
103
|
-
| `
|
|
135
|
+
| Export | What it does |
|
|
136
|
+
| ------------------------------------- | ------------------------------------------------------------------------------- |
|
|
137
|
+
| `startServer(options)` | Spawns `dist/index.js` over real stdio and returns a `LiveHarness` |
|
|
138
|
+
| `harness.call(name, args)` | Calls a tool, records it for coverage, returns the joined text parts |
|
|
139
|
+
| `harness.raw(name, args)` | The same, returning the whole result — for a tool that answers with an image |
|
|
140
|
+
| `harness.confirmed(…)` | Drives **both halves** of the two-call token, for the no-dialog fallback path |
|
|
141
|
+
| `harness.prompts` | Every message the server put in front of the user, in order |
|
|
142
|
+
| `harness.stderr()` | Everything the server wrote to stderr, including before the handshake completed |
|
|
143
|
+
| `expectEveryToolExercised` | The three-way coverage assertion above |
|
|
144
|
+
| `toolCoverage` | The same comparison without asserting, for printing the numbers |
|
|
145
|
+
| `expectEveryToolDeclaresOutputSchema` | Every advertised tool declares an output schema with an object root |
|
|
146
|
+
| `outputSchemaCoverage` | The same comparison without asserting |
|
|
147
|
+
| `assertLoopback(url)` | Throws unless the URL is on this machine |
|
|
148
|
+
| `waitForHttp(url, opts)` | Polls until an HTTP backend is ready, and says what the last attempt got |
|
|
149
|
+
| `waitForTcp(host, port)` | The same for a backend that is not HTTP — IMAP, SMTP — optionally on a greeting |
|
|
104
150
|
|
|
105
151
|
`elicit: 'accept' | 'decline' | 'cancel'` makes the harness declare the
|
|
106
152
|
elicitation capability and answer the dialog, which is the path a real client
|
package/dist/coverage.d.ts
CHANGED
|
@@ -54,3 +54,59 @@ export declare function toolCoverage(harness: Pick<LiveHarness, 'called'>, allTo
|
|
|
54
54
|
* Throws rather than returning, so it reads as one line in a test.
|
|
55
55
|
*/
|
|
56
56
|
export declare function expectEveryToolExercised(harness: Pick<LiveHarness, 'called'>, allTools: readonly string[], skipped?: SkipReasons): void;
|
|
57
|
+
/**
|
|
58
|
+
* One entry of `tools/list`, as much of it as this check reads.
|
|
59
|
+
*
|
|
60
|
+
* Structural rather than the SDK's `Tool`, like {@link expectEveryToolExercised}
|
|
61
|
+
* takes a `Pick<LiveHarness, …>`: the result of `client.listTools()` satisfies it
|
|
62
|
+
* as it comes, and the library stays free of a type-only import from a package it
|
|
63
|
+
* only peer-depends on.
|
|
64
|
+
*/
|
|
65
|
+
export interface AdvertisedTool {
|
|
66
|
+
name: string;
|
|
67
|
+
outputSchema?: unknown;
|
|
68
|
+
}
|
|
69
|
+
export interface OutputSchemaReport {
|
|
70
|
+
declared: readonly string[];
|
|
71
|
+
exempt: readonly string[];
|
|
72
|
+
/** Advertised without an `outputSchema`, and not exempt. */
|
|
73
|
+
missing: readonly string[];
|
|
74
|
+
/** Exempt, but declares one after all — the reason is stale. */
|
|
75
|
+
staleReasons: readonly string[];
|
|
76
|
+
/** Exempt, but no longer a tool — the reason outlived its tool. */
|
|
77
|
+
unknownReasons: readonly string[];
|
|
78
|
+
/** Declared with a root that is not `"object"`. */
|
|
79
|
+
nonObjectRoot: readonly string[];
|
|
80
|
+
}
|
|
81
|
+
/**
|
|
82
|
+
* Compares the advertised tools against the rule, without asserting.
|
|
83
|
+
*
|
|
84
|
+
* Separate from the assertion for the reason {@link toolCoverage} is: "62 of 62
|
|
85
|
+
* tools declare an output schema" belongs in a CI log on a green run too.
|
|
86
|
+
*/
|
|
87
|
+
export declare function outputSchemaCoverage(tools: readonly AdvertisedTool[], exempt?: SkipReasons): OutputSchemaReport;
|
|
88
|
+
/**
|
|
89
|
+
* Fails unless every advertised tool declares an output schema with an object
|
|
90
|
+
* root.
|
|
91
|
+
*
|
|
92
|
+
* The presence half only. What the schema *says* needs no check here: a server
|
|
93
|
+
* that declares an `outputSchema` and then answers with something else never
|
|
94
|
+
* gets that answer past its own SDK, which validates `structuredContent`
|
|
95
|
+
* against the advertised schema before it goes on the wire and turns a mismatch
|
|
96
|
+
* into a failed call. So every ordinary assertion in the suite is already a
|
|
97
|
+
* schema-against-reality check, and a validator in here would only re-examine
|
|
98
|
+
* data that could not have arrived if it were wrong.
|
|
99
|
+
*
|
|
100
|
+
* The object root is checked, though, and is not pedantry. SEP-2106 lets an
|
|
101
|
+
* output schema describe an array or a scalar, but a 2025-era client is served
|
|
102
|
+
* that same tool with the schema rewritten to `{result: …}` — so a tool with a
|
|
103
|
+
* non-object root answers in two different shapes depending on who asked. A
|
|
104
|
+
* list is `{ items: [...] }`.
|
|
105
|
+
*
|
|
106
|
+
* Exemptions take a written reason and rot in the same three directions
|
|
107
|
+
* {@link expectEveryToolExercised} guards against. The one that has earned its
|
|
108
|
+
* place so far:
|
|
109
|
+
*
|
|
110
|
+
* call_tool: 'forwards a child server's result; the shape is the child's'
|
|
111
|
+
*/
|
|
112
|
+
export declare function expectEveryToolDeclaresOutputSchema(tools: readonly AdvertisedTool[], exempt?: SkipReasons): void;
|
package/dist/coverage.js
CHANGED
|
@@ -55,4 +55,89 @@ export function expectEveryToolExercised(harness, allTools, skipped = {}) {
|
|
|
55
55
|
`${report.skipped.length} excused.\n\n${problems.join('\n\n')}`);
|
|
56
56
|
}
|
|
57
57
|
}
|
|
58
|
+
/** Whether a value is a JSON Schema whose instance root is an object. */
|
|
59
|
+
function hasObjectRoot(schema) {
|
|
60
|
+
if (typeof schema !== 'object' || schema === null)
|
|
61
|
+
return false;
|
|
62
|
+
return schema.type === 'object';
|
|
63
|
+
}
|
|
64
|
+
/**
|
|
65
|
+
* Compares the advertised tools against the rule, without asserting.
|
|
66
|
+
*
|
|
67
|
+
* Separate from the assertion for the reason {@link toolCoverage} is: "62 of 62
|
|
68
|
+
* tools declare an output schema" belongs in a CI log on a green run too.
|
|
69
|
+
*/
|
|
70
|
+
export function outputSchemaCoverage(tools, exempt = {}) {
|
|
71
|
+
const names = new Set(tools.map((tool) => tool.name));
|
|
72
|
+
const reasons = Object.keys(exempt);
|
|
73
|
+
const withSchema = tools.filter((tool) => tool.outputSchema !== undefined);
|
|
74
|
+
return {
|
|
75
|
+
declared: withSchema.map((tool) => tool.name).sort(),
|
|
76
|
+
exempt: reasons.sort(),
|
|
77
|
+
missing: tools
|
|
78
|
+
.filter((tool) => tool.outputSchema === undefined && !(tool.name in exempt))
|
|
79
|
+
.map((tool) => tool.name)
|
|
80
|
+
.sort(),
|
|
81
|
+
staleReasons: withSchema
|
|
82
|
+
.filter((tool) => tool.name in exempt)
|
|
83
|
+
.map((tool) => tool.name)
|
|
84
|
+
.sort(),
|
|
85
|
+
unknownReasons: reasons.filter((name) => !names.has(name)).sort(),
|
|
86
|
+
nonObjectRoot: withSchema
|
|
87
|
+
.filter((tool) => !hasObjectRoot(tool.outputSchema))
|
|
88
|
+
.map((tool) => tool.name)
|
|
89
|
+
.sort(),
|
|
90
|
+
};
|
|
91
|
+
}
|
|
92
|
+
/**
|
|
93
|
+
* Fails unless every advertised tool declares an output schema with an object
|
|
94
|
+
* root.
|
|
95
|
+
*
|
|
96
|
+
* The presence half only. What the schema *says* needs no check here: a server
|
|
97
|
+
* that declares an `outputSchema` and then answers with something else never
|
|
98
|
+
* gets that answer past its own SDK, which validates `structuredContent`
|
|
99
|
+
* against the advertised schema before it goes on the wire and turns a mismatch
|
|
100
|
+
* into a failed call. So every ordinary assertion in the suite is already a
|
|
101
|
+
* schema-against-reality check, and a validator in here would only re-examine
|
|
102
|
+
* data that could not have arrived if it were wrong.
|
|
103
|
+
*
|
|
104
|
+
* The object root is checked, though, and is not pedantry. SEP-2106 lets an
|
|
105
|
+
* output schema describe an array or a scalar, but a 2025-era client is served
|
|
106
|
+
* that same tool with the schema rewritten to `{result: …}` — so a tool with a
|
|
107
|
+
* non-object root answers in two different shapes depending on who asked. A
|
|
108
|
+
* list is `{ items: [...] }`.
|
|
109
|
+
*
|
|
110
|
+
* Exemptions take a written reason and rot in the same three directions
|
|
111
|
+
* {@link expectEveryToolExercised} guards against. The one that has earned its
|
|
112
|
+
* place so far:
|
|
113
|
+
*
|
|
114
|
+
* call_tool: 'forwards a child server's result; the shape is the child's'
|
|
115
|
+
*/
|
|
116
|
+
export function expectEveryToolDeclaresOutputSchema(tools, exempt = {}) {
|
|
117
|
+
const report = outputSchemaCoverage(tools, exempt);
|
|
118
|
+
const problems = [];
|
|
119
|
+
if (report.missing.length > 0) {
|
|
120
|
+
problems.push(`${report.missing.length} tool(s) declare no outputSchema: ` +
|
|
121
|
+
`${report.missing.join(', ')}. Declare one and return structuredContent, ` +
|
|
122
|
+
'or give each a reason saying why the shape is not this server to state.');
|
|
123
|
+
}
|
|
124
|
+
if (report.staleReasons.length > 0) {
|
|
125
|
+
problems.push(`${report.staleReasons.length} exempt tool(s) declare one after all: ` +
|
|
126
|
+
`${report.staleReasons.join(', ')}. Remove the reason — it is no longer true.`);
|
|
127
|
+
}
|
|
128
|
+
if (report.unknownReasons.length > 0) {
|
|
129
|
+
problems.push(`${report.unknownReasons.length} reason(s) name a tool that no longer ` +
|
|
130
|
+
`exists: ${report.unknownReasons.join(', ')}.`);
|
|
131
|
+
}
|
|
132
|
+
if (report.nonObjectRoot.length > 0) {
|
|
133
|
+
problems.push(`${report.nonObjectRoot.length} tool(s) declare a non-object root: ` +
|
|
134
|
+
`${report.nonObjectRoot.join(', ')}. A 2025-era client is served that ` +
|
|
135
|
+
'schema wrapped as {result: …}, so the answer has two shapes. Wrap the ' +
|
|
136
|
+
'value in an object — a list is { items: [...] }.');
|
|
137
|
+
}
|
|
138
|
+
if (problems.length > 0) {
|
|
139
|
+
throw new Error(`${report.declared.length} of ${tools.length} tools declare an output ` +
|
|
140
|
+
`schema, ${report.exempt.length} exempt.\n\n${problems.join('\n\n')}`);
|
|
141
|
+
}
|
|
142
|
+
}
|
|
58
143
|
//# sourceMappingURL=coverage.js.map
|
package/dist/coverage.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"coverage.js","sourceRoot":"","sources":["../src/coverage.ts"],"names":[],"mappings":"AAkCA;;;;;;GAMG;AACH,MAAM,UAAU,YAAY,CAC1B,OAAoC,EACpC,QAA2B,EAC3B,OAAoB;IAEpB,MAAM,SAAS,GAAG,IAAI,GAAG,CAAC,QAAQ,CAAC,CAAC;IACpC,MAAM,OAAO,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;IACrC,OAAO;QACL,MAAM,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE;QAClC,OAAO,EAAE,OAAO,CAAC,IAAI,EAAE;QACvB,OAAO,EAAE,QAAQ;aACd,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,IAAI,OAAO,CAAC,CAAC;aACjE,IAAI,EAAE;QACT,YAAY,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,EAAE;QACvE,cAAc,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,EAAE;KACtE,CAAC;AACJ,CAAC;AAED;;;;;;;;;;;;;;;GAeG;AACH,MAAM,UAAU,wBAAwB,CACtC,OAAoC,EACpC,QAA2B,EAC3B,OAAO,GAAgB,EAAE;IAEzB,MAAM,MAAM,GAAG,YAAY,CAAC,OAAO,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IACxD,MAAM,QAAQ,GAAa,EAAE,CAAC;IAE9B,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9B,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,yCAAyC;YAC/D,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,4CAA4C;YACxE,qDAAqD,CACxD,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACnC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,YAAY,CAAC,MAAM,0CAA0C;YACrE,GAAG,MAAM,CAAC,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,6CAA6C,CACjF,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACrC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,cAAc,CAAC,MAAM,wCAAwC;YACrE,WAAW,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CACjD,CAAC;IACJ,CAAC;IAED,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CACb,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM,OAAO,QAAQ,CAAC,MAAM,oBAAoB;YAC/D,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,gBAAgB,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAClE,CAAC;IACJ,CAAC;AACH,CAAC"}
|
|
1
|
+
{"version":3,"file":"coverage.js","sourceRoot":"","sources":["../src/coverage.ts"],"names":[],"mappings":"AAkCA;;;;;;GAMG;AACH,MAAM,UAAU,YAAY,CAC1B,OAAoC,EACpC,QAA2B,EAC3B,OAAoB;IAEpB,MAAM,SAAS,GAAG,IAAI,GAAG,CAAC,QAAQ,CAAC,CAAC;IACpC,MAAM,OAAO,GAAG,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;IACrC,OAAO;QACL,MAAM,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE;QAClC,OAAO,EAAE,OAAO,CAAC,IAAI,EAAE;QACvB,OAAO,EAAE,QAAQ;aACd,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,IAAI,OAAO,CAAC,CAAC;aACjE,IAAI,EAAE;QACT,YAAY,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,EAAE;QACvE,cAAc,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,EAAE;KACtE,CAAC;AACJ,CAAC;AAED;;;;;;;;;;;;;;;GAeG;AACH,MAAM,UAAU,wBAAwB,CACtC,OAAoC,EACpC,QAA2B,EAC3B,OAAO,GAAgB,EAAE;IAEzB,MAAM,MAAM,GAAG,YAAY,CAAC,OAAO,EAAE,QAAQ,EAAE,OAAO,CAAC,CAAC;IACxD,MAAM,QAAQ,GAAa,EAAE,CAAC;IAE9B,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9B,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,yCAAyC;YAC/D,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,4CAA4C;YACxE,qDAAqD,CACxD,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACnC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,YAAY,CAAC,MAAM,0CAA0C;YACrE,GAAG,MAAM,CAAC,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,6CAA6C,CACjF,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACrC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,cAAc,CAAC,MAAM,wCAAwC;YACrE,WAAW,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CACjD,CAAC;IACJ,CAAC;IAED,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CACb,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM,OAAO,QAAQ,CAAC,MAAM,oBAAoB;YAC/D,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,gBAAgB,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAClE,CAAC;IACJ,CAAC;AACH,CAAC;AA4BD,yEAAyE;AACzE,SAAS,aAAa,CAAC,MAAe;IACpC,IAAI,OAAO,MAAM,KAAK,QAAQ,IAAI,MAAM,KAAK,IAAI;QAAE,OAAO,KAAK,CAAC;IAChE,OAAQ,MAA6B,CAAC,IAAI,KAAK,QAAQ,CAAC;AAC1D,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,oBAAoB,CAClC,KAAgC,EAChC,MAAM,GAAgB,EAAE;IAExB,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC;IACtD,MAAM,OAAO,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACpC,MAAM,UAAU,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,YAAY,KAAK,SAAS,CAAC,CAAC;IAC3E,OAAO;QACL,QAAQ,EAAE,UAAU,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE;QACpD,MAAM,EAAE,OAAO,CAAC,IAAI,EAAE;QACtB,OAAO,EAAE,KAAK;aACX,MAAM,CACL,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,CAAC,CAAC,IAAI,CAAC,IAAI,IAAI,MAAM,CAAC,CACpE;aACA,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;aACxB,IAAI,EAAE;QACT,YAAY,EAAE,UAAU;aACrB,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,IAAI,MAAM,CAAC;aACrC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;aACxB,IAAI,EAAE;QACT,cAAc,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,EAAE;QACjE,aAAa,EAAE,UAAU;aACtB,MAAM,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;aACnD,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;aACxB,IAAI,EAAE;KACV,CAAC;AACJ,CAAC;AAED;;;;;;;;;;;;;;;;;;;;;;;GAuBG;AACH,MAAM,UAAU,mCAAmC,CACjD,KAAgC,EAChC,MAAM,GAAgB,EAAE;IAExB,MAAM,MAAM,GAAG,oBAAoB,CAAC,KAAK,EAAE,MAAM,CAAC,CAAC;IACnD,MAAM,QAAQ,GAAa,EAAE,CAAC;IAE9B,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9B,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,oCAAoC;YAC1D,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,8CAA8C;YAC1E,yEAAyE,CAC5E,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACnC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,YAAY,CAAC,MAAM,yCAAyC;YACpE,GAAG,MAAM,CAAC,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,6CAA6C,CACjF,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,cAAc,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACrC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,cAAc,CAAC,MAAM,wCAAwC;YACrE,WAAW,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CACjD,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,CAAC,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACpC,QAAQ,CAAC,IAAI,CACX,GAAG,MAAM,CAAC,aAAa,CAAC,MAAM,sCAAsC;YAClE,GAAG,MAAM,CAAC,aAAa,CAAC,IAAI,CAAC,IAAI,CAAC,qCAAqC;YACvE,wEAAwE;YACxE,kDAAkD,CACrD,CAAC;IACJ,CAAC;IAED,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CACb,GAAG,MAAM,CAAC,QAAQ,CAAC,MAAM,OAAO,KAAK,CAAC,MAAM,2BAA2B;YACrE,WAAW,MAAM,CAAC,MAAM,CAAC,MAAM,eAAe,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CACxE,CAAC;IACJ,CAAC;AACH,CAAC"}
|
package/dist/harness.d.ts
CHANGED
|
@@ -21,8 +21,17 @@ export interface StartServerOptions {
|
|
|
21
21
|
*
|
|
22
22
|
* Deliberately not merged with `process.env`. A `WIKIJS_URL` left in a shell
|
|
23
23
|
* is otherwise enough to point an integration run — deletes included — at
|
|
24
|
-
* whatever that variable happens to name.
|
|
25
|
-
*
|
|
24
|
+
* whatever that variable happens to name.
|
|
25
|
+
*
|
|
26
|
+
* "Nothing is inherited" needs enforcing rather than merely not asking for
|
|
27
|
+
* it: `StdioClientTransport` merges `getDefaultEnvironment()` under whatever
|
|
28
|
+
* it is given, which carries `HOME`, `LOGNAME`, `SHELL`, `TERM`, `USER` and
|
|
29
|
+
* on Windows the `APPDATA` family through from the parent. A server — or any
|
|
30
|
+
* dependency of one — that reads `~/.netrc`, `~/.npmrc` or a credential file
|
|
31
|
+
* under `os.homedir()` would then run the suite as the developer. Those names
|
|
32
|
+
* are blanked here, and `HOME` points at a temporary directory rather than
|
|
33
|
+
* being empty, because an empty `HOME` breaks tools in a way that reads like
|
|
34
|
+
* a bug in the server.
|
|
26
35
|
*/
|
|
27
36
|
env: Record<string, string>;
|
|
28
37
|
/**
|
|
@@ -35,8 +44,17 @@ export interface StartServerOptions {
|
|
|
35
44
|
timeoutSeconds?: number;
|
|
36
45
|
}
|
|
37
46
|
export interface CallOptions {
|
|
38
|
-
/**
|
|
39
|
-
|
|
47
|
+
/**
|
|
48
|
+
* Assert that the call fails. Refusals are behaviour worth pinning too.
|
|
49
|
+
*
|
|
50
|
+
* `true` only asserts that *something* failed, which is weaker than it looks:
|
|
51
|
+
* a renamed parameter makes the schema reject the call, and a guard test
|
|
52
|
+
* written this way stays green while the guard it names is no longer reached.
|
|
53
|
+
* Pass a string or a `RegExp` to require the reason as well — the returned
|
|
54
|
+
* text has to contain it, or match it — and prefer that wherever the refusal
|
|
55
|
+
* is the point of the test.
|
|
56
|
+
*/
|
|
57
|
+
expectError?: boolean | string | RegExp;
|
|
40
58
|
}
|
|
41
59
|
export interface ToolResult {
|
|
42
60
|
content?: {
|
|
@@ -45,6 +63,17 @@ export interface ToolResult {
|
|
|
45
63
|
mimeType?: string;
|
|
46
64
|
data?: string;
|
|
47
65
|
}[];
|
|
66
|
+
/**
|
|
67
|
+
* The machine-readable half of an answer, for a tool that declares an
|
|
68
|
+
* `outputSchema`.
|
|
69
|
+
*
|
|
70
|
+
* `unknown` rather than a shape, because the shape is the server's to declare
|
|
71
|
+
* and the suite's to assert. Worth knowing: a value that reaches here has
|
|
72
|
+
* already been validated against the advertised schema — the SDK does that
|
|
73
|
+
* server-side and turns a mismatch into a failed call — so a test that reads
|
|
74
|
+
* this field is checking *which* data came back, not whether it conformed.
|
|
75
|
+
*/
|
|
76
|
+
structuredContent?: unknown;
|
|
48
77
|
isError?: boolean;
|
|
49
78
|
}
|
|
50
79
|
export interface LiveHarness {
|
|
@@ -73,6 +102,12 @@ export interface LiveHarness {
|
|
|
73
102
|
* Only meaningful on a harness started **without** `elicit`: with a dialog
|
|
74
103
|
* available the server refuses to offer a token at all, which is the whole
|
|
75
104
|
* point of the dialog. Use it to prove the fallback path still works.
|
|
105
|
+
*
|
|
106
|
+
* The first half is expected to **fail** — `mcp-approval` marks the prompt
|
|
107
|
+
* `isError`, because the operation was asked for and did not happen, and
|
|
108
|
+
* because a tool that declares an `outputSchema` may not answer without
|
|
109
|
+
* `structuredContent` unless the result is an error. The second half is
|
|
110
|
+
* expected to succeed, and its text is what comes back.
|
|
76
111
|
*/
|
|
77
112
|
confirmed(name: string, args?: Record<string, unknown>): Promise<string>;
|
|
78
113
|
/** Every message the server put in front of the user, in order. */
|
package/dist/harness.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
|
+
import { tmpdir } from 'node:os';
|
|
1
2
|
import { Client } from '@modelcontextprotocol/client';
|
|
2
|
-
import { StdioClientTransport } from '@modelcontextprotocol/client/stdio';
|
|
3
|
+
import { DEFAULT_INHERITED_ENV_VARS, StdioClientTransport, } from '@modelcontextprotocol/client/stdio';
|
|
3
4
|
/** Pulls the fallback token out of a refusal. */
|
|
4
5
|
export function tokenOf(text) {
|
|
5
6
|
const match = /confirm_token="([0-9a-f]+)"/.exec(text);
|
|
@@ -22,6 +23,7 @@ export async function startServer(options) {
|
|
|
22
23
|
const prompts = [];
|
|
23
24
|
const called = new Set();
|
|
24
25
|
const errors = [];
|
|
26
|
+
const protocolErrors = [];
|
|
25
27
|
const client = new Client({ name: 'mcp-integration-harness', version: '0.1.0' }, options.elicit === undefined ? {} : { capabilities: { elicitation: {} } });
|
|
26
28
|
if (options.elicit !== undefined) {
|
|
27
29
|
const behaviour = options.elicit;
|
|
@@ -37,11 +39,33 @@ export async function startServer(options) {
|
|
|
37
39
|
return { action: 'accept', content: { confirm: true } };
|
|
38
40
|
});
|
|
39
41
|
}
|
|
42
|
+
// Out-of-band protocol failures. The transport reports a line that parses as
|
|
43
|
+
// JSON but is not a JSON-RPC message here — the `console.log(JSON.stringify(x))`
|
|
44
|
+
// that a server picks up from a dependency — and without a listener it is
|
|
45
|
+
// discarded, leaving a suite green while the framing this library exists to
|
|
46
|
+
// exercise is broken. Also catches era mismatches, unknown message ids and
|
|
47
|
+
// dropped inbound requests.
|
|
48
|
+
//
|
|
49
|
+
// The one case this does *not* reach is a line that is not JSON at all:
|
|
50
|
+
// `ReadBuffer.readMessage` swallows the SyntaxError inside the buffer, below
|
|
51
|
+
// any hook a client can install. Such a line without a trailing newline
|
|
52
|
+
// corrupts the next real message instead, which surfaces as a request
|
|
53
|
+
// timeout — see the hint in the failure below.
|
|
54
|
+
client.onerror = (error) => {
|
|
55
|
+
protocolErrors.push(error);
|
|
56
|
+
};
|
|
40
57
|
const transport = new StdioClientTransport({
|
|
41
58
|
command: process.execPath,
|
|
42
59
|
args: [entry],
|
|
43
|
-
// PATH only
|
|
44
|
-
|
|
60
|
+
// PATH only, and the SDK's inherit list explicitly blanked — it merges
|
|
61
|
+
// getDefaultEnvironment() underneath whatever it is handed. See the comment
|
|
62
|
+
// on StartServerOptions.env.
|
|
63
|
+
env: {
|
|
64
|
+
...Object.fromEntries(DEFAULT_INHERITED_ENV_VARS.map((name) => [name, ''])),
|
|
65
|
+
HOME: tmpdir(),
|
|
66
|
+
PATH: process.env.PATH ?? '',
|
|
67
|
+
...options.env,
|
|
68
|
+
},
|
|
45
69
|
...(options.cwd === undefined ? {} : { cwd: options.cwd }),
|
|
46
70
|
stderr: 'pipe',
|
|
47
71
|
});
|
|
@@ -53,7 +77,9 @@ export async function startServer(options) {
|
|
|
53
77
|
errors.push(chunk.toString());
|
|
54
78
|
});
|
|
55
79
|
try {
|
|
56
|
-
await client.connect(transport
|
|
80
|
+
await client.connect(transport, {
|
|
81
|
+
timeout: (options.timeoutSeconds ?? 30) * 1000,
|
|
82
|
+
});
|
|
57
83
|
}
|
|
58
84
|
catch (error) {
|
|
59
85
|
// A server that dies during the handshake reports "Connection closed" and
|
|
@@ -63,7 +89,16 @@ export async function startServer(options) {
|
|
|
63
89
|
throw new Error(`mcp-integration-harness: ${process.execPath} ${entry} did not start.\n` +
|
|
64
90
|
`${String(error)}\n\nIts stderr:\n${errors.join('') || '(nothing)'}`);
|
|
65
91
|
}
|
|
66
|
-
|
|
92
|
+
/**
|
|
93
|
+
* The call itself: coverage bookkeeping, the transport, the framing check.
|
|
94
|
+
*
|
|
95
|
+
* Separate from {@link raw} because one caller has no expectation to state.
|
|
96
|
+
* `confirmed()` drives a path whose first half is an error result on a
|
|
97
|
+
* current `mcp-approval` and was not on an older one, and neither branch of
|
|
98
|
+
* `expectError` describes "I am about to read a token out of this, and if it
|
|
99
|
+
* is not there I have a better sentence than either".
|
|
100
|
+
*/
|
|
101
|
+
const invoke = async (name, args) => {
|
|
67
102
|
called.add(name);
|
|
68
103
|
let result;
|
|
69
104
|
try {
|
|
@@ -79,18 +114,60 @@ export async function startServer(options) {
|
|
|
79
114
|
// since it started.
|
|
80
115
|
throw new Error(`mcp-integration-harness: calling ${name} failed at the transport.\n` +
|
|
81
116
|
`${String(error)}\n\nThe server's stderr so far:\n` +
|
|
82
|
-
`${errors.join('') || '(nothing)'}`
|
|
117
|
+
`${errors.join('') || '(nothing)'}\n\n` +
|
|
118
|
+
'If that stderr is empty and this was a timeout, suspect stdout: a ' +
|
|
119
|
+
'write there without a trailing newline is prepended to the next ' +
|
|
120
|
+
'JSON-RPC message, and the reply is discarded inside the read buffer ' +
|
|
121
|
+
'where no hook can see it. stdout belongs to the transport.');
|
|
83
122
|
}
|
|
123
|
+
assertFramingIntact(`calling ${name}`);
|
|
124
|
+
return result;
|
|
125
|
+
};
|
|
126
|
+
const raw = async (name, args = {}, callOptions = {}) => {
|
|
127
|
+
const result = await invoke(name, args);
|
|
128
|
+
const expectation = callOptions.expectError ?? false;
|
|
129
|
+
const wantFailure = expectation !== false;
|
|
84
130
|
const failed = result.isError === true;
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
throw new Error(
|
|
131
|
+
const text = textOf(result);
|
|
132
|
+
if (failed !== wantFailure) {
|
|
133
|
+
throw new Error(wantFailure
|
|
88
134
|
? `${name} was expected to fail and did not: ${text.slice(0, 500)}`
|
|
89
135
|
: `${name} failed: ${text.slice(0, 500)}`);
|
|
90
136
|
}
|
|
137
|
+
// A refusal that does not say why is a refusal that could have come from
|
|
138
|
+
// anywhere — the schema, a 500, a renamed argument. Where the caller named
|
|
139
|
+
// the reason, it has to be the reason.
|
|
140
|
+
if (typeof expectation === 'string' && !text.includes(expectation)) {
|
|
141
|
+
throw new Error(`${name} failed as expected, but not for the stated reason.\n` +
|
|
142
|
+
`Expected the message to contain: ${expectation}\n` +
|
|
143
|
+
`Got: ${text.slice(0, 500)}`);
|
|
144
|
+
}
|
|
145
|
+
if (expectation instanceof RegExp && !expectation.test(text)) {
|
|
146
|
+
throw new Error(`${name} failed as expected, but not for the stated reason.\n` +
|
|
147
|
+
`Expected the message to match: ${String(expectation)}\n` +
|
|
148
|
+
`Got: ${text.slice(0, 500)}`);
|
|
149
|
+
}
|
|
91
150
|
return result;
|
|
92
151
|
};
|
|
93
152
|
const call = async (name, args = {}, callOptions = {}) => textOf(await raw(name, args, callOptions));
|
|
153
|
+
/**
|
|
154
|
+
* Fails the run if the transport reported anything out of band.
|
|
155
|
+
*
|
|
156
|
+
* Checked after each call rather than only at the end, so the failure names
|
|
157
|
+
* the tool whose turn produced it instead of the whole suite.
|
|
158
|
+
*/
|
|
159
|
+
function assertFramingIntact(during) {
|
|
160
|
+
if (protocolErrors.length === 0)
|
|
161
|
+
return;
|
|
162
|
+
const reported = protocolErrors.map((error) => error.message).join('\n');
|
|
163
|
+
protocolErrors.length = 0;
|
|
164
|
+
throw new Error(`mcp-integration-harness: the server broke the stdio framing while ${during}.\n` +
|
|
165
|
+
`${reported}\n\n` +
|
|
166
|
+
'Something reached stdout that is not a JSON-RPC message — a stray ' +
|
|
167
|
+
'console.log in the server or in one of its dependencies is the usual ' +
|
|
168
|
+
'cause. Route it to stderr.\n\n' +
|
|
169
|
+
`The server's stderr so far:\n${errors.join('') || '(nothing)'}`);
|
|
170
|
+
}
|
|
94
171
|
return {
|
|
95
172
|
client,
|
|
96
173
|
call,
|
|
@@ -99,11 +176,16 @@ export async function startServer(options) {
|
|
|
99
176
|
called,
|
|
100
177
|
stderr: () => errors.join(''),
|
|
101
178
|
confirmed: async (name, args = {}) => {
|
|
102
|
-
|
|
179
|
+
// `invoke`, not `call`: the first half is an error result — the prompt
|
|
180
|
+
// says the operation did not happen — and asserting on that here would
|
|
181
|
+
// replace `tokenOf`'s sentence, which names the real mistake (calling
|
|
182
|
+
// this on a harness started *with* `elicit`), with a generic one.
|
|
183
|
+
const first = textOf(await invoke(name, args));
|
|
103
184
|
return call(name, { ...args, confirm_token: tokenOf(first) });
|
|
104
185
|
},
|
|
105
186
|
close: async () => {
|
|
106
187
|
await client.close();
|
|
188
|
+
assertFramingIntact('closing the session');
|
|
107
189
|
},
|
|
108
190
|
};
|
|
109
191
|
}
|
package/dist/harness.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"harness.js","sourceRoot":"","sources":["../src/harness.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,EAAE,MAAM,8BAA8B,CAAC;AACtD,OAAO,
|
|
1
|
+
{"version":3,"file":"harness.js","sourceRoot":"","sources":["../src/harness.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,MAAM,EAAE,MAAM,SAAS,CAAC;AAEjC,OAAO,EAAE,MAAM,EAAE,MAAM,8BAA8B,CAAC;AACtD,OAAO,EACL,0BAA0B,EAC1B,oBAAoB,GACrB,MAAM,oCAAoC,CAAC;AAkI5C,iDAAiD;AACjD,MAAM,UAAU,OAAO,CAAC,IAAY;IAClC,MAAM,KAAK,GAAG,6BAA6B,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACvD,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;QAChB,MAAM,IAAI,KAAK,CACb,uEAAuE;YACrE,QAAQ,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAC/B,CAAC;IACJ,CAAC;IACD,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC;AAED,+CAA+C;AAC/C,SAAS,MAAM,CAAC,MAA6B;IAC3C,MAAM,KAAK,GAAG,CAAC,MAAM,CAAC,OAAO,IAAI,EAAE,CAAsC,CAAC;IAC1E,OAAO,KAAK;SACT,MAAM,CACL,CAAC,IAAI,EAA0C,EAAE,CAAC,IAAI,CAAC,IAAI,KAAK,MAAM,CACvE;SACA,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC;SACxB,IAAI,CAAC,IAAI,CAAC,CAAC;AAChB,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,WAAW,CAC/B,OAA2B;IAE3B,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,IAAI,eAAe,CAAC;IAC/C,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,MAAM,MAAM,GAAG,IAAI,GAAG,EAAU,CAAC;IACjC,MAAM,MAAM,GAAa,EAAE,CAAC;IAC5B,MAAM,cAAc,GAAY,EAAE,CAAC;IAEnC,MAAM,MAAM,GAAG,IAAI,MAAM,CACvB,EAAE,IAAI,EAAE,yBAAyB,EAAE,OAAO,EAAE,OAAO,EAAE,EACrD,OAAO,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,EAAE,WAAW,EAAE,EAAE,EAAE,EAAE,CAC1E,CAAC;IACF,IAAI,OAAO,CAAC,MAAM,KAAK,SAAS,EAAE,CAAC;QACjC,MAAM,SAAS,GAAG,OAAO,CAAC,MAAM,CAAC;QACjC,MAAM,CAAC,iBAAiB,CAAC,oBAAoB,EAAE,CAAC,OAAO,EAAE,EAAE;YACzD,qEAAqE;YACrE,kEAAkE;YAClE,uCAAuC;YACvC,OAAO,CAAC,IAAI,CAAE,OAAO,CAAC,MAA8B,CAAC,OAAO,CAAC,CAAC;YAC9D,IAAI,SAAS,KAAK,QAAQ;gBAAE,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,CAAC;YACxD,IAAI,SAAS,KAAK,SAAS;gBAAE,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,CAAC;YAC1D,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,EAAE,OAAO,EAAE,IAAI,EAAE,EAAE,CAAC;QAC1D,CAAC,CAAC,CAAC;IACL,CAAC;IAED,6EAA6E;IAC7E,iFAAiF;IACjF,0EAA0E;IAC1E,4EAA4E;IAC5E,2EAA2E;IAC3E,4BAA4B;IAC5B,EAAE;IACF,wEAAwE;IACxE,6EAA6E;IAC7E,wEAAwE;IACxE,sEAAsE;IACtE,+CAA+C;IAC/C,MAAM,CAAC,OAAO,GAAG,CAAC,KAAY,EAAE,EAAE;QAChC,cAAc,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IAC7B,CAAC,CAAC;IAEF,MAAM,SAAS,GAAG,IAAI,oBAAoB,CAAC;QACzC,OAAO,EAAE,OAAO,CAAC,QAAQ;QACzB,IAAI,EAAE,CAAC,KAAK,CAAC;QACb,uEAAuE;QACvE,4EAA4E;QAC5E,6BAA6B;QAC7B,GAAG,EAAE;YACH,GAAG,MAAM,CAAC,WAAW,CACnB,0BAA0B,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,CACrD;YACD,IAAI,EAAE,MAAM,EAAE;YACd,IAAI,EAAE,OAAO,CAAC,GAAG,CAAC,IAAI,IAAI,EAAE;YAC5B,GAAG,OAAO,CAAC,GAAG;SACf;QACD,GAAG,CAAC,OAAO,CAAC,GAAG,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE,CAAC;QAC1D,MAAM,EAAE,MAAM;KACf,CAAC,CAAC;IAEH,4EAA4E;IAC5E,0EAA0E;IAC1E,kEAAkE;IAClE,uCAAuC;IACvC,SAAS,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,EAAE,CAAC,KAAa,EAAE,EAAE;QAC7C,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,QAAQ,EAAE,CAAC,CAAC;IAChC,CAAC,CAAC,CAAC;IAEH,IAAI,CAAC;QACH,MAAM,MAAM,CAAC,OAAO,CAAC,SAAS,EAAE;YAC9B,OAAO,EAAE,CAAC,OAAO,CAAC,cAAc,IAAI,EAAE,CAAC,GAAG,IAAI;SAC/C,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,0EAA0E;QAC1E,sEAAsE;QACtE,4EAA4E;QAC5E,0EAA0E;QAC1E,MAAM,IAAI,KAAK,CACb,4BAA4B,OAAO,CAAC,QAAQ,IAAI,KAAK,mBAAmB;YACtE,GAAG,MAAM,CAAC,KAAK,CAAC,oBAAoB,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,WAAW,EAAE,CACvE,CAAC;IACJ,CAAC;IAED;;;;;;;;OAQG;IACH,MAAM,MAAM,GAAG,KAAK,EAClB,IAAY,EACZ,IAA6B,EACR,EAAE;QACvB,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QACjB,IAAI,MAAkB,CAAC;QACvB,IAAI,CAAC;YACH,MAAM,GAAG,CAAC,MAAM,MAAM,CAAC,QAAQ,CAAC;gBAC9B,IAAI;gBACJ,SAAS,EAAE,IAAI;aAChB,CAAC,CAAe,CAAC;QACpB,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,uEAAuE;YACvE,uEAAuE;YACvE,oEAAoE;YACpE,oBAAoB;YACpB,MAAM,IAAI,KAAK,CACb,oCAAoC,IAAI,6BAA6B;gBACnE,GAAG,MAAM,CAAC,KAAK,CAAC,mCAAmC;gBACnD,GAAG,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,WAAW,MAAM;gBACvC,oEAAoE;gBACpE,kEAAkE;gBAClE,sEAAsE;gBACtE,4DAA4D,CAC/D,CAAC;QACJ,CAAC;QACD,mBAAmB,CAAC,WAAW,IAAI,EAAE,CAAC,CAAC;QACvC,OAAO,MAAM,CAAC;IAChB,CAAC,CAAC;IAEF,MAAM,GAAG,GAAG,KAAK,EACf,IAAY,EACZ,IAAI,GAA4B,EAAE,EAClC,WAAW,GAAgB,EAAE,EACR,EAAE;QACvB,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACxC,MAAM,WAAW,GAAG,WAAW,CAAC,WAAW,IAAI,KAAK,CAAC;QACrD,MAAM,WAAW,GAAG,WAAW,KAAK,KAAK,CAAC;QAC1C,MAAM,MAAM,GAAG,MAAM,CAAC,OAAO,KAAK,IAAI,CAAC;QACvC,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC;QAC5B,IAAI,MAAM,KAAK,WAAW,EAAE,CAAC;YAC3B,MAAM,IAAI,KAAK,CACb,WAAW;gBACT,CAAC,CAAC,GAAG,IAAI,sCAAsC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE;gBACnE,CAAC,CAAC,GAAG,IAAI,YAAY,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAC5C,CAAC;QACJ,CAAC;QACD,yEAAyE;QACzE,2EAA2E;QAC3E,uCAAuC;QACvC,IAAI,OAAO,WAAW,KAAK,QAAQ,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,EAAE,CAAC;YACnE,MAAM,IAAI,KAAK,CACb,GAAG,IAAI,uDAAuD;gBAC5D,oCAAoC,WAAW,IAAI;gBACnD,QAAQ,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAC/B,CAAC;QACJ,CAAC;QACD,IAAI,WAAW,YAAY,MAAM,IAAI,CAAC,WAAW,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YAC7D,MAAM,IAAI,KAAK,CACb,GAAG,IAAI,uDAAuD;gBAC5D,kCAAkC,MAAM,CAAC,WAAW,CAAC,IAAI;gBACzD,QAAQ,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAC/B,CAAC;QACJ,CAAC;QACD,OAAO,MAAM,CAAC;IAChB,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,KAAK,EAChB,IAAY,EACZ,IAAI,GAA4B,EAAE,EAClC,WAAW,GAAgB,EAAE,EACZ,EAAE,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,IAAI,EAAE,WAAW,CAAC,CAAC,CAAC;IAEjE;;;;;OAKG;IACH,SAAS,mBAAmB,CAAC,MAAc;QACzC,IAAI,cAAc,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO;QACxC,MAAM,QAAQ,GAAG,cAAc,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACzE,cAAc,CAAC,MAAM,GAAG,CAAC,CAAC;QAC1B,MAAM,IAAI,KAAK,CACb,qEAAqE,MAAM,KAAK;YAC9E,GAAG,QAAQ,MAAM;YACjB,oEAAoE;YACpE,uEAAuE;YACvE,gCAAgC;YAChC,gCAAgC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,WAAW,EAAE,CACnE,CAAC;IACJ,CAAC;IAED,OAAO;QACL,MAAM;QACN,IAAI;QACJ,GAAG;QACH,OAAO;QACP,MAAM;QACN,MAAM,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;QAC7B,SAAS,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,GAAG,EAAE,EAAE,EAAE;YACnC,uEAAuE;YACvE,uEAAuE;YACvE,sEAAsE;YACtE,kEAAkE;YAClE,MAAM,KAAK,GAAG,MAAM,CAAC,MAAM,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC;YAC/C,OAAO,IAAI,CAAC,IAAI,EAAE,EAAE,GAAG,IAAI,EAAE,aAAa,EAAE,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QAChE,CAAC;QACD,KAAK,EAAE,KAAK,IAAI,EAAE;YAChB,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;YACrB,mBAAmB,CAAC,qBAAqB,CAAC,CAAC;QAC7C,CAAC;KACF,CAAC;AACJ,CAAC"}
|
package/dist/index.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
export { startServer, tokenOf, type CallOptions, type ElicitBehaviour, type LiveHarness, type StartServerOptions, type ToolResult, } from './harness.js';
|
|
2
|
-
export { expectEveryToolExercised, toolCoverage, type CoverageReport, type SkipReasons, } from './coverage.js';
|
|
2
|
+
export { expectEveryToolDeclaresOutputSchema, expectEveryToolExercised, outputSchemaCoverage, toolCoverage, type AdvertisedTool, type CoverageReport, type OutputSchemaReport, type SkipReasons, } from './coverage.js';
|
|
3
3
|
export { assertLoopback, assertLoopbackHost } from './loopback.js';
|
|
4
4
|
export { waitForHttp, waitForTcp, type TcpWaitOptions, type WaitOptions, } from './wait.js';
|
package/dist/index.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
export { startServer, tokenOf, } from './harness.js';
|
|
2
|
-
export { expectEveryToolExercised, toolCoverage, } from './coverage.js';
|
|
2
|
+
export { expectEveryToolDeclaresOutputSchema, expectEveryToolExercised, outputSchemaCoverage, toolCoverage, } from './coverage.js';
|
|
3
3
|
export { assertLoopback, assertLoopbackHost } from './loopback.js';
|
|
4
4
|
export { waitForHttp, waitForTcp, } from './wait.js';
|
|
5
5
|
//# sourceMappingURL=index.js.map
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,WAAW,EACX,OAAO,GAMR,MAAM,cAAc,CAAC;AAEtB,OAAO,EACL,wBAAwB,EACxB,YAAY,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,WAAW,EACX,OAAO,GAMR,MAAM,cAAc,CAAC;AAEtB,OAAO,EACL,mCAAmC,EACnC,wBAAwB,EACxB,oBAAoB,EACpB,YAAY,GAKb,MAAM,eAAe,CAAC;AAEvB,OAAO,EAAE,cAAc,EAAE,kBAAkB,EAAE,MAAM,eAAe,CAAC;AAEnE,OAAO,EACL,WAAW,EACX,UAAU,GAGX,MAAM,WAAW,CAAC"}
|