stubsmith 0.0.0-stage → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +619 -2
- package/dist/cli.js +575 -0
- package/dist/cli.js.map +1 -0
- package/dist/index.cjs +5254 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +806 -0
- package/dist/index.d.ts +806 -0
- package/dist/index.js +5199 -0
- package/dist/index.js.map +1 -0
- package/package.json +65 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Bram Hoven, trading as Stubsmith
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,620 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Stubsmith Node.js SDK
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Instrument **outbound** HTTP calls made by your Node.js application and forward
|
|
4
|
+
them, request **and** response, to the Stubsmith ingest service for capture,
|
|
5
|
+
anonymization, and replay.
|
|
6
|
+
|
|
7
|
+
Covers Node's `http` and `https` modules and the global `fetch`, which between
|
|
8
|
+
them reach axios, node-fetch, got, superagent and native fetch. Sending is
|
|
9
|
+
non-blocking and fire-and-forget: captures go onto a bounded in-memory queue and
|
|
10
|
+
a single sender drains it; any failure (network, serialization, queue overflow)
|
|
11
|
+
is silently discarded and never propagates to your application.
|
|
12
|
+
|
|
13
|
+
Masking is applied **in this SDK**, in your own process, before any capture
|
|
14
|
+
leaves it: the ingest service receives masked bodies plus structural names
|
|
15
|
+
(field paths, header names, query-parameter names), never original values. A
|
|
16
|
+
field with no explicit `keep` rule is masked, so a field the rules do not
|
|
17
|
+
mention cannot leak. The server does not inspect values; the only content rules
|
|
18
|
+
it enforces are replacing image bodies with a placeholder, and storing a path
|
|
19
|
+
under its path pattern.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Install
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
npm install stubsmith
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Requires Node.js 22.12 or later. The package ships ESM and CommonJS builds with
|
|
30
|
+
bundled type declarations and has no runtime dependencies. Bun and Deno are not
|
|
31
|
+
supported.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## Quickstart
|
|
36
|
+
|
|
37
|
+
```js
|
|
38
|
+
import stubsmith from 'stubsmith';
|
|
39
|
+
|
|
40
|
+
// One call patches both hooks (http/https and global fetch)
|
|
41
|
+
const client = stubsmith.install({ apiKey: 'sk-your-project-key' });
|
|
42
|
+
|
|
43
|
+
// Now every outbound call is captured automatically:
|
|
44
|
+
await fetch('https://api.stripe.com/v1/charges');
|
|
45
|
+
|
|
46
|
+
import axios from 'axios';
|
|
47
|
+
await axios.get('https://api.example.com/users');
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Or configure via environment variables and call `install()` with no arguments:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
export STUBSMITH_URL="http://ingest:8081/v1/captures"
|
|
54
|
+
export STUBSMITH_API_KEY="sk-your-project-key"
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
```js
|
|
58
|
+
import stubsmith from 'stubsmith';
|
|
59
|
+
stubsmith.install();
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
With no API key the SDK disables itself: nothing is patched into the capture
|
|
63
|
+
path and nothing is sent, so the same code runs unchanged in environments
|
|
64
|
+
without a key.
|
|
65
|
+
|
|
66
|
+
CommonJS works the same way: `const stubsmith = require('stubsmith').default`.
|
|
67
|
+
|
|
68
|
+
**Call `install()` first.** Run it before importing any library that grabs
|
|
69
|
+
`http.request` or `fetch` at import time, such as a client that destructures
|
|
70
|
+
`const { request } = require('node:https')` at the top of its module. Such a
|
|
71
|
+
library keeps the original function and its calls are not captured.
|
|
72
|
+
|
|
73
|
+
**Servers must shut down explicitly.** See [Shutdown and
|
|
74
|
+
flushing](#shutdown-and-flushing).
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## What gets captured
|
|
79
|
+
|
|
80
|
+
`install()` patches two things:
|
|
81
|
+
|
|
82
|
+
| Hook | Covers |
|
|
83
|
+
|------|--------|
|
|
84
|
+
| `http.request`, `http.get`, `https.request`, `https.get` | axios (its default http adapter), node-fetch, got, superagent, and anything else built on Node's core modules |
|
|
85
|
+
| global `fetch` | native fetch, undici's `fetch`, axios with the fetch adapter, and frameworks that call the global |
|
|
86
|
+
|
|
87
|
+
Not captured:
|
|
88
|
+
|
|
89
|
+
- undici's `request()` and `Agent` API used directly
|
|
90
|
+
- `http2`
|
|
91
|
+
- `XMLHttpRequest` polyfills
|
|
92
|
+
- WebSockets
|
|
93
|
+
- calls made through a function a library copied from `http` or `fetch` before
|
|
94
|
+
`install()` ran
|
|
95
|
+
|
|
96
|
+
Only the outermost frame captures: one call produces one capture, and a redirect
|
|
97
|
+
chain produces one capture describing the request you made. Capture is skipped
|
|
98
|
+
entirely while a [replay](#offline-replay-in-tests) is active.
|
|
99
|
+
|
|
100
|
+
Request and response bodies are read without consuming them: the caller still
|
|
101
|
+
gets every byte. A body larger than 16 MiB is not captured, and the SDK warns
|
|
102
|
+
once per endpoint.
|
|
103
|
+
|
|
104
|
+
The SDK's own traffic to the ingest and rules hosts is never captured.
|
|
105
|
+
|
|
106
|
+
### Network egress
|
|
107
|
+
|
|
108
|
+
`install()` talks to **two** hosts, which matters if you allow-list egress:
|
|
109
|
+
|
|
110
|
+
| Host | Direction | Purpose |
|
|
111
|
+
|------|-----------|---------|
|
|
112
|
+
| `ingest.stubsmith.dev` | outbound POST | masked captures (`POST /v1/captures`) |
|
|
113
|
+
| `app.stubsmith.dev` | outbound GET | masking rules (`GET /v1/sdk/sync`), polled every `rulesPollInterval` (default 60 seconds) |
|
|
114
|
+
|
|
115
|
+
Both are overridable (`url` and `backendUrl`) for a project pointed at a
|
|
116
|
+
different environment. The rules timer is `unref()`ed, so the SDK never keeps
|
|
117
|
+
your process alive.
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Privacy model
|
|
122
|
+
|
|
123
|
+
Values are masked in your process, before a capture is uploaded: the server never
|
|
124
|
+
receives the originals. A field with no `keep` rule is replaced, by a constant
|
|
125
|
+
of the original's type (`"<masked>"`, `0`, `false`). URLs are templated (a path
|
|
126
|
+
segment such as `1042` becomes `{id}`) and query *values* are masked while their
|
|
127
|
+
names survive. What the ingest service sees is structure: field paths, header
|
|
128
|
+
names and query-parameter names.
|
|
129
|
+
|
|
130
|
+
Six header values are always sent as they are, in requests and responses:
|
|
131
|
+
`content-type`, `content-length`, `accept`, `accept-encoding`, `host` and
|
|
132
|
+
`transfer-encoding`. Every other header value, `user-agent`, `cache-control`
|
|
133
|
+
and `accept-language` included, is `<masked>` unless a keep rule
|
|
134
|
+
(`header.<name>` for the request, `resp_header.<name>` for the response) keeps
|
|
135
|
+
it. Names are structure and are sent as they are: header names, JSON keys,
|
|
136
|
+
form field names and query-parameter names. In a form body that is all
|
|
137
|
+
`key=value` pairs, whatever stands before each `=` is a name, so free text such
|
|
138
|
+
as `Hi Bob, ref=123` sends `Hi Bob, ref` as a name.
|
|
139
|
+
|
|
140
|
+
A fingerprint nobody has approved yet is **novel**: every value is masked and the
|
|
141
|
+
capture is flagged `novel`. You review it in the Request Types editor and
|
|
142
|
+
approve the fields that are safe to keep; only then do those scalars appear in
|
|
143
|
+
captures verbatim, and that is the only way a real value deliberately reaches
|
|
144
|
+
Stubsmith.
|
|
145
|
+
|
|
146
|
+
The approved rules are fetched from the rules API and re-polled every
|
|
147
|
+
`rulesPollInterval`. **An unreachable rules API degrades open, not closed.** A
|
|
148
|
+
failed sync keeps the last known-good rules and never stops capture. Because
|
|
149
|
+
masking is fail-closed, a client that has never reached the API masks everything
|
|
150
|
+
and still delivers captures: you get `novel=true` and no cleartext, not silence.
|
|
151
|
+
A rejected key (`401`) behaves the same way.
|
|
152
|
+
|
|
153
|
+
A payload without the SDK's masked marker is rejected by the ingest service with
|
|
154
|
+
`400 sdk_required`, so an unmasked body cannot be posted through that endpoint at
|
|
155
|
+
all.
|
|
156
|
+
|
|
157
|
+
Set `STUBSMITH_MASK_SALT` (or the `maskSalt` option) to a non-empty string to get
|
|
158
|
+
format-preserving placeholders for fields that carry a semantic type hint from an
|
|
159
|
+
approved fingerprint: a parseable timestamp, an RFC 4122 UUID, an IBAN with a
|
|
160
|
+
correct checksum. Before a fingerprint is approved no field has a hint, so
|
|
161
|
+
every masked value is the constant form regardless of the salt.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## Shutdown and flushing
|
|
166
|
+
|
|
167
|
+
Captures are masked on the calling path (CPU only, no I/O) and handed to a
|
|
168
|
+
bounded queue; a single sender does the HTTP POST and swallows every error. If
|
|
169
|
+
the ingest service is slow, unreachable or returning errors, your request path is
|
|
170
|
+
unaffected: the queue fills (1000 items or 32 MiB, whichever comes first),
|
|
171
|
+
excess captures are dropped, and nothing propagates to your code.
|
|
172
|
+
|
|
173
|
+
The one place an outage is visible is process exit. For a script, the SDK
|
|
174
|
+
registers a `beforeExit` hook that waits for the queue to drain, capped at
|
|
175
|
+
`STUBSMITH_FLUSH_TIMEOUT` seconds (default `1.0`) and abandoned as soon as a send
|
|
176
|
+
fails, so an unreachable endpoint costs about a second rather than the full
|
|
177
|
+
budget. Set `STUBSMITH_FLUSH_TIMEOUT=0` to disable the wait; captures still in
|
|
178
|
+
the queue are discarded.
|
|
179
|
+
|
|
180
|
+
**`beforeExit` only fires when the event loop drains.** A long-lived server never
|
|
181
|
+
drains it, so the hook never runs. Call `await client.close()` from your shutdown
|
|
182
|
+
path:
|
|
183
|
+
|
|
184
|
+
```js
|
|
185
|
+
const client = stubsmith.install();
|
|
186
|
+
const server = app.listen(3000);
|
|
187
|
+
|
|
188
|
+
process.on('SIGTERM', () => {
|
|
189
|
+
server.close(async () => {
|
|
190
|
+
await client.close(); // flushes the queue, stops the rules timer, un-patches
|
|
191
|
+
process.exit(0);
|
|
192
|
+
});
|
|
193
|
+
});
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
In a serverless function, or anywhere the process can be frozen after you
|
|
197
|
+
respond, flush before returning:
|
|
198
|
+
|
|
199
|
+
```js
|
|
200
|
+
export async function handler(event) {
|
|
201
|
+
const result = await doWork(event);
|
|
202
|
+
await client.flush(); // waits up to 5 s by default
|
|
203
|
+
// await client.flush({ timeoutMs: 500 }) // or pick your own budget
|
|
204
|
+
return result;
|
|
205
|
+
}
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
`flush()` returns early if a send fails, so an unreachable endpoint does not
|
|
209
|
+
stall the response. `close()` flushes, aborts a send still in flight, and
|
|
210
|
+
un-patches the hooks; it is idempotent. `STUBSMITH_FLUSH_TIMEOUT` is the budget
|
|
211
|
+
for the exit hook only and does not apply to `flush()` or `close()`.
|
|
212
|
+
|
|
213
|
+
The rules are fetched asynchronously, so captures made in the first moments of a
|
|
214
|
+
short script are masked with no rules and are novel. To wait for the first sync:
|
|
215
|
+
|
|
216
|
+
```js
|
|
217
|
+
const synced = await client.waitForRules(3000); // true once rules loaded, false on timeout
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
`install()` is idempotent: calling it again returns the client already installed,
|
|
221
|
+
ignoring the new options; call `close()` first to reconfigure. The SDK is per
|
|
222
|
+
isolate, so each `cluster` worker or `worker_thread` calls `install()` itself.
|
|
223
|
+
|
|
224
|
+
Errors are swallowed, refusals are not. Ingest can accept the HTTP request and
|
|
225
|
+
still decline the capture: at the fingerprint cap it answers `200` with
|
|
226
|
+
`{"ok": false, "error": "fingerprint_limit_reached"}`, and a quarantined or
|
|
227
|
+
rejected shape answers `422` with the same `ok: false` body. The SDK reads that
|
|
228
|
+
body and prints one warning per distinct error code per process, naming the code,
|
|
229
|
+
the HTTP status, the consequence and the remedy. Once per code rather than per
|
|
230
|
+
capture, and never thrown.
|
|
231
|
+
|
|
232
|
+
When a payload exceeds `limits.maxPayloadBytes` (10 MiB), both bodies are dropped
|
|
233
|
+
together and the capture is sent with `bodies_omitted`: the endpoint stays
|
|
234
|
+
visible and nothing half-captured is stored. Bodies are never truncated.
|
|
235
|
+
|
|
236
|
+
---
|
|
237
|
+
|
|
238
|
+
## Offline replay in tests
|
|
239
|
+
|
|
240
|
+
`replay()` serves recorded responses to your code's outbound `http`, `https` and
|
|
241
|
+
`fetch` calls. While it is active no network call is made and the dependency
|
|
242
|
+
does not need to be running.
|
|
243
|
+
|
|
244
|
+
Fetch the bundle once and commit it:
|
|
245
|
+
|
|
246
|
+
```bash
|
|
247
|
+
export STUBSMITH_API_KEY=<your project key>
|
|
248
|
+
npx stubsmith pull --out .stubsmith/bundle.json
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
From then on the test suite needs no key, no network and no `pull` step: it reads
|
|
252
|
+
the committed file. Refresh it when the recording should change: the upstream
|
|
253
|
+
API's shape changed, you added a call the bundle does not cover, or you approved
|
|
254
|
+
new fingerprints. Treat it like a lockfile or a golden file: an occasional,
|
|
255
|
+
reviewed, committed change.
|
|
256
|
+
|
|
257
|
+
`replay()` finds the bundle without configuration: an explicit `bundle` option
|
|
258
|
+
(a path or a parsed `Bundle` object), then `$STUBSMITH_BUNDLE`, then an upward
|
|
259
|
+
search from the working directory for `.stubsmith/bundle.json` that stops at the
|
|
260
|
+
first directory containing `.git` or `package.json`. In a monorepo, where the
|
|
261
|
+
nearest `package.json` belongs to a package below the one holding the bundle, set
|
|
262
|
+
`STUBSMITH_BUNDLE` or pass a path, because the upward walk stops at that
|
|
263
|
+
`package.json`.
|
|
264
|
+
|
|
265
|
+
`replay()` returns a context and installs its hooks immediately. Call
|
|
266
|
+
`restore()` when the test ends. Restore contexts in the reverse order you created
|
|
267
|
+
them.
|
|
268
|
+
|
|
269
|
+
Replay patches `http.request`, `http.get`, `https.request`, `https.get` and the
|
|
270
|
+
global `fetch`. `undici.request()`, `http2`, and a `fetch` or `http.request`
|
|
271
|
+
reference captured before `replay()` ran bypass replay and still hit the network.
|
|
272
|
+
A recorded 3xx under global `fetch` is returned as-is: the redirect is not followed.
|
|
273
|
+
|
|
274
|
+
### Vitest
|
|
275
|
+
|
|
276
|
+
```js
|
|
277
|
+
import { afterEach, beforeEach, expect, test } from 'vitest';
|
|
278
|
+
import { replay, byStatus, StubNotFound } from 'stubsmith';
|
|
279
|
+
|
|
280
|
+
let ctx;
|
|
281
|
+
beforeEach(() => {
|
|
282
|
+
ctx = replay();
|
|
283
|
+
});
|
|
284
|
+
afterEach(() => {
|
|
285
|
+
ctx.restore();
|
|
286
|
+
});
|
|
287
|
+
|
|
288
|
+
test('reads a recorded user', async () => {
|
|
289
|
+
const res = await fetch('http://localhost:8081/api/users/4821');
|
|
290
|
+
expect(res.status).toBe(200);
|
|
291
|
+
expect(ctx.served().map((s) => s.status)).toEqual([200]);
|
|
292
|
+
});
|
|
293
|
+
|
|
294
|
+
test('an unrecorded call never reaches the network', async () => {
|
|
295
|
+
await expect(fetch('http://localhost:8081/api/nope')).rejects.toBeInstanceOf(StubNotFound);
|
|
296
|
+
});
|
|
297
|
+
|
|
298
|
+
test('handles the recorded 404', async () => {
|
|
299
|
+
ctx.restore();
|
|
300
|
+
ctx = replay({ select: byStatus(404) });
|
|
301
|
+
const res = await fetch('http://localhost:8081/api/users/1');
|
|
302
|
+
expect(res.status).toBe(404);
|
|
303
|
+
});
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
To install it for every test file, put the `beforeEach`/`afterEach` pair in a
|
|
307
|
+
module listed under `setupFiles` in the Vitest config.
|
|
308
|
+
|
|
309
|
+
### Jest
|
|
310
|
+
|
|
311
|
+
Jest gives each test file its own module registry, so `replay()` has to be called
|
|
312
|
+
from a file Jest runs inside that registry: either the test file itself, or a
|
|
313
|
+
module listed under `setupFilesAfterEnv`, which Jest runs once per test file.
|
|
314
|
+
|
|
315
|
+
```js
|
|
316
|
+
// jest.setup.js, listed in jest.config.js as setupFilesAfterEnv: ['./jest.setup.js']
|
|
317
|
+
const { replay } = require('stubsmith');
|
|
318
|
+
|
|
319
|
+
let ctx;
|
|
320
|
+
beforeEach(() => {
|
|
321
|
+
ctx = replay();
|
|
322
|
+
});
|
|
323
|
+
afterEach(() => {
|
|
324
|
+
ctx.restore();
|
|
325
|
+
});
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
```js
|
|
329
|
+
// orders.test.js
|
|
330
|
+
const { replay, byStatus, StubNotFound } = require('stubsmith');
|
|
331
|
+
|
|
332
|
+
test('handles the recorded 404', async () => {
|
|
333
|
+
const pinned = replay({ select: byStatus(404) });
|
|
334
|
+
try {
|
|
335
|
+
const res = await fetch('http://localhost:8081/api/users/1');
|
|
336
|
+
expect(res.status).toBe(404);
|
|
337
|
+
} finally {
|
|
338
|
+
pinned.restore();
|
|
339
|
+
}
|
|
340
|
+
});
|
|
341
|
+
|
|
342
|
+
test('an unrecorded call never reaches the network', async () => {
|
|
343
|
+
await expect(fetch('http://localhost:8081/api/nope')).rejects.toBeInstanceOf(StubNotFound);
|
|
344
|
+
});
|
|
345
|
+
```
|
|
346
|
+
|
|
347
|
+
### Every recorded response, not just the newest
|
|
348
|
+
|
|
349
|
+
`replay()` serves **one** response per request shape: the newest recording of the
|
|
350
|
+
status that occurred most often. That leaves everything else in the recording
|
|
351
|
+
unexercised, which is usually where the bugs are. Stubsmith keeps a rolling
|
|
352
|
+
window of recent samples for each distinct response a shape returned, so the 429
|
|
353
|
+
and the 500 your API really produced are sitting there unused.
|
|
354
|
+
|
|
355
|
+
`replayAll()` is an async generator that runs the block once per recorded
|
|
356
|
+
response. Pass `statuses` to loop one kind:
|
|
357
|
+
|
|
358
|
+
```js
|
|
359
|
+
import { replayAll } from 'stubsmith';
|
|
360
|
+
|
|
361
|
+
test('parses every recorded success', async () => {
|
|
362
|
+
for await (const pass of replayAll({ statuses: [200] })) {
|
|
363
|
+
const result = await connector.syncOrders();
|
|
364
|
+
expect(result.orders).toBeDefined();
|
|
365
|
+
}
|
|
366
|
+
});
|
|
367
|
+
|
|
368
|
+
test('degrades on every recorded failure', async () => {
|
|
369
|
+
for await (const pass of replayAll({ statuses: [429, 500] })) {
|
|
370
|
+
await expect(connector.syncOrders()).rejects.toThrow();
|
|
371
|
+
}
|
|
372
|
+
});
|
|
373
|
+
```
|
|
374
|
+
|
|
375
|
+
Each iteration yields an active context, restored when the loop body finishes,
|
|
376
|
+
throws or `break`s. `pass.passNumber` is the 1-based pass.
|
|
377
|
+
|
|
378
|
+
**Filter by status unless you have a reason not to.** It decides what a red
|
|
379
|
+
build means. Unfiltered, the loop feeds successes and failures into one test
|
|
380
|
+
body, so the body has to hold for both; the day a 500 first appears in the
|
|
381
|
+
window, assertions written when only 200s existed start failing, and the build
|
|
382
|
+
breaks because the recording changed rather than because the code did. Filtered,
|
|
383
|
+
every pass is the same kind of response, the assertions can be specific, and a
|
|
384
|
+
failure means your code cannot handle a response your API genuinely returns. The
|
|
385
|
+
filter also bounds the work: `maxPasses` (default 64) is a backstop, not a plan.
|
|
386
|
+
|
|
387
|
+
A shape with recordings but none of the requested status throws `StubNotFound`,
|
|
388
|
+
naming the filter and listing the statuses it does have. Serving a different one
|
|
389
|
+
would make the loop assert against a response it was told to exclude.
|
|
390
|
+
|
|
391
|
+
You never name an endpoint. Which ones a pass touches is discovered by running
|
|
392
|
+
your code, so this works for a connector whose call sequence changes between
|
|
393
|
+
passes because it branches on the response it got. Pass one serves exactly what
|
|
394
|
+
`replay()` serves (without a `statuses` filter). The loop stops when every shape
|
|
395
|
+
that was actually touched has served its last recording; a shape with a shorter
|
|
396
|
+
window keeps serving its final recording rather than throwing.
|
|
397
|
+
|
|
398
|
+
To pin a single response instead, pass `select`:
|
|
399
|
+
|
|
400
|
+
```js
|
|
401
|
+
const ctx = replay({ select: byStatus(429) });
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
`byStatus` throws `StubNotFound` when no recording of that status exists, rather
|
|
405
|
+
than quietly serving a different one: a rate-limit test that silently ran against
|
|
406
|
+
a 200 would pass while testing nothing. `select` receives every recording of the
|
|
407
|
+
shape as an array and returns one, so any other rule is a function away. Each
|
|
408
|
+
recording has these fields: `status`, `count`, `durationMs`, `headers`, `body`,
|
|
409
|
+
`bodyEncoding`, `bodyCapped`, `captureId` and `capturedAt`. They are camelCase,
|
|
410
|
+
unlike the snake_case keys in the bundle JSON (`duration_ms`, `body_encoding`).
|
|
411
|
+
|
|
412
|
+
Either way, `served()` reports what actually ran, so a test can assert coverage
|
|
413
|
+
instead of assuming it:
|
|
414
|
+
|
|
415
|
+
```js
|
|
416
|
+
const ctx = replay();
|
|
417
|
+
try {
|
|
418
|
+
await connector.syncOrders();
|
|
419
|
+
console.log(ctx.served().map((s) => [s.endpoint, s.status]));
|
|
420
|
+
} finally {
|
|
421
|
+
ctx.restore();
|
|
422
|
+
}
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
Each entry carries `endpoint`, `status`, `captureId`, `capturedAt`, and
|
|
426
|
+
`index`/`total`/`exhausted` describing its place in the shape's window.
|
|
427
|
+
|
|
428
|
+
#### Getting the window into the bundle
|
|
429
|
+
|
|
430
|
+
A default bundle holds one recording per response, so `replayAll()` gives one pass
|
|
431
|
+
per status. Ask for the window explicitly:
|
|
432
|
+
|
|
433
|
+
```bash
|
|
434
|
+
npx stubsmith pull --endpoint "GET /admin/orders" --samples all \
|
|
435
|
+
--out tests/bundles/admin-orders.json
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
`--samples` above 1 requires `--endpoint`: the full window for a whole project is
|
|
439
|
+
not served. Repeat `--endpoint` to cover several in one file, which is one
|
|
440
|
+
request per endpoint, merged for you.
|
|
441
|
+
|
|
442
|
+
Or fetch at test time, with no file involved:
|
|
443
|
+
|
|
444
|
+
```js
|
|
445
|
+
import { fetchBundle, replayAll } from 'stubsmith';
|
|
446
|
+
|
|
447
|
+
const bundle = await fetchBundle('GET /admin/orders', { samples: 'all' });
|
|
448
|
+
|
|
449
|
+
for await (const pass of replayAll({ bundle })) {
|
|
450
|
+
await connector.listOrders();
|
|
451
|
+
}
|
|
452
|
+
```
|
|
453
|
+
|
|
454
|
+
Fetch once, not per test: it is a network call. It needs a live key wherever the
|
|
455
|
+
tests run, and live recordings roll as the window rolls, so a failure you see
|
|
456
|
+
today may not reproduce next week. Fetch live while iterating locally; commit a
|
|
457
|
+
pulled bundle for CI.
|
|
458
|
+
|
|
459
|
+
### Matching
|
|
460
|
+
|
|
461
|
+
A request is matched on `(domain, method, path_template, fingerprint)`. The
|
|
462
|
+
fingerprint covers body key-paths, query parameter names and the normalised
|
|
463
|
+
content-type, not values, and not the host or path, which is why the other three
|
|
464
|
+
parts of the key are needed. Every body-less `GET` shares one fingerprint.
|
|
465
|
+
|
|
466
|
+
Dynamic path segments are templated from the recording, so `/api/users/4821`
|
|
467
|
+
matches a stub recorded as `/api/users/{id}`.
|
|
468
|
+
|
|
469
|
+
### When nothing matches
|
|
470
|
+
|
|
471
|
+
`StubNotFound` is thrown with a diff of what was sent against the closest
|
|
472
|
+
recording. It never falls through to the network, so a test cannot silently start
|
|
473
|
+
calling a real service. It carries `method`, `pathTemplate` and `fingerprint`.
|
|
474
|
+
|
|
475
|
+
A stub whose fingerprint has no recorded captures is reported as `degraded` by
|
|
476
|
+
`stubsmith pull` and throws the same error at replay time, rather than serving an
|
|
477
|
+
empty response.
|
|
478
|
+
|
|
479
|
+
See [`examples/express/`](examples/express/) for a runnable service with a replay
|
|
480
|
+
test that passes offline.
|
|
481
|
+
|
|
482
|
+
---
|
|
483
|
+
|
|
484
|
+
## Fetching fixtures
|
|
485
|
+
|
|
486
|
+
`fixtures()` fetches the recorded response variants for one request type, newest
|
|
487
|
+
first, for tests that want a raw recorded exchange rather than a replay:
|
|
488
|
+
|
|
489
|
+
```js
|
|
490
|
+
import { fixtures, fixturesBundle } from 'stubsmith';
|
|
491
|
+
|
|
492
|
+
const fxs = await fixtures('POST /v1/charges/{id}', { distinct: 'status' });
|
|
493
|
+
console.log(fxs[0].status, fxs[0].response.json());
|
|
494
|
+
|
|
495
|
+
const bundle = await fixturesBundle('POST /v1/charges/{id}');
|
|
496
|
+
const declined = bundle.byStatus(402); // throws, naming the statuses present, if there is none
|
|
497
|
+
console.log(declined.response.json());
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
Both take `"METHOD /path"`. Options are `distinct: 'status'` (at most one fixture
|
|
501
|
+
per HTTP status), `limit` (1 to 100), and `apiUrl` and `apiKey`, which default to
|
|
502
|
+
`$STUBSMITH_API_URL` and `$STUBSMITH_API_KEY`. `fixtures()` returns the
|
|
503
|
+
`Fixture[]`; `fixturesBundle()` returns the full envelope, including `count` and
|
|
504
|
+
`requestType`. Each `Fixture` has `id`, `capturedAt`, `method`, `path`, `status`,
|
|
505
|
+
`durationMs`, and `request` and `response` sides with `headers`, `body` and
|
|
506
|
+
`json()`. These requests are never captured by the SDK.
|
|
507
|
+
|
|
508
|
+
---
|
|
509
|
+
|
|
510
|
+
## CLI
|
|
511
|
+
|
|
512
|
+
```bash
|
|
513
|
+
npx stubsmith pull [--out PATH] [--endpoint "METHOD /path"]... [--samples N|all]
|
|
514
|
+
npx stubsmith --version
|
|
515
|
+
```
|
|
516
|
+
|
|
517
|
+
| Flag | Default | Description |
|
|
518
|
+
|------|---------|-------------|
|
|
519
|
+
| `--out PATH` | `.stubsmith/bundle.json` | Destination file; parent directories are created |
|
|
520
|
+
| `--endpoint "METHOD /path"` | whole project | Filter to an endpoint such as `"GET /api/users/{id}"`. Repeat to pull several into one bundle: each is one request, merged for you |
|
|
521
|
+
| `--samples N\|all` | `1` | Recordings to keep per response (the newest). Above 1 requires `--endpoint` |
|
|
522
|
+
|
|
523
|
+
`pull` requires `STUBSMITH_API_KEY` and exits `1` on any error, `0` on success.
|
|
524
|
+
It prints a summary and warns when the server truncated the bundle or marked
|
|
525
|
+
stubs `degraded`. A project-wide bundle is capped at 2000 request shapes, and
|
|
526
|
+
what it drops is reported in `truncated` rather than carried; per-endpoint
|
|
527
|
+
scoping is what you want anyway.
|
|
528
|
+
|
|
529
|
+
The same operations are available from code as `fetchBundle()` (one request) and
|
|
530
|
+
`pullBundle()` (several, merged, with a summary).
|
|
531
|
+
|
|
532
|
+
---
|
|
533
|
+
|
|
534
|
+
## Configuration
|
|
535
|
+
|
|
536
|
+
### Environment variables
|
|
537
|
+
|
|
538
|
+
| Variable | Default | Description |
|
|
539
|
+
|----------|---------|-------------|
|
|
540
|
+
| `STUBSMITH_API_KEY` | none | Project key, sent as a bearer token. An empty value disables the SDK |
|
|
541
|
+
| `STUBSMITH_URL` | `https://ingest.stubsmith.dev/v1/captures` | Full URL of the ingest endpoint |
|
|
542
|
+
| `STUBSMITH_BACKEND_URL` | `https://app.stubsmith.dev/api` | API the masking rules are fetched from. A **second host**, not the ingest host |
|
|
543
|
+
| `STUBSMITH_API_URL` | falls back to `STUBSMITH_BACKEND_URL` for `pull`, then the default above | API for `stubsmith pull` and `fetchBundle()`; for `fixtures()` it falls back to the default above |
|
|
544
|
+
| `STUBSMITH_DEBUG` | off | `1`, `true` or `yes` logs send and sync failures to stderr. Does not change the fire-and-forget contract |
|
|
545
|
+
| `STUBSMITH_FLUSH_TIMEOUT` | `1.0` (seconds) | How long process exit waits for queued captures to drain. `0` disables the wait. A malformed value falls back to the default |
|
|
546
|
+
| `STUBSMITH_MASK_SALT` | none | Non-empty switches on format-preserving placeholders for fields with a type hint |
|
|
547
|
+
| `STUBSMITH_BUNDLE` | none | Path to the replay bundle, used when `replay()` is not given one |
|
|
548
|
+
|
|
549
|
+
### `install()` options
|
|
550
|
+
|
|
551
|
+
| Option | Default | Description |
|
|
552
|
+
|--------|---------|-------------|
|
|
553
|
+
| `apiKey` | `$STUBSMITH_API_KEY` | Bearer token; empty auto-disables the client |
|
|
554
|
+
| `url` | `$STUBSMITH_URL` / `https://ingest.stubsmith.dev/v1/captures` | Full URL of the ingest endpoint |
|
|
555
|
+
| `backendUrl` | `$STUBSMITH_BACKEND_URL` / `https://app.stubsmith.dev/api` | API the masking rules are fetched from |
|
|
556
|
+
| `timeoutMs` | `5000` | HTTP timeout for an ingest POST |
|
|
557
|
+
| `sampleRate` | `1` | Fraction of captures to forward (0 to 1) |
|
|
558
|
+
| `rulesPollInterval` | `60000` (ms) | How often the rules cache re-polls the API |
|
|
559
|
+
| `compress` | `true` | gzip capture payloads of 1 KiB or more before sending |
|
|
560
|
+
| `debug` | `$STUBSMITH_DEBUG` | Log send and sync failures to stderr |
|
|
561
|
+
| `maskSalt` | `$STUBSMITH_MASK_SALT` | Placeholder salt; given here it is used directly and the environment is not modified |
|
|
562
|
+
| `limits.maxQueueItems` | `1000` | Bound on the send queue; excess captures are dropped |
|
|
563
|
+
| `limits.maxQueueBytes` | `33554432` (32 MiB) | Ceiling on bytes held in the queue, separate from the item bound |
|
|
564
|
+
| `limits.maxPayloadBytes` | `10485760` (10 MiB) | Largest capture payload sent. Over it, bodies are omitted rather than cut. Must match the server's limit |
|
|
565
|
+
|
|
566
|
+
**An explicit option always wins over the environment variable**, which is read
|
|
567
|
+
only when the option is omitted, and only once, when the client is built.
|
|
568
|
+
`install({ apiKey: 'sk-from-arg' })` authenticates with `sk-from-arg` even if
|
|
569
|
+
`$STUBSMITH_API_KEY` is set to something else. That matters when the key comes
|
|
570
|
+
from a database row or a secrets manager rather than the process environment.
|
|
571
|
+
|
|
572
|
+
The client returned by `install()` has `flush()`, `close()`, `waitForRules()` and
|
|
573
|
+
an `enabled` flag that is `false` when no key was given. `stubsmith.isInstalled()`
|
|
574
|
+
is `true` while a client is installed and not closed.
|
|
575
|
+
|
|
576
|
+
---
|
|
577
|
+
|
|
578
|
+
## Limitations
|
|
579
|
+
|
|
580
|
+
- **Hook coverage.** Only `http`, `https` and the global `fetch` are patched.
|
|
581
|
+
undici's `request()` and `Agent` API, `http2`, `XMLHttpRequest` polyfills and
|
|
582
|
+
WebSockets are not captured, and neither is a library that copied
|
|
583
|
+
`http.request` before `install()` ran.
|
|
584
|
+
- **Runtimes.** Node.js 22.12 and later. Bun and Deno are not supported.
|
|
585
|
+
- **Long-lived servers** never fire `beforeExit`, so they must `await
|
|
586
|
+
client.close()` on shutdown or the last captures can be lost.
|
|
587
|
+
- **Unread responses.** A response whose body is first read more than one second
|
|
588
|
+
after the headers arrived is captured with an empty body. The reader still gets
|
|
589
|
+
every byte; the capture is not made a second time.
|
|
590
|
+
- **Body size.** A body over 16 MiB is not captured. A capture whose payload
|
|
591
|
+
exceeds 10 MiB is sent with both bodies omitted.
|
|
592
|
+
- **Per isolate.** Each `cluster` worker or `worker_thread` calls `install()` for
|
|
593
|
+
itself.
|
|
594
|
+
- **Jest.** Each test file has its own module registry, so call `replay()` from
|
|
595
|
+
the test file or a `setupFilesAfterEnv` module (see above).
|
|
596
|
+
|
|
597
|
+
---
|
|
598
|
+
|
|
599
|
+
## Development
|
|
600
|
+
|
|
601
|
+
```bash
|
|
602
|
+
npm install
|
|
603
|
+
npm test # vitest, including the conformance vectors
|
|
604
|
+
npm run typecheck
|
|
605
|
+
npm run lint
|
|
606
|
+
npm run build
|
|
607
|
+
```
|
|
608
|
+
|
|
609
|
+
The masking and fingerprinting code is checked against golden vectors in
|
|
610
|
+
`conformance/vectors/`, produced by the reference Python SDK; see
|
|
611
|
+
`conformance/README.md`. Never edit the vectors by hand. To regenerate them,
|
|
612
|
+
point `PYTHONPATH` at a checkout of the Python SDK:
|
|
613
|
+
|
|
614
|
+
```bash
|
|
615
|
+
PYTHONPATH=/path/to/stubsmith-python python3 tools/gen-vectors.py
|
|
616
|
+
```
|
|
617
|
+
|
|
618
|
+
## Licence
|
|
619
|
+
|
|
620
|
+
MIT. See [LICENSE](LICENSE).
|