@ssheleg/sheleg-dev 0.7.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,176 @@
1
+ # Sentry setup — install, auth, projects, DSN, MCP
2
+
3
+ Read when adding Sentry to a project for the first time, or when a CLI call is
4
+ refused and you need to tell a policy problem from an auth problem.
5
+
6
+ ## Contents
7
+
8
+ - [Install](#install)
9
+ - [Authenticate](#authenticate)
10
+ - [Token taxonomy — which kind can do what](#token-taxonomy--which-kind-can-do-what)
11
+ - [Find what already exists](#find-what-already-exists)
12
+ - [Create a project](#create-a-project)
13
+ - [Get the DSN and wire it in](#get-the-dsn-and-wire-it-in)
14
+ - [Declare the MCP server](#declare-the-mcp-server)
15
+ - [SDK options for a backend service](#sdk-options-for-a-backend-service)
16
+
17
+ All commands below were executed on 2026-08-24 against `sentry` 0.43.0 and
18
+ `sentry-sdk` 2.19.2. Where a version matters, it is named.
19
+
20
+ ## Install
21
+
22
+ ```bash
23
+ brew install getsentry/tools/sentry # macOS, preferred: no piping a remote script to a shell
24
+ curl https://cli.sentry.dev/install -fsS | bash # the documented alternative
25
+ ```
26
+
27
+ Then check for a shadow — two prefixes, and the older copy usually wins:
28
+
29
+ ```bash
30
+ which -a sentry | while read p; do printf '%-34s %s\n' "$p" "$("$p" --version 2>&1 | head -1)"; done
31
+ ```
32
+
33
+ Remove the stale copy rather than reordering `PATH`. Credentials live in
34
+ `~/.sentry/`, not beside the binary, so this does not log you out.
35
+
36
+ ## Authenticate
37
+
38
+ ```bash
39
+ sentry auth # OAuth device flow: prints a URL and a code
40
+ sentry auth --token <TOKEN> # non-interactive, for CI
41
+ sentry auth status # who, which token, when it expires
42
+ sentry auth logout
43
+ ```
44
+
45
+ `sentry auth` writes to `~/.sentry/cli.db`. Override with `SENTRY_CONFIG_DIR`.
46
+ `SENTRY_URL` points at self-hosted instances.
47
+
48
+ **The device-flow token expires.** `auth status` prints the expiry — one
49
+ observed token had three days left. A CI job depending on it fails on a
50
+ Wednesday for no visible reason; CI wants a token, not the device flow.
51
+
52
+ ## Token taxonomy — which kind can do what
53
+
54
+ | Kind | Created in | Scopes | Use it for |
55
+ |---|---|---|---|
56
+ | **OAuth device flow** (`sentry auth`) | the CLI | fixed by the flow; measured: `event:read`, `event:write`, `member:read`, `org:read`, `project:admin`, `project:read`, `project:write`, `team:read`, `team:write` — **no `org:write`/`org:admin`** | interactive work at a desk |
57
+ | **Organization Token** | Settings → Developer Settings → Organization Tokens | limited, not customizable | CI. Sentry's own recommendation for it |
58
+ | **Internal Integration** | the integration platform | customizable, **full API access** | org-level automation, including creating projects |
59
+ | **Personal Token** | Account → Personal Tokens | chosen at creation, never editable after | a one-off at a desk. Explicitly discouraged for CI |
60
+
61
+ Print the scopes a token actually carries, rather than inferring them from how
62
+ it was made:
63
+
64
+ ```bash
65
+ sentry api "/" --json # {"auth": {"scopes": [...]}, "user": {...}}
66
+ ```
67
+
68
+ ## Create a project
69
+
70
+ ```bash
71
+ sentry project create <org>/<name>:<platform> --team <team> --dry-run
72
+ sentry project create <org>/<name>:<platform> --team <team> --json
73
+ ```
74
+
75
+ `--dry-run` prints the resolved org, team, slug and platform without creating
76
+ anything. Use it: the org must be in the argument or the command refuses.
77
+
78
+ ### When it returns 403
79
+
80
+ ```
81
+ Your organization has disabled this feature for members.
82
+ This is an org-level policy setting, not an auth issue.
83
+ ```
84
+
85
+ The message is accurate and easy to disbelieve, because it appears **even for an
86
+ org owner**. The policy is evaluated against the token's scopes, not the human's
87
+ role, so an owner acting through a device-flow token is treated as a member.
88
+
89
+ ```bash
90
+ sentry org view <org> --json | grep -E 'allowMemberProjectCreation|allowMemberInvite'
91
+ ```
92
+
93
+ `allowMemberProjectCreation: false` is the cause. `POST /teams/{org}/{team}/projects/`
94
+ fails identically — the error names it as an alternative, but it is the same
95
+ policy underneath.
96
+
97
+ Resolutions, cheapest first:
98
+
99
+ 1. Create the project once in the web UI. One browser action, then everything
100
+ else is scriptable forever.
101
+ 2. Flip the org setting. Also a browser action, and it loosens a policy that
102
+ exists for a reason — decide deliberately.
103
+ 3. Issue an Internal Integration token and `sentry auth --token` with it. The
104
+ only route that makes project creation itself scriptable.
105
+
106
+ ## Get the DSN and wire it in
107
+
108
+ ```bash
109
+ sentry project view <org>/<project> --json \
110
+ | python3 -c "import json,sys; print(json.load(sys.stdin)[0]['dsn'])"
111
+ ```
112
+
113
+ The response is a **list**, hence the `[0]`. The DSN is the app's ingest
114
+ address; put it wherever that host keeps configuration:
115
+
116
+ ```bash
117
+ heroku config:set SENTRY_DSN='https://…@o….ingest.sentry.io/…' -a <app>
118
+ fly secrets set SENTRY_DSN='…'
119
+ gh secret set SENTRY_DSN --body '…' # only if CI itself reports errors
120
+ ```
121
+
122
+ A DSN is write-only ingest, not an API credential — see the table in `SKILL.md`.
123
+ Treat it as configuration, not as a secret to be guarded like a token, and still
124
+ do not commit it.
125
+
126
+ ## Declare the MCP server
127
+
128
+ ```bash
129
+ curl -si https://mcp.sentry.dev/mcp | grep -i www-authenticate
130
+ ```
131
+
132
+ A `401` whose `www-authenticate` carries `resource_metadata=` means OAuth, which
133
+ means **the agent's own config, never a shared MCP gateway** — an OAuth upstream
134
+ names itself as the resource at its own address, so a client pointed at a
135
+ gateway must reject the mismatch and the flow cannot complete.
136
+
137
+ ```bash
138
+ claude mcp add --transport http sentry https://mcp.sentry.dev/mcp/<org>/<project>
139
+ ```
140
+
141
+ Scoping is optional and narrows what the server sees:
142
+ `…/mcp`, `…/mcp/<org>`, `…/mcp/<org>/<project>`. Narrow it.
143
+
144
+ Everything the MCP does, the CLI also does. A machine without it loses no
145
+ capability, so never let a procedure depend on it.
146
+
147
+ ## SDK options for a backend service
148
+
149
+ ```python
150
+ sentry_sdk.init(
151
+ dsn=SENTRY_DSN,
152
+ environment=ENVIRONMENT, # "production" | "staging" | "local"
153
+ release=RELEASE, # MUST equal the version you create in Sentry
154
+ event_scrubber=EventScrubber( # see references/scrubbing.md
155
+ denylist=DEFAULT_DENYLIST + ["dsn", "database_url", "auth_key"],
156
+ recursive=True,
157
+ ),
158
+ before_send=scrub_values,
159
+ send_default_pii=False, # excludes headers, IPs, and similar
160
+ traces_sample_rate=0.0, # tracing off unless someone reads the spans
161
+ )
162
+ ```
163
+
164
+ Notes that decide behaviour:
165
+
166
+ - `environment` defaults to `"production"`. A local run that forgets to set it
167
+ pollutes the production feed with a developer's laptop.
168
+ - `release` defaults to `None` and the SDK's auto-detection is not reliable
169
+ enough to depend on. Set it.
170
+ - `traces_sample_rate` defaults to `None`; tracing is off unless it or
171
+ `traces_sampler` is set. Turning it on costs quota — do it when someone is
172
+ going to read the result, not by default.
173
+ - `profiles_sample_rate` is the current profiling option and is not deprecated.
174
+ - `send_default_pii=False` (the default) keeps integrations from attaching
175
+ request headers and IP addresses. It does **not** scrub your own payloads;
176
+ that is the scrubber's job.
@@ -2,14 +2,15 @@
2
2
 
3
3
  **Copy this whole directory into your repository.** It is the executable half of what
4
4
  `SKILL.md` and `references/webhook-events.md` state in prose: nine real Stripe event
5
- bodies, twelve assertions a correct webhook handler must pass, and a self-test that
6
- breaks the handler one rule at a time so you can watch every assertion fail.
5
+ bodies, twelve invariants a correct webhook handler must hold — 45 assertions between
6
+ them — and a self-test that breaks the handler one rule at a time so you can watch each
7
+ assertion fail on its own.
7
8
 
8
9
  Two commands, no dependencies, no network, no key:
9
10
 
10
11
  ```bash
11
- node assert-money-invariants.mjs # 12 assertions against the reference handler
12
- node assert-money-invariants.mjs --self-test # break one rule at a time; every assertion must go red
12
+ node assert-money-invariants.mjs # 12 invariants / 45 assertions
13
+ node assert-money-invariants.mjs --self-test # break one rule at a time; EACH ASSERTION must go red
13
14
  ```
14
15
 
15
16
  Point it at **your** handler by replacing the two imports at the top of
@@ -72,14 +72,20 @@ export const INVARIANTS = [
72
72
  fixtures: [CYCLE_JAN],
73
73
  states: 'a paid renewal becomes exactly one grant, and the mirror carries its period',
74
74
  breaks: ['grant-on-renewal'],
75
- async run({ store, handler }) {
75
+ async run({ store, handler, assert }) {
76
76
  const event = fixture(CYCLE_JAN);
77
77
  const response = await handler.deliver(event);
78
- assert.equal(response.status, 200);
79
- assert.deepEqual(response.body, { received: true });
78
+ // `unmutated`: no rule in RULES makes this route answer anything but 200 with that
79
+ // body, so neither assertion has ever been watched failing. They pin the envelope a
80
+ // reader copies; they are not evidence, and the verdict line counts them apart.
81
+ assert.unmutated.equal(response.status, 200);
82
+ assert.unmutated.deepEqual(response.body, { received: true });
80
83
  assert.equal(store.grants.length, 1, 'a paid renewal granted nothing');
81
84
  assert.equal(store.credits.get('usr_PLACEHOLDER_alice'), CREDITS_PER_PERIOD);
82
- const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice');
85
+ // `|| {}` on purpose: with no mirror row at all these two assertions must FAIL
86
+ // rather than throw a TypeError on the line above them. A throw aborts the run, and
87
+ // an assertion the runner never reached is one no mutant has been watched breaking.
88
+ const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice') || {};
83
89
  assert.equal(mirror.periodStart, 1767225600);
84
90
  assert.equal(mirror.periodEnd, 1769904000, 'the period is not the line period');
85
91
  },
@@ -89,10 +95,11 @@ export const INVARIANTS = [
89
95
  fixtures: [CYCLE_JAN, PRORATION],
90
96
  states: 'billing_reason subscription_update is a $40 proration, not a month of product',
91
97
  breaks: ['billing-reason', 'grant-on-renewal'],
92
- async run({ store, handler }) {
98
+ async run({ store, handler, assert }) {
93
99
  await handler.deliver(fixture(CYCLE_JAN));
94
100
  const response = await handler.deliver(fixture(PRORATION));
95
- assert.equal(response.status, 200, 'a proration invoice must not fail the endpoint');
101
+ // unmutated: nothing in RULES makes the endpoint answer 500 for a proration.
102
+ assert.unmutated.equal(response.status, 200, 'a proration invoice must not fail the endpoint');
96
103
  assert.equal(store.grants.length, 1,
97
104
  'the proration granted a second period — a user who adds and removes a seat '
98
105
  + 'four times has been given four months of product');
@@ -107,10 +114,10 @@ export const INVARIANTS = [
107
114
  fixtures: [CYCLE_JAN, CYCLE_JAN_REDELIVERY],
108
115
  states: 'the same evt_ delivered twice grants once AND runs its side effects once',
109
116
  breaks: ['claim', 'grant-on-renewal'],
110
- async run({ store, handler }) {
117
+ async run({ store, handler, assert }) {
111
118
  const first = await handler.deliver(fixture(CYCLE_JAN));
112
119
  const second = await handler.deliver(fixture(CYCLE_JAN_REDELIVERY));
113
- assert.deepEqual(first.body, { received: true });
120
+ assert.unmutated.deepEqual(first.body, { received: true }); // the envelope, again
114
121
  assert.deepEqual(second.body, { received: true, duplicate: true },
115
122
  'the second delivery was not recognised as a duplicate');
116
123
  assert.equal(store.grants.length, 1, 'the redelivery granted a second time');
@@ -124,7 +131,7 @@ export const INVARIANTS = [
124
131
  fixtures: [CYCLE_JAN, CYCLE_JAN_REDELIVERY],
125
132
  states: 'two deliveries in flight at once grant once — the claim is an INSERT, not a SELECT',
126
133
  breaks: ['claim', 'grant-on-renewal'],
127
- async run({ store, handler }) {
134
+ async run({ store, handler, assert }) {
128
135
  // 40 ms apart in production; here, both before either has finished its first read.
129
136
  // This is the fixture that separates the event claim from the grant marker: the
130
137
  // marker reads before it writes, and a read is a round trip.
@@ -146,7 +153,7 @@ export const INVARIANTS = [
146
153
  fixtures: [CYCLE_JAN],
147
154
  states: 'the nightly repair reuses the webhook marker — a nightly job is not a nightly gift',
148
155
  breaks: ['grant-marker', 'grant-on-renewal'],
149
- async run({ store, handler }) {
156
+ async run({ store, handler, assert }) {
150
157
  const event = fixture(CYCLE_JAN);
151
158
  await handler.deliver(event);
152
159
  const subId = subscriptionIdOf(event.data.object);
@@ -163,11 +170,11 @@ export const INVARIANTS = [
163
170
  fixtures: [CYCLE_FEB, CYCLE_JAN],
164
171
  states: 'February delivered before January leaves February in the mirror, and grants both',
165
172
  breaks: ['ordering', 'grant-on-renewal'],
166
- async run({ store, handler }) {
173
+ async run({ store, handler, assert }) {
167
174
  await handler.deliver(fixture(CYCLE_FEB)); // arrives first
168
175
  await handler.deliver(fixture(CYCLE_JAN)); // the earlier period, delivered late
169
176
  assert.equal(store.grants.length, 2, 'both periods were paid, so both are owed');
170
- const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice');
177
+ const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice') || {};
171
178
  assert.equal(mirror.periodStart, 1769904000,
172
179
  'the mirror was rewound to the older period — state derived from arrival order');
173
180
  assert.equal(mirror.periodEnd, 1772323200,
@@ -179,7 +186,7 @@ export const INVARIANTS = [
179
186
  fixtures: [REFUND_PARTIAL, REFUND_REMAINDER],
180
187
  states: 'amount_refunded is the total so far: two partials claw back $90, not $130',
181
188
  breaks: ['cumulative-refund'],
182
- async run({ store, handler }) {
189
+ async run({ store, handler, assert }) {
183
190
  seedOneOffPurchase(store);
184
191
  await handler.deliver(fixture(REFUND_PARTIAL));
185
192
  await handler.deliver(fixture(REFUND_REMAINDER));
@@ -201,7 +208,7 @@ export const INVARIANTS = [
201
208
  // and the claim still refuses the second delivery. A suite that judged a redelivery by
202
209
  // its grant count alone would ship with no idempotency and stay green.
203
210
  breaks: [['claim', 'grant-marker'], 'grant-on-renewal'],
204
- async run({ store, handler }) {
211
+ async run({ store, handler, assert }) {
205
212
  await handler.deliver(fixture(CYCLE_JAN));
206
213
  await handler.deliver(fixture(CYCLE_JAN_REDELIVERY));
207
214
  assert.equal(store.grants.length, 1, 'the redelivery granted a second time');
@@ -216,7 +223,7 @@ export const INVARIANTS = [
216
223
  // `increment <= 0` catches a sequential replay by itself, and the claim catches it
217
224
  // without the swap. Neither is tested by this fixture alone.
218
225
  breaks: [['claim', 'cumulative-refund']],
219
- async run({ store, handler }) {
226
+ async run({ store, handler, assert }) {
220
227
  seedOneOffPurchase(store);
221
228
  await handler.deliver(fixture(REFUND_PARTIAL));
222
229
  await handler.deliver(fixture(REFUND_PARTIAL));
@@ -229,16 +236,20 @@ export const INVARIANTS = [
229
236
  fixtures: [SESSION_UNPAID, SESSION_FAILED],
230
237
  states: 'the redirect proves a browser: an unpaid async session grants and converts nothing',
231
238
  breaks: ['async-gate'],
232
- async run({ store, handler }) {
239
+ async run({ store, handler, assert }) {
233
240
  const completed = await handler.deliver(fixture(SESSION_UNPAID));
234
- assert.equal(completed.status, 200, 'an unpaid session must still answer 200');
241
+ // unmutated: the envelope. Removing `async-gate` grants the session and still answers 200.
242
+ assert.unmutated.equal(completed.status, 200, 'an unpaid session must still answer 200');
235
243
  assert.equal(store.grants.length, 0,
236
244
  'a session whose payment_status is unpaid was granted — the money has not moved');
237
- assert.equal(store.conversions.length, 0,
245
+ // unmutated, and the reason is a gap worth seeing: this handler emits conversions on
246
+ // `invoice.paid` only, so no rule here can make a checkout session report one. The
247
+ // assertion states the rule for a reader whose handler DOES emit from a session.
248
+ assert.unmutated.equal(store.conversions.length, 0,
238
249
  'a purchase conversion was reported for a charge that has not cleared');
239
250
  await handler.deliver(fixture(SESSION_FAILED));
240
251
  assert.equal(store.grants.length, 0, 'the failed async payment still granted');
241
- assert.equal(store.conversions.length, 0);
252
+ assert.unmutated.equal(store.conversions.length, 0); // same gap as above
242
253
  assert.equal(store.credits.get('usr_PLACEHOLDER_bob'), undefined);
243
254
  },
244
255
  },
@@ -249,9 +260,9 @@ export const INVARIANTS = [
249
260
  // Without it, a handler that refuses every checkout session satisfies the assertion
250
261
  // above and this pack would have taught the opposite defect.
251
262
  breaks: ['grant-on-checkout'],
252
- async run({ store, handler }) {
263
+ async run({ store, handler, assert }) {
253
264
  const response = await handler.deliver(fixture(SESSION_PAID));
254
- assert.deepEqual(response.body, { received: true });
265
+ assert.unmutated.deepEqual(response.body, { received: true }); // the envelope
255
266
  assert.equal(store.grants.length, 1, 'a cleared card session granted nothing');
256
267
  assert.equal(store.credits.get('usr_PLACEHOLDER_carol'), CREDITS_PER_PERIOD);
257
268
  assert.ok(store.subscriptions.has('sub_PLACEHOLDER_carol'));
@@ -262,10 +273,10 @@ export const INVARIANTS = [
262
273
  fixtures: [CYCLE_JAN],
263
274
  states: 'the conversion id comes from subscription metadata, so browser and server share it',
264
275
  breaks: ['conversion-id-from-metadata', 'grant-on-renewal'],
265
- async run({ store, handler }) {
276
+ async run({ store, handler, assert }) {
266
277
  await handler.deliver(fixture(CYCLE_JAN));
267
278
  assert.equal(store.conversions.length, 1);
268
- const [conversion] = store.conversions;
279
+ const [conversion = {}] = store.conversions;
269
280
  assert.equal(conversion.eventId, 'evtid_PLACEHOLDER_purchase_jan',
270
281
  'the server generated its own id — nothing will ever deduplicate against the pixel');
271
282
  assert.equal(conversion.eventName, 'Purchase', 'the event name is not byte-equal');
@@ -276,34 +287,131 @@ export const INVARIANTS = [
276
287
  },
277
288
  ];
278
289
 
290
+ // ------------------------------------------------- one assertion, one measurement
291
+ //
292
+ // Why the assertions are not called straight through `node:assert`. Until 2026-08-20 the
293
+ // runner returned on the FIRST throw and the matrix compared one row per invariant, so an
294
+ // invariant was measured as a single unit: any assertion that was not the sole
295
+ // discriminator could be replaced by `assert.ok(true)` and nothing went red. Measured that
296
+ // day — three neutered assertions (the `clawbacks.map` deepEqual, the `mirror.periodEnd`
297
+ // equality, and the `response.body` deepEqual) left `--self-test` at exit 0 still printing
298
+ // *"12 assertions, each watched failing"*, and `npm test` at 0 with them.
299
+ //
300
+ // So the `assert` an invariant receives is a RECORDER: same API, but a failure is
301
+ // remembered instead of thrown and the assertions after it still run. Each call site is one
302
+ // assertion with its own identity — its line in this file — and `--self-test` asks the
303
+ // question per assertion rather than per invariant.
304
+ //
305
+ // Two kinds, and the difference is declared at the call site rather than in a table:
306
+ //
307
+ // assert.equal(...) a DISCRIMINATING assertion. At least one mutant must turn it
308
+ // red, or it is not evidence of anything.
309
+ // assert.unmutated.equal() an assertion NO mutant in this pack varies. It pins the
310
+ // envelope or a field the rules do not touch, so it has never
311
+ // been watched failing and the verdict line counts it apart
312
+ // rather than as evidence. Measured, not asserted: the self-test
313
+ // fails if a mutant does break one, because that is a
314
+ // discriminator hiding in the wrong list.
315
+ //
316
+ // A reader pointing this at their own handler keeps writing `assert.equal`; the recorder is
317
+ // API-compatible with `node:assert/strict` for the methods below.
318
+
319
+ const SELF = import.meta.url;
320
+ const METHODS = ['equal', 'notEqual', 'deepEqual', 'notDeepEqual', 'ok', 'match', 'doesNotMatch'];
321
+
322
+ /**
323
+ * The line in THIS file that called the recorder.
324
+ *
325
+ * `captureStackTrace(holder, boundary)` drops `boundary` and everything above it, so the
326
+ * top frame is the assertion's own line — no frame counting, which is what makes the
327
+ * identity survive a refactor of the recorder itself.
328
+ */
329
+ function siteOf(boundary) {
330
+ const holder = {};
331
+ Error.captureStackTrace(holder, boundary);
332
+ const frame = (holder.stack || '').split('\n').slice(1).find((f) => f.includes(SELF));
333
+ const m = frame && frame.match(/:(\d+):\d+\)?\s*$/);
334
+ return m ? Number(m[1]) : 0;
335
+ }
336
+
337
+ function recorder() {
338
+ const seen = new Map(); // line -> { site, unmutated, ok, why }
339
+
340
+ function record(site, unmutated, name, args) {
341
+ let ok = true;
342
+ let why = null;
343
+ try {
344
+ assert[name](...args);
345
+ } catch (error) {
346
+ ok = false;
347
+ why = error.message.split('\n')[0];
348
+ }
349
+ const prev = seen.get(site);
350
+ if (prev) {
351
+ prev.ok = prev.ok && ok;
352
+ prev.why = prev.why || why;
353
+ } else {
354
+ seen.set(site, { site, unmutated, ok, why });
355
+ }
356
+ }
357
+
358
+ const bind = (unmutated) => {
359
+ const api = {};
360
+ for (const name of METHODS) {
361
+ const call = (...args) => record(siteOf(call), unmutated, name, args);
362
+ api[name] = call;
363
+ }
364
+ return api;
365
+ };
366
+
367
+ const api = bind(false);
368
+ api.unmutated = bind(true);
369
+ return { api, seen };
370
+ }
371
+
279
372
  // ------------------------------------------------------------------------- runners
280
373
 
281
374
  async function runOne(invariant, without) {
282
- const context = harness(without);
375
+ const { api, seen } = recorder();
376
+ const { store, handler } = harness(without);
377
+ let threw = null;
283
378
  try {
284
- await invariant.run(context);
285
- return null;
379
+ await invariant.run({ store, handler, assert: api });
286
380
  } catch (error) {
287
- return error.message.split('\n')[0];
381
+ // A real error — a `TypeError` on a row the mutant never wrote, say. It aborts the
382
+ // remaining assertions, and it is still the invariant going red.
383
+ threw = error.message.split('\n')[0];
288
384
  }
385
+ const sites = [...seen.values()];
386
+ return {
387
+ threw,
388
+ sites,
389
+ failed: sites.filter((s) => !s.ok),
390
+ broke: Boolean(threw) || sites.some((s) => !s.ok),
391
+ };
289
392
  }
290
393
 
291
394
  async function runAll() {
292
395
  let failed = 0;
396
+ let asserted = 0;
293
397
  for (const invariant of INVARIANTS) {
294
- const why = await runOne(invariant, []);
295
- if (why) {
398
+ const result = await runOne(invariant, []);
399
+ asserted += result.sites.length;
400
+ if (result.broke) {
296
401
  failed += 1;
297
- console.error(`FAIL ${invariant.id}\n ${why}`);
402
+ console.error(`FAIL ${invariant.id}`);
403
+ for (const s of result.failed) console.error(` :${s.site} ${s.why}`);
404
+ if (result.threw) console.error(` threw: ${result.threw}`);
298
405
  } else {
299
- console.log(`pass ${invariant.id} — ${invariant.states}`);
406
+ console.log(`pass ${invariant.id} (${result.sites.length} assertions) — ${invariant.states}`);
300
407
  }
301
408
  }
302
409
  if (failed) {
303
410
  console.error(`\n${failed} of ${INVARIANTS.length} money invariants failed`);
304
411
  return 1;
305
412
  }
306
- console.log(`\nOK: ${INVARIANTS.length} money invariants hold against the handler`);
413
+ console.log(`\nOK: ${INVARIANTS.length} money invariants hold against the handler `
414
+ + `(${asserted} assertions)`);
307
415
  return 0;
308
416
  }
309
417
 
@@ -333,21 +441,49 @@ async function selfTest() {
333
441
  const mutants = [...RULES.map((rule) => [rule]), ...combos];
334
442
 
335
443
  const matrix = [];
444
+ const coverage = []; // one row per ASSERTION: which mutants turn that call site red
336
445
  for (const invariant of INVARIANTS) {
446
+ // The clean run is the census: it names every assertion this invariant executes, and
447
+ // whether each is declared discriminating or shape. An invariant that records nothing
448
+ // is one whose `run` never took the recorder — the failure mode of the injection.
449
+ const clean = await runOne(invariant, []);
450
+ if (clean.broke) {
451
+ problems.push(`${invariant.id}: fails against the unmutated handler `
452
+ + `(${(clean.failed[0] && `:${clean.failed[0].site} ${clean.failed[0].why}`) || clean.threw})`);
453
+ }
454
+ if (!clean.sites.length) {
455
+ problems.push(`${invariant.id}: recorded no assertion — its run() does not take the `
456
+ + '`assert` the runner passes it, so nothing about it is measured');
457
+ }
458
+ const broken = new Map(clean.sites.map((s) => [s.site, []]));
459
+
337
460
  const singles = [];
338
461
  for (const rule of RULES) {
339
- if (await runOne(invariant, [rule])) singles.push(rule);
462
+ const r = await runOne(invariant, [rule]);
463
+ if (r.broke) singles.push(rule);
464
+ for (const s of r.failed) if (broken.has(s.site)) broken.get(s.site).push(rule);
340
465
  }
341
466
  // Minimal mutants only: a combination containing a rule that already breaks the
342
467
  // assertion on its own says nothing new.
343
468
  const minimalCombos = [];
344
469
  for (const combo of combos) {
345
470
  if (combo.some((rule) => singles.includes(rule))) continue;
346
- if (await runOne(invariant, combo)) minimalCombos.push(combo);
471
+ const r = await runOne(invariant, combo);
472
+ if (r.broke) minimalCombos.push(combo);
473
+ for (const s of r.failed) if (broken.has(s.site)) broken.get(s.site).push(key(combo));
347
474
  }
348
475
  const broke = [...singles.map((r) => [r]), ...minimalCombos];
349
476
  matrix.push({ id: invariant.id, broke, singles });
350
477
 
478
+ for (const site of clean.sites) {
479
+ coverage.push({
480
+ invariant: invariant.id,
481
+ site: site.site,
482
+ unmutated: site.unmutated,
483
+ by: broken.get(site.site) || [],
484
+ });
485
+ }
486
+
351
487
  const expected = invariant.breaks
352
488
  .map((b) => key(Array.isArray(b) ? b : [b])).sort().join(', ');
353
489
  const measured = broke.map(key).sort().join(', ');
@@ -368,6 +504,27 @@ async function selfTest() {
368
504
  console.log(` ${row.id.padEnd(width)} ${row.broke.map(key).join(', ') || '(nothing)'}`);
369
505
  }
370
506
 
507
+ // The per-ASSERTION half, and the one the per-invariant table above cannot give: an
508
+ // invariant goes red as a unit, so a dead assertion inside a live one is invisible there.
509
+ console.log('\nassertion x mutant — every call site, and what turns IT red\n');
510
+ for (const row of coverage) {
511
+ const label = `${row.invariant}:${row.site}`.padEnd(width + 6);
512
+ if (row.unmutated) {
513
+ console.log(` ${label} unmutated — no rule varies what it checks`);
514
+ if (row.by.length) {
515
+ problems.push(`${row.invariant} :${row.site} is declared assert.unmutated and `
516
+ + `[${row.by.join(', ')}] breaks it — it discriminates, so drop the .unmutated`);
517
+ }
518
+ } else {
519
+ console.log(` ${label} ${row.by.join(', ') || '(nothing)'}`);
520
+ if (!row.by.length) {
521
+ problems.push(`${row.invariant} :${row.site} is broken by no mutant — the assertion `
522
+ + 'cannot fail. Either it is not testing anything, or no rule varies what it '
523
+ + 'checks and it belongs on assert.unmutated, which the verdict counts apart');
524
+ }
525
+ }
526
+ }
527
+
371
528
  console.log('\nisolation — the fixture that separates each rule from every other\n');
372
529
  for (const rule of RULES) {
373
530
  const isolating = matrix.filter((row) => {
@@ -392,7 +549,10 @@ async function selfTest() {
392
549
  for (const p of problems) console.error(` - ${p}`);
393
550
  return 1;
394
551
  }
395
- console.log(`\nOK: ${INVARIANTS.length} assertions, each watched failing; `
552
+ const pinned = coverage.filter((r) => r.unmutated).length;
553
+ console.log(`\nOK: ${INVARIANTS.length} invariants over ${coverage.length} assertions — `
554
+ + `${coverage.length - pinned} watched failing ONE CALL SITE AT A TIME against `
555
+ + `${mutants.length} mutants; ${pinned} declared unmutated and measured unbreakable; `
396
556
  + `${RULES.length} rules, each isolated by a fixture`);
397
557
  return 0;
398
558
  }
@@ -158,12 +158,13 @@ the helper works and says nothing about the route Stripe posts to.
158
158
  ## The shipped fixtures and the assertion pack
159
159
 
160
160
  Everything above tells you what to write. `fixtures/` is that suite already written:
161
- nine Stripe event bodies, twelve assertions, and the mutants that prove each assertion can
162
- fail. Copy the directory into your repository and run it.
161
+ nine Stripe event bodies, twelve invariants over 45 assertions, and the mutants that prove
162
+ each assertion can fail — per call site, not per invariant, which is a distinction this
163
+ pack got wrong until 2026-08-20. Copy the directory into your repository and run it.
163
164
 
164
165
  ```bash
165
166
  node fixtures/assert-money-invariants.mjs # the assertions
166
- node fixtures/assert-money-invariants.mjs --self-test # every one of them, watched failing
167
+ node fixtures/assert-money-invariants.mjs --self-test # each ASSERTION, watched failing
167
168
  ```
168
169
 
169
170
  Point it at your handler by replacing the two imports at the top of