@ssheleg/sheleg-dev 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +62 -0
- package/README.md +20 -6
- package/SECURITY.md +20 -11
- package/package.json +1 -1
- package/plugins/sheleg-dev/.claude-plugin/plugin.json +1 -1
- package/plugins/sheleg-dev/skills/ad-tracking/fixtures/README.md +5 -4
- package/plugins/sheleg-dev/skills/ad-tracking/fixtures/assert-dedup-contract.mjs +168 -20
- package/plugins/sheleg-dev/skills/ad-tracking/references/meta-linkedin.md +6 -2
- package/plugins/sheleg-dev/skills/crypto-payments/SKILL.md +17 -80
- package/plugins/sheleg-dev/skills/crypto-payments/references/callback-route-hardening.md +55 -0
- package/plugins/sheleg-dev/skills/crypto-payments/references/testing-and-local-dev.md +60 -0
- package/plugins/sheleg-dev/skills/stripe-billing/fixtures/README.md +5 -4
- package/plugins/sheleg-dev/skills/stripe-billing/fixtures/assert-money-invariants.mjs +195 -35
- package/plugins/sheleg-dev/skills/stripe-billing/references/testing-and-local-dev.md +4 -3
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,68 @@ All notable changes to this project are documented here.
|
|
|
4
4
|
Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
5
5
|
Versioning: [SemVer](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## v0.8.0 — 2026-08-20
|
|
8
|
+
|
|
9
|
+
**The self-test that proved the money invariants was measuring one thing and claiming
|
|
10
|
+
another.** `--self-test` deleted a rule, ran an invariant, and asked whether the invariant
|
|
11
|
+
went red. Twelve invariants carried **45 `assert.` calls**, and the verdict line called
|
|
12
|
+
them *"12 assertions, each watched failing"* — so any assertion that was not the sole
|
|
13
|
+
discriminator of a rule could be replaced by `assert.ok(true)` and nothing anywhere would
|
|
14
|
+
notice. Measured against the shipped v0.7.0 pack: three neutered assertions (the
|
|
15
|
+
`clawbacks.map` deepEqual, the `mirror.periodEnd` equality, the `response.body` deepEqual)
|
|
16
|
+
left `--self-test` at exit 0 and `npm test` at 0. The same hole in `ad-tracking`, in the PII
|
|
17
|
+
guard `references/meta-linkedin.md` advertises by name.
|
|
18
|
+
|
|
19
|
+
The `assert` an invariant receives is now a recorder: a failure is remembered instead of
|
|
20
|
+
thrown, so the assertions after it still run, and each call site is one measurement
|
|
21
|
+
identified by its line. **37 of 45 watched failing, one call site at a time. Eight declared
|
|
22
|
+
`assert.unmutated`** — an assertion no rule in the pack varies — **and measured unbreakable,
|
|
23
|
+
with the reason at the call site.** Both directions: an `unmutated` assertion a mutant does
|
|
24
|
+
break is a failure too, or the escape hatch is the bypass. Four assertions that were being
|
|
25
|
+
pre-empted by a `TypeError` on the line above them became measurable rather than declared
|
|
26
|
+
away. `test/fixtures_test.js` grew three plants, one of them the NON-first assertion inside
|
|
27
|
+
a multi-assert invariant — the plant that used to pass.
|
|
28
|
+
|
|
29
|
+
Then the sweep that started there and did not stop:
|
|
30
|
+
|
|
31
|
+
- **The validator's check count did not count checks.** `checks = 10 + len(skill_dirs)`, so
|
|
32
|
+
adding a skill moved the number and adding a check did not — and five ledger rows read it
|
|
33
|
+
as evidence a guard had arrived. Every check is registered; the verdict prints the
|
|
34
|
+
registry's length, and the ledger's quoted copy of that line is compared against it.
|
|
35
|
+
- **The ledger described an artifact nobody ships.** Its shipped block was headed `v0.6.0`
|
|
36
|
+
while `v0.7.0` was tagged and on npm, and 37 rows read *verified locally · unreleased*
|
|
37
|
+
under four blocks closing *"Nothing was released."* The heading is now compared against
|
|
38
|
+
`git describe --tags` and against `package.json`.
|
|
39
|
+
- **The `mcp__.*` half of the manual gate had no guard.** Deleting that entry from
|
|
40
|
+
`hooks/hooks.json` left all three suites green, while the decision module does refuse
|
|
41
|
+
`mcp__…__create_refund` by name. And `README.md`'s copy-channel snippet registered one
|
|
42
|
+
matcher against the plugin's two — a weaker gate handed out by the document that exists
|
|
43
|
+
because the channel has none. Both are now checked; the snippet by matcher set.
|
|
44
|
+
- **This repository measured no body budget at all**, so `crypto-payments/SKILL.md` sat past
|
|
45
|
+
the house working limit and was caught by another repository's auditor. Measured here now,
|
|
46
|
+
and the skill was **split rather than trimmed**: `references/callback-route-hardening.md`
|
|
47
|
+
and `references/testing-and-local-dev.md`, 4894 → 4387 tokens. All six skills `0 GAP`.
|
|
48
|
+
- **Counted numbers became computed ones.** Seven in `SECURITY.md` (all four correct on the
|
|
49
|
+
day they were written, three of them moved by this change) and two in
|
|
50
|
+
`docs/evals/stripe-billing.md`, which restated `4994` tokens / `441` lines / `0 GAP, 13
|
|
51
|
+
PASS` against a measured 4747 / 409 / `0 GAP, 14 PASS`.
|
|
52
|
+
- **A document that ships now resolves against the tarball, not only against a clone.**
|
|
53
|
+
`SECURITY.md` sent a reader to two documents `npm pack` does not contain.
|
|
54
|
+
- **`docs/` joined the checked corpus**, with every document in it classified as live or as
|
|
55
|
+
a dated record — `docs/AGENT_SYNC.md` had been naming six paths this repository does not
|
|
56
|
+
have.
|
|
57
|
+
- **`.claude/agent-sync.json` pointed at a file that did not exist** and guarded the ledger
|
|
58
|
+
without guarding the board.
|
|
59
|
+
- **`CONTRIBUTING.md` called the gate two suites and it runs three**; the PR template asked
|
|
60
|
+
for one third of it. Both derive the list from `package.json` → `scripts.test` now.
|
|
61
|
+
- **`install.sh` — the channel that `rm -rf`s each destination — printed no notice** that
|
|
62
|
+
the gate does not travel with it, while the npm installer did.
|
|
63
|
+
|
|
64
|
+
Fourteen negative self-tests were added, one per guard, and all **42** were watched refusing
|
|
65
|
+
their plants on a green tree. CI run `32293489020` at `6f66255` — 39 steps, 39 `success`,
|
|
66
|
+
28 of 28 negatives — retires the *"CI has not seen any of this"* limitation the ledger had
|
|
67
|
+
recorded four separate times.
|
|
68
|
+
|
|
7
69
|
## v0.7.0 — 2026-08-19
|
|
8
70
|
|
|
9
71
|
**The pack that takes money had the weakest controls in the family, and every one of them
|
package/README.md
CHANGED
|
@@ -22,8 +22,9 @@ Each carries its own references and loads them only when the work reaches them.
|
|
|
22
22
|
|
|
23
23
|
**Two of them ship a suite rather than a warning.** `stripe-billing` and `ad-tracking`
|
|
24
24
|
carry a `fixtures/` directory — real provider webhook bodies, an assertion pack, and a
|
|
25
|
-
`--self-test` that deletes one rule at a time so you can watch every assertion fail
|
|
26
|
-
the
|
|
25
|
+
`--self-test` that deletes one rule at a time so you can watch every assertion fail — one
|
|
26
|
+
call site at a time, which is not the same thing as one invariant at a time. Copy the
|
|
27
|
+
directory in and run it:
|
|
27
28
|
|
|
28
29
|
```bash
|
|
29
30
|
node plugins/sheleg-dev/skills/stripe-billing/fixtures/assert-money-invariants.mjs --self-test
|
|
@@ -163,12 +164,24 @@ settings, by design.
|
|
|
163
164
|
`.claude/settings.json`:
|
|
164
165
|
|
|
165
166
|
```json
|
|
166
|
-
{ "hooks": { "PreToolUse": [
|
|
167
|
+
{ "hooks": { "PreToolUse": [
|
|
168
|
+
{ "matcher": "Bash",
|
|
167
169
|
"hooks": [ { "type": "command",
|
|
168
170
|
"command": "node \"$HOME/.claude/skills/.sheleg-dev-hooks/money-gate.js\"",
|
|
169
|
-
"timeout": 15 } ] }
|
|
171
|
+
"timeout": 15 } ] },
|
|
172
|
+
{ "matcher": "mcp__.*",
|
|
173
|
+
"hooks": [ { "type": "command",
|
|
174
|
+
"command": "node \"$HOME/.claude/skills/.sheleg-dev-hooks/money-gate.js\"",
|
|
175
|
+
"timeout": 15 } ] }
|
|
176
|
+
] } }
|
|
170
177
|
```
|
|
171
178
|
|
|
179
|
+
**Both entries, not just the first.** A refund does not care which door it came
|
|
180
|
+
through: `create_refund` on an MCP server never touches a shell, and the table above
|
|
181
|
+
advertises it. Until 2026-08-20 this snippet carried the `Bash` matcher alone — a
|
|
182
|
+
weaker gate than the plugin's, handed out by the document that exists because the copy
|
|
183
|
+
channels have no gate at all. `test/validate.py` now compares the two matcher sets.
|
|
184
|
+
|
|
172
185
|
Copy `plugins/sheleg-dev/hooks/` to that path first. `npx @ssheleg/sheleg-dev` prints
|
|
173
186
|
this reminder; **nothing enforces it**, which is why the plugin channel is the
|
|
174
187
|
recommended one.
|
|
@@ -219,8 +232,9 @@ matter does not error — hosts truncate it silently, which is worse); and
|
|
|
219
232
|
dangling link nor a file nobody loads can ship.
|
|
220
233
|
|
|
221
234
|
Plus, for the money fixtures, that every assertion has been watched failing and that no
|
|
222
|
-
two rules are covering for each other — `test/fixtures_test.js` neuters
|
|
223
|
-
|
|
235
|
+
two rules are covering for each other — `test/fixtures_test.js` neuters assertions,
|
|
236
|
+
including one that is **not** the first inside a multi-assert invariant, and requires each
|
|
237
|
+
pack's `--self-test` to notice. That last plant is the one that used to pass.
|
|
224
238
|
|
|
225
239
|
CI plants a defect against every one of those guards and requires the validator
|
|
226
240
|
to fail. A green from a check nobody has watched fail is not evidence.
|
package/SECURITY.md
CHANGED
|
@@ -16,7 +16,7 @@ documents name that does not exist here.
|
|
|
16
16
|
|
|
17
17
|
## What ships, and what of it executes
|
|
18
18
|
|
|
19
|
-
`git ls-files plugins` returns **
|
|
19
|
+
`git ls-files plugins` returns **52 files: 30 markdown, one plugin manifest, three
|
|
20
20
|
files of the manual gate, and 18 files of money fixtures.** Two of the six skills now
|
|
21
21
|
ship code you can run — the assertion packs under `fixtures/` — and that is a change
|
|
22
22
|
from every release before v0.7.0, so it is stated here rather than left to be noticed.
|
|
@@ -26,19 +26,23 @@ no socket; the greps below prove all three.
|
|
|
26
26
|
| Component | Count | Runtime behavior |
|
|
27
27
|
|---|---|---|
|
|
28
28
|
| `SKILL.md`, one per skill | 6 | Text. Read by the agent, executes nothing. |
|
|
29
|
-
| `references/` files, loaded on demand |
|
|
29
|
+
| `references/` files, loaded on demand | 22 | Text. Same. |
|
|
30
30
|
| `plugins/sheleg-dev/.claude-plugin/plugin.json` | 1 | Manifest read by Claude Code. |
|
|
31
31
|
| `plugins/sheleg-dev/hooks/hooks.json` | 1 | Hook manifest read by Claude Code. Declares one `PreToolUse` hook and no `if` filter. |
|
|
32
32
|
| `plugins/sheleg-dev/hooks/money-gate.js` | 1 | Runs on a tool call, when the plugin is enabled. Requires `path` and its own decision module — nothing else. Reads stdin, writes one JSON line, exits 0 on every path. |
|
|
33
33
|
| `plugins/sheleg-dev/hooks/lib/moneygate.js` | 1 | Pure. Payload and environment in, verdict out. No `require` at all, no filesystem, no clock. |
|
|
34
34
|
| `bin/sheleg-dev.js` — the npm installer | 1 | Runs only when you invoke it. Node built-ins only: `fs`, `path`, `os`. |
|
|
35
|
-
| `install.sh` — the shell installer | 1 |
|
|
35
|
+
| `install.sh` — the shell installer | 1 | **Not in the tarball** — `files` ships `bin/` and `plugins/`, so this one reaches you only through a clone. Runs when you invoke it. Coreutils only: `cd`, `pwd`, `dirname`, `basename`, `mkdir`, `rm`, `cp`, `echo`. It is the destructive channel: `rm -rf` per skill, then `cp -R`. |
|
|
36
36
|
| `plugins/sheleg-dev/skills/*/fixtures/*.json` — webhook payloads | 12 | Data. Placeholder ids only; no key, token, signing secret or real customer id. |
|
|
37
37
|
| `plugins/sheleg-dev/skills/*/fixtures/*.mjs` — the assertion packs and their reference implementations | 4 | Runs only when you invoke it. Reads the JSON beside it with `readFileSync` and nothing else: no `child_process`, no `fetch`, no write, no `process.env`. |
|
|
38
38
|
| `plugins/sheleg-dev/skills/*/fixtures/manifest.json` and its `README.md` | 4 | Text. The invariant-to-fixture-to-document map, which `test/validate.py` reads. |
|
|
39
39
|
|
|
40
|
-
The published tarball is
|
|
41
|
-
`LICENSE` and `package.json` —
|
|
40
|
+
The published tarball is `bin/` and `plugins/` plus `README.md`, `CHANGELOG.md`,
|
|
41
|
+
`SECURITY.md`, `LICENSE` and `package.json` — 58 files, listed by
|
|
42
|
+
`npm pack --dry-run`. `install.sh`, `test/`, `docs/` and `CONTRIBUTING.md` are **not**
|
|
43
|
+
in it; `test/validate.py` now refuses a path these shipped documents name that the
|
|
44
|
+
tarball does not carry, because resolving in a clone is the wrong question for a
|
|
45
|
+
document that ships.
|
|
42
46
|
|
|
43
47
|
**The gate can refuse a command and can never run one.** `money-gate.js` requires
|
|
44
48
|
`path` and the decision module; the decision module requires nothing. There is no
|
|
@@ -140,8 +144,11 @@ Not the bytes — the advice. These six skills describe Stripe, Heleket and BTCP
|
|
|
140
144
|
Google Sign-In, Workload Identity Federation, four ad platforms and browser
|
|
141
145
|
defaults, and every one of those changes. **A claim here that a provider has since
|
|
142
146
|
changed is a security defect in this pack**, not merely a stale doc, and it is the
|
|
143
|
-
exposure
|
|
144
|
-
|
|
147
|
+
exposure the verification ledger names as the largest one this repository does not
|
|
148
|
+
test. **Neither that ledger nor the contributor guide is in this tarball** —
|
|
149
|
+
`docs/evidence/verification.md` and `CONTRIBUTING.md` live in the git repository at
|
|
150
|
+
<https://github.com/ssheleg/sheleg-dev>, and until 2026-08-20 this paragraph named
|
|
151
|
+
both as if you had them. The guide requires a claim about what a provider does
|
|
145
152
|
*today* to carry the date it was true. If you find one that has rotted, report it
|
|
146
153
|
with what the correct claim is and what backs it.
|
|
147
154
|
|
|
@@ -161,7 +168,7 @@ git clone https://github.com/ssheleg/sheleg-dev && cd sheleg-dev
|
|
|
161
168
|
# self-describing documents name -- this file included.
|
|
162
169
|
python3 test/validate.py
|
|
163
170
|
|
|
164
|
-
# The shipped payload:
|
|
171
|
+
# The shipped payload: 52 files.
|
|
165
172
|
git ls-files plugins
|
|
166
173
|
|
|
167
174
|
# 22 lines: the plugin manifest, the three files of the manual gate, and the 18
|
|
@@ -176,11 +183,13 @@ grep -nE "\brequire\(" plugins/sheleg-dev/hooks/money-gate.js plugins/sheleg-dev
|
|
|
176
183
|
grep -nE "child_process|\bfetch\(|require\('(http|https|net|fs|dns|tls)'\)|spawn\(|execFile|writeFile|appendFile|unlink" \
|
|
177
184
|
plugins/sheleg-dev/hooks/money-gate.js plugins/sheleg-dev/hooks/lib/moneygate.js
|
|
178
185
|
|
|
179
|
-
# Both directions of the gate, watched:
|
|
186
|
+
# Both directions of the gate, watched: every refusal and every false-positive
|
|
187
|
+
# plant. The command prints its own count -- this document does not restate it.
|
|
180
188
|
node test/moneygate_test.js
|
|
181
189
|
|
|
182
190
|
# The money fixtures: both assertion packs against their reference implementation, both
|
|
183
|
-
# --self-test runs, and
|
|
191
|
+
# --self-test runs (per ASSERTION, not per invariant), and the plants that neuter one.
|
|
192
|
+
# The command prints its own count.
|
|
184
193
|
node test/fixtures_test.js
|
|
185
194
|
|
|
186
195
|
# What the assertion packs cannot reach: NO OUTPUT, and grep exits 1 because it
|
|
@@ -196,6 +205,6 @@ grep -nE "require|child_process|exec|spawn|fetch|socket|rm -rf|cp -R|copyFileSyn
|
|
|
196
205
|
# Live-key shapes across the skill payload: one line, and it is a placeholder.
|
|
197
206
|
grep -rnE "sk_live_[A-Za-z0-9]|rk_live_[A-Za-z0-9]|whsec_[A-Za-z0-9]{8}|BEGIN [A-Z ]*PRIVATE KEY" plugins
|
|
198
207
|
|
|
199
|
-
# What npm publishes, and nothing else:
|
|
208
|
+
# What npm publishes, and nothing else: 58 files.
|
|
200
209
|
npm pack --dry-run
|
|
201
210
|
```
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ssheleg/sheleg-dev",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.0",
|
|
4
4
|
"description": "Six integration skills for the layer a product reaches once it has users: Stripe subscription billing, crypto payments, ad and conversion tracking under Consent Mode v2, Google sign-in and the wider Google auth surface, and Core Web Vitals. This package is the installer CLI.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"sheleg-dev": "bin/sheleg-dev.js"
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"name": "sheleg-dev",
|
|
3
3
|
"displayName": "Sheleg Dev",
|
|
4
4
|
"description": "The integration layer a product needs once it has users: Stripe subscription billing reconciled into your own database, crypto payments that survive over-payment and duplicate webhooks, ad and conversion tracking under Consent Mode v2, Google sign-in with the account-linking guards, the wider Google auth surface, and Core Web Vitals work that moves the score.",
|
|
5
|
-
"version": "0.
|
|
5
|
+
"version": "0.8.0",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "ssheleg",
|
|
8
8
|
"url": "https://x.com/sshlg93"
|
|
@@ -3,12 +3,13 @@
|
|
|
3
3
|
**Copy this whole directory into your repository.** It is the executable half of what
|
|
4
4
|
`SKILL.md` → *E-commerce* and `references/meta-linkedin.md` →
|
|
5
5
|
*Deduplication with the Conversions API* state in prose: the three payloads one purchase
|
|
6
|
-
produces, six
|
|
7
|
-
emitter one rule at a time so you can watch
|
|
6
|
+
produces, six invariants a correct emitter must hold — 12 assertions between them — and a
|
|
7
|
+
self-test that breaks the emitter one rule at a time so you can watch each assertion fail
|
|
8
|
+
on its own.
|
|
8
9
|
|
|
9
10
|
```bash
|
|
10
|
-
node assert-dedup-contract.mjs # 6
|
|
11
|
-
node assert-dedup-contract.mjs --self-test # break one rule at a time;
|
|
11
|
+
node assert-dedup-contract.mjs # 6 invariants / 12 assertions
|
|
12
|
+
node assert-dedup-contract.mjs --self-test # break one rule at a time; EACH ASSERTION must go red
|
|
12
13
|
```
|
|
13
14
|
|
|
14
15
|
No dependencies, no network, **no access token anywhere in this directory** — the sink
|
|
@@ -77,7 +77,7 @@ export const INVARIANTS = [
|
|
|
77
77
|
fixtures: [PIXEL_BROWSER, CAPI_SERVER],
|
|
78
78
|
states: 'one id, generated once and used by both sides, or the revenue counts twice',
|
|
79
79
|
breaks: ['shared-event-id'],
|
|
80
|
-
run({ sink, emitter }) {
|
|
80
|
+
run({ sink, emitter, assert }) {
|
|
81
81
|
runPurchase({ sink, emitter }, CLEARED());
|
|
82
82
|
// The browser side is read from the SHIPPED fixture rather than from the sink. That
|
|
83
83
|
// is deliberate: it keeps this assertion independent of whether the browser event
|
|
@@ -95,7 +95,7 @@ export const INVARIANTS = [
|
|
|
95
95
|
fixtures: [PIXEL_BROWSER, CAPI_SERVER],
|
|
96
96
|
states: 'Meta compares the name as well as the id, and the name is the half that is missed',
|
|
97
97
|
breaks: ['exact-event-name'],
|
|
98
|
-
run({ sink, emitter }) {
|
|
98
|
+
run({ sink, emitter, assert }) {
|
|
99
99
|
runPurchase({ sink, emitter }, CLEARED());
|
|
100
100
|
const pixelName = fixture(PIXEL_BROWSER)[1]; // the browser contract, not the sink
|
|
101
101
|
const serverName = sink.capi[0].data[0].event_name;
|
|
@@ -110,7 +110,7 @@ export const INVARIANTS = [
|
|
|
110
110
|
fixtures: [CAPI_FORBIDDEN],
|
|
111
111
|
states: 'the thank-you page has no idea whether the money moved, so it reports nothing',
|
|
112
112
|
breaks: ['webhook-sourced'],
|
|
113
|
-
run({ sink, emitter }) {
|
|
113
|
+
run({ sink, emitter, assert }) {
|
|
114
114
|
runPurchase({ sink, emitter }, NOT_CLEARED());
|
|
115
115
|
assert.equal(sink.capi.length, 0,
|
|
116
116
|
'a Conversions API purchase was sent for a charge that has not cleared — this is '
|
|
@@ -125,12 +125,16 @@ export const INVARIANTS = [
|
|
|
125
125
|
fixtures: [PIXEL_BROWSER],
|
|
126
126
|
states: 'the browser event stays and stays subordinate — it carries the click ids',
|
|
127
127
|
breaks: ['keep-browser-event'],
|
|
128
|
-
run({ sink, emitter }) {
|
|
128
|
+
run({ sink, emitter, assert }) {
|
|
129
129
|
runPurchase({ sink, emitter }, CLEARED());
|
|
130
130
|
assert.equal(sink.pixel.length, 1,
|
|
131
131
|
'no browser event fired: the click id, the consent state and the session are lost, '
|
|
132
132
|
+ 'and only the browser had them');
|
|
133
|
-
|
|
133
|
+
// `|| []` on purpose: with no browser event at all this must FAIL rather than throw a
|
|
134
|
+
// TypeError, because an assertion the runner never reached is one no mutant has been
|
|
135
|
+
// watched breaking — which is how it sat unmeasured until 2026-08-20.
|
|
136
|
+
assert.equal((sink.pixel[0] || [])[0], 'track',
|
|
137
|
+
'the browser event is not a `track` call — the pixel contract is `fbq(\'track\', …)`');
|
|
134
138
|
},
|
|
135
139
|
},
|
|
136
140
|
{
|
|
@@ -138,7 +142,7 @@ export const INVARIANTS = [
|
|
|
138
142
|
fixtures: [CAPI_SERVER],
|
|
139
143
|
states: 'advanced matching sends hashes, never the address itself',
|
|
140
144
|
breaks: ['hashed-identifiers'],
|
|
141
|
-
run({ sink, emitter }) {
|
|
145
|
+
run({ sink, emitter, assert }) {
|
|
142
146
|
runPurchase({ sink, emitter }, CLEARED());
|
|
143
147
|
const userData = sink.capi[0].data[0].user_data;
|
|
144
148
|
for (const value of userData.em) {
|
|
@@ -156,7 +160,7 @@ export const INVARIANTS = [
|
|
|
156
160
|
// Broad by construction: it compares the whole body, so every rule that changes any
|
|
157
161
|
// field turns it red. It is here so a reader can trust the fixtures they copy.
|
|
158
162
|
breaks: ['shared-event-id', 'exact-event-name', 'hashed-identifiers', 'keep-browser-event'],
|
|
159
|
-
run({ sink, emitter }) {
|
|
163
|
+
run({ sink, emitter, assert }) {
|
|
160
164
|
runPurchase({ sink, emitter }, CLEARED());
|
|
161
165
|
assert.deepEqual(sink.capi[0], fixture(CAPI_SERVER));
|
|
162
166
|
assert.deepEqual(sink.pixel[0], fixture(PIXEL_BROWSER));
|
|
@@ -164,46 +168,166 @@ export const INVARIANTS = [
|
|
|
164
168
|
},
|
|
165
169
|
];
|
|
166
170
|
|
|
171
|
+
// ------------------------------------------------- one assertion, one measurement
|
|
172
|
+
//
|
|
173
|
+
// Why the assertions are not called straight through `node:assert`. Until 2026-08-20 the
|
|
174
|
+
// runner returned on the FIRST throw and the matrix compared one row per invariant, so an
|
|
175
|
+
// invariant was measured as a single unit: any assertion that was not the sole
|
|
176
|
+
// discriminator could be replaced by `assert.ok(true)` and nothing went red. Measured that
|
|
177
|
+
// day in this pack too — the PII guard below (*"user_data carries something with an @ in
|
|
178
|
+
// it"*, advertised by name at `references/meta-linkedin.md`) was neutered and
|
|
179
|
+
// `--self-test` stayed at exit 0, because the `assert.match` above it discriminated the
|
|
180
|
+
// same rule.
|
|
181
|
+
//
|
|
182
|
+
// So the `assert` an invariant receives is a RECORDER: same API, but a failure is
|
|
183
|
+
// remembered instead of thrown and the assertions after it still run. Each call site is one
|
|
184
|
+
// assertion with its own identity — its line in this file — and `--self-test` asks the
|
|
185
|
+
// question per assertion rather than per invariant.
|
|
186
|
+
//
|
|
187
|
+
// `assert.unmutated.equal(...)` marks an assertion NO mutant in this pack varies: it pins
|
|
188
|
+
// something the rules do not touch, so it has never been watched failing and the verdict
|
|
189
|
+
// counts it apart rather than as evidence. Measured, not asserted — the self-test fails
|
|
190
|
+
// both when a plain assertion turns out unbreakable and when an `unmutated` one turns out
|
|
191
|
+
// breakable.
|
|
192
|
+
//
|
|
193
|
+
// Duplicated from `stripe-billing/fixtures/assert-money-invariants.mjs` for the reason
|
|
194
|
+
// given at the top: a reader installs one skill, not six.
|
|
195
|
+
|
|
196
|
+
const SELF = import.meta.url;
|
|
197
|
+
const METHODS = ['equal', 'notEqual', 'deepEqual', 'notDeepEqual', 'ok', 'match', 'doesNotMatch'];
|
|
198
|
+
|
|
199
|
+
/**
|
|
200
|
+
* The line in THIS file that called the recorder.
|
|
201
|
+
*
|
|
202
|
+
* `captureStackTrace(holder, boundary)` drops `boundary` and everything above it, so the
|
|
203
|
+
* top frame is the assertion's own line — no frame counting, which is what makes the
|
|
204
|
+
* identity survive a refactor of the recorder itself.
|
|
205
|
+
*/
|
|
206
|
+
function siteOf(boundary) {
|
|
207
|
+
const holder = {};
|
|
208
|
+
Error.captureStackTrace(holder, boundary);
|
|
209
|
+
const frame = (holder.stack || '').split('\n').slice(1).find((f) => f.includes(SELF));
|
|
210
|
+
const m = frame && frame.match(/:(\d+):\d+\)?\s*$/);
|
|
211
|
+
return m ? Number(m[1]) : 0;
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
function recorder() {
|
|
215
|
+
const seen = new Map(); // line -> { site, unmutated, ok, why }
|
|
216
|
+
|
|
217
|
+
function record(site, unmutated, name, args) {
|
|
218
|
+
let ok = true;
|
|
219
|
+
let why = null;
|
|
220
|
+
try {
|
|
221
|
+
assert[name](...args);
|
|
222
|
+
} catch (error) {
|
|
223
|
+
ok = false;
|
|
224
|
+
why = error.message.split('\n')[0];
|
|
225
|
+
}
|
|
226
|
+
const prev = seen.get(site);
|
|
227
|
+
if (prev) {
|
|
228
|
+
prev.ok = prev.ok && ok;
|
|
229
|
+
prev.why = prev.why || why;
|
|
230
|
+
} else {
|
|
231
|
+
seen.set(site, { site, unmutated, ok, why });
|
|
232
|
+
}
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
const bind = (unmutated) => {
|
|
236
|
+
const api = {};
|
|
237
|
+
for (const name of METHODS) {
|
|
238
|
+
const call = (...args) => record(siteOf(call), unmutated, name, args);
|
|
239
|
+
api[name] = call;
|
|
240
|
+
}
|
|
241
|
+
return api;
|
|
242
|
+
};
|
|
243
|
+
|
|
244
|
+
const api = bind(false);
|
|
245
|
+
api.unmutated = bind(true);
|
|
246
|
+
return { api, seen };
|
|
247
|
+
}
|
|
248
|
+
|
|
167
249
|
// ------------------------------------------------------------------------- runners
|
|
168
250
|
|
|
169
251
|
async function runOne(invariant, without) {
|
|
170
|
-
const
|
|
252
|
+
const { api, seen } = recorder();
|
|
253
|
+
const { sink, emitter } = harness(without);
|
|
254
|
+
let threw = null;
|
|
171
255
|
try {
|
|
172
|
-
await invariant.run(
|
|
173
|
-
return null;
|
|
256
|
+
await invariant.run({ sink, emitter, assert: api });
|
|
174
257
|
} catch (error) {
|
|
175
|
-
|
|
258
|
+
// A real error — a read off a row the mutant never wrote, say. It aborts the remaining
|
|
259
|
+
// assertions, and it is still the invariant going red.
|
|
260
|
+
threw = error.message.split('\n')[0];
|
|
176
261
|
}
|
|
262
|
+
const sites = [...seen.values()];
|
|
263
|
+
return {
|
|
264
|
+
threw,
|
|
265
|
+
sites,
|
|
266
|
+
failed: sites.filter((s) => !s.ok),
|
|
267
|
+
broke: Boolean(threw) || sites.some((s) => !s.ok),
|
|
268
|
+
};
|
|
177
269
|
}
|
|
178
270
|
|
|
179
271
|
async function runAll() {
|
|
180
272
|
let failed = 0;
|
|
273
|
+
let asserted = 0;
|
|
181
274
|
for (const invariant of INVARIANTS) {
|
|
182
|
-
const
|
|
183
|
-
|
|
275
|
+
const result = await runOne(invariant, []);
|
|
276
|
+
asserted += result.sites.length;
|
|
277
|
+
if (result.broke) {
|
|
184
278
|
failed += 1;
|
|
185
|
-
console.error(`FAIL ${invariant.id}
|
|
279
|
+
console.error(`FAIL ${invariant.id}`);
|
|
280
|
+
for (const s of result.failed) console.error(` :${s.site} ${s.why}`);
|
|
281
|
+
if (result.threw) console.error(` threw: ${result.threw}`);
|
|
186
282
|
} else {
|
|
187
|
-
console.log(`pass ${invariant.id} — ${invariant.states}`);
|
|
283
|
+
console.log(`pass ${invariant.id} (${result.sites.length} assertions) — ${invariant.states}`);
|
|
188
284
|
}
|
|
189
285
|
}
|
|
190
286
|
if (failed) {
|
|
191
287
|
console.error(`\n${failed} of ${INVARIANTS.length} deduplication assertions failed`);
|
|
192
288
|
return 1;
|
|
193
289
|
}
|
|
194
|
-
console.log(`\nOK: ${INVARIANTS.length} deduplication
|
|
290
|
+
console.log(`\nOK: ${INVARIANTS.length} deduplication invariants hold against the emitter `
|
|
291
|
+
+ `(${asserted} assertions)`);
|
|
195
292
|
return 0;
|
|
196
293
|
}
|
|
197
294
|
|
|
198
295
|
async function selfTest() {
|
|
199
296
|
const problems = [];
|
|
200
297
|
const matrix = [];
|
|
298
|
+
const coverage = []; // one row per ASSERTION: which mutants turn that call site red
|
|
201
299
|
for (const invariant of INVARIANTS) {
|
|
300
|
+
// The clean run is the census: it names every assertion this invariant executes. One
|
|
301
|
+
// that records nothing has a run() which never took the recorder, so nothing about it
|
|
302
|
+
// is measured — the failure mode of the injection itself.
|
|
303
|
+
const clean = await runOne(invariant, []);
|
|
304
|
+
if (clean.broke) {
|
|
305
|
+
problems.push(`${invariant.id}: fails against the unmutated emitter `
|
|
306
|
+
+ `(${(clean.failed[0] && `:${clean.failed[0].site} ${clean.failed[0].why}`) || clean.threw})`);
|
|
307
|
+
}
|
|
308
|
+
if (!clean.sites.length) {
|
|
309
|
+
problems.push(`${invariant.id}: recorded no assertion — its run() does not take the `
|
|
310
|
+
+ '`assert` the runner passes it, so nothing about it is measured');
|
|
311
|
+
}
|
|
312
|
+
const broken = new Map(clean.sites.map((s) => [s.site, []]));
|
|
313
|
+
|
|
202
314
|
const broke = [];
|
|
203
315
|
for (const rule of RULES) {
|
|
204
|
-
|
|
316
|
+
const r = await runOne(invariant, [rule]);
|
|
317
|
+
if (r.broke) broke.push(rule);
|
|
318
|
+
for (const s of r.failed) if (broken.has(s.site)) broken.get(s.site).push(rule);
|
|
205
319
|
}
|
|
206
320
|
matrix.push({ id: invariant.id, broke });
|
|
321
|
+
|
|
322
|
+
for (const site of clean.sites) {
|
|
323
|
+
coverage.push({
|
|
324
|
+
invariant: invariant.id,
|
|
325
|
+
site: site.site,
|
|
326
|
+
unmutated: site.unmutated,
|
|
327
|
+
by: broken.get(site.site) || [],
|
|
328
|
+
});
|
|
329
|
+
}
|
|
330
|
+
|
|
207
331
|
const expected = [...invariant.breaks].sort().join(', ');
|
|
208
332
|
const measured = [...broke].sort().join(', ');
|
|
209
333
|
if (expected !== measured) {
|
|
@@ -215,12 +339,33 @@ async function selfTest() {
|
|
|
215
339
|
}
|
|
216
340
|
|
|
217
341
|
const width = Math.max(...matrix.map((row) => row.id.length));
|
|
218
|
-
console.log(`\
|
|
219
|
-
+ 'and which removed rule turns each
|
|
342
|
+
console.log(`\ninvariant x mutant — ${RULES.length} mutants, `
|
|
343
|
+
+ 'and which removed rule turns each invariant red\n');
|
|
220
344
|
for (const row of matrix) {
|
|
221
345
|
console.log(` ${row.id.padEnd(width)} ${row.broke.join(', ') || '(nothing)'}`);
|
|
222
346
|
}
|
|
223
347
|
|
|
348
|
+
// The per-ASSERTION half, and the one the table above cannot give: an invariant goes red
|
|
349
|
+
// as a unit, so a dead assertion inside a live one is invisible there.
|
|
350
|
+
console.log('\nassertion x mutant — every call site, and what turns IT red\n');
|
|
351
|
+
for (const row of coverage) {
|
|
352
|
+
const label = `${row.invariant}:${row.site}`.padEnd(width + 6);
|
|
353
|
+
if (row.unmutated) {
|
|
354
|
+
console.log(` ${label} unmutated — no rule varies what it checks`);
|
|
355
|
+
if (row.by.length) {
|
|
356
|
+
problems.push(`${row.invariant} :${row.site} is declared assert.unmutated and `
|
|
357
|
+
+ `[${row.by.join(', ')}] breaks it — it discriminates, so drop the .unmutated`);
|
|
358
|
+
}
|
|
359
|
+
} else {
|
|
360
|
+
console.log(` ${label} ${row.by.join(', ') || '(nothing)'}`);
|
|
361
|
+
if (!row.by.length) {
|
|
362
|
+
problems.push(`${row.invariant} :${row.site} is broken by no mutant — the assertion `
|
|
363
|
+
+ 'cannot fail. Either it is not testing anything, or no rule varies what it '
|
|
364
|
+
+ 'checks and it belongs on assert.unmutated, which the verdict counts apart');
|
|
365
|
+
}
|
|
366
|
+
}
|
|
367
|
+
}
|
|
368
|
+
|
|
224
369
|
console.log('\nisolation — the fixture that separates each rule from every other\n');
|
|
225
370
|
for (const rule of RULES) {
|
|
226
371
|
const isolating = matrix.filter((row) => row.broke.length === 1 && row.broke[0] === rule);
|
|
@@ -237,7 +382,10 @@ async function selfTest() {
|
|
|
237
382
|
for (const p of problems) console.error(` - ${p}`);
|
|
238
383
|
return 1;
|
|
239
384
|
}
|
|
240
|
-
|
|
385
|
+
const pinned = coverage.filter((r) => r.unmutated).length;
|
|
386
|
+
console.log(`\nOK: ${INVARIANTS.length} invariants over ${coverage.length} assertions — `
|
|
387
|
+
+ `${coverage.length - pinned} watched failing ONE CALL SITE AT A TIME against `
|
|
388
|
+
+ `${RULES.length} mutants; ${pinned} declared unmutated and measured unbreakable; `
|
|
241
389
|
+ `${RULES.length} rules, each isolated by a fixture`);
|
|
242
390
|
return 0;
|
|
243
391
|
}
|
|
@@ -102,8 +102,12 @@ by accident, because the wrapper accepts whatever you hand it and hashes it
|
|
|
102
102
|
without judgement. SHA-256 does **not** make a value safe to send: hashing is a
|
|
103
103
|
matching mechanism, not a permission. That the wrapper receives a hash at all is checkable:
|
|
104
104
|
`identifiers-reach-the-server-hashed` in `fixtures/assert-dedup-contract.mjs` requires every
|
|
105
|
-
`user_data.em` entry to be 64 hex characters and refuses anything with an `@` in it.
|
|
106
|
-
|
|
105
|
+
`user_data.em` entry to be 64 hex characters and refuses anything with an `@` in it. **Both
|
|
106
|
+
halves are separately watched failing**, which they were not until 2026-08-20: the pack's
|
|
107
|
+
`--self-test` measured one row per invariant, so the `@` guard could be neutered while the
|
|
108
|
+
hex-shape assertion above it kept the invariant red, and this paragraph advertised a control
|
|
109
|
+
nothing had ever seen fail. The matrix is now per call site. What may be hashed in the first
|
|
110
|
+
place is the table below, and no assertion decides it.
|
|
107
111
|
|
|
108
112
|
Meta's Business Tools Terms prohibit sharing data that "includes or is based on"
|
|
109
113
|
any of these, and the prohibition covers the Pixel, the Conversions API, the
|
|
@@ -39,15 +39,13 @@ and anything else that issues an invoice and calls you back.
|
|
|
39
39
|
- [Status mapping — and finality](#status-mapping--and-finality)
|
|
40
40
|
- [Webhook signature verification](#webhook-signature-verification)
|
|
41
41
|
- [Idempotent webhook processing](#idempotent-webhook-processing)
|
|
42
|
-
- [
|
|
43
|
-
- [CSRF exemption for callback routes](#csrf-exemption-for-callback-routes)
|
|
42
|
+
- [Callback route hardening](#callback-route-hardening)
|
|
44
43
|
- [The conversion buffer](#the-conversion-buffer)
|
|
45
44
|
- [Reconciliation fields](#reconciliation-fields)
|
|
46
45
|
- [Crediting: the amount waterfall](#crediting-the-amount-waterfall)
|
|
47
46
|
- [Refunds and AML holds](#refunds-and-aml-holds)
|
|
48
47
|
- [The test/live credential boundary](#the-testlive-credential-boundary)
|
|
49
|
-
- [Local development](#local-development)
|
|
50
|
-
- [Test matrix](#test-matrix)
|
|
48
|
+
- [Local development and the test matrix](#local-development-and-the-test-matrix)
|
|
51
49
|
- [Security checklist](#security-checklist)
|
|
52
50
|
- [Common pitfalls](#common-pitfalls)
|
|
53
51
|
|
|
@@ -192,46 +190,17 @@ the read.
|
|
|
192
190
|
|
|
193
191
|
---
|
|
194
192
|
|
|
195
|
-
##
|
|
193
|
+
## Callback route hardening
|
|
196
194
|
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
195
|
+
Two things about the *request* rather than about the money in it, and each one is the
|
|
196
|
+
usual cause of "webhooks stopped working after we moved to a load balancer": reading the
|
|
197
|
+
caller IP from the proxy hop you actually own instead of `x-forwarded-for[0]`, and
|
|
198
|
+
exempting the **one** callback path from CSRF by exact match rather than by prefix — a
|
|
199
|
+
prefix on `/api/payments` exempts the checkout endpoint too, which is where the money is.
|
|
200
200
|
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
const xff = req.headers['x-forwarded-for'];
|
|
205
|
-
if (typeof xff === 'string') {
|
|
206
|
-
const hops = xff.split(',').map((s) => s.trim());
|
|
207
|
-
return hops[hops.length - TRUSTED_PROXY_COUNT] ?? null;
|
|
208
|
-
}
|
|
209
|
-
return req.socket.remoteAddress ?? null;
|
|
210
|
-
}
|
|
211
|
-
```
|
|
212
|
-
|
|
213
|
-
Taking `xff[0]` trusts a header the client controls — anyone can claim to be the
|
|
214
|
-
gateway. Count from the right by the number of proxies you actually run, and
|
|
215
|
-
keep that number in configuration, because it changes when infrastructure does.
|
|
216
|
-
|
|
217
|
-
Allowlist the provider's published callback IPs, log a rejection with the
|
|
218
|
-
observed IP, and **never** fall back to "allow if the header is missing".
|
|
219
|
-
|
|
220
|
-
---
|
|
221
|
-
|
|
222
|
-
## CSRF exemption for callback routes
|
|
223
|
-
|
|
224
|
-
A gateway cannot present your CSRF token. Exempt the callback path explicitly
|
|
225
|
-
and narrowly:
|
|
226
|
-
|
|
227
|
-
```ts
|
|
228
|
-
// middleware.ts (Next.js)
|
|
229
|
-
export const config = { matcher: ['/((?!api/payments/webhook).*)'] };
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
Exempt the **one** path, by exact match, and make it the only route in your app
|
|
233
|
-
that skips CSRF. A prefix match on `/api/payments` exempts the checkout endpoint
|
|
234
|
-
too, which is where the money is.
|
|
201
|
+
Both, with the code:
|
|
202
|
+
[`references/callback-route-hardening.md`](references/callback-route-hardening.md).
|
|
203
|
+
Signature verification above is the real gate; these are defence in depth.
|
|
235
204
|
|
|
236
205
|
---
|
|
237
206
|
|
|
@@ -374,45 +343,13 @@ boundary* and *Residual exposure*.
|
|
|
374
343
|
|
|
375
344
|
---
|
|
376
345
|
|
|
377
|
-
## Local development
|
|
378
|
-
|
|
379
|
-
Two paths, and you want both:
|
|
380
|
-
|
|
381
|
-
**Skip the gateway.** A `SKIP_BILLING=true` branch that credits immediately and
|
|
382
|
-
logs loudly. Everyone building a feature that merely *touches* checkout should
|
|
383
|
-
use this. Make it impossible in production: assert `NODE_ENV !== 'production'`
|
|
384
|
-
at the branch, not just in config.
|
|
385
|
-
|
|
386
|
-
**Real gateway, tunnelled callback.** Callbacks cannot reach `localhost`:
|
|
387
|
-
|
|
388
|
-
```bash
|
|
389
|
-
cloudflared tunnel --url http://localhost:3000
|
|
390
|
-
# → https://random-name.trycloudflare.com → set as the callback base URL, restart the app
|
|
391
|
-
```
|
|
392
|
-
|
|
393
|
-
Then keep a **signed mock callback** in your test tooling — a script that builds
|
|
394
|
-
a real payload, signs it with the dev key and POSTs it. This is what makes the
|
|
395
|
-
sad paths (`wrong_amount`, `refund_fail`, `locked`) testable at all; you cannot
|
|
396
|
-
ask a gateway to underpay you on demand.
|
|
397
|
-
|
|
398
|
-
---
|
|
399
|
-
|
|
400
|
-
## Test matrix
|
|
401
|
-
|
|
402
|
-
Cover these, and plant a defect against each before believing the green:
|
|
346
|
+
## Local development and the test matrix
|
|
403
347
|
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
| duplicate delivery of a settling status | idempotency guard |
|
|
410
|
-
| `paid_over` | crediting the received amount, not the invoiced |
|
|
411
|
-
| `wrong_amount` | no full credit on a partial payment |
|
|
412
|
-
| out-of-order delivery (final, then earlier) | the `notIn` guard, not timestamps |
|
|
413
|
-
| callback from a non-allowlisted IP | proxy-aware extraction |
|
|
414
|
-
| `refund_process` then `refund_fail` | refund is a state machine |
|
|
415
|
-
| `locked` | no credit, no auto-cancel |
|
|
348
|
+
Two development paths you want both of — a `SKIP_BILLING=true` branch that is impossible
|
|
349
|
+
in production, and a real gateway behind a tunnel with a signed mock callback for the sad
|
|
350
|
+
paths a provider will not produce on demand — plus the ten cases to cover before believing
|
|
351
|
+
a green suite, each with the defect it catches:
|
|
352
|
+
[`references/testing-and-local-dev.md`](references/testing-and-local-dev.md).
|
|
416
353
|
|
|
417
354
|
---
|
|
418
355
|
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Callback route hardening — the proxy hop and the CSRF hole
|
|
2
|
+
|
|
3
|
+
Split out of `SKILL.md` on 2026-08-20, when its body measured ~4894 tokens against the
|
|
4
|
+
house working limit of 4750. Both sections below are web plumbing rather than payment
|
|
5
|
+
logic — they are about the request that carries a callback, not about the money in it —
|
|
6
|
+
which is why they are the seam. Signature verification (`SKILL.md`, *Webhook signature
|
|
7
|
+
verification*) is the real gate; these two are defence in depth, and each has a failure
|
|
8
|
+
mode that looks like "webhooks stopped working" rather than like a security event.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## IP allowlisting behind a proxy
|
|
13
|
+
|
|
14
|
+
Signature verification is the real gate; the allowlist is defence in depth and
|
|
15
|
+
cheap. It is also the single most common cause of "webhooks stopped working
|
|
16
|
+
after we moved to a load balancer".
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
function callerIp(req): string | null {
|
|
20
|
+
// Trust ONLY the hop your own infrastructure appends.
|
|
21
|
+
const xff = req.headers['x-forwarded-for'];
|
|
22
|
+
if (typeof xff === 'string') {
|
|
23
|
+
const hops = xff.split(',').map((s) => s.trim());
|
|
24
|
+
return hops[hops.length - TRUSTED_PROXY_COUNT] ?? null;
|
|
25
|
+
}
|
|
26
|
+
return req.socket.remoteAddress ?? null;
|
|
27
|
+
}
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Taking `xff[0]` trusts a header the client controls — anyone can claim to be the
|
|
31
|
+
gateway. Count from the right by the number of proxies you actually run, and
|
|
32
|
+
keep that number in configuration, because it changes when infrastructure does.
|
|
33
|
+
|
|
34
|
+
Allowlist the provider's published callback IPs, log a rejection with the
|
|
35
|
+
observed IP, and **never** fall back to "allow if the header is missing".
|
|
36
|
+
|
|
37
|
+
## CSRF exemption for callback routes
|
|
38
|
+
|
|
39
|
+
A gateway cannot present your CSRF token. Exempt the callback path explicitly
|
|
40
|
+
and narrowly:
|
|
41
|
+
|
|
42
|
+
```ts
|
|
43
|
+
// middleware.ts (Next.js)
|
|
44
|
+
export const config = { matcher: ['/((?!api/payments/webhook).*)'] };
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Exempt the **one** path, by exact match, and make it the only route in your app
|
|
48
|
+
that skips CSRF. A prefix match on `/api/payments` exempts the checkout endpoint
|
|
49
|
+
too, which is where the money is.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
Back to [`SKILL.md`](../SKILL.md) — *Webhook signature verification* is the gate these two
|
|
54
|
+
sit behind, and *Test matrix* in
|
|
55
|
+
[`testing-and-local-dev.md`](testing-and-local-dev.md) carries the non-allowlisted-IP case.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Testing and local development
|
|
2
|
+
|
|
3
|
+
Split out of `SKILL.md` on 2026-08-20 for the same reason as
|
|
4
|
+
[`callback-route-hardening.md`](callback-route-hardening.md): the body was ~4894 tokens
|
|
5
|
+
against a 4750 working limit, and the house answer at that point is a split rather than a
|
|
6
|
+
trim. This is the operational half — how you run the thing on a laptop, and what you have
|
|
7
|
+
to cover before believing a green suite.
|
|
8
|
+
|
|
9
|
+
`stripe-billing` ships the same pair of subjects at
|
|
10
|
+
[`../../stripe-billing/references/testing-and-local-dev.md`](../../stripe-billing/references/testing-and-local-dev.md),
|
|
11
|
+
and its *Mutation testing* section is the argument this matrix assumes: plant a defect
|
|
12
|
+
against every row before believing it.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Local development
|
|
17
|
+
|
|
18
|
+
Two paths, and you want both:
|
|
19
|
+
|
|
20
|
+
**Skip the gateway.** A `SKIP_BILLING=true` branch that credits immediately and
|
|
21
|
+
logs loudly. Everyone building a feature that merely *touches* checkout should
|
|
22
|
+
use this. Make it impossible in production: assert `NODE_ENV !== 'production'`
|
|
23
|
+
at the branch, not just in config.
|
|
24
|
+
|
|
25
|
+
**Real gateway, tunnelled callback.** Callbacks cannot reach `localhost`:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
cloudflared tunnel --url http://localhost:3000
|
|
29
|
+
# → https://random-name.trycloudflare.com → set as the callback base URL, restart the app
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Then keep a **signed mock callback** in your test tooling — a script that builds
|
|
33
|
+
a real payload, signs it with the dev key and POSTs it. This is what makes the
|
|
34
|
+
sad paths (`wrong_amount`, `refund_fail`, `locked`) testable at all; you cannot
|
|
35
|
+
ask a gateway to underpay you on demand.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## Test matrix
|
|
40
|
+
|
|
41
|
+
Cover these, and plant a defect against each before believing the green:
|
|
42
|
+
|
|
43
|
+
| Case | What it catches |
|
|
44
|
+
|---|---|
|
|
45
|
+
| valid signature | the happy path |
|
|
46
|
+
| tampered signature | constant-time compare, 403 not 500 |
|
|
47
|
+
| signature over a payload containing a URL | the escaping trap |
|
|
48
|
+
| duplicate delivery of a settling status | idempotency guard |
|
|
49
|
+
| `paid_over` | crediting the received amount, not the invoiced |
|
|
50
|
+
| `wrong_amount` | no full credit on a partial payment |
|
|
51
|
+
| out-of-order delivery (final, then earlier) | the `notIn` guard, not timestamps |
|
|
52
|
+
| callback from a non-allowlisted IP | proxy-aware extraction |
|
|
53
|
+
| `refund_process` then `refund_fail` | refund is a state machine |
|
|
54
|
+
| `locked` | no credit, no auto-cancel |
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
Back to [`SKILL.md`](../SKILL.md). The provider-specific half — one credential, no sandbox
|
|
59
|
+
host, and what "test mode" actually toggles — is
|
|
60
|
+
[`heleket-provider.md`](heleket-provider.md), *The test/live boundary*.
|
|
@@ -2,14 +2,15 @@
|
|
|
2
2
|
|
|
3
3
|
**Copy this whole directory into your repository.** It is the executable half of what
|
|
4
4
|
`SKILL.md` and `references/webhook-events.md` state in prose: nine real Stripe event
|
|
5
|
-
bodies, twelve
|
|
6
|
-
breaks the handler one rule at a time so you can watch
|
|
5
|
+
bodies, twelve invariants a correct webhook handler must hold — 45 assertions between
|
|
6
|
+
them — and a self-test that breaks the handler one rule at a time so you can watch each
|
|
7
|
+
assertion fail on its own.
|
|
7
8
|
|
|
8
9
|
Two commands, no dependencies, no network, no key:
|
|
9
10
|
|
|
10
11
|
```bash
|
|
11
|
-
node assert-money-invariants.mjs # 12
|
|
12
|
-
node assert-money-invariants.mjs --self-test # break one rule at a time;
|
|
12
|
+
node assert-money-invariants.mjs # 12 invariants / 45 assertions
|
|
13
|
+
node assert-money-invariants.mjs --self-test # break one rule at a time; EACH ASSERTION must go red
|
|
13
14
|
```
|
|
14
15
|
|
|
15
16
|
Point it at **your** handler by replacing the two imports at the top of
|
|
@@ -72,14 +72,20 @@ export const INVARIANTS = [
|
|
|
72
72
|
fixtures: [CYCLE_JAN],
|
|
73
73
|
states: 'a paid renewal becomes exactly one grant, and the mirror carries its period',
|
|
74
74
|
breaks: ['grant-on-renewal'],
|
|
75
|
-
async run({ store, handler }) {
|
|
75
|
+
async run({ store, handler, assert }) {
|
|
76
76
|
const event = fixture(CYCLE_JAN);
|
|
77
77
|
const response = await handler.deliver(event);
|
|
78
|
-
|
|
79
|
-
|
|
78
|
+
// `unmutated`: no rule in RULES makes this route answer anything but 200 with that
|
|
79
|
+
// body, so neither assertion has ever been watched failing. They pin the envelope a
|
|
80
|
+
// reader copies; they are not evidence, and the verdict line counts them apart.
|
|
81
|
+
assert.unmutated.equal(response.status, 200);
|
|
82
|
+
assert.unmutated.deepEqual(response.body, { received: true });
|
|
80
83
|
assert.equal(store.grants.length, 1, 'a paid renewal granted nothing');
|
|
81
84
|
assert.equal(store.credits.get('usr_PLACEHOLDER_alice'), CREDITS_PER_PERIOD);
|
|
82
|
-
|
|
85
|
+
// `|| {}` on purpose: with no mirror row at all these two assertions must FAIL
|
|
86
|
+
// rather than throw a TypeError on the line above them. A throw aborts the run, and
|
|
87
|
+
// an assertion the runner never reached is one no mutant has been watched breaking.
|
|
88
|
+
const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice') || {};
|
|
83
89
|
assert.equal(mirror.periodStart, 1767225600);
|
|
84
90
|
assert.equal(mirror.periodEnd, 1769904000, 'the period is not the line period');
|
|
85
91
|
},
|
|
@@ -89,10 +95,11 @@ export const INVARIANTS = [
|
|
|
89
95
|
fixtures: [CYCLE_JAN, PRORATION],
|
|
90
96
|
states: 'billing_reason subscription_update is a $40 proration, not a month of product',
|
|
91
97
|
breaks: ['billing-reason', 'grant-on-renewal'],
|
|
92
|
-
async run({ store, handler }) {
|
|
98
|
+
async run({ store, handler, assert }) {
|
|
93
99
|
await handler.deliver(fixture(CYCLE_JAN));
|
|
94
100
|
const response = await handler.deliver(fixture(PRORATION));
|
|
95
|
-
|
|
101
|
+
// unmutated: nothing in RULES makes the endpoint answer 500 for a proration.
|
|
102
|
+
assert.unmutated.equal(response.status, 200, 'a proration invoice must not fail the endpoint');
|
|
96
103
|
assert.equal(store.grants.length, 1,
|
|
97
104
|
'the proration granted a second period — a user who adds and removes a seat '
|
|
98
105
|
+ 'four times has been given four months of product');
|
|
@@ -107,10 +114,10 @@ export const INVARIANTS = [
|
|
|
107
114
|
fixtures: [CYCLE_JAN, CYCLE_JAN_REDELIVERY],
|
|
108
115
|
states: 'the same evt_ delivered twice grants once AND runs its side effects once',
|
|
109
116
|
breaks: ['claim', 'grant-on-renewal'],
|
|
110
|
-
async run({ store, handler }) {
|
|
117
|
+
async run({ store, handler, assert }) {
|
|
111
118
|
const first = await handler.deliver(fixture(CYCLE_JAN));
|
|
112
119
|
const second = await handler.deliver(fixture(CYCLE_JAN_REDELIVERY));
|
|
113
|
-
assert.deepEqual(first.body, { received: true });
|
|
120
|
+
assert.unmutated.deepEqual(first.body, { received: true }); // the envelope, again
|
|
114
121
|
assert.deepEqual(second.body, { received: true, duplicate: true },
|
|
115
122
|
'the second delivery was not recognised as a duplicate');
|
|
116
123
|
assert.equal(store.grants.length, 1, 'the redelivery granted a second time');
|
|
@@ -124,7 +131,7 @@ export const INVARIANTS = [
|
|
|
124
131
|
fixtures: [CYCLE_JAN, CYCLE_JAN_REDELIVERY],
|
|
125
132
|
states: 'two deliveries in flight at once grant once — the claim is an INSERT, not a SELECT',
|
|
126
133
|
breaks: ['claim', 'grant-on-renewal'],
|
|
127
|
-
async run({ store, handler }) {
|
|
134
|
+
async run({ store, handler, assert }) {
|
|
128
135
|
// 40 ms apart in production; here, both before either has finished its first read.
|
|
129
136
|
// This is the fixture that separates the event claim from the grant marker: the
|
|
130
137
|
// marker reads before it writes, and a read is a round trip.
|
|
@@ -146,7 +153,7 @@ export const INVARIANTS = [
|
|
|
146
153
|
fixtures: [CYCLE_JAN],
|
|
147
154
|
states: 'the nightly repair reuses the webhook marker — a nightly job is not a nightly gift',
|
|
148
155
|
breaks: ['grant-marker', 'grant-on-renewal'],
|
|
149
|
-
async run({ store, handler }) {
|
|
156
|
+
async run({ store, handler, assert }) {
|
|
150
157
|
const event = fixture(CYCLE_JAN);
|
|
151
158
|
await handler.deliver(event);
|
|
152
159
|
const subId = subscriptionIdOf(event.data.object);
|
|
@@ -163,11 +170,11 @@ export const INVARIANTS = [
|
|
|
163
170
|
fixtures: [CYCLE_FEB, CYCLE_JAN],
|
|
164
171
|
states: 'February delivered before January leaves February in the mirror, and grants both',
|
|
165
172
|
breaks: ['ordering', 'grant-on-renewal'],
|
|
166
|
-
async run({ store, handler }) {
|
|
173
|
+
async run({ store, handler, assert }) {
|
|
167
174
|
await handler.deliver(fixture(CYCLE_FEB)); // arrives first
|
|
168
175
|
await handler.deliver(fixture(CYCLE_JAN)); // the earlier period, delivered late
|
|
169
176
|
assert.equal(store.grants.length, 2, 'both periods were paid, so both are owed');
|
|
170
|
-
const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice');
|
|
177
|
+
const mirror = store.subscriptions.get('sub_PLACEHOLDER_alice') || {};
|
|
171
178
|
assert.equal(mirror.periodStart, 1769904000,
|
|
172
179
|
'the mirror was rewound to the older period — state derived from arrival order');
|
|
173
180
|
assert.equal(mirror.periodEnd, 1772323200,
|
|
@@ -179,7 +186,7 @@ export const INVARIANTS = [
|
|
|
179
186
|
fixtures: [REFUND_PARTIAL, REFUND_REMAINDER],
|
|
180
187
|
states: 'amount_refunded is the total so far: two partials claw back $90, not $130',
|
|
181
188
|
breaks: ['cumulative-refund'],
|
|
182
|
-
async run({ store, handler }) {
|
|
189
|
+
async run({ store, handler, assert }) {
|
|
183
190
|
seedOneOffPurchase(store);
|
|
184
191
|
await handler.deliver(fixture(REFUND_PARTIAL));
|
|
185
192
|
await handler.deliver(fixture(REFUND_REMAINDER));
|
|
@@ -201,7 +208,7 @@ export const INVARIANTS = [
|
|
|
201
208
|
// and the claim still refuses the second delivery. A suite that judged a redelivery by
|
|
202
209
|
// its grant count alone would ship with no idempotency and stay green.
|
|
203
210
|
breaks: [['claim', 'grant-marker'], 'grant-on-renewal'],
|
|
204
|
-
async run({ store, handler }) {
|
|
211
|
+
async run({ store, handler, assert }) {
|
|
205
212
|
await handler.deliver(fixture(CYCLE_JAN));
|
|
206
213
|
await handler.deliver(fixture(CYCLE_JAN_REDELIVERY));
|
|
207
214
|
assert.equal(store.grants.length, 1, 'the redelivery granted a second time');
|
|
@@ -216,7 +223,7 @@ export const INVARIANTS = [
|
|
|
216
223
|
// `increment <= 0` catches a sequential replay by itself, and the claim catches it
|
|
217
224
|
// without the swap. Neither is tested by this fixture alone.
|
|
218
225
|
breaks: [['claim', 'cumulative-refund']],
|
|
219
|
-
async run({ store, handler }) {
|
|
226
|
+
async run({ store, handler, assert }) {
|
|
220
227
|
seedOneOffPurchase(store);
|
|
221
228
|
await handler.deliver(fixture(REFUND_PARTIAL));
|
|
222
229
|
await handler.deliver(fixture(REFUND_PARTIAL));
|
|
@@ -229,16 +236,20 @@ export const INVARIANTS = [
|
|
|
229
236
|
fixtures: [SESSION_UNPAID, SESSION_FAILED],
|
|
230
237
|
states: 'the redirect proves a browser: an unpaid async session grants and converts nothing',
|
|
231
238
|
breaks: ['async-gate'],
|
|
232
|
-
async run({ store, handler }) {
|
|
239
|
+
async run({ store, handler, assert }) {
|
|
233
240
|
const completed = await handler.deliver(fixture(SESSION_UNPAID));
|
|
234
|
-
|
|
241
|
+
// unmutated: the envelope. Removing `async-gate` grants the session and still answers 200.
|
|
242
|
+
assert.unmutated.equal(completed.status, 200, 'an unpaid session must still answer 200');
|
|
235
243
|
assert.equal(store.grants.length, 0,
|
|
236
244
|
'a session whose payment_status is unpaid was granted — the money has not moved');
|
|
237
|
-
|
|
245
|
+
// unmutated, and the reason is a gap worth seeing: this handler emits conversions on
|
|
246
|
+
// `invoice.paid` only, so no rule here can make a checkout session report one. The
|
|
247
|
+
// assertion states the rule for a reader whose handler DOES emit from a session.
|
|
248
|
+
assert.unmutated.equal(store.conversions.length, 0,
|
|
238
249
|
'a purchase conversion was reported for a charge that has not cleared');
|
|
239
250
|
await handler.deliver(fixture(SESSION_FAILED));
|
|
240
251
|
assert.equal(store.grants.length, 0, 'the failed async payment still granted');
|
|
241
|
-
assert.equal(store.conversions.length, 0);
|
|
252
|
+
assert.unmutated.equal(store.conversions.length, 0); // same gap as above
|
|
242
253
|
assert.equal(store.credits.get('usr_PLACEHOLDER_bob'), undefined);
|
|
243
254
|
},
|
|
244
255
|
},
|
|
@@ -249,9 +260,9 @@ export const INVARIANTS = [
|
|
|
249
260
|
// Without it, a handler that refuses every checkout session satisfies the assertion
|
|
250
261
|
// above and this pack would have taught the opposite defect.
|
|
251
262
|
breaks: ['grant-on-checkout'],
|
|
252
|
-
async run({ store, handler }) {
|
|
263
|
+
async run({ store, handler, assert }) {
|
|
253
264
|
const response = await handler.deliver(fixture(SESSION_PAID));
|
|
254
|
-
assert.deepEqual(response.body, { received: true });
|
|
265
|
+
assert.unmutated.deepEqual(response.body, { received: true }); // the envelope
|
|
255
266
|
assert.equal(store.grants.length, 1, 'a cleared card session granted nothing');
|
|
256
267
|
assert.equal(store.credits.get('usr_PLACEHOLDER_carol'), CREDITS_PER_PERIOD);
|
|
257
268
|
assert.ok(store.subscriptions.has('sub_PLACEHOLDER_carol'));
|
|
@@ -262,10 +273,10 @@ export const INVARIANTS = [
|
|
|
262
273
|
fixtures: [CYCLE_JAN],
|
|
263
274
|
states: 'the conversion id comes from subscription metadata, so browser and server share it',
|
|
264
275
|
breaks: ['conversion-id-from-metadata', 'grant-on-renewal'],
|
|
265
|
-
async run({ store, handler }) {
|
|
276
|
+
async run({ store, handler, assert }) {
|
|
266
277
|
await handler.deliver(fixture(CYCLE_JAN));
|
|
267
278
|
assert.equal(store.conversions.length, 1);
|
|
268
|
-
const [conversion] = store.conversions;
|
|
279
|
+
const [conversion = {}] = store.conversions;
|
|
269
280
|
assert.equal(conversion.eventId, 'evtid_PLACEHOLDER_purchase_jan',
|
|
270
281
|
'the server generated its own id — nothing will ever deduplicate against the pixel');
|
|
271
282
|
assert.equal(conversion.eventName, 'Purchase', 'the event name is not byte-equal');
|
|
@@ -276,34 +287,131 @@ export const INVARIANTS = [
|
|
|
276
287
|
},
|
|
277
288
|
];
|
|
278
289
|
|
|
290
|
+
// ------------------------------------------------- one assertion, one measurement
|
|
291
|
+
//
|
|
292
|
+
// Why the assertions are not called straight through `node:assert`. Until 2026-08-20 the
|
|
293
|
+
// runner returned on the FIRST throw and the matrix compared one row per invariant, so an
|
|
294
|
+
// invariant was measured as a single unit: any assertion that was not the sole
|
|
295
|
+
// discriminator could be replaced by `assert.ok(true)` and nothing went red. Measured that
|
|
296
|
+
// day — three neutered assertions (the `clawbacks.map` deepEqual, the `mirror.periodEnd`
|
|
297
|
+
// equality, and the `response.body` deepEqual) left `--self-test` at exit 0 still printing
|
|
298
|
+
// *"12 assertions, each watched failing"*, and `npm test` at 0 with them.
|
|
299
|
+
//
|
|
300
|
+
// So the `assert` an invariant receives is a RECORDER: same API, but a failure is
|
|
301
|
+
// remembered instead of thrown and the assertions after it still run. Each call site is one
|
|
302
|
+
// assertion with its own identity — its line in this file — and `--self-test` asks the
|
|
303
|
+
// question per assertion rather than per invariant.
|
|
304
|
+
//
|
|
305
|
+
// Two kinds, and the difference is declared at the call site rather than in a table:
|
|
306
|
+
//
|
|
307
|
+
// assert.equal(...) a DISCRIMINATING assertion. At least one mutant must turn it
|
|
308
|
+
// red, or it is not evidence of anything.
|
|
309
|
+
// assert.unmutated.equal() an assertion NO mutant in this pack varies. It pins the
|
|
310
|
+
// envelope or a field the rules do not touch, so it has never
|
|
311
|
+
// been watched failing and the verdict line counts it apart
|
|
312
|
+
// rather than as evidence. Measured, not asserted: the self-test
|
|
313
|
+
// fails if a mutant does break one, because that is a
|
|
314
|
+
// discriminator hiding in the wrong list.
|
|
315
|
+
//
|
|
316
|
+
// A reader pointing this at their own handler keeps writing `assert.equal`; the recorder is
|
|
317
|
+
// API-compatible with `node:assert/strict` for the methods below.
|
|
318
|
+
|
|
319
|
+
const SELF = import.meta.url;
|
|
320
|
+
const METHODS = ['equal', 'notEqual', 'deepEqual', 'notDeepEqual', 'ok', 'match', 'doesNotMatch'];
|
|
321
|
+
|
|
322
|
+
/**
|
|
323
|
+
* The line in THIS file that called the recorder.
|
|
324
|
+
*
|
|
325
|
+
* `captureStackTrace(holder, boundary)` drops `boundary` and everything above it, so the
|
|
326
|
+
* top frame is the assertion's own line — no frame counting, which is what makes the
|
|
327
|
+
* identity survive a refactor of the recorder itself.
|
|
328
|
+
*/
|
|
329
|
+
function siteOf(boundary) {
|
|
330
|
+
const holder = {};
|
|
331
|
+
Error.captureStackTrace(holder, boundary);
|
|
332
|
+
const frame = (holder.stack || '').split('\n').slice(1).find((f) => f.includes(SELF));
|
|
333
|
+
const m = frame && frame.match(/:(\d+):\d+\)?\s*$/);
|
|
334
|
+
return m ? Number(m[1]) : 0;
|
|
335
|
+
}
|
|
336
|
+
|
|
337
|
+
function recorder() {
|
|
338
|
+
const seen = new Map(); // line -> { site, unmutated, ok, why }
|
|
339
|
+
|
|
340
|
+
function record(site, unmutated, name, args) {
|
|
341
|
+
let ok = true;
|
|
342
|
+
let why = null;
|
|
343
|
+
try {
|
|
344
|
+
assert[name](...args);
|
|
345
|
+
} catch (error) {
|
|
346
|
+
ok = false;
|
|
347
|
+
why = error.message.split('\n')[0];
|
|
348
|
+
}
|
|
349
|
+
const prev = seen.get(site);
|
|
350
|
+
if (prev) {
|
|
351
|
+
prev.ok = prev.ok && ok;
|
|
352
|
+
prev.why = prev.why || why;
|
|
353
|
+
} else {
|
|
354
|
+
seen.set(site, { site, unmutated, ok, why });
|
|
355
|
+
}
|
|
356
|
+
}
|
|
357
|
+
|
|
358
|
+
const bind = (unmutated) => {
|
|
359
|
+
const api = {};
|
|
360
|
+
for (const name of METHODS) {
|
|
361
|
+
const call = (...args) => record(siteOf(call), unmutated, name, args);
|
|
362
|
+
api[name] = call;
|
|
363
|
+
}
|
|
364
|
+
return api;
|
|
365
|
+
};
|
|
366
|
+
|
|
367
|
+
const api = bind(false);
|
|
368
|
+
api.unmutated = bind(true);
|
|
369
|
+
return { api, seen };
|
|
370
|
+
}
|
|
371
|
+
|
|
279
372
|
// ------------------------------------------------------------------------- runners
|
|
280
373
|
|
|
281
374
|
async function runOne(invariant, without) {
|
|
282
|
-
const
|
|
375
|
+
const { api, seen } = recorder();
|
|
376
|
+
const { store, handler } = harness(without);
|
|
377
|
+
let threw = null;
|
|
283
378
|
try {
|
|
284
|
-
await invariant.run(
|
|
285
|
-
return null;
|
|
379
|
+
await invariant.run({ store, handler, assert: api });
|
|
286
380
|
} catch (error) {
|
|
287
|
-
|
|
381
|
+
// A real error — a `TypeError` on a row the mutant never wrote, say. It aborts the
|
|
382
|
+
// remaining assertions, and it is still the invariant going red.
|
|
383
|
+
threw = error.message.split('\n')[0];
|
|
288
384
|
}
|
|
385
|
+
const sites = [...seen.values()];
|
|
386
|
+
return {
|
|
387
|
+
threw,
|
|
388
|
+
sites,
|
|
389
|
+
failed: sites.filter((s) => !s.ok),
|
|
390
|
+
broke: Boolean(threw) || sites.some((s) => !s.ok),
|
|
391
|
+
};
|
|
289
392
|
}
|
|
290
393
|
|
|
291
394
|
async function runAll() {
|
|
292
395
|
let failed = 0;
|
|
396
|
+
let asserted = 0;
|
|
293
397
|
for (const invariant of INVARIANTS) {
|
|
294
|
-
const
|
|
295
|
-
|
|
398
|
+
const result = await runOne(invariant, []);
|
|
399
|
+
asserted += result.sites.length;
|
|
400
|
+
if (result.broke) {
|
|
296
401
|
failed += 1;
|
|
297
|
-
console.error(`FAIL ${invariant.id}
|
|
402
|
+
console.error(`FAIL ${invariant.id}`);
|
|
403
|
+
for (const s of result.failed) console.error(` :${s.site} ${s.why}`);
|
|
404
|
+
if (result.threw) console.error(` threw: ${result.threw}`);
|
|
298
405
|
} else {
|
|
299
|
-
console.log(`pass ${invariant.id} — ${invariant.states}`);
|
|
406
|
+
console.log(`pass ${invariant.id} (${result.sites.length} assertions) — ${invariant.states}`);
|
|
300
407
|
}
|
|
301
408
|
}
|
|
302
409
|
if (failed) {
|
|
303
410
|
console.error(`\n${failed} of ${INVARIANTS.length} money invariants failed`);
|
|
304
411
|
return 1;
|
|
305
412
|
}
|
|
306
|
-
console.log(`\nOK: ${INVARIANTS.length} money invariants hold against the handler`
|
|
413
|
+
console.log(`\nOK: ${INVARIANTS.length} money invariants hold against the handler `
|
|
414
|
+
+ `(${asserted} assertions)`);
|
|
307
415
|
return 0;
|
|
308
416
|
}
|
|
309
417
|
|
|
@@ -333,21 +441,49 @@ async function selfTest() {
|
|
|
333
441
|
const mutants = [...RULES.map((rule) => [rule]), ...combos];
|
|
334
442
|
|
|
335
443
|
const matrix = [];
|
|
444
|
+
const coverage = []; // one row per ASSERTION: which mutants turn that call site red
|
|
336
445
|
for (const invariant of INVARIANTS) {
|
|
446
|
+
// The clean run is the census: it names every assertion this invariant executes, and
|
|
447
|
+
// whether each is declared discriminating or shape. An invariant that records nothing
|
|
448
|
+
// is one whose `run` never took the recorder — the failure mode of the injection.
|
|
449
|
+
const clean = await runOne(invariant, []);
|
|
450
|
+
if (clean.broke) {
|
|
451
|
+
problems.push(`${invariant.id}: fails against the unmutated handler `
|
|
452
|
+
+ `(${(clean.failed[0] && `:${clean.failed[0].site} ${clean.failed[0].why}`) || clean.threw})`);
|
|
453
|
+
}
|
|
454
|
+
if (!clean.sites.length) {
|
|
455
|
+
problems.push(`${invariant.id}: recorded no assertion — its run() does not take the `
|
|
456
|
+
+ '`assert` the runner passes it, so nothing about it is measured');
|
|
457
|
+
}
|
|
458
|
+
const broken = new Map(clean.sites.map((s) => [s.site, []]));
|
|
459
|
+
|
|
337
460
|
const singles = [];
|
|
338
461
|
for (const rule of RULES) {
|
|
339
|
-
|
|
462
|
+
const r = await runOne(invariant, [rule]);
|
|
463
|
+
if (r.broke) singles.push(rule);
|
|
464
|
+
for (const s of r.failed) if (broken.has(s.site)) broken.get(s.site).push(rule);
|
|
340
465
|
}
|
|
341
466
|
// Minimal mutants only: a combination containing a rule that already breaks the
|
|
342
467
|
// assertion on its own says nothing new.
|
|
343
468
|
const minimalCombos = [];
|
|
344
469
|
for (const combo of combos) {
|
|
345
470
|
if (combo.some((rule) => singles.includes(rule))) continue;
|
|
346
|
-
|
|
471
|
+
const r = await runOne(invariant, combo);
|
|
472
|
+
if (r.broke) minimalCombos.push(combo);
|
|
473
|
+
for (const s of r.failed) if (broken.has(s.site)) broken.get(s.site).push(key(combo));
|
|
347
474
|
}
|
|
348
475
|
const broke = [...singles.map((r) => [r]), ...minimalCombos];
|
|
349
476
|
matrix.push({ id: invariant.id, broke, singles });
|
|
350
477
|
|
|
478
|
+
for (const site of clean.sites) {
|
|
479
|
+
coverage.push({
|
|
480
|
+
invariant: invariant.id,
|
|
481
|
+
site: site.site,
|
|
482
|
+
unmutated: site.unmutated,
|
|
483
|
+
by: broken.get(site.site) || [],
|
|
484
|
+
});
|
|
485
|
+
}
|
|
486
|
+
|
|
351
487
|
const expected = invariant.breaks
|
|
352
488
|
.map((b) => key(Array.isArray(b) ? b : [b])).sort().join(', ');
|
|
353
489
|
const measured = broke.map(key).sort().join(', ');
|
|
@@ -368,6 +504,27 @@ async function selfTest() {
|
|
|
368
504
|
console.log(` ${row.id.padEnd(width)} ${row.broke.map(key).join(', ') || '(nothing)'}`);
|
|
369
505
|
}
|
|
370
506
|
|
|
507
|
+
// The per-ASSERTION half, and the one the per-invariant table above cannot give: an
|
|
508
|
+
// invariant goes red as a unit, so a dead assertion inside a live one is invisible there.
|
|
509
|
+
console.log('\nassertion x mutant — every call site, and what turns IT red\n');
|
|
510
|
+
for (const row of coverage) {
|
|
511
|
+
const label = `${row.invariant}:${row.site}`.padEnd(width + 6);
|
|
512
|
+
if (row.unmutated) {
|
|
513
|
+
console.log(` ${label} unmutated — no rule varies what it checks`);
|
|
514
|
+
if (row.by.length) {
|
|
515
|
+
problems.push(`${row.invariant} :${row.site} is declared assert.unmutated and `
|
|
516
|
+
+ `[${row.by.join(', ')}] breaks it — it discriminates, so drop the .unmutated`);
|
|
517
|
+
}
|
|
518
|
+
} else {
|
|
519
|
+
console.log(` ${label} ${row.by.join(', ') || '(nothing)'}`);
|
|
520
|
+
if (!row.by.length) {
|
|
521
|
+
problems.push(`${row.invariant} :${row.site} is broken by no mutant — the assertion `
|
|
522
|
+
+ 'cannot fail. Either it is not testing anything, or no rule varies what it '
|
|
523
|
+
+ 'checks and it belongs on assert.unmutated, which the verdict counts apart');
|
|
524
|
+
}
|
|
525
|
+
}
|
|
526
|
+
}
|
|
527
|
+
|
|
371
528
|
console.log('\nisolation — the fixture that separates each rule from every other\n');
|
|
372
529
|
for (const rule of RULES) {
|
|
373
530
|
const isolating = matrix.filter((row) => {
|
|
@@ -392,7 +549,10 @@ async function selfTest() {
|
|
|
392
549
|
for (const p of problems) console.error(` - ${p}`);
|
|
393
550
|
return 1;
|
|
394
551
|
}
|
|
395
|
-
|
|
552
|
+
const pinned = coverage.filter((r) => r.unmutated).length;
|
|
553
|
+
console.log(`\nOK: ${INVARIANTS.length} invariants over ${coverage.length} assertions — `
|
|
554
|
+
+ `${coverage.length - pinned} watched failing ONE CALL SITE AT A TIME against `
|
|
555
|
+
+ `${mutants.length} mutants; ${pinned} declared unmutated and measured unbreakable; `
|
|
396
556
|
+ `${RULES.length} rules, each isolated by a fixture`);
|
|
397
557
|
return 0;
|
|
398
558
|
}
|
|
@@ -158,12 +158,13 @@ the helper works and says nothing about the route Stripe posts to.
|
|
|
158
158
|
## The shipped fixtures and the assertion pack
|
|
159
159
|
|
|
160
160
|
Everything above tells you what to write. `fixtures/` is that suite already written:
|
|
161
|
-
nine Stripe event bodies, twelve assertions, and the mutants that prove
|
|
162
|
-
fail
|
|
161
|
+
nine Stripe event bodies, twelve invariants over 45 assertions, and the mutants that prove
|
|
162
|
+
each assertion can fail — per call site, not per invariant, which is a distinction this
|
|
163
|
+
pack got wrong until 2026-08-20. Copy the directory into your repository and run it.
|
|
163
164
|
|
|
164
165
|
```bash
|
|
165
166
|
node fixtures/assert-money-invariants.mjs # the assertions
|
|
166
|
-
node fixtures/assert-money-invariants.mjs --self-test #
|
|
167
|
+
node fixtures/assert-money-invariants.mjs --self-test # each ASSERTION, watched failing
|
|
167
168
|
```
|
|
168
169
|
|
|
169
170
|
Point it at your handler by replacing the two imports at the top of
|