@openwop/openwop-conformance 1.138.1 → 1.139.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -78,7 +78,7 @@ Run `npm run test` for normal CI cadence; `npm run test:strict` when claiming fu
78
78
  | `OPENWOP_OTEL_COLLECTOR_GRPC=true` | Boots the parallel OTLP/gRPC collector alongside the HTTP one (h2c HTTP/2 on a separate port). Shares the same `spans()` + `metrics()` store; spans captured over either transport surface in `getCollector().spans()`. Requires `OPENWOP_OTEL_COLLECTOR=true`. Same `--no-file-parallelism` requirement applies. |
79
79
  | `OPENWOP_OTEL_COLLECTOR_GRPC_PORT=4317` | Bind the OTLP/gRPC collector on a specific port (default `4317`, OTLP/gRPC convention). The host MUST be configured with `OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:<port>` AND `OTEL_EXPORTER_OTLP_PROTOCOL=grpc`. |
80
80
  | `OPENWOP_OTEL_COLLECTOR_PORT=14318` | Bind the OTel collector on a specific port (default `4318`). The host MUST be configured with `OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:<port>`. |
81
- | `OPENWOP_WEBHOOK_ALLOW_PRIVATE=true` | Hosts implementing the webhook SSRF guard (rejecting loopback / RFC1918 / link-local destinations) MUST advertise this flag for the loopback test receiver in `webhook-signed-delivery.test.ts` to be accepted. The SQLite reference host honors this env var; the scenario soft-skips when the host rejects the URL. |
81
+ | `OPENWOP_WEBHOOK_ALLOW_PRIVATE=true` | Relaxes the webhook egress guard for the loopback test receiver used by `webhook-signed-delivery.test.ts`, `webhook-negative.test.ts`, and `replay-fanout-suppression.test.ts`. **The receiver is `http://127.0.0.1:{port}/`, which `webhooks.md` forbids three separate times, and the opt-in MUST relax all three to be witnessable:** (1) the **scheme** check — §"SSRF protection" bullet 1 rejects non-`https://`, and this gate fires FIRST on a host that validates scheme before address; (2) the **registration-time** address check §"SSRF protection"; and (3) **delivery-time re-resolution** — §"Delivery-time egress validation (RFC 0093)". Gates 2 and 3 are independent MUSTs at different layers, so an opt-in reaching only one layer **cannot** produce a witness: delivery-only leaves registration returning `400 webhook_url_rejected`, registration-only leaves the dispatcher refusing to connect. A host whose opt-in reaches one layer is **not** non-conformant — it cannot witness these scenarios, which is a property of the test posture and not of its webhook signing. The scheme gate went unstated here until 2026-08-25; a tier-2 host validating scheme-then-address was blocked by a gate the documented contract never named. Relaxing gates 2 and 3 is **test-only posture** — see `SECURITY/threat-model-secret-leakage.md` §4.9 for why a registration-time relaxation is the more dangerous of the two. When the host rejects, the scenario records `blocked` (RFC 0148 §A), not a pass. |
82
82
  | `OPENWOP_MCP_FAKE_SERVER=true` | Boots the synthetic MCP peer for `mcp-tool-roundtrip.test.ts`. |
83
83
  | `OPENWOP_MCP_REAL_SERVER_URL=<base-url>` | Points the MCP wire-shape probe at a real MCP server. The probe POSTs JSON-RPC and reads a single-JSON response — matches MCP's `streamable-http` transport in single-response mode. **Does NOT support** stdio transport (which is what most `modelcontextprotocol/servers` references default to) or SSE-streamed responses; an operator collecting interop evidence today runs a custom `StreamableHTTPServerTransport`-style server that returns a single JSON body per request. Adding SSE-frame parsing is tracked in `docs/PROTOCOL-GAP-CLOSURE-PLAN.md` Track 6. Assertions relax to shape-only. When both this and `OPENWOP_MCP_FAKE_SERVER` are set, the real URL wins. Phase 3 T3.4 interop-evidence path. |
84
84
  | `OPENWOP_A2A_FAKE_PEER=true` | Boots the synthetic A2A peer for `a2a-task-roundtrip.test.ts`. |
package/coverage.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # OpenWOP Conformance Coverage Map
2
2
 
3
- > **Status: Living document. Updated 2026-06-29.** This map connects the current scenario files to the protocol surfaces they protect and records the remaining gaps from the protocol deep dive. Scenario names are source-of-truth file names under `conformance/src/scenarios/`.
3
+ > **Status: Living document. Updated 2026-08-25.** This map connects the current scenario files to the protocol surfaces they protect and records the remaining gaps from the protocol deep dive. Scenario names are source-of-truth file names under `conformance/src/scenarios/`.
4
4
 
5
5
  > **Shape grade vs behavior grade.** Some optional-profile scenarios validate **capability shape** (the host's discovery advertisement is well-formed) without yet exercising **behavior** (the host actually implements the profile end-to-end). The "Current grade" column reflects shape; see §"Capability-gated scenarios: shape vs behavior" below for the dual-grade view and the `OPENWOP_REQUIRE_BEHAVIOR=true` strict-mode runner flag.
6
6
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@openwop/openwop-conformance",
3
- "version": "1.138.1",
3
+ "version": "1.139.0",
4
4
  "description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
5
5
  "repository": {
6
6
  "type": "git",
@@ -1,5 +1,5 @@
1
1
  {
2
2
  "_comment": "Provenance of this vendored schemas/ copy. See conformance/README.md \u00a7\"Resolving the contract\". Compare against the stamp in your installed @openwop/openwop-conformance to detect a stale hand-copied contract.",
3
- "suiteVersion": "1.138.1",
4
- "corpusCommit": "3555363038ce8ea7e40dc2f2a41c39066228e69c"
3
+ "suiteVersion": "1.139.0",
4
+ "corpusCommit": "f34a41ac4c1737ca8af34d0f7f17876268f58e69"
5
5
  }
@@ -0,0 +1,169 @@
1
+ /**
2
+ * Run-end disposition summary (RFC 0148 §A, 2026-08-25).
3
+ *
4
+ * ## The gap this closes
5
+ *
6
+ * Every scenario file already records ONE honest disposition when it
7
+ * finishes (`setup.ts`'s `afterAll` → `resolveFileRecord`): a file that
8
+ * returned early with zero assertions is `blocked` / `skipped` /
9
+ * `inapplicable` with a reason, never a pass. That machinery is correct.
10
+ *
11
+ * It was **published on one invocation out of two**. The worker appends
12
+ * each recording to `OPENWOP_LEDGER_PATH`, and the ONLY writer of that
13
+ * variable was `cli.ts` inside `runCertify`. Under a plain `vitest run`
14
+ * — what `README.md` documents for host operators, and what host
15
+ * implementers actually run — the worker computed `blocked`, held it in
16
+ * memory, and discarded it at process exit. The single surviving artifact
17
+ * was vitest's `1 passed`.
18
+ *
19
+ * So the suite computed an honest disposition on every run and published
20
+ * it on one. Two host implementers independently read the console line as
21
+ * coverage and reported the suite as claiming a pass it had internally
22
+ * classified `blocked` — correctly, given the only artifact they had.
23
+ * RFC 0158 §"Operator preconditions are declared, not hidden" requires a
24
+ * blocked row be surfaced "never silently skipped or reported as a pass";
25
+ * the bundle honored that and the console contradicted it.
26
+ *
27
+ * ## What this does
28
+ *
29
+ * When `OPENWOP_LEDGER_PATH` is unset, point it at a temp file so the
30
+ * workers record there anyway, then read it at run end and print a
31
+ * summary. `--certify` sets the variable itself, so this defers to it
32
+ * completely and never interferes with bundle generation.
33
+ *
34
+ * The summary is deliberately NOT a pass/fail signal — it does not change
35
+ * the exit code. `vitest`'s exit code answers "did any assertion fail";
36
+ * this answers "what did the run actually witness", and those are
37
+ * different questions. Conflating them is what produced the gap.
38
+ *
39
+ * Set `OPENWOP_DISPOSITION_SUMMARY=false` to silence it.
40
+ */
41
+
42
+ import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
43
+ import { join } from 'node:path';
44
+ import { tmpdir } from 'node:os';
45
+
46
+ /** Dispositions that are NOT a witnessed execution — the rows worth printing. */
47
+ const NON_EXECUTING = new Set(['blocked', 'skipped', 'inapplicable']);
48
+
49
+ /** Cap on printed rows; the overflow is always announced, never silent. */
50
+ const MAX_ROWS = 40;
51
+
52
+ // Import the REAL entry type rather than restating its shape. A hand-written
53
+ // structural guess here read `entry.id` — a key the ledger has never written
54
+ // (it is `requirementId`) — and the `(unnamed)` fallback made the mismatch
55
+ // look like missing data instead of a wrong reader. A type-only import erases
56
+ // at runtime, so this costs nothing and makes the compiler the oracle: rename
57
+ // a ledger field and this file stops building instead of silently degrading.
58
+ import type { LedgerEntry } from './lib/requirement-ledger.js';
59
+
60
+ /** What a JSONL line parses to before validation — every field may be absent. */
61
+ type LedgerLine = Partial<Record<keyof LedgerEntry, unknown>>;
62
+
63
+ let ownedDir: string | null = null;
64
+ let ledgerPath: string | null = null;
65
+
66
+ export function setup(): void {
67
+ if (process.env['OPENWOP_DISPOSITION_SUMMARY'] === 'false') return;
68
+ // `--certify` already routes the ledger to its own report dir and reads it
69
+ // there. Never take it over — this exists only for the path that had no
70
+ // reader at all.
71
+ if ((process.env['OPENWOP_LEDGER_PATH'] ?? '') !== '') return;
72
+ ownedDir = mkdtempSync(join(tmpdir(), 'owp-dispositions-'));
73
+ ledgerPath = join(ownedDir, 'requirement-ledger.jsonl');
74
+ process.env['OPENWOP_LEDGER_PATH'] = ledgerPath;
75
+ }
76
+
77
+ /**
78
+ * Fold raw JSONL into the printable summary. PURE — no I/O, no env — so
79
+ * `global-setup.test.ts` can pin it without a vitest subprocess or a host.
80
+ * Returns `null` when there is nothing to say.
81
+ */
82
+ export function summarise(raw: string): string | null {
83
+ const counts = new Map<string, number>();
84
+ const rows: Array<{ id: string; disposition: string; detail: string }> = [];
85
+ for (const line of raw.split('\n')) {
86
+ if (line.trim() === '') continue;
87
+ let entry: LedgerLine;
88
+ try {
89
+ entry = JSON.parse(line) as LedgerLine;
90
+ } catch {
91
+ continue;
92
+ }
93
+ const disposition = typeof entry.disposition === 'string' ? entry.disposition : 'unknown';
94
+ const id = typeof entry.requirementId === 'string' ? entry.requirementId : '(unnamed)';
95
+ const detail = typeof entry.detail === 'string' ? entry.detail : '';
96
+ counts.set(disposition, (counts.get(disposition) ?? 0) + 1);
97
+ if (NON_EXECUTING.has(disposition)) {
98
+ rows.push({ id, disposition, detail });
99
+ } else if (disposition === 'executed-pass' && entry.assertionCount === 0) {
100
+ // The other shape RFC 0148 §A rejects: a pass that asserted nothing.
101
+ rows.push({ id, disposition: 'executed-pass (0 assertions)', detail });
102
+ }
103
+ }
104
+ if (counts.size === 0) return null;
105
+ const total = [...counts.values()].reduce((a, b) => a + b, 0);
106
+ const order = ['executed-pass', 'executed-fail', 'blocked', 'skipped', 'inapplicable'];
107
+ const summary = [...order, ...[...counts.keys()].filter((k) => !order.includes(k))]
108
+ .filter((k) => counts.has(k))
109
+ .map((k) => `${k} ${counts.get(k)}`)
110
+ .join(' \u00b7 ');
111
+ const out: string[] = [
112
+ '',
113
+ `[openwop-conformance] RFC 0148 \u00a7A dispositions \u2014 ${total} requirement(s) recorded`,
114
+ ` ${summary}`,
115
+ ];
116
+ if (rows.length > 0) {
117
+ out.push(
118
+ ` ${rows.length} requirement(s) did NOT witness. vitest reports these as passes because`,
119
+ ' no assertion failed; that is a test outcome, not a conformance disposition.',
120
+ );
121
+ for (const r of rows.slice(0, MAX_ROWS)) {
122
+ const detail = r.detail.length > 120 ? `${r.detail.slice(0, 117)}...` : r.detail;
123
+ out.push(` ${r.disposition.padEnd(28)} ${r.id}${detail === '' ? '' : `\n ${detail}`}`);
124
+ }
125
+ if (rows.length > MAX_ROWS) {
126
+ // Never truncate silently: a capped list that does not say it was capped
127
+ // reads as a complete one, which is the same defect this whole summary
128
+ // exists to remove.
129
+ out.push(` ... and ${rows.length - MAX_ROWS} more not listed (run with --certify for the full bundle)`);
130
+ }
131
+ }
132
+ out.push('');
133
+ return out.join('\n');
134
+ }
135
+
136
+ export function teardown(): void {
137
+ if (ledgerPath === null || ownedDir === null) return;
138
+ // We set the variable; unset it so a subsequent in-process run re-derives.
139
+ delete process.env['OPENWOP_LEDGER_PATH'];
140
+ let raw = '';
141
+ try {
142
+ raw = readFileSync(ledgerPath, 'utf8');
143
+ } catch {
144
+ // No file means no worker ever recorded \u2014 nothing to summarise, and a
145
+ // missing summary must never be mistaken for "nothing was blocked", so
146
+ // say so rather than printing an empty clean bill.
147
+ process.stderr.write(
148
+ '\n[openwop-conformance] no disposition ledger was written \u2014 the run recorded nothing, ' +
149
+ 'which is NOT the same as every scenario having witnessed its requirement.\n',
150
+ );
151
+ cleanup();
152
+ return;
153
+ }
154
+ const text = summarise(raw);
155
+ if (text !== null) process.stderr.write(text);
156
+ cleanup();
157
+ }
158
+
159
+ function cleanup(): void {
160
+ if (ownedDir !== null) {
161
+ try {
162
+ rmSync(ownedDir, { recursive: true, force: true });
163
+ } catch {
164
+ /* best effort */
165
+ }
166
+ }
167
+ ownedDir = null;
168
+ ledgerPath = null;
169
+ }
@@ -0,0 +1,69 @@
1
+ /**
2
+ * Pins `forkDeclined` (RFC 0148 §A).
3
+ *
4
+ * The risk in a helper like this is not that it misses a status — that shows up
5
+ * as a red test somebody investigates. It is that it swallows one: every status
6
+ * it accepts turns an assertion into a skip, silently, and a suite that skips
7
+ * is indistinguishable from a suite that passes unless something records why.
8
+ * So the load-bearing cases here are the NEGATIVE ones.
9
+ */
10
+
11
+ import { describe, expect, it, beforeEach } from 'vitest';
12
+ import { forkDeclined } from './fork-availability.js';
13
+ import { resetSoftSkips, softSkipDisposition } from './soft-skip.js';
14
+
15
+ describe('forkDeclined', () => {
16
+ beforeEach(() => resetSoftSkips());
17
+
18
+ it('treats 404 and 403 the same as 501 — route absent is not weaker than route-declines', () => {
19
+ // The asymmetry this fixes: every replay scenario handled 501 and none
20
+ // handled 404. `501` means "I know this route and decline"; `404` means
21
+ // "there is no such route" — strictly less implemented. The suite treated
22
+ // the weaker signal as a skip and the stronger one as a defect, so a host
23
+ // had to implement the route in order to say it had not implemented it.
24
+ for (const status of [404, 403, 501]) {
25
+ expect(forkDeclined(status, 'leg'), `status ${status} means the fork did not happen`).toBe(true);
26
+ }
27
+ });
28
+
29
+ it('does NOT swallow any other status — a 500 is still a defect, not a skip', () => {
30
+ // If this ever goes green for one of these, a real failure has become a
31
+ // silent skip and no assertion downstream will ever run again.
32
+ for (const status of [200, 201, 202, 400, 409, 422, 500, 502, 503]) {
33
+ expect(forkDeclined(status, 'leg'), `status ${status} must reach the assertion`).toBe(false);
34
+ }
35
+ });
36
+
37
+ it('records WHY, so the ledger row is not an unclassified return', () => {
38
+ // Two of the eleven original call sites were a bare `return`. A bare return
39
+ // in a file whose OTHER tests assert is invisible: the file records
40
+ // `executed-pass` and the leg that never ran leaves no trace.
41
+ forkDeclined(404, 'fanout-suppression replay fork');
42
+ const noted = softSkipDisposition('fork-availability.test.ts');
43
+ expect(noted?.kind).toBe('blocked');
44
+ expect(noted?.reason).toContain('404');
45
+ expect(noted?.reason).toContain('fanout-suppression replay fork');
46
+ expect(noted?.reason).toContain('not mounted');
47
+ });
48
+
49
+ it('records nothing when the status is not a decline', () => {
50
+ forkDeclined(201, 'leg');
51
+ expect(softSkipDisposition('fork-availability.test.ts')).toBeNull();
52
+ });
53
+
54
+ it('names the leg, so one file with several forks stays attributable', () => {
55
+ forkDeclined(501, 'branch fork');
56
+ expect(softSkipDisposition('fork-availability.test.ts')?.reason).toContain('branch fork');
57
+ });
58
+
59
+ it('distinguishes the three declines in its reason text', () => {
60
+ forkDeclined(404, 'a');
61
+ const a = softSkipDisposition('fork-availability.test.ts')?.reason ?? '';
62
+ resetSoftSkips();
63
+ forkDeclined(501, 'a');
64
+ const b = softSkipDisposition('fork-availability.test.ts')?.reason ?? '';
65
+ // `blocked` alone does not tell a bundle reader whether the host lacks the
66
+ // route or declines the range; the reason must.
67
+ expect(a).not.toBe(b);
68
+ });
69
+ });
@@ -0,0 +1,61 @@
1
+ /**
2
+ * "The fork seam did not run" — one rule, one place (RFC 0148 §A).
3
+ *
4
+ * `POST /v1/runs/{runId}:fork` can decline in more than one way, and eleven
5
+ * call sites across six scenarios each decided for themselves which ways
6
+ * counted:
7
+ *
8
+ * - **501** — every site handled this. "Advertised but not implemented for
9
+ * this range", per the suite convention their comments cite.
10
+ * - **404 / 403** — *no* site in the replay family handled it. A host that
11
+ * never mounts the route hard-failed `expect(status).toBe(201)`.
12
+ *
13
+ * That asymmetry is backwards. `501` means "I know this route and decline";
14
+ * `404` means "there is no such route" — strictly less implemented, and the
15
+ * suite treated the weaker signal as a skip and the stronger one as a defect.
16
+ * The postgres reference host 404s, so `replay-fanout-suppression` was red in
17
+ * CI on every run while the other four replay scenarios skipped past the same
18
+ * seam for unrelated fixture reasons. **The suite required a host to implement
19
+ * the route in order to say it had not implemented the route.**
20
+ *
21
+ * Recording matters as much as the predicate. Two of those eleven sites were a
22
+ * bare `return` with no note: `replay-fork.test.ts` reported "6 tests | 6
23
+ * skipped" against a host whose fork surface is entirely unmeasured, and the
24
+ * ledger had nothing to say about why. §A resolves an unclassified return to
25
+ * `blocked`, but only when the file records nothing at all — a bare return in a
26
+ * file whose *other* tests assert is invisible.
27
+ *
28
+ * Disposition is `blocked`, not `inapplicable`: the obligation applies to this
29
+ * host, it simply could not be witnessed. `inapplicable` would claim the host
30
+ * is outside the requirement's scope, which nothing here establishes — these
31
+ * scenarios carry no capability gate, so the suite does not actually know
32
+ * whether the host advertises fork. Saying `blocked` is the claim the evidence
33
+ * supports.
34
+ */
35
+
36
+ import { softSkip } from './soft-skip.js';
37
+
38
+ /** Statuses that mean "the fork did not happen", with what each one tells us. */
39
+ const DECLINED: ReadonlyMap<number, string> = new Map([
40
+ [404, 'the route is not mounted on this host'],
41
+ [403, 'the caller is not permitted to fork (route present, access refused)'],
42
+ [501, 'the host knows the route and declines this range as not implemented'],
43
+ ]);
44
+
45
+ /**
46
+ * True when `status` means the fork seam did not run — and records WHY for the
47
+ * RFC 0148 §A ledger as a side effect, so a caller can `if (forkDeclined(...))
48
+ * return;` without leaving an unclassified return behind.
49
+ *
50
+ * `where` names the leg, so a bundle reader can tell which of several forks in
51
+ * one file declined.
52
+ */
53
+ export function forkDeclined(status: number, where: string): boolean {
54
+ const why = DECLINED.get(status);
55
+ if (why === undefined) return false;
56
+ softSkip(
57
+ 'blocked',
58
+ `${where}: POST /v1/runs/{runId}:fork returned ${status} — ${why}, so the behaviour this requirement is stated over never occurred and its absence below would prove nothing`,
59
+ );
60
+ return true;
61
+ }
@@ -0,0 +1,76 @@
1
+ /**
2
+ * Pins the run-end disposition summary (`src/global-setup.ts`, RFC 0148 §A).
3
+ *
4
+ * The defect this guards is not "the summary is ugly" — it is that the suite
5
+ * computed an honest disposition on every run and PUBLISHED it on one. Two
6
+ * host implementers read vitest's `1 passed` as coverage for a requirement the
7
+ * suite had internally classified `blocked`, because the classification had no
8
+ * reader outside `--certify`.
9
+ *
10
+ * Server-free and pure: `summarise` takes the raw JSONL and returns the text.
11
+ */
12
+
13
+ import { describe, expect, it } from 'vitest';
14
+ import { summarise } from '../global-setup.js';
15
+
16
+ const line = (o: Record<string, unknown>): string => JSON.stringify(o);
17
+
18
+ describe('global-setup: RFC 0148 §A disposition summary', () => {
19
+ it('reads the ledger key the ledger actually writes (`requirementId`, not `id`)', () => {
20
+ // Regression: the first draft of this reader keyed on `entry.id`, a field
21
+ // `recordRequirement` has never written, and fell back to `(unnamed)`.
22
+ // The fallback made a WRONG READER look like MISSING DATA — the same
23
+ // failure shape the summary exists to expose, reproduced in the exposer.
24
+ const out = summarise(
25
+ line({ requirementId: 'openwop.scenario.webhook-signed-delivery', disposition: 'blocked', detail: 'guard rejected loopback' }),
26
+ );
27
+ expect(out).toContain('openwop.scenario.webhook-signed-delivery');
28
+ expect(out).not.toContain('(unnamed)');
29
+ });
30
+
31
+ it('lists blocked / skipped / inapplicable as NOT witnessed, with the reason', () => {
32
+ const out = summarise(
33
+ [
34
+ line({ requirementId: 'a', disposition: 'blocked', detail: 'seam absent' }),
35
+ line({ requirementId: 'b', disposition: 'skipped', detail: 'operator opted out' }),
36
+ line({ requirementId: 'c', disposition: 'inapplicable', detail: 'profile not advertised' }),
37
+ ].join('\n'),
38
+ );
39
+ expect(out).toContain('did NOT witness');
40
+ for (const s of ['seam absent', 'operator opted out', 'profile not advertised']) {
41
+ expect(out).toContain(s);
42
+ }
43
+ });
44
+
45
+ it('flags an executed-pass that asserted nothing — the other RFC 0148 §A vacuity shape', () => {
46
+ const out = summarise(line({ requirementId: 'vacuous', disposition: 'executed-pass', assertionCount: 0 }));
47
+ expect(out).toContain('executed-pass (0 assertions)');
48
+ expect(out).toContain('vacuous');
49
+ });
50
+
51
+ it('does NOT flag a real executed-pass', () => {
52
+ const out = summarise(line({ requirementId: 'real', disposition: 'executed-pass', assertionCount: 12 }));
53
+ expect(out).toContain('executed-pass 1');
54
+ expect(out).not.toContain('did NOT witness');
55
+ });
56
+
57
+ it('announces truncation rather than capping silently', () => {
58
+ // A capped list that does not say it was capped reads as a complete one.
59
+ const many = Array.from({ length: 45 }, (_, i) =>
60
+ line({ requirementId: `r${i}`, disposition: 'blocked', detail: 'why' }),
61
+ ).join('\n');
62
+ const out = summarise(many);
63
+ expect(out).toContain('5 more not listed');
64
+ });
65
+
66
+ it('returns null on an empty ledger so teardown prints nothing', () => {
67
+ expect(summarise('')).toBeNull();
68
+ expect(summarise('\n\n')).toBeNull();
69
+ });
70
+
71
+ it('skips unparseable lines without discarding the rest of the run', () => {
72
+ const out = summarise(['{ not json', line({ requirementId: 'ok', disposition: 'blocked', detail: 'r' })].join('\n'));
73
+ expect(out).toContain('ok');
74
+ expect(out).toContain('blocked 1');
75
+ });
76
+ });
@@ -47,6 +47,7 @@ import { pollUntilTerminal, pollUntilStatus } from '../lib/polling.js';
47
47
  import { SCHEMAS_DIR } from '../lib/paths.js';
48
48
  import { behaviorGate } from '../lib/behavior-gate.js';
49
49
  import { readCapabilityFamily } from '../lib/discovery-capabilities.js';
50
+ import { seamAbsent } from '../lib/soft-skip.js';
50
51
 
51
52
  /**
52
53
  * Callback-shaped: the host issues A2A JSON-RPC calls to the suite's fake peer.
@@ -403,21 +404,76 @@ describe.skipIf(HTTP_SKIP)('a2a-task-roundtrip: durable tasks/get after disconne
403
404
  });
404
405
  });
405
406
 
406
- describe.skipIf(HTTP_SKIP)('a2a-task-roundtrip: push-config SSRF (gated on a2a.pushNotifications; RFC 0100)', () => {
407
- it('registering a pushConfig.url at a private address is refused (a2a-push-egress-ssrf)', async () => {
407
+ /**
408
+ * Push-config SSRF, TWO-SIDED (RFC 0100 §4, a2a-integration.md §D.6).
409
+ *
410
+ * The RFC 0093 webhook-egress guard has two arms — `webhooks.md`
411
+ * §"SSRF protection" rejects non-`https://` protocols AND private/loopback/
412
+ * link-local/ULA/metadata addresses — and a host may implement either one
413
+ * alone.
414
+ *
415
+ * Until 2026-08-25 this file probed with a single `http://10.0.0.5/push`,
416
+ * which **violates both arms at once**. Either arm alone refuses it, so a
417
+ * `>= 400` witnessed *that something refused* and never *which guard ran*.
418
+ * A host with only the address arm passed; so did a host with only the
419
+ * scheme arm; so would a host that refused every push URL for an unrelated
420
+ * reason. The assertion was real and the conclusion drawn from it was not.
421
+ *
422
+ * The probes below isolate one arm each: `https` at a private address can
423
+ * only be refused by the address arm, and `http` at a public host can only
424
+ * be refused by the scheme arm. Two legs is the minimum that distinguishes
425
+ * them — the same reason a negative control is not optional.
426
+ *
427
+ * The scheme leg is new, and the obligation it checks was previously stated
428
+ * only by reference (every prior wording abbreviated the guard to its
429
+ * address arm). `COMPATIBILITY.md` §3 records the Class 3 classification: a
430
+ * host accepting a plaintext push target was never conforming. An
431
+ * implementer reddened by this leg is reading a clarification, not a new
432
+ * requirement — the failure message says so.
433
+ */
434
+ describe.skipIf(HTTP_SKIP)('a2a-task-roundtrip: push-config SSRF, two-sided (gated on a2a.pushNotifications; RFC 0100)', () => {
435
+ async function registerPush(url: string): Promise<{ status: number } | null> {
408
436
  const a2a = await readCapabilityFamily<{ pushNotifications?: boolean }>('a2a');
409
- if (!behaviorGate('a2a.pushNotifications', a2a?.pushNotifications === true)) return;
437
+ if (!behaviorGate('a2a.pushNotifications', a2a?.pushNotifications === true)) return null;
438
+ const res = await driver.post('/v1/host/sample/a2a/tasks/push-config', { taskId: 'run_x', url });
439
+ if (res.status === 404 || res.status === 403) {
440
+ // Previously a bare `return`, invisible to the RFC 0148 §A ledger because
441
+ // this file's other tests assert — so the file recorded `executed-pass`
442
+ // while this leg had witnessed nothing. Say why instead.
443
+ return seamAbsent(
444
+ 'a2a-push-egress-ssrf — the `/v1/host/sample/a2a/tasks/push-config` seam is not mounted (404/403), so neither guard arm is observable',
445
+ ) ?? null;
446
+ }
447
+ return { status: res.status };
448
+ }
449
+
450
+ it('ADDRESS arm: an https pushConfig.url at a private address is refused', async () => {
451
+ // `https` on purpose: this probe is refusable ONLY by the address arm, so a
452
+ // host that implements the scheme arm alone cannot pass it by accident.
453
+ const res = await registerPush('https://10.0.0.5/push');
454
+ if (res === null) return;
455
+ expect(
456
+ res.status >= 400,
457
+ driver.describe(
458
+ 'a2a-integration.md §D.6 (address arm)',
459
+ 'a2a-push-egress-ssrf — a pushConfig.url at a private/loopback address MUST be refused before any push, even over https',
460
+ ),
461
+ ).toBe(true);
462
+ });
410
463
 
411
- const res = await driver.post('/v1/host/sample/a2a/tasks/push-config', {
412
- taskId: 'run_x',
413
- url: 'http://10.0.0.5/push',
414
- });
415
- if (res.status === 404 || res.status === 403) return; // seam unwired — soft-skip
464
+ it('SCHEME arm: an http pushConfig.url at a public host is refused', async () => {
465
+ // Public hostname on purpose: refusable ONLY by the scheme arm.
466
+ const res = await registerPush('http://push.example.com/push');
467
+ if (res === null) return;
416
468
  expect(
417
469
  res.status >= 400,
418
470
  driver.describe(
419
- 'a2a-integration.md §"Async / durable Tasks"',
420
- 'a2a-push-egress-ssrf — a caller-supplied pushConfig.url at a private/loopback address MUST be refused before any push',
471
+ 'a2a-integration.md §D.6 (scheme arm)',
472
+ 'a2a-push-egress-ssrf — a plaintext `http://` pushConfig.url MUST be refused before any push. '
473
+ + 'The RFC 0093 webhook-egress guard is the `webhooks.md` §"SSRF protection" list IN FULL, whose first entry is '
474
+ + '"Non-`https://` protocols". Every prior wording of this requirement abbreviated the guard to its address arm; '
475
+ + 'COMPATIBILITY.md §3 records this as a Class 3 clarification, so a host failing here was never conforming rather '
476
+ + 'than newly non-conforming.',
421
477
  ),
422
478
  ).toBe(true);
423
479
  });
@@ -38,6 +38,7 @@
38
38
 
39
39
  import { describe, it, expect } from 'vitest';
40
40
  import { driver } from '../lib/driver.js';
41
+ import { forkDeclined } from '../lib/fork-availability.js';
41
42
  import { pollUntilTerminal } from '../lib/polling.js';
42
43
  import { behaviorGate } from '../lib/behavior-gate.js';
43
44
  import { isFixtureAdvertised } from '../lib/fixtures.js';
@@ -149,8 +150,7 @@ describe.skipIf(HTTP_SKIP)('agent-channel-dispatch (RFC 0082 §B): production ru
149
150
  `/v1/runs/${encodeURIComponent(sourceRunId)}:fork`,
150
151
  { fromSeq: 0, mode: 'replay' },
151
152
  );
152
- if (fork1.status === 501) {
153
- // replay advertised but not implemented for this run — skip-equivalent.
153
+ if (forkDeclined(fork1.status, 'channel-dispatch replay fork 1')) {
154
154
  ctx.skip();
155
155
  return;
156
156
  }
@@ -213,7 +213,7 @@ describe.skipIf(HTTP_SKIP)('agent-channel-dispatch (RFC 0082 §B): production ru
213
213
  `/v1/runs/${encodeURIComponent(sourceRunId)}:fork`,
214
214
  { fromSeq: 0, mode: 'replay' },
215
215
  );
216
- if (fork2.status === 501) {
216
+ if (forkDeclined(fork2.status, 'channel-dispatch replay fork 2')) {
217
217
  ctx.skip();
218
218
  return;
219
219
  }
@@ -72,7 +72,8 @@ import { afterEach, describe, expect, it } from 'vitest';
72
72
  import { createServer, type IncomingMessage, type Server, type ServerResponse } from 'node:http';
73
73
  import { softSkip } from '../lib/soft-skip.js';
74
74
  import { driver } from '../lib/driver.js';
75
- import { discoveryFamilies } from '../lib/discovery-capabilities.js';
75
+ import { forkDeclined } from '../lib/fork-availability.js';
76
+ import { discoveryFamilies, readCapabilityFamily } from '../lib/discovery-capabilities.js';
76
77
  import { pollUntilTerminal, scaledTimeoutMs } from '../lib/polling.js';
77
78
  import { isFixtureAdvertised } from '../lib/fixtures.js';
78
79
  import { discoverOwnedTenant } from '../lib/webhook-receiver.js';
@@ -213,18 +214,50 @@ describe('replay-fanout-suppression: a replay fork MUST NOT fan out re-emitted e
213
214
  ),
214
215
  ).toBeGreaterThan(0);
215
216
 
217
+ // ── CAPABILITY GATE (added 2026-08-25) ──────────────────────────────────
218
+ // This is the ONLY scenario in the replay family that never checked whether
219
+ // the host advertises replay before forking — every sibling reads
220
+ // `replay.supported` first. That omission is why it was the one scenario
221
+ // reaching the fork seam on a host that does not implement it, and why it
222
+ // hard-failed `expected 404 to be 201` on every CI run of `main` while the
223
+ // siblings quietly returned at their capability check.
224
+ //
225
+ // `inapplicable`, not `blocked`: a host that does not advertise replay is
226
+ // outside this MUST NOT's scope entirely, and `blocked` would claim the
227
+ // requirement applies but could not be witnessed — a stronger claim than
228
+ // the evidence supports. Recorded explicitly because LEG 1 above asserted,
229
+ // so the file-level disposition would otherwise be `executed-pass`.
230
+ const replayCap = await readCapabilityFamily<{ supported?: boolean; modes?: unknown }>('replay');
231
+ if (replayCap?.supported !== true) {
232
+ recordRequirement(
233
+ REQUIREMENT_ID,
234
+ 'inapplicable',
235
+ 'host does not advertise `replay.supported: true`, so a replay fork cannot occur and this MUST NOT '
236
+ + 'has nothing to constrain on this host',
237
+ );
238
+ ctx.skip();
239
+ return;
240
+ }
241
+
216
242
  // ── LEG 2 — THE MUST NOT. A replay fork re-emits; it must not deliver. ───
217
243
  const replay = await driver.post(`/v1/runs/${encodeURIComponent(sourceRunId)}:fork`, {
218
244
  mode: 'replay',
219
245
  });
220
- if (replay.status === 501) {
221
- // Advertised but not implemented for this range. Leg 1 already asserted,
222
- // so the FILE is `executed-pass` but the MUST NOT was never exercised,
223
- // and that is what this row must say.
246
+ if (forkDeclined(replay.status, 'fanout-suppression replay fork')) {
247
+ // Leg 1 already asserted, so the FILE is `executed-pass` but the MUST
248
+ // NOT was never exercised, and that is what this row must say. The
249
+ // explicit record wins over the file-level one (setup.ts).
250
+ //
251
+ // 404 and 403 were NOT handled here until 2026-08-25, only 501. The
252
+ // postgres reference host 404s this route, so this was the one scenario
253
+ // in the replay family that actually reached the seam — and it hard-
254
+ // failed `expected 404 to be 201` on every CI run of `main`, absorbed by
255
+ // the 85% pass-rate floor. The suite required a host to implement the
256
+ // route in order to say it had not implemented the route.
224
257
  recordRequirement(
225
258
  REQUIREMENT_ID,
226
259
  'blocked',
227
- 'replay fork returned 501 — the re-emission this requirement is stated over never happened, '
260
+ `replay fork returned ${replay.status} — the re-emission this requirement is stated over never happened, `
228
261
  + 'so the absence of deliveries below would prove nothing',
229
262
  );
230
263
  ctx.skip();
@@ -30,6 +30,8 @@
30
30
 
31
31
  import { describe, it, expect } from 'vitest';
32
32
  import { driver } from '../lib/driver.js';
33
+ import { softSkip } from '../lib/soft-skip.js';
34
+ import { forkDeclined } from '../lib/fork-availability.js';
33
35
  import { pollUntilTerminal } from '../lib/polling.js';
34
36
  import { isFixtureAdvertised } from '../lib/fixtures.js';
35
37
 
@@ -122,6 +124,7 @@ describe.skipIf(SKIP_NO_MULTI)(
122
124
  it('mid-fromSeq replay fork produces a new run that reaches `completed`', async (ctx) => {
123
125
  const replay = await fetchReplayCapability();
124
126
  if (replay?.supported !== true) {
127
+ softSkip('inapplicable', "host does not advertise `replay.supported: true` — the replay contract does not apply to it");
125
128
  ctx.skip();
126
129
  return;
127
130
  }
@@ -129,6 +132,7 @@ describe.skipIf(SKIP_NO_MULTI)(
129
132
  ? replay.modes.filter((m): m is string => typeof m === 'string')
130
133
  : [];
131
134
  if (!modes.includes('replay')) {
135
+ softSkip('inapplicable', "host does not advertise the `replay` fork mode — this leg's rule has no path to apply");
132
136
  ctx.skip();
133
137
  return;
134
138
  }
@@ -140,6 +144,7 @@ describe.skipIf(SKIP_NO_MULTI)(
140
144
  // Fixture's wire shape doesn't expose node.completed(b) with a
141
145
  // numeric sequence — skip rather than fail. Conformant hosts
142
146
  // with the standard event shape will hit the assertions below.
147
+ softSkip('blocked', "the advertised fixture's wire shape exposes no numeric sequence for node.completed(b), so there is no mid-run point to fork from");
143
148
  ctx.skip();
144
149
  return;
145
150
  }
@@ -149,7 +154,7 @@ describe.skipIf(SKIP_NO_MULTI)(
149
154
  { fromSeq, mode: 'replay' },
150
155
  );
151
156
 
152
- if (fork.status === 501) {
157
+ if (forkDeclined(fork.status, 'arbitrary-event fork')) {
153
158
  ctx.skip();
154
159
  return;
155
160
  }
@@ -226,7 +231,7 @@ describe.skipIf(SKIP_NO_MULTI)(
226
231
  `/v1/runs/${encodeURIComponent(sourceRunId)}:fork`,
227
232
  { fromSeq, mode: 'replay' },
228
233
  );
229
- if (fork1.status === 501) {
234
+ if (forkDeclined(fork1.status, 'arbitrary-event fork 1')) {
230
235
  ctx.skip();
231
236
  return;
232
237
  }
@@ -238,7 +243,7 @@ describe.skipIf(SKIP_NO_MULTI)(
238
243
  `/v1/runs/${encodeURIComponent(sourceRunId)}:fork`,
239
244
  { fromSeq, mode: 'replay' },
240
245
  );
241
- if (fork2.status === 501) {
246
+ if (forkDeclined(fork2.status, 'arbitrary-event fork 2')) {
242
247
  ctx.skip();
243
248
  return;
244
249
  }
@@ -294,6 +299,7 @@ describe.skipIf(SKIP_NO_MULTI)(
294
299
  it('mid-fromSeq branch fork with empty overlay produces a new run that reaches `completed`', async (ctx) => {
295
300
  const replay = await fetchReplayCapability();
296
301
  if (replay?.supported !== true) {
302
+ softSkip('inapplicable', "host does not advertise `replay.supported: true` — the replay contract does not apply to it");
297
303
  ctx.skip();
298
304
  return;
299
305
  }
@@ -25,6 +25,8 @@
25
25
 
26
26
  import { describe, it, expect } from 'vitest';
27
27
  import { driver } from '../lib/driver.js';
28
+ import { softSkip } from '../lib/soft-skip.js';
29
+ import { forkDeclined } from '../lib/fork-availability.js';
28
30
  import { pollUntilTerminal } from '../lib/polling.js';
29
31
  import { isFixtureAdvertised } from '../lib/fixtures.js';
30
32
 
@@ -60,6 +62,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: fork from fromSeq=0 in replay mode', () =
60
62
  // Visible skip — earlier this was a silent `return` that
61
63
  // collapsed to a vacuous pass and made it impossible to tell
62
64
  // unexercised tests apart from honest passes.
65
+ softSkip('inapplicable', "host does not advertise the `replay` fork mode — this leg's rule has no path to apply");
63
66
  ctx.skip();
64
67
  return;
65
68
  }
@@ -70,7 +73,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: fork from fromSeq=0 in replay mode', () =
70
73
  { fromSeq: 0, mode: 'replay' },
71
74
  );
72
75
 
73
- if (fork.status === 501) return; // mode advertised but not implemented; skip-equivalent
76
+ if (forkDeclined(fork.status, 'replay fork')) return;
74
77
  expect(fork.status, driver.describe(
75
78
  'rest-endpoints.md POST /v1/runs/{runId}:fork',
76
79
  'fork MUST return 201 on accepted replay',
@@ -101,6 +104,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: fork from fromSeq=0 in branch mode with e
101
104
  it('produces a new run that reaches terminal `completed`', async (ctx) => {
102
105
  const modes = await fetchReplayModes();
103
106
  if (!modes.includes('branch')) {
107
+ softSkip('inapplicable', "host does not advertise the `branch` fork mode — this leg's rule has no path to apply");
104
108
  ctx.skip();
105
109
  return;
106
110
  }
@@ -111,7 +115,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: fork from fromSeq=0 in branch mode with e
111
115
  { fromSeq: 0, mode: 'branch', runOptionsOverlay: {} },
112
116
  );
113
117
 
114
- if (fork.status === 501) return; // mode advertised but not implemented; skip-equivalent
118
+ if (forkDeclined(fork.status, 'branch fork')) return;
115
119
  expect(fork.status, driver.describe(
116
120
  'rest-endpoints.md POST /v1/runs/{runId}:fork',
117
121
  'branch fork MUST return 201',
@@ -137,6 +141,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: validation errors', () => {
137
141
  it('rejects negative fromSeq with 400', async (ctx) => {
138
142
  const modes = await fetchReplayModes();
139
143
  if (modes.length === 0) {
144
+ softSkip('inapplicable', "host advertises no usable fork mode for this leg");
140
145
  ctx.skip();
141
146
  return;
142
147
  }
@@ -155,6 +160,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: validation errors', () => {
155
160
  it('rejects fromSeq beyond source event log length with 422', async (ctx) => {
156
161
  const modes = await fetchReplayModes();
157
162
  if (modes.length === 0) {
163
+ softSkip('inapplicable', "host advertises no usable fork mode for this leg");
158
164
  ctx.skip();
159
165
  return;
160
166
  }
@@ -179,6 +185,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: validation errors', () => {
179
185
  // → 400) only applies on hosts that advertise the `replay` mode.
180
186
  // A `branch`-only host has no path to even attempt the request.
181
187
  // Visible skip rather than silent vacuous pass.
188
+ softSkip('inapplicable', "host does not advertise the `replay` fork mode — the runOptionsOverlay rejection rule only applies to hosts that do");
182
189
  ctx.skip();
183
190
  return;
184
191
  }
@@ -200,6 +207,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay: validation errors', () => {
200
207
  it('rejects fork on a non-existent run with 404', async (ctx) => {
201
208
  const modes = await fetchReplayModes();
202
209
  if (modes.length === 0) {
210
+ softSkip('inapplicable', "host advertises no usable fork mode for this leg");
203
211
  ctx.skip();
204
212
  return;
205
213
  }
@@ -48,6 +48,7 @@ import { describe, it, expect } from 'vitest';
48
48
  import { recordRequirement } from '../lib/requirement-ledger.js';
49
49
  import { requirementIdForScenario } from '../lib/requirement-registry.js';
50
50
  import { driver } from '../lib/driver.js';
51
+ import { forkDeclined } from '../lib/fork-availability.js';
51
52
  import { pollUntilTerminal } from '../lib/polling.js';
52
53
  import { isFixtureAdvertised } from '../lib/fixtures.js';
53
54
  import { readFileSync } from 'node:fs';
@@ -173,8 +174,8 @@ describe.skipIf(SKIP_NO_FIXTURE)('replay-side-effect-suppression: a replay does
173
174
  const fork = await driver.post(`/v1/runs/${encodeURIComponent(sourceRunId)}:fork`, {
174
175
  mode: 'replay',
175
176
  });
176
- if (fork.status === 501) {
177
- ctx.skip(); // advertised but not implemented for this range — suite convention
177
+ if (forkDeclined(fork.status, 'side-effect-suppression replay fork')) {
178
+ ctx.skip();
178
179
  return;
179
180
  }
180
181
  expect(fork.status, 'fork should be accepted').toBe(201);
@@ -18,6 +18,8 @@
18
18
 
19
19
  import { describe, it, expect } from 'vitest';
20
20
  import { driver } from '../lib/driver.js';
21
+ import { softSkip } from '../lib/soft-skip.js';
22
+ import { forkDeclined } from '../lib/fork-availability.js';
21
23
  import { pollUntilTerminal } from '../lib/polling.js';
22
24
  import { isFixtureAdvertised } from '../lib/fixtures.js';
23
25
 
@@ -69,7 +71,7 @@ function structuralShape(events: readonly RawEvent[]): Array<{ type: unknown; no
69
71
  describe('replay-determinism: openwop-replay-fork profile gate', () => {
70
72
  it('host advertising replay.supported MUST also advertise replay.modes', async () => {
71
73
  const replay = await fetchReplayCapability();
72
- if (replay === null || replay.supported !== true) return; // skip-equivalent
74
+ if (replay === null || replay.supported !== true) return softSkip('inapplicable', 'host does not advertise `replay.supported: true` — the replay contract does not apply to it');
73
75
 
74
76
  expect(Array.isArray(replay.modes), driver.describe(
75
77
  'spec/v1/replay.md',
@@ -91,8 +93,8 @@ describe.skipIf(SKIP_NO_NOOP)('replay-determinism: same fromSeq + same workflow
91
93
  'two replay forks of the same point produce structurally-identical event lists',
92
94
  async () => {
93
95
  const replay = await fetchReplayCapability();
94
- if (replay === null || replay.supported !== true) return; // host doesn't claim replay
95
- if (!Array.isArray(replay.modes) || !replay.modes.includes('replay')) return; // mode not supported
96
+ if (replay === null || replay.supported !== true) return softSkip('inapplicable', 'host does not advertise `replay.supported: true` — the replay contract does not apply to it');
97
+ if (!Array.isArray(replay.modes) || !replay.modes.includes('replay')) return softSkip('inapplicable', 'host advertises replay but not the `replay` mode — this leg is out of scope for it');
96
98
 
97
99
  // Phase 1: complete an original run.
98
100
  const create = await driver.post('/v1/runs', { workflowId: NOOP_WORKFLOW_ID });
@@ -105,7 +107,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay-determinism: same fromSeq + same workflow
105
107
  mode: 'replay',
106
108
  fromSeq: 0,
107
109
  });
108
- if (fork1.status === 501) return; // mode not implemented; skip-equivalent
110
+ if (forkDeclined(fork1.status, 'determinism fork 1')) return;
109
111
  expect(fork1.status, driver.describe(
110
112
  'spec/v1/replay.md',
111
113
  'POST /v1/runs/{runId}:fork with mode=replay MUST return 201',
@@ -118,7 +120,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay-determinism: same fromSeq + same workflow
118
120
  mode: 'replay',
119
121
  fromSeq: 0,
120
122
  });
121
- if (fork2.status === 501) return;
123
+ if (forkDeclined(fork2.status, 'determinism fork 2')) return;
122
124
  expect(fork2.status).toBe(201);
123
125
  const fork2Id = (fork2.json as { runId: string }).runId;
124
126
  await pollUntilTerminal(fork2Id, { timeoutMs: 10_000 });
@@ -152,7 +154,7 @@ describe.skipIf(SKIP_NO_NOOP)('replay-determinism: same fromSeq + same workflow
152
154
  describe.skipIf(SKIP_NO_NOOP)('replay-determinism: branch-mode is permitted to diverge', () => {
153
155
  it('branch mode does NOT need to produce identical event sequences (negative-control)', async () => {
154
156
  const replay = await fetchReplayCapability();
155
- if (replay === null || replay.supported !== true) return;
157
+ if (replay === null || replay.supported !== true) return softSkip('inapplicable', 'host does not advertise `replay.supported: true` — the replay contract does not apply to it');
156
158
  if (!Array.isArray(replay.modes) || !replay.modes.includes('branch')) return;
157
159
 
158
160
  // Self-test on the spec interpretation: branch and replay are
@@ -16,13 +16,59 @@
16
16
  * Capability-gated: skips when the host does not advertise
17
17
  * `capabilities.webhooks.supported = true`.
18
18
  *
19
- * Operator contract: hosts that implement a SSRF guard on
20
- * `POST /v1/webhooks` (rejecting loopback / RFC1918 / link-local
21
- * destinations to protect deployer infrastructure) MUST allow the test
22
- * receiver. The SQLite reference host bypasses the guard when the
23
- * `OPENWOP_WEBHOOK_ALLOW_PRIVATE=true` env var is set at boot. Test-only
24
- * hosts SHOULD provide an equivalent opt-in. When the host rejects with
25
- * `400 webhook_url_rejected`, this scenario skips with a warning.
19
+ * Operator contract THREE gates, not one (clarified 2026-08-25).
20
+ *
21
+ * The test receiver is `http://127.0.0.1:{port}/`, and `webhooks.md`
22
+ * forbids it three separate times. A host honoring
23
+ * `OPENWOP_WEBHOOK_ALLOW_PRIVATE=true` (or an equivalent opt-in) MUST
24
+ * relax ALL THREE for this scenario to be witnessable:
25
+ *
26
+ * 1. **Scheme.** §"SSRF protection" bullet 1 rejects non-`https://`
27
+ * protocols, and §"Register" says `url` MUST be `https://`. The
28
+ * receiver is plain `http`. This gate is the one the flag's
29
+ * description omitted until 2026-08-25, and it fires FIRST on a
30
+ * host that validates scheme before address.
31
+ * 2. **Registration-time address check.** §"SSRF protection": the
32
+ * server MUST validate subscription URLs at registration time and
33
+ * reject loopback / RFC1918 / link-local / ULA / metadata.
34
+ * 3. **Delivery-time re-resolution.** §"Delivery-time egress
35
+ * validation (RFC 0093)": the dispatcher MUST re-resolve at
36
+ * delivery time and validate every resolved address against the
37
+ * same ranges.
38
+ *
39
+ * Gates 2 and 3 are INDEPENDENT MUSTs at different layers, so "which
40
+ * layer must the opt-in reach" is not a matter of taste: a relaxation
41
+ * reaching only one layer cannot produce a witness. Delivery-only leaves
42
+ * registration returning `400 webhook_url_rejected` (observed on a tier-2
43
+ * host, 2026-08-25); registration-only leaves the dispatcher re-resolving
44
+ * `127.0.0.1` and refusing to connect. A host whose opt-in reaches one
45
+ * layer is not non-conformant — it simply cannot witness this scenario,
46
+ * and that is a property of the test posture, not of its webhook signing.
47
+ *
48
+ * What that costs is more than a checkmark, and it is worth stating plainly
49
+ * because the cost is invisible from inside such a host. This scenario is
50
+ * typically the ONLY automated oracle a host has on its delivery header
51
+ * NAMES: a host-local delivery test almost always compares a hand-written
52
+ * literal in the test to a hand-written literal in the implementation, which
53
+ * is a mirror — typo both and it stays green — while the HMAC assertion
54
+ * beside it recomputes from `node:crypto` and is real. So the names go
55
+ * unchecked exactly where the bytes are checked well. All three reference
56
+ * hosts emitted invented header names for 16 days (openwop-examples#22) and
57
+ * the runs that would have caught it were the ones this scenario could not
58
+ * reach. A host that runs it is not thereby careful; it is thereby measured.
59
+ *
60
+ * Relaxing gates 2 and 3 is test-only posture. See
61
+ * `SECURITY/threat-model-secret-leakage.md` §4.9 for why a
62
+ * registration-time relaxation is the more dangerous of the two: it
63
+ * writes a durable subscription row that survives the flag being turned
64
+ * back off.
65
+ *
66
+ * When the host rejects with `400 webhook_url_rejected`, this scenario
67
+ * records `blocked` (RFC 0148 §A) — NOT a pass. Under a plain
68
+ * `vitest run` the console still prints "1 passed", because that is
69
+ * vitest reporting test outcomes rather than conformance dispositions;
70
+ * the run-end disposition summary (`src/global-setup.ts`) is where the
71
+ * `blocked` becomes visible without `--certify`.
26
72
  *
27
73
  * @see spec/v1/webhooks.md §"Signature scheme"
28
74
  */
package/vitest.config.ts CHANGED
@@ -17,5 +17,12 @@ export default defineConfig({
17
17
  // populates `lib/fixtures.ts` so `describe.skipIf(...)` predicates
18
18
  // see the cached set when scenarios register their tests.
19
19
  setupFiles: ['src/setup.ts'],
20
+ // RFC 0148 §A run-end disposition summary. `setup.ts` records an honest
21
+ // disposition per scenario file on EVERY run; before this, only
22
+ // `--certify` set OPENWOP_LEDGER_PATH, so on a plain `vitest run` the
23
+ // recording was computed and then discarded, leaving "N passed" as the
24
+ // sole artifact for rows the suite had classified `blocked`. This routes
25
+ // the ledger to a temp file and prints what did not witness.
26
+ globalSetup: ['src/global-setup.ts'],
20
27
  },
21
28
  });