@openwop/openwop-conformance 2.39.5 → 2.40.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,10 @@
1
1
  # `@openwop/openwop-conformance` Changelog
2
2
 
3
+ ## [2.40.0] — 2026-09-26 — RFC 0215: one subscription's dead receiver cannot hold another's delivery, and unregistering stops the retries
4
+
5
+ - **Two new scenarios, gated on `webhooks` (RFC 0215, `Active` 2026-09-25).** `v2-webhook-delivery-isolation` (`openwop.requirement.0215.no-head-of-line`) holds 8 subscriptions' attempts unanswered, then checks that a ninth subscription's attempt still starts. The 8 filter `run.started` and the ninth `run.completed` of a 2 s `conformance-delay` run, so contention exists before the healthy delivery falls due and the host's dispatch order cannot decide the row. It fails when the held attempts were still open at the due time and the healthy one waited for the host to release one. It records `partial-witness` when the host's own delivery timeout closed them first. `v2-webhook-unregister-stops-delivery` (`openwop.requirement.0215.unregister-stops-delivery`) answers `500` to two subscriptions on one run, unregisters one after its first attempt, and fails if that one is attempted more than 5 s after the `204`. It passes only when the other, the control, was retried after that point: a silence counts only when a retry demonstrably would have come. Sabotage on the v2 reference host: a sequential worker and a 5-slot pool each fail the first scenario, and an attempt that falls back to a cached subscription after the delete fails the second.
6
+ - **Suite `2.40.0`**: 561 scenario files, a minor because two are added. `@openwop/spec-artifacts` moves in lockstep at the same exact pin, and ships the new `webhooks.md` §Durability text and the two invariant rows' threat-model home.
7
+
3
8
  ## [2.39.5] — 2026-09-26 — RFC 0213 §B has a deterministic witness, and a host front whose URLs follow the request origin verifies
4
9
 
5
10
  - **RFC 0213 §B has a deterministic witness: `0213.in-flight-refused-under-hold`.** On a host whose create answers in milliseconds the unaided `v2-idempotency-in-flight` leg records `partial-witness` — same-key creates never overlap. The new leg arms the seams-profile seam `armIdempotencyHold` (`POST /conformance/seams/sample/test/idempotency/hold`, host-sample-test-seams.md §26), which holds the caller's next real create's claim in flight; a concurrent same-key create MUST then be refused `409 idempotency_in_flight` by the host's own path (no retry timing in `details`), and the held create still wins. Seams not advertised ⇒ `inapplicable`; advertised but the path unserved ⇒ `blocked`.
package/README.md CHANGED
@@ -11,7 +11,7 @@
11
11
  # --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
12
12
  # default resolver refuses. npm 10.9 fails outright with
13
13
  # "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
14
- npm install --legacy-peer-deps @openwop/openwop-conformance@2.39.5 @openwop/spec-artifacts@2.39.5
14
+ npm install --legacy-peer-deps @openwop/openwop-conformance@2.40.0 @openwop/spec-artifacts@2.40.0
15
15
  # or run without install:
16
16
  npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
17
17
  ```
@@ -135,7 +135,7 @@ Exit code is non-zero on any failed assertion. `--certify` distinguishes: `0`
135
135
 
136
136
  ## What's Covered
137
137
 
138
- The current suite has 559 scenario files under `src/scenarios/`.
138
+ The current suite has 561 scenario files under `src/scenarios/`.
139
139
  - 2026-09-23 (suite 2.37.0 cycle, RFC 0213): NEW `v2-sse-last-event-id-cursor.test.ts` (a `Last-Event-ID` past the log is an exclusive cursor; a malformed id, when refused, is `400 validation_error`; the cursor never changes the answer for an unknown or foreign-tenant run — public test of `event-cursor-after-authorization`), `v2-idempotency-in-flight.test.ts` (five concurrent same-key creates yield one run; each loser is a marked replay or `409 idempotency_in_flight` with no retry timing in `details`; `partial-witness` when no loser was refused in flight) and `v2-interrupt-resolve-terminal.test.ts` (a run-scoped resolve after cancel or completion is `409 interrupt_already_resolved`, never `interrupt_cancelled`). All three sit off the core-standard floor until measured on the three bundle hosts.
140
140
  - 2026-09-03 (suite `1.157.0 -> 1.158.0`, gap G17): NEW `idempotency-concurrent-claim.test.ts` — drives the new `host-sample-test-seams.md` §25 concurrent duplicate-delivery seam for the RFC 0150 §B / `idempotency.md` §"Concurrent duplicates (Layer 2)" atomic-claim MUST, which is unconditional and had no witness of any kind. Asserts every executor mints the SAME `logicalInvocationId` **before** asserting `delivered === 1` — without the identity check a host passes by minting different ids and never colliding, one effect because nothing raced. Not profile-gated and so not opt-out-able (the obligation is unconditional); an unmounted seam records `blocked`, which is not certifiable. Graduates `layer2-invocation-claim-atomic` reference-impl -> protocol.
141
141
  - 2026-08-19 (suite `1.137.0 → 1.138.0`): NEW `durability-poison-exhaustion.test.ts` — RFC 0158 §C.8, the FIRST row of that RFC's conformance table to land. Asserts what `failure-path.test.ts` cannot: not just that deterministically failing work reaches terminal, but that attempts STOP — counted on the log, re-counted after a scaled quiet window, asserted unchanged. A host still redelivering records more. Seam-gated on the existing event-log seam (`blocked` = unobservable, not unmet) and outside every profile floor.
@@ -481,7 +481,7 @@ Server-required (added in 1.7.0):
481
481
  | ------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
482
482
  | **Redaction** | [`capabilities.md`](../spec/v1/capabilities.md) §"Secrets" + NFR-7 + §"aiProviders" | Vendor-neutral assertions that the server doesn't leak secret material. Three scenario groups: (a) discovery shape contract — `secrets` + `aiProviders` advertisements are well-formed regardless of `secrets.supported`; when `supported === true`, scopes MUST be non-empty + `resolution === 'host-managed'`; `byok ⊆ supported`. (b) bearer-token redaction — invalid Bearer canary in `Authorization` header is not echoed in the 401 response body. (c) credentialRef echo control — gated on `secrets.supported === true`; canary planted in `configurable.ai.credentialRef` MUST NOT appear in any RunEvent payload (poll-based capture; transport-agnostic). Uses runtime-built canary fixtures (`lib/canaries.ts`) that defeat static secret scanners. 6 scenarios. |
483
483
 
484
- Current source tree: 559 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
484
+ Current source tree: 561 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
485
485
 
486
486
  ## Remaining Gaps
487
487
 
@@ -1,5 +1,5 @@
1
1
  {
2
2
  "package": "@openwop/spec-artifacts",
3
- "version": "2.39.5",
4
- "stampSha256": "db86d304e8d1c9042e51cc62a10c3d952826f9f3d8eddbcee08c29128c4e4252"
3
+ "version": "2.40.0",
4
+ "stampSha256": "f65c56d3a36eb7301c4baceef9ce0899091151fb12bc72c1508eb7a7d7b4ca4c"
5
5
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@openwop/openwop-conformance",
3
- "version": "2.39.5",
3
+ "version": "2.40.0",
4
4
  "description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
5
5
  "repository": {
6
6
  "type": "git",
@@ -56,6 +56,6 @@
56
56
  "@openwop/spec-artifacts": "file:../spec-artifacts"
57
57
  },
58
58
  "peerDependencies": {
59
- "@openwop/spec-artifacts": "2.39.5"
59
+ "@openwop/spec-artifacts": "2.40.0"
60
60
  }
61
61
  }
package/requirements.json CHANGED
@@ -2,11 +2,11 @@
2
2
  "$comment": "GENERATED by conformance/scripts/generate-requirement-registry.mjs — do not edit. One record per it()/test() in src/scenarios. Ids: openwop.it.<file-stem>.<title-slug>[~n] (src/lib/requirement-ids.ts). A record with id null has an interpolated title; its run-time row is keyed by the rendered title and maps here by file+line only. Renamed ids need a row in requirement-aliases.json.",
3
3
  "generatedFrom": "src/scenarios/*.test.ts",
4
4
  "counts": {
5
- "files": 633,
6
- "tests": 2425,
7
- "withStableId": 2425,
5
+ "files": 635,
6
+ "tests": 2427,
7
+ "withStableId": 2427,
8
8
  "interpolatedTitles": 0,
9
- "explicitIds": 2305
9
+ "explicitIds": 2307
10
10
  },
11
11
  "records": [
12
12
  {
@@ -33150,6 +33150,47 @@
33150
33150
  }
33151
33151
  ]
33152
33152
  },
33153
+ {
33154
+ "id": "openwop.requirement.0215.no-head-of-line",
33155
+ "file": "v2-webhook-delivery-isolation.test.ts",
33156
+ "line": 109,
33157
+ "title": null,
33158
+ "explicitId": "openwop.requirement.0215.no-head-of-line",
33159
+ "citations": [
33160
+ {
33161
+ "section": "webhooks.md §Surfaces",
33162
+ "requirement": "POST /webhooks MUST answer 201 { webhookId }"
33163
+ },
33164
+ {
33165
+ "section": "webhooks.md §Surfaces",
33166
+ "requirement": "the 201 body MUST carry `webhookId`"
33167
+ },
33168
+ {
33169
+ "section": "runs.md §Create",
33170
+ "requirement": null,
33171
+ "interpolated": true
33172
+ },
33173
+ {
33174
+ "section": "webhooks.md §Durability (RFC 0215 §A.1, §A.2)",
33175
+ "requirement": null,
33176
+ "interpolated": true
33177
+ },
33178
+ {
33179
+ "section": "webhooks.md §Durability (RFC 0215 §A.1, §A.2)",
33180
+ "requirement": null,
33181
+ "interpolated": true
33182
+ },
33183
+ {
33184
+ "section": "webhooks.md §Durability (RFC 0215 §A.1, §A.2)",
33185
+ "requirement": "the held subscriptions' attempts MUST have been dispatched"
33186
+ },
33187
+ {
33188
+ "section": "webhooks.md §Durability (RFC 0215 §A.1, §A.2)",
33189
+ "requirement": null,
33190
+ "interpolated": true
33191
+ }
33192
+ ]
33193
+ },
33153
33194
  {
33154
33195
  "id": "openwop.it.v2-webhook-delivery-shape.a-major-2-subscriber-receives-the-v2-rendering-the-delivery-validates-and-run-st",
33155
33196
  "file": "v2-webhook-delivery-shape.test.ts",
@@ -33640,6 +33681,37 @@
33640
33681
  }
33641
33682
  ]
33642
33683
  },
33684
+ {
33685
+ "id": "openwop.it.v2-webhook-unregister-stops-delivery.no-attempt-for-an-unregistered-subscription-starts-after-the-204-while-its-contr",
33686
+ "file": "v2-webhook-unregister-stops-delivery.test.ts",
33687
+ "line": 83,
33688
+ "title": "no attempt for an unregistered subscription starts after the 204, while its control keeps being retried",
33689
+ "explicitId": "openwop.requirement.0215.unregister-stops-delivery",
33690
+ "citations": [
33691
+ {
33692
+ "section": "webhooks.md §Surfaces",
33693
+ "requirement": "POST /webhooks MUST answer 201 { webhookId }"
33694
+ },
33695
+ {
33696
+ "section": "webhooks.md §Surfaces",
33697
+ "requirement": "the 201 body MUST carry `webhookId`"
33698
+ },
33699
+ {
33700
+ "section": "runs.md §Create",
33701
+ "requirement": null,
33702
+ "interpolated": true
33703
+ },
33704
+ {
33705
+ "section": "webhooks.md §Surfaces",
33706
+ "requirement": "DELETE /webhooks/{webhookId} MUST answer 204 for the caller's own subscription"
33707
+ },
33708
+ {
33709
+ "section": "webhooks.md §Durability (RFC 0215 §B)",
33710
+ "requirement": null,
33711
+ "interpolated": true
33712
+ }
33713
+ ]
33714
+ },
33643
33715
  {
33644
33716
  "id": "openwop.it.v2-well-known-one-resource.a-header-less-get-and-an-openwop-version-2-0-get-are-two-representations-of-one",
33645
33717
  "file": "v2-well-known-one-resource.test.ts",
@@ -1,9 +1,9 @@
1
1
  {
2
2
  "$comment": "GENERATED by conformance/scripts/generate-scenario-majors.mjs (RFC 0168 §D.3). Do not edit; add a file to BOTH_MAJORS in the generator to target both majors.",
3
3
  "counts": {
4
- "files": 559,
4
+ "files": 561,
5
5
  "v1": 449,
6
- "v2": 119
6
+ "v2": 121
7
7
  },
8
8
  "majors": {
9
9
  "a2a-1-0-agent-card.test.ts": [
@@ -1557,6 +1557,9 @@
1557
1557
  "v2-version-header-honored.test.ts": [
1558
1558
  2
1559
1559
  ],
1560
+ "v2-webhook-delivery-isolation.test.ts": [
1561
+ 2
1562
+ ],
1560
1563
  "v2-webhook-delivery-shape.test.ts": [
1561
1564
  2
1562
1565
  ],
@@ -1578,6 +1581,9 @@
1578
1581
  "v2-webhook-standard-webhooks-delivery.test.ts": [
1579
1582
  2
1580
1583
  ],
1584
+ "v2-webhook-unregister-stops-delivery.test.ts": [
1585
+ 2
1586
+ ],
1581
1587
  "v2-well-known-one-resource.test.ts": [
1582
1588
  2
1583
1589
  ],
@@ -1,17 +1,17 @@
1
1
  {
2
2
  "_comment": "Provenance of @openwop/spec-artifacts (RFC 0168 §D.2). files: SHA-256 per file; the conformance suite compares the installed peer against dist/spec-artifacts.lock.json at start.",
3
3
  "package": "@openwop/spec-artifacts",
4
- "version": "2.39.5",
5
- "corpusTag": "v2.39.5",
4
+ "version": "2.40.0",
5
+ "corpusTag": "v2.40.0",
6
6
  "files": {
7
7
  "api/.redocly.lint-ignore.yaml": "bf5a8350b88a72fa43f59605ed8d903ed24b6cfccda5e45509c9f6ed9ee4e712",
8
8
  "api/asyncapi.yaml": "d5ecb9ee6114582be3b1f662c84bfac9ae96dae7bacb853e461168f70a8e1c7d",
9
9
  "api/grpc/openwop.proto": "c3e72bb17cba514ee98feb6434e6c9b6ea6795bfd086489ec69fd882dd1ad977",
10
10
  "api/openapi.yaml": "6366b5514be71098e9ea41fc81ff52bc39db1a547b353c0b344cc2a2bf5370aa",
11
11
  "api/redocly.yaml": "b0604c89b2ca6d5076ec25725c539dad44a741a811fe524439ee6daef8baa09f",
12
- "api/seams-v2.yaml": "ed925b14e1e3e6f82ec0d0c844c035ae1a3b0dbcf10b3e53b96106327298f5cb",
13
- "api/v2/asyncapi.yaml": "a3ba4c688bd4ee33400ef23718eac3d03bc333df2b3d74644adc6f0b2b3a5b2e",
14
- "api/v2/openapi.yaml": "e4dd5dafff70be8c64de8344c3ae5dd5525515a069fef6f421a226813a305344",
12
+ "api/seams-v2.yaml": "acd23b627e68bee6c886f21f4b8252f0dcb49d9deb5d4abc4c19d390f56b426a",
13
+ "api/v2/asyncapi.yaml": "de935d89b72a1196c0f8f9731f1c3d32cee118e81b4271365e76ffdc5c9fced6",
14
+ "api/v2/openapi.yaml": "0d12490c1739defdb302ec384d9dd81306d1d865c7023fda040baa846ff35c8b",
15
15
  "api/v2/redocly.yaml": "1e66b60e6118ad11a823bb620678be464d99dfe50a40e3e6f93ec9429b88b34c",
16
16
  "schemas/README.md": "0c0b737ffcf8f30e7d2809cec8a498232de710f41443212922ad8337cdde0b51",
17
17
  "schemas/a2a-task-state.schema.json": "75d5049dea8bd873ff0e7546f1c60c8d36c219bec7084264360be54a8be30ae1",
@@ -204,13 +204,13 @@
204
204
  "schemas/workspace-file.schema.json": "464de85c2a068243084ee9c1d969bc7cd5d8f7948574e58450d6493c38a0e1e4",
205
205
  "spec/v1/alias-detectors.json": "2401fcb1c18cdd688c018b3d716ae6bca85c5e356220ee2b793bd9d81872412d",
206
206
  "spec/v1/capability-declaration-classes.json": "e7729aed5c4b4e1dd02abab0530f14cc95f5d4070fe51fb139e7f5cccefa00c6",
207
- "spec/v1/core-standard-manifest.json": "a2f4f866565b5da642b36384a235bf57a293366b7f9faeb0e7f62c3132337932",
207
+ "spec/v1/core-standard-manifest.json": "ca5a83168c0e6a4d73d22d32acdd4ca37bfc62297ad730df6698134caa99ebbb",
208
208
  "spec/v1/deprecations.json": "2d03f4729810280147ea08c630ea65434f2dc5370567d0aac41c3595f8337d40",
209
209
  "spec/v1/deprecations.schema.json": "3e393c405d2a41b467d8c5e3c468549078df1ce6a6d2588fc488097b95d9b55b",
210
210
  "spec/v1/event-codemap.json": "3da60d884157793a360da532a9fcbbfb5285636db325a74cec94b34622186d97",
211
211
  "spec/v1/event-codemap.schema.json": "d05933b2e88103aff51a2f774df97b9bda065e0fc88dbd9b114cc47a32189174",
212
212
  "spec/v1/extensions.json": "79a60754aa16cbdbdb604f8e5c00038af535a1690a4d07bd5ff7c38f0256a7bc",
213
- "spec/v1/gaps.json": "9424f94441ba5c4d1a8027e1f59be832f46af1a67ff0ca140adca9bb501ee993",
213
+ "spec/v1/gaps.json": "468d5b60f5e8b276f895833ac072c3984ab104d0ad97c4d31b757290c6077172",
214
214
  "spec/v1/gaps.schema.json": "8fd83259f556553c9df0f53e7a82ca8c2a4771d471197f69b4e2ae0ceeacfd66",
215
215
  "spec/v1/migrations.json": "8839497d6a4830b53fa7867014a8336f0d725ef2346352e54fb580b7947897af",
216
216
  "spec/v1/migrations.schema.json": "886779aa6c22e646db097f5df210adb018a4dd14a7b815465a18c8a7056c8f72",
@@ -240,7 +240,7 @@
240
240
  "spec/v2/core/security-defaults.md": "500471a8db7af9b776ac40b4a9a278c9f3d45e1c1dead880114e7fadc7bee37d",
241
241
  "spec/v2/core/tool-catalog.md": "35f1a3fd509db0fc0490ddf400686fe94dc4b20c98064d5ebe776da516153e60",
242
242
  "spec/v2/core/versioning.md": "0aded71d090c8358c3ce17763c120cfc1cc42531a5eaa1b8a2a7016ed9594069",
243
- "spec/v2/core/webhooks.md": "9c9169d775875f72982ed3506c1bfe27546ad05806d114af103dd7c28d4d5122",
243
+ "spec/v2/core/webhooks.md": "95880896e37e5b7096693a75203cf0ed70cefa09959230e17b45aa9193078eff",
244
244
  "spec/v2/core/workflow-chain-packs.md": "ee45d3fede3bb6f0cc6abcf0d6c929c7cd213a0fbe862b738a998f929c4d1102",
245
245
  "spec/v2/corrections.json": "cc74ffde74384b0f261a4f625871d922e8661d654d647a8534e21036595cb280",
246
246
  "spec/v2/corrections.schema.json": "4ac595b9a6d7f66d53de03fe0edbe0dc58de46426932387bfdc0ceb3cf949a3e",
@@ -292,9 +292,9 @@
292
292
  "spec/v2/path-manifest.json": "ae9b56d03a701061fd48ad0b73cb0373493f6a4547da92763b422295b5e19535",
293
293
  "spec/v2/peer-dependency-aliases.json": "d10299280abee08258502925bc327293ee413e0108cd6e6ec75ff6110653308d",
294
294
  "spec/v2/profiles.json": "0636f19fceae625390003a347e70ef4797d84766b5c24ce8a02cea52aadebca4",
295
- "spec/v2/release.json": "c3d35fbfae242c3fa3a504edb08fbdbc7ee9dd003250ea7225a0f6641dc614b9",
295
+ "spec/v2/release.json": "f17a7e0066d7851adfb246c586f12199d14fee60bf7a0eda203a124116dcff9e",
296
296
  "spec/v2/retention-floors.json": "eaf3722d95c79947af1d4269ef85117e126518c588cfcf1a2b21b97269f51624",
297
- "spec/v2/surface-baseline.json": "4273f2b75cfde4c620c995b5a4c334cfc15d936dfa7f5b6fa0aa4bf257e8b8b5"
297
+ "spec/v2/surface-baseline.json": "2ba53481583261c59008da86d1db82ed9f657675dfe2f8b019ec7d82f5bdb4b4"
298
298
  },
299
- "corpusCommit": "718fae2de800b73f48a59237cfb843578b0433f1"
299
+ "corpusCommit": "44b89e1e50f56dd2a0834771225b8e04d2bd9e6f"
300
300
  }
@@ -0,0 +1,200 @@
1
+ /**
2
+ * RFC 0215 §A — `webhook-delivery-isolation` (suite 2.40.0, target major 2; gated on `webhooks`).
3
+ *
4
+ * `spec/v2/core/webhooks.md` §Durability: a host MUST NOT make the start of an
5
+ * attempt to one subscription wait for an attempt to a DIFFERENT subscription to
6
+ * finish, and MUST sustain that while at least 8 subscriptions have attempts
7
+ * outstanding that their receivers have not answered (§A.1 + §A.2, one
8
+ * observation at the floor, so one requirement id).
9
+ *
10
+ * The shape (architect review, 2026-09-25): contention first, then the healthy
11
+ * delivery falls due — so the dispatch order the host chooses cannot decide the
12
+ * row. The RFC's first draft registered 9 subscriptions on one event and read a
13
+ * healthy attempt that beat the held ones as `partial-witness`; this file
14
+ * removes that case instead of recording it.
15
+ *
16
+ * - 8 HELD subscriptions filter `run.started`. Their receiver accepts each
17
+ * attempt and never answers, until the leg ends.
18
+ * - 1 HEALTHY subscription filters `run.completed`, and answers 204.
19
+ * - One `conformance-delay` run of `DELAY_MS`. `run.started` makes the held
20
+ * attempts due at t0; `run.completed` makes the healthy one due at
21
+ * t0 + DELAY_MS, by which time 8 attempts are outstanding.
22
+ *
23
+ * Verdicts, each on a positive observation:
24
+ * - pass — the healthy attempt ARRIVED while all 8 held attempts were open.
25
+ * - fail — the healthy attempt arrived only once fewer than 8 were open (the
26
+ * host released one to make room), or never within the window, AND the held
27
+ * attempts had stayed open until the run was terminal: the contention the
28
+ * floor names existed when the healthy delivery fell due, and it waited.
29
+ * - partial-witness — the host's own delivery timeout closed held attempts
30
+ * before the run was terminal. Contention was not sustained when the
31
+ * healthy delivery fell due, so this run of the leg cannot judge §A. The
32
+ * timeout is the host's to choose; a host with a timeout shorter than
33
+ * DELAY_MS is simply not measured by this instrument.
34
+ * - blocked — fewer than 8 held attempts ever arrived AND the healthy one
35
+ * never did either: nothing reached the suite, which says nothing about
36
+ * isolation (`noDeliveryCause`).
37
+ *
38
+ * A bounded pool of k < 9 opens k held attempts, queues the rest and the
39
+ * healthy one, and releases a slot only at its timeout — so it FAILS: its held
40
+ * attempts were open when the run completed, and the healthy attempt waited for
41
+ * one to close. That is the openwop-app WHD-1 defect (RFC 0215 §Motivation 1).
42
+ *
43
+ * @see spec/v2/core/webhooks.md §Durability
44
+ * @see RFCS/0215-webhook-delivery-isolation.md §A
45
+ * @see SECURITY/invariants.yaml `webhook-delivery-isolation`
46
+ */
47
+
48
+ import { afterEach, describe, it, expect } from 'vitest';
49
+ import type { ServerResponse } from 'node:http';
50
+ import { driver } from '../lib/driver.js';
51
+ import { v2Discovery, gateFamily } from '../lib/v2.js';
52
+ import { absenceIsUnmeasured, noDeliveryCause, startScopedReceiver, type ScopedReceiver } from '../lib/scoped-receiver.js';
53
+ import { readErrorCode } from '../lib/error-envelope.js';
54
+ import { blockedDespiteAssertions, softSkip } from '../lib/soft-skip.js';
55
+ import { req } from '../lib/requirement-ids.js';
56
+ import { RETRY_WAIT_FLOOR_MS } from '../lib/webhook-retry-window.js';
57
+
58
+ const ID = 'openwop.requirement.0215.no-head-of-line';
59
+ const DOC = 'webhooks.md §Durability (RFC 0215 §A.1, §A.2)';
60
+ const FIXTURE = 'conformance-delay';
61
+ /** §A.2's floor. Changing it is a spec change, not a tuning knob. */
62
+ const FLOOR = 8;
63
+ /**
64
+ * How long after `run.started` the healthy delivery falls due. Short, so a host
65
+ * whose delivery timeout is a few seconds still holds its 8 attempts open when
66
+ * it does; long enough that the held attempts are dispatched first on any host
67
+ * that dispatches at all.
68
+ */
69
+ const DELAY_MS = 2_000;
70
+ /** How long the leg waits for the healthy attempt once the run is terminal. */
71
+ const WINDOW_MS = RETRY_WAIT_FLOOR_MS;
72
+ const TERMINAL = new Set(['completed', 'failed', 'cancelled']);
73
+
74
+ interface Held { readonly webhookId: string; readonly res: ServerResponse; readonly openedAt: number; closedAt: number | null }
75
+
76
+ let receiver: ScopedReceiver | null = null;
77
+ const registered: string[] = [];
78
+ let release: (() => void) | null = null;
79
+ afterEach(async () => {
80
+ release?.();
81
+ release = null;
82
+ // RFC 0215 §B is what makes this cleanup work on a conforming host; on any
83
+ // host it stops this file's subscriptions outliving the exercise.
84
+ for (const id of registered.splice(0)) await driver.delete(`/webhooks/${encodeURIComponent(id)}`).catch(() => undefined);
85
+ if (receiver) { const rx = receiver; receiver = null; await rx.close(); }
86
+ });
87
+
88
+ async function waitTerminal(runId: string, timeoutMs: number): Promise<number | null> {
89
+ const deadline = Date.now() + timeoutMs;
90
+ for (;;) {
91
+ const res = await driver.get(`/runs/${encodeURIComponent(runId)}`);
92
+ const status = res.status === 200 ? String((res.json as { status?: unknown } | null)?.status ?? '') : '';
93
+ if (TERMINAL.has(status)) return Date.now();
94
+ if (Date.now() > deadline) return null;
95
+ await new Promise((r) => setTimeout(r, 100));
96
+ }
97
+ }
98
+
99
+ async function waitFor(pred: () => boolean, timeoutMs: number): Promise<boolean> {
100
+ const deadline = Date.now() + timeoutMs;
101
+ while (Date.now() < deadline) {
102
+ if (pred()) return true;
103
+ await new Promise((r) => setTimeout(r, 50));
104
+ }
105
+ return pred();
106
+ }
107
+
108
+ describe('RFC 0215 §A — one subscription\'s receiver does not hold another\'s delivery (gated on webhooks)', () => {
109
+ it(`a healthy subscription's attempt starts while ${FLOOR} other subscriptions' attempts are unanswered`, async () => {
110
+ let doc: Record<string, unknown> | null = null;
111
+ try { doc = await v2Discovery(); } catch { /* recorded below */ }
112
+ if (doc === null) return softSkip('blocked', 'discovery unreachable');
113
+ if (!(await gateFamily('webhooks'))) return softSkip('inapplicable', 'webhooks family not advertised (gate recorded under openwop.family.webhooks)');
114
+ if (!(Array.isArray(doc['fixtures']) && (doc['fixtures'] as unknown[]).includes(FIXTURE))) {
115
+ return softSkip('inapplicable', `${FIXTURE} fixture not advertised — no run whose completion falls due after its start`);
116
+ }
117
+
118
+ const held = new Map<string, Held>();
119
+ let healthyId: string | null = null;
120
+ let healthyAt: number | null = null;
121
+ let openAtHealthy = -1;
122
+ const openCount = (): number => [...held.values()].filter((h) => h.closedAt === null).length;
123
+
124
+ const rx = await startScopedReceiver((hit, res) => {
125
+ const webhookId = String(hit.headers['openwop-webhook-id'] ?? hit.headers['x-openwop-webhook-id'] ?? '');
126
+ if (webhookId !== '' && webhookId === healthyId) {
127
+ if (healthyAt === null) { healthyAt = Date.now(); openAtHealthy = openCount(); }
128
+ res.writeHead(204); res.end();
129
+ return;
130
+ }
131
+ if (webhookId === '' || held.has(webhookId) || !registered.includes(webhookId)) {
132
+ // A retry of a held attempt, or a stranger: answered, not counted as a new held attempt.
133
+ res.writeHead(204); res.end();
134
+ return;
135
+ }
136
+ const h: Held = { webhookId, res, openedAt: Date.now(), closedAt: null };
137
+ held.set(webhookId, h);
138
+ // The HOST closing the socket before we answer is the host abandoning the attempt.
139
+ res.on('close', () => { if (h.closedAt === null) h.closedAt = Date.now(); });
140
+ });
141
+ receiver = rx;
142
+ release = () => {
143
+ for (const h of held.values()) {
144
+ if (h.closedAt !== null) continue;
145
+ h.closedAt = Date.now();
146
+ try { h.res.writeHead(204); h.res.end(); } catch { /* host already gone */ }
147
+ }
148
+ };
149
+
150
+ const register = async (events: string[]): Promise<string | null> => {
151
+ const reg = await driver.post('/webhooks', { url: rx.url, events });
152
+ if (reg.status === 400 && readErrorCode(reg.json) === 'webhook_url_rejected' && !rx.tunnelled) return null;
153
+ expect(reg.status, req(ID, 'webhooks.md §Surfaces', 'POST /webhooks MUST answer 201 { webhookId }')).toBe(201);
154
+ const id = (reg.json as { webhookId?: unknown } | null)?.webhookId;
155
+ expect(typeof id, req(ID, 'webhooks.md §Surfaces', 'the 201 body MUST carry `webhookId`')).toBe('string');
156
+ registered.push(id as string);
157
+ return id as string;
158
+ };
159
+ for (let i = 0; i < FLOOR; i += 1) {
160
+ if ((await register(['run.started'])) === null) {
161
+ return blockedDespiteAssertions('host SSRF guard rejected the loopback receiver (webhooks.md §Egress requires it); set OPENWOP_WEBHOOK_RECEIVER_URL to a public https tunnel in front of the suite receiver');
162
+ }
163
+ }
164
+ healthyId = await register(['run.completed']);
165
+ if (healthyId === null) return blockedDespiteAssertions('host SSRF guard rejected the loopback receiver');
166
+
167
+ const create = await driver.post('/runs', { workflowId: FIXTURE, inputs: { delayMs: DELAY_MS } });
168
+ expect(create.status, req(ID, 'runs.md §Create', `POST /runs MUST answer 201 for ${FIXTURE}`)).toBe(201);
169
+ const runId = (create.json as { runId: string }).runId;
170
+ const terminalAt = await waitTerminal(runId, DELAY_MS + 30_000);
171
+ if (terminalAt === null) return blockedDespiteAssertions(`${FIXTURE} run ${runId} did not reach a terminal status — the healthy delivery never fell due`);
172
+ const openAtTerminal = openCount();
173
+ const everOpened = held.size;
174
+ await waitFor(() => healthyAt !== null, WINDOW_MS);
175
+ const earliestClose = Math.min(...[...held.values()].map((h) => h.closedAt ?? Infinity));
176
+ const detail = `${everOpened} held attempt(s) arrived; ${openAtTerminal} were still open when the run was seen terminal; `
177
+ + (healthyAt === null ? `the healthy attempt did not arrive within ${WINDOW_MS}ms of that` : `the healthy attempt arrived ${healthyAt - terminalAt}ms after it, with ${openAtHealthy} held open`);
178
+
179
+ if (healthyAt === null && everOpened === 0) {
180
+ if (absenceIsUnmeasured(rx)) return blockedDespiteAssertions(noDeliveryCause(rx, 'webhook attempt'));
181
+ return blockedDespiteAssertions(`no attempt for any of the ${FLOOR + 1} subscriptions arrived — ${noDeliveryCause(rx, 'webhook attempt')}`);
182
+ }
183
+ if (healthyAt !== null && openAtHealthy >= FLOOR) {
184
+ expect(openAtHealthy, req(ID, DOC, `an attempt to one subscription MUST NOT wait for attempts to other subscriptions — ${detail}`)).toBeGreaterThanOrEqual(FLOOR);
185
+ } else if (openAtTerminal >= FLOOR) {
186
+ // The healthy attempt waited (or never came), and the contention §A.2
187
+ // names was present when it fell due.
188
+ expect.fail(req(ID, DOC, `with ${FLOOR} subscriptions' attempts unanswered, a further subscription's attempt MUST still start; it waited for the host to release one — ${detail}`));
189
+ } else if (everOpened >= FLOOR || earliestClose < terminalAt) {
190
+ // Contention was not sustained to the due time. Either the host's own
191
+ // delivery timeout released the held attempts early (its choice), or it
192
+ // never opened FLOOR of them at once. The second is the defect, the first
193
+ // is not, and they differ in WHEN the first held attempt closed.
194
+ expect(everOpened, req(ID, DOC, 'the held subscriptions\' attempts MUST have been dispatched')).toBeGreaterThan(0);
195
+ return softSkip('skipped', `partial-witness: the host closed held attempts ${Number.isFinite(earliestClose) ? `${terminalAt - earliestClose}ms ` : ''}before the run was terminal (its delivery timeout is shorter than this leg's ${DELAY_MS}ms delay), so ${FLOOR} attempts were not outstanding when the healthy delivery fell due — ${detail}`);
196
+ } else {
197
+ expect.fail(req(ID, DOC, `the host never had ${FLOOR} subscriptions' attempts outstanding at once, and the healthy attempt waited — a bounded dispatcher below the floor — ${detail}`));
198
+ }
199
+ }, DELAY_MS + 30_000 + RETRY_WAIT_FLOOR_MS + 30_000);
200
+ });
@@ -0,0 +1,176 @@
1
+ /**
2
+ * RFC 0215 §B — `webhook-unregister-stops-delivery` (suite 2.40.0, target major 2; gated on `webhooks`).
3
+ *
4
+ * `spec/v2/core/webhooks.md` §Durability: after `unregisterWebhook` answers
5
+ * `204`, a host MUST NOT start any further attempt for that subscription,
6
+ * including attempts already scheduled for retry. An attempt the host had begun
7
+ * sending before the `204` MAY complete.
8
+ *
9
+ * Absence is not evidence on its own (architect review, 2026-09-25). "No retry
10
+ * arrived after the 204" proves nothing about a host whose next retry falls
11
+ * outside the window, and `webhooks.retryPolicy` carries no interval, so the
12
+ * suite cannot compute when that is. The leg therefore runs a CONTROL:
13
+ *
14
+ * - two subscriptions on one receiver, both filtering `run.completed`, both
15
+ * answered `500` on every attempt, so both have a retry scheduled;
16
+ * - the TARGET is unregistered the moment its first attempt has been
17
+ * answered; the CONTROL is left alone;
18
+ * - the control's retries are what the target's would have been — same
19
+ * event, same moment, same policy.
20
+ *
21
+ * Verdicts:
22
+ * - pass — the control was retried after `204 + GRACE_MS` and the target
23
+ * was not: a retry that would have come did not.
24
+ * - fail — an attempt for the target arrived after `204 + GRACE_MS`.
25
+ * - partial-witness — neither had an attempt after `204 + GRACE_MS` within
26
+ * the window: the schedule finished inside the grace, or runs past the
27
+ * window. Nothing was contradicted and nothing was shown.
28
+ * - inapplicable — `retryPolicy.maxAttempts: 1`: there is no retry to stop.
29
+ *
30
+ * GRACE_MS is an instrument setting, not a spec number: webhooks.md states the
31
+ * in-flight allowance as "an attempt the host had begun sending", and the
32
+ * receiver keys on ARRIVAL, so the grace only has to cover transit of a request
33
+ * already on the wire (RFC 0215 Unresolved question 3).
34
+ *
35
+ * @see spec/v2/core/webhooks.md §Durability
36
+ * @see RFCS/0215-webhook-delivery-isolation.md §B
37
+ * @see SECURITY/invariants.yaml `webhook-unregister-stops-delivery`
38
+ */
39
+
40
+ import { afterEach, describe, it, expect } from 'vitest';
41
+ import { driver } from '../lib/driver.js';
42
+ import { v2Discovery, gateFamily } from '../lib/v2.js';
43
+ import { absenceIsUnmeasured, noDeliveryCause, startScopedReceiver, type ScopedReceiver } from '../lib/scoped-receiver.js';
44
+ import { readErrorCode } from '../lib/error-envelope.js';
45
+ import { blockedDespiteAssertions, softSkip } from '../lib/soft-skip.js';
46
+ import { req } from '../lib/requirement-ids.js';
47
+ import { retryWaitCapMs, retryWaitFor } from '../lib/webhook-retry-window.js';
48
+
49
+ const ID = 'openwop.requirement.0215.unregister-stops-delivery';
50
+ const DOC = 'webhooks.md §Durability (RFC 0215 §B)';
51
+ const FIXTURE = 'conformance-noop';
52
+ const GRACE_MS = 5_000;
53
+ /** Once the control has been retried past the grace, how long to keep listening for the target's matching retry. */
54
+ const SETTLE_MS = 5_000;
55
+ const CAP_MS = retryWaitCapMs();
56
+ const TERMINAL = new Set(['completed', 'failed', 'cancelled']);
57
+
58
+ let receiver: ScopedReceiver | null = null;
59
+ const registered: string[] = [];
60
+ afterEach(async () => {
61
+ for (const id of registered.splice(0)) await driver.delete(`/webhooks/${encodeURIComponent(id)}`).catch(() => undefined);
62
+ if (receiver) { const rx = receiver; receiver = null; await rx.close(); }
63
+ });
64
+
65
+ function advertisedRetryPolicy(doc: Record<string, unknown>): { maxAttempts?: number; backoff?: string } | null {
66
+ const read = (holder: unknown): { maxAttempts?: number; backoff?: string } | null => {
67
+ const rp = holder && typeof holder === 'object' ? (holder as { retryPolicy?: unknown }).retryPolicy : undefined;
68
+ return rp && typeof rp === 'object' ? (rp as { maxAttempts?: number; backoff?: string }) : null;
69
+ };
70
+ return read(doc['webhooks']) ?? read(doc['triggerBridge']);
71
+ }
72
+
73
+ async function waitFor(pred: () => boolean, timeoutMs: number): Promise<boolean> {
74
+ const deadline = Date.now() + timeoutMs;
75
+ while (Date.now() < deadline) {
76
+ if (pred()) return true;
77
+ await new Promise((r) => setTimeout(r, 100));
78
+ }
79
+ return pred();
80
+ }
81
+
82
+ describe('RFC 0215 §B — unregistering stops the attempts (gated on webhooks)', () => {
83
+ it('no attempt for an unregistered subscription starts after the 204, while its control keeps being retried', async () => {
84
+ let doc: Record<string, unknown> | null = null;
85
+ try { doc = await v2Discovery(); } catch { /* recorded below */ }
86
+ if (doc === null) return softSkip('blocked', 'discovery unreachable');
87
+ if (!(await gateFamily('webhooks'))) return softSkip('inapplicable', 'webhooks family not advertised (gate recorded under openwop.family.webhooks)');
88
+ if (!(Array.isArray(doc['fixtures']) && (doc['fixtures'] as unknown[]).includes(FIXTURE))) {
89
+ return softSkip('inapplicable', `${FIXTURE} fixture not advertised — no run to deliver`);
90
+ }
91
+ const policy = advertisedRetryPolicy(doc);
92
+ if (policy?.maxAttempts === 1) return softSkip('inapplicable', 'webhooks.retryPolicy.maxAttempts is 1 — the host schedules no retry, so there is none for unregistering to stop');
93
+ const windowMs = retryWaitFor(policy, CAP_MS);
94
+
95
+ const arrivals = new Map<string, number[]>();
96
+ let targetId: string | null = null;
97
+ let unregisteredAt: number | null = null;
98
+ let unregisterStatus = 0;
99
+ let unregistering: Promise<void> | null = null;
100
+
101
+ const rx = await startScopedReceiver((hit, res) => {
102
+ const webhookId = String(hit.headers['openwop-webhook-id'] ?? hit.headers['x-openwop-webhook-id'] ?? '');
103
+ const list = arrivals.get(webhookId) ?? [];
104
+ list.push(Date.now());
105
+ arrivals.set(webhookId, list);
106
+ res.writeHead(500);
107
+ res.end();
108
+ // Unregister the target once its first attempt has been ANSWERED, so the
109
+ // host has a retry scheduled and nothing of its own in flight.
110
+ if (webhookId === targetId && unregistering === null) {
111
+ unregistering = (async () => {
112
+ const del = await driver.delete(`/webhooks/${encodeURIComponent(webhookId)}`);
113
+ unregisterStatus = del.status;
114
+ if (del.status === 204) {
115
+ unregisteredAt = Date.now();
116
+ const i = registered.indexOf(webhookId);
117
+ if (i >= 0) registered.splice(i, 1);
118
+ }
119
+ })();
120
+ }
121
+ });
122
+ receiver = rx;
123
+
124
+ const register = async (): Promise<string | null> => {
125
+ const reg = await driver.post('/webhooks', { url: rx.url, events: ['run.completed'] });
126
+ if (reg.status === 400 && readErrorCode(reg.json) === 'webhook_url_rejected' && !rx.tunnelled) return null;
127
+ expect(reg.status, req(ID, 'webhooks.md §Surfaces', 'POST /webhooks MUST answer 201 { webhookId }')).toBe(201);
128
+ const id = (reg.json as { webhookId?: unknown } | null)?.webhookId;
129
+ expect(typeof id, req(ID, 'webhooks.md §Surfaces', 'the 201 body MUST carry `webhookId`')).toBe('string');
130
+ registered.push(id as string);
131
+ return id as string;
132
+ };
133
+ const controlId = await register();
134
+ targetId = await register();
135
+ if (controlId === null || targetId === null) {
136
+ return blockedDespiteAssertions('host SSRF guard rejected the loopback receiver (webhooks.md §Egress requires it); set OPENWOP_WEBHOOK_RECEIVER_URL to a public https tunnel in front of the suite receiver');
137
+ }
138
+ const target = targetId;
139
+
140
+ const create = await driver.post('/runs', { workflowId: FIXTURE });
141
+ expect(create.status, req(ID, 'runs.md §Create', `POST /runs MUST answer 201 for ${FIXTURE}`)).toBe(201);
142
+ const runId = (create.json as { runId: string }).runId;
143
+ const deadline = Date.now() + 30_000;
144
+ for (;;) {
145
+ const r = await driver.get(`/runs/${encodeURIComponent(runId)}`);
146
+ if (r.status === 200 && TERMINAL.has(String((r.json as { status?: unknown } | null)?.status ?? ''))) break;
147
+ if (Date.now() > deadline) return blockedDespiteAssertions(`${FIXTURE} run ${runId} did not reach a terminal status — no run.completed to deliver`);
148
+ await new Promise((r2) => setTimeout(r2, 250));
149
+ }
150
+
151
+ const firstTarget = await waitFor(() => (arrivals.get(target)?.length ?? 0) > 0, windowMs);
152
+ if (!firstTarget) {
153
+ if (absenceIsUnmeasured(rx)) return blockedDespiteAssertions(noDeliveryCause(rx, 'run.completed attempt'));
154
+ return blockedDespiteAssertions(`the target subscription's first attempt never arrived, so there was nothing to unregister after — ${noDeliveryCause(rx, 'run.completed attempt')}`);
155
+ }
156
+ await waitFor(() => unregistering !== null, 5_000);
157
+ if (unregistering !== null) await unregistering;
158
+ expect(unregisterStatus, req(ID, 'webhooks.md §Surfaces', 'DELETE /webhooks/{webhookId} MUST answer 204 for the caller\'s own subscription')).toBe(204);
159
+ const cutoff = (unregisteredAt as number | null ?? Date.now()) + GRACE_MS;
160
+
161
+ const after = (id: string): number[] => (arrivals.get(id) ?? []).filter((t) => t > cutoff);
162
+ await waitFor(() => after(controlId).length > 0 || after(target).length > 0, windowMs);
163
+ if (after(controlId).length > 0) await new Promise((r) => setTimeout(r, SETTLE_MS));
164
+
165
+ const targetLate = after(target);
166
+ const controlLate = after(controlId);
167
+ const detail = `target ${target}: ${arrivals.get(target)?.length ?? 0} attempt(s), ${targetLate.length} after the 204 + ${GRACE_MS}ms; control ${controlId}: ${arrivals.get(controlId)?.length ?? 0} attempt(s), ${controlLate.length} after that point`;
168
+ expect(
169
+ targetLate.length,
170
+ req(ID, DOC, `after unregisterWebhook answers 204 the host MUST NOT start another attempt for that subscription, including retries already scheduled — ${detail}`),
171
+ ).toBe(0);
172
+ if (controlLate.length === 0) {
173
+ return softSkip('skipped', `partial-witness: the control subscription was not retried after the 204 + ${GRACE_MS}ms within ${windowMs}ms either, so the target's silence shows nothing — the schedule finished inside the grace or runs past the window (raise OPENWOP_WEBHOOK_RETRY_WAIT_MS above the host's backoff sum) — ${detail}`);
174
+ }
175
+ }, CAP_MS * 2 + 90_000);
176
+ });