@openwop/openwop-conformance 2.34.0 → 2.34.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/README.md +3 -1
- package/dist/lib/scenario-disposition.js +4 -0
- package/dist/lib/soft-skip.js +27 -1
- package/dist/spec-artifacts.lock.json +2 -2
- package/package.json +2 -2
- package/requirements.json +21 -9
- package/schemas/CORPUS-STAMP.json +4 -4
- package/src/lib/durability-watch.ts +29 -5
- package/src/lib/scenario-disposition.ts +4 -1
- package/src/lib/soft-skip.ts +30 -4
- package/src/lib/webhook-retry-window.ts +56 -0
- package/src/scenarios/v2-durability-recovery.test.ts +10 -1
- package/src/scenarios/v2-webhook-durable-delivery.test.ts +71 -25
- package/src/setup.ts +17 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,24 @@
|
|
|
1
1
|
# `@openwop/openwop-conformance` Changelog
|
|
2
2
|
|
|
3
|
+
## [2.34.2] — 2026-09-21 — a host that died on schedule failed the rows that were watching it recover
|
|
4
|
+
|
|
5
|
+
- **The RFC 0158 kill rows read the host through two calls that did not survive the host dying.** `waitBack()` treated a refused connection as "down, keep waiting"; the watch's `readStatus` / `readLog` called `driver.get` bare. A seam may answer and *then* die — `during-execution` dies at the first `node.started`, seconds after its response — so `waitBack` saw a 200 from the still-living process, the process died mid-watch, and `fetch failed` escaped the scenario as an **`executed-fail` — for recovering.** Measured by a tier-1 host at its exact production image under a real restart supervisor, behind a port proxy that accepts and then closes: four of five rows `executed-fail` on `UND_ERR_SOCKET: other side closed`, reproduced twice; the one that passed was the row whose kill lands before the seam answers.
|
|
6
|
+
- **A transport failure inside the watch is now an unreadable observation, never a verdict:** a status that cannot be read is `null`, a log that cannot be read is `readable: false` — both already meant "conclude nothing, poll again" — and the budget still bounds the wait, so a host that never comes back still records no resumption. `Watch.transportErrors` counts them. Unit-tested in `lib/durability-watch.test.ts` (dies after answering and comes back → resumption observed; never comes back → none; an unreadable read never latches completed-un-re-executed).
|
|
7
|
+
- **One kill no longer contaminates the rest of the run.** Every seam row now waits for the host to answer at all before probing the seam, and the bound read retries once across a restart. Suite setup retried discovery twice back-to-back, giving a restarting host about a millisecond: each file set up during the restart logged "discovery unreadable" and recorded its fixture gates `blocked`. Setup now waits out a transport failure (refused / reset / closed) for up to 45 s, 1 s apart, without spending an attempt; an HTTP answer and a 20 s timeout still count as attempts, so a host that is up and wrong is not retried into looking right.
|
|
8
|
+
- False FAIL only. Suite patch; corpus release stays `2.34.0`. Reported by openwop-app.
|
|
9
|
+
|
|
10
|
+
## [2.34.1] — 2026-09-21 — a 90 s window convicted a host for retrying at its production pace
|
|
11
|
+
|
|
12
|
+
- **`v2-webhook-durable-delivery` capped every wait at a hard 90 s, and a conformant host with a slower schedule could not advertise `webhooks.deadLetter` without failing its bundle.** A host retrying at 15 / 30 / 60 / 120 s with `maxAttempts: 5` makes its attempts at t0, +15, +45, +105, +225 s. The dead-letter leg waited for a second attempt, waited 90 s more, and read the sink ~120 s before that host exhausts: **`0173.webhook-durable-delivery.dead-letter` → `executed-fail`, "an exhausted delivery MUST be routed to the sink, not dropped"**, about a delivery still in flight; `0188.dead-letter-content-free` → `blocked`, which denies certification. To pass, the host would have had to cut its production retry window from ~225 s to ~75 s for every real subscriber — the instrument choosing the host's durability. Reported by a tier-2 host from its own constants *before* advertising the facet; reproduced on the reference host configured to the same schedule.
|
|
13
|
+
- **The window is now operator-RAISABLE and never lowerable:** `OPENWOP_WEBHOOK_RETRY_WAIT_MS` (default 90 000 per wait, at most 3 600 000). A smaller or non-numeric value is ignored, so no operator can shrink the window to hide a slow retry, and a raised window is still bounded — a host that never retries still fails, later. The `it` timeouts derive from it, as they already did from the constant. Same shape as `OPENWOP_DURABILITY_OBSERVATION_CEILING_MS`; both are now documented in the README, which had neither.
|
|
14
|
+
- **The dead-letter leg no longer asserts "not dropped" before exhaustion was observable.** Fewer attempts than the advertised `maxAttempts` AND nothing in the sink is a window that closed early *or* a host that stopped retrying — indistinguishable without an interval on the wire (`webhooks.retryPolicy` is closed over `{ maxAttempts, backoff }`) — so it records **`blocked`**, naming the variable and the numbers the run used, never `executed-fail`. **A host that reaches `maxAttempts` and has nothing in the sink still FAILS.** `blocked` denies certification exactly as the false fail did; what changes is that the row stops saying something untrue about the host.
|
|
15
|
+
- **`blocked` had to be made to stand, and two rows had no id — both found by measuring the fix, not by reading it.** (1) A leg that asserts and *then* soft-skips records `executed-pass` with a `partial-witness:` detail (`resolveItRecord`): the acceptance predicate refuses it, **certification counts it as a pass**. Both dead-letter legs assert inside `register()` before they can observe anything, so the first draft of this fix turned a false FAIL into a certifying pass that never looked at the sink. New opt-in `blockedDespiteAssertions(reason)` (`lib/soft-skip.ts`) records a `blocked` that survives setup assertions; the ordinary soft-skip convention is unchanged everywhere else. (2) `register()` asserts under the base `0173.webhook-durable-delivery` id and `req()` is last-wins, so a leg that returned before naming its own id recorded under the base id — **`0173…dead-letter` and `0188.dead-letter-content-free` then had no row in the bundle at all.** That was already true of `0188.dead-letter-content-free` on 2.34.0 and earlier whenever the sink was empty: it read as a partial-witness pass of the base row, not the `blocked` it appeared to be. Each leg now names its row first. Measured on the reference host at 15 / 30 / 60 / 120 s: both rows `blocked` under their own ids; at the host's default schedule: all `executed-pass`, same ids; with the window raised to 300 s: all `executed-pass`, exhaustion observed at 226 s.
|
|
16
|
+
- The dead-letter leg also observes before it asserts: it waits, reads the sink and deletes the subscription, returns on the inconclusive case, and only then asserts the obligations in their original order. A create failure still fails at once.
|
|
17
|
+
- **The cost, stated:** a host that advertises 5 attempts, stops at 3 and drops the delivery used to read `executed-fail` and now reads `blocked`. Neither certifies. Detection of that case comes back only by putting the interval on the wire — normative surface, an RFC, not a patch.
|
|
18
|
+
- Pure window logic moved to `lib/webhook-retry-window.ts` with unit tests (raisable, not lowerable, bounded, floor unchanged for a host advertising no policy).
|
|
19
|
+
- `spec/v2/core/persistence.md`'s Stable banner now lists RFC 0158 §A–§D — owed since the RFC's flip, which could not edit a file shipped inside the published 2.34.0.
|
|
20
|
+
- False FAIL only. Suite patch: corpus release stays `2.34.0`.
|
|
21
|
+
|
|
3
22
|
## [2.34.0] — 2026-09-21 — the rung and the recovery bound, in the bundle RFC 0158 said they were in
|
|
4
23
|
|
|
5
24
|
- **RFC 0158 §E publishes a host's rung and recovery bound in its certification bundle instead of in discovery — and the bundle had no seat for either.** `certification-bundle.schema.json` is closed; a reader saw five pass/fail rows and could neither tell which rung was claimed nor recompute the bound, whose terms lived only on a non-normative seam route. Found by reading the RFC's first acceptance criterion at the moment of flipping it, with every row green.
|
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
# --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
|
|
12
12
|
# default resolver refuses. npm 10.9 fails outright with
|
|
13
13
|
# "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
|
|
14
|
-
npm install --legacy-peer-deps @openwop/openwop-conformance@2.34.
|
|
14
|
+
npm install --legacy-peer-deps @openwop/openwop-conformance@2.34.2 @openwop/spec-artifacts@2.34.2
|
|
15
15
|
# or run without install:
|
|
16
16
|
npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
|
|
17
17
|
```
|
|
@@ -110,6 +110,8 @@ Run `npm run test` for normal CI cadence; `npm run test:strict` when claiming fu
|
|
|
110
110
|
| `OPENWOP_WEBHOOK_ALLOW_PRIVATE=true` | Relaxes the webhook egress guard for the loopback test receiver used by `webhook-signed-delivery.test.ts`, `webhook-negative.test.ts`, and `replay-fanout-suppression.test.ts`. **The receiver is `http://127.0.0.1:{port}/`, which `webhooks.md` forbids three separate times, and the opt-in MUST relax all three to be witnessable:** (1) the **scheme** check — §"SSRF protection" bullet 1 rejects non-`https://`, and this gate fires FIRST on a host that validates scheme before address; (2) the **registration-time** address check — §"SSRF protection"; and (3) **delivery-time re-resolution** — §"Delivery-time egress validation (RFC 0093)". Gates 2 and 3 are independent MUSTs at different layers, so an opt-in reaching only one layer **cannot** produce a witness: delivery-only leaves registration returning `400 webhook_url_rejected`, registration-only leaves the dispatcher refusing to connect. A host whose opt-in reaches one layer is **not** non-conformant — it cannot witness these scenarios, which is a property of the test posture and not of its webhook signing. The scheme gate went unstated here until 2026-08-25; a tier-2 host validating scheme-then-address was blocked by a gate the documented contract never named. Relaxing gates 2 and 3 is **test-only posture** — see `SECURITY/threat-model-secret-leakage.md` §4.9 for why a registration-time relaxation is the more dangerous of the two. When the host rejects, the scenario records `blocked` (RFC 0148 §A), not a pass. |
|
|
111
111
|
| `OPENWOP_WEBHOOK_RECEIVER_URL=<https-url>` | **The route to a webhook witness that relaxes nothing.** A host refusing the loopback receiver on *two* independent grounds — private address **and** non-`https:` — cannot be unblocked by `OPENWOP_WEBHOOK_ALLOW_PRIVATE`, because the scheme arm still stands. This supplies **the public `https:` front for THIS SUITE'S OWN receiver** (tunnel / TLS-terminating proxy), never an arbitrary endpoint: the scenario asserts on what *this process* received, so pointing registration elsewhere makes every header assertion vacuous while the row turns green. Zero deliveries with it set is a **hard failure**, never a skip. Preferred over the flag — it waives nothing and writes no durable subscription row aimed at a private address (`SECURITY/threat-model-secret-leakage.md` §4.9). Pair with `OPENWOP_WEBHOOK_RECEIVER_PORT`. |
|
|
112
112
|
| `OPENWOP_WEBHOOK_RECEIVER_PORT=<port>` | Pins the in-process webhook receiver to a known port instead of an ephemeral one. Required in practice by `OPENWOP_WEBHOOK_RECEIVER_URL`: a tunnel must be aimed at a port known **in advance**, and the receiver binds `0` by default. Unset ⇒ ephemeral, as before. |
|
|
113
|
+
| `OPENWOP_WEBHOOK_RETRY_WAIT_MS=<ms>` | **Raises** how long `v2-webhook-durable-delivery` waits for a retry schedule to play out (default `90000`, per wait; at most `3600000`). `webhooks.retryPolicy` carries `{ maxAttempts, backoff }` and no interval, so the suite cannot derive how long exhaustion takes: set this above the SUM of your backoff intervals (a host retrying at 15/30/60/120 s needs > 225 s). **It can be raised, never lowered** — a smaller or non-numeric value is ignored — and the window stays bounded, so a host that never retries still fails. A window that closes before the advertised `maxAttempts` arrive records `blocked` naming this variable, never a conviction. |
|
|
114
|
+
| `OPENWOP_DURABILITY_OBSERVATION_CEILING_MS=<ms>` | **Raises** how long the RFC 0158 kill rows observe for resumption (default `240000`). A host with a long *declared* recovery bound — a 750 s dispatch lease is conformant — sets this above its bound and waits it out. Raisable, never lowerable. |
|
|
113
115
|
| `OPENWOP_MCP_FAKE_SERVER=true` | Boots the synthetic MCP peer for `mcp-tool-roundtrip.test.ts`. |
|
|
114
116
|
| `OPENWOP_MCP_REAL_SERVER_URL=<base-url>` | Points the MCP wire-shape probe at a real MCP server. The probe POSTs JSON-RPC and reads a single-JSON response — matches MCP's `streamable-http` transport in single-response mode. **Does NOT support** stdio transport (which is what most `modelcontextprotocol/servers` references default to) or SSE-streamed responses; an operator collecting interop evidence today runs a custom `StreamableHTTPServerTransport`-style server that returns a single JSON body per request. Adding SSE-frame parsing is tracked in `docs/PROTOCOL-GAP-CLOSURE-PLAN.md` Track 6. Assertions relax to shape-only. When both this and `OPENWOP_MCP_FAKE_SERVER` are set, the real URL wins. Phase 3 T3.4 interop-evidence path. |
|
|
115
117
|
| `OPENWOP_A2A_FAKE_PEER=true` | Boots the synthetic A2A peer for `a2a-task-roundtrip.test.ts`. |
|
|
@@ -100,6 +100,10 @@ export function resolveItRecord(state, assertionCalls, gate, noted, firstError)
|
|
|
100
100
|
if (state === 'fail')
|
|
101
101
|
return { disposition: 'executed-fail', detail: `the test executed and failed: ${(firstError ?? 'no message').slice(0, 300)}` };
|
|
102
102
|
if (state === 'pass' && assertionCalls > 0) {
|
|
103
|
+
// `blockedDespiteAssertions` (soft-skip.ts): the leg says its setup
|
|
104
|
+
// assertions are not the requirement, and the requirement went unobserved.
|
|
105
|
+
if (noted !== null && noted.kind === 'blocked' && noted.conclusive === true)
|
|
106
|
+
return { disposition: 'blocked', detail: noted.reason };
|
|
103
107
|
// A leg that asserted AND THEN soft-skipped is only a partial witness, and
|
|
104
108
|
// the file-level record has always said so (`resolveFileRecord` below).
|
|
105
109
|
// This `it`-level record dropped the note — and the `it`-level rows are the
|
package/dist/lib/soft-skip.js
CHANGED
|
@@ -74,6 +74,32 @@ export function seamAbsent(reason) {
|
|
|
74
74
|
}
|
|
75
75
|
return softSkip('blocked', reason);
|
|
76
76
|
}
|
|
77
|
+
/**
|
|
78
|
+
* `blocked` that STANDS even though the test already asserted something.
|
|
79
|
+
*
|
|
80
|
+
* A plain `softSkip` after an assertion records `executed-pass` with a
|
|
81
|
+
* `partial-witness:` detail (`resolveItRecord`): the acceptance predicate
|
|
82
|
+
* refuses such a row, but certification counts it as a pass. That is right for
|
|
83
|
+
* a leg that finished its requirement and skipped an optional extra. It is
|
|
84
|
+
* wrong for a leg whose REQUIREMENT went unobserved after setup assertions it
|
|
85
|
+
* could not avoid — a helper like `register()` asserts `201` before the leg has
|
|
86
|
+
* observed anything. Use this only there: the row records `blocked`, which
|
|
87
|
+
* denies certification (RFC 0168 §E.1) without convicting the host.
|
|
88
|
+
*
|
|
89
|
+
* Opt-in and per-call on purpose. Honouring every note-after-assertion as
|
|
90
|
+
* `blocked` would downgrade legs that legitimately completed; that is a suite-
|
|
91
|
+
* wide semantic change, not a patch. First use: `v2-webhook-durable-delivery`'s
|
|
92
|
+
* dead-letter leg when the retry window closes before exhaustion (2.34.1).
|
|
93
|
+
*/
|
|
94
|
+
export function blockedDespiteAssertions(reason) {
|
|
95
|
+
const file = currentFile();
|
|
96
|
+
if (file === null)
|
|
97
|
+
return undefined;
|
|
98
|
+
const arr = notes.get(file) ?? [];
|
|
99
|
+
arr.push({ kind: 'blocked', reason, seq: ++seq, conclusive: true });
|
|
100
|
+
notes.set(file, arr);
|
|
101
|
+
return undefined;
|
|
102
|
+
}
|
|
77
103
|
const RANK = { blocked: 0, skipped: 1, inapplicable: 2 };
|
|
78
104
|
function fold(arr) {
|
|
79
105
|
if (arr.length === 0)
|
|
@@ -84,7 +110,7 @@ function fold(arr) {
|
|
|
84
110
|
uniq.push(n);
|
|
85
111
|
const kind = [...uniq].sort((a, b) => RANK[a.kind] - RANK[b.kind])[0].kind;
|
|
86
112
|
const reason = uniq.map((n) => (uniq.length > 1 ? `[${n.kind}] ${n.reason}` : n.reason)).join('; ');
|
|
87
|
-
return { kind, reason };
|
|
113
|
+
return uniq.some((n) => n.conclusive === true) ? { kind, reason, conclusive: true } : { kind, reason };
|
|
88
114
|
}
|
|
89
115
|
/**
|
|
90
116
|
* The noted disposition for a file, worst-first when mixed (`blocked` beats
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openwop/openwop-conformance",
|
|
3
|
-
"version": "2.34.
|
|
3
|
+
"version": "2.34.2",
|
|
4
4
|
"description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -56,6 +56,6 @@
|
|
|
56
56
|
"@openwop/spec-artifacts": "file:../spec-artifacts"
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
|
-
"@openwop/spec-artifacts": "2.34.
|
|
59
|
+
"@openwop/spec-artifacts": "2.34.2"
|
|
60
60
|
}
|
|
61
61
|
}
|
package/requirements.json
CHANGED
|
@@ -26650,7 +26650,7 @@
|
|
|
26650
26650
|
{
|
|
26651
26651
|
"id": "openwop.it.v2-durability-recovery.accepted-work-survives-a-kill-before-dispatch-and-dispatches-on-resume",
|
|
26652
26652
|
"file": "v2-durability-recovery.test.ts",
|
|
26653
|
-
"line":
|
|
26653
|
+
"line": 292,
|
|
26654
26654
|
"title": "accepted work survives a kill before dispatch and dispatches on resume",
|
|
26655
26655
|
"explicitId": "openwop.requirement.0158.kill-after-accept",
|
|
26656
26656
|
"citations": [
|
|
@@ -26664,7 +26664,7 @@
|
|
|
26664
26664
|
{
|
|
26665
26665
|
"id": "openwop.it.v2-durability-recovery.work-executing-at-a-real-process-death-is-never-reported-complete-and-resumes",
|
|
26666
26666
|
"file": "v2-durability-recovery.test.ts",
|
|
26667
|
-
"line":
|
|
26667
|
+
"line": 346,
|
|
26668
26668
|
"title": "work executing at a real process death is never reported complete, and resumes",
|
|
26669
26669
|
"explicitId": "openwop.requirement.0158.kill-during-execution",
|
|
26670
26670
|
"citations": [
|
|
@@ -26683,7 +26683,7 @@
|
|
|
26683
26683
|
{
|
|
26684
26684
|
"id": "openwop.it.v2-durability-recovery.the-same-accepted-work-delivered-twice-fires-each-effect-exactly-once",
|
|
26685
26685
|
"file": "v2-durability-recovery.test.ts",
|
|
26686
|
-
"line":
|
|
26686
|
+
"line": 402,
|
|
26687
26687
|
"title": "the same accepted work delivered twice fires each effect exactly once",
|
|
26688
26688
|
"explicitId": "openwop.requirement.0158.duplicate-delivery",
|
|
26689
26689
|
"citations": [
|
|
@@ -26702,7 +26702,7 @@
|
|
|
26702
26702
|
{
|
|
26703
26703
|
"id": "openwop.it.v2-durability-recovery.the-declared-recovery-bound-is-derived-from-the-mechanism-that-enforces-it",
|
|
26704
26704
|
"file": "v2-durability-recovery.test.ts",
|
|
26705
|
-
"line":
|
|
26705
|
+
"line": 476,
|
|
26706
26706
|
"title": "the declared recovery bound is derived from the mechanism that enforces it",
|
|
26707
26707
|
"explicitId": "openwop.requirement.0158.bound-is-derived",
|
|
26708
26708
|
"citations": [
|
|
@@ -26716,7 +26716,7 @@
|
|
|
26716
26716
|
{
|
|
26717
26717
|
"id": "openwop.it.v2-durability-recovery.deterministically-failing-work-reaches-a-terminal-state-and-stops-being-retried",
|
|
26718
26718
|
"file": "v2-durability-recovery.test.ts",
|
|
26719
|
-
"line":
|
|
26719
|
+
"line": 507,
|
|
26720
26720
|
"title": "deterministically failing work reaches a terminal state and stops being retried",
|
|
26721
26721
|
"explicitId": "openwop.requirement.0158.poison-exhaustion",
|
|
26722
26722
|
"citations": [
|
|
@@ -29507,7 +29507,7 @@
|
|
|
29507
29507
|
{
|
|
29508
29508
|
"id": "openwop.it.v2-webhook-durable-delivery.a-failed-attempt-is-retried-and-the-event-is-delivered-at-least-once",
|
|
29509
29509
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29510
|
-
"line":
|
|
29510
|
+
"line": 235,
|
|
29511
29511
|
"title": "a failed attempt is retried and the event is delivered at least once",
|
|
29512
29512
|
"explicitId": "openwop.requirement.0173.webhook-durable-delivery",
|
|
29513
29513
|
"citations": [
|
|
@@ -29538,10 +29538,18 @@
|
|
|
29538
29538
|
{
|
|
29539
29539
|
"id": "openwop.it.v2-webhook-durable-delivery.an-exhausted-delivery-is-dead-lettered-never-dropped",
|
|
29540
29540
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29541
|
-
"line":
|
|
29541
|
+
"line": 323,
|
|
29542
29542
|
"title": "an exhausted delivery is dead-lettered, never dropped",
|
|
29543
29543
|
"explicitId": "openwop.requirement.0173.webhook-durable-delivery.dead-letter",
|
|
29544
29544
|
"citations": [
|
|
29545
|
+
{
|
|
29546
|
+
"section": "webhooks.md §Durability",
|
|
29547
|
+
"requirement": "an exhausted delivery MUST be routed to the sink, not dropped"
|
|
29548
|
+
},
|
|
29549
|
+
{
|
|
29550
|
+
"section": "runs.md §Create",
|
|
29551
|
+
"requirement": "POST /runs MUST answer 201 for the noop fixture"
|
|
29552
|
+
},
|
|
29545
29553
|
{
|
|
29546
29554
|
"section": "runs.md §Create",
|
|
29547
29555
|
"requirement": "POST /runs MUST answer 201 for the noop fixture"
|
|
@@ -29564,7 +29572,7 @@
|
|
|
29564
29572
|
{
|
|
29565
29573
|
"id": "openwop.it.v2-webhook-durable-delivery.the-dead-letter-read-is-served-and-its-records-carry-no-payload",
|
|
29566
29574
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29567
|
-
"line":
|
|
29575
|
+
"line": 417,
|
|
29568
29576
|
"title": "the dead-letter read is served and its records carry no payload",
|
|
29569
29577
|
"explicitId": "openwop.requirement.0188.dead-letter-read",
|
|
29570
29578
|
"citations": [
|
|
@@ -29577,10 +29585,14 @@
|
|
|
29577
29585
|
{
|
|
29578
29586
|
"id": "openwop.it.v2-webhook-durable-delivery.a-dead-letter-record-carries-no-delivered-payload",
|
|
29579
29587
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29580
|
-
"line":
|
|
29588
|
+
"line": 438,
|
|
29581
29589
|
"title": "a dead-letter record carries no delivered payload",
|
|
29582
29590
|
"explicitId": "openwop.requirement.0188.dead-letter-content-free",
|
|
29583
29591
|
"citations": [
|
|
29592
|
+
{
|
|
29593
|
+
"section": "RFC 0188 §B.1",
|
|
29594
|
+
"requirement": "a dead-letter record MUST NOT carry the delivered body, headers or subscription secret"
|
|
29595
|
+
},
|
|
29584
29596
|
{
|
|
29585
29597
|
"section": "RFC 0188 §B.1",
|
|
29586
29598
|
"requirement": null,
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_comment": "Provenance of @openwop/spec-artifacts (RFC 0168 §D.2). files: SHA-256 per file; the conformance suite compares the installed peer against dist/spec-artifacts.lock.json at start.",
|
|
3
3
|
"package": "@openwop/spec-artifacts",
|
|
4
|
-
"version": "2.34.
|
|
4
|
+
"version": "2.34.2",
|
|
5
5
|
"corpusTag": "v2.34.0",
|
|
6
6
|
"files": {
|
|
7
7
|
"api/.redocly.lint-ignore.yaml": "bf5a8350b88a72fa43f59605ed8d903ed24b6cfccda5e45509c9f6ed9ee4e712",
|
|
@@ -201,7 +201,7 @@
|
|
|
201
201
|
"schemas/workspace-file.schema.json": "464de85c2a068243084ee9c1d969bc7cd5d8f7948574e58450d6493c38a0e1e4",
|
|
202
202
|
"spec/v1/alias-detectors.json": "40069d5976eeb6ba1384a648e57e5cfd673db3fce36175115fb8293bee9664d4",
|
|
203
203
|
"spec/v1/capability-declaration-classes.json": "e7729aed5c4b4e1dd02abab0530f14cc95f5d4070fe51fb139e7f5cccefa00c6",
|
|
204
|
-
"spec/v1/core-standard-manifest.json": "
|
|
204
|
+
"spec/v1/core-standard-manifest.json": "67e8053f7355380ba2b7a1b3f56dfb761de38565ee9cdd9a86302307cb956736",
|
|
205
205
|
"spec/v1/deprecations.json": "307083ce29c23fd406015951f99a30d78d6187ff061d38dc9732f62191b40f3f",
|
|
206
206
|
"spec/v1/deprecations.schema.json": "18c87e78bedc210431f795ae44c5b5d202f2f3317850d5cf86867d4f1fa1cdfb",
|
|
207
207
|
"spec/v1/event-codemap.json": "3da60d884157793a360da532a9fcbbfb5285636db325a74cec94b34622186d97",
|
|
@@ -229,7 +229,7 @@
|
|
|
229
229
|
"spec/v2/core/interrupt.md": "3a926d84b942d4ee7daccc4443c93dc29f6a69f2936ba813182f2c1336db2cbb",
|
|
230
230
|
"spec/v2/core/overview.md": "7dc00ace32d4e4ce929a383651f0029fc4b7091ec87487b1bf26ead1be209e0e",
|
|
231
231
|
"spec/v2/core/packs.md": "1c0919059760f9a3940a93264a76b7e30f9a0f6f27ff19753cc88f15753d8088",
|
|
232
|
-
"spec/v2/core/persistence.md": "
|
|
232
|
+
"spec/v2/core/persistence.md": "9785e536362ec2000dab7fee9847543240d40a907745efc70cf2c8b4c1b8344e",
|
|
233
233
|
"spec/v2/core/replay.md": "b980a66e2543107240e3dc4f58131b9f4ff52478737e79ade6d5777b26ad1119",
|
|
234
234
|
"spec/v2/core/runs.md": "136319842bacbbb300178980e4b83444917d5b193cc5fdd92e39a4902ba8d3c0",
|
|
235
235
|
"spec/v2/core/security-defaults.md": "2a54ceaa4cef02c26ee0fbcf44f822ceeea79537c7d5a51ed300f21aa4fea576",
|
|
@@ -282,5 +282,5 @@
|
|
|
282
282
|
"spec/v2/release.json": "eaf652e3a01f02a1d9c4eee7d7ea2c2a0396fe5694b5dce4cb01ca2af2891cc5",
|
|
283
283
|
"spec/v2/retention-floors.json": "eaf3722d95c79947af1d4269ef85117e126518c588cfcf1a2b21b97269f51624"
|
|
284
284
|
},
|
|
285
|
-
"corpusCommit": "
|
|
285
|
+
"corpusCommit": "1a06f6f3c070c043a959f82ed3e91e22d26c658e"
|
|
286
286
|
}
|
|
@@ -29,7 +29,7 @@
|
|
|
29
29
|
* Readers are injected so the ordering itself is testable without a host.
|
|
30
30
|
*/
|
|
31
31
|
export interface Observation { readonly readable: boolean; readonly runStarted: number; readonly nodeStarted: number; readonly restored: number }
|
|
32
|
-
export interface Watch { readonly resumedAfterMs: number | null; readonly last: Observation; readonly completedUnresumed: boolean; readonly waitedMs: number }
|
|
32
|
+
export interface Watch { readonly resumedAfterMs: number | null; readonly last: Observation; readonly completedUnresumed: boolean; readonly waitedMs: number; readonly transportErrors: number }
|
|
33
33
|
export interface WatchIo {
|
|
34
34
|
readStatus(): Promise<string | null>;
|
|
35
35
|
readLog(): Promise<Observation>;
|
|
@@ -37,17 +37,41 @@ export interface WatchIo {
|
|
|
37
37
|
sleep(ms: number): Promise<void>;
|
|
38
38
|
}
|
|
39
39
|
|
|
40
|
+
const UNREADABLE: Observation = { readable: false, runStarted: 0, nodeStarted: 0, restored: 0 };
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* A TRANSPORT failure is an unreadable observation, never a verdict (2.34.2).
|
|
44
|
+
*
|
|
45
|
+
* The kill rows are the one place the suite causes the host to disappear, and a
|
|
46
|
+
* seam may kill at a point AFTER it has answered - `during-execution` dies at
|
|
47
|
+
* the first `node.started`, which can land seconds later. `waitBack()` already
|
|
48
|
+
* treated a refused connection as "down, keep waiting"; these two reads did
|
|
49
|
+
* not, so a process that died mid-watch threw `fetch failed` out of the
|
|
50
|
+
* scenario and failed the row - for recovering. Measured on a tier-1 host at
|
|
51
|
+
* its production image, behind a port proxy that accepts and then closes: four
|
|
52
|
+
* of five RFC 0158 rows `executed-fail` on `UND_ERR_SOCKET: other side closed`,
|
|
53
|
+
* twice. A status that cannot be read is `null`; a log that cannot be read is
|
|
54
|
+
* `readable: false` - both already mean "conclude nothing, poll again", and the
|
|
55
|
+
* budget still bounds the wait, so a host that never comes back still fails.
|
|
56
|
+
*/
|
|
57
|
+
async function orNull<T>(read: () => Promise<T>, fallback: T): Promise<{ value: T; failed: boolean }> {
|
|
58
|
+
try { return { value: await read(), failed: false }; } catch { return { value: fallback, failed: true }; }
|
|
59
|
+
}
|
|
60
|
+
|
|
40
61
|
export async function watchForResumption(io: WatchIo, budgetMs: number, resumed: (o: Observation) => boolean, pollMs = 500): Promise<Watch> {
|
|
41
62
|
const t0 = io.now();
|
|
42
63
|
let completedUnresumed = false;
|
|
64
|
+
let transportErrors = 0;
|
|
43
65
|
for (;;) {
|
|
44
|
-
const
|
|
45
|
-
const
|
|
66
|
+
const st = await orNull(() => io.readStatus(), null); // FIRST
|
|
67
|
+
const lg = await orNull(() => io.readLog(), UNREADABLE); // SECOND — never older than the status it is judged against
|
|
68
|
+
if (st.failed || lg.failed) transportErrors++;
|
|
69
|
+
const status = st.value; const last = lg.value;
|
|
46
70
|
const waitedMs = io.now() - t0;
|
|
47
71
|
const isResumed = last.readable && resumed(last);
|
|
48
72
|
if (status === 'completed' && last.readable && !isResumed) completedUnresumed = true;
|
|
49
|
-
if (isResumed) return { resumedAfterMs: waitedMs, last, completedUnresumed, waitedMs };
|
|
50
|
-
if (waitedMs >= budgetMs) return { resumedAfterMs: null, last, completedUnresumed, waitedMs };
|
|
73
|
+
if (isResumed) return { resumedAfterMs: waitedMs, last, completedUnresumed, waitedMs, transportErrors };
|
|
74
|
+
if (waitedMs >= budgetMs) return { resumedAfterMs: null, last, completedUnresumed, waitedMs, transportErrors };
|
|
51
75
|
await io.sleep(pollMs);
|
|
52
76
|
}
|
|
53
77
|
}
|
|
@@ -102,11 +102,14 @@ export function resolveItRecord(
|
|
|
102
102
|
state: FileTestState,
|
|
103
103
|
assertionCalls: number,
|
|
104
104
|
gate: { disposition: 'inapplicable' | 'skipped'; detail?: string } | undefined,
|
|
105
|
-
noted: { kind: 'inapplicable' | 'skipped' | 'blocked'; reason: string } | null,
|
|
105
|
+
noted: { kind: 'inapplicable' | 'skipped' | 'blocked'; reason: string; conclusive?: true } | null,
|
|
106
106
|
firstError?: string,
|
|
107
107
|
): { disposition: Disposition; detail?: string } {
|
|
108
108
|
if (state === 'fail') return { disposition: 'executed-fail', detail: `the test executed and failed: ${(firstError ?? 'no message').slice(0, 300)}` };
|
|
109
109
|
if (state === 'pass' && assertionCalls > 0) {
|
|
110
|
+
// `blockedDespiteAssertions` (soft-skip.ts): the leg says its setup
|
|
111
|
+
// assertions are not the requirement, and the requirement went unobserved.
|
|
112
|
+
if (noted !== null && noted.kind === 'blocked' && noted.conclusive === true) return { disposition: 'blocked', detail: noted.reason };
|
|
110
113
|
// A leg that asserted AND THEN soft-skipped is only a partial witness, and
|
|
111
114
|
// the file-level record has always said so (`resolveFileRecord` below).
|
|
112
115
|
// This `it`-level record dropped the note — and the `it`-level rows are the
|
package/src/lib/soft-skip.ts
CHANGED
|
@@ -43,7 +43,7 @@ export type SoftSkipKind = 'inapplicable' | 'skipped' | 'blocked';
|
|
|
43
43
|
/** Detail marker the runner writes for a zero-assertion file that noted nothing. */
|
|
44
44
|
export const UNCLASSIFIED_RETURN_DETAIL = 'every test returned early with zero assertions and no recorded reason — unclassified return; RFC 0148 §A resolves it to blocked (add softSkip(kind, reason) at the early return)';
|
|
45
45
|
|
|
46
|
-
interface Note { readonly kind: SoftSkipKind; readonly reason: string; readonly seq: number }
|
|
46
|
+
interface Note { readonly kind: SoftSkipKind; readonly reason: string; readonly seq: number; readonly conclusive?: true }
|
|
47
47
|
|
|
48
48
|
const notes = new Map<string, Note[]>();
|
|
49
49
|
let seq = 0;
|
|
@@ -83,15 +83,41 @@ export function seamAbsent(reason: string): undefined {
|
|
|
83
83
|
return softSkip('blocked', reason);
|
|
84
84
|
}
|
|
85
85
|
|
|
86
|
+
/**
|
|
87
|
+
* `blocked` that STANDS even though the test already asserted something.
|
|
88
|
+
*
|
|
89
|
+
* A plain `softSkip` after an assertion records `executed-pass` with a
|
|
90
|
+
* `partial-witness:` detail (`resolveItRecord`): the acceptance predicate
|
|
91
|
+
* refuses such a row, but certification counts it as a pass. That is right for
|
|
92
|
+
* a leg that finished its requirement and skipped an optional extra. It is
|
|
93
|
+
* wrong for a leg whose REQUIREMENT went unobserved after setup assertions it
|
|
94
|
+
* could not avoid — a helper like `register()` asserts `201` before the leg has
|
|
95
|
+
* observed anything. Use this only there: the row records `blocked`, which
|
|
96
|
+
* denies certification (RFC 0168 §E.1) without convicting the host.
|
|
97
|
+
*
|
|
98
|
+
* Opt-in and per-call on purpose. Honouring every note-after-assertion as
|
|
99
|
+
* `blocked` would downgrade legs that legitimately completed; that is a suite-
|
|
100
|
+
* wide semantic change, not a patch. First use: `v2-webhook-durable-delivery`'s
|
|
101
|
+
* dead-letter leg when the retry window closes before exhaustion (2.34.1).
|
|
102
|
+
*/
|
|
103
|
+
export function blockedDespiteAssertions(reason: string): undefined {
|
|
104
|
+
const file = currentFile();
|
|
105
|
+
if (file === null) return undefined;
|
|
106
|
+
const arr = notes.get(file) ?? [];
|
|
107
|
+
arr.push({ kind: 'blocked', reason, seq: ++seq, conclusive: true });
|
|
108
|
+
notes.set(file, arr);
|
|
109
|
+
return undefined;
|
|
110
|
+
}
|
|
111
|
+
|
|
86
112
|
const RANK: Record<SoftSkipKind, number> = { blocked: 0, skipped: 1, inapplicable: 2 };
|
|
87
113
|
|
|
88
|
-
function fold(arr: readonly Note[]): { kind: SoftSkipKind; reason: string } | null {
|
|
114
|
+
function fold(arr: readonly Note[]): { kind: SoftSkipKind; reason: string; conclusive?: true } | null {
|
|
89
115
|
if (arr.length === 0) return null;
|
|
90
116
|
const uniq: Note[] = [];
|
|
91
117
|
for (const n of arr) if (!uniq.some((u) => u.kind === n.kind && u.reason === n.reason)) uniq.push(n);
|
|
92
118
|
const kind = [...uniq].sort((a, b) => RANK[a.kind] - RANK[b.kind])[0]!.kind;
|
|
93
119
|
const reason = uniq.map((n) => (uniq.length > 1 ? `[${n.kind}] ${n.reason}` : n.reason)).join('; ');
|
|
94
|
-
return { kind, reason };
|
|
120
|
+
return uniq.some((n) => n.conclusive === true) ? { kind, reason, conclusive: true } : { kind, reason };
|
|
95
121
|
}
|
|
96
122
|
|
|
97
123
|
/**
|
|
@@ -113,7 +139,7 @@ export function softSkipMark(): number {
|
|
|
113
139
|
* The noted disposition for a file counting only notes written AFTER `mark`
|
|
114
140
|
* — the notes of the test that is ending. Same fold as the file rule.
|
|
115
141
|
*/
|
|
116
|
-
export function softSkipDispositionSince(file: string, mark: number): { kind: SoftSkipKind; reason: string } | null {
|
|
142
|
+
export function softSkipDispositionSince(file: string, mark: number): { kind: SoftSkipKind; reason: string; conclusive?: true } | null {
|
|
117
143
|
return fold((notes.get(file) ?? []).filter((n) => n.seq > mark));
|
|
118
144
|
}
|
|
119
145
|
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* How long a webhook scenario waits for a host's retry schedule to play out.
|
|
3
|
+
*
|
|
4
|
+
* The advertised `webhooks.retryPolicy` facet is closed over exactly
|
|
5
|
+
* `{ maxAttempts, backoff }`: a host has NO way to put its intervals on the
|
|
6
|
+
* wire, so the suite cannot derive how long exhaustion takes and has to choose
|
|
7
|
+
* a window. Every window it has chosen so far has convicted a durable host:
|
|
8
|
+
*
|
|
9
|
+
* 2.0.1 a hard 20 s failed a host whose first backoff was 30 s.
|
|
10
|
+
* 2.34.1 a hard 90 s cap fails a host retrying at 15 / 30 / 60 / 120 s with
|
|
11
|
+
* `maxAttempts: 5` — attempts at t0, +15, +45, +105, +225 s. The
|
|
12
|
+
* dead-letter leg read the sink ~120 s before that host exhausts, and
|
|
13
|
+
* recorded `executed-fail` ("MUST be routed to the sink, not dropped")
|
|
14
|
+
* about a host that delivers, retries five times and dead-letters
|
|
15
|
+
* correctly. To pass it would have had to cut its PRODUCTION retry
|
|
16
|
+
* window from ~225 s to ~75 s for every real subscriber. Reported by a
|
|
17
|
+
* tier-2 host before it advertised the facet, from its own constants.
|
|
18
|
+
*
|
|
19
|
+
* An instrument must not choose a host's durability. So the cap is
|
|
20
|
+
* OPERATOR-RAISABLE — the shape `OPENWOP_DURABILITY_OBSERVATION_CEILING_MS`
|
|
21
|
+
* already has for RFC 0158's kill rows — and NEVER LOWERABLE: a value below the
|
|
22
|
+
* default, or not a number, is ignored, so no operator can shrink the window to
|
|
23
|
+
* hide a slow retry. A raised window is still bounded, so a host that never
|
|
24
|
+
* retries still fails; it only fails later.
|
|
25
|
+
*/
|
|
26
|
+
export const RETRY_WAIT_FLOOR_MS = 20_000;
|
|
27
|
+
export const DEFAULT_RETRY_WAIT_CAP_MS = 90_000;
|
|
28
|
+
/** A typo guard, not a policy: an hour per wait is longer than any retry schedule worth certifying in one sitting. */
|
|
29
|
+
export const MAX_RETRY_WAIT_CAP_MS = 3_600_000;
|
|
30
|
+
export const RETRY_WAIT_ENV = 'OPENWOP_WEBHOOK_RETRY_WAIT_MS';
|
|
31
|
+
|
|
32
|
+
export function retryWaitCapMs(env: Record<string, string | undefined> = process.env): number {
|
|
33
|
+
const raw = Number(env[RETRY_WAIT_ENV]);
|
|
34
|
+
if (!Number.isFinite(raw) || raw < DEFAULT_RETRY_WAIT_CAP_MS) return DEFAULT_RETRY_WAIT_CAP_MS;
|
|
35
|
+
return Math.min(Math.floor(raw), MAX_RETRY_WAIT_CAP_MS);
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/**
|
|
39
|
+
* The floor stays 20 s so a host that advertises nothing is measured exactly as
|
|
40
|
+
* before; an advertised `exponential` / `fixed` backoff widens it to the cap.
|
|
41
|
+
*/
|
|
42
|
+
export function retryWaitFor(policy: { backoff?: string } | null, capMs: number): number {
|
|
43
|
+
if (policy === null) return RETRY_WAIT_FLOOR_MS;
|
|
44
|
+
const backoff = String(policy.backoff ?? '');
|
|
45
|
+
return backoff === 'exponential' || backoff === 'fixed' ? capMs : RETRY_WAIT_FLOOR_MS;
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
/** What a row says when its window closed before the host's schedule did. Computed, so the numbers a host reads are the ones the run used. */
|
|
49
|
+
export function windowClosedNote(seen: number, maxAttempts: number, waitedMs: number, capMs: number): string {
|
|
50
|
+
const raised = capMs > DEFAULT_RETRY_WAIT_CAP_MS;
|
|
51
|
+
return `${seen} of the advertised ${maxAttempts} attempts arrived inside the ${waitedMs}ms this scenario waits, and the delivery is not in the sink yet. `
|
|
52
|
+
+ 'webhooks.retryPolicy carries only { maxAttempts, backoff } — no interval — so the suite cannot tell a slow conformant schedule from a host that stopped retrying, and it does not convict on a deadline it chose. '
|
|
53
|
+
+ (raised
|
|
54
|
+
? `${RETRY_WAIT_ENV} is already raised to ${capMs}ms; set it above the SUM of this host's backoff intervals (at most ${MAX_RETRY_WAIT_CAP_MS}ms).`
|
|
55
|
+
: `Set ${RETRY_WAIT_ENV} above the SUM of this host's backoff intervals (default ${DEFAULT_RETRY_WAIT_CAP_MS}ms; it can be raised, never lowered) and re-run.`);
|
|
56
|
+
}
|
|
@@ -127,6 +127,12 @@ type SeamState =
|
|
|
127
127
|
* would terminate a host that never claimed the rung.
|
|
128
128
|
*/
|
|
129
129
|
async function probeSeam(): Promise<SeamState> {
|
|
130
|
+
// 2.34.2: the rows before this one KILLED the host. A row whose watch ran out
|
|
131
|
+
// its budget can hand over a host that is still restarting, and a bare GET
|
|
132
|
+
// then threw `fetch failed` and failed a row that never touched the kill -
|
|
133
|
+
// four rows of one measured cut were contaminated by the first. Wait for the
|
|
134
|
+
// service to answer at all (immediately, when it is up), then probe.
|
|
135
|
+
await waitBack(scaledTimeoutMs(RESUME_WINDOW_MS));
|
|
130
136
|
const r = await driver.get(KILL_SEAM);
|
|
131
137
|
if (r.status === 404 || r.status === 405) {
|
|
132
138
|
return { kind: 'absent', why: `no RFC 0158 durability seam at ${KILL_SEAM} (HTTP ${r.status}) — §E: the seam advertises nothing and a host that never runs the durability exercises exposes no such route, so this host claims no durable-execution rung` };
|
|
@@ -236,7 +242,10 @@ async function declaredBoundMs(fired: unknown): Promise<{ ms: number; declared:
|
|
|
236
242
|
// interval against a bound that does not govern the staged run.
|
|
237
243
|
const governing = (fired as { recoveryBoundMs?: unknown } | null)?.recoveryBoundMs;
|
|
238
244
|
if (typeof governing === 'number' && Number.isFinite(governing) && governing > 0) return { ms: governing, declared: true, recoveryClass };
|
|
239
|
-
|
|
245
|
+
// A seam may answer and THEN die (during-execution dies at the first
|
|
246
|
+
// node.started), so this read can meet a closed socket: wait the restart out
|
|
247
|
+
// once and read again rather than fail the row on transport (2.34.2).
|
|
248
|
+
const r = await driver.get('/host/durability/bound').catch(async () => { await waitBack(scaledTimeoutMs(RESUME_WINDOW_MS)); return driver.get('/host/durability/bound'); });
|
|
240
249
|
const bound = (r.json as { bound?: unknown } | null)?.bound;
|
|
241
250
|
return r.status === 200 && typeof bound === 'number' && Number.isFinite(bound) && bound > 0
|
|
242
251
|
? { ms: bound, declared: true, recoveryClass }
|
|
@@ -34,8 +34,9 @@ import { v2Discovery, gateFamily } from '../lib/v2.js';
|
|
|
34
34
|
import { projectBoundId } from '../lib/bound-id.js';
|
|
35
35
|
import { receiverBinding, resolveRegistrationUrl } from '../lib/webhook-receiver.js';
|
|
36
36
|
import { readErrorCode } from '../lib/error-envelope.js';
|
|
37
|
-
import { softSkip } from '../lib/soft-skip.js';
|
|
37
|
+
import { blockedDespiteAssertions, softSkip } from '../lib/soft-skip.js';
|
|
38
38
|
import { req } from '../lib/requirement-ids.js';
|
|
39
|
+
import { retryWaitCapMs, retryWaitFor, windowClosedNote } from '../lib/webhook-retry-window.js';
|
|
39
40
|
|
|
40
41
|
const FIXTURE = 'conformance-noop';
|
|
41
42
|
const TERMINAL = new Set(['completed', 'failed', 'cancelled']);
|
|
@@ -167,14 +168,14 @@ function advertisedRetryPolicy(doc: Record<string, unknown>): { maxAttempts?: nu
|
|
|
167
168
|
* which covers a 30 s first attempt with room for the second. The cap is
|
|
168
169
|
* deliberate: unbounded waiting would let a host that never retries hold the
|
|
169
170
|
* suite open instead of failing.
|
|
171
|
+
*
|
|
172
|
+
* 2.34.1: and 90 s was the same defect again, one schedule further out — see
|
|
173
|
+
* `lib/webhook-retry-window.ts`. The cap is now operator-RAISABLE through
|
|
174
|
+
* `OPENWOP_WEBHOOK_RETRY_WAIT_MS` and never lowerable; still bounded.
|
|
170
175
|
*/
|
|
171
|
-
const
|
|
172
|
-
const RETRY_WAIT_CAP_MS = 90_000;
|
|
176
|
+
const RETRY_WAIT_CAP_MS = retryWaitCapMs();
|
|
173
177
|
function retryWaitMs(doc: Record<string, unknown>): number {
|
|
174
|
-
|
|
175
|
-
if (policy === null) return RETRY_WAIT_FLOOR_MS;
|
|
176
|
-
const backoff = String(policy.backoff ?? '');
|
|
177
|
-
return backoff === 'exponential' || backoff === 'fixed' ? RETRY_WAIT_CAP_MS : RETRY_WAIT_FLOOR_MS;
|
|
178
|
+
return retryWaitFor(advertisedRetryPolicy(doc), RETRY_WAIT_CAP_MS);
|
|
178
179
|
}
|
|
179
180
|
|
|
180
181
|
/**
|
|
@@ -330,8 +331,26 @@ describe('RFC 0173 §B — webhook-durable-delivery (gated on webhooks)', () =>
|
|
|
330
331
|
const sub = await register(receiver.url);
|
|
331
332
|
if (sub === null) return softSkip('blocked', 'registration refused (reason recorded above)');
|
|
332
333
|
|
|
334
|
+
// The row id is the FIRST thing this leg names: `register()` asserts under
|
|
335
|
+
// the base `0173.webhook-durable-delivery` id, and a leg that returned before
|
|
336
|
+
// any `.dead-letter` req() ran recorded its outcome under THAT id, leaving
|
|
337
|
+
// `.dead-letter` with no row in the bundle at all (measured, 2.34.1 draft).
|
|
338
|
+
req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'webhooks.md §Durability', 'an exhausted delivery MUST be routed to the sink, not dropped');
|
|
333
339
|
const create = await driver.post('/runs', { workflowId: FIXTURE });
|
|
334
|
-
|
|
340
|
+
// 2.34.1 — NO OBLIGATION IS ASSERTED UNTIL THE WINDOW QUESTION IS ANSWERED. A leg
|
|
341
|
+
// that asserts and THEN soft-skips records `executed-pass` with a
|
|
342
|
+
// `partial-witness:` detail (`resolveItRecord`), which certifies. The first
|
|
343
|
+
// draft of this fix asserted `create.status` and the retry, then
|
|
344
|
+
// soft-skipped `blocked` on a closed window, and so turned a false FAIL into
|
|
345
|
+
// a pass that never looked at the sink — caught by measuring it: five rows
|
|
346
|
+
// `executed-pass`, 106 s, on a host whose first delivery exhausts at 225 s.
|
|
347
|
+
// So everything is OBSERVED first, the one inconclusive case returns with
|
|
348
|
+
// zero assertions (a real `blocked`, which denies certification), and only
|
|
349
|
+
// then are the obligations asserted, in their original order. A failure the
|
|
350
|
+
// observations already show still fails: the create check fires at once.
|
|
351
|
+
if (create.status !== 201) {
|
|
352
|
+
expect(create.status, req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'runs.md §Create', 'POST /runs MUST answer 201 for the noop fixture')).toBe(201);
|
|
353
|
+
}
|
|
335
354
|
const runId = (create.json as { runId: string }).runId;
|
|
336
355
|
await waitTerminal(runId, 10_000);
|
|
337
356
|
// Filter by the delivery's own SUBSCRIPTION, not just its run. A host that
|
|
@@ -344,19 +363,11 @@ describe('RFC 0173 §B — webhook-durable-delivery (gated on webhooks)', () =>
|
|
|
344
363
|
const ours = () => receiver.attempts.filter((a) => a.runId === runId && a.webhookId === sub.webhookId);
|
|
345
364
|
await waitFor(() => ours().length > 1, retryWaitMs(doc));
|
|
346
365
|
const attempts = ours();
|
|
347
|
-
expect(
|
|
348
|
-
attempts.length,
|
|
349
|
-
req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'webhooks.md §Durability', 'a delivery that keeps failing MUST be retried before it can be exhausted (one attempt is a drop)'),
|
|
350
|
-
).toBeGreaterThan(1);
|
|
351
366
|
const policy = advertisedRetryPolicy(doc);
|
|
352
|
-
if (policy?.maxAttempts !== undefined) {
|
|
367
|
+
if (policy?.maxAttempts !== undefined && attempts.length > 1) {
|
|
353
368
|
// Give the policy time to exhaust, then the host MUST stop.
|
|
354
369
|
await waitFor(() => ours().length >= policy.maxAttempts!, retryWaitMs(doc));
|
|
355
370
|
await new Promise((r) => setTimeout(r, 1_000));
|
|
356
|
-
expect(
|
|
357
|
-
ours().length,
|
|
358
|
-
req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'webhooks.md §Durability', `retries MUST stop at the advertised retryPolicy.maxAttempts (${policy.maxAttempts}) — exhaustion routes to the dead-letter sink, not to an unbounded loop`),
|
|
359
|
-
).toBeLessThanOrEqual(policy.maxAttempts);
|
|
360
371
|
}
|
|
361
372
|
// The sink itself. Until RFC 0188 this was an UNCONDITIONAL soft-skip on
|
|
362
373
|
// every host — `webhooks.md` §Durability said the sink was "inspectable for
|
|
@@ -364,14 +375,41 @@ describe('RFC 0173 §B — webhook-durable-delivery (gated on webhooks)', () =>
|
|
|
364
375
|
// every bundle recorded "exhaustion was observed, routing to the sink was
|
|
365
376
|
// not". The read exists now; read it before deleting the subscription.
|
|
366
377
|
const fam = await gateFamily('webhooks');
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
378
|
+
let inSink: boolean | null = null;
|
|
379
|
+
if (fam?.['deadLetter']) {
|
|
380
|
+
const sink = await driver.get(`/webhooks/${projectBoundId(sub.webhookId)}/dead-letters`);
|
|
381
|
+
inSink = sink.status === 200 && ((sink.json as { deliveries?: unknown[] } | null)?.deliveries ?? []).some((r) => (r as Record<string, unknown>)['runId'] === runId);
|
|
370
382
|
}
|
|
371
|
-
const sink = await driver.get(`/webhooks/${projectBoundId(sub.webhookId)}/dead-letters`);
|
|
372
383
|
await driver.delete(`/webhooks/${encodeURIComponent(sub.webhookId)}`);
|
|
384
|
+
// "Routed to the sink, not dropped" is a claim about an EXHAUSTED delivery,
|
|
385
|
+
// and until 2.34.1 it was asserted whether or not exhaustion had been
|
|
386
|
+
// observed: a host retrying at 15 / 30 / 60 / 120 s had made 4 of its 5
|
|
387
|
+
// attempts when the 90 s window closed, and this row said it had dropped a
|
|
388
|
+
// delivery still in flight. Retried, fewer attempts than advertised, and
|
|
389
|
+
// nothing in the sink is a window that closed early OR a host that stopped
|
|
390
|
+
// retrying — indistinguishable without an interval on the wire — so it is
|
|
391
|
+
// `blocked`, never a conviction. Reached maxAttempts with nothing in the
|
|
392
|
+
// sink still FAILS below; so does a delivery that was never retried.
|
|
393
|
+
const seen = ours().length;
|
|
394
|
+
if (inSink === false && policy?.maxAttempts !== undefined && attempts.length > 1 && seen < policy.maxAttempts) {
|
|
395
|
+
return blockedDespiteAssertions(windowClosedNote(seen, policy.maxAttempts, retryWaitMs(doc), RETRY_WAIT_CAP_MS));
|
|
396
|
+
}
|
|
397
|
+
expect(create.status, req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'runs.md §Create', 'POST /runs MUST answer 201 for the noop fixture')).toBe(201);
|
|
398
|
+
expect(
|
|
399
|
+
attempts.length,
|
|
400
|
+
req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'webhooks.md §Durability', 'a delivery that keeps failing MUST be retried before it can be exhausted (one attempt is a drop)'),
|
|
401
|
+
).toBeGreaterThan(1);
|
|
402
|
+
if (policy?.maxAttempts !== undefined) {
|
|
403
|
+
expect(
|
|
404
|
+
ours().length,
|
|
405
|
+
req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'webhooks.md §Durability', `retries MUST stop at the advertised retryPolicy.maxAttempts (${policy.maxAttempts}) — exhaustion routes to the dead-letter sink, not to an unbounded loop`),
|
|
406
|
+
).toBeLessThanOrEqual(policy.maxAttempts);
|
|
407
|
+
}
|
|
408
|
+
if (inSink === null) {
|
|
409
|
+
return softSkip('inapplicable', 'host does not advertise the webhooks.deadLetter facet — RFC 0188 §A.5 makes the read a 404 rather than an obligation, so the sink half of §Durability is unwitnessable here (the retry half above passed)');
|
|
410
|
+
}
|
|
373
411
|
expect(
|
|
374
|
-
|
|
412
|
+
inSink,
|
|
375
413
|
req('openwop.requirement.0173.webhook-durable-delivery.dead-letter', 'webhooks.md §Durability', 'an exhausted delivery MUST be routed to the sink, not dropped — the half no bundle could witness before RFC 0188 served a read'),
|
|
376
414
|
).toBe(true);
|
|
377
415
|
}, DEAD_LETTER_TEST_TIMEOUT_MS);
|
|
@@ -424,10 +462,14 @@ describe('RFC 0173 §B — webhook-durable-delivery (gated on webhooks)', () =>
|
|
|
424
462
|
active = receiver.server;
|
|
425
463
|
const sub = await register(receiver.url);
|
|
426
464
|
if (sub === null) return softSkip('blocked', 'registration refused (reason recorded above)');
|
|
465
|
+
// Same two traps as the leg above (2.34.1): `register()` asserts under the base
|
|
466
|
+
// id, so name this row first; and every `blocked` after it must STAND rather
|
|
467
|
+
// than fold into a partial-witness pass that certifies.
|
|
468
|
+
req('openwop.requirement.0188.dead-letter-content-free', 'RFC 0188 §B.1', 'a dead-letter record MUST NOT carry the delivered body, headers or subscription secret');
|
|
427
469
|
const create = await driver.post('/runs', { workflowId: FIXTURE });
|
|
428
470
|
if (create.status !== 201) {
|
|
429
471
|
await driver.delete(`/webhooks/${encodeURIComponent(sub.webhookId)}`);
|
|
430
|
-
return
|
|
472
|
+
return blockedDespiteAssertions(`POST /runs answered ${create.status} — no delivery to exhaust`);
|
|
431
473
|
}
|
|
432
474
|
const runId = (create.json as { runId: string }).runId;
|
|
433
475
|
await waitTerminal(runId, 10_000);
|
|
@@ -441,12 +483,16 @@ describe('RFC 0173 §B — webhook-durable-delivery (gated on webhooks)', () =>
|
|
|
441
483
|
|
|
442
484
|
const sink = await driver.get(`/webhooks/${projectBoundId(sub.webhookId)}/dead-letters`);
|
|
443
485
|
await driver.delete(`/webhooks/${encodeURIComponent(sub.webhookId)}`);
|
|
444
|
-
if (sink.status !== 200) return
|
|
486
|
+
if (sink.status !== 200) return blockedDespiteAssertions(`the dead-letter read answered ${sink.status}`);
|
|
445
487
|
const rows = ((sink.json as { deliveries?: Array<Record<string, unknown>> } | null)?.deliveries ?? []);
|
|
446
488
|
// An empty sink is NOT a pass. Recording one as a pass is exactly the
|
|
447
489
|
// vacuous witness this leg used to produce; say so instead.
|
|
448
490
|
if (rows.length === 0) {
|
|
449
|
-
|
|
491
|
+
const seen = ours().length;
|
|
492
|
+
const why = policy?.maxAttempts !== undefined && seen < policy.maxAttempts
|
|
493
|
+
? windowClosedNote(seen, policy.maxAttempts, retryWaitMs(doc), RETRY_WAIT_CAP_MS)
|
|
494
|
+
: `the delivery made ${seen} attempt(s) and is not in the sink`;
|
|
495
|
+
return blockedDespiteAssertions(`§B.1 is a claim about a real record and there is none here to read — ${why}`);
|
|
450
496
|
}
|
|
451
497
|
expect(
|
|
452
498
|
rows.every((r) => r['body'] === undefined && r['headers'] === undefined && r['secret'] === undefined),
|
package/src/setup.ts
CHANGED
|
@@ -50,6 +50,17 @@ import { takeNotedEvidence } from './lib/durability-evidence.js';
|
|
|
50
50
|
// vacuous `inapplicable`. Measured 2026-09-05 on a host answering in 200 ms.
|
|
51
51
|
const SUITE_INIT_TIMEOUT_MS = 20_000;
|
|
52
52
|
const SUITE_INIT_ATTEMPTS = 2;
|
|
53
|
+
/**
|
|
54
|
+
* 2.34.2: a TRANSPORT failure (refused, reset, closed) is retried for up to
|
|
55
|
+
* this long, 1 s apart, before discovery is declared unreadable. Two
|
|
56
|
+
* back-to-back attempts gave a host restarted by the RFC 0158 kill rows about
|
|
57
|
+
* a millisecond to come back: every file set up while it was restarting logged
|
|
58
|
+
* "discovery unreadable" and recorded its fixture gates `blocked`, so one kill
|
|
59
|
+
* contaminated the rest of the run. An HTTP answer is not retried here - a host
|
|
60
|
+
* that answers 500 is up and its answer stands. A timeout (the 20 s abort) is
|
|
61
|
+
* not a transport failure either and still counts as an attempt.
|
|
62
|
+
*/
|
|
63
|
+
const SUITE_INIT_RESTART_WINDOW_MS = 45_000;
|
|
53
64
|
|
|
54
65
|
async function loadHostFixtures(): Promise<void> {
|
|
55
66
|
const baseUrl = process.env.OPENWOP_BASE_URL?.trim();
|
|
@@ -67,6 +78,7 @@ async function loadHostFixtures(): Promise<void> {
|
|
|
67
78
|
if (targetMajor() === 2) headers['OpenWOP-Version'] = '2.0';
|
|
68
79
|
|
|
69
80
|
let lastFailure = 'unknown';
|
|
81
|
+
const initStarted = Date.now();
|
|
70
82
|
for (let attempt = 1; attempt <= SUITE_INIT_ATTEMPTS; attempt += 1) {
|
|
71
83
|
const controller = new AbortController();
|
|
72
84
|
const timer = setTimeout(() => controller.abort(), SUITE_INIT_TIMEOUT_MS);
|
|
@@ -82,6 +94,11 @@ async function loadHostFixtures(): Promise<void> {
|
|
|
82
94
|
return;
|
|
83
95
|
} catch (err) {
|
|
84
96
|
lastFailure = `${(err as Error).message ?? 'unknown'} on attempt ${attempt}`;
|
|
97
|
+
const aborted = (err as Error).name === 'AbortError';
|
|
98
|
+
if (!aborted && Date.now() - initStarted < SUITE_INIT_RESTART_WINDOW_MS) {
|
|
99
|
+
attempt -= 1; // a restarting host: wait, and do not spend an attempt on it
|
|
100
|
+
await new Promise((r) => setTimeout(r, 1_000));
|
|
101
|
+
}
|
|
85
102
|
} finally {
|
|
86
103
|
clearTimeout(timer);
|
|
87
104
|
}
|