@openwop/openwop-conformance 2.33.2 → 2.34.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +25 -0
- package/README.md +3 -1
- package/coverage.md +1 -1
- package/dist/cli.js +23 -1
- package/dist/lib/certification-bundle-v3.js +9 -1
- package/dist/lib/durability-evidence.js +182 -0
- package/dist/lib/requirement-ledger.js +1 -0
- package/dist/lib/scenario-disposition.js +4 -0
- package/dist/lib/soft-skip.js +27 -1
- package/dist/spec-artifacts.lock.json +2 -2
- package/package.json +2 -2
- package/requirements.json +21 -9
- package/schemas/CORPUS-STAMP.json +11 -11
- package/src/cli.ts +23 -1
- package/src/lib/certification-bundle-v3.ts +13 -1
- package/src/lib/durability-evidence.ts +175 -0
- package/src/lib/requirement-ledger.ts +7 -1
- package/src/lib/scenario-disposition.ts +4 -1
- package/src/lib/soft-skip.ts +30 -4
- package/src/lib/webhook-retry-window.ts +56 -0
- package/src/scenarios/v2-durability-recovery.test.ts +61 -13
- package/src/scenarios/v2-webhook-durable-delivery.test.ts +71 -25
- package/src/setup.ts +3 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,30 @@
|
|
|
1
1
|
# `@openwop/openwop-conformance` Changelog
|
|
2
2
|
|
|
3
|
+
## [2.34.1] — 2026-09-21 — a 90 s window convicted a host for retrying at its production pace
|
|
4
|
+
|
|
5
|
+
- **`v2-webhook-durable-delivery` capped every wait at a hard 90 s, and a conformant host with a slower schedule could not advertise `webhooks.deadLetter` without failing its bundle.** A host retrying at 15 / 30 / 60 / 120 s with `maxAttempts: 5` makes its attempts at t0, +15, +45, +105, +225 s. The dead-letter leg waited for a second attempt, waited 90 s more, and read the sink ~120 s before that host exhausts: **`0173.webhook-durable-delivery.dead-letter` → `executed-fail`, "an exhausted delivery MUST be routed to the sink, not dropped"**, about a delivery still in flight; `0188.dead-letter-content-free` → `blocked`, which denies certification. To pass, the host would have had to cut its production retry window from ~225 s to ~75 s for every real subscriber — the instrument choosing the host's durability. Reported by a tier-2 host from its own constants *before* advertising the facet; reproduced on the reference host configured to the same schedule.
|
|
6
|
+
- **The window is now operator-RAISABLE and never lowerable:** `OPENWOP_WEBHOOK_RETRY_WAIT_MS` (default 90 000 per wait, at most 3 600 000). A smaller or non-numeric value is ignored, so no operator can shrink the window to hide a slow retry, and a raised window is still bounded — a host that never retries still fails, later. The `it` timeouts derive from it, as they already did from the constant. Same shape as `OPENWOP_DURABILITY_OBSERVATION_CEILING_MS`; both are now documented in the README, which had neither.
|
|
7
|
+
- **The dead-letter leg no longer asserts "not dropped" before exhaustion was observable.** Fewer attempts than the advertised `maxAttempts` AND nothing in the sink is a window that closed early *or* a host that stopped retrying — indistinguishable without an interval on the wire (`webhooks.retryPolicy` is closed over `{ maxAttempts, backoff }`) — so it records **`blocked`**, naming the variable and the numbers the run used, never `executed-fail`. **A host that reaches `maxAttempts` and has nothing in the sink still FAILS.** `blocked` denies certification exactly as the false fail did; what changes is that the row stops saying something untrue about the host.
|
|
8
|
+
- **`blocked` had to be made to stand, and two rows had no id — both found by measuring the fix, not by reading it.** (1) A leg that asserts and *then* soft-skips records `executed-pass` with a `partial-witness:` detail (`resolveItRecord`): the acceptance predicate refuses it, **certification counts it as a pass**. Both dead-letter legs assert inside `register()` before they can observe anything, so the first draft of this fix turned a false FAIL into a certifying pass that never looked at the sink. New opt-in `blockedDespiteAssertions(reason)` (`lib/soft-skip.ts`) records a `blocked` that survives setup assertions; the ordinary soft-skip convention is unchanged everywhere else. (2) `register()` asserts under the base `0173.webhook-durable-delivery` id and `req()` is last-wins, so a leg that returned before naming its own id recorded under the base id — **`0173…dead-letter` and `0188.dead-letter-content-free` then had no row in the bundle at all.** That was already true of `0188.dead-letter-content-free` on 2.34.0 and earlier whenever the sink was empty: it read as a partial-witness pass of the base row, not the `blocked` it appeared to be. Each leg now names its row first. Measured on the reference host at 15 / 30 / 60 / 120 s: both rows `blocked` under their own ids; at the host's default schedule: all `executed-pass`, same ids; with the window raised to 300 s: all `executed-pass`, exhaustion observed at 226 s.
|
|
9
|
+
- The dead-letter leg also observes before it asserts: it waits, reads the sink and deletes the subscription, returns on the inconclusive case, and only then asserts the obligations in their original order. A create failure still fails at once.
|
|
10
|
+
- **The cost, stated:** a host that advertises 5 attempts, stops at 3 and drops the delivery used to read `executed-fail` and now reads `blocked`. Neither certifies. Detection of that case comes back only by putting the interval on the wire — normative surface, an RFC, not a patch.
|
|
11
|
+
- Pure window logic moved to `lib/webhook-retry-window.ts` with unit tests (raisable, not lowerable, bounded, floor unchanged for a host advertising no policy).
|
|
12
|
+
- `spec/v2/core/persistence.md`'s Stable banner now lists RFC 0158 §A–§D — owed since the RFC's flip, which could not edit a file shipped inside the published 2.34.0.
|
|
13
|
+
- False FAIL only. Suite patch: corpus release stays `2.34.0`.
|
|
14
|
+
|
|
15
|
+
## [2.34.0] — 2026-09-21 — the rung and the recovery bound, in the bundle RFC 0158 said they were in
|
|
16
|
+
|
|
17
|
+
- **RFC 0158 §E publishes a host's rung and recovery bound in its certification bundle instead of in discovery — and the bundle had no seat for either.** `certification-bundle.schema.json` is closed; a reader saw five pass/fail rows and could neither tell which rung was claimed nor recompute the bound, whose terms lived only on a non-normative seam route. Found by reading the RFC's first acceptance criterion at the moment of flipping it, with every row green.
|
|
18
|
+
- **Evidence rides on ROWS, because that is the only place it is signed.** The attestation covers `{ witnessSha256, host.build, suite.version, discovery.sha256 }` and the witness digest covers rows, so a root-level block would be editable after signing on a bundle that still verifies. A row gains an OPTIONAL closed `evidence` object that enters the digest **only when present** — every earlier bundle digests byte-identically, pinned against the three committed host bundles.
|
|
19
|
+
- `0158.bound-is-derived` → `evidence.recoveryBounds[]` of `{ class, bound, terms[{ name, ms }] }`, **per recovery class, no aggregate** (Unresolved Question 1: a scalar would have to be the maximum, which overstates recovery for every faster class).
|
|
20
|
+
- `0158.kill-after-accept`, `0158.kill-during-execution` → `evidence.recovery { class, boundMs, observedMs }`. The kill → resumption interval was always measured and appeared only in a FAILURE message; a passing row recorded nothing.
|
|
21
|
+
- **`durability.rung` is a claim, and the verifier does not trust it.** It sits outside the signature, which is sound for the reason `claimedProfiles[].certified` is: `--verify` re-derives it from signed rows — all five rows `executed-pass`, each kill row's `class` naming a declared entry whose `bound` equals its `boundMs` and is not exceeded by its `observedMs`, each entry's terms summing to its bound — and rejects what it cannot derive (new rejection `rung-not-derivable`). Only `durable-single-instance` is derivable; `peer-resume` evidence is not carried yet, so a higher claim is **refused rather than assumed**. `--certify` claims a rung only when the same derivation supports it, and says on stderr which rung it derived and why.
|
|
22
|
+
- **`class` and term names are opaque, host-chosen, and pattern-constrained** (`^[a-z][A-Za-z0-9._-]{0,63}$`, ≤ 16 classes × ≤ 16 terms). Never an enum: "leased" names one host's mechanism and "boot re-entry" another's. Only `{ name, ms }` survives from a term — free text a host attaches to its seam response never reaches a published bundle.
|
|
23
|
+
- **What class-binding does NOT catch, stated and pinned by a test so nobody reads more into it.** It refuses an undeclared class, a bound label that disagrees with its class, and a resumption outside the bound. It does not refuse a seam that kills *before* the execution claim is held and labels the exercise with the slower class — a tier-1 host shipped exactly that (11.6 s against a 750 s leased class, every row green), and no arithmetic separates it from a fast recovery. The evidence makes it **visible** where before nothing was recorded; killing only once the claim is held stays the host's obligation.
|
|
24
|
+
- **Seam contract (non-normative):** the bound response may be `{ classes: [{ class, bound, terms[] }] }`, a map keyed by class, or the flat `{ bound, terms[], class? }` two hosts already serve. A top-level scalar beside `classes` is ignored. A flat response that names no class is recorded as `default`, which will not match a kill response naming a real class — so the rung is not derivable until the bound says which class it governs.
|
|
25
|
+
- **`--verify` gains a `rung:` line and a newer-suite notice.** A verifier older than the bundle it reads recomputes a different witness digest and fails closed, correctly — but `witness-digest` reads as tampering when the truth is "upgrade the verifier".
|
|
26
|
+
- Additive. No scenario added or removed; no host that passed before fails now. `bundleVersion` stays `"3"`.
|
|
27
|
+
|
|
3
28
|
## [2.33.2] — 2026-09-21 — `v2-run-snapshot-etag` took its tag from a run that was still moving
|
|
4
29
|
|
|
5
30
|
- **The scenario slept a fixed 1 s after `POST /runs`, took the snapshot's `ETag`, and demanded `304`.** A host whose ETag tracks the run's event-log sequence — the strong tag `runs.md` §Snapshot asks for — is still appending while a noop executes, so between the two GETs the representation **changes** and `200` with a new tag is the only correct answer; a `304` there would be a stale-cache bug. **Measured on a tier-2 production host, 4 of 4:** immediately after create `etag1 ≠ etag2` and the conditional GET answers `200`; after the run settles the same request answers `304` every time. The row had passed three earlier cuts of that host and failed the fourth — a timing lottery, won whenever the run finished inside the sleep.
|
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
# --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
|
|
12
12
|
# default resolver refuses. npm 10.9 fails outright with
|
|
13
13
|
# "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
|
|
14
|
-
npm install --legacy-peer-deps @openwop/openwop-conformance@2.
|
|
14
|
+
npm install --legacy-peer-deps @openwop/openwop-conformance@2.34.1 @openwop/spec-artifacts@2.34.1
|
|
15
15
|
# or run without install:
|
|
16
16
|
npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
|
|
17
17
|
```
|
|
@@ -110,6 +110,8 @@ Run `npm run test` for normal CI cadence; `npm run test:strict` when claiming fu
|
|
|
110
110
|
| `OPENWOP_WEBHOOK_ALLOW_PRIVATE=true` | Relaxes the webhook egress guard for the loopback test receiver used by `webhook-signed-delivery.test.ts`, `webhook-negative.test.ts`, and `replay-fanout-suppression.test.ts`. **The receiver is `http://127.0.0.1:{port}/`, which `webhooks.md` forbids three separate times, and the opt-in MUST relax all three to be witnessable:** (1) the **scheme** check — §"SSRF protection" bullet 1 rejects non-`https://`, and this gate fires FIRST on a host that validates scheme before address; (2) the **registration-time** address check — §"SSRF protection"; and (3) **delivery-time re-resolution** — §"Delivery-time egress validation (RFC 0093)". Gates 2 and 3 are independent MUSTs at different layers, so an opt-in reaching only one layer **cannot** produce a witness: delivery-only leaves registration returning `400 webhook_url_rejected`, registration-only leaves the dispatcher refusing to connect. A host whose opt-in reaches one layer is **not** non-conformant — it cannot witness these scenarios, which is a property of the test posture and not of its webhook signing. The scheme gate went unstated here until 2026-08-25; a tier-2 host validating scheme-then-address was blocked by a gate the documented contract never named. Relaxing gates 2 and 3 is **test-only posture** — see `SECURITY/threat-model-secret-leakage.md` §4.9 for why a registration-time relaxation is the more dangerous of the two. When the host rejects, the scenario records `blocked` (RFC 0148 §A), not a pass. |
|
|
111
111
|
| `OPENWOP_WEBHOOK_RECEIVER_URL=<https-url>` | **The route to a webhook witness that relaxes nothing.** A host refusing the loopback receiver on *two* independent grounds — private address **and** non-`https:` — cannot be unblocked by `OPENWOP_WEBHOOK_ALLOW_PRIVATE`, because the scheme arm still stands. This supplies **the public `https:` front for THIS SUITE'S OWN receiver** (tunnel / TLS-terminating proxy), never an arbitrary endpoint: the scenario asserts on what *this process* received, so pointing registration elsewhere makes every header assertion vacuous while the row turns green. Zero deliveries with it set is a **hard failure**, never a skip. Preferred over the flag — it waives nothing and writes no durable subscription row aimed at a private address (`SECURITY/threat-model-secret-leakage.md` §4.9). Pair with `OPENWOP_WEBHOOK_RECEIVER_PORT`. |
|
|
112
112
|
| `OPENWOP_WEBHOOK_RECEIVER_PORT=<port>` | Pins the in-process webhook receiver to a known port instead of an ephemeral one. Required in practice by `OPENWOP_WEBHOOK_RECEIVER_URL`: a tunnel must be aimed at a port known **in advance**, and the receiver binds `0` by default. Unset ⇒ ephemeral, as before. |
|
|
113
|
+
| `OPENWOP_WEBHOOK_RETRY_WAIT_MS=<ms>` | **Raises** how long `v2-webhook-durable-delivery` waits for a retry schedule to play out (default `90000`, per wait; at most `3600000`). `webhooks.retryPolicy` carries `{ maxAttempts, backoff }` and no interval, so the suite cannot derive how long exhaustion takes: set this above the SUM of your backoff intervals (a host retrying at 15/30/60/120 s needs > 225 s). **It can be raised, never lowered** — a smaller or non-numeric value is ignored — and the window stays bounded, so a host that never retries still fails. A window that closes before the advertised `maxAttempts` arrive records `blocked` naming this variable, never a conviction. |
|
|
114
|
+
| `OPENWOP_DURABILITY_OBSERVATION_CEILING_MS=<ms>` | **Raises** how long the RFC 0158 kill rows observe for resumption (default `240000`). A host with a long *declared* recovery bound — a 750 s dispatch lease is conformant — sets this above its bound and waits it out. Raisable, never lowerable. |
|
|
113
115
|
| `OPENWOP_MCP_FAKE_SERVER=true` | Boots the synthetic MCP peer for `mcp-tool-roundtrip.test.ts`. |
|
|
114
116
|
| `OPENWOP_MCP_REAL_SERVER_URL=<base-url>` | Points the MCP wire-shape probe at a real MCP server. The probe POSTs JSON-RPC and reads a single-JSON response — matches MCP's `streamable-http` transport in single-response mode. **Does NOT support** stdio transport (which is what most `modelcontextprotocol/servers` references default to) or SSE-streamed responses; an operator collecting interop evidence today runs a custom `StreamableHTTPServerTransport`-style server that returns a single JSON body per request. Adding SSE-frame parsing is tracked in `docs/PROTOCOL-GAP-CLOSURE-PLAN.md` Track 6. Assertions relax to shape-only. When both this and `OPENWOP_MCP_FAKE_SERVER` are set, the real URL wins. Phase 3 T3.4 interop-evidence path. |
|
|
115
117
|
| `OPENWOP_A2A_FAKE_PEER=true` | Boots the synthetic A2A peer for `a2a-task-roundtrip.test.ts`. |
|
package/coverage.md
CHANGED
|
@@ -145,7 +145,7 @@ The scenario groups in the table below (one row per group; count the rows — th
|
|
|
145
145
|
| `egress-audience-binding.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §C, `host-capabilities.md`) + `SECURITY/invariants.yaml` `egress-credential-audience-bound` | A (KEYSTONE — the §C confused-deputy MUST via `POST /v1/host/sample/egress/decide`: an out-of-audience egress is `denied`/`downgraded` with `reason:"out-of-audience"` and the credential is NOT attached (`credentialAttached !== true`); a provenance-unevaluable egress fails closed `denied`+`reason:"provenance-unevaluable"`; decision/reason ∈ the closed enums) | `host-pending` | `behaviorGate('openwop-egress-audience-binding', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0079 → Accepted bar** (the `egress-credential-audience-bound` invariant graduates reference-impl → protocol tier when this passes against a host). First adopter: MyndHyve `httpClient.egressPolicy`. |
|
|
146
146
|
| `egress-decision-content-free.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §F / SR-1) | A (the secret non-leak — a `canary` credential's sentinel never surfaces in the decision (`canaryLeaked !== true`), the `egress.decided` payload carries no forbidden content key, and `reason` stays in the CLOSED vocabulary so no blocked destination spills into a free-form field) | `host-pending` | `behaviorGate('openwop-egress-decision-content-free', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0079 → Accepted bar.** First adopter: MyndHyve `httpClient.egressPolicy`. |
|
|
147
147
|
| `v2-webhook-egress-refusal.test.ts` | the `webhooks` family (capability-gated) | A (`webhooks.md` §SSRF at MAJOR 2 — eight registration probes, one per class the sentence names: non-`https`, IPv4/IPv6 loopback, `localhost`, two RFC 1918 ranges, link-local / cloud metadata, IPv6 ULA; each MUST answer `400 webhook_url_rejected`. Until 2.33.0 no major-2 row asserted the guard refuses anything — the refusal scenarios were major-1 only. THREE OUTCOMES: all refused → `executed-pass`; a probe accepted with a `webhooks.*` relaxation DECLARED in `OPENWOP_HOST_RELAXATIONS` → `inapplicable`, and the declared relaxation denies the owning profile; a probe accepted with NOTHING declared → `executed-fail`, which the emitter and verifier read as an OBSERVED undeclared relaxation and deny every profile built on `webhooks` (`undeclared-relaxation-observed`). Registration-time only: no probe is a destination the suite owns, nothing is delivered, an accepted registration is deleted at once) |
|
|
148
|
-
| `v2-durability-recovery.test.ts` | the RFC 0158 host-extension durability seam `POST /host/durability/kill` (non-normative; §E mints no capability field, so seam PRESENCE is how a host says it claims a rung) | A (all five `durable-single-instance` rows: `kill-after-accept` as a HOLD-DISPATCH exercise, `kill-during-execution` asserting that work executing at a real process death is never observable as completed without re-execution, `duplicate-delivery` counted AT THE EFFECT'S DESTINATION — a receiver the suite owns, `effectUrl` — because a ledger keyed on effect identity admits one row per identity by construction and cannot show a double-fire (2.32.0), `bound-is-derived` as an explicitly-labelled paper check, and `poison-exhaustion` ported from the major-1 scenario — at major 2 it reads the canonical JSON log read `GET /runs/{runId}/events/poll` (NOT `/events`, which is SSE-only at major 2 — reading it as JSON made this row pass vacuously on a host serving exactly the OpenAPI; 2.32.0), so unlike its v1 twin it needs NO seam and can never record `blocked` for want of one a host did not wire) | `host-pending` | **Disposition rule, and it protects fleet certification:** no seam route at all ⇒ `inapplicable` (the host claims no rung); seam present but an operator precondition unmet — a restart supervisor, which a black-box suite cannot supply — ⇒ `blocked` with the precondition NAMED. Collapsing the two would deny certification to every host the day these rows entered the lane. `peer-resume` is deliberately NOT written: §E makes it bundle-witnessed, so a scenario could only ever record `blocked`. **This is the RFC 0158 → Accepted bar, and it is now complete at major 2** — before the port, a major-2 bundle could carry at most four of the five rows and the rung was unwitnessable no matter what a host did, and its witness is openwop-app, which the RFC records as having observed `kill-during-execution` across a real `SIGKILL`. |
|
|
148
|
+
| `v2-durability-recovery.test.ts` | the RFC 0158 host-extension durability seam `POST /host/durability/kill` (non-normative; §E mints no capability field, so seam PRESENCE is how a host says it claims a rung) | A (all five `durable-single-instance` rows: `kill-after-accept` as a HOLD-DISPATCH exercise, `kill-during-execution` asserting that work executing at a real process death is never observable as completed without re-execution, `duplicate-delivery` counted AT THE EFFECT'S DESTINATION — a receiver the suite owns, `effectUrl` — because a ledger keyed on effect identity admits one row per identity by construction and cannot show a double-fire (2.32.0), `bound-is-derived` as an explicitly-labelled paper check, and `poison-exhaustion` ported from the major-1 scenario — at major 2 it reads the canonical JSON log read `GET /runs/{runId}/events/poll` (NOT `/events`, which is SSE-only at major 2 — reading it as JSON made this row pass vacuously on a host serving exactly the OpenAPI; 2.32.0), so unlike its v1 twin it needs NO seam and can never record `blocked` for want of one a host did not wire) | `host-pending` | **2.34.0 — the rows now WRITE evidence into the bundle (RFC 0158 §E):** `bound-is-derived` records `evidence.recoveryBounds[]` per recovery class and each kill row records `evidence.recovery { class, boundMs, observedMs }` — the interval was always measured and, on a pass, never recorded. Both ride on the ROW so the witness digest, and through it the signature, covers them; the bundle claims `durability.rung` only when `deriveRung` supports it and `--verify` rejects a claim it cannot re-derive (`rung-not-derivable`). A kill row records nothing unless the seam NAMES the class it exercised (`recoveryClass`) and the bound response declares that class — no name, no rung, rather than a guessed one. **Disposition rule, and it protects fleet certification:** no seam route at all ⇒ `inapplicable` (the host claims no rung); seam present but an operator precondition unmet — a restart supervisor, which a black-box suite cannot supply — ⇒ `blocked` with the precondition NAMED. Collapsing the two would deny certification to every host the day these rows entered the lane. `peer-resume` is deliberately NOT written: §E makes it bundle-witnessed, so a scenario could only ever record `blocked`. **This is the RFC 0158 → Accepted bar, and it is now complete at major 2** — before the port, a major-2 bundle could carry at most four of the five rows and the rung was unwitnessable no matter what a host did, and its witness is openwop-app, which the RFC records as having observed `kill-during-execution` across a real `SIGKILL`. |
|
|
149
149
|
| `memory-degraded-projection.test.ts` | `capabilities.agents.manifestRuntime.supported` + `capabilities.memory.supported` (RFC 0080 §C, `agent-memory.md`) | A (the §C iff-contract on the NORMATIVE `GET /v1/agents`: a degraded entry MUST carry `memoryDegraded:true` + a non-empty, unique `degradedMemoryDimensions[]` drawn from the closed §A-name enum [read/write/search/long-term-durability/compaction/attribution/replay-snapshot/retention]; a non-degraded entry MUST NOT carry a non-empty list; the inventory is non-empty; the degraded branch runs non-vacuously when `OPENWOP_DEGRADED_AGENT_ID` names a known-degraded agent) | `host-pending` | `behaviorGate('openwop-memory-degraded', …)`. Black-box on the normative path (no POST seam); soft-skips on 404 / when the host computes no degradation. **This is the RFC 0080 → Accepted bar.** First adopter: MyndHyve `memory`. |
|
|
150
150
|
| `budget-enforcement.test.ts` | `capabilities.budget.supported` (RFC 0084 §C/§D, `budget-policy.md`) + `SECURITY/invariants.yaml` `budget-no-pricing-leak` | A (the §C/§D enforcement via `POST /v1/host/sample/budget/run` + the test event-log seam: a `hard-cost-exhaust` run emits the strict-ordered `budget.reserved → budget.consumed → budget.threshold.crossed{percent} → budget.exhausted → cap.breached{kind:"budget-cost"} → run.failed{error:"budget_exhausted"}` chain; a `model-denied` run is refused `budget_model_denied` BEFORE the provider call (fail-closed); an `advisory` host emits the `budget.*` events without stopping; every `budget.*` payload content-free — no pricing/rate) | `host-pending` | `behaviorGate('openwop-budget-enforcement', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0084 → Accepted bar.** First adopter: MyndHyve `budget`. |
|
|
151
151
|
| `agent-platform-aggregate-evidence.test.ts` | `openwop-agent-platform` claim — live discovery `profiles[]` includes it (RFC 0085 §C, `agent-platform-profile.md`) | A (the §C/§D honest-advertisement on live `/.well-known/openwop`: a host claiming `openwop-agent-platform` MUST satisfy the §B floor predicate (`isAgentPlatformPartial` → `partial`/`full`, never `none`), the claim backed by per-capability evidence not the profile string; `OPENWOP_AGENT_PLATFORM_TIER=full` forces the full-predicate bar — all governance terms + tenant installScope + all 16 §D terms) | `host-pending` | `behaviorGate('openwop-agent-platform', …)`. Black-box on the discovery doc (no POST seam); soft-skips until a host claims the profile. **This is the RFC 0085 → Accepted bar.** First adopter: MyndHyve (after the memory batch surfaces the floor's `memory.supported`). |
|
package/dist/cli.js
CHANGED
|
@@ -41,6 +41,7 @@ import Ajv2020 from 'ajv/dist/2020.js';
|
|
|
41
41
|
import addFormats from 'ajv-formats';
|
|
42
42
|
import { SCHEMAS_DIR } from './lib/paths.js';
|
|
43
43
|
import { readLedgerFile } from './lib/requirement-ledger.js';
|
|
44
|
+
import { deriveRung, emittedByNewerSuite } from './lib/durability-evidence.js';
|
|
44
45
|
import { deriveRequirementDispositions } from './lib/scenario-disposition.js';
|
|
45
46
|
import { scrubEvidence, evidenceSecretsFromEnv, verifyBundleV2 } from './lib/certification-bundle-verify.js';
|
|
46
47
|
import { publicKeyFromPrivate, signBundleV3, verifierSign, verifyBundleV3, witnessDigest } from './lib/certification-bundle-v3.js';
|
|
@@ -619,7 +620,15 @@ async function runCertify(args, baseUrl, apiKey) {
|
|
|
619
620
|
process.stderr.write('openwop-conformance --certify: a v3 bundle needs --host-build <kind>:<id> (or OPENWOP_HOST_BUILD), --signing-key <pem> (or OPENWOP_BUNDLE_SIGNING_KEY) and --signing-key-id (or OPENWOP_BUNDLE_SIGNING_KEY_ID) — an unsigned bundle does not exist in v3 (RFC 0168 §E.2).\n');
|
|
620
621
|
process.exit(2);
|
|
621
622
|
}
|
|
622
|
-
const
|
|
623
|
+
const evidenceById = new Map();
|
|
624
|
+
for (const e of ledgerEntries)
|
|
625
|
+
if (e.evidence !== undefined && e.disposition === 'executed-pass')
|
|
626
|
+
evidenceById.set(e.requirementId, e.evidence);
|
|
627
|
+
const rows3 = derived.requirements.map((r) => ({ id: r.requirementId, scenario: r.scenarioId, result: r.disposition, ...(r.assertionCount === undefined ? {} : { assertions: r.assertionCount }), ...(r.detail === undefined ? {} : { detail: r.detail }),
|
|
628
|
+
// RFC 0158 §E: structured evidence rides on the ROW, so the witness digest —
|
|
629
|
+
// and through it the signature — covers it. Lifted from the raw ledger by
|
|
630
|
+
// requirement id, and only onto a row that is itself `executed-pass`.
|
|
631
|
+
...(r.disposition === 'executed-pass' && evidenceById.has(r.requirementId) ? { evidence: evidenceById.get(r.requirementId) } : {}) }));
|
|
623
632
|
const totals3 = derived.totals;
|
|
624
633
|
const doc3 = document;
|
|
625
634
|
const protocolVersions = Array.isArray(doc3['protocolVersions']) ? doc3['protocolVersions'] : [String(doc3['protocolVersion'] ?? '')];
|
|
@@ -672,6 +681,10 @@ async function runCertify(args, baseUrl, apiKey) {
|
|
|
672
681
|
const lockPath = resolvePath(conformanceRoot, 'dist', 'spec-artifacts.lock.json');
|
|
673
682
|
const lock = existsSync(lockPath) ? JSON.parse(readFileSync(lockPath, 'utf8')) : undefined;
|
|
674
683
|
const nonPass = rows3.filter((r) => r.result !== 'executed-pass');
|
|
684
|
+
const rung3 = deriveRung(rows3);
|
|
685
|
+
if (rows3.some((r) => r.id.startsWith('openwop.requirement.0158.') && r.result === 'executed-pass' && r.id !== 'openwop.requirement.0158.poison-exhaustion')) {
|
|
686
|
+
process.stderr.write(`openwop-conformance --certify: RFC 0158 rung — ${rung3.rung ?? 'NONE'} (${rung3.why})\n`);
|
|
687
|
+
}
|
|
675
688
|
const unsigned = {
|
|
676
689
|
bundleVersion: '3',
|
|
677
690
|
generatedAt: new Date().toISOString(),
|
|
@@ -687,6 +700,10 @@ async function runCertify(args, baseUrl, apiKey) {
|
|
|
687
700
|
witnessSha256: witnessDigest(rows3),
|
|
688
701
|
assertionCount: rows3.reduce((n, r) => n + (r.assertions ?? 0), 0),
|
|
689
702
|
...(nonPass.length ? { detail: { nonPass: nonPass.map((r) => ({ id: r.id, result: r.result, reason: r.detail ?? '' })) } } : {}),
|
|
703
|
+
// RFC 0158 §D: claimed ONLY when these rows support it. The verifier
|
|
704
|
+
// re-derives the same answer from the same signed rows, so an emitter that
|
|
705
|
+
// claimed more would be writing a bundle its own `--verify` rejects.
|
|
706
|
+
...(rung3.rung === null ? {} : { durability: { rung: rung3.rung } }),
|
|
690
707
|
};
|
|
691
708
|
const signature = signBundleV3(unsigned, signingKeyPem, keyId);
|
|
692
709
|
const v3 = { ...unsigned, signature };
|
|
@@ -889,6 +906,8 @@ async function main() {
|
|
|
889
906
|
`suite: ${bundle.suite?.version ?? '?'} (this CLI is ${suiteVersion()})`,
|
|
890
907
|
`totals: executedPass=${t.executedPass ?? '?'} executedFail=${t.executedFail ?? '?'} blocked=${t.blocked ?? '?'} inapplicable=${t.inapplicable ?? '?'} skipped=${t.skipped ?? '?'}`,
|
|
891
908
|
`certified: ${verdict.certifiedProfiles.length > 0 ? verdict.certifiedProfiles.join(', ') : '(none)'}`,
|
|
909
|
+
// RFC 0158 §D/§E: the rung is a CLAIM; a claim the signed rows do not support is a rejection below.
|
|
910
|
+
`rung: ${bundle.durability?.rung ?? '(none claimed)'}`,
|
|
892
911
|
'',
|
|
893
912
|
'What this command does NOT do:',
|
|
894
913
|
' · It does not re-run anything. A host that measured itself wrongly, and signed',
|
|
@@ -899,6 +918,9 @@ async function main() {
|
|
|
899
918
|
' an older bundle measured less, and whose fact that is belongs to its emitter.',
|
|
900
919
|
'',
|
|
901
920
|
];
|
|
921
|
+
if (emittedByNewerSuite(bundle.suite?.version, suiteVersion())) {
|
|
922
|
+
out.push(`NOTE \u2014 this bundle was emitted by suite ${String(bundle.suite?.version)}, NEWER than this verifier (${suiteVersion()}).`, ' A newer suite may digest row members this verifier does not know. If a `witness-digest`', ' rejection follows, upgrade the verifier before reading it as tampering.', '');
|
|
923
|
+
}
|
|
902
924
|
if (verdict.rejections.length > 0) {
|
|
903
925
|
out.push(`REJECTED \u2014 ${verdict.rejections.length} problem(s):`);
|
|
904
926
|
for (const r of verdict.rejections)
|
|
@@ -19,6 +19,7 @@
|
|
|
19
19
|
*/
|
|
20
20
|
import { createHash, createPrivateKey, createPublicKey, sign as edSign, verify as edVerify } from 'node:crypto';
|
|
21
21
|
import { profileDerivable } from './profiles.js';
|
|
22
|
+
import { checkRungClaim } from './durability-evidence.js';
|
|
22
23
|
import { profilesDeniedByObservedRelaxation, profilesRelaxedBy, v2RegistryAvailable } from './v2-profiles.js';
|
|
23
24
|
export const SIGNATURE_OVER = ['witnessSha256', 'host.build', 'suite.version', 'discovery.sha256'];
|
|
24
25
|
/** Deterministic JSON: keys sorted at every level, no whitespace. */
|
|
@@ -32,7 +33,9 @@ export function canonicalJSON(value) {
|
|
|
32
33
|
}
|
|
33
34
|
/** RFC 0148 §C — the digest over the reporter record (the requirement rows). */
|
|
34
35
|
export function witnessDigest(rows) {
|
|
35
|
-
const canonicalRows = [...rows].sort((a, b) => a.id.localeCompare(b.id)).map((r) => ({ id: r.id, scenario: r.scenario, result: r.result, ...(r.assertions === undefined ? {} : { assertions: r.assertions }), ...(r.detail === undefined ? {} : { detail: r.detail })
|
|
36
|
+
const canonicalRows = [...rows].sort((a, b) => a.id.localeCompare(b.id)).map((r) => ({ id: r.id, scenario: r.scenario, result: r.result, ...(r.assertions === undefined ? {} : { assertions: r.assertions }), ...(r.detail === undefined ? {} : { detail: r.detail }),
|
|
37
|
+
// ONLY WHEN PRESENT: every bundle cut before 2.34.0 has no `evidence` and digests byte-identically.
|
|
38
|
+
...(r.evidence === undefined ? {} : { evidence: r.evidence }) }));
|
|
36
39
|
return createHash('sha256').update(canonicalJSON(canonicalRows), 'utf8').digest('hex');
|
|
37
40
|
}
|
|
38
41
|
/** The bytes the attestation covers. */
|
|
@@ -120,6 +123,11 @@ export function verifyBundleV3(bundle, opts = {}) {
|
|
|
120
123
|
const relaxed = new Set((bundle.host?.relaxations ?? []).map((r) => r.obligation.split('.')[0]));
|
|
121
124
|
// Ownership comes from the profile registry, not from the profile's name — see profilesRelaxedBy.
|
|
122
125
|
const relaxedProfiles = profilesRelaxedBy((bundle.host?.relaxations ?? []).map((r) => r.obligation), (bundle.claimedProfiles ?? []).map((p) => p.id));
|
|
126
|
+
// RFC 0158 §D: a rung is CLAIMED at the top level, outside the signature, and
|
|
127
|
+
// is therefore only ever as good as its re-derivation from the signed rows.
|
|
128
|
+
const rung = checkRungClaim(bundle.durability?.rung, rows);
|
|
129
|
+
if (!rung.ok)
|
|
130
|
+
rejections.push({ kind: 'rung-not-derivable', detail: rung.detail });
|
|
123
131
|
// An OBSERVED relaxation nobody declared: the bundle's own results show the
|
|
124
132
|
// host accepting a destination its egress guard MUST refuse. Same denial as a
|
|
125
133
|
// declared one, so declaring nothing is not a way around the rule.
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* RFC 0158 §E — the rung and the recovery bound, IN THE BUNDLE.
|
|
3
|
+
*
|
|
4
|
+
* §E.10 mints no discovery capability: "a qualification ladder is evidence about
|
|
5
|
+
* behaviour that already exists, so the useful place for a rung and a recovery
|
|
6
|
+
* bound is the host's conformance evidence bundle — where a claim without
|
|
7
|
+
* evidence is already a defect." Until 2.34.0 that sentence had nothing behind
|
|
8
|
+
* it. `certification-bundle.schema.json` had no seat for a rung, a bound or its
|
|
9
|
+
* terms; a bundle showed five pass/fail rows, and a reader could neither tell
|
|
10
|
+
* which rung was claimed nor recompute the bound. The terms lived only on a
|
|
11
|
+
* non-normative seam route. Found by READING acceptance criterion 1 at the
|
|
12
|
+
* moment of flipping the RFC, with every row already green.
|
|
13
|
+
*
|
|
14
|
+
* ── Why the evidence rides on ROWS ───────────────────────────────────────────
|
|
15
|
+
* The attestation covers exactly `{ witnessSha256, host.build, suite.version,
|
|
16
|
+
* discovery.sha256 }` (`conformance.md` §Bundle v3), and `witnessSha256` digests
|
|
17
|
+
* the requirement rows. A top-level block would therefore be UNSIGNED — a bound
|
|
18
|
+
* or a rung editable after signing on a bundle that still verifies. Evidence
|
|
19
|
+
* carried on a row is inside the witness digest, and so inside the signature.
|
|
20
|
+
* It enters the digest ONLY WHEN PRESENT, so every bundle cut before 2.34.0
|
|
21
|
+
* digests byte-identically and still verifies (pinned against the three
|
|
22
|
+
* committed bundles in `durability-evidence.test.ts`).
|
|
23
|
+
*
|
|
24
|
+
* ── Why the rung claim is NOT trusted ────────────────────────────────────────
|
|
25
|
+
* `durability.rung` sits at the top level, outside the signature, and that is
|
|
26
|
+
* sound for the same reason `claimedProfiles[].certified` is: the verifier
|
|
27
|
+
* RE-DERIVES it from signed rows and rejects a claim it cannot derive. §D: "a
|
|
28
|
+
* host MAY claim a rung only with the evidence named for it."
|
|
29
|
+
*
|
|
30
|
+
* ── Why a kill row names its recovery CLASS ──────────────────────────────────
|
|
31
|
+
* Unresolved Question 1 resolved PER WORK CLASS, not a scalar. A host with an
|
|
32
|
+
* outbox lane (65 s) and a dispatch lease (750 s) has two bounds, so each kill
|
|
33
|
+
* row records `{ class, boundMs, observedMs }`: the class MUST name a declared
|
|
34
|
+
* entry, `boundMs` MUST equal that entry's `bound`, and `observedMs` MUST NOT
|
|
35
|
+
* exceed it.
|
|
36
|
+
*
|
|
37
|
+
* WHAT THAT CATCHES, and what it cannot. It refuses an undeclared class, a bound
|
|
38
|
+
* label that disagrees with its class, and a resumption outside the bound. It
|
|
39
|
+
* does NOT catch a host that names the WRONG DECLARED class while resuming
|
|
40
|
+
* inside that class's (longer) bound — which a tier-1 host actually shipped: the
|
|
41
|
+
* seam killed before the execution claim was held, the 65 s lane rescued the run
|
|
42
|
+
* in 11.6 s, and the exercise was labelled leased / 750 s, every row green. No
|
|
43
|
+
* arithmetic separates that from a fast leased recovery. What the evidence
|
|
44
|
+
* changes is that it becomes VISIBLE — 11.6 s recorded against a class whose own
|
|
45
|
+
* terms say 720 s of lease — where before nothing was recorded at all. Killing
|
|
46
|
+
* only once the claim is held stays the HOST's obligation (§E).
|
|
47
|
+
*
|
|
48
|
+
* `class` is an OPAQUE host-chosen string, never an enum: "leased" / "unleased"
|
|
49
|
+
* name one host's mechanisms and boot re-entry is another's. An enum would
|
|
50
|
+
* select for an architecture, which this RFC refuses to do everywhere else.
|
|
51
|
+
* There is NO scalar "overall bound": that is UQ1's lie by aggregation with a
|
|
52
|
+
* friendlier name, and a reader would use it.
|
|
53
|
+
*
|
|
54
|
+
* What this block proves: ARITHMETIC (Σ terms = bound) and, through the kill
|
|
55
|
+
* rows' `observedMs`, that the mechanism RAN at least once inside the bound.
|
|
56
|
+
* `bound-is-derived` alone remains a paper check and MUST NOT be cited for
|
|
57
|
+
* liveness.
|
|
58
|
+
*/
|
|
59
|
+
/** Host-chosen identifiers enter a PUBLISHED bundle: short, plain, no free text. */
|
|
60
|
+
export const EVIDENCE_NAME_PATTERN = /^[a-z][A-Za-z0-9._-]{0,63}$/;
|
|
61
|
+
export const MAX_RECOVERY_CLASSES = 16;
|
|
62
|
+
export const MAX_TERMS_PER_CLASS = 16;
|
|
63
|
+
export const RUNGS = ['durable-single-instance', 'durable-multi-instance', 'multi-region-qualified'];
|
|
64
|
+
const R = 'openwop.requirement.0158.';
|
|
65
|
+
/** The rows §Acceptance names for the lowest rung. */
|
|
66
|
+
export const SINGLE_INSTANCE_ROWS = ['kill-after-accept', 'kill-during-execution', 'duplicate-delivery', 'poison-exhaustion', 'bound-is-derived'].map((r) => R + r);
|
|
67
|
+
const KILL_ROWS = [R + 'kill-after-accept', R + 'kill-during-execution'];
|
|
68
|
+
const BOUND_ROW = R + 'bound-is-derived';
|
|
69
|
+
const isCount = (n) => typeof n === 'number' && Number.isInteger(n) && n >= 0;
|
|
70
|
+
const isName = (s) => typeof s === 'string' && EVIDENCE_NAME_PATTERN.test(s);
|
|
71
|
+
/** Normalise what a host's seam returned into bundle evidence, or say exactly why it cannot be. */
|
|
72
|
+
export function parseRecoveryBounds(raw) {
|
|
73
|
+
if (!Array.isArray(raw) || raw.length === 0)
|
|
74
|
+
return { ok: false, why: 'no recovery classes were declared' };
|
|
75
|
+
if (raw.length > MAX_RECOVERY_CLASSES)
|
|
76
|
+
return { ok: false, why: `more than ${MAX_RECOVERY_CLASSES} recovery classes` };
|
|
77
|
+
const bounds = [];
|
|
78
|
+
const seen = new Set();
|
|
79
|
+
for (const entry of raw) {
|
|
80
|
+
const cls = entry?.['class'];
|
|
81
|
+
if (!isName(cls))
|
|
82
|
+
return { ok: false, why: `a recovery class name does not match ${String(EVIDENCE_NAME_PATTERN)}` };
|
|
83
|
+
if (seen.has(cls))
|
|
84
|
+
return { ok: false, why: `recovery class ${cls} is declared twice` };
|
|
85
|
+
seen.add(cls);
|
|
86
|
+
const terms = entry['terms'];
|
|
87
|
+
if (!Array.isArray(terms) || terms.length === 0 || terms.length > MAX_TERMS_PER_CLASS)
|
|
88
|
+
return { ok: false, why: `class ${cls}: terms[] MUST carry 1..${MAX_TERMS_PER_CLASS} entries — a total alone cannot be recomputed` };
|
|
89
|
+
const clean = [];
|
|
90
|
+
for (const t of terms) {
|
|
91
|
+
if (!isName(t?.['name']) || !isCount(t['ms']))
|
|
92
|
+
return { ok: false, why: `class ${cls}: every term is { name, ms } with ms a non-negative integer` };
|
|
93
|
+
clean.push({ name: t['name'], ms: t['ms'] });
|
|
94
|
+
}
|
|
95
|
+
if (!isCount(entry['bound']))
|
|
96
|
+
return { ok: false, why: `class ${cls}: bound is a non-negative integer of milliseconds` };
|
|
97
|
+
const sum = clean.reduce((a, t) => a + t.ms, 0);
|
|
98
|
+
if (sum !== entry['bound'])
|
|
99
|
+
return { ok: false, why: `class ${cls}: terms sum to ${sum} ms but the declared bound is ${String(entry['bound'])} ms — a host that states a bound its terms do not produce fails §B.5` };
|
|
100
|
+
bounds.push({ class: cls, bound: entry['bound'], terms: clean });
|
|
101
|
+
}
|
|
102
|
+
return { ok: true, bounds };
|
|
103
|
+
}
|
|
104
|
+
/**
|
|
105
|
+
* The highest rung these SIGNED rows support, and — when it is lower than a
|
|
106
|
+
* reader might hope — the first reason why. Only `durable-single-instance` is
|
|
107
|
+
* derivable today: `peer-resume` is bundle-witnessed by a per-boot incarnation
|
|
108
|
+
* token this revision does not yet carry (§E), so a higher claim is REFUSED
|
|
109
|
+
* rather than waved through.
|
|
110
|
+
*/
|
|
111
|
+
export function deriveRung(rows) {
|
|
112
|
+
const byId = new Map(rows.map((r) => [r.id, r]));
|
|
113
|
+
for (const id of SINGLE_INSTANCE_ROWS) {
|
|
114
|
+
const row = byId.get(id);
|
|
115
|
+
if (row === undefined)
|
|
116
|
+
return { rung: null, why: `${id} is absent from the bundle` };
|
|
117
|
+
if (row.result !== 'executed-pass')
|
|
118
|
+
return { rung: null, why: `${id} is ${row.result}, not executed-pass` };
|
|
119
|
+
}
|
|
120
|
+
const declared = parseRecoveryBounds(byId.get(BOUND_ROW)?.evidence?.recoveryBounds);
|
|
121
|
+
if (!declared.ok)
|
|
122
|
+
return { rung: null, why: `${BOUND_ROW} carries no usable evidence.recoveryBounds — ${declared.why}` };
|
|
123
|
+
const bounds = new Map(declared.bounds.map((b) => [b.class, b.bound]));
|
|
124
|
+
for (const id of KILL_ROWS) {
|
|
125
|
+
const obs = byId.get(id)?.evidence?.recovery;
|
|
126
|
+
if (obs === undefined)
|
|
127
|
+
return { rung: null, why: `${id} passed but recorded no evidence.recovery — the class it exercised and the interval it observed are unknown` };
|
|
128
|
+
if (!isName(obs.class) || !isCount(obs.boundMs) || !isCount(obs.observedMs))
|
|
129
|
+
return { rung: null, why: `${id}: evidence.recovery is malformed` };
|
|
130
|
+
const bound = bounds.get(obs.class);
|
|
131
|
+
if (bound === undefined)
|
|
132
|
+
return { rung: null, why: `${id} exercised recovery class "${obs.class}", which ${BOUND_ROW} does not declare (declared: ${[...bounds.keys()].join(', ')})` };
|
|
133
|
+
if (obs.boundMs !== bound)
|
|
134
|
+
return { rung: null, why: `${id} was judged against ${obs.boundMs} ms but class "${obs.class}" declares ${bound} ms — the exercise was labelled with a bound that does not govern it` };
|
|
135
|
+
if (obs.observedMs > bound)
|
|
136
|
+
return { rung: null, why: `${id} resumed after ${obs.observedMs} ms, outside the ${bound} ms bound of class "${obs.class}"` };
|
|
137
|
+
}
|
|
138
|
+
return { rung: 'durable-single-instance', why: 'all five rows executed-pass; each kill row names a declared class and resumed inside its bound' };
|
|
139
|
+
}
|
|
140
|
+
/** Is the rung a bundle CLAIMS supported by its own signed rows? */
|
|
141
|
+
export function checkRungClaim(claimed, rows) {
|
|
142
|
+
if (claimed === undefined)
|
|
143
|
+
return { ok: true };
|
|
144
|
+
if (typeof claimed !== 'string' || !RUNGS.includes(claimed))
|
|
145
|
+
return { ok: false, detail: `durability.rung ${JSON.stringify(claimed)} is not one of ${RUNGS.join(' | ')}` };
|
|
146
|
+
const derived = deriveRung(rows);
|
|
147
|
+
if (derived.rung === null)
|
|
148
|
+
return { ok: false, detail: `durability.rung claims ${claimed}, but the signed rows do not support any rung: ${derived.why}` };
|
|
149
|
+
if (claimed !== derived.rung)
|
|
150
|
+
return { ok: false, detail: `durability.rung claims ${claimed}, but the signed rows support only ${derived.rung} — a higher rung is bundle-witnessed by evidence this revision does not yet carry (RFC 0158 §E, peer-resume), so it is refused rather than assumed` };
|
|
151
|
+
return { ok: true };
|
|
152
|
+
}
|
|
153
|
+
/**
|
|
154
|
+
* Was this bundle cut by a NEWER suite than the verifier reading it?
|
|
155
|
+
*
|
|
156
|
+
* Row evidence enters the witness digest, so a verifier older than 2.34.0 that
|
|
157
|
+
* meets a bundle carrying it recomputes a different digest and reports
|
|
158
|
+
* `witness-digest` — it FAILS CLOSED, which is right, but the message reads as
|
|
159
|
+
* tampering when the truth is "upgrade the verifier". Every later digested
|
|
160
|
+
* member will repeat that. The notice is advice; it never changes a verdict.
|
|
161
|
+
*/
|
|
162
|
+
export function emittedByNewerSuite(bundleSuite, verifierSuite) {
|
|
163
|
+
const parse = (v) => {
|
|
164
|
+
const m = typeof v === 'string' ? /^(\d+)\.(\d+)\.(\d+)/.exec(v) : null;
|
|
165
|
+
return m === null ? null : [Number(m[1]), Number(m[2]), Number(m[3])];
|
|
166
|
+
};
|
|
167
|
+
const b = parse(bundleSuite);
|
|
168
|
+
const mine = parse(verifierSuite);
|
|
169
|
+
if (b === null || mine === null)
|
|
170
|
+
return false;
|
|
171
|
+
for (let i = 0; i < 3; i++) {
|
|
172
|
+
if (b[i] !== mine[i])
|
|
173
|
+
return b[i] > mine[i];
|
|
174
|
+
}
|
|
175
|
+
return false;
|
|
176
|
+
}
|
|
177
|
+
// ── the scenario → setup.ts → ledger channel ─────────────────────────────────
|
|
178
|
+
let pending = null;
|
|
179
|
+
/** Called by a scenario inside an `it`; setup.ts attaches it to that test's ledger row. Later notes merge. */
|
|
180
|
+
export function noteEvidence(evidence) { pending = { ...(pending ?? {}), ...evidence }; }
|
|
181
|
+
/** setup.ts reads and clears it after each test. */
|
|
182
|
+
export function takeNotedEvidence() { const e = pending; pending = null; return e; }
|
|
@@ -64,6 +64,7 @@ export function recordRequirement(requirementId, disposition, detail, extras) {
|
|
|
64
64
|
...(detail === undefined ? {} : { detail }),
|
|
65
65
|
...(extras?.assertionCount === undefined ? {} : { assertionCount: extras.assertionCount }),
|
|
66
66
|
...(extras?.scenarioFile === undefined ? {} : { scenarioFile: extras.scenarioFile }),
|
|
67
|
+
...(extras?.evidence === undefined || disposition !== 'executed-pass' ? {} : { evidence: extras.evidence }),
|
|
67
68
|
};
|
|
68
69
|
ledger.set(requirementId, entry);
|
|
69
70
|
journal.push(entry);
|
|
@@ -100,6 +100,10 @@ export function resolveItRecord(state, assertionCalls, gate, noted, firstError)
|
|
|
100
100
|
if (state === 'fail')
|
|
101
101
|
return { disposition: 'executed-fail', detail: `the test executed and failed: ${(firstError ?? 'no message').slice(0, 300)}` };
|
|
102
102
|
if (state === 'pass' && assertionCalls > 0) {
|
|
103
|
+
// `blockedDespiteAssertions` (soft-skip.ts): the leg says its setup
|
|
104
|
+
// assertions are not the requirement, and the requirement went unobserved.
|
|
105
|
+
if (noted !== null && noted.kind === 'blocked' && noted.conclusive === true)
|
|
106
|
+
return { disposition: 'blocked', detail: noted.reason };
|
|
103
107
|
// A leg that asserted AND THEN soft-skipped is only a partial witness, and
|
|
104
108
|
// the file-level record has always said so (`resolveFileRecord` below).
|
|
105
109
|
// This `it`-level record dropped the note — and the `it`-level rows are the
|
package/dist/lib/soft-skip.js
CHANGED
|
@@ -74,6 +74,32 @@ export function seamAbsent(reason) {
|
|
|
74
74
|
}
|
|
75
75
|
return softSkip('blocked', reason);
|
|
76
76
|
}
|
|
77
|
+
/**
|
|
78
|
+
* `blocked` that STANDS even though the test already asserted something.
|
|
79
|
+
*
|
|
80
|
+
* A plain `softSkip` after an assertion records `executed-pass` with a
|
|
81
|
+
* `partial-witness:` detail (`resolveItRecord`): the acceptance predicate
|
|
82
|
+
* refuses such a row, but certification counts it as a pass. That is right for
|
|
83
|
+
* a leg that finished its requirement and skipped an optional extra. It is
|
|
84
|
+
* wrong for a leg whose REQUIREMENT went unobserved after setup assertions it
|
|
85
|
+
* could not avoid — a helper like `register()` asserts `201` before the leg has
|
|
86
|
+
* observed anything. Use this only there: the row records `blocked`, which
|
|
87
|
+
* denies certification (RFC 0168 §E.1) without convicting the host.
|
|
88
|
+
*
|
|
89
|
+
* Opt-in and per-call on purpose. Honouring every note-after-assertion as
|
|
90
|
+
* `blocked` would downgrade legs that legitimately completed; that is a suite-
|
|
91
|
+
* wide semantic change, not a patch. First use: `v2-webhook-durable-delivery`'s
|
|
92
|
+
* dead-letter leg when the retry window closes before exhaustion (2.34.1).
|
|
93
|
+
*/
|
|
94
|
+
export function blockedDespiteAssertions(reason) {
|
|
95
|
+
const file = currentFile();
|
|
96
|
+
if (file === null)
|
|
97
|
+
return undefined;
|
|
98
|
+
const arr = notes.get(file) ?? [];
|
|
99
|
+
arr.push({ kind: 'blocked', reason, seq: ++seq, conclusive: true });
|
|
100
|
+
notes.set(file, arr);
|
|
101
|
+
return undefined;
|
|
102
|
+
}
|
|
77
103
|
const RANK = { blocked: 0, skipped: 1, inapplicable: 2 };
|
|
78
104
|
function fold(arr) {
|
|
79
105
|
if (arr.length === 0)
|
|
@@ -84,7 +110,7 @@ function fold(arr) {
|
|
|
84
110
|
uniq.push(n);
|
|
85
111
|
const kind = [...uniq].sort((a, b) => RANK[a.kind] - RANK[b.kind])[0].kind;
|
|
86
112
|
const reason = uniq.map((n) => (uniq.length > 1 ? `[${n.kind}] ${n.reason}` : n.reason)).join('; ');
|
|
87
|
-
return { kind, reason };
|
|
113
|
+
return uniq.some((n) => n.conclusive === true) ? { kind, reason, conclusive: true } : { kind, reason };
|
|
88
114
|
}
|
|
89
115
|
/**
|
|
90
116
|
* The noted disposition for a file, worst-first when mixed (`blocked` beats
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openwop/openwop-conformance",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.34.1",
|
|
4
4
|
"description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -56,6 +56,6 @@
|
|
|
56
56
|
"@openwop/spec-artifacts": "file:../spec-artifacts"
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
|
-
"@openwop/spec-artifacts": "2.
|
|
59
|
+
"@openwop/spec-artifacts": "2.34.1"
|
|
60
60
|
}
|
|
61
61
|
}
|
package/requirements.json
CHANGED
|
@@ -26650,7 +26650,7 @@
|
|
|
26650
26650
|
{
|
|
26651
26651
|
"id": "openwop.it.v2-durability-recovery.accepted-work-survives-a-kill-before-dispatch-and-dispatches-on-resume",
|
|
26652
26652
|
"file": "v2-durability-recovery.test.ts",
|
|
26653
|
-
"line":
|
|
26653
|
+
"line": 283,
|
|
26654
26654
|
"title": "accepted work survives a kill before dispatch and dispatches on resume",
|
|
26655
26655
|
"explicitId": "openwop.requirement.0158.kill-after-accept",
|
|
26656
26656
|
"citations": [
|
|
@@ -26664,7 +26664,7 @@
|
|
|
26664
26664
|
{
|
|
26665
26665
|
"id": "openwop.it.v2-durability-recovery.work-executing-at-a-real-process-death-is-never-reported-complete-and-resumes",
|
|
26666
26666
|
"file": "v2-durability-recovery.test.ts",
|
|
26667
|
-
"line":
|
|
26667
|
+
"line": 337,
|
|
26668
26668
|
"title": "work executing at a real process death is never reported complete, and resumes",
|
|
26669
26669
|
"explicitId": "openwop.requirement.0158.kill-during-execution",
|
|
26670
26670
|
"citations": [
|
|
@@ -26683,7 +26683,7 @@
|
|
|
26683
26683
|
{
|
|
26684
26684
|
"id": "openwop.it.v2-durability-recovery.the-same-accepted-work-delivered-twice-fires-each-effect-exactly-once",
|
|
26685
26685
|
"file": "v2-durability-recovery.test.ts",
|
|
26686
|
-
"line":
|
|
26686
|
+
"line": 393,
|
|
26687
26687
|
"title": "the same accepted work delivered twice fires each effect exactly once",
|
|
26688
26688
|
"explicitId": "openwop.requirement.0158.duplicate-delivery",
|
|
26689
26689
|
"citations": [
|
|
@@ -26702,7 +26702,7 @@
|
|
|
26702
26702
|
{
|
|
26703
26703
|
"id": "openwop.it.v2-durability-recovery.the-declared-recovery-bound-is-derived-from-the-mechanism-that-enforces-it",
|
|
26704
26704
|
"file": "v2-durability-recovery.test.ts",
|
|
26705
|
-
"line":
|
|
26705
|
+
"line": 467,
|
|
26706
26706
|
"title": "the declared recovery bound is derived from the mechanism that enforces it",
|
|
26707
26707
|
"explicitId": "openwop.requirement.0158.bound-is-derived",
|
|
26708
26708
|
"citations": [
|
|
@@ -26716,7 +26716,7 @@
|
|
|
26716
26716
|
{
|
|
26717
26717
|
"id": "openwop.it.v2-durability-recovery.deterministically-failing-work-reaches-a-terminal-state-and-stops-being-retried",
|
|
26718
26718
|
"file": "v2-durability-recovery.test.ts",
|
|
26719
|
-
"line":
|
|
26719
|
+
"line": 498,
|
|
26720
26720
|
"title": "deterministically failing work reaches a terminal state and stops being retried",
|
|
26721
26721
|
"explicitId": "openwop.requirement.0158.poison-exhaustion",
|
|
26722
26722
|
"citations": [
|
|
@@ -29507,7 +29507,7 @@
|
|
|
29507
29507
|
{
|
|
29508
29508
|
"id": "openwop.it.v2-webhook-durable-delivery.a-failed-attempt-is-retried-and-the-event-is-delivered-at-least-once",
|
|
29509
29509
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29510
|
-
"line":
|
|
29510
|
+
"line": 235,
|
|
29511
29511
|
"title": "a failed attempt is retried and the event is delivered at least once",
|
|
29512
29512
|
"explicitId": "openwop.requirement.0173.webhook-durable-delivery",
|
|
29513
29513
|
"citations": [
|
|
@@ -29538,10 +29538,18 @@
|
|
|
29538
29538
|
{
|
|
29539
29539
|
"id": "openwop.it.v2-webhook-durable-delivery.an-exhausted-delivery-is-dead-lettered-never-dropped",
|
|
29540
29540
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29541
|
-
"line":
|
|
29541
|
+
"line": 323,
|
|
29542
29542
|
"title": "an exhausted delivery is dead-lettered, never dropped",
|
|
29543
29543
|
"explicitId": "openwop.requirement.0173.webhook-durable-delivery.dead-letter",
|
|
29544
29544
|
"citations": [
|
|
29545
|
+
{
|
|
29546
|
+
"section": "webhooks.md §Durability",
|
|
29547
|
+
"requirement": "an exhausted delivery MUST be routed to the sink, not dropped"
|
|
29548
|
+
},
|
|
29549
|
+
{
|
|
29550
|
+
"section": "runs.md §Create",
|
|
29551
|
+
"requirement": "POST /runs MUST answer 201 for the noop fixture"
|
|
29552
|
+
},
|
|
29545
29553
|
{
|
|
29546
29554
|
"section": "runs.md §Create",
|
|
29547
29555
|
"requirement": "POST /runs MUST answer 201 for the noop fixture"
|
|
@@ -29564,7 +29572,7 @@
|
|
|
29564
29572
|
{
|
|
29565
29573
|
"id": "openwop.it.v2-webhook-durable-delivery.the-dead-letter-read-is-served-and-its-records-carry-no-payload",
|
|
29566
29574
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29567
|
-
"line":
|
|
29575
|
+
"line": 417,
|
|
29568
29576
|
"title": "the dead-letter read is served and its records carry no payload",
|
|
29569
29577
|
"explicitId": "openwop.requirement.0188.dead-letter-read",
|
|
29570
29578
|
"citations": [
|
|
@@ -29577,10 +29585,14 @@
|
|
|
29577
29585
|
{
|
|
29578
29586
|
"id": "openwop.it.v2-webhook-durable-delivery.a-dead-letter-record-carries-no-delivered-payload",
|
|
29579
29587
|
"file": "v2-webhook-durable-delivery.test.ts",
|
|
29580
|
-
"line":
|
|
29588
|
+
"line": 438,
|
|
29581
29589
|
"title": "a dead-letter record carries no delivered payload",
|
|
29582
29590
|
"explicitId": "openwop.requirement.0188.dead-letter-content-free",
|
|
29583
29591
|
"citations": [
|
|
29592
|
+
{
|
|
29593
|
+
"section": "RFC 0188 §B.1",
|
|
29594
|
+
"requirement": "a dead-letter record MUST NOT carry the delivered body, headers or subscription secret"
|
|
29595
|
+
},
|
|
29584
29596
|
{
|
|
29585
29597
|
"section": "RFC 0188 §B.1",
|
|
29586
29598
|
"requirement": null,
|