@openwop/openwop-conformance 2.34.1 → 2.35.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,36 @@
1
1
  # `@openwop/openwop-conformance` Changelog
2
2
 
3
+ ## [2.35.0] — 2026-09-21 — blocked means blocked, the declarations are signed, and the header a subscriber reads is checked
4
+
5
+ ### Security
6
+
7
+ - **`v2-webhook-egress-refusal` (`0171.webhook-egress-refused`) now registers loopback, cloud metadata and RFC 1918 written as IPv4-mapped IPv6** — `https://[::ffff:7f00:1]/`, `https://[::ffff:a9fe:a9fe]/`, `https://[::ffff:a00:1]/` — eleven probes, up from eight. A URL parser emits the mapped form in hex (`[::ffff:127.0.0.1]` has hostname `::ffff:7f00:1`), and an egress guard that recognises only the dotted mapped form lets all three through at registration and at delivery. They are loopback, link-local and RFC 1918 addresses under `webhooks.md` §SSRF as written, so this enforces an existing MUST under the same id. Measured: the steward's reference host before its fix accepted exactly these three (`3 of 11`); after the fix, none.
8
+
9
+ ### Changed
10
+
11
+ > **The three rules below are RFC 0195's.** They were merged before that RFC was filed; it was then made `Active` by steward override of RFC 0147 §A.6, whose full comment window it did not run.
12
+
13
+ - **A `blocked` note stands at major 2 even after setup assertions.** A test that asserted something and then soft-skipped `blocked` recorded `executed-pass` with a `partial-witness:` detail — refused by the acceptance predicate, but counted by certification, so a profile could certify on a requirement nobody observed. At major 2 it now records `blocked`, which denies certification (`conformance.md` §Bundle v3). Swept first: every one of the 24 major-2 sites is "requirement unobserved" (unreachable, a control that did not answer, a window that closed), none an optional extra. Major 1 keeps its convention; the 146 v1-side sites were not measured and v1 bundles are read through its end of support. Measured on the committed bundles: the next cut changes 0 rows on the reference host and MyndHyve, 1 on openwop-app.
14
+ - **`host.relaxations[]` is signed.** It sits outside the attestation, so a relaxation deleted after signing let the verifier re-derive the relaxed profile as certified on a bundle that still verified. `witnessSha256` now digests `{ rows, relaxations }` whenever relaxations are declared, and the rows alone otherwise — every committed bundle digests unchanged. An older verifier fails closed (`witness-digest`) on a 2.35.0 bundle that declares relaxations. `v2-relaxation-recorded` now strips a signed relaxation and requires the rejection.
15
+ - **Opt-outs are derived and checked.** The verifier derives the operator's opt-outs from the signed `skipped` rows the behavior gate writes and rejects a bundle whose captured discovery document advertises one (`opted-out-but-advertised`); `--verify` prints the set. No schema change — a root-level list would have been unsigned. The file-level detail now says "profile or family".
16
+
17
+ ### Added
18
+
19
+ - **`v2-terminal-event-once` — RFC 0194 (`0194.terminal-once`, `0194.terminal-once.duplicate-delivery`).** A run's log carries exactly one terminal run event and no forward-execution event after it; `compensation.*` and `run.dead-lettered` may follow. The ordinary leg drives a noop run, and the failure and cancellable fixtures where advertised; the duplicate-delivery leg needs the RFC 0158 durability test hook (`inapplicable` without it). The log is read after it settles. Pure rule in `lib/terminal-shape.ts`, unit-tested against the two logs observed in openwop#1445.
20
+ - **`v2-callback-url-guarded` — RFC 0196 §A.3 (`0196.callback-url-guarded`), invariant `callback-url-egress-guarded`.** A host advertising `interrupt.callbackDelivery` must refuse at `createRun` a loopback, cloud-metadata or hex-mapped-loopback `callbackUrl` with `400 validation_error`, `details.field: callbackUrl`. `inapplicable` where the facet is absent — no committed host advertises it today.
21
+ - **`0187.bound-id-kinds.webhook-emitted` (openwop#1450).** The mint row reads the `webhookId` a host returns; a subscriber identifies its deliveries by the one it emits. A tier-2 host bound the 201 and left the delivery headers bare, and the mint row stayed green — the defect surfaced only as a durability failure elsewhere. The new leg drives one delivery, selects it by the run in the body (never by the header under test), and requires `OpenWOP-Webhook-Id` — and `X-openwop-Webhook-Id` on a host advertising both majors — to equal the minted id.
22
+
23
+ ### Fixed
24
+
25
+ - **`v2-bound-id-kinds` left a subscription behind on every run** (its `deliveryId` leg registered and never deleted). Every registering leg now deletes in `finally`. Measured: 1 leftover subscription per run before, 0 after.
26
+
27
+ ## [2.34.2] — 2026-09-21 — a host that died on schedule failed the rows that were watching it recover
28
+
29
+ - **The RFC 0158 kill rows read the host through two calls that did not survive the host dying.** `waitBack()` treated a refused connection as "down, keep waiting"; the watch's `readStatus` / `readLog` called `driver.get` bare. A seam may answer and *then* die — `during-execution` dies at the first `node.started`, seconds after its response — so `waitBack` saw a 200 from the still-living process, the process died mid-watch, and `fetch failed` escaped the scenario as an **`executed-fail` — for recovering.** Measured by a tier-1 host at its exact production image under a real restart supervisor, behind a port proxy that accepts and then closes: four of five rows `executed-fail` on `UND_ERR_SOCKET: other side closed`, reproduced twice; the one that passed was the row whose kill lands before the seam answers.
30
+ - **A transport failure inside the watch is now an unreadable observation, never a verdict:** a status that cannot be read is `null`, a log that cannot be read is `readable: false` — both already meant "conclude nothing, poll again" — and the budget still bounds the wait, so a host that never comes back still records no resumption. `Watch.transportErrors` counts them. Unit-tested in `lib/durability-watch.test.ts` (dies after answering and comes back → resumption observed; never comes back → none; an unreadable read never latches completed-un-re-executed).
31
+ - **One kill no longer contaminates the rest of the run.** Every seam row now waits for the host to answer at all before probing the seam, and the bound read retries once across a restart. Suite setup retried discovery twice back-to-back, giving a restarting host about a millisecond: each file set up during the restart logged "discovery unreadable" and recorded its fixture gates `blocked`. Setup now waits out a transport failure (refused / reset / closed) for up to 45 s, 1 s apart, without spending an attempt; an HTTP answer and a 20 s timeout still count as attempts, so a host that is up and wrong is not retried into looking right.
32
+ - False FAIL only. Suite patch; corpus release stays `2.34.0`. Reported by openwop-app.
33
+
3
34
  ## [2.34.1] — 2026-09-21 — a 90 s window convicted a host for retrying at its production pace
4
35
 
5
36
  - **`v2-webhook-durable-delivery` capped every wait at a hard 90 s, and a conformant host with a slower schedule could not advertise `webhooks.deadLetter` without failing its bundle.** A host retrying at 15 / 30 / 60 / 120 s with `maxAttempts: 5` makes its attempts at t0, +15, +45, +105, +225 s. The dead-letter leg waited for a second attempt, waited 90 s more, and read the sink ~120 s before that host exhausts: **`0173.webhook-durable-delivery.dead-letter` → `executed-fail`, "an exhausted delivery MUST be routed to the sink, not dropped"**, about a delivery still in flight; `0188.dead-letter-content-free` → `blocked`, which denies certification. To pass, the host would have had to cut its production retry window from ~225 s to ~75 s for every real subscriber — the instrument choosing the host's durability. Reported by a tier-2 host from its own constants *before* advertising the facet; reproduced on the reference host configured to the same schedule.
package/README.md CHANGED
@@ -11,7 +11,7 @@
11
11
  # --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
12
12
  # default resolver refuses. npm 10.9 fails outright with
13
13
  # "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
14
- npm install --legacy-peer-deps @openwop/openwop-conformance@2.34.1 @openwop/spec-artifacts@2.34.1
14
+ npm install --legacy-peer-deps @openwop/openwop-conformance@2.35.0 @openwop/spec-artifacts@2.35.0
15
15
  # or run without install:
16
16
  npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
17
17
  ```
@@ -135,7 +135,7 @@ Exit code is non-zero on any failed assertion. `--certify` distinguishes: `0`
135
135
 
136
136
  ## What's Covered
137
137
 
138
- The current suite has 524 scenario files under `src/scenarios/`.
138
+ The current suite has 526 scenario files under `src/scenarios/`.
139
139
  - 2026-09-03 (suite `1.157.0 -> 1.158.0`, gap G17): NEW `idempotency-concurrent-claim.test.ts` — drives the new `host-sample-test-seams.md` §25 concurrent duplicate-delivery seam for the RFC 0150 §B / `idempotency.md` §"Concurrent duplicates (Layer 2)" atomic-claim MUST, which is unconditional and had no witness of any kind. Asserts every executor mints the SAME `logicalInvocationId` **before** asserting `delivered === 1` — without the identity check a host passes by minting different ids and never colliding, one effect because nothing raced. Not profile-gated and so not opt-out-able (the obligation is unconditional); an unmounted seam records `blocked`, which is not certifiable. Graduates `layer2-invocation-claim-atomic` reference-impl -> protocol.
140
140
  - 2026-08-19 (suite `1.137.0 → 1.138.0`): NEW `durability-poison-exhaustion.test.ts` — RFC 0158 §C.8, the FIRST row of that RFC's conformance table to land. Asserts what `failure-path.test.ts` cannot: not just that deterministically failing work reaches terminal, but that attempts STOP — counted on the log, re-counted after a scaled quiet window, asserted unchanged. A host still redelivering records more. Seam-gated on the existing event-log seam (`blocked` = unobservable, not unmet) and outside every profile floor.
141
141
  - 2026-08-19 (suite `1.136.15 → 1.137.0`): NEW `replay-fanout-suppression.test.ts` — capability-gated on `webhooks.supported`, **outside every profile floor**; witnesses `replay.md` §"Host-initiated fan-out is an external effect", which was the largest normative MUST NOT on the replay surface with no scenario and no SECURITY invariant. Three legs in ONE `it` against ONE receiver and ONE subscription — a positive control, the MUST NOT, and a `branch` boundary leg — because "no delivery arrived" passes identically when delivery never worked, so absence is asserted only after presence is proven on that exact wiring. A host with an SSRF guard correctly refuses the loopback receiver and records `blocked`: **unobservable, not unmet.**
@@ -480,7 +480,7 @@ Server-required (added in 1.7.0):
480
480
  | ------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
481
481
  | **Redaction** | [`capabilities.md`](../spec/v1/capabilities.md) §"Secrets" + NFR-7 + §"aiProviders" | Vendor-neutral assertions that the server doesn't leak secret material. Three scenario groups: (a) discovery shape contract — `secrets` + `aiProviders` advertisements are well-formed regardless of `secrets.supported`; when `supported === true`, scopes MUST be non-empty + `resolution === 'host-managed'`; `byok ⊆ supported`. (b) bearer-token redaction — invalid Bearer canary in `Authorization` header is not echoed in the 401 response body. (c) credentialRef echo control — gated on `secrets.supported === true`; canary planted in `configurable.ai.credentialRef` MUST NOT appear in any RunEvent payload (poll-based capture; transport-agnostic). Uses runtime-built canary fixtures (`lib/canaries.ts`) that defeat static secret scanners. 6 scenarios. |
482
482
 
483
- Current source tree: 524 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
483
+ Current source tree: 526 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
484
484
 
485
485
  ## Remaining Gaps
486
486
 
package/coverage.md CHANGED
@@ -144,7 +144,9 @@ The scenario groups in the table below (one row per group; count the rows — th
144
144
  | `tool-session-lifecycle.test.ts` | `capabilities.toolCatalog.sessionLifecycle` (RFC 0078 §D, `tool-catalog.md`) | A (the §D bracket via `POST /v1/host/sample/tools/session-run` + the test event-log seam: `tool.session.opened` before the first RFC 0064 call event → `tool.session.closed` after the last, one shared `sessionId`, each carrying a `toolId`, `closed.outcome` ∈ {completed,failed,aborted,expired}, both content-free) | `host-pending` | `behaviorGate('openwop-tool-session-lifecycle', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0078 → Accepted bar.** First adopter: MyndHyve `toolCatalog`. |
145
145
  | `egress-audience-binding.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §C, `host-capabilities.md`) + `SECURITY/invariants.yaml` `egress-credential-audience-bound` | A (KEYSTONE — the §C confused-deputy MUST via `POST /v1/host/sample/egress/decide`: an out-of-audience egress is `denied`/`downgraded` with `reason:"out-of-audience"` and the credential is NOT attached (`credentialAttached !== true`); a provenance-unevaluable egress fails closed `denied`+`reason:"provenance-unevaluable"`; decision/reason ∈ the closed enums) | `host-pending` | `behaviorGate('openwop-egress-audience-binding', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0079 → Accepted bar** (the `egress-credential-audience-bound` invariant graduates reference-impl → protocol tier when this passes against a host). First adopter: MyndHyve `httpClient.egressPolicy`. |
146
146
  | `egress-decision-content-free.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §F / SR-1) | A (the secret non-leak — a `canary` credential's sentinel never surfaces in the decision (`canaryLeaked !== true`), the `egress.decided` payload carries no forbidden content key, and `reason` stays in the CLOSED vocabulary so no blocked destination spills into a free-form field) | `host-pending` | `behaviorGate('openwop-egress-decision-content-free', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0079 → Accepted bar.** First adopter: MyndHyve `httpClient.egressPolicy`. |
147
- | `v2-webhook-egress-refusal.test.ts` | the `webhooks` family (capability-gated) | A (`webhooks.md` §SSRF at MAJOR 2 — eight registration probes, one per class the sentence names: non-`https`, IPv4/IPv6 loopback, `localhost`, two RFC 1918 ranges, link-local / cloud metadata, IPv6 ULA; each MUST answer `400 webhook_url_rejected`. Until 2.33.0 no major-2 row asserted the guard refuses anything — the refusal scenarios were major-1 only. THREE OUTCOMES: all refused → `executed-pass`; a probe accepted with a `webhooks.*` relaxation DECLARED in `OPENWOP_HOST_RELAXATIONS` → `inapplicable`, and the declared relaxation denies the owning profile; a probe accepted with NOTHING declared → `executed-fail`, which the emitter and verifier read as an OBSERVED undeclared relaxation and deny every profile built on `webhooks` (`undeclared-relaxation-observed`). Registration-time only: no probe is a destination the suite owns, nothing is delivered, an accepted registration is deleted at once) |
147
+ | `v2-webhook-egress-refusal.test.ts` | the `webhooks` family (capability-gated) | A (`webhooks.md` §SSRF at MAJOR 2 — **eleven** registration probes since 2.35.0: the eight below plus loopback, cloud metadata and RFC 1918 written as IPv4-mapped IPv6, which a URL parser emits in HEX (`[::ffff:127.0.0.1]` has hostname `::ffff:7f00:1`) and a guard matching only the dotted mapped form lets through — same id, the same MUST. The original eight probes, one per class the sentence names: non-`https`, IPv4/IPv6 loopback, `localhost`, two RFC 1918 ranges, link-local / cloud metadata, IPv6 ULA; each MUST answer `400 webhook_url_rejected`. Until 2.33.0 no major-2 row asserted the guard refuses anything — the refusal scenarios were major-1 only. THREE OUTCOMES: all refused → `executed-pass`; a probe accepted with a `webhooks.*` relaxation DECLARED in `OPENWOP_HOST_RELAXATIONS` → `inapplicable`, and the declared relaxation denies the owning profile; a probe accepted with NOTHING declared → `executed-fail`, which the emitter and verifier read as an OBSERVED undeclared relaxation and deny every profile built on `webhooks` (`undeclared-relaxation-observed`). Registration-time only: no probe is a destination the suite owns, nothing is delivered, an accepted registration is deleted at once) |
148
+ | `v2-terminal-event-once.test.ts` | none for the ordinary leg (every host has runs; the failed and cancelled fixtures are each gated on being advertised); the RFC 0158 durability test hook for the duplicate-delivery leg | A (RFC 0194 §A: a run's log carries exactly one terminal run event and no forward-execution event after it — `compensation.*` and `run.dead-lettered` may follow. The log is read after it SETTLES, two reads ≥ 2 s apart agreeing, bounded at 20 s, because a second delivery can finish after the first terminal event. Two ids: `0194.terminal-once`, `0194.terminal-once.duplicate-delivery`) | `host-pending` | A dedicated file on purpose: a violation must fail this RFC's rows, not unrelated ones. |
149
+ | `v2-callback-url-guarded.test.ts` | the `interrupt` family with `interrupt.callbackDelivery: true` | A (RFC 0196 §A.3: `createRun` with a loopback, cloud-metadata or hex-mapped-loopback `callbackUrl` MUST answer `400 validation_error`, `details.field: callbackUrl`; invariant `callback-url-egress-guarded`) | `host-pending` | No committed host advertises the facet; witnessed on none today. |
148
150
  | `v2-durability-recovery.test.ts` | the RFC 0158 host-extension durability seam `POST /host/durability/kill` (non-normative; §E mints no capability field, so seam PRESENCE is how a host says it claims a rung) | A (all five `durable-single-instance` rows: `kill-after-accept` as a HOLD-DISPATCH exercise, `kill-during-execution` asserting that work executing at a real process death is never observable as completed without re-execution, `duplicate-delivery` counted AT THE EFFECT'S DESTINATION — a receiver the suite owns, `effectUrl` — because a ledger keyed on effect identity admits one row per identity by construction and cannot show a double-fire (2.32.0), `bound-is-derived` as an explicitly-labelled paper check, and `poison-exhaustion` ported from the major-1 scenario — at major 2 it reads the canonical JSON log read `GET /runs/{runId}/events/poll` (NOT `/events`, which is SSE-only at major 2 — reading it as JSON made this row pass vacuously on a host serving exactly the OpenAPI; 2.32.0), so unlike its v1 twin it needs NO seam and can never record `blocked` for want of one a host did not wire) | `host-pending` | **2.34.0 — the rows now WRITE evidence into the bundle (RFC 0158 §E):** `bound-is-derived` records `evidence.recoveryBounds[]` per recovery class and each kill row records `evidence.recovery { class, boundMs, observedMs }` — the interval was always measured and, on a pass, never recorded. Both ride on the ROW so the witness digest, and through it the signature, covers them; the bundle claims `durability.rung` only when `deriveRung` supports it and `--verify` rejects a claim it cannot re-derive (`rung-not-derivable`). A kill row records nothing unless the seam NAMES the class it exercised (`recoveryClass`) and the bound response declares that class — no name, no rung, rather than a guessed one. **Disposition rule, and it protects fleet certification:** no seam route at all ⇒ `inapplicable` (the host claims no rung); seam present but an operator precondition unmet — a restart supervisor, which a black-box suite cannot supply — ⇒ `blocked` with the precondition NAMED. Collapsing the two would deny certification to every host the day these rows entered the lane. `peer-resume` is deliberately NOT written: §E makes it bundle-witnessed, so a scenario could only ever record `blocked`. **This is the RFC 0158 → Accepted bar, and it is now complete at major 2** — before the port, a major-2 bundle could carry at most four of the five rows and the rung was unwitnessable no matter what a host did, and its witness is openwop-app, which the RFC records as having observed `kill-during-execution` across a real `SIGKILL`. |
149
151
  | `memory-degraded-projection.test.ts` | `capabilities.agents.manifestRuntime.supported` + `capabilities.memory.supported` (RFC 0080 §C, `agent-memory.md`) | A (the §C iff-contract on the NORMATIVE `GET /v1/agents`: a degraded entry MUST carry `memoryDegraded:true` + a non-empty, unique `degradedMemoryDimensions[]` drawn from the closed §A-name enum [read/write/search/long-term-durability/compaction/attribution/replay-snapshot/retention]; a non-degraded entry MUST NOT carry a non-empty list; the inventory is non-empty; the degraded branch runs non-vacuously when `OPENWOP_DEGRADED_AGENT_ID` names a known-degraded agent) | `host-pending` | `behaviorGate('openwop-memory-degraded', …)`. Black-box on the normative path (no POST seam); soft-skips on 404 / when the host computes no degradation. **This is the RFC 0080 → Accepted bar.** First adopter: MyndHyve `memory`. |
150
152
  | `budget-enforcement.test.ts` | `capabilities.budget.supported` (RFC 0084 §C/§D, `budget-policy.md`) + `SECURITY/invariants.yaml` `budget-no-pricing-leak` | A (the §C/§D enforcement via `POST /v1/host/sample/budget/run` + the test event-log seam: a `hard-cost-exhaust` run emits the strict-ordered `budget.reserved → budget.consumed → budget.threshold.crossed{percent} → budget.exhausted → cap.breached{kind:"budget-cost"} → run.failed{error:"budget_exhausted"}` chain; a `model-denied` run is refused `budget_model_denied` BEFORE the provider call (fail-closed); an `advisory` host emits the `budget.*` events without stopping; every `budget.*` payload content-free — no pricing/rate) | `host-pending` | `behaviorGate('openwop-budget-enforcement', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0084 → Accepted bar.** First adopter: MyndHyve `budget`. |
package/dist/cli.js CHANGED
@@ -697,7 +697,7 @@ async function runCertify(args, baseUrl, apiKey) {
697
697
  discovery: { url: discoveryUrl, sha256, protocolVersions, preferredVersion, document },
698
698
  claimedProfiles: claimed3,
699
699
  results: { totals: totals3, requirements: rows3 },
700
- witnessSha256: witnessDigest(rows3),
700
+ witnessSha256: witnessDigest(rows3, relaxations),
701
701
  assertionCount: rows3.reduce((n, r) => n + (r.assertions ?? 0), 0),
702
702
  ...(nonPass.length ? { detail: { nonPass: nonPass.map((r) => ({ id: r.id, result: r.result, reason: r.detail ?? '' })) } } : {}),
703
703
  // RFC 0158 §D: claimed ONLY when these rows support it. The verifier
@@ -908,6 +908,8 @@ async function main() {
908
908
  `certified: ${verdict.certifiedProfiles.length > 0 ? verdict.certifiedProfiles.join(', ') : '(none)'}`,
909
909
  // RFC 0158 §D/§E: the rung is a CLAIM; a claim the signed rows do not support is a rejection below.
910
910
  `rung: ${bundle.durability?.rung ?? '(none claimed)'}`,
911
+ // 2.35.0: derived from the signed `skipped` rows; checked against the signed discovery document.
912
+ `opted out: ${verdict.optedOut.length > 0 ? verdict.optedOut.join(', ') : '(none)'}`,
911
913
  '',
912
914
  'What this command does NOT do:',
913
915
  ' · It does not re-run anything. A host that measured itself wrongly, and signed',
@@ -31,12 +31,26 @@ export function canonicalJSON(value) {
31
31
  }
32
32
  return JSON.stringify(value);
33
33
  }
34
- /** RFC 0148 §C — the digest over the reporter record (the requirement rows). */
35
- export function witnessDigest(rows) {
34
+ /**
35
+ * RFC 0148 §C — the digest over the reporter record (the requirement rows),
36
+ * and, from 2.35.0, the operator's declared relaxations when there are any.
37
+ *
38
+ * Why relaxations are in here: the attestation covers exactly
39
+ * `{ witnessSha256, host.build, suite.version, discovery.sha256 }`, so a
40
+ * `host.relaxations[]` beside the rows was UNSIGNED — strip it after signing
41
+ * and the verifier re-derives the relaxed profile as certified, on a bundle
42
+ * that still verifies. Included ONLY WHEN NON-EMPTY: the preimage is the rows
43
+ * array exactly as before for every bundle that declares nothing (all
44
+ * committed bundles digest unchanged, pinned in durability-evidence.test.ts),
45
+ * and `{ rows, relaxations }` otherwise. An older verifier fails closed on a
46
+ * 2.35.0 bundle that declares relaxations — `witness-digest`, never a pass.
47
+ */
48
+ export function witnessDigest(rows, relaxations) {
36
49
  const canonicalRows = [...rows].sort((a, b) => a.id.localeCompare(b.id)).map((r) => ({ id: r.id, scenario: r.scenario, result: r.result, ...(r.assertions === undefined ? {} : { assertions: r.assertions }), ...(r.detail === undefined ? {} : { detail: r.detail }),
37
50
  // ONLY WHEN PRESENT: every bundle cut before 2.34.0 has no `evidence` and digests byte-identically.
38
51
  ...(r.evidence === undefined ? {} : { evidence: r.evidence }) }));
39
- return createHash('sha256').update(canonicalJSON(canonicalRows), 'utf8').digest('hex');
52
+ const preimage = relaxations !== undefined && relaxations.length > 0 ? { rows: canonicalRows, relaxations } : canonicalRows;
53
+ return createHash('sha256').update(canonicalJSON(preimage), 'utf8').digest('hex');
40
54
  }
41
55
  /** The bytes the attestation covers. */
42
56
  export function attestationPayload(b) {
@@ -58,6 +72,14 @@ export function verifierSign(bundle, privateKeyPem, keyId) {
58
72
  export function publicKeyFromPrivate(privateKeyPem) {
59
73
  return createPublicKey(createPrivateKey(privateKeyPem)).export({ type: 'spki', format: 'pem' });
60
74
  }
75
+ const OPT_OUT_ROW = 'openwop.profile.';
76
+ /** The opt-outs a bundle's signed rows record. */
77
+ export function optedOutFromRows(rows) {
78
+ return rows
79
+ .filter((r) => r.id.startsWith(OPT_OUT_ROW) && r.result === 'skipped' && (r.detail ?? '').includes('OPENWOP_OPTED_OUT_PROFILES'))
80
+ .map((r) => r.id.slice(OPT_OUT_ROW.length))
81
+ .sort();
82
+ }
61
83
  export function verifyBundleV3(bundle, opts = {}) {
62
84
  const rejections = [];
63
85
  if (bundle.bundleVersion !== '3')
@@ -78,9 +100,9 @@ export function verifyBundleV3(bundle, opts = {}) {
78
100
  for (const k of Object.keys(expected))
79
101
  if (bundle.results?.totals?.[k] !== expected[k])
80
102
  rejections.push({ kind: 'totals-mismatch', detail: `totals.${k} is ${String(bundle.results?.totals?.[k])} but the rows count ${expected[k]}` });
81
- const digest = witnessDigest(rows);
103
+ const digest = witnessDigest(rows, bundle.host?.relaxations);
82
104
  if (bundle.witnessSha256 !== digest)
83
- rejections.push({ kind: 'witness-digest', detail: `witnessSha256 ${String(bundle.witnessSha256).slice(0, 12)} does not equal the digest of the rows (${digest.slice(0, 12)})` });
105
+ rejections.push({ kind: 'witness-digest', detail: `witnessSha256 ${String(bundle.witnessSha256).slice(0, 12)} does not equal the digest of the rows and declared relaxations (${digest.slice(0, 12)})` });
84
106
  const assertions = rows.reduce((n, r) => n + (r.assertions ?? 0), 0);
85
107
  if (bundle.assertionCount !== assertions)
86
108
  rejections.push({ kind: 'assertion-count', detail: `assertionCount is ${String(bundle.assertionCount)} but the rows sum to ${assertions}` });
@@ -183,6 +205,21 @@ export function verifyBundleV3(bundle, opts = {}) {
183
205
  }
184
206
  else {
185
207
  derivabilityChecked = true;
208
+ // RFC 0148 §B: "A host MUST NOT both advertise and opt out of the same
209
+ // profile." The suite refuses that at run time (behaviorGate throws),
210
+ // so a bundle that records both was edited after the run or emitted by
211
+ // something else. Both inputs are signed — the rows through
212
+ // witnessSha256, the document through discovery.sha256 — so the
213
+ // contradiction is checkable offline, and it is bundle-wide: an opt-out
214
+ // list that is false makes every `skipped` row in the bundle suspect.
215
+ const root = document;
216
+ for (const name of optedOutFromRows(rows)) {
217
+ const advertised = name.startsWith('family.')
218
+ ? (() => { const v = root[name.slice('family.'.length)]; return typeof v === 'object' && v !== null && !Array.isArray(v); })()
219
+ : profileDerivable(document, name, targetMajor);
220
+ if (advertised)
221
+ rejections.push({ kind: 'opted-out-but-advertised', detail: `${name} is recorded as an operator opt-out, but the captured discovery document advertises it — a host MUST NOT both advertise and opt out of the same profile (RFC 0148 §B)` });
222
+ }
186
223
  for (const p of bundle.claimedProfiles ?? []) {
187
224
  // §B(1) binds the CERTIFICATION, not the listing: a profile a bundle
188
225
  // names without certifying makes no claim for derivability to falsify.
@@ -209,5 +246,6 @@ export function verifyBundleV3(bundle, opts = {}) {
209
246
  verifierSignatureVerified,
210
247
  certifiedProfiles: bundleWide.length > 0 ? [] : certifiedProfiles.filter((p) => !scoped.has(p)),
211
248
  derivabilityChecked,
249
+ optedOut: optedOutFromRows(rows),
212
250
  };
213
251
  }
@@ -96,13 +96,27 @@ export const PARTIAL_WITNESS_PREFIX = 'partial-witness: ';
96
96
  * dishonest per-`it` rows denied certification to every profile on a host that
97
97
  * simply did not advertise the gated surface.
98
98
  */
99
- export function resolveItRecord(state, assertionCalls, gate, noted, firstError) {
99
+ export function resolveItRecord(state, assertionCalls, gate, noted, firstError,
100
+ /**
101
+ * 2.35.0, major 2 (`conformance.md` §Bundle v3): a requirement the test did
102
+ * not observe records `blocked` even when the test asserted setup facts
103
+ * first. At major 1 a `blocked` note after assertions stays a
104
+ * `partial-witness` pass unless the leg used `blockedDespiteAssertions`.
105
+ *
106
+ * Why the rule changed: a `partial-witness` row is refused by the acceptance
107
+ * predicate but COUNTED by certification, so a profile could certify on a
108
+ * requirement nobody observed. Swept before the change: all 24 major-2
109
+ * sites were "requirement unobserved" (unreachable, control silent, window
110
+ * closed), none an optional extra. Scoped to major 2 because the 146 v1-side
111
+ * sites were not measured and v1 bundles are read through its EOS.
112
+ */
113
+ blockedStands = false) {
100
114
  if (state === 'fail')
101
115
  return { disposition: 'executed-fail', detail: `the test executed and failed: ${(firstError ?? 'no message').slice(0, 300)}` };
102
116
  if (state === 'pass' && assertionCalls > 0) {
103
117
  // `blockedDespiteAssertions` (soft-skip.ts): the leg says its setup
104
118
  // assertions are not the requirement, and the requirement went unobserved.
105
- if (noted !== null && noted.kind === 'blocked' && noted.conclusive === true)
119
+ if (noted !== null && noted.kind === 'blocked' && (noted.conclusive === true || blockedStands))
106
120
  return { disposition: 'blocked', detail: noted.reason };
107
121
  // A leg that asserted AND THEN soft-skipped is only a partial witness, and
108
122
  // the file-level record has always said so (`resolveFileRecord` below).
@@ -141,14 +155,14 @@ export function fileDisposition(states, gateReason, assertionCount) {
141
155
  return { disposition: 'inapplicable', detail: 'every test returned early through behaviorGate with zero assertions: profile not advertised in the captured discovery set' };
142
156
  }
143
157
  if (assertionCount === 0 && gateReason === 'skipped') {
144
- return { disposition: 'skipped', detail: 'every test returned early through behaviorGate with zero assertions: operator opted the profile out (OPENWOP_OPTED_OUT_PROFILES)' };
158
+ return { disposition: 'skipped', detail: 'every test returned early through behaviorGate with zero assertions: operator opted the profile or family out (OPENWOP_OPTED_OUT_PROFILES)' };
145
159
  }
146
160
  return { disposition: 'executed-pass' };
147
161
  }
148
162
  if (gateReason === 'inapplicable')
149
163
  return { disposition: 'inapplicable', detail: 'every test skipped: profile not advertised in the captured discovery set (behaviorGate)' };
150
164
  if (gateReason === 'skipped')
151
- return { disposition: 'skipped', detail: 'every test skipped: operator opted the profile out (OPENWOP_OPTED_OUT_PROFILES)' };
165
+ return { disposition: 'skipped', detail: 'every test skipped: operator opted the profile or family out (OPENWOP_OPTED_OUT_PROFILES)' };
152
166
  return {
153
167
  disposition: 'blocked',
154
168
  detail: states.length === 0
@@ -1,5 +1,5 @@
1
1
  {
2
2
  "package": "@openwop/spec-artifacts",
3
- "version": "2.34.1",
4
- "stampSha256": "3f491c227a43a26ab8f77081df1443d066b404d3af64e50c5e9786d8f03462e5"
3
+ "version": "2.35.0",
4
+ "stampSha256": "164c05f7216f16c89a05636375b4e5eaebce3c461eb3c24fb5a992750633765e"
5
5
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@openwop/openwop-conformance",
3
- "version": "2.34.1",
3
+ "version": "2.35.0",
4
4
  "description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
5
5
  "repository": {
6
6
  "type": "git",
@@ -56,6 +56,6 @@
56
56
  "@openwop/spec-artifacts": "file:../spec-artifacts"
57
57
  },
58
58
  "peerDependencies": {
59
- "@openwop/spec-artifacts": "2.34.1"
59
+ "@openwop/spec-artifacts": "2.35.0"
60
60
  }
61
61
  }
package/requirements.json CHANGED
@@ -2,11 +2,11 @@
2
2
  "$comment": "GENERATED by conformance/scripts/generate-requirement-registry.mjs — do not edit. One record per it()/test() in src/scenarios. Ids: openwop.it.<file-stem>.<title-slug>[~n] (src/lib/requirement-ids.ts). A record with id null has an interpolated title; its run-time row is keyed by the rendered title and maps here by file+line only. Renamed ids need a row in requirement-aliases.json.",
3
3
  "generatedFrom": "src/scenarios/*.test.ts",
4
4
  "counts": {
5
- "files": 582,
6
- "tests": 2209,
7
- "withStableId": 2209,
5
+ "files": 584,
6
+ "tests": 2213,
7
+ "withStableId": 2213,
8
8
  "interpolatedTitles": 0,
9
- "explicitIds": 2163
9
+ "explicitIds": 2167
10
10
  },
11
11
  "records": [
12
12
  {
@@ -26009,7 +26009,7 @@
26009
26009
  {
26010
26010
  "id": "openwop.it.v2-bound-id-kinds.subscriptionid-post-webhooks-mints-a-bound-webhookid-the-projected-segment-is-ac",
26011
26011
  "file": "v2-bound-id-kinds.test.ts",
26012
- "line": 61,
26012
+ "line": 89,
26013
26013
  "title": "subscriptionId: POST /webhooks mints a bound webhookId, the projected segment is accepted, and a foreign tenant segment is refused 403",
26014
26014
  "explicitId": "openwop.requirement.0187.bound-id-kinds.webhook",
26015
26015
  "citations": [
@@ -26033,10 +26033,29 @@
26033
26033
  }
26034
26034
  ]
26035
26035
  },
26036
+ {
26037
+ "id": "openwop.it.v2-bound-id-kinds.subscriptionid-the-webhookid-a-host-emits-on-a-delivery-is-the-one-it-minted",
26038
+ "file": "v2-bound-id-kinds.test.ts",
26039
+ "line": 132,
26040
+ "title": "subscriptionId: the webhookId a host EMITS on a delivery is the one it minted",
26041
+ "explicitId": "openwop.requirement.0187.bound-id-kinds.webhook-emitted",
26042
+ "citations": [
26043
+ {
26044
+ "section": "webhooks.md §Headers (RFC 0187 §A.1)",
26045
+ "requirement": null,
26046
+ "interpolated": true
26047
+ },
26048
+ {
26049
+ "section": "webhooks.md §Headers (dual emission, RFC 0176 §D.2)",
26050
+ "requirement": null,
26051
+ "interpolated": true
26052
+ }
26053
+ ]
26054
+ },
26036
26055
  {
26037
26056
  "id": "openwop.it.v2-bound-id-kinds.interruptid-the-id-a-suspension-puts-on-the-wire-is-bound",
26038
26057
  "file": "v2-bound-id-kinds.test.ts",
26039
- "line": 95,
26058
+ "line": 179,
26040
26059
  "title": "interruptId: the id a suspension puts on the wire is bound",
26041
26060
  "explicitId": null,
26042
26061
  "citations": []
@@ -26044,7 +26063,7 @@
26044
26063
  {
26045
26064
  "id": "openwop.it.v2-bound-id-kinds.effectid-every-row-get-runs-runid-effects-serves-is-bound",
26046
26065
  "file": "v2-bound-id-kinds.test.ts",
26047
- "line": 116,
26066
+ "line": 200,
26048
26067
  "title": "effectId: every row GET /runs/{runId}/effects serves is bound",
26049
26068
  "explicitId": "openwop.requirement.0187.bound-id-kinds.per-kind",
26050
26069
  "citations": [
@@ -26057,7 +26076,7 @@
26057
26076
  {
26058
26077
  "id": "openwop.it.v2-bound-id-kinds.deliveryid-every-record-in-the-dead-letter-read-carries-a-bound-id-in-the-caller",
26059
26078
  "file": "v2-bound-id-kinds.test.ts",
26060
- "line": 131,
26079
+ "line": 215,
26061
26080
  "title": "deliveryId: every record in the dead-letter read carries a bound id in the caller's tenant",
26062
26081
  "explicitId": "openwop.requirement.0187.bound-id-kinds.per-kind",
26063
26082
  "citations": [
@@ -26184,6 +26203,23 @@
26184
26203
  }
26185
26204
  ]
26186
26205
  },
26206
+ {
26207
+ "id": "openwop.it.v2-callback-url-guarded.a-host-advertising-callback-delivery-refuses-a-callbackurl-its-egress-guard-woul",
26208
+ "file": "v2-callback-url-guarded.test.ts",
26209
+ "line": 36,
26210
+ "title": "a host advertising callback delivery refuses a callbackUrl its egress guard would refuse",
26211
+ "explicitId": "openwop.requirement.0196.callback-url-guarded",
26212
+ "citations": [
26213
+ {
26214
+ "section": "spec/v2/core/interrupt.md §Callback delivery (RFC 0196 §A.3)",
26215
+ "requirement": "a host advertising interrupt.callbackDelivery MUST refuse at createRun a callbackUrl the webhooks.md §SSRF registration guard would refuse"
26216
+ },
26217
+ {
26218
+ "section": "spec/v2/core/interrupt.md §Callback delivery (RFC 0196 §A.3)",
26219
+ "requirement": "the refusal MUST be 400 validation_error naming details.field \"callbackUrl\""
26220
+ }
26221
+ ]
26222
+ },
26187
26223
  {
26188
26224
  "id": "openwop.it.v2-capabilities-root-closed.the-host-document-validates-against-capabilities-schema-json",
26189
26225
  "file": "v2-capabilities-root-closed.test.ts",
@@ -26650,7 +26686,7 @@
26650
26686
  {
26651
26687
  "id": "openwop.it.v2-durability-recovery.accepted-work-survives-a-kill-before-dispatch-and-dispatches-on-resume",
26652
26688
  "file": "v2-durability-recovery.test.ts",
26653
- "line": 283,
26689
+ "line": 292,
26654
26690
  "title": "accepted work survives a kill before dispatch and dispatches on resume",
26655
26691
  "explicitId": "openwop.requirement.0158.kill-after-accept",
26656
26692
  "citations": [
@@ -26664,7 +26700,7 @@
26664
26700
  {
26665
26701
  "id": "openwop.it.v2-durability-recovery.work-executing-at-a-real-process-death-is-never-reported-complete-and-resumes",
26666
26702
  "file": "v2-durability-recovery.test.ts",
26667
- "line": 337,
26703
+ "line": 346,
26668
26704
  "title": "work executing at a real process death is never reported complete, and resumes",
26669
26705
  "explicitId": "openwop.requirement.0158.kill-during-execution",
26670
26706
  "citations": [
@@ -26683,7 +26719,7 @@
26683
26719
  {
26684
26720
  "id": "openwop.it.v2-durability-recovery.the-same-accepted-work-delivered-twice-fires-each-effect-exactly-once",
26685
26721
  "file": "v2-durability-recovery.test.ts",
26686
- "line": 393,
26722
+ "line": 402,
26687
26723
  "title": "the same accepted work delivered twice fires each effect exactly once",
26688
26724
  "explicitId": "openwop.requirement.0158.duplicate-delivery",
26689
26725
  "citations": [
@@ -26702,7 +26738,7 @@
26702
26738
  {
26703
26739
  "id": "openwop.it.v2-durability-recovery.the-declared-recovery-bound-is-derived-from-the-mechanism-that-enforces-it",
26704
26740
  "file": "v2-durability-recovery.test.ts",
26705
- "line": 467,
26741
+ "line": 476,
26706
26742
  "title": "the declared recovery bound is derived from the mechanism that enforces it",
26707
26743
  "explicitId": "openwop.requirement.0158.bound-is-derived",
26708
26744
  "citations": [
@@ -26716,7 +26752,7 @@
26716
26752
  {
26717
26753
  "id": "openwop.it.v2-durability-recovery.deterministically-failing-work-reaches-a-terminal-state-and-stops-being-retried",
26718
26754
  "file": "v2-durability-recovery.test.ts",
26719
- "line": 498,
26755
+ "line": 507,
26720
26756
  "title": "deterministically failing work reaches a terminal state and stops being retried",
26721
26757
  "explicitId": "openwop.requirement.0158.poison-exhaustion",
26722
26758
  "citations": [
@@ -28418,13 +28454,17 @@
28418
28454
  {
28419
28455
  "section": "security-defaults.md §Relaxations",
28420
28456
  "requirement": "a relaxation recorded against an unclaimed profile is not a rejection — recording is the obligation, claiming is the defect"
28457
+ },
28458
+ {
28459
+ "section": "conformance.md §Bundle v3",
28460
+ "requirement": "a relaxation removed after signing MUST be detected — the declaration is covered by witnessSha256, so the stripped bundle fails `witness-digest`"
28421
28461
  }
28422
28462
  ]
28423
28463
  },
28424
28464
  {
28425
28465
  "id": "openwop.it.v2-relaxation-recorded.a-relaxation-denies-the-real-profile-that-owns-its-family-not-only-one-named-aft",
28426
28466
  "file": "v2-relaxation-recorded.test.ts",
28427
- "line": 118,
28467
+ "line": 130,
28428
28468
  "title": "a relaxation denies the REAL profile that owns its family, not only one named after it",
28429
28469
  "explicitId": "openwop.requirement.0173.relaxation-recorded.owned-by-registry",
28430
28470
  "citations": [
@@ -28449,7 +28489,7 @@
28449
28489
  {
28450
28490
  "id": "openwop.it.v2-relaxation-recorded.durability-is-a-closed-set-session-deployment-persisted",
28451
28491
  "file": "v2-relaxation-recorded.test.ts",
28452
- "line": 143,
28492
+ "line": 155,
28453
28493
  "title": "durability is a closed set: session | deployment | persisted",
28454
28494
  "explicitId": "openwop.requirement.0173.relaxation-recorded.durability-closed",
28455
28495
  "citations": [
@@ -29253,6 +29293,46 @@
29253
29293
  }
29254
29294
  ]
29255
29295
  },
29296
+ {
29297
+ "id": "openwop.it.v2-terminal-event-once.every-run-observed-to-a-terminal-state-has-exactly-one-terminal-event-and-no-for",
29298
+ "file": "v2-terminal-event-once.test.ts",
29299
+ "line": 71,
29300
+ "title": "every run observed to a terminal state has exactly one terminal event and no forward execution after it",
29301
+ "explicitId": "openwop.requirement.0194.terminal-once",
29302
+ "citations": [
29303
+ {
29304
+ "section": "spec/v2/core/events.md §The terminal event (RFC 0194 §A)",
29305
+ "requirement": "a run's log MUST contain exactly one terminal run event and no forward-execution event after it"
29306
+ },
29307
+ {
29308
+ "section": "spec/v2/core/events.md §The terminal event (RFC 0194 §A)",
29309
+ "requirement": null,
29310
+ "interpolated": true
29311
+ }
29312
+ ]
29313
+ },
29314
+ {
29315
+ "id": "openwop.it.v2-terminal-event-once.work-delivered-twice-still-leaves-one-terminal-event-and-no-forward-execution-af",
29316
+ "file": "v2-terminal-event-once.test.ts",
29317
+ "line": 99,
29318
+ "title": "work delivered twice still leaves one terminal event and no forward execution after it",
29319
+ "explicitId": "openwop.requirement.0194.terminal-once.duplicate-delivery",
29320
+ "citations": [
29321
+ {
29322
+ "section": "spec/v2/core/events.md §The terminal event (RFC 0194 §A)",
29323
+ "requirement": "under a duplicate delivery the log MUST still contain exactly one terminal run event and no forward-execution event after it"
29324
+ },
29325
+ {
29326
+ "section": "spec/v2/core/events.md §The terminal event (RFC 0194 §A)",
29327
+ "requirement": "the redelivered run's log reached a terminal event"
29328
+ },
29329
+ {
29330
+ "section": "spec/v2/core/events.md §The terminal event (RFC 0194 §A)",
29331
+ "requirement": null,
29332
+ "interpolated": true
29333
+ }
29334
+ ]
29335
+ },
29256
29336
  {
29257
29337
  "id": "openwop.it.v2-unmapped-type-refused.poll-over-a-log-with-an-unmapped-unprefixed-type-fails-with-500-event-type-unmap",
29258
29338
  "file": "v2-unmapped-type-refused.test.ts",
@@ -29603,7 +29683,7 @@
29603
29683
  {
29604
29684
  "id": "openwop.it.v2-webhook-egress-refusal.a-webhook-registration-naming-a-destination-the-guard-must-refuse-is-refused-400",
29605
29685
  "file": "v2-webhook-egress-refusal.test.ts",
29606
- "line": 82,
29686
+ "line": 92,
29607
29687
  "title": "a webhook registration naming a destination the guard MUST refuse is refused 400 webhook_url_rejected",
29608
29688
  "explicitId": "openwop.requirement.0171.webhook-egress-refused",
29609
29689
  "citations": [
@@ -1,9 +1,9 @@
1
1
  {
2
2
  "$comment": "GENERATED by conformance/scripts/generate-scenario-majors.mjs (RFC 0168 §D.3). Do not edit; add a file to BOTH_MAJORS in the generator to target both majors.",
3
3
  "counts": {
4
- "files": 524,
4
+ "files": 526,
5
5
  "v1": 445,
6
- "v2": 81
6
+ "v2": 83
7
7
  },
8
8
  "majors": {
9
9
  "a2a-1-0-agent-card.test.ts": [
@@ -1253,6 +1253,9 @@
1253
1253
  "v2-bundle-v3-signed.test.ts": [
1254
1254
  2
1255
1255
  ],
1256
+ "v2-callback-url-guarded.test.ts": [
1257
+ 2
1258
+ ],
1256
1259
  "v2-capabilities-root-closed.test.ts": [
1257
1260
  2
1258
1261
  ],
@@ -1445,6 +1448,9 @@
1445
1448
  "v2-subject-link-record.test.ts": [
1446
1449
  2
1447
1450
  ],
1451
+ "v2-terminal-event-once.test.ts": [
1452
+ 2
1453
+ ],
1448
1454
  "v2-unmapped-type-refused.test.ts": [
1449
1455
  2
1450
1456
  ],