@openwop/openwop-conformance 2.32.0 → 2.33.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +5 -4
- package/coverage.md +2 -1
- package/dist/cli.js +4 -2
- package/dist/lib/certification-bundle-v3.js +12 -2
- package/dist/lib/v2-profiles.js +66 -0
- package/dist/spec-artifacts.lock.json +2 -2
- package/package.json +2 -2
- package/requirements.json +50 -6
- package/scenario-majors.json +5 -2
- package/schemas/CORPUS-STAMP.json +8 -8
- package/src/cli.ts +4 -2
- package/src/lib/a2a-fake-peer.ts +16 -3
- package/src/lib/certification-bundle-v3.ts +11 -2
- package/src/lib/mcp-fake-server.ts +13 -0
- package/src/lib/v2-profiles.ts +66 -0
- package/src/lib/webhook-receiver.ts +24 -5
- package/src/scenarios/a2a-1-0-agent-card.test.ts +1 -1
- package/src/scenarios/a2a-peer-authority.test.ts +1 -1
- package/src/scenarios/a2a-task-roundtrip.test.ts +1 -1
- package/src/scenarios/a2a-version-negotiation.test.ts +3 -3
- package/src/scenarios/mcp-extension-opacity.test.ts +1 -1
- package/src/scenarios/mcp-mrtr-roundtrip.test.ts +1 -1
- package/src/scenarios/mcp-tool-roundtrip.test.ts +1 -1
- package/src/scenarios/mcp-version-negotiation.test.ts +3 -3
- package/src/scenarios/v2-minimum-version-refused.test.ts +2 -2
- package/src/scenarios/v2-mrtr-rounds-ceiling.test.ts +1 -1
- package/src/scenarios/v2-negotiation-authenticated.test.ts +2 -2
- package/src/scenarios/v2-negotiation-decided-emitted.test.ts +3 -3
- package/src/scenarios/v2-relaxation-recorded.test.ts +37 -2
- package/src/scenarios/v2-webhook-egress-refusal.test.ts +115 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# `@openwop/openwop-conformance` Changelog
|
|
2
2
|
|
|
3
|
+
## [2.33.0] — 2026-09-21 — a declared relaxation denied nothing, an undeclared one was invisible, and two fakes made both unavoidable
|
|
4
|
+
|
|
5
|
+
Step 2 of closing `docs/KNOWN-LIMITS.md` §"The reference host's certified bundles were cut under an undeclared relaxation". Three suite defects, all the corpus's.
|
|
6
|
+
|
|
7
|
+
- **A host advertising `a2a` or `mcp` could not be cut relaxation-free.** The A2A fake peer and MCP fake server advertised `http://127.0.0.1:<port>` and nothing else; only the webhook receiver had a public front. So such a host could be measured ONLY with its egress guard relaxed, and a relaxed guard cannot certify — the rule made a host implementing *more* of the protocol *less* able to certify. The two production hosts cut cleanly only because they advertise neither family. New `OPENWOP_A2A_FAKE_PEER_URL` / `OPENWOP_MCP_FAKE_SERVER_URL`, validated by the same rule as `OPENWOP_WEBHOOK_RECEIVER_URL` (now `resolvePublicFront(envName, …)`; `resolveRegistrationUrl` is a wrapper). `hostFacingEndpoint()` is what is handed to the host — every invoke seam's `peerUrl` / `serverUrl`, and the AgentCard's `url` fields — while the suite's own requests to its own fake keep using the loopback `endpoint()`. Twelve scenario files updated.
|
|
8
|
+
- **A DECLARED relaxation denied nothing on any real host.** `verifyBundleV3` matched a relaxed obligation to a profile by testing whether the profile **id** contained the family name; the emitter tested whether it contained the **whole obligation string** — two different rules under a comment saying they matched. No real v2 profile id contains a family name, so `webhooks.egress-guard` denied neither `openwop-core-standard` nor anything else. `v2-relaxation-recorded` stayed green because its fixture profile was called `openwop-webhooks`, the one id the rule could match. New `profilesRelaxedBy()` in `lib/v2-profiles.ts` reads ownership from where it was always stated — a profile's `predicate.families` in `spec/v2/profiles.json` — and is the ONE function both the emitter and the verifier call. The id-named arm is kept, so nothing denied today is un-denied. New leg `openwop.requirement.0173.relaxation-recorded.owned-by-registry` asserts against the real `openwop-core-standard`. **Sabotage-proven: restoring the old rule leaves the old leg green and reddens only the new one** — which is the blindness, reproduced.
|
|
9
|
+
- **NEW `v2-webhook-egress-refusal.test.ts` — the egress guard's refusal, at major 2, for the first time.** `webhooks.md` §SSRF is a plain MUST with a registered code, and no major-2 row asserted it: `http-client-ssrf` and `webhook-negative` never left the major-1 lane, so a v2 bundle could certify `openwop-core-standard` on a host whose guard refused nothing. Eight registration probes, one per class the sentence names (non-`https`, IPv4 and IPv6 loopback, `localhost`, two RFC 1918 ranges, link-local / cloud metadata, IPv6 ULA); each MUST answer `400 webhook_url_rejected`. Registration-time only — no probe is a destination the suite owns, nothing is delivered, an accepted registration is deleted at once. New id `openwop.requirement.0171.webhook-egress-refused`.
|
|
10
|
+
- **An UNDECLARED relaxation is now detectable, and denied.** This README said the suite "cannot detect an undeclared one". For the egress guard that was never true: a host that accepts `https://127.0.0.1/…` has shown on the wire that its guard is open. Three outcomes — all refused → `executed-pass`; accepted with a `webhooks.*` relaxation DECLARED → `inapplicable`, the declaration quoted, the owning profile denied by the declaration; accepted with NOTHING declared → `executed-fail`, which `profilesDeniedByObservedRelaxation()` turns into a denial of every profile built on `webhooks` in both the emitter (`certified: false`) and `--verify` (new rejection `undeclared-relaxation-observed`, scoped to the profile). Declaring nothing is no longer a way around the rule.
|
|
11
|
+
- **Measured on the v2 reference host in all three postures:** guarded → pass (8/8 refused, right code); `OPENWOP_WEBHOOK_ALLOW_PRIVATE=1` with nothing declared → **fail, "ACCEPTED 8 of 8"** — that flag opens the scheme check and the metadata address too, not only loopback; same flag with the relaxation declared → `inapplicable`. The first attempt at that measurement reported pass/pass/pass, because the harness had not killed the first host and all three runs hit the guarded one; it now refuses to measure while the port is held.
|
|
12
|
+
- `coverage.md`: new row; and the `v2-durability-recovery` row is corrected — it still said `duplicate-delivery` was "counted per effect identity" and that the log was read from `/events`, both wrong since 2.32.0.
|
|
13
|
+
- **Expect this release to un-certify a loopback cut.** That is its purpose. A host cut through public fronts (both production hosts) is unaffected.
|
|
14
|
+
- Suite pinned to corpus `2.33.0`.
|
|
15
|
+
|
|
3
16
|
## [2.32.0] — 2026-09-20 — every published v2 pack failed the corpus schema, and two kill rows that mandated a fast bound
|
|
4
17
|
|
|
5
18
|
- **`v2-one-signing-scheme` now covers every bare `*-pack-manifest`, enumerated rather than listed** (openwop#1367). It pinned one file — `registry-version-manifest` — while six bare manifests kept the v1 `{ publicKeyRef, signatureRef, method }` block, closed, and `connection-pack-manifest` had no `signing` seat at all. Measured against the published v2 registry: **0 of 190** `pack.json` documents validated. New id `openwop.requirement.0177.one-signing-scheme.bare-manifests`. Sabotage-proven: fails on the 2.31.1 `node-pack-manifest`.
|
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
# --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
|
|
12
12
|
# default resolver refuses. npm 10.9 fails outright with
|
|
13
13
|
# "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
|
|
14
|
-
npm install --legacy-peer-deps @openwop/openwop-conformance@2.
|
|
14
|
+
npm install --legacy-peer-deps @openwop/openwop-conformance@2.33.0 @openwop/spec-artifacts@2.33.0
|
|
15
15
|
# or run without install:
|
|
16
16
|
npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
|
|
17
17
|
```
|
|
@@ -99,7 +99,7 @@ Run `npm run test` for normal CI cadence; `npm run test:strict` when claiming fu
|
|
|
99
99
|
|
|
100
100
|
| Variable | Effect |
|
|
101
101
|
| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
102
|
-
| `OPENWOP_HOST_RELAXATIONS` | Sets `host.relaxations[]` in the bundle. **It relaxes nothing in the suite** — it is your declaration that the HOST runs relaxed, and the suite cannot detect an undeclared one. A profile carrying a relaxation **cannot certify** (`security-defaults.md` §Relaxations, RFC 0173 §A.2), and since 2.6.0 the emitter enforces that rather than writing `certified: true` and leaving the contradiction to `--verify`. JSON array of `{obligation, durability, reason}`; `durability` is `session \| deployment \| persisted` — **not** `permanent`. Unparseable JSON exits 2; a wrong enum value exits 2 later, at schema validation. |
|
|
102
|
+
| `OPENWOP_HOST_RELAXATIONS` | Sets `host.relaxations[]` in the bundle. **It relaxes nothing in the suite** — it is your declaration that the HOST runs relaxed, and the suite cannot detect an undeclared one **in general**. It can detect ONE: since 2.33.0 `v2-webhook-egress-refusal` registers eight destinations `webhooks.md` §SSRF says a host MUST refuse, and a host that accepts one with no `webhooks.*` relaxation declared here records `executed-fail` — which the emitter and `--verify` read as an observed undeclared relaxation, denying every profile built on `webhooks`. An obligation is `<family>.<name>`; it denies each profile whose `spec/v2/profiles.json` predicate lists that family (so `webhooks.egress-guard` denies `openwop-core-standard`). Until 2.33.0 the match was on the profile *id*, which contains no family name, so a declared relaxation denied nothing on any real host. A profile carrying a relaxation **cannot certify** (`security-defaults.md` §Relaxations, RFC 0173 §A.2), and since 2.6.0 the emitter enforces that rather than writing `certified: true` and leaving the contradiction to `--verify`. JSON array of `{obligation, durability, reason}`; `durability` is `session \| deployment \| persisted` — **not** `permanent`. Unparseable JSON exits 2; a wrong enum value exits 2 later, at schema validation. |
|
|
103
103
|
| `OPENWOP_TARGET_MAJOR` | Selects the scenario lane. **Only the literal `"2"` selects major 2** — unlike `--target-major`, which validates and exits 2 on a bad value, the env var falls back to major 1 for `"1"`, `"3"`, `"two"` or empty. The runner and every vitest worker MUST see the same value: when they disagreed, the worker minted `openwop.floor.*` while the runner looked up `openwop.scenario.*`, producing a vacuous-pass emitter defect and **no bundle at all**. Prefer `--target-major`; set this only when driving `vitest` directly. |
|
|
104
104
|
| `OPENWOP_REQUIRE_BEHAVIOR=true` | Capability-gated scenarios (audit-log integrity, rate-limit envelope, multi-region idempotency, `configurableSchema`, webhook sig versioning, etc.) FAIL instead of skipping when the host doesn't advertise the profile. Lets a host claim "full coverage" mechanically. See [`coverage.md`](./coverage.md) §"Capability-gated scenarios". |
|
|
105
105
|
| `OPENWOP_TEST_PUBLIC_REGISTRY=true` | Runs `registry-public.test.ts` against the hosted registry at `packs.openwop.dev`. Skipped by default so the suite doesn't depend on outbound connectivity. |
|
|
@@ -113,6 +113,7 @@ Run `npm run test` for normal CI cadence; `npm run test:strict` when claiming fu
|
|
|
113
113
|
| `OPENWOP_MCP_FAKE_SERVER=true` | Boots the synthetic MCP peer for `mcp-tool-roundtrip.test.ts`. |
|
|
114
114
|
| `OPENWOP_MCP_REAL_SERVER_URL=<base-url>` | Points the MCP wire-shape probe at a real MCP server. The probe POSTs JSON-RPC and reads a single-JSON response — matches MCP's `streamable-http` transport in single-response mode. **Does NOT support** stdio transport (which is what most `modelcontextprotocol/servers` references default to) or SSE-streamed responses; an operator collecting interop evidence today runs a custom `StreamableHTTPServerTransport`-style server that returns a single JSON body per request. Adding SSE-frame parsing is tracked in `docs/PROTOCOL-GAP-CLOSURE-PLAN.md` Track 6. Assertions relax to shape-only. When both this and `OPENWOP_MCP_FAKE_SERVER` are set, the real URL wins. Phase 3 T3.4 interop-evidence path. |
|
|
115
115
|
| `OPENWOP_A2A_FAKE_PEER=true` | Boots the synthetic A2A peer for `a2a-task-roundtrip.test.ts`. |
|
|
116
|
+
| `OPENWOP_A2A_FAKE_PEER_URL=<https-url>` / `OPENWOP_MCP_FAKE_SERVER_URL=<https-url>` | **The public front for each fake — the route to an A2A / MCP witness that relaxes nothing** (2.33.0). Same rule and the same loud validation as `OPENWOP_WEBHOOK_RECEIVER_URL`: `https:`, publicly resolvable. The fake still LISTENS on loopback; pin it with the matching `_PORT` so the tunnel has a fixed port to forward to. When set, the URL handed to the host under test — the `peerUrl` / `serverUrl` of every invoke seam, and the `url` fields of the A2A AgentCard — is the front; the suite's own requests to its own fake stay local. Until 2.33.0 neither fake had a front, so a host advertising `a2a` or `mcp` could be measured ONLY with its egress guard relaxed, and a relaxed guard cannot certify: the rule made a host implementing more of the protocol less able to certify. |
|
|
116
117
|
| `OPENWOP_A2A_REAL_PEER_URL=<base-url>` | Points the A2A AgentCard + task-lifecycle probe at a real reference A2A peer. Drift-point subtests (AUTH_REQUIRED / REJECTED) stay fake-peer-only — real peers don't expose a state-forcing API. Phase 3 T3.4 interop-evidence path. |
|
|
117
118
|
| `OPENWOP_FORCE_RATE_LIMIT=true` | Signals the host (test-only key) to fabricate a 429 so `rate-limit-envelope.test.ts` can exercise envelope shape deterministically. |
|
|
118
119
|
| `OPENWOP_TEST_PROMPT_PACK_INSTALLED=true` | Promotes the `prompt-pack-install.test.ts` existence claim from soft-skip to hard assertion (RFC 0028 §B). Set when the target host is known to have at least one prompt pack installed at boot — RFC 0028 §B does NOT require any host with `endpointsSupported: true` to have packs installed, so the suite stays conformant against a fresh production host with no pack subscriptions when the flag is unset. The in-tree workflow-engine sample auto-installs `vendor.openwop.prompt-sample` via `promptPackLoader.ts`, so operators running against it should set the flag. |
|
|
@@ -132,7 +133,7 @@ Exit code is non-zero on any failed assertion. `--certify` distinguishes: `0`
|
|
|
132
133
|
|
|
133
134
|
## What's Covered
|
|
134
135
|
|
|
135
|
-
The current suite has
|
|
136
|
+
The current suite has 524 scenario files under `src/scenarios/`.
|
|
136
137
|
- 2026-09-03 (suite `1.157.0 -> 1.158.0`, gap G17): NEW `idempotency-concurrent-claim.test.ts` — drives the new `host-sample-test-seams.md` §25 concurrent duplicate-delivery seam for the RFC 0150 §B / `idempotency.md` §"Concurrent duplicates (Layer 2)" atomic-claim MUST, which is unconditional and had no witness of any kind. Asserts every executor mints the SAME `logicalInvocationId` **before** asserting `delivered === 1` — without the identity check a host passes by minting different ids and never colliding, one effect because nothing raced. Not profile-gated and so not opt-out-able (the obligation is unconditional); an unmounted seam records `blocked`, which is not certifiable. Graduates `layer2-invocation-claim-atomic` reference-impl -> protocol.
|
|
137
138
|
- 2026-08-19 (suite `1.137.0 → 1.138.0`): NEW `durability-poison-exhaustion.test.ts` — RFC 0158 §C.8, the FIRST row of that RFC's conformance table to land. Asserts what `failure-path.test.ts` cannot: not just that deterministically failing work reaches terminal, but that attempts STOP — counted on the log, re-counted after a scaled quiet window, asserted unchanged. A host still redelivering records more. Seam-gated on the existing event-log seam (`blocked` = unobservable, not unmet) and outside every profile floor.
|
|
138
139
|
- 2026-08-19 (suite `1.136.15 → 1.137.0`): NEW `replay-fanout-suppression.test.ts` — capability-gated on `webhooks.supported`, **outside every profile floor**; witnesses `replay.md` §"Host-initiated fan-out is an external effect", which was the largest normative MUST NOT on the replay surface with no scenario and no SECURITY invariant. Three legs in ONE `it` against ONE receiver and ONE subscription — a positive control, the MUST NOT, and a `branch` boundary leg — because "no delivery arrived" passes identically when delivery never worked, so absence is asserted only after presence is proven on that exact wiring. A host with an SSRF guard correctly refuses the loopback receiver and records `blocked`: **unobservable, not unmet.**
|
|
@@ -477,7 +478,7 @@ Server-required (added in 1.7.0):
|
|
|
477
478
|
| ------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
478
479
|
| **Redaction** | [`capabilities.md`](../spec/v1/capabilities.md) §"Secrets" + NFR-7 + §"aiProviders" | Vendor-neutral assertions that the server doesn't leak secret material. Three scenario groups: (a) discovery shape contract — `secrets` + `aiProviders` advertisements are well-formed regardless of `secrets.supported`; when `supported === true`, scopes MUST be non-empty + `resolution === 'host-managed'`; `byok ⊆ supported`. (b) bearer-token redaction — invalid Bearer canary in `Authorization` header is not echoed in the 401 response body. (c) credentialRef echo control — gated on `secrets.supported === true`; canary planted in `configurable.ai.credentialRef` MUST NOT appear in any RunEvent payload (poll-based capture; transport-agnostic). Uses runtime-built canary fixtures (`lib/canaries.ts`) that defeat static secret scanners. 6 scenarios. |
|
|
479
480
|
|
|
480
|
-
Current source tree:
|
|
481
|
+
Current source tree: 524 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
|
|
481
482
|
|
|
482
483
|
## Remaining Gaps
|
|
483
484
|
|
package/coverage.md
CHANGED
|
@@ -144,7 +144,8 @@ The scenario groups in the table below (one row per group; count the rows — th
|
|
|
144
144
|
| `tool-session-lifecycle.test.ts` | `capabilities.toolCatalog.sessionLifecycle` (RFC 0078 §D, `tool-catalog.md`) | A (the §D bracket via `POST /v1/host/sample/tools/session-run` + the test event-log seam: `tool.session.opened` before the first RFC 0064 call event → `tool.session.closed` after the last, one shared `sessionId`, each carrying a `toolId`, `closed.outcome` ∈ {completed,failed,aborted,expired}, both content-free) | `host-pending` | `behaviorGate('openwop-tool-session-lifecycle', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0078 → Accepted bar.** First adopter: MyndHyve `toolCatalog`. |
|
|
145
145
|
| `egress-audience-binding.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §C, `host-capabilities.md`) + `SECURITY/invariants.yaml` `egress-credential-audience-bound` | A (KEYSTONE — the §C confused-deputy MUST via `POST /v1/host/sample/egress/decide`: an out-of-audience egress is `denied`/`downgraded` with `reason:"out-of-audience"` and the credential is NOT attached (`credentialAttached !== true`); a provenance-unevaluable egress fails closed `denied`+`reason:"provenance-unevaluable"`; decision/reason ∈ the closed enums) | `host-pending` | `behaviorGate('openwop-egress-audience-binding', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0079 → Accepted bar** (the `egress-credential-audience-bound` invariant graduates reference-impl → protocol tier when this passes against a host). First adopter: MyndHyve `httpClient.egressPolicy`. |
|
|
146
146
|
| `egress-decision-content-free.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §F / SR-1) | A (the secret non-leak — a `canary` credential's sentinel never surfaces in the decision (`canaryLeaked !== true`), the `egress.decided` payload carries no forbidden content key, and `reason` stays in the CLOSED vocabulary so no blocked destination spills into a free-form field) | `host-pending` | `behaviorGate('openwop-egress-decision-content-free', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0079 → Accepted bar.** First adopter: MyndHyve `httpClient.egressPolicy`. |
|
|
147
|
-
| `v2-
|
|
147
|
+
| `v2-webhook-egress-refusal.test.ts` | the `webhooks` family (capability-gated) | A (`webhooks.md` §SSRF at MAJOR 2 — eight registration probes, one per class the sentence names: non-`https`, IPv4/IPv6 loopback, `localhost`, two RFC 1918 ranges, link-local / cloud metadata, IPv6 ULA; each MUST answer `400 webhook_url_rejected`. Until 2.33.0 no major-2 row asserted the guard refuses anything — the refusal scenarios were major-1 only. THREE OUTCOMES: all refused → `executed-pass`; a probe accepted with a `webhooks.*` relaxation DECLARED in `OPENWOP_HOST_RELAXATIONS` → `inapplicable`, and the declared relaxation denies the owning profile; a probe accepted with NOTHING declared → `executed-fail`, which the emitter and verifier read as an OBSERVED undeclared relaxation and deny every profile built on `webhooks` (`undeclared-relaxation-observed`). Registration-time only: no probe is a destination the suite owns, nothing is delivered, an accepted registration is deleted at once) |
|
|
148
|
+
| `v2-durability-recovery.test.ts` | the RFC 0158 host-extension durability seam `POST /host/durability/kill` (non-normative; §E mints no capability field, so seam PRESENCE is how a host says it claims a rung) | A (all five `durable-single-instance` rows: `kill-after-accept` as a HOLD-DISPATCH exercise, `kill-during-execution` asserting that work executing at a real process death is never observable as completed without re-execution, `duplicate-delivery` counted AT THE EFFECT'S DESTINATION — a receiver the suite owns, `effectUrl` — because a ledger keyed on effect identity admits one row per identity by construction and cannot show a double-fire (2.32.0), `bound-is-derived` as an explicitly-labelled paper check, and `poison-exhaustion` ported from the major-1 scenario — at major 2 it reads the canonical JSON log read `GET /runs/{runId}/events/poll` (NOT `/events`, which is SSE-only at major 2 — reading it as JSON made this row pass vacuously on a host serving exactly the OpenAPI; 2.32.0), so unlike its v1 twin it needs NO seam and can never record `blocked` for want of one a host did not wire) | `host-pending` | **Disposition rule, and it protects fleet certification:** no seam route at all ⇒ `inapplicable` (the host claims no rung); seam present but an operator precondition unmet — a restart supervisor, which a black-box suite cannot supply — ⇒ `blocked` with the precondition NAMED. Collapsing the two would deny certification to every host the day these rows entered the lane. `peer-resume` is deliberately NOT written: §E makes it bundle-witnessed, so a scenario could only ever record `blocked`. **This is the RFC 0158 → Accepted bar, and it is now complete at major 2** — before the port, a major-2 bundle could carry at most four of the five rows and the rung was unwitnessable no matter what a host did, and its witness is openwop-app, which the RFC records as having observed `kill-during-execution` across a real `SIGKILL`. |
|
|
148
149
|
| `memory-degraded-projection.test.ts` | `capabilities.agents.manifestRuntime.supported` + `capabilities.memory.supported` (RFC 0080 §C, `agent-memory.md`) | A (the §C iff-contract on the NORMATIVE `GET /v1/agents`: a degraded entry MUST carry `memoryDegraded:true` + a non-empty, unique `degradedMemoryDimensions[]` drawn from the closed §A-name enum [read/write/search/long-term-durability/compaction/attribution/replay-snapshot/retention]; a non-degraded entry MUST NOT carry a non-empty list; the inventory is non-empty; the degraded branch runs non-vacuously when `OPENWOP_DEGRADED_AGENT_ID` names a known-degraded agent) | `host-pending` | `behaviorGate('openwop-memory-degraded', …)`. Black-box on the normative path (no POST seam); soft-skips on 404 / when the host computes no degradation. **This is the RFC 0080 → Accepted bar.** First adopter: MyndHyve `memory`. |
|
|
149
150
|
| `budget-enforcement.test.ts` | `capabilities.budget.supported` (RFC 0084 §C/§D, `budget-policy.md`) + `SECURITY/invariants.yaml` `budget-no-pricing-leak` | A (the §C/§D enforcement via `POST /v1/host/sample/budget/run` + the test event-log seam: a `hard-cost-exhaust` run emits the strict-ordered `budget.reserved → budget.consumed → budget.threshold.crossed{percent} → budget.exhausted → cap.breached{kind:"budget-cost"} → run.failed{error:"budget_exhausted"}` chain; a `model-denied` run is refused `budget_model_denied` BEFORE the provider call (fail-closed); an `advisory` host emits the `budget.*` events without stopping; every `budget.*` payload content-free — no pricing/rate) | `host-pending` | `behaviorGate('openwop-budget-enforcement', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0084 → Accepted bar.** First adopter: MyndHyve `budget`. |
|
|
150
151
|
| `agent-platform-aggregate-evidence.test.ts` | `openwop-agent-platform` claim — live discovery `profiles[]` includes it (RFC 0085 §C, `agent-platform-profile.md`) | A (the §C/§D honest-advertisement on live `/.well-known/openwop`: a host claiming `openwop-agent-platform` MUST satisfy the §B floor predicate (`isAgentPlatformPartial` → `partial`/`full`, never `none`), the claim backed by per-capability evidence not the profile string; `OPENWOP_AGENT_PLATFORM_TIER=full` forces the full-predicate bar — all governance terms + tenant installScope + all 16 §D terms) | `host-pending` | `behaviorGate('openwop-agent-platform', …)`. Black-box on the discovery doc (no POST seam); soft-skips until a host claims the profile. **This is the RFC 0085 → Accepted bar.** First adopter: MyndHyve (after the memory batch surfaces the floor's `memory.supported`). |
|
package/dist/cli.js
CHANGED
|
@@ -46,7 +46,7 @@ import { scrubEvidence, evidenceSecretsFromEnv, verifyBundleV2 } from './lib/cer
|
|
|
46
46
|
import { publicKeyFromPrivate, signBundleV3, verifierSign, verifyBundleV3, witnessDigest } from './lib/certification-bundle-v3.js';
|
|
47
47
|
import { deriveProfiles, isCoreStandard, agentPlatformStatus, DEPRECATED_PROFILE_ALIASES, PROFILE_FLOOR_SCENARIOS, } from './lib/profiles.js';
|
|
48
48
|
import { setV2ProfileFloors, v2ProfileFloorFiles } from './lib/requirement-registry.js';
|
|
49
|
-
import { v2ProfileIds } from './lib/v2-profiles.js';
|
|
49
|
+
import { profilesDeniedByObservedRelaxation, profilesRelaxedBy, v2ProfileIds } from './lib/v2-profiles.js';
|
|
50
50
|
import { childEnv } from './lib/child-env.js';
|
|
51
51
|
function parseArgs(argv) {
|
|
52
52
|
let baseUrl;
|
|
@@ -665,7 +665,9 @@ async function runCertify(args, baseUrl, apiKey) {
|
|
|
665
665
|
// belongs to. Matched the same way the verifier matches it, so the emitter
|
|
666
666
|
// and `verifyBundleV3` cannot disagree about the same bundle.
|
|
667
667
|
const relaxedObligations = new Set((relaxations ?? []).map((r) => r.obligation));
|
|
668
|
-
const
|
|
668
|
+
const relaxedProfileIds = profilesRelaxedBy([...relaxedObligations], claimedProfiles);
|
|
669
|
+
const observedRelaxed = profilesDeniedByObservedRelaxation(rows3, claimedProfiles).profiles;
|
|
670
|
+
const relaxedProfile = (p) => relaxedProfileIds.has(p) || observedRelaxed.has(p);
|
|
669
671
|
const claimed3 = claimedProfiles.filter((p) => !(p in DEPRECATED_PROFILE_ALIASES)).map((p) => ({ id: p, evidenceTier: args.evidenceTier, witnessCount: witnessCountFor(p), certified: !relaxedProfile(p) && (verdictFor(p)?.certifiable ?? false) && !notHeld.has(p) && !rejectedProfiles.some((v) => v.profile === p) && totals3.blocked === 0 }));
|
|
670
672
|
const lockPath = resolvePath(conformanceRoot, 'dist', 'spec-artifacts.lock.json');
|
|
671
673
|
const lock = existsSync(lockPath) ? JSON.parse(readFileSync(lockPath, 'utf8')) : undefined;
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
*/
|
|
20
20
|
import { createHash, createPrivateKey, createPublicKey, sign as edSign, verify as edVerify } from 'node:crypto';
|
|
21
21
|
import { profileDerivable } from './profiles.js';
|
|
22
|
-
import { v2RegistryAvailable } from './v2-profiles.js';
|
|
22
|
+
import { profilesDeniedByObservedRelaxation, profilesRelaxedBy, v2RegistryAvailable } from './v2-profiles.js';
|
|
23
23
|
export const SIGNATURE_OVER = ['witnessSha256', 'host.build', 'suite.version', 'discovery.sha256'];
|
|
24
24
|
/** Deterministic JSON: keys sorted at every level, no whitespace. */
|
|
25
25
|
export function canonicalJSON(value) {
|
|
@@ -118,9 +118,19 @@ export function verifyBundleV3(bundle, opts = {}) {
|
|
|
118
118
|
}
|
|
119
119
|
// Relaxations: a relaxed obligation's profile cannot certify (RFC 0173 §A.2).
|
|
120
120
|
const relaxed = new Set((bundle.host?.relaxations ?? []).map((r) => r.obligation.split('.')[0]));
|
|
121
|
+
// Ownership comes from the profile registry, not from the profile's name — see profilesRelaxedBy.
|
|
122
|
+
const relaxedProfiles = profilesRelaxedBy((bundle.host?.relaxations ?? []).map((r) => r.obligation), (bundle.claimedProfiles ?? []).map((p) => p.id));
|
|
123
|
+
// An OBSERVED relaxation nobody declared: the bundle's own results show the
|
|
124
|
+
// host accepting a destination its egress guard MUST refuse. Same denial as a
|
|
125
|
+
// declared one, so declaring nothing is not a way around the rule.
|
|
126
|
+
const observed = profilesDeniedByObservedRelaxation(bundle.results?.requirements ?? [], (bundle.claimedProfiles ?? []).map((p) => p.id));
|
|
127
|
+
for (const p of bundle.claimedProfiles ?? []) {
|
|
128
|
+
if (p.certified && observed.profiles.has(p.id) && !relaxedProfiles.has(p.id))
|
|
129
|
+
rejections.push({ kind: 'undeclared-relaxation-observed', profile: p.id, detail: `${p.id} is marked certified, but this bundle's own results show the host running relaxed on ${observed.obligations.join(', ')} with no host.relaxations[] entry recording it (security-defaults.md §Relaxations: every relaxation a host runs under MUST be recorded, and its profile MUST NOT certify)` });
|
|
130
|
+
}
|
|
121
131
|
const certifiedProfiles = [];
|
|
122
132
|
for (const p of bundle.claimedProfiles ?? []) {
|
|
123
|
-
if (p.certified &&
|
|
133
|
+
if (p.certified && relaxedProfiles.has(p.id))
|
|
124
134
|
rejections.push({ kind: 'relaxed-profile-certified', profile: p.id, detail: `${p.id} is marked certified while a relaxation on ${[...relaxed].join(', ')} is recorded (RFC 0173 §A.2)` });
|
|
125
135
|
else if (p.certified)
|
|
126
136
|
certifiedProfiles.push(p.id);
|
package/dist/lib/v2-profiles.js
CHANGED
|
@@ -104,3 +104,69 @@ export function v2ProfileIds(doc) {
|
|
|
104
104
|
export function v2ProfileDerivable(doc, profile) {
|
|
105
105
|
return v2ProfileIds(doc)?.includes(profile) ?? false;
|
|
106
106
|
}
|
|
107
|
+
/**
|
|
108
|
+
* Which of `profileIds` a relaxed obligation denies (RFC 0173 §A.2;
|
|
109
|
+
* `security-defaults.md` §Relaxations: "a bundle that records a relaxation MUST
|
|
110
|
+
* NOT certify the profile the relaxed obligation belongs to").
|
|
111
|
+
*
|
|
112
|
+
* An obligation is `<family>.<name>`. A profile OWNS a family when its registry
|
|
113
|
+
* predicate lists it — `openwop-core-standard` is built on `interrupt`, `replay`,
|
|
114
|
+
* `webhooks`, `idempotency`, `eventLog`, so a relaxation on any of those denies
|
|
115
|
+
* it. That ownership was always stated in `spec/v2/profiles.json`; nothing read
|
|
116
|
+
* it. Until 2.33.0 the verifier tested whether the profile ID contained the
|
|
117
|
+
* family name, and the emitter tested whether it contained the WHOLE obligation
|
|
118
|
+
* string — two different rules under a comment saying they matched — and since
|
|
119
|
+
* no real v2 profile id contains a family name, a declared relaxation denied
|
|
120
|
+
* nothing on any real host. The one scenario pinning the rule used a fixture
|
|
121
|
+
* profile called `openwop-webhooks`, which is why it stayed green.
|
|
122
|
+
*
|
|
123
|
+
* The id-substring rule is KEPT as a second arm, not replaced: it is how a
|
|
124
|
+
* non-registry profile id (a v1 alias, a fixture) names its family, and
|
|
125
|
+
* dropping it would un-deny something that is denied today. When the registry
|
|
126
|
+
* is unavailable only that arm can run; the caller already records that layout
|
|
127
|
+
* as `derivabilityChecked: false`.
|
|
128
|
+
*
|
|
129
|
+
* ONE function, called by the emitter and the verifier, so the file and the
|
|
130
|
+
* verdict on the file cannot disagree again.
|
|
131
|
+
*/
|
|
132
|
+
export function profilesRelaxedBy(obligations, profileIds) {
|
|
133
|
+
const families = new Set(obligations.map((o) => o.split('.')[0] ?? '').filter((f) => f.length > 0));
|
|
134
|
+
const out = new Set();
|
|
135
|
+
if (families.size === 0)
|
|
136
|
+
return out;
|
|
137
|
+
const registry = readRegistry();
|
|
138
|
+
const owned = new Map();
|
|
139
|
+
for (const p of registry ?? []) {
|
|
140
|
+
if (typeof p.id === 'string')
|
|
141
|
+
owned.set(p.id, Array.isArray(p.predicate?.families) ? p.predicate.families.map(String) : []);
|
|
142
|
+
}
|
|
143
|
+
for (const id of profileIds) {
|
|
144
|
+
const ownsOne = (owned.get(id) ?? []).some((f) => families.has(f));
|
|
145
|
+
const namesOne = [...families].some((f) => id.includes(f));
|
|
146
|
+
if (ownsOne || namesOne)
|
|
147
|
+
out.add(id);
|
|
148
|
+
}
|
|
149
|
+
return out;
|
|
150
|
+
}
|
|
151
|
+
/**
|
|
152
|
+
* The requirement whose FAILURE is an observed, undeclared relaxation.
|
|
153
|
+
*
|
|
154
|
+
* `conformance/README.md` said of `OPENWOP_HOST_RELAXATIONS` that "the suite
|
|
155
|
+
* cannot detect an undeclared one". For the egress guard that was never true:
|
|
156
|
+
* a host that ACCEPTS `https://127.0.0.1/…` as a webhook destination has shown
|
|
157
|
+
* on the wire that its guard is open (`webhooks.md`: "a host MUST reject (400
|
|
158
|
+
* webhook_url_rejected) … RFC 1918 and loopback and link-local ranges").
|
|
159
|
+
* `v2-webhook-egress-refusal` records that as `executed-fail` on this id unless
|
|
160
|
+
* the operator DECLARED the relaxation — in which case {@link profilesRelaxedBy}
|
|
161
|
+
* already denies the profile. Either way a profile built on `webhooks` does not
|
|
162
|
+
* certify on a host whose guard is open; this closes the path where it did,
|
|
163
|
+
* silently, because nobody declared anything.
|
|
164
|
+
*/
|
|
165
|
+
export const UNDECLARED_RELAXATION_WITNESSES = [
|
|
166
|
+
{ requirementId: 'openwop.requirement.0171.webhook-egress-refused', obligation: 'webhooks.egress-guard' },
|
|
167
|
+
];
|
|
168
|
+
/** Profiles that MUST NOT certify because a row above is `executed-fail` in this bundle's own results. */
|
|
169
|
+
export function profilesDeniedByObservedRelaxation(rows, profileIds) {
|
|
170
|
+
const obligations = UNDECLARED_RELAXATION_WITNESSES.filter((w) => rows.some((r) => r.id === w.requirementId && r.result === 'executed-fail')).map((w) => w.obligation);
|
|
171
|
+
return { profiles: profilesRelaxedBy(obligations, profileIds), obligations };
|
|
172
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openwop/openwop-conformance",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.33.0",
|
|
4
4
|
"description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -56,6 +56,6 @@
|
|
|
56
56
|
"@openwop/spec-artifacts": "file:../spec-artifacts"
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
|
-
"@openwop/spec-artifacts": "2.
|
|
59
|
+
"@openwop/spec-artifacts": "2.33.0"
|
|
60
60
|
}
|
|
61
61
|
}
|
package/requirements.json
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
"$comment": "GENERATED by conformance/scripts/generate-requirement-registry.mjs — do not edit. One record per it()/test() in src/scenarios. Ids: openwop.it.<file-stem>.<title-slug>[~n] (src/lib/requirement-ids.ts). A record with id null has an interpolated title; its run-time row is keyed by the rendered title and maps here by file+line only. Renamed ids need a row in requirement-aliases.json.",
|
|
3
3
|
"generatedFrom": "src/scenarios/*.test.ts",
|
|
4
4
|
"counts": {
|
|
5
|
-
"files":
|
|
6
|
-
"tests":
|
|
7
|
-
"withStableId":
|
|
5
|
+
"files": 582,
|
|
6
|
+
"tests": 2209,
|
|
7
|
+
"withStableId": 2209,
|
|
8
8
|
"interpolatedTitles": 0,
|
|
9
|
-
"explicitIds":
|
|
9
|
+
"explicitIds": 2163
|
|
10
10
|
},
|
|
11
11
|
"records": [
|
|
12
12
|
{
|
|
@@ -28387,7 +28387,7 @@
|
|
|
28387
28387
|
{
|
|
28388
28388
|
"id": "openwop.it.v2-relaxation-recorded.a-bundle-recording-a-relaxation-cannot-certify-the-relaxed-profile",
|
|
28389
28389
|
"file": "v2-relaxation-recorded.test.ts",
|
|
28390
|
-
"line":
|
|
28390
|
+
"line": 70,
|
|
28391
28391
|
"title": "a bundle recording a relaxation cannot certify the relaxed profile",
|
|
28392
28392
|
"explicitId": "openwop.requirement.0173.relaxation-recorded",
|
|
28393
28393
|
"citations": [
|
|
@@ -28421,10 +28421,35 @@
|
|
|
28421
28421
|
}
|
|
28422
28422
|
]
|
|
28423
28423
|
},
|
|
28424
|
+
{
|
|
28425
|
+
"id": "openwop.it.v2-relaxation-recorded.a-relaxation-denies-the-real-profile-that-owns-its-family-not-only-one-named-aft",
|
|
28426
|
+
"file": "v2-relaxation-recorded.test.ts",
|
|
28427
|
+
"line": 118,
|
|
28428
|
+
"title": "a relaxation denies the REAL profile that owns its family, not only one named after it",
|
|
28429
|
+
"explicitId": "openwop.requirement.0173.relaxation-recorded.owned-by-registry",
|
|
28430
|
+
"citations": [
|
|
28431
|
+
{
|
|
28432
|
+
"section": "security-defaults.md §Relaxations",
|
|
28433
|
+
"requirement": "control: with no relaxation recorded, the real profile certifies — so the rejection below is the relaxation's"
|
|
28434
|
+
},
|
|
28435
|
+
{
|
|
28436
|
+
"section": "security-defaults.md §Relaxations",
|
|
28437
|
+
"requirement": "a relaxation on `webhooks.*` MUST deny `openwop-core-standard`, whose registry predicate is built on the `webhooks` family — ownership is read from spec/v2/profiles.json, never from the spelling of the profile id"
|
|
28438
|
+
},
|
|
28439
|
+
{
|
|
28440
|
+
"section": "security-defaults.md §Relaxations",
|
|
28441
|
+
"requirement": "the owning profile is not certified, and a profile that does not own the family still is"
|
|
28442
|
+
},
|
|
28443
|
+
{
|
|
28444
|
+
"section": "security-defaults.md §Relaxations",
|
|
28445
|
+
"requirement": "a relaxation on a family outside every claimed profile's predicate is scoped to nothing claimed — it does not poison an unrelated profile"
|
|
28446
|
+
}
|
|
28447
|
+
]
|
|
28448
|
+
},
|
|
28424
28449
|
{
|
|
28425
28450
|
"id": "openwop.it.v2-relaxation-recorded.durability-is-a-closed-set-session-deployment-persisted",
|
|
28426
28451
|
"file": "v2-relaxation-recorded.test.ts",
|
|
28427
|
-
"line":
|
|
28452
|
+
"line": 143,
|
|
28428
28453
|
"title": "durability is a closed set: session | deployment | persisted",
|
|
28429
28454
|
"explicitId": "openwop.requirement.0173.relaxation-recorded.durability-closed",
|
|
28430
28455
|
"citations": [
|
|
@@ -29563,6 +29588,25 @@
|
|
|
29563
29588
|
}
|
|
29564
29589
|
]
|
|
29565
29590
|
},
|
|
29591
|
+
{
|
|
29592
|
+
"id": "openwop.it.v2-webhook-egress-refusal.a-webhook-registration-naming-a-destination-the-guard-must-refuse-is-refused-400",
|
|
29593
|
+
"file": "v2-webhook-egress-refusal.test.ts",
|
|
29594
|
+
"line": 82,
|
|
29595
|
+
"title": "a webhook registration naming a destination the guard MUST refuse is refused 400 webhook_url_rejected",
|
|
29596
|
+
"explicitId": "openwop.requirement.0171.webhook-egress-refused",
|
|
29597
|
+
"citations": [
|
|
29598
|
+
{
|
|
29599
|
+
"section": "spec/v2/core/webhooks.md §SSRF",
|
|
29600
|
+
"requirement": null,
|
|
29601
|
+
"interpolated": true
|
|
29602
|
+
},
|
|
29603
|
+
{
|
|
29604
|
+
"section": "spec/v2/core/webhooks.md §SSRF",
|
|
29605
|
+
"requirement": null,
|
|
29606
|
+
"interpolated": true
|
|
29607
|
+
}
|
|
29608
|
+
]
|
|
29609
|
+
},
|
|
29566
29610
|
{
|
|
29567
29611
|
"id": "openwop.it.v2-well-known-one-resource.a-header-less-get-and-an-openwop-version-2-0-get-are-two-representations-of-one",
|
|
29568
29612
|
"file": "v2-well-known-one-resource.test.ts",
|
package/scenario-majors.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$comment": "GENERATED by conformance/scripts/generate-scenario-majors.mjs (RFC 0168 §D.3). Do not edit; add a file to BOTH_MAJORS in the generator to target both majors.",
|
|
3
3
|
"counts": {
|
|
4
|
-
"files":
|
|
4
|
+
"files": 524,
|
|
5
5
|
"v1": 445,
|
|
6
|
-
"v2":
|
|
6
|
+
"v2": 81
|
|
7
7
|
},
|
|
8
8
|
"majors": {
|
|
9
9
|
"a2a-1-0-agent-card.test.ts": [
|
|
@@ -1463,6 +1463,9 @@
|
|
|
1463
1463
|
"v2-webhook-durable-delivery.test.ts": [
|
|
1464
1464
|
2
|
|
1465
1465
|
],
|
|
1466
|
+
"v2-webhook-egress-refusal.test.ts": [
|
|
1467
|
+
2
|
|
1468
|
+
],
|
|
1466
1469
|
"v2-well-known-one-resource.test.ts": [
|
|
1467
1470
|
2
|
|
1468
1471
|
],
|
|
@@ -1,17 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_comment": "Provenance of @openwop/spec-artifacts (RFC 0168 §D.2). files: SHA-256 per file; the conformance suite compares the installed peer against dist/spec-artifacts.lock.json at start.",
|
|
3
3
|
"package": "@openwop/spec-artifacts",
|
|
4
|
-
"version": "2.
|
|
5
|
-
"corpusTag": "v2.
|
|
4
|
+
"version": "2.33.0",
|
|
5
|
+
"corpusTag": "v2.33.0",
|
|
6
6
|
"files": {
|
|
7
7
|
"api/.redocly.lint-ignore.yaml": "bf5a8350b88a72fa43f59605ed8d903ed24b6cfccda5e45509c9f6ed9ee4e712",
|
|
8
8
|
"api/asyncapi.yaml": "d5ecb9ee6114582be3b1f662c84bfac9ae96dae7bacb853e461168f70a8e1c7d",
|
|
9
9
|
"api/grpc/openwop.proto": "c3e72bb17cba514ee98feb6434e6c9b6ea6795bfd086489ec69fd882dd1ad977",
|
|
10
10
|
"api/openapi.yaml": "39081c59fb696159806b0f2f9a42e7e9ff830d622fcf2ad4159b21357580a955",
|
|
11
11
|
"api/redocly.yaml": "b0604c89b2ca6d5076ec25725c539dad44a741a811fe524439ee6daef8baa09f",
|
|
12
|
-
"api/seams-v2.yaml": "
|
|
13
|
-
"api/v2/asyncapi.yaml": "
|
|
14
|
-
"api/v2/openapi.yaml": "
|
|
12
|
+
"api/seams-v2.yaml": "975b6cd87fa679857df9fcdc60b6a2bed98510b831e82a5eff5a114e3b263f6e",
|
|
13
|
+
"api/v2/asyncapi.yaml": "96ba13176119bbf54a94bd41d789ebaa919f67c81d529bdc3c66ffb3f419ee36",
|
|
14
|
+
"api/v2/openapi.yaml": "428ab881d0a8d08d388dd89d4b4457f3454cd0433b665674180d3e85821b978d",
|
|
15
15
|
"api/v2/redocly.yaml": "1e66b60e6118ad11a823bb620678be464d99dfe50a40e3e6f93ec9429b88b34c",
|
|
16
16
|
"schemas/README.md": "0c0b737ffcf8f30e7d2809cec8a498232de710f41443212922ad8337cdde0b51",
|
|
17
17
|
"schemas/a2a-task-state.schema.json": "c9365918f993f943b4b619d42551eb066a1ed33a08d895d51b395432a5b1f1bc",
|
|
@@ -201,7 +201,7 @@
|
|
|
201
201
|
"schemas/workspace-file.schema.json": "464de85c2a068243084ee9c1d969bc7cd5d8f7948574e58450d6493c38a0e1e4",
|
|
202
202
|
"spec/v1/alias-detectors.json": "40069d5976eeb6ba1384a648e57e5cfd673db3fce36175115fb8293bee9664d4",
|
|
203
203
|
"spec/v1/capability-declaration-classes.json": "e7729aed5c4b4e1dd02abab0530f14cc95f5d4070fe51fb139e7f5cccefa00c6",
|
|
204
|
-
"spec/v1/core-standard-manifest.json": "
|
|
204
|
+
"spec/v1/core-standard-manifest.json": "9f5d065a0a84cf4745f53957f395b1e868453f910531e030f2da4039ee2c9140",
|
|
205
205
|
"spec/v1/deprecations.json": "307083ce29c23fd406015951f99a30d78d6187ff061d38dc9732f62191b40f3f",
|
|
206
206
|
"spec/v1/deprecations.schema.json": "18c87e78bedc210431f795ae44c5b5d202f2f3317850d5cf86867d4f1fa1cdfb",
|
|
207
207
|
"spec/v1/event-codemap.json": "3da60d884157793a360da532a9fcbbfb5285636db325a74cec94b34622186d97",
|
|
@@ -279,8 +279,8 @@
|
|
|
279
279
|
"spec/v2/path-manifest.json": "c123c9fd77dc1b3f9c2e9346ace80cdae007e6511138894956abda01e61ec1c6",
|
|
280
280
|
"spec/v2/peer-dependency-aliases.json": "d10299280abee08258502925bc327293ee413e0108cd6e6ec75ff6110653308d",
|
|
281
281
|
"spec/v2/profiles.json": "0636f19fceae625390003a347e70ef4797d84766b5c24ce8a02cea52aadebca4",
|
|
282
|
-
"spec/v2/release.json": "
|
|
282
|
+
"spec/v2/release.json": "94a3ea302324d9f7df87ce62a186b0b98936de6ce5ac5ec01a3fb3208c38efc3",
|
|
283
283
|
"spec/v2/retention-floors.json": "eaf3722d95c79947af1d4269ef85117e126518c588cfcf1a2b21b97269f51624"
|
|
284
284
|
},
|
|
285
|
-
"corpusCommit": "
|
|
285
|
+
"corpusCommit": "e9825b0da6ca8e2dbc76353e52aea73a56473393"
|
|
286
286
|
}
|
package/src/cli.ts
CHANGED
|
@@ -54,7 +54,7 @@ import {
|
|
|
54
54
|
PROFILE_FLOOR_SCENARIOS,
|
|
55
55
|
} from './lib/profiles.js';
|
|
56
56
|
import { setV2ProfileFloors, v2ProfileFloorFiles } from './lib/requirement-registry.js';
|
|
57
|
-
import { v2ProfileIds } from './lib/v2-profiles.js';
|
|
57
|
+
import { profilesDeniedByObservedRelaxation, profilesRelaxedBy, v2ProfileIds } from './lib/v2-profiles.js';
|
|
58
58
|
import { childEnv } from './lib/child-env.js';
|
|
59
59
|
|
|
60
60
|
interface ParsedArgs {
|
|
@@ -694,7 +694,9 @@ async function runCertify(args: ParsedArgs, baseUrl: string, apiKey: string): Pr
|
|
|
694
694
|
// belongs to. Matched the same way the verifier matches it, so the emitter
|
|
695
695
|
// and `verifyBundleV3` cannot disagree about the same bundle.
|
|
696
696
|
const relaxedObligations = new Set((relaxations ?? []).map((r) => r.obligation));
|
|
697
|
-
const
|
|
697
|
+
const relaxedProfileIds = profilesRelaxedBy([...relaxedObligations], claimedProfiles);
|
|
698
|
+
const observedRelaxed = profilesDeniedByObservedRelaxation(rows3, claimedProfiles).profiles;
|
|
699
|
+
const relaxedProfile = (p: string): boolean => relaxedProfileIds.has(p) || observedRelaxed.has(p);
|
|
698
700
|
const claimed3 = claimedProfiles.filter((p) => !(p in DEPRECATED_PROFILE_ALIASES)).map((p) => ({ id: p, evidenceTier: args.evidenceTier, witnessCount: witnessCountFor(p), certified: !relaxedProfile(p) && (verdictFor(p)?.certifiable ?? false) && !notHeld.has(p) && !rejectedProfiles.some((v) => v.profile === p) && totals3.blocked === 0 }));
|
|
699
701
|
const lockPath = resolvePath(conformanceRoot, 'dist', 'spec-artifacts.lock.json');
|
|
700
702
|
const lock = existsSync(lockPath) ? (JSON.parse(readFileSync(lockPath, 'utf8')) as { version: string; stampSha256: string }) : undefined;
|
package/src/lib/a2a-fake-peer.ts
CHANGED
|
@@ -38,6 +38,7 @@
|
|
|
38
38
|
* @see https://a2a-protocol.org/v0.3.0/specification
|
|
39
39
|
*/
|
|
40
40
|
|
|
41
|
+
import { resolvePublicFront } from './webhook-receiver.js';
|
|
41
42
|
import { createServer, type Server } from 'node:http';
|
|
42
43
|
import type { AddressInfo } from 'node:net';
|
|
43
44
|
|
|
@@ -198,6 +199,18 @@ export class A2AFakePeer {
|
|
|
198
199
|
return `http://127.0.0.1:${this._boundPort}`;
|
|
199
200
|
}
|
|
200
201
|
|
|
202
|
+
/**
|
|
203
|
+
* The address to hand THE HOST UNDER TEST — `OPENWOP_A2A_FAKE_PEER_URL` when the operator fronts
|
|
204
|
+
* this peer publicly (https, publicly resolvable; validated loudly, same rule
|
|
205
|
+
* as the webhook receiver), else `endpoint()`. The suite's own requests to its
|
|
206
|
+
* own fake keep using `endpoint()`: they need no tunnel and must not depend on
|
|
207
|
+
* one. Pin the listener with the matching `_PORT` variable so the front has a
|
|
208
|
+
* fixed port to forward to.
|
|
209
|
+
*/
|
|
210
|
+
hostFacingEndpoint(): string {
|
|
211
|
+
return resolvePublicFront('OPENWOP_A2A_FAKE_PEER_URL', this.endpoint()).url;
|
|
212
|
+
}
|
|
213
|
+
|
|
201
214
|
reset(): void {
|
|
202
215
|
this._tasks.clear();
|
|
203
216
|
this._invocations.length = 0;
|
|
@@ -293,7 +306,7 @@ export class A2AFakePeer {
|
|
|
293
306
|
description: 'Synthetic A2A peer for openwop conformance suite (dual-era: 1.0 + 0.3-legacy)',
|
|
294
307
|
version: '1.1.0',
|
|
295
308
|
supportedInterfaces: [...this._protocolVersions].sort((a, b) => (a === '1.0' ? -1 : b === '1.0' ? 1 : 0)).map((v) => ({
|
|
296
|
-
url: `${this.
|
|
309
|
+
url: `${this.hostFacingEndpoint()}/a2a/jsonrpc`,
|
|
297
310
|
protocolBinding: 'JSONRPC',
|
|
298
311
|
protocolVersion: v,
|
|
299
312
|
})),
|
|
@@ -310,13 +323,13 @@ export class A2AFakePeer {
|
|
|
310
323
|
protocolVersion: '0.3.0',
|
|
311
324
|
name: 'openwop-conformance-fake-a2a',
|
|
312
325
|
description: 'Synthetic A2A peer for openwop conformance suite',
|
|
313
|
-
url: `${this.
|
|
326
|
+
url: `${this.hostFacingEndpoint()}/a2a/jsonrpc`,
|
|
314
327
|
version: '1.0.0',
|
|
315
328
|
capabilities: { streaming: false, pushNotifications: false },
|
|
316
329
|
skills: [{ id: 'echo', name: 'echo', description: 'Returns input verbatim', tags: ['echo'] }],
|
|
317
330
|
defaultInputModes: ['text'],
|
|
318
331
|
defaultOutputModes: ['text'],
|
|
319
|
-
additionalInterfaces: [{ url: `${this.
|
|
332
|
+
additionalInterfaces: [{ url: `${this.hostFacingEndpoint()}/a2a/jsonrpc`, transport: 'JSONRPC' }],
|
|
320
333
|
};
|
|
321
334
|
}
|
|
322
335
|
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
*/
|
|
20
20
|
import { createHash, createPrivateKey, createPublicKey, sign as edSign, verify as edVerify, type KeyObject } from 'node:crypto';
|
|
21
21
|
import { profileDerivable, type DiscoveryPayload } from './profiles.js';
|
|
22
|
-
import { v2RegistryAvailable } from './v2-profiles.js';
|
|
22
|
+
import { profilesDeniedByObservedRelaxation, profilesRelaxedBy, v2RegistryAvailable } from './v2-profiles.js';
|
|
23
23
|
|
|
24
24
|
export type BundleV3Result = 'executed-pass' | 'executed-fail' | 'skipped' | 'inapplicable' | 'blocked';
|
|
25
25
|
|
|
@@ -179,9 +179,18 @@ export function verifyBundleV3(bundle: BundleV3, opts: VerifyV3Options = {}): V3
|
|
|
179
179
|
}
|
|
180
180
|
// Relaxations: a relaxed obligation's profile cannot certify (RFC 0173 §A.2).
|
|
181
181
|
const relaxed = new Set((bundle.host?.relaxations ?? []).map((r) => r.obligation.split('.')[0]));
|
|
182
|
+
// Ownership comes from the profile registry, not from the profile's name — see profilesRelaxedBy.
|
|
183
|
+
const relaxedProfiles = profilesRelaxedBy((bundle.host?.relaxations ?? []).map((r) => r.obligation), (bundle.claimedProfiles ?? []).map((p) => p.id));
|
|
184
|
+
// An OBSERVED relaxation nobody declared: the bundle's own results show the
|
|
185
|
+
// host accepting a destination its egress guard MUST refuse. Same denial as a
|
|
186
|
+
// declared one, so declaring nothing is not a way around the rule.
|
|
187
|
+
const observed = profilesDeniedByObservedRelaxation(bundle.results?.requirements ?? [], (bundle.claimedProfiles ?? []).map((p) => p.id));
|
|
188
|
+
for (const p of bundle.claimedProfiles ?? []) {
|
|
189
|
+
if (p.certified && observed.profiles.has(p.id) && !relaxedProfiles.has(p.id)) rejections.push({ kind: 'undeclared-relaxation-observed', profile: p.id, detail: `${p.id} is marked certified, but this bundle's own results show the host running relaxed on ${observed.obligations.join(', ')} with no host.relaxations[] entry recording it (security-defaults.md §Relaxations: every relaxation a host runs under MUST be recorded, and its profile MUST NOT certify)` });
|
|
190
|
+
}
|
|
182
191
|
const certifiedProfiles: string[] = [];
|
|
183
192
|
for (const p of bundle.claimedProfiles ?? []) {
|
|
184
|
-
if (p.certified &&
|
|
193
|
+
if (p.certified && relaxedProfiles.has(p.id)) rejections.push({ kind: 'relaxed-profile-certified', profile: p.id, detail: `${p.id} is marked certified while a relaxation on ${[...relaxed].join(', ')} is recorded (RFC 0173 §A.2)` });
|
|
185
194
|
else if (p.certified) certifiedProfiles.push(p.id);
|
|
186
195
|
}
|
|
187
196
|
if (expected.blocked > 0 && certifiedProfiles.length > 0) rejections.push({ kind: 'blocked-certified', detail: `${expected.blocked} blocked row(s): a bundle with blocked > 0 does not certify (RFC 0168 §E.1)` });
|
|
@@ -34,6 +34,7 @@
|
|
|
34
34
|
* @see SECURITY/threat-model-prompt-injection.md §"UNTRUSTED marker"
|
|
35
35
|
*/
|
|
36
36
|
|
|
37
|
+
import { resolvePublicFront } from './webhook-receiver.js';
|
|
37
38
|
import { createServer, type Server } from 'node:http';
|
|
38
39
|
import type { AddressInfo } from 'node:net';
|
|
39
40
|
|
|
@@ -149,6 +150,18 @@ export class McpFakeServer {
|
|
|
149
150
|
return `http://127.0.0.1:${this._boundPort}`;
|
|
150
151
|
}
|
|
151
152
|
|
|
153
|
+
/**
|
|
154
|
+
* The address to hand THE HOST UNDER TEST — `OPENWOP_MCP_FAKE_SERVER_URL` when the operator fronts
|
|
155
|
+
* this server publicly (https, publicly resolvable; validated loudly, same rule
|
|
156
|
+
* as the webhook receiver), else `endpoint()`. The suite's own requests to its
|
|
157
|
+
* own fake keep using `endpoint()`: they need no tunnel and must not depend on
|
|
158
|
+
* one. Pin the listener with the matching `_PORT` variable so the front has a
|
|
159
|
+
* fixed port to forward to.
|
|
160
|
+
*/
|
|
161
|
+
hostFacingEndpoint(): string {
|
|
162
|
+
return resolvePublicFront('OPENWOP_MCP_FAKE_SERVER_URL', this.endpoint()).url;
|
|
163
|
+
}
|
|
164
|
+
|
|
152
165
|
invocations(): readonly McpInvocation[] {
|
|
153
166
|
return this._invocations;
|
|
154
167
|
}
|
package/src/lib/v2-profiles.ts
CHANGED
|
@@ -110,3 +110,69 @@ export function v2ProfileIds(doc: DiscoveryPayload): readonly string[] | null {
|
|
|
110
110
|
export function v2ProfileDerivable(doc: DiscoveryPayload, profile: string): boolean {
|
|
111
111
|
return v2ProfileIds(doc)?.includes(profile) ?? false;
|
|
112
112
|
}
|
|
113
|
+
|
|
114
|
+
/**
|
|
115
|
+
* Which of `profileIds` a relaxed obligation denies (RFC 0173 §A.2;
|
|
116
|
+
* `security-defaults.md` §Relaxations: "a bundle that records a relaxation MUST
|
|
117
|
+
* NOT certify the profile the relaxed obligation belongs to").
|
|
118
|
+
*
|
|
119
|
+
* An obligation is `<family>.<name>`. A profile OWNS a family when its registry
|
|
120
|
+
* predicate lists it — `openwop-core-standard` is built on `interrupt`, `replay`,
|
|
121
|
+
* `webhooks`, `idempotency`, `eventLog`, so a relaxation on any of those denies
|
|
122
|
+
* it. That ownership was always stated in `spec/v2/profiles.json`; nothing read
|
|
123
|
+
* it. Until 2.33.0 the verifier tested whether the profile ID contained the
|
|
124
|
+
* family name, and the emitter tested whether it contained the WHOLE obligation
|
|
125
|
+
* string — two different rules under a comment saying they matched — and since
|
|
126
|
+
* no real v2 profile id contains a family name, a declared relaxation denied
|
|
127
|
+
* nothing on any real host. The one scenario pinning the rule used a fixture
|
|
128
|
+
* profile called `openwop-webhooks`, which is why it stayed green.
|
|
129
|
+
*
|
|
130
|
+
* The id-substring rule is KEPT as a second arm, not replaced: it is how a
|
|
131
|
+
* non-registry profile id (a v1 alias, a fixture) names its family, and
|
|
132
|
+
* dropping it would un-deny something that is denied today. When the registry
|
|
133
|
+
* is unavailable only that arm can run; the caller already records that layout
|
|
134
|
+
* as `derivabilityChecked: false`.
|
|
135
|
+
*
|
|
136
|
+
* ONE function, called by the emitter and the verifier, so the file and the
|
|
137
|
+
* verdict on the file cannot disagree again.
|
|
138
|
+
*/
|
|
139
|
+
export function profilesRelaxedBy(obligations: readonly string[], profileIds: readonly string[]): ReadonlySet<string> {
|
|
140
|
+
const families = new Set(obligations.map((o) => o.split('.')[0] ?? '').filter((f) => f.length > 0));
|
|
141
|
+
const out = new Set<string>();
|
|
142
|
+
if (families.size === 0) return out;
|
|
143
|
+
const registry = readRegistry();
|
|
144
|
+
const owned = new Map<string, readonly string[]>();
|
|
145
|
+
for (const p of registry ?? []) {
|
|
146
|
+
if (typeof p.id === 'string') owned.set(p.id, Array.isArray(p.predicate?.families) ? (p.predicate.families as unknown[]).map(String) : []);
|
|
147
|
+
}
|
|
148
|
+
for (const id of profileIds) {
|
|
149
|
+
const ownsOne = (owned.get(id) ?? []).some((f) => families.has(f));
|
|
150
|
+
const namesOne = [...families].some((f) => id.includes(f));
|
|
151
|
+
if (ownsOne || namesOne) out.add(id);
|
|
152
|
+
}
|
|
153
|
+
return out;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
/**
|
|
157
|
+
* The requirement whose FAILURE is an observed, undeclared relaxation.
|
|
158
|
+
*
|
|
159
|
+
* `conformance/README.md` said of `OPENWOP_HOST_RELAXATIONS` that "the suite
|
|
160
|
+
* cannot detect an undeclared one". For the egress guard that was never true:
|
|
161
|
+
* a host that ACCEPTS `https://127.0.0.1/…` as a webhook destination has shown
|
|
162
|
+
* on the wire that its guard is open (`webhooks.md`: "a host MUST reject (400
|
|
163
|
+
* webhook_url_rejected) … RFC 1918 and loopback and link-local ranges").
|
|
164
|
+
* `v2-webhook-egress-refusal` records that as `executed-fail` on this id unless
|
|
165
|
+
* the operator DECLARED the relaxation — in which case {@link profilesRelaxedBy}
|
|
166
|
+
* already denies the profile. Either way a profile built on `webhooks` does not
|
|
167
|
+
* certify on a host whose guard is open; this closes the path where it did,
|
|
168
|
+
* silently, because nobody declared anything.
|
|
169
|
+
*/
|
|
170
|
+
export const UNDECLARED_RELAXATION_WITNESSES: ReadonlyArray<{ readonly requirementId: string; readonly obligation: string }> = [
|
|
171
|
+
{ requirementId: 'openwop.requirement.0171.webhook-egress-refused', obligation: 'webhooks.egress-guard' },
|
|
172
|
+
];
|
|
173
|
+
|
|
174
|
+
/** Profiles that MUST NOT certify because a row above is `executed-fail` in this bundle's own results. */
|
|
175
|
+
export function profilesDeniedByObservedRelaxation(rows: ReadonlyArray<{ readonly id: string; readonly result: string }>, profileIds: readonly string[]): { readonly profiles: ReadonlySet<string>; readonly obligations: readonly string[] } {
|
|
176
|
+
const obligations = UNDECLARED_RELAXATION_WITNESSES.filter((w) => rows.some((r) => r.id === w.requirementId && r.result === 'executed-fail')).map((w) => w.obligation);
|
|
177
|
+
return { profiles: profilesRelaxedBy(obligations, profileIds), obligations };
|
|
178
|
+
}
|
|
@@ -235,7 +235,26 @@ export async function discoverOwnedTenant(
|
|
|
235
235
|
* "the host could not be exercised".
|
|
236
236
|
*/
|
|
237
237
|
export function resolveRegistrationUrl(localUrl: string): { url: string; tunnelled: boolean } {
|
|
238
|
-
|
|
238
|
+
return resolvePublicFront('OPENWOP_WEBHOOK_RECEIVER_URL', localUrl);
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
/**
|
|
242
|
+
* The same rule, for ANY suite fixture the host under test must reach — the
|
|
243
|
+
* webhook receiver, the A2A fake peer, the MCP fake server. `envName` names the
|
|
244
|
+
* operator's public front for that one fixture; unset means the local address.
|
|
245
|
+
*
|
|
246
|
+
* Generalised in 2.33.0. Until then only the webhook receiver had a front, so a
|
|
247
|
+
* host that advertised `a2a` or `mcp` could be measured ONLY with its egress
|
|
248
|
+
* guard relaxed — those fakes advertised `http://127.0.0.1:<port>` and nothing
|
|
249
|
+
* else — and a relaxed guard is a relaxation the bundle must declare and cannot
|
|
250
|
+
* certify under (`security-defaults.md` §Relaxations). The two production hosts
|
|
251
|
+
* cut relaxation-free only because they advertise neither family. The rule made
|
|
252
|
+
* a host that implements MORE of the protocol LESS able to certify; found when
|
|
253
|
+
* the steward's own reference host turned out to be certifying under an
|
|
254
|
+
* undeclared one.
|
|
255
|
+
*/
|
|
256
|
+
export function resolvePublicFront(envName: string, localUrl: string): { url: string; tunnelled: boolean } {
|
|
257
|
+
const raw = process.env[envName]?.trim();
|
|
239
258
|
if (!raw) return { url: localUrl, tunnelled: false };
|
|
240
259
|
|
|
241
260
|
let parsed: URL;
|
|
@@ -243,7 +262,7 @@ export function resolveRegistrationUrl(localUrl: string): { url: string; tunnell
|
|
|
243
262
|
parsed = new URL(raw);
|
|
244
263
|
} catch {
|
|
245
264
|
throw new Error(
|
|
246
|
-
|
|
265
|
+
`${envName} is not a valid URL: ${JSON.stringify(raw)}`,
|
|
247
266
|
);
|
|
248
267
|
}
|
|
249
268
|
|
|
@@ -252,7 +271,7 @@ export function resolveRegistrationUrl(localUrl: string): { url: string; tunnell
|
|
|
252
271
|
// ALLOW_PRIVATE flag insufficient in the first place.
|
|
253
272
|
if (parsed.protocol !== 'https:') {
|
|
254
273
|
throw new Error(
|
|
255
|
-
|
|
274
|
+
`${envName} MUST be https: (got ${parsed.protocol}). ` +
|
|
256
275
|
'A plain-http front cannot clear the scheme gate, so it cannot witness this scenario.',
|
|
257
276
|
);
|
|
258
277
|
}
|
|
@@ -272,13 +291,13 @@ export function resolveRegistrationUrl(localUrl: string): { url: string; tunnell
|
|
|
272
291
|
/^(fc|fd)/.test(host);
|
|
273
292
|
if (isLoopback || isPrivate) {
|
|
274
293
|
throw new Error(
|
|
275
|
-
|
|
294
|
+
`${envName} MUST be a publicly-resolvable host (got ${parsed.hostname}). ` +
|
|
276
295
|
'It is the PUBLIC front for the local receiver — a tunnel or TLS-terminating proxy — ' +
|
|
277
296
|
'not the receiver address itself.',
|
|
278
297
|
);
|
|
279
298
|
}
|
|
280
299
|
|
|
281
|
-
return { url: raw, tunnelled: true };
|
|
300
|
+
return { url: envName === 'OPENWOP_WEBHOOK_RECEIVER_URL' ? raw : raw.replace(/\/+$/, ''), tunnelled: true };
|
|
282
301
|
}
|
|
283
302
|
|
|
284
303
|
/**
|
|
@@ -62,7 +62,7 @@ describe('RFC 0152 — the suite peer speaks A2A 1.0 (dual-era A2AFakePeer)', ()
|
|
|
62
62
|
expect(ifaces.map((i) => i.protocolVersion)).toEqual(['1.0', '0.3']);
|
|
63
63
|
for (const i of ifaces) {
|
|
64
64
|
expect(i.protocolBinding).toBe('JSONRPC');
|
|
65
|
-
expect(i.url.startsWith(peer.
|
|
65
|
+
expect(i.url.startsWith(peer.hostFacingEndpoint())).toBe(true);
|
|
66
66
|
}
|
|
67
67
|
expect(card['url'], req('openwop.it.a2a-1-0-agent-card.the-agent-card-is-1-0-shaped-supportedinterfaces-no-top-level-url-protocolversio', 'RFC 0152 §A/§B/§C/§D', '1.0 removed top-level url')).toBeUndefined();
|
|
68
68
|
expect(card['protocolVersion'], req('openwop.it.a2a-1-0-agent-card.the-agent-card-is-1-0-shaped-supportedinterfaces-no-top-level-url-protocolversio', 'RFC 0152 §A/§B/§C/§D', '1.0 removed top-level protocolVersion (it is per interface)')).toBeUndefined();
|
|
@@ -64,7 +64,7 @@ describe('RFC 0152 §E — a2a-peer-no-authority-escalation (host as A2A client,
|
|
|
64
64
|
peer.reset();
|
|
65
65
|
peer.setNextPeerAssertsAuthority(true);
|
|
66
66
|
const drive = await driver.post('/v1/host/sample/a2a/invoke', {
|
|
67
|
-
peerUrl: peer.
|
|
67
|
+
peerUrl: peer.hostFacingEndpoint(),
|
|
68
68
|
scenario: 'peer-asserts-authority',
|
|
69
69
|
});
|
|
70
70
|
if (drive.status === 404 || drive.status === 403) {
|
|
@@ -66,7 +66,7 @@ function probePeer(): { url: string; isReal: boolean } | null {
|
|
|
66
66
|
const real = process.env.OPENWOP_A2A_REAL_PEER_URL;
|
|
67
67
|
if (real && real.length > 0) return { url: real.replace(/\/$/, ''), isReal: true };
|
|
68
68
|
const fake = getA2AFakePeer();
|
|
69
|
-
if (fake) return { url: fake.
|
|
69
|
+
if (fake) return { url: fake.hostFacingEndpoint(), isReal: false };
|
|
70
70
|
return null;
|
|
71
71
|
}
|
|
72
72
|
|
|
@@ -78,7 +78,7 @@ describe('RFC 0152 §B — A2A version negotiation', () => {
|
|
|
78
78
|
const peer = getA2AFakePeer();
|
|
79
79
|
if (peer === null) return softSkip('blocked', 'precondition not met — `peer === null` returned early (no fake peer wired in this run) (seam, prior step, or fixture unavailable)'); // no fake peer wired in this run
|
|
80
80
|
peer.reset();
|
|
81
|
-
const drive = await driver.post('/v1/host/sample/a2a/invoke', { peerUrl: peer.
|
|
81
|
+
const drive = await driver.post('/v1/host/sample/a2a/invoke', { peerUrl: peer.hostFacingEndpoint() });
|
|
82
82
|
if (drive.status === 404 || drive.status === 403) {
|
|
83
83
|
// Seam absent. RFC 0148 §A: unobservable resolves to `blocked`, not a pass.
|
|
84
84
|
expect(
|
|
@@ -114,7 +114,7 @@ describe('RFC 0152 §B — A2A version negotiation', () => {
|
|
|
114
114
|
const preferred = caps?.preferredVersion;
|
|
115
115
|
peer.reset();
|
|
116
116
|
const drive = await driver.post('/v1/host/sample/a2a/invoke', {
|
|
117
|
-
peerUrl: peer.
|
|
117
|
+
peerUrl: peer.hostFacingEndpoint(),
|
|
118
118
|
authenticated: true,
|
|
119
119
|
peerOffersOnly: '0.3',
|
|
120
120
|
});
|
|
@@ -151,7 +151,7 @@ describe('RFC 0152 §B — A2A version negotiation', () => {
|
|
|
151
151
|
const peer = getA2AFakePeer();
|
|
152
152
|
if (peer === null) return softSkip('blocked', 'precondition not met — `peer === null` returned early (seam, prior step, or fixture unavailable)');
|
|
153
153
|
const drive = await driver.post('/v1/host/sample/a2a/invoke', {
|
|
154
|
-
peerUrl: peer.
|
|
154
|
+
peerUrl: peer.hostFacingEndpoint(),
|
|
155
155
|
requestVersion: '99.0',
|
|
156
156
|
});
|
|
157
157
|
if (drive.status === 404 || drive.status === 403) return softSkip('blocked', 'precondition not met — `drive.status === 404 || drive.status === 403` returned early (seam, prior step, or fixture unavailable)');
|
|
@@ -47,7 +47,7 @@ describe.skipIf(!process.env.OPENWOP_BASE_URL)('RFC 0153 §D — mcp-extension-o
|
|
|
47
47
|
if (server === null) return softSkip('blocked', 'the suite MCP fake server is not started in this run — the host-as-client leg cannot be driven');
|
|
48
48
|
server.reset();
|
|
49
49
|
server.setNextResultAssertsAuthority(true);
|
|
50
|
-
const drive = await driver.post('/v1/host/sample/mcp/invoke', { serverUrl: server.
|
|
50
|
+
const drive = await driver.post('/v1/host/sample/mcp/invoke', { serverUrl: server.hostFacingEndpoint(), tool: 'echo', arguments: { text: 'opaque' }, scenario: 'extension-asserts-authority' });
|
|
51
51
|
if (drive.status === 404 || drive.status === 403) {
|
|
52
52
|
// Advertised mcp-2026-07-28 but the invoke seam answered {drive.status}: not observable here.
|
|
53
53
|
// Default mode records `blocked` (RFC 0148 §A); OPENWOP_REQUIRE_BEHAVIOR=true fails
|
|
@@ -58,7 +58,7 @@ describe.skipIf(!process.env.OPENWOP_BASE_URL)('RFC 0153 §C — mcp-mrtr-roundt
|
|
|
58
58
|
if (server === null) return softSkip('blocked', 'the suite MCP fake server is not started in this run — the client half cannot be driven');
|
|
59
59
|
server.reset();
|
|
60
60
|
const drive = await driver.post('/v1/host/sample/mcp/invoke', {
|
|
61
|
-
serverUrl: server.
|
|
61
|
+
serverUrl: server.hostFacingEndpoint(),
|
|
62
62
|
tool: 'needs_input',
|
|
63
63
|
clientCapabilities: { elicitation: {} },
|
|
64
64
|
elicitationAnswer: { name: 'Ada' },
|
|
@@ -179,7 +179,7 @@ function probeEndpoint(): { url: string; isReal: boolean } | null {
|
|
|
179
179
|
const real = process.env.OPENWOP_MCP_REAL_SERVER_URL;
|
|
180
180
|
if (real && real.length > 0) return { url: real.replace(/\/$/, ''), isReal: true };
|
|
181
181
|
const fake = getMcpFakeServer();
|
|
182
|
-
if (fake) return { url: fake.
|
|
182
|
+
if (fake) return { url: fake.hostFacingEndpoint(), isReal: false };
|
|
183
183
|
return null;
|
|
184
184
|
}
|
|
185
185
|
|
|
@@ -86,7 +86,7 @@ describe('RFC 0153 §A/§B — MCP revision negotiation', () => {
|
|
|
86
86
|
const server = getMcpFakeServer();
|
|
87
87
|
if (server === null) return softSkip('blocked', 'precondition not met — `server === null` returned early (seam, prior step, or fixture unavailable)');
|
|
88
88
|
server.reset();
|
|
89
|
-
const drive = await driver.post('/v1/host/sample/mcp/invoke', { serverUrl: server.
|
|
89
|
+
const drive = await driver.post('/v1/host/sample/mcp/invoke', { serverUrl: server.hostFacingEndpoint() });
|
|
90
90
|
if (drive.status === 404 || drive.status === 403) {
|
|
91
91
|
expect(
|
|
92
92
|
drive.status,
|
|
@@ -125,7 +125,7 @@ describe('RFC 0153 §A/§B — MCP revision negotiation', () => {
|
|
|
125
125
|
if (server === null) return softSkip('blocked', 'precondition not met — `server === null` returned early (seam, prior step, or fixture unavailable)');
|
|
126
126
|
const caps = await mcp();
|
|
127
127
|
server.reset();
|
|
128
|
-
const drive = await driver.post('/v1/host/sample/mcp/invoke', { serverUrl: server.
|
|
128
|
+
const drive = await driver.post('/v1/host/sample/mcp/invoke', { serverUrl: server.hostFacingEndpoint() });
|
|
129
129
|
if (drive.status === 404 || drive.status === 403) return seamAbsent(`host advertises mcp version negotiation but the invoke seam /v1/host/sample/mcp/invoke answered ${drive.status} — the host-as-client legs are unobservable (host-sample-test-seams.md)`);
|
|
130
130
|
// it never advertised has made its own discovery document unreliable, which
|
|
131
131
|
// is worse than advertising nothing — a consumer that read it made a
|
|
@@ -148,7 +148,7 @@ describe('RFC 0153 §A/§B — MCP revision negotiation', () => {
|
|
|
148
148
|
const server = getMcpFakeServer();
|
|
149
149
|
if (server === null) return softSkip('blocked', 'precondition not met — `server === null` returned early (seam, prior step, or fixture unavailable)');
|
|
150
150
|
const drive = await driver.post('/v1/host/sample/mcp/invoke', {
|
|
151
|
-
serverUrl: server.
|
|
151
|
+
serverUrl: server.hostFacingEndpoint(),
|
|
152
152
|
requestVersion: '1999-01-01',
|
|
153
153
|
});
|
|
154
154
|
if (drive.status === 404 || drive.status === 403) return seamAbsent(`host advertises mcp version negotiation but the invoke seam /v1/host/sample/mcp/invoke answered ${drive.status} — the host-as-client legs are unobservable (host-sample-test-seams.md)`);
|
|
@@ -96,7 +96,7 @@ describe('RFC 0175 §D.2 — minimum-version-refused (gated on a2a/mcp + seams)'
|
|
|
96
96
|
const peer = new A2AFakePeer({ protocolVersions: ['0.3'] });
|
|
97
97
|
await peer.start();
|
|
98
98
|
try {
|
|
99
|
-
const res = await driver.post(`${SEAMS_PREFIX}/sample/a2a/invoke`, { peerUrl: peer.
|
|
99
|
+
const res = await driver.post(`${SEAMS_PREFIX}/sample/a2a/invoke`, { peerUrl: peer.hostFacingEndpoint(), authenticated: true, peerOffersOnly: A2A_LOWEST });
|
|
100
100
|
if (res.status === 404 || res.status === 403 || res.status === 405) return seamAbsent(`host advertises a2a but ${SEAMS_PREFIX}/sample/a2a/invoke answered ${res.status} (host-sample-test-seams.md §22)`);
|
|
101
101
|
await assertRefused('openwop.requirement.0175.minimum-version-refused', 'a2a', floor, res);
|
|
102
102
|
// The wire leg: the host MUST NOT have spoken the below-floor version to the peer.
|
|
@@ -124,7 +124,7 @@ describe('RFC 0175 §D.2 — minimum-version-refused (gated on a2a/mcp + seams)'
|
|
|
124
124
|
const server = new McpFakeServer({ protocolVersions: ['2025-06-18'] });
|
|
125
125
|
await server.start();
|
|
126
126
|
try {
|
|
127
|
-
const res = await driver.post(`${SEAMS_PREFIX}/sample/mcp/invoke`, { serverUrl: server.
|
|
127
|
+
const res = await driver.post(`${SEAMS_PREFIX}/sample/mcp/invoke`, { serverUrl: server.hostFacingEndpoint(), requestVersion: MCP_LOWEST });
|
|
128
128
|
if (res.status === 404 || res.status === 403 || res.status === 405) return seamAbsent(`host advertises mcp but ${SEAMS_PREFIX}/sample/mcp/invoke answered ${res.status} (host-sample-test-seams.md §23)`);
|
|
129
129
|
await assertRefused('openwop.requirement.0175.minimum-version-refused.mcp', 'mcp', floor, res);
|
|
130
130
|
for (const c of server.invocations()) {
|
|
@@ -73,7 +73,7 @@ describe('RFC 0175 §E.1 — mrtr-rounds-ceiling (gated on mcp + mrtr)', () => {
|
|
|
73
73
|
// input_required for `arguments.rounds` retries before completing.
|
|
74
74
|
const rounds = (maxRounds as number) + 1;
|
|
75
75
|
const res = await driver.post(`${SEAMS_PREFIX}/sample/mcp/invoke`, {
|
|
76
|
-
serverUrl: server.
|
|
76
|
+
serverUrl: server.hostFacingEndpoint(),
|
|
77
77
|
tool: LOOP_TOOL,
|
|
78
78
|
arguments: { rounds },
|
|
79
79
|
clientCapabilities: { elicitation: {} },
|
|
@@ -99,7 +99,7 @@ describe('RFC 0175 §D.1 — negotiation-authenticated (gated on a2a/mcp + seams
|
|
|
99
99
|
const peer = new A2AFakePeer({ protocolVersions: ['0.3'] });
|
|
100
100
|
await peer.start();
|
|
101
101
|
try {
|
|
102
|
-
const res = await driver.post(`${SEAMS_PREFIX}/sample/a2a/invoke`, { peerUrl: peer.
|
|
102
|
+
const res = await driver.post(`${SEAMS_PREFIX}/sample/a2a/invoke`, { peerUrl: peer.hostFacingEndpoint(), authenticated: false, peerOffersOnly: A2A_LOWER });
|
|
103
103
|
if (res.status === 404 || res.status === 403 || res.status === 405) return seamAbsent(`host advertises a2a but ${SEAMS_PREFIX}/sample/a2a/invoke answered ${res.status} (host-sample-test-seams.md §22)`);
|
|
104
104
|
const wire = peer.invocations().filter((i) => i.method !== 'GET').map((i) => String(i.headers['a2a-version'] ?? ''));
|
|
105
105
|
await assertNotLowered('openwop.requirement.0175.negotiation-authenticated', preferred, A2A_LOWER, res, wire);
|
|
@@ -119,7 +119,7 @@ describe('RFC 0175 §D.1 — negotiation-authenticated (gated on a2a/mcp + seams
|
|
|
119
119
|
const server = new McpFakeServer({ protocolVersions: ['2025-06-18'] });
|
|
120
120
|
await server.start();
|
|
121
121
|
try {
|
|
122
|
-
const res = await driver.post(`${SEAMS_PREFIX}/sample/mcp/invoke`, { serverUrl: server.
|
|
122
|
+
const res = await driver.post(`${SEAMS_PREFIX}/sample/mcp/invoke`, { serverUrl: server.hostFacingEndpoint(), authenticated: false });
|
|
123
123
|
if (res.status === 404 || res.status === 403 || res.status === 405) return seamAbsent(`host advertises mcp but ${SEAMS_PREFIX}/sample/mcp/invoke answered ${res.status} (host-sample-test-seams.md §23)`);
|
|
124
124
|
const wire = server.invocations().map((i) => String(i.headers['mcp-protocol-version'] ?? ''));
|
|
125
125
|
await assertNotLowered('openwop.requirement.0175.negotiation-authenticated.mcp', preferred, MCP_LOWER, res, wire);
|
|
@@ -95,7 +95,7 @@ async function leg(protocol: 'a2a' | 'mcp', id: string): Promise<void> {
|
|
|
95
95
|
const validate = payloadValidator();
|
|
96
96
|
if (!validate) return softSkip('blocked', 'run-event-payloads.schema.json#/$defs/negotiationDecided not readable from SCHEMAS_DIR');
|
|
97
97
|
|
|
98
|
-
const driven = await drive(protocol, peer.
|
|
98
|
+
const driven = await drive(protocol, peer.hostFacingEndpoint());
|
|
99
99
|
if (!driven) return softSkip('blocked', 'invoke seam unavailable (reason recorded above)');
|
|
100
100
|
const runId = driven.body['runId'];
|
|
101
101
|
if (typeof runId !== 'string') {
|
|
@@ -117,11 +117,11 @@ async function leg(protocol: 'a2a' | 'mcp', id: string): Promise<void> {
|
|
|
117
117
|
expect(
|
|
118
118
|
p['peerDigest'],
|
|
119
119
|
req(id, 'interop.md §The audit event', 'peerDigest MUST be the SHA-256 of the peer origin — never the origin in clear (RFC 0175 §D.3)'),
|
|
120
|
-
).toBe(originDigest(peer.
|
|
120
|
+
).toBe(originDigest(peer.hostFacingEndpoint()));
|
|
121
121
|
}
|
|
122
122
|
for (const v of Object.values(p)) {
|
|
123
123
|
expect(
|
|
124
|
-
typeof v === 'string' && v.includes(new URL(peer.
|
|
124
|
+
typeof v === 'string' && v.includes(new URL(peer.hostFacingEndpoint()).host),
|
|
125
125
|
req(id, 'interop.md §The audit event', 'the event is content-free: the peer origin MUST NOT appear in clear in any field'),
|
|
126
126
|
).toBe(false);
|
|
127
127
|
}
|
|
@@ -28,6 +28,8 @@ import { generateKeyPairSync } from 'node:crypto';
|
|
|
28
28
|
import { signBundleV3, verifyBundleV3, witnessDigest, type BundleV3, type BundleV3Relaxation } from '../lib/certification-bundle-v3.js';
|
|
29
29
|
import { v2Validator } from '../lib/v2.js';
|
|
30
30
|
import { req } from '../lib/requirement-ids.js';
|
|
31
|
+
import { softSkip } from '../lib/soft-skip.js';
|
|
32
|
+
import { v2RegistryAvailable } from '../lib/v2-profiles.js';
|
|
31
33
|
|
|
32
34
|
const host = generateKeyPairSync('ed25519');
|
|
33
35
|
const hostPem = host.privateKey.export({ type: 'pkcs8', format: 'pem' }) as string;
|
|
@@ -38,7 +40,7 @@ const ROWS: BundleV3['results']['requirements'] = [
|
|
|
38
40
|
];
|
|
39
41
|
|
|
40
42
|
/** A schema-valid, verifier-clean bundle v3; `relaxations` and the claimed profile vary per leg. */
|
|
41
|
-
function bundle(opts: { relaxations?: BundleV3Relaxation[]; certified: boolean }): BundleV3 {
|
|
43
|
+
function bundle(opts: { relaxations?: BundleV3Relaxation[]; certified: boolean; profileId?: string }): BundleV3 {
|
|
42
44
|
const unsigned: Omit<BundleV3, 'signature'> = {
|
|
43
45
|
bundleVersion: '3',
|
|
44
46
|
generatedAt: '2026-09-03T00:00:00Z',
|
|
@@ -52,7 +54,7 @@ function bundle(opts: { relaxations?: BundleV3Relaxation[]; certified: boolean }
|
|
|
52
54
|
},
|
|
53
55
|
discovery: { url: 'https://fixture.invalid/.well-known/openwop', sha256: 'a'.repeat(64), protocolVersions: ['2.0'], preferredVersion: '2.0' },
|
|
54
56
|
claimedProfiles: [
|
|
55
|
-
{ id: 'openwop-webhooks', evidenceTier: 'self', witnessCount: 1, certified: opts.certified },
|
|
57
|
+
{ id: opts.profileId ?? 'openwop-webhooks', evidenceTier: 'self', witnessCount: 1, certified: opts.certified },
|
|
56
58
|
{ id: 'openwop-discovery-core', evidenceTier: 'self', witnessCount: 1, certified: true },
|
|
57
59
|
],
|
|
58
60
|
results: { totals: { executedPass: ROWS.length, executedFail: 0, skipped: 0, inapplicable: 0, blocked: 0 }, requirements: ROWS },
|
|
@@ -105,6 +107,39 @@ describe('RFC 0173 §A.2 — relaxation-recorded (unaided, fixture bundle)', ()
|
|
|
105
107
|
).not.toContain('relaxed-profile-certified');
|
|
106
108
|
});
|
|
107
109
|
|
|
110
|
+
// Until 2.33.0 every leg above used the FIXTURE profile `openwop-webhooks`, and
|
|
111
|
+
// the verifier matched a relaxation to a profile by testing whether the profile
|
|
112
|
+
// ID contained the family name. No real v2 profile id contains one —
|
|
113
|
+
// `openwop-discovery-core`, `openwop-core-standard`,
|
|
114
|
+
// `openwop-conformance-seams-v2` — so on every real host a DECLARED relaxation
|
|
115
|
+
// denied nothing, and this file stayed green because its fixture was the one
|
|
116
|
+
// id the rule could match. Ownership is what `spec/v2/profiles.json` states:
|
|
117
|
+
// `openwop-core-standard`'s predicate is built on `webhooks`.
|
|
118
|
+
it('a relaxation denies the REAL profile that owns its family, not only one named after it', () => {
|
|
119
|
+
if (!v2RegistryAvailable()) return softSkip('inapplicable', 'spec/v2/profiles.json is absent from this layout — profile ownership cannot be read, so only the id-named arm of the rule can run');
|
|
120
|
+
const EGRESS: BundleV3Relaxation = { obligation: 'webhooks.egress-guard', durability: 'session', reason: 'conformance cut against loopback fixtures' };
|
|
121
|
+
const control = verifyBundleV3(bundle({ certified: true, profileId: 'openwop-core-standard' }));
|
|
122
|
+
expect(
|
|
123
|
+
control.certifiedProfiles,
|
|
124
|
+
req('openwop.requirement.0173.relaxation-recorded.owned-by-registry', 'security-defaults.md §Relaxations', 'control: with no relaxation recorded, the real profile certifies — so the rejection below is the relaxation\'s'),
|
|
125
|
+
).toContain('openwop-core-standard');
|
|
126
|
+
const relaxed = verifyBundleV3(bundle({ relaxations: [EGRESS], certified: true, profileId: 'openwop-core-standard' }));
|
|
127
|
+
expect(
|
|
128
|
+
relaxed.rejections.find((r) => r.kind === 'relaxed-profile-certified')?.profile,
|
|
129
|
+
req('openwop.requirement.0173.relaxation-recorded.owned-by-registry', 'security-defaults.md §Relaxations', 'a relaxation on `webhooks.*` MUST deny `openwop-core-standard`, whose registry predicate is built on the `webhooks` family — ownership is read from spec/v2/profiles.json, never from the spelling of the profile id'),
|
|
130
|
+
).toBe('openwop-core-standard');
|
|
131
|
+
expect(
|
|
132
|
+
relaxed.certifiedProfiles,
|
|
133
|
+
req('openwop.requirement.0173.relaxation-recorded.owned-by-registry', 'security-defaults.md §Relaxations', 'the owning profile is not certified, and a profile that does not own the family still is'),
|
|
134
|
+
).toEqual(['openwop-discovery-core']);
|
|
135
|
+
// A family no claimed profile owns denies nothing — the relaxation is still recorded, which is the obligation.
|
|
136
|
+
const unowned = verifyBundleV3(bundle({ relaxations: [{ ...EGRESS, obligation: 'mcp.egress-guard' }], certified: true, profileId: 'openwop-core-standard' }));
|
|
137
|
+
expect(
|
|
138
|
+
unowned.certifiedProfiles,
|
|
139
|
+
req('openwop.requirement.0173.relaxation-recorded.owned-by-registry', 'security-defaults.md §Relaxations', 'a relaxation on a family outside every claimed profile\'s predicate is scoped to nothing claimed — it does not poison an unrelated profile'),
|
|
140
|
+
).toContain('openwop-core-standard');
|
|
141
|
+
});
|
|
142
|
+
|
|
108
143
|
it('durability is a closed set: session | deployment | persisted', () => {
|
|
109
144
|
const validate = v2Validator('certification-bundle');
|
|
110
145
|
const withDurability = (durability: string): unknown => {
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* v2-webhook-egress-refusal — the webhook egress guard REFUSES, at major 2.
|
|
3
|
+
*
|
|
4
|
+
* `spec/v2/core/webhooks.md` §SSRF: "At registration a host MUST reject (`400
|
|
5
|
+
* webhook_url_rejected`) non-`https://` URLs, RFC 1918 and loopback and
|
|
6
|
+
* link-local ranges, IPv6 ULA, cloud metadata hosts, and `localhost`."
|
|
7
|
+
*
|
|
8
|
+
* ── Why this file exists ─────────────────────────────────────────────────────
|
|
9
|
+
* Until 2.33.0 NO major-2 row asserted that sentence. The scenarios that do
|
|
10
|
+
* (`http-client-ssrf`, `webhook-negative`) are registered at major 1 only, and
|
|
11
|
+
* the three major-2 files that name `webhook_url_rejected` name it only as the
|
|
12
|
+
* soft-skip taken when a host refuses the suite's own receiver. So a v2 bundle
|
|
13
|
+
* could certify `openwop-core-standard` — the profile built on `webhooks` — on a
|
|
14
|
+
* host whose guard refused nothing at all.
|
|
15
|
+
*
|
|
16
|
+
* It was found the expensive way: the steward's own reference host had been
|
|
17
|
+
* cutting its certified bundles with the guard open (`OPENWOP_WEBHOOK_ALLOW_PRIVATE`,
|
|
18
|
+
* because every suite fixture was a loopback listener) and recording no
|
|
19
|
+
* relaxation, which `security-defaults.md` §Relaxations forbids. Nothing in the
|
|
20
|
+
* suite could see it. `conformance/README.md` even said so: "the suite cannot
|
|
21
|
+
* detect an undeclared one." For THIS relaxation that was never true — a host
|
|
22
|
+
* that accepts `https://127.0.0.1/…` has shown on the wire that its guard is open.
|
|
23
|
+
*
|
|
24
|
+
* ── The three outcomes ───────────────────────────────────────────────────────
|
|
25
|
+
* every probe refused → executed-pass. The guard holds.
|
|
26
|
+
* a probe ACCEPTED, relaxation
|
|
27
|
+
* DECLARED (OPENWOP_HOST_RELAXATIONS
|
|
28
|
+
* names a `webhooks.*` obligation) → inapplicable, with the declaration
|
|
29
|
+
* quoted. The bundle carries
|
|
30
|
+
* `host.relaxations[]` and the owning
|
|
31
|
+
* profile is denied by that (RFC 0173 §A.2).
|
|
32
|
+
* a probe ACCEPTED, nothing declared → executed-FAIL — and the emitter and
|
|
33
|
+
* verifier deny every profile that owns
|
|
34
|
+
* `webhooks` (`lib/v2-profiles.ts`
|
|
35
|
+
* `profilesDeniedByObservedRelaxation`).
|
|
36
|
+
*
|
|
37
|
+
* Registration-time only, by design: every probe is a destination the suite
|
|
38
|
+
* does not own, so nothing is ever delivered and an accepted registration is
|
|
39
|
+
* deleted at once. Delivery-time re-resolution is a separate MUST that needs a
|
|
40
|
+
* resolver the suite controls; it is not claimed here.
|
|
41
|
+
*
|
|
42
|
+
* @see spec/v2/core/webhooks.md §SSRF
|
|
43
|
+
* @see spec/v2/core/security-defaults.md §Relaxations
|
|
44
|
+
*/
|
|
45
|
+
|
|
46
|
+
import { describe, expect, it } from 'vitest';
|
|
47
|
+
import { driver } from '../lib/driver.js';
|
|
48
|
+
import { gateFamily, v2Discovery } from '../lib/v2.js';
|
|
49
|
+
import { readErrorCode } from '../lib/error-envelope.js';
|
|
50
|
+
import { softSkip } from '../lib/soft-skip.js';
|
|
51
|
+
import { req } from '../lib/requirement-ids.js';
|
|
52
|
+
|
|
53
|
+
const ID = 'openwop.requirement.0171.webhook-egress-refused';
|
|
54
|
+
const DOC = 'spec/v2/core/webhooks.md §SSRF';
|
|
55
|
+
const EVENTS = ['run.completed'];
|
|
56
|
+
|
|
57
|
+
/** One destination per class the sentence names. None is ever contacted. */
|
|
58
|
+
const PROBES: ReadonlyArray<{ readonly url: string; readonly cls: string }> = [
|
|
59
|
+
{ url: 'http://webhook-egress-probe.example.com/hook', cls: 'non-https scheme' },
|
|
60
|
+
{ url: 'https://127.0.0.1/openwop-egress-probe', cls: 'loopback (IPv4)' },
|
|
61
|
+
{ url: 'https://[::1]/openwop-egress-probe', cls: 'loopback (IPv6)' },
|
|
62
|
+
{ url: 'https://localhost/openwop-egress-probe', cls: 'localhost' },
|
|
63
|
+
{ url: 'https://10.255.255.1/openwop-egress-probe', cls: 'RFC 1918 (10/8)' },
|
|
64
|
+
{ url: 'https://192.168.255.1/openwop-egress-probe', cls: 'RFC 1918 (192.168/16)' },
|
|
65
|
+
{ url: 'https://169.254.169.254/latest/meta-data/', cls: 'link-local / cloud metadata' },
|
|
66
|
+
{ url: 'https://[fd00::1]/openwop-egress-probe', cls: 'IPv6 ULA' },
|
|
67
|
+
];
|
|
68
|
+
|
|
69
|
+
/** The `webhooks.*` obligations the operator declared relaxed, if any. */
|
|
70
|
+
function declaredWebhookRelaxations(): string[] {
|
|
71
|
+
const raw = process.env['OPENWOP_HOST_RELAXATIONS'];
|
|
72
|
+
if (!raw) return [];
|
|
73
|
+
try {
|
|
74
|
+
const parsed = JSON.parse(raw) as Array<{ obligation?: unknown }>;
|
|
75
|
+
return (Array.isArray(parsed) ? parsed : []).map((r) => String(r.obligation ?? '')).filter((o) => o.split('.')[0] === 'webhooks');
|
|
76
|
+
} catch {
|
|
77
|
+
return []; // the emitter rejects unparseable JSON loudly; this file does not second-guess it
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
describe('v2-webhook-egress-refusal (webhooks.md §SSRF — the guard refuses, at major 2)', () => {
|
|
82
|
+
it('a webhook registration naming a destination the guard MUST refuse is refused 400 webhook_url_rejected', async () => {
|
|
83
|
+
const doc = await v2Discovery();
|
|
84
|
+
if (!doc) return softSkip('blocked', 'v2 discovery unreachable');
|
|
85
|
+
if (!(await gateFamily('webhooks'))) return softSkip('inapplicable', 'webhooks family not advertised (gate recorded under openwop.family.webhooks)');
|
|
86
|
+
|
|
87
|
+
const accepted: string[] = [];
|
|
88
|
+
const wrongCode: string[] = [];
|
|
89
|
+
for (const probe of PROBES) {
|
|
90
|
+
const reg = await driver.post('/webhooks', { url: probe.url, events: EVENTS });
|
|
91
|
+
if (reg.status === 201) {
|
|
92
|
+
accepted.push(`${probe.cls} (${probe.url})`);
|
|
93
|
+
const webhookId = (reg.json as { webhookId?: unknown } | null)?.webhookId;
|
|
94
|
+
if (typeof webhookId === 'string') await driver.delete(`/webhooks/${encodeURIComponent(webhookId)}`);
|
|
95
|
+
} else if (!(reg.status === 400 && readErrorCode(reg.json) === 'webhook_url_rejected')) {
|
|
96
|
+
wrongCode.push(`${probe.cls}: ${reg.status} ${String(readErrorCode(reg.json))}`);
|
|
97
|
+
}
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
if (accepted.length > 0) {
|
|
101
|
+
const declared = declaredWebhookRelaxations();
|
|
102
|
+
if (declared.length > 0) {
|
|
103
|
+
return softSkip('inapplicable', `the egress guard is relaxed and the operator DECLARED it (OPENWOP_HOST_RELAXATIONS: ${declared.join(', ')}) — the host accepted ${accepted.length} of ${PROBES.length} destinations the guard MUST refuse (${accepted.join('; ')}). The refusal cannot be witnessed on a host running relaxed; the bundle carries host.relaxations[] and the profile that owns \`webhooks\` does not certify (security-defaults.md §Relaxations)`);
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
expect(
|
|
107
|
+
accepted,
|
|
108
|
+
req(ID, DOC, `a host MUST reject (400 webhook_url_rejected) non-https URLs, RFC 1918, loopback, link-local, IPv6 ULA, cloud-metadata hosts and localhost at registration — this host ACCEPTED ${accepted.length} of ${PROBES.length} (${accepted.join('; ')}) and no \`webhooks.*\` relaxation is declared in OPENWOP_HOST_RELAXATIONS. An open guard nobody declared is an UNDECLARED RELAXATION: security-defaults.md §Relaxations requires it in host.relaxations[], and no profile built on \`webhooks\` certifies either way`),
|
|
109
|
+
).toEqual([]);
|
|
110
|
+
expect(
|
|
111
|
+
wrongCode,
|
|
112
|
+
req(ID, DOC, `the refusal MUST be 400 webhook_url_rejected (spec/v2/errors.json) — other answers: ${wrongCode.join('; ')}`),
|
|
113
|
+
).toEqual([]);
|
|
114
|
+
}, 60_000);
|
|
115
|
+
});
|