@openwop/openwop-conformance 2.45.23 → 2.45.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +227 -608
- package/coverage.md +118 -90
- package/dist/spec-artifacts.lock.json +2 -2
- package/fixtures.md +77 -55
- package/package.json +3 -3
- package/requirements.json +68 -68
- package/schemas/CORPUS-STAMP.json +15 -15
- package/src/lib/effect-receiver.ts +12 -1
package/coverage.md
CHANGED
|
@@ -1,18 +1,103 @@
|
|
|
1
1
|
# OpenWOP Conformance Coverage Map
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
This map connects the scenario files in `src/scenarios/` to the protocol surfaces they test. v2 comes first; the v1 map below it is frozen.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
- The suite has 649 scenario files. 209 run against a v2 host and 455 against a v1 host; 15 run at both ([`scenario-majors.json`](./scenario-majors.json)).
|
|
6
|
+
- File names here are the source of truth. A row lists the files; each file's docblock says what it asserts and how it is gated.
|
|
7
|
+
- Paths of the form `<repo>:<path>` point into a sibling repository under `https://github.com/openwop` (for example `openwop-examples:` or `openwop-app:`). Bare paths are in this repository.
|
|
6
8
|
|
|
7
|
-
|
|
9
|
+
**Run stateful hosts serially.** Some scenarios drive a host test seam that holds global state (a projected event log plus a `POST .../test/reset`). vitest runs files in parallel by default, so against one in-process host instance concurrent `reset` calls clobber each other and produce phantom failures or passes. Use `npm run test:strict` (`vitest run --no-file-parallelism`), or pass `--no-file-parallelism` into your own wrapper.
|
|
8
10
|
|
|
9
|
-
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## v2 coverage by core document
|
|
14
|
+
|
|
15
|
+
Each scenario that targets major 2 is listed once, under the [`spec/v2/core/`](../spec/v2/core/) document its docblock cites first. Files whose name does not start with `v2-` run at both majors. For per-family witness status, see the generated [`docs/V2-WITNESS-COVERAGE.md`](../docs/V2-WITNESS-COVERAGE.md).
|
|
16
|
+
|
|
17
|
+
| Core document | Scenario files |
|
|
18
|
+
| --- | --- |
|
|
19
|
+
| [`artifact-type-packs.md`](../spec/v2/core/artifact-type-packs.md) | `v2-artifact-type-schema-url.test.ts`, `v2-artifact-types-advertisement.test.ts` |
|
|
20
|
+
| [`capabilities.md`](../spec/v2/core/capabilities.md) | `v2-agent-org-chart-served-shape.test.ts`, `v2-capabilities-root-closed.test.ts`, `v2-capability-maturity-bounded.test.ts`, `v2-capability-record-shape.test.ts`, `v2-discovery-etag.test.ts`, `v2-implementation-informational.test.ts`, `v2-profiles-derived-only.test.ts` |
|
|
21
|
+
| [`conformance.md`](../spec/v2/core/conformance.md) | `fixtures-valid.test.ts`, `jcs-vectors.test.ts`, `v2-advertised-fixtures-exist.test.ts`, `v2-bundle-v3-signed.test.ts`, `v2-coherence-not-in-bundle.test.ts`, `v2-production-backpressure.test.ts` |
|
|
22
|
+
| [`connection-packs.md`](../spec/v2/core/connection-packs.md) | `v2-provider-conflict.test.ts` |
|
|
23
|
+
| [`conversation.md`](../spec/v2/core/conversation.md) | `v2-channel-presence-advertisement.test.ts`, `v2-channel-presence-delivery.test.ts`, `v2-conversation-turn-model-provenance-advertisement.test.ts`, `v2-conversation-turn-model-provenance.test.ts`, `v2-multi-party-conversation-advertisement.test.ts`, `v2-multi-party-council.test.ts` |
|
|
24
|
+
| [`errors.md`](../spec/v2/core/errors.md) | `v2-error-registry.test.ts`, `v2-interrupt-resolve-terminal.test.ts`, `v2-unadvertised-operation-not-found.test.ts` |
|
|
25
|
+
| [`events.md`](../spec/v2/core/events.md) | `v2-enum-growth-rule.test.ts`, `v2-envelope-contracts-advertisement.test.ts`, `v2-envelope-reasoning-shape.test.ts`, `v2-envelope-tier-one-subset-static.test.ts`, `v2-envelopes-advertisement.test.ts`, `v2-event-type-closed.test.ts`, `v2-host-event-delivery.test.ts`, `v2-payload-registry-closed.test.ts`, `v2-payload-seats-0186.test.ts`, `v2-payload-vendor-hatch.test.ts`, `v2-poll-cursor-v2.test.ts`, `v2-provider-usage-advertisement.test.ts`, `v2-provider-usage-emission.test.ts`, `v2-run-completed-outputs.test.ts`, `v2-sse-last-event-id-cursor.test.ts`, `v2-sse-last-event-id.test.ts`, `v2-stream-mode-refusal.test.ts`, `v2-stream-sse-projection.test.ts`, `v2-terminal-event-once.test.ts` |
|
|
26
|
+
| [`execution.md`](../spec/v2/core/execution.md) | `context-budget-transcript-bound.test.ts`, `v2-dispatch-input-mapping.test.ts`, `v2-dispatch-loop.test.ts`, `v2-self-hosted-runner-advertisement.test.ts`, `v2-sub-workflow-advertisement.test.ts`, `v2-subworkflow-input-mapping.test.ts`, `v2-subworkflow-linkage.test.ts` |
|
|
27
|
+
| [`form-content-packs.md`](../spec/v2/core/form-content-packs.md) | `v2-form-when-reuses-edge-conditions.test.ts` |
|
|
28
|
+
| [`headers.md`](../spec/v2/core/headers.md) | `v2-cors-preflight.test.ts`, `v2-header-scheme.test.ts` |
|
|
29
|
+
| [`host-services.md`](../spec/v2/core/host-services.md) | `v2-agent-runtime-advertisement.test.ts`, `v2-ai-envelope-advertisement.test.ts`, `v2-ai-providers-advertisement.test.ts`, `v2-mcp-client-results.test.ts`, `v2-memory-cross-tenant-isolation.test.ts`, `v2-model-capabilities-advertisement.test.ts`, `v2-model-capability-insufficient.test.ts`, `v2-prompt-all-four-kinds-events.test.ts`, `v2-prompt-end-to-end-events.test.ts`, `v2-prompt-library-advertisement.test.ts`, `v2-prompt-list-and-fetch.test.ts`, `v2-prompt-mutable-lifecycle.test.ts`, `v2-prompt-pack-install.test.ts`, `v2-prompt-render-deterministic.test.ts`, `v2-prompt-resolution-chain-event.test.ts`, `v2-prompt-template-shape.test.ts`, `v2-queue-cross-tenant-isolation.test.ts`, `v2-safefetch-ssrf-refused.test.ts`, `v2-scheduling-advertisement.test.ts`, `v2-secret-canary-absent.test.ts`, `v2-secrets-run-witness.test.ts`, `v2-tool-authorization-fail-closed.test.ts`, `v2-workspace-scope-from-identity.test.ts` |
|
|
30
|
+
| [`i18n.md`](../spec/v2/core/i18n.md) | `v2-content-locale-keys.test.ts`, `v2-i18n-negotiation.test.ts`, `v2-localized-content-delivery.test.ts` |
|
|
31
|
+
| [`idempotency.md`](../spec/v2/core/idempotency.md) | `v2-durability-recovery.test.ts`, `v2-idempotency-in-flight.test.ts`, `v2-idempotency-key-grammar.test.ts` |
|
|
32
|
+
| [`identity.md`](../spec/v2/core/identity.md) | `auth-challenge-no-oracle.test.ts`, `v2-assurance-downgrade-audited.test.ts`, `v2-auth-challenge.test.ts`, `v2-authorization-advertisement.test.ts`, `v2-bound-id-kinds.test.ts`, `v2-bound-id-path-projection.test.ts`, `v2-id-grammar.test.ts`, `v2-interrupt-token-scheme.test.ts`, `v2-lane-exp-only-bound.test.ts`, `v2-lane-issuer-advertised.test.ts`, `v2-oidc-id-token-audience.test.ts`, `v2-owner-subject-required.test.ts`, `v2-protected-resource-metadata.test.ts`, `v2-revocation-honored.test.ts`, `v2-subject-link-record.test.ts` |
|
|
33
|
+
| [`interop.md`](../spec/v2/core/interop.md) | `otel-mcp-semconv-projection.test.ts`, `v2-a2a-agent-cards.test.ts`, `v2-a2a-client-error-details.test.ts`, `v2-a2a-operation-map.test.ts`, `v2-a2a-push-delivery.test.ts`, `v2-interop-trace-context.test.ts`, `v2-legacy-profiles-absent.test.ts`, `v2-mcp-mount-map.test.ts`, `v2-mcp-tasks.test.ts`, `v2-minimum-version-refused.test.ts`, `v2-mrtr-rounds-ceiling.test.ts`, `v2-negotiation-authenticated.test.ts`, `v2-negotiation-decided-emitted.test.ts`, `v2-no-transport-advertisement.test.ts`, `v2-refresh-sla.test.ts` |
|
|
34
|
+
| [`interrupt.md`](../spec/v2/core/interrupt.md) | `v2-approval-reject-disposition.test.ts`, `v2-approver-enforced.test.ts`, `v2-callback-url-guarded.test.ts` |
|
|
35
|
+
| [`node-pack-runtimes.md`](../spec/v2/core/node-pack-runtimes.md) | `v2-node-pack-runtimes-advertisement.test.ts`, `v2-wasm-memory-cap.test.ts` |
|
|
36
|
+
| [`oauth.md`](../spec/v2/core/oauth.md) | `v2-credential-interrupt.test.ts`, `v2-credentials-advertisement.test.ts`, `v2-oauth-client-pkce-state-iss.test.ts`, `v2-oauth-mcp-reach-discovery.test.ts` |
|
|
37
|
+
| [`packs.md`](../spec/v2/core/packs.md) | `v2-manifest-ceiling-refused.test.ts`, `v2-manifest-hatch-carried.test.ts`, `v2-peer-dependency-declared.test.ts`, `v2-registry-lifecycle.test.ts`, `v2-ui-plugin-boundary.test.ts`, `v2-ui-plugins-advertisement.test.ts` |
|
|
38
|
+
| [`persistence.md`](../spec/v2/core/persistence.md) | `v2-era-2-append-vocabulary.test.ts`, `v2-era-key.test.ts`, `v2-era-stamp-universal.test.ts`, `v2-pinned-run-disposition.test.ts`, `v2-unmapped-type-refused.test.ts`, `v2-v1-events-translated.test.ts` |
|
|
39
|
+
| [`portability.md`](../spec/v2/core/portability.md) | `v2-portability-advertisement.test.ts` |
|
|
40
|
+
| [`replay.md`](../spec/v2/core/replay.md) | `context-summarization-replay.test.ts`, `memory-attribution-replay-stable.test.ts`, `v2-effect-seam-manifest.test.ts`, `v2-effect-seam-no-refire.test.ts`, `v2-fork-a-v1-run.test.ts`, `v2-nondeterminism-policy-advertisement.test.ts`, `v2-nondeterminism-sources.test.ts`, `v2-replay-suppression-ordinal.test.ts`, `v2-run-fork-prefix.test.ts` |
|
|
41
|
+
| [`runs.md`](../spec/v2/core/runs.md) | `v2-artifact-a2a-shape.test.ts`, `v2-budget-enforcement.test.ts`, `v2-budget-exhaustion-facet.test.ts`, `v2-configurable-closed.test.ts`, `v2-conversation-turn-parts.test.ts`, `v2-created-run-readable.test.ts`, `v2-data-residency-admission.test.ts`, `v2-data-residency-advertisement.test.ts`, `v2-dead-letter-advertisement.test.ts`, `v2-eval-mode-unadvertised-refused.test.ts`, `v2-limits-envelope-caps-advertisement.test.ts`, `v2-run-annotation-not-event.test.ts`, `v2-run-bulk-cancel.test.ts`, `v2-run-cancel.test.ts`, `v2-run-diff-identical.test.ts`, `v2-run-execution-bounds.test.ts`, `v2-run-fork-ancestry.test.ts`, `v2-run-fork-refusals.test.ts`, `v2-run-list.test.ts`, `v2-run-options-limits.test.ts`, `v2-run-pause-resume.test.ts`, `v2-run-snapshot-etag.test.ts` |
|
|
42
|
+
| [`security-defaults.md`](../spec/v2/core/security-defaults.md) | `audit-anomaly-shape.test.ts`, `audit-checkpoint-signature.test.ts`, `audit-checkpoint-vectors.test.ts`, `audit-log-integrity.test.ts`, `inbound-credential-no-passthrough.test.ts`, `v2-compensation-read-projection.test.ts`, `v2-effect-identity-business-key.test.ts`, `v2-pack-isolation.test.ts`, `v2-purpose-propagation-advertisement.test.ts`, `v2-purpose-propagation-onward.test.ts`, `v2-relaxation-recorded.test.ts` |
|
|
43
|
+
| [`storage.md`](../spec/v2/core/storage.md) | `v2-fs-sandbox-escape-refused.test.ts`, `v2-storage-cross-tenant-isolation.test.ts`, `v2-table-schema-enforcement.test.ts` |
|
|
44
|
+
| [`tool-catalog.md`](../spec/v2/core/tool-catalog.md) | `tool-catalog-compact-projection.test.ts`, `tool-catalog-projection.test.ts`, `tool-descriptor-shape.test.ts`, `v2-tool-catalog-annotations.test.ts` |
|
|
45
|
+
| [`versioning.md`](../spec/v2/core/versioning.md) | `v2-advertised-path-space-served.test.ts`, `v2-client-version-header.test.ts`, `v2-dual-stack-negotiation.test.ts`, `v2-malformed-body-envelope.test.ts`, `v2-min-client-version.test.ts`, `v2-preferred-version-default.test.ts`, `v2-version-header-honored.test.ts`, `v2-well-known-one-resource.test.ts` |
|
|
46
|
+
| [`webhooks.md`](../spec/v2/core/webhooks.md) | `v2-trigger-bridge-delivery.test.ts`, `v2-trigger-dead-letter-read.test.ts`, `v2-v1-signed-webhook-accepted.test.ts`, `v2-webhook-delivery-isolation.test.ts`, `v2-webhook-delivery-shape.test.ts`, `v2-webhook-durable-delivery.test.ts`, `v2-webhook-egress-refusal.test.ts`, `v2-webhook-endpoint-verification.test.ts`, `v2-webhook-generated-secret.test.ts`, `v2-webhook-message-id-stable.test.ts`, `v2-webhook-secret-rotation.test.ts`, `v2-webhook-standard-webhooks-delivery.test.ts`, `v2-webhook-unregister-stops-delivery.test.ts` |
|
|
47
|
+
| [`workflow-chain-packs.md`](../spec/v2/core/workflow-chain-packs.md) | `v2-chain-pin-exact.test.ts` |
|
|
48
|
+
|
|
49
|
+
`overview.md`, the introduction to the other documents, has no scenario filed under it.
|
|
50
|
+
|
|
51
|
+
### v2 extensions
|
|
52
|
+
|
|
53
|
+
| Extension | Scenario files |
|
|
54
|
+
| --- | --- |
|
|
55
|
+
| [`ext/README.md`](../spec/v2/ext/README.md) | `v2-ext-family-claims.test.ts` |
|
|
56
|
+
| [`ext/a2uiSurface`](../spec/v2/ext/a2uiSurface/README.md) | `v2-a2ui-v09-surface.test.ts` |
|
|
57
|
+
| [`ext/restTransport`](../spec/v2/ext/restTransport/README.md) | `v2-ext-rest-transport.test.ts` |
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Requirement dispositions and certification
|
|
62
|
+
|
|
63
|
+
Every scenario file records a **disposition** into the requirement ledger (`OPENWOP_LEDGER_PATH`, set by `--certify`):
|
|
64
|
+
|
|
65
|
+
- `executed-pass` — with `assertionCount` > 0.
|
|
66
|
+
- `executed-fail`.
|
|
67
|
+
- `skipped` — operator opt-out via `OPENWOP_OPTED_OUT_PROFILES`.
|
|
68
|
+
- `inapplicable` — the capability, profile or fixture is not advertised (`behaviorGate` or `softSkip('inapplicable', …)`).
|
|
69
|
+
- `blocked` — a seam is absent, a prior step failed, or a precondition was unmet (`seamAbsent` or `softSkip('blocked', …)`).
|
|
70
|
+
|
|
71
|
+
A test that returns before its first `expect` made zero assertions. The runner never records that as a pass: it takes the file's noted reason, or records `blocked` with a fixed marker, which certification treats as an *unclassified return*. Authoring rules: `CONTRIBUTING.md` §Conformance suite ("never return early in silence").
|
|
72
|
+
|
|
73
|
+
How a claim certifies: `claimedProfiles` are derived from the captured discovery document, and every profile has a floor of scenarios. A profile is *certifiable* when every floor row is `executed-pass`, `skipped` or `inapplicable`. A `blocked` or `executed-fail` floor row makes it *not certifiable* (valid evidence, honest state). A floor row with no ledger row, a zero-assertion pass or the runner marker is an *unclassified return* and **rejects** the claim (`--certify` exits 3; the bundle is still written). `--verify` re-derives all of this from the bundle's own rows. The bundle format, signatures and certification rules are in [`spec/v2/core/conformance.md`](../spec/v2/core/conformance.md) §"Bundle v3".
|
|
74
|
+
|
|
75
|
+
## Capability-gated scenarios: shape vs behavior
|
|
10
76
|
|
|
11
|
-
|
|
77
|
+
Many scenarios test an optional capability family or profile. They check two things at different grades:
|
|
78
|
+
|
|
79
|
+
- **Shape** — the host's discovery advertisement is well-formed. These legs usually run always-on or server-free.
|
|
80
|
+
- **Behavior** — the host actually does what it advertises, end to end. These legs run only when the host advertises the capability, and often need a host test seam or fixture.
|
|
81
|
+
|
|
82
|
+
By default a behavior leg the host does not advertise records `inapplicable`, and one the suite cannot reach (seam absent) records `blocked`. In strict mode an **advertised** behavior that cannot be witnessed fails instead:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
OPENWOP_REQUIRE_BEHAVIOR=true npx vitest run # or: openwop-conformance --require-behavior
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
The flag is read at scenario startup (`src/lib/env.ts` → `loadEnv().requireBehavior`). Scenarios gate through `behaviorGate(profileName, advertised)` in `src/lib/behavior-gate.ts`, so the strict-mode failure message cites the relevant spec section. A host that deliberately does not implement a profile lists it in `OPENWOP_OPTED_OUT_PROFILES=name1,name2,...`; those rows record `skipped` even in strict mode, so a minimal host can run strict without falsifying its advertisements.
|
|
89
|
+
|
|
90
|
+
The v1 table of capability-gated scenario groups is in the frozen section below.
|
|
12
91
|
|
|
13
92
|
---
|
|
14
93
|
|
|
15
|
-
##
|
|
94
|
+
## v1 coverage (frozen)
|
|
95
|
+
|
|
96
|
+
v1 reached end of support on 2026-10-04 ([RFC 0234](../RFCS/0234-maintainer-set-v1-end-of-support.md)); [`spec/v1/`](../spec/v1/) is a frozen tree. The v1 scenarios remain in the package for hosts that still measure against it (`--target-major 1`). This section is kept as the v1 record and is not updated for v2 work. Paths in it are v1 paths (`/v1/...`) and spec references are to `spec/v1/`.
|
|
97
|
+
|
|
98
|
+
v1 hosts read multi-agent capability flags such as `conversationPrimitive` at the discovery **root**, not under a nested `capabilities` object. A host that advertises a flag only under `capabilities.` resolves to `false`, and every conversation scenario then skips — an all-skip run that looks like a pass.
|
|
99
|
+
|
|
100
|
+
### Coverage by protocol surface
|
|
16
101
|
|
|
17
102
|
| Surface | Scenario files | Current grade | Remaining gaps |
|
|
18
103
|
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
@@ -73,17 +158,9 @@
|
|
|
73
158
|
| Agent org-chart (RFC 0087 — `capabilities.agents.orgChart`) | `agent-org-chart-shape.test.ts` | A (always-on, server-free shape probe; doubles as the public test for `org-position-no-authority-escalation`) | RFC 0087 promoted Draft → Active 2026-05-30. Always-on shape probe asserts `capabilities.agents.orgChart` (+ `supported`/`installScope`/`departmentNesting`/`responsibilityView` sub-flags); the `AgentOrgChart` record compiles + round-trips and rejects malformed ones (a non-`host:` member `rosterId`; a chart missing `members`). The **§B structural non-authority guarantee**: the schema **rejects** an authority-bearing field on a member (`scopes`/`canDispatch`/`permissions`/`authority` — every object is `additionalProperties:false`), and a conforming member's key set is exactly `{rosterId, departmentId, roleId, reportsTo}`. These are the public test for protocol-tier SECURITY invariant `org-position-no-authority-escalation` (an org edge confers no authority — position describes, never authorizes). NO new RunEventType (the org-chart is structure + a read, not an event surface). **Behavioral scenarios deferred** per RFC 0087 §Conformance (reference host): the live-dispatch refusal of a manager's tool over-reach; an RFC 0049 decision invariant to org position; the cross-tenant `GET /v1/agents/org-chart` 404; the §D responsibility roll-up over live roster portfolios — gate on `capabilities.agents.orgChart.supported` + the org-store seam and soft-skip until a host wires it (the host-extension at `/v1/host/sample/org-chart`, `openwop-app` #371 — formerly `apps/workflow-engine` — is the reference demonstration). Path to `Accepted`: a non-steward host advertises `agents.orgChart` + passes the behavioral non-authority scenario. |
|
|
74
159
|
| Anonymous-actor authorization (RFC 0132 — `capabilities.anonymousActor`) | `anonymous-actor-shape.test.ts` (always-on) + `anonymous-actor-default-deny.test.ts`, `anonymous-actor-no-secret-reach.test.ts`, `anonymous-actor-egress-guarded.test.ts`, `anonymous-actor-write-gated.test.ts`, `anonymous-actor-audit-opaque.test.ts` (gated) | A (always-on server-free shape probe) / behavior host-pass (openwop-app tier-1, non-vacuous) | RFC 0132 promoted Draft → Active 2026-07-22, **Active → Accepted 2026-07-22**. Always-on shape probe asserts the `capabilities.anonymousActor` advert (`supported` const-true + `tiers` non-empty subset of `{read, bounded-write-egress}` + the §B.2 `writeEgressControls`-iff-`bounded-write-egress` `if/then/else` conditional + `failClosed` const-true, closed) validates conforming read + write adverts and rejects the negatives (`tiers:[]`; write-tier-without-control; controls-without-write-tier; `supported:false`; unknown property); `run-snapshot.owner.principalKind` accepts `"anonymous"` + rejects `"guest"`; the anon audit reuses the existing RFC 0049 `authorization.decided` payload (no new event). **Behavioral scenarios deferred** per RFC 0132 §Conformance (reference host): default-deny grant (the RFC 0078 anon tool-catalog fails empty; a non-granted tool → `authorization.decided{allowed:false,reason:"anon-not-granted"}`), no-secret/cross-tenant reach (planted BYOK canary never surfaces; cross-tenant read fails closed), SSRF-guarded audience-bound egress (out-of-audience → `denied`/`downgraded`, no credential attached), gated bounded-write (ungated → denied; hitl → suspends on an approval interrupt before the write), opaque non-PII audit — gate on `capabilities.anonymousActor.supported` + the `/v1/host/sample/anon-surface/{tools,dispatch}` seam and soft-skip until a host wires a tool-enabled public surface. Each backs a SECURITY invariant. **Witnessed 2026-07-22:** the openwop-app tier-1 reference host (PR openwop-app#2403, rev `03d06d1f2`) wired the `/v1/host/sample/anon-surface/{tools,dispatch}` seam (both tiers + a `hitl` control, gated behind `OPENWOP_ANON_ACTOR_ENABLED`) and the five gated scenarios passed **10/10 non-vacuously** under `OPENWOP_REQUIRE_BEHAVIOR=true` (steward-verified by booting the in-memory host; seam served `200`, not a soft-skip) — graduating the five `anon-actor-*` invariants reference-impl → protocol tier and RFC 0132 `Active → Accepted` (RFC 0079 precedent). |
|
|
75
160
|
|
|
76
|
-
|
|
161
|
+
### Capability-gated scenarios (v1)
|
|
77
162
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
Since suite `1.114.0` every scenario file records a **disposition** into the RFC 0148 §A ledger (`OPENWOP_LEDGER_PATH`, set by `--certify`): `executed-pass` (with `assertionCount` > 0), `executed-fail`, `skipped` (operator opt-out via `OPENWOP_OPTED_OUT_PROFILES`), `inapplicable` (capability / profile / fixture not advertised — `behaviorGate` or `softSkip('inapplicable', …)`), or `blocked` (seam absent, prior step failed, precondition unmet — `seamAbsent` / `softSkip('blocked', …)`). A test that returns before its first `expect` is a pass with **zero assertions**; the runner never records that as a pass — it takes the file's noted reason, or resolves it to `blocked` with a fixed marker (`UNCLASSIFIED_RETURN_DETAIL`), which certification still treats as an *unclassified return*. Rules for authors: `CONTRIBUTING.md` §Conformance suite ("never return early in silence").
|
|
81
|
-
|
|
82
|
-
**How a claim certifies (bundle v2, `--certify --bundle-version 2`):** `claimedProfiles` are derived from the captured discovery document (canonical ids; `openwop-core` goes in `aliases`); every catalog profile has a floor in `PROFILE_FLOOR_SCENARIOS` (`discoveryOnly`, plain, `runtimeDerived` for `openwop-node-packs` — claimed only when held — or discovery-`conditional` for `openwop-replay-fork`); a profile is *certifiable* when every floor row is `executed-pass` (witnessed) / `skipped` / `inapplicable`; a floor row that is `blocked` or `executed-fail` makes it *not certifiable* (valid evidence, honest state); a floor row with **no ledger row, a zero-assertion pass, or the runner marker** is an *unclassified return* and **rejects** the claim (`--certify` exits 3, bundle still written). The consumer verifier `verifyBundleV2` re-derives all of this from the bundle's own rows and document (`certification-bundle-non-vacuous.test.ts`), and both emitter and verifier scrub / reject the SR-1 secret canary (`certification-bundle-redaction.test.ts`). Under `OPENWOP_REQUIRE_BEHAVIOR=true` an advertised profile with a missing seam **fails** instead of recording `blocked` (`conformance-advertised-seam-required.test.ts`); an early return can never become a pass at any layer (`conformance-execution-witness.test.ts`). Strict runs need the host's opt-out list (its conformance boot sets it) or every unadvertised gated profile fails — those totals are not findings.
|
|
83
|
-
|
|
84
|
-
## Capability-gated scenarios: shape vs behavior
|
|
85
|
-
|
|
86
|
-
The scenario groups in the table below (one row per group; count the rows — the number is no longer hard-coded here because it drifts) validate optional profiles where the host's discovery advertisement is well-formed (shape grade) but no reference host yet implements the profile end-to-end (behavior grade is `host-pending`). Default suite runs skip these with a warning; set `OPENWOP_REQUIRE_BEHAVIOR=true` to convert skips into hard failures.
|
|
163
|
+
The scenario groups in the table below validate optional profiles where the host's discovery advertisement is well-formed (shape grade) but no reference host yet implements the profile end-to-end (behavior grade is `host-pending`). Default suite runs skip these with a warning; set `OPENWOP_REQUIRE_BEHAVIOR=true` to convert skips into hard failures.
|
|
87
164
|
|
|
88
165
|
| Scenario | Profile / capability | Shape grade | Behavior grade | Behavior-unlock dependency |
|
|
89
166
|
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
@@ -202,24 +279,9 @@ The scenario groups in the table below (one row per group; count the rows — th
|
|
|
202
279
|
| `connection-provider-resolution.test.ts`, `connection-pack-write-reconsent.test.ts` | `capabilities.connections.packsSupported: true` (RFC 0095 §C) | A — always-on schema coverage lives in the three `connection-pack-*` probes (manifest-valid incl. the §C capability declaration; no-credential-material; reach-exclusive) | Gated — §B.6 install/resolve precedence (incl. the SemVer §11 prerelease conflict), §B.8 rejection isolation, §B.4 write-re-consent | A host advertising `connections.packsSupported` + the `POST /v1/host/sample/connection-packs/{install,resolve,consent-plan}` seams (`host-sample-test-seams.md` §10). Soft-skip when unadvertised / seam 404; strict under `OPENWOP_REQUIRE_BEHAVIOR=true`. |
|
|
203
280
|
| `localized-content-delivery.test.ts` | `capabilities.content.supported: true` (RFC 0103; `localized-content.md`) | A — always-on, server-free: the `content` capability block shape, the four content schemas (section/page/settings/page-response) + negatives, the §C `resolveSection` merge (exact/family/base/partial), and the §A coherence predicate (requires i18n; baseLocale==i18n.defaultLocale; ⊆ i18n.supportedLocales; base ∉ supported). Doubles as the public test for the protocol-tier invariants `content-published-cache-no-draft`, `content-response-tenant-scoped`, `content-no-cross-tenant-enumeration`. | Gated — live advertisement coherence + malformed `Accept-Language` → base locale, `Content-Language` reflects, published-only delivery, tenant isolation + no cross-tenant enumeration via `GET /v1/content/pages/{slug}`. **Dual non-vacuous witnesses 2026-06-18 — bar MET:** MyndHyve `workflow-runtime` prod rev `00273-6rf` AND openwop-app reference prod rev `00234-mg8` (`app.openwop.dev`) each ran 18/18 incl. the live §A leg, both steward-curl-verified. | A host advertising `content.supported` (+ `i18n.supported`) serving the `/v1/content/*` surface. Soft-skip when unadvertised / no `OPENWOP_BASE_URL`; strict under `OPENWOP_REQUIRE_BEHAVIOR=true` (`behaviorGate('openwop-content', …)`). Dual-witness bar cleared (MyndHyve + openwop-app, both non-vacuous). |
|
|
204
281
|
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
```bash
|
|
208
|
-
OPENWOP_REQUIRE_BEHAVIOR=true npx vitest run
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
The flag is read at scenario startup via `conformance/src/lib/env.ts` → `loadEnv().requireBehavior`. Scenarios use the `behaviorGate(profileName, advertised)` helper from `conformance/src/lib/behavior-gate.ts` so the strict-mode failure message cites the relevant spec section. The wired-and-passing examples (host-pass behavior grade) include `audit-log-integrity.test.ts` (landed 2026-05-11), the 5 host-capability-surface families gated on the `OPENWOP_TEST_SEAM_ENABLED=true` test seam, and the prompt-_ family of 10 scenarios across the five `prompts-_`profile names (landed 2026-05-20). The remaining`host-pending`rows in the table above adopt the helper as their host-side profiles land (tracked in`docs/PROTOCOL-GAP-CLOSURE-PLAN.md`Phase-1 tracks T1.1 onward). Hosts that deliberately don't implement a profile can list it in`OPENWOP_OPTED_OUT_PROFILES=name1,name2,...` to skip even in strict mode — minimal hosts can claim full strict-mode coverage without falsifying advertisements.
|
|
212
|
-
|
|
213
|
-
---
|
|
282
|
+
### Endpoint coverage manifest (v1 operations)
|
|
214
283
|
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
Every OpenAPI operation should have:
|
|
218
|
-
|
|
219
|
-
1. At least one positive scenario.
|
|
220
|
-
2. At least one auth failure scenario where auth applies.
|
|
221
|
-
3. At least one validation or conflict scenario where the operation accepts input.
|
|
222
|
-
4. A cited spec section in each assertion message.
|
|
284
|
+
Every operation in the v1 OpenAPI document (`api/openapi.yaml`) is listed here; `spec-corpus-validity.test.ts` checks that none is missing. Each operation should have a positive scenario, an auth-failure scenario where auth applies, a validation or conflict scenario where it takes input, and a cited spec section in each assertion message.
|
|
223
285
|
|
|
224
286
|
| Operation ID | Positive coverage | Negative / auth / validation coverage | Gap |
|
|
225
287
|
| ------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
@@ -274,43 +336,10 @@ Every OpenAPI operation should have:
|
|
|
274
336
|
| `putContentSection` | `localized-content-delivery.test.ts` (RFC 0103 §D) — `localized-content-section.schema.json` shape + the locale-targeted write body. | `401`/`403`/`404`/`400`; base-locale key + bad-key-case negatives. | Behavioral locale-targeted upsert pending a host admin seam. |
|
|
275
337
|
| `getContentSettings`, `putContentSettings` | `localized-content-delivery.test.ts` (RFC 0103 §B) — `localized-content-language-settings.schema.json` shape (base ∉ supported). | `401`/`403`; put documents `400` for the base ∈ supported violation. | Behavioral settings round-trip pending a host admin seam. |
|
|
276
338
|
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
## Gap closure plan
|
|
280
|
-
|
|
281
|
-
| Priority | Work item | Target docs |
|
|
282
|
-
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
|
283
|
-
| ✅ done | ~~Add production-profile scenarios for backpressure envelope, retry durability, stale-claim recovery, and debug-bundle truncation.~~ Shipped: `production-backpressure.test.ts`, `production-retention-expiry.test.ts`, `restart-during-run.test.ts`, `staleClaim.test.ts`, `debug-bundle-truncation.test.ts`, `idempotency.test.ts`/`idempotencyRetry.test.ts` (see the `openwop-production` row in §"Capability-gated scenarios"). | `production-profile.md`, `scale-profiles.md`, `storage-adapters.md`, `debug-bundle.md` |
|
|
284
|
-
| ✅ done | ~~Add auth-profile scenarios for API-key rotation and OAuth2 client-credentials where test issuer metadata is available.~~ Shipped under RFC 0010: `auth-api-key-rotation.test.ts`, `auth-oauth2-client-credentials.test.ts`, `auth-oidc-user-bearer.test.ts`, `auth-mtls.test.ts` (capability-gated; see §"Capability-gated scenarios"). | `auth.md`, `auth-profiles.md` |
|
|
285
|
-
| ✅ done | Interrupt-profile scenarios for quorum, external-event, auth-required, and parent/child cascade — landed 2026-05-10. | `interrupt.md`, `interrupt-profiles.md` |
|
|
286
|
-
| P1 | Convert endpoint manifest into generated coverage evidence from `api/openapi.yaml` operation IDs. | `rest-endpoints.md` |
|
|
287
|
-
| ✅ done | MCP and A2A synthetic-peer roundtrip scenarios landed 2026-05-10 (`mcp-tool-roundtrip.test.ts`, `a2a-task-roundtrip.test.ts`); opt-in via `OPENWOP_MCP_FAKE_SERVER=true` / `OPENWOP_A2A_FAKE_PEER=true`. | `mcp-integration.md`, `a2a-integration.md` |
|
|
288
|
-
| ✅ done | ~~Add replay retention and fork-from-arbitrary-event coverage.~~ Shipped: `replay-fork-arbitrary.test.ts` + `replay-retention-expiry.test.ts` (see the "Replay and fork" surface row). | `replay.md` |
|
|
289
|
-
| ~~P1~~ | ~~Deterministic 429-induction harness so `rate-limit-envelope.test.ts` triggers reliably under CI (currently observational).~~ ✅ Closed 2026-05-15 (CF-6): Postgres reference host honors `OPENWOP_FORCE_RATE_LIMIT=true`. | `rest-endpoints.md` |
|
|
290
|
-
| ✅ done | ~~Add tamper-detection scenario for `audit-log-integrity.test.ts` — requires admin write access to the host's audit store.~~ Covered host-internally at `openwop-examples:examples/hosts/sqlite/test/audit-tamper.test.ts` (mutate-entry + forge-signature) + the CF-11 checkpoint-export verifier path (`scripts/verify-audit-checkpoints.mjs`, regression-guarded in `openwop:check` step 7) — see the "Audit-log integrity profile" surface row. | `auth-profiles.md` |
|
|
291
|
-
| ✅ done | ~~Cross-engine append-ordering scenario (multi-engine fixture).~~ Shipped 2026-05-22: `cross-engine-append-ordering.test.ts` + `cross-engine-append-behavior.test.ts` (Lamport-clock monotonicity via the RFC 0036 cross-engine harness seam — see the RFC 0036 surface row). | `channels-and-reducers.md` |
|
|
292
|
-
| ✅ done | ~~End-to-end webhook signed-delivery test exercising `X-openwop-Signature-Algorithm: v1`.~~ Shipped: `webhook-signed-delivery.test.ts` (forward direction) + `webhook-receiver-adversarial.test.ts` (reverse direction, CF-5) — see the "Webhook signature algorithms" surface row. | `webhooks.md` |
|
|
293
|
-
| P2 | Conformance scenarios that cite normative RFC docs (not just schemas) for the multi-agent surfaces. | RFCS/0002–0007 |
|
|
294
|
-
| ✅ Closed 2026-05-17 (HV-1) | `agentPackHandoffSchemaValidation.test.ts` verifies RFC 0003 §D — host MUST validate dispatch payloads against `handoff.taskSchemaRef` AND return payloads against `handoff.returnSchemaRef`. Fixture `conformance-agent-pack-handoff-schema-validation` exercises 3 branches (valid-task / invalid-task / mock-return-violation). Capability-gated on `capabilities.agents.{supported,dispatch}`. | RFCS/0003-agent-packs.md §D |
|
|
295
|
-
| ✅ Closed 2026-05-19 (HVMAP-1/2) | RFC 0022 conformance fully landed across two cycles: 2026-05-18 promoted HVMAP-1a/1b/1c/2 happy paths from `it.todo()` to live behavioral tests against the Postgres reference host (advertising `capabilities.agents.dispatchMapping: true` + `capabilities.subWorkflow.inputMapping: true`); 2026-05-19 promoted all remaining negative-path cases — HVMAP-1a-null, HVMAP-1a-refusal, HVMAP-1b-failed, HVMAP-1b-cancelled, HVMAP-1c-override, HVMAP-2-unset, HVMAP-2-no-midrun-propagation, HVMAP-2-refusal — via the new capability-toggle seam (`conformance/src/lib/host-toggle.ts` + `POST /v1/host/sample/test/capability-toggle`) and 5 new fixtures (`-no-default` variants, `-per-worker-override`, `-deterministic-fail-child`, `-cancellable-child`). Republished as `@openwop/openwop-conformance@1.3.0`. | RFCS/0022-dispatch-input-output-mapping.md §A + §B + §C |
|
|
296
|
-
|
|
297
|
-
---
|
|
298
|
-
|
|
299
|
-
## Scenario inventory backfill (2026-06-11)
|
|
300
|
-
|
|
301
|
-
<!-- NOTE: this inventory should be machine-generated in future (diff
|
|
302
|
-
`ls conformance/src/scenarios/*.test.ts` against the file names cited in
|
|
303
|
-
this document and emit rows from each file's docblock header). It was
|
|
304
|
-
hand-compiled on 2026-06-11 from each file's docblock / behaviorGate /
|
|
305
|
-
capability-gate source; ~90 of the 330 scenario files had no row anywhere
|
|
306
|
-
in this document before this section. -->
|
|
339
|
+
### Scenario inventory (v1)
|
|
307
340
|
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
covers, and the capability (or seam/fixture) that gates it — "always-on" means
|
|
311
|
-
server-free or shape-probe assertions that run unconditionally.
|
|
312
|
-
|
|
313
|
-
### Agent identity, reasoning + orchestrator (Multi-Agent Shift)
|
|
341
|
+
Scenario files not named in the tables above. Columns: the spec doc or RFC the scenario covers, and the capability, seam or fixture that gates it. "Always-on" means server-free or shape assertions that run unconditionally.
|
|
342
|
+
#### Agent identity, reasoning + orchestrator (Multi-Agent Shift)
|
|
314
343
|
|
|
315
344
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
316
345
|
| -------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
|
@@ -323,7 +352,7 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
323
352
|
| `orchestratorConservativePath.test.ts` | RFC 0006 (CP-1 conservative-path suspend) | `capabilities.agents.orchestrator` |
|
|
324
353
|
| `orchestratorTermination.test.ts` | RFC 0006 (CO-3 terminate decision) | `capabilities.agents.orchestrator` |
|
|
325
354
|
|
|
326
|
-
|
|
355
|
+
#### Agent packs (RFC 0003)
|
|
327
356
|
|
|
328
357
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
329
358
|
| ----------------------------- | ------------------------------------------------------------------------------------- | ------------------------------- |
|
|
@@ -332,7 +361,7 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
332
361
|
| `agentPackProvenance.test.ts` | RFC 0003 (`sourceManifestId` provenance round-trip) | `capabilities.agents.supported` |
|
|
333
362
|
| `agentPackCatalog.test.ts` | RFC 0003 (`core.openwop.agents.{deep-research,react,supervisor}` catalog evidence) | `capabilities.agents.supported` |
|
|
334
363
|
|
|
335
|
-
|
|
364
|
+
#### Agent memory: adapter (RFC 0004), compaction (RFC 0012), attribution (RFC 0057), distillation (RFC 0062), consolidation + commitments (RFC 0068)
|
|
336
365
|
|
|
337
366
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
338
367
|
| --------------------------------------------- | ----------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
|
|
@@ -358,7 +387,7 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
358
387
|
| `memory-consolidation-idempotent.test.ts` | RFC 0068 (idempotence + SR-1 carry-forward) | `capabilities.agents.memoryConsolidation.supported` + seam |
|
|
359
388
|
| `commitment-fired.test.ts` | RFC 0068 (fire-once + content-free commitment) | `capabilities.agents.commitments.supported` + seam |
|
|
360
389
|
|
|
361
|
-
|
|
390
|
+
#### Agent loop (RFC 0061)
|
|
362
391
|
|
|
363
392
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
364
393
|
| ---------------------------------------- | -------------------------------------------------------------------- | ---------------------------------------------------------- |
|
|
@@ -367,7 +396,7 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
367
396
|
| `agent-loop-workspace-snapshot.test.ts` | RFC 0061 §C (per-iteration snapshot immutability) | `version >= 5` + `host.workspace.supported` + seam |
|
|
368
397
|
| `agent-loop-stateful-resume.test.ts` | RFC 0061 §D (HITL resume at same iteration) | `executionModel.statefulResume` + seam |
|
|
369
398
|
|
|
370
|
-
|
|
399
|
+
#### RFC 0090 / 0091 / 0092 (verifier, multimodal input, capability requirements)
|
|
371
400
|
|
|
372
401
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
373
402
|
| ---------------------------------------------- | ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
|
@@ -378,7 +407,7 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
378
407
|
| `agent-requires-capabilities-shape.test.ts` | RFC 0092 (`AgentManifest.requiresCapabilities[]` shape) | always-on |
|
|
379
408
|
| `agent-capability-degraded-projection.test.ts` | RFC 0092 §B (degraded projection on `GET /v1/agents`, behavioral) | `capabilities.agents.manifestRuntime` (behaviorGate `openwop-agent-capability-degraded`) |
|
|
380
409
|
|
|
381
|
-
|
|
410
|
+
#### RFC 0105 (speech synthesis adapter)
|
|
382
411
|
|
|
383
412
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
384
413
|
| ------------------------------------------ | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
@@ -388,7 +417,7 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
388
417
|
| `speech-synthesis-roundtrip.test.ts` | RFC 0105 §A (`ctx.callSpeechSynthesizer` audio round-trip, behavioral) | `aiProviders.speechSynthesis === 'supported'` (behaviorGate `openwop-speech-synthesis`) |
|
|
389
418
|
| `speech-synthesis-unadvertised.test.ts` | RFC 0105 §C (unadvertised host rejects with `speech_synthesis_unsupported`) | `aiProviders.speechSynthesis` ABSENT (behaviorGate `openwop-speech-synthesis-unadvertised`) |
|
|
390
419
|
|
|
391
|
-
|
|
420
|
+
#### Conversation primitive (RFC 0005)
|
|
392
421
|
|
|
393
422
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
394
423
|
| ------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------ |
|
|
@@ -397,16 +426,16 @@ server-free or shape-probe assertions that run unconditionally.
|
|
|
397
426
|
| `conversationVsLegacySuspend.test.ts` | RFC 0005 (`conversation.exchanged` ≠ `clarification.requested`) | `capabilities.conversationPrimitive` |
|
|
398
427
|
| `conversationReplayDeterminism.test.ts` | RFC 0005 (replay-fork yields byte-equal conversation log) | `capabilities.conversationPrimitive` + replay-fork + fixture |
|
|
399
428
|
|
|
400
|
-
|
|
429
|
+
#### Multi-party group conversation (RFC 0101)
|
|
401
430
|
|
|
402
431
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
403
432
|
| ---------------------------------------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
|
404
433
|
| `multi-party-conversation-shape.test.ts` | RFC 0101 §Schema (participant roster + conditional `speakerId` + `multiPartyConversation` block) | always-on (server-free) |
|
|
405
434
|
| `multi-party-conversation-behavioral.test.ts` | RFC 0101 §Spec (roster membership + attribution + `maxParticipants`, behavioral) | `capabilities.multiPartyConversation.supported` (behaviorGate `openwop-multi-party-conversation`) + the `POST /v1/host/sample/conversation/multi-party/{open,exchange}` seam |
|
|
406
435
|
|
|
407
|
-
The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative client trigger to open a council) and asserts the cross-field MUSTs JSON Schema cannot express: a roster-valid attributed agent turn is accepted, while a `role:'agent'` turn missing `speakerId`, a non-participant `speakerId`, and an over-`maxParticipants` open are each rejected with `error.code:'validation_error'` (status-tolerant `400`/`422` per RFC 0005 §E). Soft-skips on `404`/`405`. **This is the RFC 0101 → behavioral-conformance bar** (
|
|
436
|
+
The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative client trigger to open a council) and asserts the cross-field MUSTs JSON Schema cannot express: a roster-valid attributed agent turn is accepted, while a `role:'agent'` turn missing `speakerId`, a non-participant `speakerId`, and an over-`maxParticipants` open are each rejected with `error.code:'validation_error'` (status-tolerant `400`/`422` per RFC 0005 §E). Soft-skips on `404`/`405`. **This is the RFC 0101 → behavioral-conformance bar** (implemented by the postgres example host; a product-flow-bound host — e.g. openwop-app ADR 0040's advisory-board council — witnesses instead via its host-side test + an `INTEROP-MATRIX.md` row, the RFC 0086 dual-staging).
|
|
408
437
|
|
|
409
|
-
|
|
438
|
+
#### Dispatch / sub-workflow mapping (RFC 0022) + sub-run attestation (RFC 0063)
|
|
410
439
|
|
|
411
440
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
412
441
|
| --------------------------------------- | -------------------------------------------------------------------------- | ---------------------------------------------- |
|
|
@@ -419,7 +448,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
419
448
|
| `subrun-approval-gate.test.ts` | RFC 0063 §C (suspend-before-merge; accept merges, reject doesn't) | `capabilities.agents.subRunAttestation` + seam |
|
|
420
449
|
| `subrun-approval-fail-closed.test.ts` | RFC 0063 §C + `SECURITY/invariants.yaml#subrun-merge-approval-fail-closed` | `capabilities.agents.subRunAttestation` + seam |
|
|
421
450
|
|
|
422
|
-
|
|
451
|
+
#### Tool hooks (RFC 0064)
|
|
423
452
|
|
|
424
453
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
425
454
|
| ---------------------------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------- |
|
|
@@ -429,7 +458,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
429
458
|
| `tool-hooks-authorization-fail-closed.test.ts` | RFC 0064 §C (per-tool RFC 0049 fail-closed; also listed under `authorization-fail-closed` invariant) | `toolHooks.perToolAuthorization` + seam |
|
|
430
459
|
| `tool-hooks-rate-limit.test.ts` | RFC 0064 §D (per-`(principal, tool)` token bucket → `rate_limited`) | `toolHooks.perToolRateLimit` + seam |
|
|
431
460
|
|
|
432
|
-
|
|
461
|
+
#### Heartbeat (RFC 0060)
|
|
433
462
|
|
|
434
463
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
435
464
|
| --------------------------------------- | ------------------------------------------------------------------ | --------------------------------- |
|
|
@@ -438,7 +467,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
438
467
|
| `heartbeat-runtime-bound.test.ts` | RFC 0060 §B.2 (`maxRuntimeMs` → `{status:"timeout"}`) | `heartbeat.supported` + tick seam |
|
|
439
468
|
| `heartbeat-idempotent-no-spam.test.ts` | RFC 0060 §B.5 (action on transition, not on tick) | `heartbeat.supported` + tick seam |
|
|
440
469
|
|
|
441
|
-
|
|
470
|
+
#### Host storage-surface roundtrips (RFCs 0015–0019, beyond the cross-tenant-isolation rows above)
|
|
442
471
|
|
|
443
472
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
444
473
|
| ----------------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------- |
|
|
@@ -455,7 +484,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
455
484
|
| `blob-presign-expiry.test.ts` | RFC 0019 (presigned-URL expiry) | `capabilities.blobStorage` + seam |
|
|
456
485
|
| `cache-ttl-expiry.test.ts` | RFC 0019 (cache TTL expiry) | `capabilities.cache` + seam |
|
|
457
486
|
|
|
458
|
-
|
|
487
|
+
#### Replay determinism + cross-host causation (RFC 0040 / 0041)
|
|
459
488
|
|
|
460
489
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
461
490
|
| -------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
@@ -463,7 +492,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
463
492
|
| `replay-llm-cache-key-portable.test.ts` | RFC 0041 §E + `SECURITY/invariants.yaml#replay-llm-cache-key-portable` | `multiAgent.executionModel.version >= 4` + `replayDeterminism.llmCacheKeyRecipe: "spec-rfc-0041"` |
|
|
464
493
|
| `cross-host-traceparent-propagation.test.ts` | RFC 0040 §B, carriers named by RFC 0207 §A/§B (trace context into outbound MCP/A2A) | `executionModel.version >= 3` + fixture `conformance-mcp-tool-roundtrip` / `conformance-a2a-task-roundtrip` + the suite fake MCP server / A2A peer (`blocked` without them). **Honesty note (rewritten 2.36.0):** no longer `it.skip` — the two legs run against the suite's own fake peers and fail when NEITHER carrier (`params._meta.traceparent` / `Message.metadata.openwop.traceparent`, or the HTTP `traceparent` header) is present or the trace id is not the suite's; a header-only host passes (logged `header-only`, decisions log D4). Major 1 only and supplementary: the gating home of `0207.mcp-traceparent-carried` / `.a2a-traceparent-carried` is `v2-interop-trace-context.test.ts`. |
|
|
465
494
|
|
|
466
|
-
|
|
495
|
+
#### Miscellaneous protocol surfaces
|
|
467
496
|
|
|
468
497
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
469
498
|
| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
|
|
@@ -480,7 +509,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
480
509
|
| `feedback-capability-shape.test.ts` | RFC 0056 §A (`capabilities.feedback` advertisement) | always-on |
|
|
481
510
|
| `workflow-primary-output-annotation.test.ts` | RFC 0065 (`outputRole` closed enum on `WorkflowNode`) | always-on, server-free |
|
|
482
511
|
|
|
483
|
-
|
|
512
|
+
#### Version fold, text streams, i18n, gRPC and webhook tenancy
|
|
484
513
|
|
|
485
514
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
486
515
|
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- |
|
|
@@ -490,7 +519,7 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
490
519
|
| `grpc-transport.test.ts` | `spec/v1/grpc-transport.md` (gRPC transport profile) | `capabilities.grpc` (schema block added by RFC 0094 on this branch) |
|
|
491
520
|
| `webhook-tenant-isolation.test.ts` | RFC 0093 §A3 (`spec/v1/webhooks.md` delivery tenant scope; backs the protocol-tier `webhook-cross-tenant-isolation` invariant) | `capabilities.webhooks.supported` (two-tenant legs via the `host/sample/test/surface` seam, soft-skip on 404) |
|
|
492
521
|
|
|
493
|
-
|
|
522
|
+
#### Reviewable learning, standing goals and portability (RFCs 0096–0098)
|
|
494
523
|
|
|
495
524
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
496
525
|
| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
|
@@ -498,13 +527,13 @@ The behavioral leg drives the conformance-only seam (RFC 0101 mints no normative
|
|
|
498
527
|
| `goal-standing-continuation.test.ts` | `spec/v1/agent-runtime.md` §"Standing goals" (RFC 0097); backs `goal-continuation-bounded` + `goal-completion-judge-only` | always-on schema legs; behavioral bounds-422 + judge-only legs gated on `capabilities.agents.goals` (soft-skip on 404) |
|
|
499
528
|
| `export-bundle-portability.test.ts` | `spec/v1/portability.md` (RFC 0098); backs `export-bundle-no-credential-material` | always-on schema legs; behavioral literal-credential-import-422 leg gated on `capabilities.portability.import` (soft-skip) |
|
|
500
529
|
|
|
501
|
-
|
|
530
|
+
#### Portable approver routing (RFC 0104)
|
|
502
531
|
|
|
503
532
|
| Scenario file | Spec doc / RFC | Gating capability |
|
|
504
533
|
| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
|
505
534
|
| `interrupt-approver-routing.test.ts` | `spec/v1/interrupt.md` §"Portable approver routing" (RFC 0104); the `interrupt.approverRouting` capability + the `approverGroupRefs` / `approverRoleRefs` / `audience` advisory fields on the approval `InterruptPayload` | always-on schema + `notifyTargets` reference-rule legs; advertisement-coherence leg gated on `capabilities.interrupt.approverRouting.supported` (soft-skip when unadvertised) |
|
|
506
535
|
|
|
507
|
-
|
|
536
|
+
#### Extension opacity is verified DIFFERENTIALLY, and only at install (RFC 0139)
|
|
508
537
|
|
|
509
538
|
`pack-manifest-extension-opacity.test.ts` witnesses RFC 0138's "ignore means ignore" clause against a host. The load-bearing assertion is **not** that an extension-bearing manifest installs — a host that stashes the blob and interprets it later passes that trivially. It is that the host's registration projection is **identical** with and without an unrecognized `vendor.conformance.*` extension.
|
|
510
539
|
|
|
@@ -512,7 +541,7 @@ Measured against stub hosts: a violating stub that routes an unrecognized extens
|
|
|
512
541
|
|
|
513
542
|
**What a green run does NOT prove.** The differential covers **install-time** sinks. A host that stores an extension and interprets it at a later moment the seam never reaches still passes. No finite suite closes that; it is recorded as RFC 0139 gap G3 rather than papered over. The leg also runs on the artifact-type seam only — the one kind with a real host behind it (gap G2) — and does not witness `artifact.created` (RFC 0138 gap G8), because this seam emits no run events.
|
|
514
543
|
|
|
515
|
-
|
|
544
|
+
#### Advertise-and-skip now fails in strict mode (RFC 0139, the G14 flip)
|
|
516
545
|
|
|
517
546
|
`artifact-type-pack-install`, `artifact-type-store-without-render`, and `chat-card-pack-execution` previously used a bare `return` for both the unadvertised-capability and seam-absent cases. They reported green while exercising nothing.
|
|
518
547
|
|
|
@@ -524,11 +553,10 @@ Measured against stub hosts: a violating stub that routes an unrecognized extens
|
|
|
524
553
|
|
|
525
554
|
Advertise-and-skip is the only combination that can lie. Measured: a stub advertising both capabilities and serving no seam gives **9 passed** in default mode and **9 failed** under `OPENWOP_REQUIRE_BEHAVIOR=true`. Before the flip, the strict column read 9 passed. Hosts that deliberately do not implement a capability should advertise honestly or use `OPENWOP_OPTED_OUT_PROFILES`.
|
|
526
555
|
|
|
527
|
-
|
|
556
|
+
#### Open seams — `artifacttypes/runproduce` (RFC 0142)
|
|
528
557
|
|
|
529
558
|
`POST /v1/host/sample/artifacttypes/runproduce` `{artifactTypeId}` → `{runId}` — starts a **REAL run**, through the host's normal execution path, producing one artifact of the given registered type. Host-extension test surface: env-gated, never production, never advertised in discovery. The witness then reads the **standard** `GET /v1/runs/{runId}/events/poll` and asserts `artifact.created` is present with the bound `artifactType` and `registered: true` — the evidence is the event log itself, which is what the `store` MUST is about.
|
|
530
559
|
|
|
531
560
|
**Why this leg gates differently from everything else artifact-type:** the corpus's only emission MUST hangs off the **`store`** facet (`artifact-type-packs.md`: "a host advertising `store: true` MUST do so"), while every other leg gates on `supported`, which carries no emission obligation. So: `store` absent/false at BOTH scopes (the facet is per-type-scoped) ⇒ **inapplicable, plain return in both modes** — `store` is optional and strict mode must not coerce the advertisement; `store: true` + seam absent ⇒ `behaviorGate` (skip default, FAIL strict); `store: true` + seam present ⇒ the witness runs. Measured against stub hosts: an honest stub passes; a stub that persists without emitting reds the presence assertion; a stub emitting the **wrong** `artifactType` also reds — the leg discriminates payload correctness, not mere event presence.
|
|
532
561
|
|
|
533
562
|
**The gap this closes was symmetric.** A `store: true` host emitting nothing was undetectable — and so was a host emitting *correctly*: the reference host emitted `artifact.created` from its documents pack for months and no leg ever credited it. Leg A of the same file pins, always-on and server-free, that `artifactCreated` **requires** `artifactType` — the fact RFC 0138's replay correction and RFC 0141's rewrite prohibition rest on, previously exercised only by a formerly-soft-skipping leg.
|
|
534
|
-
|