@openwop/openwop-conformance 2.25.0 → 2.27.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +3 -3
- package/coverage.md +1 -0
- package/dist/spec-artifacts.lock.json +2 -2
- package/package.json +2 -2
- package/requirements.json +60 -4
- package/scenario-majors.json +5 -2
- package/schemas/CORPUS-STAMP.json +8 -8
- package/src/scenarios/v2-durability-recovery.test.ts +243 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# `@openwop/openwop-conformance` Changelog
|
|
2
2
|
|
|
3
|
+
## [2.27.0] — 2026-09-19 — the four RFC 0158 recovery rows
|
|
4
|
+
|
|
5
|
+
- `v2-durability-recovery.test.ts` — `kill-after-accept` (hold-dispatch), `kill-during-execution` (never observable as completed without re-execution), `duplicate-delivery` (counted per effect identity), `bound-is-derived` (an explicitly-labelled paper check that MUST NOT be cited as evidence the mechanism runs).
|
|
6
|
+
- **Disposition rule:** no `POST /host/durability/kill` route ⇒ `inapplicable` (the host claims no rung); route present but an operator precondition unmet ⇒ `blocked` with the precondition named. Collapsing the two would deny certification to every host the day these rows entered the lane.
|
|
7
|
+
- `peer-resume` is deliberately not written — §E makes it bundle-witnessed, so a scenario could only ever record `blocked`.
|
|
8
|
+
- README scenario counts 522 → 523. Suite pinned to corpus `2.27.0`.
|
|
9
|
+
|
|
10
|
+
## [2.26.0] — 2026-09-19 — RFC 0188 Accepted
|
|
11
|
+
|
|
12
|
+
- No scenario change. Suite pinned to corpus `2.26.0`; the reference-host bundle it certified is checked in at suite 2.25.0.
|
|
13
|
+
|
|
3
14
|
## [2.25.0] — 2026-09-19 — a leg that asserted nothing
|
|
4
15
|
|
|
5
16
|
- `v2-webhook-durable-delivery` — the `0188.dead-letter-content-free` leg read a **fresh** subscription's sink, looped over zero rows, and asserted nothing; RFC 0148 §A resolves that silent return to `blocked`, denying certification. It now exhausts a real delivery first and asserts over the records it gets, and records `blocked` with a reason when the sink is empty.
|
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
# --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
|
|
12
12
|
# default resolver refuses. npm 10.9 fails outright with
|
|
13
13
|
# "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
|
|
14
|
-
npm install --legacy-peer-deps @openwop/openwop-conformance@2.
|
|
14
|
+
npm install --legacy-peer-deps @openwop/openwop-conformance@2.27.0 @openwop/spec-artifacts@2.27.0
|
|
15
15
|
# or run without install:
|
|
16
16
|
npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
|
|
17
17
|
```
|
|
@@ -132,7 +132,7 @@ Exit code is non-zero on any failed assertion. `--certify` distinguishes: `0`
|
|
|
132
132
|
|
|
133
133
|
## What's Covered
|
|
134
134
|
|
|
135
|
-
The current suite has
|
|
135
|
+
The current suite has 523 scenario files under `src/scenarios/`.
|
|
136
136
|
- 2026-09-03 (suite `1.157.0 -> 1.158.0`, gap G17): NEW `idempotency-concurrent-claim.test.ts` — drives the new `host-sample-test-seams.md` §25 concurrent duplicate-delivery seam for the RFC 0150 §B / `idempotency.md` §"Concurrent duplicates (Layer 2)" atomic-claim MUST, which is unconditional and had no witness of any kind. Asserts every executor mints the SAME `logicalInvocationId` **before** asserting `delivered === 1` — without the identity check a host passes by minting different ids and never colliding, one effect because nothing raced. Not profile-gated and so not opt-out-able (the obligation is unconditional); an unmounted seam records `blocked`, which is not certifiable. Graduates `layer2-invocation-claim-atomic` reference-impl -> protocol.
|
|
137
137
|
- 2026-08-19 (suite `1.137.0 → 1.138.0`): NEW `durability-poison-exhaustion.test.ts` — RFC 0158 §C.8, the FIRST row of that RFC's conformance table to land. Asserts what `failure-path.test.ts` cannot: not just that deterministically failing work reaches terminal, but that attempts STOP — counted on the log, re-counted after a scaled quiet window, asserted unchanged. A host still redelivering records more. Seam-gated on the existing event-log seam (`blocked` = unobservable, not unmet) and outside every profile floor.
|
|
138
138
|
- 2026-08-19 (suite `1.136.15 → 1.137.0`): NEW `replay-fanout-suppression.test.ts` — capability-gated on `webhooks.supported`, **outside every profile floor**; witnesses `replay.md` §"Host-initiated fan-out is an external effect", which was the largest normative MUST NOT on the replay surface with no scenario and no SECURITY invariant. Three legs in ONE `it` against ONE receiver and ONE subscription — a positive control, the MUST NOT, and a `branch` boundary leg — because "no delivery arrived" passes identically when delivery never worked, so absence is asserted only after presence is proven on that exact wiring. A host with an SSRF guard correctly refuses the loopback receiver and records `blocked`: **unobservable, not unmet.**
|
|
@@ -477,7 +477,7 @@ Server-required (added in 1.7.0):
|
|
|
477
477
|
| ------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
478
478
|
| **Redaction** | [`capabilities.md`](../spec/v1/capabilities.md) §"Secrets" + NFR-7 + §"aiProviders" | Vendor-neutral assertions that the server doesn't leak secret material. Three scenario groups: (a) discovery shape contract — `secrets` + `aiProviders` advertisements are well-formed regardless of `secrets.supported`; when `supported === true`, scopes MUST be non-empty + `resolution === 'host-managed'`; `byok ⊆ supported`. (b) bearer-token redaction — invalid Bearer canary in `Authorization` header is not echoed in the 401 response body. (c) credentialRef echo control — gated on `secrets.supported === true`; canary planted in `configurable.ai.credentialRef` MUST NOT appear in any RunEvent payload (poll-based capture; transport-agnostic). Uses runtime-built canary fixtures (`lib/canaries.ts`) that defeat static secret scanners. 6 scenarios. |
|
|
479
479
|
|
|
480
|
-
Current source tree:
|
|
480
|
+
Current source tree: 523 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
|
|
481
481
|
|
|
482
482
|
## Remaining Gaps
|
|
483
483
|
|
package/coverage.md
CHANGED
|
@@ -144,6 +144,7 @@ The scenario groups in the table below (one row per group; count the rows — th
|
|
|
144
144
|
| `tool-session-lifecycle.test.ts` | `capabilities.toolCatalog.sessionLifecycle` (RFC 0078 §D, `tool-catalog.md`) | A (the §D bracket via `POST /v1/host/sample/tools/session-run` + the test event-log seam: `tool.session.opened` before the first RFC 0064 call event → `tool.session.closed` after the last, one shared `sessionId`, each carrying a `toolId`, `closed.outcome` ∈ {completed,failed,aborted,expired}, both content-free) | `host-pending` | `behaviorGate('openwop-tool-session-lifecycle', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0078 → Accepted bar.** First adopter: MyndHyve `toolCatalog`. |
|
|
145
145
|
| `egress-audience-binding.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §C, `host-capabilities.md`) + `SECURITY/invariants.yaml` `egress-credential-audience-bound` | A (KEYSTONE — the §C confused-deputy MUST via `POST /v1/host/sample/egress/decide`: an out-of-audience egress is `denied`/`downgraded` with `reason:"out-of-audience"` and the credential is NOT attached (`credentialAttached !== true`); a provenance-unevaluable egress fails closed `denied`+`reason:"provenance-unevaluable"`; decision/reason ∈ the closed enums) | `host-pending` | `behaviorGate('openwop-egress-audience-binding', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0079 → Accepted bar** (the `egress-credential-audience-bound` invariant graduates reference-impl → protocol tier when this passes against a host). First adopter: MyndHyve `httpClient.egressPolicy`. |
|
|
146
146
|
| `egress-decision-content-free.test.ts` | `capabilities.httpClient.egressPolicy.supported` (RFC 0079 §F / SR-1) | A (the secret non-leak — a `canary` credential's sentinel never surfaces in the decision (`canaryLeaked !== true`), the `egress.decided` payload carries no forbidden content key, and `reason` stays in the CLOSED vocabulary so no blocked destination spills into a free-form field) | `host-pending` | `behaviorGate('openwop-egress-decision-content-free', …)`. Seam-gated; soft-skips on 404. **Part of the RFC 0079 → Accepted bar.** First adopter: MyndHyve `httpClient.egressPolicy`. |
|
|
147
|
+
| `v2-durability-recovery.test.ts` | the RFC 0158 host-extension durability seam `POST /host/durability/kill` (non-normative; §E mints no capability field, so seam PRESENCE is how a host says it claims a rung) | A (the four `durable-single-instance` rows: `kill-after-accept` as a HOLD-DISPATCH exercise, `kill-during-execution` asserting that work executing at a real process death is never observable as completed without re-execution, `duplicate-delivery` counted PER EFFECT IDENTITY rather than by end state, and `bound-is-derived` as an explicitly-labelled paper check) | `host-pending` | **Disposition rule, and it protects fleet certification:** no seam route at all ⇒ `inapplicable` (the host claims no rung); seam present but an operator precondition unmet — a restart supervisor, which a black-box suite cannot supply — ⇒ `blocked` with the precondition NAMED. Collapsing the two would deny certification to every host the day these rows entered the lane. `peer-resume` is deliberately NOT written: §E makes it bundle-witnessed, so a scenario could only ever record `blocked`. **This is the RFC 0158 → Accepted bar**, and its witness is openwop-app, which the RFC records as having observed `kill-during-execution` across a real `SIGKILL`. |
|
|
147
148
|
| `memory-degraded-projection.test.ts` | `capabilities.agents.manifestRuntime.supported` + `capabilities.memory.supported` (RFC 0080 §C, `agent-memory.md`) | A (the §C iff-contract on the NORMATIVE `GET /v1/agents`: a degraded entry MUST carry `memoryDegraded:true` + a non-empty, unique `degradedMemoryDimensions[]` drawn from the closed §A-name enum [read/write/search/long-term-durability/compaction/attribution/replay-snapshot/retention]; a non-degraded entry MUST NOT carry a non-empty list; the inventory is non-empty; the degraded branch runs non-vacuously when `OPENWOP_DEGRADED_AGENT_ID` names a known-degraded agent) | `host-pending` | `behaviorGate('openwop-memory-degraded', …)`. Black-box on the normative path (no POST seam); soft-skips on 404 / when the host computes no degradation. **This is the RFC 0080 → Accepted bar.** First adopter: MyndHyve `memory`. |
|
|
148
149
|
| `budget-enforcement.test.ts` | `capabilities.budget.supported` (RFC 0084 §C/§D, `budget-policy.md`) + `SECURITY/invariants.yaml` `budget-no-pricing-leak` | A (the §C/§D enforcement via `POST /v1/host/sample/budget/run` + the test event-log seam: a `hard-cost-exhaust` run emits the strict-ordered `budget.reserved → budget.consumed → budget.threshold.crossed{percent} → budget.exhausted → cap.breached{kind:"budget-cost"} → run.failed{error:"budget_exhausted"}` chain; a `model-denied` run is refused `budget_model_denied` BEFORE the provider call (fail-closed); an `advisory` host emits the `budget.*` events without stopping; every `budget.*` payload content-free — no pricing/rate) | `host-pending` | `behaviorGate('openwop-budget-enforcement', …)`. Seam-gated; soft-skips on 404. **This is the RFC 0084 → Accepted bar.** First adopter: MyndHyve `budget`. |
|
|
149
150
|
| `agent-platform-aggregate-evidence.test.ts` | `openwop-agent-platform` claim — live discovery `profiles[]` includes it (RFC 0085 §C, `agent-platform-profile.md`) | A (the §C/§D honest-advertisement on live `/.well-known/openwop`: a host claiming `openwop-agent-platform` MUST satisfy the §B floor predicate (`isAgentPlatformPartial` → `partial`/`full`, never `none`), the claim backed by per-capability evidence not the profile string; `OPENWOP_AGENT_PLATFORM_TIER=full` forces the full-predicate bar — all governance terms + tenant installScope + all 16 §D terms) | `host-pending` | `behaviorGate('openwop-agent-platform', …)`. Black-box on the discovery doc (no POST seam); soft-skips until a host claims the profile. **This is the RFC 0085 → Accepted bar.** First adopter: MyndHyve (after the memory batch surfaces the floor's `memory.supported`). |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openwop/openwop-conformance",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.27.0",
|
|
4
4
|
"description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -56,6 +56,6 @@
|
|
|
56
56
|
"@openwop/spec-artifacts": "file:../spec-artifacts"
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
|
-
"@openwop/spec-artifacts": "2.
|
|
59
|
+
"@openwop/spec-artifacts": "2.27.0"
|
|
60
60
|
}
|
|
61
61
|
}
|
package/requirements.json
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
"$comment": "GENERATED by conformance/scripts/generate-requirement-registry.mjs — do not edit. One record per it()/test() in src/scenarios. Ids: openwop.it.<file-stem>.<title-slug>[~n] (src/lib/requirement-ids.ts). A record with id null has an interpolated title; its run-time row is keyed by the rendered title and maps here by file+line only. Renamed ids need a row in requirement-aliases.json.",
|
|
3
3
|
"generatedFrom": "src/scenarios/*.test.ts",
|
|
4
4
|
"counts": {
|
|
5
|
-
"files":
|
|
6
|
-
"tests":
|
|
7
|
-
"withStableId":
|
|
5
|
+
"files": 580,
|
|
6
|
+
"tests": 2202,
|
|
7
|
+
"withStableId": 2202,
|
|
8
8
|
"interpolatedTitles": 0,
|
|
9
|
-
"explicitIds":
|
|
9
|
+
"explicitIds": 2156
|
|
10
10
|
},
|
|
11
11
|
"records": [
|
|
12
12
|
{
|
|
@@ -26647,6 +26647,62 @@
|
|
|
26647
26647
|
}
|
|
26648
26648
|
]
|
|
26649
26649
|
},
|
|
26650
|
+
{
|
|
26651
|
+
"id": "openwop.it.v2-durability-recovery.accepted-work-survives-a-kill-before-dispatch-and-dispatches-on-resume",
|
|
26652
|
+
"file": "v2-durability-recovery.test.ts",
|
|
26653
|
+
"line": 105,
|
|
26654
|
+
"title": "accepted work survives a kill before dispatch and dispatches on resume",
|
|
26655
|
+
"explicitId": "openwop.requirement.0158.kill-after-accept",
|
|
26656
|
+
"citations": [
|
|
26657
|
+
{
|
|
26658
|
+
"section": "RFC 0158 §B.4",
|
|
26659
|
+
"requirement": null,
|
|
26660
|
+
"interpolated": true
|
|
26661
|
+
}
|
|
26662
|
+
]
|
|
26663
|
+
},
|
|
26664
|
+
{
|
|
26665
|
+
"id": "openwop.it.v2-durability-recovery.work-executing-at-a-real-process-death-is-never-reported-complete-and-resumes",
|
|
26666
|
+
"file": "v2-durability-recovery.test.ts",
|
|
26667
|
+
"line": 141,
|
|
26668
|
+
"title": "work executing at a real process death is never reported complete, and resumes",
|
|
26669
|
+
"explicitId": "openwop.requirement.0158.kill-during-execution",
|
|
26670
|
+
"citations": [
|
|
26671
|
+
{
|
|
26672
|
+
"section": "RFC 0158 §B.4 / §E item 11",
|
|
26673
|
+
"requirement": null,
|
|
26674
|
+
"interpolated": true
|
|
26675
|
+
}
|
|
26676
|
+
]
|
|
26677
|
+
},
|
|
26678
|
+
{
|
|
26679
|
+
"id": "openwop.it.v2-durability-recovery.the-same-accepted-work-delivered-twice-fires-each-effect-exactly-once",
|
|
26680
|
+
"file": "v2-durability-recovery.test.ts",
|
|
26681
|
+
"line": 175,
|
|
26682
|
+
"title": "the same accepted work delivered twice fires each effect exactly once",
|
|
26683
|
+
"explicitId": "openwop.requirement.0158.duplicate-delivery",
|
|
26684
|
+
"citations": [
|
|
26685
|
+
{
|
|
26686
|
+
"section": "RFC 0158 §C",
|
|
26687
|
+
"requirement": null,
|
|
26688
|
+
"interpolated": true
|
|
26689
|
+
}
|
|
26690
|
+
]
|
|
26691
|
+
},
|
|
26692
|
+
{
|
|
26693
|
+
"id": "openwop.it.v2-durability-recovery.the-declared-recovery-bound-is-derived-from-the-mechanism-that-enforces-it",
|
|
26694
|
+
"file": "v2-durability-recovery.test.ts",
|
|
26695
|
+
"line": 214,
|
|
26696
|
+
"title": "the declared recovery bound is derived from the mechanism that enforces it",
|
|
26697
|
+
"explicitId": "openwop.requirement.0158.bound-is-derived",
|
|
26698
|
+
"citations": [
|
|
26699
|
+
{
|
|
26700
|
+
"section": "RFC 0158 §B.5",
|
|
26701
|
+
"requirement": null,
|
|
26702
|
+
"interpolated": true
|
|
26703
|
+
}
|
|
26704
|
+
]
|
|
26705
|
+
},
|
|
26650
26706
|
{
|
|
26651
26707
|
"id": "openwop.it.v2-effect-identity-business-key.get-runs-runid-effects-validates-and-every-row-is-keyed-on-business-identity-or",
|
|
26652
26708
|
"file": "v2-effect-identity-business-key.test.ts",
|
package/scenario-majors.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$comment": "GENERATED by conformance/scripts/generate-scenario-majors.mjs (RFC 0168 §D.3). Do not edit; add a file to BOTH_MAJORS in the generator to target both majors.",
|
|
3
3
|
"counts": {
|
|
4
|
-
"files":
|
|
4
|
+
"files": 523,
|
|
5
5
|
"v1": 445,
|
|
6
|
-
"v2":
|
|
6
|
+
"v2": 80
|
|
7
7
|
},
|
|
8
8
|
"majors": {
|
|
9
9
|
"a2a-1-0-agent-card.test.ts": [
|
|
@@ -1277,6 +1277,9 @@
|
|
|
1277
1277
|
"v2-dual-stack-negotiation.test.ts": [
|
|
1278
1278
|
2
|
|
1279
1279
|
],
|
|
1280
|
+
"v2-durability-recovery.test.ts": [
|
|
1281
|
+
2
|
|
1282
|
+
],
|
|
1280
1283
|
"v2-effect-identity-business-key.test.ts": [
|
|
1281
1284
|
2
|
|
1282
1285
|
],
|
|
@@ -1,17 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_comment": "Provenance of @openwop/spec-artifacts (RFC 0168 §D.2). files: SHA-256 per file; the conformance suite compares the installed peer against dist/spec-artifacts.lock.json at start.",
|
|
3
3
|
"package": "@openwop/spec-artifacts",
|
|
4
|
-
"version": "2.
|
|
5
|
-
"corpusTag": "v2.
|
|
4
|
+
"version": "2.27.0",
|
|
5
|
+
"corpusTag": "v2.27.0",
|
|
6
6
|
"files": {
|
|
7
7
|
"api/.redocly.lint-ignore.yaml": "bf5a8350b88a72fa43f59605ed8d903ed24b6cfccda5e45509c9f6ed9ee4e712",
|
|
8
8
|
"api/asyncapi.yaml": "d5ecb9ee6114582be3b1f662c84bfac9ae96dae7bacb853e461168f70a8e1c7d",
|
|
9
9
|
"api/grpc/openwop.proto": "c3e72bb17cba514ee98feb6434e6c9b6ea6795bfd086489ec69fd882dd1ad977",
|
|
10
10
|
"api/openapi.yaml": "39081c59fb696159806b0f2f9a42e7e9ff830d622fcf2ad4159b21357580a955",
|
|
11
11
|
"api/redocly.yaml": "b0604c89b2ca6d5076ec25725c539dad44a741a811fe524439ee6daef8baa09f",
|
|
12
|
-
"api/seams-v2.yaml": "
|
|
13
|
-
"api/v2/asyncapi.yaml": "
|
|
14
|
-
"api/v2/openapi.yaml": "
|
|
12
|
+
"api/seams-v2.yaml": "1dce94833d8e1c8701b45f41280b592ca65c2eb989933635bdf2bda4c5ec92e1",
|
|
13
|
+
"api/v2/asyncapi.yaml": "e662cd1be3c79e3ca89ff24f0a021c1ce8e9e77d239c3a1f931edde241914525",
|
|
14
|
+
"api/v2/openapi.yaml": "04cba9484da8fb7a7708d8c927d273daa4775cf9ae9d7dadeb92f9e7a19a174f",
|
|
15
15
|
"api/v2/redocly.yaml": "1e66b60e6118ad11a823bb620678be464d99dfe50a40e3e6f93ec9429b88b34c",
|
|
16
16
|
"schemas/README.md": "0c0b737ffcf8f30e7d2809cec8a498232de710f41443212922ad8337cdde0b51",
|
|
17
17
|
"schemas/a2a-task-state.schema.json": "c9365918f993f943b4b619d42551eb066a1ed33a08d895d51b395432a5b1f1bc",
|
|
@@ -201,7 +201,7 @@
|
|
|
201
201
|
"schemas/workspace-file.schema.json": "464de85c2a068243084ee9c1d969bc7cd5d8f7948574e58450d6493c38a0e1e4",
|
|
202
202
|
"spec/v1/alias-detectors.json": "40069d5976eeb6ba1384a648e57e5cfd673db3fce36175115fb8293bee9664d4",
|
|
203
203
|
"spec/v1/capability-declaration-classes.json": "e7729aed5c4b4e1dd02abab0530f14cc95f5d4070fe51fb139e7f5cccefa00c6",
|
|
204
|
-
"spec/v1/core-standard-manifest.json": "
|
|
204
|
+
"spec/v1/core-standard-manifest.json": "7a1f2b95997a643482a2223033ad47de828f91b0c1c38f57fe132d80d87dd651",
|
|
205
205
|
"spec/v1/deprecations.json": "307083ce29c23fd406015951f99a30d78d6187ff061d38dc9732f62191b40f3f",
|
|
206
206
|
"spec/v1/deprecations.schema.json": "18c87e78bedc210431f795ae44c5b5d202f2f3317850d5cf86867d4f1fa1cdfb",
|
|
207
207
|
"spec/v1/event-codemap.json": "3da60d884157793a360da532a9fcbbfb5285636db325a74cec94b34622186d97",
|
|
@@ -279,8 +279,8 @@
|
|
|
279
279
|
"spec/v2/path-manifest.json": "c123c9fd77dc1b3f9c2e9346ace80cdae007e6511138894956abda01e61ec1c6",
|
|
280
280
|
"spec/v2/peer-dependency-aliases.json": "d10299280abee08258502925bc327293ee413e0108cd6e6ec75ff6110653308d",
|
|
281
281
|
"spec/v2/profiles.json": "0636f19fceae625390003a347e70ef4797d84766b5c24ce8a02cea52aadebca4",
|
|
282
|
-
"spec/v2/release.json": "
|
|
282
|
+
"spec/v2/release.json": "b1577166696dbdb289d57b52b15c31e62f46848ded4d92b3bf2076920bdcb185",
|
|
283
283
|
"spec/v2/retention-floors.json": "eaf3722d95c79947af1d4269ef85117e126518c588cfcf1a2b21b97269f51624"
|
|
284
284
|
},
|
|
285
|
-
"corpusCommit": "
|
|
285
|
+
"corpusCommit": "7032550dc695e608f809cc097196952e80764abe"
|
|
286
286
|
}
|
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* RFC 0158 — the `durable-single-instance` recovery rows.
|
|
3
|
+
*
|
|
4
|
+
* These are the four rows the RFC names that no scenario has ever existed for.
|
|
5
|
+
* `durability/poison-exhaustion` is the fifth and is written
|
|
6
|
+
* (`durability-poison-exhaustion.test.ts`) — but it is registered at MAJOR 1
|
|
7
|
+
* ONLY, so no v2 bundle has ever carried a durability row of any kind. Porting
|
|
8
|
+
* it to major 2 needs the v2 event-log read and is tracked separately; without
|
|
9
|
+
* it, the rung is still unwitnessable at major 2 even when these four pass.
|
|
10
|
+
*
|
|
11
|
+
* ── The disposition ruling, which is the load-bearing design decision ────────
|
|
12
|
+
* §E says an unmet OPERATOR PRECONDITION is `blocked` with the precondition
|
|
13
|
+
* named, never a silent skip or a pass. §E also says the seam is "a
|
|
14
|
+
* non-normative host-extension route … it advertises nothing … a host that
|
|
15
|
+
* never runs the durability exercises exposes no such route. It is test
|
|
16
|
+
* infrastructure, not protocol surface."
|
|
17
|
+
*
|
|
18
|
+
* Those two sentences describe DIFFERENT states, and collapsing them would
|
|
19
|
+
* strip certification from the entire fleet the day these rows enter the
|
|
20
|
+
* major-2 lane, because a `blocked` row denies certification (RFC 0168 §E.1):
|
|
21
|
+
*
|
|
22
|
+
* - NO SEAM ROUTE AT ALL → `inapplicable`. The host exposes no durability
|
|
23
|
+
* seam, which under §E.10 (no capability field is minted) is exactly how a
|
|
24
|
+
* host says it claims no durable-execution rung. Blocking a host for a
|
|
25
|
+
* claim it never made is the false-refusal half of the same error as a
|
|
26
|
+
* vacuous pass.
|
|
27
|
+
* - SEAM PRESENT, PRECONDITION UNMET → `blocked`, precondition NAMED. The
|
|
28
|
+
* host offered the exercise and could not complete it. That is §E's case.
|
|
29
|
+
*
|
|
30
|
+
* ── Why a real kill, and why the interval is kill → RESUMPTION ───────────────
|
|
31
|
+
* §D.9 rejects "claim semantics asserted without a process death", and §B.4
|
|
32
|
+
* bounds the interval until another instance becomes eligible to RESUME the
|
|
33
|
+
* work — not until the work finishes. Timing kill → terminal adds the work's
|
|
34
|
+
* own execution time and makes a conformant host report a figure over its own
|
|
35
|
+
* derived bound: a §B.5 violation that did not happen. Both measured failures
|
|
36
|
+
* are recorded in the RFC.
|
|
37
|
+
*
|
|
38
|
+
* ── What this file deliberately does not do ─────────────────────────────────
|
|
39
|
+
* `durability/peer-resume` is the `durable-multi-instance` discriminator and is
|
|
40
|
+
* NOT required for the rung these rows witness. §E makes it bundle-witnessed
|
|
41
|
+
* via an opaque per-boot incarnation token, not black-box observable, so a
|
|
42
|
+
* scenario here could only ever record `blocked` on the ≥2-instances
|
|
43
|
+
* precondition. It is omitted rather than written as a row that can never pass
|
|
44
|
+
* — the defect this session found in `0188.dead-letter-content-free`.
|
|
45
|
+
*
|
|
46
|
+
* @see RFCS/0158-durable-execution-and-disaster-recovery-qualification.md §B.4 §D.9 §E
|
|
47
|
+
*/
|
|
48
|
+
|
|
49
|
+
import { describe, expect, it } from 'vitest';
|
|
50
|
+
import { driver } from '../lib/driver.js';
|
|
51
|
+
import { v2Discovery } from '../lib/v2.js';
|
|
52
|
+
import { isFixtureAdvertised } from '../lib/fixtures.js';
|
|
53
|
+
import { softSkip } from '../lib/soft-skip.js';
|
|
54
|
+
import { scaledTimeoutMs } from '../lib/polling.js';
|
|
55
|
+
import { req } from '../lib/requirement-ids.js';
|
|
56
|
+
|
|
57
|
+
const FIXTURE = 'conformance-noop';
|
|
58
|
+
|
|
59
|
+
/** The host-extension seam these rows drive. Non-normative; advertises nothing. */
|
|
60
|
+
const KILL_SEAM = '/host/durability/kill';
|
|
61
|
+
|
|
62
|
+
/** How long to wait for the service to answer again after a real termination. */
|
|
63
|
+
const RESUME_WINDOW_MS = 30_000;
|
|
64
|
+
|
|
65
|
+
type SeamState =
|
|
66
|
+
| { kind: 'absent'; why: string }
|
|
67
|
+
| { kind: 'present' };
|
|
68
|
+
|
|
69
|
+
/**
|
|
70
|
+
* Probe the seam WITHOUT firing it. A `404`/`405` means the route is not
|
|
71
|
+
* mounted; anything else means the host offered it and the exercise is on.
|
|
72
|
+
* A GET is used on purpose: firing the kill to discover whether it exists
|
|
73
|
+
* would terminate a host that never claimed the rung.
|
|
74
|
+
*/
|
|
75
|
+
async function probeSeam(): Promise<SeamState> {
|
|
76
|
+
const r = await driver.get(KILL_SEAM);
|
|
77
|
+
if (r.status === 404 || r.status === 405) {
|
|
78
|
+
return { kind: 'absent', why: `no RFC 0158 durability seam at ${KILL_SEAM} (HTTP ${r.status}) — §E: the seam advertises nothing and a host that never runs the durability exercises exposes no such route, so this host claims no durable-execution rung` };
|
|
79
|
+
}
|
|
80
|
+
return { kind: 'present' };
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
/** Wait for the service to answer discovery again; null when it never does. */
|
|
84
|
+
async function waitBack(deadlineMs: number): Promise<number | null> {
|
|
85
|
+
const started = Date.now();
|
|
86
|
+
while (Date.now() - started < deadlineMs) {
|
|
87
|
+
try {
|
|
88
|
+
const r = await driver.get('/.well-known/openwop');
|
|
89
|
+
if (r.status === 200) return Date.now() - started;
|
|
90
|
+
} catch { /* the process is down; that is the point */ }
|
|
91
|
+
await new Promise((r) => setTimeout(r, 250));
|
|
92
|
+
}
|
|
93
|
+
return null;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
/** Count the run-lifecycle re-starts on a run: §B.4's resumption observation. */
|
|
97
|
+
async function runStartedCount(runId: string): Promise<number> {
|
|
98
|
+
const r = await driver.get(`/runs/${encodeURIComponent(runId)}/events`);
|
|
99
|
+
if (r.status !== 200) return -1;
|
|
100
|
+
const events = (r.json as { events?: Array<{ type?: string }> } | null)?.events ?? [];
|
|
101
|
+
return events.filter((e) => e.type === 'run.started').length;
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
describe('v2-durability-recovery (RFC 0158 §B.4, §D.9, §E — the durable-single-instance rows)', () => {
|
|
105
|
+
it('accepted work survives a kill before dispatch and dispatches on resume', async () => {
|
|
106
|
+
const doc = await v2Discovery();
|
|
107
|
+
if (!doc) return softSkip('blocked', 'v2 discovery unreachable');
|
|
108
|
+
if (!isFixtureAdvertised(FIXTURE)) return softSkip('inapplicable', `${FIXTURE} fixture not advertised — no work to accept`);
|
|
109
|
+
const seam = await probeSeam();
|
|
110
|
+
if (seam.kind === 'absent') return softSkip('inapplicable', seam.why);
|
|
111
|
+
|
|
112
|
+
// §E item 11: kill-after-accept is a HOLD-DISPATCH row, not a
|
|
113
|
+
// termination-timing row. On a host where acceptance and dispatch are
|
|
114
|
+
// microseconds apart, racing a kill into that window cannot reliably hit
|
|
115
|
+
// it — so the seam holds dispatch, the kill lands during the hold, and the
|
|
116
|
+
// accepted-but-undispatched work dispatches on resume.
|
|
117
|
+
const fired = await driver.post(KILL_SEAM, { mode: 'after-accept', workflowId: FIXTURE });
|
|
118
|
+
if (fired.status >= 400) {
|
|
119
|
+
return softSkip('blocked', `the durability seam answered ${fired.status} for mode=after-accept — the host exposes the route but could not stage the exercise`);
|
|
120
|
+
}
|
|
121
|
+
const runId = (fired.json as { runId?: unknown } | null)?.runId;
|
|
122
|
+
if (typeof runId !== 'string') {
|
|
123
|
+
return softSkip('blocked', 'the seam staged a kill but named no runId, so there is no accepted work to follow across the death');
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
const backIn = await waitBack(scaledTimeoutMs(RESUME_WINDOW_MS));
|
|
127
|
+
if (backIn === null) {
|
|
128
|
+
// §E: kill rows need a RESTART SUPERVISOR — a black-box suite cannot
|
|
129
|
+
// restart a killed single instance. Name the precondition rather than
|
|
130
|
+
// reporting a host defect that is really a harness gap.
|
|
131
|
+
return softSkip('blocked', 'the service never answered again within the window — the operator precondition for this row is a restart supervisor (something must restart the killed instance; the suite cannot)');
|
|
132
|
+
}
|
|
133
|
+
|
|
134
|
+
const starts = await runStartedCount(runId);
|
|
135
|
+
expect(
|
|
136
|
+
starts >= 1,
|
|
137
|
+
req('openwop.requirement.0158.kill-after-accept', 'RFC 0158 §B.4', `work accepted before a real process death MUST dispatch on resume — the run's log MUST show it being executed after the kill, observed ${starts} run.started (service answered again after ${backIn}ms)`),
|
|
138
|
+
).toBe(true);
|
|
139
|
+
}, 120_000);
|
|
140
|
+
|
|
141
|
+
it('work executing at a real process death is never reported complete, and resumes', async () => {
|
|
142
|
+
const doc = await v2Discovery();
|
|
143
|
+
if (!doc) return softSkip('blocked', 'v2 discovery unreachable');
|
|
144
|
+
if (!isFixtureAdvertised(FIXTURE)) return softSkip('inapplicable', `${FIXTURE} fixture not advertised — no work to execute`);
|
|
145
|
+
const seam = await probeSeam();
|
|
146
|
+
if (seam.kind === 'absent') return softSkip('inapplicable', seam.why);
|
|
147
|
+
|
|
148
|
+
const fired = await driver.post(KILL_SEAM, { mode: 'during-execution', workflowId: FIXTURE });
|
|
149
|
+
if (fired.status >= 400) {
|
|
150
|
+
return softSkip('blocked', `the durability seam answered ${fired.status} for mode=during-execution — the host exposes the route but could not stage the exercise`);
|
|
151
|
+
}
|
|
152
|
+
const runId = (fired.json as { runId?: unknown } | null)?.runId;
|
|
153
|
+
if (typeof runId !== 'string') {
|
|
154
|
+
return softSkip('blocked', 'the seam staged a kill but named no runId, so the in-flight work cannot be followed across the death');
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
const backIn = await waitBack(scaledTimeoutMs(RESUME_WINDOW_MS));
|
|
158
|
+
if (backIn === null) {
|
|
159
|
+
return softSkip('blocked', 'the service never answered again within the window — the operator precondition for this row is a restart supervisor (something must restart the killed instance; the suite cannot)');
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
// §E item 11's first clause, and the one a host is most likely to get
|
|
163
|
+
// wrong: work that was executing when the process died MUST NOT be
|
|
164
|
+
// observable as completed. A host that marks it complete on restart has
|
|
165
|
+
// reported success for work it never finished.
|
|
166
|
+
const snap = await driver.get(`/runs/${encodeURIComponent(runId)}`);
|
|
167
|
+
const status = (snap.json as { status?: unknown } | null)?.status;
|
|
168
|
+
const starts = await runStartedCount(runId);
|
|
169
|
+
expect(
|
|
170
|
+
status !== 'completed' || starts > 1,
|
|
171
|
+
req('openwop.requirement.0158.kill-during-execution', 'RFC 0158 §B.4 / §E item 11', `work executing at a real process death MUST NOT be observable as completed without having been re-executed — read status ${String(status)} with ${starts} run.started (service answered again after ${backIn}ms); §B.4 measures kill → RESUMPTION, never kill → terminal`),
|
|
172
|
+
).toBe(true);
|
|
173
|
+
}, 120_000);
|
|
174
|
+
|
|
175
|
+
it('the same accepted work delivered twice fires each effect exactly once', async () => {
|
|
176
|
+
const doc = await v2Discovery();
|
|
177
|
+
if (!doc) return softSkip('blocked', 'v2 discovery unreachable');
|
|
178
|
+
const seam = await probeSeam();
|
|
179
|
+
if (seam.kind === 'absent') return softSkip('inapplicable', seam.why);
|
|
180
|
+
|
|
181
|
+
const fired = await driver.post(KILL_SEAM, { mode: 'duplicate-delivery', workflowId: FIXTURE });
|
|
182
|
+
if (fired.status >= 400) {
|
|
183
|
+
return softSkip('blocked', `the durability seam answered ${fired.status} for mode=duplicate-delivery — the host exposes the route but could not stage a double delivery`);
|
|
184
|
+
}
|
|
185
|
+
const runId = (fired.json as { runId?: unknown } | null)?.runId;
|
|
186
|
+
if (typeof runId !== 'string') {
|
|
187
|
+
return softSkip('blocked', 'the seam staged a duplicate delivery but named no runId');
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
const eff = await driver.get(`/runs/${encodeURIComponent(runId)}/effects`);
|
|
191
|
+
if (eff.status !== 200) {
|
|
192
|
+
return softSkip('blocked', `GET /runs/{runId}/effects answered ${eff.status} — per-identity invocation counts are unobservable, so the assertion would be vacuous`);
|
|
193
|
+
}
|
|
194
|
+
const effects = (eff.json as { effects?: Array<Record<string, unknown>> } | null)?.effects ?? [];
|
|
195
|
+
if (effects.length === 0) {
|
|
196
|
+
return softSkip('blocked', 'the staged run recorded no effects — there is no identity to count invocations against, and an end-state assertion is exactly what §C rules out');
|
|
197
|
+
}
|
|
198
|
+
// §C: assert INVOCATION COUNTS PER IDENTITY, not final state. A legal end
|
|
199
|
+
// state is precisely what a double-fire produces, so an end-state
|
|
200
|
+
// assertion passes on the defect it exists to catch.
|
|
201
|
+
const byIdentity = new Map<string, number>();
|
|
202
|
+
for (const e of effects) {
|
|
203
|
+
const id = String(e['effectId'] ?? e['keying'] ?? '');
|
|
204
|
+
if (id === '') continue;
|
|
205
|
+
byIdentity.set(id, (byIdentity.get(id) ?? 0) + 1);
|
|
206
|
+
}
|
|
207
|
+
const doubled = [...byIdentity.entries()].filter(([, n]) => n > 1);
|
|
208
|
+
expect(
|
|
209
|
+
doubled.length === 0,
|
|
210
|
+
req('openwop.requirement.0158.duplicate-delivery', 'RFC 0158 §C', `the same accepted work delivered twice MUST fire each effect exactly once, asserted per effect identity — ${byIdentity.size} identity/identities recorded, ${doubled.length} fired more than once${doubled.length ? ` (${doubled.map(([k, n]) => `${k}×${n}`).join(', ')})` : ''}`),
|
|
211
|
+
).toBe(true);
|
|
212
|
+
}, 120_000);
|
|
213
|
+
|
|
214
|
+
it('the declared recovery bound is derived from the mechanism that enforces it', async () => {
|
|
215
|
+
const doc = await v2Discovery();
|
|
216
|
+
if (!doc) return softSkip('blocked', 'v2 discovery unreachable');
|
|
217
|
+
const seam = await probeSeam();
|
|
218
|
+
if (seam.kind === 'absent') return softSkip('inapplicable', seam.why);
|
|
219
|
+
|
|
220
|
+
const terms = await driver.get('/host/durability/bound');
|
|
221
|
+
if (terms.status === 404 || terms.status === 405) {
|
|
222
|
+
return softSkip('blocked', 'the host exposes the durability seam but serves no recovery-bound terms — §E puts the per-class arithmetic in the RFC 0148 evidence bundle where a reader can recompute it, and there is nothing here to recompute');
|
|
223
|
+
}
|
|
224
|
+
const body = (terms.json as { bound?: unknown; terms?: unknown } | null) ?? {};
|
|
225
|
+
const bound = typeof body.bound === 'number' ? body.bound : null;
|
|
226
|
+
const parts = Array.isArray(body.terms) ? (body.terms as unknown[]) : null;
|
|
227
|
+
if (bound === null || parts === null || parts.length === 0) {
|
|
228
|
+
return softSkip('blocked', 'the recovery-bound response names no { bound, terms[] } — §E asks for the per-class arithmetic, not a single total (Unresolved Question 1), and a total alone cannot be recomputed');
|
|
229
|
+
}
|
|
230
|
+
const summed = parts.reduce<number>((acc, t) => acc + (typeof (t as { ms?: unknown }).ms === 'number' ? (t as { ms: number }).ms : Number.NaN), 0);
|
|
231
|
+
// THIS ROW IS A PAPER CHECK BY CONSTRUCTION AND THE RFC SAYS SO. It checks
|
|
232
|
+
// that the declared number follows from the stated mechanism. It cannot
|
|
233
|
+
// check that the mechanism RUNS: a host whose sweeper wedges has a
|
|
234
|
+
// derivation that stays perfectly correct while the bound is not produced
|
|
235
|
+
// at all — a run sat unclaimed for 16 minutes against a derived bound of
|
|
236
|
+
// 12.5 with every isolated check of the mechanism passing. Only the kill
|
|
237
|
+
// rows above witness liveness, and this row MUST NOT be cited for it.
|
|
238
|
+
expect(
|
|
239
|
+
Number.isFinite(summed) && Math.abs(summed - bound) <= 1,
|
|
240
|
+
req('openwop.requirement.0158.bound-is-derived', 'RFC 0158 §B.5', `the declared recovery bound MUST follow from the per-class terms that produce it — declared ${bound}, terms sum to ${summed}. A host that states a bound it cannot produce fails; a host whose sweeper wedges still passes, which is why this row MUST NOT be read as evidence that the mechanism runs`),
|
|
241
|
+
).toBe(true);
|
|
242
|
+
}, 120_000);
|
|
243
|
+
});
|