@openwop/openwop-conformance 2.45.5 → 2.45.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -0
- package/README.md +2 -2
- package/dist/spec-artifacts.lock.json +2 -2
- package/package.json +2 -2
- package/requirements.json +34 -6
- package/scenario-majors.json +5 -2
- package/schemas/CORPUS-STAMP.json +14 -14
- package/src/lib/exhaustion-facet-witness.ts +104 -0
- package/src/lib/scratch-host.ts +4 -0
- package/src/scenarios/v2-budget-exhaustion-facet.test.ts +72 -0
- package/src/scenarios/v2-workspace-scope-from-identity.test.ts +10 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,27 @@
|
|
|
1
1
|
# `@openwop/openwop-conformance` Changelog
|
|
2
2
|
|
|
3
|
+
## [2.45.7] — 2026-10-02 — a strict-mode host that does not advertise `workspace` is not failed; RFC 0231 Accepted
|
|
4
|
+
|
|
5
|
+
- **Fix: `v2-workspace-scope-from-identity` no longer fails a strict-mode host that does not advertise `workspace`.** The file gated with `gateFamily('workspace')`, which under `--require-behavior` fails an unadvertised family unless the operator opts it out. Its own header says an unadvertised family is `inapplicable`, and its sibling isolation scenarios (`v2-storage-`, `v2-queue-cross-tenant-isolation`) read with `familyAdvertised` and recorded `inapplicable` in the same run. It now reads with `familyAdvertised`.
|
|
6
|
+
- **Found by** MyndHyve's 2.45.5 cut: both `0059.workspace-scope-*` rows `executed-fail` on a host that does not serve `workspace`.
|
|
7
|
+
- **Proof.** Against a scratch host that does not advertise the family, in strict mode: both legs failed before the change; the three rows are `inapplicable` after it. A host that advertises `workspace` is unchanged (the v2 reference host passes both rows on its certified 2.45.6 cut).
|
|
8
|
+
- **Workaround on 2.45.3–2.45.6:** `OPENWOP_OPTED_OUT_PROFILES=family.workspace`, which records the rows `skipped`.
|
|
9
|
+
- **The 2.45.7 cycle opens.** RFC 0231's gap register changes the packed `spec/v1/gaps.json`: G3 and G4 close, and G2 and G7 are `externally-gated`. No scenario changes.
|
|
10
|
+
|
|
11
|
+
## [2.45.6] — 2026-10-02 — RFC 0231 (Active): a host says which budget exhaustion behaviours it serves
|
|
12
|
+
|
|
13
|
+
- **The 2.45.6 cycle opens.** RFC 0231's gap register changes the packed `spec/v1/gaps.json`.
|
|
14
|
+
- **New scenario `v2-budget-exhaustion-facet`** (major 2, unaided; RFC 0231). Logic in `lib/exhaustion-facet-witness.ts`.
|
|
15
|
+
- `openwop.requirement.runs.budget-exhaustion-facet-contains-fail`: an advertised `budget.onExhaustion` is a non-empty list of `fail` and `interrupt` that contains `fail`.
|
|
16
|
+
- `openwop.requirement.runs.budget-unserved-exhaustion-refused`: where the list leaves out `interrupt`, a create asking for it answers `422 capability_not_provided`. The suite sends one create and cancels any run a non-conforming host starts.
|
|
17
|
+
|
|
18
|
+
**Dispositions.** No `budget`, no list, or `enforce` not `hard`: `inapplicable`. A list that includes `interrupt`: the refusal row is `inapplicable`, and the served path has no witness (RFC 0231 gap G2). No advertised fixture: the refusal row is `inapplicable`.
|
|
19
|
+
|
|
20
|
+
**Sabotage.** `lib/exhaustion-facet-witness.test.ts`, nine cases against the scratch host. The refusal row fails on a host that accepts the run, answers `400 validation_error`, or answers `422` with another code. The first row fails on a list without `fail`, an empty list and a bare string.
|
|
21
|
+
|
|
22
|
+
**Host impact: none today.** No host advertises `budget.onExhaustion`, so both rows record `inapplicable` everywhere.
|
|
23
|
+
- **`lib/scratch-host.ts` can refuse a create by its body** (`createRefusal`). Unset, every create is admitted as before.
|
|
24
|
+
|
|
3
25
|
## [2.45.5] — 2026-10-01 — a prefix floor row is written only for a claimed profile
|
|
4
26
|
|
|
5
27
|
- **The 2.45.5 cycle opens.** A runner change alters the packed content.
|
package/README.md
CHANGED
|
@@ -135,7 +135,7 @@ Exit code is non-zero on any failed assertion. `--certify` distinguishes: `0`
|
|
|
135
135
|
|
|
136
136
|
## What's Covered
|
|
137
137
|
|
|
138
|
-
The current suite has
|
|
138
|
+
The current suite has 590 scenario files under `src/scenarios/`.
|
|
139
139
|
- 2026-09-29 (suite 2.45.0, openwop#1763): NEW `v2-cors-preflight.test.ts` — `headers.md` §Cross-origin preflight. For every operation it preflights with `Origin`, the method, and the operation's declared request headers (plus `authorization` / `content-type` where they apply, from `spec/v2/path-manifest.json`), and reads the answer as the Fetch standard's CORS-preflight check does. Self-gated: `inapplicable` when no operation's preflight grants `OPENWOP_CORS_ORIGIN` (default `https://conformance.invalid`).
|
|
140
140
|
- 2026-09-23 (suite 2.37.0 cycle, RFC 0213): NEW `v2-sse-last-event-id-cursor.test.ts` (a `Last-Event-ID` past the log is an exclusive cursor; a malformed id, when refused, is `400 validation_error`; the cursor never changes the answer for an unknown or foreign-tenant run — public test of `event-cursor-after-authorization`), `v2-idempotency-in-flight.test.ts` (five concurrent same-key creates yield one run; each loser is a marked replay or `409 idempotency_in_flight` with no retry timing in `details`; `partial-witness` when no loser was refused in flight) and `v2-interrupt-resolve-terminal.test.ts` (a run-scoped resolve after cancel or completion is `409 interrupt_already_resolved`, never `interrupt_cancelled`). All three sit off the core-standard floor until measured on the three bundle hosts.
|
|
141
141
|
- 2026-09-03 (suite `1.157.0 -> 1.158.0`, gap G17): NEW `idempotency-concurrent-claim.test.ts` — drives the new `host-sample-test-seams.md` §25 concurrent duplicate-delivery seam for the RFC 0150 §B / `idempotency.md` §"Concurrent duplicates (Layer 2)" atomic-claim MUST, which is unconditional and had no witness of any kind. Asserts every executor mints the SAME `logicalInvocationId` **before** asserting `delivered === 1` — without the identity check a host passes by minting different ids and never colliding, one effect because nothing raced. Not profile-gated and so not opt-out-able (the obligation is unconditional); an unmounted seam records `blocked`, which is not certifiable. Graduates `layer2-invocation-claim-atomic` reference-impl -> protocol.
|
|
@@ -482,7 +482,7 @@ Server-required (added in 1.7.0):
|
|
|
482
482
|
| ------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
483
483
|
| **Redaction** | [`capabilities.md`](../spec/v1/capabilities.md) §"Secrets" + NFR-7 + §"aiProviders" | Vendor-neutral assertions that the server doesn't leak secret material. Three scenario groups: (a) discovery shape contract — `secrets` + `aiProviders` advertisements are well-formed regardless of `secrets.supported`; when `supported === true`, scopes MUST be non-empty + `resolution === 'host-managed'`; `byok ⊆ supported`. (b) bearer-token redaction — invalid Bearer canary in `Authorization` header is not echoed in the 401 response body. (c) credentialRef echo control — gated on `secrets.supported === true`; canary planted in `configurable.ai.credentialRef` MUST NOT appear in any RunEvent payload (poll-based capture; transport-agnostic). Uses runtime-built canary fixtures (`lib/canaries.ts`) that defeat static secret scanners. 6 scenarios. |
|
|
484
484
|
|
|
485
|
-
Current source tree:
|
|
485
|
+
Current source tree: 590 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
|
|
486
486
|
|
|
487
487
|
## Remaining Gaps
|
|
488
488
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openwop/openwop-conformance",
|
|
3
|
-
"version": "2.45.
|
|
3
|
+
"version": "2.45.7",
|
|
4
4
|
"description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -56,6 +56,6 @@
|
|
|
56
56
|
"@openwop/spec-artifacts": "file:../spec-artifacts"
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
|
-
"@openwop/spec-artifacts": "2.45.
|
|
59
|
+
"@openwop/spec-artifacts": "2.45.7"
|
|
60
60
|
}
|
|
61
61
|
}
|
package/requirements.json
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
"$comment": "GENERATED by conformance/scripts/generate-requirement-registry.mjs — do not edit. One record per it()/test() in src/scenarios. Ids: openwop.it.<file-stem>.<title-slug>[~n] (src/lib/requirement-ids.ts). A record with id null has an interpolated title; its run-time row is keyed by the rendered title and maps here by file+line only. Renamed ids need a row in requirement-aliases.json.",
|
|
3
3
|
"generatedFrom": "src/scenarios/*.test.ts",
|
|
4
4
|
"counts": {
|
|
5
|
-
"files":
|
|
6
|
-
"tests":
|
|
7
|
-
"withStableId":
|
|
5
|
+
"files": 665,
|
|
6
|
+
"tests": 2542,
|
|
7
|
+
"withStableId": 2542,
|
|
8
8
|
"interpolatedTitles": 0,
|
|
9
|
-
"explicitIds":
|
|
9
|
+
"explicitIds": 2418
|
|
10
10
|
},
|
|
11
11
|
"records": [
|
|
12
12
|
{
|
|
@@ -28660,6 +28660,34 @@
|
|
|
28660
28660
|
}
|
|
28661
28661
|
]
|
|
28662
28662
|
},
|
|
28663
|
+
{
|
|
28664
|
+
"id": "openwop.it.v2-budget-exhaustion-facet.an-advertised-budget-onexhaustion-contains-fail",
|
|
28665
|
+
"file": "v2-budget-exhaustion-facet.test.ts",
|
|
28666
|
+
"line": 61,
|
|
28667
|
+
"title": "an advertised budget.onExhaustion contains fail",
|
|
28668
|
+
"explicitId": "openwop.requirement.runs.budget-exhaustion-facet-contains-fail",
|
|
28669
|
+
"citations": [
|
|
28670
|
+
{
|
|
28671
|
+
"section": null,
|
|
28672
|
+
"requirement": null,
|
|
28673
|
+
"interpolated": true
|
|
28674
|
+
}
|
|
28675
|
+
]
|
|
28676
|
+
},
|
|
28677
|
+
{
|
|
28678
|
+
"id": "openwop.it.v2-budget-exhaustion-facet.an-onexhaustion-value-the-host-does-not-list-is-refused-422-capability-not-provi",
|
|
28679
|
+
"file": "v2-budget-exhaustion-facet.test.ts",
|
|
28680
|
+
"line": 67,
|
|
28681
|
+
"title": "an onExhaustion value the host does not list is refused 422 capability_not_provided",
|
|
28682
|
+
"explicitId": "openwop.requirement.runs.budget-unserved-exhaustion-refused",
|
|
28683
|
+
"citations": [
|
|
28684
|
+
{
|
|
28685
|
+
"section": null,
|
|
28686
|
+
"requirement": null,
|
|
28687
|
+
"interpolated": true
|
|
28688
|
+
}
|
|
28689
|
+
]
|
|
28690
|
+
},
|
|
28663
28691
|
{
|
|
28664
28692
|
"id": "openwop.it.v2-bundle-v3-signed.a-signed-v3-bundle-validates-against-the-closed-root-schema-an-extra-root-key-is",
|
|
28665
28693
|
"file": "v2-bundle-v3-signed.test.ts",
|
|
@@ -36037,7 +36065,7 @@
|
|
|
36037
36065
|
{
|
|
36038
36066
|
"id": "openwop.it.v2-workspace-scope-from-identity.a-second-tenant-cannot-read-or-list-a-file-the-first-tenant-wrote",
|
|
36039
36067
|
"file": "v2-workspace-scope-from-identity.test.ts",
|
|
36040
|
-
"line":
|
|
36068
|
+
"line": 107,
|
|
36041
36069
|
"title": "a second tenant cannot read or list a file the first tenant wrote",
|
|
36042
36070
|
"explicitId": "openwop.requirement.0059.workspace-scope-from-identity",
|
|
36043
36071
|
"citations": [
|
|
@@ -36071,7 +36099,7 @@
|
|
|
36071
36099
|
{
|
|
36072
36100
|
"id": "openwop.it.v2-workspace-scope-from-identity.naming-the-first-tenant-s-scope-in-the-request-does-not-widen-the-second-tenant",
|
|
36073
36101
|
"file": "v2-workspace-scope-from-identity.test.ts",
|
|
36074
|
-
"line":
|
|
36102
|
+
"line": 129,
|
|
36075
36103
|
"title": "naming the first tenant's scope in the request does not widen the second tenant's scope",
|
|
36076
36104
|
"explicitId": "openwop.requirement.0059.workspace-scope-not-request-named",
|
|
36077
36105
|
"citations": [
|
package/scenario-majors.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$comment": "GENERATED by conformance/scripts/generate-scenario-majors.mjs (RFC 0168 §D.3). Do not edit; add a file to BOTH_MAJORS in the generator to target both majors.",
|
|
3
3
|
"counts": {
|
|
4
|
-
"files":
|
|
4
|
+
"files": 590,
|
|
5
5
|
"v1": 454,
|
|
6
|
-
"v2":
|
|
6
|
+
"v2": 151
|
|
7
7
|
},
|
|
8
8
|
"majors": {
|
|
9
9
|
"a2a-1-0-agent-card.test.ts": [
|
|
@@ -1320,6 +1320,9 @@
|
|
|
1320
1320
|
"v2-budget-enforcement.test.ts": [
|
|
1321
1321
|
2
|
|
1322
1322
|
],
|
|
1323
|
+
"v2-budget-exhaustion-facet.test.ts": [
|
|
1324
|
+
2
|
|
1325
|
+
],
|
|
1323
1326
|
"v2-bundle-v3-signed.test.ts": [
|
|
1324
1327
|
2
|
|
1325
1328
|
],
|
|
@@ -1,17 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_comment": "Provenance of @openwop/spec-artifacts (RFC 0168 §D.2). files: SHA-256 per file; the conformance suite compares the installed peer against dist/spec-artifacts.lock.json at start.",
|
|
3
3
|
"package": "@openwop/spec-artifacts",
|
|
4
|
-
"version": "2.45.
|
|
5
|
-
"corpusTag": "v2.45.
|
|
4
|
+
"version": "2.45.7",
|
|
5
|
+
"corpusTag": "v2.45.7",
|
|
6
6
|
"files": {
|
|
7
7
|
"api/.redocly.lint-ignore.yaml": "bf5a8350b88a72fa43f59605ed8d903ed24b6cfccda5e45509c9f6ed9ee4e712",
|
|
8
8
|
"api/asyncapi.yaml": "d5ecb9ee6114582be3b1f662c84bfac9ae96dae7bacb853e461168f70a8e1c7d",
|
|
9
9
|
"api/grpc/openwop.proto": "c3e72bb17cba514ee98feb6434e6c9b6ea6795bfd086489ec69fd882dd1ad977",
|
|
10
10
|
"api/openapi.yaml": "ad37a5a5c732ad46100fc03c3f4243fdd4f2976e330f02a8791b6b5b10a8a0c1",
|
|
11
11
|
"api/redocly.yaml": "b0604c89b2ca6d5076ec25725c539dad44a741a811fe524439ee6daef8baa09f",
|
|
12
|
-
"api/seams-v2.yaml": "
|
|
13
|
-
"api/v2/asyncapi.yaml": "
|
|
14
|
-
"api/v2/openapi.yaml": "
|
|
12
|
+
"api/seams-v2.yaml": "c248954e6382322c769fd44e6888f739644bcee2becfacabff3de173a9f40d71",
|
|
13
|
+
"api/v2/asyncapi.yaml": "63f837b7f56a5b5d17b1c34ffb47337ceb72e0027f8682c2f082c4df3e9dd856",
|
|
14
|
+
"api/v2/openapi.yaml": "d3c23e3c30db36ebe2ace9273f0262f927a4a44dc0ea4807ea9e94f607b661d3",
|
|
15
15
|
"api/v2/redocly.yaml": "1e66b60e6118ad11a823bb620678be464d99dfe50a40e3e6f93ec9429b88b34c",
|
|
16
16
|
"schemas/README.md": "0c0b737ffcf8f30e7d2809cec8a498232de710f41443212922ad8337cdde0b51",
|
|
17
17
|
"schemas/a2a-task-state.schema.json": "75d5049dea8bd873ff0e7546f1c60c8d36c219bec7084264360be54a8be30ae1",
|
|
@@ -31,7 +31,7 @@
|
|
|
31
31
|
"schemas/artifact-type-pack-manifest.schema.json": "60342fcb15eb7f3ab46271f0641409a30d3a4a8f318b1020c53070886b191252",
|
|
32
32
|
"schemas/audit-verify-result.schema.json": "0d3a3088ac1c32f9df66d346c33d7adbd8037940dba040e14ea847565a1a14bd",
|
|
33
33
|
"schemas/budget-policy.schema.json": "3d4fe30ae129fe136ab9ece395f2a2a42a76360786b1b9a1d5f1cbc753eb5b39",
|
|
34
|
-
"schemas/capabilities.schema.json": "
|
|
34
|
+
"schemas/capabilities.schema.json": "a7e5910bf10e6d8238e909fee5b81e405846f91688cccf1bb0d13fb081336a33",
|
|
35
35
|
"schemas/certification-bundle-v2.schema.json": "2f85530dd04824a15810c94e96869362dee1e78396eca21fdd18e5501416bb7d",
|
|
36
36
|
"schemas/channel-presence-payload.schema.json": "f9d0ef4219814c2b765baf3e68cbe5eea2cb42379f516a93a0459dcd5f7db47e",
|
|
37
37
|
"schemas/channel-written-payload.schema.json": "54723a56bc5a35232f64544ca686ac1ac959a5fb05b32b2401facca6b3ef51a3",
|
|
@@ -116,7 +116,7 @@
|
|
|
116
116
|
"schemas/v2/artifact.schema.json": "85b9dc284d39b99d2a66fa8cbf47f2dd3d8526d3bd6cc411ad315b372de38975",
|
|
117
117
|
"schemas/v2/audit-verify-result.schema.json": "14af8e9283510f0a40e48a2059c929ee44e0fc9fa3cc66a95bfbe90b17c48297",
|
|
118
118
|
"schemas/v2/budget-policy.schema.json": "1de68095db0beb5c7a4be629de26e79f62ebbe748a88d46f36e768edfd1f0d68",
|
|
119
|
-
"schemas/v2/capabilities.schema.json": "
|
|
119
|
+
"schemas/v2/capabilities.schema.json": "8d151a098a3cf7285fcadec8c1f192bed963ed6da3146e62d25155eb18afba9a",
|
|
120
120
|
"schemas/v2/certification-bundle.schema.json": "82ce31a1ed6c7e21b0ba88ff62a7b8b768fc3b176b1d06fce023203830b94725",
|
|
121
121
|
"schemas/v2/channel-presence-payload.schema.json": "4eeb224910e6f7eee3ab5e7c3899559c25c83a26d0022e0ad195c4672611977a",
|
|
122
122
|
"schemas/v2/channel-written-payload.schema.json": "ccecff3c71a3275ad8db035ac0e04db25e6f8d23cf091ddbcd5a60d1be1cbb6b",
|
|
@@ -204,13 +204,13 @@
|
|
|
204
204
|
"schemas/workspace-file.schema.json": "464de85c2a068243084ee9c1d969bc7cd5d8f7948574e58450d6493c38a0e1e4",
|
|
205
205
|
"spec/v1/alias-detectors.json": "0b2f28808ffe0b9d98a187775038c3406bb70919747a788a6e4f866e293ae6d3",
|
|
206
206
|
"spec/v1/capability-declaration-classes.json": "e7729aed5c4b4e1dd02abab0530f14cc95f5d4070fe51fb139e7f5cccefa00c6",
|
|
207
|
-
"spec/v1/core-standard-manifest.json": "
|
|
207
|
+
"spec/v1/core-standard-manifest.json": "592b87877bb6f713f77e6f4f64d671a63a119a71372405370d8a426cfab4113a",
|
|
208
208
|
"spec/v1/deprecations.json": "a3293072ea951857395d05f0437264e88965f1e9019ff61c55bacfa9c1aaa17c",
|
|
209
209
|
"spec/v1/deprecations.schema.json": "3e393c405d2a41b467d8c5e3c468549078df1ce6a6d2588fc488097b95d9b55b",
|
|
210
210
|
"spec/v1/event-codemap.json": "3da60d884157793a360da532a9fcbbfb5285636db325a74cec94b34622186d97",
|
|
211
211
|
"spec/v1/event-codemap.schema.json": "d05933b2e88103aff51a2f774df97b9bda065e0fc88dbd9b114cc47a32189174",
|
|
212
212
|
"spec/v1/extensions.json": "79a60754aa16cbdbdb604f8e5c00038af535a1690a4d07bd5ff7c38f0256a7bc",
|
|
213
|
-
"spec/v1/gaps.json": "
|
|
213
|
+
"spec/v1/gaps.json": "95c1c9242a97fafd75d5ca040c2ca07e146dbfa6d9be1fcee9ba882da0efd8ed",
|
|
214
214
|
"spec/v1/gaps.schema.json": "8fd83259f556553c9df0f53e7a82ca8c2a4771d471197f69b4e2ae0ceeacfd66",
|
|
215
215
|
"spec/v1/migrations.json": "4c50760be70cb644d02427bcaf5749606700edf4513f423903cf3b1d12f5c300",
|
|
216
216
|
"spec/v1/migrations.schema.json": "886779aa6c22e646db097f5df210adb018a4dd14a7b815465a18c8a7056c8f72",
|
|
@@ -240,7 +240,7 @@
|
|
|
240
240
|
"spec/v2/core/persistence.md": "249a1de353f421cdf92b9006a13a10afdb002f4029eb6a13520cc95252edb2ec",
|
|
241
241
|
"spec/v2/core/portability.md": "ab13a6ca43544b9a6a91edfb377ec680c768d14be841c84fb79125b231426891",
|
|
242
242
|
"spec/v2/core/replay.md": "4a8b53325e18aa53400ece81168c76b4402b932e5e304a391754f293b126eabf",
|
|
243
|
-
"spec/v2/core/runs.md": "
|
|
243
|
+
"spec/v2/core/runs.md": "bee5da861b31a583257f3d6020fbd245c36ba5ec40340d54ae6741f0c6f50ede",
|
|
244
244
|
"spec/v2/core/security-defaults.md": "887d8c4bc592094e3b931c23f1c2175c18ebbd47528b074016f9d2091bc513b2",
|
|
245
245
|
"spec/v2/core/storage.md": "dc94b985847c1341c7a064f04a6010f0fdbe6c3ab3e555d4362878c81d764c18",
|
|
246
246
|
"spec/v2/core/tool-catalog.md": "d16a9004418719c48301804aefa4d7ab6f41112a4ed83727f552aa1b92b418b0",
|
|
@@ -249,7 +249,7 @@
|
|
|
249
249
|
"spec/v2/core/workflow-chain-packs.md": "fdce7a8477e269950a2bac05ee6eddbd66eb0b670b464d00054f7196ec05d29c",
|
|
250
250
|
"spec/v2/corrections.json": "3a8d93e33865b692d26a3b6bfd3819766e206b2e0c2a69669baa4bf833d05a49",
|
|
251
251
|
"spec/v2/corrections.schema.json": "46bbffb954d13bd9995aebc912ceeb2544f7064331abf25fd6ab85c81a5cec94",
|
|
252
|
-
"spec/v2/declaration.json": "
|
|
252
|
+
"spec/v2/declaration.json": "45ca70a0d74fdade13192565cb3b35e446ffab537f2caaf23da73a4050b3c6ac",
|
|
253
253
|
"spec/v2/declaration.schema.json": "ef951633e4070899f64828f5cf1b5f4870270a0360f81dbb49b5b4d6cd407550",
|
|
254
254
|
"spec/v2/errors.json": "83f048b0a06952b4af6f125fb66c4212acab51056b411390f1010acd6ab9cdc0",
|
|
255
255
|
"spec/v2/errors.schema.json": "47404ad53e6147cf9017b813483712466573e419fb33e6c4b6c12d3e7deba1bc",
|
|
@@ -303,9 +303,9 @@
|
|
|
303
303
|
"spec/v2/path-manifest.json": "27c2307b84376b2f22bea4e2213fc7e33ef3bf8fd1d18039f7955edb00dc9c9a",
|
|
304
304
|
"spec/v2/peer-dependency-aliases.json": "d10299280abee08258502925bc327293ee413e0108cd6e6ec75ff6110653308d",
|
|
305
305
|
"spec/v2/profiles.json": "180beb9ef2d161766fa23b2987e7e829851e3d316273147cf6a4a3f324befd6b",
|
|
306
|
-
"spec/v2/release.json": "
|
|
306
|
+
"spec/v2/release.json": "4cc5a5efe139e3ef205ccba16dcc52a684cc65d682a24c195af7ad8a236896ad",
|
|
307
307
|
"spec/v2/retention-floors.json": "eaf3722d95c79947af1d4269ef85117e126518c588cfcf1a2b21b97269f51624",
|
|
308
|
-
"spec/v2/surface-baseline.json": "
|
|
308
|
+
"spec/v2/surface-baseline.json": "b7b01715d4b07dcefd055a937e8f865896e35017ff89f99a3b1aa1a801bbdb87"
|
|
309
309
|
},
|
|
310
|
-
"corpusCommit": "
|
|
310
|
+
"corpusCommit": "501121df013767d1adfcc3a342d82737d6fbd054"
|
|
311
311
|
}
|
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The budget exhaustion-facet witness (RFC 0231), shared by every major that
|
|
3
|
+
* gives a budget a run wire surface (`major-profile.ts` `runBudget`).
|
|
4
|
+
*
|
|
5
|
+
* A host that lists `budget.onExhaustion` serves exactly those values. The
|
|
6
|
+
* suite reads the list, and where it leaves out `interrupt` posts one create
|
|
7
|
+
* that asks for it. The answer must be `422 capability_not_provided`.
|
|
8
|
+
*
|
|
9
|
+
* Unaided: one discovery read and one create. Nothing is spent, because a
|
|
10
|
+
* conforming host creates no run.
|
|
11
|
+
*
|
|
12
|
+
* Two halves, as in `budget-witness.ts`: {@link drive} observes and asserts
|
|
13
|
+
* nothing; {@link judge} is pure.
|
|
14
|
+
*
|
|
15
|
+
* Not witnessed here: a host that lists `interrupt` and serves it. The resume
|
|
16
|
+
* that extends a budget has no specified shape (RFC 0231 gap G2), so no suite
|
|
17
|
+
* can drive it.
|
|
18
|
+
*/
|
|
19
|
+
|
|
20
|
+
import { driver, type OpenWOPResponse } from './driver.js';
|
|
21
|
+
import { readErrorCode } from './error-envelope.js';
|
|
22
|
+
import { getAdvertisedFixtures, isFixtureAdvertised } from './fixtures.js';
|
|
23
|
+
import { BUDGET_FIXTURE } from './budget-witness.js';
|
|
24
|
+
import type { MajorProfile } from './major-profile.js';
|
|
25
|
+
|
|
26
|
+
/** A valid policy whose only unserved part is the exhaustion behaviour. */
|
|
27
|
+
export const UNSERVED_POLICY = { maxToolCalls: 2, onExhaustion: 'interrupt' } as const;
|
|
28
|
+
const SERVED = new Set(['fail', 'interrupt']);
|
|
29
|
+
|
|
30
|
+
export interface FacetObservation {
|
|
31
|
+
/** The advertised list, as the host sent it. */
|
|
32
|
+
readonly facet: unknown;
|
|
33
|
+
/** The answer to a create asking for `interrupt`, or `undefined` when the list serves it and none was sent. */
|
|
34
|
+
readonly create: { readonly status: number; readonly code: string | undefined } | undefined;
|
|
35
|
+
}
|
|
36
|
+
export type FacetRun =
|
|
37
|
+
| { readonly kind: 'skip'; readonly disposition: 'inapplicable' | 'blocked'; readonly reason: string }
|
|
38
|
+
| { readonly kind: 'observed'; readonly observation: FacetObservation };
|
|
39
|
+
|
|
40
|
+
export interface FacetFinding {
|
|
41
|
+
readonly rule: 'contains-fail' | 'refused';
|
|
42
|
+
readonly ok: boolean;
|
|
43
|
+
readonly doc: string;
|
|
44
|
+
readonly message: string;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
async function http(fn: () => Promise<OpenWOPResponse>): Promise<OpenWOPResponse | null> {
|
|
48
|
+
try { return await fn(); } catch { return null; }
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
/** A workflow the host says it has, so the create is valid apart from the exhaustion behaviour. */
|
|
52
|
+
function advertisedWorkflow(): string | undefined {
|
|
53
|
+
if (isFixtureAdvertised(BUDGET_FIXTURE)) return BUDGET_FIXTURE;
|
|
54
|
+
return [...(getAdvertisedFixtures() ?? [])].sort()[0];
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
/** Read the facet and, where it leaves out `interrupt`, ask for it once. Asserts nothing. */
|
|
58
|
+
export async function drive(profile: MajorProfile, doc: unknown): Promise<FacetRun> {
|
|
59
|
+
const family = profile.family(doc, 'budget');
|
|
60
|
+
if (family === null) return { kind: 'skip', disposition: 'inapplicable', reason: 'the host does not advertise budget' };
|
|
61
|
+
const facet = family['onExhaustion'];
|
|
62
|
+
if (facet === undefined) return { kind: 'skip', disposition: 'inapplicable', reason: 'budget.onExhaustion is not advertised — the host is taken to serve both values' };
|
|
63
|
+
if (family['enforce'] !== 'hard') return { kind: 'skip', disposition: 'inapplicable', reason: 'budget.enforce is not hard — no exhaustion behaviour applies, and a consumer ignores the list' };
|
|
64
|
+
|
|
65
|
+
if (!Array.isArray(facet) || facet.includes('interrupt')) return { kind: 'observed', observation: { facet, create: undefined } };
|
|
66
|
+
|
|
67
|
+
const fragment = profile.runBudget(UNSERVED_POLICY);
|
|
68
|
+
if (fragment === null) return { kind: 'skip', disposition: 'inapplicable', reason: `major ${profile.major} gives a run budget no createRun surface` };
|
|
69
|
+
const workflowId = advertisedWorkflow();
|
|
70
|
+
if (workflowId === undefined) return { kind: 'skip', disposition: 'inapplicable', reason: 'no fixture is advertised — the suite has no workflow to name in an otherwise valid create' };
|
|
71
|
+
|
|
72
|
+
const res = await http(() => driver.post(profile.runsPath, { workflowId, ...fragment }));
|
|
73
|
+
if (res === null) return { kind: 'skip', disposition: 'blocked', reason: `POST ${profile.runsPath} unreachable (fetch failed)` };
|
|
74
|
+
if (res.status === 429) return { kind: 'skip', disposition: 'blocked', reason: `POST ${profile.runsPath} answered 429 — the run budget of the host, not the wire` };
|
|
75
|
+
|
|
76
|
+
// A host that accepted the create started a run. Stop it, so a failing row leaves none behind.
|
|
77
|
+
const runId = (res.json as { runId?: unknown } | undefined)?.runId;
|
|
78
|
+
if (typeof runId === 'string') await http(() => driver.post(`${profile.runsPath}/${encodeURIComponent(runId)}/cancel`, {}));
|
|
79
|
+
|
|
80
|
+
return { kind: 'observed', observation: { facet, create: { status: res.status, code: readErrorCode(res.json) ?? undefined } } };
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
/** What the list and the answer must show. Pure. */
|
|
84
|
+
export function judge(profile: MajorProfile, o: FacetObservation): FacetFinding[] {
|
|
85
|
+
const home = profile.major === 1 ? 'spec/v1/budget-policy.md §D' : `${profile.specRoot}/runs.md §Refusals`;
|
|
86
|
+
const out: FacetFinding[] = [];
|
|
87
|
+
const list = Array.isArray(o.facet) ? o.facet : null;
|
|
88
|
+
const wellFormed = list !== null && list.length > 0 && list.every((v) => typeof v === 'string' && SERVED.has(v));
|
|
89
|
+
out.push({
|
|
90
|
+
rule: 'contains-fail',
|
|
91
|
+
ok: wellFormed && list.includes('fail'),
|
|
92
|
+
doc: 'schemas/capabilities.schema.json §budget.onExhaustion',
|
|
93
|
+
message: `an advertised budget.onExhaustion MUST be a non-empty list of fail and interrupt that contains fail (got ${JSON.stringify(o.facet)})`,
|
|
94
|
+
});
|
|
95
|
+
if (o.create !== undefined) {
|
|
96
|
+
out.push({
|
|
97
|
+
rule: 'refused',
|
|
98
|
+
ok: o.create.status === 422 && o.create.code === 'capability_not_provided',
|
|
99
|
+
doc: home,
|
|
100
|
+
message: `a host whose budget.onExhaustion does not list interrupt MUST reject a create asking for it with 422 capability_not_provided and create no run (got ${o.create.status} ${o.create.code ?? ''})`.trim(),
|
|
101
|
+
});
|
|
102
|
+
}
|
|
103
|
+
return out;
|
|
104
|
+
}
|
package/src/lib/scratch-host.ts
CHANGED
|
@@ -37,6 +37,8 @@ export interface ScratchOptions {
|
|
|
37
37
|
/** Concurrent in-flight requests admitted before {@link refusal} answers. Unset: no cap. */
|
|
38
38
|
readonly inflightCap?: number | undefined;
|
|
39
39
|
readonly refusal?: ScratchRefusal | undefined;
|
|
40
|
+
/** Refuse a create by its body. `undefined` admits it. Unset: every create is admitted. */
|
|
41
|
+
readonly createRefusal?: ((workflowId: string, body: Readonly<Record<string, unknown>>) => ScratchRefusal | undefined) | undefined;
|
|
40
42
|
}
|
|
41
43
|
|
|
42
44
|
interface StoredRun { readonly id: string; run: ScriptedRun }
|
|
@@ -101,6 +103,8 @@ export class ScratchHost {
|
|
|
101
103
|
if (req.method === 'POST' && path === runs) {
|
|
102
104
|
const body = await this.body(req);
|
|
103
105
|
const workflowId = String(body['workflowId'] ?? '');
|
|
106
|
+
const refused = this.opts.createRefusal?.(workflowId, body);
|
|
107
|
+
if (refused !== undefined) return this.send(res, refused.status, refused.body, refused.headers);
|
|
104
108
|
const id = `scratch/run-${++this.seq}`;
|
|
105
109
|
this.runs.set(id, { id, run: this.opts.script?.(workflowId, body) ?? { status: 'running', events: [] } });
|
|
106
110
|
return this.send(res, 201, { runId: id, status: 'running' });
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* v2 — a host that lists `budget.onExhaustion` serves exactly those values
|
|
3
|
+
* (RFC 0231; `spec/v2/core/runs.md` §Refusals). The logic is
|
|
4
|
+
* `lib/exhaustion-facet-witness.ts`.
|
|
5
|
+
*
|
|
6
|
+
* contains-fail the advertised list is a non-empty list of `fail` and
|
|
7
|
+
* `interrupt`, and contains `fail`.
|
|
8
|
+
* refused where the list leaves out `interrupt`, a create with
|
|
9
|
+
* `budget.onExhaustion: "interrupt"` answers
|
|
10
|
+
* `422 capability_not_provided`. Unaided: one create, and a
|
|
11
|
+
* conforming host starts no run.
|
|
12
|
+
*
|
|
13
|
+
* Dispositions: no `budget`, no `onExhaustion` list, or `enforce` not `hard`
|
|
14
|
+
* ⇒ `inapplicable`. A list that includes `interrupt` ⇒ the refusal leg is
|
|
15
|
+
* `inapplicable`; the served path is not witnessed (RFC 0231 gap G2). No
|
|
16
|
+
* advertised fixture ⇒ the refusal leg is `inapplicable`, since the suite has
|
|
17
|
+
* no workflow to name.
|
|
18
|
+
*
|
|
19
|
+
* No major-1 twin: `lib/major-profile.ts` gives a major-1 run budget no
|
|
20
|
+
* `createRun` surface the suite drives.
|
|
21
|
+
*
|
|
22
|
+
* Proven both ways against the scratch host in
|
|
23
|
+
* `lib/exhaustion-facet-witness.test.ts`.
|
|
24
|
+
*
|
|
25
|
+
* @see spec/v2/core/runs.md §Refusals
|
|
26
|
+
* @see RFCS/0231-budget-exhaustion-facet.md
|
|
27
|
+
*/
|
|
28
|
+
|
|
29
|
+
import { describe, it, expect } from 'vitest';
|
|
30
|
+
import { v2Discovery } from '../lib/v2.js';
|
|
31
|
+
import { softSkip } from '../lib/soft-skip.js';
|
|
32
|
+
import { req } from '../lib/requirement-ids.js';
|
|
33
|
+
import { majorProfile } from '../lib/major-profile.js';
|
|
34
|
+
import { drive, judge, type FacetFinding, type FacetRun } from '../lib/exhaustion-facet-witness.js';
|
|
35
|
+
|
|
36
|
+
const PROFILE = majorProfile(2);
|
|
37
|
+
const ID_CONTAINS_FAIL = 'openwop.requirement.runs.budget-exhaustion-facet-contains-fail';
|
|
38
|
+
const ID_REFUSED = 'openwop.requirement.runs.budget-unserved-exhaustion-refused';
|
|
39
|
+
|
|
40
|
+
/** One observation per file: both legs read it. */
|
|
41
|
+
let once: Promise<FacetRun> | undefined;
|
|
42
|
+
function facetRun(): Promise<FacetRun> {
|
|
43
|
+
once ??= (async (): Promise<FacetRun> => {
|
|
44
|
+
let doc: Record<string, unknown> | null;
|
|
45
|
+
try { doc = await v2Discovery(); } catch { doc = null; }
|
|
46
|
+
if (!doc) return { kind: 'skip', disposition: 'blocked', reason: 'v2 discovery unreachable — /.well-known/openwop did not answer 200 with a JSON body under OpenWOP-Version: 2.0' };
|
|
47
|
+
return drive(PROFILE, doc);
|
|
48
|
+
})();
|
|
49
|
+
return once;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
type Leg = { readonly skip: readonly ['inapplicable' | 'blocked', string] } | { readonly findings: FacetFinding[] };
|
|
53
|
+
async function leg(rule: FacetFinding['rule'], absent: string): Promise<Leg> {
|
|
54
|
+
const r = await facetRun();
|
|
55
|
+
if (r.kind === 'skip') return { skip: [r.disposition, r.reason] };
|
|
56
|
+
const findings = judge(PROFILE, r.observation).filter((f) => f.rule === rule);
|
|
57
|
+
return findings.length === 0 ? { skip: ['inapplicable', absent] } : { findings };
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
describe('v2 budget exhaustion facet (RFC 0231; runs.md §Refusals)', () => {
|
|
61
|
+
it('an advertised budget.onExhaustion contains fail', async () => {
|
|
62
|
+
const l = await leg('contains-fail', 'budget.onExhaustion is not advertised');
|
|
63
|
+
if ('skip' in l) return softSkip(...l.skip);
|
|
64
|
+
for (const f of l.findings) expect(f.ok, req(ID_CONTAINS_FAIL, f.doc, f.message)).toBe(true);
|
|
65
|
+
});
|
|
66
|
+
|
|
67
|
+
it('an onExhaustion value the host does not list is refused 422 capability_not_provided', async () => {
|
|
68
|
+
const l = await leg('refused', 'budget.onExhaustion lists interrupt — the host serves it, and the served path has no witness (RFC 0231 gap G2)');
|
|
69
|
+
if ('skip' in l) return softSkip(...l.skip);
|
|
70
|
+
for (const f of l.findings) expect(f.ok, req(ID_REFUSED, f.doc, f.message)).toBe(true);
|
|
71
|
+
});
|
|
72
|
+
});
|
|
@@ -24,7 +24,9 @@
|
|
|
24
24
|
* request instead of the credential serves A's file here.
|
|
25
25
|
*
|
|
26
26
|
* Dispositions: family or seams profile not advertised ⇒ `inapplicable`
|
|
27
|
-
* (lib/seams.ts: the host has not claimed the instrument).
|
|
27
|
+
* (lib/seams.ts: the host has not claimed the instrument). That holds in
|
|
28
|
+
* strict mode too: the family is read with `familyAdvertised`, so no
|
|
29
|
+
* `family.workspace` opt-out is needed from a host that does not serve it. Advertised, but the
|
|
28
30
|
* seam answers 404/405/501, the second-tenant key is absent / identical /
|
|
29
31
|
* refused, or (leg 2) A's scope cannot be read from a run ⇒ `blocked` — the host
|
|
30
32
|
* took the obligation on and the suite could not measure it.
|
|
@@ -33,7 +35,7 @@ import { describe, it, expect } from 'vitest';
|
|
|
33
35
|
import { randomUUID } from 'node:crypto';
|
|
34
36
|
import { driver, type OpenWOPResponse } from '../lib/driver.js';
|
|
35
37
|
import { loadEnv } from '../lib/env.js';
|
|
36
|
-
import { v2Discovery,
|
|
38
|
+
import { v2Discovery, familyAdvertised } from '../lib/v2.js';
|
|
37
39
|
import { seamsProfileAdvertised, SEAMS_PREFIX } from '../lib/seams.js';
|
|
38
40
|
import { isFixtureAdvertised } from '../lib/fixtures.js';
|
|
39
41
|
import { softSkip, seamAbsent } from '../lib/soft-skip.js';
|
|
@@ -55,7 +57,12 @@ async function http(fn: () => Promise<OpenWOPResponse>): Promise<OpenWOPResponse
|
|
|
55
57
|
async function gate(): Promise<Gate> {
|
|
56
58
|
const doc = await v2Discovery();
|
|
57
59
|
if (!doc) return { ok: false, kind: 'blocked', reason: 'v2 discovery unreachable — /.well-known/openwop did not answer 200 JSON under OpenWOP-Version: 2.0' };
|
|
58
|
-
|
|
60
|
+
// `familyAdvertised`, not `gateFamily`: `workspace` is optional, and a host that does not advertise it has
|
|
61
|
+
// taken on no obligation here. Until 2.45.7 this was `gateFamily`, which in strict mode
|
|
62
|
+
// (`--require-behavior`) FAILS an unadvertised family unless the operator opts it out — so this file
|
|
63
|
+
// recorded `executed-fail` on a host its own header calls `inapplicable` (MyndHyve, 2.45.5), while
|
|
64
|
+
// its sibling isolation scenarios recorded `inapplicable` in the same run.
|
|
65
|
+
if (!(await familyAdvertised('workspace'))) return { ok: false, kind: 'inapplicable', reason: 'workspace family not advertised — no obligation' };
|
|
59
66
|
if (!seamsProfileAdvertised(doc)) return { ok: false, kind: 'inapplicable', reason: 'workspace files have no protocol path; the only observation surface is the seams profile (conformance.seamsProfile = openwop-conformance-seams-v2), which is not advertised' };
|
|
60
67
|
const probe = await http(() => driver.get(FILES));
|
|
61
68
|
if (probe === null) return { ok: false, kind: 'blocked', reason: `GET ${FILES} unreachable (fetch failed)` };
|