@openwop/openwop-conformance 2.42.8 → 2.42.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +1 -1
- package/dist/cli.js +3 -4
- package/dist/spec-artifacts.lock.json +2 -2
- package/package.json +2 -2
- package/requirements.json +24 -17
- package/schemas/CORPUS-STAMP.json +9 -9
- package/src/cli.ts +3 -4
- package/src/lib/mock-ai-budgets.ts +26 -0
- package/src/lib/mock-node-lock.ts +66 -0
- package/src/scenarios/envelope-completion-distinguishes-truncation.test.ts +17 -8
- package/src/scenarios/envelope-retry-attempted.test.ts +4 -0
- package/src/scenarios/envelope-truncated.test.ts +16 -16
- package/src/scenarios/v2-webhook-delivery-shape.test.ts +5 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# `@openwop/openwop-conformance` Changelog
|
|
2
2
|
|
|
3
|
+
## [2.42.9] — 2026-09-28 — a v2 delivery's workspaceId is checked, and two test-isolation and SHOULD-strength fixes
|
|
4
|
+
|
|
5
|
+
- **Scenario files that share a mock-AI node no longer race.** The mock-AI program hook is keyed by node id, and a node id belongs to a fixture. `envelope-completion-distinguishes-truncation` drives both envelope fixtures that `envelope-truncated` and `envelope-retry-attempted` also drive, so under the default parallel run one file could overwrite another's program. New `lib/mock-node-lock.ts` holds a cross-process lock per node id for the whole file, so files sharing a node run one after another. Serial runs (`--certify`, openwop-app's harness) are unchanged.
|
|
6
|
+
- **The RFC 0033 §B truncation budget legs compare real attempts and record the SHOULD.** Both legs asserted "retry budget > 50" (the fixture's value) as a MUST. §B makes the retry a MAY and the larger budget a SHOULD, so the old legs failed a conforming host that retried at the same budget and passed a host that ignored the fixture's `maxTokens`. They now read the first and retry budgets through the `dispatch-budgets` hook. A larger retry budget passes, and an unmet SHOULD is recorded as `inapplicable` with the observed values. An unserved hook is a partial witness. On openwop-app the rows are identical before and after, and the truncation leg passes with a real comparison.
|
|
7
|
+
- **`v2-webhook-delivery-shape` checks that `workspaceId` is present exactly when the run has a workspace.** `webhooks.md` §Delivery requires it and forbids substituting a value, but the leg only validated the schema, which cannot say "exactly when". It now compares the delivered `workspaceId` with the owner echo on the event. The v2 reference host sent `"default"` and fails without openwop-examples #99 (fixed there); openwop-app already conformed.
|
|
8
|
+
- **`--help` says `--bundle-version` defaults to 3** (#1693). The code has defaulted to 3 since 2.0.0-rc.1 and the deprecation note for `2` already said so; the help text still read `Default 2`.
|
|
9
|
+
- **Suite `2.42.9`**. `@openwop/spec-artifacts` moves in lockstep at the same exact pin.
|
|
10
|
+
|
|
3
11
|
## [2.42.8] — 2026-09-27 — RFC 0221, a closed audit schema again, and a dispatch race fixed
|
|
4
12
|
|
|
5
13
|
- **`v2-preferred-version-default`: a header-less request on an unversioned operation path is served major 2.** This follows a Class 3 correction to `versioning.md` §1.3: the `preferredVersion` default applies to `/.well-known/openwop` only. The new leg reads `OpenWOP-Version` on a header-less `GET /runs/{unknown}`. It passes on the v2 reference host and fails when the host applies `preferredVersion` to every path.
|
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
# --legacy-peer-deps is REQUIRED, not optional: the exact peer pin is what npm's
|
|
12
12
|
# default resolver refuses. npm 10.9 fails outright with
|
|
13
13
|
# "Cannot read properties of null (reading 'edgesOut')" — use npm >= 11.
|
|
14
|
-
npm install --legacy-peer-deps @openwop/openwop-conformance@2.42.
|
|
14
|
+
npm install --legacy-peer-deps @openwop/openwop-conformance@2.42.9 @openwop/spec-artifacts@2.42.9
|
|
15
15
|
# or run without install:
|
|
16
16
|
npx @openwop/openwop-conformance --base-url https://api.example.com --api-key hk_test_...
|
|
17
17
|
```
|
package/dist/cli.js
CHANGED
|
@@ -60,9 +60,8 @@ function parseArgs(argv) {
|
|
|
60
60
|
let impl;
|
|
61
61
|
let implVersion;
|
|
62
62
|
let certify;
|
|
63
|
-
// Suite
|
|
64
|
-
// `--bundle-version
|
|
65
|
-
// 2026-11-10) and is removed at v2.0 (spec/v1/deprecations.json).
|
|
63
|
+
// Suite 2.0.0-rc.1: bundle v3 (RFC 0168 §E) is the default. v2 stays reachable
|
|
64
|
+
// via `--bundle-version 2`, deprecated (RFC 0168 §E.3); v1 is gone.
|
|
66
65
|
let bundleVersion = '3';
|
|
67
66
|
let hostBuild;
|
|
68
67
|
let evidenceTier = 'self';
|
|
@@ -265,7 +264,7 @@ Certification (RFC 0089):
|
|
|
265
264
|
--signing-key <pem> v3: Ed25519 private key (PKCS8 PEM) that signs the bundle, or OPENWOP_BUNDLE_SIGNING_KEY.
|
|
266
265
|
--signing-key-id <id> v3: the keyId the host publishes for that key, or OPENWOP_BUNDLE_SIGNING_KEY_ID.
|
|
267
266
|
--evidence-tier <t> v3: self | steward | independent (independent needs --verifier-key/--verifier-key-id).
|
|
268
|
-
--bundle-version <2|3> Certification bundle format. Default
|
|
267
|
+
--bundle-version <2|3> Certification bundle format. Default 3 (since suite 2.0.0-rc.1; 2 is DEPRECATED, RFC 0168 §E.3). Version 2 (RFC 0148
|
|
269
268
|
§C) records per-requirement DISPOSITIONS instead of pass/fail/skip
|
|
270
269
|
file lists, so "we could not check" stops being indistinguishable
|
|
271
270
|
from "checked and it holds". See the note it prints.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openwop/openwop-conformance",
|
|
3
|
-
"version": "2.42.
|
|
3
|
+
"version": "2.42.9",
|
|
4
4
|
"description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -56,6 +56,6 @@
|
|
|
56
56
|
"@openwop/spec-artifacts": "file:../spec-artifacts"
|
|
57
57
|
},
|
|
58
58
|
"peerDependencies": {
|
|
59
|
-
"@openwop/spec-artifacts": "2.42.
|
|
59
|
+
"@openwop/spec-artifacts": "2.42.9"
|
|
60
60
|
}
|
|
61
61
|
}
|
package/requirements.json
CHANGED
|
@@ -11541,7 +11541,7 @@
|
|
|
11541
11541
|
{
|
|
11542
11542
|
"id": "openwop.it.envelope-completion-distinguishes-truncation.capabilities-envelopes-reliability-completion-when-present-conforms-to-rfc-0033",
|
|
11543
11543
|
"file": "envelope-completion-distinguishes-truncation.test.ts",
|
|
11544
|
-
"line":
|
|
11544
|
+
"line": 108,
|
|
11545
11545
|
"title": "capabilities.envelopes.reliability.completion (when present) conforms to RFC 0033 §E",
|
|
11546
11546
|
"explicitId": "openwop.it.envelope-completion-distinguishes-truncation.capabilities-envelopes-reliability-completion-when-present-conforms-to-rfc-0033",
|
|
11547
11547
|
"citations": [
|
|
@@ -11558,7 +11558,7 @@
|
|
|
11558
11558
|
{
|
|
11559
11559
|
"id": "openwop.it.envelope-completion-distinguishes-truncation.truncation-emits-envelope-truncated-envelope-retry-attempted-with-reason-truncat",
|
|
11560
11560
|
"file": "envelope-completion-distinguishes-truncation.test.ts",
|
|
11561
|
-
"line":
|
|
11561
|
+
"line": 131,
|
|
11562
11562
|
"title": "truncation: emits envelope.truncated + envelope.retry.attempted with reason: \"truncation\"",
|
|
11563
11563
|
"explicitId": "openwop.it.envelope-completion-distinguishes-truncation.truncation-emits-envelope-truncated-envelope-retry-attempted-with-reason-truncat",
|
|
11564
11564
|
"citations": [
|
|
@@ -11579,20 +11579,21 @@
|
|
|
11579
11579
|
{
|
|
11580
11580
|
"id": "openwop.it.envelope-completion-distinguishes-truncation.truncation-retry-budget-strictly-greater-than-initial-rfc-0033-b-truncationbudge",
|
|
11581
11581
|
"file": "envelope-completion-distinguishes-truncation.test.ts",
|
|
11582
|
-
"line":
|
|
11582
|
+
"line": 156,
|
|
11583
11583
|
"title": "truncation: retry budget strictly greater than initial (RFC 0033 §B truncationBudgetMultiplier)",
|
|
11584
11584
|
"explicitId": "openwop.it.envelope-completion-distinguishes-truncation.truncation-retry-budget-strictly-greater-than-initial-rfc-0033-b-truncationbudge",
|
|
11585
11585
|
"citations": [
|
|
11586
11586
|
{
|
|
11587
11587
|
"section": "RFCS/0033-envelope-completion-contract.md §B",
|
|
11588
|
-
"requirement":
|
|
11588
|
+
"requirement": null,
|
|
11589
|
+
"interpolated": true
|
|
11589
11590
|
}
|
|
11590
11591
|
]
|
|
11591
11592
|
},
|
|
11592
11593
|
{
|
|
11593
11594
|
"id": "openwop.it.envelope-completion-distinguishes-truncation.schema-violation-no-envelope-truncated-envelope-retry-attempted-reason-schema-vi",
|
|
11594
11595
|
"file": "envelope-completion-distinguishes-truncation.test.ts",
|
|
11595
|
-
"line":
|
|
11596
|
+
"line": 185,
|
|
11596
11597
|
"title": "schema-violation: NO envelope.truncated; envelope.retry.attempted reason ∈ {schema-violation, parse-error}",
|
|
11597
11598
|
"explicitId": "openwop.it.envelope-completion-distinguishes-truncation.schema-violation-no-envelope-truncated-envelope-retry-attempted-reason-schema-vi",
|
|
11598
11599
|
"citations": [
|
|
@@ -11609,7 +11610,7 @@
|
|
|
11609
11610
|
{
|
|
11610
11611
|
"id": "openwop.it.envelope-completion-distinguishes-truncation.schema-violation-retry-budget-unchanged-from-initial-no-budget-multiplication-on",
|
|
11611
11612
|
"file": "envelope-completion-distinguishes-truncation.test.ts",
|
|
11612
|
-
"line":
|
|
11613
|
+
"line": 215,
|
|
11613
11614
|
"title": "schema-violation: retry budget UNCHANGED from initial (no budget multiplication on this path)",
|
|
11614
11615
|
"explicitId": "openwop.it.envelope-completion-distinguishes-truncation.schema-violation-retry-budget-unchanged-from-initial-no-budget-multiplication-on",
|
|
11615
11616
|
"citations": [
|
|
@@ -12261,7 +12262,7 @@
|
|
|
12261
12262
|
{
|
|
12262
12263
|
"id": "openwop.it.envelope-retry-attempted.when-mock-llm-emits-invalid-envelope-on-attempt-1-then-valid-on-attempt-2-exactl",
|
|
12263
12264
|
"file": "envelope-retry-attempted.test.ts",
|
|
12264
|
-
"line":
|
|
12265
|
+
"line": 162,
|
|
12265
12266
|
"title": "when mock LLM emits invalid envelope on attempt 1 then valid on attempt 2, exactly one `envelope.retry.attempted` event fires before the second attempt",
|
|
12266
12267
|
"explicitId": "openwop.it.envelope-retry-attempted.when-mock-llm-emits-invalid-envelope-on-attempt-1-then-valid-on-attempt-2-exactl",
|
|
12267
12268
|
"citations": [
|
|
@@ -12274,7 +12275,7 @@
|
|
|
12274
12275
|
{
|
|
12275
12276
|
"id": "openwop.it.envelope-retry-attempted.event-payload-carries-attempt-2-1-indexed-first-attempt-does-not-emit",
|
|
12276
12277
|
"file": "envelope-retry-attempted.test.ts",
|
|
12277
|
-
"line":
|
|
12278
|
+
"line": 183,
|
|
12278
12279
|
"title": "event payload carries `attempt: 2` (1-indexed; first attempt does not emit)",
|
|
12279
12280
|
"explicitId": "openwop.it.envelope-retry-attempted.event-payload-carries-attempt-2-1-indexed-first-attempt-does-not-emit",
|
|
12280
12281
|
"citations": [
|
|
@@ -12291,7 +12292,7 @@
|
|
|
12291
12292
|
{
|
|
12292
12293
|
"id": "openwop.it.envelope-retry-attempted.reason-is-one-of-the-spec-reserved-closed-enum-values-or-matches-the-x-host-host",
|
|
12293
12294
|
"file": "envelope-retry-attempted.test.ts",
|
|
12294
|
-
"line":
|
|
12295
|
+
"line": 201,
|
|
12295
12296
|
"title": "`reason` is one of the spec-reserved closed-enum values OR matches the `x-host-<host>-<key>` extension pattern",
|
|
12296
12297
|
"explicitId": "openwop.it.envelope-retry-attempted.reason-is-one-of-the-spec-reserved-closed-enum-values-or-matches-the-x-host-host",
|
|
12297
12298
|
"citations": [
|
|
@@ -12304,7 +12305,7 @@
|
|
|
12304
12305
|
{
|
|
12305
12306
|
"id": "openwop.it.envelope-retry-attempted.eventual-success-records-normally-via-envelope-acceptance-downstream-runeventdoc",
|
|
12306
12307
|
"file": "envelope-retry-attempted.test.ts",
|
|
12307
|
-
"line":
|
|
12308
|
+
"line": 221,
|
|
12308
12309
|
"title": "eventual success records normally via envelope acceptance + downstream RunEventDoc",
|
|
12309
12310
|
"explicitId": "openwop.it.envelope-retry-attempted.eventual-success-records-normally-via-envelope-acceptance-downstream-runeventdoc",
|
|
12310
12311
|
"citations": [
|
|
@@ -12317,7 +12318,7 @@
|
|
|
12317
12318
|
{
|
|
12318
12319
|
"id": "openwop.it.envelope-retry-attempted.previouserror-when-populated-must-not-contain-prompt-or-response-substring-excer",
|
|
12319
12320
|
"file": "envelope-retry-attempted.test.ts",
|
|
12320
|
-
"line":
|
|
12321
|
+
"line": 240,
|
|
12321
12322
|
"title": "`previousError` (when populated) MUST NOT contain prompt or response substring excerpts — limit to validator output",
|
|
12322
12323
|
"explicitId": "openwop.it.envelope-retry-attempted.previouserror-when-populated-must-not-contain-prompt-or-response-substring-excer",
|
|
12323
12324
|
"citations": [
|
|
@@ -12428,7 +12429,7 @@
|
|
|
12428
12429
|
{
|
|
12429
12430
|
"id": "openwop.it.envelope-truncated.when-mock-returns-stopreason-max-tokens-on-attempt-1-exactly-one-envelope-trunca",
|
|
12430
12431
|
"file": "envelope-truncated.test.ts",
|
|
12431
|
-
"line":
|
|
12432
|
+
"line": 59,
|
|
12432
12433
|
"title": "when mock returns stopReason: max_tokens on attempt 1, exactly one envelope.truncated event fires with stopReason: max_tokens",
|
|
12433
12434
|
"explicitId": "openwop.it.envelope-truncated.when-mock-returns-stopreason-max-tokens-on-attempt-1-exactly-one-envelope-trunca",
|
|
12434
12435
|
"citations": [
|
|
@@ -12441,7 +12442,7 @@
|
|
|
12441
12442
|
{
|
|
12442
12443
|
"id": "openwop.it.envelope-truncated.payload-includes-nodeid-provider-model-partialpayloadavailable-boolean",
|
|
12443
12444
|
"file": "envelope-truncated.test.ts",
|
|
12444
|
-
"line":
|
|
12445
|
+
"line": 85,
|
|
12445
12446
|
"title": "payload includes nodeId + provider + model + partialPayloadAvailable boolean",
|
|
12446
12447
|
"explicitId": "openwop.it.envelope-truncated.payload-includes-nodeid-provider-model-partialpayloadavailable-boolean",
|
|
12447
12448
|
"citations": [
|
|
@@ -12454,13 +12455,14 @@
|
|
|
12454
12455
|
{
|
|
12455
12456
|
"id": "openwop.it.envelope-truncated.retry-attempt-receives-a-maxtokens-value-strictly-greater-than-the-previous-atte",
|
|
12456
12457
|
"file": "envelope-truncated.test.ts",
|
|
12457
|
-
"line":
|
|
12458
|
+
"line": 104,
|
|
12458
12459
|
"title": "retry attempt receives a maxTokens value strictly greater than the previous attempt (RFC 0033 §B truncationBudgetMultiplier)",
|
|
12459
12460
|
"explicitId": "openwop.it.envelope-truncated.retry-attempt-receives-a-maxtokens-value-strictly-greater-than-the-previous-atte",
|
|
12460
12461
|
"citations": [
|
|
12461
12462
|
{
|
|
12462
12463
|
"section": "RFCS/0033-envelope-completion-contract.md §B",
|
|
12463
|
-
"requirement":
|
|
12464
|
+
"requirement": null,
|
|
12465
|
+
"interpolated": true
|
|
12464
12466
|
}
|
|
12465
12467
|
]
|
|
12466
12468
|
},
|
|
@@ -33869,13 +33871,18 @@
|
|
|
33869
33871
|
{
|
|
33870
33872
|
"section": "spec/v2/core/webhooks.md §Delivery",
|
|
33871
33873
|
"requirement": "a major-2 owner echo carries tenant and subject"
|
|
33874
|
+
},
|
|
33875
|
+
{
|
|
33876
|
+
"section": "spec/v2/core/webhooks.md §Delivery",
|
|
33877
|
+
"requirement": null,
|
|
33878
|
+
"interpolated": true
|
|
33872
33879
|
}
|
|
33873
33880
|
]
|
|
33874
33881
|
},
|
|
33875
33882
|
{
|
|
33876
33883
|
"id": "openwop.it.v2-webhook-delivery-shape.a-major-1-subscriber-still-receives-the-v1-rendering-the-v1-wire-does-not-move-m",
|
|
33877
33884
|
"file": "v2-webhook-delivery-shape.test.ts",
|
|
33878
|
-
"line":
|
|
33885
|
+
"line": 244,
|
|
33879
33886
|
"title": "a major-1 subscriber still receives the v1 rendering — the v1 wire does not move mid-overlap",
|
|
33880
33887
|
"explicitId": "openwop.requirement.0171.webhook-delivery-shape",
|
|
33881
33888
|
"citations": [
|
|
@@ -33893,7 +33900,7 @@
|
|
|
33893
33900
|
{
|
|
33894
33901
|
"id": "openwop.it.v2-webhook-delivery-shape.a-seeded-era-2-run-is-delivered-projected-too-the-fan-out-branch-a-fresh-run-can",
|
|
33895
33902
|
"file": "v2-webhook-delivery-shape.test.ts",
|
|
33896
|
-
"line":
|
|
33903
|
+
"line": 268,
|
|
33897
33904
|
"title": "a seeded era-2 run is delivered projected too — the fan-out branch a fresh run cannot reach (seam-gated)",
|
|
33898
33905
|
"explicitId": "openwop.requirement.0171.webhook-delivery-shape",
|
|
33899
33906
|
"citations": [
|
|
@@ -1,17 +1,17 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_comment": "Provenance of @openwop/spec-artifacts (RFC 0168 §D.2). files: SHA-256 per file; the conformance suite compares the installed peer against dist/spec-artifacts.lock.json at start.",
|
|
3
3
|
"package": "@openwop/spec-artifacts",
|
|
4
|
-
"version": "2.42.
|
|
5
|
-
"corpusTag": "v2.42.
|
|
4
|
+
"version": "2.42.9",
|
|
5
|
+
"corpusTag": "v2.42.9",
|
|
6
6
|
"files": {
|
|
7
7
|
"api/.redocly.lint-ignore.yaml": "bf5a8350b88a72fa43f59605ed8d903ed24b6cfccda5e45509c9f6ed9ee4e712",
|
|
8
8
|
"api/asyncapi.yaml": "d5ecb9ee6114582be3b1f662c84bfac9ae96dae7bacb853e461168f70a8e1c7d",
|
|
9
9
|
"api/grpc/openwop.proto": "c3e72bb17cba514ee98feb6434e6c9b6ea6795bfd086489ec69fd882dd1ad977",
|
|
10
10
|
"api/openapi.yaml": "69464bc0e1e71ef3b543a73d4fa4f67202343d2603c80af8033f6c3619ef1b30",
|
|
11
11
|
"api/redocly.yaml": "b0604c89b2ca6d5076ec25725c539dad44a741a811fe524439ee6daef8baa09f",
|
|
12
|
-
"api/seams-v2.yaml": "
|
|
13
|
-
"api/v2/asyncapi.yaml": "
|
|
14
|
-
"api/v2/openapi.yaml": "
|
|
12
|
+
"api/seams-v2.yaml": "91a84ef4a8d38824e528ee7f59b29df34c167f2e5e695f200ad19b0fd4b47cdd",
|
|
13
|
+
"api/v2/asyncapi.yaml": "b93593055cf033fdf37b8a18fd36a2045b989a97a97b49adda23efb6ff1654e6",
|
|
14
|
+
"api/v2/openapi.yaml": "d698d93dd3ef46c060bcdb68f8950aa983ef8904ad18cf14b87ce96304ae59ed",
|
|
15
15
|
"api/v2/redocly.yaml": "1e66b60e6118ad11a823bb620678be464d99dfe50a40e3e6f93ec9429b88b34c",
|
|
16
16
|
"schemas/README.md": "0c0b737ffcf8f30e7d2809cec8a498232de710f41443212922ad8337cdde0b51",
|
|
17
17
|
"schemas/a2a-task-state.schema.json": "75d5049dea8bd873ff0e7546f1c60c8d36c219bec7084264360be54a8be30ae1",
|
|
@@ -204,7 +204,7 @@
|
|
|
204
204
|
"schemas/workspace-file.schema.json": "464de85c2a068243084ee9c1d969bc7cd5d8f7948574e58450d6493c38a0e1e4",
|
|
205
205
|
"spec/v1/alias-detectors.json": "29f21e0fb2c7a232a92a5c19b4cba3a86bc3b5e16f06780452f0421c66c6ec38",
|
|
206
206
|
"spec/v1/capability-declaration-classes.json": "e7729aed5c4b4e1dd02abab0530f14cc95f5d4070fe51fb139e7f5cccefa00c6",
|
|
207
|
-
"spec/v1/core-standard-manifest.json": "
|
|
207
|
+
"spec/v1/core-standard-manifest.json": "717b478af97d11a4c7844fceecd536a70783ca1cbb7079ec1434e710a24fc6e5",
|
|
208
208
|
"spec/v1/deprecations.json": "46df807fb4adf0aae24671c20f69975cf3496e3bc3a521cb7e6ea556bdce563c",
|
|
209
209
|
"spec/v1/deprecations.schema.json": "3e393c405d2a41b467d8c5e3c468549078df1ce6a6d2588fc488097b95d9b55b",
|
|
210
210
|
"spec/v1/event-codemap.json": "3da60d884157793a360da532a9fcbbfb5285636db325a74cec94b34622186d97",
|
|
@@ -297,9 +297,9 @@
|
|
|
297
297
|
"spec/v2/path-manifest.json": "17298a245c5da9a70aa7ab0b7f2db4277e06209d6fcdd7187a8b4cea179c08f3",
|
|
298
298
|
"spec/v2/peer-dependency-aliases.json": "d10299280abee08258502925bc327293ee413e0108cd6e6ec75ff6110653308d",
|
|
299
299
|
"spec/v2/profiles.json": "180beb9ef2d161766fa23b2987e7e829851e3d316273147cf6a4a3f324befd6b",
|
|
300
|
-
"spec/v2/release.json": "
|
|
300
|
+
"spec/v2/release.json": "1ac31e3dd517036b357bfa5743b760e21d28b8cda7f8550e65d8444e964751c6",
|
|
301
301
|
"spec/v2/retention-floors.json": "eaf3722d95c79947af1d4269ef85117e126518c588cfcf1a2b21b97269f51624",
|
|
302
|
-
"spec/v2/surface-baseline.json": "
|
|
302
|
+
"spec/v2/surface-baseline.json": "5cab3b39eaeb344361ac7d9038d15be5cf6a5620f1912fdd0c184cef95bae857"
|
|
303
303
|
},
|
|
304
|
-
"corpusCommit": "
|
|
304
|
+
"corpusCommit": "67e27a37bbd22f24ca2b6a2bce8f0a0f0971e3c8"
|
|
305
305
|
}
|
package/src/cli.ts
CHANGED
|
@@ -104,9 +104,8 @@ function parseArgs(argv: readonly string[]): ParsedArgs {
|
|
|
104
104
|
let impl: string | undefined;
|
|
105
105
|
let implVersion: string | undefined;
|
|
106
106
|
let certify: string | undefined;
|
|
107
|
-
// Suite
|
|
108
|
-
// `--bundle-version
|
|
109
|
-
// 2026-11-10) and is removed at v2.0 (spec/v1/deprecations.json).
|
|
107
|
+
// Suite 2.0.0-rc.1: bundle v3 (RFC 0168 §E) is the default. v2 stays reachable
|
|
108
|
+
// via `--bundle-version 2`, deprecated (RFC 0168 §E.3); v1 is gone.
|
|
110
109
|
let bundleVersion: '2' | '3' = '3';
|
|
111
110
|
let hostBuild: ParsedArgs['hostBuild'];
|
|
112
111
|
let evidenceTier: ParsedArgs['evidenceTier'] = 'self';
|
|
@@ -294,7 +293,7 @@ Certification (RFC 0089):
|
|
|
294
293
|
--signing-key <pem> v3: Ed25519 private key (PKCS8 PEM) that signs the bundle, or OPENWOP_BUNDLE_SIGNING_KEY.
|
|
295
294
|
--signing-key-id <id> v3: the keyId the host publishes for that key, or OPENWOP_BUNDLE_SIGNING_KEY_ID.
|
|
296
295
|
--evidence-tier <t> v3: self | steward | independent (independent needs --verifier-key/--verifier-key-id).
|
|
297
|
-
--bundle-version <2|3> Certification bundle format. Default
|
|
296
|
+
--bundle-version <2|3> Certification bundle format. Default 3 (since suite 2.0.0-rc.1; 2 is DEPRECATED, RFC 0168 §E.3). Version 2 (RFC 0148
|
|
298
297
|
§C) records per-requirement DISPOSITIONS instead of pass/fail/skip
|
|
299
298
|
file lists, so "we could not check" stops being indistinguishable
|
|
300
299
|
from "checked and it holds". See the note it prints.
|
|
@@ -38,3 +38,29 @@ export async function readDispatchBudgets(nodeId: string): Promise<DispatchBudge
|
|
|
38
38
|
}
|
|
39
39
|
return { ok: true, attempts: out };
|
|
40
40
|
}
|
|
41
|
+
|
|
42
|
+
/**
|
|
43
|
+
* RFC 0033 §B: a host MAY retry a truncated emission, and the new output budget
|
|
44
|
+
* SHOULD be greater than the previous one. A SHOULD is recorded, not failed: the
|
|
45
|
+
* caller asserts only on `met`, and records `unmet` as `inapplicable` with the
|
|
46
|
+
* observed budgets. The comparison is between the attempts the provider actually
|
|
47
|
+
* received (the dispatch-budgets seam), never against the fixture's configured value.
|
|
48
|
+
*/
|
|
49
|
+
export type TruncationBudgetVerdict =
|
|
50
|
+
| { kind: 'met'; first: number; retry: number }
|
|
51
|
+
| { kind: 'unmet'; first: number; retry: number }
|
|
52
|
+
| { kind: 'skip'; skip: 'inapplicable' | 'blocked'; reason: string };
|
|
53
|
+
|
|
54
|
+
export async function truncationBudgetVerdict(nodeId: string): Promise<TruncationBudgetVerdict> {
|
|
55
|
+
const budgets = await readDispatchBudgets(nodeId);
|
|
56
|
+
if (!budgets.ok && budgets.unserved) {
|
|
57
|
+
return { kind: 'skip', skip: 'inapplicable', reason: `partial witness — the §B budget SHOULD is not observed: ${budgets.reason}, so the retry's budget cannot be compared with the first attempt's` };
|
|
58
|
+
}
|
|
59
|
+
if (!budgets.ok) return { kind: 'skip', skip: 'blocked', reason: budgets.reason };
|
|
60
|
+
const [first, retry] = budgets.attempts;
|
|
61
|
+
if (budgets.attempts.length < 2) return { kind: 'skip', skip: 'inapplicable', reason: `the host made no truncation retry (RFC 0033 §B: MAY retry); per-attempt budgets ${JSON.stringify(budgets.attempts)}` };
|
|
62
|
+
if (typeof first !== 'number' || typeof retry !== 'number') {
|
|
63
|
+
return { kind: 'skip', skip: 'inapplicable', reason: `partial witness — a call carried no output budget (per-attempt budgets ${JSON.stringify(budgets.attempts)}), so the §B SHOULD cannot be compared` };
|
|
64
|
+
}
|
|
65
|
+
return retry > first ? { kind: 'met', first, retry } : { kind: 'unmet', first, retry };
|
|
66
|
+
}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Cross-file isolation for the mock-AI program seam.
|
|
3
|
+
*
|
|
4
|
+
* `POST /v1/host/sample/test/mock-ai/program` is keyed by `nodeId` alone
|
|
5
|
+
* (host-sample-test-seams.md §5), and a node id belongs to a fixture. When two
|
|
6
|
+
* scenario files drive the same fixture, vitest runs them in parallel workers
|
|
7
|
+
* by default and each can overwrite the other's program between seeding it and
|
|
8
|
+
* reading the run: a flaky result that says nothing about the host.
|
|
9
|
+
*
|
|
10
|
+
* `holdMockNodes(...ids)` holds a lock per node id for the whole file, taken in
|
|
11
|
+
* `beforeAll` and released in `afterAll`, so files sharing a node run one after
|
|
12
|
+
* another. The lock is an atomic `mkdir` in the OS temp dir, keyed by the target
|
|
13
|
+
* base URL and the node id, so it is shared by every worker process on the
|
|
14
|
+
* machine and separate hosts never contend. Ids are taken in sorted order (no
|
|
15
|
+
* deadlock between files holding overlapping sets); a lock older than
|
|
16
|
+
* STALE_MS is presumed left by a killed run and is broken.
|
|
17
|
+
*/
|
|
18
|
+
|
|
19
|
+
import { beforeAll, afterAll } from 'vitest';
|
|
20
|
+
import { mkdirSync, rmSync, statSync, writeFileSync } from 'node:fs';
|
|
21
|
+
import { createHash } from 'node:crypto';
|
|
22
|
+
import { tmpdir } from 'node:os';
|
|
23
|
+
import { join } from 'node:path';
|
|
24
|
+
|
|
25
|
+
const POLL_MS = 200;
|
|
26
|
+
const WAIT_MS = 300_000;
|
|
27
|
+
const STALE_MS = 600_000;
|
|
28
|
+
|
|
29
|
+
function lockDir(nodeId: string): string {
|
|
30
|
+
const key = createHash('sha256').update(`${process.env.OPENWOP_BASE_URL ?? ''}\u0000${nodeId}`).digest('hex').slice(0, 24);
|
|
31
|
+
return join(tmpdir(), `openwop-mock-node-${key}`);
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
async function acquire(dir: string, nodeId: string): Promise<void> {
|
|
35
|
+
const deadline = Date.now() + WAIT_MS;
|
|
36
|
+
for (;;) {
|
|
37
|
+
try {
|
|
38
|
+
mkdirSync(dir);
|
|
39
|
+
writeFileSync(join(dir, 'owner'), `${process.pid} ${nodeId} ${new Date().toISOString()}\n`);
|
|
40
|
+
return;
|
|
41
|
+
} catch (e) {
|
|
42
|
+
if ((e as NodeJS.ErrnoException).code !== 'EEXIST') throw e;
|
|
43
|
+
try {
|
|
44
|
+
if (Date.now() - statSync(dir).mtimeMs > STALE_MS) { rmSync(dir, { recursive: true, force: true }); continue; }
|
|
45
|
+
} catch { continue; }
|
|
46
|
+
if (Date.now() > deadline) throw new Error(`mock-node lock for ${nodeId} not acquired within ${WAIT_MS / 1000}s (${dir}) — another scenario file still holds it`);
|
|
47
|
+
await new Promise((r) => setTimeout(r, POLL_MS));
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
/** Hold the mock-AI program slot for these node ids for the duration of the calling file. */
|
|
53
|
+
export function holdMockNodes(...nodeIds: string[]): void {
|
|
54
|
+
const held: string[] = [];
|
|
55
|
+
beforeAll(async () => {
|
|
56
|
+
if (!process.env.OPENWOP_BASE_URL) return; // server-free run: nothing to isolate
|
|
57
|
+
for (const id of [...new Set(nodeIds)].sort()) {
|
|
58
|
+
const dir = lockDir(id);
|
|
59
|
+
await acquire(dir, id);
|
|
60
|
+
held.push(dir);
|
|
61
|
+
}
|
|
62
|
+
}, WAIT_MS + 10_000);
|
|
63
|
+
afterAll(() => {
|
|
64
|
+
for (const dir of held.splice(0).reverse()) rmSync(dir, { recursive: true, force: true });
|
|
65
|
+
});
|
|
66
|
+
}
|
|
@@ -34,7 +34,11 @@ import { isFixtureAdvertised } from '../lib/fixtures.js';
|
|
|
34
34
|
import { capabilityFamily } from '../lib/discovery-capabilities.js';
|
|
35
35
|
import { req } from '../lib/requirement-ids.js';
|
|
36
36
|
import { softSkip, blockedDespiteAssertions } from '../lib/soft-skip.js';
|
|
37
|
-
import { readDispatchBudgets } from '../lib/mock-ai-budgets.js';
|
|
37
|
+
import { readDispatchBudgets, truncationBudgetVerdict } from '../lib/mock-ai-budgets.js';
|
|
38
|
+
import { holdMockNodes } from '../lib/mock-node-lock.js';
|
|
39
|
+
|
|
40
|
+
// The mock-AI program seam is keyed by node id; files sharing a fixture's node run one at a time.
|
|
41
|
+
holdMockNodes('truncated-structured-call', 'retry-attempted-structured-call');
|
|
38
42
|
|
|
39
43
|
const HTTP_SKIP = !process.env.OPENWOP_BASE_URL;
|
|
40
44
|
/**
|
|
@@ -159,16 +163,21 @@ describe.skipIf(HTTP_SKIP)('envelope-completion-distinguishes-truncation: trunca
|
|
|
159
163
|
]);
|
|
160
164
|
if (seed.status === 404) return softSkip('blocked', 'precondition not met — `seed.status === 404` returned early (seam, prior step, or fixture unavailable)');
|
|
161
165
|
|
|
162
|
-
await startRunAndRead(TRUNCATED_FIXTURE);
|
|
163
|
-
|
|
164
|
-
|
|
166
|
+
const run = await startRunAndRead(TRUNCATED_FIXTURE);
|
|
167
|
+
if (run === null) return softSkip('blocked', 'precondition not met — the run create or event-log read failed');
|
|
168
|
+
// RFC 0033 §B: MAY retry, and the retry budget SHOULD exceed the first
|
|
169
|
+
// attempt's. Compared through the dispatch-budgets seam; an unmet SHOULD is
|
|
170
|
+
// recorded, not failed (suite 2.42.9; this leg asserted `> 50` as a MUST).
|
|
171
|
+
const v = await truncationBudgetVerdict(NODE_OF[TRUNCATED_FIXTURE] as string);
|
|
172
|
+
if (v.kind === 'skip') return softSkip(v.skip, v.reason);
|
|
173
|
+
if (v.kind === 'unmet') return softSkip('inapplicable', `RFC 0033 §B SHOULD not met — the retry carried maxTokens ${v.retry} after the first attempt carried ${v.first}; recorded, not failed`);
|
|
165
174
|
expect(
|
|
166
|
-
|
|
167
|
-
req('openwop.it.envelope-completion-distinguishes-truncation.truncation-retry-budget-strictly-greater-than-initial-rfc-0033-b-truncationbudge',
|
|
175
|
+
v.retry,
|
|
176
|
+
req('openwop.it.envelope-completion-distinguishes-truncation.truncation-retry-budget-strictly-greater-than-initial-rfc-0033-b-truncationbudge',
|
|
168
177
|
'RFCS/0033-envelope-completion-contract.md §B',
|
|
169
|
-
|
|
178
|
+
`the truncation retry's budget SHOULD exceed the first attempt's (first ${v.first}, retry ${v.retry})`,
|
|
170
179
|
),
|
|
171
|
-
).toBeGreaterThan(
|
|
180
|
+
).toBeGreaterThan(v.first);
|
|
172
181
|
});
|
|
173
182
|
});
|
|
174
183
|
|
|
@@ -117,6 +117,10 @@ import { isFixtureAdvertised } from '../lib/fixtures.js';
|
|
|
117
117
|
import { capabilityFamily } from '../lib/discovery-capabilities.js';
|
|
118
118
|
import { req } from '../lib/requirement-ids.js';
|
|
119
119
|
import { softSkip, blockedDespiteAssertions } from '../lib/soft-skip.js';
|
|
120
|
+
import { holdMockNodes } from '../lib/mock-node-lock.js';
|
|
121
|
+
|
|
122
|
+
// The mock-AI program seam is keyed by node id; files sharing a fixture's node run one at a time.
|
|
123
|
+
holdMockNodes('retry-attempted-structured-call');
|
|
120
124
|
|
|
121
125
|
const FIXTURE = 'conformance-envelope-retry-attempted';
|
|
122
126
|
const NODE_ID = 'retry-attempted-structured-call';
|
|
@@ -22,6 +22,11 @@ import { pollUntilTerminal } from '../lib/polling.js';
|
|
|
22
22
|
import { isFixtureAdvertised } from '../lib/fixtures.js';
|
|
23
23
|
import { req } from '../lib/requirement-ids.js';
|
|
24
24
|
import { softSkip, blockedDespiteAssertions } from '../lib/soft-skip.js';
|
|
25
|
+
import { holdMockNodes } from '../lib/mock-node-lock.js';
|
|
26
|
+
import { truncationBudgetVerdict } from '../lib/mock-ai-budgets.js';
|
|
27
|
+
|
|
28
|
+
// The mock-AI program seam is keyed by node id; files sharing a fixture's node run one at a time.
|
|
29
|
+
holdMockNodes('truncated-structured-call');
|
|
25
30
|
|
|
26
31
|
const HTTP_SKIP = !process.env.OPENWOP_BASE_URL;
|
|
27
32
|
const FIXTURE = 'conformance-envelope-truncated';
|
|
@@ -50,12 +55,6 @@ async function startRunAndRead(): Promise<{ events: RunEvent[]; terminal: unknow
|
|
|
50
55
|
return { events, terminal };
|
|
51
56
|
}
|
|
52
57
|
|
|
53
|
-
async function lastBudget(): Promise<number | null> {
|
|
54
|
-
const res = await driver.get(`/v1/host/sample/test/mock-ai/last-dispatch-budget?nodeId=${encodeURIComponent(NODE_ID)}`);
|
|
55
|
-
if (res.status !== 200) return null;
|
|
56
|
-
return (res.json as { maxTokens?: number | null }).maxTokens ?? null;
|
|
57
|
-
}
|
|
58
|
-
|
|
59
58
|
describe.skipIf(HTTP_SKIP)('envelope-truncated: runtime behavior (RFC 0032 §B.4 + RFC 0033 §B)', () => {
|
|
60
59
|
it('when mock returns stopReason: max_tokens on attempt 1, exactly one envelope.truncated event fires with stopReason: max_tokens', async () => {
|
|
61
60
|
if (!isFixtureAdvertised(FIXTURE)) return softSkip('inapplicable', 'capability or profile not advertised by this host — gate `!isFixtureAdvertised(FIXTURE)` returned early');
|
|
@@ -112,19 +111,20 @@ describe.skipIf(HTTP_SKIP)('envelope-truncated: runtime behavior (RFC 0032 §B.4
|
|
|
112
111
|
|
|
113
112
|
const result = await startRunAndRead();
|
|
114
113
|
if (result === null) return softSkip('blocked', 'precondition not met — `result === null` returned early (seam, prior step, or fixture unavailable)');
|
|
115
|
-
//
|
|
116
|
-
//
|
|
117
|
-
//
|
|
118
|
-
//
|
|
119
|
-
const
|
|
120
|
-
if (
|
|
114
|
+
// RFC 0033 §B: the retry budget SHOULD exceed the first attempt's (a MAY retry,
|
|
115
|
+
// a SHOULD increase). Suite 2.42.9 compares the budgets the provider actually
|
|
116
|
+
// received, and records an unmet SHOULD rather than failing it; until then this
|
|
117
|
+
// leg asserted `> 50` (the fixture value) as a MUST.
|
|
118
|
+
const v = await truncationBudgetVerdict(NODE_ID);
|
|
119
|
+
if (v.kind === 'skip') return softSkip(v.skip, v.reason);
|
|
120
|
+
if (v.kind === 'unmet') return softSkip('inapplicable', `RFC 0033 §B SHOULD not met — the retry carried maxTokens ${v.retry} after the first attempt carried ${v.first}; recorded, not failed`);
|
|
121
121
|
expect(
|
|
122
|
-
|
|
123
|
-
req('openwop.it.envelope-truncated.retry-attempt-receives-a-maxtokens-value-strictly-greater-than-the-previous-atte',
|
|
122
|
+
v.retry,
|
|
123
|
+
req('openwop.it.envelope-truncated.retry-attempt-receives-a-maxtokens-value-strictly-greater-than-the-previous-atte',
|
|
124
124
|
'RFCS/0033-envelope-completion-contract.md §B',
|
|
125
|
-
|
|
125
|
+
`the truncation retry's budget SHOULD exceed the first attempt's (first ${v.first}, retry ${v.retry})`,
|
|
126
126
|
),
|
|
127
|
-
).toBeGreaterThan(
|
|
127
|
+
).toBeGreaterThan(v.first);
|
|
128
128
|
});
|
|
129
129
|
|
|
130
130
|
it('run terminates `completed` after the second attempt succeeds', async () => {
|
|
@@ -234,6 +234,11 @@ describe('webhook delivery shape is per-contract (webhooks.md §Delivery, versio
|
|
|
234
234
|
expect(owner !== null, req(ID, DOC, 'a major-2 run.started payload MUST carry the owner echo { tenant, subject } (identity.md §1; run-event-payloads runStarted.owner requires both)')).toBe(true);
|
|
235
235
|
expect(Object.keys(owner ?? {}).filter((k) => k === 'principal' || k === 'principalKind'), req(ID, DOC, 'a major-2 owner echo MUST NOT carry the v1 keys principal / principalKind — that is the fan-out forwarding the in-process dialect instead of projecting')).toEqual([]);
|
|
236
236
|
expect(typeof owner?.['tenant'] === 'string' && owner?.['subject'] !== undefined, req(ID, DOC, 'a major-2 owner echo carries tenant and subject')).toBe(true);
|
|
237
|
+
// webhooks.md §Delivery: `workspaceId` is present exactly when RunSnapshot.owner.workspace is,
|
|
238
|
+
// equal to it, and a host MUST NOT substitute its tenant id (or any placeholder) for an absent one.
|
|
239
|
+
const ws = owner?.['workspace'];
|
|
240
|
+
const sent = (d.envelope as Record<string, unknown>)['workspaceId'];
|
|
241
|
+
expect(ws === undefined ? sent === undefined : sent === ws, req(ID, DOC, `workspaceId MUST be present exactly when owner.workspace is, and equal to it — nothing is substituted for an absent workspace (owner.workspace ${JSON.stringify(ws)}, delivered workspaceId ${JSON.stringify(sent)})`)).toBe(true);
|
|
237
242
|
});
|
|
238
243
|
|
|
239
244
|
it('a major-1 subscriber still receives the v1 rendering — the v1 wire does not move mid-overlap', async () => {
|