@openwop/openwop-conformance 2.46.43 → 2.47.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,14 @@
1
+ ## [2.47.0] — 2026-11-13 — Cache-key recipe 2, telemetry export, sabotage ledger
2
+
3
+ - **RFC 0260.** `v2-replay-llm-cache-key`: a server-free shape leg, an unaided implies-idempotency leg, and four host legs on `GET /runs/{runId}/effects` (key recorded, parameters in the key, stable, portable), gated on `replay.llmCacheKeyRecipe` and the `conformance-llm-cache-key` fixture. Judges in `lib/llm-cache-key-2.ts`.
4
+ - **RFC 0262.** `v2-telemetry-export`: a server-free shape leg and five legs against the suite OTLP collector (run span, node span, `traceparent`, no secret in spans, GenAI attributes), gated on the `telemetry` family.
5
+ - **RFC 0263.** `v2-sabotage-ledger` (coherence) drives `check-accepted-predicate` rules 5 and 6 on fixture trees, with positive and negative controls.
6
+
7
+ ## [2.46.44] — 2026-11-08 — Bounded loopback and resolvedVia go Active
8
+
9
+ - **RFC 0258.** `v2-bounded-loopback`: a server-free shape leg and six host legs (re-arm, history from the log, `ApprovalData.loopback`, withdraw, fail on exhaustion, replay re-arm), gated on the `loopback` family and the `conformance-loopback-refine` / `-fail` fixtures. Judges in `lib/bounded-loopback-witness.ts`.
10
+ - **RFC 0259.** `v2-interrupt-resolved-via`: a server-free shape leg and six host legs on the `readInterruptToken` seam, gated on `interrupt.resolvedVia`.
11
+
1
12
  ## [2.46.43] — 2026-11-04 — Two RFCs filed as Draft: bounded loopback and resolvedVia
2
13
 
3
14
  - **No suite change.** RFCs 0258 and 0259 add gap rows, which change the packed `spec/v1/gaps.json` and `CORPUS-STAMP.json`; no scenario, fixture or schema moves.
package/README.md CHANGED
@@ -137,7 +137,7 @@ Each scenario file records its disposition and assertion count in `<report-dir>/
137
137
 
138
138
  ## What's Covered
139
139
 
140
- The current suite has 666 scenario files under `src/scenarios/`. By target major ([`scenario-majors.json`](./scenario-majors.json)): 226 files run against a v2 host and 455 against a v1 host; 15 of them run at both.
140
+ The current suite has 670 scenario files under `src/scenarios/`. By target major ([`scenario-majors.json`](./scenario-majors.json)): 230 files run against a v2 host and 455 against a v1 host; 15 of them run at both.
141
141
 
142
142
  The v2 scenarios (`src/scenarios/v2-*.test.ts`) cover, by area:
143
143
 
@@ -166,7 +166,7 @@ Three kinds of scenario share the tree:
166
166
  - **Host** checks drive a live host over HTTP.
167
167
  - **Gated** checks run only when the host advertises the family, profile or fixture they need, and otherwise record `inapplicable` or `blocked` with a reason. `--require-behavior` turns an advertised-but-unwitnessable behaviour into a failure.
168
168
 
169
- Current source tree: 666 scenario files. [`coverage.md`](./coverage.md) maps each spec document and every OpenAPI operation to its scenarios. [`CHANGELOG.md`](./CHANGELOG.md) records when each scenario landed.
169
+ Current source tree: 670 scenario files. [`coverage.md`](./coverage.md) maps each spec document and every OpenAPI operation to its scenarios. [`CHANGELOG.md`](./CHANGELOG.md) records when each scenario landed.
170
170
 
171
171
  ## How the 2.x suite is built
172
172
 
@@ -1,5 +1,5 @@
1
1
  {
2
2
  "package": "@openwop/spec-artifacts",
3
- "version": "2.46.43",
4
- "stampSha256": "35bbc415dd07e23421b1ce986ea81f727c03bcfa9259c80fc96373180a7d712d"
3
+ "version": "2.47.0",
4
+ "stampSha256": "f03caded5ddba2a0fd12bf7b6a93995789afdd4ae5ffd56d0cfcde8701f7a657"
5
5
  }
@@ -0,0 +1,23 @@
1
+ {
2
+ "id": "conformance-llm-cache-key",
3
+ "name": "Conformance: LLM cache key recipe 2 (RFC 0260)",
4
+ "version": "1.0",
5
+ "description": "Fixture for `v2-replay-llm-cache-key.test.ts` (RFC 0260; the rules are the `replay.llmCacheKeyRecipe` facet description and the effect-ledger `cacheKey` description). One node, `conformance.callai.cache-key` (conformance-RESERVED). It makes exactly one `ctx.callAI` call through the host's conformance mock provider, whose provider id is `conformance-mock` and model id `conformance-mock-1`. The call sends one message `{role: \"user\", content: inputs.request.prompt}`, no tools, no toolChoice, no responseSchema, and exactly the parameters in `inputs.request.params` (keys from recipe 2's named set: `temperature`, `maxTokens`, `stop`, `seed`, `topP`, `topK`, `frequencyPenalty`, `presencePenalty`), nothing more. The mock replies `ok`. The host records the call on `GET /runs/{runId}/effects` under this node's id, carrying `cacheKey`. The node completes with `outputs.text`, the reply. A host that does not advertise `replay.llmCacheKeyRecipe` MUST NOT advertise this fixture.",
6
+ "nodes": [
7
+ {
8
+ "id": "llm-call",
9
+ "typeId": "conformance.callai.cache-key",
10
+ "name": "One ctx.callAI call through the conformance mock",
11
+ "position": { "x": 0, "y": 0 },
12
+ "config": {},
13
+ "inputs": {}
14
+ }
15
+ ],
16
+ "edges": [],
17
+ "triggers": [{ "id": "manual", "type": "manual", "enabled": true }],
18
+ "variables": [
19
+ { "name": "request", "type": "object" }
20
+ ],
21
+ "metadata": { "tags": ["conformance", "replay", "llmCacheKeyRecipe"] },
22
+ "settings": { "timeout": 0 }
23
+ }
@@ -0,0 +1,92 @@
1
+ {
2
+ "id": "conformance-loopback-refine-fail",
3
+ "name": "Conformance: Bounded loopback (refine, fail on exhaustion)",
4
+ "version": "1.0",
5
+ "description": "RFC 0258 §D. The same graph with maxIterations 1 and onExhausted: fail. The first refine loops; a refine on the exhausted second visit emits workflow.loopback-limit, then cap.breached {kind: loopback-iterations}, and fails the run with loop_limit_exceeded naming gate. Conformance scenario: v2-bounded-loopback.test.ts.",
6
+ "nodes": [
7
+ {
8
+ "id": "draft",
9
+ "typeId": "core.identity",
10
+ "name": "Draft",
11
+ "position": {
12
+ "x": 0,
13
+ "y": 0
14
+ },
15
+ "config": {},
16
+ "inputs": {}
17
+ },
18
+ {
19
+ "id": "gate",
20
+ "typeId": "core.approvalGate",
21
+ "name": "Review",
22
+ "position": {
23
+ "x": 200,
24
+ "y": 0
25
+ },
26
+ "config": {
27
+ "title": "Conformance loopback review (conformance-loopback-refine-fail)",
28
+ "description": "Conformance suite — refine to loop back to draft.",
29
+ "actions": [
30
+ "accept",
31
+ "reject",
32
+ "refine"
33
+ ]
34
+ },
35
+ "inputs": {}
36
+ },
37
+ {
38
+ "id": "apply",
39
+ "typeId": "core.noop",
40
+ "name": "Apply",
41
+ "position": {
42
+ "x": 400,
43
+ "y": 0
44
+ },
45
+ "config": {},
46
+ "inputs": {}
47
+ }
48
+ ],
49
+ "edges": [
50
+ {
51
+ "id": "draft-gate",
52
+ "sourceNodeId": "draft",
53
+ "targetNodeId": "gate"
54
+ },
55
+ {
56
+ "id": "again",
57
+ "sourceNodeId": "gate",
58
+ "targetNodeId": "draft",
59
+ "loopback": {
60
+ "on": "refine",
61
+ "maxIterations": 1,
62
+ "historyInput": "rounds",
63
+ "onExhausted": "fail"
64
+ }
65
+ },
66
+ {
67
+ "id": "gate-apply",
68
+ "sourceNodeId": "gate",
69
+ "targetNodeId": "apply",
70
+ "triggerRule": "all_success"
71
+ }
72
+ ],
73
+ "triggers": [
74
+ {
75
+ "id": "manual",
76
+ "type": "manual",
77
+ "enabled": true
78
+ }
79
+ ],
80
+ "variables": [],
81
+ "metadata": {
82
+ "tags": [
83
+ "conformance",
84
+ "hitl",
85
+ "loopback",
86
+ "v2-only"
87
+ ]
88
+ },
89
+ "settings": {
90
+ "timeout": 0
91
+ }
92
+ }
@@ -0,0 +1,91 @@
1
+ {
2
+ "id": "conformance-loopback-refine",
3
+ "name": "Conformance: Bounded loopback (refine, withdraw)",
4
+ "version": "1.0",
5
+ "description": "RFC 0258. draft (core.identity, echoes its inputs) → gate (core.approvalGate, accept/reject/refine) → apply (core.noop), plus a loopback edge gate → draft {on: refine, maxIterations: 2, historyInput: rounds}. Each refine re-runs draft and asks gate again; draft echoes the round history it received. After two refines the third visit omits refine (onExhausted defaults to withdraw), and accept runs apply once. Conformance scenario: v2-bounded-loopback.test.ts.",
6
+ "nodes": [
7
+ {
8
+ "id": "draft",
9
+ "typeId": "core.identity",
10
+ "name": "Draft",
11
+ "position": {
12
+ "x": 0,
13
+ "y": 0
14
+ },
15
+ "config": {},
16
+ "inputs": {}
17
+ },
18
+ {
19
+ "id": "gate",
20
+ "typeId": "core.approvalGate",
21
+ "name": "Review",
22
+ "position": {
23
+ "x": 200,
24
+ "y": 0
25
+ },
26
+ "config": {
27
+ "title": "Conformance loopback review (conformance-loopback-refine)",
28
+ "description": "Conformance suite — refine to loop back to draft.",
29
+ "actions": [
30
+ "accept",
31
+ "reject",
32
+ "refine"
33
+ ]
34
+ },
35
+ "inputs": {}
36
+ },
37
+ {
38
+ "id": "apply",
39
+ "typeId": "core.noop",
40
+ "name": "Apply",
41
+ "position": {
42
+ "x": 400,
43
+ "y": 0
44
+ },
45
+ "config": {},
46
+ "inputs": {}
47
+ }
48
+ ],
49
+ "edges": [
50
+ {
51
+ "id": "draft-gate",
52
+ "sourceNodeId": "draft",
53
+ "targetNodeId": "gate"
54
+ },
55
+ {
56
+ "id": "again",
57
+ "sourceNodeId": "gate",
58
+ "targetNodeId": "draft",
59
+ "loopback": {
60
+ "on": "refine",
61
+ "maxIterations": 2,
62
+ "historyInput": "rounds"
63
+ }
64
+ },
65
+ {
66
+ "id": "gate-apply",
67
+ "sourceNodeId": "gate",
68
+ "targetNodeId": "apply",
69
+ "triggerRule": "all_success"
70
+ }
71
+ ],
72
+ "triggers": [
73
+ {
74
+ "id": "manual",
75
+ "type": "manual",
76
+ "enabled": true
77
+ }
78
+ ],
79
+ "variables": [],
80
+ "metadata": {
81
+ "tags": [
82
+ "conformance",
83
+ "hitl",
84
+ "loopback",
85
+ "v2-only"
86
+ ]
87
+ },
88
+ "settings": {
89
+ "timeout": 0
90
+ }
91
+ }
package/fixtures.md CHANGED
@@ -55,6 +55,8 @@ All fixtures MUST advertise:
55
55
  | Error route only | `conformance-error-route-only` | RFC 0257 — the only terminal sits behind `any_failed`; it is skipped with no condition anywhere | `completed` with `skippedTerminals` | < 1s |
56
56
  | All terminals skipped | `conformance-all-terminals-skipped` | RFC 0257 — a `truthy` edge on `payload.ok` declines (`{ok:false}`) or fires (`{ok:true}`) | `completed`, with or without `skippedTerminals` | < 1s |
57
57
  | Sub-workflow (child reaches no terminal) | `conformance-subworkflow-skip-parent` | RFC 0257 §B — a `core.subWorkflow` child completes with `skippedTerminals`; the parent continues | `completed` | < 2s |
58
+ | Bounded loopback (withdraw) | `conformance-loopback-refine` | RFC 0258 — a refine loops `gate` back to `draft` with the round history; after two refines `refine` is withdrawn | `completed` after accept | unbounded (suspends) |
59
+ | Bounded loopback (fail) | `conformance-loopback-refine-fail` | RFC 0258 §D — `maxIterations: 1`, `onExhausted: fail`; a refine on the exhausted visit fails the run | `failed` (`loop_limit_exceeded`) | unbounded (suspends) |
58
60
  | Approval (timeout) | `conformance-approval-timeout` | RFC 0223 G2 — a 1500 ms `timeoutMs`, no `onTimeout`: the host's timer resolves the gate rejected | `failed` (`approval_rejected`) after the timeout | ~1.5 s + the host's sweep |
59
61
  | Approval (timeout, onTimeout approve) | `conformance-approval-timeout-approve` | openwop#1696 — a 1500 ms `timeoutMs` with `onTimeout: approve`: a timeout never grants, so the gate still resolves rejected | `failed` (`approval_rejected`) after the timeout | ~1.5 s + the host's sweep |
60
62
  | Approval (reject loops back) | `conformance-approval-reject-loopback` | openwop#1697 — a rejected gate routed back to itself through `revise` is asked again under a new `key` | `waiting-approval` again after one reject (the suite cancels) | unbounded (suspends) |
@@ -148,6 +150,7 @@ All fixtures MUST advertise:
148
150
  | v2 fs Sandbox Probe | `conformance-fs-probe` | `storage.md` §`fs` — one `core.conformance.fs-probe` node (conformance-RESERVED) calls the host's own `ctx.fs.read` / `write` with the run's `path`, unchanged; a rejection fails the node with its code and details. Operator symlink `<sandboxRoot>/conformance/escape-link` points outside the root. See §"The storage probe fixtures" | `completed` inside the root; `failed` (`forbidden`, `details.reason: path-outside-sandbox`) for an escape | ≤ 30s |
149
151
  | v2 tableStorage Schema Probe | `conformance-table-schema-probe` | `storage.md` §`tableStorage` — one `core.conformance.table-schema-probe` node (conformance-RESERVED) creates a table `{ k: string, n: number }` with the host's own `ctx.storage.table`, then inserts a well-typed row (`control`), a row whose `n` is a string (`insert-mistyped`), or updates `n` to a string (`update-mistyped`). See §"The tableStorage schema probe fixture" | `completed` for `control`; `failed` (`validation_error`, `details.service: tableStorage`) for a mistyped write | ≤ 30s |
150
152
  | v2 ctx.callAI Attempt Accounting | `conformance-callai-attempt-accounting` | RFC 0248 — one `conformance.callai.echo-attempts` node (conformance-RESERVED) that makes one `ctx.callAI` call through the host's conformance mock, serves the run's `script`, and echoes the result's `attempts` (`v2-callai-attempt-accounting`) | each case run completes or fails as its script dictates | ≤ 30s |
153
+ | v2 LLM Cache Key | `conformance-llm-cache-key` | RFC 0260 — one `conformance.callai.cache-key` node (conformance-RESERVED) making one `ctx.callAI` call through the suite's mock provider with the run's prompt and params unchanged (`v2-replay-llm-cache-key`) | completes; the call's effect record carries `cacheKey` | ≤ 30s |
151
154
  | v2 safeFetch SSRF Probe | `conformance-safefetch-probe` | `host-services.md` §`httpClient` — one `core.conformance.safefetch-probe` node (conformance-RESERVED) calls the host's own `ctx.http.safeFetch(url)` with the run's `url`, unchanged; completes with the response `status` only; a rejection fails the node with its code and details. See §"The safeFetch probe fixture" | `failed` (`egress_denied`, `details.reason: ssrf-blocked`) for a refused target | ≤ 60s |
152
155
  | A2A Task Roundtrip | `conformance-a2a-task-roundtrip` | Track 6 — host consumes the conformance suite's synthetic A2A peer; covers drift points #3 (`AUTH_REQUIRED`) and #4 (`REJECTED`). Node `core.conformance.a2a-invoke` is a **conformance-RESERVED** typeId (renamed 2026-08-16 from `core.a2a.invoke`, a multi-segment `core.*` id no host shipped and the corpus never defined): a host that consumes A2A peers MUST map it to its A2A bridge node (a host-extension node — `a2a-integration.md` §"State projection"); a host that does not consume A2A MUST NOT advertise this fixture | `failed` or `waiting-input` (per `driftScenario` input) | ≤ 30s |
153
156
  | WASM Pack Roundtrip | `conformance-wasm-pack-roundtrip` | RFC 0008 — invokes `vendor.openwop.rust-hello.greet` (loaded WASM pack); exercises required exports + at least one import | `completed` | ≤ 10s |
@@ -308,6 +311,29 @@ The `messages`-mode stream fixture (AI token streaming) is covered by the determ
308
311
  4. With `{ok: true}`, `act` MUST run and the run MUST end `run.completed` with no `skippedTerminals`.
309
312
  - **Terminal status**: `completed` on both inputs.
310
313
 
314
+ ### `conformance-loopback-refine`
315
+
316
+ - **Tagged** `v2-only`: `fixtures-valid` validates it against `schemas/v2/workflow-definition.schema.json`, because the frozen v1 wire cannot carry `loopback`.
317
+ - **Purpose**: witness RFC 0258 §§C–D: the re-arm, the round history and withdrawal on exhaustion.
318
+ - **Inputs**: none.
319
+ - **Behavior**:
320
+ 1. `draft` (`core.identity`) echoes its inputs. Its only input is the loopback's `historyInput` port `rounds`, so its `node.completed` `outputs` is `{rounds: [...]}`: `[]` on the first visit, then one `{round, outputs, refineFeedback}` entry per round, oldest first.
321
+ 2. `gate` (`core.approvalGate`, actions accept/reject/refine) suspends. Each request's `ApprovalData.loopback` reads `{round, maxIterations: 2, exhausted}`.
322
+ 3. A `refine` with `refineFeedback` completes `gate` with the marker `loopback {edgeId: "again", targetNodeId: "draft", round, rearmed: ["draft", "gate"]}`; `draft` runs again and `gate` is asked again under a new key. `apply` MUST NOT start.
323
+ 4. After two refines, the third request omits `refine`, and a `refine` resume answers `400 validation_error` with the run still `waiting-approval`.
324
+ 5. `accept` completes `gate`; `apply` runs once.
325
+ - **Terminal status**: `completed`.
326
+ - **Requires**: the `loopback` family with `maxIterations >= 2`.
327
+
328
+ ### `conformance-loopback-refine-fail`
329
+
330
+ - **Tagged** `v2-only`: `fixtures-valid` validates it against `schemas/v2/workflow-definition.schema.json`, because the frozen v1 wire cannot carry `loopback`.
331
+ - **Purpose**: witness RFC 0258 §D `onExhausted: fail`.
332
+ - **Inputs**: none.
333
+ - **Behavior**: the same graph with `maxIterations: 1` and `onExhausted: "fail"`. The first refine loops. A refine on the exhausted second visit emits `workflow.loopback-limit {nodeId: "gate", iterations: 1, limit: 1}`, then `cap.breached {kind: "loopback-iterations", limit: 1, observed: 2}`, then `run.failed` with `loop_limit_exceeded` and `failedNodeId: "gate"`.
334
+ - **Terminal status**: `failed`.
335
+ - **Requires**: the `loopback` family.
336
+
311
337
  ### `conformance-subworkflow-skip-parent`
312
338
 
313
339
  - **Purpose**: witness RFC 0257 §B (`spec/v2/core/execution.md` §`subWorkflow`, Output).
@@ -899,6 +925,8 @@ For `v2-tool-catalog-annotations`' unclassified-MCP leg, a host that advertises
899
925
 
900
926
  `conformance-callai-attempt-accounting` is what `v2-callai-attempt-accounting` drives. Its node, `conformance.callai.echo-attempts`, is conformance-RESERVED. A host that advertises `aiProviders.callAccounting` maps it to a node that makes exactly one `ctx.callAI` call through the host's conformance mock provider. It does no counting of its own. A host that does not advertise `callAccounting` MUST NOT advertise the fixture.
901
927
 
928
+ `conformance-llm-cache-key` is what `v2-replay-llm-cache-key` drives. Its node, `conformance.callai.cache-key`, is conformance-RESERVED. A host that advertises `replay.llmCacheKeyRecipe` maps it to a node that makes exactly one `ctx.callAI` call through the host's conformance mock provider, passing the run's prompt and parameters unchanged, so the suite can recompute the recipe-2 key from the inputs it sent. A host that does not advertise `llmCacheKeyRecipe` MUST NOT advertise the fixture.
929
+
902
930
  **Inputs.** `request` is passed to `ctx.callAI` unchanged: `prompt`, and optionally `responseSchema`, `maxTokens` and `retry`. `script` is the mock's replies.
903
931
 
904
932
  **The mock.** It serves `script` to the call's provider requests in order, one entry per request, and repeats the last entry past the end. Each entry is `{ kind, outputTokens?, usage?, structuredMode? }`:
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@openwop/openwop-conformance",
3
- "version": "2.46.43",
3
+ "version": "2.47.0",
4
4
  "description": "Black-box conformance suite for OpenWOP hosts (v2 current; v1 scenarios retained for the frozen v1 tree).",
5
5
  "repository": {
6
6
  "type": "git",
@@ -56,6 +56,6 @@
56
56
  "@openwop/spec-artifacts": "file:../spec-artifacts"
57
57
  },
58
58
  "peerDependencies": {
59
- "@openwop/spec-artifacts": "2.46.43"
59
+ "@openwop/spec-artifacts": "2.47.0"
60
60
  }
61
61
  }