@ggui-ai/protocol-conformance 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -3
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +10 -1
- package/dist/fixtures/index.d.ts +7 -0
- package/dist/fixtures/index.d.ts.map +1 -1
- package/dist/fixtures/index.js +7 -0
- package/dist/index.d.ts +4 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +11 -0
- package/dist/refusal-envelope-conformance/cases/refuse-after-fix-caller.json +27 -0
- package/dist/refusal-envelope-conformance/cases/refuse-after-fix-owner-with-balance.json +29 -0
- package/dist/refusal-envelope-conformance/cases/refuse-later.json +27 -0
- package/dist/refusal-envelope-conformance/cases/refuse-never.json +27 -0
- package/dist/refusal-envelope-conformance/cases/refuse-next-period.json +27 -0
- package/dist/refusal-envelope-conformance/cases/refuse-non-render-surface.json +12 -0
- package/dist/refusal-envelope-conformance/index.d.ts +108 -0
- package/dist/refusal-envelope-conformance/index.d.ts.map +1 -0
- package/dist/refusal-envelope-conformance/index.js +138 -0
- package/dist/registry-completeness/index.d.ts +105 -0
- package/dist/registry-completeness/index.d.ts.map +1 -0
- package/dist/registry-completeness/index.js +110 -0
- package/dist/reporter.d.ts.map +1 -1
- package/dist/reporter.js +26 -9
- package/dist/run-conformance.d.ts +29 -0
- package/dist/run-conformance.d.ts.map +1 -1
- package/dist/run-conformance.js +102 -4
- package/package.json +15 -5
package/README.md
CHANGED
|
@@ -51,13 +51,16 @@ One declared grading gap on the action loop: `action-ack` proves the append half
|
|
|
51
51
|
|
|
52
52
|
The behavioral fixture catalog above asserts what an implementation _does_ over the wire. Some protocol obligations aren't transport-observable at all — they are deterministic _validation functions_: given an input, accept or reject it. SPEC §7.7.2's gadget obligations are exactly this. Modeling them as WebSocket fixtures would mean faking wire frames the protocol never emits; instead the kit ships **pure-function catalogs** — accept/reject cases graded against a caller-supplied function, with no host, render, or transport.
|
|
53
53
|
|
|
54
|
-
Each catalog ships its cases as raw JSON so a non-TypeScript implementer can grade their own implementation, and each runner takes the implementation as a callback — the kit never hard-binds a concrete one.
|
|
54
|
+
Each catalog ships its cases as raw JSON so a non-TypeScript implementer can grade their own implementation, and each runner takes the implementation as a callback — the kit never hard-binds a concrete one. Eight catalogs ship today:
|
|
55
55
|
|
|
56
56
|
- **`@ggui-ai/protocol-conformance/schema-conformance`** — which `DataContract.clientCapabilities` payloads a conformant parser MUST accept / reject (the gadget wire shape).
|
|
57
57
|
- **`@ggui-ai/protocol-conformance/registration-conformance`** — which `(contract, appGadgets, appPublicEnv)` triples the push-time gadget gate stack MUST accept / reject, and with which precise SPEC §7.9 reject code (`gadget_not_registered` / `gadget_package_mismatch` / `gadget_public_env_missing` / `duplicate_gadget_hook`).
|
|
58
58
|
- **`@ggui-ai/protocol-conformance/resolution-conformance`** — which bundle + style URLs the server MUST compute for a gadget descriptor's transport fields (`bundleHost` precedence, default host, loopback `http` scheme).
|
|
59
59
|
- **`@ggui-ai/protocol-conformance/binding-conformance`** — which MCP tool-binding set a manifest resolves to (a declared `mcpTools` list wins entirely; blueprints without it derive bare tool-name entries from contract tool names) and which artifacts the registry `tool=` / `server=` search filters MUST match (SPEC §7.7.4.1).
|
|
60
60
|
- **`@ggui-ai/protocol-conformance/props-schema-conformance`** — the schema-precise render arbiter (SPEC §2.3.2): the enforced-props-schema builder (drift), its RFC 8785 sha256 hash, the grammar-safe profile verdict, and returned-schema AUTHORITY (sample validity under the case's own `propsSchema`, including the 2026-08-19 out-of-vocabulary enum incident pinned as a permanent sample).
|
|
61
|
+
- **`@ggui-ai/protocol-conformance/host-helper-conformance`** — which render coordinates a HOST-HELPER library MUST extract from a `tools/call` result, and which malformed results it MUST decline rather than guess at.
|
|
62
|
+
- **`@ggui-ai/protocol-conformance/refusal-envelope-conformance`** — the tool result a server MUST project for a PRE-GENERATION refusal (SPEC §7.1's refused arm): an in-result `isError` (never a throw), `content[0].text` leading with the code, `structuredContent` carrying `{outcome: 'refused', refusal}` and nothing else, no `_meta` — and no envelope at all for a registered code whose surfaces exclude the render gate.
|
|
63
|
+
- **`@ggui-ai/protocol-conformance/registry-completeness`** — the structural obligations a deployment's closed refusal-code registry MUST satisfy: `surfaces` non-empty, `code === key`, every `after-fix` entry naming who acts (`fixBy`), and `retry` inside the closed four-value set. Grades DATA rather than a function — the caller passes their registry.
|
|
61
64
|
|
|
62
65
|
```ts
|
|
63
66
|
import { runSchemaConformance } from "@ggui-ai/protocol-conformance/schema-conformance";
|
|
@@ -77,7 +80,7 @@ if (schema.failed.length + gate.failed.length + urls.failed.length > 0) {
|
|
|
77
80
|
}
|
|
78
81
|
```
|
|
79
82
|
|
|
80
|
-
The `schema-conformance` meta-test binds its catalog to the live `@ggui-ai/protocol` `clientCapabilitiesSpecSchema` — a drift-catch if the wire schema diverges from the §7.7.2 obligations the catalog freezes. The `registration-conformance`, `resolution-conformance`, and `binding-conformance` meta-tests verify catalog coherence against faithful in-test implementations (the kit stays vendor-neutral — it does not depend on a server implementation); grading the _shipping_ gate / resolver is an implementation-side test that drives the corresponding runner.
|
|
83
|
+
The `schema-conformance` meta-test binds its catalog to the live `@ggui-ai/protocol` `clientCapabilitiesSpecSchema` — a drift-catch if the wire schema diverges from the §7.7.2 obligations the catalog freezes. The `registration-conformance`, `resolution-conformance`, and `binding-conformance` meta-tests verify catalog coherence against faithful in-test implementations (the kit stays vendor-neutral — it does not depend on a server implementation); grading the _shipping_ gate / resolver is an implementation-side test that drives the corresponding runner. The two ggui#786 catalogs split the same way, and for the same reason: `registry-completeness` binds the LIVE `@ggui-ai/protocol` `PRE_GENERATION_REFUSAL_CODES` (the obligation IS a protocol declaration, so the drift-catch belongs kit-side, like `schema-conformance`), while `refusal-envelope-conformance` grades a kit-local reference projector (the obligation is an implementation behaviour, like `resolution-conformance`). Two of that ruling's six registry checks read repo source — the no-auto-retry docstring rule and the grep for code literals minted outside the registry file — and are therefore NOT in the kit: it publishes only `dist`, so a source walk could never run for an adopter. They live in `@ggui-ai/protocol`'s own suite. That split is declared, not hidden.
|
|
81
84
|
|
|
82
85
|
## Resource-read conformance — the MCP binding
|
|
83
86
|
|
|
@@ -116,7 +119,7 @@ It deliberately does **not** grade: the order in which a substrate-less server a
|
|
|
116
119
|
|
|
117
120
|
What is actually graded against shipping code today, and what is still awaiting a driver. This is an inventory, not a roadmap — a mechanism is listed as proven only if a test in the repo drives it against the real implementation on every push.
|
|
118
121
|
|
|
119
|
-
Every test below is a plain `*.test.ts` inside a package's `src/`, so it runs in that package's ordinary vitest suite. The ggui repo's per-push CI runs `turbo run test` across all non-e2e packages, so **all
|
|
122
|
+
Every test below is a plain `*.test.ts` inside a package's `src/`, so it runs in that package's ordinary vitest suite. The ggui repo's per-push CI runs `turbo run test` across all non-e2e packages, so **all ten rows below are gated on every push** — the kit needs no dedicated workflow, and adopters wiring it into their own CI need nothing more than their existing unit-test task.
|
|
120
123
|
|
|
121
124
|
| Obligation | Graded against | Driver |
|
|
122
125
|
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- |
|
|
@@ -128,6 +131,8 @@ Every test below is a plain `*.test.ts` inside a package's `src/`, so it runs in
|
|
|
128
131
|
| `resources/read` typed failures + mount invariant | the shipping `registerGguiRenderResourceTemplate` in `@ggui-ai/mcp-server` | `mcp-server/src/mcp-apps-outbound.resource-read.conformance.test.ts` — **12 pass / 0 skip / 0 fail** |
|
|
129
132
|
| Tool-binding resolution — declared wins, contract derivation (§7.7.4.1) | the shipping `resolveMcpToolBindings` in `@ggui-ai/artifact-manifest` | `artifact-manifest/src/mcp-tool-bindings.conformance.test.ts` |
|
|
130
133
|
| Tool-binding search filters — `tool=` / `server=` semantics (§7.7.4.1) | the shipping `matchesMcpToolFilters` in `@ggui-ai/registry-core` | `registry-core/src/mcp-tool-filters.conformance.test.ts` |
|
|
134
|
+
| Pre-generation refusal envelope (§7.1, refused arm) | the shipping `ggui_render` refusal projection in `@ggui-ai/mcp-server-handlers` | `mcp-server-handlers/src/renders/render-refusal-projection.conformance.test.ts` |
|
|
135
|
+
| Refusal-registry completeness (ggui#786 ruling 5b) | the shipping `PRE_GENERATION_REFUSAL_CODES` in `@ggui-ai/protocol` | this kit's own `src/registry-completeness/registry-completeness.test.ts` |
|
|
131
136
|
|
|
132
137
|
Both WS drivers pin their pass set _and_ their skip set as exact sets, and the resource-read driver pins its skip set as exactly empty. A fixture silently degrading to a skip fails the build, and so does a skipped fixture that quietly starts passing — a skip set that can grow unnoticed is a false gate, and re-pinning it is meant to be a deliberate act.
|
|
133
138
|
|
package/dist/cli.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AA+BA,OAAO,EAAkB,KAAK,iBAAiB,EAAE,MAAM,sBAAsB,CAAC;AAC9E,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAM7C,UAAU,UAAU;IAClB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IAClC,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAC1B,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;CACxB;AAED,iBAAS,SAAS,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,GAAG,UAAU,CA8CtD;AAED,iBAAS,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,UAAU,CAY3C;
|
|
1
|
+
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AA+BA,OAAO,EAAkB,KAAK,iBAAiB,EAAE,MAAM,sBAAsB,CAAC;AAC9E,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAM7C,UAAU,UAAU;IAClB,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IAClC,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAC1B,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;CACxB;AAED,iBAAS,SAAS,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,GAAG,UAAU,CA8CtD;AAED,iBAAS,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,UAAU,CAY3C;AA6CD,iBAAe,IAAI,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,GAAG,OAAO,CAAC,MAAM,CAAC,CA0D5D;AAED;;;;;;;GAOG;AACH,iBAAS,iBAAiB,CAAC,MAAM,EAAE,iBAAiB,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAI/D;AA+BD,OAAO,EAAE,iBAAiB,EAAE,IAAI,EAAE,SAAS,EAAE,SAAS,EAAE,CAAC"}
|
package/dist/cli.js
CHANGED
|
@@ -109,11 +109,20 @@ Required:
|
|
|
109
109
|
session-cookie:COOKIE — Cookie header
|
|
110
110
|
|
|
111
111
|
Options:
|
|
112
|
-
--only <fixture-name> Run only this fixture. May be repeated.
|
|
112
|
+
--only <fixture-name> Run only this fixture. May be repeated. Also
|
|
113
|
+
filters the pure-function catalog rows, whose
|
|
114
|
+
names are prefixed <catalog>/<case>.
|
|
113
115
|
--timeout-ms <N> Per-fixture observation window. Default 2000.
|
|
114
116
|
--verbose, -v Print failure details + skip reasons at the end.
|
|
115
117
|
--help, -h Show this help.
|
|
116
118
|
|
|
119
|
+
Pure-function catalogs (refusal-envelope, registry-completeness):
|
|
120
|
+
These grade an in-process function and a data table, not the wire, so
|
|
121
|
+
the CLI cannot supply them over a URL. They appear on every scorecard
|
|
122
|
+
as SKIPPED with the reason — an ungraded obligation stays visible
|
|
123
|
+
rather than vanishing. To GRADE them, call runConformance()
|
|
124
|
+
programmatically with refusalProjector / refusalRegistry.
|
|
125
|
+
|
|
117
126
|
Exit codes:
|
|
118
127
|
0 — at least one fixture executed and none failed.
|
|
119
128
|
1 — at least one fixture failed.
|
package/dist/fixtures/index.d.ts
CHANGED
|
@@ -36,6 +36,13 @@
|
|
|
36
36
|
* frame to drive — so they do NOT belong in this behavioral fixture
|
|
37
37
|
* catalog.
|
|
38
38
|
*
|
|
39
|
+
* The same is true of SPEC §7.1's PRE-GENERATION refusal obligations
|
|
40
|
+
* (ggui#786): what a server projects for a refusal
|
|
41
|
+
* (`../refusal-envelope-conformance`) and what a deployment's closed
|
|
42
|
+
* refusal registry must satisfy (`../registry-completeness`) are a
|
|
43
|
+
* projection and a data check. Neither is WebSocket-observable, so
|
|
44
|
+
* neither is registered below.
|
|
45
|
+
*
|
|
39
46
|
* ## An MCP-binding driver EXISTS for `resources/read` — `../resource-read-conformance`
|
|
40
47
|
*
|
|
41
48
|
* This catalog used to declare the MCP-binding driver as missing. It is
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/fixtures/index.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/fixtures/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAsGG;AAEH,OAAO,EAAE,yBAAyB,EAAE,MAAM,+BAA+B,CAAC;AAC1E,OAAO,EAAE,qBAAqB,EAAE,MAAM,2BAA2B,CAAC;AAClE,OAAO,EAAE,mBAAmB,EAAE,MAAM,yBAAyB,CAAC;AAC9D,OAAO,EAAE,gCAAgC,EAAE,MAAM,uCAAuC,CAAC;AACzF,OAAO,EAAE,8BAA8B,EAAE,MAAM,qCAAqC,CAAC;AACrF,OAAO,EAAE,wBAAwB,EAAE,MAAM,8BAA8B,CAAC;AAExE,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,aAAa,CAAC;AAQ5C;;;;;GAKG;AACH,MAAM,MAAM,YAAY,GACpB,oBAAoB,GACpB,gBAAgB,GAChB,cAAc,GACd,4BAA4B,GAC5B,0BAA0B,GAC1B,mBAAmB,GACnB,CAAC,MAAM,GAAG,EAAE,CAAC,CAAC;AAElB;;;;;;;GAOG;AACH,eAAO,MAAM,kBAAkB,EAAE,QAAQ,CAAC,MAAM,CAAC,YAAY,EAAE,SAAS,QAAQ,EAAE,CAAC,CAOlF,CAAC;AAEF;;;;GAIG;AACH,eAAO,MAAM,WAAW,EAAE,SAAS,QAAQ,EAGwB,CAAC"}
|
package/dist/fixtures/index.js
CHANGED
|
@@ -36,6 +36,13 @@
|
|
|
36
36
|
* frame to drive — so they do NOT belong in this behavioral fixture
|
|
37
37
|
* catalog.
|
|
38
38
|
*
|
|
39
|
+
* The same is true of SPEC §7.1's PRE-GENERATION refusal obligations
|
|
40
|
+
* (ggui#786): what a server projects for a refusal
|
|
41
|
+
* (`../refusal-envelope-conformance`) and what a deployment's closed
|
|
42
|
+
* refusal registry must satisfy (`../registry-completeness`) are a
|
|
43
|
+
* projection and a data check. Neither is WebSocket-observable, so
|
|
44
|
+
* neither is registered below.
|
|
45
|
+
*
|
|
39
46
|
* ## An MCP-binding driver EXISTS for `resources/read` — `../resource-read-conformance`
|
|
40
47
|
*
|
|
41
48
|
* This catalog used to declare the MCP-binding driver as missing. It is
|
package/dist/index.d.ts
CHANGED
|
@@ -24,6 +24,10 @@ export { gadgetResolutionCases, runResolutionConformance, } from './resolution-c
|
|
|
24
24
|
export type { GadgetUrlEntry, ResolvedGadgetUrls, ResolutionConformanceCase, ResolutionConformanceMismatch, ResolutionConformanceResult, } from './resolution-conformance/index.js';
|
|
25
25
|
export { bindingFilterCases, bindingResolutionCases, runBindingFilterCases, runBindingResolutionCases, } from './binding-conformance/index.js';
|
|
26
26
|
export type { BindingFilterCase, BindingFilterMismatch, BindingFilterResult, BindingManifestEntry, BindingRejectCode, BindingResolutionCase, BindingResolutionMismatch, BindingResolutionOutcome, BindingResolutionResult, McpToolBindingDecl, McpToolFilterDecl, } from './binding-conformance/index.js';
|
|
27
|
+
export { refusalEnvelopeCases, runRefusalEnvelopeConformance, } from './refusal-envelope-conformance/index.js';
|
|
28
|
+
export type { PreGenerationRefusalInput, ProjectedRefusalResult, RefusalEnvelopeConformanceCase, RefusalEnvelopeConformanceResult, RefusalEnvelopeMismatch, } from './refusal-envelope-conformance/index.js';
|
|
29
|
+
export { registryCompletenessPins, runRegistryCompletenessConformance, } from './registry-completeness/index.js';
|
|
30
|
+
export type { RefusalRegistryRow, RefusalRegistryView, RegistryCompletenessMismatch, RegistryCompletenessPin, RegistryCompletenessResult, } from './registry-completeness/index.js';
|
|
27
31
|
export { declaresDeliveryChannel, renderLocatorUri, resourceReadCases, runResourceReadConformance, GGUI_RENDER_RESOURCE_URI, } from './resource-read-conformance/index.js';
|
|
28
32
|
export type { DurableSubstrateWiring, JsonRpcErrorFrame, PreparedResourceReadScenario, ResourceReadCaller, ResourceReadConformanceCase, ResourceReadConformanceFailure, ResourceReadConformanceResult, ResourceReadConformanceSkip, ResourceReadErrorCodeDecl, ResourceReadExpectation, ResourceReadKey, ResourceReadLocator, ResourceReadOutcome, ResourceReadProbe, ResourceReadRenderMeta, ResourceReadScenario, ResourceReadScenarioDriver, ResourceReadSeed, ResourceReadServerShape, RunResourceReadConformanceOptions, } from './resource-read-conformance/index.js';
|
|
29
33
|
export { loadFixture, listFixtures, loadAllFixtures } from './loader.js';
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAKH,YAAY,EAEV,eAAe,EACf,wBAAwB,EACxB,sBAAsB,EACtB,UAAU,EAEV,QAAQ,EACR,qBAAqB,EAErB,SAAS,EACT,qBAAqB,EACrB,mBAAmB,EACnB,cAAc,EACd,uBAAuB,EACvB,yBAAyB,EACzB,gCAAgC,EAChC,gBAAgB,EAEhB,YAAY,EAEZ,gBAAgB,EAChB,iBAAiB,EACjB,kBAAkB,EAClB,oBAAoB,EACpB,wBAAwB,EACxB,wBAAwB,EACxB,uBAAuB,EACvB,mBAAmB,EACnB,oBAAoB,EACpB,YAAY,EACZ,eAAe,EAEf,aAAa,EACb,sBAAsB,GACvB,MAAM,YAAY,CAAC;AAMpB,YAAY,EACV,eAAe,EAEf,SAAS,IAAI,aAAa,EAC1B,sBAAsB,EACtB,wBAAwB,EACxB,0BAA0B,EAC1B,iCAAiC,EACjC,iBAAiB,EAEjB,YAAY,IAAI,gBAAgB,GACjC,MAAM,uBAAuB,CAAC;AAK/B,OAAO,EACL,WAAW,EACX,kBAAkB,EAClB,yBAAyB,EACzB,qBAAqB,EACrB,mBAAmB,EACnB,gCAAgC,EAChC,8BAA8B,EAC9B,wBAAwB,GACzB,MAAM,qBAAqB,CAAC;AAC7B,YAAY,EAAE,YAAY,EAAE,MAAM,qBAAqB,CAAC;AASxD,OAAO,EACL,qBAAqB,EACrB,oBAAoB,GACrB,MAAM,+BAA+B,CAAC;AACvC,YAAY,EACV,qBAAqB,EACrB,yBAAyB,EACzB,uBAAuB,GACxB,MAAM,+BAA+B,CAAC;AAOvC,OAAO,EACL,2BAA2B,EAC3B,yBAAyB,GAC1B,MAAM,qCAAqC,CAAC;AAC7C,YAAY,EACV,0BAA0B,EAC1B,6BAA6B,EAC7B,4BAA4B,EAC5B,4BAA4B,EAC5B,yBAAyB,GAC1B,MAAM,qCAAqC,CAAC;AAK7C,OAAO,EACL,uBAAuB,EACvB,0BAA0B,GAC3B,MAAM,qCAAqC,CAAC;AAC7C,YAAY,EACV,oBAAoB,EACpB,WAAW,EACX,2BAA2B,EAC3B,+BAA+B,EAC/B,6BAA6B,GAC9B,MAAM,qCAAqC,CAAC;AAK7C,OAAO,EACL,qBAAqB,EACrB,wBAAwB,GACzB,MAAM,mCAAmC,CAAC;AAC3C,YAAY,EACV,cAAc,EACd,kBAAkB,EAClB,yBAAyB,EACzB,6BAA6B,EAC7B,2BAA2B,GAC5B,MAAM,mCAAmC,CAAC;AAM3C,OAAO,EACL,kBAAkB,EAClB,sBAAsB,EACtB,qBAAqB,EACrB,yBAAyB,GAC1B,MAAM,gCAAgC,CAAC;AACxC,YAAY,EACV,iBAAiB,EACjB,qBAAqB,EACrB,mBAAmB,EACnB,oBAAoB,EACpB,iBAAiB,EACjB,qBAAqB,EACrB,yBAAyB,EACzB,wBAAwB,EACxB,uBAAuB,EACvB,kBAAkB,EAClB,iBAAiB,GAClB,MAAM,gCAAgC,CAAC;
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AAKH,YAAY,EAEV,eAAe,EACf,wBAAwB,EACxB,sBAAsB,EACtB,UAAU,EAEV,QAAQ,EACR,qBAAqB,EAErB,SAAS,EACT,qBAAqB,EACrB,mBAAmB,EACnB,cAAc,EACd,uBAAuB,EACvB,yBAAyB,EACzB,gCAAgC,EAChC,gBAAgB,EAEhB,YAAY,EAEZ,gBAAgB,EAChB,iBAAiB,EACjB,kBAAkB,EAClB,oBAAoB,EACpB,wBAAwB,EACxB,wBAAwB,EACxB,uBAAuB,EACvB,mBAAmB,EACnB,oBAAoB,EACpB,YAAY,EACZ,eAAe,EAEf,aAAa,EACb,sBAAsB,GACvB,MAAM,YAAY,CAAC;AAMpB,YAAY,EACV,eAAe,EAEf,SAAS,IAAI,aAAa,EAC1B,sBAAsB,EACtB,wBAAwB,EACxB,0BAA0B,EAC1B,iCAAiC,EACjC,iBAAiB,EAEjB,YAAY,IAAI,gBAAgB,GACjC,MAAM,uBAAuB,CAAC;AAK/B,OAAO,EACL,WAAW,EACX,kBAAkB,EAClB,yBAAyB,EACzB,qBAAqB,EACrB,mBAAmB,EACnB,gCAAgC,EAChC,8BAA8B,EAC9B,wBAAwB,GACzB,MAAM,qBAAqB,CAAC;AAC7B,YAAY,EAAE,YAAY,EAAE,MAAM,qBAAqB,CAAC;AASxD,OAAO,EACL,qBAAqB,EACrB,oBAAoB,GACrB,MAAM,+BAA+B,CAAC;AACvC,YAAY,EACV,qBAAqB,EACrB,yBAAyB,EACzB,uBAAuB,GACxB,MAAM,+BAA+B,CAAC;AAOvC,OAAO,EACL,2BAA2B,EAC3B,yBAAyB,GAC1B,MAAM,qCAAqC,CAAC;AAC7C,YAAY,EACV,0BAA0B,EAC1B,6BAA6B,EAC7B,4BAA4B,EAC5B,4BAA4B,EAC5B,yBAAyB,GAC1B,MAAM,qCAAqC,CAAC;AAK7C,OAAO,EACL,uBAAuB,EACvB,0BAA0B,GAC3B,MAAM,qCAAqC,CAAC;AAC7C,YAAY,EACV,oBAAoB,EACpB,WAAW,EACX,2BAA2B,EAC3B,+BAA+B,EAC/B,6BAA6B,GAC9B,MAAM,qCAAqC,CAAC;AAK7C,OAAO,EACL,qBAAqB,EACrB,wBAAwB,GACzB,MAAM,mCAAmC,CAAC;AAC3C,YAAY,EACV,cAAc,EACd,kBAAkB,EAClB,yBAAyB,EACzB,6BAA6B,EAC7B,2BAA2B,GAC5B,MAAM,mCAAmC,CAAC;AAM3C,OAAO,EACL,kBAAkB,EAClB,sBAAsB,EACtB,qBAAqB,EACrB,yBAAyB,GAC1B,MAAM,gCAAgC,CAAC;AACxC,YAAY,EACV,iBAAiB,EACjB,qBAAqB,EACrB,mBAAmB,EACnB,oBAAoB,EACpB,iBAAiB,EACjB,qBAAqB,EACrB,yBAAyB,EACzB,wBAAwB,EACxB,uBAAuB,EACvB,kBAAkB,EAClB,iBAAiB,GAClB,MAAM,gCAAgC,CAAC;AAOxC,OAAO,EACL,oBAAoB,EACpB,6BAA6B,GAC9B,MAAM,yCAAyC,CAAC;AACjD,YAAY,EACV,yBAAyB,EACzB,sBAAsB,EACtB,8BAA8B,EAC9B,gCAAgC,EAChC,uBAAuB,GACxB,MAAM,yCAAyC,CAAC;AAMjD,OAAO,EACL,wBAAwB,EACxB,kCAAkC,GACnC,MAAM,kCAAkC,CAAC;AAC1C,YAAY,EACV,kBAAkB,EAClB,mBAAmB,EACnB,4BAA4B,EAC5B,uBAAuB,EACvB,0BAA0B,GAC3B,MAAM,kCAAkC,CAAC;AAQ1C,OAAO,EACL,uBAAuB,EACvB,gBAAgB,EAChB,iBAAiB,EACjB,0BAA0B,EAC1B,wBAAwB,GACzB,MAAM,sCAAsC,CAAC;AAC9C,YAAY,EACV,sBAAsB,EACtB,iBAAiB,EACjB,4BAA4B,EAC5B,kBAAkB,EAClB,2BAA2B,EAC3B,8BAA8B,EAC9B,6BAA6B,EAC7B,2BAA2B,EAC3B,yBAAyB,EACzB,uBAAuB,EACvB,eAAe,EACf,mBAAmB,EACnB,mBAAmB,EACnB,iBAAiB,EACjB,sBAAsB,EACtB,oBAAoB,EACpB,0BAA0B,EAC1B,gBAAgB,EAChB,uBAAuB,EACvB,iCAAiC,GAClC,MAAM,sCAAsC,CAAC;AAG9C,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,aAAa,CAAC;AAGzE,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,YAAY,EACV,oBAAoB,EACpB,iBAAiB,EACjB,kBAAkB,EAClB,cAAc,EACd,mBAAmB,EAInB,qBAAqB,EACrB,gCAAgC,GACjC,MAAM,sBAAsB,CAAC;AAK9B,OAAO,EACL,qBAAqB,EACrB,eAAe,EACf,aAAa,EACb,cAAc,EACd,WAAW,GACZ,MAAM,eAAe,CAAC;AACvB,YAAY,EAAE,sBAAsB,EAAE,MAAM,eAAe,CAAC"}
|
package/dist/index.js
CHANGED
|
@@ -41,6 +41,17 @@ export { gadgetResolutionCases, runResolutionConformance, } from './resolution-c
|
|
|
41
41
|
// filter semantics (`tool=` / `server=` case-sensitive exact,
|
|
42
42
|
// AND-composed per entry).
|
|
43
43
|
export { bindingFilterCases, bindingResolutionCases, runBindingFilterCases, runBindingResolutionCases, } from './binding-conformance/index.js';
|
|
44
|
+
// Refusal-envelope conformance — the tool result a server MUST emit for
|
|
45
|
+
// a PRE-GENERATION refusal (SPEC §7.1's refused arm): an in-result
|
|
46
|
+
// `isError`, the code leading the text, `{outcome, refusal}` and
|
|
47
|
+
// nothing else on structuredContent, no `_meta`, and no envelope at all
|
|
48
|
+
// for a code whose surfaces exclude the render gate.
|
|
49
|
+
export { refusalEnvelopeCases, runRefusalEnvelopeConformance, } from './refusal-envelope-conformance/index.js';
|
|
50
|
+
// Registry-completeness — the structural obligations a deployment's
|
|
51
|
+
// closed refusal-code registry MUST satisfy. Grades DATA rather than a
|
|
52
|
+
// function: `surfaces` non-empty, `code === key`, every `after-fix`
|
|
53
|
+
// entry naming who acts, and `retry` inside the closed four-value set.
|
|
54
|
+
export { registryCompletenessPins, runRegistryCompletenessConformance, } from './registry-completeness/index.js';
|
|
44
55
|
// Resource-read conformance — the kit's first MCP-binding driver, and
|
|
45
56
|
// it binds `resources/read` ONLY (`tools/call` has none). A read of a
|
|
46
57
|
// render locator has exactly two exits: a result whose contents
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "refuse-after-fix-caller",
|
|
3
|
+
"description": "An after-fix refusal the CALLER can act on itself: the projector emits an isError result whose text leads with the code, carries outcome 'refused' plus the refusal and nothing else, and exposes no _meta. This is the only fixBy class an agent may retry on its own.",
|
|
4
|
+
"refusal": {
|
|
5
|
+
"code": "model_not_in_tier",
|
|
6
|
+
"message": "the requested model is not among those this app may use",
|
|
7
|
+
"fix": "pick a model the app is allowed to use and call again with the same handshakeId",
|
|
8
|
+
"retry": "after-fix",
|
|
9
|
+
"handshake": "intact"
|
|
10
|
+
},
|
|
11
|
+
"expect": {
|
|
12
|
+
"isError": true,
|
|
13
|
+
"text": "model_not_in_tier: the requested model is not among those this app may use pick a model the app is allowed to use and call again with the same handshakeId",
|
|
14
|
+
"structuredContent": {
|
|
15
|
+
"outcome": "refused",
|
|
16
|
+
"refusal": {
|
|
17
|
+
"code": "model_not_in_tier",
|
|
18
|
+
"message": "the requested model is not among those this app may use",
|
|
19
|
+
"fix": "pick a model the app is allowed to use and call again with the same handshakeId",
|
|
20
|
+
"retry": "after-fix",
|
|
21
|
+
"handshake": "intact"
|
|
22
|
+
}
|
|
23
|
+
},
|
|
24
|
+
"hasMeta": false,
|
|
25
|
+
"identityFields": []
|
|
26
|
+
}
|
|
27
|
+
}
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "refuse-after-fix-owner-with-balance",
|
|
3
|
+
"description": "An after-fix refusal whose fix belongs to the app OWNER, carrying the optional balance the refusing check read. The optional field rides the envelope unchanged \u2014 a projector must not drop it, and an agent must not auto-retry a fix it cannot perform.",
|
|
4
|
+
"refusal": {
|
|
5
|
+
"code": "insufficient_credit",
|
|
6
|
+
"message": "the balance this deployment funds the app from is exhausted",
|
|
7
|
+
"fix": "the app owner adds funds; the same handshakeId is still valid afterwards",
|
|
8
|
+
"retry": "after-fix",
|
|
9
|
+
"handshake": "intact",
|
|
10
|
+
"balanceCentsAtCheck": 0
|
|
11
|
+
},
|
|
12
|
+
"expect": {
|
|
13
|
+
"isError": true,
|
|
14
|
+
"text": "insufficient_credit: the balance this deployment funds the app from is exhausted the app owner adds funds; the same handshakeId is still valid afterwards",
|
|
15
|
+
"structuredContent": {
|
|
16
|
+
"outcome": "refused",
|
|
17
|
+
"refusal": {
|
|
18
|
+
"code": "insufficient_credit",
|
|
19
|
+
"message": "the balance this deployment funds the app from is exhausted",
|
|
20
|
+
"fix": "the app owner adds funds; the same handshakeId is still valid afterwards",
|
|
21
|
+
"retry": "after-fix",
|
|
22
|
+
"handshake": "intact",
|
|
23
|
+
"balanceCentsAtCheck": 0
|
|
24
|
+
}
|
|
25
|
+
},
|
|
26
|
+
"hasMeta": false,
|
|
27
|
+
"identityFields": []
|
|
28
|
+
}
|
|
29
|
+
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "refuse-later",
|
|
3
|
+
"description": "A transient refusal: unbounded time rather than a period boundary. Handshake intact, so the SAME id is valid on the retry the fix invites \u2014 the case exists so a projector cannot special-case the retryable class.",
|
|
4
|
+
"refusal": {
|
|
5
|
+
"code": "issuer_rate_limited",
|
|
6
|
+
"message": "calls under this issuing identity are over the rate this deployment allows",
|
|
7
|
+
"fix": "retry shortly \u2014 time restores it",
|
|
8
|
+
"retry": "later",
|
|
9
|
+
"handshake": "intact"
|
|
10
|
+
},
|
|
11
|
+
"expect": {
|
|
12
|
+
"isError": true,
|
|
13
|
+
"text": "issuer_rate_limited: calls under this issuing identity are over the rate this deployment allows retry shortly \u2014 time restores it",
|
|
14
|
+
"structuredContent": {
|
|
15
|
+
"outcome": "refused",
|
|
16
|
+
"refusal": {
|
|
17
|
+
"code": "issuer_rate_limited",
|
|
18
|
+
"message": "calls under this issuing identity are over the rate this deployment allows",
|
|
19
|
+
"fix": "retry shortly \u2014 time restores it",
|
|
20
|
+
"retry": "later",
|
|
21
|
+
"handshake": "intact"
|
|
22
|
+
}
|
|
23
|
+
},
|
|
24
|
+
"hasMeta": false,
|
|
25
|
+
"identityFields": []
|
|
26
|
+
}
|
|
27
|
+
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "refuse-never",
|
|
3
|
+
"description": "A refusal no caller action can lift. It still projects to a normal isError tool result — a terminal refusal is a wire state, never a thrown error. Carried on `billing_mode_anomaly` since registry v9 retired `tier_unrecognized`; the graded property is the `never` retry class, not the particular code.",
|
|
4
|
+
"refusal": {
|
|
5
|
+
"code": "billing_mode_anomaly",
|
|
6
|
+
"message": "the app record declares a funding mode this deployment does not recognise",
|
|
7
|
+
"fix": "no caller or owner action restores rendering under this identity; the operator reconciles the record",
|
|
8
|
+
"retry": "never",
|
|
9
|
+
"handshake": "intact"
|
|
10
|
+
},
|
|
11
|
+
"expect": {
|
|
12
|
+
"isError": true,
|
|
13
|
+
"text": "billing_mode_anomaly: the app record declares a funding mode this deployment does not recognise no caller or owner action restores rendering under this identity; the operator reconciles the record",
|
|
14
|
+
"structuredContent": {
|
|
15
|
+
"outcome": "refused",
|
|
16
|
+
"refusal": {
|
|
17
|
+
"code": "billing_mode_anomaly",
|
|
18
|
+
"message": "the app record declares a funding mode this deployment does not recognise",
|
|
19
|
+
"fix": "no caller or owner action restores rendering under this identity; the operator reconciles the record",
|
|
20
|
+
"retry": "never",
|
|
21
|
+
"handshake": "intact"
|
|
22
|
+
}
|
|
23
|
+
},
|
|
24
|
+
"hasMeta": false,
|
|
25
|
+
"identityFields": []
|
|
26
|
+
}
|
|
27
|
+
}
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "refuse-next-period",
|
|
3
|
+
"description": "A next-period refusal names no party at all \u2014 time restores it. The envelope is otherwise identical, which is the point: the retry class is data on the wire, not a different shape.",
|
|
4
|
+
"refusal": {
|
|
5
|
+
"code": "hard_cap_exceeded",
|
|
6
|
+
"message": "the app reached the hard ceiling set for the current period",
|
|
7
|
+
"fix": "the ceiling resets at the start of the next period; no action restores it sooner",
|
|
8
|
+
"retry": "next-period",
|
|
9
|
+
"handshake": "intact"
|
|
10
|
+
},
|
|
11
|
+
"expect": {
|
|
12
|
+
"isError": true,
|
|
13
|
+
"text": "hard_cap_exceeded: the app reached the hard ceiling set for the current period the ceiling resets at the start of the next period; no action restores it sooner",
|
|
14
|
+
"structuredContent": {
|
|
15
|
+
"outcome": "refused",
|
|
16
|
+
"refusal": {
|
|
17
|
+
"code": "hard_cap_exceeded",
|
|
18
|
+
"message": "the app reached the hard ceiling set for the current period",
|
|
19
|
+
"fix": "the ceiling resets at the start of the next period; no action restores it sooner",
|
|
20
|
+
"retry": "next-period",
|
|
21
|
+
"handshake": "intact"
|
|
22
|
+
}
|
|
23
|
+
},
|
|
24
|
+
"hasMeta": false,
|
|
25
|
+
"identityFields": []
|
|
26
|
+
}
|
|
27
|
+
}
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "refuse-non-render-surface",
|
|
3
|
+
"description": "A REGISTERED code whose surfaces exclude the render gate. It has no render envelope at all, so a conformant projector produces nothing for it. Projecting it anyway is the drift the per-surface namespace rule forbids \u2014 this case is what makes that rule observable rather than prose.",
|
|
4
|
+
"refusal": {
|
|
5
|
+
"code": "subscription_exists",
|
|
6
|
+
"message": "an active plan already exists on this app",
|
|
7
|
+
"fix": "cancel the existing plan first, then call again",
|
|
8
|
+
"retry": "after-fix",
|
|
9
|
+
"handshake": "intact"
|
|
10
|
+
},
|
|
11
|
+
"expect": null
|
|
12
|
+
}
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* The refusal a deployment's gate returned, as the projector receives
|
|
3
|
+
* it. Authored here rather than imported from `@ggui-ai/protocol`:
|
|
4
|
+
* this is the kit's own vocabulary, and `code` is typed loosely on
|
|
5
|
+
* purpose so a catalog case can carry a registered code from ANY
|
|
6
|
+
* surface — including one that must not project.
|
|
7
|
+
*/
|
|
8
|
+
export interface PreGenerationRefusalInput {
|
|
9
|
+
/** A code from the deployment's refusal registry. */
|
|
10
|
+
readonly code: string;
|
|
11
|
+
/** Precise diagnostic — what was checked, against what. */
|
|
12
|
+
readonly message: string;
|
|
13
|
+
/** The one recovery step, addressed to the party that can take it. */
|
|
14
|
+
readonly fix: string;
|
|
15
|
+
/** How the call becomes possible again. */
|
|
16
|
+
readonly retry: string;
|
|
17
|
+
/** Always `intact` — a refusal reads nothing, so it consumes nothing. */
|
|
18
|
+
readonly handshake: string;
|
|
19
|
+
/** Present only when the refusing check read a balance. */
|
|
20
|
+
readonly balanceCentsAtCheck?: number;
|
|
21
|
+
}
|
|
22
|
+
/**
|
|
23
|
+
* The observable shape of the tool result a conformant server emits for
|
|
24
|
+
* a refusal. Deliberately flattened to the four facts the obligation is
|
|
25
|
+
* about, so an implementation in any language can report them without
|
|
26
|
+
* reproducing an MCP result object.
|
|
27
|
+
*/
|
|
28
|
+
export interface ProjectedRefusalResult {
|
|
29
|
+
/** MUST be true — a refusal is an in-result error, never a throw. */
|
|
30
|
+
readonly isError: boolean;
|
|
31
|
+
/** `content[0].text` — leads with the code, then message and fix. */
|
|
32
|
+
readonly text: string;
|
|
33
|
+
/** The whole `structuredContent`. */
|
|
34
|
+
readonly structuredContent: {
|
|
35
|
+
readonly outcome: string;
|
|
36
|
+
readonly refusal: PreGenerationRefusalInput;
|
|
37
|
+
};
|
|
38
|
+
/** Whether the result carries any `_meta`. MUST be false. */
|
|
39
|
+
readonly hasMeta: boolean;
|
|
40
|
+
/**
|
|
41
|
+
* Identity fields the projector put on the envelope. MUST be empty —
|
|
42
|
+
* a refusal commits nothing, so any identity field means the
|
|
43
|
+
* projection leaked state that does not exist.
|
|
44
|
+
*/
|
|
45
|
+
readonly identityFields: readonly string[];
|
|
46
|
+
}
|
|
47
|
+
/**
|
|
48
|
+
* One refusal-envelope case. Authored as JSON under `./cases/`,
|
|
49
|
+
* consumed via {@link refusalEnvelopeCases}, graded by
|
|
50
|
+
* {@link runRefusalEnvelopeConformance}.
|
|
51
|
+
*
|
|
52
|
+
* The shape IS the public API — additive changes only.
|
|
53
|
+
*/
|
|
54
|
+
export interface RefusalEnvelopeConformanceCase {
|
|
55
|
+
/** Unique case name. MUST match the JSON filename without `.json`. */
|
|
56
|
+
readonly name: string;
|
|
57
|
+
/** Which projection obligation this case proves. */
|
|
58
|
+
readonly description: string;
|
|
59
|
+
/** The refusal fed to the projector. */
|
|
60
|
+
readonly refusal: PreGenerationRefusalInput;
|
|
61
|
+
/**
|
|
62
|
+
* The result a spec-conformant projector MUST produce — or `null`
|
|
63
|
+
* when the code has no render envelope at all (its registered
|
|
64
|
+
* surfaces exclude the render gate).
|
|
65
|
+
*/
|
|
66
|
+
readonly expect: ProjectedRefusalResult | null;
|
|
67
|
+
}
|
|
68
|
+
/**
|
|
69
|
+
* Every refusal-envelope case the kit ships, in deterministic order:
|
|
70
|
+
* the projectable cases by retry class (after-fix, next-period, later,
|
|
71
|
+
* never), then the non-projectable surface case.
|
|
72
|
+
*/
|
|
73
|
+
export declare const refusalEnvelopeCases: readonly RefusalEnvelopeConformanceCase[];
|
|
74
|
+
/** One case the projector under test graded wrong. */
|
|
75
|
+
export interface RefusalEnvelopeMismatch {
|
|
76
|
+
readonly name: string;
|
|
77
|
+
/** What the catalog says a conformant projector MUST produce. */
|
|
78
|
+
readonly expected: ProjectedRefusalResult | null;
|
|
79
|
+
/**
|
|
80
|
+
* What the projector produced — or, when it THREW instead of
|
|
81
|
+
* returning, a one-line report of the throw. A gate that throws to
|
|
82
|
+
* refuse is non-conformant, so the throw is graded, never rethrown.
|
|
83
|
+
*/
|
|
84
|
+
readonly actual: ProjectedRefusalResult | null | string;
|
|
85
|
+
}
|
|
86
|
+
/** Outcome of grading a projector against the catalog. */
|
|
87
|
+
export interface RefusalEnvelopeConformanceResult {
|
|
88
|
+
/** Names of cases the projector graded correctly. */
|
|
89
|
+
readonly passed: readonly string[];
|
|
90
|
+
/** Cases the projector graded wrong — empty iff fully conformant. */
|
|
91
|
+
readonly failed: readonly RefusalEnvelopeMismatch[];
|
|
92
|
+
}
|
|
93
|
+
/**
|
|
94
|
+
* Grade a refusal projector against the catalog.
|
|
95
|
+
*
|
|
96
|
+
* `project` MUST be a pure computation of `(refusal) → result | null`,
|
|
97
|
+
* where `null` means "this code has no render envelope". The kit
|
|
98
|
+
* deliberately does NOT import a concrete projector — adopters pass
|
|
99
|
+
* their own. A conformant projector produces an empty `failed` array.
|
|
100
|
+
*
|
|
101
|
+
* `project` is invoked exactly once per case, in
|
|
102
|
+
* {@link refusalEnvelopeCases} order. It MUST NOT throw: throwing to
|
|
103
|
+
* refuse is precisely the non-conformance this catalog exists to
|
|
104
|
+
* catch, so a throw is recorded as a graded FAILURE on that case
|
|
105
|
+
* rather than escaping the run.
|
|
106
|
+
*/
|
|
107
|
+
export declare function runRefusalEnvelopeConformance(project: (refusal: PreGenerationRefusalInput) => ProjectedRefusalResult | null): RefusalEnvelopeConformanceResult;
|
|
108
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/refusal-envelope-conformance/index.ts"],"names":[],"mappings":"AA0DA;;;;;;GAMG;AACH,MAAM,WAAW,yBAAyB;IACxC,qDAAqD;IACrD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,2DAA2D;IAC3D,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,sEAAsE;IACtE,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;IACrB,2CAA2C;IAC3C,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,yEAAyE;IACzE,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,2DAA2D;IAC3D,QAAQ,CAAC,mBAAmB,CAAC,EAAE,MAAM,CAAC;CACvC;AAED;;;;;GAKG;AACH,MAAM,WAAW,sBAAsB;IACrC,qEAAqE;IACrE,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAC1B,qEAAqE;IACrE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,qCAAqC;IACrC,QAAQ,CAAC,iBAAiB,EAAE;QAC1B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;QACzB,QAAQ,CAAC,OAAO,EAAE,yBAAyB,CAAC;KAC7C,CAAC;IACF,6DAA6D;IAC7D,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAC1B;;;;OAIG;IACH,QAAQ,CAAC,cAAc,EAAE,SAAS,MAAM,EAAE,CAAC;CAC5C;AAED;;;;;;GAMG;AACH,MAAM,WAAW,8BAA8B;IAC7C,sEAAsE;IACtE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,oDAAoD;IACpD,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,wCAAwC;IACxC,QAAQ,CAAC,OAAO,EAAE,yBAAyB,CAAC;IAC5C;;;;OAIG;IACH,QAAQ,CAAC,MAAM,EAAE,sBAAsB,GAAG,IAAI,CAAC;CAChD;AAED;;;;GAIG;AACH,eAAO,MAAM,oBAAoB,EAAE,SAAS,8BAA8B,EAOzE,CAAC;AAEF,sDAAsD;AACtD,MAAM,WAAW,uBAAuB;IACtC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,iEAAiE;IACjE,QAAQ,CAAC,QAAQ,EAAE,sBAAsB,GAAG,IAAI,CAAC;IACjD;;;;OAIG;IACH,QAAQ,CAAC,MAAM,EAAE,sBAAsB,GAAG,IAAI,GAAG,MAAM,CAAC;CACzD;AAED,0DAA0D;AAC1D,MAAM,WAAW,gCAAgC;IAC/C,qDAAqD;IACrD,QAAQ,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IACnC,qEAAqE;IACrE,QAAQ,CAAC,MAAM,EAAE,SAAS,uBAAuB,EAAE,CAAC;CACrD;AAkCD;;;;;;;;;;;;;GAaG;AACH,wBAAgB,6BAA6B,CAC3C,OAAO,EAAE,CACP,OAAO,EAAE,yBAAyB,KAC/B,sBAAsB,GAAG,IAAI,GACjC,gCAAgC,CAwBlC"}
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Refusal-envelope conformance catalog — the projection obligation for a
|
|
3
|
+
* PRE-GENERATION refusal (SPEC §7.1's refused arm, ggui#786).
|
|
4
|
+
*
|
|
5
|
+
* ## Why this is a pure-function catalog
|
|
6
|
+
*
|
|
7
|
+
* Given a refusal a deployment's gate returned, the tool result a
|
|
8
|
+
* conformant server emits is a pure, deterministic projection — no
|
|
9
|
+
* render, no transport, no wire frame. So it gets the same treatment as
|
|
10
|
+
* the wire schema (`../schema-conformance`) and the gadget URL resolver
|
|
11
|
+
* (`../resolution-conformance`): a catalog of cases graded against a
|
|
12
|
+
* caller-supplied projector.
|
|
13
|
+
*
|
|
14
|
+
* The obligation, restated from SPEC §7.1 so an implementer never has
|
|
15
|
+
* to guess:
|
|
16
|
+
*
|
|
17
|
+
* - The result is an `isError: true` TOOL RESULT — never a thrown
|
|
18
|
+
* error, never a JSON-RPC error. A gate that throws instead of
|
|
19
|
+
* returning a refusal is non-conformant, which is why the runner
|
|
20
|
+
* grades a throwing projector as a failure on every case rather
|
|
21
|
+
* than letting the exception escape.
|
|
22
|
+
* - `content[0].text` LEADS with the code, then the message and the
|
|
23
|
+
* fix. A courtesy line for the model; the machine-readable code is
|
|
24
|
+
* the mechanism.
|
|
25
|
+
* - `structuredContent` is `{outcome: 'refused', refusal}` and
|
|
26
|
+
* NOTHING else. A refusal commits nothing, so no identity field
|
|
27
|
+
* may appear on it.
|
|
28
|
+
* - No `_meta`. A refusal exposes no mount affordance.
|
|
29
|
+
* - A code whose registered surfaces exclude the render gate has NO
|
|
30
|
+
* render envelope; a conformant projector produces none.
|
|
31
|
+
*
|
|
32
|
+
* ## Polyglot, decoupled
|
|
33
|
+
*
|
|
34
|
+
* Each case ships as raw JSON under `./cases/`.
|
|
35
|
+
* {@link runRefusalEnvelopeConformance} takes the projector as a
|
|
36
|
+
* callback — the kit never imports a concrete server here, so adopters
|
|
37
|
+
* drive their OWN. The reference binding
|
|
38
|
+
* (`./refusal-envelope-conformance.test.ts`) builds a faithful in-test
|
|
39
|
+
* projector from `@ggui-ai/protocol` primitives only; the drift-catch
|
|
40
|
+
* against a SHIPPING projector belongs implementation-side, in the
|
|
41
|
+
* package that owns it.
|
|
42
|
+
*
|
|
43
|
+
* ## Not a WS fixture
|
|
44
|
+
*
|
|
45
|
+
* These cases are deliberately NOT registered in `fixturesByContract`.
|
|
46
|
+
* The obligation is a projection, not a WebSocket-observable behavior,
|
|
47
|
+
* so registering it there would be a permanent skip on every run —
|
|
48
|
+
* exactly the false gate the kit's exact skip-set pinning prevents.
|
|
49
|
+
* `runConformance()` / the CLI / the reporter are untouched; the entry
|
|
50
|
+
* point is the programmatic runner below.
|
|
51
|
+
*/
|
|
52
|
+
import refuseAfterFixCaller from './cases/refuse-after-fix-caller.json' with { type: 'json' };
|
|
53
|
+
import refuseAfterFixOwnerWithBalance from './cases/refuse-after-fix-owner-with-balance.json' with { type: 'json' };
|
|
54
|
+
import refuseLater from './cases/refuse-later.json' with { type: 'json' };
|
|
55
|
+
import refuseNever from './cases/refuse-never.json' with { type: 'json' };
|
|
56
|
+
import refuseNextPeriod from './cases/refuse-next-period.json' with { type: 'json' };
|
|
57
|
+
import refuseNonRenderSurface from './cases/refuse-non-render-surface.json' with { type: 'json' };
|
|
58
|
+
/**
|
|
59
|
+
* Every refusal-envelope case the kit ships, in deterministic order:
|
|
60
|
+
* the projectable cases by retry class (after-fix, next-period, later,
|
|
61
|
+
* never), then the non-projectable surface case.
|
|
62
|
+
*/
|
|
63
|
+
export const refusalEnvelopeCases = [
|
|
64
|
+
refuseAfterFixCaller,
|
|
65
|
+
refuseAfterFixOwnerWithBalance,
|
|
66
|
+
refuseNextPeriod,
|
|
67
|
+
refuseLater,
|
|
68
|
+
refuseNever,
|
|
69
|
+
refuseNonRenderSurface,
|
|
70
|
+
];
|
|
71
|
+
/** Structural equality over plain JSON data. */
|
|
72
|
+
function jsonEqual(a, b) {
|
|
73
|
+
if (a === b)
|
|
74
|
+
return true;
|
|
75
|
+
if (typeof a !== 'object' || typeof b !== 'object' || a === null || b === null) {
|
|
76
|
+
return false;
|
|
77
|
+
}
|
|
78
|
+
if (Array.isArray(a) || Array.isArray(b)) {
|
|
79
|
+
if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length)
|
|
80
|
+
return false;
|
|
81
|
+
return a.every((item, index) => jsonEqual(item, b[index]));
|
|
82
|
+
}
|
|
83
|
+
const left = a;
|
|
84
|
+
const right = b;
|
|
85
|
+
const leftKeys = Object.keys(left).sort();
|
|
86
|
+
const rightKeys = Object.keys(right).sort();
|
|
87
|
+
if (!jsonEqual(leftKeys, rightKeys))
|
|
88
|
+
return false;
|
|
89
|
+
return leftKeys.every((key) => jsonEqual(left[key], right[key]));
|
|
90
|
+
}
|
|
91
|
+
function projectionEqual(a, b) {
|
|
92
|
+
if (a === null || b === null)
|
|
93
|
+
return a === b;
|
|
94
|
+
return (a.isError === b.isError &&
|
|
95
|
+
a.text === b.text &&
|
|
96
|
+
a.hasMeta === b.hasMeta &&
|
|
97
|
+
jsonEqual([...a.identityFields], [...b.identityFields]) &&
|
|
98
|
+
jsonEqual(a.structuredContent, b.structuredContent));
|
|
99
|
+
}
|
|
100
|
+
/**
|
|
101
|
+
* Grade a refusal projector against the catalog.
|
|
102
|
+
*
|
|
103
|
+
* `project` MUST be a pure computation of `(refusal) → result | null`,
|
|
104
|
+
* where `null` means "this code has no render envelope". The kit
|
|
105
|
+
* deliberately does NOT import a concrete projector — adopters pass
|
|
106
|
+
* their own. A conformant projector produces an empty `failed` array.
|
|
107
|
+
*
|
|
108
|
+
* `project` is invoked exactly once per case, in
|
|
109
|
+
* {@link refusalEnvelopeCases} order. It MUST NOT throw: throwing to
|
|
110
|
+
* refuse is precisely the non-conformance this catalog exists to
|
|
111
|
+
* catch, so a throw is recorded as a graded FAILURE on that case
|
|
112
|
+
* rather than escaping the run.
|
|
113
|
+
*/
|
|
114
|
+
export function runRefusalEnvelopeConformance(project) {
|
|
115
|
+
const passed = [];
|
|
116
|
+
const failed = [];
|
|
117
|
+
for (const testCase of refusalEnvelopeCases) {
|
|
118
|
+
let actual;
|
|
119
|
+
try {
|
|
120
|
+
actual = project(testCase.refusal);
|
|
121
|
+
}
|
|
122
|
+
catch (err) {
|
|
123
|
+
failed.push({
|
|
124
|
+
name: testCase.name,
|
|
125
|
+
expected: testCase.expect,
|
|
126
|
+
actual: `threw instead of returning a refusal envelope: ${err instanceof Error ? err.message : String(err)}`,
|
|
127
|
+
});
|
|
128
|
+
continue;
|
|
129
|
+
}
|
|
130
|
+
if (projectionEqual(actual, testCase.expect)) {
|
|
131
|
+
passed.push(testCase.name);
|
|
132
|
+
}
|
|
133
|
+
else {
|
|
134
|
+
failed.push({ name: testCase.name, expected: testCase.expect, actual });
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
return { passed, failed };
|
|
138
|
+
}
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Registry-completeness catalog — the structural obligations a
|
|
3
|
+
* deployment's refusal-code registry MUST satisfy (ggui#786, ruling
|
|
4
|
+
* 5b).
|
|
5
|
+
*
|
|
6
|
+
* ## Why this is a pure-function catalog, graded differently
|
|
7
|
+
*
|
|
8
|
+
* Unlike every other catalog here, this one grades DATA rather than a
|
|
9
|
+
* function: the closed set of refusal codes a deployment declares. The
|
|
10
|
+
* caller passes the registry; each pin inspects every row and reports
|
|
11
|
+
* the rows that violate it.
|
|
12
|
+
*
|
|
13
|
+
* ## The four PORTABLE pins, and the two that are deliberately absent
|
|
14
|
+
*
|
|
15
|
+
* Ruling 5b names six checks. Four are portable — a third-party adopter
|
|
16
|
+
* with their own registry can run them, and they are the four below.
|
|
17
|
+
*
|
|
18
|
+
* Two read repo SOURCE and therefore cannot ship as conformance: the
|
|
19
|
+
* docstring rule ("an agent MUST NOT auto-retry an after-fix refusal
|
|
20
|
+
* whose fixBy is not caller") and the grep for code literals minted
|
|
21
|
+
* outside the registry file. This package publishes only `dist` +
|
|
22
|
+
* `README.md`, so a source walk could never run for an adopter, and the
|
|
23
|
+
* files that rule names live outside this tree entirely. Both live in
|
|
24
|
+
* `@ggui-ai/protocol`'s own suite, where the invariant is owned and the
|
|
25
|
+
* paths are stable. The omission is DECLARED here rather than hidden.
|
|
26
|
+
*
|
|
27
|
+
* ## Why the `fixBy` pin is one-directional
|
|
28
|
+
*
|
|
29
|
+
* The pin below asserts only that an `after-fix` row NAMES who acts. It
|
|
30
|
+
* does not assert `fixBy` is absent elsewhere, because that is a naming
|
|
31
|
+
* discipline rather than a reader-observable obligation: a reader
|
|
32
|
+
* branches on `retry`, and a `fixBy` beside a `never` misleads a human
|
|
33
|
+
* without changing what any implementation does.
|
|
34
|
+
*
|
|
35
|
+
* ggui's own registry is stricter — `fixBy` is REQUIRED on `after-fix`,
|
|
36
|
+
* PERMITTED on `later` (the operator who restores a transiently
|
|
37
|
+
* unavailable surface) and FORBIDDEN on `next-period` and `never` — and
|
|
38
|
+
* enforces it at the type level in its definer, pinned in
|
|
39
|
+
* `@ggui-ai/protocol`'s own suite. Adopters are welcome to that rule;
|
|
40
|
+
* the kit does not make it a conformance obligation. The one thing that
|
|
41
|
+
* must never happen, and IS graded here, is an `after-fix` row that
|
|
42
|
+
* does not say WHO acts — the one an agent could misread as its own to
|
|
43
|
+
* retry.
|
|
44
|
+
*/
|
|
45
|
+
/** One registry row, in the kit's own loose vocabulary. */
|
|
46
|
+
export interface RefusalRegistryRow {
|
|
47
|
+
/** MUST equal the row's key in the registry. */
|
|
48
|
+
readonly code: string;
|
|
49
|
+
/** MUST be non-empty — the surfaces this state is refused on. */
|
|
50
|
+
readonly surfaces: readonly string[];
|
|
51
|
+
/** MUST be one of the four recognized retry classes. */
|
|
52
|
+
readonly retry: string;
|
|
53
|
+
/** Who acts. REQUIRED when `retry` is `after-fix`. */
|
|
54
|
+
readonly fixBy?: string | undefined;
|
|
55
|
+
/** Where the code is produced. */
|
|
56
|
+
readonly emitter: string;
|
|
57
|
+
/** What state the code names. */
|
|
58
|
+
readonly description: string;
|
|
59
|
+
}
|
|
60
|
+
/**
|
|
61
|
+
* A registry as the kit reads it: code → row. Decoupled from
|
|
62
|
+
* `@ggui-ai/protocol`'s own types on purpose — an adopter's registry is
|
|
63
|
+
* graded through this view, and a deliberately broken registry must be
|
|
64
|
+
* expressible for the negative cases.
|
|
65
|
+
*/
|
|
66
|
+
export type RefusalRegistryView = {
|
|
67
|
+
readonly [code: string]: RefusalRegistryRow;
|
|
68
|
+
};
|
|
69
|
+
/** One structural pin, applied to every row of a registry. */
|
|
70
|
+
export interface RegistryCompletenessPin {
|
|
71
|
+
/** Unique pin name. */
|
|
72
|
+
readonly name: string;
|
|
73
|
+
/** The obligation this pin fixes in place. */
|
|
74
|
+
readonly description: string;
|
|
75
|
+
/**
|
|
76
|
+
* Report why `row` (declared under `code`) violates the pin, or
|
|
77
|
+
* `null` when it satisfies it.
|
|
78
|
+
*/
|
|
79
|
+
readonly violation: (code: string, row: RefusalRegistryRow) => string | null;
|
|
80
|
+
}
|
|
81
|
+
/** Every portable pin the kit ships, in deterministic order. */
|
|
82
|
+
export declare const registryCompletenessPins: readonly RegistryCompletenessPin[];
|
|
83
|
+
/** One pin a registry failed, with every offending row named. */
|
|
84
|
+
export interface RegistryCompletenessMismatch {
|
|
85
|
+
readonly name: string;
|
|
86
|
+
readonly description: string;
|
|
87
|
+
/** One entry per violating row: `<code>: <why>`. */
|
|
88
|
+
readonly violations: readonly string[];
|
|
89
|
+
}
|
|
90
|
+
/** Outcome of grading a registry against the pins. */
|
|
91
|
+
export interface RegistryCompletenessResult {
|
|
92
|
+
/** Names of pins the registry satisfies. */
|
|
93
|
+
readonly passed: readonly string[];
|
|
94
|
+
/** Pins the registry violates — empty iff fully conformant. */
|
|
95
|
+
readonly failed: readonly RegistryCompletenessMismatch[];
|
|
96
|
+
}
|
|
97
|
+
/**
|
|
98
|
+
* Grade a refusal registry against {@link registryCompletenessPins}.
|
|
99
|
+
*
|
|
100
|
+
* Every pin is applied to every row, in declaration order; a pin passes
|
|
101
|
+
* only when NO row violates it. A conformant registry produces an empty
|
|
102
|
+
* `failed` array.
|
|
103
|
+
*/
|
|
104
|
+
export declare function runRegistryCompletenessConformance(registry: RefusalRegistryView): RegistryCompletenessResult;
|
|
105
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/registry-completeness/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA2CG;AAEH,2DAA2D;AAC3D,MAAM,WAAW,kBAAkB;IACjC,gDAAgD;IAChD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,iEAAiE;IACjE,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,wDAAwD;IACxD,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,sDAAsD;IACtD,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IACpC,kCAAkC;IAClC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,iCAAiC;IACjC,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;CAC9B;AAED;;;;;GAKG;AACH,MAAM,MAAM,mBAAmB,GAAG;IAChC,QAAQ,EAAE,IAAI,EAAE,MAAM,GAAG,kBAAkB,CAAC;CAC7C,CAAC;AAUF,8DAA8D;AAC9D,MAAM,WAAW,uBAAuB;IACtC,uBAAuB;IACvB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,8CAA8C;IAC9C,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B;;;OAGG;IACH,QAAQ,CAAC,SAAS,EAAE,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,EAAE,kBAAkB,KAAK,MAAM,GAAG,IAAI,CAAC;CAC9E;AAED,gEAAgE;AAChE,eAAO,MAAM,wBAAwB,EAAE,SAAS,uBAAuB,EAoCtE,CAAC;AAEF,iEAAiE;AACjE,MAAM,WAAW,4BAA4B;IAC3C,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,oDAAoD;IACpD,QAAQ,CAAC,UAAU,EAAE,SAAS,MAAM,EAAE,CAAC;CACxC;AAED,sDAAsD;AACtD,MAAM,WAAW,0BAA0B;IACzC,4CAA4C;IAC5C,QAAQ,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IACnC,+DAA+D;IAC/D,QAAQ,CAAC,MAAM,EAAE,SAAS,4BAA4B,EAAE,CAAC;CAC1D;AAED;;;;;;GAMG;AACH,wBAAgB,kCAAkC,CAChD,QAAQ,EAAE,mBAAmB,GAC5B,0BAA0B,CAiB5B"}
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Registry-completeness catalog — the structural obligations a
|
|
3
|
+
* deployment's refusal-code registry MUST satisfy (ggui#786, ruling
|
|
4
|
+
* 5b).
|
|
5
|
+
*
|
|
6
|
+
* ## Why this is a pure-function catalog, graded differently
|
|
7
|
+
*
|
|
8
|
+
* Unlike every other catalog here, this one grades DATA rather than a
|
|
9
|
+
* function: the closed set of refusal codes a deployment declares. The
|
|
10
|
+
* caller passes the registry; each pin inspects every row and reports
|
|
11
|
+
* the rows that violate it.
|
|
12
|
+
*
|
|
13
|
+
* ## The four PORTABLE pins, and the two that are deliberately absent
|
|
14
|
+
*
|
|
15
|
+
* Ruling 5b names six checks. Four are portable — a third-party adopter
|
|
16
|
+
* with their own registry can run them, and they are the four below.
|
|
17
|
+
*
|
|
18
|
+
* Two read repo SOURCE and therefore cannot ship as conformance: the
|
|
19
|
+
* docstring rule ("an agent MUST NOT auto-retry an after-fix refusal
|
|
20
|
+
* whose fixBy is not caller") and the grep for code literals minted
|
|
21
|
+
* outside the registry file. This package publishes only `dist` +
|
|
22
|
+
* `README.md`, so a source walk could never run for an adopter, and the
|
|
23
|
+
* files that rule names live outside this tree entirely. Both live in
|
|
24
|
+
* `@ggui-ai/protocol`'s own suite, where the invariant is owned and the
|
|
25
|
+
* paths are stable. The omission is DECLARED here rather than hidden.
|
|
26
|
+
*
|
|
27
|
+
* ## Why the `fixBy` pin is one-directional
|
|
28
|
+
*
|
|
29
|
+
* The pin below asserts only that an `after-fix` row NAMES who acts. It
|
|
30
|
+
* does not assert `fixBy` is absent elsewhere, because that is a naming
|
|
31
|
+
* discipline rather than a reader-observable obligation: a reader
|
|
32
|
+
* branches on `retry`, and a `fixBy` beside a `never` misleads a human
|
|
33
|
+
* without changing what any implementation does.
|
|
34
|
+
*
|
|
35
|
+
* ggui's own registry is stricter — `fixBy` is REQUIRED on `after-fix`,
|
|
36
|
+
* PERMITTED on `later` (the operator who restores a transiently
|
|
37
|
+
* unavailable surface) and FORBIDDEN on `next-period` and `never` — and
|
|
38
|
+
* enforces it at the type level in its definer, pinned in
|
|
39
|
+
* `@ggui-ai/protocol`'s own suite. Adopters are welcome to that rule;
|
|
40
|
+
* the kit does not make it a conformance obligation. The one thing that
|
|
41
|
+
* must never happen, and IS graded here, is an `after-fix` row that
|
|
42
|
+
* does not say WHO acts — the one an agent could misread as its own to
|
|
43
|
+
* retry.
|
|
44
|
+
*/
|
|
45
|
+
/** The four recognized retry classes (ggui#786 rulings v2 + v4). */
|
|
46
|
+
const RECOGNIZED_RETRIES = [
|
|
47
|
+
'after-fix',
|
|
48
|
+
'next-period',
|
|
49
|
+
'later',
|
|
50
|
+
'never',
|
|
51
|
+
];
|
|
52
|
+
/** Every portable pin the kit ships, in deterministic order. */
|
|
53
|
+
export const registryCompletenessPins = [
|
|
54
|
+
{
|
|
55
|
+
name: 'surfaces-non-empty',
|
|
56
|
+
description: 'Every entry lists at least one surface it is emitted on. A row nobody emits anywhere is a name reserved with no obligation behind it, which is the drift a closed registry exists to prevent. One code MAY list several surfaces — one state can surface in more than one place.',
|
|
57
|
+
violation: (_code, row) => row.surfaces.length > 0 ? null : 'surfaces is empty',
|
|
58
|
+
},
|
|
59
|
+
{
|
|
60
|
+
name: 'code-equals-key',
|
|
61
|
+
description: "Every entry's `code` equals the key it is declared under. The key is what emitters look a row up by and the code is what reaches the wire; if they disagree, the registry stops being the single namespace it claims to be.",
|
|
62
|
+
violation: (code, row) => row.code === code ? null : `code '${row.code}' does not equal key '${code}'`,
|
|
63
|
+
},
|
|
64
|
+
{
|
|
65
|
+
name: 'after-fix-names-fixby',
|
|
66
|
+
description: "Every `after-fix` entry names WHO acts, in `fixBy`. Without it, 'retry after the fix' cannot be told apart from 'retry now' — and an agent may only act on a fix that is its own. A row whose `retry` is not a recognized class ALSO fails this pin: the obligation is undecidable for it, and a pin that silently passes a row it could not evaluate is a false gate.",
|
|
67
|
+
violation: (_code, row) => {
|
|
68
|
+
if (!RECOGNIZED_RETRIES.includes(row.retry)) {
|
|
69
|
+
return `retry '${row.retry}' is not a recognized class, so the fixBy obligation cannot be evaluated`;
|
|
70
|
+
}
|
|
71
|
+
if (row.retry !== 'after-fix')
|
|
72
|
+
return null;
|
|
73
|
+
return row.fixBy === undefined ? 'after-fix entry carries no fixBy' : null;
|
|
74
|
+
},
|
|
75
|
+
},
|
|
76
|
+
{
|
|
77
|
+
name: 'retry-in-closed-set',
|
|
78
|
+
description: "Every entry's `retry` is one of `after-fix`, `next-period`, `later`, `never`. The class is what a reader branches on to decide whether to wait, act, or stop; an unrecognized value leaves it with no defined behaviour.",
|
|
79
|
+
violation: (_code, row) => RECOGNIZED_RETRIES.includes(row.retry)
|
|
80
|
+
? null
|
|
81
|
+
: `retry '${row.retry}' is outside the closed set`,
|
|
82
|
+
},
|
|
83
|
+
];
|
|
84
|
+
/**
|
|
85
|
+
* Grade a refusal registry against {@link registryCompletenessPins}.
|
|
86
|
+
*
|
|
87
|
+
* Every pin is applied to every row, in declaration order; a pin passes
|
|
88
|
+
* only when NO row violates it. A conformant registry produces an empty
|
|
89
|
+
* `failed` array.
|
|
90
|
+
*/
|
|
91
|
+
export function runRegistryCompletenessConformance(registry) {
|
|
92
|
+
const entries = Object.entries(registry);
|
|
93
|
+
const passed = [];
|
|
94
|
+
const failed = [];
|
|
95
|
+
for (const pin of registryCompletenessPins) {
|
|
96
|
+
const violations = [];
|
|
97
|
+
for (const [code, row] of entries) {
|
|
98
|
+
const why = pin.violation(code, row);
|
|
99
|
+
if (why !== null)
|
|
100
|
+
violations.push(`${code}: ${why}`);
|
|
101
|
+
}
|
|
102
|
+
if (violations.length === 0) {
|
|
103
|
+
passed.push(pin.name);
|
|
104
|
+
}
|
|
105
|
+
else {
|
|
106
|
+
failed.push({ name: pin.name, description: pin.description, violations });
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
return { passed, failed };
|
|
110
|
+
}
|
package/dist/reporter.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"reporter.d.ts","sourceRoot":"","sources":["../src/reporter.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"reporter.d.ts","sourceRoot":"","sources":["../src/reporter.ts"],"names":[],"mappings":"AAiBA,OAAO,KAAK,EACV,kBAAkB,EAClB,mBAAmB,EACnB,iBAAiB,EACjB,cAAc,EACf,MAAM,sBAAsB,CAAC;AAM9B,MAAM,WAAW,sBAAsB;IACrC;;;OAGG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACxC;;;OAGG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;CAC9B;AAED;;;;GAIG;AACH,wBAAgB,qBAAqB,CACnC,OAAO,GAAE,sBAA2B,GACnC,mBAAmB,CA0CrB;AAMD,wBAAgB,eAAe,CAAC,MAAM,EAAE,iBAAiB,GAAG,MAAM,CAejE;AAyBD,wBAAgB,aAAa,CAAC,MAAM,EAAE,iBAAiB,GAAG,MAAM,CAW/D;AAED;;;GAGG;AACH,wBAAgB,cAAc,CAAC,QAAQ,EAAE,SAAS,kBAAkB,EAAE,GAAG,MAAM,CAW9E;AAED;;;GAGG;AACH,wBAAgB,WAAW,CAAC,OAAO,EAAE,SAAS,cAAc,EAAE,GAAG,MAAM,CAStE"}
|
package/dist/reporter.js
CHANGED
|
@@ -14,6 +14,7 @@
|
|
|
14
14
|
* `reporter` config.
|
|
15
15
|
*/
|
|
16
16
|
import { fixturesByContract } from './fixtures/index.js';
|
|
17
|
+
import { PURE_FUNCTION_CATALOG_SLUGS } from './run-conformance.js';
|
|
17
18
|
/**
|
|
18
19
|
* Build a {@link ConformanceReporter} that writes the bar-scorecard
|
|
19
20
|
* report. The return value plugs directly into
|
|
@@ -69,21 +70,37 @@ export function createDefaultReporter(options = {}) {
|
|
|
69
70
|
// =============================================================================
|
|
70
71
|
export function formatScorecard(result) {
|
|
71
72
|
// Group fixtures by contract so operators see pass rates per
|
|
72
|
-
// criterion, mirroring `fixturesByContract`'s classification
|
|
73
|
+
// criterion, mirroring `fixturesByContract`'s classification, then
|
|
74
|
+
// the pure-function catalogs, whose rows carry a `<slug>/` prefix
|
|
75
|
+
// instead of living in that map (they are not WS-observable).
|
|
73
76
|
const lines = [];
|
|
74
77
|
for (const [slug, fixtures] of Object.entries(fixturesByContract)) {
|
|
75
78
|
const names = new Set(fixtures.map((f) => f.name));
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
const
|
|
80
|
-
|
|
81
|
-
continue;
|
|
82
|
-
const tone = failedCount > 0 ? MARK_FAIL : passedCount > 0 ? MARK_PASS : MARK_SKIP;
|
|
83
|
-
lines.push(` ${tone} ${padRight(slug, 38)} ${passedCount}/${total} pass${failedCount > 0 ? ` · ${failedCount} fail` : ''}${skippedCount > 0 ? ` · ${skippedCount} skip` : ''}`);
|
|
79
|
+
lines.push(...scorecardRow(slug, result, (name) => names.has(name)));
|
|
80
|
+
}
|
|
81
|
+
for (const slug of PURE_FUNCTION_CATALOG_SLUGS) {
|
|
82
|
+
const prefix = `${slug}/`;
|
|
83
|
+
lines.push(...scorecardRow(slug, result, (name) => name.startsWith(prefix)));
|
|
84
84
|
}
|
|
85
85
|
return lines.join('\n');
|
|
86
86
|
}
|
|
87
|
+
/**
|
|
88
|
+
* One scorecard line for the rows `belongs` selects, or no line at all
|
|
89
|
+
* when the group graded nothing. A group that is entirely SKIPPED still
|
|
90
|
+
* prints — an obligation nobody graded must be visible, not absent.
|
|
91
|
+
*/
|
|
92
|
+
function scorecardRow(slug, result, belongs) {
|
|
93
|
+
const passedCount = result.passed.filter(belongs).length;
|
|
94
|
+
const failedCount = result.failed.filter((f) => belongs(f.name)).length;
|
|
95
|
+
const skippedCount = result.skipped.filter((s) => belongs(s.name)).length;
|
|
96
|
+
const total = passedCount + failedCount + skippedCount;
|
|
97
|
+
if (total === 0)
|
|
98
|
+
return [];
|
|
99
|
+
const tone = failedCount > 0 ? MARK_FAIL : passedCount > 0 ? MARK_PASS : MARK_SKIP;
|
|
100
|
+
return [
|
|
101
|
+
` ${tone} ${padRight(slug, 38)} ${passedCount}/${total} pass${failedCount > 0 ? ` · ${failedCount} fail` : ''}${skippedCount > 0 ? ` · ${skippedCount} skip` : ''}`,
|
|
102
|
+
];
|
|
103
|
+
}
|
|
87
104
|
export function formatSummary(result) {
|
|
88
105
|
const passed = result.passed.length;
|
|
89
106
|
const failed = result.failed.length;
|
|
@@ -2,6 +2,8 @@ import type { HostContextObservedPayload } from '@ggui-ai/protocol';
|
|
|
2
2
|
import type { ConformanceHost, SetupStep as HostSetupStep } from './conformance-host.js';
|
|
3
3
|
import { type MatchResult } from './match-behavior.js';
|
|
4
4
|
import type { AuthConfig, SessionStateBehavior, StreamUpdateBehavior, VersionMismatchBehavior } from './types.js';
|
|
5
|
+
import { type PreGenerationRefusalInput, type ProjectedRefusalResult } from './refusal-envelope-conformance/index.js';
|
|
6
|
+
import { type RefusalRegistryView } from './registry-completeness/index.js';
|
|
5
7
|
export interface RunConformanceConfig {
|
|
6
8
|
/**
|
|
7
9
|
* URL of the implementation under test. `http://` / `https://`
|
|
@@ -27,7 +29,34 @@ export interface RunConformanceConfig {
|
|
|
27
29
|
readonly host?: ConformanceHost;
|
|
28
30
|
/** Per-fixture observation window. Default 2000ms. */
|
|
29
31
|
readonly observationTimeoutMs?: number;
|
|
32
|
+
/**
|
|
33
|
+
* The implementation's PRE-GENERATION refusal projector (SPEC §7.1's
|
|
34
|
+
* refused arm). Given a refusal, return the tool result the server
|
|
35
|
+
* would emit for it, or `null` when the code has no envelope on this
|
|
36
|
+
* surface.
|
|
37
|
+
*
|
|
38
|
+
* Supplied ⇒ the `refusal-envelope` catalog is graded as part of this
|
|
39
|
+
* run. Omitted ⇒ its cases are reported SKIPPED with that reason, so
|
|
40
|
+
* an ungraded obligation shows up on the scorecard instead of
|
|
41
|
+
* vanishing from it.
|
|
42
|
+
*/
|
|
43
|
+
readonly refusalProjector?: (refusal: PreGenerationRefusalInput) => ProjectedRefusalResult | null;
|
|
44
|
+
/**
|
|
45
|
+
* The deployment's closed refusal-code registry, keyed by code.
|
|
46
|
+
* Supplied ⇒ the `registry-completeness` pins are graded as part of
|
|
47
|
+
* this run; omitted ⇒ reported SKIPPED, same reasoning as above.
|
|
48
|
+
*/
|
|
49
|
+
readonly refusalRegistry?: RefusalRegistryView;
|
|
30
50
|
}
|
|
51
|
+
/**
|
|
52
|
+
* Row-name prefixes for the PURE-FUNCTION catalogs the runner folds
|
|
53
|
+
* into a run. Prefixed so a catalog row can never collide with a
|
|
54
|
+
* WebSocket fixture name, and so the reporter can group them without a
|
|
55
|
+
* second registry of names.
|
|
56
|
+
*/
|
|
57
|
+
export declare const PURE_FUNCTION_CATALOG_SLUGS: readonly ["refusal-envelope", "registry-completeness"];
|
|
58
|
+
/** One member of {@link PURE_FUNCTION_CATALOG_SLUGS}. */
|
|
59
|
+
export type PureFunctionCatalogSlug = (typeof PURE_FUNCTION_CATALOG_SLUGS)[number];
|
|
31
60
|
export interface ConformanceResult {
|
|
32
61
|
readonly passed: readonly string[];
|
|
33
62
|
readonly failed: readonly ConformanceFailure[];
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"run-conformance.d.ts","sourceRoot":"","sources":["../src/run-conformance.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"run-conformance.d.ts","sourceRoot":"","sources":["../src/run-conformance.ts"],"names":[],"mappings":"AA6DA,OAAO,KAAK,EACV,0BAA0B,EAG3B,MAAM,mBAAmB,CAAC;AAG3B,OAAO,KAAK,EACV,eAAe,EACf,SAAS,IAAI,aAAa,EAE3B,MAAM,uBAAuB,CAAC;AAE/B,OAAO,EAAwC,KAAK,WAAW,EAAE,MAAM,qBAAqB,CAAC;AAC7F,OAAO,KAAK,EAEV,UAAU,EAEV,oBAAoB,EACpB,oBAAoB,EAEpB,uBAAuB,EACxB,MAAM,YAAY,CAAC;AAEpB,OAAO,EAGL,KAAK,yBAAyB,EAC9B,KAAK,sBAAsB,EAC5B,MAAM,yCAAyC,CAAC;AACjD,OAAO,EAGL,KAAK,mBAAmB,EACzB,MAAM,kCAAkC,CAAC;AAM1C,MAAM,WAAW,oBAAoB;IACnC;;;;;;;OAOG;IACH,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,8DAA8D;IAC9D,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAC1B,gEAAgE;IAChE,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAC;IAC9B,qDAAqD;IACrD,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IAClC,wDAAwD;IACxD,QAAQ,CAAC,QAAQ,CAAC,EAAE,mBAAmB,CAAC;IACxC;;;OAGG;IACH,QAAQ,CAAC,IAAI,CAAC,EAAE,eAAe,CAAC;IAChC,sDAAsD;IACtD,QAAQ,CAAC,oBAAoB,CAAC,EAAE,MAAM,CAAC;IACvC;;;;;;;;;;OAUG;IACH,QAAQ,CAAC,gBAAgB,CAAC,EAAE,CAC1B,OAAO,EAAE,yBAAyB,KAC/B,sBAAsB,GAAG,IAAI,CAAC;IACnC;;;;OAIG;IACH,QAAQ,CAAC,eAAe,CAAC,EAAE,mBAAmB,CAAC;CAChD;AAED;;;;;GAKG;AACH,eAAO,MAAM,2BAA2B,wDAG9B,CAAC;AAEX,yDAAyD;AACzD,MAAM,MAAM,uBAAuB,GAAG,CAAC,OAAO,2BAA2B,CAAC,CAAC,MAAM,CAAC,CAAC;AAEnF,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC;IACnC,QAAQ,CAAC,MAAM,EAAE,SAAS,kBAAkB,EAAE,CAAC;IAC/C,QAAQ,CAAC,OAAO,EAAE,SAAS,cAAc,EAAE,CAAC;IAC5C,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,kBAAkB;IACjC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC;IAC3B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAED,MAAM,WAAW,mBAAmB;IAClC,OAAO,CAAC,CAAC,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;IACtC,aAAa,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACtD,aAAa,CAAC,CAAC,OAAO,EAAE,kBAAkB,GAAG,IAAI,CAAC;IAClD,aAAa,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IACnD;;;;OAIG;IACH,iBAAiB,CAAC,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IACxD,UAAU,CAAC,CAAC,MAAM,EAAE,iBAAiB,GAAG,IAAI,CAAC;CAC9C;AAMD,wBAAsB,cAAc,CAClC,MAAM,EAAE,oBAAoB,GAC3B,OAAO,CAAC,iBAAiB,CAAC,CA+C5B;AAwQD;;;;;;;;;;;;GAYG;AACH,wBAAgB,WAAW,CAAC,SAAS,EAAE,MAAM,GAAG,MAAM,CAWrD;AAoBD;;;;;;;;;;GAUG;AACH,MAAM,WAAW,gCAAgC;IAC/C,QAAQ,CAAC,IAAI,EAAE,uBAAuB,CAAC;IACvC,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,OAAO,EAAE,0BAA0B,CAAC;CAC9C;AAED;;;;;;;;;;;;;;;;;;GAkBG;AACH,MAAM,MAAM,qBAAqB,GAC7B;IAAE,QAAQ,CAAC,IAAI,EAAE,QAAQ,CAAC;IAAC,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAA;CAAE,GACvD;IACE,QAAQ,CAAC,IAAI,EAAE,uBAAuB,CAAC;IACvC,QAAQ,CAAC,QAAQ,EAAE,gCAAgC,CAAC;CACrD,GACD;IAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAAC;AAE9B;;;;;;;;;GASG;AACH,wBAAgB,kBAAkB,CAChC,WAAW,EAAE,MAAM,EACnB,QAAQ,EAAE,OAAO,GAChB,qBAAqB,CAcvB;AAkOD;;;;;;;;;;;;;;GAcG;AACH,wBAAsB,iBAAiB,CACrC,QAAQ,EAAE,oBAAoB,EAC9B,SAAS,EAAE,MAAM,EACjB,IAAI,EAAE,eAAe,GAAG,SAAS,GAChC,OAAO,CAAC,WAAW,CAAC,CA8BtB;AAED;;;;;GAKG;AACH,MAAM,WAAW,wBAAwB;IACvC,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAC;IAC5B,QAAQ,CAAC,iBAAiB,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;CAChD;AAED;;;;;;;;;;;;;;;;;GAiBG;AACH,wBAAgB,qBAAqB,CACnC,WAAW,EAAE,MAAM,EACnB,OAAO,EAAE,OAAO,GACf,wBAAwB,CAmC1B;AAqED;;;;;;;;;;;;;;;;;;GAkBG;AACH,wBAAgB,cAAc,CAAC,WAAW,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,GAAG,aAAa,CA4DhF;AAqHD,YAAY,EAAE,oBAAoB,EAAE,uBAAuB,EAAE,CAAC"}
|
package/dist/run-conformance.js
CHANGED
|
@@ -46,17 +46,35 @@
|
|
|
46
46
|
* (No teardown vocabulary exists in this kit version, so the
|
|
47
47
|
* loop is empty today.)
|
|
48
48
|
* l. Record the outcome via reporter + accumulate in result.
|
|
49
|
-
* 3.
|
|
49
|
+
* 3. Fold in the PURE-FUNCTION catalogs (ggui#786) —
|
|
50
|
+
* `refusal-envelope` and `registry-completeness`. They grade a
|
|
51
|
+
* caller-supplied projector / registry rather than the wire, so
|
|
52
|
+
* they need no transport; when the caller supplies neither, their
|
|
53
|
+
* rows are reported SKIPPED with that reason so an ungraded
|
|
54
|
+
* obligation stays visible on the scorecard.
|
|
55
|
+
* 4. Invoke `reporter.onComplete(result)` + return.
|
|
50
56
|
*
|
|
51
|
-
* Scope v1: pure WS transport. Browser-level
|
|
52
|
-
* failure, props-update) automatically skip via the
|
|
53
|
-
* `unmatchable-on-ws` arm.
|
|
57
|
+
* Scope v1: pure WS transport for the fixture catalog. Browser-level
|
|
58
|
+
* fixtures (bootstrap-failure, props-update) automatically skip via the
|
|
59
|
+
* matcher's `unmatchable-on-ws` arm.
|
|
54
60
|
*/
|
|
55
61
|
import { PROTOCOL_SCHEMA_VERSION } from '@ggui-ai/protocol';
|
|
56
62
|
import { allFixtures, fixturesByContract } from './fixtures/index.js';
|
|
57
63
|
import { isRecord } from '@ggui-ai/protocol';
|
|
58
64
|
import { behaviorIs, deepEqual, matchBehavior } from './match-behavior.js';
|
|
59
65
|
import { openWsTransport } from './ws-transport.js';
|
|
66
|
+
import { refusalEnvelopeCases, runRefusalEnvelopeConformance, } from './refusal-envelope-conformance/index.js';
|
|
67
|
+
import { registryCompletenessPins, runRegistryCompletenessConformance, } from './registry-completeness/index.js';
|
|
68
|
+
/**
|
|
69
|
+
* Row-name prefixes for the PURE-FUNCTION catalogs the runner folds
|
|
70
|
+
* into a run. Prefixed so a catalog row can never collide with a
|
|
71
|
+
* WebSocket fixture name, and so the reporter can group them without a
|
|
72
|
+
* second registry of names.
|
|
73
|
+
*/
|
|
74
|
+
export const PURE_FUNCTION_CATALOG_SLUGS = [
|
|
75
|
+
'refusal-envelope',
|
|
76
|
+
'registry-completeness',
|
|
77
|
+
];
|
|
60
78
|
// =============================================================================
|
|
61
79
|
// Runner
|
|
62
80
|
// =============================================================================
|
|
@@ -93,6 +111,7 @@ export async function runConformance(config) {
|
|
|
93
111
|
break;
|
|
94
112
|
}
|
|
95
113
|
}
|
|
114
|
+
runPureFunctionCatalogs(config, reporter, { passed, failed, skipped });
|
|
96
115
|
const result = {
|
|
97
116
|
passed,
|
|
98
117
|
failed,
|
|
@@ -102,6 +121,85 @@ export async function runConformance(config) {
|
|
|
102
121
|
reporter.onComplete?.(result);
|
|
103
122
|
return result;
|
|
104
123
|
}
|
|
124
|
+
/**
|
|
125
|
+
* Fold the pure-function catalogs (ggui#786) into a run.
|
|
126
|
+
*
|
|
127
|
+
* These obligations are not WebSocket-observable — a refusal projection
|
|
128
|
+
* is a pure function and a registry is data — so they are NOT in
|
|
129
|
+
* `fixturesByContract` (see `fixtures/index.ts` for why registering a
|
|
130
|
+
* non-WS case there would be a permanent skip). But leaving them
|
|
131
|
+
* reachable only through a direct import made them invisible to every
|
|
132
|
+
* CLI run: a catalog the runner cannot report is a silent gate
|
|
133
|
+
* (docs/principles/no-silent-block.md).
|
|
134
|
+
*
|
|
135
|
+
* So they run HERE, on the same verdict buckets as the fixtures, and an
|
|
136
|
+
* absent input produces a SKIP row naming what to supply rather than
|
|
137
|
+
* nothing at all.
|
|
138
|
+
*/
|
|
139
|
+
function runPureFunctionCatalogs(config, reporter, buckets) {
|
|
140
|
+
const only = config.only;
|
|
141
|
+
const wanted = (name) => only === undefined || only.length === 0 || only.includes(name);
|
|
142
|
+
const pass = (name) => {
|
|
143
|
+
if (!wanted(name))
|
|
144
|
+
return;
|
|
145
|
+
buckets.passed.push(name);
|
|
146
|
+
reporter.onFixturePass?.(name, 0);
|
|
147
|
+
};
|
|
148
|
+
const skip = (name, reason) => {
|
|
149
|
+
if (!wanted(name))
|
|
150
|
+
return;
|
|
151
|
+
buckets.skipped.push({ name, reason });
|
|
152
|
+
reporter.onFixtureSkip?.(name, reason);
|
|
153
|
+
};
|
|
154
|
+
const fail = (failure) => {
|
|
155
|
+
if (!wanted(failure.name))
|
|
156
|
+
return;
|
|
157
|
+
buckets.failed.push(failure);
|
|
158
|
+
reporter.onFixtureFail?.(failure);
|
|
159
|
+
};
|
|
160
|
+
// ── refusal-envelope ──
|
|
161
|
+
const projector = config.refusalProjector;
|
|
162
|
+
if (projector === undefined) {
|
|
163
|
+
for (const testCase of refusalEnvelopeCases) {
|
|
164
|
+
skip(`refusal-envelope/${testCase.name}`, 'no `refusalProjector` supplied — pass one to runConformance() to grade SPEC §7.1\u2019s refused arm');
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
else {
|
|
168
|
+
const envelope = runRefusalEnvelopeConformance(projector);
|
|
169
|
+
for (const name of envelope.passed)
|
|
170
|
+
pass(`refusal-envelope/${name}`);
|
|
171
|
+
for (const mismatch of envelope.failed) {
|
|
172
|
+
fail({
|
|
173
|
+
name: `refusal-envelope/${mismatch.name}`,
|
|
174
|
+
criterion: 'pre-generation refusal envelope (SPEC §7.1, refused arm)',
|
|
175
|
+
expected: mismatch.expected,
|
|
176
|
+
received: mismatch.actual,
|
|
177
|
+
message: 'the projected refusal envelope does not match the catalog case',
|
|
178
|
+
});
|
|
179
|
+
}
|
|
180
|
+
}
|
|
181
|
+
// ── registry-completeness ──
|
|
182
|
+
const registry = config.refusalRegistry;
|
|
183
|
+
if (registry === undefined) {
|
|
184
|
+
for (const pin of registryCompletenessPins) {
|
|
185
|
+
skip(`registry-completeness/${pin.name}`, 'no `refusalRegistry` supplied — pass the deployment\u2019s refusal-code registry to runConformance() to grade it');
|
|
186
|
+
}
|
|
187
|
+
}
|
|
188
|
+
else {
|
|
189
|
+
const completeness = runRegistryCompletenessConformance(registry);
|
|
190
|
+
for (const name of completeness.passed)
|
|
191
|
+
pass(`registry-completeness/${name}`);
|
|
192
|
+
for (const mismatch of completeness.failed) {
|
|
193
|
+
fail({
|
|
194
|
+
name: `registry-completeness/${mismatch.name}`,
|
|
195
|
+
criterion: 'refusal-registry completeness',
|
|
196
|
+
expected: mismatch.description,
|
|
197
|
+
received: mismatch.violations,
|
|
198
|
+
message: `${String(mismatch.violations.length)} row(s) violate this pin`,
|
|
199
|
+
});
|
|
200
|
+
}
|
|
201
|
+
}
|
|
202
|
+
}
|
|
105
203
|
async function runOneFixture(fixture, config, _reporter) {
|
|
106
204
|
const fixtureStarted = Date.now();
|
|
107
205
|
if (fixture.skipReason !== null) {
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ggui-ai/protocol-conformance",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Conformance test kit for the ggui protocol. Drives a behavioral fixture catalog against a live implementation over WebSocket, a resources/read catalog against a live MCP server, and
|
|
3
|
+
"version": "0.14.0",
|
|
4
|
+
"description": "Conformance test kit for the ggui protocol. Drives a behavioral fixture catalog against a live implementation over WebSocket, a resources/read catalog against a live MCP server, and eight pure-function catalogs (schema, registration, resolution, binding, props-schema, host-helper, refusal-envelope, registry-completeness) for the gadget, tool-binding, schema-precise-render, host-helper and pre-generation-refusal obligations, reporting a scorecard-style pass / fail / skip. MCP builders adopt this kit to prove their implementation is protocol-conformant.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"keywords": [
|
|
7
7
|
"ggui",
|
|
@@ -65,11 +65,21 @@
|
|
|
65
65
|
"types": "./dist/host-helper-conformance/index.d.ts",
|
|
66
66
|
"import": "./dist/host-helper-conformance/index.js",
|
|
67
67
|
"default": "./dist/host-helper-conformance/index.js"
|
|
68
|
+
},
|
|
69
|
+
"./refusal-envelope-conformance": {
|
|
70
|
+
"types": "./dist/refusal-envelope-conformance/index.d.ts",
|
|
71
|
+
"import": "./dist/refusal-envelope-conformance/index.js",
|
|
72
|
+
"default": "./dist/refusal-envelope-conformance/index.js"
|
|
73
|
+
},
|
|
74
|
+
"./registry-completeness": {
|
|
75
|
+
"types": "./dist/registry-completeness/index.d.ts",
|
|
76
|
+
"import": "./dist/registry-completeness/index.js",
|
|
77
|
+
"default": "./dist/registry-completeness/index.js"
|
|
68
78
|
}
|
|
69
79
|
},
|
|
70
80
|
"dependencies": {
|
|
71
81
|
"ws": "^8.20.1",
|
|
72
|
-
"@ggui-ai/protocol": "0.
|
|
82
|
+
"@ggui-ai/protocol": "0.14.0"
|
|
73
83
|
},
|
|
74
84
|
"devDependencies": {
|
|
75
85
|
"@types/node": "^22.0.0",
|
|
@@ -90,11 +100,11 @@
|
|
|
90
100
|
"access": "public"
|
|
91
101
|
},
|
|
92
102
|
"engines": {
|
|
93
|
-
"node": ">=
|
|
103
|
+
"node": ">=22.0.0"
|
|
94
104
|
},
|
|
95
105
|
"author": "ggui contributors <hello@ggui.ai>",
|
|
96
106
|
"scripts": {
|
|
97
|
-
"build": "rm -rf dist.staging && tsc -p tsconfig.build.json --outDir dist.staging && node ../scripts/fix-esm-imports.mjs dist.staging && chmod +x dist.staging
|
|
107
|
+
"build": "rm -rf dist.staging-$$ && tsc -p tsconfig.build.json --outDir dist.staging-$$ && node ../scripts/fix-esm-imports.mjs dist.staging-$$ && chmod +x dist.staging-$$/cli.js && node ../scripts/atomic-swap.mjs dist.staging-$$ dist",
|
|
98
108
|
"dev": "tsc --watch",
|
|
99
109
|
"typecheck": "tsc --noEmit",
|
|
100
110
|
"test": "vitest run --passWithNoTests",
|