@openwop/openwop-conformance 1.142.0 → 1.145.0

package/README.md

@@ -94,7 +94,7 @@ Exit code is non-zero on any failed assertion. `--certify` distinguishes: `0`
 
 ## What's Covered
 
-The current suite has 465 scenario files under `src/scenarios/`.
+The current suite has 467 scenario files under `src/scenarios/`.
 - 2026-08-19 (suite `1.137.0 → 1.138.0`): NEW `durability-poison-exhaustion.test.ts` — RFC 0158 §C.8, the FIRST row of that RFC's conformance table to land. Asserts what `failure-path.test.ts` cannot: not just that deterministically failing work reaches terminal, but that attempts STOP — counted on the log, re-counted after a scaled quiet window, asserted unchanged. A host still redelivering records more. Seam-gated on the existing event-log seam (`blocked` = unobservable, not unmet) and outside every profile floor.
 - 2026-08-19 (suite `1.136.15 → 1.137.0`): NEW `replay-fanout-suppression.test.ts` — capability-gated on `webhooks.supported`, **outside every profile floor**; witnesses `replay.md` §"Host-initiated fan-out is an external effect", which was the largest normative MUST NOT on the replay surface with no scenario and no SECURITY invariant. Three legs in ONE `it` against ONE receiver and ONE subscription — a positive control, the MUST NOT, and a `branch` boundary leg — because "no delivery arrived" passes identically when delivery never worked, so absence is asserted only after presence is proven on that exact wiring. A host with an SSRF guard correctly refuses the loopback receiver and records `blocked`: **unobservable, not unmet.**
 - 2026-08-18 (suite `1.136.7 → 1.136.8`, SP-04): NEW `spec-section-citations.test.ts` — server-free; a `<doc>.md §"<Section>"` citation of a checked doc MUST resolve to a heading that exists. `storage-adapters.md §"Claim acquisition"` was cited by four artifacts, including a normative MUST in `production-profile.md` §Durability, and the section did not exist for the life of RFC 0009. Scoped to the docs whose section citations carry the durability contract; a corpus-wide sweep finds ~260 unresolved citations that need triage before they can gate.
@@ -438,7 +438,7 @@ Server-required (added in 1.7.0):
 | ------------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | **Redaction** | [`capabilities.md`](../spec/v1/capabilities.md) §"Secrets" + NFR-7 + §"aiProviders" | Vendor-neutral assertions that the server doesn't leak secret material. Three scenario groups: (a) discovery shape contract — `secrets` + `aiProviders` advertisements are well-formed regardless of `secrets.supported`; when `supported === true`, scopes MUST be non-empty + `resolution === 'host-managed'`; `byok ⊆ supported`. (b) bearer-token redaction — invalid Bearer canary in `Authorization` header is not echoed in the 401 response body. (c) credentialRef echo control — gated on `secrets.supported === true`; canary planted in `configurable.ai.credentialRef` MUST NOT appear in any RunEvent payload (poll-based capture; transport-agnostic). Uses runtime-built canary fixtures (`lib/canaries.ts`) that defeat static secret scanners. 6 scenarios. |
 
-Current source tree: 465 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
+Current source tree: 467 scenario files. Use [`coverage.md`](./coverage.md) for current grade/gap tracking.
 
 ## Remaining Gaps
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@openwop/openwop-conformance",
-  "version": "1.142.0",
+  "version": "1.145.0",
   "description": "Production-ready black-box conformance suite for OpenWOP v1.0 compliant servers.",
   "repository": {
     "type": "git",

package/schemas/CORPUS-STAMP.json

@@ -1,5 +1,5 @@
 {
   "_comment": "Provenance of this vendored schemas/ copy. See conformance/README.md \u00a7\"Resolving the contract\". Compare against the stamp in your installed @openwop/openwop-conformance to detect a stale hand-copied contract.",
-  "suiteVersion": "1.142.0",
-  "corpusCommit": "aced8a6a82d751394c3b299fa4602b9dd5cc9971"
+  "suiteVersion": "1.145.0",
+  "corpusCommit": "55457606be653cb499906ba6f5960c2fdd62017c"
 }

package/schemas/capabilities.schema.json

@@ -3691,6 +3691,10 @@
           "uniqueItems": true,
           "description": "Auth profiles the host claims. Canonical ids: `openwop-audit-log-integrity` (auth-profiles.md \u00a7Audit-log integrity), `openwop-auth-api-key-rotation`, `openwop-auth-oauth2-client-credentials`, `openwop-auth-oidc-user-bearer`, `openwop-auth-mtls`, `openwop-auth-saml` + `openwop-auth-scim` + `openwop-auth-ldap` (RFC 0050 enterprise identity). Clients SHOULD tolerate unknown profile ids."
         },
+        "subjectLinking": {
+          "type": "boolean",
+          "description": "RFC 0159 (auth-profiles.md \u00a7Subject linking). When true, this host advertises BOTH `openwop-auth-saml` and `openwop-auth-scim` AND honours the cross-profile subject-linking obligation: it links the SAML and SCIM principals of one human on an opaque, IdP-stable subject id (SCIM `externalId` \u2194 persistent SAML `NameID`), same-tenant only, and on SCIM deactivation fail-closes the linked SAML identity too (composing RFC 0049 \u00a7C) \u2014 so a provisioned leaver cannot still SSO in. The link key MUST NOT be a mutable/PII attribute (email, userName). Optional; absent/false means the SAML and SCIM lanes are independent and the combined leaver guarantee is NOT claimed. A host MUST NOT set this true unless both profiles are advertised and the obligation holds."
+        },
         "rotation": {
           "type": "object",
           "description": "API-key rotation advertisement (auth-profiles.md \u00a7`openwop-auth-api-key-rotation`).",

package/src/lib/spec-coherence-registry.test.ts

@@ -119,3 +119,37 @@ describe('resolveFileRecord classifies a corpus scenario as inapplicable, not bl
     expect(r.disposition).toBe('executed-pass');
   });
 });
+
+describe('the published layout is what makes these rows comparable across hosts', () => {
+  // Load-bearing and, until now, tested nowhere.
+  //
+  // A spec-coherence row is `inapplicable` only when V1_DIR is null, and V1_DIR
+  // is null only when the layout resolves to `published`. Two peers established
+  // by measurement what the code implies: `resolveLayout()` keys off PKG_ROOT,
+  // not the consuming repo, so an npm-installed consumer's parent is always
+  // `node_modules/@openwop/` — which never contains `schemas/` no matter where
+  // the host's checkout sits on disk. Every npm-consuming host therefore gets
+  // the same answer, and the dispositions are comparable BY CONSTRUCTION.
+  //
+  // One peer had generalised the opposite way — "the disposition is a property
+  // of where the bundle was cut" — from a host whose own runner sets
+  // OPENWOP_CONFORMANCE_ROOT when it finds a sibling checkout. True of that
+  // host, false of the artifact. The distinction only survives if something
+  // holds the artifact to it.
+  //
+  // The thing that would break it is a change that looks HELPFUL: adding
+  // `spec` to `files` so "the corpus tests run for consumers too". That would
+  // silently give npm consumers V1_DIR, the 28 would execute instead of
+  // flipping, and every host's numbers would shift with no failure anywhere.
+  it('the published package ships no spec/ — so V1_DIR is null for npm consumers', () => {
+    const pkg = JSON.parse(readFileSync(join(SCENARIOS, '../../package.json'), 'utf8')) as { files: string[] };
+    expect(
+      pkg.files.filter((f) => f === 'spec' || f.startsWith('spec/')),
+      'adding spec/ to `files` would give npm consumers a V1_DIR, so the spec-coherence rows would execute '
+        + 'instead of reporting `inapplicable` — changing every host bundle with no test going red',
+    ).toEqual([]);
+    // schemas/ IS shipped, and is what selects the `published` layout. If this
+    // ever stops being true the layout resolves to neither branch.
+    expect(pkg.files, 'schemas/ is what makes resolveLayout() pick `published`').toContain('schemas');
+  });
+});

package/src/scenarios/auth-subject-link.test.ts

@@ -0,0 +1,152 @@
+/**
+ * auth-subject-link — RFC 0159: SCIM ⟷ SAML subject linking (the combined
+ * leaver contract).
+ *
+ * Status: DRAFT. RFC 0159 is `Active` (amends RFC 0050). The obligation is
+ * documented in `auth-profiles.md` §"Subject linking (SAML ⟷ SCIM)" and is
+ * discoverable via `capabilities.auth.subjectLinking`.
+ *
+ * Advertisement shape runs unconditionally when the flag is set. The
+ * cross-lane behavioral legs (a SCIM deactivation fail-closing the linked SAML
+ * identity; a mutable-key link never producing a cross-lane pass) are opt-in
+ * via `OPENWOP_TEST_SAML_IDP_URL` + `OPENWOP_TEST_SCIM_URL` (operator-supplied
+ * endpoints), following the `auth-scim-profile.test.ts` opt-in precedent.
+ * Soft-skips otherwise; soft-skips `blocked` until the host wires the seam.
+ *
+ * @see RFCS/0159-scim-saml-subject-linking.md
+ * @see spec/v1/auth-profiles.md §"Subject linking (SAML ⟷ SCIM)"
+ */
+
+import { describe, it, expect } from 'vitest';
+import { softSkip } from '../lib/soft-skip.js';
+import { driver } from '../lib/driver.js';
+import { capabilityFamily } from '../lib/discovery-capabilities.js';
+
+const SAML_PROFILE = 'openwop-auth-saml';
+const SCIM_PROFILE = 'openwop-auth-scim';
+
+interface DiscoveryAuth {
+  profiles?: string[];
+  subjectLinking?: boolean;
+}
+
+interface DiscoveryDoc {
+  capabilities?: { auth?: DiscoveryAuth };
+  extensions?: { auth?: DiscoveryAuth };
+}
+
+async function readAuth(): Promise<DiscoveryAuth | null> {
+  const res = await driver.get('/.well-known/openwop');
+  const body = res.json as DiscoveryDoc | undefined;
+  return capabilityFamily<DiscoveryAuth>(body, 'auth') ?? body?.extensions?.auth ?? null;
+}
+
+describe('auth-subject-link: advertisement shape (RFC 0159 §B)', () => {
+  it('subjectLinking:true is only claimed alongside both SAML and SCIM profiles', async () => {
+    const auth = await readAuth();
+    if (auth === null || auth.subjectLinking !== true) {
+      return softSkip('inapplicable', 'auth.subjectLinking not claimed');
+    }
+    const profiles = auth.profiles ?? [];
+    expect(
+      profiles.includes(SAML_PROFILE) && profiles.includes(SCIM_PROFILE),
+      driver.describe(
+        'auth-profiles.md §Subject linking',
+        'a host MUST NOT set capabilities.auth.subjectLinking:true unless BOTH openwop-auth-saml and openwop-auth-scim are advertised',
+      ),
+    ).toBe(true);
+  });
+});
+
+describe('auth-subject-link: cross-lane deactivation (RFC 0159 §A.3 — opt-in)', () => {
+  const idpUrl = process.env.OPENWOP_TEST_SAML_IDP_URL;
+  const scimUrl = process.env.OPENWOP_TEST_SCIM_URL;
+
+  it('a SCIM deactivation fail-closes the linked SAML identity', async () => {
+    const auth = await readAuth();
+    if (auth === null || auth.subjectLinking !== true) return softSkip('inapplicable', 'capability-gated');
+    if (!idpUrl || !scimUrl) return softSkip('inapplicable', 'opt-in: SAML IdP and/or SCIM endpoint not provided');
+
+    // 1. Provision a SCIM user carrying an opaque, IdP-stable externalId.
+    const externalId = 'idp-op-8f3a';
+    const provision = await driver.post('/v1/host/sample/auth/scim/provision', {
+      scimUrl,
+      op: 'create-user',
+      externalId,
+      userName: 'r.smith',
+    });
+    if (provision.status === 404) return softSkip('blocked', 'seam unwired');
+    expect(provision.status, driver.describe('auth-profiles.md §Subject linking', 'SCIM provisioning MUST succeed')).toBeLessThan(400);
+
+    // 2. A valid SAML assertion whose persistent NameID equals the externalId
+    //    links to the same subject and authenticates.
+    const before = await driver.post('/v1/host/sample/auth/saml/validate', { idpUrl, variant: 'valid', nameId: externalId });
+    if (before.status === 404) return softSkip('blocked', 'seam unwired');
+    expect(
+      (before.json as { authenticated?: boolean } | undefined)?.authenticated,
+      driver.describe('auth-profiles.md §Subject linking', 'a valid linked SAML assertion authenticates before deactivation'),
+    ).toBe(true);
+
+    // 3. SCIM-deactivate the provisioned user.
+    const deactivate = await driver.post('/v1/host/sample/auth/scim/provision', { scimUrl, op: 'deactivate-user', externalId });
+    expect(deactivate.status, driver.describe('auth-profiles.md §Subject linking', 'SCIM deactivation MUST succeed')).toBeLessThan(400);
+
+    // 4. THE CONTRACT: a subsequent SAML assertion for the linked subject MUST
+    //    NOT yield an authorized decision (fail-closed across the link).
+    const after = await driver.post('/v1/host/sample/auth/saml/validate', { idpUrl, variant: 'valid', nameId: externalId });
+    expect(
+      (after.json as { authenticated?: boolean } | undefined)?.authenticated === true,
+      driver.describe(
+        'auth-profiles.md §Subject linking',
+        'RFC 0159 §A.3: after a SCIM deactivation the LINKED SAML identity MUST be denied (a provisioned leaver cannot still SSO in)',
+      ),
+    ).toBe(false);
+  });
+});
+
+describe('auth-subject-link: link-key hygiene (RFC 0159 §A.2 — opt-in)', () => {
+  const idpUrl = process.env.OPENWOP_TEST_SAML_IDP_URL;
+  const scimUrl = process.env.OPENWOP_TEST_SCIM_URL;
+
+  it('a mutable/PII link key (email) never produces a cross-lane pass', async () => {
+    const auth = await readAuth();
+    if (auth === null || auth.subjectLinking !== true) return softSkip('inapplicable', 'capability-gated');
+    if (!idpUrl || !scimUrl) return softSkip('inapplicable', 'opt-in: SAML IdP and/or SCIM endpoint not provided');
+
+    // Attempt to form the cross-lane link on email (a mutable/PII attribute).
+    // §A.2 forbids it: the host MUST reject the mutable-key link (4xx) OR
+    // decline to apply any cross-lane effect from it. Either way, an email
+    // "link" MUST NOT yield an authorized cross-lane pass after a deactivation.
+    const link = await driver.post('/v1/host/sample/auth/scim/provision', {
+      scimUrl,
+      op: 'link',
+      linkKey: 'email',
+      email: 'r.smith@example.test',
+    });
+    if (link.status === 404) return softSkip('blocked', 'seam unwired');
+
+    if (link.status >= 400) {
+      // Conforming: the host rejected a mutable-key link outright.
+      expect(
+        link.status,
+        driver.describe('auth-profiles.md §Subject linking', 'RFC 0159 §A.2: a link on a mutable/PII key (email) MUST be rejected'),
+      ).toBeGreaterThanOrEqual(400);
+      return;
+    }
+
+    // If the host accepted the request, it MUST NOT have formed a cross-lane
+    // link on email: a deactivation via the email "link" MUST NOT deny (or
+    // otherwise act on) an unrelated SAML subject, and MUST NOT let a mutable
+    // key authorize one. Probe: deactivate by email, then a SAML assertion for
+    // a DIFFERENT opaque subject MUST be unaffected by the email operation.
+    await driver.post('/v1/host/sample/auth/scim/provision', { scimUrl, op: 'deactivate-user', email: 'r.smith@example.test' });
+    const other = await driver.post('/v1/host/sample/auth/saml/validate', { idpUrl, variant: 'valid', nameId: 'idp-op-DIFFERENT' });
+    expect(
+      (other.json as { linkedDenied?: boolean } | undefined)?.linkedDenied === true,
+      driver.describe(
+        'auth-profiles.md §Subject linking',
+        'RFC 0159 §A.2: a mutable-key (email) operation MUST NOT drive a cross-lane deny on any opaque subject',
+      ),
+    ).toBe(false);
+  });
+});

package/src/scenarios/tool-hooks-failure-honesty.test.ts

@@ -0,0 +1,62 @@
+/**
+ * tool-hooks-failure-honesty — RFC 0064 §F. A tool failure MUST be
+ * self-describing on the wire: a non-success `agent.toolReturned` MUST carry a
+ * failure discriminator — `error` populated (the `_errorObject`) and, when the
+ * host advertises `prePostEvents`, `status: 'error'` — and MUST NOT be
+ * represented as a bare/empty success. Closes the gap where a failed tool is
+ * indistinguishable from one that succeeded-empty (host-program WFAU-4).
+ *
+ * Gated on `capabilities.toolHooks.supported` + the tool-hooks seam. Drives the
+ * seam's `simulateToolError` arm; a host that advertises tool-hooks but has not
+ * wired that arm soft-skips (`blocked`) rather than failing — §F and the seam
+ * arm land together, so hosts wire it on adoption (the same soft-skip-until-wired
+ * discipline the other tool-hooks scenarios use).
+ *
+ * @see RFCS/0064-tool-invocation-hooks-and-authorization.md §F
+ */
+
+import { describe, it, expect } from 'vitest';
+import { softSkip } from '../lib/soft-skip.js';
+import { driver } from '../lib/driver.js';
+import { readToolHooksCap, invokeToolHook } from '../lib/toolHooks.js';
+
+describe('tool-hooks-failure-honesty (RFC 0064 §F)', () => {
+  it('a failing tool yields a populated error, never a bare success', async () => {
+    const cap = await readToolHooksCap();
+    if (cap?.supported !== true) return softSkip('inapplicable', 'capability or profile not advertised by this host — gate `cap?.supported !== true` returned early');
+    const res = await invokeToolHook({ principal: 'core.system', toolName: 'web.search', simulateToolError: true });
+    if (res === null) return softSkip('blocked', 'tool-hooks seam absent — soft-skip');
+
+    const returned = res.toolReturned ?? {};
+    const errRaw = returned['error'];
+    const err = errRaw != null && typeof errRaw === 'object' ? (errRaw as Record<string, unknown>) : undefined;
+    const isErrorStatus = returned['status'] === 'error';
+
+    // A host that has not wired the §F simulateToolError arm answers with a
+    // success (status 'ok' / an outcome). Soft-skip rather than red-light an
+    // honest pre-§F host — the prose MUST and this seam arm adopt together.
+    if (err === undefined && !isErrorStatus) {
+      return softSkip('blocked', 'host has not wired the RFC 0064 §F simulateToolError arm — soft-skip until adopted');
+    }
+
+    // §F: the failure carries a populated `_errorObject { code, message }`.
+    expect(
+      err !== undefined && typeof err['code'] === 'string' && (err['code'] as string).length > 0 && typeof err['message'] === 'string',
+      driver.describe('RFC 0064 §F', 'a failed tool MUST populate error {code, message} on agent.toolReturned'),
+    ).toBe(true);
+
+    // §F: `error` and `outcome` are mutually exclusive.
+    expect(
+      returned['outcome'] === undefined,
+      driver.describe('RFC 0064 §F', 'error and outcome are mutually exclusive on a failed tool-return'),
+    ).toBe(true);
+
+    // §F: a host advertising `prePostEvents` MUST also set `status: 'error'`.
+    if (cap['prePostEvents'] === true) {
+      expect(
+        returned['status'] === 'error',
+        driver.describe('RFC 0064 §F', 'a host advertising prePostEvents MUST set status:"error" on a tool failure'),
+      ).toBe(true);
+    }
+  });
+});