@holonograph/client 0.9.0 → 0.10.0

package/README.md

@@ -66,6 +66,68 @@ That is the whole loop: **`messages.create()` → score → `reportOutcome()`**.
 lens assembles and stores the evaluation event; over time those events become
 the longitudinal record you query and visualize.
 
+## Verdict reliability — let the lens check your judge
+
+A score is only as trustworthy as the judge that produced it, and an LLM judge can
+misread its own input: accuse the model of inventing a value that was right there in
+a tool result, or of skipping a tool it actually called. The lens catches that
+**deterministically** — but only if your judge hands it a structured claim to check,
+instead of burying the accusation in prose. Attach a `claim` to any judged dimension:
+
+```ts
+await handle.reportOutcome({
+  dimensions: [
+    {
+      dimensionId: 'grounded',
+      passed: false,
+      expected: 'only facts present in the tool results',
+      actual: 'claimed a refund was issued and cited a total of $84.20',
+      // What the judge ASSERTED, as fields the lens can verify — not prose.
+      claim: {
+        status: 'emitted',
+        assertions: [
+          // "the model cited a value that is not in what it read"
+          { kind: 'datum-absent', datum: '$84.20', reference: 'the lookup total was $48.20' },
+          // "the model claimed an action without calling the tool that performs it"
+          { kind: 'tool-not-invoked', tool: 'issue_refund' },
+        ],
+      },
+    },
+  ],
+});
+```
+
+The lens checks each assertion **against the captured call** — is the accused value
+actually absent from everything the model read? was that tool really never invoked,
+anywhere across the whole conversation? — and reports, per assertion, whether the
+record **corroborates** or **contradicts** the judge. A contradiction is the loud
+case: the judge's verdict rested on something it misread, so that score can't be
+trusted. Two assertion kinds ship today:
+
+- **`datum-absent`** — a value the judge says was fabricated. It splits the accused
+  `datum` from the optional `reference` the judge measured it against, so the lens
+  verifies the _accusation_, never the yardstick.
+- **`tool-not-invoked`** — a tool the judge says was never called. The check reads the
+  whole captured conversation, so a call made in an earlier turn still counts.
+
+A dimension with no `claim` is simply not claim-instrumented — the lens stays
+honest-null about it rather than guessing, so you opt in one dimension at a time. If
+your judge is _meant_ to emit a claim but its output is malformed, report
+`{ status: 'malformed', reflectionAttempted: <boolean> }` — the lens raises that as a
+judge-malfunction signal rather than silently trusting a broken instrument. The
+`JudgeClaimReport` and `JudgeAssertion` types are exported so you can build and
+validate the claim before you send it.
+
+Read the whole layer back over a window from your lens — every claim-instrumented
+dimension with its per-assertion disposition:
+
+```
+GET /holonograph/verdict-reliability
+```
+
+(There is no dedicated client method for this read yet; use `client.callDirectly` or a
+plain `fetch` against your lens `endpoint`.)
+
 ## What it speaks
 
 The client talks to a lens over plain HTTP under the `/holonograph/*` path
@@ -81,7 +143,10 @@ status, an error code, and any details the lens returned.
 - **`HolonographClient`** — construct with an `endpoint` (plus optional `token`
   / `runMode`) and the `lensVersion` + `substrate` to pin into every event.
   - `client.messages.create(request)` — send a message; returns a handle.
-  - `handle.reportOutcome(outcome)` — commit the scored outcome.
+  - `handle.reportOutcome(outcome)` — commit the scored outcome. Each dimension may
+    carry a `claim` (a `JudgeClaimReport`) for the verdict-reliability layer to check
+    (see above). The `JudgeClaimReport` / `JudgeAssertion` / `JudgeAssertionKind` types
+    are exported.
   - `handle.gradeObserver(...)` — attach grades to a cross-vendor observer call
     before reporting, when the lens returned observer records.
   - `client.contract.register(contract)` — publish a surface contract to the lens.