stilltrue 0.5.0 → 0.6.0

package/README.md

@@ -23,7 +23,7 @@ update it.
 |---|---|
 | `npx stilltrue drift` | Checks your saved facts against their live sources. Fails loudly if a fact has gone stale. |
 | `npx stilltrue report` | Turns past check runs into a simple web page — green/red grid, what broke and when. |
-| `verify` | For developers: a checkpoint that inspects AI-generated answers before they're shown to users. |
+| `verify` | For developers: a checkpoint that inspects AI-generated answers before they're shown to users. `supportStage` ships the common one — does this source support this claim? |
 
 ## Getting started
 
@@ -36,8 +36,8 @@ project folder with your facts saved in a JSON file.
 npm install --save-dev stilltrue
 ```
 
-**2. Scaffold a check** — this creates a ready-to-edit config file and an
-example facts file (it never overwrites anything you already have):
+**2. Scaffold a check** — this creates a ready-to-edit config file (it
+never overwrites anything you already have):
 
 ```bash
 npx stilltrue init
@@ -45,34 +45,52 @@ npx stilltrue init
 
 Open the generated `stilltrue.config.mjs`. Each check answers three
 questions: *where do I fetch the truth from* (`source`), *what do I
-believe* (`expect`), and *how do I compare the two* (`compare`):
+believe* (`expect`), and *how do I compare the two* (`compare`) — and in
+its simplest form, **the only editing needed is replacing strings**:
 
 ```js
-import { defineStilltrue, corpus, json, surname } from 'stilltrue';
+import { defineStilltrue, corpus } from 'stilltrue';
 
 export default defineStilltrue({
   drift: [
     {
-      name: 'school-board-roster',
-      // Where the truth lives: the district's own pages.
-      source: corpus(['https://www.myschooldistrict.org/board']),
-      // What you believe: read your saved file, keep just the last names.
-      expect: json('./data/school-board.json', (data) =>
-        data.members.map((m) => surname(m.name))
-      ),
-      // How to compare: every name must still appear on those pages.
+      name: 'my-first-check',
+      // Where the truth lives: the official page(s) your facts came from.
+      source: corpus(['https://example.org/about']),
+      // What you believe: your facts, as short strings ("markers").
+      expect: ['Jane Smith', 'Main Street Office', '12 locations'],
+      // How to compare: every marker must still appear on those pages.
       compare: 'contains-all',
     },
   ],
 });
 ```
 
-This example says: *fetch the board page, and make sure every last name in
-my `school-board.json` file still appears somewhere on it.*
+This says: *fetch that page, and make sure "Jane Smith", "Main Street
+Office", and "12 locations" all still appear on it.* Swap in your own URL
+and your own facts — that's a working check, no programming required.
+
+Every other field, compare mode, and helper is in
+[Configuration](#configuration) below. Choosing *what* to assert is the
+judgment call — surnames beat full names, `"143 schools"` beats a bare
+`"143"`. The short guide: [docs/MARKERS.md](docs/MARKERS.md).
+
+**Leveling up (optional):** you don't have to keep the facts in the config.
+If `./data/markers.json` is just a list of strings, point at it with
+`expect: json('./data/markers.json')` — still no code. And if your app
+already keeps structured data, derive the markers from it so your data file
+stays the single source of truth:
 
-Choosing what to assert is the judgment call — surnames beat full names,
-"143 schools" beats a bare "143". The short guide:
-[docs/MARKERS.md](docs/MARKERS.md).
+```js
+// ./data/school-board.json: { "members": [{ "name": "Dr. Jane Smith" }, ...] }
+expect: json('./data/school-board.json', (data) =>
+  data.members.map((m) => surname(m.name))   // keeps just "Smith" — titles
+),                                           // and first names vary on pages
+```
+
+The function receives *your* file's contents, whatever shape they are, and
+returns the list of markers — stilltrue never needs to understand your data
+structure. Update the data file and the markers update themselves.
 
 **3. Run it:**
 
@@ -97,6 +115,172 @@ You'll see one line per check:
   This only *warns*. An unreachable site doesn't mean your facts are wrong,
   and an alarm that cries wolf gets ignored — so outages never fail the build.
 
+## Configuration
+
+stilltrue loads a config file from the current directory. The first of
+these that exists wins:
+
+`stilltrue.config.ts` · `.mts` · `.js` · `.mjs` · `.cjs`
+
+(`npx stilltrue init` writes `.mjs`.) Point at a file anywhere with
+`stilltrue drift --config path/to/file`.
+
+Wrap the export with `defineStilltrue` so TypeScript can check it:
+
+```js
+import { defineStilltrue, corpus, json, surname } from 'stilltrue';
+
+export default defineStilltrue({
+  drift: [ /* one entry per fact-set */ ],
+  verify: { /* optional named pipelines — library use, see below */ },
+});
+```
+
+### Drift checks
+
+Each `drift` entry is one curated fact-set vs its live source.
+
+| Field | Required | Default | What it is |
+|---|---|---|---|
+| `name` | yes | — | Short id. Shows up in the terminal, `--only`, and the report. |
+| `source` | yes | — | `async (ctx) => actual`. Fetch/parse the live authority. **Throw** if the source is unreachable — that is outcome `error`, never `rot`. |
+| `expect` | yes | — | The curated facts. A value, or `async (ctx) => expected` (e.g. `json(...)`). A throw here is also `error` (broken config, not rot). |
+| `compare` | no | `'deep-equal'` | How to compare `expect` against what `source` returned. |
+
+`source` and function-form `expect` receive `ctx`:
+
+| | |
+|---|---|
+| `ctx.warn(message)` | Record a non-fatal note (one of several pages down, odd markup). Shows as a warning; does not fail the run. |
+| `ctx.configDir` | Absolute directory of the config file. Data paths resolve from here. |
+
+### `compare`
+
+**`'contains-all'`** — `expect` is a list of marker strings; `source` must
+return a text corpus (what `corpus()` returns). Every marker must appear
+somewhere in that text, case-insensitively. Extra text on the page is
+ignored. This is the webpage-watch check.
+
+**`'deep-equal'`** — structural equality. Extra keys, missing keys, and
+value mismatches all rot, with pathed messages
+(`$.members[2].name: expected "Smith", source has "Jones"`). Use this when
+`source()` already returns structured data that should match `expect`
+exactly.
+
+**A function** — `(expected, actual) => string[]`. Return `[]` to pass;
+each string is one mismatch (outcome `rot`). Throw if the shapes are
+unusable (outcome `error`). Use this for numeric tolerance, order-insensitive
+lists, or any comparison that isn't "markers in a page" or "these two
+objects are identical."
+
+```js
+compare: (expected, actual) =>
+  expected.rate === actual.rate
+    ? []
+    : [`tax rate ${expected.rate} → ${actual.rate}`],
+```
+
+Surnames are a *marker choice*, not a compare mode: derive them from your
+data with `surname()` and use `'contains-all'`.
+
+### Helpers
+
+These are the built-in `source` / `expect` builders. You can always write
+your own functions instead.
+
+**`corpus(urls, options?)`** — fetch pages, strip HTML to text,
+concatenate. Individual unreachable pages call `ctx.warn`; if *every* page
+fails, it throws (`error`). Pair with `'contains-all'`.
+
+| Option | Default | What it is |
+|---|---|---|
+| `timeoutMs` | `20000` | Per-page fetch timeout, milliseconds. |
+| `userAgent` | `stilltrue drift check (github.com/sg-bfoster/stilltrue)` | Sent on each request so operators can identify the check. |
+
+```js
+source: corpus(
+  ['https://example.org/about', 'https://example.org/board'],
+  { timeoutMs: 15_000, userAgent: 'my-app drift check' },
+),
+```
+
+**`json(path, derive?)`** — read a JSON file relative to the config file.
+With no `derive`, the parsed value *is* the expectation (typical with
+`'deep-equal'`). With `derive`, you map that value into whatever `compare`
+needs — usually a marker list:
+
+```js
+expect: json('./data/board.json', (data) =>
+  data.members.map((m) => surname(m.name)),
+),
+```
+
+**`surname(fullName)`** — last token of a name, with leading honorifics
+(`Dr.`, `Mr.`, `Mrs.`, `Ms.`) and trailing generational suffixes (`Jr.`,
+`Sr.`, `II`–`V`) stripped. `"Dr. Jane Smith III"` → `"Smith"`.
+
+**`htmlToText(html)`** — the same tag-stripping `corpus()` uses. For a
+custom `source` that already has HTML in hand.
+
+### Custom `source`
+
+A throw is an outage; a return value is data to compare. Warn for partial
+trouble:
+
+```js
+source: async (ctx) => {
+  const res = await fetch('https://example.org/api/rates');
+  if (!res.ok) throw new Error(`HTTP ${res.status}`);
+  const data = await res.json();
+  if (!data.updatedAt) ctx.warn('response missing updatedAt');
+  return data;
+},
+expect: json('./data/rates.json'),
+// compare defaults to 'deep-equal'
+```
+
+### `verify` pipelines
+
+`verify` is an optional map of named stage lists, for use from your app
+(`runVerify(config.verify.answers, draft)`). There is no `stilltrue verify`
+CLI command — verification runs in the request path, not in CI.
+
+Each stage:
+
+| Field | Required | Default | What it is |
+|---|---|---|---|
+| `name` | yes | — | Id of this checkpoint. The first rejection reports this name. |
+| `check` | yes | — | `(input, ctx) => { ok: true, value? } \| { ok: false, messages, revision? }`. |
+| `failPolicy` | no | `'open'` | What a **thrown** stage means. `'open'`: warn and continue (a judge outage must not block the app). `'closed'`: treat the throw as a rejection. |
+| `timeoutMs` | no | none | Abort the stage after this many ms. The timeout follows `failPolicy`. |
+| `tier` | no | `'inline'` | `'inline'` always runs; `'ci'` runs only when you call `runVerify(..., { tier: 'ci' })`. |
+
+Stages run in order and stop at the first rejection. `zodStage(name, schema)`
+builds a parse stage from a Zod schema.
+`generateVerified({ generate, stages, attempts, tier })` is the generate →
+verify → retry loop (`attempts` defaults to `2`). See
+[For developers: `verify`](#for-developers-verify) below.
+
+### CLI flags
+
+`stilltrue drift`:
+
+| Flag | What it does |
+|---|---|
+| `--config <path>` | Config file. Default: first `stilltrue.config.*` in the current directory. |
+| `--only <name,name>` | Run only these checks. |
+| `--json <path>` | Write the result array as JSON. |
+| `--record` | Append this run to history. |
+| `--history <path>` | History file for `--record`. Default: `.stilltrue/history.jsonl`. |
+
+`stilltrue report`:
+
+| Flag | What it does |
+|---|---|
+| `--history <path>` | History file to read. Default: `.stilltrue/history.jsonl`. |
+| `--out <path>` | HTML output. Default: `stilltrue-report.html`. |
+| `--title <text>` | Page `<title>` and heading. Default: `stilltrue drift report`. |
+
 ## Running it automatically every week
 
 If your project is on GitHub, add this file as
@@ -152,6 +336,67 @@ the model for a retry. A broken checker never blocks your app — it fails
 open. See the [project docs](docs/BRIEF.md) and the typed API in
 `stilltrue`'s exports (`runVerify`, `zodStage`, `generateVerified`).
 
+**Bring your own judge — any provider, or none.** stilltrue ships no AI
+and never touches a model key: a checkpoint is just a function you supply,
+so the judge stage can call Gemini, OpenAI, Anthropic, a local model — or
+be a plain schema/rules check with no AI at all. Swapping providers means
+editing your function, never this package.
+
+### The common case, in six lines
+
+Most people arrive wanting one thing: *does this source actually support
+this claim?* `supportStage` is that check, ready-made. You bring a function
+that calls your model; it brings the prompt, the response schema, and the
+guard.
+
+```js
+import { runVerify, supportStage } from 'stilltrue';
+
+const stage = supportStage({
+  // The only required option. Return the model's reply — parsed or raw JSON.
+  judge: ({ system, user, schema }) => callMyModel({ system, user, schema }),
+});
+
+const result = await runVerify([stage], { claim, source });
+// result.ok      → true when the source supports the claim
+// result.value   → { verdict, reasoning, evidence, contradiction, guard }
+// result.messages → why it was rejected, ready for a retry
+```
+
+`verdict` is one of `supported`, `not_supported` or `cant_tell`. Silence is
+not support: a source that never addresses the claim returns `cant_tell`,
+even when the claim happens to be true.
+
+**"Not supported" has to earn itself.** That verdict tells a reader the
+source *says otherwise*, which is the strongest thing a judge can assert
+and the one most worth being wrong about. So the stage requires the judge
+to quote the contradicting sentence, and checks that quote against the
+source in code. If it cannot point at real text, the verdict is downgraded
+to `cant_tell` and a warning explains why. The check only ever downgrades —
+a missing quote is evidence of nothing except a missing quote.
+
+That guard is not theoretical. Judged by a small local model, adding two
+lines to a source that never mentioned the claim flipped a verdict from
+`supported` to `not_supported` — three runs each way, so not noise — where
+a frontier model stayed stable. A package that promises engine-agnostic
+verification has to survive the weaker engine.
+
+Useful options, all defaulted: `minRun` (how much of the contradiction
+quote must match the source — 24 characters, measured on civic notices and
+fee tables, not universal), `requireContradiction: false` to switch the
+guard off, `treatCantTellAs: 'warn'` to pass unanswerable claims instead of
+rejecting them, and `system` to replace the prompt entirely. `SUPPORT_SYSTEM`
+and `SUPPORT_SCHEMA` are exported too, so a provider with constrained
+decoding can take the schema directly.
+
+### Writing your own stages
+
+`supportStage` is one stage; a pipeline is a list of them, and any function
+of the right shape will do. **[docs/VERIFY.md](docs/VERIFY.md)** is the full
+walkthrough — the judging prompt explained, response mapping, retries, and
+the generate → verify → retry loop — for when the ready-made stage is not
+the shape you need.
+
 ## What about testing prompts?
 
 That's a different job with a great existing tool:

package/dist/cli.js

@@ -103,9 +103,9 @@ try {
                 console.log(`kept existing ${f} (never overwritten)`);
             if (created.length) {
                 console.log('\nNext steps:\n' +
-                    '  1. Edit stilltrue.config.mjs — point source at the real page(s) your facts came from\n' +
-                    '  2. Edit data/example-facts.json — or point the config at a facts file you already have\n' +
-                    '  3. Run: npx stilltrue drift\n' +
+                    '  1. Edit stilltrue.config.mjs — point source at the real page(s) your facts came from,\n' +
+                    '     and replace the example markers with your own facts (plain strings)\n' +
+                    '  2. Run: npx stilltrue drift\n' +
                     'Picking good markers: docs/MARKERS.md in the stilltrue repo');
             }
             process.exit(0);

package/dist/index.d.ts

@@ -3,3 +3,4 @@ export { corpus, json, surname, htmlToText, type CorpusOptions } from './helpers
 export { runDrift, runDriftCheck, type RunDriftOptions } from './runner.ts';
 export { resolveCompare } from './compare.ts';
 export { runVerify, zodStage, formatRejection, generateVerified, type VerifyStage, type VerifyResult, type StageOutcome, type StageContext, type RunVerifyOptions, type GenerateVerifiedOptions, } from './verify.ts';
+export { supportStage, supportPrompt, guardJudgement, longestRun, SUPPORT_SYSTEM, SUPPORT_SCHEMA, type Verdict, type SupportInput, type SupportJudgement, type SupportResult, type SupportStageOptions, type Judge, type JudgeRequest, } from './support.ts';

package/dist/index.js

@@ -3,3 +3,4 @@ export { corpus, json, surname, htmlToText } from "./helpers.js";
 export { runDrift, runDriftCheck } from "./runner.js";
 export { resolveCompare } from "./compare.js";
 export { runVerify, zodStage, formatRejection, generateVerified, } from "./verify.js";
+export { supportStage, supportPrompt, guardJudgement, longestRun, SUPPORT_SYSTEM, SUPPORT_SCHEMA, } from "./support.js";

package/dist/init.d.ts

@@ -2,5 +2,5 @@ export interface InitResult {
     created: string[];
     skipped: string[];
 }
-/** Write starter config + example data file. Never overwrites. */
+/** Write the starter config. Never overwrites an existing one. */
 export declare function init(dir: string): Promise<InitResult>;

package/dist/init.js

@@ -1,4 +1,4 @@
-import { mkdir, writeFile } from 'node:fs/promises';
+import { writeFile } from 'node:fs/promises';
 import { existsSync } from 'node:fs';
 import { join } from 'node:path';
 const CONFIG_TEMPLATE = `// stilltrue configuration — one entry per fact-set you want watched.
@@ -10,47 +10,53 @@ export default defineStilltrue({
   drift: [
     {
       // A short id for this check — shows up in output and reports.
-      name: 'example-board-roster',
+      name: 'my-first-check',
 
       // WHERE THE TRUTH LIVES: the official page(s) your facts came from.
       // corpus() fetches each page and strips it down to readable text.
-      // If one page is unreachable it warns; only if ALL are down does the
-      // check report an error (which never fails your build).
       source: corpus([
-        'https://example.org/board',
-        // 'https://example.org/about',   // add more pages if facts span several
+        'https://example.org/about',
+        // 'https://example.org/board',   // add more pages if facts span several
       ]),
 
-      // WHAT YOU BELIEVE: read your saved facts file and pick out the
-      // "markers" — short, stable strings that must still appear on the
-      // page(s) above. Here: each member's last name, plus one number.
-      expect: json('./data/example-facts.json', (data) => [
-        ...data.members.map((m) => surname(m.name)),
-        \`\${data.totalSchools} schools\`,
-      ]),
+      // WHAT YOU BELIEVE: list the facts as short strings ("markers").
+      // Every one of these must still appear somewhere on the page(s)
+      // above, or the check fails and names the missing one.
+      expect: [
+        'Jane Smith',
+        'Main Street Office',
+        '12 locations',
+      ],
 
       // HOW TO COMPARE: 'contains-all' = every marker must appear in the
-      // fetched text (case-insensitive). Any missing marker = rot = the
-      // build fails and tells you which fact went stale.
+      // fetched page text (case-insensitive).
       compare: 'contains-all',
     },
+
+    // ── Leveling up (optional) ──────────────────────────────────────────
+    //
+    // Instead of listing markers inline, you can keep them in a file.
+    // If ./data/markers.json contains just a list of strings —
+    //   ["Jane Smith", "Main Street Office", "12 locations"]
+    // — then this works with no code at all:
+    //
+    //   expect: json('./data/markers.json'),
+    //
+    // And if your app already keeps structured data — say
+    // ./data/board.json looks like
+    //   { "members": [ { "name": "Dr. Jane Smith" }, { "name": "Bob Jones Jr." } ] }
+    // — you can derive the markers from it, so your data file stays the
+    // single source of truth (surname() keeps just "Smith" / "Jones",
+    // because pages change titles and first names more than surnames):
+    //
+    //   expect: json('./data/board.json', (data) =>
+    //     data.members.map((m) => surname(m.name))
+    //   ),
   ],
 });
 `;
-const DATA_TEMPLATE = `{
-  "_comment": "Example facts file - replace with your own data, or point the config at a file you already have.",
-  "members": [
-    { "name": "Dr. Jane Smith" },
-    { "name": "Robert Jones Jr." }
-  ],
-  "totalSchools": 12
-}
-`;
-/** Write starter config + example data file. Never overwrites. */
+/** Write the starter config. Never overwrites an existing one. */
 export async function init(dir) {
-    const created = [];
-    const skipped = [];
-    const configPath = join(dir, 'stilltrue.config.mjs');
     const existingConfig = [
         'stilltrue.config.mjs',
         'stilltrue.config.ts',
@@ -59,20 +65,8 @@ export async function init(dir) {
         'stilltrue.config.cjs',
     ].find((name) => existsSync(join(dir, name)));
     if (existingConfig) {
-        skipped.push(existingConfig);
-    }
-    else {
-        await writeFile(configPath, CONFIG_TEMPLATE);
-        created.push('stilltrue.config.mjs');
-    }
-    const dataPath = join(dir, 'data', 'example-facts.json');
-    if (existsSync(dataPath)) {
-        skipped.push('data/example-facts.json');
-    }
-    else {
-        await mkdir(join(dir, 'data'), { recursive: true });
-        await writeFile(dataPath, DATA_TEMPLATE);
-        created.push('data/example-facts.json');
+        return { created: [], skipped: [existingConfig] };
     }
-    return { created, skipped };
+    await writeFile(join(dir, 'stilltrue.config.mjs'), CONFIG_TEMPLATE);
+    return { created: ['stilltrue.config.mjs'], skipped: [] };
 }

package/dist/support.d.ts

@@ -0,0 +1,157 @@
+/**
+ * supportStage — does this SOURCE actually support this CLAIM?
+ *
+ * The verify half of this package shipped one ready-made stage (`zodStage`, a
+ * schema check) and otherwise handed callers an interface: implement
+ * VerifyStage, understand StageOutcome, write your own judge — prompt, response
+ * schema, parsing, guard. That is a lot of scaffolding in front of the single
+ * question people arrive here to ask.
+ *
+ * This is that scaffolding, shipped. You bring a function that calls a model;
+ * everything else has a default. stilltrue still ships no AI, no key and no
+ * provider dependency — the judge is a function, and this file never learns
+ * what is behind it.
+ *
+ * See docs/SUPPORT_STAGE_PLAN.md for the measurements that produced the
+ * defaults below. They are not taste; each one was paid for.
+ */
+import type { VerifyStage } from './verify.ts';
+export type Verdict = 'supported' | 'not_supported' | 'cant_tell';
+export interface SupportInput {
+    claim: string;
+    source: string;
+}
+export interface SupportJudgement {
+    verdict: Verdict;
+    reasoning: string;
+    /** The sentence that decided it, quoted from the source. */
+    evidence: string | null;
+    /** For not_supported only: the sentence that says otherwise, verbatim. */
+    contradiction: string | null;
+}
+/** What the stage puts in `result.value`: the input plus what the judge said. */
+export interface SupportResult extends SupportInput, SupportJudgement {
+    /**
+     * Why the verdict is what it is, when the guard intervened:
+     * - `verified` — a contradicting quote was found in the source
+     * - `no-contradiction-quoted` — none offered, verdict downgraded
+     * - `quote-not-in-source` — one offered but not present, verdict downgraded
+     * - `null` — the guard had no opinion (verdict was not not_supported)
+     */
+    guard: 'verified' | 'no-contradiction-quoted' | 'quote-not-in-source' | null;
+}
+export interface JudgeRequest {
+    system: string;
+    user: string;
+    /**
+     * The response shape, for providers that support constrained decoding. Hand
+     * this straight to the provider where you can: a grammar-enforced enum makes
+     * an out-of-band verdict unrepresentable rather than merely unlikely, which
+     * is a stronger guarantee than asking politely for JSON.
+     */
+    schema: typeof SUPPORT_SCHEMA;
+}
+/** Return parsed JSON, or the raw string — both are accepted. */
+export type Judge = (req: JudgeRequest) => unknown | Promise<unknown>;
+export interface SupportStageOptions {
+    judge: Judge;
+    name?: string;
+    /**
+     * Minimum contiguous run, in characters, that a contradiction quote must
+     * share with the source.
+     *
+     * 24 is MEASURED, not chosen, and measured on ONE corpus — civic notices,
+     * fee schedules and ordinances. Contradictions written in prose ran 76-96
+     * characters, but a contradicting line in a fee table ("Seniors 65 and over
+     * $1.00.") is only 26, and a first guess of 40 silently discarded those.
+     * Treat it as a starting point for your corpus, not a tuned constant.
+     */
+    minRun?: number;
+    /**
+     * Require a not_supported verdict to quote the contradicting sentence, and
+     * downgrade to cant_tell when it cannot. Default true.
+     *
+     * Off restores naive behaviour, which is sometimes what you want — this
+     * package should not dictate policy to a caller whose corpus differs.
+     */
+    requireContradiction?: boolean;
+    /** How to treat cant_tell. 'reject' (default) or 'warn' and pass. */
+    treatCantTellAs?: 'reject' | 'warn';
+    /** Replace the shipped judge prompt entirely. */
+    system?: string;
+    /** Default 'open': a judge outage warns rather than rejects. */
+    failPolicy?: 'open' | 'closed';
+    timeoutMs?: number;
+    tier?: 'inline' | 'ci';
+}
+/**
+ * The shipped judge prompt.
+ *
+ * The line about qualification is load-bearing rather than decorative.
+ * Measured 2026-09-04 without it: adding two lines to a source that never
+ * mention the claim flipped a verdict from supported to not_supported — three
+ * runs each way, so not sampling noise — on a local model where a frontier
+ * model stayed stable. A package promising engine-agnostic verification has to
+ * survive the weaker engine, so the instruction ships by default.
+ */
+export declare const SUPPORT_SYSTEM = "You are a strict fact-check judge. Decide whether the SOURCE supports the CLAIM using ONLY the source text \u2014 no outside knowledge, no charitable guessing.\nVerdicts:\n- \"supported\": the source affirmatively supports the claim.\n- \"not_supported\": the source contradicts the claim or asserts something incompatible with it.\n- \"cant_tell\": the source does not address the claim, or addresses it only partially. Silence is NOT support.\nA qualification is not a contradiction. If the source substantially supports the claim but adds an exception, a carve-out or a condition, it has NOT said otherwise \u2014 do not rule not_supported. Reserve not_supported for a source that asserts something the claim cannot be true alongside.\nTo rule \"not_supported\" you MUST fill \"contradiction\" with the exact sentence from the source that says otherwise, copied verbatim. If no such sentence exists, you may not use that verdict.\nReturn STRICT JSON only:\n{\"verdict\":\"supported\"|\"not_supported\"|\"cant_tell\",\"reasoning\":\"<one or two plain sentences>\",\"evidence\":\"<short verbatim quote from the source that decided it, or null>\",\"contradiction\":\"<verbatim sentence that contradicts the claim, or null>\"}";
+/** JSON Schema for the judgement, for providers that constrain decoding. */
+export declare const SUPPORT_SCHEMA: {
+    readonly type: "object";
+    readonly additionalProperties: false;
+    readonly properties: {
+        readonly verdict: {
+            readonly type: "string";
+            readonly enum: readonly ["supported", "not_supported", "cant_tell"];
+        };
+        readonly reasoning: {
+            readonly type: "string";
+        };
+        readonly evidence: {
+            readonly type: readonly ["string", "null"];
+        };
+        readonly contradiction: {
+            readonly type: readonly ["string", "null"];
+        };
+    };
+    readonly required: readonly ["verdict", "reasoning", "evidence", "contradiction"];
+};
+/**
+ * SOURCE first, CLAIM last — deliberately.
+ *
+ * Prefix caching is an exact-token match, so putting the long, stable source
+ * ahead of the short, variable claim means a second claim checked against the
+ * SAME source reuses the cached prefix. That is the obvious next thing a user
+ * does. Claim-first diverges at token one and pays full prefill every time.
+ */
+export declare function supportPrompt(input: SupportInput): string;
+/**
+ * Longest run of `needle` that appears contiguously in `haystack`.
+ *
+ * Whole-string matching was tried first and was too strict: a judge quoted
+ * "dogs must remain on a leash ..." where the source read "Dogs are permitted
+ * in all county parks BUT must remain on a leash ...". Tidying a quote's
+ * subject into a grammatical sentence is what careful writers do, and exact
+ * matching discarded a correct verdict for it. A long contiguous run still
+ * proves the judge was reading real text — it cannot invent two dozen
+ * consecutive characters that happen to be in the source.
+ */
+export declare function longestRun(needle: string, haystack: string): number;
+/**
+ * Make "not supported" earn itself.
+ *
+ * That verdict tells a reader the source SAYS OTHERWISE, which is a strong
+ * claim and the one most worth being wrong about. This only ever DOWNGRADES:
+ * a missing quote is evidence of nothing except a missing quote, so nothing is
+ * ever promoted to supported on its strength.
+ */
+export declare function guardJudgement(judgement: SupportJudgement, source: string, minRun: number): SupportResult['guard'];
+/**
+ * A ready-made verify stage that asks whether a source supports a claim.
+ *
+ * ```js
+ * const stage = supportStage({ judge: ({ system, user }) => callMyModel(system, user) });
+ * const result = await runVerify([stage], { claim, source });
+ * ```
+ */
+export declare function supportStage(options: SupportStageOptions): VerifyStage<any>;

package/dist/support.js

@@ -0,0 +1,188 @@
+/**
+ * supportStage — does this SOURCE actually support this CLAIM?
+ *
+ * The verify half of this package shipped one ready-made stage (`zodStage`, a
+ * schema check) and otherwise handed callers an interface: implement
+ * VerifyStage, understand StageOutcome, write your own judge — prompt, response
+ * schema, parsing, guard. That is a lot of scaffolding in front of the single
+ * question people arrive here to ask.
+ *
+ * This is that scaffolding, shipped. You bring a function that calls a model;
+ * everything else has a default. stilltrue still ships no AI, no key and no
+ * provider dependency — the judge is a function, and this file never learns
+ * what is behind it.
+ *
+ * See docs/SUPPORT_STAGE_PLAN.md for the measurements that produced the
+ * defaults below. They are not taste; each one was paid for.
+ */
+/**
+ * The shipped judge prompt.
+ *
+ * The line about qualification is load-bearing rather than decorative.
+ * Measured 2026-09-04 without it: adding two lines to a source that never
+ * mention the claim flipped a verdict from supported to not_supported — three
+ * runs each way, so not sampling noise — on a local model where a frontier
+ * model stayed stable. A package promising engine-agnostic verification has to
+ * survive the weaker engine, so the instruction ships by default.
+ */
+export const SUPPORT_SYSTEM = `You are a strict fact-check judge. Decide whether the SOURCE supports the CLAIM using ONLY the source text — no outside knowledge, no charitable guessing.
+Verdicts:
+- "supported": the source affirmatively supports the claim.
+- "not_supported": the source contradicts the claim or asserts something incompatible with it.
+- "cant_tell": the source does not address the claim, or addresses it only partially. Silence is NOT support.
+A qualification is not a contradiction. If the source substantially supports the claim but adds an exception, a carve-out or a condition, it has NOT said otherwise — do not rule not_supported. Reserve not_supported for a source that asserts something the claim cannot be true alongside.
+To rule "not_supported" you MUST fill "contradiction" with the exact sentence from the source that says otherwise, copied verbatim. If no such sentence exists, you may not use that verdict.
+Return STRICT JSON only:
+{"verdict":"supported"|"not_supported"|"cant_tell","reasoning":"<one or two plain sentences>","evidence":"<short verbatim quote from the source that decided it, or null>","contradiction":"<verbatim sentence that contradicts the claim, or null>"}`;
+/** JSON Schema for the judgement, for providers that constrain decoding. */
+export const SUPPORT_SCHEMA = {
+    type: 'object',
+    additionalProperties: false,
+    properties: {
+        verdict: { type: 'string', enum: ['supported', 'not_supported', 'cant_tell'] },
+        reasoning: { type: 'string' },
+        evidence: { type: ['string', 'null'] },
+        contradiction: { type: ['string', 'null'] },
+    },
+    required: ['verdict', 'reasoning', 'evidence', 'contradiction'],
+};
+/**
+ * SOURCE first, CLAIM last — deliberately.
+ *
+ * Prefix caching is an exact-token match, so putting the long, stable source
+ * ahead of the short, variable claim means a second claim checked against the
+ * SAME source reuses the cached prefix. That is the obvious next thing a user
+ * does. Claim-first diverges at token one and pays full prefill every time.
+ */
+export function supportPrompt(input) {
+    return `SOURCE:\n${input.source}\n\nCLAIM:\n${input.claim}`;
+}
+const VERDICTS = ['supported', 'not_supported', 'cant_tell'];
+/** Whitespace and quote-style differences must not decide a match. */
+function normalise(s) {
+    return String(s ?? '')
+        .toLowerCase()
+        .replace(/[\s‘’“”"']+/g, ' ')
+        .trim();
+}
+/**
+ * Longest run of `needle` that appears contiguously in `haystack`.
+ *
+ * Whole-string matching was tried first and was too strict: a judge quoted
+ * "dogs must remain on a leash ..." where the source read "Dogs are permitted
+ * in all county parks BUT must remain on a leash ...". Tidying a quote's
+ * subject into a grammatical sentence is what careful writers do, and exact
+ * matching discarded a correct verdict for it. A long contiguous run still
+ * proves the judge was reading real text — it cannot invent two dozen
+ * consecutive characters that happen to be in the source.
+ */
+export function longestRun(needle, haystack) {
+    let best = 0;
+    for (let i = 0; i < needle.length; i += 1) {
+        if (needle.length - i <= best)
+            break; // no run from here can beat `best`
+        let lo = best + 1;
+        let hi = needle.length - i;
+        while (lo <= hi) {
+            const mid = (lo + hi) >> 1;
+            if (haystack.includes(needle.slice(i, i + mid))) {
+                best = mid;
+                lo = mid + 1;
+            }
+            else {
+                hi = mid - 1;
+            }
+        }
+    }
+    return best;
+}
+/**
+ * Make "not supported" earn itself.
+ *
+ * That verdict tells a reader the source SAYS OTHERWISE, which is a strong
+ * claim and the one most worth being wrong about. This only ever DOWNGRADES:
+ * a missing quote is evidence of nothing except a missing quote, so nothing is
+ * ever promoted to supported on its strength.
+ */
+export function guardJudgement(judgement, source, minRun) {
+    if (judgement.verdict !== 'not_supported')
+        return null;
+    const quote = normalise(judgement.contradiction);
+    // Below a floor, a "quote" is not evidence — "not", "free" and "$14" would
+    // all match almost any source.
+    if (quote.length < Math.min(15, minRun))
+        return 'no-contradiction-quoted';
+    return longestRun(quote, normalise(source)) >= minRun ? 'verified' : 'quote-not-in-source';
+}
+function parseJudgement(raw) {
+    let obj = raw;
+    if (typeof raw === 'string') {
+        // Providers that cannot constrain decoding often wrap JSON in prose or a
+        // fenced block. Recover the object rather than failing the whole stage.
+        try {
+            obj = JSON.parse(raw);
+        }
+        catch {
+            const match = raw.match(/\{[\s\S]*\}/);
+            if (!match)
+                throw new Error('judge returned no JSON');
+            obj = JSON.parse(match[0]);
+        }
+    }
+    if (!obj || typeof obj !== 'object')
+        throw new Error('judge returned no object');
+    return {
+        // Coerce, do not trust: only a grammar-constrained provider guarantees the
+        // enum, and an unrecognised verdict must not become a confident answer.
+        verdict: VERDICTS.includes(obj.verdict) ? obj.verdict : 'cant_tell',
+        reasoning: typeof obj.reasoning === 'string' ? obj.reasoning : '',
+        evidence: typeof obj.evidence === 'string' ? obj.evidence : null,
+        contradiction: typeof obj.contradiction === 'string' ? obj.contradiction : null,
+    };
+}
+/**
+ * A ready-made verify stage that asks whether a source supports a claim.
+ *
+ * ```js
+ * const stage = supportStage({ judge: ({ system, user }) => callMyModel(system, user) });
+ * const result = await runVerify([stage], { claim, source });
+ * ```
+ */
+export function supportStage(options) {
+    const { judge, name = 'support', minRun = 24, requireContradiction = true, treatCantTellAs = 'reject', system = SUPPORT_SYSTEM, failPolicy = 'open', timeoutMs, tier, } = options;
+    if (typeof judge !== 'function') {
+        throw new TypeError('supportStage requires a `judge` function — you bring the model');
+    }
+    return {
+        name,
+        failPolicy,
+        ...(timeoutMs ? { timeoutMs } : {}),
+        ...(tier ? { tier } : {}),
+        check: async (input, ctx) => {
+            if (!input || typeof input.claim !== 'string' || typeof input.source !== 'string') {
+                return { ok: false, messages: ['supportStage expects { claim, source } as the input value'] };
+            }
+            // A throw here is an infrastructure failure and follows failPolicy — the
+            // stage does not swallow it into a verdict.
+            const raw = await judge({ system, user: supportPrompt(input), schema: SUPPORT_SCHEMA });
+            const judgement = parseJudgement(raw);
+            const guard = requireContradiction ? guardJudgement(judgement, input.source, minRun) : null;
+            const verdict = guard && guard !== 'verified' ? 'cant_tell' : judgement.verdict;
+            if (guard && guard !== 'verified') {
+                ctx.warn(`${name}: "not supported" was downgraded to "can't tell" (${guard})`);
+            }
+            const value = { ...input, ...judgement, verdict, guard };
+            if (verdict === 'supported')
+                return { ok: true, value };
+            if (verdict === 'cant_tell' && treatCantTellAs === 'warn') {
+                ctx.warn(`${name}: the source does not settle this claim`);
+                return { ok: true, value };
+            }
+            return {
+                ok: false,
+                messages: [judgement.reasoning || `the source does not support this claim (${verdict})`],
+                revision: value,
+            };
+        },
+    };
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "stilltrue",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "Your tests check your code. stilltrue checks your facts. CI checks for AI-in-the-loop apps: drift (curated facts vs live sources), golden (regression evals), verify (judge answers against sources).",
   "keywords": [
     "drift",
@@ -17,13 +17,14 @@
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
   "bin": {
-    "stilltrue": "./dist/cli.js"
+    "stilltrue": "dist/cli.js"
   },
   "files": [
     "dist"
   ],
   "scripts": {
     "build": "tsc",
+    "prepare": "tsc",
     "dev": "tsc --watch",
     "test": "node --test test/*.test.ts"
   },