stilltrue 0.5.1 → 0.7.0

package/CHANGELOG.md

@@ -0,0 +1,57 @@
+# Changelog
+
+## 0.7.0
+
+**A different value now counts as a contradiction.**
+
+The judge used to answer `cant_tell` when a source stated its own figure for
+the thing a claim quantified. Asked whether `apple.com/apple-card` supports
+"Apple Card gives 10% Daily Cash back on every purchase", it reasoned:
+
+> The source states that Apple Card offers 2% Daily Cash for most purchases,
+> 3% at Apple and select merchants, and 1% on the titanium card, but it does
+> not mention a 10% Daily Cash rate.
+
+It had read every rate on the page and still treated the absence of the
+claimed number as the claim being unaddressed. Nothing on that page denies
+10% in words, so the old wording — `cant_tell: the source does not address
+the claim` — left it nowhere else to land.
+
+Explicit denial is the rare shape of a false claim. The common one is a wrong
+number, and a checker that goes quiet on wrong numbers is quiet exactly when
+it is most useful. It also collapsed the distinction the verdict set exists
+for: a claim the source contradicts and a claim the source never mentions
+came back identical.
+
+- `not_supported` now explicitly covers a source stating a DIFFERENT VALUE for
+  what the claim quantifies — a rate, price, date, count, duration or name.
+- `cant_tell` explicitly excludes that case.
+- The contradiction-quote rule gains the matching line: where the conflict is
+  a differing value, the sentence to cite is the one stating the source's own
+  value.
+- "Compare like with like" ships in the same change, because the obvious
+  over-correction is ruling `not_supported` against a page that simply lists
+  several values for different cases.
+
+**Minor, not patch.** The public API is unchanged, but verdicts change for the
+same inputs — and for a library whose product IS the verdict, that should not
+reach a consumer pinned to `^0.6.0` silently.
+
+Verified against a local gpt-oss-20b (the weaker of two available engines),
+five claims against one live page:
+
+| claim | before | after |
+|---|---|---|
+| 3% Daily Cash at Apple | supported | supported |
+| **10% on every purchase** | **cant_tell** | **not_supported** |
+| works with Windows PCs | cant_tell | cant_tell |
+| 2% with Apple Pay | — | supported |
+| no annual fee | — | supported |
+
+The last two are over-correction checks. The contradiction guard did not
+downgrade the 10% verdict, so it arrived with a verbatim quote that passed the
+contiguous-run check — refused with evidence, not merely refused.
+
+## 0.6.0 and earlier
+
+Not recorded here; see the git history.

package/README.md

@@ -23,7 +23,7 @@ update it.
 |---|---|
 | `npx stilltrue drift` | Checks your saved facts against their live sources. Fails loudly if a fact has gone stale. |
 | `npx stilltrue report` | Turns past check runs into a simple web page — green/red grid, what broke and when. |
-| `verify` | For developers: a checkpoint that inspects AI-generated answers before they're shown to users. |
+| `verify` | For developers: a checkpoint that inspects AI-generated answers before they're shown to users. `supportStage` ships the common one — does this source support this claim? |
 
 ## Getting started
 
@@ -70,9 +70,10 @@ This says: *fetch that page, and make sure "Jane Smith", "Main Street
 Office", and "12 locations" all still appear on it.* Swap in your own URL
 and your own facts — that's a working check, no programming required.
 
-Choosing what to assert is the judgment call — surnames beat full names,
-"143 schools" beats a bare "143". The short guide:
-[docs/MARKERS.md](docs/MARKERS.md).
+Every other field, compare mode, and helper is in
+[Configuration](#configuration) below. Choosing *what* to assert is the
+judgment call — surnames beat full names, `"143 schools"` beats a bare
+`"143"`. The short guide: [docs/MARKERS.md](docs/MARKERS.md).
 
 **Leveling up (optional):** you don't have to keep the facts in the config.
 If `./data/markers.json` is just a list of strings, point at it with
@@ -114,6 +115,172 @@ You'll see one line per check:
   This only *warns*. An unreachable site doesn't mean your facts are wrong,
   and an alarm that cries wolf gets ignored — so outages never fail the build.
 
+## Configuration
+
+stilltrue loads a config file from the current directory. The first of
+these that exists wins:
+
+`stilltrue.config.ts` · `.mts` · `.js` · `.mjs` · `.cjs`
+
+(`npx stilltrue init` writes `.mjs`.) Point at a file anywhere with
+`stilltrue drift --config path/to/file`.
+
+Wrap the export with `defineStilltrue` so TypeScript can check it:
+
+```js
+import { defineStilltrue, corpus, json, surname } from 'stilltrue';
+
+export default defineStilltrue({
+  drift: [ /* one entry per fact-set */ ],
+  verify: { /* optional named pipelines — library use, see below */ },
+});
+```
+
+### Drift checks
+
+Each `drift` entry is one curated fact-set vs its live source.
+
+| Field | Required | Default | What it is |
+|---|---|---|---|
+| `name` | yes | — | Short id. Shows up in the terminal, `--only`, and the report. |
+| `source` | yes | — | `async (ctx) => actual`. Fetch/parse the live authority. **Throw** if the source is unreachable — that is outcome `error`, never `rot`. |
+| `expect` | yes | — | The curated facts. A value, or `async (ctx) => expected` (e.g. `json(...)`). A throw here is also `error` (broken config, not rot). |
+| `compare` | no | `'deep-equal'` | How to compare `expect` against what `source` returned. |
+
+`source` and function-form `expect` receive `ctx`:
+
+| | |
+|---|---|
+| `ctx.warn(message)` | Record a non-fatal note (one of several pages down, odd markup). Shows as a warning; does not fail the run. |
+| `ctx.configDir` | Absolute directory of the config file. Data paths resolve from here. |
+
+### `compare`
+
+**`'contains-all'`** — `expect` is a list of marker strings; `source` must
+return a text corpus (what `corpus()` returns). Every marker must appear
+somewhere in that text, case-insensitively. Extra text on the page is
+ignored. This is the webpage-watch check.
+
+**`'deep-equal'`** — structural equality. Extra keys, missing keys, and
+value mismatches all rot, with pathed messages
+(`$.members[2].name: expected "Smith", source has "Jones"`). Use this when
+`source()` already returns structured data that should match `expect`
+exactly.
+
+**A function** — `(expected, actual) => string[]`. Return `[]` to pass;
+each string is one mismatch (outcome `rot`). Throw if the shapes are
+unusable (outcome `error`). Use this for numeric tolerance, order-insensitive
+lists, or any comparison that isn't "markers in a page" or "these two
+objects are identical."
+
+```js
+compare: (expected, actual) =>
+  expected.rate === actual.rate
+    ? []
+    : [`tax rate ${expected.rate} → ${actual.rate}`],
+```
+
+Surnames are a *marker choice*, not a compare mode: derive them from your
+data with `surname()` and use `'contains-all'`.
+
+### Helpers
+
+These are the built-in `source` / `expect` builders. You can always write
+your own functions instead.
+
+**`corpus(urls, options?)`** — fetch pages, strip HTML to text,
+concatenate. Individual unreachable pages call `ctx.warn`; if *every* page
+fails, it throws (`error`). Pair with `'contains-all'`.
+
+| Option | Default | What it is |
+|---|---|---|
+| `timeoutMs` | `20000` | Per-page fetch timeout, milliseconds. |
+| `userAgent` | `stilltrue drift check (github.com/sg-bfoster/stilltrue)` | Sent on each request so operators can identify the check. |
+
+```js
+source: corpus(
+  ['https://example.org/about', 'https://example.org/board'],
+  { timeoutMs: 15_000, userAgent: 'my-app drift check' },
+),
+```
+
+**`json(path, derive?)`** — read a JSON file relative to the config file.
+With no `derive`, the parsed value *is* the expectation (typical with
+`'deep-equal'`). With `derive`, you map that value into whatever `compare`
+needs — usually a marker list:
+
+```js
+expect: json('./data/board.json', (data) =>
+  data.members.map((m) => surname(m.name)),
+),
+```
+
+**`surname(fullName)`** — last token of a name, with leading honorifics
+(`Dr.`, `Mr.`, `Mrs.`, `Ms.`) and trailing generational suffixes (`Jr.`,
+`Sr.`, `II`–`V`) stripped. `"Dr. Jane Smith III"` → `"Smith"`.
+
+**`htmlToText(html)`** — the same tag-stripping `corpus()` uses. For a
+custom `source` that already has HTML in hand.
+
+### Custom `source`
+
+A throw is an outage; a return value is data to compare. Warn for partial
+trouble:
+
+```js
+source: async (ctx) => {
+  const res = await fetch('https://example.org/api/rates');
+  if (!res.ok) throw new Error(`HTTP ${res.status}`);
+  const data = await res.json();
+  if (!data.updatedAt) ctx.warn('response missing updatedAt');
+  return data;
+},
+expect: json('./data/rates.json'),
+// compare defaults to 'deep-equal'
+```
+
+### `verify` pipelines
+
+`verify` is an optional map of named stage lists, for use from your app
+(`runVerify(config.verify.answers, draft)`). There is no `stilltrue verify`
+CLI command — verification runs in the request path, not in CI.
+
+Each stage:
+
+| Field | Required | Default | What it is |
+|---|---|---|---|
+| `name` | yes | — | Id of this checkpoint. The first rejection reports this name. |
+| `check` | yes | — | `(input, ctx) => { ok: true, value? } \| { ok: false, messages, revision? }`. |
+| `failPolicy` | no | `'open'` | What a **thrown** stage means. `'open'`: warn and continue (a judge outage must not block the app). `'closed'`: treat the throw as a rejection. |
+| `timeoutMs` | no | none | Abort the stage after this many ms. The timeout follows `failPolicy`. |
+| `tier` | no | `'inline'` | `'inline'` always runs; `'ci'` runs only when you call `runVerify(..., { tier: 'ci' })`. |
+
+Stages run in order and stop at the first rejection. `zodStage(name, schema)`
+builds a parse stage from a Zod schema.
+`generateVerified({ generate, stages, attempts, tier })` is the generate →
+verify → retry loop (`attempts` defaults to `2`). See
+[For developers: `verify`](#for-developers-verify) below.
+
+### CLI flags
+
+`stilltrue drift`:
+
+| Flag | What it does |
+|---|---|
+| `--config <path>` | Config file. Default: first `stilltrue.config.*` in the current directory. |
+| `--only <name,name>` | Run only these checks. |
+| `--json <path>` | Write the result array as JSON. |
+| `--record` | Append this run to history. |
+| `--history <path>` | History file for `--record`. Default: `.stilltrue/history.jsonl`. |
+
+`stilltrue report`:
+
+| Flag | What it does |
+|---|---|
+| `--history <path>` | History file to read. Default: `.stilltrue/history.jsonl`. |
+| `--out <path>` | HTML output. Default: `stilltrue-report.html`. |
+| `--title <text>` | Page `<title>` and heading. Default: `stilltrue drift report`. |
+
 ## Running it automatically every week
 
 If your project is on GitHub, add this file as
@@ -169,6 +336,67 @@ the model for a retry. A broken checker never blocks your app — it fails
 open. See the [project docs](docs/BRIEF.md) and the typed API in
 `stilltrue`'s exports (`runVerify`, `zodStage`, `generateVerified`).
 
+**Bring your own judge — any provider, or none.** stilltrue ships no AI
+and never touches a model key: a checkpoint is just a function you supply,
+so the judge stage can call Gemini, OpenAI, Anthropic, a local model — or
+be a plain schema/rules check with no AI at all. Swapping providers means
+editing your function, never this package.
+
+### The common case, in six lines
+
+Most people arrive wanting one thing: *does this source actually support
+this claim?* `supportStage` is that check, ready-made. You bring a function
+that calls your model; it brings the prompt, the response schema, and the
+guard.
+
+```js
+import { runVerify, supportStage } from 'stilltrue';
+
+const stage = supportStage({
+  // The only required option. Return the model's reply — parsed or raw JSON.
+  judge: ({ system, user, schema }) => callMyModel({ system, user, schema }),
+});
+
+const result = await runVerify([stage], { claim, source });
+// result.ok      → true when the source supports the claim
+// result.value   → { verdict, reasoning, evidence, contradiction, guard }
+// result.messages → why it was rejected, ready for a retry
+```
+
+`verdict` is one of `supported`, `not_supported` or `cant_tell`. Silence is
+not support: a source that never addresses the claim returns `cant_tell`,
+even when the claim happens to be true.
+
+**"Not supported" has to earn itself.** That verdict tells a reader the
+source *says otherwise*, which is the strongest thing a judge can assert
+and the one most worth being wrong about. So the stage requires the judge
+to quote the contradicting sentence, and checks that quote against the
+source in code. If it cannot point at real text, the verdict is downgraded
+to `cant_tell` and a warning explains why. The check only ever downgrades —
+a missing quote is evidence of nothing except a missing quote.
+
+That guard is not theoretical. Judged by a small local model, adding two
+lines to a source that never mentioned the claim flipped a verdict from
+`supported` to `not_supported` — three runs each way, so not noise — where
+a frontier model stayed stable. A package that promises engine-agnostic
+verification has to survive the weaker engine.
+
+Useful options, all defaulted: `minRun` (how much of the contradiction
+quote must match the source — 24 characters, measured on civic notices and
+fee tables, not universal), `requireContradiction: false` to switch the
+guard off, `treatCantTellAs: 'warn'` to pass unanswerable claims instead of
+rejecting them, and `system` to replace the prompt entirely. `SUPPORT_SYSTEM`
+and `SUPPORT_SCHEMA` are exported too, so a provider with constrained
+decoding can take the schema directly.
+
+### Writing your own stages
+
+`supportStage` is one stage; a pipeline is a list of them, and any function
+of the right shape will do. **[docs/VERIFY.md](docs/VERIFY.md)** is the full
+walkthrough — the judging prompt explained, response mapping, retries, and
+the generate → verify → retry loop — for when the ready-made stage is not
+the shape you need.
+
 ## What about testing prompts?
 
 That's a different job with a great existing tool:

package/dist/index.d.ts

@@ -3,3 +3,4 @@ export { corpus, json, surname, htmlToText, type CorpusOptions } from './helpers
 export { runDrift, runDriftCheck, type RunDriftOptions } from './runner.ts';
 export { resolveCompare } from './compare.ts';
 export { runVerify, zodStage, formatRejection, generateVerified, type VerifyStage, type VerifyResult, type StageOutcome, type StageContext, type RunVerifyOptions, type GenerateVerifiedOptions, } from './verify.ts';
+export { supportStage, supportPrompt, guardJudgement, longestRun, SUPPORT_SYSTEM, SUPPORT_SCHEMA, type Verdict, type SupportInput, type SupportJudgement, type SupportResult, type SupportStageOptions, type Judge, type JudgeRequest, } from './support.ts';

package/dist/index.js

@@ -3,3 +3,4 @@ export { corpus, json, surname, htmlToText } from "./helpers.js";
 export { runDrift, runDriftCheck } from "./runner.js";
 export { resolveCompare } from "./compare.js";
 export { runVerify, zodStage, formatRejection, generateVerified, } from "./verify.js";
+export { supportStage, supportPrompt, guardJudgement, longestRun, SUPPORT_SYSTEM, SUPPORT_SCHEMA, } from "./support.js";

package/dist/support.d.ts

@@ -0,0 +1,171 @@
+/**
+ * supportStage — does this SOURCE actually support this CLAIM?
+ *
+ * The verify half of this package shipped one ready-made stage (`zodStage`, a
+ * schema check) and otherwise handed callers an interface: implement
+ * VerifyStage, understand StageOutcome, write your own judge — prompt, response
+ * schema, parsing, guard. That is a lot of scaffolding in front of the single
+ * question people arrive here to ask.
+ *
+ * This is that scaffolding, shipped. You bring a function that calls a model;
+ * everything else has a default. stilltrue still ships no AI, no key and no
+ * provider dependency — the judge is a function, and this file never learns
+ * what is behind it.
+ *
+ * See docs/SUPPORT_STAGE_PLAN.md for the measurements that produced the
+ * defaults below. They are not taste; each one was paid for.
+ */
+import type { VerifyStage } from './verify.ts';
+export type Verdict = 'supported' | 'not_supported' | 'cant_tell';
+export interface SupportInput {
+    claim: string;
+    source: string;
+}
+export interface SupportJudgement {
+    verdict: Verdict;
+    reasoning: string;
+    /** The sentence that decided it, quoted from the source. */
+    evidence: string | null;
+    /** For not_supported only: the sentence that says otherwise, verbatim. */
+    contradiction: string | null;
+}
+/** What the stage puts in `result.value`: the input plus what the judge said. */
+export interface SupportResult extends SupportInput, SupportJudgement {
+    /**
+     * Why the verdict is what it is, when the guard intervened:
+     * - `verified` — a contradicting quote was found in the source
+     * - `no-contradiction-quoted` — none offered, verdict downgraded
+     * - `quote-not-in-source` — one offered but not present, verdict downgraded
+     * - `null` — the guard had no opinion (verdict was not not_supported)
+     */
+    guard: 'verified' | 'no-contradiction-quoted' | 'quote-not-in-source' | null;
+}
+export interface JudgeRequest {
+    system: string;
+    user: string;
+    /**
+     * The response shape, for providers that support constrained decoding. Hand
+     * this straight to the provider where you can: a grammar-enforced enum makes
+     * an out-of-band verdict unrepresentable rather than merely unlikely, which
+     * is a stronger guarantee than asking politely for JSON.
+     */
+    schema: typeof SUPPORT_SCHEMA;
+}
+/** Return parsed JSON, or the raw string — both are accepted. */
+export type Judge = (req: JudgeRequest) => unknown | Promise<unknown>;
+export interface SupportStageOptions {
+    judge: Judge;
+    name?: string;
+    /**
+     * Minimum contiguous run, in characters, that a contradiction quote must
+     * share with the source.
+     *
+     * 24 is MEASURED, not chosen, and measured on ONE corpus — civic notices,
+     * fee schedules and ordinances. Contradictions written in prose ran 76-96
+     * characters, but a contradicting line in a fee table ("Seniors 65 and over
+     * $1.00.") is only 26, and a first guess of 40 silently discarded those.
+     * Treat it as a starting point for your corpus, not a tuned constant.
+     */
+    minRun?: number;
+    /**
+     * Require a not_supported verdict to quote the contradicting sentence, and
+     * downgrade to cant_tell when it cannot. Default true.
+     *
+     * Off restores naive behaviour, which is sometimes what you want — this
+     * package should not dictate policy to a caller whose corpus differs.
+     */
+    requireContradiction?: boolean;
+    /** How to treat cant_tell. 'reject' (default) or 'warn' and pass. */
+    treatCantTellAs?: 'reject' | 'warn';
+    /** Replace the shipped judge prompt entirely. */
+    system?: string;
+    /** Default 'open': a judge outage warns rather than rejects. */
+    failPolicy?: 'open' | 'closed';
+    timeoutMs?: number;
+    tier?: 'inline' | 'ci';
+}
+/**
+ * The shipped judge prompt.
+ *
+ * The "different value" clause was added 2026-09-11 after a measured failure:
+ * asked whether apple.com/apple-card supports "Apple Card gives 10% Daily Cash",
+ * the judge answered cant_tell, reasoning "the source states 2%... 3%... and 1%,
+ * but it does not mention a 10% Daily Cash rate". It had read every rate on the
+ * page and still treated the absence of the claimed number as the claim being
+ * unaddressed. Nothing on that page denies 10% in words, so the original
+ * wording gave it nowhere else to land.
+ *
+ * That is the common shape of a false claim in the wild — a wrong number, not
+ * an explicit denial — so a checker that goes quiet on it is quiet exactly when
+ * it is most needed. The compare-like-with-like line ships alongside because
+ * the obvious over-correction is ruling not_supported on a page that simply
+ * lists several values for different cases.
+ *
+ * The line about qualification is load-bearing rather than decorative.
+ * Measured 2026-09-04 without it: adding two lines to a source that never
+ * mention the claim flipped a verdict from supported to not_supported — three
+ * runs each way, so not sampling noise — on a local model where a frontier
+ * model stayed stable. A package promising engine-agnostic verification has to
+ * survive the weaker engine, so the instruction ships by default.
+ */
+export declare const SUPPORT_SYSTEM = "You are a strict fact-check judge. Decide whether the SOURCE supports the CLAIM using ONLY the source text \u2014 no outside knowledge, no charitable guessing.\nVerdicts:\n- \"supported\": the source affirmatively supports the claim.\n- \"not_supported\": the source contradicts the claim or asserts something incompatible with it. This INCLUDES the source stating a DIFFERENT VALUE for the same thing the claim quantifies \u2014 a rate, price, date, count, duration or name. If the claim says 10% and the source says the rate is 3%, the source has contradicted it; it does not have to mention 10% to do so.\n- \"cant_tell\": the source never addresses the subject of the claim at all, or addresses it only partially. A source that states its own value for the very thing the claim asserts HAS addressed it \u2014 that is not_supported, not cant_tell. Silence is NOT support.\nA qualification is not a contradiction. If the source substantially supports the claim but adds an exception, a carve-out or a condition, it has NOT said otherwise \u2014 do not rule not_supported. Reserve not_supported for a source that asserts something the claim cannot be true alongside.\nTo rule \"not_supported\" you MUST fill \"contradiction\" with the exact sentence from the source that says otherwise, copied verbatim. Where the conflict is a differing value, that sentence is the one stating the SOURCE's own value. If no such sentence exists, you may not use that verdict.\nCompare like with like: a different value only contradicts when it describes the same thing under the same conditions. A source listing several values for different cases has not contradicted a claim that matches one of them.\nReturn STRICT JSON only:\n{\"verdict\":\"supported\"|\"not_supported\"|\"cant_tell\",\"reasoning\":\"<one or two plain sentences>\",\"evidence\":\"<short verbatim quote from the source that decided it, or null>\",\"contradiction\":\"<verbatim sentence that contradicts the claim, or null>\"}";
+/** JSON Schema for the judgement, for providers that constrain decoding. */
+export declare const SUPPORT_SCHEMA: {
+    readonly type: "object";
+    readonly additionalProperties: false;
+    readonly properties: {
+        readonly verdict: {
+            readonly type: "string";
+            readonly enum: readonly ["supported", "not_supported", "cant_tell"];
+        };
+        readonly reasoning: {
+            readonly type: "string";
+        };
+        readonly evidence: {
+            readonly type: readonly ["string", "null"];
+        };
+        readonly contradiction: {
+            readonly type: readonly ["string", "null"];
+        };
+    };
+    readonly required: readonly ["verdict", "reasoning", "evidence", "contradiction"];
+};
+/**
+ * SOURCE first, CLAIM last — deliberately.
+ *
+ * Prefix caching is an exact-token match, so putting the long, stable source
+ * ahead of the short, variable claim means a second claim checked against the
+ * SAME source reuses the cached prefix. That is the obvious next thing a user
+ * does. Claim-first diverges at token one and pays full prefill every time.
+ */
+export declare function supportPrompt(input: SupportInput): string;
+/**
+ * Longest run of `needle` that appears contiguously in `haystack`.
+ *
+ * Whole-string matching was tried first and was too strict: a judge quoted
+ * "dogs must remain on a leash ..." where the source read "Dogs are permitted
+ * in all county parks BUT must remain on a leash ...". Tidying a quote's
+ * subject into a grammatical sentence is what careful writers do, and exact
+ * matching discarded a correct verdict for it. A long contiguous run still
+ * proves the judge was reading real text — it cannot invent two dozen
+ * consecutive characters that happen to be in the source.
+ */
+export declare function longestRun(needle: string, haystack: string): number;
+/**
+ * Make "not supported" earn itself.
+ *
+ * That verdict tells a reader the source SAYS OTHERWISE, which is a strong
+ * claim and the one most worth being wrong about. This only ever DOWNGRADES:
+ * a missing quote is evidence of nothing except a missing quote, so nothing is
+ * ever promoted to supported on its strength.
+ */
+export declare function guardJudgement(judgement: SupportJudgement, source: string, minRun: number): SupportResult['guard'];
+/**
+ * A ready-made verify stage that asks whether a source supports a claim.
+ *
+ * ```js
+ * const stage = supportStage({ judge: ({ system, user }) => callMyModel(system, user) });
+ * const result = await runVerify([stage], { claim, source });
+ * ```
+ */
+export declare function supportStage(options: SupportStageOptions): VerifyStage<any>;

package/dist/support.js

@@ -0,0 +1,203 @@
+/**
+ * supportStage — does this SOURCE actually support this CLAIM?
+ *
+ * The verify half of this package shipped one ready-made stage (`zodStage`, a
+ * schema check) and otherwise handed callers an interface: implement
+ * VerifyStage, understand StageOutcome, write your own judge — prompt, response
+ * schema, parsing, guard. That is a lot of scaffolding in front of the single
+ * question people arrive here to ask.
+ *
+ * This is that scaffolding, shipped. You bring a function that calls a model;
+ * everything else has a default. stilltrue still ships no AI, no key and no
+ * provider dependency — the judge is a function, and this file never learns
+ * what is behind it.
+ *
+ * See docs/SUPPORT_STAGE_PLAN.md for the measurements that produced the
+ * defaults below. They are not taste; each one was paid for.
+ */
+/**
+ * The shipped judge prompt.
+ *
+ * The "different value" clause was added 2026-09-11 after a measured failure:
+ * asked whether apple.com/apple-card supports "Apple Card gives 10% Daily Cash",
+ * the judge answered cant_tell, reasoning "the source states 2%... 3%... and 1%,
+ * but it does not mention a 10% Daily Cash rate". It had read every rate on the
+ * page and still treated the absence of the claimed number as the claim being
+ * unaddressed. Nothing on that page denies 10% in words, so the original
+ * wording gave it nowhere else to land.
+ *
+ * That is the common shape of a false claim in the wild — a wrong number, not
+ * an explicit denial — so a checker that goes quiet on it is quiet exactly when
+ * it is most needed. The compare-like-with-like line ships alongside because
+ * the obvious over-correction is ruling not_supported on a page that simply
+ * lists several values for different cases.
+ *
+ * The line about qualification is load-bearing rather than decorative.
+ * Measured 2026-09-04 without it: adding two lines to a source that never
+ * mention the claim flipped a verdict from supported to not_supported — three
+ * runs each way, so not sampling noise — on a local model where a frontier
+ * model stayed stable. A package promising engine-agnostic verification has to
+ * survive the weaker engine, so the instruction ships by default.
+ */
+export const SUPPORT_SYSTEM = `You are a strict fact-check judge. Decide whether the SOURCE supports the CLAIM using ONLY the source text — no outside knowledge, no charitable guessing.
+Verdicts:
+- "supported": the source affirmatively supports the claim.
+- "not_supported": the source contradicts the claim or asserts something incompatible with it. This INCLUDES the source stating a DIFFERENT VALUE for the same thing the claim quantifies — a rate, price, date, count, duration or name. If the claim says 10% and the source says the rate is 3%, the source has contradicted it; it does not have to mention 10% to do so.
+- "cant_tell": the source never addresses the subject of the claim at all, or addresses it only partially. A source that states its own value for the very thing the claim asserts HAS addressed it — that is not_supported, not cant_tell. Silence is NOT support.
+A qualification is not a contradiction. If the source substantially supports the claim but adds an exception, a carve-out or a condition, it has NOT said otherwise — do not rule not_supported. Reserve not_supported for a source that asserts something the claim cannot be true alongside.
+To rule "not_supported" you MUST fill "contradiction" with the exact sentence from the source that says otherwise, copied verbatim. Where the conflict is a differing value, that sentence is the one stating the SOURCE's own value. If no such sentence exists, you may not use that verdict.
+Compare like with like: a different value only contradicts when it describes the same thing under the same conditions. A source listing several values for different cases has not contradicted a claim that matches one of them.
+Return STRICT JSON only:
+{"verdict":"supported"|"not_supported"|"cant_tell","reasoning":"<one or two plain sentences>","evidence":"<short verbatim quote from the source that decided it, or null>","contradiction":"<verbatim sentence that contradicts the claim, or null>"}`;
+/** JSON Schema for the judgement, for providers that constrain decoding. */
+export const SUPPORT_SCHEMA = {
+    type: 'object',
+    additionalProperties: false,
+    properties: {
+        verdict: { type: 'string', enum: ['supported', 'not_supported', 'cant_tell'] },
+        reasoning: { type: 'string' },
+        evidence: { type: ['string', 'null'] },
+        contradiction: { type: ['string', 'null'] },
+    },
+    required: ['verdict', 'reasoning', 'evidence', 'contradiction'],
+};
+/**
+ * SOURCE first, CLAIM last — deliberately.
+ *
+ * Prefix caching is an exact-token match, so putting the long, stable source
+ * ahead of the short, variable claim means a second claim checked against the
+ * SAME source reuses the cached prefix. That is the obvious next thing a user
+ * does. Claim-first diverges at token one and pays full prefill every time.
+ */
+export function supportPrompt(input) {
+    return `SOURCE:\n${input.source}\n\nCLAIM:\n${input.claim}`;
+}
+const VERDICTS = ['supported', 'not_supported', 'cant_tell'];
+/** Whitespace and quote-style differences must not decide a match. */
+function normalise(s) {
+    return String(s ?? '')
+        .toLowerCase()
+        .replace(/[\s‘’“”"']+/g, ' ')
+        .trim();
+}
+/**
+ * Longest run of `needle` that appears contiguously in `haystack`.
+ *
+ * Whole-string matching was tried first and was too strict: a judge quoted
+ * "dogs must remain on a leash ..." where the source read "Dogs are permitted
+ * in all county parks BUT must remain on a leash ...". Tidying a quote's
+ * subject into a grammatical sentence is what careful writers do, and exact
+ * matching discarded a correct verdict for it. A long contiguous run still
+ * proves the judge was reading real text — it cannot invent two dozen
+ * consecutive characters that happen to be in the source.
+ */
+export function longestRun(needle, haystack) {
+    let best = 0;
+    for (let i = 0; i < needle.length; i += 1) {
+        if (needle.length - i <= best)
+            break; // no run from here can beat `best`
+        let lo = best + 1;
+        let hi = needle.length - i;
+        while (lo <= hi) {
+            const mid = (lo + hi) >> 1;
+            if (haystack.includes(needle.slice(i, i + mid))) {
+                best = mid;
+                lo = mid + 1;
+            }
+            else {
+                hi = mid - 1;
+            }
+        }
+    }
+    return best;
+}
+/**
+ * Make "not supported" earn itself.
+ *
+ * That verdict tells a reader the source SAYS OTHERWISE, which is a strong
+ * claim and the one most worth being wrong about. This only ever DOWNGRADES:
+ * a missing quote is evidence of nothing except a missing quote, so nothing is
+ * ever promoted to supported on its strength.
+ */
+export function guardJudgement(judgement, source, minRun) {
+    if (judgement.verdict !== 'not_supported')
+        return null;
+    const quote = normalise(judgement.contradiction);
+    // Below a floor, a "quote" is not evidence — "not", "free" and "$14" would
+    // all match almost any source.
+    if (quote.length < Math.min(15, minRun))
+        return 'no-contradiction-quoted';
+    return longestRun(quote, normalise(source)) >= minRun ? 'verified' : 'quote-not-in-source';
+}
+function parseJudgement(raw) {
+    let obj = raw;
+    if (typeof raw === 'string') {
+        // Providers that cannot constrain decoding often wrap JSON in prose or a
+        // fenced block. Recover the object rather than failing the whole stage.
+        try {
+            obj = JSON.parse(raw);
+        }
+        catch {
+            const match = raw.match(/\{[\s\S]*\}/);
+            if (!match)
+                throw new Error('judge returned no JSON');
+            obj = JSON.parse(match[0]);
+        }
+    }
+    if (!obj || typeof obj !== 'object')
+        throw new Error('judge returned no object');
+    return {
+        // Coerce, do not trust: only a grammar-constrained provider guarantees the
+        // enum, and an unrecognised verdict must not become a confident answer.
+        verdict: VERDICTS.includes(obj.verdict) ? obj.verdict : 'cant_tell',
+        reasoning: typeof obj.reasoning === 'string' ? obj.reasoning : '',
+        evidence: typeof obj.evidence === 'string' ? obj.evidence : null,
+        contradiction: typeof obj.contradiction === 'string' ? obj.contradiction : null,
+    };
+}
+/**
+ * A ready-made verify stage that asks whether a source supports a claim.
+ *
+ * ```js
+ * const stage = supportStage({ judge: ({ system, user }) => callMyModel(system, user) });
+ * const result = await runVerify([stage], { claim, source });
+ * ```
+ */
+export function supportStage(options) {
+    const { judge, name = 'support', minRun = 24, requireContradiction = true, treatCantTellAs = 'reject', system = SUPPORT_SYSTEM, failPolicy = 'open', timeoutMs, tier, } = options;
+    if (typeof judge !== 'function') {
+        throw new TypeError('supportStage requires a `judge` function — you bring the model');
+    }
+    return {
+        name,
+        failPolicy,
+        ...(timeoutMs ? { timeoutMs } : {}),
+        ...(tier ? { tier } : {}),
+        check: async (input, ctx) => {
+            if (!input || typeof input.claim !== 'string' || typeof input.source !== 'string') {
+                return { ok: false, messages: ['supportStage expects { claim, source } as the input value'] };
+            }
+            // A throw here is an infrastructure failure and follows failPolicy — the
+            // stage does not swallow it into a verdict.
+            const raw = await judge({ system, user: supportPrompt(input), schema: SUPPORT_SCHEMA });
+            const judgement = parseJudgement(raw);
+            const guard = requireContradiction ? guardJudgement(judgement, input.source, minRun) : null;
+            const verdict = guard && guard !== 'verified' ? 'cant_tell' : judgement.verdict;
+            if (guard && guard !== 'verified') {
+                ctx.warn(`${name}: "not supported" was downgraded to "can't tell" (${guard})`);
+            }
+            const value = { ...input, ...judgement, verdict, guard };
+            if (verdict === 'supported')
+                return { ok: true, value };
+            if (verdict === 'cant_tell' && treatCantTellAs === 'warn') {
+                ctx.warn(`${name}: the source does not settle this claim`);
+                return { ok: true, value };
+            }
+            return {
+                ok: false,
+                messages: [judgement.reasoning || `the source does not support this claim (${verdict})`],
+                revision: value,
+            };
+        },
+    };
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "stilltrue",
-  "version": "0.5.1",
+  "version": "0.7.0",
   "description": "Your tests check your code. stilltrue checks your facts. CI checks for AI-in-the-loop apps: drift (curated facts vs live sources), golden (regression evals), verify (judge answers against sources).",
   "keywords": [
     "drift",
@@ -17,13 +17,15 @@
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
   "bin": {
-    "stilltrue": "./dist/cli.js"
+    "stilltrue": "dist/cli.js"
   },
   "files": [
-    "dist"
+    "dist",
+    "CHANGELOG.md"
   ],
   "scripts": {
     "build": "tsc",
+    "prepare": "tsc",
     "dev": "tsc --watch",
     "test": "node --test test/*.test.ts"
   },