stilltrue 0.7.0 → 0.7.1

package/CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## 0.7.1 — 2026-09-13
+
+Docs restructure, no behavior changes except one message:
+
+- README cut from 408 lines to ~190. A fresh-eyes pass (install → first
+  green check in three minutes) showed setup was never the problem — the
+  document performed difficulty the tool doesn't have. The drift
+  quickstart, which tested perfect, is untouched.
+- New `docs/CONFIG.md`: the reference shelf (config fields, compare modes,
+  helpers, CLI flags), moved verbatim out of the tutorial path.
+- `docs/VERIFY.md` gained the `supportStage` section from the README, so
+  the ready-made stage lives beside the judge walkthrough it builds on.
+- `stilltrue init` now prints the full MARKERS.md URL — npm users have no
+  repo checkout for a relative path to resolve against.
+
 ## 0.7.0
 
 **A different value now counts as a contradiction.**

package/README.md

@@ -71,7 +71,7 @@ Office", and "12 locations" all still appear on it.* Swap in your own URL
 and your own facts — that's a working check, no programming required.
 
 Every other field, compare mode, and helper is in
-[Configuration](#configuration) below. Choosing *what* to assert is the
+[the configuration reference](docs/CONFIG.md). Choosing *what* to assert is the
 judgment call — surnames beat full names, `"143 schools"` beats a bare
 `"143"`. The short guide: [docs/MARKERS.md](docs/MARKERS.md).
 
@@ -115,172 +115,6 @@ You'll see one line per check:
   This only *warns*. An unreachable site doesn't mean your facts are wrong,
   and an alarm that cries wolf gets ignored — so outages never fail the build.
 
-## Configuration
-
-stilltrue loads a config file from the current directory. The first of
-these that exists wins:
-
-`stilltrue.config.ts` · `.mts` · `.js` · `.mjs` · `.cjs`
-
-(`npx stilltrue init` writes `.mjs`.) Point at a file anywhere with
-`stilltrue drift --config path/to/file`.
-
-Wrap the export with `defineStilltrue` so TypeScript can check it:
-
-```js
-import { defineStilltrue, corpus, json, surname } from 'stilltrue';
-
-export default defineStilltrue({
-  drift: [ /* one entry per fact-set */ ],
-  verify: { /* optional named pipelines — library use, see below */ },
-});
-```
-
-### Drift checks
-
-Each `drift` entry is one curated fact-set vs its live source.
-
-| Field | Required | Default | What it is |
-|---|---|---|---|
-| `name` | yes | — | Short id. Shows up in the terminal, `--only`, and the report. |
-| `source` | yes | — | `async (ctx) => actual`. Fetch/parse the live authority. **Throw** if the source is unreachable — that is outcome `error`, never `rot`. |
-| `expect` | yes | — | The curated facts. A value, or `async (ctx) => expected` (e.g. `json(...)`). A throw here is also `error` (broken config, not rot). |
-| `compare` | no | `'deep-equal'` | How to compare `expect` against what `source` returned. |
-
-`source` and function-form `expect` receive `ctx`:
-
-| | |
-|---|---|
-| `ctx.warn(message)` | Record a non-fatal note (one of several pages down, odd markup). Shows as a warning; does not fail the run. |
-| `ctx.configDir` | Absolute directory of the config file. Data paths resolve from here. |
-
-### `compare`
-
-**`'contains-all'`** — `expect` is a list of marker strings; `source` must
-return a text corpus (what `corpus()` returns). Every marker must appear
-somewhere in that text, case-insensitively. Extra text on the page is
-ignored. This is the webpage-watch check.
-
-**`'deep-equal'`** — structural equality. Extra keys, missing keys, and
-value mismatches all rot, with pathed messages
-(`$.members[2].name: expected "Smith", source has "Jones"`). Use this when
-`source()` already returns structured data that should match `expect`
-exactly.
-
-**A function** — `(expected, actual) => string[]`. Return `[]` to pass;
-each string is one mismatch (outcome `rot`). Throw if the shapes are
-unusable (outcome `error`). Use this for numeric tolerance, order-insensitive
-lists, or any comparison that isn't "markers in a page" or "these two
-objects are identical."
-
-```js
-compare: (expected, actual) =>
-  expected.rate === actual.rate
-    ? []
-    : [`tax rate ${expected.rate} → ${actual.rate}`],
-```
-
-Surnames are a *marker choice*, not a compare mode: derive them from your
-data with `surname()` and use `'contains-all'`.
-
-### Helpers
-
-These are the built-in `source` / `expect` builders. You can always write
-your own functions instead.
-
-**`corpus(urls, options?)`** — fetch pages, strip HTML to text,
-concatenate. Individual unreachable pages call `ctx.warn`; if *every* page
-fails, it throws (`error`). Pair with `'contains-all'`.
-
-| Option | Default | What it is |
-|---|---|---|
-| `timeoutMs` | `20000` | Per-page fetch timeout, milliseconds. |
-| `userAgent` | `stilltrue drift check (github.com/sg-bfoster/stilltrue)` | Sent on each request so operators can identify the check. |
-
-```js
-source: corpus(
-  ['https://example.org/about', 'https://example.org/board'],
-  { timeoutMs: 15_000, userAgent: 'my-app drift check' },
-),
-```
-
-**`json(path, derive?)`** — read a JSON file relative to the config file.
-With no `derive`, the parsed value *is* the expectation (typical with
-`'deep-equal'`). With `derive`, you map that value into whatever `compare`
-needs — usually a marker list:
-
-```js
-expect: json('./data/board.json', (data) =>
-  data.members.map((m) => surname(m.name)),
-),
-```
-
-**`surname(fullName)`** — last token of a name, with leading honorifics
-(`Dr.`, `Mr.`, `Mrs.`, `Ms.`) and trailing generational suffixes (`Jr.`,
-`Sr.`, `II`–`V`) stripped. `"Dr. Jane Smith III"` → `"Smith"`.
-
-**`htmlToText(html)`** — the same tag-stripping `corpus()` uses. For a
-custom `source` that already has HTML in hand.
-
-### Custom `source`
-
-A throw is an outage; a return value is data to compare. Warn for partial
-trouble:
-
-```js
-source: async (ctx) => {
-  const res = await fetch('https://example.org/api/rates');
-  if (!res.ok) throw new Error(`HTTP ${res.status}`);
-  const data = await res.json();
-  if (!data.updatedAt) ctx.warn('response missing updatedAt');
-  return data;
-},
-expect: json('./data/rates.json'),
-// compare defaults to 'deep-equal'
-```
-
-### `verify` pipelines
-
-`verify` is an optional map of named stage lists, for use from your app
-(`runVerify(config.verify.answers, draft)`). There is no `stilltrue verify`
-CLI command — verification runs in the request path, not in CI.
-
-Each stage:
-
-| Field | Required | Default | What it is |
-|---|---|---|---|
-| `name` | yes | — | Id of this checkpoint. The first rejection reports this name. |
-| `check` | yes | — | `(input, ctx) => { ok: true, value? } \| { ok: false, messages, revision? }`. |
-| `failPolicy` | no | `'open'` | What a **thrown** stage means. `'open'`: warn and continue (a judge outage must not block the app). `'closed'`: treat the throw as a rejection. |
-| `timeoutMs` | no | none | Abort the stage after this many ms. The timeout follows `failPolicy`. |
-| `tier` | no | `'inline'` | `'inline'` always runs; `'ci'` runs only when you call `runVerify(..., { tier: 'ci' })`. |
-
-Stages run in order and stop at the first rejection. `zodStage(name, schema)`
-builds a parse stage from a Zod schema.
-`generateVerified({ generate, stages, attempts, tier })` is the generate →
-verify → retry loop (`attempts` defaults to `2`). See
-[For developers: `verify`](#for-developers-verify) below.
-
-### CLI flags
-
-`stilltrue drift`:
-
-| Flag | What it does |
-|---|---|
-| `--config <path>` | Config file. Default: first `stilltrue.config.*` in the current directory. |
-| `--only <name,name>` | Run only these checks. |
-| `--json <path>` | Write the result array as JSON. |
-| `--record` | Append this run to history. |
-| `--history <path>` | History file for `--record`. Default: `.stilltrue/history.jsonl`. |
-
-`stilltrue report`:
-
-| Flag | What it does |
-|---|---|
-| `--history <path>` | History file to read. Default: `.stilltrue/history.jsonl`. |
-| `--out <path>` | HTML output. Default: `stilltrue-report.html`. |
-| `--title <text>` | Page `<title>` and heading. Default: `stilltrue drift report`. |
-
 ## Running it automatically every week
 
 If your project is on GitHub, add this file as
@@ -329,79 +163,29 @@ comparison stays mechanical.)
 ## For developers: `verify`
 
 `verify` is a library for teams shipping AI-generated answers: an ordered
-series of checkpoints (a schema parse, a rules check, an AI judge — your
-choice) that each answer must pass before publishing. Rejections say
-exactly which checkpoint failed and why, in a format you can feed back to
-the model for a retry. A broken checker never blocks your app — it fails
-open. See the [project docs](docs/BRIEF.md) and the typed API in
-`stilltrue`'s exports (`runVerify`, `zodStage`, `generateVerified`).
-
-**Bring your own judge — any provider, or none.** stilltrue ships no AI
-and never touches a model key: a checkpoint is just a function you supply,
-so the judge stage can call Gemini, OpenAI, Anthropic, a local model — or
-be a plain schema/rules check with no AI at all. Swapping providers means
-editing your function, never this package.
-
-### The common case, in six lines
-
-Most people arrive wanting one thing: *does this source actually support
-this claim?* `supportStage` is that check, ready-made. You bring a function
-that calls your model; it brings the prompt, the response schema, and the
-guard.
-
-```js
-import { runVerify, supportStage } from 'stilltrue';
-
-const stage = supportStage({
-  // The only required option. Return the model's reply — parsed or raw JSON.
-  judge: ({ system, user, schema }) => callMyModel({ system, user, schema }),
-});
-
-const result = await runVerify([stage], { claim, source });
-// result.ok      → true when the source supports the claim
-// result.value   → { verdict, reasoning, evidence, contradiction, guard }
-// result.messages → why it was rejected, ready for a retry
-```
-
-`verdict` is one of `supported`, `not_supported` or `cant_tell`. Silence is
-not support: a source that never addresses the claim returns `cant_tell`,
-even when the claim happens to be true.
-
-**"Not supported" has to earn itself.** That verdict tells a reader the
-source *says otherwise*, which is the strongest thing a judge can assert
-and the one most worth being wrong about. So the stage requires the judge
-to quote the contradicting sentence, and checks that quote against the
-source in code. If it cannot point at real text, the verdict is downgraded
-to `cant_tell` and a warning explains why. The check only ever downgrades —
-a missing quote is evidence of nothing except a missing quote.
-
-That guard is not theoretical. Judged by a small local model, adding two
-lines to a source that never mentioned the claim flipped a verdict from
-`supported` to `not_supported` — three runs each way, so not noise — where
-a frontier model stayed stable. A package that promises engine-agnostic
-verification has to survive the weaker engine.
-
-Useful options, all defaulted: `minRun` (how much of the contradiction
-quote must match the source — 24 characters, measured on civic notices and
-fee tables, not universal), `requireContradiction: false` to switch the
-guard off, `treatCantTellAs: 'warn'` to pass unanswerable claims instead of
-rejecting them, and `system` to replace the prompt entirely. `SUPPORT_SYSTEM`
-and `SUPPORT_SCHEMA` are exported too, so a provider with constrained
-decoding can take the schema directly.
-
-### Writing your own stages
-
-`supportStage` is one stage; a pipeline is a list of them, and any function
-of the right shape will do. **[docs/VERIFY.md](docs/VERIFY.md)** is the full
-walkthrough — the judging prompt explained, response mapping, retries, and
-the generate → verify → retry loop — for when the ready-made stage is not
-the shape you need.
-
-## What about testing prompts?
-
-That's a different job with a great existing tool:
-[promptfoo](https://promptfoo.dev) tests your prompts; stilltrue tests your
-facts. How they fit together: [docs/GOLDEN.md](docs/GOLDEN.md).
+series of checkpoints — a schema parse, a rules check, an AI judge, your
+choice — that each answer must pass before publishing. Rejections say which
+checkpoint failed and why, in a format you can feed back to the model for a
+retry; a broken checker fails open rather than blocking your app. stilltrue
+ships no AI and never touches a model key — the judge is a function you
+supply, so swapping providers means editing your function, never this
+package.
+
+**[docs/VERIFY.md](docs/VERIFY.md) is the whole story**, including a
+complete copy-paste judge (Gemini shown, OpenAI-compatible variant inline —
+which also covers LM Studio and every hosted provider) and `supportStage`,
+the ready-made "does this source support this claim?" check.
+
+## Reference
+
+- **[Configuration](docs/CONFIG.md)** — every config field, compare mode,
+  helper, and CLI flag.
+- **[Picking good markers](docs/MARKERS.md)** — the judgment calls
+  (`surnames beat full names`, `"143 schools"` beats `"143"`).
+- **[Verify](docs/VERIFY.md)** — building a judge, writing stages,
+  `generateVerified`.
+- **[Prompts vs facts](docs/GOLDEN.md)** — [promptfoo](https://promptfoo.dev)
+  tests your prompts; stilltrue tests your facts. How they fit together.
 
 ## License
 

package/dist/cli.js

@@ -106,7 +106,7 @@ try {
                     '  1. Edit stilltrue.config.mjs — point source at the real page(s) your facts came from,\n' +
                     '     and replace the example markers with your own facts (plain strings)\n' +
                     '  2. Run: npx stilltrue drift\n' +
-                    'Picking good markers: docs/MARKERS.md in the stilltrue repo');
+                    'Picking good markers: https://github.com/sg-bfoster/stilltrue/blob/main/docs/MARKERS.md');
             }
             process.exit(0);
             break;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "stilltrue",
-  "version": "0.7.0",
+  "version": "0.7.1",
   "description": "Your tests check your code. stilltrue checks your facts. CI checks for AI-in-the-loop apps: drift (curated facts vs live sources), golden (regression evals), verify (judge answers against sources).",
   "keywords": [
     "drift",