backdraft 0.6.0__tar.gz → 0.8.0__tar.gz

{backdraft-0.6.0 → backdraft-0.8.0}/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "backdraft",
   "displayName": "Backdraft",
-  "version": "0.6.0",
+  "version": "0.8.0",
   "description": "Drop-in provenance for factual claims: one click from any claim to the evidence behind it. Ships the backdraft writing, backfill, and artifact-reading skills; the CLI installs separately (uv tool install backdraft).",
   "author": {
     "name": "Spencer Braun",

{backdraft-0.6.0 → backdraft-0.8.0}/DESIGN.md

@@ -140,6 +140,13 @@ Self-contained HTML (plus the machine-readable sidecar): **quotes + receipts + e
 | 2026-08-04 | **Failure speaks at the claim, not in the masthead (amends the 2026-07-28 reader doctrine).** The artifact no longer prints a failure summary under the title. "Success is silent; only failure speaks" stands, and so does the principle under it — failures are data, never warn-and-drop — but *where* it speaks moves: the wavy alarm-coloured mark on the affected claim, and that claim's note with the reason, both of which a reader meets in context. The line that went was a bare count, and a count is the one thing about a failed citation a reader cannot act on: whether `1 of 16 citations could not be traced` matters depends entirely on which claim it was, which the headline does not say and the mark and the note do. It was also aimed at the wrong reader — the writer verifying the work learns about failures from `bind`, which exits 2 and lists them, while the recipient got a number out of context before the first sentence. Nothing loses its last surface: an unmatched claim with no citations ranks `unresolved` and is marked in the body like any other failure, and a claim that is not in the document at all was already the Notes' job under artifact rule 4. This is a spec change — `spec/artifact.md` rule 6 is rewritten, and the summary is now explicitly *not* required of a conforming producer rather than merely unmentioned, so another implementation reads the same intent. |
 | 2026-08-05 | **A web page's identity is the sha256 of the snapshot fetched at ingest time; the URL is provenance, not identity.** `ingest` accepts an http(s) URL alongside files (graduating the "HTML pages" question from Later). The fetched bytes are hashed and snapshotted exactly as a file's are, so everything downstream — anchors, the ledger, `bind`, generations — is unchanged, and a page that changed since the last fetch is a *new generation of the same document*, which is the machinery drift already describes. Continuity across a re-fetch follows the URL rather than the path, because the path is a temporary staging file; `documents.sha256` is UNIQUE, so bytes still win over origin when the same content arrives twice under two URLs. The URL and `fetched_at` live in a new `document_meta` table — a JSON sibling of `page_meta`, chosen over columns on `documents` so an existing registry needs no migration and so the next provenance question (`.eml` headers) needs no second one; `fetched_at` moves on a re-fetch that changed nothing, since "when was this last confirmed to say this" is the question a reader of an old citation actually has. Fetching is stdlib `urllib` and lives in `fetch.py`, *outside* `extract/`: an extractor stays a pure function of a file's bytes, which is what keeps `deterministic` honest and lets a saved `.html` file and a fetched page extract identically. The CLI owns the network the way it already owns page-snapshot capture. The new `html` extractor is a parse, not a readability heuristic — no boilerplate stripping, no main-content guess, because a heuristic that changes its mind moves anchors, which is the failure mode this system exists to prevent; navigation and footers are part of the page. Its representation follows `docx`: tables as markdown pipe tables, lists as `- `/`1. ` lines kept in one block, `<pre>` verbatim. Decoding is BOM, then the document's own `<meta charset>`, then UTF-8 with replacement — the server's `Content-Type` charset is deliberately *not* consulted, so the bytes alone determine the text. Known limitations, accepted and documented rather than worked around: JavaScript-rendered pages yield whatever the server sends, pages behind a login are out of reach, and a non-UTF-8 page that declares its encoding only in the HTTP header will render as mojibake in the snapshot — visibly, rather than silently. |
 | 2026-08-06 | **A fetched source's origin travels into the artifact as a live link, dated.** The capture half of the row above stored `url` and `fetched_at` and stopped at the registry, so an artifact citing a web page named `q4-2025.html` — a staging filename that never existed on anyone's disk — and a reader holding the file had the frozen receipt with no way back to the page. `bind`'s evidence `documents` map now carries `url` and `fetched_at` through to the record, and both renderers show them: the receipt card, the script-free note, the source list and the rail all print the URL as a real `<a href>` with the fetch date beside it, and the markdown projection carries it as an autolink. This is an artifact-format change, so `spec/artifact.md` and the `$legend` name the two keys together, both OPTIONAL. The keys are emitted *only* where a document has a URL, which is what keeps an artifact built from files byte-identical to one built before this existed — the conditional is the compatibility guarantee, and it is a test rather than an intention. Linking does not weaken the self-containment rule: the CSP (`default-src 'none'`) forbids *fetching*, and an anchor fetches nothing until a reader chooses to follow it, so the file still renders identically offline. The href is allowlisted to `http`/`https` rather than merely screened for `javascript:`, since artifact rule 3 is a conformance requirement and an allowlist cannot be outflanked by a scheme nobody thought of; a URL under any other scheme is printed as plain text rather than dropped, because failures are data and provenance the reader must paste by hand still beats provenance withheld. In the source list the URL *replaces* the filename rather than joining it — showing both would give a reader two names for one thing and let the fictional one look authoritative. The date is the day, not the timestamp: "as of" is what makes a frozen quote from a live page defensible, and no reader needs the seconds. |
+| 2026-08-11 | **An extractor declares the `--config` keys it reads, and an undeclared key is an error.** `--config dpy=300` used to exit 0, ingest, and change nothing: the config boundary was the one place this repo's "failures are data, never warn-and-drop" rule did not hold, and nothing — help, docs, `llms.txt` — said which keys existed, so learning them meant reading `extract/vlm.py` and `extract/snapshots.py`. The `Extractor` protocol gains an optional `config_keys: Mapping[str, str]` (key → one-line meaning) and `check_config` validates against it. Validation happens **after selection and inside `Registry.ingest`**, not in the CLI: `auto` picks per file, so only the chosen extractor can say whether `dpi` is a render setting or a typo, and putting it at the registry means every substrate is covered rather than only the one with a `--config` flag. It raises `ExtractionError` rather than the `UsageError` a CLI-side check would have used — `cli_context` imports the registry, so the registry cannot import `UsageError` back — and both are `BackdraftError`, so the guard maps either to exit 1 and the user sees the same line. Each key is declared once by the module that already owns its default and reads it: the page-render budget (`dpi`, `snapshot_quality`, `snapshot_max_height`) in `extract/snapshots.py`, the provider surface (`api_key`, `base_url`, `model`, `timeout`, `retries`) in `extract/vlm_settings.py`, and the extractors compose those by reference. `pdf-text` declares the page-render keys despite reading none of them itself — the snapshot capture that runs beside it at ingest does, and a user passing `dpi` to a text-layer ingest means the images; declaring by *what one flag on one ingest affects* beats declaring by which function happens to call `config.get`. Tradeoff accepted: declaring nothing means accepting nothing, so an out-of-tree extractor that reads config without declaring it starts rejecting keys it used to receive. That is the intended direction — a silently ignored key is the failure being fixed — and the fix is one attribute. The help text that lists the keys is pinned by a test against the declarations, so a new key cannot be added in code and left undocumented. |
+| 2026-08-12 | **`show` is a gate verb, not a lookup tool.** `backdraft show <token>` answers what a token says — the inverse of minting, for a token that arrived out of an artifact, a draft, or somebody's message, which nothing in the tool could answer before without rebuilding the slug and selector by hand and re-reading a whole page. It therefore *mints what it shows*, exactly as `read` and `search` do: an agent that has seen a snippet through the gate has genuinely read it, so recording it is honest rather than a loophole, and `not_shown` stays true by construction — a status `show` can never produce. That makes a token laundering into a citable one by being looked at, which is the intended behavior and the reason this is in `gate/` and not somewhere neutral. Resolution is bind's path rather than a second reading of what a token means: `kernel.claims.parse_citation` decides `malformed` (so the reserved `bd:calc(...)` form lands identically in both) and `Registry.resolve` decides the rest, which is what makes a status printed here the status bind would print. Two smaller calls. `drifted` prints both snippets *and* mints the anchor standing at the locator now, since that anchor's token is the one worth citing and printing it unrecorded would be the under-minting the gate must never do. And drift does **not** fail the exit code: exit 1 covers `unresolved` and `malformed`, the two statuses where there was nothing to show, because whether a changed source still supports a sentence is bind's question about a document, not a lookup's about a token. Tradeoff accepted: auditing someone else's artifact against a registry now writes ledger rows into the auditor's session, so `skills/backdraft-artifact/SKILL.md` says so and keeps that check reported separately from the artifact's self-checks. |
+| 2026-08-13 | **`ingest` finishes its list; an unreadable source is data, not the end of the run.** `backdraft ingest a.md broken.pdf c.md` used to ingest `a.md`, raise on `broken.pdf`, and never attempt `c.md` — leaving a half-built registry, a message naming only the first failure, and no way to tell which sources landed from which were never tried, so ingesting a folder meant re-running and inferring the state. The loop now catches `BackdraftError` per source and records `(source, reason)`, so every source is attempted, and prints one `!` line per failure under a count (`2 of 3 sources ingested; 1 failed`) whose two numbers add to the whole list — which is precisely the fact the old behavior could not state. This is principle 4 ("failures are data, never warn-and-drop") applied at the boundary it had not reached, and the same shape the page-snapshot note already uses one level down. Grouping differs deliberately: the snapshot note groups by *reason* because one missing poppler explains every document at once, while this lists by *source*, because the source is what the caller has to go fix. The exit code is unchanged at 1 and `guard` remains the only place a `BackdraftError` becomes one — nothing caught per source is re-raised, and the report exits through `fail` like any other error line, on stderr, so a clean run's stdout is byte-identical to before and that is pinned by a test. Tradeoff accepted: a run that fails partway now leaves *more* in the registry than it used to, so "the command failed" no longer implies "nothing happened". The closing line of the report answers that directly — re-running the same list is safe, because a source already ingested with unchanged bytes, extractor and config is a no-op — which is cheaper for a caller than reconstructing the failed half by diffing `ls` against its own arguments. |
+| 2026-08-17 | **The demo cites a web page at a fixed revision, so the capability is visible and stays `resolved`.** URL sources shipped whole — fetch, snapshot, `url`/`fetched_at`, the artifact's live dated link — and none of it appeared at backdraft.dev/demo.html, because both demo sources were local files and the byte-identity test that proved the artifact change safe was the same fact that kept it invisible. The demo gains a third source: the Wikipedia article for Franklin County, Ohio — the county the fictional property sits in and the one its tax appeal is already pending before — cited for two market figures a T12 cannot supply, the county's population and its renter-occupied share. It is ingested at Wikipedia's **permanent link** (`?oldid=<rev>`) rather than the bare article URL. What that gives up is named: the demo no longer shows a live page changing under a citation, which is the better story and belongs in a `drifted` walkthrough rather than the shop window. What it buys is that the front page does not permanently display `drifted` within days of the next edit — a demo that always shows drift teaches that the tool is broken rather than that the source moved — and an immutable revision plus the rendered "as of" date is what a careful analyst cites anyway. The same reasoning is now advice rather than a one-off: the docs, `llms.txt` and the writing skill tell an agent to prefer a fixed-revision address (`?oldid=`, a DOI, an archived snapshot) and to pass `--slug` with it. `--slug` is load-bearing here, not decoration: a permanent link is served from `/w/index.php`, so the default stem would be `index` and every token in the memo would carry it. That is the queued slug-assignment item's problem and is fixed at the fetch layer there, not worked around here. One renderer change came with it. `source_title` took a source's display name from its filename's stem, which for a fetched page is the staging name `fetch.filename_for` invented — so the article titled itself "Index". Where a document carries a URL the slug is the name instead, which is the 2026-08-06 row's rule ("showing both would let the fictional one look authoritative") applied to the one surface that had kept the invented name. It changes nothing for a page ingested without `--slug`, since the slug is derived from that same stem; it defers to whoever named the document when someone did. |
+| 2026-08-18 | **One rule names a source, and every surface that names one uses it.** A fetched page's `filename` is the temporary file `fetch.filename_for` staged the bytes in — `index.html` for a permanent link, a name on nobody's disk — and the 2026-08-06 row settled what to do about it ("showing both would give a reader two names for one thing and let the fictional one look authoritative") for the artifact alone. The four CLI surfaces that name a document kept the fiction: `ingest`'s line, the gate's document list, its table-of-contents headline, and `ls` — which showed *both*, filename in column two and URL in a fifth field, the exact shape that row rejected. `gate.source_name` is now the single owner, exported the way `gate.unit` is and imported downward by `cli.py`, so a writing agent learns a source's provenance from the surface it already lives on instead of being sent to `ls` by the skill. Display only: the slug, the token and everything a citation resolves through are untouched, which is the 2026-08-05 "provenance, never identity" rule. Two consequences named rather than discovered. `ls` loses a field — a fetched row is four columns, not five — so anything parsing column five stops seeing one; that is the point, since column two was a lie and one true name beats two names of which one is fictional. And in the gate's aligned list a URL *overflows* the name column rather than widening it: the column is sized on filenames alone, so one 80-character address pushes its own row's later columns right instead of pushing every row's past column 100. The cost is a ragged right on fetched rows, accepted because the file rows a mostly-file registry is made of stay byte-identical to what they printed before — pinned by a test, the way the artifact's byte-identity is. `bind`'s unmatched-claim report still names a source by filename: the fix cannot be shared into it without a sideways `bind` → `gate` import, which SPEC § Dependency rule forbids, and duplicating the rule there is the reinvention this row exists to end. |
+| 2026-08-19 | **A slug is assigned by a rule, and the rule is written down.** The Open list carried "slug assignment/collision rules" from the first field trial, and URL sources sharpened it into a live failure: `fetch.filename_for` took the stem from a URL's last path segment, so `https://a.example/reports/2025/index.html` and `https://b.example/docs/index.html` both staged as `index.html` and became `index` and `index-2` — two handles that say which is which to nobody, carried by every token in every document written against them, permanently, since a slug is stable once assigned. The rule now has two clauses and they are stated together for files and URLs. A **file** is named by its filename's stem, kebabed and cut to 32 characters, which is unchanged and was never the problem. A **URL** is named by its last path segment, *except* when that segment names the site's plumbing rather than its page — a directory index or generic handler (`_GENERIC_STEMS`: `index`, `view`, `page`, `show`, and a dozen more), an all-digit id, or no segment at all — in which case the host goes in front of it, minus `www.` and minus the port, so the two pages above become `a-example-index` and `b-example-index`. The common case is deliberately untouched: `https://example.com/reports/q4-2025` is still `q4-2025`, because a rule that made every good name uglier to fix the bad ones would be the wrong trade. `_dedupe`'s `-2` stays, demoted from the normal outcome to the backstop it was meant to be — the host is a *better* stem, not a unique one, and two `index.html` pages on one site still collide. Truncation is trimmed on both cuts now (`slug_for` and `_dedupe`), which is the other half of the Open line: a 32-character cut lands on a hyphen often enough that `q4-report-` was a slug this repo could produce, legal in the grammar and reading like a bug. The compatibility statement, said out loud because nothing else will say it: two registries built on either side of this change disagree about a fetched page's slug, and an artifact's tokens are not portable between them. Existing registries are unaffected, and that is not luck — `_find_document` matches a re-fetch on the URL, never the slug, so re-ingesting a page keeps the slug it already has. `--slug` remains the way to overrule all of this and is still the advice for a fetched page worth citing, because `a-example-index` names a site rather than nothing, which is better and is not good. |
+| 2026-08-20 | **`ingest` says which of its three outcomes happened, and how much text came out.** One line served three different events — a document created, a *new generation* of one whose bytes moved, and a no-op that wrote nothing — so an agent re-ingesting after a fix could not tell whether anything had changed, at exactly the moment it most needed to: a new generation is when citations into the previous snapshot begin resolving to a superseded one and `bind` starts calling them `drifted`. The registry already distinguished all three internally (`_find_document`, `_is_noop`, `_upsert_document`) and threw the answer away, so `ingest` now returns an `Ingested` — a `Document` subclass carrying `outcome`, the shape `SearchResults` established for `search`: Addendum A's pinned `-> Document` still holds, every `.slug` caller and every fake is untouched, and the one consumer that cares reads the extra field. Re-deriving the outcome in the CLI was the alternative and would have meant a second implementation of `_is_noop` living away from the first. The wart is named rather than papered over: a dataclass compares by class, so an `Ingested` never equals the plain `Document` the read side returns for the same row, and the two tests that compared them now compare what persists. The second half is a number: the extracted character count, beside the page count on every line. A scanned PDF with no text layer and a page behind a login both ingest cleanly, exit 0 and print a page count like any success, which is how a shell of a source gets cited — and the writing skill's instruction to notice "thin or empty" gave the agent nothing to notice it with. Under `THIN_SOURCE_CHARS` (200) a note names the likely cause by media type and what to do about it. That threshold is a heuristic, which this repo refuses elsewhere on purpose — the `html` extractor is a parse and not a readability guess, because a heuristic that changes its mind moves anchors. This one is admissible for the reason those are not: it is display only, at exit 0, and no token, anchor, status or byte of the artifact derives from it, so being wrong costs one line of advice and never a citation. `chars` counts sheets too, where `read`'s window unit is rows: this is the extraction's volume, not a window into it. Grouping follows the 2026-08-13 rule — thin sources group by cause, since one missing text layer explains every scan at once and each document's own count is already on its own line, while the failure report still lists by source. |
 
 ## Open
 
@@ -147,8 +154,6 @@ Queued and parked work lives in ROADMAP.md; this list is questions still
 awaiting a decision:
 
 - Re-bind/orphan pass on re-ingest of changed docs (chunk ordinal drift)
-- Slug assignment/collision rules (the trailing-hyphen truncation from the
-  first field trial lives here)
 - Distribution: published to PyPI as `backdraft` 0.2.0 (2026-07-28); domain
   backdraft.dev acquired; still open: skill registration, deploying site/
 - First external user (design-partner validation layer vs eval substrate vs

{backdraft-0.6.0 → backdraft-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: backdraft
-Version: 0.6.0
+Version: 0.8.0
 Summary: Drop-in provenance for factual claims: one click from any claim to the evidence behind it.
 Project-URL: Homepage, https://backdraft.dev
 Project-URL: Documentation, https://backdraft.dev/docs.html
@@ -56,7 +56,9 @@ cd demo
 cells for spreadsheet sheets. Formats: PDF, XLSX/XLSM, XLS, CSV/TSV, DOCX,
 PPTX, HTML, images (png, jpeg, tiff, through the vision model), plain text and
 Markdown. A source can also be an `http(s)` URL — the page is fetched once and
-snapshotted like any other source, and the URL travels with it.
+snapshotted like any other source, and the URL travels with it. A source that
+cannot be read does not end the run: the rest of the list is ingested anyway,
+and the command exits 1 naming each source that failed and why.
 
 ```console
 $ backdraft init
@@ -65,11 +67,27 @@ documents: 0
 … plus where settings live and what to do next
 
 $ backdraft ingest sources/t12-summary.pdf sources/underwriting-model.xlsx
-t12-summary  t12-summary.pdf  pdf  3 pages
-underwriting-model  underwriting-model.xlsx  xlsx  2 sheets
+t12-summary  t12-summary.pdf  pdf  3 pages  8865 chars
+underwriting-model  underwriting-model.xlsx  xlsx  2 sheets  2357 chars
 note: extracted with pdf-text (the embedded text layer). Glossy or scanned PDFs extract better through a vision model: set BACKDRAFT_VLM_API_KEY in .backdraft/env.
+
+$ backdraft ingest "https://en.wikipedia.org/w/index.php?title=Franklin_County,_Ohio&oldid=1367935775" --slug franklin-county
+franklin-county  https://en.wikipedia.org/w/index.php?title=Franklin_County,_Ohio&oldid=1367935775  html  1 page  34141 chars
 ```
 
+The demo cites that page for market context. It is Wikipedia's *permanent
+link* — `?oldid=` serves one revision's bytes forever — because a citation into
+a live article reports `drifted` the moment somebody edits it, and the quoted
+sentence would no longer be the sentence there.
+
+Every ingest line ends with how much text came out, and says which of three
+things happened: a document created, a **new generation** of one whose bytes
+moved — which is when citations into the previous snapshot start reporting
+`drifted` — or `unchanged`, a no-op because re-running would reproduce what is
+already there. When almost no text came out, a note names the likely cause (a
+scan with no text layer, a page behind a login) and what to do; it is a note at
+exit 0, because a thin snapshot is still a real one.
+
 **Read.** The page arrives with a citable name over each chunk. This is the whole
 mechanism: what you can cite is exactly what you were shown.
 
@@ -117,6 +135,25 @@ $ backdraft cell underwriting-model "assumptions!B10"
 [bd:underwriting-model:assumptions!B10:964a]  24850000
 ```
 
+And a token you were handed — out of an artifact, a draft, someone else's
+message — runs backwards to what it actually says:
+
+```console
+$ backdraft show bd:t12-summary:p1.c1:c2e8
+[bd:t12-summary:p1.c1:c2e8]  resolved  t12-summary p1.c1
+Bridgeview Commons — Trailing Twelve Month Summary
+
+Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
+garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. …
+
+[Read the page: backdraft read t12-summary p1]
+```
+
+`show` is the gate too, so what it shows is minted and citable. A token whose
+source has changed since prints both snippets, the one that was cited and the
+one standing there now; a token that names nothing exits 1 and says whether the
+slug or the locator is the wrong half.
+
 **Cite** by writing the token as the href of a markdown link on the claim span.
 Multiple citations are `;`-separated in one href.
 
@@ -129,12 +166,12 @@ Multiple citations are `;`-separated in one href.
 
 ```console
 $ backdraft bind memo.md --session s-bridgeview --check value-trace,overlap
-bound 15 claim(s), 16 citation(s) [frontwalk]
-  resolved: 15
+bound 17 claim(s), 18 citation(s) [frontwalk]
+  resolved: 17
   unresolved: 1
-  overlap: pass 11, skip 4
-  value-trace: pass 15
-  ! unresolved: bd:t12-summary:p4.c1:1a2b
+  overlap: pass 13, skip 4
+  value-trace: pass 17
+  ! unresolved: bd:t12-summary:p4.c1:1a2b — replacement reserve of $250 per unit per year @3629
 wrote .backdraft/records/memo.backdraft.json
 
 $ echo $?
@@ -223,18 +260,23 @@ address.
 
 ```console
 $ backdraft ingest https://example.com/reports/q4-2025
-q4-2025  q4-2025.html  html  1 page
+q4-2025  https://example.com/reports/q4-2025  html  1 page  18402 chars
 
 $ backdraft ls
-q4-2025	q4-2025.html	html	1 page	https://example.com/reports/q4-2025
+q4-2025	https://example.com/reports/q4-2025	html	1 page
 ```
 
+Every surface names a fetched source by its page. The filename you never see —
+`q4-2025.html` — is the temporary file the fetch staged the bytes in, and it
+exists on nobody's disk, so `ingest`, `ls`, `backdraft read` and the artifact
+all show the URL in its place rather than beside it: two names for one thing
+would let the invented one look authoritative.
+
 The origin travels into the artifact too. A claim citing a fetched page shows
-the URL as a link on its receipt, with the date the bytes were taken, and the
-source list names the page rather than the staged filename. That pointer is the
-half of citing a web page a frozen receipt cannot answer on its own: the
-snippet says what the page said, the link is how a reader asks whether it still
-says it.
+the URL as a link on its receipt, with the date the bytes were taken. That
+pointer is the half of citing a web page a frozen receipt cannot answer on its
+own: the snippet says what the page said, the link is how a reader asks whether
+it still says it.
 
 What it does not do, stated rather than worked around: JavaScript-rendered
 pages give you whatever the server sends to a plain GET, pages behind a login
@@ -269,7 +311,11 @@ explicit, backdraft-scoped consent: `BACKDRAFT_VLM_API_KEY` (env or
 `OPENAI_API_KEY`/`OPENROUTER_API_KEY` are deliberately never read; a generic
 key in the environment is not consent to send documents to its provider. The
 default provider is OpenRouter running Gemini 3.1 Flash Lite; direct OpenAI is
-base_url + model, set explicitly. Without a scoped key, `auto` falls back to
+base_url + model, set explicitly (`--config base_url=...` and `--config
+model=...`, one key per flag). Every `--config` key is declared by the extractor
+that reads it and checked against the one `ingest` chose, so a misspelled key
+fails and names the keys that apply rather than being silently ignored.
+Without a scoped key, `auto` falls back to
 the keyless text layer and says so — still storing each page's image, which it
 renders locally. PDF page rendering needs poppler, installed separately; without
 it ingest succeeds without the images and names `snapshot-pages` as the backfill.

{backdraft-0.6.0 → backdraft-0.8.0}/README.md

@@ -31,7 +31,9 @@ cd demo
 cells for spreadsheet sheets. Formats: PDF, XLSX/XLSM, XLS, CSV/TSV, DOCX,
 PPTX, HTML, images (png, jpeg, tiff, through the vision model), plain text and
 Markdown. A source can also be an `http(s)` URL — the page is fetched once and
-snapshotted like any other source, and the URL travels with it.
+snapshotted like any other source, and the URL travels with it. A source that
+cannot be read does not end the run: the rest of the list is ingested anyway,
+and the command exits 1 naming each source that failed and why.
 
 ```console
 $ backdraft init
@@ -40,11 +42,27 @@ documents: 0
 … plus where settings live and what to do next
 
 $ backdraft ingest sources/t12-summary.pdf sources/underwriting-model.xlsx
-t12-summary  t12-summary.pdf  pdf  3 pages
-underwriting-model  underwriting-model.xlsx  xlsx  2 sheets
+t12-summary  t12-summary.pdf  pdf  3 pages  8865 chars
+underwriting-model  underwriting-model.xlsx  xlsx  2 sheets  2357 chars
 note: extracted with pdf-text (the embedded text layer). Glossy or scanned PDFs extract better through a vision model: set BACKDRAFT_VLM_API_KEY in .backdraft/env.
+
+$ backdraft ingest "https://en.wikipedia.org/w/index.php?title=Franklin_County,_Ohio&oldid=1367935775" --slug franklin-county
+franklin-county  https://en.wikipedia.org/w/index.php?title=Franklin_County,_Ohio&oldid=1367935775  html  1 page  34141 chars
 ```
 
+The demo cites that page for market context. It is Wikipedia's *permanent
+link* — `?oldid=` serves one revision's bytes forever — because a citation into
+a live article reports `drifted` the moment somebody edits it, and the quoted
+sentence would no longer be the sentence there.
+
+Every ingest line ends with how much text came out, and says which of three
+things happened: a document created, a **new generation** of one whose bytes
+moved — which is when citations into the previous snapshot start reporting
+`drifted` — or `unchanged`, a no-op because re-running would reproduce what is
+already there. When almost no text came out, a note names the likely cause (a
+scan with no text layer, a page behind a login) and what to do; it is a note at
+exit 0, because a thin snapshot is still a real one.
+
 **Read.** The page arrives with a citable name over each chunk. This is the whole
 mechanism: what you can cite is exactly what you were shown.
 
@@ -92,6 +110,25 @@ $ backdraft cell underwriting-model "assumptions!B10"
 [bd:underwriting-model:assumptions!B10:964a]  24850000
 ```
 
+And a token you were handed — out of an artifact, a draft, someone else's
+message — runs backwards to what it actually says:
+
+```console
+$ backdraft show bd:t12-summary:p1.c1:c2e8
+[bd:t12-summary:p1.c1:c2e8]  resolved  t12-summary p1.c1
+Bridgeview Commons — Trailing Twelve Month Summary
+
+Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
+garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. …
+
+[Read the page: backdraft read t12-summary p1]
+```
+
+`show` is the gate too, so what it shows is minted and citable. A token whose
+source has changed since prints both snippets, the one that was cited and the
+one standing there now; a token that names nothing exits 1 and says whether the
+slug or the locator is the wrong half.
+
 **Cite** by writing the token as the href of a markdown link on the claim span.
 Multiple citations are `;`-separated in one href.
 
@@ -104,12 +141,12 @@ Multiple citations are `;`-separated in one href.
 
 ```console
 $ backdraft bind memo.md --session s-bridgeview --check value-trace,overlap
-bound 15 claim(s), 16 citation(s) [frontwalk]
-  resolved: 15
+bound 17 claim(s), 18 citation(s) [frontwalk]
+  resolved: 17
   unresolved: 1
-  overlap: pass 11, skip 4
-  value-trace: pass 15
-  ! unresolved: bd:t12-summary:p4.c1:1a2b
+  overlap: pass 13, skip 4
+  value-trace: pass 17
+  ! unresolved: bd:t12-summary:p4.c1:1a2b — replacement reserve of $250 per unit per year @3629
 wrote .backdraft/records/memo.backdraft.json
 
 $ echo $?
@@ -198,18 +235,23 @@ address.
 
 ```console
 $ backdraft ingest https://example.com/reports/q4-2025
-q4-2025  q4-2025.html  html  1 page
+q4-2025  https://example.com/reports/q4-2025  html  1 page  18402 chars
 
 $ backdraft ls
-q4-2025	q4-2025.html	html	1 page	https://example.com/reports/q4-2025
+q4-2025	https://example.com/reports/q4-2025	html	1 page
 ```
 
+Every surface names a fetched source by its page. The filename you never see —
+`q4-2025.html` — is the temporary file the fetch staged the bytes in, and it
+exists on nobody's disk, so `ingest`, `ls`, `backdraft read` and the artifact
+all show the URL in its place rather than beside it: two names for one thing
+would let the invented one look authoritative.
+
 The origin travels into the artifact too. A claim citing a fetched page shows
-the URL as a link on its receipt, with the date the bytes were taken, and the
-source list names the page rather than the staged filename. That pointer is the
-half of citing a web page a frozen receipt cannot answer on its own: the
-snippet says what the page said, the link is how a reader asks whether it still
-says it.
+the URL as a link on its receipt, with the date the bytes were taken. That
+pointer is the half of citing a web page a frozen receipt cannot answer on its
+own: the snippet says what the page said, the link is how a reader asks whether
+it still says it.
 
 What it does not do, stated rather than worked around: JavaScript-rendered
 pages give you whatever the server sends to a plain GET, pages behind a login
@@ -244,7 +286,11 @@ explicit, backdraft-scoped consent: `BACKDRAFT_VLM_API_KEY` (env or
 `OPENAI_API_KEY`/`OPENROUTER_API_KEY` are deliberately never read; a generic
 key in the environment is not consent to send documents to its provider. The
 default provider is OpenRouter running Gemini 3.1 Flash Lite; direct OpenAI is
-base_url + model, set explicitly. Without a scoped key, `auto` falls back to
+base_url + model, set explicitly (`--config base_url=...` and `--config
+model=...`, one key per flag). Every `--config` key is declared by the extractor
+that reads it and checked against the one `ingest` chose, so a misspelled key
+fails and names the keys that apply rather than being silently ignored.
+Without a scoped key, `auto` falls back to
 the keyless text layer and says so — still storing each page's image, which it
 renders locally. PDF page rendering needs poppler, installed separately; without
 it ingest succeeds without the images and names `snapshot-pages` as the backfill.