backdraft 0.6.0__tar.gz → 0.7.0__tar.gz

{backdraft-0.6.0 → backdraft-0.7.0}/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "backdraft",
   "displayName": "Backdraft",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "Drop-in provenance for factual claims: one click from any claim to the evidence behind it. Ships the backdraft writing, backfill, and artifact-reading skills; the CLI installs separately (uv tool install backdraft).",
   "author": {
     "name": "Spencer Braun",

{backdraft-0.6.0 → backdraft-0.7.0}/DESIGN.md

@@ -140,6 +140,9 @@ Self-contained HTML (plus the machine-readable sidecar): **quotes + receipts + e
 | 2026-08-04 | **Failure speaks at the claim, not in the masthead (amends the 2026-07-28 reader doctrine).** The artifact no longer prints a failure summary under the title. "Success is silent; only failure speaks" stands, and so does the principle under it — failures are data, never warn-and-drop — but *where* it speaks moves: the wavy alarm-coloured mark on the affected claim, and that claim's note with the reason, both of which a reader meets in context. The line that went was a bare count, and a count is the one thing about a failed citation a reader cannot act on: whether `1 of 16 citations could not be traced` matters depends entirely on which claim it was, which the headline does not say and the mark and the note do. It was also aimed at the wrong reader — the writer verifying the work learns about failures from `bind`, which exits 2 and lists them, while the recipient got a number out of context before the first sentence. Nothing loses its last surface: an unmatched claim with no citations ranks `unresolved` and is marked in the body like any other failure, and a claim that is not in the document at all was already the Notes' job under artifact rule 4. This is a spec change — `spec/artifact.md` rule 6 is rewritten, and the summary is now explicitly *not* required of a conforming producer rather than merely unmentioned, so another implementation reads the same intent. |
 | 2026-08-05 | **A web page's identity is the sha256 of the snapshot fetched at ingest time; the URL is provenance, not identity.** `ingest` accepts an http(s) URL alongside files (graduating the "HTML pages" question from Later). The fetched bytes are hashed and snapshotted exactly as a file's are, so everything downstream — anchors, the ledger, `bind`, generations — is unchanged, and a page that changed since the last fetch is a *new generation of the same document*, which is the machinery drift already describes. Continuity across a re-fetch follows the URL rather than the path, because the path is a temporary staging file; `documents.sha256` is UNIQUE, so bytes still win over origin when the same content arrives twice under two URLs. The URL and `fetched_at` live in a new `document_meta` table — a JSON sibling of `page_meta`, chosen over columns on `documents` so an existing registry needs no migration and so the next provenance question (`.eml` headers) needs no second one; `fetched_at` moves on a re-fetch that changed nothing, since "when was this last confirmed to say this" is the question a reader of an old citation actually has. Fetching is stdlib `urllib` and lives in `fetch.py`, *outside* `extract/`: an extractor stays a pure function of a file's bytes, which is what keeps `deterministic` honest and lets a saved `.html` file and a fetched page extract identically. The CLI owns the network the way it already owns page-snapshot capture. The new `html` extractor is a parse, not a readability heuristic — no boilerplate stripping, no main-content guess, because a heuristic that changes its mind moves anchors, which is the failure mode this system exists to prevent; navigation and footers are part of the page. Its representation follows `docx`: tables as markdown pipe tables, lists as `- `/`1. ` lines kept in one block, `<pre>` verbatim. Decoding is BOM, then the document's own `<meta charset>`, then UTF-8 with replacement — the server's `Content-Type` charset is deliberately *not* consulted, so the bytes alone determine the text. Known limitations, accepted and documented rather than worked around: JavaScript-rendered pages yield whatever the server sends, pages behind a login are out of reach, and a non-UTF-8 page that declares its encoding only in the HTTP header will render as mojibake in the snapshot — visibly, rather than silently. |
 | 2026-08-06 | **A fetched source's origin travels into the artifact as a live link, dated.** The capture half of the row above stored `url` and `fetched_at` and stopped at the registry, so an artifact citing a web page named `q4-2025.html` — a staging filename that never existed on anyone's disk — and a reader holding the file had the frozen receipt with no way back to the page. `bind`'s evidence `documents` map now carries `url` and `fetched_at` through to the record, and both renderers show them: the receipt card, the script-free note, the source list and the rail all print the URL as a real `<a href>` with the fetch date beside it, and the markdown projection carries it as an autolink. This is an artifact-format change, so `spec/artifact.md` and the `$legend` name the two keys together, both OPTIONAL. The keys are emitted *only* where a document has a URL, which is what keeps an artifact built from files byte-identical to one built before this existed — the conditional is the compatibility guarantee, and it is a test rather than an intention. Linking does not weaken the self-containment rule: the CSP (`default-src 'none'`) forbids *fetching*, and an anchor fetches nothing until a reader chooses to follow it, so the file still renders identically offline. The href is allowlisted to `http`/`https` rather than merely screened for `javascript:`, since artifact rule 3 is a conformance requirement and an allowlist cannot be outflanked by a scheme nobody thought of; a URL under any other scheme is printed as plain text rather than dropped, because failures are data and provenance the reader must paste by hand still beats provenance withheld. In the source list the URL *replaces* the filename rather than joining it — showing both would give a reader two names for one thing and let the fictional one look authoritative. The date is the day, not the timestamp: "as of" is what makes a frozen quote from a live page defensible, and no reader needs the seconds. |
+| 2026-08-11 | **An extractor declares the `--config` keys it reads, and an undeclared key is an error.** `--config dpy=300` used to exit 0, ingest, and change nothing: the config boundary was the one place this repo's "failures are data, never warn-and-drop" rule did not hold, and nothing — help, docs, `llms.txt` — said which keys existed, so learning them meant reading `extract/vlm.py` and `extract/snapshots.py`. The `Extractor` protocol gains an optional `config_keys: Mapping[str, str]` (key → one-line meaning) and `check_config` validates against it. Validation happens **after selection and inside `Registry.ingest`**, not in the CLI: `auto` picks per file, so only the chosen extractor can say whether `dpi` is a render setting or a typo, and putting it at the registry means every substrate is covered rather than only the one with a `--config` flag. It raises `ExtractionError` rather than the `UsageError` a CLI-side check would have used — `cli_context` imports the registry, so the registry cannot import `UsageError` back — and both are `BackdraftError`, so the guard maps either to exit 1 and the user sees the same line. Each key is declared once by the module that already owns its default and reads it: the page-render budget (`dpi`, `snapshot_quality`, `snapshot_max_height`) in `extract/snapshots.py`, the provider surface (`api_key`, `base_url`, `model`, `timeout`, `retries`) in `extract/vlm_settings.py`, and the extractors compose those by reference. `pdf-text` declares the page-render keys despite reading none of them itself — the snapshot capture that runs beside it at ingest does, and a user passing `dpi` to a text-layer ingest means the images; declaring by *what one flag on one ingest affects* beats declaring by which function happens to call `config.get`. Tradeoff accepted: declaring nothing means accepting nothing, so an out-of-tree extractor that reads config without declaring it starts rejecting keys it used to receive. That is the intended direction — a silently ignored key is the failure being fixed — and the fix is one attribute. The help text that lists the keys is pinned by a test against the declarations, so a new key cannot be added in code and left undocumented. |
+| 2026-08-12 | **`show` is a gate verb, not a lookup tool.** `backdraft show <token>` answers what a token says — the inverse of minting, for a token that arrived out of an artifact, a draft, or somebody's message, which nothing in the tool could answer before without rebuilding the slug and selector by hand and re-reading a whole page. It therefore *mints what it shows*, exactly as `read` and `search` do: an agent that has seen a snippet through the gate has genuinely read it, so recording it is honest rather than a loophole, and `not_shown` stays true by construction — a status `show` can never produce. That makes a token laundering into a citable one by being looked at, which is the intended behavior and the reason this is in `gate/` and not somewhere neutral. Resolution is bind's path rather than a second reading of what a token means: `kernel.claims.parse_citation` decides `malformed` (so the reserved `bd:calc(...)` form lands identically in both) and `Registry.resolve` decides the rest, which is what makes a status printed here the status bind would print. Two smaller calls. `drifted` prints both snippets *and* mints the anchor standing at the locator now, since that anchor's token is the one worth citing and printing it unrecorded would be the under-minting the gate must never do. And drift does **not** fail the exit code: exit 1 covers `unresolved` and `malformed`, the two statuses where there was nothing to show, because whether a changed source still supports a sentence is bind's question about a document, not a lookup's about a token. Tradeoff accepted: auditing someone else's artifact against a registry now writes ledger rows into the auditor's session, so `skills/backdraft-artifact/SKILL.md` says so and keeps that check reported separately from the artifact's self-checks. |
+| 2026-08-13 | **`ingest` finishes its list; an unreadable source is data, not the end of the run.** `backdraft ingest a.md broken.pdf c.md` used to ingest `a.md`, raise on `broken.pdf`, and never attempt `c.md` — leaving a half-built registry, a message naming only the first failure, and no way to tell which sources landed from which were never tried, so ingesting a folder meant re-running and inferring the state. The loop now catches `BackdraftError` per source and records `(source, reason)`, so every source is attempted, and prints one `!` line per failure under a count (`2 of 3 sources ingested; 1 failed`) whose two numbers add to the whole list — which is precisely the fact the old behavior could not state. This is principle 4 ("failures are data, never warn-and-drop") applied at the boundary it had not reached, and the same shape the page-snapshot note already uses one level down. Grouping differs deliberately: the snapshot note groups by *reason* because one missing poppler explains every document at once, while this lists by *source*, because the source is what the caller has to go fix. The exit code is unchanged at 1 and `guard` remains the only place a `BackdraftError` becomes one — nothing caught per source is re-raised, and the report exits through `fail` like any other error line, on stderr, so a clean run's stdout is byte-identical to before and that is pinned by a test. Tradeoff accepted: a run that fails partway now leaves *more* in the registry than it used to, so "the command failed" no longer implies "nothing happened". The closing line of the report answers that directly — re-running the same list is safe, because a source already ingested with unchanged bytes, extractor and config is a no-op — which is cheaper for a caller than reconstructing the failed half by diffing `ls` against its own arguments. |
 
 ## Open
 

{backdraft-0.6.0 → backdraft-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: backdraft
-Version: 0.6.0
+Version: 0.7.0
 Summary: Drop-in provenance for factual claims: one click from any claim to the evidence behind it.
 Project-URL: Homepage, https://backdraft.dev
 Project-URL: Documentation, https://backdraft.dev/docs.html
@@ -56,7 +56,9 @@ cd demo
 cells for spreadsheet sheets. Formats: PDF, XLSX/XLSM, XLS, CSV/TSV, DOCX,
 PPTX, HTML, images (png, jpeg, tiff, through the vision model), plain text and
 Markdown. A source can also be an `http(s)` URL — the page is fetched once and
-snapshotted like any other source, and the URL travels with it.
+snapshotted like any other source, and the URL travels with it. A source that
+cannot be read does not end the run: the rest of the list is ingested anyway,
+and the command exits 1 naming each source that failed and why.
 
 ```console
 $ backdraft init
@@ -117,6 +119,25 @@ $ backdraft cell underwriting-model "assumptions!B10"
 [bd:underwriting-model:assumptions!B10:964a]  24850000
 ```
 
+And a token you were handed — out of an artifact, a draft, someone else's
+message — runs backwards to what it actually says:
+
+```console
+$ backdraft show bd:t12-summary:p1.c1:c2e8
+[bd:t12-summary:p1.c1:c2e8]  resolved  t12-summary p1.c1
+Bridgeview Commons — Trailing Twelve Month Summary
+
+Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
+garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. …
+
+[Read the page: backdraft read t12-summary p1]
+```
+
+`show` is the gate too, so what it shows is minted and citable. A token whose
+source has changed since prints both snippets, the one that was cited and the
+one standing there now; a token that names nothing exits 1 and says whether the
+slug or the locator is the wrong half.
+
 **Cite** by writing the token as the href of a markdown link on the claim span.
 Multiple citations are `;`-separated in one href.
 
@@ -134,7 +155,7 @@ bound 15 claim(s), 16 citation(s) [frontwalk]
   unresolved: 1
   overlap: pass 11, skip 4
   value-trace: pass 15
-  ! unresolved: bd:t12-summary:p4.c1:1a2b
+  ! unresolved: bd:t12-summary:p4.c1:1a2b — replacement reserve of $250 per unit per year @2900
 wrote .backdraft/records/memo.backdraft.json
 
 $ echo $?
@@ -269,7 +290,11 @@ explicit, backdraft-scoped consent: `BACKDRAFT_VLM_API_KEY` (env or
 `OPENAI_API_KEY`/`OPENROUTER_API_KEY` are deliberately never read; a generic
 key in the environment is not consent to send documents to its provider. The
 default provider is OpenRouter running Gemini 3.1 Flash Lite; direct OpenAI is
-base_url + model, set explicitly. Without a scoped key, `auto` falls back to
+base_url + model, set explicitly (`--config base_url=...` and `--config
+model=...`, one key per flag). Every `--config` key is declared by the extractor
+that reads it and checked against the one `ingest` chose, so a misspelled key
+fails and names the keys that apply rather than being silently ignored.
+Without a scoped key, `auto` falls back to
 the keyless text layer and says so — still storing each page's image, which it
 renders locally. PDF page rendering needs poppler, installed separately; without
 it ingest succeeds without the images and names `snapshot-pages` as the backfill.

{backdraft-0.6.0 → backdraft-0.7.0}/README.md

@@ -31,7 +31,9 @@ cd demo
 cells for spreadsheet sheets. Formats: PDF, XLSX/XLSM, XLS, CSV/TSV, DOCX,
 PPTX, HTML, images (png, jpeg, tiff, through the vision model), plain text and
 Markdown. A source can also be an `http(s)` URL — the page is fetched once and
-snapshotted like any other source, and the URL travels with it.
+snapshotted like any other source, and the URL travels with it. A source that
+cannot be read does not end the run: the rest of the list is ingested anyway,
+and the command exits 1 naming each source that failed and why.
 
 ```console
 $ backdraft init
@@ -92,6 +94,25 @@ $ backdraft cell underwriting-model "assumptions!B10"
 [bd:underwriting-model:assumptions!B10:964a]  24850000
 ```
 
+And a token you were handed — out of an artifact, a draft, someone else's
+message — runs backwards to what it actually says:
+
+```console
+$ backdraft show bd:t12-summary:p1.c1:c2e8
+[bd:t12-summary:p1.c1:c2e8]  resolved  t12-summary p1.c1
+Bridgeview Commons — Trailing Twelve Month Summary
+
+Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
+garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. …
+
+[Read the page: backdraft read t12-summary p1]
+```
+
+`show` is the gate too, so what it shows is minted and citable. A token whose
+source has changed since prints both snippets, the one that was cited and the
+one standing there now; a token that names nothing exits 1 and says whether the
+slug or the locator is the wrong half.
+
 **Cite** by writing the token as the href of a markdown link on the claim span.
 Multiple citations are `;`-separated in one href.
 
@@ -109,7 +130,7 @@ bound 15 claim(s), 16 citation(s) [frontwalk]
   unresolved: 1
   overlap: pass 11, skip 4
   value-trace: pass 15
-  ! unresolved: bd:t12-summary:p4.c1:1a2b
+  ! unresolved: bd:t12-summary:p4.c1:1a2b — replacement reserve of $250 per unit per year @2900
 wrote .backdraft/records/memo.backdraft.json
 
 $ echo $?
@@ -244,7 +265,11 @@ explicit, backdraft-scoped consent: `BACKDRAFT_VLM_API_KEY` (env or
 `OPENAI_API_KEY`/`OPENROUTER_API_KEY` are deliberately never read; a generic
 key in the environment is not consent to send documents to its provider. The
 default provider is OpenRouter running Gemini 3.1 Flash Lite; direct OpenAI is
-base_url + model, set explicitly. Without a scoped key, `auto` falls back to
+base_url + model, set explicitly (`--config base_url=...` and `--config
+model=...`, one key per flag). Every `--config` key is declared by the extractor
+that reads it and checked against the one `ingest` chose, so a misspelled key
+fails and names the keys that apply rather than being silently ignored.
+Without a scoped key, `auto` falls back to
 the keyless text layer and says so — still storing each page's image, which it
 renders locally. PDF page rendering needs poppler, installed separately; without
 it ingest succeeds without the images and names `snapshot-pages` as the backfill.

backdraft-0.7.0/ROADMAP.md

@@ -0,0 +1,316 @@
+# Roadmap
+
+Sequenced intent. DESIGN.md records what was *decided*; this file records what
+is *queued*, so tabling something is an act of writing it down here rather than
+losing it. Items graduate by getting built and deleted.
+
+## How the queue runs
+
+The Now section is an ordered queue written for an implementing agent with no
+session context: each item carries its intent, its shape, and its acceptance
+test. The working agreement is one item at a time, top first, shipped complete
+— code, tests, docs, and a DESIGN.md decision row where the item calls for one
+— then deleted from this file in the same commit. Releases cut Friday, so an
+item that lands mid-week waits on PyPI until the release; nothing ships
+half-done to make a release. An item that turns out bigger than its sizing
+gets split here, not stretched silently.
+
+The cadence is daily and the runner is an agent: one item each morning Monday
+through Thursday, landing as a direct commit to `main` — this repo does not
+use pull requests, so a week of commits is the only review surface there is.
+
+Friday builds nothing new. It reads that week's diff instead — duplication,
+drift from the repo's own idioms, branches that landed untested, help text and
+error messages that confuse, commands the docs describe wrongly — and lands
+the fixes. Then the release, and only if something landed since the last tag:
+the version moves in both `pyproject.toml` and `.claude-plugin/plugin.json`,
+and publishing the GitHub release is what ships to PyPI. Then this section is
+refilled with five items for the week ahead, drawn against the core — the four
+verbs, the registry, the grammar, the artifact — and against where an agent
+using backdraft actually stumbles. What the maintenance pass just found is the
+best evidence available for what those five should be.
+
+## Now
+
+### 1. The demo cites a web page, so the feature is visible
+
+**Intent.** URL sources ship complete — `ingest` fetches and snapshots, the
+registry keeps `url` and `fetched_at`, and as of 2026-08-06 the artifact links
+back to the page with its fetch date. None of it appears at
+`backdraft.dev/demo.html`, because both demo sources are local files
+(`t12-summary.pdf`, `underwriting-model.xlsx`). The one artifact anybody
+actually looks at does not show the capability, and the byte-identity test that
+proved the change was safe is the same fact that keeps it invisible. A reader
+evaluating backdraft cannot see that a web citation is a citation like any
+other.
+
+**Shape.** Add a third source to `demo/`: the Wikipedia article for the county
+the property sits in, cited for market context — population, median household
+income, the kind of external figure a screening memo genuinely leans on and
+cannot get from a T12. Bridgeview Commons is fictional and says so in its own
+first line, so give it a real county and let the memo name it; one new
+paragraph under "The asset" or a short "Market" section, with one or two
+claims. Cite Wikipedia's **permanent link** (`?oldid=<rev>`), not the bare
+article URL: an article is edited constantly, a citation into the live URL
+would report `drifted` within days, and a front-page demo that permanently
+shows drift teaches that the tool is broken rather than that the source moved.
+An immutable revision plus the rendered "as of" date is also what a careful
+analyst cites. Write the tradeoff as a DESIGN row, naming the thing given up —
+the demo no longer shows a live page changing under a citation, which is the
+better story and belongs in a `drifted` walkthrough, not the shop window.
+
+Practical notes for whoever builds this: `demo/.backdraft/` is gitignored, so
+the fetch happens once on the implementer's machine and the snapshot then
+travels in the local registry; `demo/generate_sources.py` builds the two
+fixture files and is *not* where this goes, since the page is fetched rather
+than generated. Re-bind and re-render (`bind memo.md --session s-bridgeview
+--check value-trace,overlap`, then `render memo.md --to html`), and copy the
+result over `site/demo.html` — the two are byte-identical by convention.
+`demo/walkthrough.md` shows real CLI output and its `ls` and `bind` blocks both
+move.
+
+**Acceptance.** `backdraft.dev/demo.html` shows three sources, one of them a
+clickable Wikipedia link with a fetch date, on both the resting rail and the
+end-matter list; the claim citing it opens a card whose source line carries the
+same link. `demo/walkthrough.md` matches what the commands now print. A re-bind
+some weeks later still reports `resolved`, not `drifted` — that is the check
+that the permanent link did its job.
+
+**Size.** One day.
+
+### 2. The gate names a fetched source by its page, not by its staging filename
+
+**Intent.** A fetched page's `filename` is `q4-2025.html`, a name invented by
+`fetch.filename_for` for a temporary file that no longer exists. The
+2026-08-06 decision row ruled on this for the artifact — the URL *replaces* the
+filename there, because "showing both would give a reader two names for one
+thing and let the fictional one look authoritative" — but the same fictional
+name is still what `backdraft read` prints in its document list, what its table
+of contents headline prints, and what `ls` prints in column two. The gate is
+where a writing agent actually lives, and there it learns nothing about
+provenance at all: `skills/backdraft/SKILL.md` has to send it to `ls` for a
+fact the surface it is already using could tell it. One rule, applied in one
+place, is the fix.
+
+**Shape.** `gate/reader.py`'s `render_documents` and `_document_headline`, plus
+`cli.py`'s `ls` (which now shares the gate's page-count vocabulary through
+`gate.unit` and should share this too). For a document whose `meta` carries a
+`url`, show the origin in place of the filename, exactly as the artifact's
+source list does. The gate's output is a contract — its module docstring says
+"stable enough to diff" — so keep it line-oriented and keep the column
+alignment: a URL is long, so put it on the row rather than in the aligned
+filename column if alignment would collapse, and say which choice you made in
+the docstring. `Document.meta` is already carried by `Registry.documents()` and
+`Registry.document()`, so no registry change is needed. Do not touch the slug,
+the token, or anything a citation resolves through — this is display, and the
+2026-08-05 row's "provenance, never identity" governs.
+
+**Acceptance.** In a registry holding one file and one fetched page, `backdraft
+read` shows the file by its filename and the page by its URL; `backdraft read
+<slug>` for the fetched page carries the URL in its headline; `backdraft ls`
+matches. A registry of files prints exactly what it printed before — pin that
+as a test, the way the artifact's byte-identity is pinned. `README.md`, the
+docs page and `skills/backdraft/SKILL.md` drop the detour through `ls` for a
+fact `read` now carries.
+
+**Size.** Two days.
+
+### 3. A fetched page's slug is decided, not defaulted
+
+**Intent.** DESIGN.md's Open list has carried "slug assignment/collision rules"
+since the first field trial, and URL sources sharpened it into a real failure:
+`fetch.filename_for` takes the URL's last path segment, so
+`https://a.example/reports/2025/index.html` and
+`https://b.example/docs/index.html` both stage as `index.html` and become slugs
+`index` and `index-2`. An agent then cites `bd:index-2:p1.c3:...` with no way to
+tell which site that is, and a slug is the handle every token in every authored
+document carries — it is stable once assigned, so a bad one is permanent. The
+same collapse hits any site whose pages are `index.html`, `page`, `view`, or a
+bare numeric id.
+
+**Shape.** `fetch.filename_for`, and a decision row that closes the Open
+question for files and URLs together. For a URL, build the stem from enough of
+the address to distinguish it — the host plus the last meaningful path segment
+is the obvious candidate, with the registry's existing `slug_for` slugification
+doing the rest — and keep `_dedupe`'s `-2` as the collision *backstop* rather
+than the normal outcome. `store._assign_slug` and `_dedupe` do not need to
+change; this is about giving them a stem worth deduping. Note the constraint
+that makes this delicate: continuity across a re-fetch is matched on the URL
+(see `_find_document`), not on the slug, so changing the stem rule does not
+break an existing registry's documents — but it does mean two registries built
+on either side of this change disagree about a page's slug, which is a
+compatibility statement the decision row has to make out loud.
+
+**Acceptance.** The two `index.html` URLs above ingest as two slugs that each
+name their site. `https://example.com/reports/q4-2025` still ingests as
+`q4-2025` — the common case does not get uglier to fix the collision case. A
+URL with an empty path still falls back to the host. Tests for each, plus one
+that two pages colliding even after the new rule still dedupe rather than
+error. DESIGN.md gains the decision row and the Open list loses the line.
+
+**Size.** Two days.
+
+### 4. Ingest says what it did and what it got
+
+**Intent.** Two things `ingest` knows and does not say. First, it prints the
+same line whether it created a document, created a *new generation* of one
+whose bytes changed, or did nothing at all because the bytes and config were
+identical — so an agent re-ingesting after a fix cannot tell whether anything
+happened, and the moment a new generation appears is exactly the moment
+existing bindings may start reporting `drifted`. Second, it never says how much
+text came out. `skills/backdraft/SKILL.md` instructs the agent that a
+JavaScript-rendered page or one behind a login "will come back thin or empty,
+and if it does, say so to the user rather than citing the shell of it" — and
+gives it no signal to notice that with. A login wall extracts as a few dozen
+characters, exits 0, and prints `1 page` like any success. The same silence
+covers a scanned PDF with no text layer.
+
+**Shape.** `ingest` in `cli.py`, output only — no registry or format change.
+`Registry.ingest` already distinguishes the three outcomes internally
+(`_find_document`, `_is_noop`, `_upsert_document`); surface which one happened
+rather than re-deriving it in the CLI, which may mean returning it alongside
+the `Document` or exposing it on the returned value. Print the extracted
+character count beside the page count, and when it falls under a threshold the
+module declares as a named constant, add one line naming the likely cause and
+what to do — the shape the poppler and `pdf-text` notes already use, which is a
+note at exit 0 and never a failure, because a thin page is still a real
+snapshot and failures here are data. A new generation gets its own line saying
+that citations into the previous one may now report `drifted`, and that `bind`
+is what will say so.
+
+**Acceptance.** Ingesting a file twice unchanged says so the second time;
+editing it and re-ingesting says a new generation was made and names drift;
+both still exit 0. A near-empty HTML file ingests, exits 0, and prints the thin
+-source note; a normal source does not. `backdraft ingest` on a fresh document
+prints what it prints today plus the character count. Tests for all five
+branches. `demo/walkthrough.md`, `README.md` and `site/docs.html` show the real
+new output, and `skills/backdraft/SKILL.md` points at the note instead of
+asking the agent to judge thinness unaided.
+
+**Size.** Two days.
+
+### 5. `backdraft verify <artifact>`: checking a receipt without the registry
+
+**Intent.** `skills/backdraft-artifact/SKILL.md` exists because there is no
+command: it walks an agent through opening a `.backdraft.json` or
+`.backdraft.html`, recomputing each `snippet_sha256`, and reporting claim
+statuses by hand. That is a deterministic check over a self-describing file,
+which is precisely the kind of thing that should not be a prose procedure an
+agent re-implements every time — and doing it by hand is how a recipient's
+check ends up weaker than the producer's. The artifact is the product; nothing in the
+tool reads one back.
+
+**Shape.** A new command in `render/cli.py` (which already owns the artifact's
+reader half through `sidecar.read`), taking the artifact or its sidecar.
+Two tiers, and the output must be explicit about which ran. Tier one needs
+nothing but the file: parse it against `kernel/artifact.py`'s format, recompute
+every `snippet_sha256` from the snippet the file carries, and confirm the
+`$format` and the claim/citation counts — this is the check a recipient with
+only the file can make, and it catches an edited artifact. Tier two runs only
+when a registry is discoverable: re-resolve each token through the same path
+`bind` uses and report the closed status set. Exit 0 when everything checked
+passed, 1 on a usage error, and — matching `bind` — 2 when something did not
+verify, so a hook can gate on it. Read-only: it opens no session and mints
+nothing, which is what distinguishes it from `read` and from `backdraft show`
+(landed 2026-08-12), whose whole point is that showing mints — an audit must not
+make its subject citable.
+
+**Acceptance.** `backdraft verify demo/memo.backdraft.html` from outside
+`demo/` reports tier one passing and says plainly that no registry was found,
+exiting 0. From inside `demo/` it also re-resolves and reports the one
+unresolved citation, exiting 2. A file with one snippet byte changed fails by
+naming the claim and the hash that did not match. A file that is not an
+artifact exits 1 saying so. `skills/backdraft-artifact/SKILL.md` is rewritten
+around the command, keeping the prose procedure only as the fallback for an
+agent that has no backdraft install.
+
+**Size.** Three days.
+
+### 6. `spec/registry.md`: the export format is normative, or it is not a format
+
+**Intent.** `backdraft export` writes `"$format": "backdraft/registry-v1"`, a
+version string that promises a specification and does not have one. `spec/`
+holds `artifact.md`, `tokens.md` and `chunking.md`, and the repo's own rule is
+that another implementation reads those and nothing else — the export is the
+only declared format outside that set. It is also the only complete, portable
+representation of a registry, so it is what a second implementation, a
+migration, or an audit would read. The gap is not theoretical: the 2026-08-05
+work added a conditional `meta` key to every exported document, a format change
+to a format with nothing to change.
+
+**Shape.** A new `spec/registry.md` in the voice of `spec/artifact.md`:
+normative prose, MUST/OPTIONAL where it means them, and written to be readable
+without the code. Cover the top-level shape, the document entry including the
+`meta` key and its `url`/`fetched_at` (OPTIONAL, present only for a fetched
+source — the same conditional the artifact spec already states, for the same
+compatibility reason), every extraction generation with `is_current`, pages
+and cells, and anchors with their receipts. State what identity is and what it
+is not, matching the concept table in `SPEC.md`. Say what a conforming reader
+must do with a key it does not recognize. Pin it the way the artifact format is
+pinned: a golden-file test over the demo registry's export, so the next
+conditional key is a test failure rather than an undocumented change.
+
+**Acceptance.** `spec/registry.md` describes every key `Registry.export`
+actually emits — check that mechanically, with a test that walks the exported
+JSON and fails on a key the spec does not name. `SPEC.md`'s file map and
+`README.md`'s spec list gain it. Exporting the demo registry and reading only
+the spec is enough to say what each field means.
+
+**Size.** Two days.
+
+## Parked
+
+Deliberately not queued, each with the reason, so picking one up starts from
+the objection rather than rediscovering it.
+
+- **Exhibits view** — the evidence-first inversion. Risk: it grows the
+  artifact and pollutes a design whose strength is restraint. Needs a
+  deliberate design pass and a real audience before any code; not a
+  weekend-feature shape.
+- **Excel region maps and range reads** — the research notes exist, but no
+  confidence the deterministic detection cascade lands well on the first
+  try. Revisit when large-sheet navigation hurts in a real corpus, with
+  test workbooks in hand.
+- **`bd:calc` derivations** — reserved in the grammar and parsed as
+  malformed-on-purpose, which is enough for now. Full support complicates
+  binding and verification for one field-trial case.
+- **Merged-cell and frozen-pane rendering** — ingest captures `merged` and
+  `frozen` in sheet meta and keeps them (cheap, and capture is the part you
+  cannot backfill). Frozen-pane rendering is cut; merged-cell rendering
+  waits for a real sheet that renders wrong without it.
+
+## Later
+
+- **Living documents** — re-ingest, re-bind, and a drift-first report: "these
+  claims cite figures that changed; cited-then vs. now." The primitives
+  (generations, `drifted`, the word-diff) all exist; the missing piece is a
+  diff-shaped report and a demo.
+- **Substrates beyond the CLI** — SDK middleware for pipelines (the exit-code
+  and `--to json` contracts are already shaped for it), and a client-side
+  drag-drop viewer page (no upload — the file never leaves the reader's
+  machine).
+- **Email (.eml)** — arrives in diligence folders alongside everything else;
+  identity questions (headers vs. body, attachments as child sources) deserve
+  a decision row before code. The web-page half of this item shipped
+  2026-08-05; `document_meta` is where an email's provenance would land.
+- **Anthropic-API provider for the VLM extractor** — Cowork's sandbox
+  egress allowlist includes api.anthropic.com but not the OpenAI-compatible
+  providers, so an Anthropic-native option would let sandboxed sessions
+  ingest at full fidelity. Needs a client abstraction the vlm module
+  currently doesn't have; the documented workaround (ingest locally, the
+  registry travels with the folder) covers it meanwhile.
+- **Distribution** — published on PyPI; the repo ships
+  `.claude-plugin/plugin.json` and `marketplace.json`, so it installs as a
+  Claude Code plugin and self-hosts its marketplace, and the AGENTS.md
+  snippet is published in the README and site docs. The community-directory
+  submission went in 2026-07-29 and is awaiting review — approval is what
+  surfaces the skills in Cowork's directory.
+
+## Someday
+
+- **Hosted team features** — upload, access control, shared registries.
+  Deliberately parked: the product's trust story is the self-contained file,
+  and a hosted viewer inverts it. Revisit only as a distinct teams product.
+- **Entail at scale** — the model-judge verifier is wired (`[entail]` extra)
+  but has never been field-calibrated; needs a corpus and a rubric before it
+  is recommended anywhere.

{backdraft-0.6.0 → backdraft-0.7.0}/SPEC.md

@@ -231,11 +231,12 @@ class Extractor(Protocol):
     name: str
     version: str
     deterministic: bool
+    config_keys: Mapping[str, str]         # optional; --config surface, key -> meaning
     def can_handle(self, path: Path, media_type: str) -> bool: ...
     def extract(self, path: Path, config: dict) -> Iterator[ExtractedPage]: ...
 ```
 
-Registered in a plain dict; `ingest --extractor auto` picks the first `can_handle`, built-ins ordered `xlsx, xls, csv, docx, pptx, pdf-text, image, html, text` (an extractor whose optional extra is missing is skipped, not fatal). `vlm` is pdf→images→VLM per page, OpenAI-compatible client, model configurable; its deps ship by default and the backdraft-scoped key gates use. **For PDFs, `auto` prefers `vlm` when it is ready (importable, key configured)** (glossy layouts and info boxes extract badly from the text layer, and the snapshot is the receipt); otherwise it falls back to `pdf-text` and prints a one-line nudge naming the VLM option. An explicit `--extractor` always wins. The sheet representation (shared by xlsx, xls, csv): `Row | A | B...` header, `[B10] value` prefixes, dimension caps, inflated-sheet placeholder pages; cell values are never rounded, since the snapshot is the receipt a claim is traced against. `docx` synthesizes pages from heading sections (the document's smallest present outline level among Heading 1–2 splits; no headings → one page), so locators stay `pN.cM`. `pptx` is one page per slide, text floor only; ingest notes that slide visuals need the PDF-export path. `pdf-text` reconstructs paragraph breaks from line geometry before handing the page over, so the chunker's blank-line rule has something to fire on. `html` is one page for the whole document, named by its `<title>`: block elements end a block so the chunker's blank-line rule fires, `script`/`style`/`noscript`/`template`/`svg` are dropped whole, lists become `- `/`1. ` lines in a single block, and tables become pipe tables as `docx` renders them. It is a parse and not a readability heuristic — no boilerplate stripping, since a guess that changes between two versions of a site moves anchors. Decoding is a pure function of the bytes (BOM, then `<meta charset>`, then UTF-8 with replacement), so a saved page and a fetched one extract identically.
+Registered in a plain dict; `ingest --extractor auto` picks the first `can_handle`, built-ins ordered `xlsx, xls, csv, docx, pptx, pdf-text, image, html, text` (an extractor whose optional extra is missing is skipped, not fatal). `config_keys` declares the `--config` keys an extractor reads; `Registry.ingest` validates the parsed config against the *chosen* extractor (after selection, since `auto` picks per file) and raises on an undeclared key, naming it and the ones that apply. Declaring nothing is the default and means accepting nothing. The page-render keys (`dpi`, `snapshot_quality`, `snapshot_max_height`) are declared once in `extract/snapshots.py` and taken by both PDF paths, since both rasterize; the provider keys (`api_key`, `base_url`, `model`, `timeout`, `retries`) are declared in `extract/vlm_settings.py` and taken by both vision paths; `concurrency` is `vlm`'s alone. `vlm` is pdf→images→VLM per page, OpenAI-compatible client, model configurable; its deps ship by default and the backdraft-scoped key gates use. **For PDFs, `auto` prefers `vlm` when it is ready (importable, key configured)** (glossy layouts and info boxes extract badly from the text layer, and the snapshot is the receipt); otherwise it falls back to `pdf-text` and prints a one-line nudge naming the VLM option. An explicit `--extractor` always wins. The sheet representation (shared by xlsx, xls, csv): `Row | A | B...` header, `[B10] value` prefixes, dimension caps, inflated-sheet placeholder pages; cell values are never rounded, since the snapshot is the receipt a claim is traced against. `docx` synthesizes pages from heading sections (the document's smallest present outline level among Heading 1–2 splits; no headings → one page), so locators stay `pN.cM`. `pptx` is one page per slide, text floor only; ingest notes that slide visuals need the PDF-export path. `pdf-text` reconstructs paragraph breaks from line geometry before handing the page over, so the chunker's blank-line rule has something to fire on. `html` is one page for the whole document, named by its `<title>`: block elements end a block so the chunker's blank-line rule fires, `script`/`style`/`noscript`/`template`/`svg` are dropped whole, lists become `- `/`1. ` lines in a single block, and tables become pipe tables as `docx` renders them. It is a parse and not a readability heuristic — no boilerplate stripping, since a guess that changes between two versions of a site moves anchors. Decoding is a pure function of the bytes (BOM, then `<meta charset>`, then UTF-8 with replacement), so a saved page and a fetched one extract identically.
 
 ## Gate (gate/)
 
@@ -243,6 +244,7 @@ CLI-facing behavior contract:
 - `read` with no args → document list (slug, filename, media, pages). `read <slug>` → TOC (page/sheet, name, summary-or-first-120-chars). `read <slug> p3` / `p3-5` / `rent-roll` → token-marked content.
 - PDF pages render as chunks: `[bd:slug:p3.c1:a7f3]` on its own line above each chunk's text. Sheets render the markdown table as-is with a header line carrying the page token; cell tokens are NOT inlined per-cell (the `[B10]` refs are already in-band; bind composes `bd:slug:sheet!B10:hash` from the registry).
 - `search "<query>"` → FTS5 over anchors; each result: token, slug, page, snippet. Results are minted (ledger-recorded) — a searched snippet is citable without a page read.
+- `show <token>...` → the inverse of minting: per token, its bind status, its slug and locator, and the verbatim snippet, in argument order. `resolved` prints the snippet; `drifted` prints both the cited snippet and what stands at the locator now (and mints that anchor too, since it is the one worth citing); `unresolved` says whether the slug or the locator is what named nothing; `malformed` carries the kernel's reason and the grammar. Statuses and resolution are bind's, reached through `kernel.claims.parse_citation` and `Registry.resolve` rather than a second reading of what a token means; `not_shown` cannot occur, because showing mints. Exit 1 if any token was `unresolved` or `malformed`, with the block still on stdout.
 - Every emitted token is recorded to the ledger under the session (`--session` flag or `BACKDRAFT_SESSION` env; auto-created default session otherwise).
 - Long pages: `--offset/--limit` on chars for PDFs; sheets paginate by rows, never mid-row, header row repeated.
 
@@ -293,6 +295,7 @@ backdraft ingest <sources...> [--extractor auto] [--slug S] [--config k=v]
                                     # a source is a path or an http(s) URL
 backdraft ls | backdraft read ...   # gate, above
 backdraft search "<query>" [--in slug]
+backdraft show <token>...           # gate: what does this token say?
 backdraft bind <doc.md> [--session S] [--check ...] [--mode ...]
 backdraft render <doc.md> [--to html|footnotes|json] [-o out] [--theme name|file]
 backdraft theme [list|show <name|file>]
@@ -327,6 +330,8 @@ Integration invariants (every workstream's tests must respect): any token the ga
 
 `registry/store.py` exposes exactly this surface. W1 implements it; W2/W3 consume it and MUST NOT implement anything under `registry/`; their tests use a lightweight fake implementing these names.
 
+Beside the class, `registry` exports one module function, `current_at(registry, anchor) -> Anchor | None`: the current generation's anchor at `anchor`'s locator, or None if the locator is gone. It is the other half of drift — `resolve` finds what the writer saw, this finds what stands there now — and it lives here because both `bind` and the gate's `show` need it and the dependency rule forbids either importing the other. A function rather than a method so it composes the surface above without widening it, and so the fakes keep working unchanged.
+
 ```python
 class Registry:
     @classmethod
@@ -389,4 +394,4 @@ class Registry:
 
 ## Addendum B — CLI assembly
 
-Top-level `cli.py` (W1) owns: typer app, registry discovery (nearest `.backdraft/` walking up from cwd; `BACKDRAFT_HOME` override), session resolution (`--session` flag > `BACKDRAFT_SESSION` env > default session), and the `init` / `ingest` / `ls` / `export` / `snapshot-pages` commands. After each ingest, `cli.py` captures page snapshots for any PDF whose extraction carries none — the text-layer path, since `vlm` stores its own — via `extract/snapshots.py` (poppler through pdf2image, `snapshot-pages`' internals). It sits outside the extractor so `pdf-text` stays deterministic regardless of whether poppler is installed, and it is best-effort: a `SnapshotError` leaves ingest at exit 0 with a one-line note naming `snapshot-pages` as the backfill. `cli.py` also owns the network for the same reason: an `ingest` argument with a scheme is fetched through `fetch.py` (stdlib `urllib`, http and https only, redirects followed, 32 MiB cap) and staged in a temporary file named for the content type the server declared — which is what selects the extractor — so extractors stay pure functions of bytes and the registry opens no sockets. It mounts sub-apps `gate/cli.py` (W2: `read`, `search`, `session`), `bind/cli.py` (W3: `bind`), `render/cli.py` (W4: `render`) — each exposes `app = typer.Typer()`; the top level mounts each inside a try/except ImportError so partial merges still run. The `backdraft` console script is declared by W1.
+Top-level `cli.py` (W1) owns: typer app, registry discovery (nearest `.backdraft/` walking up from cwd; `BACKDRAFT_HOME` override), session resolution (`--session` flag > `BACKDRAFT_SESSION` env > default session), and the `init` / `ingest` / `ls` / `export` / `snapshot-pages` commands. `ingest` runs its whole source list: a `BackdraftError` from one source is caught, recorded against that source, and the loop continues, so a folder ingest is never half-attempted. Every failure is printed at the end as one `!` line naming the source and the reason, under a count of what landed, and the command exits 1 — the guard in `cli_context` stays the only place a `BackdraftError` becomes an exit code, since nothing caught per source is re-raised. After each ingest, `cli.py` captures page snapshots for any PDF whose extraction carries none — the text-layer path, since `vlm` stores its own — via `extract/snapshots.py` (poppler through pdf2image, `snapshot-pages`' internals). It sits outside the extractor so `pdf-text` stays deterministic regardless of whether poppler is installed, and it is best-effort: a `SnapshotError` leaves ingest at exit 0 with a one-line note naming `snapshot-pages` as the backfill. `cli.py` also owns the network for the same reason: an `ingest` argument with a scheme is fetched through `fetch.py` (stdlib `urllib`, http and https only, redirects followed, 32 MiB cap) and staged in a temporary file named for the content type the server declared — which is what selects the extractor — so extractors stay pure functions of bytes and the registry opens no sockets. It mounts sub-apps `gate/cli.py` (W2: `read`, `search`, `session`), `bind/cli.py` (W3: `bind`), `render/cli.py` (W4: `render`) — each exposes `app = typer.Typer()`; the top level mounts each inside a try/except ImportError so partial merges still run. The `backdraft` console script is declared by W1.

{backdraft-0.6.0 → backdraft-0.7.0}/demo/walkthrough.md

@@ -339,7 +339,7 @@ bound 15 claim(s), 16 citation(s) [frontwalk]
   unresolved: 1
   overlap: pass 11, skip 4
   value-trace: pass 15
-  ! unresolved: bd:t12-summary:p4.c1:1a2b
+  ! unresolved: bd:t12-summary:p4.c1:1a2b — replacement reserve of $250 per unit per year @2900
 wrote .backdraft/records/memo.backdraft.json
 exit: 2
 ```
@@ -351,6 +351,13 @@ the current extraction of their source. One did not, and it is named. Bind did
 not delete it, did not warn-and-continue, and did not quietly attach it to
 something nearby.
 
+**The line item names the sentence, not just the token.** Exit 2 sends you back
+into `memo.md` to fix something, and `bd:t12-summary:p4.c1:1a2b` is the one
+thing you already had. The claim's own words follow it, then `@2900` — the
+claim's character offset into the authored document, which is what tells two
+line items carrying the same token apart when one token is cited from several
+sentences.
+
 **Exit code 2 means unresolved citations.** Not a failed check — a citation that
 does not point at anything. It is the code a `Stop` hook or a CI job gates on.
 Exit 1 is a usage error; exit 0 means every citation resolved.
@@ -492,7 +499,29 @@ backdraft theme show default > ~/.config/backdraft/theme.toml
 
 ## 10. Fixing the failure
 
-The memo's broken token was a typo for a real anchor. Find it the same way you
+Start by asking the token itself. `show` is the inverse of minting: hand it a
+token from anywhere — a bind report, an artifact, a draft — and it says what
+that token names.
+
+```bash
+backdraft show bd:t12-summary:p4.c1:1a2b
+echo "exit: $?"
+```
+
+```
+[bd:t12-summary:p4.c1:1a2b]  unresolved
+t12-summary carries no anchor named by this token, in any extraction; the locator or the hash is wrong
+
+[Table of contents: backdraft read t12-summary]
+exit: 1
+```
+
+The slug is real — that document was ingested — so what was invented is the page
+or the hash. (A token whose source had *changed* rather than never existed would
+come back `drifted`, printing both the snippet that was cited and the snippet
+standing at that locator now.)
+
+So the broken token was a typo for a real anchor. Find it the same way you
 found everything else:
 
 ```bash
@@ -512,8 +541,26 @@ backdraft search "replacement reserve"
 [Read the page: backdraft read t12-summary p3]
 ```
 
-`bd:t12-summary:p3.c4:6f0f` is the snippet that actually says it. Swapping it in
-and re-binding would take the run to `resolved: 16` and exit 0.
+`bd:t12-summary:p3.c4:6f0f` is the snippet that actually says it — and `show`
+confirms that before the edit, rather than after the re-bind:
+
+```bash
+backdraft show bd:t12-summary:p3.c4:6f0f
+```
+
+```
+[bd:t12-summary:p3.c4:6f0f]  resolved  t12-summary p3.c4
+Capital expenditures of $317,000 were incurred outside the operating statement and are not deducted in arriving at
+net operating income. The largest items were roof replacement on the A and B buildings ($186,000) and parking
+lot resurfacing ($64,500). A replacement reserve of $250 per unit per year, or $32,000 annually, would be
+customary for an asset of this vintage and is not reflected in the figures above.
+
+[Read the page: backdraft read t12-summary p3]
+```
+
+`show` is the gate like `read` and `search` are, so that snippet is now minted
+into the session and the token is citable. Swapping it in and re-binding would
+take the run to `resolved: 16` and exit 0.
 
 **This demo deliberately does not make that fix.** The checked-in `memo.md`
 and `memo.backdraft.html` both carry the unresolved citation, because an

{backdraft-0.6.0 → backdraft-0.7.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "backdraft"
-version = "0.6.0"
+version = "0.7.0"
 description = "Drop-in provenance for factual claims: one click from any claim to the evidence behind it."
 readme = "README.md"
 requires-python = ">=3.13"