backdraft 0.10.0__tar.gz → 0.11.0__tar.gz

{backdraft-0.10.0 → backdraft-0.11.0}/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "backdraft",
   "displayName": "Backdraft",
-  "version": "0.10.0",
+  "version": "0.11.0",
   "description": "Drop-in provenance for factual claims: one click from any claim to the evidence behind it. Ships the backdraft writing, backfill, and artifact-reading skills; the CLI installs separately (uv tool install backdraft).",
   "author": {
     "name": "Spencer Braun",

{backdraft-0.10.0 → backdraft-0.11.0}/DESIGN.md

@@ -155,6 +155,10 @@ Self-contained HTML (plus the machine-readable sidecar): **quotes + receipts + e
 | 2026-09-01 | **An unreadable source says what to do about it, and an empty one is a failure.** `ingest` finishing its list (2026-08-13) turned the reason string into something a calling agent actually reads, and the reason was Python: `cannot read adir: [Errno 21] Is a directory: 'adir'` named the source three times, leaked an errno nobody should have to know, and said nothing about a fix that differs completely between the causes and is obvious in each one. The read now goes through `store._read_source`, which maps the four causes worth distinguishing — a directory (name the files inside it, or pass a glob), a path with nothing at it (check the spelling), a file the process may not open (fix the permissions, or ingest a copy), and zero bytes — and `fetch.py`'s messages follow the same rule for the URL half. Anything outside those four keeps the operating system's own `strerror` rather than being guessed at: an unknown cause reported plainly beats a wrong next step, and that is the one place an errno-free message loses information, which is the trade taken. The reason never names the source, because every caller already leads with it — which means the exception read on its own, by a library caller rather than through `ingest`'s `!` line, is now less self-contained than it was; the CLI is the only caller and the report is the surface that matters. Empty is the substantive change rather than a rewording: zero bytes read cleanly, so an empty file used to become a document — listed by `ls`, offered by `read`, competing in `search`, citable in name only — and a fetched page whose server sent an empty body did the same, which is the shape a JavaScript-rendered page takes here. That is not the thin-source case the 2026-08-20 row admitted a heuristic for: thin is a judgement about how much text came out of real bytes and stays display-only at exit 0, while zero bytes is a fact and there is no snapshot to make. Staging a fetched page also became a per-source failure rather than a traceback, so a full temporary directory no longer ends the run the rest of the list was promised. |
 | 2026-09-02 | **A registry can forget a source, and forgetting removes nothing.** Ingest was one-way, and now that `ingest` finishes its list the unattended folder run is the normal case — so a scratch copy, the same report under two names, or a file nobody meant to include lands and stays: listed by `read`, competing in `search`, and citable by an agent choosing evidence without anything looking wrong. The only recovery was deleting `.backdraft/`, which throws away every other document and the whole ledger. `backdraft forget <slug>` **withdraws** rather than deletes, and the whole design is in that word: the document, its generations, its anchors and its receipts all stay, so `resolve` still finds every anchor and a token already written into somebody's draft or artifact still shows its snippet under `show`. What a withdrawal changes is that the registry stops *offering* the source — `documents()` excludes it, and with it `ls`, the gate's list, its table of contents, page reads and `search`, whose filter sits in the shared `_match_sql` so the pre-limit `total` a caller is offered to widen to drops too. A deletion would have stranded those citations with nothing to say about them, which is the failure this system exists to prevent. The status is the second half of the decision and the temptation was a sixth one: `citation_for` reports a withdrawn source's token `unresolved` carrying `withdrawn_reason` as its `error`, because the closed status set is the artifact format's and a new value would break every reader written against it, while `resolved` would let a document keep citing a source the registry no longer offers and report nothing. That widens `error` from `malformed`'s explanation to any status the status alone does not explain — an optional key appearing where it applies, not a new format version — and it made the legend's own `unresolved` line wrong, since a withdrawn citation *does* carry an anchor; corrected in `LEGEND` and `spec/artifact.md` at the cost of regenerated goldens and a re-rendered demo, the 2026-08-24 trade. Storage is a `withdrawals` table rather than a column on `documents`, which is the 2026-08-05 reason `document_meta` is one: an existing registry needs no migration, and this repo has no migration machinery to introduce for one flag. Ingesting the source again is the undo and deletes the row, bringing back the *same* document under the same slug — `_find_document` matches a withdrawn document like any other, which is exactly what a delete could not do, since a second slug would leave every token carrying the first one stranded. That undo is silent-by-default territory, so it is not: `Ingested.restored` is a fact beside `outcome` rather than a fourth value of it (a forgotten file that was also edited is both restored and a new generation, and one field could only say one), the line says `restored`, and a grouped note names what came back. Two costs stated rather than discovered. A registry only ever grows: nothing is reclaimed, and a corpus you must be able to shrink is a new registry, not a `forget`. And the rule is drawn at *offering a source to read*, so `snapshot-pages` still backfills page images for a withdrawn document — it serves no text and cannot reach a writer's context, and widening the rule to every command that touches a document would make it a rule nobody can state. `forget` asks before withdrawing and, where nothing can answer, names `--yes` instead of hanging on a prompt or aborting without a reason, since the usual caller here is an agent with no terminal. |
 | 2026-09-03 | **The rule moved down to the type it is about, and the last surface naming a source wrongly stopped.** The 2026-08-18 row gave `gate.source_name` one owner and accepted one exception on a dependency argument, and the argument was sound about imports and wrong about where the rule belonged. Bind's `_doc_name` returned `Document.filename`, and it feeds *both* halves of the markdown projection — the generated `## References` section and backfill's proposed anchors — so `bind --bound` called the demo's Wikipedia article `index.html` while `render --to footnotes` named its URL and the artifact named it *Franklin County*. Three of backdraft's own outputs, two answers, and the one that disagreed was the one a reader gets without a browser: the bound markdown is the form that travels into a pull request or an email, where the HTML artifact cannot follow. The fix is to stop working around the dependency rule and read it properly. `source_name` is a pure function of a `Document`, `Document` is defined in `kernel/model.py`, and gate, bind and the CLI all import that module *downward* — so the kernel was the owner the rule was looking for all along, and it stays stdlib-pure there. SPEC § Dependency rule now says so in general, because the reasoning that produced the exception ("we cannot share it sideways, so we duplicate or we live with it") will recur for the next shared rule and the escape hatch should not have to be rediscovered. `gate.source_name` is **not** left as a redirect: callers import `kernel.model.source_name` and the gate's re-export is gone, since two import paths for one function is the same shape as two names for one source — the thing the 2026-08-06 row rejected — and the gate's stated export style is names it owns, not names it passes through. The cost is that anything importing `from backdraft.gate import source_name` breaks; it is one internal caller and a library surface nobody outside this repo is known to use, and leaving both paths alive is what makes an owner stop being one. `_origin` moved with it as public `source_origin`, because the gate's document list needs the *question* ("is this name a URL, and therefore must not size the name column?") and not just the answer, and two readings of one `meta` key drift apart the moment the key does. Display only, unchanged: the slug, the token and everything a citation resolves through are untouched, and a References entry for a file source is byte-identical to what it printed before — pinned as a whole line, the way `ls`'s file rows are. The render side was deliberately not touched and was never wrong: `render/html/text.source_title` and `render/footnotes._origin` read a `url` off the *artifact's* `docs` dict, which is the artifact format's shape rather than a `Document`, and they already prefer the slug and the URL over the invented filename — they look like misses and are not. One thing this closes that was already promised in prose: `demo/walkthrough.md` told readers that a staging filename appears at "no surface that names a source", which was false for as long as this row's exception stood. |
+| 2026-09-07 | **A one-page source's table of contents is its chunks, and a page that named itself is called by that name.** `backdraft read franklin-county` opened with `Jump to content Main menu move to sidebar hide Navigation`, which is the site's chrome and not the page, and it was the *only* thing the table of contents said about a 34,000-character article — because a web page has no pagination, so the page list is one row and does no navigating at all. Two halves, one fact. The listing half: a document that is a single page of prose now lists that page's chunks underneath it, `p1.c1  opening words` per row, read off `anchors_for_page` and never re-chunked (rule 1 of the reader), so every locator printed is one a token already carries. The page's own preview goes with it, because it *was* the first chunk's opening words said twice, and a page row left carrying nothing but `p1` is dropped, since every chunk row below already begins with it. Scoped to one page rather than to HTML: a one-page PDF has the same problem, and a single-sheet workbook mints cell anchors and no chunk anchors, so it lists nothing and prints what it always did — as do all three of the demo's sources except the article, pinned byte for byte. The naming half: `extract/html.py` already parsed the `<title>` and threw away the fact that it *found* one, so `Page.meta` now carries `title` and carries it only where the markup declared it. `Page.name` could not answer the question, and that is the whole reason for a second field holding the same string: `name` falls back to the file stem, and a fetched page's stem is the one `fetch.filename_for` invented — so a page called `index` and a page called *Franklin County, Ohio - Wikipedia* were indistinguishable to anything downstream. Bind copies it into the evidence entry (`spec/artifact.md` § evidence, optional, absent for everything else) and `render/html/text.source_title` prefers it, which is what stops a URL ingested without `--slug` from titling itself *En Wikipedia Org Index* in the artifact's source list. Only a single-page source answers, because page 1's title is the document's title exactly when page 1 is the document; a section heading must not name a whole report. Display only, and the 2026-08-20 thin-source argument is the one that admits it: no chunk boundary moves, no anchor moves, no token changes, the slug is still not derived from the title, and the demo's 18 tokens are identical before and after. That is also why this is not the boilerplate stripping the extractor refuses — a heuristic that changes its mind moves anchors; this reorders a preview. One deviation from the queued shape, stated rather than buried: the item asked for `_preview` to prefer the title too, and it must not, because the gate already had the title in `Page.name` and every page that would reach that branch has its chunks listed instead — the branch is unreachable, and unreachable code is what Friday deletes. One cost taken knowingly: the demo's registry predates the page meta and re-ingesting the same bytes is a no-op by design, so the demo artifact still names that source from its slug and will pick up the title the next time the registry is rebuilt. Re-fetching to force it is the one thing that must not happen — Wikipedia's chrome carries the render date, so the bytes differ, the chunks differ and every token in the demo memo breaks. |
+| 2026-09-08 | **A page read has a budget, and the line that closes a short read is a command that runs.** `backdraft read franklin-county p1` printed 36,442 characters and stopped on `49 languages Add topic`, which reads exactly like a truncation and was not one: `--limit` defaulted to *all of them*, so the continuation line this repo is proud of — `[Showing 0-464 of 3115 chars. Continue with: …]` — only ever appeared to a caller who already knew to pass `--limit`. The gate is the one surface an agent is required to use, which makes the default the case that matters, and unbounded meant a single read could spend a large fraction of a context window with nothing in the output to say so — the same silent-cap family as the 2026-08-26 `search` row, from the other end. The choice was a default budget against staying unbounded and *always* closing with the size, and byte-identity settled it: always-closing puts a new line on every read of every small page, and a small page is both the common case and a contract, so `DEFAULT_BUDGET` in `gate/reader.py` is 12,000 characters and 200 sheet rows — roughly three full pages of prose, and about the same spend counted the way a sheet is — while the demo's `read t12-summary p1`, `p1-3` and `assumptions` come back byte for byte what they were. The budget is **soft**, which is the half that keeps the format honest: `_Window.take` was already offering whole chunks and whole rows, so a chunk longer than the whole budget is still shown whole rather than cut under its token, and no token ever stands above text the reader received only part of — now pinned by a test that walks every window at five limits and compares each printed block against the anchor's own snippet, rather than left as a property the code happens to have. What a read shows is unchanged; only how much of it arrives at once. The closing line then had to become the thing it claimed to be, because a budget makes it the normal path rather than the expert one, and it was wrong in three ways nobody had hit: it dropped the caller's `--limit`, so continuing a deliberately narrow read silently widened it to the default; it dropped `--session`, so continuing a read performed under `--session s-deal` minted the rest of the page into the default ledger and the writer's own reading came back `not_shown` — the one failure this mechanism exists to catch, manufactured by the hint telling the truth badly; and it did not quote the selector, so a page read by its name (`Franklin County, Ohio - Wikipedia`) named a command no shell could run, which `searcher._widen_hint` had already learned to avoid. The session flag is keyed on the **rule** that supplied the id and not on the id, which is the opposite of the 2026-08-31 note and right for the opposite reason: that note warns about a shared ledger however you reached it, while this asks whether the id survives the command — an exported `BACKDRAFT_SESSION` does and needs no flag, a typed `--session` does not. `gate.cli` makes that call and the reader owns the wording, the same split `session show` makes for its note, so the gate's library half still does not import the CLI's constants. The default is not echoed back as a `--limit`: a command carrying 12000 forward would go on meaning 12000 after the number moved. Two costs stated. There is no `--all`; the way to read a long page in one call is the `--limit` the closing line already names, which is why that line reports the total and not just the remainder. And `demo/walkthrough.md` quotes the closing line rather than adding the page read that produces it — the walkthrough's `session show` block is the ledger a clean run leaves behind, a new minting command would move those counts, and the numbers cannot be regenerated without re-fetching a source whose bytes carry a render date and would break every token in the demo. |
+| 2026-09-09 | **A source that came back a shell says so on every surface that lists it, not only in the `ingest` that printed the note.** The 2026-08-20 row gave `ingest` a character count and a `note: little text extracted` naming the cause, and both were printed once and then gone — nothing stored the signal and no other surface carried it, so off `backdraft read`'s list a 34,000-character article and a 27-character login wall were the same row. The asymmetry is the point: the skill says to ingest everything up front and then write, a registry travels with the project folder, and the agent choosing evidence is usually not the process that ingested — so the one signal that matters was the one it was least likely to be present for. `gate.thin_mark` now closes the row of a thin source in the gate's document list, in its table-of-contents headline (inside the parenthesis, where everything else describing the source already lives) and in `ls`, reading `little text: 27 chars`; the count is there because 0 characters and 190 are different problems. Reporting, not storage: the count is `sum(len(page.text))` over the current extraction, so nothing was added to the registry and no migration exists to have gone wrong. The owner is `gate/reader.py` beside `unit` rather than `kernel.model` beside `source_name`, which is the other precedent and the wrong one here: the kernel is the spec's vocabulary, `source_name` is a pure rule about a `Document` that bind also needs, and this is a **heuristic** about how a surface describes an extraction — putting a tunable threshold down there would read as normative to the next implementation, and only the gate and the CLI (which imports downward from it, as it already does for `unit`) ever ask. `cli.THIN_SOURCE_CHARS` is gone; `ingest`'s own line, its grouped note and all three list surfaces now derive the same number from `gate.extracted_chars`, which is the whole of the item — a second implementation of the count would have defeated it. Only thin rows are marked, so a registry of ordinary sources gains no column and prints byte for byte what it always did — the gate's count column is padded and `_block`'s existing rstrip takes the padding back off an unmarked row, and `ls` adds a fifth tab-separated field only where there is one to add. That fifth field is the same slot the 2026-08-18 row deliberately emptied when `ls` stopped printing a fetched source's URL beside its invented filename, and the difference is worth stating: that field was a second name for a thing that already had one, this one appears only on the rows nothing else says anything about. Two costs. The heuristic now lives on a persistent surface rather than in a note that scrolled away, so a genuinely tiny real source carries the mark for as long as it is in the registry — accepted on the 2026-08-20 terms, which still hold exactly: display only, exit 0, and no token, anchor, status or byte of the artifact derives from it. And the gate's own fixtures had to grow: the miniature standing in for a 34,000-character Wikipedia article held 112 characters, which would have pinned `little text` into the golden strings that document the list format — a fixture that small was already lying about what it represented, and the marks in `tests/gate` are now on sources that really are shells. |
+| 2026-09-10 | **What a source will be called is a question `ingest` answers with `--dry-run`, not a sibling command.** Three docs tell an agent to pass `--slug` when it ingests a URL, because the default may name a site rather than a page and a slug is permanent the moment a token carries it — and none of them let the agent find out what the default would be. The fix is that the question is cheap: `fetch.filename_for` settles the stem from the address alone (the 2026-08-19 row's own words), so the only way to ask it was to ingest and live with the answer, and getting it wrong costs a re-ingest under a new slug plus a rewritten draft. The shape considered against a flag was a sibling command — `backdraft name <source>` — and the objection to a flag is real: a verb that does not do its verb reads badly, and `ingest` is a writing verb. `--dry-run` answers it, and that is the whole reason it is spelled that way rather than `--plan` or `--name`: the convention is universal and says exactly the thing the objection worries about, that the verb did not run. Two things settled it beyond the wording. The question arises *while composing the ingest command*, so the answer belongs on that command's own help, next to the `--slug` it exists to inform — a separate noun is discoverable only by reading the top-level help of a CLI whose surface is deliberately four verbs. And the prediction has to be the outcome, which means taking `ingest`'s arguments and walking `ingest`'s path: `Registry.naming` matches identity through the same `_find_document` and assigns through the same `_assign_slug`, so it cannot drift from the thing it predicts. Where the two must differ is stated rather than hidden. A file's bytes are read, because identity is the bytes and a copy under another name is the same document keeping its first slug. A URL's are not, so continuity rests on the URL — which is what a re-fetch matches on anyway — and the honest gap, printed as a note rather than guessed at, is the media type: a slug comes from the address and is settled here, a media type comes from the content type the server sends and is not. A source already ingested reports its recorded media type instead, because that one *was* settled by a server. The registry's part of the answer is the part a name alone cannot give: already ingested (and withdrawn, since `ls` denies those exist), the numbered slug a taken name would land on, and that `--slug` will not rename a document that has one. Read-only throughout — no socket, no document, no anchor, no ledger row — and the test that pins it blocks `fetch.fetch` outright rather than checking the output for absence. |
 
 ## Open
 

{backdraft-0.10.0 → backdraft-0.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: backdraft
-Version: 0.10.0
+Version: 0.11.0
 Summary: Drop-in provenance for factual claims: one click from any claim to the evidence behind it.
 Project-URL: Homepage, https://backdraft.dev
 Project-URL: Documentation, https://backdraft.dev/docs.html
@@ -86,13 +86,26 @@ link* — `?oldid=` serves one revision's bytes forever — because a citation i
 a live article reports `drifted` the moment somebody edits it, and the quoted
 sentence would no longer be the sentence there.
 
+`--slug` is there because that URL's default name would be
+`en-wikipedia-org-index` — its last segment is `/index.php`, a handler rather
+than a page, so the host stands in — and a slug is permanent once tokens carry
+it. `backdraft ingest <source> --dry-run` is how to see that before choosing:
+it prints the slug and media type each source would take and stops, fetching
+nothing and writing nothing, and it says when a source is already ingested or
+when the name it wants is taken.
+
 Every ingest line ends with how much text came out, and says which of three
 things happened: a document created, a **new generation** of one whose bytes
 moved — which is when citations into the previous snapshot start reporting
 `drifted` — or `unchanged`, a no-op because re-running would reproduce what is
 already there. When almost no text came out, a note names the likely cause (a
 scan with no text layer, a page behind a login) and what to do; it is a note at
-exit 0, because a thin snapshot is still a real one.
+exit 0, because a thin snapshot is still a real one. That source keeps the mark
+afterwards — `backdraft read`'s document list, its table of contents and
+`backdraft ls` end its row with `little text: N chars` — because a registry
+travels with the project folder and whoever writes against it is usually not
+whoever ingested it. Ordinary sources gain nothing: only the rows worth doubting
+say anything.
 
 **Read.** The page arrives with a citable name over each chunk. This is the whole
 mechanism: what you can cite is exactly what you were shown.
@@ -102,7 +115,7 @@ $ backdraft read t12-summary p1
 # t12-summary p1  (page 1 of 3)
 
 [bd:t12-summary:p1.c1:c2e8]
-Bridgeview Commons, Trailing Twelve Month Summary
+Bridgeview Commons — Trailing Twelve Month Summary
 
 Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
 garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. This summary
@@ -122,6 +135,33 @@ Chunks fall on the document's own paragraphs. A PDF stores glyphs at
 coordinates, not paragraphs, so the extractor rebuilds the breaks from the line
 geometry before the chunker sees the page.
 
+`backdraft read <slug>` on its own is the table of contents: one line per page
+for a source that paginates, and — for a web page, which has none — one line per
+chunk, under the name the page gave itself in its `<title>`. That list is the map
+you read before deciding whether to spend 34,000 characters on the page.
+
+And a page read will not spend all 34,000 of them unasked. One read shows
+12,000 characters by default — 200 rows of a sheet — which every ordinary page
+and small range fits inside, so the output above is what it always was. A page
+that runs past it stops on a chunk boundary and says so:
+
+```console
+$ backdraft read franklin-county p1
+# franklin-county p1  (page 1 of 1)
+
+[bd:franklin-county:p1.c1:d9b5]
+Jump to content
+…
+[Showing 0-11531 of 34031 chars. Continue with: backdraft read franklin-county p1 --offset 11531]
+```
+
+That last line is the whole of the contract: how much you got, how much there
+is, and the command that continues where this one stopped — which is also how
+you read a long page in one call, by passing the total it names as `--limit`.
+A read that prints no such line showed you the whole page. The cut falls between
+chunks and never inside one, so every token you are handed still names text you
+were shown all of.
+
 Search results are citable too, no page read required to get an anchor:
 
 ```console
@@ -331,7 +371,9 @@ Every surface names a fetched source by its page. The filename you never see —
 exists on nobody's disk, so `ingest`, `ls`, `backdraft read`, the artifact and
 the `## References` section `bind --bound` writes all show the URL in its place
 rather than beside it: two names for one thing would let the invented one look
-authoritative.
+authoritative. The artifact goes one further and calls the source by its
+`<title>` where the page declared one, because that is the only name in the
+record the source chose for itself.
 
 The origin travels into the artifact too. A claim citing a fetched page shows
 the URL as a link on its receipt, with the date the bytes were taken. That

{backdraft-0.10.0 → backdraft-0.11.0}/README.md

@@ -59,13 +59,26 @@ link* — `?oldid=` serves one revision's bytes forever — because a citation i
 a live article reports `drifted` the moment somebody edits it, and the quoted
 sentence would no longer be the sentence there.
 
+`--slug` is there because that URL's default name would be
+`en-wikipedia-org-index` — its last segment is `/index.php`, a handler rather
+than a page, so the host stands in — and a slug is permanent once tokens carry
+it. `backdraft ingest <source> --dry-run` is how to see that before choosing:
+it prints the slug and media type each source would take and stops, fetching
+nothing and writing nothing, and it says when a source is already ingested or
+when the name it wants is taken.
+
 Every ingest line ends with how much text came out, and says which of three
 things happened: a document created, a **new generation** of one whose bytes
 moved — which is when citations into the previous snapshot start reporting
 `drifted` — or `unchanged`, a no-op because re-running would reproduce what is
 already there. When almost no text came out, a note names the likely cause (a
 scan with no text layer, a page behind a login) and what to do; it is a note at
-exit 0, because a thin snapshot is still a real one.
+exit 0, because a thin snapshot is still a real one. That source keeps the mark
+afterwards — `backdraft read`'s document list, its table of contents and
+`backdraft ls` end its row with `little text: N chars` — because a registry
+travels with the project folder and whoever writes against it is usually not
+whoever ingested it. Ordinary sources gain nothing: only the rows worth doubting
+say anything.
 
 **Read.** The page arrives with a citable name over each chunk. This is the whole
 mechanism: what you can cite is exactly what you were shown.
@@ -75,7 +88,7 @@ $ backdraft read t12-summary p1
 # t12-summary p1  (page 1 of 3)
 
 [bd:t12-summary:p1.c1:c2e8]
-Bridgeview Commons, Trailing Twelve Month Summary
+Bridgeview Commons — Trailing Twelve Month Summary
 
 Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
 garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. This summary
@@ -95,6 +108,33 @@ Chunks fall on the document's own paragraphs. A PDF stores glyphs at
 coordinates, not paragraphs, so the extractor rebuilds the breaks from the line
 geometry before the chunker sees the page.
 
+`backdraft read <slug>` on its own is the table of contents: one line per page
+for a source that paginates, and — for a web page, which has none — one line per
+chunk, under the name the page gave itself in its `<title>`. That list is the map
+you read before deciding whether to spend 34,000 characters on the page.
+
+And a page read will not spend all 34,000 of them unasked. One read shows
+12,000 characters by default — 200 rows of a sheet — which every ordinary page
+and small range fits inside, so the output above is what it always was. A page
+that runs past it stops on a chunk boundary and says so:
+
+```console
+$ backdraft read franklin-county p1
+# franklin-county p1  (page 1 of 1)
+
+[bd:franklin-county:p1.c1:d9b5]
+Jump to content
+…
+[Showing 0-11531 of 34031 chars. Continue with: backdraft read franklin-county p1 --offset 11531]
+```
+
+That last line is the whole of the contract: how much you got, how much there
+is, and the command that continues where this one stopped — which is also how
+you read a long page in one call, by passing the total it names as `--limit`.
+A read that prints no such line showed you the whole page. The cut falls between
+chunks and never inside one, so every token you are handed still names text you
+were shown all of.
+
 Search results are citable too, no page read required to get an anchor:
 
 ```console
@@ -304,7 +344,9 @@ Every surface names a fetched source by its page. The filename you never see —
 exists on nobody's disk, so `ingest`, `ls`, `backdraft read`, the artifact and
 the `## References` section `bind --bound` writes all show the URL in its place
 rather than beside it: two names for one thing would let the invented one look
-authoritative.
+authoritative. The artifact goes one further and calls the source by its
+`<title>` where the page declared one, because that is the only name in the
+record the source chose for itself.
 
 The origin travels into the artifact too. A claim citing a fetched page shows
 the URL as a link on its receipt, with the date the bytes were taken. That

{backdraft-0.10.0 → backdraft-0.11.0}/ROADMAP.md

@@ -32,157 +32,7 @@ best evidence available for what those five should be.
 
 ## Now
 
-### 1. A web page has a name, and `read` shows it the navigation menu instead
-
-**Intent.** `backdraft read franklin-county` — the demo's own web source —
-prints `p1  Franklin County, Ohio - Wikipedia  Jump to content Main menu Main
-menu move to sidebar hide Navigation - Main page - Contents - Current events -
-Random art...`. The table-of-contents line is the one surface that tells an
-agent what is on a page before it spends context reading it, and for every web
-source it is site chrome. The page's real name is sitting in the extracted text
-and in the markup's `<title>`, unused: a fetched page ingested without `--slug`
-also titles itself `En Wikipedia Org Index` in the artifact's source list,
-because `render/html/text.source_title` falls back to the slug and title-cases
-it. Two surfaces need one fact nothing captures. Separately, an HTML source is
-a single page, so its table of contents is that one line and does no
-navigating at all — for a 34,000-character article the citable unit is the
-chunk and nothing lists the chunks.
-
-**Shape.** Capture the fact once, at extraction: `extract/html.py` already
-parses the markup, so the document's `<title>` becomes page or document meta
-there, and everything downstream reads it instead of guessing. Then
-`gate/reader.py`'s TOC preview (`TOC_PREVIEW_CHARS`, `_preview`) prefers it,
-and `render/html/text.source_title` prefers it over the slug fallback for a
-document that has one. For the single-page case, `render_toc` lists the page's
-chunks with their opening words, so a one-page source's table of contents is
-one. **This is display only and must stay that way**: no chunk boundary moves,
-no anchor moves, no token changes, and the slug is not derived from the title —
-that is the admissibility argument the 2026-08-20 thin-source note made, and
-this row must make it again, because the neighbouring rule ("no boilerplate
-stripping: a heuristic that changes its mind moves anchors") is about
-extraction and is not being touched. A page with no `<title>`, and every
-non-HTML source, keeps exactly what it prints today.
-
-**Acceptance.** In `demo/`, `backdraft read franklin-county` names the county
-rather than "Jump to content", and lists the page's chunks. `backdraft read
-t12-summary` and `backdraft read underwriting-model` are byte-identical to
-today's — pin both. `backdraft bind memo.md --session s-bridgeview --check
-value-trace,overlap` still reports 17 resolved and the same one unresolved,
-with the same tokens: mint them before and after and diff. An HTML fixture with
-no `<title>` falls back to what it prints now. `demo/walkthrough.md`'s read
-blocks and `README.md`'s show the real output. DESIGN row.
-
-**Size.** Three days.
-
-### 2. A page read has no budget and no closing line
-
-**Intent.** `backdraft read franklin-county p1` prints 36,442 characters and
-stops, with nothing to say how much that was or whether it was all of it — it
-ends mid-navigation on `49 languages Add topic`, which reads exactly like a
-truncation and is not one. `--limit` defaults to "all of them", so the
-continuation hint this repo is proud of — `[Showing 0-464 of 3115 chars.
-Continue with: …]` — only ever appears to a caller who already knew to pass
-`--limit`. The gate is the one surface an agent is *required* to use, which
-makes the default the case that matters: one `read` can spend a large fraction
-of a context window with no warning, and the agent cannot tell a complete page
-from a cut one. `search` caps at 20 by default; `read` caps at nothing. This is
-the same silent-cap family as the queued `search` item, from the other end — an
-uncapped read that looks capped, rather than a capped search that looks
-complete.
-
-**Shape.** `gate/reader.py`'s page rendering and `gate/cli.py`'s `--limit`. The
-decision to make and write down is a default budget versus staying unbounded
-and always closing with the size and the continuation line; prefer whichever
-keeps a small page byte-identical, since most pages are small and their output
-is a contract. Whatever lands, a read that shows part of a page must close by
-naming the exact command that continues it, as the existing hint does. One
-constraint to state and test rather than discover: the window slices by
-characters and a chunk token stands above its chunk, so a cut must not leave a
-token over a chunk the reader only half received — either the window respects
-chunk boundaries or the closing line says the last chunk is partial. Do not add
-an offset story to `search` here; that is the queued item's ground.
-
-**Acceptance.** `backdraft read t12-summary p1` in `demo/` is byte-identical to
-what it prints today — pin it. `backdraft read franklin-county p1` closes by
-saying how much of the page it showed and how to see the rest, and running the
-command it names continues where it stopped. No read output ever shows a token
-above a chunk it did not finish. `demo/walkthrough.md`, `README.md`,
-`site/llms.txt` and `skills/backdraft/SKILL.md` show the real output and tell
-the agent to continue rather than assume it saw the page. DESIGN row.
-
-**Size.** Two days.
-
-### 3. The thin-source signal exists only in the ingest that printed it
-
-**Intent.** 2026-08-20 gave `ingest` a character count and a `note: little text
-extracted` naming the likely cause — the signal that a source is a shell and
-must not be cited. Both are printed once and then gone: nothing stores the
-count, and `ls`, the gate's document list and a document's table of contents
-say nothing about it. The agent that writes is usually not the process that
-ingested — the skill says to ingest everything up front and then read, and a
-registry travels with the project folder — so the one signal that matters is
-the one an agent is least likely to be present for. Off `backdraft read`'s
-list, a 34,000-character article and a 60-character login wall are the same
-row.
-
-**Shape.** The count is derivable from the current extraction
-(`sum(len(page.text))`), so this is a reporting change rather than a storage
-one unless measurement says otherwise. One owner, shared between `cli.py`'s
-ingest line and `gate/reader.py`'s list and headline, the way `unit` and
-`source_name` already are — the point of this item is that three surfaces give
-one answer, so a second implementation of the count would defeat it. Display
-only, as the ingest note is: no token, anchor or status derives from it. Hold
-the byte-identity rule the source-naming work established — a registry of
-ordinary sources must not gain a column for this, so mark only the sources that
-are thin rather than annotating every row.
-
-**Acceptance.** Ingest a login-wall HTML file beside a real source: `backdraft
-read` and `backdraft ls` mark the thin one and say nothing new about the other,
-and `backdraft read <thin-slug>` says it in the headline too. A registry with
-no thin source prints what it prints today, byte for byte — pinned, including
-`demo/`'s three sources. `THIN_SOURCE_CHARS` is read from one place.
-`skills/backdraft/SKILL.md` and `site/llms.txt` tell the agent that the
-document list is where it learns this, not only the ingest it may not have run.
-
-**Size.** Two days.
-
-### 4. What a URL will be called, before the answer is permanent
-
-**Intent.** Three docs now tell an agent to pass `--slug` when it ingests a URL,
-because a slug is permanent once tokens carry it and the default may name a
-site rather than a page. None of them lets the agent find out what the default
-would be. `fetch.filename_for` settles the answer from the address alone,
-before any bytes exist — that is the 2026-08-19 row's own words — so the
-question is cheap and answerable, and today the only way to ask it is to ingest
-and live with the result. An agent that guesses wrong has written a name into
-every token of the document, and the fix is re-ingesting under a new slug and
-rewriting the draft.
-
-**Shape.** Small and read-only. The natural home is beside the thing it
-predicts: a flag on `ingest` that resolves each source to the slug and media
-type it would take and prints them without fetching, writing or minting
-anything, or a sibling command if a flag on a writing verb reads badly — pick
-one and say why in the DESIGN row, since "a verb that does not do its verb" is
-the objection to answer. For a URL the answer comes from `fetch.filename_for`
-plus `registry.slug_for` and needs no network; the media type is the honest
-gap, because only the served content type settles it, and the output must say
-so rather than guessing. For a file, both are already knowable. Existing
-collisions matter: a slug already taken must be reported as taken, since that is
-what turns a predicted `index` into a real `index-2`.
-
-**Acceptance.** Against `demo/`, asking about
-`https://en.wikipedia.org/w/index.php?title=Franklin_County,_Ohio&oldid=1367935775`
-prints `en-wikipedia-org-index` and does not create a document, open a socket,
-or touch the ledger — check the registry's document count and `ls` before and
-after. Asking about a path prints the slug the file would take. Asking about a
-source whose slug is already in the registry says so. `README.md`,
-`site/docs.html`, `site/llms.txt` and `skills/backdraft/SKILL.md` name it where
-they currently say "pass `--slug`", so the advice comes with the way to check
-it.
-
-**Size.** One day.
-
-### 5. A calling agent parses prose to find out what happened
+### 1. A calling agent parses prose to find out what happened
 
 **Intent.** `bind` and `verify` are the two commands whose *output* is the
 product — the exit code says clean or not, and everything actionable is in the
@@ -223,7 +73,7 @@ relay the human report to the user.
 
 **Size.** Two to three days.
 
-### 6. An artifact you were sent cannot be checked against a registry you have
+### 2. An artifact you were sent cannot be checked against a registry you have
 
 **Intent.** `verify`'s second tier runs only where a `.backdraft/` is
 discoverable from cwd, and the reason is good: an artifact is a file people
@@ -259,7 +109,7 @@ runs only in the project it was bound in.
 
 **Size.** One day.
 
-### 7. A claim that straddles a chunk boundary gets one token instead of two
+### 3. A claim that straddles a chunk boundary gets one token instead of two
 
 **Intent.** `skills/backdraft/SKILL.md` tells the writing agent that "a claim
 that spans two chunks needs both tokens, not the nearest one" — a correct
@@ -294,7 +144,7 @@ one" with the surface that now says which both are.
 
 **Size.** Two to three days.
 
-### 8. A re-ingested source strands citations one at a time
+### 4. A re-ingested source strands citations one at a time
 
 **Intent.** This is DESIGN.md's oldest Open line — "re-bind/orphan pass on
 re-ingest of changed docs (chunk ordinal drift)" — and the week that taught
@@ -336,7 +186,7 @@ extraction and ledger counts are identical before and after.
 
 **Size.** Three days.
 
-### 9. What this install can do, said before a verb needs it
+### 5. What this install can do, said before a verb needs it
 
 **Intent.** backdraft degrades rather than fails, which is right, and the price
 is that its capabilities are discovered one at a time at the moment each is
@@ -373,6 +223,208 @@ run in an unfamiliar environment.
 
 **Size.** Two days.
 
+### 6. `bind` never says which ledger it judged `not_shown` against
+
+**Intent.** `bind`'s report names the mode, the claim and citation counts, every
+status, every check that ran and every failure — everything except the one input
+that decides `not_shown`. The session is resolved silently by
+`cli_context.resolve_session` (flag, then `BACKDRAFT_SESSION`, then the default),
+so `backdraft bind memo.md` in `demo/` with nothing exported reports `not_shown:
+1` off the shared default ledger and says nothing about it. That is the weakest
+form the check has: the 2026-08-31 row established that the default session
+accumulates across every run in a registry, so a `resolved` there can mean "some
+earlier run read that page" rather than "this writer did". The row fixed it at
+`session show`, the command that reports the ledger, and left `bind`, the
+command that acts on it. The recipient inherits the gap: the record carries
+`session_id` and `spec/artifact.md` calls it "the ledger session, when the run
+had one", so a reader holding a record that says `"session_id": "default"` has
+the fact and no way to know it weakens every `not_shown` and every `resolved`
+beside it.
+
+**Shape.** `bind/cli.py`'s report header, and `spec/artifact.md`. Name the
+session on the summary line, and where it is the default one, close with
+`gate.reader.DEFAULT_SESSION_NOTE` — imported, not re-written, which is the whole
+point of the item: two wordings for one cost is how the `session show` note and
+this one drift apart. Keyed on the resolved id rather than on which rule supplied
+it, exactly as `session show` keys it, so an explicit `--session default` is
+named too. Then say it in the format: `spec/artifact.md`'s `session_id` row and
+the legend's `not_shown` line must tell a reader what a default session means for
+the status, since the record travels and the CLI does not. That is a legend
+change, so goldens and the demo regenerate — the 2026-08-24 trade, taken twice
+before. Not the ground of "A calling agent parses prose to find out what
+happened": that item makes the report machine-readable, this one adds a fact the
+report does not currently carry in any form.
+
+**Acceptance.** In `demo/`, `backdraft bind memo.md` with no session and no
+`BACKDRAFT_SESSION` names `default` and prints the note; `--session s-bridgeview`
+names that id and prints no note; `--session default` prints the note. Below the
+header the two runs are byte-identical. A test asserts the printed note is
+`gate.reader.DEFAULT_SESSION_NOTE` by constant rather than by substring, so a
+second copy cannot pass. `backdraft verify` on a record bound in the default
+session says so from the record alone, with no registry present. Golden sidecars
+and `demo/memo.backdraft.html`/`site/demo.html` regenerate together and stay
+byte-identical to each other. `README.md`, `site/llms.txt`,
+`skills/backdraft/SKILL.md` and `skills/backdraft-artifact/SKILL.md` follow.
+DESIGN row.
+
+**Size.** Two days.
+
+### 7. A withdrawn source is invisible, including to the person looking for it
+
+**Intent.** `forget` withdraws a source from every surface that offers one, which
+is right, and the result is that nothing lists what was withdrawn. `ls` says `no
+documents ingested` in a registry holding two withdrawn documents — true about
+sources, false about the registry, and the wording admits nothing. The three
+places a withdrawn document still appears are `session show`'s mark, the JSON
+export, and the unknown-slug error added 2026-09-04; none of them is where
+somebody looks. So an agent handed a project, meeting a citation that binds
+`unresolved — withdrawn from the registry on ...`, can learn that one source went
+and cannot learn what else did. The documented undo makes it worse: re-ingesting
+the file is the way back, and the path to re-ingest lives on the document, which
+is exactly what no surface will show once it is out of the list.
+
+**Shape.** `cli.list_documents`, off `documents(include_withdrawn=True)`, which
+already exists. Reuse `registry.withdrawn_reason` for the date and
+`gate.WITHDRAWN_HINT` for the way back — both are single owners and this must not
+mint a fourth wording for a withdrawal. The decision to make and write down is
+whether this is a flag on `ls` or a block printed under it when the registry has
+any: the objection to answer is that `ls` is the readable set, so anything that
+widens it must not make a withdrawn source look available — mark every row and
+never sort a withdrawn one among the live ones. The gate's own document list is
+out of scope and must not change: the gate serves sources, and a withdrawn source
+is not one. Hold the byte-identity rule the source-naming work established — a
+registry with nothing withdrawn prints exactly what it prints today, everywhere.
+
+**Acceptance.** In a registry with two documents, one withdrawn, `ls`'s default
+output is byte-identical to the same registry with only the live document — pin
+it. The withdrawn view names the slug, the date it went and the exact `backdraft
+ingest <path>` that brings it back, asserted against `WITHDRAWN_HINT` by
+constant. A registry where everything has been withdrawn no longer reports itself
+empty without qualification. `backdraft read` and the gate's list are unchanged,
+pinned. `README.md`'s "Forgetting a source", `site/docs.html`, `site/llms.txt`
+and `skills/backdraft/SKILL.md` say where to look. DESIGN row.
+
+**Size.** One day.
+
+### 8. `--slug` is dropped without a word when the document is already there
+
+**Intent.** `Registry.ingest`'s docstring says "`slug` is honoured only when the
+document is new — a slug is stable once assigned", which is the right rule and is
+stated nowhere a caller reads. `backdraft ingest report.pdf --slug q4-report`
+against a `report.pdf` already in the registry prints the old slug, exits 0, and
+never mentions that the flag did nothing; the same is true of a URL re-fetch,
+where the fallback slug the docs warn about is exactly what the caller was trying
+to replace. This is the one input `ingest` accepts and does not report on, in the
+week it learned to report everything else it did — `unchanged`, `new generation`,
+`restored`, thin extractions, and every source that never landed. It is not
+`ingest --dry-run`, which predicts what a source would be called *before* the
+ingest and already says there that `--slug` would not rename a document that has
+one; this is the ingest itself saying what it did with the flag it was handed.
+
+**Shape.** `cli.ingest`, in the shape `_outcome_note` and the grouped notes
+already use — a mark on the source's line or one grouped note, matching how
+`restored` was added rather than inventing a third reporting shape. No schema
+change and no registry change: the CLI holds the requested slug and the returned
+`Ingested` carries the assigned one, so the comparison is a string equality in
+the layer that already owns the reporting. Say the next step, which fd8d00f made
+the rule for this command: the slug is permanent because every token carries it,
+so the honest options are keep it, or `forget` the document and ingest the file
+under the new slug as a *new* document — which strands every token minted under
+the old one, and the note must name that cost rather than recommending it. Exit 0
+throughout: getting back the document you already had is not a failure.
+
+**Acceptance.** Ingest a file, then ingest it again with `--slug something-else`:
+the run says the slug was not applied, names the slug the document actually has,
+names what re-slugging would cost, and exits 0. Same for a URL re-fetched with a
+`--slug` it did not have the first time. A `--slug` on a genuinely new document,
+and every ingest with no `--slug` at all, is byte-identical to today — pin both,
+including `demo/`'s three sources. `README.md`, `site/llms.txt` and
+`skills/backdraft/SKILL.md` say a slug is permanent where they currently say to
+pass `--slug`. DESIGN row.
+
+**Size.** One day.
+
+### 9. `verify` cannot re-check the one status only the ledger can settle
+
+**Intent.** `verify`'s source tier re-resolves every token and reports the
+statuses "as `bind` would", with one gap the code names out loud: `not_shown`
+cannot appear, because it is a fact about a ledger session and verify opens none
+(`render/cli.py`'s `_against_sources`). So a record that says a claim cited
+something its author never read comes back `resolved` from the tier that is
+supposed to be the stronger check, and the strongest claim this product makes —
+that the set of citable tokens is exactly the set the gate emitted — is the one
+claim a recipient holding the registry cannot re-check. That recipient is the
+`backdraft-artifact` skill's whole case: they have the artifact, they have the
+project, and the check they most want is the one that says the author read it.
+
+**Shape.** `render/cli.py`'s source tier plus one registry read. The record
+already carries `session_id`, and `Registry.was_shown(session_id, token)` already
+answers the question one token at a time — that is the call `bind` makes, so a
+status printed here stays the status a re-bind would print. The standing
+objection is in the docstring and must be answered rather than overridden:
+"read-only, it opens no session and mints nothing, which is what separates it
+from `backdraft show`". Reading a ledger is not minting into one, and the session
+read is the one the record names rather than one verify chose — so
+`ensure_session` must not be called, a session id absent from the registry is
+reported as absent rather than created, and a `session_id` of `null` is reported
+as a run that had none. Nothing about `show`'s minting changes. The `sources:`
+line gains the status; the existing counts must keep meaning what they mean.
+
+**Acceptance.** In `demo/`, bind a memo under a session that was never shown one
+of its citations, render, then `backdraft verify` in the project: the source tier
+reports that citation `not_shown` and says which session it read. Verified from
+outside the project, the tier does not run and the output is byte-identical to
+today. A record whose `session_id` names no session in this registry says so and
+does not create it — assert the registry's session count before and after. A
+record with `session_id: null` says the run had no session. Every existing verify
+output where the ledger agrees with the record is byte-identical, pinned,
+including `demo/`'s. `spec/artifact.md` § Checking an artifact,
+`skills/backdraft-artifact/SKILL.md`, `site/llms.txt` and `README.md` follow.
+DESIGN row.
+
+**Size.** Two to three days.
+
+### 10. `bind` and `verify` name the failure and not the move
+
+**Intent.** 2026-09-01 made `ingest`'s failures say what to do as well as what
+went wrong, on the argument that a calling agent reads the reason and acts on it.
+`bind` and `verify` are the two commands whose output *is* the product, and
+neither says a next step anywhere: `! not_shown: <token> — <claim> @<offset>` is
+a complete diagnosis and a blank instruction. The move differs sharply per status
+and is not guessable — `not_shown` clears by showing the token, `unresolved`
+needs a search and a new token or an uncited claim, `drifted` needs both snippets
+read and a judgement, `malformed` is an href to fix, and a `receipt:` finding
+means the artifact was edited rather than the sources moved. Today that mapping
+lives only in `skills/backdraft/SKILL.md`, which means it works for an agent
+running the skill and for nobody else — a hook, a CI job, a person, or any agent
+that reached the command another way.
+
+**Shape.** A closing block on both commands, one line per status actually
+present, naming the exact command that addresses it — the shape `ingest`'s
+grouped notes already use, and never a hint appended to each of eighteen line
+items. One owner for the mapping, imported by `bind/cli.py` and
+`render/cli.py` rather than written twice; the status set is the artifact
+format's and closed, so the mapping is total and a test asserts every
+`CitationStatus` has a line. `verify` adds the `receipt` case, which is not a
+citation status and is the finding that means something categorically different.
+Display only: no exit code moves, no status moves, no record field is added — a
+run that came out clean prints exactly what it prints today. Not "A calling agent
+parses prose to find out what happened", which gives the same two commands a
+machine-readable payload; that item serves a caller
+that parses, this one serves the caller that reads, and both are wanted because
+the human report is what gets relayed to the user.
+
+**Acceptance.** In `demo/`, `backdraft bind memo.md --session s-bridgeview
+--check value-trace,overlap` closes by naming `backdraft search` for its one
+`unresolved`, and says nothing about the statuses it did not produce. A bind that
+comes out clean is byte-identical to today, as is a clean `verify` — pin both. A
+test enumerates `CitationStatus` and asserts each has a line and a command that
+`backdraft --help` lists. `verify` on an artifact with one edited snippet says
+the file was changed rather than the sources. `demo/walkthrough.md`'s bind and
+verify blocks show the real output. DESIGN row.
+
+**Size.** Two days.
+
 ## Parked
 
 Deliberately not queued, each with the reason, so picking one up starts from