backdraft 0.10.0__tar.gz → 0.12.0__tar.gz

{backdraft-0.10.0 → backdraft-0.12.0}/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "backdraft",
   "displayName": "Backdraft",
-  "version": "0.10.0",
+  "version": "0.12.0",
   "description": "Drop-in provenance for factual claims: one click from any claim to the evidence behind it. Ships the backdraft writing, backfill, and artifact-reading skills; the CLI installs separately (uv tool install backdraft).",
   "author": {
     "name": "Spencer Braun",

{backdraft-0.10.0 → backdraft-0.12.0}/DESIGN.md

@@ -155,13 +155,20 @@ Self-contained HTML (plus the machine-readable sidecar): **quotes + receipts + e
 | 2026-09-01 | **An unreadable source says what to do about it, and an empty one is a failure.** `ingest` finishing its list (2026-08-13) turned the reason string into something a calling agent actually reads, and the reason was Python: `cannot read adir: [Errno 21] Is a directory: 'adir'` named the source three times, leaked an errno nobody should have to know, and said nothing about a fix that differs completely between the causes and is obvious in each one. The read now goes through `store._read_source`, which maps the four causes worth distinguishing — a directory (name the files inside it, or pass a glob), a path with nothing at it (check the spelling), a file the process may not open (fix the permissions, or ingest a copy), and zero bytes — and `fetch.py`'s messages follow the same rule for the URL half. Anything outside those four keeps the operating system's own `strerror` rather than being guessed at: an unknown cause reported plainly beats a wrong next step, and that is the one place an errno-free message loses information, which is the trade taken. The reason never names the source, because every caller already leads with it — which means the exception read on its own, by a library caller rather than through `ingest`'s `!` line, is now less self-contained than it was; the CLI is the only caller and the report is the surface that matters. Empty is the substantive change rather than a rewording: zero bytes read cleanly, so an empty file used to become a document — listed by `ls`, offered by `read`, competing in `search`, citable in name only — and a fetched page whose server sent an empty body did the same, which is the shape a JavaScript-rendered page takes here. That is not the thin-source case the 2026-08-20 row admitted a heuristic for: thin is a judgement about how much text came out of real bytes and stays display-only at exit 0, while zero bytes is a fact and there is no snapshot to make. Staging a fetched page also became a per-source failure rather than a traceback, so a full temporary directory no longer ends the run the rest of the list was promised. |
 | 2026-09-02 | **A registry can forget a source, and forgetting removes nothing.** Ingest was one-way, and now that `ingest` finishes its list the unattended folder run is the normal case — so a scratch copy, the same report under two names, or a file nobody meant to include lands and stays: listed by `read`, competing in `search`, and citable by an agent choosing evidence without anything looking wrong. The only recovery was deleting `.backdraft/`, which throws away every other document and the whole ledger. `backdraft forget <slug>` **withdraws** rather than deletes, and the whole design is in that word: the document, its generations, its anchors and its receipts all stay, so `resolve` still finds every anchor and a token already written into somebody's draft or artifact still shows its snippet under `show`. What a withdrawal changes is that the registry stops *offering* the source — `documents()` excludes it, and with it `ls`, the gate's list, its table of contents, page reads and `search`, whose filter sits in the shared `_match_sql` so the pre-limit `total` a caller is offered to widen to drops too. A deletion would have stranded those citations with nothing to say about them, which is the failure this system exists to prevent. The status is the second half of the decision and the temptation was a sixth one: `citation_for` reports a withdrawn source's token `unresolved` carrying `withdrawn_reason` as its `error`, because the closed status set is the artifact format's and a new value would break every reader written against it, while `resolved` would let a document keep citing a source the registry no longer offers and report nothing. That widens `error` from `malformed`'s explanation to any status the status alone does not explain — an optional key appearing where it applies, not a new format version — and it made the legend's own `unresolved` line wrong, since a withdrawn citation *does* carry an anchor; corrected in `LEGEND` and `spec/artifact.md` at the cost of regenerated goldens and a re-rendered demo, the 2026-08-24 trade. Storage is a `withdrawals` table rather than a column on `documents`, which is the 2026-08-05 reason `document_meta` is one: an existing registry needs no migration, and this repo has no migration machinery to introduce for one flag. Ingesting the source again is the undo and deletes the row, bringing back the *same* document under the same slug — `_find_document` matches a withdrawn document like any other, which is exactly what a delete could not do, since a second slug would leave every token carrying the first one stranded. That undo is silent-by-default territory, so it is not: `Ingested.restored` is a fact beside `outcome` rather than a fourth value of it (a forgotten file that was also edited is both restored and a new generation, and one field could only say one), the line says `restored`, and a grouped note names what came back. Two costs stated rather than discovered. A registry only ever grows: nothing is reclaimed, and a corpus you must be able to shrink is a new registry, not a `forget`. And the rule is drawn at *offering a source to read*, so `snapshot-pages` still backfills page images for a withdrawn document — it serves no text and cannot reach a writer's context, and widening the rule to every command that touches a document would make it a rule nobody can state. `forget` asks before withdrawing and, where nothing can answer, names `--yes` instead of hanging on a prompt or aborting without a reason, since the usual caller here is an agent with no terminal. |
 | 2026-09-03 | **The rule moved down to the type it is about, and the last surface naming a source wrongly stopped.** The 2026-08-18 row gave `gate.source_name` one owner and accepted one exception on a dependency argument, and the argument was sound about imports and wrong about where the rule belonged. Bind's `_doc_name` returned `Document.filename`, and it feeds *both* halves of the markdown projection — the generated `## References` section and backfill's proposed anchors — so `bind --bound` called the demo's Wikipedia article `index.html` while `render --to footnotes` named its URL and the artifact named it *Franklin County*. Three of backdraft's own outputs, two answers, and the one that disagreed was the one a reader gets without a browser: the bound markdown is the form that travels into a pull request or an email, where the HTML artifact cannot follow. The fix is to stop working around the dependency rule and read it properly. `source_name` is a pure function of a `Document`, `Document` is defined in `kernel/model.py`, and gate, bind and the CLI all import that module *downward* — so the kernel was the owner the rule was looking for all along, and it stays stdlib-pure there. SPEC § Dependency rule now says so in general, because the reasoning that produced the exception ("we cannot share it sideways, so we duplicate or we live with it") will recur for the next shared rule and the escape hatch should not have to be rediscovered. `gate.source_name` is **not** left as a redirect: callers import `kernel.model.source_name` and the gate's re-export is gone, since two import paths for one function is the same shape as two names for one source — the thing the 2026-08-06 row rejected — and the gate's stated export style is names it owns, not names it passes through. The cost is that anything importing `from backdraft.gate import source_name` breaks; it is one internal caller and a library surface nobody outside this repo is known to use, and leaving both paths alive is what makes an owner stop being one. `_origin` moved with it as public `source_origin`, because the gate's document list needs the *question* ("is this name a URL, and therefore must not size the name column?") and not just the answer, and two readings of one `meta` key drift apart the moment the key does. Display only, unchanged: the slug, the token and everything a citation resolves through are untouched, and a References entry for a file source is byte-identical to what it printed before — pinned as a whole line, the way `ls`'s file rows are. The render side was deliberately not touched and was never wrong: `render/html/text.source_title` and `render/footnotes._origin` read a `url` off the *artifact's* `docs` dict, which is the artifact format's shape rather than a `Document`, and they already prefer the slug and the URL over the invented filename — they look like misses and are not. One thing this closes that was already promised in prose: `demo/walkthrough.md` told readers that a staging filename appears at "no surface that names a source", which was false for as long as this row's exception stood. |
+| 2026-09-07 | **A one-page source's table of contents is its chunks, and a page that named itself is called by that name.** `backdraft read franklin-county` opened with `Jump to content Main menu move to sidebar hide Navigation`, which is the site's chrome and not the page, and it was the *only* thing the table of contents said about a 34,000-character article — because a web page has no pagination, so the page list is one row and does no navigating at all. Two halves, one fact. The listing half: a document that is a single page of prose now lists that page's chunks underneath it, `p1.c1  opening words` per row, read off `anchors_for_page` and never re-chunked (rule 1 of the reader), so every locator printed is one a token already carries. The page's own preview goes with it, because it *was* the first chunk's opening words said twice, and a page row left carrying nothing but `p1` is dropped, since every chunk row below already begins with it. Scoped to one page rather than to HTML: a one-page PDF has the same problem, and a single-sheet workbook mints cell anchors and no chunk anchors, so it lists nothing and prints what it always did — as do all three of the demo's sources except the article, pinned byte for byte. The naming half: `extract/html.py` already parsed the `<title>` and threw away the fact that it *found* one, so `Page.meta` now carries `title` and carries it only where the markup declared it. `Page.name` could not answer the question, and that is the whole reason for a second field holding the same string: `name` falls back to the file stem, and a fetched page's stem is the one `fetch.filename_for` invented — so a page called `index` and a page called *Franklin County, Ohio - Wikipedia* were indistinguishable to anything downstream. Bind copies it into the evidence entry (`spec/artifact.md` § evidence, optional, absent for everything else) and `render/html/text.source_title` prefers it, which is what stops a URL ingested without `--slug` from titling itself *En Wikipedia Org Index* in the artifact's source list. Only a single-page source answers, because page 1's title is the document's title exactly when page 1 is the document; a section heading must not name a whole report. Display only, and the 2026-08-20 thin-source argument is the one that admits it: no chunk boundary moves, no anchor moves, no token changes, the slug is still not derived from the title, and the demo's 18 tokens are identical before and after. That is also why this is not the boilerplate stripping the extractor refuses — a heuristic that changes its mind moves anchors; this reorders a preview. One deviation from the queued shape, stated rather than buried: the item asked for `_preview` to prefer the title too, and it must not, because the gate already had the title in `Page.name` and every page that would reach that branch has its chunks listed instead — the branch is unreachable, and unreachable code is what Friday deletes. One cost taken knowingly: the demo's registry predates the page meta and re-ingesting the same bytes is a no-op by design, so the demo artifact still names that source from its slug and will pick up the title the next time the registry is rebuilt. Re-fetching to force it is the one thing that must not happen — Wikipedia's chrome carries the render date, so the bytes differ, the chunks differ and every token in the demo memo breaks. |
+| 2026-09-08 | **A page read has a budget, and the line that closes a short read is a command that runs.** `backdraft read franklin-county p1` printed 36,442 characters and stopped on `49 languages Add topic`, which reads exactly like a truncation and was not one: `--limit` defaulted to *all of them*, so the continuation line this repo is proud of — `[Showing 0-464 of 3115 chars. Continue with: …]` — only ever appeared to a caller who already knew to pass `--limit`. The gate is the one surface an agent is required to use, which makes the default the case that matters, and unbounded meant a single read could spend a large fraction of a context window with nothing in the output to say so — the same silent-cap family as the 2026-08-26 `search` row, from the other end. The choice was a default budget against staying unbounded and *always* closing with the size, and byte-identity settled it: always-closing puts a new line on every read of every small page, and a small page is both the common case and a contract, so `DEFAULT_BUDGET` in `gate/reader.py` is 12,000 characters and 200 sheet rows — roughly three full pages of prose, and about the same spend counted the way a sheet is — while the demo's `read t12-summary p1`, `p1-3` and `assumptions` come back byte for byte what they were. The budget is **soft**, which is the half that keeps the format honest: `_Window.take` was already offering whole chunks and whole rows, so a chunk longer than the whole budget is still shown whole rather than cut under its token, and no token ever stands above text the reader received only part of — now pinned by a test that walks every window at five limits and compares each printed block against the anchor's own snippet, rather than left as a property the code happens to have. What a read shows is unchanged; only how much of it arrives at once. The closing line then had to become the thing it claimed to be, because a budget makes it the normal path rather than the expert one, and it was wrong in three ways nobody had hit: it dropped the caller's `--limit`, so continuing a deliberately narrow read silently widened it to the default; it dropped `--session`, so continuing a read performed under `--session s-deal` minted the rest of the page into the default ledger and the writer's own reading came back `not_shown` — the one failure this mechanism exists to catch, manufactured by the hint telling the truth badly; and it did not quote the selector, so a page read by its name (`Franklin County, Ohio - Wikipedia`) named a command no shell could run, which `searcher._widen_hint` had already learned to avoid. The session flag is keyed on the **rule** that supplied the id and not on the id, which is the opposite of the 2026-08-31 note and right for the opposite reason: that note warns about a shared ledger however you reached it, while this asks whether the id survives the command — an exported `BACKDRAFT_SESSION` does and needs no flag, a typed `--session` does not. `gate.cli` makes that call and the reader owns the wording, the same split `session show` makes for its note, so the gate's library half still does not import the CLI's constants. The default is not echoed back as a `--limit`: a command carrying 12000 forward would go on meaning 12000 after the number moved. Two costs stated. There is no `--all`; the way to read a long page in one call is the `--limit` the closing line already names, which is why that line reports the total and not just the remainder. And `demo/walkthrough.md` quotes the closing line rather than adding the page read that produces it — the walkthrough's `session show` block is the ledger a clean run leaves behind, a new minting command would move those counts, and the numbers cannot be regenerated without re-fetching a source whose bytes carry a render date and would break every token in the demo. |
+| 2026-09-09 | **A source that came back a shell says so on every surface that lists it, not only in the `ingest` that printed the note.** The 2026-08-20 row gave `ingest` a character count and a `note: little text extracted` naming the cause, and both were printed once and then gone — nothing stored the signal and no other surface carried it, so off `backdraft read`'s list a 34,000-character article and a 27-character login wall were the same row. The asymmetry is the point: the skill says to ingest everything up front and then write, a registry travels with the project folder, and the agent choosing evidence is usually not the process that ingested — so the one signal that matters was the one it was least likely to be present for. `gate.thin_mark` now closes the row of a thin source in the gate's document list, in its table-of-contents headline (inside the parenthesis, where everything else describing the source already lives) and in `ls`, reading `little text: 27 chars`; the count is there because 0 characters and 190 are different problems. Reporting, not storage: the count is `sum(len(page.text))` over the current extraction, so nothing was added to the registry and no migration exists to have gone wrong. The owner is `gate/reader.py` beside `unit` rather than `kernel.model` beside `source_name`, which is the other precedent and the wrong one here: the kernel is the spec's vocabulary, `source_name` is a pure rule about a `Document` that bind also needs, and this is a **heuristic** about how a surface describes an extraction — putting a tunable threshold down there would read as normative to the next implementation, and only the gate and the CLI (which imports downward from it, as it already does for `unit`) ever ask. `cli.THIN_SOURCE_CHARS` is gone; `ingest`'s own line, its grouped note and all three list surfaces now derive the same number from `gate.extracted_chars`, which is the whole of the item — a second implementation of the count would have defeated it. Only thin rows are marked, so a registry of ordinary sources gains no column and prints byte for byte what it always did — the gate's count column is padded and `_block`'s existing rstrip takes the padding back off an unmarked row, and `ls` adds a fifth tab-separated field only where there is one to add. That fifth field is the same slot the 2026-08-18 row deliberately emptied when `ls` stopped printing a fetched source's URL beside its invented filename, and the difference is worth stating: that field was a second name for a thing that already had one, this one appears only on the rows nothing else says anything about. Two costs. The heuristic now lives on a persistent surface rather than in a note that scrolled away, so a genuinely tiny real source carries the mark for as long as it is in the registry — accepted on the 2026-08-20 terms, which still hold exactly: display only, exit 0, and no token, anchor, status or byte of the artifact derives from it. And the gate's own fixtures had to grow: the miniature standing in for a 34,000-character Wikipedia article held 112 characters, which would have pinned `little text` into the golden strings that document the list format — a fixture that small was already lying about what it represented, and the marks in `tests/gate` are now on sources that really are shells. |
+| 2026-09-10 | **What a source will be called is a question `ingest` answers with `--dry-run`, not a sibling command.** Three docs tell an agent to pass `--slug` when it ingests a URL, because the default may name a site rather than a page and a slug is permanent the moment a token carries it — and none of them let the agent find out what the default would be. The fix is that the question is cheap: `fetch.filename_for` settles the stem from the address alone (the 2026-08-19 row's own words), so the only way to ask it was to ingest and live with the answer, and getting it wrong costs a re-ingest under a new slug plus a rewritten draft. The shape considered against a flag was a sibling command — `backdraft name <source>` — and the objection to a flag is real: a verb that does not do its verb reads badly, and `ingest` is a writing verb. `--dry-run` answers it, and that is the whole reason it is spelled that way rather than `--plan` or `--name`: the convention is universal and says exactly the thing the objection worries about, that the verb did not run. Two things settled it beyond the wording. The question arises *while composing the ingest command*, so the answer belongs on that command's own help, next to the `--slug` it exists to inform — a separate noun is discoverable only by reading the top-level help of a CLI whose surface is deliberately four verbs. And the prediction has to be the outcome, which means taking `ingest`'s arguments and walking `ingest`'s path: `Registry.naming` matches identity through the same `_find_document` and assigns through the same `_assign_slug`, so it cannot drift from the thing it predicts. Where the two must differ is stated rather than hidden. A file's bytes are read, because identity is the bytes and a copy under another name is the same document keeping its first slug. A URL's are not, so continuity rests on the URL — which is what a re-fetch matches on anyway — and the honest gap, printed as a note rather than guessed at, is the media type: a slug comes from the address and is settled here, a media type comes from the content type the server sends and is not. A source already ingested reports its recorded media type instead, because that one *was* settled by a server. The registry's part of the answer is the part a name alone cannot give: already ingested (and withdrawn, since `ls` denies those exist), the numbered slug a taken name would land on, and that `--slug` will not rename a document that has one. Read-only throughout — no socket, no document, no anchor, no ledger row — and the test that pins it blocks `fetch.fetch` outright rather than checking the output for absence. |
+| 2026-09-14 | **`bind` and `verify` answer a caller that parses with `--json`, and the exit codes stay exactly where they are.** Both commands' output is the product and an agent is their main caller, which made every wording fix this repo lands — and it lands them weekly — a silent break for anything that scraped the last wording. The sharper problem was `verify`'s exit 2, which means both "a receipt did not hold" (the file was edited) and "a citation does not resolve against the sources" (the file is honest and the sources do not stand behind it): opposite conclusions, one code, and the prose the only place the difference lived. A third exit code was the obvious fix and was rejected, because the codes are the contract every Stop hook is already written against, and `bind` and `verify` sharing 2 is what lets one hook catch both (2026-08-24). The distinction goes in a payload instead. `bind --json` needed no new format: it prints the record, byte for byte the sidecar that run wrote, so there is no second serialization to keep in step with `spec/artifact.md`. `verify --json` did need one, `backdraft/verify-v1`, specified beside the checks it reports in § Checking an artifact and matched exactly under the artifact's no-guessing rule: each tier with a `ran` flag, and a `findings` list whose `kind` — `receipt`, `recount`, `source` — is the field a caller branches on, empty exactly when the run exits 0. A `receipt` finding also names the failed step in `check` (`snippet_sha256`, `token_parses`, `token_hash`, `token_slug`, `token_locator`), so a rehashed snippet and a re-pointed token are told apart without the sentence, and a `source` finding carries today's status beside the record's, so drift since binding is two values differing rather than a clause to find. `detail` and `error` stay prose and are declared rewordable; that line between keys and sentences is the point of the format. The same rule reached the implementation: `verify` now computes its findings once and hands them to two presenters, and the exit code reads the findings rather than either rendering, so the plain report is byte-identical to before and cannot drift from the object. Tradeoff accepted, and named in every skill that mentions the flag: `bind --json` is the whole record, evidence included, which for the demo is some 450 KB of mostly page images — the price of refusing to invent a second, slimmer bind format. The skills therefore tell an agent to pipe it into a parser (or read the record file bind already wrote) rather than print it into context, and to relay the plain report, not the JSON, to the user. |
+| 2026-09-15 | **`verify --against <project>` checks a file you were sent against a registry you have, and discovery still infers nothing.** The 2026-08-24 row put `verify`'s source tier behind discovery from cwd, because an artifact is a file people forward and the folder it landed in says nothing about which registry produced it. That rule is kept exactly, and it had left no way to say what the recipient often does know — this file came out of *that* project — so the only route to a source check was to copy the artifact into the project first: a filesystem move performed to change a discovery result, repeated per artifact by an agent auditing several, and impossible for a reviewer without write access to the checkout. The flag answers the objection rather than overriding it. Discovery stays refusal-by-default and never looks at the artifact's location; `--against` is the caller asserting a link the tool must not infer, the same shape as `--slug` overruling a derived name — the tool derives nothing it is not told, and what it is told it takes as said. Three consequences follow from treating it as an assertion rather than a hint. It bypasses discovery instead of joining it, so it outranks `BACKDRAFT_HOME` as well as the walk: a name typed on one command is more specific than one exported for a whole shell, and a test pins flag, then environment, then walk. A path holding no registry is a usage error (exit 1) naming what was expected, never a fall back to tier one — a caller who named a registry and got the file-only check would read `sources: no .backdraft/ found from here` as a fact about the file rather than a typo — and it is refused *before* anything opens, because `Registry.open` creates what it does not find and a read-only audit must not leave a registry in whatever directory was typed. And the report names the registry that answered either way, on the `sources:` line and in `--json`'s `sources.registry`, resolved to an absolute root so the line from an inbox is the line from inside the project, which is what makes a wrong `--against` visible rather than just a run of `unresolved`. Both forms `BACKDRAFT_HOME` accepts — the project root or the `.backdraft` directory — are accepted through one rule (`cli_context.named_root` shares `find_root`'s), so the two ways of naming a registry cannot drift. Without the flag nothing moved: seven scenarios, in and out of the demo project, with and without `--json` and `BACKDRAFT_HOME`, diff byte-identical against the previous CLI, including the closing `[Re-check against the sources: …]` hint, which still names only the in-project route; pointing that hint at the flag is a wording change the acceptance ruled out of this item. |
+| 2026-09-16 | **A chunk whose text may run on names the chunk it runs on into, decided by the chunks' own edges and never by the prose — so every page break counts.** The writing skill told an agent that "a claim that spans two chunks needs both tokens, not the nearest one", which is right and gave it nothing to act on: the gate handed back chunks, and whether the sentence being cited ended inside one was for the agent to notice by eye, mid-draft. When it did not, nothing complained — one token resolved, the receipt was real, the artifact rendered clean, and the half of the claim in the next chunk was uncited while looking cited, the one failure worse than `unresolved`, because `unresolved` says so. `gate.reader.adjoining` now answers it off the registry's own rows. On one page, two chunks adjoin when the whitespace between their stored offsets holds no blank line, tested with the chunker's own separator — `kernel.chunking.PARAGRAPH_BREAK`, made public so the question is asked of one pattern rather than a copy — because the chunker splits on blank lines before anything else, so two chunks with none between them are a paragraph rule 3 cut for length. A search hit carries that neighbour indented under its excerpt, labelled `same paragraph, before` or `after`, over the edge that touches the hit: the tail of the chunk before, the head of the chunk after. The page break was the decision, and it went the uncomfortable way on purpose. Each page is chunked alone and `pdf-text` writes no whitespace at a page's edges, so nothing a chunk carries tells a paragraph that crosses the break from one that ended at it; the only way to tell would be a sentence test at the join, and the obvious one — rule 3's own terminal-punctuation definition — was rejected twice over. It would decide what gets minted, which a guess must not. And its miss is exactly this row's failure: a paragraph that crosses the break at a full stop, under a claim spanning those two sentences. So a page break always adjoins, labelled `previous page ends` or `next page begins` rather than `same paragraph`, since that is all that is known, and the excerpt shows the reader in one line which case it is. The page read takes the same rule where it adds something: every chunk inside a read already sits beside the ones it adjoins, so only a read that shows the last chunk of its last page names anything — the next page's first chunk — and a read that stopped short already names the command that goes on. The costs, named rather than discovered. Emitting is minting, so the adjoining chunk is recorded as shown though only an excerpt of it was: it now binds `resolved` for a claim drawn from text the writer saw 160 characters of, the trade `search` already makes for every hit, and `ADJOINING_NOTE` says so once per run. It is noisy on clean documents: a read of any page of a paginated source but the last gains three lines and the note, and a hit on a page's first or last chunk gains its neighbour. The demo's own `read t12-summary p1`, a page ending on a full stop before a heading, now names `p2.c1`, which moves the walkthrough's `session show` from 77 anchors to 78; the gate tests' toy fixtures, one sentence a page, show it hardest. Display and ledger only everywhere else: no chunk boundary, token, anchor or receipt moves, and a hit or a read with nothing adjoining prints byte for byte what it did — pinned over the fakes, which now carry offsets the way the store does, and over a real ingest of a PDF whose paragraph the typesetter carried onto the next page. `EXCERPT_CHARS` and the excerpt cut moved from `searcher.py` to `reader.py` for the reason `LIST_HINT` lives there: both surfaces print the neighbour in one shape, and the searcher imports the reader, never the reverse. |
+| 2026-09-17 | **A drifted citation whose words did not change is found where they stand now, by exact text and never by resemblance, and printing its new token is not minting it.** A re-ingested source re-chunks, so a paragraph inserted near the top of a page moves every chunk ordinal below it, and a citation whose words are untouched comes back `drifted` because a chunk's ordinal is its address — the Open list's oldest line, sharpened when `ingest` learned to announce a new generation (2026-08-20) and still left the agent a list of failures and a manual hunt per line. `backdraft locate <doc.md>` does the hunt: for each citation `citation_for` calls `drifted`, it looks the cited anchor's snippet hash up in the document's current generation (`Registry.anchors_with_snippet`, one query on the new `idx_anchors_snippet`, which the idempotent DDL adds to an existing registry on its next open) through `current_with`, `current_at`'s other half — that one keeps the locator and asks what text stands there, this keeps the text and asks which locators hold it. Matching is the normalized hash, same anchor kind, and nothing looser, because a near match proposed as a move rewrites provenance; the price is named rather than discovered — a sentence edited by one figure is `gone`, and so is a chunk that an insertion under 200 characters merged into (chunking rule 2), since its text did change. Three outcomes, and only one proposes: `moved` is exactly one place holding the words of a chunk or a page. Several places are `ambiguous` and named, because choosing among identical text is a guess. So is a cell with one match, which was the real decision: a cell's receipt is its value, and an equal value at another address is as often a second figure that agrees — two rates of 6.25% — as the same cell after a row went in, so a workbook gets lines naming the right cell for a person to confirm rather than tokens to paste. Read-only is the queued shape and the gate's doctrine argued against it, since every gate surface that hands a writer a token to cite mints it; it holds anyway, and the reason is that printing is not what makes a token citable — `bind` is. A moved token binds `not_shown` until the session is shown it, which is the ledger being right about names even where the words were seen, so the queued acceptance ("applying those tokens by hand and re-binding gives a clean run") holds one step later than written: the closing line names the `show` that records exactly the moved tokens `--session` has not seen (read, never created, and carried when typed through `gate.session_argument`), or the re-bind itself once none remain. Minting here was the alternative and was rejected, because a command that proposes should not write the ledger it proposes against, and `verify` already prints tokens without minting them. Two more departures from the queued shape, stated. It walks the document's own tokens (`kernel.claims.parse_claims`) rather than the stored `bindings`, which go stale the moment one token is swapped by hand and key on the path as it was typed. And it lives in `cli.py` beside the `ingest` whose note now names it, not in `bind/cli.py`, since that closing line needs the gate's session wording and bind may not import the gate. Exit 0 whatever it finds: `bind` is the check and this is the advice. Not Later's living documents, which is a cited-then-versus-now presentation; this is the lookup that presentation would stand on. |
 
 ## Open
 
 Queued and parked work lives in ROADMAP.md; this list is questions still
 awaiting a decision:
 
-- Re-bind/orphan pass on re-ingest of changed docs (chunk ordinal drift)
 - Distribution: published to PyPI as `backdraft` 0.2.0 (2026-07-28); domain
   backdraft.dev acquired; still open: skill registration, deploying site/
 - First external user (design-partner validation layer vs eval substrate vs

{backdraft-0.10.0 → backdraft-0.12.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: backdraft
-Version: 0.10.0
+Version: 0.12.0
 Summary: Drop-in provenance for factual claims: one click from any claim to the evidence behind it.
 Project-URL: Homepage, https://backdraft.dev
 Project-URL: Documentation, https://backdraft.dev/docs.html
@@ -86,13 +86,27 @@ link* — `?oldid=` serves one revision's bytes forever — because a citation i
 a live article reports `drifted` the moment somebody edits it, and the quoted
 sentence would no longer be the sentence there.
 
+`--slug` is there because that URL's default name would be
+`en-wikipedia-org-index` — its last segment is `/index.php`, a handler rather
+than a page, so the host stands in — and a slug is permanent once tokens carry
+it. `backdraft ingest <source> --dry-run` is how to see that before choosing:
+it prints the slug and media type each source would take and stops, fetching
+nothing and writing nothing, and it says when a source is already ingested or
+when the name it wants is taken.
+
 Every ingest line ends with how much text came out, and says which of three
 things happened: a document created, a **new generation** of one whose bytes
 moved — which is when citations into the previous snapshot start reporting
-`drifted` — or `unchanged`, a no-op because re-running would reproduce what is
+`drifted`, and [`backdraft locate`](#when-a-source-changes) says where their
+text went — or `unchanged`, a no-op because re-running would reproduce what is
 already there. When almost no text came out, a note names the likely cause (a
 scan with no text layer, a page behind a login) and what to do; it is a note at
-exit 0, because a thin snapshot is still a real one.
+exit 0, because a thin snapshot is still a real one. That source keeps the mark
+afterwards — `backdraft read`'s document list, its table of contents and
+`backdraft ls` end its row with `little text: N chars` — because a registry
+travels with the project folder and whoever writes against it is usually not
+whoever ingested it. Ordinary sources gain nothing: only the rows worth doubting
+say anything.
 
 **Read.** The page arrives with a citable name over each chunk. This is the whole
 mechanism: what you can cite is exactly what you were shown.
@@ -102,7 +116,7 @@ $ backdraft read t12-summary p1
 # t12-summary p1  (page 1 of 3)
 
 [bd:t12-summary:p1.c1:c2e8]
-Bridgeview Commons, Trailing Twelve Month Summary
+Bridgeview Commons — Trailing Twelve Month Summary
 
 Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
 garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. This summary
@@ -115,12 +129,48 @@ $2,972,160. The resulting economic occupancy of 90.3% reflects an average physic
 by concessions, bad debt, and vacancy loss. Physical occupancy was 96.4% in the most recent month and has not
 fallen below 92.0% in any month of the period.
 
-… five more chunks, c3 through c7.
+… five more chunks, c3 through c7, and then:
+
+next page begins: [bd:t12-summary:p2.c1:7bd3]
+  Revenue Detail Rental income of $2,548,900 accounts for 94.9% of effective gross income. Other income of $135,500 comprises pet rent, parking, utility reimburse...
+
+note: text can run on across a paragraph split in two or a page break, so the chunk on the other side is named and recorded as shown. A claim whose words come from both sides cites both tokens.
 ```
 
 Chunks fall on the document's own paragraphs. A PDF stores glyphs at
 coordinates, not paragraphs, so the extractor rebuilds the breaks from the line
-geometry before the chunker sees the page.
+geometry before the chunker sees the page. Where a claim could still straddle
+two chunks — a paragraph the chunker cut for length, or a page break, which
+chunking never sees past — the gate names the chunk on the other side, minted
+like the rest, so the sentence that runs on is cited by both tokens rather than
+the one you happened to be reading.
+
+`backdraft read <slug>` on its own is the table of contents: one line per page
+for a source that paginates, and — for a web page, which has none — one line per
+chunk, under the name the page gave itself in its `<title>`. That list is the map
+you read before deciding whether to spend 34,000 characters on the page.
+
+And a page read will not spend all 34,000 of them unasked. One read shows
+12,000 characters by default — 200 rows of a sheet — which every ordinary page
+and small range fits inside, so the output above is what it always was. A page
+that runs past it stops on a chunk boundary and says so:
+
+```console
+$ backdraft read franklin-county p1
+# franklin-county p1  (page 1 of 1)
+
+[bd:franklin-county:p1.c1:d9b5]
+Jump to content
+…
+[Showing 0-11531 of 34031 chars. Continue with: backdraft read franklin-county p1 --offset 11531]
+```
+
+That last line is the whole of the contract: how much you got, how much there
+is, and the command that continues where this one stopped — which is also how
+you read a long page in one call, by passing the total it names as `--limit`.
+A read that prints no such line showed you the whole page. The cut falls between
+chunks and never inside one, so every token you are handed still names text you
+were shown all of.
 
 Search results are citable too, no page read required to get an anchor:
 
@@ -174,9 +224,9 @@ enough to write this yet?", asked before the draft exists rather than after
 $ backdraft session show
 session s-bridgeview  (from BACKDRAFT_SESSION)
 
-77 anchors shown across 3 documents
+78 anchors shown across 3 documents
 
-  t12-summary          7
+  t12-summary          8
   underwriting-model  68
   franklin-county      2
 
@@ -253,6 +303,29 @@ going unnoticed. Exit 0 when everything it checked passed, 2 when something did
 not, so a hook can gate on it. It opens no session and mints nothing: an audit
 must not make its subject citable.
 
+Holding a file that was sent to you beside a checkout of the project it came
+out of, say so rather than moving the file:
+
+```console
+$ backdraft verify ~/Downloads/memo.backdraft.html --against ~/work/project
+```
+
+That runs the source check too, and its `sources:` line names the registry that
+answered. `verify` never works out which project a file belongs to from where it
+landed, so the link is yours to assert; a path with no registry in it is refused
+rather than quietly checking the file alone.
+
+**Scripts and agents read keys, not lines.** The reports above are worded for
+people, and the wording changes. `backdraft bind memo.md --json` prints the
+record in place of the report (the same bytes bind writes), and
+`backdraft verify memo.backdraft.html --json` prints the check as one object whose
+`findings` each carry a `kind`: `receipt` when the file was edited, `source`
+when a citation does not resolve against the sources today (its status beside
+the record's says whether they moved since binding), `recount` when the summary
+disagrees with the claims. Exit codes are unchanged; the payload says which kind
+of exit 2 you got. The format is in
+[spec/artifact.md](spec/artifact.md#the-verify-report).
+
 **Theming.** The artifact ships with a default look and two alternates,
 `press` and `slate`:
 
@@ -331,7 +404,9 @@ Every surface names a fetched source by its page. The filename you never see —
 exists on nobody's disk, so `ingest`, `ls`, `backdraft read`, the artifact and
 the `## References` section `bind --bound` writes all show the URL in its place
 rather than beside it: two names for one thing would let the invented one look
-authoritative.
+authoritative. The artifact goes one further and calls the source by its
+`<title>` where the page declared one, because that is the only name in the
+record the source chose for itself.
 
 The origin travels into the artifact too. A claim citing a fetched page shows
 the URL as a link on its receipt, with the date the bytes were taken. That
@@ -346,6 +421,42 @@ are out of reach, and the extractor is a parse rather than a readability guess
 its mind between two versions of a site would move anchors. Responses are
 capped at 32 MiB.
 
+## When a source changes
+
+A re-ingested source re-chunks, so an edit near the top of a page moves every
+citation below it: a paragraph inserted above the one you cited shifts its chunk
+ordinal, and a citation whose words never changed comes back `drifted` because
+its address did. `bind` can say that much and no more. `locate` finds where the
+text went:
+
+```console
+$ backdraft locate memo.md
+3 citation(s) in memo.md, 3 drifted
+  gone: 1
+  moved: 2
+  ! moved: bd:q3-letter:p1.c1:afd2 — now at bd:q3-letter:p1.c2:afd2 — closed Q3 at 92.0% @24
+  ! moved: bd:q3-letter:p1.c2:d140 — now at bd:q3-letter:p1.c3:d140 — 1.42x @84
+  ! gone: bd:q3-letter:p1.c3:00ce — taxes of $412,300 @122
+[Nothing was rewritten. Put each moved token in place of the old one in memo.md, then show the new ones, which this session has not seen, and re-bind: backdraft show bd:q3-letter:p1.c2:afd2 bd:q3-letter:p1.c3:d140]
+[A gone citation's text was edited or removed, so nothing is proposed. Read what it cited, then search for the new wording: backdraft show bd:q3-letter:p1.c3:00ce]
+```
+
+For each citation `bind` would call `drifted`, it looks for the exact text the
+citation quotes — the normalized snippet hash — in the source's current
+snapshot. `moved` is one place holding it, and names the token there.
+`ambiguous` names the places when there are several, or when the citation is a
+cell: an equal value at another address is as often a second figure that
+happens to agree as the same cell after a row was inserted. `gone` is the text
+edited or removed, here a tax figure revised from $412,300, and nothing is
+proposed for it — matching is exact on purpose, because a near match offered as
+a move would rewrite provenance.
+
+It proposes and never rewrites, and it is read-only: the document, the registry
+and the ledger are untouched. That includes minting — a token printed here was
+never shown to your session, so swap the moved tokens in, run the `show` the
+closing line names (and read that the text is what you cited), then re-bind.
+Exit 0 whatever it finds: `bind` is the check.
+
 ## Forgetting a source
 
 Ingest is otherwise one-way, and an unattended folder ingest picks up what it

{backdraft-0.10.0 → backdraft-0.12.0}/README.md

@@ -59,13 +59,27 @@ link* — `?oldid=` serves one revision's bytes forever — because a citation i
 a live article reports `drifted` the moment somebody edits it, and the quoted
 sentence would no longer be the sentence there.
 
+`--slug` is there because that URL's default name would be
+`en-wikipedia-org-index` — its last segment is `/index.php`, a handler rather
+than a page, so the host stands in — and a slug is permanent once tokens carry
+it. `backdraft ingest <source> --dry-run` is how to see that before choosing:
+it prints the slug and media type each source would take and stops, fetching
+nothing and writing nothing, and it says when a source is already ingested or
+when the name it wants is taken.
+
 Every ingest line ends with how much text came out, and says which of three
 things happened: a document created, a **new generation** of one whose bytes
 moved — which is when citations into the previous snapshot start reporting
-`drifted` — or `unchanged`, a no-op because re-running would reproduce what is
+`drifted`, and [`backdraft locate`](#when-a-source-changes) says where their
+text went — or `unchanged`, a no-op because re-running would reproduce what is
 already there. When almost no text came out, a note names the likely cause (a
 scan with no text layer, a page behind a login) and what to do; it is a note at
-exit 0, because a thin snapshot is still a real one.
+exit 0, because a thin snapshot is still a real one. That source keeps the mark
+afterwards — `backdraft read`'s document list, its table of contents and
+`backdraft ls` end its row with `little text: N chars` — because a registry
+travels with the project folder and whoever writes against it is usually not
+whoever ingested it. Ordinary sources gain nothing: only the rows worth doubting
+say anything.
 
 **Read.** The page arrives with a citable name over each chunk. This is the whole
 mechanism: what you can cite is exactly what you were shown.
@@ -75,7 +89,7 @@ $ backdraft read t12-summary p1
 # t12-summary p1  (page 1 of 3)
 
 [bd:t12-summary:p1.c1:c2e8]
-Bridgeview Commons, Trailing Twelve Month Summary
+Bridgeview Commons — Trailing Twelve Month Summary
 
 Property: Bridgeview Commons, 4400 Halsted Avenue, Columbus, OH 43214. 128 units across four three-story
 garden buildings on 6.2 acres, built in 1998 and substantially renovated between 2019 and 2021. This summary
@@ -88,12 +102,48 @@ $2,972,160. The resulting economic occupancy of 90.3% reflects an average physic
 by concessions, bad debt, and vacancy loss. Physical occupancy was 96.4% in the most recent month and has not
 fallen below 92.0% in any month of the period.
 
-… five more chunks, c3 through c7.
+… five more chunks, c3 through c7, and then:
+
+next page begins: [bd:t12-summary:p2.c1:7bd3]
+  Revenue Detail Rental income of $2,548,900 accounts for 94.9% of effective gross income. Other income of $135,500 comprises pet rent, parking, utility reimburse...
+
+note: text can run on across a paragraph split in two or a page break, so the chunk on the other side is named and recorded as shown. A claim whose words come from both sides cites both tokens.
 ```
 
 Chunks fall on the document's own paragraphs. A PDF stores glyphs at
 coordinates, not paragraphs, so the extractor rebuilds the breaks from the line
-geometry before the chunker sees the page.
+geometry before the chunker sees the page. Where a claim could still straddle
+two chunks — a paragraph the chunker cut for length, or a page break, which
+chunking never sees past — the gate names the chunk on the other side, minted
+like the rest, so the sentence that runs on is cited by both tokens rather than
+the one you happened to be reading.
+
+`backdraft read <slug>` on its own is the table of contents: one line per page
+for a source that paginates, and — for a web page, which has none — one line per
+chunk, under the name the page gave itself in its `<title>`. That list is the map
+you read before deciding whether to spend 34,000 characters on the page.
+
+And a page read will not spend all 34,000 of them unasked. One read shows
+12,000 characters by default — 200 rows of a sheet — which every ordinary page
+and small range fits inside, so the output above is what it always was. A page
+that runs past it stops on a chunk boundary and says so:
+
+```console
+$ backdraft read franklin-county p1
+# franklin-county p1  (page 1 of 1)
+
+[bd:franklin-county:p1.c1:d9b5]
+Jump to content
+…
+[Showing 0-11531 of 34031 chars. Continue with: backdraft read franklin-county p1 --offset 11531]
+```
+
+That last line is the whole of the contract: how much you got, how much there
+is, and the command that continues where this one stopped — which is also how
+you read a long page in one call, by passing the total it names as `--limit`.
+A read that prints no such line showed you the whole page. The cut falls between
+chunks and never inside one, so every token you are handed still names text you
+were shown all of.
 
 Search results are citable too, no page read required to get an anchor:
 
@@ -147,9 +197,9 @@ enough to write this yet?", asked before the draft exists rather than after
 $ backdraft session show
 session s-bridgeview  (from BACKDRAFT_SESSION)
 
-77 anchors shown across 3 documents
+78 anchors shown across 3 documents
 
-  t12-summary          7
+  t12-summary          8
   underwriting-model  68
   franklin-county      2
 
@@ -226,6 +276,29 @@ going unnoticed. Exit 0 when everything it checked passed, 2 when something did
 not, so a hook can gate on it. It opens no session and mints nothing: an audit
 must not make its subject citable.
 
+Holding a file that was sent to you beside a checkout of the project it came
+out of, say so rather than moving the file:
+
+```console
+$ backdraft verify ~/Downloads/memo.backdraft.html --against ~/work/project
+```
+
+That runs the source check too, and its `sources:` line names the registry that
+answered. `verify` never works out which project a file belongs to from where it
+landed, so the link is yours to assert; a path with no registry in it is refused
+rather than quietly checking the file alone.
+
+**Scripts and agents read keys, not lines.** The reports above are worded for
+people, and the wording changes. `backdraft bind memo.md --json` prints the
+record in place of the report (the same bytes bind writes), and
+`backdraft verify memo.backdraft.html --json` prints the check as one object whose
+`findings` each carry a `kind`: `receipt` when the file was edited, `source`
+when a citation does not resolve against the sources today (its status beside
+the record's says whether they moved since binding), `recount` when the summary
+disagrees with the claims. Exit codes are unchanged; the payload says which kind
+of exit 2 you got. The format is in
+[spec/artifact.md](spec/artifact.md#the-verify-report).
+
 **Theming.** The artifact ships with a default look and two alternates,
 `press` and `slate`:
 
@@ -304,7 +377,9 @@ Every surface names a fetched source by its page. The filename you never see —
 exists on nobody's disk, so `ingest`, `ls`, `backdraft read`, the artifact and
 the `## References` section `bind --bound` writes all show the URL in its place
 rather than beside it: two names for one thing would let the invented one look
-authoritative.
+authoritative. The artifact goes one further and calls the source by its
+`<title>` where the page declared one, because that is the only name in the
+record the source chose for itself.
 
 The origin travels into the artifact too. A claim citing a fetched page shows
 the URL as a link on its receipt, with the date the bytes were taken. That
@@ -319,6 +394,42 @@ are out of reach, and the extractor is a parse rather than a readability guess
 its mind between two versions of a site would move anchors. Responses are
 capped at 32 MiB.
 
+## When a source changes
+
+A re-ingested source re-chunks, so an edit near the top of a page moves every
+citation below it: a paragraph inserted above the one you cited shifts its chunk
+ordinal, and a citation whose words never changed comes back `drifted` because
+its address did. `bind` can say that much and no more. `locate` finds where the
+text went:
+
+```console
+$ backdraft locate memo.md
+3 citation(s) in memo.md, 3 drifted
+  gone: 1
+  moved: 2
+  ! moved: bd:q3-letter:p1.c1:afd2 — now at bd:q3-letter:p1.c2:afd2 — closed Q3 at 92.0% @24
+  ! moved: bd:q3-letter:p1.c2:d140 — now at bd:q3-letter:p1.c3:d140 — 1.42x @84
+  ! gone: bd:q3-letter:p1.c3:00ce — taxes of $412,300 @122
+[Nothing was rewritten. Put each moved token in place of the old one in memo.md, then show the new ones, which this session has not seen, and re-bind: backdraft show bd:q3-letter:p1.c2:afd2 bd:q3-letter:p1.c3:d140]
+[A gone citation's text was edited or removed, so nothing is proposed. Read what it cited, then search for the new wording: backdraft show bd:q3-letter:p1.c3:00ce]
+```
+
+For each citation `bind` would call `drifted`, it looks for the exact text the
+citation quotes — the normalized snippet hash — in the source's current
+snapshot. `moved` is one place holding it, and names the token there.
+`ambiguous` names the places when there are several, or when the citation is a
+cell: an equal value at another address is as often a second figure that
+happens to agree as the same cell after a row was inserted. `gone` is the text
+edited or removed, here a tax figure revised from $412,300, and nothing is
+proposed for it — matching is exact on purpose, because a near match offered as
+a move would rewrite provenance.
+
+It proposes and never rewrites, and it is read-only: the document, the registry
+and the ledger are untouched. That includes minting — a token printed here was
+never shown to your session, so swap the moved tokens in, run the `show` the
+closing line names (and read that the text is what you cited), then re-bind.
+Exit 0 whatever it finds: `bind` is the check.
+
 ## Forgetting a source
 
 Ingest is otherwise one-way, and an unattended folder ingest picks up what it