proofpath 0.0.1__tar.gz → 0.1.0__tar.gz

proofpath-0.1.0/.env.example

@@ -0,0 +1,4 @@
+# Copy to .env (gitignored) and fill in. Only the optional LLM judge needs a key;
+# everything else runs offline. See `proofpath judge` for where this file is read.
+GROQ_API_KEY=
+# GEMINI_API_KEY=

proofpath-0.1.0/CHANGELOG.md

@@ -0,0 +1,162 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project adheres to
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-09-12
+
+First working release: `proofpath check` verifies a document's citations end to end
+and writes a report that states its own coverage.
+
+### Added
+- `proofpath check TARGET` as the v0.1 surface: compiler-style diagnostics, a
+  markdown `report.md`, `--format json`, `-q`, and exit codes `0` / `1` / `2`.
+- Confidence tiers calibrated on SciFact dev rather than chosen by hand
+  (`decide=0.45`, `medium=0.457948`, `high=0.99933`;
+  `docs/eval/2026-09-12-tiers.md`), shipped as `pipeline.DEFAULT_THRESHOLDS`.
+- Nine live user-like runs recorded in `docs/eval/2026-09-12-v0.1-live.md`, and
+  `scripts/zero_network_check.py`, which guards both HTTP clients and re-verifies a
+  document: with the network gone the fetch ladder makes no attempt at all, because
+  resolution produces no identifiers to fetch with.
+- Config and permissions module: `config.toml` under the platform config dir,
+  `proofpath config` / `proofpath config set permissions.<key>`, and the rule that an
+  `ask` permission without a TTY resolves to `deny` and is reported (spec §7.1).
+- Core types (`Verdict` cannot be `SUPPORTED`/`REFUTED` without a passage), device
+  selection (CUDA → CoreML → CPU), sentence retrieval over `fastembed` embeddings
+  scanned with numpy, ONNX NLI entailment on `cross-encoder/nli-deberta-v3-base`, and the aggregation
+  pipeline.
+- Reference resolution (spec §8): Crossref + Semantic Scholar first, then arXiv,
+  Open Library and OpenAlex before any ghost call; identity decided only by
+  field agreement against the raw string; DOI / arXiv id resolved directly;
+  retraction check via Crossref's Retraction Watch data and OpenAlex; new state
+  `UNVERIFIED (not in bibliographic indexes)`; per-host throttling, `Retry-After`,
+  OpenAlex daily-budget handling. `proofpath resolve "<reference>"` on the CLI.
+  Hand-built ghost set (106 real, 100 fabricated, 20 mutated) and
+  `scripts/eval_ghosts.py`: false-ghost rate 0 %.
+- Persistent cache: one plain SQLite file (`sources`, `raw_text` with 7-day TTL,
+  `chunks` with float32 embeddings, `verdicts`), a schema `CHECK` that refuses an
+  asserted verdict without a passage, and `proofpath cache` / `cache path` / `ls` /
+  `show` / `clear [--expired]`.
+- Numeric claim layer (spec §10): percentages, factors and unit counts with
+  direction are compared before NLI; an unambiguous contradiction is refuted by
+  rule with both figures named (`Verdict.reason`). Conservative by design: one
+  comparable figure on each side, change never against level.
+- Judge settings (`[judge]` in config, Groq default) with `proofpath config check`
+  and `proofpath config set judge.<key>`; API key resolved from the environment or
+  `.env`, never stored or printed. `.env.example` added.
+- SciFact loader pinned to the AI2 tarball by sha256, evaluation metrics, and
+  `scripts/eval_scifact.py`. First measured result: dev accuracy 0.606 vs 0.406
+  trivial baseline (`docs/eval/2026-09-11-scifact-dev.md`).
+
+- Fetch ladder (spec §7): `httpx` → `curl_cffi` TLS impersonation → browser engine
+  behind the §7.1 consent prompt → Wayback Machine; `robots.txt` via `protego`;
+  content type from headers; per-host throttling and backoff shared in `polite.py`;
+  fetched text cached with the 7-day TTL. Distinct honesty states for blocked,
+  blocked-by-robots, browser-not-permitted, unreachable, provider-unavailable and
+  network-denied — never collapsed.
+- Consent gate for the ~280 MB browser engine: asks at most once per run, never
+  without a TTY, `always`/`never` persist to config, installs with `pip` (or `uv`)
+  and `scrapling install`, and reports how many sources were skipped.
+- Open-access chain: Semantic Scholar → Crossref TDM links → Unpaywall (only with a
+  contact address) → Europe PMC → arXiv → landing page → abstract (OpenAlex last);
+  abstract-only results labelled `LOW CONFIDENCE (abstract only)`. DataCite arXiv
+  DOIs resolve straight to the arXiv PDF. `proofpath fetch <url|doi|arXiv id>`.
+- `scripts/eval_coverage.py`: measured 72 % full text / 18 % abstract / 10 % none
+  on 50 DOIs (`docs/eval/2026-09-11-coverage.md`).
+
+- Document ingest and claim extraction (spec §9): `document.py` value types with
+  page/line locators; `ingest.py` for PDF (pymupdf blocks, per-page line numbers,
+  running header/footer removal, superscript citations), docx (paragraphs and tables),
+  markdown and plain text, with the bibliography kept as raw strings, every
+  unparseable page reported as a `PageError` and a page holding an image and no text
+  reported as a scan rather than passed on as an empty page; `claims.py` pairs numeric
+  markers (`[12]`, `[12,15]`, `[12-15]`) with their sentence, applies the
+  `PARAGRAPH-SCOPED` rule, reports author-year markers as `UNSUPPORTED CITATION STYLE`
+  and any number the bibliography does not print as unresolved. Hand-built pairing set
+  (61 passages, rate 0.98) and `scripts/eval_pairing.py`; four real documents (three
+  PDFs and one extracted text) measured in `docs/eval/2026-09-11-pairing.md`.
+
+- `proofpath check` (spec §9, §13.2, §15): one `verify()` entry point built as
+  `prepare()` (parse, claims, resolve, retractions, fetch) and `decide_all()` (retrieval,
+  numeric rule, NLI, cached per source and claim); compiler-style diagnostics with the
+  quoted passage, the confidence tier and the exact honesty state; a coverage block in
+  every run and a "coverage is weak" line when a quarter or more of the sources could
+  not be read; `report.md` written by default, `--format json`, `-q`, `--out`, `check -`
+  for stdin, a real Ctrl-C that keeps what was decided (exit 2). A second run of the
+  same document re-decides nothing: chunks and verdicts come from the cache and neither
+  model scores again, although the models are still loaded and reference resolution and
+  the retraction check still go to the network. Cache schema v2: chunks carry the text
+  digest they were cut from and a source's verdicts are dropped when its text changes.
+  `permissions.network = deny` now also skips reference resolution and the retraction
+  check, each reported as not attempted.
+
+### Changed
+- CLI surface (spec §13.3): `permissions` and `judge` groups replaced by `config`
+  (`config` / `show` / `path` / `set SECTION.KEY VALUE` / `check`); global
+  `--no-color` and `-q`; one `ui.py` layer over `rich` owns every colour and the
+  10-column key/value layout; `resolve --format json`; "provider unavailable"
+  exits `1` (a finding), no longer `2`. A resolved but retracted reference and a
+  URL that was reached but yielded no text (`reached but no text extracted`) are
+  findings too (`1`). `skipped N source(s)` counts sources, not the URLs tried for
+  them; `--format json` adds `browser.skipped_urls`. `permissions.network` binds
+  the open-access providers as well as the ladder, and `ask` without a TTY is
+  `deny`, reported.
+- Retrieval no longer depends on `sqlite-vec`: a numpy cosine scan is faster at
+  every measured scale and the plain SQLite file opens in any GUI.
+- Spec: `PARAGRAPH-SCOPED` and `UNSUPPORTED CITATION STYLE` states, three-tier
+  confidence display, 7-day raw-text cache TTL, v0.1 limited to numeric citation
+  markers.
+
+### Fixed (found by the release's own live runs, `docs/eval/2026-09-12-v0.1-live.md`)
+- A real reference whose author list carries a surname particle (`van der Walt`) was
+  called a `GHOST REFERENCE` — product rule 3. Reference resolution now understands
+  particles (including glued `al-`/`el-` forms), scores the title against every
+  title-like segment, and accepts a DOI whose record agrees on first author and year
+  as `RESOLVED (low confidence)` instead of a ghost. The ghost set grew to 258 rows
+  (`docs/eval/2026-09-12-ghosts.md`): false-ghost rate 0.0 %, fabricated recall 100 %.
+- A fabricated reference in a numbered bibliography was reported as
+  `UNVERIFIED (not in bibliographic indexes)` instead of a ghost, because the printed
+  marker (`[7] `) blinded the "is this even a paper" check. The marker is stripped once
+  at the resolver's entry, guarded so it can never remove a year, an identifier or a
+  title that begins with a number.
+- A markdown draft piped through `proofpath -q check -` lost its `## References`.
+- The consent-gated browser step (fetch-ladder step 3) could not install itself
+  (`scrapling` has no `__main__`); it now runs the package's own CLI after `pip`/`uv`,
+  and a paywalled Cell landing page was read through it live (3,632 words).
+- A piped run occasionally aborted with exit `134` from ONNX runtime teardown after
+  printing a complete report; the engine now releases both models explicitly and the
+  CLI flushes its streams before exiting (20 of 20 consecutive runs exit `1`).
+- A document with citation markers but no detected bibliography printed a clean-looking
+  `0 %` coverage block; it now says `no bibliography was found; N citation markers could
+  not be checked`.
+
+### Known issues
+- Reference resolution and the retraction check are not cached, so a warm re-run is
+  still a network run: a 68-reference PDF takes about 13 minutes cold and 3.7 minutes
+  cached, a 129-reference one 24 minutes cold. Concurrent resolution and a resolution
+  cache are v0.2 work.
+- Two-author lists written `First Last and First Last` resolve to `AMBIGUOUS`, not
+  `RESOLVED`; an arXiv id whose record agrees on author and year is not yet rescued the
+  way a DOI is.
+- `browser.is_installed()` checks that the packages import, not that a browser binary
+  exists; a half-installed environment skips the installer and fails inside the browser
+  fetch (reported as an unverified source, never a crash).
+
+## [0.0.1] - 2026-09-10
+
+First release. The verification pipeline is not implemented; this reserves the name
+and establishes the interface, packaging and CI that later phases build on.
+
+### Added
+- Design specification with measured source-access data (spec §6), the fetch ladder
+  and its permission model (§7), and corrected reference resolution (§8).
+- Phased implementation plan, ordered by risk retired rather than user-visible
+  progress.
+- `proofpath` command. A bare invocation is a first-class entry point rather than a
+  help screen, which is where the TUI will attach. Exit codes are fixed: `0` clean,
+  `1` findings, `2` the run itself failed.
+- Cross-platform CI on Linux, macOS and Windows, and PyPI publishing through trusted
+  publishing rather than a stored API token.

proofpath-0.1.0/PKG-INFO

@@ -0,0 +1,224 @@
+Metadata-Version: 2.5
+Name: proofpath
+Version: 0.1.0
+Summary: Check whether the sources behind a claim actually say what the claim says.
+Project-URL: Homepage, https://github.com/Yigtwxx/proofpath
+Project-URL: Repository, https://github.com/Yigtwxx/proofpath
+Project-URL: Issues, https://github.com/Yigtwxx/proofpath/issues
+Project-URL: Changelog, https://github.com/Yigtwxx/proofpath/blob/main/CHANGELOG.md
+Author: Yigit Erdogan
+License-Expression: MIT
+License-File: LICENSE
+Keywords: citation,cli,entailment,fact-checking,nli,rag,research-integrity,retrieval-augmented-generation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: curl-cffi>=0.16.1
+Requires-Dist: fastembed>=0.5
+Requires-Dist: httpx>=0.28
+Requires-Dist: huggingface-hub>=0.30
+Requires-Dist: numpy>=1.26
+Requires-Dist: onnxruntime>=1.20
+Requires-Dist: orjson>=3.11
+Requires-Dist: platformdirs>=4.3
+Requires-Dist: protego>=0.3
+Requires-Dist: pymupdf>=1.25
+Requires-Dist: python-docx>=1.1
+Requires-Dist: rich>=13
+Requires-Dist: scrapling>=0.4.15
+Requires-Dist: textual>=1.0
+Requires-Dist: tokenizers>=0.20
+Requires-Dist: tomli>=2.0; python_version < '3.11'
+Requires-Dist: typer>=0.15
+Provides-Extra: browser
+Requires-Dist: scrapling[fetchers]>=0.4.15; extra == 'browser'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.14; extra == 'dev'
+Requires-Dist: pytest-cov>=6.0; extra == 'dev'
+Requires-Dist: pytest>=8.3; extra == 'dev'
+Requires-Dist: respx>=0.22; extra == 'dev'
+Requires-Dist: ruff>=0.9; extra == 'dev'
+Provides-Extra: gpu
+Requires-Dist: sentence-transformers>=3.3; extra == 'gpu'
+Requires-Dist: torch>=2.5; extra == 'gpu'
+Description-Content-Type: text/markdown
+
+# proofpath
+
+> Don't guess. Show the evidence.
+
+`proofpath` checks whether the sources behind a claim actually say what the claim
+says. Point it at a paper or a draft and it verifies every citation on three levels,
+then shows you the passage behind each verdict.
+
+1. **Does the source exist?** — Crossref, Semantic Scholar, arXiv, Open Library, OpenAlex.
+2. **Is it still valid?** — Crossref's Retraction Watch data, and OpenAlex.
+3. **Does it support the claim?** — retrieval and entailment against the source text,
+   with a numeric rule that runs before the model.
+
+Most tools stop at step 1. Step 3 is the point.
+
+Runs locally and free: no API key, no Docker, no server. Windows, Linux, macOS.
+
+```bash
+uv tool install proofpath
+
+proofpath check paper.pdf          # one-shot report → report.md, exit 0/1/2
+proofpath check draft.md --format json | jq '.coverage'
+proofpath -q check - < draft.md    # stdin; findings and coverage only
+```
+
+The install carries **no browser engine**. When a publisher blocks the plain fetch,
+proofpath asks once before downloading one — about 280 MB, into its own environment
+and the shared browser cache, never system-wide — and remembers the answer.
+`proofpath config set permissions.install_browser never` stops it asking at all.
+
+## Honesty
+
+**Most verdicts are "not enough information", and that is the honest answer.** On
+SciFact dev only **135 of 340** claim–source pairs score above the decision threshold
+at all; the other 60 % are `NEI` rather than guessed. Full text is openly reachable
+for well under half of published citations, so a real document loses more on top.
+
+**Every report states its own coverage** (and adds a warning line when it is thin):
+
+```
+7 refs: 1 ghost, 2 unsupported, 4 ok
+fulltext   72%
+abstract   14%
+unverified 14%
+```
+
+When a quarter or more of the sources could not be read, a further line says so:
+`coverage is weak: unread sources may hold more, so this is a lower bound`.
+
+**No verdict without its passage.** `SUPPORTED` and `REFUTED` cannot exist without
+the quoted sentence they rest on — the cache schema itself refuses to store one.
+
+**The confidence tiers are measured, not chosen** — read off a sweep on SciFact dev
+([details](docs/eval/2026-09-12-tiers.md)): `decide = 0.45`, `medium = 0.457948`,
+`high = 0.99933`. Two caveats belong next to those numbers. `high` was fitted on the
+same split it is reported on: an in-sample point estimate over **21 verdicts** (at
+least 18 correct — roughly 0.65–0.95 at 95 % confidence), so read it as "the model
+was near-certain here", not as a guarantee of 85 % precision. And `medium` lands
+almost exactly on `decide`, so `low` is practically empty among asserted verdicts —
+the display is effectively **two tiers**: near-certain, and asserted at all.
+
+## What the states mean
+
+Absence of evidence is never reported as evidence of absence. Each of these is a
+distinct, printed state (spec §15), never collapsed into a verdict:
+
+| State | Cause |
+|---|---|
+| `LOW CONFIDENCE (abstract only)` | full text unavailable, abstract used |
+| `UNVERIFIED (blocked)` | 403 or bot protection |
+| `UNVERIFIED (blocked, robots.txt)` | the site's `robots.txt` disallows the fetch |
+| `UNVERIFIED (blocked, browser not permitted)` | steps 1–2 blocked and the browser consent was denied or impossible |
+| `UNVERIFIED (unreachable)` | dead link, Wayback miss |
+| `UNVERIFIED (reached, no text extracted)` | 200 answered, nothing readable came back |
+| `UNVERIFIED (network not permitted)` | `permissions.network = deny` |
+| `UNVERIFIED (provider unavailable)` | API down or rate limited after backoff |
+| `UNVERIFIED (not in bibliographic indexes)` | web page, blog, report — indexes do not cover it, so absence proves nothing |
+| `AMBIGUOUS` | several plausible records, all listed |
+| `NEI` | the source was read and neither supports nor contradicts |
+| `PARAGRAPH-SCOPED` | the citation covers a paragraph; each sentence is judged separately |
+| `UNSUPPORTED CITATION STYLE` | an author–year marker; v0.1 pairs numeric markers only |
+
+Exit codes: `0` clean, `1` findings (every `UNVERIFIED` and `LOW CONFIDENCE` counts),
+`2` the run itself failed — no text parsing needed to gate a CI job. An earlier build
+sometimes aborted with `134` after printing a complete report (ONNX runtime
+teardown); fixed in this release — 20 of 20 piped runs exit `1` ([live runs](docs/eval/2026-09-12-v0.1-live.md)).
+
+## What v0.1 cannot do yet
+
+- **Numeric citation markers only** — `[12]`, `[12,15]`, `[12-15]`; an author–year
+  citation is listed as `UNSUPPORTED CITATION STYLE`, not judged.
+- **Superscript citations only when the PDF marks them as superscript**; a PDF that
+  draws them as ordinary digits loses them, and `km²` can be read as `[2]`.
+- **Abstract fallback**: when only an abstract is reachable the verdict is labelled
+  `LOW CONFIDENCE (abstract only)`. Three sentences is not a source.
+- **A reference list without a `References` heading is not found at all**, so its
+  markers are reported as uncheckable rather than checked.
+- **Resolution and the retraction check are not cached**, so even a warm re-run goes
+  to the network for them.
+- **The 0 % false-ghost rate is a property of the hand set's citation style.** Live
+  runs hit styles that set does not contain, and misjudged real references because of
+  it ([the live runs](docs/eval/2026-09-12-v0.1-live.md)).
+- **A refused or failed browser install is reported, not hidden**: the source is
+  `UNVERIFIED (blocked, browser not permitted)` with the install log, never silently
+  counted as unreachable.
+- **Coverage is not perfectly reproducible**: two runs minutes apart can read a
+  different number of sources, depending on which providers answered.
+- The TUI and `--format sarif` arrive in **v0.2**; the LLM judge and `--summarize`
+  in **v0.3**.
+
+## Speed
+
+Apple Silicon Mac, models already downloaded ([live runs](docs/eval/2026-09-12-v0.1-live.md)):
+
+| document | first run | cached re-run |
+|---|---|---|
+| 1-page markdown draft, 7 references | 79.3 s | 15.8 s * |
+| 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s |
+
+ * the draft's cached re-run followed the earlier of the two recorded cold runs; the live doc keeps both.
+
+The **first ever** run also downloads about 250 MB of ONNX models. A cached re-run
+reads its chunks and verdicts back from the cache instead of recomputing them, but it
+is not a no-op: both models are still loaded, reference resolution and the retraction
+check still query the network, and any source whose text has expired or was never
+read is fetched again — the PDF re-run above still spent 22 s fetching (it went out to
+Wayback and arXiv) and re-scored 3 of 102 claims.
+
+## Looking inside the cache
+
+Everything proofpath fetches, embeds and decides lands in one plain SQLite file:
+
+```bash
+proofpath cache            # where it is and what it holds
+proofpath cache ls         # sources, chunk/verdict counts, text expiry
+proofpath cache show <id>  # one source's chunks and verdicts
+proofpath cache clear --expired
+```
+
+Open `proofpath cache path` in [DB Browser for SQLite](https://sqlitebrowser.org/),
+TablePlus or DBeaver — plain tables, no extension. Raw publisher text expires after
+7 days; verdicts keep the passage they quote. `proofpath resolve REF` and `proofpath
+fetch URL|DOI` run either half on its own.
+
+## Optional LLM judge — arrives in v0.3
+
+Everything above runs locally. An LLM is used only at the end, as an opt-in second
+opinion on low-confidence verdicts — it never sees a source document and cannot
+change a verdict. The settings exist today (`proofpath config check` proves a key
+works, default Groq); `check --judge` and `--summarize` are wired in v0.3. The key
+comes from the environment or a `.env` file, never from config, and is never printed.
+
+## Measured
+
+| What | Set | Result |
+|---|---|---|
+| Retrieval + entailment | SciFact dev, 340 pairs | 0.609 accuracy, 0.597 macro-F1, against a 0.406 trivial baseline ([details](docs/eval/2026-09-12-scifact-dev.md)) |
+| Reference resolution | hand-built ghost set, 258 references | 0 % false-ghost, 100 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) — but see the live-run caveat above |
+| Source access | 50 DOIs | 72 % full text, 18 % abstract only, 10 % nothing ([details](docs/eval/2026-09-11-coverage.md)) — a real biomedical paper in the live runs reached 33 % full text |
+| Citation pairing | 61 hand-built passages | 0.98 ([details](docs/eval/2026-09-11-pairing.md)) |
+
+Published SciFact results sit around 70–75 F1, not 95. Nothing is tuned on a test
+split, and no number is quoted without the run that produced it.
+
+- [Design specification](docs/superpowers/specs/2026-09-10-proofpath-design.md) — what it does and the measurements behind each decision
+- [Open items](docs/superpowers/OPEN-ITEMS.md) — what is unresolved, and what has not been verified yet · [Changelog](CHANGELOG.md)
+
+## License
+
+MIT

proofpath-0.1.0/README.md

@@ -0,0 +1,170 @@
+# proofpath
+
+> Don't guess. Show the evidence.
+
+`proofpath` checks whether the sources behind a claim actually say what the claim
+says. Point it at a paper or a draft and it verifies every citation on three levels,
+then shows you the passage behind each verdict.
+
+1. **Does the source exist?** — Crossref, Semantic Scholar, arXiv, Open Library, OpenAlex.
+2. **Is it still valid?** — Crossref's Retraction Watch data, and OpenAlex.
+3. **Does it support the claim?** — retrieval and entailment against the source text,
+   with a numeric rule that runs before the model.
+
+Most tools stop at step 1. Step 3 is the point.
+
+Runs locally and free: no API key, no Docker, no server. Windows, Linux, macOS.
+
+```bash
+uv tool install proofpath
+
+proofpath check paper.pdf          # one-shot report → report.md, exit 0/1/2
+proofpath check draft.md --format json | jq '.coverage'
+proofpath -q check - < draft.md    # stdin; findings and coverage only
+```
+
+The install carries **no browser engine**. When a publisher blocks the plain fetch,
+proofpath asks once before downloading one — about 280 MB, into its own environment
+and the shared browser cache, never system-wide — and remembers the answer.
+`proofpath config set permissions.install_browser never` stops it asking at all.
+
+## Honesty
+
+**Most verdicts are "not enough information", and that is the honest answer.** On
+SciFact dev only **135 of 340** claim–source pairs score above the decision threshold
+at all; the other 60 % are `NEI` rather than guessed. Full text is openly reachable
+for well under half of published citations, so a real document loses more on top.
+
+**Every report states its own coverage** (and adds a warning line when it is thin):
+
+```
+7 refs: 1 ghost, 2 unsupported, 4 ok
+fulltext   72%
+abstract   14%
+unverified 14%
+```
+
+When a quarter or more of the sources could not be read, a further line says so:
+`coverage is weak: unread sources may hold more, so this is a lower bound`.
+
+**No verdict without its passage.** `SUPPORTED` and `REFUTED` cannot exist without
+the quoted sentence they rest on — the cache schema itself refuses to store one.
+
+**The confidence tiers are measured, not chosen** — read off a sweep on SciFact dev
+([details](docs/eval/2026-09-12-tiers.md)): `decide = 0.45`, `medium = 0.457948`,
+`high = 0.99933`. Two caveats belong next to those numbers. `high` was fitted on the
+same split it is reported on: an in-sample point estimate over **21 verdicts** (at
+least 18 correct — roughly 0.65–0.95 at 95 % confidence), so read it as "the model
+was near-certain here", not as a guarantee of 85 % precision. And `medium` lands
+almost exactly on `decide`, so `low` is practically empty among asserted verdicts —
+the display is effectively **two tiers**: near-certain, and asserted at all.
+
+## What the states mean
+
+Absence of evidence is never reported as evidence of absence. Each of these is a
+distinct, printed state (spec §15), never collapsed into a verdict:
+
+| State | Cause |
+|---|---|
+| `LOW CONFIDENCE (abstract only)` | full text unavailable, abstract used |
+| `UNVERIFIED (blocked)` | 403 or bot protection |
+| `UNVERIFIED (blocked, robots.txt)` | the site's `robots.txt` disallows the fetch |
+| `UNVERIFIED (blocked, browser not permitted)` | steps 1–2 blocked and the browser consent was denied or impossible |
+| `UNVERIFIED (unreachable)` | dead link, Wayback miss |
+| `UNVERIFIED (reached, no text extracted)` | 200 answered, nothing readable came back |
+| `UNVERIFIED (network not permitted)` | `permissions.network = deny` |
+| `UNVERIFIED (provider unavailable)` | API down or rate limited after backoff |
+| `UNVERIFIED (not in bibliographic indexes)` | web page, blog, report — indexes do not cover it, so absence proves nothing |
+| `AMBIGUOUS` | several plausible records, all listed |
+| `NEI` | the source was read and neither supports nor contradicts |
+| `PARAGRAPH-SCOPED` | the citation covers a paragraph; each sentence is judged separately |
+| `UNSUPPORTED CITATION STYLE` | an author–year marker; v0.1 pairs numeric markers only |
+
+Exit codes: `0` clean, `1` findings (every `UNVERIFIED` and `LOW CONFIDENCE` counts),
+`2` the run itself failed — no text parsing needed to gate a CI job. An earlier build
+sometimes aborted with `134` after printing a complete report (ONNX runtime
+teardown); fixed in this release — 20 of 20 piped runs exit `1` ([live runs](docs/eval/2026-09-12-v0.1-live.md)).
+
+## What v0.1 cannot do yet
+
+- **Numeric citation markers only** — `[12]`, `[12,15]`, `[12-15]`; an author–year
+  citation is listed as `UNSUPPORTED CITATION STYLE`, not judged.
+- **Superscript citations only when the PDF marks them as superscript**; a PDF that
+  draws them as ordinary digits loses them, and `km²` can be read as `[2]`.
+- **Abstract fallback**: when only an abstract is reachable the verdict is labelled
+  `LOW CONFIDENCE (abstract only)`. Three sentences is not a source.
+- **A reference list without a `References` heading is not found at all**, so its
+  markers are reported as uncheckable rather than checked.
+- **Resolution and the retraction check are not cached**, so even a warm re-run goes
+  to the network for them.
+- **The 0 % false-ghost rate is a property of the hand set's citation style.** Live
+  runs hit styles that set does not contain, and misjudged real references because of
+  it ([the live runs](docs/eval/2026-09-12-v0.1-live.md)).
+- **A refused or failed browser install is reported, not hidden**: the source is
+  `UNVERIFIED (blocked, browser not permitted)` with the install log, never silently
+  counted as unreachable.
+- **Coverage is not perfectly reproducible**: two runs minutes apart can read a
+  different number of sources, depending on which providers answered.
+- The TUI and `--format sarif` arrive in **v0.2**; the LLM judge and `--summarize`
+  in **v0.3**.
+
+## Speed
+
+Apple Silicon Mac, models already downloaded ([live runs](docs/eval/2026-09-12-v0.1-live.md)):
+
+| document | first run | cached re-run |
+|---|---|---|
+| 1-page markdown draft, 7 references | 79.3 s | 15.8 s * |
+| 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s |
+
+ * the draft's cached re-run followed the earlier of the two recorded cold runs; the live doc keeps both.
+
+The **first ever** run also downloads about 250 MB of ONNX models. A cached re-run
+reads its chunks and verdicts back from the cache instead of recomputing them, but it
+is not a no-op: both models are still loaded, reference resolution and the retraction
+check still query the network, and any source whose text has expired or was never
+read is fetched again — the PDF re-run above still spent 22 s fetching (it went out to
+Wayback and arXiv) and re-scored 3 of 102 claims.
+
+## Looking inside the cache
+
+Everything proofpath fetches, embeds and decides lands in one plain SQLite file:
+
+```bash
+proofpath cache            # where it is and what it holds
+proofpath cache ls         # sources, chunk/verdict counts, text expiry
+proofpath cache show <id>  # one source's chunks and verdicts
+proofpath cache clear --expired
+```
+
+Open `proofpath cache path` in [DB Browser for SQLite](https://sqlitebrowser.org/),
+TablePlus or DBeaver — plain tables, no extension. Raw publisher text expires after
+7 days; verdicts keep the passage they quote. `proofpath resolve REF` and `proofpath
+fetch URL|DOI` run either half on its own.
+
+## Optional LLM judge — arrives in v0.3
+
+Everything above runs locally. An LLM is used only at the end, as an opt-in second
+opinion on low-confidence verdicts — it never sees a source document and cannot
+change a verdict. The settings exist today (`proofpath config check` proves a key
+works, default Groq); `check --judge` and `--summarize` are wired in v0.3. The key
+comes from the environment or a `.env` file, never from config, and is never printed.
+
+## Measured
+
+| What | Set | Result |
+|---|---|---|
+| Retrieval + entailment | SciFact dev, 340 pairs | 0.609 accuracy, 0.597 macro-F1, against a 0.406 trivial baseline ([details](docs/eval/2026-09-12-scifact-dev.md)) |
+| Reference resolution | hand-built ghost set, 258 references | 0 % false-ghost, 100 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) — but see the live-run caveat above |
+| Source access | 50 DOIs | 72 % full text, 18 % abstract only, 10 % nothing ([details](docs/eval/2026-09-11-coverage.md)) — a real biomedical paper in the live runs reached 33 % full text |
+| Citation pairing | 61 hand-built passages | 0.98 ([details](docs/eval/2026-09-11-pairing.md)) |
+
+Published SciFact results sit around 70–75 F1, not 95. Nothing is tuned on a test
+split, and no number is quoted without the run that produced it.
+
+- [Design specification](docs/superpowers/specs/2026-09-10-proofpath-design.md) — what it does and the measurements behind each decision
+- [Open items](docs/superpowers/OPEN-ITEMS.md) — what is unresolved, and what has not been verified yet · [Changelog](CHANGELOG.md)
+
+## License
+
+MIT