proofpath 0.0.1__tar.gz → 0.2.0__tar.gz

proofpath-0.2.0/.env.example

@@ -0,0 +1,4 @@
+# Copy to .env (gitignored) and fill in. Only the optional LLM judge needs a key;
+# everything else runs offline. See `proofpath judge` for where this file is read.
+GROQ_API_KEY=
+# GEMINI_API_KEY=

proofpath-0.2.0/.gitattributes

@@ -0,0 +1,3 @@
+# Test fixtures whose bytes are pinned by a digest must not be CRLF-converted.
+tests/data/*.json text eol=lf
+tests/data/*.jsonl text eol=lf

proofpath-0.2.0/CHANGELOG.md

@@ -0,0 +1,284 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project adheres to
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.2.0] - 2026-09-15
+
+The interactive front-end, author-year citations, SARIF output and a cache for the
+two lookups that made a warm re-run slow. Bare `proofpath` now opens the TUI; the
+one-shot verbs are unchanged for CI. Live checks in `docs/eval/2026-09-15-v0.2-live.md`.
+
+### Added
+- **TUI** (spec §13.1): bare `proofpath` opens a `textual` session — a four-line banner
+  with the pet, one prompt bar, and a scrolling log of run blocks. Every CLI verb is a
+  slash command (`/check`, `/resolve`, `/fetch`, `/config`, `/cache`; plus `/allow`,
+  `/cancel #n`, `/help`, `/quit`), a bare verb waits for its argument (`Esc` leaves
+  it), and a line that is not a command is something to check. Runs are scheduled
+  concurrently through one `Scheduler` (network stages in parallel under a shared
+  politeness limiter, the NLI model one run at a time), each run is **cancellable**
+  mid-flight and keeps what it had decided, the coverage footer never scrolls away,
+  and the §7.1 permission prompt is drawn **inline under the stage that hit the wall**
+  with `[allow once] [always] [no] [never]` buttons (or `/allow …`). Every log line
+  works by mouse and by keyboard: run headers fold, stage rows hide their summary,
+  findings open the full quoted passage, `⧉`/`c` copies it, a finding's reference is
+  an OSC 8 link to its source. The pet's eyes blink, watch a run and react to its
+  result — only with colour, never under `--no-color` or `-q`.
+- **Author-year citations** (spec §9, §17): `(Smith et al., 2020)`, `Smith (2020)`,
+  `(Smith, 2020; Jones, 2019)`, `2020a`/`2020b` collisions, `ibid.` and `op. cit.`
+  back-references, and mixed `(Smith, 2020; [12])` all pair with their bibliography
+  entry. Hand-built set of 55 passages, 83 expectations, rate **0.940**
+  (`docs/eval/2026-09-12-pairing-author-year.md`); the numeric set is unchanged at
+  108/109. A marker no entry matches is reported as `UNRESOLVED MARKER`, never guessed.
+- **`check --format sarif`** (spec §13.2): the run as a SARIF 2.1.0 log for VS Code or
+  any SARIF 2.1.0 viewer — one result per finding on its line, the quoted passage in
+  the message of every asserting result, the exact honesty state in `properties`, and
+  the run's coverage in `runs[0].properties`. `--out FILE` writes the same document.
+  Validated against the schemastore schema, vendored in
+  `tests/data/sarif-schema-2.1.0.json`; not yet exercised against GitHub code scanning.
+- **Resolution and retraction cache** (cache schema v3): `resolutions` (30-day TTL,
+  keyed by the marker-free folded entry) and `retractions` (30 days for a notice, 7
+  for its absence). Neither an `UNVERIFIED (provider unavailable)` nor a retraction
+  check every provider failed is ever stored. A warm re-run of the seven-reference
+  draft: **1.35 s** wall clock, 0 network calls (v0.1: 15.8 s). `proofpath cache`
+  counts both tables.
+- The §7.1 aggregate line `skipped N source(s) because the browser was not permitted`
+  is printed by `check` too (terminal and markdown), never dropped by `-q`.
+- `browser_binary_present()`: the consent gate checks for a chromium build under
+  `PLAYWRIGHT_BROWSERS_PATH` or the platform default, so a half-installed environment
+  runs the idempotent installer instead of failing inside the fetch; the consent log
+  gains `browser binary: found|missing`.
+- `commands.py`: the shared wiring behind every mirrored verb (`resolve_reference`,
+  `fetch_target`, `config_*`, `cache_*`) returns result objects and never prints, so
+  `cli.py` and the TUI can only differ in how a result is drawn.
+- Bibliography fallback for PDFs without a `References` heading (paged formats):
+  the last contiguous run of numbered paragraphs is read as the list. AlphaFold:
+  0 → 17 references found, 97 markers now reported as unresolved instead of invisible.
+- Resolver rescues: `First Last and First Last` author lists resolve instead of landing
+  in `AMBIGUOUS`; an arXiv id whose record agrees on author and year is accepted as
+  `RESOLVED (low confidence)` the way a DOI already was. Ghost set 274 rows: false-ghost
+  **0.0 %**, ghost recall 99.1 % (`docs/eval/2026-09-12-ghosts.md`).
+- `tests/data/draft-author-year.md`, a committed author-year draft with an offline
+  smoke test; live: 6 citations paired, 0 unresolved.
+
+### Changed
+- Cache schema **v3** (migrated in place from v1/v2; versions compared as integers).
+- `Claims.unsupported` now means "a style this version cannot pair" and is empty by
+  construction; `UNSUPPORTED CITATION STYLE` is left for footnote-only and
+  superscript-letter styles. Markers that pair with nothing are `unresolved`.
+- `resolve.looks_unindexed` reads only the initials-path author patterns: a full-name
+  list (`First Last and First Last`) no longer counts as "this entry printed an
+  author list", so a title-first book keeps its title (see Known issues for the price).
+- `--format sarif` on `resolve` or `fetch` now says `applies to check only` instead of
+  naming a future version.
+- Stage row `Claims` reads `N citations, M unresolved` (was `M unsupported`).
+
+### Fixed
+- A provider answering with a non-JSON body (a bot wall or maintenance page under a
+  200) escaped `resolve.py` as a raw `JSONDecodeError` traceback. Every provider body
+  is decoded in one place and that failure is a `ProviderError`, so `resolve`, `fetch`
+  and `check` report `UNVERIFIED (provider unavailable)` and never read it as evidence
+  the work does not exist.
+- The §7.1 prompt said `blocked this request (HTTP 200)` for the empty-body bot wall;
+  it now says `answered without readable text (HTTP 200)`.
+- `resolve` on the CLI leaked one HTTP client per invocation; `commands.resolve_reference`
+  owns and closes it.
+- A retraction check every provider failed was recorded as "not retracted". It now
+  raises, is reported as `retraction check unavailable` and counted in the stage
+  summary (`1 unavailable`), and nothing is cached, so the next run asks again
+  (product rule 2). Found alongside it: the fetching stage replaced a source's notes
+  instead of appending, which would have dropped that very note.
+- Versions in the cache file compared as strings (`"10" < "9"`); compared as integers now.
+
+### Known issues
+- `api_calls` in the footer counts LLM calls only (none yet), so a cold run that spent
+  30 s on Crossref and Semantic Scholar still prints `0 API calls`; the provider lookups
+  are shown on the stage lines instead.
+- A cached resolution reprints the notes it was stored with (for example an `openalex
+  unavailable (HTTP 429)` from the day it was resolved) as if they were current.
+- `proofpath check -` names its SARIF artifact `-`; give the draft a file name when the
+  log is meant for a viewer.
+- Author-year pairing matches the first author and the exact year; `(Lindqvist, 2019)`
+  against "Okafor, C. and Lindqvist, S." and a year off by one are reported unresolved,
+  and two surnames sharing a last word (`Berg` / `van der Berg`) are reported ambiguous
+  rather than guessed. `Smith 2020` with no comma is not a marker; `(WHO, 2020)` does
+  not pair with "World Health Organization".
+- An unnumbered two-column bibliography is cut at line breaks (RoBERTa: 103 entries
+  for ~50), so author-year pairing over such a list resolves few items. The largest
+  open item for author-year pairing.
+- The headless bibliography fallback takes only the *last* run of numbered paragraphs
+  (AlphaFold: 17 of 84 entries) and drops the prose printed before a block's first entry.
+- A fabricated `First Last and First Last. Title. Venue.` with no year between the
+  names and the title reaches `UNVERIFIED (not in bibliographic indexes)`, not `GHOST`
+  (the price of the rescue above; rule 3 outranks recall). A proceedings-*volume*
+  record can accept a fabricated paper cited into that volume as `RESOLVED (low
+  confidence)` — the ghost set's one fabricated acceptance (0.9 %).
+- A cold run of a long bibliography is still serial through the providers (129
+  references: minutes); only the TUI's concurrent scheduler runs them in parallel.
+- In the TUI the `loading models …` note is drawn after the `Verifying` row it precedes
+  (it is emitted inside that stage).
+- A `#n` run reference in the TUI is a click target that folds its block, not a
+  hyperlink (spec §13.1); a run number has no address to open.
+- A provider body that is valid JSON but not an object (a bare list or string) still
+  escapes `resolve` as an error, exit 2, rather than being reported as `UNVERIFIED
+  (provider unavailable)`.
+- `/quit` in the TUI waits for an in-flight mirrored `/fetch` to finish before the
+  app exits; a `/check` run is cancelled, a `/fetch` is not.
+
+## [0.1.0] - 2026-09-12
+
+First working release: `proofpath check` verifies a document's citations end to end
+and writes a report that states its own coverage.
+
+### Added
+- `proofpath check TARGET` as the v0.1 surface: compiler-style diagnostics, a
+  markdown `report.md`, `--format json`, `-q`, and exit codes `0` / `1` / `2`.
+- Confidence tiers calibrated on SciFact dev rather than chosen by hand
+  (`decide=0.45`, `medium=0.457948`, `high=0.99933`;
+  `docs/eval/2026-09-12-tiers.md`), shipped as `pipeline.DEFAULT_THRESHOLDS`.
+- Nine live user-like runs recorded in `docs/eval/2026-09-12-v0.1-live.md`, and
+  `scripts/zero_network_check.py`, which guards both HTTP clients and re-verifies a
+  document: with the network gone the fetch ladder makes no attempt at all, because
+  resolution produces no identifiers to fetch with.
+- Config and permissions module: `config.toml` under the platform config dir,
+  `proofpath config` / `proofpath config set permissions.<key>`, and the rule that an
+  `ask` permission without a TTY resolves to `deny` and is reported (spec §7.1).
+- Core types (`Verdict` cannot be `SUPPORTED`/`REFUTED` without a passage), device
+  selection (CUDA → CoreML → CPU), sentence retrieval over `fastembed` embeddings
+  scanned with numpy, ONNX NLI entailment on `cross-encoder/nli-deberta-v3-base`, and the aggregation
+  pipeline.
+- Reference resolution (spec §8): Crossref + Semantic Scholar first, then arXiv,
+  Open Library and OpenAlex before any ghost call; identity decided only by
+  field agreement against the raw string; DOI / arXiv id resolved directly;
+  retraction check via Crossref's Retraction Watch data and OpenAlex; new state
+  `UNVERIFIED (not in bibliographic indexes)`; per-host throttling, `Retry-After`,
+  OpenAlex daily-budget handling. `proofpath resolve "<reference>"` on the CLI.
+  Hand-built ghost set (106 real, 100 fabricated, 20 mutated) and
+  `scripts/eval_ghosts.py`: false-ghost rate 0 %.
+- Persistent cache: one plain SQLite file (`sources`, `raw_text` with 7-day TTL,
+  `chunks` with float32 embeddings, `verdicts`), a schema `CHECK` that refuses an
+  asserted verdict without a passage, and `proofpath cache` / `cache path` / `ls` /
+  `show` / `clear [--expired]`.
+- Numeric claim layer (spec §10): percentages, factors and unit counts with
+  direction are compared before NLI; an unambiguous contradiction is refuted by
+  rule with both figures named (`Verdict.reason`). Conservative by design: one
+  comparable figure on each side, change never against level.
+- Judge settings (`[judge]` in config, Groq default) with `proofpath config check`
+  and `proofpath config set judge.<key>`; API key resolved from the environment or
+  `.env`, never stored or printed. `.env.example` added.
+- SciFact loader pinned to the AI2 tarball by sha256, evaluation metrics, and
+  `scripts/eval_scifact.py`. First measured result: dev accuracy 0.606 vs 0.406
+  trivial baseline (`docs/eval/2026-09-11-scifact-dev.md`).
+
+- Fetch ladder (spec §7): `httpx` → `curl_cffi` TLS impersonation → browser engine
+  behind the §7.1 consent prompt → Wayback Machine; `robots.txt` via `protego`;
+  content type from headers; per-host throttling and backoff shared in `polite.py`;
+  fetched text cached with the 7-day TTL. Distinct honesty states for blocked,
+  blocked-by-robots, browser-not-permitted, unreachable, provider-unavailable and
+  network-denied — never collapsed.
+- Consent gate for the ~280 MB browser engine: asks at most once per run, never
+  without a TTY, `always`/`never` persist to config, installs with `pip` (or `uv`)
+  and `scrapling install`, and reports how many sources were skipped.
+- Open-access chain: Semantic Scholar → Crossref TDM links → Unpaywall (only with a
+  contact address) → Europe PMC → arXiv → landing page → abstract (OpenAlex last);
+  abstract-only results labelled `LOW CONFIDENCE (abstract only)`. DataCite arXiv
+  DOIs resolve straight to the arXiv PDF. `proofpath fetch <url|doi|arXiv id>`.
+- `scripts/eval_coverage.py`: measured 72 % full text / 18 % abstract / 10 % none
+  on 50 DOIs (`docs/eval/2026-09-11-coverage.md`).
+
+- Document ingest and claim extraction (spec §9): `document.py` value types with
+  page/line locators; `ingest.py` for PDF (pymupdf blocks, per-page line numbers,
+  running header/footer removal, superscript citations), docx (paragraphs and tables),
+  markdown and plain text, with the bibliography kept as raw strings, every
+  unparseable page reported as a `PageError` and a page holding an image and no text
+  reported as a scan rather than passed on as an empty page; `claims.py` pairs numeric
+  markers (`[12]`, `[12,15]`, `[12-15]`) with their sentence, applies the
+  `PARAGRAPH-SCOPED` rule, reports author-year markers as `UNSUPPORTED CITATION STYLE`
+  and any number the bibliography does not print as unresolved. Hand-built pairing set
+  (61 passages, rate 0.98) and `scripts/eval_pairing.py`; four real documents (three
+  PDFs and one extracted text) measured in `docs/eval/2026-09-11-pairing.md`.
+
+- `proofpath check` (spec §9, §13.2, §15): one `verify()` entry point built as
+  `prepare()` (parse, claims, resolve, retractions, fetch) and `decide_all()` (retrieval,
+  numeric rule, NLI, cached per source and claim); compiler-style diagnostics with the
+  quoted passage, the confidence tier and the exact honesty state; a coverage block in
+  every run and a "coverage is weak" line when a quarter or more of the sources could
+  not be read; `report.md` written by default, `--format json`, `-q`, `--out`, `check -`
+  for stdin, a real Ctrl-C that keeps what was decided (exit 2). A second run of the
+  same document re-decides nothing: chunks and verdicts come from the cache and neither
+  model scores again, although the models are still loaded and reference resolution and
+  the retraction check still go to the network. Cache schema v2: chunks carry the text
+  digest they were cut from and a source's verdicts are dropped when its text changes.
+  `permissions.network = deny` now also skips reference resolution and the retraction
+  check, each reported as not attempted.
+
+### Changed
+- CLI surface (spec §13.3): `permissions` and `judge` groups replaced by `config`
+  (`config` / `show` / `path` / `set SECTION.KEY VALUE` / `check`); global
+  `--no-color` and `-q`; one `ui.py` layer over `rich` owns every colour and the
+  10-column key/value layout; `resolve --format json`; "provider unavailable"
+  exits `1` (a finding), no longer `2`. A resolved but retracted reference and a
+  URL that was reached but yielded no text (`reached but no text extracted`) are
+  findings too (`1`). `skipped N source(s)` counts sources, not the URLs tried for
+  them; `--format json` adds `browser.skipped_urls`. `permissions.network` binds
+  the open-access providers as well as the ladder, and `ask` without a TTY is
+  `deny`, reported.
+- Retrieval no longer depends on `sqlite-vec`: a numpy cosine scan is faster at
+  every measured scale and the plain SQLite file opens in any GUI.
+- Spec: `PARAGRAPH-SCOPED` and `UNSUPPORTED CITATION STYLE` states, three-tier
+  confidence display, 7-day raw-text cache TTL, v0.1 limited to numeric citation
+  markers.
+
+### Fixed (found by the release's own live runs, `docs/eval/2026-09-12-v0.1-live.md`)
+- A real reference whose author list carries a surname particle (`van der Walt`) was
+  called a `GHOST REFERENCE` — product rule 3. Reference resolution now understands
+  particles (including glued `al-`/`el-` forms), scores the title against every
+  title-like segment, and accepts a DOI whose record agrees on first author and year
+  as `RESOLVED (low confidence)` instead of a ghost. The ghost set grew to 258 rows
+  (`docs/eval/2026-09-12-ghosts.md`): false-ghost rate 0.0 %, fabricated recall 100 %.
+- A fabricated reference in a numbered bibliography was reported as
+  `UNVERIFIED (not in bibliographic indexes)` instead of a ghost, because the printed
+  marker (`[7] `) blinded the "is this even a paper" check. The marker is stripped once
+  at the resolver's entry, guarded so it can never remove a year, an identifier or a
+  title that begins with a number.
+- A markdown draft piped through `proofpath -q check -` lost its `## References`.
+- The consent-gated browser step (fetch-ladder step 3) could not install itself
+  (`scrapling` has no `__main__`); it now runs the package's own CLI after `pip`/`uv`,
+  and a paywalled Cell landing page was read through it live (3,632 words).
+- A piped run occasionally aborted with exit `134` from ONNX runtime teardown after
+  printing a complete report; the engine now releases both models explicitly and the
+  CLI flushes its streams before exiting (20 of 20 consecutive runs exit `1`).
+- A document with citation markers but no detected bibliography printed a clean-looking
+  `0 %` coverage block; it now says `no bibliography was found; N citation markers could
+  not be checked`.
+
+### Known issues
+- Reference resolution and the retraction check are not cached, so a warm re-run is
+  still a network run: a 68-reference PDF takes about 13 minutes cold and 3.7 minutes
+  cached, a 129-reference one 24 minutes cold. Concurrent resolution and a resolution
+  cache are v0.2 work.
+- Two-author lists written `First Last and First Last` resolve to `AMBIGUOUS`, not
+  `RESOLVED`; an arXiv id whose record agrees on author and year is not yet rescued the
+  way a DOI is.
+- `browser.is_installed()` checks that the packages import, not that a browser binary
+  exists; a half-installed environment skips the installer and fails inside the browser
+  fetch (reported as an unverified source, never a crash).
+
+## [0.0.1] - 2026-09-10
+
+First release. The verification pipeline is not implemented; this reserves the name
+and establishes the interface, packaging and CI that later phases build on.
+
+### Added
+- Design specification with measured source-access data (spec §6), the fetch ladder
+  and its permission model (§7), and corrected reference resolution (§8).
+- Phased implementation plan, ordered by risk retired rather than user-visible
+  progress.
+- `proofpath` command. A bare invocation is a first-class entry point rather than a
+  help screen, which is where the TUI will attach. Exit codes are fixed: `0` clean,
+  `1` findings, `2` the run itself failed.
+- Cross-platform CI on Linux, macOS and Windows, and PyPI publishing through trusted
+  publishing rather than a stored API token.

proofpath-0.2.0/PKG-INFO

@@ -0,0 +1,275 @@
+Metadata-Version: 2.5
+Name: proofpath
+Version: 0.2.0
+Summary: Check whether the sources behind a claim actually say what the claim says.
+Project-URL: Homepage, https://github.com/Yigtwxx/proofpath
+Project-URL: Repository, https://github.com/Yigtwxx/proofpath
+Project-URL: Issues, https://github.com/Yigtwxx/proofpath/issues
+Project-URL: Changelog, https://github.com/Yigtwxx/proofpath/blob/main/CHANGELOG.md
+Author: Yigit Erdogan
+License-Expression: MIT
+License-File: LICENSE
+Keywords: citation,cli,entailment,fact-checking,nli,rag,research-integrity,retrieval-augmented-generation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: curl-cffi>=0.16.1
+Requires-Dist: fastembed>=0.5
+Requires-Dist: httpx>=0.28
+Requires-Dist: huggingface-hub>=0.30
+Requires-Dist: numpy>=1.26
+Requires-Dist: onnxruntime>=1.20
+Requires-Dist: orjson>=3.11
+Requires-Dist: platformdirs>=4.3
+Requires-Dist: protego>=0.3
+Requires-Dist: pymupdf>=1.25
+Requires-Dist: python-docx>=1.1
+Requires-Dist: rich>=13
+Requires-Dist: scrapling>=0.4.15
+Requires-Dist: textual>=1.0
+Requires-Dist: tokenizers>=0.20
+Requires-Dist: tomli>=2.0; python_version < '3.11'
+Requires-Dist: typer>=0.15
+Provides-Extra: browser
+Requires-Dist: scrapling[fetchers]>=0.4.15; extra == 'browser'
+Provides-Extra: dev
+Requires-Dist: jsonschema>=4.23; extra == 'dev'
+Requires-Dist: mypy>=1.14; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest-cov>=6.0; extra == 'dev'
+Requires-Dist: pytest>=8.3; extra == 'dev'
+Requires-Dist: respx>=0.22; extra == 'dev'
+Requires-Dist: ruff>=0.9; extra == 'dev'
+Provides-Extra: gpu
+Requires-Dist: sentence-transformers>=3.3; extra == 'gpu'
+Requires-Dist: torch>=2.5; extra == 'gpu'
+Description-Content-Type: text/markdown
+
+# proofpath
+
+> Don't guess. Show the evidence.
+
+`proofpath` checks whether the sources behind a claim actually say what the claim
+says. Point it at a paper or a draft and it verifies every citation on three levels,
+then shows you the passage behind each verdict.
+
+1. **Does the source exist?** — Crossref, Semantic Scholar, arXiv, Open Library, OpenAlex.
+2. **Is it still valid?** — Crossref's Retraction Watch data, and OpenAlex.
+3. **Does it support the claim?** — retrieval and entailment against the source text,
+   with a numeric rule that runs before the model.
+
+Most tools stop at step 1. Step 3 is the point.
+
+Runs locally and free: no API key, no Docker, no server. Windows, Linux, macOS.
+
+```bash
+uv tool install proofpath
+proofpath
+```
+
+```
+   ,_,
+  (o.o)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[PROOF]
+   " "    proofpath v0.2.0                                              academic . online . coreml
+          paste a file path, a URL, or a claim.            /help  /config  /quit
+```
+
+Bare `proofpath` opens the terminal UI ([a recorded session](docs/eval/2026-09-15-v0.2-live.md)).
+Paste a path and it runs; every one-shot verb is a slash command (`/check`, `/resolve`,
+`/fetch`, `/config`, `/cache`), runs can be started while others are in flight and
+stopped with `/cancel #n` — a stopped run keeps what it had decided — and the
+coverage footer never scrolls away. Click (or press `enter` on) a finding to read the
+whole quoted passage; `⧉` copies it; a finding's reference is a link to its source.
+When a publisher blocks the plain fetch, the permission question is asked **inline,
+under the stage that hit the wall**, with `[allow once] [always] [no] [never]`.
+
+The same engine behind a pipe or in CI:
+
+```bash
+proofpath check paper.pdf                       # report → report.md, exit 0/1/2
+proofpath check draft.md --format json | jq '.coverage'
+proofpath check draft.md --format sarif --out draft.sarif   # any SARIF 2.1.0 viewer (VS Code's SARIF Viewer, …)
+proofpath -q check - < draft.md                 # stdin; findings and coverage only
+```
+
+`--format sarif` writes a SARIF 2.1.0 log: one result per finding on its line, the
+quoted passage in the message, the exact honesty state in `properties`, and the run's
+coverage in the run's properties, so a log opened without the terminal still says how
+much was read. Exit codes are the interface: `0` clean, `1` findings (every
+`UNVERIFIED` and `LOW CONFIDENCE` counts), `2` the run itself failed — no text
+parsing needed to gate a job. Piped or in CI there is **no prompt**: an `ask`
+permission is treated as `deny` and reported.
+
+The install carries **no browser engine**. When a publisher blocks the plain fetch,
+proofpath asks once before downloading one — about 280 MB, into its own environment
+and the shared browser cache, never system-wide — and remembers the answer.
+`proofpath config set permissions.install_browser never` stops it asking at all.
+
+## Honesty
+
+**Most verdicts are "not enough information", and that is the honest answer.** On
+SciFact dev only **135 of 340** claim–source pairs score above the decision threshold
+at all; the other 60 % are `NEI` rather than guessed. Full text is openly reachable
+for well under half of published citations, so a real document loses more on top.
+
+**Every report states its own coverage** (and adds a warning line when it is thin):
+
+```
+7 refs: 1 ghost, 2 unsupported, 4 ok
+fulltext   72%
+abstract   14%
+unverified 14%
+```
+
+When a quarter or more of the sources could not be read, a further line says so:
+`coverage is weak: unread sources may hold more, so this is a lower bound`. When the
+browser step was not permitted, a line counts the sources it cost:
+`skipped N source(s) because the browser was not permitted`.
+
+**No verdict without its passage.** `SUPPORTED` and `REFUTED` cannot exist without
+the quoted sentence they rest on — the cache schema itself refuses to store one.
+
+**The confidence tiers are measured, not chosen** — read off a sweep on SciFact dev
+([details](docs/eval/2026-09-12-tiers.md)): `decide = 0.45`, `medium = 0.457948`,
+`high = 0.99933`. Two caveats belong next to those numbers. `high` was fitted on the
+same split it is reported on: an in-sample point estimate over **21 verdicts** (at
+least 18 correct — roughly 0.65–0.95 at 95 % confidence), so read it as "the model
+was near-certain here", not as a guarantee of 85 % precision. And `medium` lands
+almost exactly on `decide`, so `low` is practically empty among asserted verdicts —
+the display is effectively **two tiers**: near-certain, and asserted at all.
+
+## Citations it reads
+
+Numeric markers — `[12]`, `[12,15]`, `[12-15]`, Nature-style superscripts — and, since
+v0.2, **author-year**: `(Smith et al., 2020)`, `Smith (2020)`, `(Smith, 2020; Jones,
+2019)`, `2020a`/`2020b` collisions, `ibid.` and `op. cit.`, and a mixed
+`(Smith, 2020; [12])`. Pairing rate on the hand-built author-year set: **0.940**
+(83 expectations over 55 passages, [details](docs/eval/2026-09-12-pairing-author-year.md));
+the numeric set is at 108 of 109. Still unpaired, and reported rather than guessed:
+a surname that is not the entry's *first* author, a year off by one, two surnames
+sharing a last word (`Berg` / `van der Berg`), `Smith 2020` with no comma, and an
+initialism such as `(WHO, 2020)` against "World Health Organization". A marker no
+entry matches is listed as an unresolved marker.
+
+## What the states mean
+
+Absence of evidence is never reported as evidence of absence. Each of these is a
+distinct, printed state (spec §15), never collapsed into a verdict:
+
+| State | Cause |
+|---|---|
+| `LOW CONFIDENCE (abstract only)` | full text unavailable, abstract used |
+| `UNVERIFIED (blocked)` | 403 or bot protection |
+| `UNVERIFIED (blocked, robots.txt)` | the site's `robots.txt` disallows the fetch |
+| `UNVERIFIED (blocked, browser not permitted)` | steps 1–2 blocked and the browser consent was denied or impossible |
+| `UNVERIFIED (unreachable)` | dead link, Wayback miss |
+| `UNVERIFIED (reached, no text extracted)` | 200 answered, nothing readable came back |
+| `UNVERIFIED (network not permitted)` | `permissions.network = deny` |
+| `UNVERIFIED (provider unavailable)` | API down, rate limited after backoff, or answering with a page instead of a record |
+| `UNVERIFIED (not in bibliographic indexes)` | web page, blog, report — indexes do not cover it, so absence proves nothing |
+| `AMBIGUOUS` | several plausible records, all listed |
+| `NEI` | the source was read and neither supports nor contradicts |
+| `PARAGRAPH-SCOPED` | the citation covers a paragraph; each sentence is judged separately |
+| `UNSUPPORTED CITATION STYLE` | reserved; no detected style produces it in v0.2 |
+
+An earlier build sometimes aborted with `134` after printing a complete report (ONNX
+runtime teardown); fixed in v0.1 — 20 of 20 piped runs exit `1`
+([live runs](docs/eval/2026-09-12-v0.1-live.md)).
+
+## Known limitations
+
+- **Superscript citations only when the PDF marks them as superscript**; a PDF that
+  draws them as ordinary digits loses them, and `km²` can be read as `[2]`.
+- **Abstract fallback**: when only an abstract is reachable the verdict is labelled
+  `LOW CONFIDENCE (abstract only)`. Three sentences is not a source.
+- **An unnumbered two-column bibliography is cut at line breaks** (RoBERTa's ACL
+  list: 103 entries for ~50, half of them with no author), so author-year pairing over
+  such a list resolves few items — 15 claims where a rejoined list would give 65. The
+  largest open item for author-year pairing.
+- **A reference list without a `References` heading** is read by its shape in paged
+  formats — the last contiguous run of numbered paragraphs — which recovers only that
+  last block (AlphaFold: 17 of 84 entries) and drops the prose printed before a
+  block's first entry.
+- **The 0 % false-ghost rate is measured on 274 hand-built references**, and rule 3
+  is paid for in recall: a fabricated `First Last and First Last. Title. Venue.` with
+  no year between the names and the title is reported as `not in bibliographic
+  indexes`, not as a ghost, and a proceedings-*volume* record can accept a fabricated
+  paper cited into that volume as `RESOLVED (low confidence)` — the set's one
+  fabricated acceptance (0.9 %) ([details](docs/eval/2026-09-12-ghosts.md)).
+- **A cold run of a long bibliography is still serial** through the providers (129
+  references: minutes). Only the TUI runs several documents' network stages at once.
+- **A refused or failed browser install is reported, not hidden**: the source is
+  `UNVERIFIED (blocked, browser not permitted)` with the install log, never silently
+  counted as unreachable.
+- **Coverage is not perfectly reproducible**: two runs minutes apart can read a
+  different number of sources, depending on which providers answered.
+- The LLM judge and `--summarize` arrive in **v0.3**.
+
+## Speed
+
+Apple Silicon Mac, models already downloaded:
+
+| document | first run | cached re-run |
+|---|---|---|
+| 1-page markdown draft, 7 references | 79.3 s (v0.1 cold) | **1.35 s** ([v0.2](docs/eval/2026-09-15-v0.2-live.md)) — 15.8 s in v0.1 |
+| 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s ([v0.1](docs/eval/2026-09-12-v0.1-live.md)) |
+
+The **first ever** run also downloads about 250 MB of ONNX models. Since v0.2 a
+cached re-run asks the network for nothing: reference resolution and the retraction
+check are cached (resolutions 30 days; a retraction hit 30 days, a miss 7 days), the
+fetched text for 7 days, and chunks and verdicts for as long as the text is unchanged. The models are still loaded, and any source whose text
+has expired or was never read is fetched again — the PDF re-run above (v0.1) still
+spent 22 s fetching and re-scored 3 of 102 claims.
+
+## Looking inside the cache
+
+Everything proofpath fetches, embeds and decides lands in one plain SQLite file:
+
+```bash
+proofpath cache            # where it is, what it holds, how many lookups it remembers
+proofpath cache ls         # sources, chunk/verdict counts, text expiry
+proofpath cache show <id>  # one source's chunks and verdicts
+proofpath cache clear --expired
+```
+
+Open `proofpath cache path` in [DB Browser for SQLite](https://sqlitebrowser.org/),
+TablePlus or DBeaver — plain tables, no extension. Raw publisher text expires after
+7 days; verdicts keep the passage they quote; a provider outage is never stored.
+`proofpath resolve REF` and `proofpath fetch URL|DOI` run either half on its own.
+
+## Optional LLM judge — arrives in v0.3
+
+Everything above runs locally. An LLM is used only at the end, as an opt-in second
+opinion on low-confidence verdicts — it never sees a source document and cannot
+change a verdict. The settings exist today (`proofpath config check` proves a key
+works, default Groq); `check --judge` and `--summarize` are wired in v0.3. The key
+comes from the environment or a `.env` file, never from config, and is never printed.
+
+## Measured
+
+| What | Set | Result |
+|---|---|---|
+| Retrieval + entailment | SciFact dev, 340 pairs | 0.609 accuracy, 0.597 macro-F1, against a 0.406 trivial baseline ([details](docs/eval/2026-09-12-scifact-dev.md)) |
+| Reference resolution | hand-built ghost set, 274 references | 0.0 % false-ghost, 99.1 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) |
+| Source access | 50 DOIs | 72 % full text, 18 % abstract only, 10 % nothing ([details](docs/eval/2026-09-11-coverage.md)) — a real biomedical paper in the live runs reached 33 % full text |
+| Citation pairing, numeric | 61 hand-built passages | 0.99 ([details](docs/eval/2026-09-11-pairing.md)) |
+| Citation pairing, author-year | 55 hand-built passages, 83 expectations | 0.940 ([details](docs/eval/2026-09-12-pairing-author-year.md)) |
+
+Published SciFact results sit around 70–75 F1, not 95. Nothing is tuned on a test
+split, and no number is quoted without the run that produced it.
+
+- [Design specification](docs/superpowers/specs/2026-09-10-proofpath-design.md) — what it does and the measurements behind each decision
+- [Open items](docs/superpowers/OPEN-ITEMS.md) — what is unresolved, and what has not been verified yet · [Changelog](CHANGELOG.md)
+
+## License
+
+MIT