proofpath 0.1.0__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (159) hide show
  1. proofpath-0.2.0/.gitattributes +3 -0
  2. {proofpath-0.1.0 → proofpath-0.2.0}/CHANGELOG.md +122 -0
  3. {proofpath-0.1.0 → proofpath-0.2.0}/PKG-INFO +89 -38
  4. {proofpath-0.1.0 → proofpath-0.2.0}/README.md +86 -37
  5. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-11-pairing.md +65 -4
  6. proofpath-0.2.0/docs/eval/2026-09-12-ghosts.md +222 -0
  7. proofpath-0.2.0/docs/eval/2026-09-12-pairing-author-year.md +238 -0
  8. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-12-v0.1-live.md +48 -0
  9. proofpath-0.2.0/docs/eval/2026-09-15-v0.2-live.md +264 -0
  10. {proofpath-0.1.0 → proofpath-0.2.0}/docs/superpowers/OPEN-ITEMS.md +71 -12
  11. {proofpath-0.1.0 → proofpath-0.2.0}/docs/superpowers/plans/2026-09-10-proofpath-implementation-plan.md +20 -0
  12. proofpath-0.2.0/docs/superpowers/plans/2026-09-12-phases-9-10-plan.md +414 -0
  13. {proofpath-0.1.0 → proofpath-0.2.0}/docs/superpowers/specs/2026-09-10-proofpath-design.md +25 -5
  14. {proofpath-0.1.0 → proofpath-0.2.0}/pyproject.toml +6 -0
  15. {proofpath-0.1.0 → proofpath-0.2.0}/scripts/eval_pairing.py +85 -21
  16. {proofpath-0.1.0 → proofpath-0.2.0}/scripts/zero_network_check.py +10 -6
  17. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/__init__.py +1 -1
  18. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/browser.py +78 -3
  19. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/cache.py +256 -15
  20. proofpath-0.2.0/src/proofpath/claims.py +652 -0
  21. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/cli.py +122 -135
  22. proofpath-0.2.0/src/proofpath/commands.py +337 -0
  23. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/ingest.py +64 -4
  24. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/polite.py +59 -10
  25. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/report.py +29 -2
  26. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/resolve.py +157 -22
  27. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/retrieval.py +15 -1
  28. proofpath-0.2.0/src/proofpath/sarif.py +240 -0
  29. proofpath-0.2.0/src/proofpath/tui/__init__.py +7 -0
  30. proofpath-0.2.0/src/proofpath/tui/app.py +1813 -0
  31. proofpath-0.2.0/src/proofpath/tui/banner.py +89 -0
  32. proofpath-0.2.0/src/proofpath/tui/commands.py +119 -0
  33. proofpath-0.2.0/src/proofpath/tui/runs.py +459 -0
  34. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/ui.py +45 -4
  35. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/verify.py +80 -9
  36. proofpath-0.2.0/tests/conftest.py +23 -0
  37. proofpath-0.2.0/tests/data/draft-author-year.md +25 -0
  38. {proofpath-0.1.0 → proofpath-0.2.0}/tests/data/ghost_set.jsonl +16 -0
  39. proofpath-0.2.0/tests/data/pairing_author_year.jsonl +55 -0
  40. {proofpath-0.1.0 → proofpath-0.2.0}/tests/data/pairing_set.jsonl +6 -6
  41. proofpath-0.2.0/tests/data/sarif-schema-2.1.0.json +2882 -0
  42. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_browser.py +119 -0
  43. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_cache.py +143 -8
  44. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_cache_cli.py +27 -0
  45. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_check_cli.py +133 -7
  46. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_claims.py +299 -6
  47. proofpath-0.2.0/tests/test_cli.py +87 -0
  48. proofpath-0.2.0/tests/test_commands.py +411 -0
  49. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_eval_pairing.py +78 -15
  50. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_fetch_cli.py +1 -1
  51. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_ingest.py +123 -0
  52. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_polite.py +70 -0
  53. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_report.py +73 -0
  54. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_resolve.py +291 -8
  55. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_resolve_cli.py +55 -1
  56. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_retrieval.py +34 -0
  57. proofpath-0.2.0/tests/test_sarif.py +380 -0
  58. proofpath-0.2.0/tests/test_tui_app.py +1464 -0
  59. proofpath-0.2.0/tests/test_tui_banner.py +117 -0
  60. proofpath-0.2.0/tests/test_tui_commands.py +141 -0
  61. proofpath-0.2.0/tests/test_tui_runs.py +522 -0
  62. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_ui.py +42 -0
  63. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_verify.py +196 -9
  64. {proofpath-0.1.0 → proofpath-0.2.0}/uv.lock +334 -0
  65. proofpath-0.1.0/docs/eval/2026-09-12-ghosts.md +0 -121
  66. proofpath-0.1.0/src/proofpath/claims.py +0 -327
  67. proofpath-0.1.0/tests/test_cli.py +0 -20
  68. {proofpath-0.1.0 → proofpath-0.2.0}/.editorconfig +0 -0
  69. {proofpath-0.1.0 → proofpath-0.2.0}/.env.example +0 -0
  70. {proofpath-0.1.0 → proofpath-0.2.0}/.github/ISSUE_TEMPLATE/bug_report.yml +0 -0
  71. {proofpath-0.1.0 → proofpath-0.2.0}/.github/ISSUE_TEMPLATE/config.yml +0 -0
  72. {proofpath-0.1.0 → proofpath-0.2.0}/.github/ISSUE_TEMPLATE/feature_request.yml +0 -0
  73. {proofpath-0.1.0 → proofpath-0.2.0}/.github/ISSUE_TEMPLATE/wrong_verdict.yml +0 -0
  74. {proofpath-0.1.0 → proofpath-0.2.0}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
  75. {proofpath-0.1.0 → proofpath-0.2.0}/.github/workflows/ci.yml +0 -0
  76. {proofpath-0.1.0 → proofpath-0.2.0}/.github/workflows/release.yml +0 -0
  77. {proofpath-0.1.0 → proofpath-0.2.0}/.gitignore +0 -0
  78. {proofpath-0.1.0 → proofpath-0.2.0}/CLAUDE.md +0 -0
  79. {proofpath-0.1.0 → proofpath-0.2.0}/CONTRIBUTING.md +0 -0
  80. {proofpath-0.1.0 → proofpath-0.2.0}/LICENSE +0 -0
  81. {proofpath-0.1.0 → proofpath-0.2.0}/SECURITY.md +0 -0
  82. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-11-coverage.md +0 -0
  83. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-11-ghosts.md +0 -0
  84. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-11-scifact-dev.md +0 -0
  85. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-12-scifact-dev.md +0 -0
  86. {proofpath-0.1.0 → proofpath-0.2.0}/docs/eval/2026-09-12-tiers.md +0 -0
  87. {proofpath-0.1.0 → proofpath-0.2.0}/docs/research/2026-09-11-free-llm-api-tiers.md +0 -0
  88. {proofpath-0.1.0 → proofpath-0.2.0}/scripts/eval_coverage.py +0 -0
  89. {proofpath-0.1.0 → proofpath-0.2.0}/scripts/eval_ghosts.py +0 -0
  90. {proofpath-0.1.0 → proofpath-0.2.0}/scripts/eval_scifact.py +0 -0
  91. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/config.py +0 -0
  92. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/device.py +0 -0
  93. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/document.py +0 -0
  94. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/entailment.py +0 -0
  95. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/eval/__init__.py +0 -0
  96. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/eval/metrics.py +0 -0
  97. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/eval/scifact.py +0 -0
  98. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/events.py +0 -0
  99. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/fetch.py +0 -0
  100. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/judge.py +0 -0
  101. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/models.py +0 -0
  102. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/numerics.py +0 -0
  103. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/oa.py +0 -0
  104. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/paths.py +0 -0
  105. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/pipeline.py +0 -0
  106. {proofpath-0.1.0 → proofpath-0.2.0}/src/proofpath/py.typed +0 -0
  107. {proofpath-0.1.0 → proofpath-0.2.0}/tests/__init__.py +0 -0
  108. {proofpath-0.1.0 → proofpath-0.2.0}/tests/data/draft-live.md +0 -0
  109. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fakes.py +0 -0
  110. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/crossref_work.json +0 -0
  111. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/crossref_work_no_abstract.json +0 -0
  112. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/europepmc_fulltext.xml +0 -0
  113. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/europepmc_search.json +0 -0
  114. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/europepmc_search_empty.json +0 -0
  115. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/europepmc_search_not_in_epmc.json +0 -0
  116. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/openalex_work.json +0 -0
  117. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/s2_paper.json +0 -0
  118. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/s2_paper_no_abstract.json +0 -0
  119. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/unpaywall.json +0 -0
  120. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/unpaywall_closed.json +0 -0
  121. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/oa/unpaywall_landing_only.json +0 -0
  122. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/arxiv_id_roberta.xml +0 -0
  123. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/arxiv_title_fabricated.xml +0 -0
  124. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/arxiv_title_roberta.xml +0 -0
  125. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_alphafold.json +0 -0
  126. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_fabricated.json +0 -0
  127. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_numpy.json +0 -0
  128. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_rag.json +0 -0
  129. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_retracted_wakefield.json +0 -0
  130. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_roberta.json +0 -0
  131. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_work_alphafold.json +0 -0
  132. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/crossref_work_numpy.json +0 -0
  133. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openalex_alphafold.json +0 -0
  134. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openalex_fabricated.json +0 -0
  135. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openalex_rag.json +0 -0
  136. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openalex_retracted_wakefield.json +0 -0
  137. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openalex_roberta.json +0 -0
  138. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openlibrary_fabricated.json +0 -0
  139. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/openlibrary_using_language.json +0 -0
  140. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/s2_alphafold.json +0 -0
  141. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/s2_fabricated.json +0 -0
  142. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/s2_rag.json +0 -0
  143. {proofpath-0.1.0 → proofpath-0.2.0}/tests/fixtures/resolve/s2_roberta.json +0 -0
  144. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_config.py +0 -0
  145. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_config_cli.py +0 -0
  146. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_device.py +0 -0
  147. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_document.py +0 -0
  148. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_entailment.py +0 -0
  149. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_eval_coverage.py +0 -0
  150. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_eval_scifact.py +0 -0
  151. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_fetch.py +0 -0
  152. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_judge.py +0 -0
  153. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_metrics.py +0 -0
  154. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_models.py +0 -0
  155. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_numerics.py +0 -0
  156. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_oa.py +0 -0
  157. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_pipeline.py +0 -0
  158. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_scifact.py +0 -0
  159. {proofpath-0.1.0 → proofpath-0.2.0}/tests/test_version.py +0 -0
@@ -0,0 +1,3 @@
1
+ # Test fixtures whose bytes are pinned by a digest must not be CRLF-converted.
2
+ tests/data/*.json text eol=lf
3
+ tests/data/*.jsonl text eol=lf
@@ -6,6 +6,128 @@ All notable changes to this project are documented here. The format follows
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.2.0] - 2026-09-15
10
+
11
+ The interactive front-end, author-year citations, SARIF output and a cache for the
12
+ two lookups that made a warm re-run slow. Bare `proofpath` now opens the TUI; the
13
+ one-shot verbs are unchanged for CI. Live checks in `docs/eval/2026-09-15-v0.2-live.md`.
14
+
15
+ ### Added
16
+ - **TUI** (spec §13.1): bare `proofpath` opens a `textual` session — a four-line banner
17
+ with the pet, one prompt bar, and a scrolling log of run blocks. Every CLI verb is a
18
+ slash command (`/check`, `/resolve`, `/fetch`, `/config`, `/cache`; plus `/allow`,
19
+ `/cancel #n`, `/help`, `/quit`), a bare verb waits for its argument (`Esc` leaves
20
+ it), and a line that is not a command is something to check. Runs are scheduled
21
+ concurrently through one `Scheduler` (network stages in parallel under a shared
22
+ politeness limiter, the NLI model one run at a time), each run is **cancellable**
23
+ mid-flight and keeps what it had decided, the coverage footer never scrolls away,
24
+ and the §7.1 permission prompt is drawn **inline under the stage that hit the wall**
25
+ with `[allow once] [always] [no] [never]` buttons (or `/allow …`). Every log line
26
+ works by mouse and by keyboard: run headers fold, stage rows hide their summary,
27
+ findings open the full quoted passage, `⧉`/`c` copies it, a finding's reference is
28
+ an OSC 8 link to its source. The pet's eyes blink, watch a run and react to its
29
+ result — only with colour, never under `--no-color` or `-q`.
30
+ - **Author-year citations** (spec §9, §17): `(Smith et al., 2020)`, `Smith (2020)`,
31
+ `(Smith, 2020; Jones, 2019)`, `2020a`/`2020b` collisions, `ibid.` and `op. cit.`
32
+ back-references, and mixed `(Smith, 2020; [12])` all pair with their bibliography
33
+ entry. Hand-built set of 55 passages, 83 expectations, rate **0.940**
34
+ (`docs/eval/2026-09-12-pairing-author-year.md`); the numeric set is unchanged at
35
+ 108/109. A marker no entry matches is reported as `UNRESOLVED MARKER`, never guessed.
36
+ - **`check --format sarif`** (spec §13.2): the run as a SARIF 2.1.0 log for VS Code or
37
+ any SARIF 2.1.0 viewer — one result per finding on its line, the quoted passage in
38
+ the message of every asserting result, the exact honesty state in `properties`, and
39
+ the run's coverage in `runs[0].properties`. `--out FILE` writes the same document.
40
+ Validated against the schemastore schema, vendored in
41
+ `tests/data/sarif-schema-2.1.0.json`; not yet exercised against GitHub code scanning.
42
+ - **Resolution and retraction cache** (cache schema v3): `resolutions` (30-day TTL,
43
+ keyed by the marker-free folded entry) and `retractions` (30 days for a notice, 7
44
+ for its absence). Neither an `UNVERIFIED (provider unavailable)` nor a retraction
45
+ check every provider failed is ever stored. A warm re-run of the seven-reference
46
+ draft: **1.35 s** wall clock, 0 network calls (v0.1: 15.8 s). `proofpath cache`
47
+ counts both tables.
48
+ - The §7.1 aggregate line `skipped N source(s) because the browser was not permitted`
49
+ is printed by `check` too (terminal and markdown), never dropped by `-q`.
50
+ - `browser_binary_present()`: the consent gate checks for a chromium build under
51
+ `PLAYWRIGHT_BROWSERS_PATH` or the platform default, so a half-installed environment
52
+ runs the idempotent installer instead of failing inside the fetch; the consent log
53
+ gains `browser binary: found|missing`.
54
+ - `commands.py`: the shared wiring behind every mirrored verb (`resolve_reference`,
55
+ `fetch_target`, `config_*`, `cache_*`) returns result objects and never prints, so
56
+ `cli.py` and the TUI can only differ in how a result is drawn.
57
+ - Bibliography fallback for PDFs without a `References` heading (paged formats):
58
+ the last contiguous run of numbered paragraphs is read as the list. AlphaFold:
59
+ 0 → 17 references found, 97 markers now reported as unresolved instead of invisible.
60
+ - Resolver rescues: `First Last and First Last` author lists resolve instead of landing
61
+ in `AMBIGUOUS`; an arXiv id whose record agrees on author and year is accepted as
62
+ `RESOLVED (low confidence)` the way a DOI already was. Ghost set 274 rows: false-ghost
63
+ **0.0 %**, ghost recall 99.1 % (`docs/eval/2026-09-12-ghosts.md`).
64
+ - `tests/data/draft-author-year.md`, a committed author-year draft with an offline
65
+ smoke test; live: 6 citations paired, 0 unresolved.
66
+
67
+ ### Changed
68
+ - Cache schema **v3** (migrated in place from v1/v2; versions compared as integers).
69
+ - `Claims.unsupported` now means "a style this version cannot pair" and is empty by
70
+ construction; `UNSUPPORTED CITATION STYLE` is left for footnote-only and
71
+ superscript-letter styles. Markers that pair with nothing are `unresolved`.
72
+ - `resolve.looks_unindexed` reads only the initials-path author patterns: a full-name
73
+ list (`First Last and First Last`) no longer counts as "this entry printed an
74
+ author list", so a title-first book keeps its title (see Known issues for the price).
75
+ - `--format sarif` on `resolve` or `fetch` now says `applies to check only` instead of
76
+ naming a future version.
77
+ - Stage row `Claims` reads `N citations, M unresolved` (was `M unsupported`).
78
+
79
+ ### Fixed
80
+ - A provider answering with a non-JSON body (a bot wall or maintenance page under a
81
+ 200) escaped `resolve.py` as a raw `JSONDecodeError` traceback. Every provider body
82
+ is decoded in one place and that failure is a `ProviderError`, so `resolve`, `fetch`
83
+ and `check` report `UNVERIFIED (provider unavailable)` and never read it as evidence
84
+ the work does not exist.
85
+ - The §7.1 prompt said `blocked this request (HTTP 200)` for the empty-body bot wall;
86
+ it now says `answered without readable text (HTTP 200)`.
87
+ - `resolve` on the CLI leaked one HTTP client per invocation; `commands.resolve_reference`
88
+ owns and closes it.
89
+ - A retraction check every provider failed was recorded as "not retracted". It now
90
+ raises, is reported as `retraction check unavailable` and counted in the stage
91
+ summary (`1 unavailable`), and nothing is cached, so the next run asks again
92
+ (product rule 2). Found alongside it: the fetching stage replaced a source's notes
93
+ instead of appending, which would have dropped that very note.
94
+ - Versions in the cache file compared as strings (`"10" < "9"`); compared as integers now.
95
+
96
+ ### Known issues
97
+ - `api_calls` in the footer counts LLM calls only (none yet), so a cold run that spent
98
+ 30 s on Crossref and Semantic Scholar still prints `0 API calls`; the provider lookups
99
+ are shown on the stage lines instead.
100
+ - A cached resolution reprints the notes it was stored with (for example an `openalex
101
+ unavailable (HTTP 429)` from the day it was resolved) as if they were current.
102
+ - `proofpath check -` names its SARIF artifact `-`; give the draft a file name when the
103
+ log is meant for a viewer.
104
+ - Author-year pairing matches the first author and the exact year; `(Lindqvist, 2019)`
105
+ against "Okafor, C. and Lindqvist, S." and a year off by one are reported unresolved,
106
+ and two surnames sharing a last word (`Berg` / `van der Berg`) are reported ambiguous
107
+ rather than guessed. `Smith 2020` with no comma is not a marker; `(WHO, 2020)` does
108
+ not pair with "World Health Organization".
109
+ - An unnumbered two-column bibliography is cut at line breaks (RoBERTa: 103 entries
110
+ for ~50), so author-year pairing over such a list resolves few items. The largest
111
+ open item for author-year pairing.
112
+ - The headless bibliography fallback takes only the *last* run of numbered paragraphs
113
+ (AlphaFold: 17 of 84 entries) and drops the prose printed before a block's first entry.
114
+ - A fabricated `First Last and First Last. Title. Venue.` with no year between the
115
+ names and the title reaches `UNVERIFIED (not in bibliographic indexes)`, not `GHOST`
116
+ (the price of the rescue above; rule 3 outranks recall). A proceedings-*volume*
117
+ record can accept a fabricated paper cited into that volume as `RESOLVED (low
118
+ confidence)` — the ghost set's one fabricated acceptance (0.9 %).
119
+ - A cold run of a long bibliography is still serial through the providers (129
120
+ references: minutes); only the TUI's concurrent scheduler runs them in parallel.
121
+ - In the TUI the `loading models …` note is drawn after the `Verifying` row it precedes
122
+ (it is emitted inside that stage).
123
+ - A `#n` run reference in the TUI is a click target that folds its block, not a
124
+ hyperlink (spec §13.1); a run number has no address to open.
125
+ - A provider body that is valid JSON but not an object (a bare list or string) still
126
+ escapes `resolve` as an error, exit 2, rather than being reported as `UNVERIFIED
127
+ (provider unavailable)`.
128
+ - `/quit` in the TUI waits for an in-flight mirrored `/fetch` to finish before the
129
+ app exits; a `/check` run is cancelled, a `/fetch` is not.
130
+
9
131
  ## [0.1.0] - 2026-09-12
10
132
 
11
133
  First working release: `proofpath check` verifies a document's citations end to end
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: proofpath
3
- Version: 0.1.0
3
+ Version: 0.2.0
4
4
  Summary: Check whether the sources behind a claim actually say what the claim says.
5
5
  Project-URL: Homepage, https://github.com/Yigtwxx/proofpath
6
6
  Project-URL: Repository, https://github.com/Yigtwxx/proofpath
@@ -42,7 +42,9 @@ Requires-Dist: typer>=0.15
42
42
  Provides-Extra: browser
43
43
  Requires-Dist: scrapling[fetchers]>=0.4.15; extra == 'browser'
44
44
  Provides-Extra: dev
45
+ Requires-Dist: jsonschema>=4.23; extra == 'dev'
45
46
  Requires-Dist: mypy>=1.14; extra == 'dev'
47
+ Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
46
48
  Requires-Dist: pytest-cov>=6.0; extra == 'dev'
47
49
  Requires-Dist: pytest>=8.3; extra == 'dev'
48
50
  Requires-Dist: respx>=0.22; extra == 'dev'
@@ -71,12 +73,42 @@ Runs locally and free: no API key, no Docker, no server. Windows, Linux, macOS.
71
73
 
72
74
  ```bash
73
75
  uv tool install proofpath
76
+ proofpath
77
+ ```
78
+
79
+ ```
80
+ ,_,
81
+ (o.o)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[PROOF]
82
+ " " proofpath v0.2.0 academic . online . coreml
83
+ paste a file path, a URL, or a claim. /help /config /quit
84
+ ```
85
+
86
+ Bare `proofpath` opens the terminal UI ([a recorded session](docs/eval/2026-09-15-v0.2-live.md)).
87
+ Paste a path and it runs; every one-shot verb is a slash command (`/check`, `/resolve`,
88
+ `/fetch`, `/config`, `/cache`), runs can be started while others are in flight and
89
+ stopped with `/cancel #n` — a stopped run keeps what it had decided — and the
90
+ coverage footer never scrolls away. Click (or press `enter` on) a finding to read the
91
+ whole quoted passage; `⧉` copies it; a finding's reference is a link to its source.
92
+ When a publisher blocks the plain fetch, the permission question is asked **inline,
93
+ under the stage that hit the wall**, with `[allow once] [always] [no] [never]`.
94
+
95
+ The same engine behind a pipe or in CI:
74
96
 
75
- proofpath check paper.pdf # one-shot report → report.md, exit 0/1/2
97
+ ```bash
98
+ proofpath check paper.pdf # report → report.md, exit 0/1/2
76
99
  proofpath check draft.md --format json | jq '.coverage'
77
- proofpath -q check - < draft.md # stdin; findings and coverage only
100
+ proofpath check draft.md --format sarif --out draft.sarif # any SARIF 2.1.0 viewer (VS Code's SARIF Viewer, …)
101
+ proofpath -q check - < draft.md # stdin; findings and coverage only
78
102
  ```
79
103
 
104
+ `--format sarif` writes a SARIF 2.1.0 log: one result per finding on its line, the
105
+ quoted passage in the message, the exact honesty state in `properties`, and the run's
106
+ coverage in the run's properties, so a log opened without the terminal still says how
107
+ much was read. Exit codes are the interface: `0` clean, `1` findings (every
108
+ `UNVERIFIED` and `LOW CONFIDENCE` counts), `2` the run itself failed — no text
109
+ parsing needed to gate a job. Piped or in CI there is **no prompt**: an `ask`
110
+ permission is treated as `deny` and reported.
111
+
80
112
  The install carries **no browser engine**. When a publisher blocks the plain fetch,
81
113
  proofpath asks once before downloading one — about 280 MB, into its own environment
82
114
  and the shared browser cache, never system-wide — and remembers the answer.
@@ -99,7 +131,9 @@ unverified 14%
99
131
  ```
100
132
 
101
133
  When a quarter or more of the sources could not be read, a further line says so:
102
- `coverage is weak: unread sources may hold more, so this is a lower bound`.
134
+ `coverage is weak: unread sources may hold more, so this is a lower bound`. When the
135
+ browser step was not permitted, a line counts the sources it cost:
136
+ `skipped N source(s) because the browser was not permitted`.
103
137
 
104
138
  **No verdict without its passage.** `SUPPORTED` and `REFUTED` cannot exist without
105
139
  the quoted sentence they rest on — the cache schema itself refuses to store one.
@@ -113,6 +147,19 @@ was near-certain here", not as a guarantee of 85 % precision. And `medium` lands
113
147
  almost exactly on `decide`, so `low` is practically empty among asserted verdicts —
114
148
  the display is effectively **two tiers**: near-certain, and asserted at all.
115
149
 
150
+ ## Citations it reads
151
+
152
+ Numeric markers — `[12]`, `[12,15]`, `[12-15]`, Nature-style superscripts — and, since
153
+ v0.2, **author-year**: `(Smith et al., 2020)`, `Smith (2020)`, `(Smith, 2020; Jones,
154
+ 2019)`, `2020a`/`2020b` collisions, `ibid.` and `op. cit.`, and a mixed
155
+ `(Smith, 2020; [12])`. Pairing rate on the hand-built author-year set: **0.940**
156
+ (83 expectations over 55 passages, [details](docs/eval/2026-09-12-pairing-author-year.md));
157
+ the numeric set is at 108 of 109. Still unpaired, and reported rather than guessed:
158
+ a surname that is not the entry's *first* author, a year off by one, two surnames
159
+ sharing a last word (`Berg` / `van der Berg`), `Smith 2020` with no comma, and an
160
+ initialism such as `(WHO, 2020)` against "World Health Organization". A marker no
161
+ entry matches is listed as an unresolved marker.
162
+
116
163
  ## What the states mean
117
164
 
118
165
  Absence of evidence is never reported as evidence of absence. Each of these is a
@@ -127,65 +174,68 @@ distinct, printed state (spec §15), never collapsed into a verdict:
127
174
  | `UNVERIFIED (unreachable)` | dead link, Wayback miss |
128
175
  | `UNVERIFIED (reached, no text extracted)` | 200 answered, nothing readable came back |
129
176
  | `UNVERIFIED (network not permitted)` | `permissions.network = deny` |
130
- | `UNVERIFIED (provider unavailable)` | API down or rate limited after backoff |
177
+ | `UNVERIFIED (provider unavailable)` | API down, rate limited after backoff, or answering with a page instead of a record |
131
178
  | `UNVERIFIED (not in bibliographic indexes)` | web page, blog, report — indexes do not cover it, so absence proves nothing |
132
179
  | `AMBIGUOUS` | several plausible records, all listed |
133
180
  | `NEI` | the source was read and neither supports nor contradicts |
134
181
  | `PARAGRAPH-SCOPED` | the citation covers a paragraph; each sentence is judged separately |
135
- | `UNSUPPORTED CITATION STYLE` | an author–year marker; v0.1 pairs numeric markers only |
182
+ | `UNSUPPORTED CITATION STYLE` | reserved; no detected style produces it in v0.2 |
136
183
 
137
- Exit codes: `0` clean, `1` findings (every `UNVERIFIED` and `LOW CONFIDENCE` counts),
138
- `2` the run itself failedno text parsing needed to gate a CI job. An earlier build
139
- sometimes aborted with `134` after printing a complete report (ONNX runtime
140
- teardown); fixed in this release — 20 of 20 piped runs exit `1` ([live runs](docs/eval/2026-09-12-v0.1-live.md)).
184
+ An earlier build sometimes aborted with `134` after printing a complete report (ONNX
185
+ runtime teardown); fixed in v0.120 of 20 piped runs exit `1`
186
+ ([live runs](docs/eval/2026-09-12-v0.1-live.md)).
141
187
 
142
- ## What v0.1 cannot do yet
188
+ ## Known limitations
143
189
 
144
- - **Numeric citation markers only** — `[12]`, `[12,15]`, `[12-15]`; an author–year
145
- citation is listed as `UNSUPPORTED CITATION STYLE`, not judged.
146
190
  - **Superscript citations only when the PDF marks them as superscript**; a PDF that
147
191
  draws them as ordinary digits loses them, and `km²` can be read as `[2]`.
148
192
  - **Abstract fallback**: when only an abstract is reachable the verdict is labelled
149
193
  `LOW CONFIDENCE (abstract only)`. Three sentences is not a source.
150
- - **A reference list without a `References` heading is not found at all**, so its
151
- markers are reported as uncheckable rather than checked.
152
- - **Resolution and the retraction check are not cached**, so even a warm re-run goes
153
- to the network for them.
154
- - **The 0 % false-ghost rate is a property of the hand set's citation style.** Live
155
- runs hit styles that set does not contain, and misjudged real references because of
156
- it ([the live runs](docs/eval/2026-09-12-v0.1-live.md)).
194
+ - **An unnumbered two-column bibliography is cut at line breaks** (RoBERTa's ACL
195
+ list: 103 entries for ~50, half of them with no author), so author-year pairing over
196
+ such a list resolves few items 15 claims where a rejoined list would give 65. The
197
+ largest open item for author-year pairing.
198
+ - **A reference list without a `References` heading** is read by its shape in paged
199
+ formats the last contiguous run of numbered paragraphs which recovers only that
200
+ last block (AlphaFold: 17 of 84 entries) and drops the prose printed before a
201
+ block's first entry.
202
+ - **The 0 % false-ghost rate is measured on 274 hand-built references**, and rule 3
203
+ is paid for in recall: a fabricated `First Last and First Last. Title. Venue.` with
204
+ no year between the names and the title is reported as `not in bibliographic
205
+ indexes`, not as a ghost, and a proceedings-*volume* record can accept a fabricated
206
+ paper cited into that volume as `RESOLVED (low confidence)` — the set's one
207
+ fabricated acceptance (0.9 %) ([details](docs/eval/2026-09-12-ghosts.md)).
208
+ - **A cold run of a long bibliography is still serial** through the providers (129
209
+ references: minutes). Only the TUI runs several documents' network stages at once.
157
210
  - **A refused or failed browser install is reported, not hidden**: the source is
158
211
  `UNVERIFIED (blocked, browser not permitted)` with the install log, never silently
159
212
  counted as unreachable.
160
213
  - **Coverage is not perfectly reproducible**: two runs minutes apart can read a
161
214
  different number of sources, depending on which providers answered.
162
- - The TUI and `--format sarif` arrive in **v0.2**; the LLM judge and `--summarize`
163
- in **v0.3**.
215
+ - The LLM judge and `--summarize` arrive in **v0.3**.
164
216
 
165
217
  ## Speed
166
218
 
167
- Apple Silicon Mac, models already downloaded ([live runs](docs/eval/2026-09-12-v0.1-live.md)):
219
+ Apple Silicon Mac, models already downloaded:
168
220
 
169
221
  | document | first run | cached re-run |
170
222
  |---|---|---|
171
- | 1-page markdown draft, 7 references | 79.3 s | 15.8 s * |
172
- | 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s |
173
-
174
-  * the draft's cached re-run followed the earlier of the two recorded cold runs; the live doc keeps both.
223
+ | 1-page markdown draft, 7 references | 79.3 s (v0.1 cold) | **1.35 s** ([v0.2](docs/eval/2026-09-15-v0.2-live.md)) — 15.8 s in v0.1 |
224
+ | 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s ([v0.1](docs/eval/2026-09-12-v0.1-live.md)) |
175
225
 
176
- The **first ever** run also downloads about 250 MB of ONNX models. A cached re-run
177
- reads its chunks and verdicts back from the cache instead of recomputing them, but it
178
- is not a no-op: both models are still loaded, reference resolution and the retraction
179
- check still query the network, and any source whose text has expired or was never
180
- read is fetched again — the PDF re-run above still spent 22 s fetching (it went out to
181
- Wayback and arXiv) and re-scored 3 of 102 claims.
226
+ The **first ever** run also downloads about 250 MB of ONNX models. Since v0.2 a
227
+ cached re-run asks the network for nothing: reference resolution and the retraction
228
+ check are cached (resolutions 30 days; a retraction hit 30 days, a miss 7 days), the
229
+ fetched text for 7 days, and chunks and verdicts for as long as the text is unchanged. The models are still loaded, and any source whose text
230
+ has expired or was never read is fetched again — the PDF re-run above (v0.1) still
231
+ spent 22 s fetching and re-scored 3 of 102 claims.
182
232
 
183
233
  ## Looking inside the cache
184
234
 
185
235
  Everything proofpath fetches, embeds and decides lands in one plain SQLite file:
186
236
 
187
237
  ```bash
188
- proofpath cache # where it is and what it holds
238
+ proofpath cache # where it is, what it holds, how many lookups it remembers
189
239
  proofpath cache ls # sources, chunk/verdict counts, text expiry
190
240
  proofpath cache show <id> # one source's chunks and verdicts
191
241
  proofpath cache clear --expired
@@ -193,8 +243,8 @@ proofpath cache clear --expired
193
243
 
194
244
  Open `proofpath cache path` in [DB Browser for SQLite](https://sqlitebrowser.org/),
195
245
  TablePlus or DBeaver — plain tables, no extension. Raw publisher text expires after
196
- 7 days; verdicts keep the passage they quote. `proofpath resolve REF` and `proofpath
197
- fetch URL|DOI` run either half on its own.
246
+ 7 days; verdicts keep the passage they quote; a provider outage is never stored.
247
+ `proofpath resolve REF` and `proofpath fetch URL|DOI` run either half on its own.
198
248
 
199
249
  ## Optional LLM judge — arrives in v0.3
200
250
 
@@ -209,9 +259,10 @@ comes from the environment or a `.env` file, never from config, and is never pri
209
259
  | What | Set | Result |
210
260
  |---|---|---|
211
261
  | Retrieval + entailment | SciFact dev, 340 pairs | 0.609 accuracy, 0.597 macro-F1, against a 0.406 trivial baseline ([details](docs/eval/2026-09-12-scifact-dev.md)) |
212
- | Reference resolution | hand-built ghost set, 258 references | 0 % false-ghost, 100 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) — but see the live-run caveat above |
262
+ | Reference resolution | hand-built ghost set, 274 references | 0.0 % false-ghost, 99.1 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) |
213
263
  | Source access | 50 DOIs | 72 % full text, 18 % abstract only, 10 % nothing ([details](docs/eval/2026-09-11-coverage.md)) — a real biomedical paper in the live runs reached 33 % full text |
214
- | Citation pairing | 61 hand-built passages | 0.98 ([details](docs/eval/2026-09-11-pairing.md)) |
264
+ | Citation pairing, numeric | 61 hand-built passages | 0.99 ([details](docs/eval/2026-09-11-pairing.md)) |
265
+ | Citation pairing, author-year | 55 hand-built passages, 83 expectations | 0.940 ([details](docs/eval/2026-09-12-pairing-author-year.md)) |
215
266
 
216
267
  Published SciFact results sit around 70–75 F1, not 95. Nothing is tuned on a test
217
268
  split, and no number is quoted without the run that produced it.
@@ -17,12 +17,42 @@ Runs locally and free: no API key, no Docker, no server. Windows, Linux, macOS.
17
17
 
18
18
  ```bash
19
19
  uv tool install proofpath
20
+ proofpath
21
+ ```
22
+
23
+ ```
24
+ ,_,
25
+ (o.o)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~[PROOF]
26
+ " " proofpath v0.2.0 academic . online . coreml
27
+ paste a file path, a URL, or a claim. /help /config /quit
28
+ ```
29
+
30
+ Bare `proofpath` opens the terminal UI ([a recorded session](docs/eval/2026-09-15-v0.2-live.md)).
31
+ Paste a path and it runs; every one-shot verb is a slash command (`/check`, `/resolve`,
32
+ `/fetch`, `/config`, `/cache`), runs can be started while others are in flight and
33
+ stopped with `/cancel #n` — a stopped run keeps what it had decided — and the
34
+ coverage footer never scrolls away. Click (or press `enter` on) a finding to read the
35
+ whole quoted passage; `⧉` copies it; a finding's reference is a link to its source.
36
+ When a publisher blocks the plain fetch, the permission question is asked **inline,
37
+ under the stage that hit the wall**, with `[allow once] [always] [no] [never]`.
38
+
39
+ The same engine behind a pipe or in CI:
20
40
 
21
- proofpath check paper.pdf # one-shot report → report.md, exit 0/1/2
41
+ ```bash
42
+ proofpath check paper.pdf # report → report.md, exit 0/1/2
22
43
  proofpath check draft.md --format json | jq '.coverage'
23
- proofpath -q check - < draft.md # stdin; findings and coverage only
44
+ proofpath check draft.md --format sarif --out draft.sarif # any SARIF 2.1.0 viewer (VS Code's SARIF Viewer, …)
45
+ proofpath -q check - < draft.md # stdin; findings and coverage only
24
46
  ```
25
47
 
48
+ `--format sarif` writes a SARIF 2.1.0 log: one result per finding on its line, the
49
+ quoted passage in the message, the exact honesty state in `properties`, and the run's
50
+ coverage in the run's properties, so a log opened without the terminal still says how
51
+ much was read. Exit codes are the interface: `0` clean, `1` findings (every
52
+ `UNVERIFIED` and `LOW CONFIDENCE` counts), `2` the run itself failed — no text
53
+ parsing needed to gate a job. Piped or in CI there is **no prompt**: an `ask`
54
+ permission is treated as `deny` and reported.
55
+
26
56
  The install carries **no browser engine**. When a publisher blocks the plain fetch,
27
57
  proofpath asks once before downloading one — about 280 MB, into its own environment
28
58
  and the shared browser cache, never system-wide — and remembers the answer.
@@ -45,7 +75,9 @@ unverified 14%
45
75
  ```
46
76
 
47
77
  When a quarter or more of the sources could not be read, a further line says so:
48
- `coverage is weak: unread sources may hold more, so this is a lower bound`.
78
+ `coverage is weak: unread sources may hold more, so this is a lower bound`. When the
79
+ browser step was not permitted, a line counts the sources it cost:
80
+ `skipped N source(s) because the browser was not permitted`.
49
81
 
50
82
  **No verdict without its passage.** `SUPPORTED` and `REFUTED` cannot exist without
51
83
  the quoted sentence they rest on — the cache schema itself refuses to store one.
@@ -59,6 +91,19 @@ was near-certain here", not as a guarantee of 85 % precision. And `medium` lands
59
91
  almost exactly on `decide`, so `low` is practically empty among asserted verdicts —
60
92
  the display is effectively **two tiers**: near-certain, and asserted at all.
61
93
 
94
+ ## Citations it reads
95
+
96
+ Numeric markers — `[12]`, `[12,15]`, `[12-15]`, Nature-style superscripts — and, since
97
+ v0.2, **author-year**: `(Smith et al., 2020)`, `Smith (2020)`, `(Smith, 2020; Jones,
98
+ 2019)`, `2020a`/`2020b` collisions, `ibid.` and `op. cit.`, and a mixed
99
+ `(Smith, 2020; [12])`. Pairing rate on the hand-built author-year set: **0.940**
100
+ (83 expectations over 55 passages, [details](docs/eval/2026-09-12-pairing-author-year.md));
101
+ the numeric set is at 108 of 109. Still unpaired, and reported rather than guessed:
102
+ a surname that is not the entry's *first* author, a year off by one, two surnames
103
+ sharing a last word (`Berg` / `van der Berg`), `Smith 2020` with no comma, and an
104
+ initialism such as `(WHO, 2020)` against "World Health Organization". A marker no
105
+ entry matches is listed as an unresolved marker.
106
+
62
107
  ## What the states mean
63
108
 
64
109
  Absence of evidence is never reported as evidence of absence. Each of these is a
@@ -73,65 +118,68 @@ distinct, printed state (spec §15), never collapsed into a verdict:
73
118
  | `UNVERIFIED (unreachable)` | dead link, Wayback miss |
74
119
  | `UNVERIFIED (reached, no text extracted)` | 200 answered, nothing readable came back |
75
120
  | `UNVERIFIED (network not permitted)` | `permissions.network = deny` |
76
- | `UNVERIFIED (provider unavailable)` | API down or rate limited after backoff |
121
+ | `UNVERIFIED (provider unavailable)` | API down, rate limited after backoff, or answering with a page instead of a record |
77
122
  | `UNVERIFIED (not in bibliographic indexes)` | web page, blog, report — indexes do not cover it, so absence proves nothing |
78
123
  | `AMBIGUOUS` | several plausible records, all listed |
79
124
  | `NEI` | the source was read and neither supports nor contradicts |
80
125
  | `PARAGRAPH-SCOPED` | the citation covers a paragraph; each sentence is judged separately |
81
- | `UNSUPPORTED CITATION STYLE` | an author–year marker; v0.1 pairs numeric markers only |
126
+ | `UNSUPPORTED CITATION STYLE` | reserved; no detected style produces it in v0.2 |
82
127
 
83
- Exit codes: `0` clean, `1` findings (every `UNVERIFIED` and `LOW CONFIDENCE` counts),
84
- `2` the run itself failedno text parsing needed to gate a CI job. An earlier build
85
- sometimes aborted with `134` after printing a complete report (ONNX runtime
86
- teardown); fixed in this release — 20 of 20 piped runs exit `1` ([live runs](docs/eval/2026-09-12-v0.1-live.md)).
128
+ An earlier build sometimes aborted with `134` after printing a complete report (ONNX
129
+ runtime teardown); fixed in v0.120 of 20 piped runs exit `1`
130
+ ([live runs](docs/eval/2026-09-12-v0.1-live.md)).
87
131
 
88
- ## What v0.1 cannot do yet
132
+ ## Known limitations
89
133
 
90
- - **Numeric citation markers only** — `[12]`, `[12,15]`, `[12-15]`; an author–year
91
- citation is listed as `UNSUPPORTED CITATION STYLE`, not judged.
92
134
  - **Superscript citations only when the PDF marks them as superscript**; a PDF that
93
135
  draws them as ordinary digits loses them, and `km²` can be read as `[2]`.
94
136
  - **Abstract fallback**: when only an abstract is reachable the verdict is labelled
95
137
  `LOW CONFIDENCE (abstract only)`. Three sentences is not a source.
96
- - **A reference list without a `References` heading is not found at all**, so its
97
- markers are reported as uncheckable rather than checked.
98
- - **Resolution and the retraction check are not cached**, so even a warm re-run goes
99
- to the network for them.
100
- - **The 0 % false-ghost rate is a property of the hand set's citation style.** Live
101
- runs hit styles that set does not contain, and misjudged real references because of
102
- it ([the live runs](docs/eval/2026-09-12-v0.1-live.md)).
138
+ - **An unnumbered two-column bibliography is cut at line breaks** (RoBERTa's ACL
139
+ list: 103 entries for ~50, half of them with no author), so author-year pairing over
140
+ such a list resolves few items 15 claims where a rejoined list would give 65. The
141
+ largest open item for author-year pairing.
142
+ - **A reference list without a `References` heading** is read by its shape in paged
143
+ formats the last contiguous run of numbered paragraphs which recovers only that
144
+ last block (AlphaFold: 17 of 84 entries) and drops the prose printed before a
145
+ block's first entry.
146
+ - **The 0 % false-ghost rate is measured on 274 hand-built references**, and rule 3
147
+ is paid for in recall: a fabricated `First Last and First Last. Title. Venue.` with
148
+ no year between the names and the title is reported as `not in bibliographic
149
+ indexes`, not as a ghost, and a proceedings-*volume* record can accept a fabricated
150
+ paper cited into that volume as `RESOLVED (low confidence)` — the set's one
151
+ fabricated acceptance (0.9 %) ([details](docs/eval/2026-09-12-ghosts.md)).
152
+ - **A cold run of a long bibliography is still serial** through the providers (129
153
+ references: minutes). Only the TUI runs several documents' network stages at once.
103
154
  - **A refused or failed browser install is reported, not hidden**: the source is
104
155
  `UNVERIFIED (blocked, browser not permitted)` with the install log, never silently
105
156
  counted as unreachable.
106
157
  - **Coverage is not perfectly reproducible**: two runs minutes apart can read a
107
158
  different number of sources, depending on which providers answered.
108
- - The TUI and `--format sarif` arrive in **v0.2**; the LLM judge and `--summarize`
109
- in **v0.3**.
159
+ - The LLM judge and `--summarize` arrive in **v0.3**.
110
160
 
111
161
  ## Speed
112
162
 
113
- Apple Silicon Mac, models already downloaded ([live runs](docs/eval/2026-09-12-v0.1-live.md)):
163
+ Apple Silicon Mac, models already downloaded:
114
164
 
115
165
  | document | first run | cached re-run |
116
166
  |---|---|---|
117
- | 1-page markdown draft, 7 references | 79.3 s | 15.8 s * |
118
- | 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s |
119
-
120
-  * the draft's cached re-run followed the earlier of the two recorded cold runs; the live doc keeps both.
167
+ | 1-page markdown draft, 7 references | 79.3 s (v0.1 cold) | **1.35 s** ([v0.2](docs/eval/2026-09-15-v0.2-live.md)) — 15.8 s in v0.1 |
168
+ | 19-page arXiv PDF, 68 references | 12 m 57 s | 3 m 42 s ([v0.1](docs/eval/2026-09-12-v0.1-live.md)) |
121
169
 
122
- The **first ever** run also downloads about 250 MB of ONNX models. A cached re-run
123
- reads its chunks and verdicts back from the cache instead of recomputing them, but it
124
- is not a no-op: both models are still loaded, reference resolution and the retraction
125
- check still query the network, and any source whose text has expired or was never
126
- read is fetched again — the PDF re-run above still spent 22 s fetching (it went out to
127
- Wayback and arXiv) and re-scored 3 of 102 claims.
170
+ The **first ever** run also downloads about 250 MB of ONNX models. Since v0.2 a
171
+ cached re-run asks the network for nothing: reference resolution and the retraction
172
+ check are cached (resolutions 30 days; a retraction hit 30 days, a miss 7 days), the
173
+ fetched text for 7 days, and chunks and verdicts for as long as the text is unchanged. The models are still loaded, and any source whose text
174
+ has expired or was never read is fetched again — the PDF re-run above (v0.1) still
175
+ spent 22 s fetching and re-scored 3 of 102 claims.
128
176
 
129
177
  ## Looking inside the cache
130
178
 
131
179
  Everything proofpath fetches, embeds and decides lands in one plain SQLite file:
132
180
 
133
181
  ```bash
134
- proofpath cache # where it is and what it holds
182
+ proofpath cache # where it is, what it holds, how many lookups it remembers
135
183
  proofpath cache ls # sources, chunk/verdict counts, text expiry
136
184
  proofpath cache show <id> # one source's chunks and verdicts
137
185
  proofpath cache clear --expired
@@ -139,8 +187,8 @@ proofpath cache clear --expired
139
187
 
140
188
  Open `proofpath cache path` in [DB Browser for SQLite](https://sqlitebrowser.org/),
141
189
  TablePlus or DBeaver — plain tables, no extension. Raw publisher text expires after
142
- 7 days; verdicts keep the passage they quote. `proofpath resolve REF` and `proofpath
143
- fetch URL|DOI` run either half on its own.
190
+ 7 days; verdicts keep the passage they quote; a provider outage is never stored.
191
+ `proofpath resolve REF` and `proofpath fetch URL|DOI` run either half on its own.
144
192
 
145
193
  ## Optional LLM judge — arrives in v0.3
146
194
 
@@ -155,9 +203,10 @@ comes from the environment or a `.env` file, never from config, and is never pri
155
203
  | What | Set | Result |
156
204
  |---|---|---|
157
205
  | Retrieval + entailment | SciFact dev, 340 pairs | 0.609 accuracy, 0.597 macro-F1, against a 0.406 trivial baseline ([details](docs/eval/2026-09-12-scifact-dev.md)) |
158
- | Reference resolution | hand-built ghost set, 258 references | 0 % false-ghost, 100 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) — but see the live-run caveat above |
206
+ | Reference resolution | hand-built ghost set, 274 references | 0.0 % false-ghost, 99.1 % ghost recall ([details](docs/eval/2026-09-12-ghosts.md)) |
159
207
  | Source access | 50 DOIs | 72 % full text, 18 % abstract only, 10 % nothing ([details](docs/eval/2026-09-11-coverage.md)) — a real biomedical paper in the live runs reached 33 % full text |
160
- | Citation pairing | 61 hand-built passages | 0.98 ([details](docs/eval/2026-09-11-pairing.md)) |
208
+ | Citation pairing, numeric | 61 hand-built passages | 0.99 ([details](docs/eval/2026-09-11-pairing.md)) |
209
+ | Citation pairing, author-year | 55 hand-built passages, 83 expectations | 0.940 ([details](docs/eval/2026-09-12-pairing-author-year.md)) |
161
210
 
162
211
  Published SciFact results sit around 70–75 F1, not 95. Nothing is tuned on a test
163
212
  split, and no number is quoted without the run that produced it.