paperlint 3.1.1 → 3.1.2

package/skills/verify-citations/SKILL.md

@@ -4,7 +4,7 @@ description: Verify every citation is a real work with correct metadata, and tha
 allowed-tools: [Read, Write, Edit, Grep, Glob, Bash, WebSearch, WebFetch, Agent]
 ---
 
-<!-- vigiles:sha256:9973bd8fef7e0927 compiled from skills/verify-citations/SKILL.md.spec.ts -->
+<!-- vigiles:sha256:3a7e10662d0084c4 compiled from skills/verify-citations/SKILL.md.spec.ts -->
 
 # verify-citations — every cite real, the delta explicit
 
@@ -133,17 +133,26 @@ node scripts/verify-cites.mjs cites.json --offline   # no network (everything de
 ```
 It prints per-cite `{id, verdict, reason, matched_db, matched_title?, flags?}` + a summary, and **exits 1 iff
 any `false` (fabrication)** — `unresolvable` alone is advisory and does NOT fail the gate, matching the
-narrowed-false philosophy. API responses are cached to `scripts/.cite-cache.json` (gitignored) so re-runs are
-cheap and deterministic.
+narrowed-false philosophy. API responses are cached to `~/.cache/paperlint/cite-cache.json`
+(`VERIFY_CITES_CACHE` overrides) so re-runs are cheap and deterministic — outside the installed package,
+which `npm ci` wipes.
 
-## 5a. Both checks run on every `paperlint build`
+## 5a. Both checks run on every `paperlint build` — from a committed cache
 
 `npx paperlint build` runs `verify-cites` and `bib-authors` over the paper's bibliography after the PDF is
 built and records the verdicts, with the SHA-256 of the bibliography it checked, in
 `<paper>/_build/references.json`. `paperlint lint` reads that record offline: `paper/cite-exists` and
 `paper/author-list` report a failing entry on its own line, `paper/refs-fresh` says when the bibliography
 changed since, and `paper/refs-checked` warns when nothing was recorded or the build had no network. The
-step never fails the build. Run the scripts by hand (below) to read a single verdict in full.
+step never fails the build.
+
+What the services ANSWERED is kept in `<paper>/repro/references-cache.json` — **commit it**. A build asks
+only the questions that file cannot answer (a new or edited entry, or an answer older than 30 days), so an
+unchanged bibliography builds with no network at all, in CI too, until its answers age out. It holds responses, never verdicts: the verdicts are derived again on
+every build. A service that refuses (429, HTML instead of JSON, a timeout) is not asked again for the rest of the
+build (the requests already in flight still land); the entries it would have answered say `not asked: <service>: <reason>`, and DBLP's become
+`authors: unchecked`. Answers are asked again after 30 days; to refresh one sooner, delete its entry, or delete the file. Details:
+`docs/references.md` in the package. Run the scripts by hand (below) to read a single verdict in full.
 
 ## 5b. The author-list gate — `scripts/bib-authors.mjs`
 
@@ -179,11 +188,18 @@ difference is real and not a metadata glitch. Entries that themselves declare a
 scope (a preprint entry is allowed to carry preprint metadata).
 
 ⚠️ **Discovery credit, and why it is NOT a dependency.** The class was found by running **rebiber**
-(`yuchenlin/rebiber`), which rewrites bib entries into their DBLP records. We deliberately do not depend on
-it: its install fights modern setuptools (the `bibtexparser` wheel dies with `AttributeError:
-install_layout`), and its *output* replaces our entries with DBLP's very long official booktitles — wrong for
-a page-limited paper. We needed the **comparison**, not the rewrite, and that is one fetch with zero deps.
-Note also that live DBLP beat rebiber's bundled dump: `raji2021benchmark` was found only by the live query.
+(`yuchenlin/rebiber`), which rewrites bib entries into their DBLP records. Re-measured 2026-09-27 on rebiber
+1.4.0 (GitHub commit `b917e36`, the pin in `render-paper/checkers-requirements.txt`; the PyPI package is
+a 2021 release upstream disowns):
+- **Install: no longer broken, but heavy.** It installs in 36 s in a clean venv (bibtexparser 2.0.1 wheel,
+  rebiber built from source) — and carries **409 MB** of bundled venue data.
+- **Rewrites: now optional.** `--dry-run --report` previews and leaves the file untouched; `-s True` shortens
+  venues (NAACL 2019 becomes `Proc. of NAACL-HLT` instead of a 160-character booktitle).
+- **Bundled data still misses what live DBLP finds.** `raji2021benchmark` stayed an arXiv entry, local
+  index and `--live-lookup` alike, as it did in August.
+
+So the reason not to depend on it is now scope, not breakage: we need the **comparison** of an author list
+against the published record, not a rewrite of the entry, and that is one DBLP query with zero dependencies.
 
 🔴 **Two design decisions that are load-bearing, both learned by measurement the same day:**
 1. **Compare surname SEQUENCES, nothing else.** A first cut reported 13 differences of which **11 were

package/skills/verify-citations/SKILL.md.spec.ts

@@ -149,17 +149,26 @@ node scripts/verify-cites.mjs cites.json --offline   # no network (everything de
 \`\`\`
 It prints per-cite \`{id, verdict, reason, matched_db, matched_title?, flags?}\` + a summary, and **exits 1 iff
 any \`false\` (fabrication)** — \`unresolvable\` alone is advisory and does NOT fail the gate, matching the
-narrowed-false philosophy. API responses are cached to \`scripts/.cite-cache.json\` (gitignored) so re-runs are
-cheap and deterministic.
+narrowed-false philosophy. API responses are cached to \`~/.cache/paperlint/cite-cache.json\`
+(\`VERIFY_CITES_CACHE\` overrides) so re-runs are cheap and deterministic — outside the installed package,
+which \`npm ci\` wipes.
 
-## 5a. Both checks run on every \`paperlint build\`
+## 5a. Both checks run on every \`paperlint build\` — from a committed cache
 
 \`npx paperlint build\` runs \`verify-cites\` and \`bib-authors\` over the paper's bibliography after the PDF is
 built and records the verdicts, with the SHA-256 of the bibliography it checked, in
 \`<paper>/_build/references.json\`. \`paperlint lint\` reads that record offline: \`paper/cite-exists\` and
 \`paper/author-list\` report a failing entry on its own line, \`paper/refs-fresh\` says when the bibliography
 changed since, and \`paper/refs-checked\` warns when nothing was recorded or the build had no network. The
-step never fails the build. Run the scripts by hand (below) to read a single verdict in full.
+step never fails the build.
+
+What the services ANSWERED is kept in \`<paper>/repro/references-cache.json\` — **commit it**. A build asks
+only the questions that file cannot answer (a new or edited entry, or an answer older than 30 days), so an
+unchanged bibliography builds with no network at all, in CI too, until its answers age out. It holds responses, never verdicts: the verdicts are derived again on
+every build. A service that refuses (429, HTML instead of JSON, a timeout) is not asked again for the rest of the
+build (the requests already in flight still land); the entries it would have answered say \`not asked: <service>: <reason>\`, and DBLP's become
+\`authors: unchecked\`. Answers are asked again after 30 days; to refresh one sooner, delete its entry, or delete the file. Details:
+\`docs/references.md\` in the package. Run the scripts by hand (below) to read a single verdict in full.
 
 ## 5b. The author-list gate — \`scripts/bib-authors.mjs\`
 
@@ -195,11 +204,18 @@ difference is real and not a metadata glitch. Entries that themselves declare a
 scope (a preprint entry is allowed to carry preprint metadata).
 
 ⚠️ **Discovery credit, and why it is NOT a dependency.** The class was found by running **rebiber**
-(\`yuchenlin/rebiber\`), which rewrites bib entries into their DBLP records. We deliberately do not depend on
-it: its install fights modern setuptools (the \`bibtexparser\` wheel dies with \`AttributeError:
-install_layout\`), and its *output* replaces our entries with DBLP's very long official booktitles — wrong for
-a page-limited paper. We needed the **comparison**, not the rewrite, and that is one fetch with zero deps.
-Note also that live DBLP beat rebiber's bundled dump: \`raji2021benchmark\` was found only by the live query.
+(\`yuchenlin/rebiber\`), which rewrites bib entries into their DBLP records. Re-measured 2026-09-27 on rebiber
+1.4.0 (GitHub commit \`b917e36\`, the pin in \`render-paper/checkers-requirements.txt\`; the PyPI package is
+a 2021 release upstream disowns):
+- **Install: no longer broken, but heavy.** It installs in 36 s in a clean venv (bibtexparser 2.0.1 wheel,
+  rebiber built from source) — and carries **409 MB** of bundled venue data.
+- **Rewrites: now optional.** \`--dry-run --report\` previews and leaves the file untouched; \`-s True\` shortens
+  venues (NAACL 2019 becomes \`Proc. of NAACL-HLT\` instead of a 160-character booktitle).
+- **Bundled data still misses what live DBLP finds.** \`raji2021benchmark\` stayed an arXiv entry, local
+  index and \`--live-lookup\` alike, as it did in August.
+
+So the reason not to depend on it is now scope, not breakage: we need the **comparison** of an author list
+against the published record, not a rewrite of the entry, and that is one DBLP query with zero dependencies.
 
 🔴 **Two design decisions that are load-bearing, both learned by measurement the same day:**
 1. **Compare surname SEQUENCES, nothing else.** A first cut reported 13 differences of which **11 were

package/skills/verify-citations/scripts/__snapshots__/verify-cites.net.test.mjs.snap

@@ -34,6 +34,9 @@ exports[`CVEs: found by count, found by list, not found, NVD down, and a malform
       "id": "CVE-2024-0004",
       "matched_db": null,
       "reason": "all databases unreachable (network/rate-limit) — degraded to unresolvable, never to false",
+      "refused": [
+        "nvd: http 500",
+      ],
       "verdict": "unresolvable",
     },
     {
@@ -62,6 +65,10 @@ exports[`S2 and arXiv unreachable for an arXiv id 1`] = `
     "id": "x",
     "matched_db": null,
     "reason": "all databases unreachable (network/rate-limit) — degraded to unresolvable, never to false",
+    "refused": [
+      "semantic_scholar: http 500",
+      "arxiv: http 500",
+    ],
     "verdict": "unresolvable",
   },
 }
@@ -71,9 +78,6 @@ exports[`a DOI every registry resolves to the claimed paper, and the authority k
 {
   "calls": [
     "https://api.crossref.org/works/10.1%2Fx",
-    "https://api.openalex.org/works/doi:10.1%2Fx",
-    "https://api.semanticscholar.org/graph/v1/paper/DOI%3A10.1%2Fx?fields=title,year",
-    "https://doi.org/api/handles/10.1/x",
   ],
   "result": {
     "id": "v",
@@ -112,9 +116,9 @@ exports[`a title only: every registry's search endpoint, hits and all 1`] = `
   ],
   "result": {
     "id": "t",
-    "matched_db": "crossref",
+    "matched_db": "arxiv",
     "matched_title": "Attention Is All You Need",
-    "reason": "confirmed by crossref — "Attention Is All You Need"",
+    "reason": "confirmed by arxiv — "Attention Is All You Need"",
     "verdict": "true",
   },
 }
@@ -132,6 +136,12 @@ exports[`a title search that cannot reach any registry is unresolvable 1`] = `
     "id": "d",
     "matched_db": null,
     "reason": "all databases unreachable (network/rate-limit) — degraded to unresolvable, never to false",
+    "refused": [
+      "crossref: http 500",
+      "openalex: http 502",
+      "semantic_scholar: timeout",
+      "arxiv: http 503",
+    ],
     "verdict": "unresolvable",
   },
 }
@@ -175,13 +185,16 @@ exports[`an arXiv id: Semantic Scholar by arXiv id, the arXiv API by id_list 1`]
     "https://api.crossref.org/works?query.bibliographic=Attention%20Is%20All%20You%20Need&rows=5",
     "https://api.openalex.org/works?search=Attention%20Is%20All%20You%20Need&per-page=5",
     "https://api.semanticscholar.org/graph/v1/paper/arXiv%3A1706.03762?fields=title,year",
-    "https://export.arxiv.org/api/query?id_list=1706.03762&max_results=1",
   ],
   "result": {
     "id": "a",
     "matched_db": "semantic_scholar",
     "matched_title": "Attention Is All You Need",
     "reason": "confirmed by semantic_scholar — "Attention Is All You Need"",
+    "refused": [
+      "crossref: Error: unrouted request: https://api.crossref.org/works?query.bibliographic=Attention%20Is%20All%20You%20Need&rows=5",
+      "openalex: Error: unrouted request: https://api.openalex.org/works?search=Attention%20Is%20All%20You%20Need&per-page=5",
+    ],
     "verdict": "true",
   },
 }
@@ -199,6 +212,12 @@ exports[`every registry down (429, 503, 403, a thrown error): unresolvable, neve
     "id": "u",
     "matched_db": null,
     "reason": "all databases unreachable (network/rate-limit) — degraded to unresolvable, never to false",
+    "refused": [
+      "crossref: http 429",
+      "openalex: http 503",
+      "semantic_scholar: http 403",
+      "doi_authority: TypeError: fetch failed",
+    ],
     "verdict": "unresolvable",
   },
 }
@@ -226,10 +245,9 @@ exports[`records with fields missing: no title, a subtitle, a year only in \`pub
   ],
   "result": {
     "id": "m",
-    "matched_db": "crossref",
-    "matched_title": "",
-    "reason": "confirmed by crossref",
-    "verdict": "true",
+    "matched_db": null,
+    "reason": "DOI_MISMATCH: identifier resolves via crossref to an UNRELATED paper "" (claimed "Attention Is All You Need") — DOI misdirection",
+    "verdict": "false",
   },
 }
 `;

package/skills/verify-citations/scripts/bib-authors.mjs

@@ -28,11 +28,10 @@
  *
  * ⚠️ Discovery credit and a deliberate NON-dependency: this class was found by running
  * `rebiber` (yuchenlin/rebiber), which rewrites bib entries to their DBLP records. We do
- * NOT depend on it here. Two reasons, both practical: its install needs a Python toolchain
- * that fights modern setuptools (bibtexparser wheel fails with `AttributeError:
- * install_layout`; the workaround is vendoring a tarball), and its output REPLACES entries
- * with DBLP's very long official booktitles, which is wrong for a page-limited paper. What
- * we actually needed was the comparison, and that is one HTTPS call with no dependencies.
+ * NOT depend on it here. Re-measured 2026-09-27 on 1.4.0 (GitHub b917e36): it installs cleanly
+ * now but carries 409 MB of venue data, its bundled data still misses entries live DBLP finds,
+ * and what it does is REWRITE an entry — we need the comparison of an author list against the
+ * published record, and that is one HTTPS call with no dependencies.
  *
  * 🔴 The normalisation below is load-bearing, not tidiness. A first cut of this comparison
  * reported 13 mismatches on our bibliography, of which ELEVEN were "Last, First" vs
@@ -279,7 +278,7 @@ export function surnames(authorField) {
 }
 
 // Only called for an entry that has an author (checkAuthors filters on it first).
-const truncated = (authorField) => /\band\s+others\b/i.test(authorField);
+export const truncated = (authorField) => /\band\s+others\b/i.test(authorField);
 
 /** Is the entry claiming a PUBLISHED venue (as opposed to a preprint)? */
 export function claimsPublished(e) {