paperstack-cli 0.1.0__tar.gz

paperstack_cli-0.1.0/.gitignore

@@ -0,0 +1,7 @@
+_site/
+.ruff_cache/
+.claude/
+__pycache__/
+.env
+# uv tool resolves fresh; uv run uses the PEP-723 metadata.
+uv.lock

paperstack_cli-0.1.0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: paperstack-cli
+Version: 0.1.0
+Summary: Review, inspect, and retrieve research papers from one CLI
+Project-URL: Repository, https://github.com/MilkClouds/my-paperstack
+Requires-Python: >=3.11.4
+Requires-Dist: polars>=1.43
+Requires-Dist: python-dotenv>=1.2.2
+Requires-Dist: pyyaml>=6
+Provides-Extra: pdf
+Requires-Dist: pymupdf4llm>=0.0.17; extra == 'pdf'
+Description-Content-Type: text/markdown
+
+# Critical Reads
+
+Some papers mistake their own story for evidence. Some are optimized for acceptance rather than truth. Others are
+careful, competent work on directions that do not matter. Critical Reads exists to tell them apart, preserve the
+insights that survive scrutiny, and choose research directions worth pursuing.
+
+Body-read reviews of papers, posts, and talks. Each entry lives in `entries/<citation-key>.md` and carries its identity in frontmatter.
+
+- [Collections](#collections)
+- [CLI](#cli)
+    - [DBLP index](#dblp-index)
+    - [Configuration](#configuration)
+- [Adding a review](#adding-a-review)
+    - [Getting the source in front of you](#getting-the-source-in-front-of-you)
+- [Review guide](#review-guide)
+    - [For robotics papers](#for-robotics-papers)
+
+## Collections
+
+Curated, ordered lists live in [`collections.json`](collections.json), the single source used by the viewer and validation.
+Every collection has a `published` or `draft` status. Published collections are stable reference catalogs; draft
+collections are provisional reading paths shown in a separate collapsed section. Select one to filter entries in its
+declared order.
+
+CI publishes the generated viewer after GitHub Pages is enabled with GitHub Actions as its source.
+
+## CLI
+
+Run `make serve` for the local viewer. Entry links open rendered HTML; `.md` URLs expose UTF-8 Markdown source. Install the CLI from PyPI, then authenticate with GitHub for review syncing and DBLP index downloads:
+
+```bash
+uv tool install paperstack-cli
+gh auth login
+```
+
+The command groups have separate data and authority boundaries:
+
+| Group | Data | Behavior |
+|---|---|---|
+| `review` | `entries/` review database | Read, initialize, validate, or audit authored judgments |
+| `paper` | External metadata and arXiv content | Return source records or content without choosing a citation |
+| `index` | Optional local indexes | Install and manage lookup data |
+
+Review commands:
+
+```bash
+paperstack review show black2024pi0 --brief
+paperstack review show arxiv:2410.24164 --json
+paperstack review list --quality poor --tag vla
+paperstack review search "flow matching"
+paperstack review sync --force
+paperstack review init <key> --id arxiv:NNNN.NNNNN --title "Verbatim title" --editor <name>
+paperstack review check --style
+paperstack review audit
+paperstack review citations --fetch
+```
+
+Paper commands:
+
+```bash
+paperstack paper search "Attention Is All You Need" --source dblp
+paperstack paper metadata arxiv:2106.09685
+paperstack paper metadata arxiv:2410.15549 --source semantic_scholar --json
+paperstack paper metadata doi:10.1109/CVPR.2016.90 --source crossref
+paperstack paper read arxiv:2604.23073
+paperstack paper read arxiv:2604.23073 --outline
+paperstack paper read arxiv:2604.23073 --section 6
+paperstack paper pdf arxiv:2602.09017
+```
+
+`metadata` accepts `arxiv:`, `doi:`, `dblp:`, and `openreview:` references. Semantic Scholar records include total,
+influential, and reference counts. `read` and `pdf` require an `arxiv:` reference. Metadata output keeps
+source records separate and includes provenance; it never selects or generates a citation.
+
+Options are scoped to commands that use them. `--json` is available for structured records and `--offline` for
+commands with a local or cached path. PDF conversion requires the optional dependency:
+
+```bash
+uv tool install 'paperstack-cli[pdf]'
+```
+
+From a clone, replace `paperstack` with `uv run paperstack`.
+For PDF conversion in a clone, use `uv run --extra pdf paperstack paper pdf <arxiv-ref>`.
+`review init` requires a writable review tree; `review check` requires a clone containing `scripts/build/check.sh`.
+`review citations --fetch` updates `citations.json` for every arXiv-backed entry through Semantic Scholar's batch API;
+without `--fetch`, it only removes cached records for entries that no longer exist. The static viewer exposes a minimum-citation
+filter and shows counts alongside entries. `SEMANTIC_SCHOLAR_API_KEY` is optional but raises the API rate limit.
+
+### DBLP index
+
+The optional index accelerates DBLP search and is required by `review audit`. It covers selected CS venues, not all
+of DBLP. The `2026.08` Parquet snapshot contains 285,521 structured records and is about 25 MiB. Installation is explicit:
+
+```bash
+paperstack index dblp status
+paperstack index dblp install
+paperstack index dblp update
+paperstack index dblp remove --yes
+```
+
+`install` downloads the pinned Parquet snapshot through authenticated `gh`. `update` discovers the newest
+`dblp-index-YYYY.MM` paperstack Release and replaces an older index only after its SHA-256, schema, and embedded metadata checks pass. `status` reports the version,
+coverage, size, record count, and location. `review audit` reports title and venue matches without editing reviews.
+Snapshots with mismatched metadata or fewer than 250,000 records are rejected. Installed files are immutable and
+content-addressed; an atomic pointer switch preserves the previous index if an update is interrupted.
+See [DBLP snapshot releases](docs/DBLP_RELEASES.md) for the publishing procedure.
+
+### Configuration
+
+| Variable | Purpose |
+|---|---|
+| `PAPERSTACK_DIR` | Review database to use instead of the surrounding clone or GitHub cache |
+| `PAPERSTACK_REPO` | GitHub review repository; default `MilkClouds/my-paperstack` |
+| `PAPERSTACK_TTL` | Review-cache refresh interval in seconds; default `3600` |
+| `PAPERSTACK_PAPERS_DIR` | arXiv source/PDF cache; default `${XDG_CACHE_HOME:-~/.cache}/paperstack/papers` |
+| `SEMANTIC_SCHOLAR_API_KEY` | Optional key for Semantic Scholar discovery |
+| `OPENREVIEW_ACCESS_TOKEN` | Optional OpenReview `openreview.accessToken` cookie value |
+| `XDG_CACHE_HOME` | Review and paper cache root |
+| `XDG_DATA_HOME` | DBLP index root |
+
+Configuration comes from the process environment. The CLI also loads the nearest `.env` without overriding exported
+variables, so a clone can keep local configuration in a gitignored `.env` file.
+
+Review lookup reads `$PAPERSTACK_DIR`, a surrounding clone, or a GitHub-backed cache, in that order. Scoped
+`--offline` flags serve cached review or paper data without network access. Exit codes are `0` for
+hits, `1` for no match, `2` for ambiguity, and `3` for unavailable data.
+
+## Adding a review
+
+```bash
+paperstack review init <key> --id arxiv:NNNN.NNNNN --title "Verbatim title" --editor <name>
+```
+
+This initializes an ungraded scaffold only. The review itself remains a reading and judgment task. Review files are
+stored internally as `entries/<key>.md`. See the [review guide](#review-guide) for editorial guidance.
+
+- Name files `<first-author surname><arXiv v1 year><first significant title word>`, lowercase; suffix collisions with `a`, `b`, and so on.
+- Use the established method name or full title as the `#` heading.
+- Use a registered CURIE (`arxiv:`, `doi:`, `hdl:`, `isbn:`) for `id`, or a URL when none exists.
+- `tags` are lowercase and singular. Reuse before inventing.
+- Include only verified affiliations in `lab`. Use a person's name in `editor`, or `model effort (harness)` for an agent.
+- Use CommonMark with GFM, including tables for tabular results.
+
+Run `make check`; `make style` adds prose-length warnings. The scripts require `jq` and mikefarah's `yq` (`go-yq` on conda-forge).
+
+### Getting the source in front of you
+
+The source fetcher uses `latexpand` from PATH, falling back to a vendored copy via Perl.
+
+```bash
+paperstack paper read arxiv:2604.23073            # the complete LaTeX body
+paperstack paper read arxiv:2604.23073 --outline  # the section outline
+paperstack paper read arxiv:2604.23073 --section 6
+paperstack paper pdf arxiv:cs/9301101            # when there is no LaTeX source
+yt-dlp --skip-download --write-auto-subs --sub-lang en -o talk <url>
+```
+
+- Prefer the LaTeX source; use the PDF fallback only when no source exists.
+- Cross-check malformed tables with `pdftotext -layout <pdf> -`.
+- Treat commented-out results as evidence only when their surviving values match the published version; a mismatched baseline may be an earlier run.
+- Strip timestamps and duplicate cues from video captions.
+
+## Review guide
+
+Read the body, then write the critical read, paper summary, reason to read, and one-liner, in that order.
+
+- Keep the review body within 2,500 visible non-whitespace characters. Use prose for argument, bullets for independent points, and tables for repeated comparisons.
+- Make the summary self-contained: a reader who has not read the paper should understand its core problem, approach, evidence, and findings. Choose the form that reads best; five bullets is one option, not a target.
+- In the critical read, consult prior and subsequent work as needed, then focus on what matters for interpreting, trusting, or using the paper.
+- Keep the one-liner to one sentence and `Why read it` to two. `Why read it` captures significance, originality, or practical value; use `none` when there is no reason.
+- `Quality` measures how much of the title and abstract's main claim survives the evidence:
+    - `excellent`: the claim stands and has lasting importance
+    - `good`: the claim stands
+    - `fair`: only a narrower claim stands
+    - `poor`: the claim is not established
+- Grade the advertised claim, not the narrower verdict. Use `fair` only when narrowing scope preserves the core claim; materially replacing it is `poor`.
+- For SOTA or efficiency claims, check the strongest comparable result and name the denominator.
+- If an official protocol fits the claim, an unjustified custom replacement caps `Quality` at `fair` unless matched, interpretable anchors restore comparability.
+- Disclosed weaknesses still count when the paper claims past them. Side contributions do not raise the grade.
+- On `fair` or `poor`, add `Read it anyway.` only with a checked citation count from Hugging Face or Semantic Scholar.
+
+### For robotics papers
+
+See [What Are We Actually Benchmarking in Robot Manipulation?](entries/jiang2026benchmarking.md).
+
+- A benchmark counts only when success requires the claimed capability. LIBERO-only evidence caps Quality at `poor`; so does an uncounted real-world result added to it.
+- Judge the hardest benchmark, note omissions, and account for benchmark age and test-set proximity.
+- Treat margins within evaluation noise as ties; check SOTA claims against the [VLA Evaluation Harness](https://allenai.github.io/vla-evaluation-harness/leaderboard/).
+- Recover exact values and trial counts where possible; otherwise state that they are unavailable.
+- Compare baselines only under the same training and evaluation protocol.

paperstack_cli-0.1.0/README.md

@@ -0,0 +1,190 @@
+# Critical Reads
+
+Some papers mistake their own story for evidence. Some are optimized for acceptance rather than truth. Others are
+careful, competent work on directions that do not matter. Critical Reads exists to tell them apart, preserve the
+insights that survive scrutiny, and choose research directions worth pursuing.
+
+Body-read reviews of papers, posts, and talks. Each entry lives in `entries/<citation-key>.md` and carries its identity in frontmatter.
+
+- [Collections](#collections)
+- [CLI](#cli)
+    - [DBLP index](#dblp-index)
+    - [Configuration](#configuration)
+- [Adding a review](#adding-a-review)
+    - [Getting the source in front of you](#getting-the-source-in-front-of-you)
+- [Review guide](#review-guide)
+    - [For robotics papers](#for-robotics-papers)
+
+## Collections
+
+Curated, ordered lists live in [`collections.json`](collections.json), the single source used by the viewer and validation.
+Every collection has a `published` or `draft` status. Published collections are stable reference catalogs; draft
+collections are provisional reading paths shown in a separate collapsed section. Select one to filter entries in its
+declared order.
+
+CI publishes the generated viewer after GitHub Pages is enabled with GitHub Actions as its source.
+
+## CLI
+
+Run `make serve` for the local viewer. Entry links open rendered HTML; `.md` URLs expose UTF-8 Markdown source. Install the CLI from PyPI, then authenticate with GitHub for review syncing and DBLP index downloads:
+
+```bash
+uv tool install paperstack-cli
+gh auth login
+```
+
+The command groups have separate data and authority boundaries:
+
+| Group | Data | Behavior |
+|---|---|---|
+| `review` | `entries/` review database | Read, initialize, validate, or audit authored judgments |
+| `paper` | External metadata and arXiv content | Return source records or content without choosing a citation |
+| `index` | Optional local indexes | Install and manage lookup data |
+
+Review commands:
+
+```bash
+paperstack review show black2024pi0 --brief
+paperstack review show arxiv:2410.24164 --json
+paperstack review list --quality poor --tag vla
+paperstack review search "flow matching"
+paperstack review sync --force
+paperstack review init <key> --id arxiv:NNNN.NNNNN --title "Verbatim title" --editor <name>
+paperstack review check --style
+paperstack review audit
+paperstack review citations --fetch
+```
+
+Paper commands:
+
+```bash
+paperstack paper search "Attention Is All You Need" --source dblp
+paperstack paper metadata arxiv:2106.09685
+paperstack paper metadata arxiv:2410.15549 --source semantic_scholar --json
+paperstack paper metadata doi:10.1109/CVPR.2016.90 --source crossref
+paperstack paper read arxiv:2604.23073
+paperstack paper read arxiv:2604.23073 --outline
+paperstack paper read arxiv:2604.23073 --section 6
+paperstack paper pdf arxiv:2602.09017
+```
+
+`metadata` accepts `arxiv:`, `doi:`, `dblp:`, and `openreview:` references. Semantic Scholar records include total,
+influential, and reference counts. `read` and `pdf` require an `arxiv:` reference. Metadata output keeps
+source records separate and includes provenance; it never selects or generates a citation.
+
+Options are scoped to commands that use them. `--json` is available for structured records and `--offline` for
+commands with a local or cached path. PDF conversion requires the optional dependency:
+
+```bash
+uv tool install 'paperstack-cli[pdf]'
+```
+
+From a clone, replace `paperstack` with `uv run paperstack`.
+For PDF conversion in a clone, use `uv run --extra pdf paperstack paper pdf <arxiv-ref>`.
+`review init` requires a writable review tree; `review check` requires a clone containing `scripts/build/check.sh`.
+`review citations --fetch` updates `citations.json` for every arXiv-backed entry through Semantic Scholar's batch API;
+without `--fetch`, it only removes cached records for entries that no longer exist. The static viewer exposes a minimum-citation
+filter and shows counts alongside entries. `SEMANTIC_SCHOLAR_API_KEY` is optional but raises the API rate limit.
+
+### DBLP index
+
+The optional index accelerates DBLP search and is required by `review audit`. It covers selected CS venues, not all
+of DBLP. The `2026.08` Parquet snapshot contains 285,521 structured records and is about 25 MiB. Installation is explicit:
+
+```bash
+paperstack index dblp status
+paperstack index dblp install
+paperstack index dblp update
+paperstack index dblp remove --yes
+```
+
+`install` downloads the pinned Parquet snapshot through authenticated `gh`. `update` discovers the newest
+`dblp-index-YYYY.MM` paperstack Release and replaces an older index only after its SHA-256, schema, and embedded metadata checks pass. `status` reports the version,
+coverage, size, record count, and location. `review audit` reports title and venue matches without editing reviews.
+Snapshots with mismatched metadata or fewer than 250,000 records are rejected. Installed files are immutable and
+content-addressed; an atomic pointer switch preserves the previous index if an update is interrupted.
+See [DBLP snapshot releases](docs/DBLP_RELEASES.md) for the publishing procedure.
+
+### Configuration
+
+| Variable | Purpose |
+|---|---|
+| `PAPERSTACK_DIR` | Review database to use instead of the surrounding clone or GitHub cache |
+| `PAPERSTACK_REPO` | GitHub review repository; default `MilkClouds/my-paperstack` |
+| `PAPERSTACK_TTL` | Review-cache refresh interval in seconds; default `3600` |
+| `PAPERSTACK_PAPERS_DIR` | arXiv source/PDF cache; default `${XDG_CACHE_HOME:-~/.cache}/paperstack/papers` |
+| `SEMANTIC_SCHOLAR_API_KEY` | Optional key for Semantic Scholar discovery |
+| `OPENREVIEW_ACCESS_TOKEN` | Optional OpenReview `openreview.accessToken` cookie value |
+| `XDG_CACHE_HOME` | Review and paper cache root |
+| `XDG_DATA_HOME` | DBLP index root |
+
+Configuration comes from the process environment. The CLI also loads the nearest `.env` without overriding exported
+variables, so a clone can keep local configuration in a gitignored `.env` file.
+
+Review lookup reads `$PAPERSTACK_DIR`, a surrounding clone, or a GitHub-backed cache, in that order. Scoped
+`--offline` flags serve cached review or paper data without network access. Exit codes are `0` for
+hits, `1` for no match, `2` for ambiguity, and `3` for unavailable data.
+
+## Adding a review
+
+```bash
+paperstack review init <key> --id arxiv:NNNN.NNNNN --title "Verbatim title" --editor <name>
+```
+
+This initializes an ungraded scaffold only. The review itself remains a reading and judgment task. Review files are
+stored internally as `entries/<key>.md`. See the [review guide](#review-guide) for editorial guidance.
+
+- Name files `<first-author surname><arXiv v1 year><first significant title word>`, lowercase; suffix collisions with `a`, `b`, and so on.
+- Use the established method name or full title as the `#` heading.
+- Use a registered CURIE (`arxiv:`, `doi:`, `hdl:`, `isbn:`) for `id`, or a URL when none exists.
+- `tags` are lowercase and singular. Reuse before inventing.
+- Include only verified affiliations in `lab`. Use a person's name in `editor`, or `model effort (harness)` for an agent.
+- Use CommonMark with GFM, including tables for tabular results.
+
+Run `make check`; `make style` adds prose-length warnings. The scripts require `jq` and mikefarah's `yq` (`go-yq` on conda-forge).
+
+### Getting the source in front of you
+
+The source fetcher uses `latexpand` from PATH, falling back to a vendored copy via Perl.
+
+```bash
+paperstack paper read arxiv:2604.23073            # the complete LaTeX body
+paperstack paper read arxiv:2604.23073 --outline  # the section outline
+paperstack paper read arxiv:2604.23073 --section 6
+paperstack paper pdf arxiv:cs/9301101            # when there is no LaTeX source
+yt-dlp --skip-download --write-auto-subs --sub-lang en -o talk <url>
+```
+
+- Prefer the LaTeX source; use the PDF fallback only when no source exists.
+- Cross-check malformed tables with `pdftotext -layout <pdf> -`.
+- Treat commented-out results as evidence only when their surviving values match the published version; a mismatched baseline may be an earlier run.
+- Strip timestamps and duplicate cues from video captions.
+
+## Review guide
+
+Read the body, then write the critical read, paper summary, reason to read, and one-liner, in that order.
+
+- Keep the review body within 2,500 visible non-whitespace characters. Use prose for argument, bullets for independent points, and tables for repeated comparisons.
+- Make the summary self-contained: a reader who has not read the paper should understand its core problem, approach, evidence, and findings. Choose the form that reads best; five bullets is one option, not a target.
+- In the critical read, consult prior and subsequent work as needed, then focus on what matters for interpreting, trusting, or using the paper.
+- Keep the one-liner to one sentence and `Why read it` to two. `Why read it` captures significance, originality, or practical value; use `none` when there is no reason.
+- `Quality` measures how much of the title and abstract's main claim survives the evidence:
+    - `excellent`: the claim stands and has lasting importance
+    - `good`: the claim stands
+    - `fair`: only a narrower claim stands
+    - `poor`: the claim is not established
+- Grade the advertised claim, not the narrower verdict. Use `fair` only when narrowing scope preserves the core claim; materially replacing it is `poor`.
+- For SOTA or efficiency claims, check the strongest comparable result and name the denominator.
+- If an official protocol fits the claim, an unjustified custom replacement caps `Quality` at `fair` unless matched, interpretable anchors restore comparability.
+- Disclosed weaknesses still count when the paper claims past them. Side contributions do not raise the grade.
+- On `fair` or `poor`, add `Read it anyway.` only with a checked citation count from Hugging Face or Semantic Scholar.
+
+### For robotics papers
+
+See [What Are We Actually Benchmarking in Robot Manipulation?](entries/jiang2026benchmarking.md).
+
+- A benchmark counts only when success requires the claimed capability. LIBERO-only evidence caps Quality at `poor`; so does an uncounted real-world result added to it.
+- Judge the hardest benchmark, note omissions, and account for benchmark age and test-set proximity.
+- Treat margins within evaluation noise as ties; check SOTA claims against the [VLA Evaluation Harness](https://allenai.github.io/vla-evaluation-harness/leaderboard/).
+- Recover exact values and trial counts where possible; otherwise state that they are unavailable.
+- Compare baselines only under the same training and evaluation protocol.

paperstack_cli-0.1.0/pyproject.toml

@@ -0,0 +1,41 @@
+[project]
+name = "paperstack-cli"
+version = "0.1.0"
+description = "Review, inspect, and retrieve research papers from one CLI"
+readme = "README.md"
+requires-python = ">=3.11.4" # First 3.11 release with tarfile.extractall(filter=).
+dependencies = [
+    "polars>=1.43",
+    "python-dotenv>=1.2.2",
+    "pyyaml>=6",
+]
+
+[project.optional-dependencies]
+pdf = ["pymupdf4llm>=0.0.17"]
+
+[dependency-groups]
+lint = ["ruff>=0.12"]
+test = ["pytest>=8"]
+dev = [
+    { include-group = "lint" },
+    { include-group = "test" },
+]
+
+[project.urls]
+Repository = "https://github.com/MilkClouds/my-paperstack"
+
+[project.scripts]
+paperstack = "paperstack.entrypoint:main"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/paperstack"]
+
+[tool.hatch.build.targets.sdist]
+include = ["/src/paperstack", "/README.md", "/pyproject.toml"]
+
+[tool.ruff]
+line-length = 119
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"

paperstack_cli-0.1.0/src/paperstack/__init__.py

@@ -0,0 +1 @@
+"""Paperstack command-line package."""

paperstack_cli-0.1.0/src/paperstack/citations.py

@@ -0,0 +1,97 @@
+"""Batch citation-count updates for the review corpus."""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+import urllib.parse
+from datetime import UTC, datetime
+from pathlib import Path
+
+from . import metadata
+
+S2_BATCH_API = "https://api.semanticscholar.org/graph/v1/paper/batch"
+BATCH_SIZE = 500
+
+
+def arxiv_id(raw: object) -> str | None:
+    """Return an unversioned arXiv ID from a CURIE or arxiv.org URL."""
+    value = str(raw or "").strip()
+    match = re.fullmatch(r"arxiv:([^?#]+)", value, re.IGNORECASE)
+    if not match:
+        match = re.search(r"arxiv\.org/(?:abs|pdf)/([^?#]+?)(?:\.pdf)?(?:[?#]|$)", value, re.IGNORECASE)
+    if not match:
+        return None
+    value = re.sub(r"v\d+$", "", match.group(1))
+    try:
+        return metadata.PaperRef.parse(f"arxiv:{value}").value
+    except ValueError:
+        return None
+
+
+def collect(entries: list[dict]) -> list[str]:
+    return sorted({paper_id for entry in entries if (paper_id := arxiv_id(entry.get("id")))})
+
+
+def fetch(arxiv_ids: list[str]) -> dict[str, int]:
+    """Fetch citation counts in aligned Semantic Scholar batches."""
+    counts: dict[str, int] = {}
+    headers = {"Content-Type": "application/json"}
+    if api_key := os.environ.get("SEMANTIC_SCHOLAR_API_KEY"):
+        headers["x-api-key"] = api_key
+
+    for start in range(0, len(arxiv_ids), BATCH_SIZE):
+        batch = arxiv_ids[start : start + BATCH_SIZE]
+        query = urllib.parse.urlencode({"fields": "citationCount"})
+        payload = json.dumps({"ids": [f"ARXIV:{paper_id}" for paper_id in batch]}).encode()
+        response = json.loads(metadata.request(f"{S2_BATCH_API}?{query}", headers=headers, data=payload))
+        if not isinstance(response, list):
+            detail = response.get("error") if isinstance(response, dict) else None
+            raise TypeError(f"unexpected Semantic Scholar batch response{f': {detail}' if detail else ''}")
+        for paper_id, paper in zip(batch, response, strict=True):
+            if paper is not None and not isinstance(paper, dict):
+                raise TypeError("unexpected paper in Semantic Scholar batch response")
+            if paper is not None and isinstance(paper.get("citationCount"), int):
+                counts[paper_id] = paper["citationCount"]
+    return counts
+
+
+def load(path: Path) -> dict:
+    if not path.is_file():
+        return {"last_updated": None, "papers": {}}
+    value = json.loads(path.read_text(encoding="utf-8"))
+    if not isinstance(value, dict) or not isinstance(value.get("papers"), dict):
+        raise TypeError(f"{path} must contain a papers object")
+    last_updated = value.get("last_updated")
+    if last_updated is not None and not isinstance(last_updated, str):
+        raise TypeError(f"{path} last_updated must be a string or null")
+    papers = value["papers"]
+    if any(
+        not isinstance(paper_id, str) or not isinstance(count, int) or isinstance(count, bool) or count < 0
+        for paper_id, count in papers.items()
+    ):
+        raise TypeError(f"{path} papers must map IDs to non-negative integers")
+    return {"last_updated": last_updated, "papers": papers}
+
+
+def update(root: Path, entries: list[dict], *, live: bool) -> tuple[dict, int]:
+    """Refresh or prune citation data and return the document and change count."""
+    path = root / "citations.json"
+    cached = load(path)
+    cached_papers = cached["papers"]
+    paper_ids = collect(entries)
+    fetched = fetch(paper_ids) if live else {}
+    papers = {
+        paper_id: fetched.get(paper_id, cached_papers.get(paper_id))
+        for paper_id in paper_ids
+        if paper_id in fetched or paper_id in cached_papers
+    }
+    changed = sum(cached_papers.get(paper_id) != count for paper_id, count in papers.items())
+    changed += sum(paper_id not in papers for paper_id in cached_papers)
+    document = {
+        "last_updated": datetime.now(UTC).date().isoformat() if live and fetched else cached["last_updated"],
+        "papers": papers,
+    }
+    path.write_text(json.dumps(document, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+    return document, changed