asta-papers 0.0.1__tar.gz

asta_papers-0.0.1/.gitignore

@@ -0,0 +1,135 @@
+# =========================
+# OS / System files
+# =========================
+.DS_Store
+.DS_Store?
+._*
+.Spotlight-V100
+.Trashes
+Thumbs.db
+ehthumbs.db
+Desktop.ini
+
+# =========================
+# Editor / IDE files
+# =========================
+.vscode/
+.idea/
+*.swp
+*.swo
+*.swn
+*~
+*.bak
+*.tmp
+*.orig
+
+# =========================
+# Logs
+# =========================
+logs/
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+
+# =========================
+# Environment variables / secrets
+# =========================
+.env
+.env.*
+!.env.example
+*.pem
+*.key
+*.crt
+
+# =========================
+# Build / dist artifacts
+# =========================
+dist/
+build/
+out/
+target/
+obj/
+
+# =========================
+# Dependency directories
+# =========================
+node_modules/
+vendor/
+.venv/
+venv/
+env/
+__pycache__/
+
+# Yarn
+**/.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/sdks
+!.yarn/versions
+.pnp.*
+
+# =========================
+# Test / coverage output
+# =========================
+coverage/
+.coverage
+.nyc_output/
+test-results/
+pytest_cache/
+
+# =========================
+# Temporary / cache files
+# =========================
+.cache/
+.tmp/
+.temp/
+*.cache
+
+# =========================
+# Archive files
+# =========================
+*.zip
+*.tar
+*.tar.gz
+*.rar
+*.7z
+
+# =========================
+# Runtime files
+# =========================
+*.pid
+*.seed
+*.pid.lock
+
+# =========================
+# Misc
+# =========================
+*.lock
+!.gitignore
+
+# =========================
+# Terraform
+# =========================
+*.tfstate
+*.tfstate.*
+.terraform/
+.terraform.lock.hcl
+infra/terraform.tfvars
+
+# =========================
+# Generated deploy files
+# =========================
+_asta_modal_*.py
+
+# =========================
+# Python packaging
+# =========================
+*.egg-info/
+
+# =========================
+# Claude Code worktrees
+# =========================
+.claude/worktrees/

asta_papers-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Allen Institute for AI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

asta_papers-0.0.1/PKG-INFO

@@ -0,0 +1,183 @@
+Metadata-Version: 2.4
+Name: asta-papers
+Version: 0.0.1
+Summary: Legal-only paper full-text retrieval and conversion. DOI/PMID/PMCID/arXiv/Corpus ID + BYO PDF/JATS → markdown with license classification.
+Project-URL: Homepage, https://github.com/allenai/asta-sdk
+Project-URL: Repository, https://github.com/allenai/asta-sdk
+Project-URL: Issues, https://github.com/allenai/asta-sdk/issues
+Project-URL: Documentation, https://github.com/allenai/asta-sdk/tree/main/src/python/asta/papers/docs
+Project-URL: Changelog, https://github.com/allenai/asta-sdk/blob/main/src/python/asta/papers/docs/CHANGELOG.md
+Author: Allen Institute for AI
+License: MIT
+License-File: LICENSE
+Keywords: arxiv,europepmc,jats,license-classification,ocr,open-access,pdf,pubmed,scientific-papers,text-mining,unpaywall
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Text Processing :: Markup :: XML
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: requests>=2.31
+Provides-Extra: all
+Requires-Dist: boto3>=1.34; extra == 'all'
+Requires-Dist: mistralai<2,>=1.5; extra == 'all'
+Requires-Dist: olmocr>=0.4; extra == 'all'
+Requires-Dist: openai>=1.40; extra == 'all'
+Requires-Dist: pillow>=10.0; extra == 'all'
+Requires-Dist: pypdf>=4.0; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: freezegun>=1.5; extra == 'dev'
+Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pytest-benchmark>=4.0; extra == 'dev'
+Requires-Dist: pytest-cov>=5.0; extra == 'dev'
+Requires-Dist: pytest-xdist>=3.6; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: responses>=0.25; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Provides-Extra: mistral
+Requires-Dist: mistralai<2,>=1.5; extra == 'mistral'
+Provides-Extra: olmocr
+Requires-Dist: olmocr>=0.4; extra == 'olmocr'
+Requires-Dist: openai>=1.40; extra == 'olmocr'
+Requires-Dist: pillow>=10.0; extra == 'olmocr'
+Requires-Dist: pypdf>=4.0; extra == 'olmocr'
+Provides-Extra: s3
+Requires-Dist: boto3>=1.34; extra == 's3'
+Description-Content-Type: text/markdown
+
+# asta-papers
+
+Legal-only paper full-text retrieval and conversion. Identifier (DOI / PMID /
+PMCID / arXiv / Semantic Scholar Corpus ID) — or BYO PDF / JATS — to markdown
+with explicit license classification.
+
+Lifts paper recovery on biomedical literature from ~22% (Mistral OCR alone)
+to ~85% using only publisher-blessed legal channels (NCBI E-utilities,
+Unpaywall, EuropePMC, bioRxiv, institutional repositories).
+
+## Install
+
+```bash
+pip install asta-papers                 # core (JATS conversion only)
+pip install 'asta-papers[mistral]'      # + Mistral OCR for PDFs
+pip install 'asta-papers[olmocr]'       # + local olmOCR for PDFs (offline)
+pip install 'asta-papers[s3]'           # + s3:// BYO support
+pip install 'asta-papers[all]'          # everything
+```
+
+## Quickstart
+
+```python
+import os
+from asta_papers import Client
+from asta_papers.converters.mistral import MistralConverter
+
+c = Client(
+    email="me@allenai.org",
+    ncbi_api_key=os.environ.get("NCBI_API_KEY"),       # optional, lifts NCBI 3→10 rps
+    converters=[MistralConverter()],                    # for PDF→markdown
+)
+
+# By identifier
+r = c.fetch(doi="10.1186/s12943-024-02093-w")
+print(r.success, r.license_class, r.markdown[:200])
+
+# Storage-tier policy
+if r.may_redistribute:                                   # CC BY / CC0 / CC BY-SA
+    save_artifact(r.bytes)
+elif r.may_use_for_tdm:                                  # TDM-permissive licenses; not BRONZE/UNKNOWN/CLOSED
+    extract_inline(r.markdown)
+
+# BYO PDF — bytes, local path, or URI
+r = c.fetch(pdf=b"%PDF-...")
+r = c.fetch(pdf="paper.pdf")
+r = c.fetch(pdf="s3://my-bucket/paper.pdf")              # requires [s3]
+
+# Batch with bounded concurrency + per-host rate limits
+results = c.fetch_many([
+    {"doi": "10.1038/foo"},
+    {"pmcid": "PMC123"},
+    {"pdf": "paper.pdf", "doi": "10.99/local"},
+])
+```
+
+## How it works
+
+A strategy ladder runs against legal aggregator APIs in order, returning the
+first successful result:
+
+1. **PMC E-utilities efetch** — JATS XML for PMC OA Subset articles
+2. **NCBI elink** — PMID → PMC self-link when S2 didn't surface it
+3. **Published-version handoff** — when input is a preprint DOI, route to
+   the published version (bioRxiv API or Crossref `relation`) so callers
+   get the most-recent public version of the paper
+4. **arXiv** — direct PDF for arXiv papers
+5. **bioRxiv / medRxiv API** — JATS XML or PDF for preprints
+6. **Unpaywall** — best legal OA URL; routes PMC URLs through efetch
+7. **Institutional repo scrape** — `hdl.handle.net`, `pure.eur.nl`, etc.
+   (now respects `robots.txt`)
+8. **EuropePMC PDF render** — text-mining-licensed PDFs for free-to-read
+   papers NCBI's OA Subset doesn't include
+
+Per-host token-bucket rate limiting honors every publisher's published quota
+exactly. arXiv at 0.33 rps (their explicit rule). NCBI at 3 rps (10 with key)
+shared across `eutils.*`, `pmc.*`, `www.*` hostnames. Independent services
+run fully in parallel.
+
+## License classification
+
+Every successful fetch carries a `LicenseClass`:
+
+```
+cc-by  cc-by-sa  cc-by-nd  cc-by-nc  cc-by-nc-sa  cc-by-nc-nd  cc0
+text-mining-only   bronze   arxiv-default   closed   unknown
+```
+
+Plus helper booleans on `FetchResult`: `may_redistribute`,
+`may_redistribute_nc`, `may_make_derivatives`, `may_train_models`,
+`may_use_for_tdm`, plus a `source_type` field (`publisher` /
+`repository` / `other`) for callers who want publisher-vs-repository
+policy without parsing strings. Storage-tier policy is a one-line check.
+
+Successful results also carry an attribution blockquote at the top of
+the markdown by default (source URL + DOI/PMCID + license + retrieval
+strategy), so the markdown is self-attributing when it travels to end
+users. Disable with `Client(include_attribution=False)`.
+
+## What's NOT here
+
+- No Sci-Hub, no archive scraping, no UA spoofing past WAFs.
+- No title-only paper search (use PaperFinder / S2 first to get an identifier).
+- No multi-tenant `Credentials` per-call object (one Client per credential set).
+- No async API (sync only in v0.1).
+
+## Configuration
+
+See `Client.__init__` and `docs/concepts.md` for the full list. Required:
+`email` (Crossref polite-pool identifier; kwarg or `ASTA_PAPERS_EMAIL` env).
+Recommended: `NCBI_API_KEY` env (free, 5-minute registration, 3.3× throughput).
+
+## Tests
+
+```bash
+pytest tests/integration -v                  # 29 real-API tests
+python tools/check_test_legitimacy.py --strict  # asserts mock-ratio < 30%
+```
+
+The full integration suite hits live upstream APIs — no mocks. Tests run in
+~90 seconds. A per-paper snapshot recovery benchmark (53 biomedical DOIs
+that fail Mistral-OCR-only retrieval; 46/53 = 87% recovered) gates
+regressions.
+
+## Design
+
+Full design at [`docs/DESIGN.md`](docs/DESIGN.md).

asta_papers-0.0.1/README.md

@@ -0,0 +1,127 @@
+# asta-papers
+
+Legal-only paper full-text retrieval and conversion. Identifier (DOI / PMID /
+PMCID / arXiv / Semantic Scholar Corpus ID) — or BYO PDF / JATS — to markdown
+with explicit license classification.
+
+Lifts paper recovery on biomedical literature from ~22% (Mistral OCR alone)
+to ~85% using only publisher-blessed legal channels (NCBI E-utilities,
+Unpaywall, EuropePMC, bioRxiv, institutional repositories).
+
+## Install
+
+```bash
+pip install asta-papers                 # core (JATS conversion only)
+pip install 'asta-papers[mistral]'      # + Mistral OCR for PDFs
+pip install 'asta-papers[olmocr]'       # + local olmOCR for PDFs (offline)
+pip install 'asta-papers[s3]'           # + s3:// BYO support
+pip install 'asta-papers[all]'          # everything
+```
+
+## Quickstart
+
+```python
+import os
+from asta_papers import Client
+from asta_papers.converters.mistral import MistralConverter
+
+c = Client(
+    email="me@allenai.org",
+    ncbi_api_key=os.environ.get("NCBI_API_KEY"),       # optional, lifts NCBI 3→10 rps
+    converters=[MistralConverter()],                    # for PDF→markdown
+)
+
+# By identifier
+r = c.fetch(doi="10.1186/s12943-024-02093-w")
+print(r.success, r.license_class, r.markdown[:200])
+
+# Storage-tier policy
+if r.may_redistribute:                                   # CC BY / CC0 / CC BY-SA
+    save_artifact(r.bytes)
+elif r.may_use_for_tdm:                                  # TDM-permissive licenses; not BRONZE/UNKNOWN/CLOSED
+    extract_inline(r.markdown)
+
+# BYO PDF — bytes, local path, or URI
+r = c.fetch(pdf=b"%PDF-...")
+r = c.fetch(pdf="paper.pdf")
+r = c.fetch(pdf="s3://my-bucket/paper.pdf")              # requires [s3]
+
+# Batch with bounded concurrency + per-host rate limits
+results = c.fetch_many([
+    {"doi": "10.1038/foo"},
+    {"pmcid": "PMC123"},
+    {"pdf": "paper.pdf", "doi": "10.99/local"},
+])
+```
+
+## How it works
+
+A strategy ladder runs against legal aggregator APIs in order, returning the
+first successful result:
+
+1. **PMC E-utilities efetch** — JATS XML for PMC OA Subset articles
+2. **NCBI elink** — PMID → PMC self-link when S2 didn't surface it
+3. **Published-version handoff** — when input is a preprint DOI, route to
+   the published version (bioRxiv API or Crossref `relation`) so callers
+   get the most-recent public version of the paper
+4. **arXiv** — direct PDF for arXiv papers
+5. **bioRxiv / medRxiv API** — JATS XML or PDF for preprints
+6. **Unpaywall** — best legal OA URL; routes PMC URLs through efetch
+7. **Institutional repo scrape** — `hdl.handle.net`, `pure.eur.nl`, etc.
+   (now respects `robots.txt`)
+8. **EuropePMC PDF render** — text-mining-licensed PDFs for free-to-read
+   papers NCBI's OA Subset doesn't include
+
+Per-host token-bucket rate limiting honors every publisher's published quota
+exactly. arXiv at 0.33 rps (their explicit rule). NCBI at 3 rps (10 with key)
+shared across `eutils.*`, `pmc.*`, `www.*` hostnames. Independent services
+run fully in parallel.
+
+## License classification
+
+Every successful fetch carries a `LicenseClass`:
+
+```
+cc-by  cc-by-sa  cc-by-nd  cc-by-nc  cc-by-nc-sa  cc-by-nc-nd  cc0
+text-mining-only   bronze   arxiv-default   closed   unknown
+```
+
+Plus helper booleans on `FetchResult`: `may_redistribute`,
+`may_redistribute_nc`, `may_make_derivatives`, `may_train_models`,
+`may_use_for_tdm`, plus a `source_type` field (`publisher` /
+`repository` / `other`) for callers who want publisher-vs-repository
+policy without parsing strings. Storage-tier policy is a one-line check.
+
+Successful results also carry an attribution blockquote at the top of
+the markdown by default (source URL + DOI/PMCID + license + retrieval
+strategy), so the markdown is self-attributing when it travels to end
+users. Disable with `Client(include_attribution=False)`.
+
+## What's NOT here
+
+- No Sci-Hub, no archive scraping, no UA spoofing past WAFs.
+- No title-only paper search (use PaperFinder / S2 first to get an identifier).
+- No multi-tenant `Credentials` per-call object (one Client per credential set).
+- No async API (sync only in v0.1).
+
+## Configuration
+
+See `Client.__init__` and `docs/concepts.md` for the full list. Required:
+`email` (Crossref polite-pool identifier; kwarg or `ASTA_PAPERS_EMAIL` env).
+Recommended: `NCBI_API_KEY` env (free, 5-minute registration, 3.3× throughput).
+
+## Tests
+
+```bash
+pytest tests/integration -v                  # 29 real-API tests
+python tools/check_test_legitimacy.py --strict  # asserts mock-ratio < 30%
+```
+
+The full integration suite hits live upstream APIs — no mocks. Tests run in
+~90 seconds. A per-paper snapshot recovery benchmark (53 biomedical DOIs
+that fail Mistral-OCR-only retrieval; 46/53 = 87% recovered) gates
+regressions.
+
+## Design
+
+Full design at [`docs/DESIGN.md`](docs/DESIGN.md).

asta_papers-0.0.1/docs/CHANGELOG.md

@@ -0,0 +1,195 @@
+# Changelog
+
+All notable changes to `asta-papers`. Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+Versioning: [Semver](https://semver.org/).
+
+## [Unreleased]
+
+### Added — license-aware fields and policies
+
+- **`LicenseClass.ARXIV_DEFAULT`**: arXiv's non-exclusive distribution
+  grant. Distinct from `UNKNOWN` — callers know they have at least the
+  arXiv default rights (academic / non-commercial redistribution). The
+  `arxiv` strategy now returns `ARXIV_DEFAULT` instead of `UNKNOWN`.
+- **`SourceType` enum** (`PUBLISHER`, `REPOSITORY`, `OTHER`) and a
+  matching `FetchResult.source_type` field. Promoted from Unpaywall's
+  `host_type` in `unpaywall`; inferred from the strategy in
+  `arxiv`/`biorxiv`/`pmc-efetch`/`europepmc-pdf`/`repo-scrape`. Lets
+  callers implement publisher-vs-repository policy without parsing
+  `notes` strings.
+- **`FetchProvenance.enriched_identifiers`**: dict of identifiers that
+  were added by the Client's enrichment step (S2 corpus_id resolve, NCBI
+  ID converter). Empty when the caller supplied all identifiers
+  themselves. Audit trail for "did this fetch hit PMC because the caller
+  asked, or because enrichment supplied a PMCID?"
+- **Attribution blockquote** auto-prepended to converted markdown so the
+  output is self-attributing when it travels to end users. Includes
+  source URL, DOI/PMCID, license class + URL, and the strategy that
+  retrieved the bytes. Controlled by a new `Client(include_attribution=True)`
+  flag (default on); pass `False` for pure-inference workflows that
+  prefer clean prose.
+- **Bronze fallback** in `unpaywall`: when Unpaywall says `is_oa=True`
+  with `host_type=publisher` but no `license` tag, the result is now
+  classified as `LicenseClass.BRONZE` (previously fell through to
+  `UNKNOWN`). A note `unpaywall:no-license-tag-classified-as-bronze` is
+  added so the audit trail records the inference.
+- **Robots.txt check** in `repo_scrape`. Before scraping an
+  institutional repo's landing page (and again for the discovered PDF
+  URL), the strategy checks the host's `robots.txt` and skips disallowed
+  paths. The check uses stdlib `urllib.robotparser`, is cached per host
+  via `ctx.api_cache`, and fails permissive (allows the scrape if
+  robots.txt is unreachable, which is the conservative-permissive
+  default — most repos don't ship robots).
+
+### Changed — strategy ladder reordered
+
+Default ladder now runs `published-version` *before* the preprint-fetching
+strategies (`arxiv`, `biorxiv`). Realises the user-facing contract "give
+me the most recent public version of the paper": when a preprint DOI
+maps to a published version with a usable license, fetch the published
+version instead of the preprint. Returns None for non-preprint DOIs and
+falls through to the preprint strategies. The `result.identifiers`
+field always reflects the actually-fetched DOI, so callers can compare
+against the request to detect handoffs.
+
+### Documented — NIH/PMC and publisher TOS posture
+
+Researched and documented the AI-training-vs-inference distinction in
+`docs/licensing.md`. Key takeaways:
+
+- PMC's E-Utilities (the `pmc-efetch` strategy) is on the explicit list
+  of permitted automated retrieval services in PMC's copyright policy —
+  not just tolerated, explicitly named.
+- PMC does NOT carve AI training out of "text mining" in its TOS.
+- The pressure to do so is coming from individual publishers (Elsevier,
+  Wiley, etc.) via website footers added since 2024 — not from PMC, NIH,
+  EuropePMC, or arXiv. Whether those footers can override CC-BY on
+  individual articles is legally contested.
+- Practical implication: when a downstream caller is **training a model**
+  (not just running inference), they should filter
+  `source_type == SourceType.REPOSITORY` to avoid the publisher-CDN
+  paths whose TOS may reserve AI rights. Library defaults remain right
+  for the inference use case.
+
+### Changed — `may_use_for_tdm` semantics tightened
+
+Previously returned `True` for any successful fetch — over-permissive
+because it equated "library may ingest" with "caller may run TDM
+downstream." Now returns `True` only for license classes that grant TDM
+explicitly (CC*, `TEXT_MINING_ONLY`, `ARXIV_DEFAULT`, etc.). Returns
+`False` for `BRONZE`, `UNKNOWN`, and `CLOSED` because TDM rights for
+those are ambiguous. Callers should pair with `may_redistribute` for
+redistribution gates; `may_use_for_tdm` is now strictly about
+*caller-side* TDM permission.
+
+### Fixed — JATS converter coverage
+
+Driven by a 26-paper diverse stress-test corpus spanning 9 publishers
+(PNAS, Cell, Lancet, JAMA, Nature, Science, PLOS, BMC, MDPI). Element-by-element
+audit found 10 categories of silently-dropped content; all closed.
+
+- **Walk `<floats-group>`** sibling of `<body>`. Recovers figures and tables
+  for MDPI, AACR, JCI, OUP, NEJM (≈ half the PMC corpus by publisher).
+- **Descend into `<p>` block children** (fig / table-wrap / disp-formula /
+  list / supplementary-material / boxed-text / def-list / ack / app /
+  media / chem-struct-wrap / sub-article / glossary). Recovers
+  Nature/BMC/OUP figures wrapped inside body paragraphs.
+- **Walk all `<back>` children**, not just `<ref-list>`. Recovers `<ack>`,
+  `<app>`, `<glossary>` (was 19 of 26 papers dropping these).
+- **Strip whitespace `<label>`** before emitting caption blocks. Lancet and
+  similar publishers emit `<label>\n</label>`, which previously produced
+  malformed `**\n: caption**` blocks that downstream regexes couldn't
+  match.
+- **Render `<def-list>`** as `**term**: definition` lines.
+- **Render `<ack>` / `<app>` / `<app-group>` / `<glossary>`** as labelled
+  sections, honouring the publisher's `<title>` if present.
+- **Nested `<list>` inside `<list-item>`** now dispatched as a block
+  (preserves nested-list structure instead of flattening).
+- **Tables now honour `<thead>` / `<tbody>` / `<tfoot>`**: thead rows
+  become the markdown header (multi-row headers preserved as pre-header
+  body rows); cells from nested tables can no longer bleed into the
+  parent table's row set.
+- **`<inline-formula>` MathML fallback**: when no `<tex-math>` is present
+  but the formula contains `<mml:math>`, render plaintext content as
+  `$mathml-text$` (or `$[math]$` if empty). Math-heavy papers no longer
+  have empty formula slots in the markdown.
+- **`<disp-formula>` always emits `$$...$$`** regardless of source
+  (tex-math / MathML / plaintext sub-sup / equation-graphic). Display
+  math is now machine-detectable in markdown without inspecting the JATS.
+- **`<supplementary-material>` and `<table-wrap-foot>`** rendered. Surfaces
+  supplementary file captions and table abbreviation glossaries needed to
+  interpret cells.
+
+### Tested
+
+- Unit suite grew from 113 → 121 tests (8 new tests in
+  `test_jats_to_md.py` lock the behaviours above).
+- Full unit suite passes; no regressions.
+- Stress-audit drop counts (papers where JATS XML had > 0 of an element
+  but markdown had 0): figures 2 → 0; tables 1 → 0; disp-formula 2 → 0;
+  inline-formula 1 → 0; ack 19 → 3 (the remaining 3 are publisher-custom
+  titles like "Sources of Funding:" — converter is correct, audit-script
+  is too narrow); def-list 3 → 0; app 3 → 0; glossary 4 → 0 (same
+  custom-title caveat as ack).
+
+### Known gaps (deferred)
+
+A separate legal review surfaced **3 P0 findings** that the *caller*
+should be aware of when implementing license-aware vending policy. See
+[`docs/licensing.md`](licensing.md#known-caveats) for the current list and
+recommended mitigations.
+
+## [0.0.1] — 2026-05-06
+
+Initial release.
+
+### Added
+
+- `Client` (sync) with `fetch(...)` and `fetch_many(...)` entry points.
+- 8-strategy ladder: PMC efetch, NCBI elink, arXiv, bioRxiv API, Unpaywall,
+  bioRxiv published-version handoff (with Crossref `relation` fallback for
+  non-bioRxiv DOIs), institutional repo scrape, EuropePMC PDF render.
+- BYO inputs via `pdf=` and `jats=` accepting bytes / `Path` / URI
+  (`file://`, `http(s)://`, `s3://`).
+- Pluggable converters: built-in `JatsConverter` (stdlib only),
+  `MistralConverter` (extra `[mistral]`), `OlmocrConverter` (extra `[olmocr]`).
+- Per-host token-bucket `RateLimiter` with hostname aliasing — independent
+  services run in parallel, same-service workers serialize at the configured
+  rps.
+- License classifier producing one of 11 `LicenseClass` values; `FetchResult`
+  helpers `may_redistribute`, `may_redistribute_nc`, `may_make_derivatives`,
+  `may_train_models`, `may_use_for_tdm`.
+- `FileCache` keyed by canonical identifier set; multi-key aliasing for
+  cross-identifier hits.
+- Identifier enrichment: NCBI ID converter (DOI ↔ PMID/PMCID) and S2 (Corpus
+  ID → others). Off by default; opt-in via `enrich_identifiers=True` (or
+  always-on for corpus-id-only inputs).
+- `tools/check_test_legitimacy.py` — fails CI if mock ratio exceeds 30% or any
+  strategy has no real-API test.
+
+### Tested
+
+- 113 unit tests (license classifier, identifier normalization, rate limiter
+  alias chain, FileCache round-trip, JATS converter, FetchResult helpers,
+  config redaction).
+- 29 real-API integration tests, zero mocks (verified by the legitimacy
+  sweep).
+- Per-paper snapshot recovery benchmark (53 biomedical DOIs that fail
+  Mistral-OCR-only retrieval): 46/53 (86.8%) recovered. Diff-based to
+  catch regressions on specific DOIs.
+- Rate-limit timing tests with real wall-clock assertions: arXiv 0.33 rps
+  strict, NCBI 3 rps shared across `eutils`/`pmc`/`www` hostnames, service
+  independence under 4-thread concurrency, 100-acquire concurrent safety.
+
+### Known caveats
+
+- `mistralai` pinned to `>=1.5,<2`; v2 reorganized imports and breaks the
+  converter.
+- `olmocr` requires LM Studio (or compatible) loaded with **8k+ context** for
+  reliable output on 2-column biomedical PDFs. Default 4k context yields
+  ~67% per-page failure rate.
+- Per-Client rate limiter — multi-instance deployments don't share quota.
+  Distributed limiter deferred to v0.2.
+- No async API. Sync only.
+- No multi-tenant `Credentials` per-call object. One Client per credential set.
+- `gs://` BYO is not implemented (placeholder enum value reserved).