open-alex-wrapper 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jose Luis Hernando
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,437 @@
+# open-alex-wrapper
+
+[![npm](https://img.shields.io/npm/v/open-alex-wrapper?color=cb3837&logo=npm)](https://www.npmjs.com/package/open-alex-wrapper)
+[![types](https://img.shields.io/badge/types-included-3178c6?logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
+[![node](https://img.shields.io/badge/node-%E2%89%A518-339933?logo=node.js&logoColor=white)](https://nodejs.org)
+[![dependencies](https://img.shields.io/badge/runtime%20deps-0-brightgreen)](package.json)
+[![tests](https://img.shields.io/badge/tests-91%20passing-brightgreen)](test)
+[![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+
+A fully-typed TypeScript SDK for the [OpenAlex](https://openalex.org) API. It covers all eight entity types, streams results past the API's paging cap, fetches pages in parallel, and prices an extraction in dollars before you run it.
+
+```ts
+import { OpenAlexClient, gt } from 'open-alex-wrapper'
+
+const client = new OpenAlexClient({ apiKey: process.env.OPENALEX_API_KEY })
+
+// 1. Know the price before you pay it
+const est = await client.works
+  .filter({ publication_year: gt(2020), 'open_access.is_oa': true })
+  .estimateCost()
+console.log(`~$${est.totalCostUsd} for ${est.matchingResults.toLocaleString()} works`)
+
+// 2. Stream every matching work, cursor paging, past the 10k cap
+for await (const work of client.works.filter({ publication_year: 2023 }).all()) {
+  console.log(work.display_name)
+}
+
+// 3. See exactly what you've spent
+console.log(client.usageReport())
+```
+
+**Jump to:** [Why not the raw API](#why-not-just-call-the-api-directly) · [Install](#install) · [Config](#configuration) · [Entities](#entities) · [Cost & budget](#cost-and-budget) · [Export](#export) · [Citation graph](#citation-graph) · [Resumable](#resumable-extractions) · [Snapshot](#snapshot-bridge-extract-all-data-at-0) · [Full text](#full-text-pdf--tei-xml) · [CLI](#cli) · [Errors](#errors) · [Performance](#performance-notes)
+
+## Why not just call the API directly?
+
+OpenAlex has a clean REST API and you can absolutely drive it with `fetch`. The question is how much of the surrounding machinery you want to write yourself, and since February 2026 that machinery includes keeping track of a bill.
+
+Here is one ordinary job, harvesting every open-access work from 2023, written both ways.
+
+**With `fetch` against the raw API:**
+
+```js
+const filter = 'publication_year:2023,open_access.is_oa:true'
+let cursor = '*'
+let spent = 0
+
+while (cursor) {
+  const url =
+    `https://api.openalex.org/works?filter=${encodeURIComponent(filter)}` +
+    `&per-page=200&cursor=${cursor}&api_key=${process.env.OPENALEX_API_KEY}`
+
+  let res
+  for (let attempt = 0; ; attempt++) {
+    res = await fetch(url)
+    if (res.status !== 429 && res.status < 500) break
+    const wait = Number(res.headers.get('retry-after') ?? 2 ** attempt)
+    await new Promise((r) => setTimeout(r, wait * 1000))
+  }
+  if (!res.ok) throw new Error(`${res.status} ${await res.text()}`)
+
+  spent += 0.0001                                    // your own running total
+  const { results, meta } = await res.json()         // results are untyped
+  for (const work of results) {
+    // ...
+  }
+  cursor = meta.next_cursor                          // crash here and you start over
+}
+```
+
+**With this SDK:**
+
+```ts
+for await (const work of client.works
+  .filter({ publication_year: 2023, 'open_access.is_oa': true })
+  .all()) {
+  // ...
+}
+```
+
+The second version also retries, throttles against your remaining budget, tracks spend from the response headers, and can checkpoint so a crash resumes instead of re-billing.
+
+### Line by line
+
+| What you want to do | With `fetch` and the raw API | With this SDK |
+| --- | --- | --- |
+| Read past the 10,000 result cap | Thread `cursor=*` and `meta.next_cursor` through your own loop | `for await (const w of q.all())` |
+| Know the price before running it | Not possible. You learn the cost from headers after you have paid | `await q.estimateCost()` |
+| Track what you have spent | Parse `x-ratelimit-*` off every response and aggregate it | `client.usageReport()` |
+| Stop before a daily cap | Catch the 402 or 429 once you have already hit it | `dailyBudgetUsd` plus adaptive throttling |
+| Write a filter | Hand-build `filter=publication_year:>2015,type:!dataset` strings | `.filter({ publication_year: gt(2015), type: not('dataset') })` |
+| Recall 206 filter field names | Keep the docs open in another tab | IDE autocomplete, per entity |
+| Look up a DOI, ORCID, ROR or ISSN | Different prefix rules for each identifier | `client.works.get('10.7717/peerj.4375')` |
+| Survive a 429 or a 5xx | Write backoff that honours `Retry-After` | Retried automatically |
+| Resume a harvest that died | Start over and pay for the same pages twice | Checkpoint resume, no double billing |
+| Read an abstract | Invert `abstract_inverted_index` yourself | `decodeAbstract(work.abstract_inverted_index)` |
+| Pull the entire dataset | A separate S3 bucket, gzip, manifest parsing | `client.snapshot.stream('works')` at $0 |
+| Get static types | There are no official ones | Typed models for all eight entities |
+
+None of this is exotic. It is the code most people write anyway, on their third project against the API, after the first two taught them why they needed it.
+
+## What you get
+
+- All eight entities, plus autocomplete, n-grams, and abstract decoding.
+- A fluent, immutable query builder with typed per-entity filter-key autocomplete.
+- Streaming auto-pagination: cursor for unbounded reads, parallel basic paging when the result set is under 10k.
+- Dollar-cost estimation and live budget tracking read from the `x-ratelimit-*` headers.
+- Budget auto-throttle and a cost-optimal query planner.
+- A snapshot bridge that extracts the full dataset from the public S3 dump at $0.
+- Resumable extractions, so a crashed harvest is never paid for twice.
+- Full-text download (PDF and TEI-XML) and citation graph helpers.
+- An `oaw` CLI, zero runtime dependencies, and JSONL / CSV / JSON export.
+
+The last six do not exist in any other OpenAlex client. [`docs/COMPETITIVE.md`](docs/COMPETITIVE.md) has the full comparison against pyalex, openalexR, and the other TS and JS libraries.
+
+## OpenAlex pricing, in brief
+
+OpenAlex moved to usage-based pricing in February 2026. An API key is now required, you get a free daily allowance ($1.00/day with a key, $0.10/day without), and every request has a price:
+
+| Request type | Price / call | Free-tier daily cap |
+| --- | --- | --- |
+| Single entity lookup (by ID, DOI, and so on) | **$0** | unlimited |
+| List / filter | **$0.0001** | 10,000 calls · 1M results |
+| Search | **$0.001** | 1,000 calls · 100k results |
+| PDF/XML full-text download | **$0.01** | 100 calls |
+
+That pricing change is why cost sits at the centre of this SDK rather than off to one side. You can estimate an extraction's dollar cost before running it, and read authoritative live spend from the `x-ratelimit-*` headers OpenAlex returns on every call. Free keys are at [openalex.org/settings/api](https://openalex.org/settings/api). The data itself is still free: [they sell services, not data](https://blog.openalex.org/openalex-api-new-features-and-usage-based-pricing/).
+
+## Install
+
+```bash
+npm install open-alex-wrapper
+```
+
+Requires Node 18+ for native `fetch`. No runtime dependencies.
+
+## Configuration
+
+```ts
+const client = new OpenAlexClient({
+  apiKey: process.env.OPENALEX_API_KEY, // or set OPENALEX_API_KEY
+  mailto: 'you@example.com',            // polite pool; or set OPENALEX_MAILTO
+  concurrency: 8,                       // max parallel requests
+  dailyBudgetUsd: 5.0,                  // optional soft cap, throws BudgetExceededError
+  throttle: true,                       // adaptive rate limiting (see Auto-throttle)
+  cache: true,                          // opt-in LRU cache for GETs
+  cacheTtlMs: 300_000,
+  timeoutMs: 60_000,
+  retry: { maxRetries: 5, baseDelayMs: 500 },
+  onBudgetWarning: ({ spentUsd, limitUsd, ratio }) => { /* ... */ },
+})
+```
+
+## Entities
+
+All eight entity types share one interface:
+
+```
+client.works        client.authors      client.sources      client.institutions
+client.topics       client.keywords     client.publishers   client.funders
+```
+
+### Fetch one
+
+`get()` takes an OpenAlex ID, DOI, ORCID, ROR, ISSN, Wikidata QID, PMID, MAG id, or a full URL, and normalizes all of them:
+
+```ts
+await client.works.get('10.7717/peerj.4375')                 // DOI
+await client.authors.get('0000-0002-1298-3089')              // ORCID
+await client.institutions.get('https://ror.org/03yrm5c26')   // ROR URL
+await client.sources.get('issn:2167-8359')                   // ISSN
+await client.works.get('W2741809807', { select: ['id', 'title'] })
+```
+
+### Fetch many (batched, parallel)
+
+```ts
+// Batches into `ids.openalex` OR-filters (default 50/call), run in parallel.
+const works = await client.works.getMany(['W2741809807', 'W2755950973', /* ... */])
+```
+
+### Query, filter, sort, select
+
+The builder is immutable, so queries branch and get reused safely:
+
+```ts
+import { gt, lt, not, or } from 'open-alex-wrapper'
+
+const q = client.works
+  .filter({
+    publication_year: gt(2015),        // >2015
+    'open_access.is_oa': true,
+    type: not('dataset'),              // !dataset
+    language: or('en', 'es'),          // en OR es  (arrays also work)
+  })
+  .searchField('title', 'genomics')
+  .sort({ cited_by_count: 'desc' })
+  .select(['id', 'display_name', 'cited_by_count'])
+
+const page = await q.get()             // one page
+const first = await q.first()          // first match or null
+const total = await q.count()          // just meta.count (cheap)
+```
+
+**Typed filter keys.** `.filter({ … })` autocompletes each entity's real filter keys in your IDE (206 for works, 43 for authors, and so on), generated from the live OpenAlex API via `npm run gen:filters`. Unknown keys still pass through, so a new OpenAlex field never breaks your build.
+
+```ts
+client.works.filter({ publication_year: 2020 })   // 'publication_year' autocompletes
+client.authors.filter({ h_index: gt(40) })        // per-entity keys
+```
+
+### Iterate everything
+
+```ts
+// Default: cursor paging. Unbounded, memory-safe, sequential.
+for await (const work of q.all()) { /* ... */ }
+
+// Fast path for ≤10k results: parallel basic paging (throttled by `concurrency`).
+for await (const work of q.all({ parallel: true })) { /* ... */ }
+
+// Page objects instead of items:
+for await (const { meta, results } of q.paginate()) { /* ... */ }
+
+// Collect (bounded):
+const top100 = await q.perPage(100).toArray({ maxResults: 100 })
+```
+
+### Aggregate, sample, autocomplete
+
+```ts
+await client.works.filter({ publication_year: 2023 }).groups('open_access.is_oa')
+await client.works.sample(50, /* seed */ 42).get()
+await client.autocomplete('einst')                      // cross-entity typeahead
+await client.authors.autocomplete('einstein')           // scoped
+```
+
+## Cost and budget
+
+```ts
+// Estimate before extracting
+const est = await client.works.search('crispr').estimateCost()
+// → { matchingResults, pageCalls, pagesCostUsd, probeCostUsd, totalCostUsd, cappedByBasicPaging }
+
+// Live, authoritative usage (from OpenAlex response headers)
+const snap = client.usageSnapshot()
+// → { spentUsd, remainingUsd, dailyLimitUsd, creditsRemaining, resetSeconds, byType, ... }
+
+console.log(client.usageReport())
+// OpenAlex usage report
+//   API key:            yes
+//   Total API calls:    12
+//   Spend today:        $0.001200 (reported by OpenAlex)
+//   Daily limit:        $1.0000
+//   Remaining:          $0.998800
+//   ...
+```
+
+Set `dailyBudgetUsd` and any call, or any estimated extraction, that would cross it throws `BudgetExceededError` before spending.
+
+### Auto-throttle
+
+Adaptive rate limiting paces requests and slows down as the budget runs low, so long extractions ease into the limit instead of hitting a 429 or 402:
+
+```ts
+const client = new OpenAlexClient({
+  throttle: { maxRequestsPerSecond: 10, minRemainingUsd: 0.05 },  // or just `throttle: true`
+})
+```
+
+Spacing doubles under 25% remaining budget and quadruples under 10%. `minRemainingUsd` is a hard stop that throws `BudgetExceededError`.
+
+### Cost-optimal planner
+
+The planner counts once, then picks the cheapest way to read the rest:
+
+```mermaid
+flowchart TD
+  A["your query"] --> B["one count probe"]
+  B --> C{"estimated bill ≥ $1<br/>and the snapshot covers it?"}
+  C -->|yes| S["free S3 snapshot · $0"]
+  C -->|no| D{"results ≤ 10,000?"}
+  D -->|yes| P["parallel basic paging · fastest"]
+  D -->|no| U["cursor streaming · unbounded"]
+```
+
+```ts
+const plan = await client.planExtraction(client.works.filter({ 'open_access.is_oa': true }))
+// { matchingResults, recommendedStrategy: 'parallel-basic'|'cursor'|'snapshot',
+//   recommendedPerPage, estimate, useSnapshot, snapshotAwsCommand?, rationale }
+
+// Or just run the optimal plan (auto per-page, plus parallel or cursor):
+for await (const work of client.works.filter({ publication_year: 2023 }).streamOptimized()) { /* ... */ }
+```
+
+## Export
+
+Stream results straight to disk without buffering:
+
+```ts
+await client.works
+  .filter({ publication_year: 2023 })
+  .select(['id', 'doi', 'display_name', 'cited_by_count'])
+  .export('works-2023.jsonl')                 // format inferred from extension
+
+await q.export('out.csv',  { format: 'csv' })  // flattened, RFC-escaped
+await q.export('out.json', { format: 'json' }) // single JSON array
+```
+
+## Citation graph
+
+```ts
+// Composable QueryBuilders. Chain .select(), .all(), .export(), and the rest.
+client.works.citedBy('W2741809807')      // works that CITE this one
+client.works.references('W2741809807')   // works this one CITES
+client.works.related('W2741809807')      // OpenAlex "related_to"
+
+// One-hop neighbourhood in parallel:
+const { seed, references, citedBy } = await client.works.citationGraph('W2741809807', {
+  limit: 50,
+  select: ['id', 'display_name', 'cited_by_count'],
+})
+```
+
+## Resumable extractions
+
+Long extractions checkpoint their cursor after every page, so a crash, a rate limit, or a Ctrl-C picks up where it stopped. You do not re-pay for pages you already fetched.
+
+```ts
+import { FileCheckpoint } from 'open-alex-wrapper'
+
+const resume = new FileCheckpoint('.harvest.checkpoint')
+
+await client.works.filter({ publication_year: 2023 }).export('works-2023.jsonl', {
+  resume,                                   // survives restarts, auto-cleared on completion
+  onProgress: ({ emitted, total }) =>
+    console.log(`${emitted.toLocaleString()} / ${total?.toLocaleString()}`),
+})
+```
+
+## Snapshot bridge: extract all data at $0
+
+OpenAlex publishes the whole dataset as a free, public S3 snapshot (gzipped JSON Lines, no credentials). For large extractions it costs a lot less than the metered API. This SDK compares the two and streams the snapshot in-process, with no dependencies.
+
+```ts
+// Should I pay the API or grab the free snapshot?
+const plan = await client.snapshotPlan(client.works.filter({ 'open_access.is_oa': true }))
+console.log(plan.recommendation)   // 'api' | 'snapshot'
+console.log(plan.rationale)        // e.g. "...covers 42% of all works. The full works snapshot (666 GB, 510M records) is free."
+console.log(plan.awsCommand)       // aws s3 sync "s3://openalex/data/jsonl/works" ... --no-sign-request
+
+// Manifest totals (all 21 entities, or one):
+await client.snapshot.totals()
+await client.snapshot.entityTotals('works')   // { recordCount: 510_000_000, contentLength: 665e9 }
+
+// Stream the snapshot directly, in-process, at $0, with a client-side filter.
+for await (const work of client.snapshot.stream('works', {
+  filter: (w) => w.open_access?.is_oa === true,
+  maxRecords: 100_000,
+  onPart: ({ index, total }) => console.log(`part ${index + 1}/${total}`),
+})) {
+  // ...no API credits spent
+}
+```
+
+## Full text (PDF / TEI-XML)
+
+OpenAlex caches full text for roughly 60M open-access works. Downloads cost $0.01 per call and are budget-tracked like everything else.
+
+```ts
+// Find works that have a downloadable PDF:
+const withPdf = client.works.withPdf().filter({ publication_year: 2024 })
+
+// Download one (bytes, or stream to a file with `dest`):
+const { bytes } = await client.works.downloadFulltext('W3038568908', { format: 'pdf' })
+await client.works.downloadFulltext('W3038568908', { format: 'xml', dest: 'paper.tei.xml' })
+
+// Or just get the (auth-redacted) content URL:
+client.works.fulltextUrl('W3038568908')          // …/works/W3038568908.pdf?api_key=REDACTED
+```
+
+## CLI
+
+Installs an `oaw` command, usable globally or through `npx`:
+
+```bash
+oaw works --filter "publication_year:>2020,open_access.is_oa:true" --limit 5
+oaw works --search crispr --select id,display_name --export out.jsonl --parallel
+oaw works --filter "publication_year:2024" --estimate     # dollar cost preview
+oaw get authors A5023888391 --select id,display_name
+oaw snapshot totals works                                 # 510,372,821 records  665.7 GB
+oaw snapshot plan works --filter "open_access.is_oa:true"  # API $ vs free snapshot
+oaw snapshot stream works --limit 100 --export works.jsonl # $0 harvest
+oaw fulltext W3038568908 --format pdf --out paper.pdf
+```
+
+Global flags: `--api-key` (or `OPENALEX_API_KEY`), `--mailto`, `--json`, `--help`.
+
+## Helpers
+
+```ts
+import { decodeAbstract, fetchNgrams } from 'open-alex-wrapper'
+
+const work = await client.works.get('W2741809807')
+const abstract = decodeAbstract(work.abstract_inverted_index)  // reconstruct text
+const ngrams = await client.ngrams('W2741809807')              // fulltext n-grams
+```
+
+## Errors
+
+Every failure is a typed subclass of `OpenAlexError`: `AuthenticationError` (401/403), `NotFoundError` (404), `ValidationError` (400/422), `RateLimitError` (429), `ServerError` (5xx), `TimeoutError`, `NetworkError`, and `BudgetExceededError`. Retryable statuses (429 and 5xx) are retried automatically with exponential backoff that honours `Retry-After`.
+
+## Performance notes
+
+- Concurrency is bounded by a semaphore (`concurrency`, default 8) across the whole client. `getMany`, parallel paging, and any concurrent `await`s share the limit.
+- Cursor paging streams unlimited results with constant memory. Parallel basic paging (`{ parallel: true }`) is the fast path for the very common case of 10k results or fewer.
+- HTTP keep-alive is on by default through Node's global `fetch` agent.
+- `select` fewer fields for smaller, faster payloads.
+- Caching (`cache: true`) de-duplicates identical GETs.
+
+## Scripts
+
+```bash
+npm run build      # ESM + CJS + .d.ts via tsup
+npm run typecheck  # tsc --noEmit
+npm test           # vitest (mocked HTTP, no network)
+npm run gen:filters # regenerate typed filter keys from the live API
+
+# Opt-in live smoke test against the real API:
+OPENALEX_LIVE=1 OPENALEX_API_KEY=... npx vitest run test/live.smoke.test.ts
+```
+
+## Changelog
+
+See [`CHANGELOG.md`](CHANGELOG.md).
+
+## License
+
+MIT © Jose Luis Hernando