scholarcli 1.23__tar.gz → 1.26__tar.gz

{scholarcli-1.23 → scholarcli-1.26}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scholarcli
-Version: 1.23
+Version: 1.26
 Summary: A tool for structured literature searches across bibliographic databases
 Author-email: Daniel Bosk <dbosk@kth.se>, Ric Glassey <glassey@kth.se>
 License-Expression: MIT
@@ -46,19 +46,36 @@ A command-line tool for conducting structured literature searches across multipl
 
 ### Multi-Database Search
 
-Search across six academic databases with a single query:
+Search across fifteen academic databases with a single query:
 
 - **Semantic Scholar** - AI-powered research database with 200M+ papers
 - **OpenAlex** - Open catalog of 250M+ scholarly works
 - **DBLP** - Computer science bibliography
 - **Web of Science** - Comprehensive citation index (requires API key)
 - **IEEE Xplore** - IEEE technical literature (requires API key)
-- **arXiv** - Preprints (no API key)
+- **Scopus** - Elsevier's abstract and citation database (requires API key)
+- **arXiv** - Preprints in physics, math, CS (no API key)
+- **HAL** - The French national open archive (no API key)
+- **Preprint servers** - SSRN, bioRxiv, medRxiv, ChemRxiv, Research
+  Square, Preprints.org, and the IACR Cryptology ePrint Archive, each
+  searched through OpenAlex pinned to that server (none of them offers
+  a usable search API of its own)
+
+The preprint-server providers are opt-in: a default search already finds
+their papers through OpenAlex, so they run only when you name them —
+either individually (`-p ssrn` scopes a search to SSRN) or all at once
+with the `preprints` group.
 
 ```bash
 # Search specific providers
 scholar search "federated learning" -p s2 -p openalex
 
+# Scope a search to one preprint server
+scholar search "corporate governance" -p ssrn
+
+# All preprint servers at once (arXiv, HAL, SSRN, bioRxiv, medRxiv, ...)
+scholar search "diffusion models" -p preprints
+
 # Start from a research question (LLM generates provider-specific queries)
 scholar rq "How can privacy-preserving ML be evaluated?" \
   --provider openalex --provider dblp \
@@ -109,6 +126,16 @@ scholar sessions resume "machine learning"
 
 # Export session to reports
 scholar sessions export "machine learning" -f all
+
+# LaTeX report as an \input-able fragment (writes review.tex + review.bib)
+scholar sessions export "machine learning" -f latex --no-standalone -o review
+
+# Audit table for a backing appendix (one row per paper with the reason;
+# always a fragment; --lang en|sv, --theme TAG=NAME gives themes readable names)
+scholar sessions export "machine learning" -f table --lang sv --label sok-A \
+    --track "sökspår 1" --theme ml=maskininlärning -o review-full
+# ... listing only the cited/supporting/qualifying rows, the rest as counts
+scholar sessions export "machine learning" -f table --bearing-only -o review-bearing
 ```
 
 ### Paper Notes
@@ -139,6 +166,18 @@ scholar cache path    # Print cache directory
 
 PDF downloads are also cached for offline viewing.
 
+### Rate-Limit and Quota Tracking
+
+Provider rate limits and daily quotas (e.g. IEEE's 200 requests/day) are
+tracked persistently across invocations: a provider whose budget is spent
+is skipped with a warning naming the reset time instead of firing doomed
+requests.
+
+```bash
+scholar providers limits              # Show usage, remaining budget, reset times
+scholar providers limits --reset ieee # Clear tracked state for one provider
+```
+
 ## Quickstart
 
 ### Install
@@ -204,15 +243,34 @@ uv pip install scholarcli
 
 ## Configuration
 
+Set `SCHOLAR_EMAIL` to a contact address: OpenAlex (and thus every
+preprint-server provider), Crossref, and the polite pools use it for
+faster responses, and Unpaywall (open-access lookup during
+`scholar enrich` and `scholar pdf open <doi>`) requires it. `OPENALEX_EMAIL`
+and `CROSSREF_MAILTO` are still honoured as per-service overrides.
+
+OpenAlex meters requests against a daily budget (a search costs $0.001,
+reset at midnight UTC). A free API key from
+[openalex.org/settings/api](https://openalex.org/settings/api) raises the
+budget tenfold; set it as `OPENALEX_API_KEY` and it is used by the
+`openalex` provider and every preprint-server provider. `scholar
+providers check` then shows how much of the day's budget is left, and a
+search that hits the spent budget reports the reset time instead of a
+bare 429.
+
 Some providers require API keys set as environment variables:
 
 | Provider | Environment Variable | Required | How to Get |
 |----------|---------------------|----------|------------|
 | Semantic Scholar | `S2_API_KEY` | No | [api.semanticscholar.org](https://api.semanticscholar.org) |
-| OpenAlex | `OPENALEX_EMAIL` | No | Any email (for polite pool) |
+| OpenAlex | `OPENALEX_API_KEY`, `SCHOLAR_EMAIL` | No | [openalex.org/settings/api](https://openalex.org/settings/api) (10x daily budget); any email (polite pool) |
 | DBLP | - | No | No key needed |
+| arXiv | - | No | No key needed |
+| HAL | - | No | No key needed |
+| SSRN, bioRxiv, medRxiv, ChemRxiv, Research Square, Preprints.org, IACR | `SCHOLAR_EMAIL` | No | Any email (they search via OpenAlex) |
 | Web of Science | `WOS_EXPANDED_API_KEY` or `WOS_STARTER_API_KEY` | Yes | [developer.clarivate.com](https://developer.clarivate.com) |
 | IEEE Xplore | `IEEE_API_KEY` | Yes | [developer.ieee.org](https://developer.ieee.org) |
+| Scopus | `SCOPUS_API_KEY` | Yes | [dev.elsevier.com](https://dev.elsevier.com) |
 
 View provider status:
 
@@ -249,6 +307,15 @@ scholar sessions resume "privacy-ml-review"
 
 # 4. Generate reports
 scholar sessions export "privacy-ml-review" -f all
+
+# 5. Synthesise the kept papers (needs a research context); the LaTeX
+#    document ends with the decision record as an appendix
+scholar llm context "privacy-ml-review" "How is privacy preserved in ML?"
+scholar llm synthesize "privacy-ml-review" -f latex -o synthesis.tex
+
+# ... or as a fragment to \input into a thesis chapter (also writes
+#     synthesis.bib; add \addbibresource{synthesis.bib} to the preamble)
+scholar llm synthesize "privacy-ml-review" -f latex --no-standalone -o synthesis.tex
 ```
 
 ### Enriching Results
@@ -263,6 +330,22 @@ scholar search "query" --enrich
 scholar enrich "session-name"
 ```
 
+When no PDF can be found for a paper, enrichment falls back to preprint
+versions: authoritative links (OpenAlex's alternate locations and
+Crossref's `has-preprint` relation) are always followed, and with
+`--preprint-search` the preprint servers are additionally searched by
+title and authors (exact match only). An adopted preprint *replaces*
+the entry — its DOI, venue, and PDF — so exported citations point at
+the version you actually read; the published DOI is kept on the entry
+for provenance, and `scholar verify` reports such entries as
+`preprint of <doi>` rather than superseded.
+
+```bash
+# Also search preprint servers for papers without a PDF
+scholar enrich "session-name" --preprint-search
+scholar search "query" --enrich --preprint-search
+```
+
 ### PDF Management
 
 ```bash
@@ -299,6 +382,12 @@ For large result sets, Scholar can use LLMs to assist with paper classification:
 # In the TUI, press 'L' to invoke LLM classification
 # Or use the CLI command directly
 scholar llm classify "session-name" --count 10
+
+# Constrain the tags to a vocabulary (tags already in the session count too)
+scholar llm classify "session-name" -t supports-claim -t qualifies-claim \
+    -t adjacent-subtopic -t off-topic-false-hit
+# ... or let the model add tags of its own when none of them fits
+scholar llm classify "session-name" --allow-new-tags
 ```
 
 ### How It Works

{scholarcli-1.23 → scholarcli-1.26}/README.md

@@ -6,19 +6,36 @@ A command-line tool for conducting structured literature searches across multipl
 
 ### Multi-Database Search
 
-Search across six academic databases with a single query:
+Search across fifteen academic databases with a single query:
 
 - **Semantic Scholar** - AI-powered research database with 200M+ papers
 - **OpenAlex** - Open catalog of 250M+ scholarly works
 - **DBLP** - Computer science bibliography
 - **Web of Science** - Comprehensive citation index (requires API key)
 - **IEEE Xplore** - IEEE technical literature (requires API key)
-- **arXiv** - Preprints (no API key)
+- **Scopus** - Elsevier's abstract and citation database (requires API key)
+- **arXiv** - Preprints in physics, math, CS (no API key)
+- **HAL** - The French national open archive (no API key)
+- **Preprint servers** - SSRN, bioRxiv, medRxiv, ChemRxiv, Research
+  Square, Preprints.org, and the IACR Cryptology ePrint Archive, each
+  searched through OpenAlex pinned to that server (none of them offers
+  a usable search API of its own)
+
+The preprint-server providers are opt-in: a default search already finds
+their papers through OpenAlex, so they run only when you name them —
+either individually (`-p ssrn` scopes a search to SSRN) or all at once
+with the `preprints` group.
 
 ```bash
 # Search specific providers
 scholar search "federated learning" -p s2 -p openalex
 
+# Scope a search to one preprint server
+scholar search "corporate governance" -p ssrn
+
+# All preprint servers at once (arXiv, HAL, SSRN, bioRxiv, medRxiv, ...)
+scholar search "diffusion models" -p preprints
+
 # Start from a research question (LLM generates provider-specific queries)
 scholar rq "How can privacy-preserving ML be evaluated?" \
   --provider openalex --provider dblp \
@@ -69,6 +86,16 @@ scholar sessions resume "machine learning"
 
 # Export session to reports
 scholar sessions export "machine learning" -f all
+
+# LaTeX report as an \input-able fragment (writes review.tex + review.bib)
+scholar sessions export "machine learning" -f latex --no-standalone -o review
+
+# Audit table for a backing appendix (one row per paper with the reason;
+# always a fragment; --lang en|sv, --theme TAG=NAME gives themes readable names)
+scholar sessions export "machine learning" -f table --lang sv --label sok-A \
+    --track "sökspår 1" --theme ml=maskininlärning -o review-full
+# ... listing only the cited/supporting/qualifying rows, the rest as counts
+scholar sessions export "machine learning" -f table --bearing-only -o review-bearing
 ```
 
 ### Paper Notes
@@ -99,6 +126,18 @@ scholar cache path    # Print cache directory
 
 PDF downloads are also cached for offline viewing.
 
+### Rate-Limit and Quota Tracking
+
+Provider rate limits and daily quotas (e.g. IEEE's 200 requests/day) are
+tracked persistently across invocations: a provider whose budget is spent
+is skipped with a warning naming the reset time instead of firing doomed
+requests.
+
+```bash
+scholar providers limits              # Show usage, remaining budget, reset times
+scholar providers limits --reset ieee # Clear tracked state for one provider
+```
+
 ## Quickstart
 
 ### Install
@@ -164,15 +203,34 @@ uv pip install scholarcli
 
 ## Configuration
 
+Set `SCHOLAR_EMAIL` to a contact address: OpenAlex (and thus every
+preprint-server provider), Crossref, and the polite pools use it for
+faster responses, and Unpaywall (open-access lookup during
+`scholar enrich` and `scholar pdf open <doi>`) requires it. `OPENALEX_EMAIL`
+and `CROSSREF_MAILTO` are still honoured as per-service overrides.
+
+OpenAlex meters requests against a daily budget (a search costs $0.001,
+reset at midnight UTC). A free API key from
+[openalex.org/settings/api](https://openalex.org/settings/api) raises the
+budget tenfold; set it as `OPENALEX_API_KEY` and it is used by the
+`openalex` provider and every preprint-server provider. `scholar
+providers check` then shows how much of the day's budget is left, and a
+search that hits the spent budget reports the reset time instead of a
+bare 429.
+
 Some providers require API keys set as environment variables:
 
 | Provider | Environment Variable | Required | How to Get |
 |----------|---------------------|----------|------------|
 | Semantic Scholar | `S2_API_KEY` | No | [api.semanticscholar.org](https://api.semanticscholar.org) |
-| OpenAlex | `OPENALEX_EMAIL` | No | Any email (for polite pool) |
+| OpenAlex | `OPENALEX_API_KEY`, `SCHOLAR_EMAIL` | No | [openalex.org/settings/api](https://openalex.org/settings/api) (10x daily budget); any email (polite pool) |
 | DBLP | - | No | No key needed |
+| arXiv | - | No | No key needed |
+| HAL | - | No | No key needed |
+| SSRN, bioRxiv, medRxiv, ChemRxiv, Research Square, Preprints.org, IACR | `SCHOLAR_EMAIL` | No | Any email (they search via OpenAlex) |
 | Web of Science | `WOS_EXPANDED_API_KEY` or `WOS_STARTER_API_KEY` | Yes | [developer.clarivate.com](https://developer.clarivate.com) |
 | IEEE Xplore | `IEEE_API_KEY` | Yes | [developer.ieee.org](https://developer.ieee.org) |
+| Scopus | `SCOPUS_API_KEY` | Yes | [dev.elsevier.com](https://dev.elsevier.com) |
 
 View provider status:
 
@@ -209,6 +267,15 @@ scholar sessions resume "privacy-ml-review"
 
 # 4. Generate reports
 scholar sessions export "privacy-ml-review" -f all
+
+# 5. Synthesise the kept papers (needs a research context); the LaTeX
+#    document ends with the decision record as an appendix
+scholar llm context "privacy-ml-review" "How is privacy preserved in ML?"
+scholar llm synthesize "privacy-ml-review" -f latex -o synthesis.tex
+
+# ... or as a fragment to \input into a thesis chapter (also writes
+#     synthesis.bib; add \addbibresource{synthesis.bib} to the preamble)
+scholar llm synthesize "privacy-ml-review" -f latex --no-standalone -o synthesis.tex
 ```
 
 ### Enriching Results
@@ -223,6 +290,22 @@ scholar search "query" --enrich
 scholar enrich "session-name"
 ```
 
+When no PDF can be found for a paper, enrichment falls back to preprint
+versions: authoritative links (OpenAlex's alternate locations and
+Crossref's `has-preprint` relation) are always followed, and with
+`--preprint-search` the preprint servers are additionally searched by
+title and authors (exact match only). An adopted preprint *replaces*
+the entry — its DOI, venue, and PDF — so exported citations point at
+the version you actually read; the published DOI is kept on the entry
+for provenance, and `scholar verify` reports such entries as
+`preprint of <doi>` rather than superseded.
+
+```bash
+# Also search preprint servers for papers without a PDF
+scholar enrich "session-name" --preprint-search
+scholar search "query" --enrich --preprint-search
+```
+
 ### PDF Management
 
 ```bash
@@ -259,6 +342,12 @@ For large result sets, Scholar can use LLMs to assist with paper classification:
 # In the TUI, press 'L' to invoke LLM classification
 # Or use the CLI command directly
 scholar llm classify "session-name" --count 10
+
+# Constrain the tags to a vocabulary (tags already in the session count too)
+scholar llm classify "session-name" -t supports-claim -t qualifies-claim \
+    -t adjacent-subtopic -t off-topic-false-hit
+# ... or let the model add tags of its own when none of them fits
+scholar llm classify "session-name" --allow-new-tags
 ```
 
 ### How It Works

{scholarcli-1.23 → scholarcli-1.26}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "scholarcli"
-version = "1.23"
+version = "1.26"
 description = "A tool for structured literature searches across bibliographic databases"
 authors = [{ name = "Daniel Bosk", email = "dbosk@kth.se" },
            { name = "Ric Glassey", email = "glassey@kth.se" }]

{scholarcli-1.23 → scholarcli-1.26}/src/scholar/__init__.py

@@ -4,6 +4,7 @@ Scholar package for structured literature searches.
 
 from .scholar import Search, SearchResult, Paper, SearchFilters
 from .scholar import search, filter_papers, deduplicate_papers
+from .scholar import deduplicate_versions, merge_version_of
 from .scholar import get_registry, isolated_registry
 from .utils import safe_get_nested, ensure_list
 
@@ -15,6 +16,8 @@ __all__ = [
     "search",
     "filter_papers",
     "deduplicate_papers",
+    "deduplicate_versions",
+    "merge_version_of",
     "get_registry",
     "isolated_registry",
     "safe_get_nested",