litsurvey 1.0.0__tar.gz

litsurvey-1.0.0/CHANGELOG.md

@@ -0,0 +1,37 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/) and the project uses
+[Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [1.0.0] - 2026-09-11
+
+First public release.
+
+### Added
+- `search`: fused, de-duplicated keyword search over OpenAlex, Semantic Scholar,
+  arXiv, TechRxiv, Research Square (via Crossref) and the IACR ePrint archive,
+  with `--sort relevance|citations|year`.
+- `paper`, `cites`, `refs`, `related`: one paper and its citation neighbourhood;
+  a title is accepted in place of an id, with a candidate list and `--pick N`.
+  `cites`/`refs` rank by citation count via OpenAlex, with Semantic Scholar as
+  fallback; `related` and `paper` fall back to OpenAlex when Semantic Scholar
+  is unavailable.
+- `oa`: legal open-access copies via Unpaywall.
+- `novelty` and `research`: LLM-driven agents with pluggable backends and a
+  reproducible search log. Backends: `cli` (delegates the task to a signed-in
+  Claude Code, Codex CLI or Gemini CLI under the user's subscription),
+  `ollama` (local), `openai` (any OpenAI-compatible server, `--base-url` for
+  OpenRouter, LM Studio and others), `anthropic`.
+- Export to BibTeX, RIS, CSV, JSON and Markdown from any result list.
+- Run history (`litsurvey history`) shared by the CLI and the web page.
+- `litsurvey web`: local browser interface with a History tab, a photo
+  banner, a three-way choice of where the LLM runs (subscription CLI, local
+  model, cloud API key) with a privacy banner, BibTeX/RIS/CSV downloads,
+  URL detection with DOI/arXiv conversion, "waiting for <source>" status
+  while a run is in progress, and a report-a-bug link.
+- `init` and `doctor` commands.
+- Per-host rate limiting (Semantic Scholar 1 req/s, arXiv 1 req/3 s) and
+  `Retry-After` handling.

litsurvey-1.0.0/CITATION.cff

@@ -0,0 +1,38 @@
+cff-version: 1.2.0
+title: "litsurvey: multi-source scientific literature search with LLM-assisted novelty assessment"
+message: >-
+  If you use this software in your research, please cite it using the
+  metadata below.
+type: software
+authors:
+  - family-names: Khankhoje
+    given-names: Uday
+    email: uday@ee.iitm.ac.in
+    affiliation: >-
+      Department of Electrical Engineering, Indian Institute of Technology
+      Madras
+    orcid: 'https://orcid.org/0000-0002-9629-3922'
+repository-code: 'https://github.com/udaykdk/litsurvey'
+abstract: >-
+  A zero-dependency Python tool for literature surveys: it searches OpenAlex,
+  Semantic Scholar, arXiv, TechRxiv, Research Square and the IACR ePrint
+  archive, fuses and de-duplicates the results, walks the citation graph,
+  finds open-access copies, and exports BibTeX/RIS/CSV. An LLM (a local
+  model, the user's Claude Code / Codex / Gemini subscription CLI, or a cloud
+  API) drives multi-step novelty assessment and cited state-of-the-art
+  surveys, with a reproducible search log. A local browser interface with
+  run history is included. Designed so that confidential manuscript content
+  never has to leave the user's machine.
+keywords:
+  - literature search
+  - peer review
+  - systematic review
+  - novelty assessment
+  - Semantic Scholar
+  - OpenAlex
+  - arXiv
+  - BibTeX
+  - large language models
+license: MIT
+version: 1.0.0
+date-released: '2026-09-11'

litsurvey-1.0.0/CONTRIBUTING.md

@@ -0,0 +1,63 @@
+# Contributing
+
+## Reporting a bug
+
+Open an issue with the bug template. Include the command, the output with
+`--debug`, and the output of `litsurvey --version` and `litsurvey doctor`.
+Remove your API key if it appears anywhere. Most problems are API changes on
+the provider side, and the exact URL from `--debug` is what makes them
+fixable.
+
+## Development setup
+
+```bash
+git clone https://github.com/udaykdk/litsurvey
+cd litsurvey
+python -m venv .venv && source .venv/bin/activate     # Windows: .venv\Scripts\activate
+pip install -e ".[dev]"
+python -m pytest
+```
+
+The tests are offline: every HTTP call is mocked. Please keep it that way, so
+CI does not depend on API availability. If you change how a source is parsed,
+add a fixture for the new response shape in `tests/test_sources.py`.
+
+## Design rules
+
+- **Zero runtime dependencies.** Standard library only. This keeps
+  installation trivial on university machines and inside agents. If a
+  feature needs a dependency, make it optional and import it lazily.
+- **Only public, official APIs, with one documented exception.** No
+  scraping of sites whose terms forbid it (Google Scholar in particular).
+  New sources should have a documented API and a rate limit we can honour in
+  `http.HOST_INTERVAL`. The IACR ePrint source reads a server-rendered
+  search page because the archive offers no search API and does not
+  prohibit it; it is rate-limited to one request per two seconds and is
+  isolated so that a markup change only silences that source.
+- **Nothing leaves the machine that the docs do not say leaves.** Any change
+  to what is sent where must be reflected in `docs/confidentiality.md`.
+- **Every successfully completed run is recorded** in the history store
+  (search runs with per-source hit counts), so results are reproducible.
+
+## Adding a source
+
+1. Create `litsurvey/sources/<name>.py` with a `search(query, limit,
+   year_from)` function that returns a list of `papers.make(...)` records
+   with `sources=["<name>"]`.
+2. Register it in `litsurvey/sources/__init__.py` (`SOURCES`).
+3. Add the host's rate limit to `http.HOST_INTERVAL`.
+4. Add a parsing test with a fixture, and a line in `docs/cli.md`.
+
+## Pull requests
+
+Small, focused pull requests are easier to review. Add a line under
+*Unreleased* in `CHANGELOG.md`. CI runs the tests on Linux, macOS and
+Windows with Python 3.10, 3.12 and 3.13.
+
+## Releasing (maintainers)
+
+1. Bump `version` in `pyproject.toml`, `litsurvey/__init__.py` and
+   `CITATION.cff`; move the changelog entries under the new version.
+2. `git tag -a vX.Y.Z -m "vX.Y.Z"` and push the tag.
+3. Create a GitHub release from the tag; the publish workflow uploads to
+   PyPI via trusted publishing.

litsurvey-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Uday K Khankhoje
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

litsurvey-1.0.0/MANIFEST.in

@@ -0,0 +1,4 @@
+include LICENSE README.md CHANGELOG.md CONTRIBUTING.md CITATION.cff
+recursive-include docs *.md
+recursive-include integrations *.md
+recursive-include litsurvey/web *.html

litsurvey-1.0.0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: litsurvey
+Version: 1.0.0
+Summary: Multi-source scientific literature search with optional LLM-driven novelty assessment and deep research. Zero dependencies.
+Author-email: Uday Khankhoje <uday@ee.iitm.ac.in>
+License: MIT
+Project-URL: Homepage, https://github.com/udaykdk/litsurvey
+Project-URL: Issues, https://github.com/udaykdk/litsurvey/issues
+Project-URL: Changelog, https://github.com/udaykdk/litsurvey/blob/main/CHANGELOG.md
+Keywords: literature search,literature survey,semantic scholar,openalex,arxiv,crossref,peer review,novelty,systematic review,bibtex,llm,ollama,claude code
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# litsurvey
+
+A research assistant for the literature: ask a question, get a cited
+state-of-the-art survey; state a claim, get a prior-art verdict; give a
+paper title, get everything that cites it. It runs on your laptop, from the
+command line or a small browser page, and uses the LLM you already have,
+whether that is a Claude, ChatGPT or Gemini subscription, a local model, or
+an API key. One Python package, no runtime dependencies, Python 3.10 or
+newer.
+
+## What it does
+
+**LLM-driven (the main draw)**
+
+- `research "question"`: the model breaks the question into sub-questions,
+  runs searches across six scholarly indexes, walks the citation graph of
+  the central papers, reads one or two arXiv papers in full, and writes a
+  survey organised by theme with numbered citations, open problems, and an
+  honest coverage caveat. Every citation comes from a real search result,
+  and the report ends with a log of every query that was run.
+- `novelty "claim"`: the same machinery aimed at one claim: closest prior
+  work, what is new versus known, a verdict (clearly novel / incremental /
+  substantially anticipated / cannot determine) with confidence, and
+  citations a reviewer can use.
+- Runs on any of: **your subscription's command-line agent** (Claude Code,
+  Codex CLI, Gemini CLI; no API key), **a local model** through Ollama
+  (nothing leaves the machine), or **a cloud API key** (OpenAI, OpenRouter,
+  Anthropic, any OpenAI-compatible server).
+
+**Search and lookup (no LLM)**
+
+- Fused search over OpenAlex, Semantic Scholar, arXiv, TechRxiv, Research
+  Square and the IACR ePrint archive, de-duplicated, sortable by relevance,
+  citations or year.
+- For any paper, by title or id: who cites it (most-cited first), what it
+  cites, similar papers from a recommender, its full record and BibTeX, and
+  a legal open-access copy.
+- Export to BibTeX, RIS, CSV, JSON or Markdown for Zotero, Overleaf, Mendeley
+  or a spreadsheet.
+- A history of every completed run, shared by the command line and the
+  browser page, with the same downloads.
+
+## Install
+
+From GitHub (works today):
+
+```bash
+pipx install git+https://github.com/udaykdk/litsurvey
+# or
+pip install git+https://github.com/udaykdk/litsurvey
+```
+
+From PyPI, after the first release is published:
+
+```bash
+pipx install litsurvey
+```
+
+Or clone and run without installing: `python3 -m litsurvey --help`.
+
+Then, optional but recommended: get a free Semantic Scholar API key
+(a one-minute form, see [docs/api-keys.md](https://github.com/udaykdk/litsurvey/blob/main/docs/api-keys.md))
+and store it with `litsurvey init`. Without it everything works, only
+slower. `litsurvey doctor` checks the setup and prints a `RESULT:` verdict.
+
+## Examples
+
+A state-of-the-art survey, written by the model from live search results:
+
+```bash
+litsurvey research "uncertainty quantification methods for physics-informed neural networks" --out uq-survey.md
+```
+
+A prior-art check on a claim, phrased as a generic topic (never a sentence
+from a confidential manuscript):
+
+```bash
+litsurvey novelty "adaptive sampling of collocation points in physics-informed neural networks" --out claim.md
+```
+
+Choosing where the model runs:
+
+```bash
+litsurvey research "..." --backend cli --model claude          # Claude Code, under your Claude Pro/Max plan
+litsurvey research "..." --backend ollama --model qwen3:30b    # local; nothing leaves the machine
+litsurvey research "..." --backend openai --base-url https://openrouter.ai/api --model anthropic/claude-sonnet-4.5
+```
+
+Search and lookup, with titles instead of ids:
+
+```bash
+litsurvey search "physics informed neural networks inverse problems" --year-from 2022 --sort citations
+litsurvey cites "Superior thermal conductivity of single-layer graphene"   # picks the paper, lists the most-cited citers
+litsurvey related "Embedding deep learning in inverse scattering problems"
+litsurvey paper "KAN: Kolmogorov-Arnold Networks" --out ref.bib          # BibTeX for a paper you know by title
+litsurvey oa "10.1016/j.cma.2022.114823"                                 # legal free PDF
+litsurvey web                                                            # the same, in your browser
+```
+
+When a title matches several papers, litsurvey shows the candidates and
+asks; scripts and agents get the list and use `--pick N`.
+
+Typical times: a search takes a few seconds; `novelty` and `research` take
+two to fifteen minutes depending on the model.
+
+## What needs an LLM, and what leaves your machine
+
+| Command | LLM | What is sent out, and to whom |
+|---|---|---|
+| `search`, `paper`, `cites`, `refs`, `related` | no | your query, or a paper id or title, to the scholarly APIs (OpenAlex, Semantic Scholar, arXiv, Crossref, eprint.iacr.org) |
+| `oa` | no | one DOI to Unpaywall (a title is first resolved through the search APIs) |
+| `history`, `init`, `doctor`, `web` | no | nothing (`doctor` makes one test query per source; actions on the web page follow the rows above) |
+| `novelty`, `research`, local model (Ollama, or an OpenAI-compatible server on localhost) | yes, local | the keyword queries the model composes and the paper ids it looks up, to the scholarly APIs; your text stays on the machine |
+| `novelty`, `research`, subscription CLI (Claude Code, Codex, Gemini) | yes, vendor cloud | your claim or question and everything the agent reads, to that vendor under your subscription |
+| `novelty`, `research`, cloud API key | yes, cloud | your claim or question and every search result, to that provider |
+
+For confidential work (a manuscript under review, an unpublished result)
+use a local model and phrase the input as generic topic terms. Completed
+runs, including inputs and results, are stored under `~/.litsurvey/` on
+your disk. LLM reports are aids, not verified reviews: check the cited
+paper and DOI before using any claim. Details in
+[docs/confidentiality.md](https://github.com/udaykdk/litsurvey/blob/main/docs/confidentiality.md).
+
+## Documentation
+
+- [docs/cli.md](https://github.com/udaykdk/litsurvey/blob/main/docs/cli.md): every command, option and output format, with sample output
+- [docs/web.md](https://github.com/udaykdk/litsurvey/blob/main/docs/web.md): the browser interface and the History tab
+- [docs/use-cases.md](https://github.com/udaykdk/litsurvey/blob/main/docs/use-cases.md): seven scenarios with the exact commands, from student discovery to confidential peer review
+- [docs/llm-integration.md](https://github.com/udaykdk/litsurvey/blob/main/docs/llm-integration.md): LLM backends and models; using litsurvey as a tool from Claude Code, Codex and other agents
+- [docs/confidentiality.md](https://github.com/udaykdk/litsurvey/blob/main/docs/confidentiality.md): what leaves the machine, per mode and backend
+- [docs/api-keys.md](https://github.com/udaykdk/litsurvey/blob/main/docs/api-keys.md): getting and storing the Semantic Scholar key; running without one
+- [CONTRIBUTING.md](https://github.com/udaykdk/litsurvey/blob/main/CONTRIBUTING.md): reporting bugs, running tests, adding a source
+
+## Coverage, and Google Scholar
+
+Six sources are searched by default: OpenAlex (about 250 million works),
+Semantic Scholar (about 220 million), arXiv, and three preprint portals:
+TechRxiv and Research Square (through Crossref, by their DOI prefixes) and
+the IACR Cryptology ePrint Archive (which has no search API, so litsurvey
+reads its search page; if that page changes, only that source goes quiet).
+`--sources` restricts the set; `crossref` (all DOI-registered works) can be
+added but overlaps OpenAlex. Together they cover most of what Google
+Scholar shows, except some grey literature (technical reports, theses,
+standards). Google Scholar has no API and its terms of service forbid
+automated access, so litsurvey does not scrape it; `litsurvey search
+--scholar` prints the matching Google Scholar URL for a manual comparison,
+and the web page has the same link.
+
+## Data attribution
+
+Results come from [OpenAlex](https://openalex.org) (CC0),
+[Semantic Scholar](https://www.semanticscholar.org) (Semantic Scholar Open
+Data Platform, Allen Institute for AI), [arXiv](https://arxiv.org) (thank you
+to arXiv for use of its open access interoperability),
+[Crossref](https://www.crossref.org), the
+[IACR Cryptology ePrint Archive](https://eprint.iacr.org) and
+[Unpaywall](https://unpaywall.org). If you publish work that used these
+results, please credit them.
+
+## Author
+
+Designed by Uday Khankhoje, Department of Electrical Engineering, IIT
+Madras; implemented with Anthropic Claude. Questions and bug reports: the
+[issue tracker](https://github.com/udaykdk/litsurvey/issues).
+
+## Credits
+
+The banner of the web page uses slivers of an aerial beach photograph by
+[Lance Asper on Unsplash](https://unsplash.com/@lance_asper).
+
+## Citing
+
+See [CITATION.cff](https://github.com/udaykdk/litsurvey/blob/main/CITATION.cff).
+GitHub shows a "Cite this repository" button on the project page.
+
+## License
+
+MIT. See [LICENSE](https://github.com/udaykdk/litsurvey/blob/main/LICENSE).

litsurvey-1.0.0/README.md

@@ -0,0 +1,178 @@
+# litsurvey
+
+A research assistant for the literature: ask a question, get a cited
+state-of-the-art survey; state a claim, get a prior-art verdict; give a
+paper title, get everything that cites it. It runs on your laptop, from the
+command line or a small browser page, and uses the LLM you already have,
+whether that is a Claude, ChatGPT or Gemini subscription, a local model, or
+an API key. One Python package, no runtime dependencies, Python 3.10 or
+newer.
+
+## What it does
+
+**LLM-driven (the main draw)**
+
+- `research "question"`: the model breaks the question into sub-questions,
+  runs searches across six scholarly indexes, walks the citation graph of
+  the central papers, reads one or two arXiv papers in full, and writes a
+  survey organised by theme with numbered citations, open problems, and an
+  honest coverage caveat. Every citation comes from a real search result,
+  and the report ends with a log of every query that was run.
+- `novelty "claim"`: the same machinery aimed at one claim: closest prior
+  work, what is new versus known, a verdict (clearly novel / incremental /
+  substantially anticipated / cannot determine) with confidence, and
+  citations a reviewer can use.
+- Runs on any of: **your subscription's command-line agent** (Claude Code,
+  Codex CLI, Gemini CLI; no API key), **a local model** through Ollama
+  (nothing leaves the machine), or **a cloud API key** (OpenAI, OpenRouter,
+  Anthropic, any OpenAI-compatible server).
+
+**Search and lookup (no LLM)**
+
+- Fused search over OpenAlex, Semantic Scholar, arXiv, TechRxiv, Research
+  Square and the IACR ePrint archive, de-duplicated, sortable by relevance,
+  citations or year.
+- For any paper, by title or id: who cites it (most-cited first), what it
+  cites, similar papers from a recommender, its full record and BibTeX, and
+  a legal open-access copy.
+- Export to BibTeX, RIS, CSV, JSON or Markdown for Zotero, Overleaf, Mendeley
+  or a spreadsheet.
+- A history of every completed run, shared by the command line and the
+  browser page, with the same downloads.
+
+## Install
+
+From GitHub (works today):
+
+```bash
+pipx install git+https://github.com/udaykdk/litsurvey
+# or
+pip install git+https://github.com/udaykdk/litsurvey
+```
+
+From PyPI, after the first release is published:
+
+```bash
+pipx install litsurvey
+```
+
+Or clone and run without installing: `python3 -m litsurvey --help`.
+
+Then, optional but recommended: get a free Semantic Scholar API key
+(a one-minute form, see [docs/api-keys.md](https://github.com/udaykdk/litsurvey/blob/main/docs/api-keys.md))
+and store it with `litsurvey init`. Without it everything works, only
+slower. `litsurvey doctor` checks the setup and prints a `RESULT:` verdict.
+
+## Examples
+
+A state-of-the-art survey, written by the model from live search results:
+
+```bash
+litsurvey research "uncertainty quantification methods for physics-informed neural networks" --out uq-survey.md
+```
+
+A prior-art check on a claim, phrased as a generic topic (never a sentence
+from a confidential manuscript):
+
+```bash
+litsurvey novelty "adaptive sampling of collocation points in physics-informed neural networks" --out claim.md
+```
+
+Choosing where the model runs:
+
+```bash
+litsurvey research "..." --backend cli --model claude          # Claude Code, under your Claude Pro/Max plan
+litsurvey research "..." --backend ollama --model qwen3:30b    # local; nothing leaves the machine
+litsurvey research "..." --backend openai --base-url https://openrouter.ai/api --model anthropic/claude-sonnet-4.5
+```
+
+Search and lookup, with titles instead of ids:
+
+```bash
+litsurvey search "physics informed neural networks inverse problems" --year-from 2022 --sort citations
+litsurvey cites "Superior thermal conductivity of single-layer graphene"   # picks the paper, lists the most-cited citers
+litsurvey related "Embedding deep learning in inverse scattering problems"
+litsurvey paper "KAN: Kolmogorov-Arnold Networks" --out ref.bib          # BibTeX for a paper you know by title
+litsurvey oa "10.1016/j.cma.2022.114823"                                 # legal free PDF
+litsurvey web                                                            # the same, in your browser
+```
+
+When a title matches several papers, litsurvey shows the candidates and
+asks; scripts and agents get the list and use `--pick N`.
+
+Typical times: a search takes a few seconds; `novelty` and `research` take
+two to fifteen minutes depending on the model.
+
+## What needs an LLM, and what leaves your machine
+
+| Command | LLM | What is sent out, and to whom |
+|---|---|---|
+| `search`, `paper`, `cites`, `refs`, `related` | no | your query, or a paper id or title, to the scholarly APIs (OpenAlex, Semantic Scholar, arXiv, Crossref, eprint.iacr.org) |
+| `oa` | no | one DOI to Unpaywall (a title is first resolved through the search APIs) |
+| `history`, `init`, `doctor`, `web` | no | nothing (`doctor` makes one test query per source; actions on the web page follow the rows above) |
+| `novelty`, `research`, local model (Ollama, or an OpenAI-compatible server on localhost) | yes, local | the keyword queries the model composes and the paper ids it looks up, to the scholarly APIs; your text stays on the machine |
+| `novelty`, `research`, subscription CLI (Claude Code, Codex, Gemini) | yes, vendor cloud | your claim or question and everything the agent reads, to that vendor under your subscription |
+| `novelty`, `research`, cloud API key | yes, cloud | your claim or question and every search result, to that provider |
+
+For confidential work (a manuscript under review, an unpublished result)
+use a local model and phrase the input as generic topic terms. Completed
+runs, including inputs and results, are stored under `~/.litsurvey/` on
+your disk. LLM reports are aids, not verified reviews: check the cited
+paper and DOI before using any claim. Details in
+[docs/confidentiality.md](https://github.com/udaykdk/litsurvey/blob/main/docs/confidentiality.md).
+
+## Documentation
+
+- [docs/cli.md](https://github.com/udaykdk/litsurvey/blob/main/docs/cli.md): every command, option and output format, with sample output
+- [docs/web.md](https://github.com/udaykdk/litsurvey/blob/main/docs/web.md): the browser interface and the History tab
+- [docs/use-cases.md](https://github.com/udaykdk/litsurvey/blob/main/docs/use-cases.md): seven scenarios with the exact commands, from student discovery to confidential peer review
+- [docs/llm-integration.md](https://github.com/udaykdk/litsurvey/blob/main/docs/llm-integration.md): LLM backends and models; using litsurvey as a tool from Claude Code, Codex and other agents
+- [docs/confidentiality.md](https://github.com/udaykdk/litsurvey/blob/main/docs/confidentiality.md): what leaves the machine, per mode and backend
+- [docs/api-keys.md](https://github.com/udaykdk/litsurvey/blob/main/docs/api-keys.md): getting and storing the Semantic Scholar key; running without one
+- [CONTRIBUTING.md](https://github.com/udaykdk/litsurvey/blob/main/CONTRIBUTING.md): reporting bugs, running tests, adding a source
+
+## Coverage, and Google Scholar
+
+Six sources are searched by default: OpenAlex (about 250 million works),
+Semantic Scholar (about 220 million), arXiv, and three preprint portals:
+TechRxiv and Research Square (through Crossref, by their DOI prefixes) and
+the IACR Cryptology ePrint Archive (which has no search API, so litsurvey
+reads its search page; if that page changes, only that source goes quiet).
+`--sources` restricts the set; `crossref` (all DOI-registered works) can be
+added but overlaps OpenAlex. Together they cover most of what Google
+Scholar shows, except some grey literature (technical reports, theses,
+standards). Google Scholar has no API and its terms of service forbid
+automated access, so litsurvey does not scrape it; `litsurvey search
+--scholar` prints the matching Google Scholar URL for a manual comparison,
+and the web page has the same link.
+
+## Data attribution
+
+Results come from [OpenAlex](https://openalex.org) (CC0),
+[Semantic Scholar](https://www.semanticscholar.org) (Semantic Scholar Open
+Data Platform, Allen Institute for AI), [arXiv](https://arxiv.org) (thank you
+to arXiv for use of its open access interoperability),
+[Crossref](https://www.crossref.org), the
+[IACR Cryptology ePrint Archive](https://eprint.iacr.org) and
+[Unpaywall](https://unpaywall.org). If you publish work that used these
+results, please credit them.
+
+## Author
+
+Designed by Uday Khankhoje, Department of Electrical Engineering, IIT
+Madras; implemented with Anthropic Claude. Questions and bug reports: the
+[issue tracker](https://github.com/udaykdk/litsurvey/issues).
+
+## Credits
+
+The banner of the web page uses slivers of an aerial beach photograph by
+[Lance Asper on Unsplash](https://unsplash.com/@lance_asper).
+
+## Citing
+
+See [CITATION.cff](https://github.com/udaykdk/litsurvey/blob/main/CITATION.cff).
+GitHub shows a "Cite this repository" button on the project page.
+
+## License
+
+MIT. See [LICENSE](https://github.com/udaykdk/litsurvey/blob/main/LICENSE).

litsurvey-1.0.0/docs/api-keys.md

@@ -0,0 +1,71 @@
+# API keys and running without them
+
+litsurvey uses six public services. None requires payment. Only one offers
+a key, and it is optional.
+
+| Service | Used for | Needs | Without it |
+|---|---|---|---|
+| OpenAlex | search, citation ranking, fallbacks | nothing; an email address gets the faster "polite pool" | works, slightly slower |
+| Semantic Scholar | search, citation graph, recommendations | nothing; a free API key gives a dedicated 1 request/second | works on a shared pool; frequent rate-limit retries, `related` most affected |
+| arXiv | search, full text for the agent | nothing | n/a |
+| Crossref | TechRxiv and Research Square search | nothing; the same email gets its polite pool | works, slightly slower |
+| IACR ePrint | cryptography preprint search | nothing | n/a |
+| Unpaywall | open-access lookup | an email address | works with a placeholder address |
+
+## Getting a Semantic Scholar key
+
+1. Open https://www.semanticscholar.org/product/api and find the API key
+   request form.
+2. Fill in your name, email, and a one-line purpose such as "literature
+   search for academic research". Institutional email helps.
+3. Approval arrives by email, usually within a day or two. The key looks like
+   `s2k-…` and the email states the limit: 1 request per second, cumulative
+   across all endpoints.
+
+litsurvey enforces that limit itself, across the CLI and the web page at the
+same time, so you will not be blocked for exceeding it.
+
+## Storing the key
+
+Interactive, recommended:
+
+```bash
+litsurvey init
+```
+
+This writes `~/.litsurvey/config.json` with file permissions 600 and also
+asks for your email and the default LLM backend. Or write the file yourself:
+
+```json
+{
+  "s2_api_key": "s2k-…",
+  "openalex_mailto": "you@university.edu"
+}
+```
+
+Or use environment variables, which override the file: `S2_API_KEY`,
+`OPENALEX_MAILTO`.
+
+Check with `litsurvey doctor`; look for the line
+`S2 API key : set (s2k-…)`.
+
+Never paste the key into an issue report; `--debug` output does not print it.
+
+## What the key changes
+
+With a key, searches return in a few seconds and the citation-graph and
+recommendation commands are reliable. Without a key, Semantic Scholar's
+shared pool often answers 429; litsurvey waits and retries (2, 4, 8, 16
+seconds), so a search may take half a minute at busy times, and a long agent
+run will be slow. OpenAlex and arXiv are unaffected, so `search` still
+returns results even when Semantic Scholar is refusing.
+
+## Other keys
+
+- `OPENAI_API_KEY` and `OPENAI_BASE_URL`: for the `openai` backend. Local
+  servers such as LM Studio need no key; set the base URL to
+  `http://localhost:1234`.
+- `ANTHROPIC_API_KEY`: for the `anthropic` backend.
+
+These are only used by `novelty` and `research`. See
+[llm-integration.md](llm-integration.md).