pysyrev 0.1.0__tar.gz

pysyrev-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Benjamin Pillot
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pysyrev-0.1.0/PKG-INFO

@@ -0,0 +1,222 @@
+Metadata-Version: 2.4
+Name: pysyrev
+Version: 0.1.0
+Summary: LLM-assisted PRISMA workflow for systematic literature review
+Author: Benjamin Pillot
+Author-email: benjamin.pillot@ird.fr
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Intended Audience :: Science/Research
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pandas>=2.0
+Requires-Dist: numpy>=1.24
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: rapidfuzz>=3.0
+Requires-Dist: requests>=2.28
+Requires-Dist: tqdm>=4.64
+Requires-Dist: nest-asyncio>=1.5
+Requires-Dist: pydantic>=2.0
+Requires-Dist: litellm>=1.0
+Requires-Dist: networkx>=3.0
+Requires-Dist: reportlab>=4.0
+Requires-Dist: bertopic>=0.16
+Requires-Dist: hdbscan>=0.8
+Requires-Dist: umap-learn>=0.5
+Requires-Dist: sentence-transformers>=2.0
+Requires-Dist: gensim>=4.3.0
+Requires-Dist: octis>=1.0
+Requires-Dist: nltk>=3.8
+Requires-Dist: spacy>=3.0
+Requires-Dist: beautifulsoup4>=4.11
+Requires-Dist: contractions>=0.1
+Provides-Extra: plotly
+Requires-Dist: plotly>=5.0; extra == "plotly"
+Requires-Dist: kaleido>=0.2; extra == "plotly"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: license
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+<div align="center">
+  <img src="docs/logo.png" alt="pysyrev" width="350"/>
+</div>
+
+<div align="center">
+
+[![tests](https://github.com/benjaminpillot/pysyrev/actions/workflows/tests.yml/badge.svg)](https://github.com/benjaminpillot/pysyrev/actions/workflows/tests.yml)
+[![docs](https://readthedocs.org/projects/pysyrev/badge/?version=latest)](https://pysyrev.readthedocs.io/en/latest/)
+[![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+[![python](https://img.shields.io/badge/python-≥3.10-blue)](https://www.python.org/)
+
+</div>
+
+**pysyrev** (PYthon SYstematic REView) is an automated, LLM-assisted PRISMA workflow for systematic literature reviews. It covers the full pipeline — from raw bibliographic records to screened, deduplicated, and thematically structured corpora — and produces a PDF report at the end.
+
+---
+
+## Features
+
+- **Multi-source ingestion** — Web of Science (file or REST API), OpenAlex (file or REST API), Scopus, PubMed
+- **Automatic deduplication** — fuzzy title matching across sources
+- **LLM-based title/abstract screening** — multi-reviewer workflows with majority or mean voting, powered by any provider supported by LiteLLM (Anthropic, OpenAI, Ollama, LiteLLM proxy…)
+- **Bibliographic network analysis** — bibliographic coupling and co-citation graphs exported as GraphML
+- **Topic modelling** — BERTopic-based clustering with UMAP + HDBSCAN grid search, ranked by coherence scores
+- **PDF report generation** — declarative, theme-aware PDF engine built on ReportLab
+
+---
+
+## Pipeline stages
+
+| Stage | Key | Description |
+|---|---|---|
+| Bibliography | `bib` | Fetch, clean, filter, deduplicate, and optionally resolve references |
+| LLM review | `review` | Screen documents against inclusion/exclusion criteria with one or more LLM reviewers |
+| Bibliographic network | `bib-network` | Build coupling and co-citation networks from the included corpus |
+| Topic modelling | `topic-model` | Cluster documents into topics using BERTopic; rank configurations by coherence |
+| Report | `topic-report` | Generate a PDF report from the selected topic model run |
+
+All sections are optional — only the stages declared in the config file are executed. Each stage auto-detects the most recent output of the previous one when run standalone.
+
+---
+
+## Installation
+
+> **Prerequisite:** Python ≥ 3.10.
+
+```bash
+# 1. Clone the repository
+git clone <repo-url>
+cd pysyrev
+
+# 2. Install dependencies
+pip install -r requirements.txt
+
+# 3. Install pysyrev itself (editable mode recommended for development)
+pip install -e .
+```
+
+To enable Plotly figure embedding in PDF reports, install the optional extras:
+
+```bash
+pip install -e ".[plotly]"
+```
+
+## Documentation
+Documentation is available from [here](https://pysyrev.readthedocs.io/en/latest/)
+
+## Quick start
+
+### CLI
+
+```bash
+# Run all configured stages (only stages present in the config are executed)
+python -m pysyrev config.yaml
+
+# Run a single stage
+python -m pysyrev config.yaml --stage bib
+python -m pysyrev config.yaml --stage review
+python -m pysyrev config.yaml --stage bib-network
+python -m pysyrev config.yaml --stage topic-model
+python -m pysyrev config.yaml --stage topic-report
+```
+
+If installed via `setup.py`, the `pysyrev` command is also available directly:
+
+```bash
+pysyrev config.yaml --stage topic-report
+```
+
+### Python API
+
+```python
+from pysyrev import Pipeline
+
+# Full pipeline in one call — runs only the stages declared in the config
+pipeline = Pipeline.from_config("config.yaml")
+pipeline.run()
+
+# Or stage by stage — results persist on the instance between calls
+pipeline.run(stages=["bib"])
+pipeline.run(stages=["review"])        # uses pipeline.bib.dataset automatically
+pipeline.run(stages=["topic-report"])  # generates the PDF report
+
+# Access results
+df_all    = pipeline.bib.dataset          # pd.DataFrame — all collected documents
+df_kept   = pipeline.review.included_docs # pd.DataFrame — LLM-screened inclusions
+network   = pipeline.network              # BibNetwork
+topic     = pipeline.topic                # TopicModel
+report    = pipeline.report               # TopicReport
+```
+
+### Report-only run
+
+A config containing only the `topic_report` (and optionally `report` and `llm`) sections is valid. This lets you generate or regenerate a report from a previous topic-model run without re-running the full pipeline:
+
+```yaml
+# report_only.yaml
+topic_report:
+  run_dir: /path/to/topic_modeling/run_2026-05-01T120000/  # or leave blank to auto-detect
+  model_index: 0
+  export_to: /path/to/output/report/
+```
+
+```bash
+python -m pysyrev report_only.yaml
+```
+
+---
+
+## Configuration
+
+A single YAML file controls all stages. Copy `pysyrev/config_examples/config_template.yaml` and fill in the sections you need. Sections not present in the file are simply skipped.
+
+Key auto-detection rules (when fields are left blank):
+
+| Blank field | Auto-detected from |
+|---|---|
+| `review.doc_dataset` | latest run in `bib.export.export_dir` |
+| `bib_network.doc_dataset` | latest run in `review.export.export_dir` |
+| `topic_model.doc_dataset` | latest run in `review.export.export_dir` |
+| `topic_report.run_dir` | latest run in `topic_model.export.export_dir` |
+| bib-network graphs in report | latest run in `bib_network.export.export_dir` |
+
+---
+
+## Getting started
+
+See the `tutorials/` folder for step-by-step Jupyter notebooks and annotated configuration examples covering each pipeline stage.
+
+---
+
+---
+
+<div align="center">
+  <img src="docs/espace-dev-ird.png" alt="organizations" width="600"/>
+</div>
+
+---
+
+## Contributing
+
+### Development and improvement
+
+- Benjamin Pillot
+- Théo Chamarande
+- Kevin Chapuis
+
+### Conceptualization and Coordination
+
+- Benjamin Pillot

pysyrev-0.1.0/README.md

@@ -0,0 +1,170 @@
+<div align="center">
+  <img src="docs/logo.png" alt="pysyrev" width="350"/>
+</div>
+
+<div align="center">
+
+[![tests](https://github.com/benjaminpillot/pysyrev/actions/workflows/tests.yml/badge.svg)](https://github.com/benjaminpillot/pysyrev/actions/workflows/tests.yml)
+[![docs](https://readthedocs.org/projects/pysyrev/badge/?version=latest)](https://pysyrev.readthedocs.io/en/latest/)
+[![license](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
+[![python](https://img.shields.io/badge/python-≥3.10-blue)](https://www.python.org/)
+
+</div>
+
+**pysyrev** (PYthon SYstematic REView) is an automated, LLM-assisted PRISMA workflow for systematic literature reviews. It covers the full pipeline — from raw bibliographic records to screened, deduplicated, and thematically structured corpora — and produces a PDF report at the end.
+
+---
+
+## Features
+
+- **Multi-source ingestion** — Web of Science (file or REST API), OpenAlex (file or REST API), Scopus, PubMed
+- **Automatic deduplication** — fuzzy title matching across sources
+- **LLM-based title/abstract screening** — multi-reviewer workflows with majority or mean voting, powered by any provider supported by LiteLLM (Anthropic, OpenAI, Ollama, LiteLLM proxy…)
+- **Bibliographic network analysis** — bibliographic coupling and co-citation graphs exported as GraphML
+- **Topic modelling** — BERTopic-based clustering with UMAP + HDBSCAN grid search, ranked by coherence scores
+- **PDF report generation** — declarative, theme-aware PDF engine built on ReportLab
+
+---
+
+## Pipeline stages
+
+| Stage | Key | Description |
+|---|---|---|
+| Bibliography | `bib` | Fetch, clean, filter, deduplicate, and optionally resolve references |
+| LLM review | `review` | Screen documents against inclusion/exclusion criteria with one or more LLM reviewers |
+| Bibliographic network | `bib-network` | Build coupling and co-citation networks from the included corpus |
+| Topic modelling | `topic-model` | Cluster documents into topics using BERTopic; rank configurations by coherence |
+| Report | `topic-report` | Generate a PDF report from the selected topic model run |
+
+All sections are optional — only the stages declared in the config file are executed. Each stage auto-detects the most recent output of the previous one when run standalone.
+
+---
+
+## Installation
+
+> **Prerequisite:** Python ≥ 3.10.
+
+```bash
+# 1. Clone the repository
+git clone <repo-url>
+cd pysyrev
+
+# 2. Install dependencies
+pip install -r requirements.txt
+
+# 3. Install pysyrev itself (editable mode recommended for development)
+pip install -e .
+```
+
+To enable Plotly figure embedding in PDF reports, install the optional extras:
+
+```bash
+pip install -e ".[plotly]"
+```
+
+## Documentation
+Documentation is available from [here](https://pysyrev.readthedocs.io/en/latest/)
+
+## Quick start
+
+### CLI
+
+```bash
+# Run all configured stages (only stages present in the config are executed)
+python -m pysyrev config.yaml
+
+# Run a single stage
+python -m pysyrev config.yaml --stage bib
+python -m pysyrev config.yaml --stage review
+python -m pysyrev config.yaml --stage bib-network
+python -m pysyrev config.yaml --stage topic-model
+python -m pysyrev config.yaml --stage topic-report
+```
+
+If installed via `setup.py`, the `pysyrev` command is also available directly:
+
+```bash
+pysyrev config.yaml --stage topic-report
+```
+
+### Python API
+
+```python
+from pysyrev import Pipeline
+
+# Full pipeline in one call — runs only the stages declared in the config
+pipeline = Pipeline.from_config("config.yaml")
+pipeline.run()
+
+# Or stage by stage — results persist on the instance between calls
+pipeline.run(stages=["bib"])
+pipeline.run(stages=["review"])        # uses pipeline.bib.dataset automatically
+pipeline.run(stages=["topic-report"])  # generates the PDF report
+
+# Access results
+df_all    = pipeline.bib.dataset          # pd.DataFrame — all collected documents
+df_kept   = pipeline.review.included_docs # pd.DataFrame — LLM-screened inclusions
+network   = pipeline.network              # BibNetwork
+topic     = pipeline.topic                # TopicModel
+report    = pipeline.report               # TopicReport
+```
+
+### Report-only run
+
+A config containing only the `topic_report` (and optionally `report` and `llm`) sections is valid. This lets you generate or regenerate a report from a previous topic-model run without re-running the full pipeline:
+
+```yaml
+# report_only.yaml
+topic_report:
+  run_dir: /path/to/topic_modeling/run_2026-05-01T120000/  # or leave blank to auto-detect
+  model_index: 0
+  export_to: /path/to/output/report/
+```
+
+```bash
+python -m pysyrev report_only.yaml
+```
+
+---
+
+## Configuration
+
+A single YAML file controls all stages. Copy `pysyrev/config_examples/config_template.yaml` and fill in the sections you need. Sections not present in the file are simply skipped.
+
+Key auto-detection rules (when fields are left blank):
+
+| Blank field | Auto-detected from |
+|---|---|
+| `review.doc_dataset` | latest run in `bib.export.export_dir` |
+| `bib_network.doc_dataset` | latest run in `review.export.export_dir` |
+| `topic_model.doc_dataset` | latest run in `review.export.export_dir` |
+| `topic_report.run_dir` | latest run in `topic_model.export.export_dir` |
+| bib-network graphs in report | latest run in `bib_network.export.export_dir` |
+
+---
+
+## Getting started
+
+See the `tutorials/` folder for step-by-step Jupyter notebooks and annotated configuration examples covering each pipeline stage.
+
+---
+
+---
+
+<div align="center">
+  <img src="docs/espace-dev-ird.png" alt="organizations" width="600"/>
+</div>
+
+---
+
+## Contributing
+
+### Development and improvement
+
+- Benjamin Pillot
+- Théo Chamarande
+- Kevin Chapuis
+
+### Conceptualization and Coordination
+
+- Benjamin Pillot

pysyrev-0.1.0/pysyrev/__init__.py

@@ -0,0 +1,6 @@
+from pysyrev.pipeline import Pipeline, ALL_STAGES
+from pysyrev.bibdata import BibDataset, WosDataset, OpenAlexDataset, ScopusDataset, PubmedDataset
+from pysyrev.topic_model import TopicModel
+from pysyrev.network import BibNetwork
+from pysyrev.topic_report import TopicReport
+from pysyrev.review import LLMReview, ReviewedDataset

pysyrev-0.1.0/pysyrev/__main__.py

@@ -0,0 +1,81 @@
+"""
+pysyrev CLI — run the literature review pipeline from a YAML config.
+
+Usage
+-----
+  # Run all configured stages (only stages present in the config are executed)
+  python -m pysyrev config.yaml
+
+  # Individual stages
+  python -m pysyrev config.yaml --stage bib
+  python -m pysyrev config.yaml --stage review
+  python -m pysyrev config.yaml --stage bib-network
+  python -m pysyrev config.yaml --stage topic-model
+  python -m pysyrev config.yaml --stage topic-report
+
+When running a single stage, the input dataset is read from the
+``doc_dataset`` field of that stage's config section.  If ``doc_dataset``
+is left blank, Config.load falls back to auto-detecting the most recent
+output of the previous stage (e.g. the latest ``reviewed_included.csv``
+for bib-network / topic-model, or the latest topic_model run for
+topic-report).
+"""
+
+import argparse
+
+from pysyrev import ALL_STAGES, Pipeline
+
+
+def _print_stage_result(stage, pipeline):
+    if stage == 'bib' and pipeline.bib is not None:
+        print(f"[bib] Done — {len(pipeline.bib.dataset)} documents.")
+    elif stage == 'review' and pipeline.review is not None:
+        print(f"[review] Done — {len(pipeline.review.included_docs)} documents included.")
+    elif stage == 'bib-network' and pipeline.network is not None:
+        net = pipeline.network
+        if pipeline.config.bib_network.export is not None:
+            print(
+                f"[bib-network] Done — coupling: {net.n_coupling_nodes} nodes / "
+                f"{net.n_coupling_edges} edges, "
+                f"co-citation: {net.n_cocitation_nodes} nodes / "
+                f"{net.n_cocitation_edges} edges."
+            )
+        else:
+            print("[bib-network] Done (no export configured).")
+    elif stage == 'topic-model':
+        print("[topic-model] Done.")
+    elif stage == 'topic-report' and pipeline.report is not None:
+        print(f"[topic-report] Done — report written to {pipeline.report.export_to}")
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        prog='python -m pysyrev',
+        description='Systematic literature review pipeline.',
+    )
+    parser.add_argument('config', help='Path to the YAML pipeline config file.')
+    parser.add_argument(
+        '--stage',
+        choices=ALL_STAGES + ['all'],
+        default='all',
+        metavar='STAGE',
+        help=(
+            'Stage to run: bib | review | bib-network | topic-model | topic-report | all '
+            '(default: all — runs only the stages present in the config file).'
+        ),
+    )
+    args = parser.parse_args()
+
+    pipeline = Pipeline.from_config(args.config)
+
+    if args.stage == 'all':
+        print("Starting pipeline…")
+        pipeline.run()
+    else:
+        print(f"[{args.stage}] Starting…")
+        pipeline.run(stages=[args.stage])
+        _print_stage_result(args.stage, pipeline)
+
+
+if __name__ == '__main__':
+    main()