papersift 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- papersift-0.1.0/LICENSE +21 -0
- papersift-0.1.0/PKG-INFO +249 -0
- papersift-0.1.0/README.md +216 -0
- papersift-0.1.0/papersift/__init__.py +8 -0
- papersift-0.1.0/papersift/cli.py +365 -0
- papersift-0.1.0/papersift/core.py +216 -0
- papersift-0.1.0/papersift/download.py +66 -0
- papersift-0.1.0/papersift/gui/__init__.py +1 -0
- papersift-0.1.0/papersift/gui/app.py +328 -0
- papersift-0.1.0/papersift/gui/ollama.py +36 -0
- papersift-0.1.0/papersift/keywords.py +40 -0
- papersift-0.1.0/papersift/models.py +35 -0
- papersift-0.1.0/papersift/providers/__init__.py +22 -0
- papersift-0.1.0/papersift/providers/_http.py +64 -0
- papersift-0.1.0/papersift/providers/base.py +50 -0
- papersift-0.1.0/papersift/providers/fallback.py +29 -0
- papersift-0.1.0/papersift/providers/gemini.py +78 -0
- papersift-0.1.0/papersift/providers/mock.py +35 -0
- papersift-0.1.0/papersift/providers/ollama.py +50 -0
- papersift-0.1.0/papersift/providers/openai_compat.py +70 -0
- papersift-0.1.0/papersift/providers/registry.py +59 -0
- papersift-0.1.0/papersift/ranking/__init__.py +6 -0
- papersift-0.1.0/papersift/ranking/abstract.py +21 -0
- papersift-0.1.0/papersift/ranking/fulltext.py +127 -0
- papersift-0.1.0/papersift/ranking/rubric.py +122 -0
- papersift-0.1.0/papersift/sources/__init__.py +19 -0
- papersift-0.1.0/papersift/sources/arxiv.py +78 -0
- papersift-0.1.0/papersift/store.py +90 -0
- papersift-0.1.0/papersift/utils.py +38 -0
- papersift-0.1.0/papersift.egg-info/PKG-INFO +249 -0
- papersift-0.1.0/papersift.egg-info/SOURCES.txt +51 -0
- papersift-0.1.0/papersift.egg-info/dependency_links.txt +1 -0
- papersift-0.1.0/papersift.egg-info/entry_points.txt +2 -0
- papersift-0.1.0/papersift.egg-info/requires.txt +14 -0
- papersift-0.1.0/papersift.egg-info/top_level.txt +1 -0
- papersift-0.1.0/pyproject.toml +66 -0
- papersift-0.1.0/setup.cfg +4 -0
- papersift-0.1.0/tests/test_arxiv_source.py +82 -0
- papersift-0.1.0/tests/test_cli.py +42 -0
- papersift-0.1.0/tests/test_core.py +110 -0
- papersift-0.1.0/tests/test_core_fulltext.py +87 -0
- papersift-0.1.0/tests/test_download.py +50 -0
- papersift-0.1.0/tests/test_fulltext.py +86 -0
- papersift-0.1.0/tests/test_gemini.py +45 -0
- papersift-0.1.0/tests/test_http_retry.py +51 -0
- papersift-0.1.0/tests/test_ollama.py +34 -0
- papersift-0.1.0/tests/test_openai_compat.py +47 -0
- papersift-0.1.0/tests/test_providers_base.py +33 -0
- papersift-0.1.0/tests/test_ranking_abstract.py +59 -0
- papersift-0.1.0/tests/test_registry_fallback.py +75 -0
- papersift-0.1.0/tests/test_smoke.py +5 -0
- papersift-0.1.0/tests/test_store.py +40 -0
- papersift-0.1.0/tests/test_utils_keywords.py +43 -0
papersift-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Seif El Islem
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
papersift-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,249 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: papersift
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: An open-source LLM-powered research assistant for discovering, evaluating, ranking, and organizing academic papers.
|
|
5
|
+
Author: Abedelkader Seif El Islem Rahmani, Mohammed Bennour
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/RAHAMNIabdelkaderseifelislem/papersift
|
|
8
|
+
Project-URL: Repository, https://github.com/RAHAMNIabdelkaderseifelislem/papersift
|
|
9
|
+
Project-URL: Issues, https://github.com/RAHAMNIabdelkaderseifelislem/papersift/issues
|
|
10
|
+
Project-URL: Changelog, https://github.com/RAHAMNIabdelkaderseifelislem/papersift/blob/main/CHANGELOG.md
|
|
11
|
+
Keywords: academic-research,research-assistant,literature-review,systematic-review,arxiv,paper-discovery,paper-ranking,llm,ai-agents,research-automation,ollama,gemini,openai-compatible,scientific-research
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
17
|
+
Classifier: Intended Audience :: Science/Research
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
License-File: LICENSE
|
|
21
|
+
Requires-Dist: arxiv>=2.0
|
|
22
|
+
Requires-Dist: requests>=2.31
|
|
23
|
+
Requires-Dist: pydantic>=2.0
|
|
24
|
+
Requires-Dist: python-dotenv>=1.0
|
|
25
|
+
Provides-Extra: gui
|
|
26
|
+
Requires-Dist: streamlit>=1.30; extra == "gui"
|
|
27
|
+
Requires-Dist: pandas>=2.0; extra == "gui"
|
|
28
|
+
Provides-Extra: fulltext
|
|
29
|
+
Requires-Dist: pymupdf>=1.23; extra == "fulltext"
|
|
30
|
+
Provides-Extra: dev
|
|
31
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
32
|
+
Dynamic: license-file
|
|
33
|
+
|
|
34
|
+
# PaperSift
|
|
35
|
+
|
|
36
|
+
**An Open-Source AI Research Assistant for Academic Paper Discovery and Evaluation.**
|
|
37
|
+
|
|
38
|
+
PaperSift is an LLM-powered research assistant that helps researchers
|
|
39
|
+
discover, evaluate, rank, and organize academic papers based on their
|
|
40
|
+
research interests.
|
|
41
|
+
|
|
42
|
+
It combines academic search, structured relevance assessment,
|
|
43
|
+
multi-provider LLM integration, and automated paper retrieval
|
|
44
|
+
into a unified workflow.
|
|
45
|
+
|
|
46
|
+
> **Project status:** Pre-release / alpha.
|
|
47
|
+
>
|
|
48
|
+
> The core CLI, multi-provider support, two-stage ranking, and caching
|
|
49
|
+
> are implemented. The graphical interface is being integrated.
|
|
50
|
+
> The first public PyPI release is planned for version 0.1.0.
|
|
51
|
+
|
|
52
|
+
## Why
|
|
53
|
+
|
|
54
|
+
Starting a literature review means reading hundreds of abstracts to find the few papers worth your time. `papersift` was first built at the start of a PhD to do that triage automatically, and it was used to select the papers behind a published systematic survey (see [Citation](#citation)).
|
|
55
|
+
|
|
56
|
+
## Features
|
|
57
|
+
|
|
58
|
+
- **Project-aware relevance scoring.** You describe your research project once; every paper is scored against it.
|
|
59
|
+
- **Two-stage ranking.** Every result is triaged from its abstract; with `--fulltext`, the best ones are then read in full (introduction, method and conclusion) for a more reliable score.
|
|
60
|
+
- **Rubric-based scores.** The model only fills a small rubric (topic match, method match, novelty); the 0-100 score is computed in code, so it is reproducible and comparable across models.
|
|
61
|
+
- **Many LLM providers.** Local Ollama, Gemini, and any OpenAI-compatible API (Groq, OpenRouter, Together, Cerebras, Mistral, OpenAI, or your own `--base-url`), with automatic retry on rate limits and optional fallback to a backup provider.
|
|
62
|
+
- **Explainable results.** Each saved paper comes with its scores, strengths, limitations and reasoning.
|
|
63
|
+
- **Local cache.** Papers already scored are not scored again, so re-running is cheap.
|
|
64
|
+
- **Optional GUI**.
|
|
65
|
+
|
|
66
|
+
## Installation
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
For installation, use:
|
|
70
|
+
```bash
|
|
71
|
+
pip install papersift
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
For the GUI:
|
|
76
|
+
```bash
|
|
77
|
+
pip install "papersift[gui]"
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
For full-text PDF processing:
|
|
82
|
+
```bash
|
|
83
|
+
pip install "papersift[fulltext]"
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
For both optional features:
|
|
87
|
+
```bash
|
|
88
|
+
pip install "papersift[gui,fulltext]"
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
For developers:
|
|
93
|
+
```bash
|
|
94
|
+
pip install "papersift[dev,gui,fulltext]"
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## Quick start
|
|
98
|
+
|
|
99
|
+
### Command line
|
|
100
|
+
|
|
101
|
+
With a local model (no API key):
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
papersift run "Generative AI methods for autonomous agents" --provider ollama --model <model-name>
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
With a free online provider:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
export GROQ_API_KEY=... # Windows PowerShell: $env:GROQ_API_KEY="..."
|
|
111
|
+
papersift run "Generative AI methods for autonomous agents" \
|
|
112
|
+
--provider groq --model <model-name> \
|
|
113
|
+
--threshold 80 --since-days 365 --fulltext -v
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
With a backup provider if the first one fails or hits its rate limit:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
papersift run "my project" --provider groq --model <model-a> --fallback gemini=<model-b>
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
List the available providers with `papersift providers`, and all options with `papersift run --help`.
|
|
123
|
+
|
|
124
|
+
### Python
|
|
125
|
+
|
|
126
|
+
```python
|
|
127
|
+
from papersift import Scout
|
|
128
|
+
from papersift.providers import create_provider
|
|
129
|
+
|
|
130
|
+
provider = create_provider("groq", "<model-name>")
|
|
131
|
+
scout = Scout(
|
|
132
|
+
"Generative AI methods for autonomous agents",
|
|
133
|
+
provider,
|
|
134
|
+
relevance_threshold=80,
|
|
135
|
+
fulltext=True,
|
|
136
|
+
)
|
|
137
|
+
for result in scout.run():
|
|
138
|
+
print(result.score, result.paper.title, result.pdf_path)
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Model names change often, so look up the current free models on your provider's website.
|
|
142
|
+
|
|
143
|
+
## Configuration
|
|
144
|
+
|
|
145
|
+
API keys are read from environment variables, never from code or command-line arguments. Copy `.env.example` to `.env` and fill in only the providers you use; `papersift` loads `.env` automatically.
|
|
146
|
+
|
|
147
|
+
| Provider | Variable |
|
|
148
|
+
|---|---|
|
|
149
|
+
| `gemini` | `GEMINI_API_KEY` |
|
|
150
|
+
| `groq` | `GROQ_API_KEY` |
|
|
151
|
+
| `openrouter` | `OPENROUTER_API_KEY` |
|
|
152
|
+
| `together` | `TOGETHER_API_KEY` |
|
|
153
|
+
| `cerebras` | `CEREBRAS_API_KEY` |
|
|
154
|
+
| `mistral` | `MISTRAL_API_KEY` |
|
|
155
|
+
| `openai` | `OPENAI_API_KEY` |
|
|
156
|
+
| `openai_compat` (with `--base-url`) | `OPENAI_COMPAT_API_KEY` |
|
|
157
|
+
| `ollama` | none (the Ollama server must be running) |
|
|
158
|
+
|
|
159
|
+
> Never commit your `.env` file. It is already listed in `.gitignore`.
|
|
160
|
+
|
|
161
|
+
## Output
|
|
162
|
+
|
|
163
|
+
Results are saved in the folder you choose (default `papersift_output/`):
|
|
164
|
+
|
|
165
|
+
```
|
|
166
|
+
papersift_output/
|
|
167
|
+
pdfs/ # downloaded PDFs, named <score>-<year>-<title>.pdf
|
|
168
|
+
metadata/ # one JSON file per paper: authors, abstract, arXiv id,
|
|
169
|
+
# scores and the reasoning behind them
|
|
170
|
+
.cache/ # SQLite cache and temporary downloads
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
## How it works
|
|
174
|
+
|
|
175
|
+
1. An LLM turns your project description into precise academic search keywords.
|
|
176
|
+
2. The arXiv API returns candidate papers from the date window you set.
|
|
177
|
+
3. **Stage 1:** each abstract is scored against your project with a rubric.
|
|
178
|
+
4. **Stage 2** (`--fulltext`): the best papers are downloaded and scored again from their full text; both scores are blended (the full text counts more).
|
|
179
|
+
5. Papers above your threshold are saved with their metadata and explanations.
|
|
180
|
+
|
|
181
|
+
## Roadmap
|
|
182
|
+
|
|
183
|
+
### Core
|
|
184
|
+
- [x] arXiv paper discovery
|
|
185
|
+
- [x] Multi-provider LLM integration
|
|
186
|
+
- [x] Retry and provider fallback
|
|
187
|
+
- [x] Rubric-based relevance scoring
|
|
188
|
+
- [x] SQLite caching and deduplication
|
|
189
|
+
- [x] Command-line interface
|
|
190
|
+
- [x] Optional full-text ranking
|
|
191
|
+
|
|
192
|
+
### Application
|
|
193
|
+
- [ ] Graphical user interface integration
|
|
194
|
+
- [ ] Unified CLI and GUI configuration
|
|
195
|
+
- [ ] End-to-end application testing
|
|
196
|
+
|
|
197
|
+
### Distribution
|
|
198
|
+
- [ ] First PyPI release (0.1.0)
|
|
199
|
+
- [ ] Installation and packaging verification
|
|
200
|
+
- [ ] Public documentation
|
|
201
|
+
|
|
202
|
+
### Future
|
|
203
|
+
- [ ] Additional academic sources (Semantic Scholar, OpenAlex)
|
|
204
|
+
- [ ] Extended export formats
|
|
205
|
+
- [ ] Improved research project management
|
|
206
|
+
|
|
207
|
+
## Contributing
|
|
208
|
+
|
|
209
|
+
Contributions are welcome, from first-time contributors too.
|
|
210
|
+
|
|
211
|
+
1. Open an issue to discuss what you want to work on, or pick one labeled `good first issue`.
|
|
212
|
+
2. Create a branch from `main` (`git checkout -b feature/short-description`).
|
|
213
|
+
3. Make your change and add or update tests (`pytest`).
|
|
214
|
+
4. Open a pull request describing what you changed and why.
|
|
215
|
+
|
|
216
|
+
Please keep pull requests small and focused so they are easy to review.
|
|
217
|
+
|
|
218
|
+
## Citation
|
|
219
|
+
|
|
220
|
+
PaperSift was developed to support academic literature discovery
|
|
221
|
+
and research paper selection.
|
|
222
|
+
|
|
223
|
+
If you use PaperSift in your research, please cite the software
|
|
224
|
+
using the citation information provided in `CITATION.cff`.
|
|
225
|
+
|
|
226
|
+
### Associated research publication
|
|
227
|
+
|
|
228
|
+
The development of PaperSift supported the literature discovery
|
|
229
|
+
and paper selection process for the following publication:
|
|
230
|
+
|
|
231
|
+
Rahmani, Abedelkader Seif El Islem, Yahiaoui, Yasser & Bouziane,
|
|
232
|
+
Abdelghani.
|
|
233
|
+
|
|
234
|
+
"Architectural Patterns for Self-Improving Intelligent Agents:
|
|
235
|
+
A Systematic Survey of Generative and Cognitive Approaches."
|
|
236
|
+
|
|
237
|
+
SN Computer Science, 7, 746 (2026).
|
|
238
|
+
|
|
239
|
+
https://doi.org/10.1007/s42979-026-05333-6
|
|
240
|
+
|
|
241
|
+
|
|
242
|
+
## Software Authors
|
|
243
|
+
|
|
244
|
+
- Abedelkader Seif El Islem Rahmani
|
|
245
|
+
- Mohammed Bennour
|
|
246
|
+
|
|
247
|
+
## License
|
|
248
|
+
|
|
249
|
+
Released under the MIT License. See `LICENSE` for details.
|
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
# PaperSift
|
|
2
|
+
|
|
3
|
+
**An Open-Source AI Research Assistant for Academic Paper Discovery and Evaluation.**
|
|
4
|
+
|
|
5
|
+
PaperSift is an LLM-powered research assistant that helps researchers
|
|
6
|
+
discover, evaluate, rank, and organize academic papers based on their
|
|
7
|
+
research interests.
|
|
8
|
+
|
|
9
|
+
It combines academic search, structured relevance assessment,
|
|
10
|
+
multi-provider LLM integration, and automated paper retrieval
|
|
11
|
+
into a unified workflow.
|
|
12
|
+
|
|
13
|
+
> **Project status:** Pre-release / alpha.
|
|
14
|
+
>
|
|
15
|
+
> The core CLI, multi-provider support, two-stage ranking, and caching
|
|
16
|
+
> are implemented. The graphical interface is being integrated.
|
|
17
|
+
> The first public PyPI release is planned for version 0.1.0.
|
|
18
|
+
|
|
19
|
+
## Why
|
|
20
|
+
|
|
21
|
+
Starting a literature review means reading hundreds of abstracts to find the few papers worth your time. `papersift` was first built at the start of a PhD to do that triage automatically, and it was used to select the papers behind a published systematic survey (see [Citation](#citation)).
|
|
22
|
+
|
|
23
|
+
## Features
|
|
24
|
+
|
|
25
|
+
- **Project-aware relevance scoring.** You describe your research project once; every paper is scored against it.
|
|
26
|
+
- **Two-stage ranking.** Every result is triaged from its abstract; with `--fulltext`, the best ones are then read in full (introduction, method and conclusion) for a more reliable score.
|
|
27
|
+
- **Rubric-based scores.** The model only fills a small rubric (topic match, method match, novelty); the 0-100 score is computed in code, so it is reproducible and comparable across models.
|
|
28
|
+
- **Many LLM providers.** Local Ollama, Gemini, and any OpenAI-compatible API (Groq, OpenRouter, Together, Cerebras, Mistral, OpenAI, or your own `--base-url`), with automatic retry on rate limits and optional fallback to a backup provider.
|
|
29
|
+
- **Explainable results.** Each saved paper comes with its scores, strengths, limitations and reasoning.
|
|
30
|
+
- **Local cache.** Papers already scored are not scored again, so re-running is cheap.
|
|
31
|
+
- **Optional GUI**.
|
|
32
|
+
|
|
33
|
+
## Installation
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
For installation, use:
|
|
37
|
+
```bash
|
|
38
|
+
pip install papersift
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
For the GUI:
|
|
43
|
+
```bash
|
|
44
|
+
pip install "papersift[gui]"
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
For full-text PDF processing:
|
|
49
|
+
```bash
|
|
50
|
+
pip install "papersift[fulltext]"
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
For both optional features:
|
|
54
|
+
```bash
|
|
55
|
+
pip install "papersift[gui,fulltext]"
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
For developers:
|
|
60
|
+
```bash
|
|
61
|
+
pip install "papersift[dev,gui,fulltext]"
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Quick start
|
|
65
|
+
|
|
66
|
+
### Command line
|
|
67
|
+
|
|
68
|
+
With a local model (no API key):
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
papersift run "Generative AI methods for autonomous agents" --provider ollama --model <model-name>
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
With a free online provider:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
export GROQ_API_KEY=... # Windows PowerShell: $env:GROQ_API_KEY="..."
|
|
78
|
+
papersift run "Generative AI methods for autonomous agents" \
|
|
79
|
+
--provider groq --model <model-name> \
|
|
80
|
+
--threshold 80 --since-days 365 --fulltext -v
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
With a backup provider if the first one fails or hits its rate limit:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
papersift run "my project" --provider groq --model <model-a> --fallback gemini=<model-b>
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
List the available providers with `papersift providers`, and all options with `papersift run --help`.
|
|
90
|
+
|
|
91
|
+
### Python
|
|
92
|
+
|
|
93
|
+
```python
|
|
94
|
+
from papersift import Scout
|
|
95
|
+
from papersift.providers import create_provider
|
|
96
|
+
|
|
97
|
+
provider = create_provider("groq", "<model-name>")
|
|
98
|
+
scout = Scout(
|
|
99
|
+
"Generative AI methods for autonomous agents",
|
|
100
|
+
provider,
|
|
101
|
+
relevance_threshold=80,
|
|
102
|
+
fulltext=True,
|
|
103
|
+
)
|
|
104
|
+
for result in scout.run():
|
|
105
|
+
print(result.score, result.paper.title, result.pdf_path)
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Model names change often, so look up the current free models on your provider's website.
|
|
109
|
+
|
|
110
|
+
## Configuration
|
|
111
|
+
|
|
112
|
+
API keys are read from environment variables, never from code or command-line arguments. Copy `.env.example` to `.env` and fill in only the providers you use; `papersift` loads `.env` automatically.
|
|
113
|
+
|
|
114
|
+
| Provider | Variable |
|
|
115
|
+
|---|---|
|
|
116
|
+
| `gemini` | `GEMINI_API_KEY` |
|
|
117
|
+
| `groq` | `GROQ_API_KEY` |
|
|
118
|
+
| `openrouter` | `OPENROUTER_API_KEY` |
|
|
119
|
+
| `together` | `TOGETHER_API_KEY` |
|
|
120
|
+
| `cerebras` | `CEREBRAS_API_KEY` |
|
|
121
|
+
| `mistral` | `MISTRAL_API_KEY` |
|
|
122
|
+
| `openai` | `OPENAI_API_KEY` |
|
|
123
|
+
| `openai_compat` (with `--base-url`) | `OPENAI_COMPAT_API_KEY` |
|
|
124
|
+
| `ollama` | none (the Ollama server must be running) |
|
|
125
|
+
|
|
126
|
+
> Never commit your `.env` file. It is already listed in `.gitignore`.
|
|
127
|
+
|
|
128
|
+
## Output
|
|
129
|
+
|
|
130
|
+
Results are saved in the folder you choose (default `papersift_output/`):
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
papersift_output/
|
|
134
|
+
pdfs/ # downloaded PDFs, named <score>-<year>-<title>.pdf
|
|
135
|
+
metadata/ # one JSON file per paper: authors, abstract, arXiv id,
|
|
136
|
+
# scores and the reasoning behind them
|
|
137
|
+
.cache/ # SQLite cache and temporary downloads
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
## How it works
|
|
141
|
+
|
|
142
|
+
1. An LLM turns your project description into precise academic search keywords.
|
|
143
|
+
2. The arXiv API returns candidate papers from the date window you set.
|
|
144
|
+
3. **Stage 1:** each abstract is scored against your project with a rubric.
|
|
145
|
+
4. **Stage 2** (`--fulltext`): the best papers are downloaded and scored again from their full text; both scores are blended (the full text counts more).
|
|
146
|
+
5. Papers above your threshold are saved with their metadata and explanations.
|
|
147
|
+
|
|
148
|
+
## Roadmap
|
|
149
|
+
|
|
150
|
+
### Core
|
|
151
|
+
- [x] arXiv paper discovery
|
|
152
|
+
- [x] Multi-provider LLM integration
|
|
153
|
+
- [x] Retry and provider fallback
|
|
154
|
+
- [x] Rubric-based relevance scoring
|
|
155
|
+
- [x] SQLite caching and deduplication
|
|
156
|
+
- [x] Command-line interface
|
|
157
|
+
- [x] Optional full-text ranking
|
|
158
|
+
|
|
159
|
+
### Application
|
|
160
|
+
- [ ] Graphical user interface integration
|
|
161
|
+
- [ ] Unified CLI and GUI configuration
|
|
162
|
+
- [ ] End-to-end application testing
|
|
163
|
+
|
|
164
|
+
### Distribution
|
|
165
|
+
- [ ] First PyPI release (0.1.0)
|
|
166
|
+
- [ ] Installation and packaging verification
|
|
167
|
+
- [ ] Public documentation
|
|
168
|
+
|
|
169
|
+
### Future
|
|
170
|
+
- [ ] Additional academic sources (Semantic Scholar, OpenAlex)
|
|
171
|
+
- [ ] Extended export formats
|
|
172
|
+
- [ ] Improved research project management
|
|
173
|
+
|
|
174
|
+
## Contributing
|
|
175
|
+
|
|
176
|
+
Contributions are welcome, from first-time contributors too.
|
|
177
|
+
|
|
178
|
+
1. Open an issue to discuss what you want to work on, or pick one labeled `good first issue`.
|
|
179
|
+
2. Create a branch from `main` (`git checkout -b feature/short-description`).
|
|
180
|
+
3. Make your change and add or update tests (`pytest`).
|
|
181
|
+
4. Open a pull request describing what you changed and why.
|
|
182
|
+
|
|
183
|
+
Please keep pull requests small and focused so they are easy to review.
|
|
184
|
+
|
|
185
|
+
## Citation
|
|
186
|
+
|
|
187
|
+
PaperSift was developed to support academic literature discovery
|
|
188
|
+
and research paper selection.
|
|
189
|
+
|
|
190
|
+
If you use PaperSift in your research, please cite the software
|
|
191
|
+
using the citation information provided in `CITATION.cff`.
|
|
192
|
+
|
|
193
|
+
### Associated research publication
|
|
194
|
+
|
|
195
|
+
The development of PaperSift supported the literature discovery
|
|
196
|
+
and paper selection process for the following publication:
|
|
197
|
+
|
|
198
|
+
Rahmani, Abedelkader Seif El Islem, Yahiaoui, Yasser & Bouziane,
|
|
199
|
+
Abdelghani.
|
|
200
|
+
|
|
201
|
+
"Architectural Patterns for Self-Improving Intelligent Agents:
|
|
202
|
+
A Systematic Survey of Generative and Cognitive Approaches."
|
|
203
|
+
|
|
204
|
+
SN Computer Science, 7, 746 (2026).
|
|
205
|
+
|
|
206
|
+
https://doi.org/10.1007/s42979-026-05333-6
|
|
207
|
+
|
|
208
|
+
|
|
209
|
+
## Software Authors
|
|
210
|
+
|
|
211
|
+
- Abedelkader Seif El Islem Rahmani
|
|
212
|
+
- Mohammed Bennour
|
|
213
|
+
|
|
214
|
+
## License
|
|
215
|
+
|
|
216
|
+
Released under the MIT License. See `LICENSE` for details.
|