sourcelens 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. sourcelens-0.0.1/.gitignore +27 -0
  2. sourcelens-0.0.1/CHANGELOG.md +75 -0
  3. sourcelens-0.0.1/LICENSE +674 -0
  4. sourcelens-0.0.1/PKG-INFO +548 -0
  5. sourcelens-0.0.1/README.md +509 -0
  6. sourcelens-0.0.1/cmd/sourcelens/defaults/default.yaml +733 -0
  7. sourcelens-0.0.1/docs/configuration.md +157 -0
  8. sourcelens-0.0.1/publish.md +217 -0
  9. sourcelens-0.0.1/pyproject.toml +85 -0
  10. sourcelens-0.0.1/src/sourcelens/__init__.py +7 -0
  11. sourcelens-0.0.1/src/sourcelens/__main__.py +7 -0
  12. sourcelens-0.0.1/src/sourcelens/buildcatalog/__init__.py +0 -0
  13. sourcelens-0.0.1/src/sourcelens/buildcatalog/build_progress.py +414 -0
  14. sourcelens-0.0.1/src/sourcelens/buildcatalog/summarise.py +162 -0
  15. sourcelens-0.0.1/src/sourcelens/buildcatalog/test_incremental.py +116 -0
  16. sourcelens-0.0.1/src/sourcelens/cli/__init__.py +0 -0
  17. sourcelens-0.0.1/src/sourcelens/cli/main.py +699 -0
  18. sourcelens-0.0.1/src/sourcelens/common/__init__.py +0 -0
  19. sourcelens-0.0.1/src/sourcelens/common/agelit.py +440 -0
  20. sourcelens-0.0.1/src/sourcelens/common/crossref.py +101 -0
  21. sourcelens-0.0.1/src/sourcelens/common/flags.py +47 -0
  22. sourcelens-0.0.1/src/sourcelens/common/hits.py +50 -0
  23. sourcelens-0.0.1/src/sourcelens/common/pubmed.py +135 -0
  24. sourcelens-0.0.1/src/sourcelens/common/settings.py +367 -0
  25. sourcelens-0.0.1/src/sourcelens/common/store.py +112 -0
  26. sourcelens-0.0.1/src/sourcelens/common/terms.py +217 -0
  27. sourcelens-0.0.1/src/sourcelens/defaults/default.yaml +1 -0
  28. sourcelens-0.0.1/src/sourcelens/localseeds/__init__.py +0 -0
  29. sourcelens-0.0.1/src/sourcelens/localseeds/extract_seeds.py +240 -0
  30. sourcelens-0.0.1/src/sourcelens/pullliturature/__init__.py +0 -0
  31. sourcelens-0.0.1/src/sourcelens/pullliturature/enrich_openalex.py +74 -0
  32. sourcelens-0.0.1/src/sourcelens/pullliturature/enrich_preprints.py +66 -0
  33. sourcelens-0.0.1/src/sourcelens/pullliturature/fetch_fulltext.py +662 -0
  34. sourcelens-0.0.1/src/sourcelens/pullliturature/jats.py +269 -0
  35. sourcelens-0.0.1/src/sourcelens/pullliturature/resolve_seeds.py +119 -0
  36. sourcelens-0.0.1/src/sourcelens/pullliturature/search_arxiv.py +141 -0
  37. sourcelens-0.0.1/src/sourcelens/pullliturature/search_europepmc.py +282 -0
  38. sourcelens-0.0.1/src/sourcelens/pullliturature/search_openalex.py +222 -0
  39. sourcelens-0.0.1/src/sourcelens/pullliturature/search_pubmed.py +111 -0
  40. sourcelens-0.0.1/src/sourcelens/pullrepos/__init__.py +0 -0
  41. sourcelens-0.0.1/src/sourcelens/pullrepos/build_repos.py +379 -0
  42. sourcelens-0.0.1/src/sourcelens/pullrepos/mine_links.py +146 -0
  43. sourcelens-0.0.1/src/sourcelens/pullrepos/search_github.py +108 -0
  44. sourcelens-0.0.1/src/sourcelens/query/__init__.py +0 -0
  45. sourcelens-0.0.1/src/sourcelens/query/catalog.py +191 -0
  46. sourcelens-0.0.1/src/sourcelens/query/export_refs.py +71 -0
  47. sourcelens-0.0.1/src/sourcelens/query/refs.py +566 -0
  48. sourcelens-0.0.1/src/sourcelens/query/search_catalog.py +53 -0
  49. sourcelens-0.0.1/src/sourcelens/websites/__init__.py +0 -0
  50. sourcelens-0.0.1/src/sourcelens/websites/build_websites.py +181 -0
  51. sourcelens-0.0.1/tests/test_core.py +147 -0
@@ -0,0 +1,27 @@
1
+ # Builds
2
+ bin/
3
+ dist/
4
+ build/
5
+ *.egg-info/
6
+
7
+ # Python
8
+ __pycache__/
9
+ *.pyc
10
+ .venv/
11
+ venv/
12
+ .pytest_cache/
13
+ .ruff_cache/
14
+
15
+ # Local settings and secrets never belong in the repository
16
+ .env
17
+ .env.*
18
+ *.local.yaml
19
+ settings.yaml
20
+
21
+ # Catalogues are written to the output folder (sourcelens config), not here
22
+ research/
23
+ exports/
24
+
25
+ *.tmp
26
+ *.tmp.gz
27
+ .DS_Store
@@ -0,0 +1,75 @@
1
+ # Changelog
2
+
3
+ All notable changes to sourcelens. Versions follow semantic versioning.
4
+
5
+ ## 0.0.1 (2026-10-04)
6
+
7
+ First public release, under the name sourcelens (developed as litsearch and
8
+ litSearch before). sourcelens is a general tool: the research topic comes from
9
+ the command line or from a configuration file, and the aging-clock catalogue
10
+ it was built for is the default topic.
11
+
12
+ ### Added
13
+
14
+ - `sourcelens "TOPIC"`: start or update a catalogue for any topic from one
15
+ line. Commas, semicolons and `OR` separate alternatives; the words of an
16
+ alternative are all required; double quotes keep a phrase together.
17
+ - One catalogue per topic: the default topic in the output folder, every other
18
+ topic in a subfolder named after it. `--topic` and `--dir` select a catalogue
19
+ in every command; `sourcelens list` shows them.
20
+ - OpenAlex search, covering every field of research, and OpenAlex
21
+ open-access PDF links as a full-text source.
22
+ - OpenAlex results are kept only when the topic words appear as whole words in
23
+ the title or abstract.
24
+ - Further versions of a stored work (same title, venue and year, another DOI)
25
+ are skipped.
26
+ - A new topic starts 12 months back. Asking later for older work with
27
+ `--from` or `--range` extends the window.
28
+ - `sourcelens config`: per-machine settings (output folder, contact email, API
29
+ keys) in `~/.config/sourcelens/settings.yaml`, with environment overrides.
30
+ - The newest additions are printed after every update.
31
+ - `pip install sourcelens` (Python 3.10+) and `go install` (one static binary)
32
+ install the same command. PyMuPDF is an optional extra (`sourcelens[pdf]`);
33
+ without it poppler-utils converts PDFs.
34
+ - Unit tests for both implementations, a parity check between them, CI and
35
+ release workflows, a Makefile and a Dockerfile.
36
+ - `publish.yml` and `publish.md`: PyPI and TestPyPI publishing with trusted
37
+ publishing, with version, README and wheel checks before every upload.
38
+ - `docs/configuration.md`: every configuration key.
39
+
40
+ ### Changed
41
+
42
+ - Topic-specific parts moved from the code into the configuration:
43
+ - reference folders (`references`, `seeds`);
44
+ - the category of registry origin papers (`origin_category`);
45
+ - the core categories and the default category;
46
+ - the summary's data layers;
47
+ - website relevance.
48
+ - Column `clocks` is now `entities`; configuration key `clock_names` is now
49
+ `entities`; query option `--clock` is now `--entity`. Older catalogues and
50
+ configurations are still read.
51
+ - Seed roles `clock_origin` and `clock_reference` are now `registry_origin`
52
+ and `registry_reference`; `seeds/falconage_clocks.csv` is now
53
+ `seeds/registry_entries.csv`; `corpus/pre2011_background.csv` is now
54
+ `corpus/before_window.csv`.
55
+ - Docker is optional: both implementations run natively.
56
+
57
+ ### Removed
58
+
59
+ - The Docker launchers (`run.sh`), the sequential `update_all.sh` and the
60
+ per-workspace `config/local.yaml`; settings replace them.
61
+
62
+ ## litSearch 2.0 (2026-10-04, unreleased)
63
+
64
+ - Python and Go implementations with the same commands, flags and outputs,
65
+ each with its own Docker image.
66
+ - `query` and `export` (APA, AMA, MLA, Chicago, Harvard, Vancouver, IEEE,
67
+ Nature, BibTeX, RIS, EndNote, CSL-JSON).
68
+ - One configuration file kept in the output folder.
69
+
70
+ ## agingcat 1.x (2026-09-29 to 2026-10-04, unreleased)
71
+
72
+ - The aging-clock literature pipeline: PubMed, Europe PMC and arXiv searches,
73
+ seeds from local projects, an append-only chronological `progress.csv`,
74
+ open-access full texts, repositories, packages and websites, and the
75
+ `agingcat` CLI with parallel stages, source, type and date-range selection.