citation-topics 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. citation_topics-0.1.0/.gitignore +17 -0
  2. citation_topics-0.1.0/CHANGELOG.md +18 -0
  3. citation_topics-0.1.0/LICENSE +21 -0
  4. citation_topics-0.1.0/PKG-INFO +156 -0
  5. citation_topics-0.1.0/README.md +124 -0
  6. citation_topics-0.1.0/pyproject.toml +67 -0
  7. citation_topics-0.1.0/src/citation_topics/__init__.py +11 -0
  8. citation_topics-0.1.0/src/citation_topics/__main__.py +3 -0
  9. citation_topics-0.1.0/src/citation_topics/claude/agents/topic-namer.md +19 -0
  10. citation_topics-0.1.0/src/citation_topics/claude/skills/topic-namer/SKILL.md +63 -0
  11. citation_topics-0.1.0/src/citation_topics/claude/skills/topics-compare/SKILL.md +41 -0
  12. citation_topics-0.1.0/src/citation_topics/claude/skills/topics-run/SKILL.md +90 -0
  13. citation_topics-0.1.0/src/citation_topics/cli.py +173 -0
  14. citation_topics-0.1.0/src/citation_topics/cluster.py +154 -0
  15. citation_topics-0.1.0/src/citation_topics/compare.py +203 -0
  16. citation_topics-0.1.0/src/citation_topics/describe.py +356 -0
  17. citation_topics-0.1.0/src/citation_topics/export.py +117 -0
  18. citation_topics-0.1.0/src/citation_topics/io.py +102 -0
  19. citation_topics-0.1.0/src/citation_topics/names.py +58 -0
  20. citation_topics-0.1.0/src/citation_topics/network.py +112 -0
  21. citation_topics-0.1.0/src/citation_topics/py.typed +0 -0
  22. citation_topics-0.1.0/src/citation_topics/run.py +171 -0
  23. citation_topics-0.1.0/src/citation_topics/secondary.py +60 -0
  24. citation_topics-0.1.0/src/citation_topics/skills.py +39 -0
  25. citation_topics-0.1.0/tests/conftest.py +54 -0
  26. citation_topics-0.1.0/tests/test_cli.py +58 -0
  27. citation_topics-0.1.0/tests/test_pipeline.py +190 -0
@@ -0,0 +1,17 @@
1
+ __pycache__/
2
+ *.py[cod]
3
+ *.egg-info/
4
+ build/
5
+ dist/
6
+ .venv/
7
+ venv/
8
+ .pytest_cache/
9
+ .ruff_cache/
10
+ .coverage
11
+ coverage.xml
12
+ htmlcov/
13
+ # Add here any generated outputs or data files that hold personal data (e.g. downloaded pages)
14
+ # Never commit credentials
15
+ .env
16
+ .pypirc
17
+ *apikey*
@@ -0,0 +1,18 @@
1
+ # Changelog
2
+
3
+ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and versions follow [Semantic Versioning](https://semver.org/).
4
+
5
+ ## [Unreleased]
6
+
7
+ ## [0.1.0] - 2026-11-15
8
+
9
+ ### Added
10
+ - `run`: direct-citation network of a JSON or JSON Lines list of works, Leiden topics (igraph) and areas
11
+ (leidenalg, best of several seeds), second and third topics of every work; `works.jsonl.gz`, `topics.json`,
12
+ `areas.json`, `meta.json`.
13
+ - `describe`: dossier of every topic and area from the optional fields.
14
+ - `names`, `compare` (fates and names carried over), `agreement` (NMI/ARI, medoid), `export` (VOSviewer, CSV).
15
+ - Claude skills `topics-run`, `topic-namer`, `topics-compare` and the agent `topic-namer` (`skills install`).
16
+
17
+ [Unreleased]: https://github.com/restrepo/citation-topics/compare/v0.1.0...HEAD
18
+ [0.1.0]: https://github.com/restrepo/citation-topics/releases/tag/v0.1.0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Diego Restrepo
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,156 @@
1
+ Metadata-Version: 2.5
2
+ Name: citation-topics
3
+ Version: 0.1.0
4
+ Summary: Research topics and areas from the citation links of any list of works, with Leiden clustering as CWTS does for OpenAlex.
5
+ Project-URL: Homepage, https://github.com/restrepo/citation-topics
6
+ Project-URL: Repository, https://github.com/restrepo/citation-topics
7
+ Project-URL: Issues, https://github.com/restrepo/citation-topics/issues
8
+ Project-URL: Changelog, https://github.com/restrepo/citation-topics/blob/main/CHANGELOG.md
9
+ Author-email: Diego Restrepo <restrepo@udea.edu.co>
10
+ License-Expression: MIT
11
+ License-File: LICENSE
12
+ Keywords: bibliometrics,citation network,community detection,leiden,scientometrics,topics
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Science/Research
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3 :: Only
19
+ Classifier: Topic :: Scientific/Engineering :: Information Analysis
20
+ Classifier: Typing :: Typed
21
+ Requires-Python: >=3.10
22
+ Requires-Dist: igraph>=0.11
23
+ Requires-Dist: leidenalg>=0.10
24
+ Requires-Dist: numpy>=1.24
25
+ Provides-Extra: dev
26
+ Requires-Dist: build>=1.2; extra == 'dev'
27
+ Requires-Dist: pytest-cov>=5; extra == 'dev'
28
+ Requires-Dist: pytest>=8; extra == 'dev'
29
+ Requires-Dist: ruff==0.17.3; extra == 'dev'
30
+ Requires-Dist: twine>=5; extra == 'dev'
31
+ Description-Content-Type: text/markdown
32
+
33
+ # citation-topics
34
+
35
+ [![CI](https://github.com/restrepo/citation-topics/actions/workflows/ci.yml/badge.svg)](https://github.com/restrepo/citation-topics/actions/workflows/ci.yml)
36
+ [![PyPI](https://img.shields.io/pypi/v/citation-topics.svg)](https://pypi.org/project/citation-topics/)
37
+ [![Python](https://img.shields.io/pypi/pyversions/citation-topics.svg)](https://pypi.org/project/citation-topics/)
38
+
39
+ Research topics and broad areas for any list of works (papers, theses, projects, patents) from their citation
40
+ links, with Leiden clustering of the direct-citation network as CWTS does for the OpenAlex topics. It also gives
41
+ each work a second and a third topic, describes every topic from whatever the input carries (titles, abstracts,
42
+ keywords, journals, categories, countries), and ships Claude skills and an agent that name the topics.
43
+
44
+ It was extracted from [restrepo/inspireauth10](https://github.com/restrepo/inspireauth10), where it groups 1.9
45
+ million INSPIRE-HEP papers into 304 topics and 6 areas
46
+ ([how the topics are made](https://restrepo.github.io/inspireauth10/leiden.html)).
47
+
48
+ ## Install
49
+
50
+ ```bash
51
+ pip install citation-topics
52
+ citation-topics --help
53
+ ```
54
+
55
+ ## Input
56
+
57
+ A JSON array or JSON Lines file (optionally gzipped), one object per work:
58
+
59
+ ```json
60
+ {"id": "W1", "references": ["W7", "W9"], "year": 2019, "title": "...", "abstract": "...",
61
+ "keywords": ["..."], "journal": "...", "categories": ["..."], "countries": ["..."],
62
+ "authors": ["..."], "citations": 41, "url": "https://..."}
63
+ ```
64
+
65
+ - `id` (unique; strings or integers, compared as strings) and `references` (ids of other works) are required.
66
+ References to ids outside the list and self-citations are ignored and counted.
67
+ - Every other field is optional. Other names are read with `--id-field`, `--refs-field` and
68
+ `--map NAME=FIELD` (for example `--map year=publication_year`).
69
+
70
+ | Optional field | Adds |
71
+ |---|---|
72
+ | `year` | years of each topic, time windows (`--from-year`, `--to-year`), comparisons between epochs |
73
+ | `title`, `abstract`, `keywords`, `journal`, `categories` | the description of each topic (`describe`), hence its name |
74
+ | `citations` | most cited works by that count (otherwise by citations inside the list) |
75
+ | `countries` | countries of each topic in the description |
76
+ | `authors`, `url` | counted in `meta.json`; `url` is copied to the most cited works of the description |
77
+
78
+ ## Commands
79
+
80
+ ```bash
81
+ citation-topics run works.jsonl.gz out/ # network, topics, areas, second and third topics
82
+ citation-topics describe works.jsonl.gz out/ # out/dossier.json: what each topic is about
83
+ citation-topics names out/ names.json # put names into topics.json and areas.json
84
+ citation-topics export out/ # VOSviewer JSON and map/network files, CSVs
85
+ citation-topics agreement out_s1/ out_s2/ ... # NMI/ARI between seeds, and the medoid to keep
86
+ citation-topics compare old/ new/ cmp/ # where earlier topics went; names carried over
87
+ citation-topics skills install # Claude skills and the topic-namer agent into ./.claude
88
+ ```
89
+
90
+ `run` options and defaults (those of the INSPIRE run): `--resolution 16` (higher = more, smaller topics),
91
+ `--min-size 100` (smaller topics are merged into the topic they share the most links with), `--passes 20`
92
+ (stops early when the quality stops rising), `--seed 1`, `--area-resolution 0.3`, `--area-seeds 50` (the best
93
+ of that many runs). Tune them for other lists; the skill `topics-run` explains how.
94
+
95
+ ## Method
96
+
97
+ 1. **Network**: one node per work, an edge for each reference to another work of the list; undirected, each
98
+ pair of works once.
99
+ 2. **Topics**: Leiden maximizing modularity with a resolution parameter (the RB configuration model), with
100
+ igraph's `community_leiden`, pass after pass until the quality stops rising. Works with no link stay
101
+ unassigned; topics under `--min-size` are merged into their most linked neighbour; topics are numbered by
102
+ size.
103
+ 3. **Areas**: Leiden (leidenalg) on the graph of topics weighted by the links between them, with the links
104
+ inside each topic as self-loops, the best of `--area-seeds` runs. Every topic is in one area.
105
+ 4. **Second and third topics**: a work's links are counted by the topic at the other end; the second and third
106
+ topics are the most linked other topics, scored as links to them / links to the main topic, capped at 1.
107
+ Works the clustering left out take as main topic the one they link to most (`inferred`).
108
+ 5. **Description** (`describe`): per topic, its distinctive terms (shares in the topic against the rest of the
109
+ list), journals, categories, keywords, countries, years, linked topics and most cited works.
110
+ 6. **Names**: written by Claude from the description (skill `topic-namer`), never by the package. `compare`
111
+ carries names over from an earlier run: a topic sharing 60% of its works both ways with an earlier one keeps
112
+ its name.
113
+
114
+ The same input, parameters and versions of igraph and leidenalg give the same topics; `meta.json` records them
115
+ with the MD5 of the input.
116
+
117
+ **Checked against inspireauth10.** On its 1,872,221 INSPIRE papers (44 million citation links) with the
118
+ defaults, `run` gives the same network, the same 304 topics and 6 areas, the same Leiden quality and the same
119
+ second and third topics as inspireauth10's own scripts, array for array (igraph 1.0.0, leidenalg 0.12.0, numpy
120
+ 2.5.4, Python 3.13). It took 45 minutes and more than 4 GB of memory; `describe` with titles, keywords,
121
+ journals, categories and countries took 10 minutes and 4.4 GB.
122
+
123
+ ## Output
124
+
125
+ | File | Content |
126
+ |---|---|
127
+ | `works.jsonl.gz` | per work: `id`, `topic`, `area` (null if none), `topics` (three, with `score` and `links`), `n_links`, `inferred` |
128
+ | `topics.json` | per topic: area, name, works, links inside and out, `share_inside`, years, most cited works |
129
+ | `areas.json` | per area: name, works, topics, links, `share_inside` |
130
+ | `meta.json` | parameters, versions, input MD5, network counts, optional fields present, quality |
131
+ | `dossier.json` | description of each topic and area (`describe`) |
132
+ | `topics_vosviewer.json`, `topics_map.txt`, `topics_network.txt` | graph of topics for VOSviewer (`export`) |
133
+ | `topics.csv`, `works_topics.csv.gz` | the same as tables (`export`) |
134
+ | `network.npz`, `clusters.npz` | arrays reused by the other commands |
135
+
136
+ ## Claude skills
137
+
138
+ `citation-topics skills install` copies into `./.claude` (or `~/.claude` with `--user`):
139
+
140
+ - `topics-run`: checks the input, chooses the parameters, runs, describes and exports, and explains the result;
141
+ - `topic-namer`: names topics and areas from `dossier.json` and applies them, after carrying over earlier names;
142
+ - `topics-compare`: seeds (which run to keep) and earlier runs or time windows (fates, flows, names);
143
+ - the agent `topic-namer`, which names a batch of topics when the skill hands it one.
144
+
145
+ In Claude Desktop or Claude Code, ask for topics of a list of works and the skills take it from there.
146
+
147
+ ## Development
148
+
149
+ ```bash
150
+ python -m pip install -e ".[dev]"
151
+ python -m pytest
152
+ ruff check . && ruff format --check .
153
+ ```
154
+
155
+ Releases are published to PyPI from GitHub Actions when a `v*` tag is pushed; see [PUBLISHING.md](PUBLISHING.md)
156
+ and [CHANGELOG.md](CHANGELOG.md).
@@ -0,0 +1,124 @@
1
+ # citation-topics
2
+
3
+ [![CI](https://github.com/restrepo/citation-topics/actions/workflows/ci.yml/badge.svg)](https://github.com/restrepo/citation-topics/actions/workflows/ci.yml)
4
+ [![PyPI](https://img.shields.io/pypi/v/citation-topics.svg)](https://pypi.org/project/citation-topics/)
5
+ [![Python](https://img.shields.io/pypi/pyversions/citation-topics.svg)](https://pypi.org/project/citation-topics/)
6
+
7
+ Research topics and broad areas for any list of works (papers, theses, projects, patents) from their citation
8
+ links, with Leiden clustering of the direct-citation network as CWTS does for the OpenAlex topics. It also gives
9
+ each work a second and a third topic, describes every topic from whatever the input carries (titles, abstracts,
10
+ keywords, journals, categories, countries), and ships Claude skills and an agent that name the topics.
11
+
12
+ It was extracted from [restrepo/inspireauth10](https://github.com/restrepo/inspireauth10), where it groups 1.9
13
+ million INSPIRE-HEP papers into 304 topics and 6 areas
14
+ ([how the topics are made](https://restrepo.github.io/inspireauth10/leiden.html)).
15
+
16
+ ## Install
17
+
18
+ ```bash
19
+ pip install citation-topics
20
+ citation-topics --help
21
+ ```
22
+
23
+ ## Input
24
+
25
+ A JSON array or JSON Lines file (optionally gzipped), one object per work:
26
+
27
+ ```json
28
+ {"id": "W1", "references": ["W7", "W9"], "year": 2019, "title": "...", "abstract": "...",
29
+ "keywords": ["..."], "journal": "...", "categories": ["..."], "countries": ["..."],
30
+ "authors": ["..."], "citations": 41, "url": "https://..."}
31
+ ```
32
+
33
+ - `id` (unique; strings or integers, compared as strings) and `references` (ids of other works) are required.
34
+ References to ids outside the list and self-citations are ignored and counted.
35
+ - Every other field is optional. Other names are read with `--id-field`, `--refs-field` and
36
+ `--map NAME=FIELD` (for example `--map year=publication_year`).
37
+
38
+ | Optional field | Adds |
39
+ |---|---|
40
+ | `year` | years of each topic, time windows (`--from-year`, `--to-year`), comparisons between epochs |
41
+ | `title`, `abstract`, `keywords`, `journal`, `categories` | the description of each topic (`describe`), hence its name |
42
+ | `citations` | most cited works by that count (otherwise by citations inside the list) |
43
+ | `countries` | countries of each topic in the description |
44
+ | `authors`, `url` | counted in `meta.json`; `url` is copied to the most cited works of the description |
45
+
46
+ ## Commands
47
+
48
+ ```bash
49
+ citation-topics run works.jsonl.gz out/ # network, topics, areas, second and third topics
50
+ citation-topics describe works.jsonl.gz out/ # out/dossier.json: what each topic is about
51
+ citation-topics names out/ names.json # put names into topics.json and areas.json
52
+ citation-topics export out/ # VOSviewer JSON and map/network files, CSVs
53
+ citation-topics agreement out_s1/ out_s2/ ... # NMI/ARI between seeds, and the medoid to keep
54
+ citation-topics compare old/ new/ cmp/ # where earlier topics went; names carried over
55
+ citation-topics skills install # Claude skills and the topic-namer agent into ./.claude
56
+ ```
57
+
58
+ `run` options and defaults (those of the INSPIRE run): `--resolution 16` (higher = more, smaller topics),
59
+ `--min-size 100` (smaller topics are merged into the topic they share the most links with), `--passes 20`
60
+ (stops early when the quality stops rising), `--seed 1`, `--area-resolution 0.3`, `--area-seeds 50` (the best
61
+ of that many runs). Tune them for other lists; the skill `topics-run` explains how.
62
+
63
+ ## Method
64
+
65
+ 1. **Network**: one node per work, an edge for each reference to another work of the list; undirected, each
66
+ pair of works once.
67
+ 2. **Topics**: Leiden maximizing modularity with a resolution parameter (the RB configuration model), with
68
+ igraph's `community_leiden`, pass after pass until the quality stops rising. Works with no link stay
69
+ unassigned; topics under `--min-size` are merged into their most linked neighbour; topics are numbered by
70
+ size.
71
+ 3. **Areas**: Leiden (leidenalg) on the graph of topics weighted by the links between them, with the links
72
+ inside each topic as self-loops, the best of `--area-seeds` runs. Every topic is in one area.
73
+ 4. **Second and third topics**: a work's links are counted by the topic at the other end; the second and third
74
+ topics are the most linked other topics, scored as links to them / links to the main topic, capped at 1.
75
+ Works the clustering left out take as main topic the one they link to most (`inferred`).
76
+ 5. **Description** (`describe`): per topic, its distinctive terms (shares in the topic against the rest of the
77
+ list), journals, categories, keywords, countries, years, linked topics and most cited works.
78
+ 6. **Names**: written by Claude from the description (skill `topic-namer`), never by the package. `compare`
79
+ carries names over from an earlier run: a topic sharing 60% of its works both ways with an earlier one keeps
80
+ its name.
81
+
82
+ The same input, parameters and versions of igraph and leidenalg give the same topics; `meta.json` records them
83
+ with the MD5 of the input.
84
+
85
+ **Checked against inspireauth10.** On its 1,872,221 INSPIRE papers (44 million citation links) with the
86
+ defaults, `run` gives the same network, the same 304 topics and 6 areas, the same Leiden quality and the same
87
+ second and third topics as inspireauth10's own scripts, array for array (igraph 1.0.0, leidenalg 0.12.0, numpy
88
+ 2.5.4, Python 3.13). It took 45 minutes and more than 4 GB of memory; `describe` with titles, keywords,
89
+ journals, categories and countries took 10 minutes and 4.4 GB.
90
+
91
+ ## Output
92
+
93
+ | File | Content |
94
+ |---|---|
95
+ | `works.jsonl.gz` | per work: `id`, `topic`, `area` (null if none), `topics` (three, with `score` and `links`), `n_links`, `inferred` |
96
+ | `topics.json` | per topic: area, name, works, links inside and out, `share_inside`, years, most cited works |
97
+ | `areas.json` | per area: name, works, topics, links, `share_inside` |
98
+ | `meta.json` | parameters, versions, input MD5, network counts, optional fields present, quality |
99
+ | `dossier.json` | description of each topic and area (`describe`) |
100
+ | `topics_vosviewer.json`, `topics_map.txt`, `topics_network.txt` | graph of topics for VOSviewer (`export`) |
101
+ | `topics.csv`, `works_topics.csv.gz` | the same as tables (`export`) |
102
+ | `network.npz`, `clusters.npz` | arrays reused by the other commands |
103
+
104
+ ## Claude skills
105
+
106
+ `citation-topics skills install` copies into `./.claude` (or `~/.claude` with `--user`):
107
+
108
+ - `topics-run`: checks the input, chooses the parameters, runs, describes and exports, and explains the result;
109
+ - `topic-namer`: names topics and areas from `dossier.json` and applies them, after carrying over earlier names;
110
+ - `topics-compare`: seeds (which run to keep) and earlier runs or time windows (fates, flows, names);
111
+ - the agent `topic-namer`, which names a batch of topics when the skill hands it one.
112
+
113
+ In Claude Desktop or Claude Code, ask for topics of a list of works and the skills take it from there.
114
+
115
+ ## Development
116
+
117
+ ```bash
118
+ python -m pip install -e ".[dev]"
119
+ python -m pytest
120
+ ruff check . && ruff format --check .
121
+ ```
122
+
123
+ Releases are published to PyPI from GitHub Actions when a `v*` tag is pushed; see [PUBLISHING.md](PUBLISHING.md)
124
+ and [CHANGELOG.md](CHANGELOG.md).
@@ -0,0 +1,67 @@
1
+ [build-system]
2
+ requires = ["hatchling>=1.24", "hatch-vcs>=0.4"]
3
+ build-backend = "hatchling.build"
4
+
5
+ [project]
6
+ name = "citation-topics"
7
+ dynamic = ["version"]
8
+ description = "Research topics and areas from the citation links of any list of works, with Leiden clustering as CWTS does for OpenAlex."
9
+ readme = "README.md"
10
+ license = "MIT"
11
+ license-files = ["LICENSE"]
12
+ requires-python = ">=3.10"
13
+ authors = [{ name = "Diego Restrepo", email = "restrepo@udea.edu.co" }]
14
+ keywords = ["bibliometrics", "citation network", "leiden", "community detection", "topics", "scientometrics"]
15
+ classifiers = [
16
+ "Development Status :: 3 - Alpha",
17
+ "Environment :: Console",
18
+ "Operating System :: OS Independent",
19
+ "Programming Language :: Python :: 3",
20
+ "Programming Language :: Python :: 3 :: Only",
21
+ "Intended Audience :: Science/Research",
22
+ "Topic :: Scientific/Engineering :: Information Analysis",
23
+ "Typing :: Typed",
24
+ ]
25
+ dependencies = ["numpy>=1.24", "igraph>=0.11", "leidenalg>=0.10"]
26
+
27
+ [project.optional-dependencies]
28
+ dev = ["pytest>=8", "pytest-cov>=5", "ruff==0.17.3", "build>=1.2", "twine>=5"]
29
+
30
+ [project.scripts]
31
+ citation-topics = "citation_topics.cli:main"
32
+
33
+ [project.urls]
34
+ Homepage = "https://github.com/restrepo/citation-topics"
35
+ Repository = "https://github.com/restrepo/citation-topics"
36
+ Issues = "https://github.com/restrepo/citation-topics/issues"
37
+ Changelog = "https://github.com/restrepo/citation-topics/blob/main/CHANGELOG.md"
38
+
39
+ # Version from the git tag (v0.1.0 -> 0.1.0); commits after a tag get 0.1.1.devN
40
+ [tool.hatch.version]
41
+ source = "vcs"
42
+
43
+ [tool.hatch.version.raw-options]
44
+ local_scheme = "no-local-version" # PyPI and TestPyPI reject local versions (+g1234abc)
45
+
46
+ [tool.hatch.build.targets.sdist]
47
+ include = ["src/", "tests/", "README.md", "CHANGELOG.md", "LICENSE"]
48
+
49
+ [tool.hatch.build.targets.wheel]
50
+ packages = ["src/citation_topics"]
51
+
52
+ [tool.pytest.ini_options]
53
+ testpaths = ["tests"]
54
+ addopts = "-ra --strict-markers"
55
+
56
+ [tool.coverage.run]
57
+ source = ["citation_topics"]
58
+ branch = true
59
+
60
+ [tool.ruff]
61
+ line-length = 120
62
+ target-version = "py310"
63
+ src = ["src", "tests"]
64
+
65
+ [tool.ruff.lint]
66
+ select = ["E", "F", "W", "I", "B", "UP", "SIM", "RUF"]
67
+ ignore = ["B905"] # zip() over arrays of equal length by construction
@@ -0,0 +1,11 @@
1
+ """Research topics and areas from the citation links of any list of works, with Leiden clustering as CWTS does
2
+ for the OpenAlex topics."""
3
+
4
+ from importlib.metadata import PackageNotFoundError, version
5
+
6
+ try:
7
+ __version__ = version("citation-topics")
8
+ except PackageNotFoundError: # running from a source tree that was never installed
9
+ __version__ = "0.0.0"
10
+
11
+ __all__ = ["__version__"]
@@ -0,0 +1,3 @@
1
+ from .cli import main
2
+
3
+ raise SystemExit(main())
@@ -0,0 +1,19 @@
1
+ ---
2
+ name: topic-namer
3
+ description: Names a batch of citation-topics topics or areas from OUT/dossier.json and writes them to a names JSON file. Give it the dossier path, the topic (or area) numbers, the output file, and the names already used elsewhere so it avoids duplicates.
4
+ tools: Read, Write, Bash
5
+ ---
6
+
7
+ You name research topics from their dossier. Read the entries of the requested topics (or areas) in the given
8
+ `dossier.json` (with `python3 -c` or `jq` if the file is large; do not load more than you need). For each:
9
+
10
+ - Write a name of 2 to 7 words (areas: 3 to 10) that says what sets it apart, grounded in its `terms`,
11
+ `most_cited` titles and abstracts, and checked against `journals`, `categories` and `keywords`.
12
+ - No generic names, no lists of terms, no duplicates among the batch or of the names you were told are taken;
13
+ use `linked_topics` to tell neighbours apart.
14
+ - Add a one-line `reason` citing the evidence.
15
+
16
+ Write the requested file as
17
+ `{"topics": {"N": {"name": "...", "reason": "..."}}, "meta": {"by": "topic-namer", "model": "<your model>", "date": "YYYY-MM-DD"}}`
18
+ (use `"areas"` instead of `"topics"` for areas) and reply with a short table of number and name. Do not run
19
+ `citation-topics names`; the caller applies the files.
@@ -0,0 +1,63 @@
1
+ ---
2
+ name: topic-namer
3
+ description: Name the topics and areas found by citation-topics from their dossier (dossier.json) - short, specific names written from each topic's distinctive terms and most cited titles, checked against its journals, categories and keywords - and apply them with `citation-topics names`; carries names over from an earlier run first so only new topics are named. Use whenever a citation-topics output folder has unnamed topics, the user asks to name, rename or check topic names, or after `citation-topics run` and `describe`.
4
+ ---
5
+
6
+ # Naming topics and areas
7
+
8
+ Names are the one step that needs reading; everything else is computed. They are written by Claude, so they
9
+ are not exactly repeatable: record who named them and when.
10
+
11
+ ## 1. Prepare
12
+
13
+ - `OUT/dossier.json` must exist (`citation-topics describe INPUT OUT/`). Check its `missing` list: without
14
+ titles, keywords, journals or categories there is nothing to name from; tell the user and stop.
15
+ - If an earlier run of the same or an overlapping list was named, carry its names over first:
16
+
17
+ ```bash
18
+ citation-topics compare OLD_OUT/ OUT/ CMP/ # CMP/names_carried.json: names + "to_name"
19
+ citation-topics names OUT/ CMP/names_carried.json
20
+ ```
21
+
22
+ Then name only the topics in `to_name`.
23
+
24
+ ## 2. Name
25
+
26
+ For each topic, read in `dossier.json`: `terms` (with the share of its works using them), `most_cited` (titles,
27
+ years, and the start of the abstract of the first three), `journals`, `categories`, `keywords`, `years`,
28
+ `linked_topics`. Write:
29
+
30
+ - **2 to 7 words**, specific to what sets the topic apart: "Neutrino oscillations", "Lattice QCD at finite
31
+ temperature", "Dark matter direct detection". Not generic ("Physics", "Theory"), not a list of every term.
32
+ - Grounded in the dossier: the most cited titles and the top terms should agree; check the name against the
33
+ journals and categories. If they disagree, prefer what most of its works share (terms, categories) over a
34
+ single famous work.
35
+ - Distinct: two topics never get the same name; when two are close, name what differs (method, object,
36
+ energy range, period). `linked_topics` shows the neighbours to tell apart.
37
+ - A one-line `reason` citing the evidence (terms, titles, categories).
38
+
39
+ Areas: broader names (3 to 10 words) covering their topics (`areas[].topics` lists them with their names).
40
+
41
+ With many topics, work in batches of about 30. If the `topic-namer` agent is installed (`.claude/agents/`),
42
+ give each batch to it: the topic numbers, the path of `dossier.json` and the file to write; check its names
43
+ for duplicates afterwards.
44
+
45
+ ## 3. Apply
46
+
47
+ Write `names.json`:
48
+
49
+ ```json
50
+ {"topics": {"0": {"name": "...", "reason": "..."}},
51
+ "areas": {"0": {"name": "...", "reason": "..."}},
52
+ "meta": {"by": "topic-namer", "model": "<the model you are>", "date": "YYYY-MM-DD"}}
53
+ ```
54
+
55
+ and merge it (it can be applied in parts; later files add to earlier ones):
56
+
57
+ ```bash
58
+ citation-topics names OUT/ names.json
59
+ citation-topics export OUT/ # VOSviewer and CSVs with the names
60
+ ```
61
+
62
+ The command lists any topic still unnamed. Show the user a table of number, works and name, and say which
63
+ names were carried over and which are new.
@@ -0,0 +1,41 @@
1
+ ---
2
+ name: topics-compare
3
+ description: Compare runs of citation-topics - agreement between seeds (NMI, ARI, medoid to keep) and where the topics of an earlier run or time window went in a later one (same, absorbed, split, scattered, with flows between topics and names carried over). Use whenever the user asks how stable the topics are, which seed to keep, how topics defined up to some year changed, how a new version of a list changes its topics, or wants to keep names from a previous run.
4
+ ---
5
+
6
+ # Comparing runs of citation-topics
7
+
8
+ ## Seeds of one list
9
+
10
+ ```bash
11
+ for s in 1 2 3 4; do citation-topics run INPUT OUT_s$s/ --seed $s; done # same other parameters
12
+ citation-topics agreement OUT_s1/ OUT_s2/ OUT_s3/ OUT_s4/
13
+ ```
14
+
15
+ It prints NMI and ARI for every pair, for topics and areas, on the works all runs assign, and the medoid (the
16
+ run with the highest mean topic NMI to the others): keep that one. Topic NMI around 0.85 between seeds is
17
+ usual for large citation networks; much lower means the resolution is too high for the list or the list is
18
+ small.
19
+
20
+ ## An earlier run against a later one
21
+
22
+ The earlier run can be a time window of the same list (`run INPUT OLD/ --to-year 2010`) or a run of an older
23
+ version of the list. Works are joined by id.
24
+
25
+ ```bash
26
+ citation-topics compare OLD/ NEW/ CMP/
27
+ ```
28
+
29
+ `CMP/flows.json`:
30
+
31
+ - `old`: each earlier topic with its best new topic, `share` (of its works there), `back` (of that topic's
32
+ works that were in the earlier run) and `fate`: **same** (60% both ways), **absorbed** (60% one way),
33
+ **split** (two or more new topics with 15% each), **scattered**.
34
+ - `new`: each new topic with its works not in the earlier run and the earlier topic it is the same as.
35
+ - `flows`: `[old, new, works]` for every pair; `meta`: counts of fates, NMI and ARI.
36
+
37
+ `CMP/names_carried.json` holds the names of the earlier topics that are the same as a new one, ready for
38
+ `citation-topics names NEW/ CMP/names_carried.json`, and the topics left to name (skill `topic-namer`).
39
+
40
+ Report the counts of each fate, NMI and ARI, the largest splits and merges by name, and how many new topics
41
+ have most of their works after the earlier run.
@@ -0,0 +1,90 @@
1
+ ---
2
+ name: topics-run
3
+ description: Build research topics and broad areas for any list of works (papers, theses, projects, patents) from their citation links with the citation-topics package - Leiden clustering of the direct-citation network as CWTS does for the OpenAlex topics, plus second and third topics of every work, a description of each topic, names, and VOSviewer/CSV exports. Use whenever the user has a JSON or JSON Lines list of works with ids and references and wants topics, fields, clusters or a classification of them, wants to rerun or tune the clustering, or asks what the output files mean.
4
+ ---
5
+
6
+ # Topics of a list of works with citation-topics
7
+
8
+ The package does the computation; this skill decides the parameters and explains the result. Every step
9
+ writes into one output folder, `OUT/`.
10
+
11
+ ## 1. Check the input
12
+
13
+ One JSON object per work, in a JSON array or JSON Lines file (`.gz` allowed):
14
+
15
+ ```json
16
+ {"id": "W1", "references": ["W7", "W9"], "year": 2019, "title": "...", "abstract": "...",
17
+ "keywords": ["..."], "journal": "...", "categories": ["..."], "countries": ["..."], "authors": ["..."],
18
+ "citations": 41, "url": "https://..."}
19
+ ```
20
+
21
+ Only `id` (unique) and `references` (ids of other works of the list) are required. Other names are read with
22
+ `--id-field`, `--refs-field` and `--map NAME=FIELD` (for example `--map year=publication_year`). References to
23
+ ids outside the list are ignored and counted. What each optional field adds:
24
+
25
+ | Field | Adds |
26
+ |---|---|
27
+ | `year` | years of each topic, time windows (`--from-year`, `--to-year`), comparisons between epochs |
28
+ | `title`, `abstract`, `keywords`, `journal`, `categories` | the description of each topic, hence its name |
29
+ | `citations` | most cited works by that count (otherwise by citations inside the list) |
30
+ | `countries`, `authors`, `url` | countries per topic in the description; kept for the boards; links |
31
+
32
+ Before a long run, look at a few lines of the input and tell the user which optional fields it has and what
33
+ will be missing.
34
+
35
+ ## 2. Run
36
+
37
+ ```bash
38
+ citation-topics run INPUT OUT/ [--resolution 16] [--min-size 100] [--passes 20] [--seed 1] \
39
+ [--area-resolution 0.3] [--area-seeds 50]
40
+ ```
41
+
42
+ The defaults are those of the HEP run this package was made from (1.9 million works, 304 topics, 6 areas).
43
+ For another list, tune them:
44
+
45
+ - `--min-size`: the smallest topic. Aim for topics of a few hundred to a few thousand works; for a list of
46
+ N works, start near N / 5,000 (at least 20).
47
+ - `--resolution`: higher gives more, smaller topics. Run two or three values (for example 4, 8, 16) and compare
48
+ the number of topics, the share of links inside topics (`share_inside` in `topics.json`, higher is more
49
+ solid) and the unassigned works (`meta.json`).
50
+ - `--area-resolution`: 0.3 gave the steadiest areas in the HEP run; try 0.2-0.5 if the areas look too few or
51
+ too many.
52
+ - Seeds: Leiden depends on its random seed. For a result to keep, run 3-4 seeds (`--seed 1..4`, into different
53
+ folders) and keep the medoid with `citation-topics agreement OUT1 OUT2 OUT3 OUT4`.
54
+
55
+ Run large lists in the background. Measured on 1.9 million works and 44 million links: `run` 45 minutes with
56
+ 20 passes (about 85 s per pass) and over 4 GB of memory; `describe` with titles 10 minutes and 4.4 GB. The same
57
+ input, parameters and versions (all in `meta.json`) give the same topics.
58
+
59
+ ## 3. Describe and name
60
+
61
+ ```bash
62
+ citation-topics describe INPUT OUT/ [--abstracts] # OUT/dossier.json
63
+ ```
64
+
65
+ Then follow the skill `topic-namer`: it names topics and areas from `dossier.json` and applies the names with
66
+ `citation-topics names OUT/ names.json`. Without titles, keywords, journals or categories the dossier has only
67
+ sizes, years and links; say so, and name only if the user can supply what a topic is about.
68
+
69
+ ## 4. Export
70
+
71
+ ```bash
72
+ citation-topics export OUT/ # topics_vosviewer.json, topics_map.txt, topics_network.txt, CSVs
73
+ ```
74
+
75
+ `topics_vosviewer.json` opens in VOSviewer Online: `https://app.vosviewer.com/?json=URL` with a public URL of
76
+ the file.
77
+
78
+ ## Output files
79
+
80
+ | File | Content |
81
+ |---|---|
82
+ | `works.jsonl.gz` | per work: `topic` and `area` from the clustering (null if none), `topics` (main, second, third with `score` = links to it / links to the main topic, capped at 1, and `links`), `n_links`, `inferred` (main topic from its links when the clustering left it out) |
83
+ | `topics.json` | per topic: area, name, works, links inside and out, `share_inside`, years, most cited works |
84
+ | `areas.json` | per area: name, works, topics, links, `share_inside` |
85
+ | `meta.json` | parameters, versions, input MD5, network counts, which optional fields were present, quality |
86
+ | `dossier.json` | description of each topic and area (after `describe`) |
87
+ | `network.npz`, `clusters.npz` | arrays used by the other commands |
88
+
89
+ Report the numbers that matter (topics, areas, works with a topic, unassigned, share of links inside) and
90
+ which steps the missing fields ruled out.