research-helpers 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. research_helpers-1.0.0/CHANGELOG.md +68 -0
  2. research_helpers-1.0.0/LICENSE +21 -0
  3. research_helpers-1.0.0/MANIFEST.in +1 -0
  4. research_helpers-1.0.0/PKG-INFO +293 -0
  5. research_helpers-1.0.0/README.md +235 -0
  6. research_helpers-1.0.0/pyproject.toml +224 -0
  7. research_helpers-1.0.0/setup.cfg +4 -0
  8. research_helpers-1.0.0/src/research_helpers/__init__.py +17 -0
  9. research_helpers-1.0.0/src/research_helpers/archive.py +162 -0
  10. research_helpers-1.0.0/src/research_helpers/arxiv.py +253 -0
  11. research_helpers-1.0.0/src/research_helpers/build.py +389 -0
  12. research_helpers-1.0.0/src/research_helpers/cli.py +76 -0
  13. research_helpers-1.0.0/src/research_helpers/display.py +282 -0
  14. research_helpers-1.0.0/src/research_helpers/figures.py +299 -0
  15. research_helpers-1.0.0/src/research_helpers/latex.py +173 -0
  16. research_helpers-1.0.0/src/research_helpers/log/__init__.py +22 -0
  17. research_helpers-1.0.0/src/research_helpers/log/config.py +197 -0
  18. research_helpers-1.0.0/src/research_helpers/log/formatters.py +88 -0
  19. research_helpers-1.0.0/src/research_helpers/log/handlers.py +72 -0
  20. research_helpers-1.0.0/src/research_helpers/log/tqdm_integration.py +32 -0
  21. research_helpers-1.0.0/src/research_helpers/project.py +425 -0
  22. research_helpers-1.0.0/src/research_helpers/py.typed +0 -0
  23. research_helpers-1.0.0/src/research_helpers/sweep/__init__.py +30 -0
  24. research_helpers-1.0.0/src/research_helpers/sweep/collect.py +143 -0
  25. research_helpers-1.0.0/src/research_helpers/sweep/grid.py +133 -0
  26. research_helpers-1.0.0/src/research_helpers/sweep/runner.py +150 -0
  27. research_helpers-1.0.0/src/research_helpers/sweep/sweep.py +339 -0
  28. research_helpers-1.0.0/src/research_helpers/templates/sweep.sbatch +78 -0
  29. research_helpers-1.0.0/src/research_helpers.egg-info/PKG-INFO +293 -0
  30. research_helpers-1.0.0/src/research_helpers.egg-info/SOURCES.txt +43 -0
  31. research_helpers-1.0.0/src/research_helpers.egg-info/dependency_links.txt +1 -0
  32. research_helpers-1.0.0/src/research_helpers.egg-info/entry_points.txt +2 -0
  33. research_helpers-1.0.0/src/research_helpers.egg-info/requires.txt +35 -0
  34. research_helpers-1.0.0/src/research_helpers.egg-info/top_level.txt +1 -0
  35. research_helpers-1.0.0/tests/test_archive.py +211 -0
  36. research_helpers-1.0.0/tests/test_arxiv.py +265 -0
  37. research_helpers-1.0.0/tests/test_build.py +387 -0
  38. research_helpers-1.0.0/tests/test_cli.py +261 -0
  39. research_helpers-1.0.0/tests/test_display.py +256 -0
  40. research_helpers-1.0.0/tests/test_figures.py +305 -0
  41. research_helpers-1.0.0/tests/test_latex.py +193 -0
  42. research_helpers-1.0.0/tests/test_log.py +354 -0
  43. research_helpers-1.0.0/tests/test_packaging.py +32 -0
  44. research_helpers-1.0.0/tests/test_project.py +357 -0
  45. research_helpers-1.0.0/tests/test_sweep.py +401 -0
@@ -0,0 +1,68 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here. The format follows
4
+ [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to
5
+ [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
+
7
+ ## [1.0.0] — 2026-09-11
8
+
9
+ First public release.
10
+
11
+ The package was generalized out of three research projects that had each grown their own copy of
12
+ the same utilities.
13
+
14
+ ### The idea
15
+
16
+ `research-helpers` reads a project's **wiring**—*i.e., where the paper lives, how wide it is, where
17
+ sweep runs are written—and nothing else. Research parameters stay in the project that owns
18
+ them. This wiring lives in one `[tool.research-helpers]` section of a project's `pyproject.toml`,
19
+ which is also the project-root marker, so finding the configuration and finding the root are one
20
+ operation. Every setting has a working default, and every entry point is callable with plain
21
+ arguments and no project file at all.
22
+
23
+ ### Added
24
+
25
+ - **`build`** — a registry of generated artefacts with three-way drift detection: a table the
26
+ paper is behind on, one that is generated but never pulled in, and one typed by hand with no
27
+ emitter behind it. `--check` exits non-zero, so "every number in the paper is generated from
28
+ data in the repository, and is current" becomes a build status.
29
+ - **`latex`** — table assembly with the conventions a paper needs: half-up rounding, escaping
30
+ for text arriving from data, en-dashes for ranges, em-dashes for empty cells.
31
+ - **`display`** — parse a generated `.tex` table back into rows, spans and alignment, for reading
32
+ in a notebook or asserting on in a test.
33
+ - **`figures`** — one matplotlib look across a project, and a `print` profile that sizes a figure
34
+ to the document it is going into, so it renders in the paper at 1:1 with body-text-sized labels.
35
+ - **`arxiv`** — inventory a submission keyed by the paths it takes *inside* the upload, and check
36
+ it against the engine and TeX Live year it will build under: filenames arXiv rejects, a `.bbl`
37
+ that is missing, stale or in the wrong `biblatex` format, and `microtype` font expansion under an
38
+ engine that has none.
39
+ - **`archive`** — byte-reproducible tarballs. Pinned `mtimes`, `uid`, and `gid`, normalised modes,
40
+ sorted members, and an emptied gzip filename field, so a published checksum means something.
41
+ - **`sweep`** — plan a parameter grid once, run it as a scheduler job array, resume what was
42
+ killed, collect the parts into one table. Resumption is per combination rather than per task,
43
+ the manifest is written once so a sparse resubmission reproduces its original slices, and
44
+ planning, running and status need only the standard library.
45
+ - **`log`** — *structlog* over the standard library, with separate console and file levels, colour
46
+ only when the stream is a terminal, and progress bars that do not fight the log.
47
+ - **`research-helpers` CLI** — `doctor` prints every setting, the value in force and where it came
48
+ from; `arxiv` assembles, checks, packs and previews a submission.
49
+ - Lengths may be written in inches, millimetres, centimetres, or TeX points by changing a key's
50
+ suffix: `text-width-pt = 468.0` is what `\showthe\textwidth` prints, copied across with no
51
+ arithmetic in between.
52
+ - Full documentation at <https://gpizzorno.github.io/research-helpers/>.
53
+
54
+ ### Notes for packagers
55
+
56
+ - Requires Python 3.11 or newer.
57
+ - **The core package has no third-party dependencies.** Capabilities are opt-in extras:
58
+ `figures` (matplotlib, seaborn), `log` (structlog, colorama, tqdm), `sweep` (pandas, pyarrow),
59
+ and `latex`, which is standard library only and exists, so the capability is discoverable.
60
+ Importing `research_helpers` pulls in none of them.
61
+ - The package ships a `py.typed` marker, so annotations are visible to type checkers.
62
+
63
+ ### Verified
64
+
65
+ - 276 tests, 97% coverage, run against Python 3.11, 3.12, and 3.13 in CI.
66
+ - The documentation builds with warnings as errors.
67
+
68
+ [1.0.0]: https://github.com/gpizzorno/research-helpers/releases/tag/v1.0.0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Gabe Pizzorno
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ include CHANGELOG.md
@@ -0,0 +1,293 @@
1
+ Metadata-Version: 2.4
2
+ Name: research-helpers
3
+ Version: 1.0.0
4
+ Summary: Utilities for code-intensive research projects and the papers they produce
5
+ Author: Gabe Pizzorno
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/gpizzorno/research-helpers
8
+ Project-URL: Documentation, https://gpizzorno.github.io/research-helpers/
9
+ Project-URL: Repository, https://github.com/gpizzorno/research-helpers
10
+ Project-URL: Issues, https://github.com/gpizzorno/research-helpers/issues
11
+ Project-URL: Changelog, https://github.com/gpizzorno/research-helpers/blob/main/CHANGELOG.md
12
+ Keywords: research,latex,arxiv,zenodo,matplotlib,reproducibility,slurm,parameter-sweep
13
+ Classifier: Development Status :: 5 - Production/Stable
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Topic :: Scientific/Engineering
16
+ Classifier: Topic :: Text Processing :: Markup :: LaTeX
17
+ Classifier: Typing :: Typed
18
+ Classifier: Programming Language :: Python
19
+ Classifier: Programming Language :: Python :: 3
20
+ Classifier: Programming Language :: Python :: 3 :: Only
21
+ Classifier: Programming Language :: Python :: 3.11
22
+ Classifier: Programming Language :: Python :: 3.12
23
+ Classifier: Programming Language :: Python :: 3.13
24
+ Classifier: Operating System :: OS Independent
25
+ Requires-Python: >=3.11
26
+ Description-Content-Type: text/markdown
27
+ License-File: LICENSE
28
+ Provides-Extra: figures
29
+ Requires-Dist: matplotlib; extra == "figures"
30
+ Requires-Dist: seaborn; extra == "figures"
31
+ Provides-Extra: log
32
+ Requires-Dist: colorama; extra == "log"
33
+ Requires-Dist: structlog; extra == "log"
34
+ Requires-Dist: tqdm; extra == "log"
35
+ Provides-Extra: latex
36
+ Provides-Extra: sweep
37
+ Requires-Dist: pandas; extra == "sweep"
38
+ Requires-Dist: pyarrow; extra == "sweep"
39
+ Provides-Extra: dev
40
+ Requires-Dist: research-helpers[figures,latex,log,sweep]; extra == "dev"
41
+ Requires-Dist: factory_boy; extra == "dev"
42
+ Requires-Dist: hypothesis; extra == "dev"
43
+ Requires-Dist: pillow; extra == "dev"
44
+ Requires-Dist: pytest-cov; extra == "dev"
45
+ Requires-Dist: pytest-mock; extra == "dev"
46
+ Requires-Dist: pytest; extra == "dev"
47
+ Requires-Dist: ruff; extra == "dev"
48
+ Requires-Dist: ty; extra == "dev"
49
+ Requires-Dist: types-requests; extra == "dev"
50
+ Provides-Extra: docs
51
+ Requires-Dist: research-helpers[figures,latex,log,sweep]; extra == "docs"
52
+ Requires-Dist: sphinx>=7.2.0; extra == "docs"
53
+ Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
54
+ Requires-Dist: sphinx-autodoc-typehints>=1.25.0; extra == "docs"
55
+ Requires-Dist: sphinxcontrib-mermaid>=1.0.0; extra == "docs"
56
+ Requires-Dist: myst-parser>=2.0.0; extra == "docs"
57
+ Dynamic: license-file
58
+
59
+ # Research Helpers
60
+
61
+ [![License](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
62
+ [![Python](https://img.shields.io/badge/Language-Python-blue.svg)](https://www.python.org)
63
+ [![Tests](https://github.com/gpizzorno/research-helpers/actions/workflows/tests.yml/badge.svg)](https://github.com/gpizzorno/research-helpers/actions/workflows/tests.yml)
64
+ [![Documentation](https://img.shields.io/badge/Docs-latest-blue.svg)](https://gpizzorno.github.io/research-helpers/)
65
+
66
+ **Research Helpers** is a set of utilities for managing code-intensive research projects and their accompanying papers.
67
+ Includes shared figure styling, LaTeX table generation and rendering, a build pipeline that keeps
68
+ a paper's generated tables and figures from drifting out of date, submission packaging for [arXiv](https://arxiv.org)
69
+ and [Zenodo](https://zenodo.org), structured logging, and a parameter-sweep engine for [scheduler](https://slurm.schedmd.com/overview.html) job arrays.
70
+
71
+ [Read the documentation](https://gpizzorno.github.io/research-helpers/)
72
+
73
+ ## Features
74
+
75
+ - **Generated Tables and Figures**: Build the paper's tables and figures from the repository's own data
76
+ - **Drift Detection**: Detect when the paper and the data differ
77
+ - **LaTeX Assembly**: Compose tables for LaTeX and also display them in Jupyter
78
+ - **Shared Figure Styling**: Consistent look across LaTeX and Jupyter
79
+ - **Submission Packaging**: Automatic preparation and pre-checking for arXiv and Zenodo
80
+ - **Parameter Sweeps**: Plan a grid, run it as a scheduler job array, resume if necessary, and collect the results
81
+ - **Structured Logging**: Console and file output at separate levels, with progress bars and colour output
82
+
83
+ ## Installation
84
+
85
+ The core package has no third-party dependencies. Capabilities are installed as extras:
86
+
87
+ ```sh
88
+ pip install research-helpers[figures] # matplotlib, seaborn
89
+ pip install research-helpers[latex] # LaTeX table assembly and rendering (stdlib only)
90
+ pip install research-helpers[log] # structlog, colorama, tqdm
91
+ pip install research-helpers[sweep] # pandas, pyarrow (planning and running need neither)
92
+ ```
93
+
94
+ Requires Python 3.11 or newer.
95
+
96
+ ## Quick Start
97
+
98
+ ### Wire up the project
99
+
100
+ A project's wiring—*i.e.*, where the paper lives, how wide it is, where runs are written—goes in the
101
+ `[tool.research-helpers]` section of the `pyproject.toml` file.
102
+
103
+ ```toml
104
+ [tool.research-helpers.paper]
105
+ main = "tex/paper.tex"
106
+ tables-dir = "tex/tables"
107
+ figures-dir = "tex/figures"
108
+ text-width-pt = 468.0 # as reported by \showthe\textwidth
109
+
110
+ [tool.research-helpers.figures]
111
+ profile = "print"
112
+ palette = "colorblind"
113
+ dpi = 300
114
+ ```
115
+
116
+ `research-helpers doctor` shows every setting, the value in force, and where it came from.
117
+
118
+ Keys and values are described in the [configuration reference](https://gpizzorno.github.io/research-helpers/configuration.html).
119
+
120
+ ### Generate a table from data
121
+
122
+ ```python
123
+ import json
124
+
125
+ from research_helpers.build import TABLES, Registry
126
+ from research_helpers.latex import half_up, header, table
127
+
128
+ tables = Registry(TABLES)
129
+
130
+
131
+ @tables.register('tab:scores')
132
+ def scores() -> str:
133
+ """Accuracy on the held-out set."""
134
+ data = json.loads(SCORES_PATH.read_text())
135
+ return table(
136
+ spec='lr',
137
+ header_rows=[[header('System'), header('Accuracy')]],
138
+ body_rows=[[name.title(), half_up(v['accuracy'] * 100)] for name, v in data.items()],
139
+ caption='Accuracy on the held-out set.',
140
+ label='tab:scores',
141
+ )
142
+
143
+
144
+ if __name__ == '__main__':
145
+ raise SystemExit(tables.main())
146
+ ```
147
+
148
+ The label names the file, so this is written to `scores.tex` and the paper reads it with
149
+ `\input{tables/scores}`:
150
+
151
+ ```console
152
+ $ python -m demo.tables --install
153
+ wrote build/tables/scores.tex
154
+ installed tex/tables/scores.tex
155
+ ```
156
+
157
+ ### Check for drift
158
+
159
+ ```console
160
+ $ python -m demo.tables --check
161
+ OK: all 1 tables in the paper are current
162
+ ```
163
+
164
+ If the data is changed without rebuilding, the check fails:
165
+
166
+ ```console
167
+ $ python -m demo.tables --check
168
+ stale, the paper is behind the data: tab:scores
169
+ to fix: rebuild and install the tables
170
+ $ echo $?
171
+ 1
172
+ ```
173
+
174
+ For a full example, see [adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html).
175
+
176
+ ### Style a figure for the page
177
+
178
+ ```python
179
+ import matplotlib.pyplot as plt
180
+
181
+ from research_helpers.figures import apply_style, save
182
+
183
+ apply_style(profile='print') # \textwidth across, paper font sizes
184
+ fig, ax = plt.subplots() # the profile's size is already in rcParams
185
+ ax.plot(iterations, accuracy, marker='o')
186
+ save(fig, 'tex/figures/learning-curve.png')
187
+ ```
188
+
189
+ See details in the [figures guide](https://gpizzorno.github.io/research-helpers/guide/figures.html).
190
+
191
+ ### Run a parameter sweep
192
+
193
+ ```python
194
+ from research_helpers.sweep import Sweep
195
+
196
+ sweep = Sweep()
197
+
198
+
199
+ @sweep.context
200
+ def prepare(manifest, run_dir):
201
+ """Loaded once per array task."""
202
+ return load_corpus(manifest.metadata['corpus'])
203
+
204
+
205
+ @sweep.evaluate
206
+ def evaluate(params, corpus):
207
+ """One parameter combination."""
208
+ return {'f1': score(corpus, **params)}
209
+
210
+
211
+ if __name__ == '__main__':
212
+ raise SystemExit(sweep.main())
213
+ ```
214
+
215
+ ```console
216
+ $ python sweep_demo.py plan --config sweep.toml --run-dir runs/demo --tasks 4
217
+ planned 12 combinations over 4 array tasks (~3 per task)
218
+ manifest: runs/demo/manifest.json
219
+
220
+ submit with:
221
+ sbatch --array=1-4 <your sbatch script> runs/demo
222
+ ```
223
+
224
+ The manifest is written once and the tasks read it, so re-submitting the two that were killed
225
+ reproduces exactly their original slices instead of repartitioning the grid. Resumption is per
226
+ combination, not per task, so a job that hits its walltime keeps everything it finished.
227
+ `status` and `run` need only the standard library, so nothing needs to be installed to run a
228
+ progress check from a login node.
229
+
230
+ Full details are discussed in the [sweeps guide](https://gpizzorno.github.io/research-helpers/guide/sweeps.html).
231
+
232
+ ### Package a submission
233
+
234
+ ```console
235
+ $ research-helpers arxiv --dry-run
236
+ paper.tex 43.1 KB
237
+ paper.bbl 82.1 KB
238
+ tables/scores.tex 0.3 KB
239
+ figures/learning-curve.png 66.3 KB
240
+
241
+ 4 files, 0.19 MB
242
+ bbl format 3.3 (TeX Live 2025)
243
+
244
+ ready to upload: select xelatex and TeX Live 2025
245
+
246
+ (dry run, nothing written)
247
+ ```
248
+
249
+ The target files are keyed by the path they take *inside* the submission, so what resolves locally
250
+ will also resolve at the destination. The checks cover filenames arXiv rejects, a `.bbl` that is missing,
251
+ stale, or in a format the selected TeX Live will not read, and `microtype` font expansion under an engine
252
+ that has none. `--tar` packs it reproducibly, so re-packing an unchanged submission gives an identical
253
+ file.
254
+
255
+ See the [submission guide](https://gpizzorno.github.io/research-helpers/guide/submission.html).
256
+
257
+ ### Logging
258
+
259
+ ```python
260
+ from research_helpers.log import get_logger, setup_logging
261
+
262
+ setup_logging()
263
+ log = get_logger(__name__)
264
+
265
+ log.info('scoring', corpus='perseus', sentences=18_000)
266
+ ```
267
+
268
+ ```console
269
+ [INFO ] 19:41:47 __main__ scoring (corpus=perseus, sentences=18000)
270
+ ```
271
+
272
+ The console and the log file take separate levels. Colour is applied only when the stream is a terminal,
273
+ so no escape codes are included in a redirected log.
274
+
275
+ Details in the [logging guide](https://gpizzorno.github.io/research-helpers/guide/logging.html).
276
+
277
+ ## Documentation
278
+
279
+ The full documentation includes:
280
+
281
+ - **[Configuration](https://gpizzorno.github.io/research-helpers/configuration.html)**: Every section, key, type, and default, plus units and settings resolution
282
+ - **[Guides](https://gpizzorno.github.io/research-helpers/guide/index.html)**: Task-focused walkthroughs
283
+ - [Adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html): Full table pipeline
284
+ - [Figures](https://gpizzorno.github.io/research-helpers/guide/figures.html): Profiles, scoped styling, saving versus rendering
285
+ - [LaTeX tables](https://gpizzorno.github.io/research-helpers/guide/tables.html): Assembling tables and reading them back
286
+ - [Parameter sweeps](https://gpizzorno.github.io/research-helpers/guide/sweeps.html): Planning, running, resuming, and collecting
287
+ - [arXiv and Zenodo](https://gpizzorno.github.io/research-helpers/guide/submission.html): Submission checks and reproducible archives
288
+ - [Logging](https://gpizzorno.github.io/research-helpers/guide/logging.html): Setup, levels, colour, and progress bars
289
+ - **[API Reference](https://gpizzorno.github.io/research-helpers/api/index.html)**: Complete API documentation
290
+
291
+ ## License
292
+
293
+ The project is licensed under the [MIT License](LICENSE), allowing free use, modification, and distribution.
@@ -0,0 +1,235 @@
1
+ # Research Helpers
2
+
3
+ [![License](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
4
+ [![Python](https://img.shields.io/badge/Language-Python-blue.svg)](https://www.python.org)
5
+ [![Tests](https://github.com/gpizzorno/research-helpers/actions/workflows/tests.yml/badge.svg)](https://github.com/gpizzorno/research-helpers/actions/workflows/tests.yml)
6
+ [![Documentation](https://img.shields.io/badge/Docs-latest-blue.svg)](https://gpizzorno.github.io/research-helpers/)
7
+
8
+ **Research Helpers** is a set of utilities for managing code-intensive research projects and their accompanying papers.
9
+ Includes shared figure styling, LaTeX table generation and rendering, a build pipeline that keeps
10
+ a paper's generated tables and figures from drifting out of date, submission packaging for [arXiv](https://arxiv.org)
11
+ and [Zenodo](https://zenodo.org), structured logging, and a parameter-sweep engine for [scheduler](https://slurm.schedmd.com/overview.html) job arrays.
12
+
13
+ [Read the documentation](https://gpizzorno.github.io/research-helpers/)
14
+
15
+ ## Features
16
+
17
+ - **Generated Tables and Figures**: Build the paper's tables and figures from the repository's own data
18
+ - **Drift Detection**: Detect when the paper and the data differ
19
+ - **LaTeX Assembly**: Compose tables for LaTeX and also display them in Jupyter
20
+ - **Shared Figure Styling**: Consistent look across LaTeX and Jupyter
21
+ - **Submission Packaging**: Automatic preparation and pre-checking for arXiv and Zenodo
22
+ - **Parameter Sweeps**: Plan a grid, run it as a scheduler job array, resume if necessary, and collect the results
23
+ - **Structured Logging**: Console and file output at separate levels, with progress bars and colour output
24
+
25
+ ## Installation
26
+
27
+ The core package has no third-party dependencies. Capabilities are installed as extras:
28
+
29
+ ```sh
30
+ pip install research-helpers[figures] # matplotlib, seaborn
31
+ pip install research-helpers[latex] # LaTeX table assembly and rendering (stdlib only)
32
+ pip install research-helpers[log] # structlog, colorama, tqdm
33
+ pip install research-helpers[sweep] # pandas, pyarrow (planning and running need neither)
34
+ ```
35
+
36
+ Requires Python 3.11 or newer.
37
+
38
+ ## Quick Start
39
+
40
+ ### Wire up the project
41
+
42
+ A project's wiring—*i.e.*, where the paper lives, how wide it is, where runs are written—goes in the
43
+ `[tool.research-helpers]` section of the `pyproject.toml` file.
44
+
45
+ ```toml
46
+ [tool.research-helpers.paper]
47
+ main = "tex/paper.tex"
48
+ tables-dir = "tex/tables"
49
+ figures-dir = "tex/figures"
50
+ text-width-pt = 468.0 # as reported by \showthe\textwidth
51
+
52
+ [tool.research-helpers.figures]
53
+ profile = "print"
54
+ palette = "colorblind"
55
+ dpi = 300
56
+ ```
57
+
58
+ `research-helpers doctor` shows every setting, the value in force, and where it came from.
59
+
60
+ Keys and values are described in the [configuration reference](https://gpizzorno.github.io/research-helpers/configuration.html).
61
+
62
+ ### Generate a table from data
63
+
64
+ ```python
65
+ import json
66
+
67
+ from research_helpers.build import TABLES, Registry
68
+ from research_helpers.latex import half_up, header, table
69
+
70
+ tables = Registry(TABLES)
71
+
72
+
73
+ @tables.register('tab:scores')
74
+ def scores() -> str:
75
+ """Accuracy on the held-out set."""
76
+ data = json.loads(SCORES_PATH.read_text())
77
+ return table(
78
+ spec='lr',
79
+ header_rows=[[header('System'), header('Accuracy')]],
80
+ body_rows=[[name.title(), half_up(v['accuracy'] * 100)] for name, v in data.items()],
81
+ caption='Accuracy on the held-out set.',
82
+ label='tab:scores',
83
+ )
84
+
85
+
86
+ if __name__ == '__main__':
87
+ raise SystemExit(tables.main())
88
+ ```
89
+
90
+ The label names the file, so this is written to `scores.tex` and the paper reads it with
91
+ `\input{tables/scores}`:
92
+
93
+ ```console
94
+ $ python -m demo.tables --install
95
+ wrote build/tables/scores.tex
96
+ installed tex/tables/scores.tex
97
+ ```
98
+
99
+ ### Check for drift
100
+
101
+ ```console
102
+ $ python -m demo.tables --check
103
+ OK: all 1 tables in the paper are current
104
+ ```
105
+
106
+ If the data is changed without rebuilding, the check fails:
107
+
108
+ ```console
109
+ $ python -m demo.tables --check
110
+ stale, the paper is behind the data: tab:scores
111
+ to fix: rebuild and install the tables
112
+ $ echo $?
113
+ 1
114
+ ```
115
+
116
+ For a full example, see [adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html).
117
+
118
+ ### Style a figure for the page
119
+
120
+ ```python
121
+ import matplotlib.pyplot as plt
122
+
123
+ from research_helpers.figures import apply_style, save
124
+
125
+ apply_style(profile='print') # \textwidth across, paper font sizes
126
+ fig, ax = plt.subplots() # the profile's size is already in rcParams
127
+ ax.plot(iterations, accuracy, marker='o')
128
+ save(fig, 'tex/figures/learning-curve.png')
129
+ ```
130
+
131
+ See details in the [figures guide](https://gpizzorno.github.io/research-helpers/guide/figures.html).
132
+
133
+ ### Run a parameter sweep
134
+
135
+ ```python
136
+ from research_helpers.sweep import Sweep
137
+
138
+ sweep = Sweep()
139
+
140
+
141
+ @sweep.context
142
+ def prepare(manifest, run_dir):
143
+ """Loaded once per array task."""
144
+ return load_corpus(manifest.metadata['corpus'])
145
+
146
+
147
+ @sweep.evaluate
148
+ def evaluate(params, corpus):
149
+ """One parameter combination."""
150
+ return {'f1': score(corpus, **params)}
151
+
152
+
153
+ if __name__ == '__main__':
154
+ raise SystemExit(sweep.main())
155
+ ```
156
+
157
+ ```console
158
+ $ python sweep_demo.py plan --config sweep.toml --run-dir runs/demo --tasks 4
159
+ planned 12 combinations over 4 array tasks (~3 per task)
160
+ manifest: runs/demo/manifest.json
161
+
162
+ submit with:
163
+ sbatch --array=1-4 <your sbatch script> runs/demo
164
+ ```
165
+
166
+ The manifest is written once and the tasks read it, so re-submitting the two that were killed
167
+ reproduces exactly their original slices instead of repartitioning the grid. Resumption is per
168
+ combination, not per task, so a job that hits its walltime keeps everything it finished.
169
+ `status` and `run` need only the standard library, so nothing needs to be installed to run a
170
+ progress check from a login node.
171
+
172
+ Full details are discussed in the [sweeps guide](https://gpizzorno.github.io/research-helpers/guide/sweeps.html).
173
+
174
+ ### Package a submission
175
+
176
+ ```console
177
+ $ research-helpers arxiv --dry-run
178
+ paper.tex 43.1 KB
179
+ paper.bbl 82.1 KB
180
+ tables/scores.tex 0.3 KB
181
+ figures/learning-curve.png 66.3 KB
182
+
183
+ 4 files, 0.19 MB
184
+ bbl format 3.3 (TeX Live 2025)
185
+
186
+ ready to upload: select xelatex and TeX Live 2025
187
+
188
+ (dry run, nothing written)
189
+ ```
190
+
191
+ The target files are keyed by the path they take *inside* the submission, so what resolves locally
192
+ will also resolve at the destination. The checks cover filenames arXiv rejects, a `.bbl` that is missing,
193
+ stale, or in a format the selected TeX Live will not read, and `microtype` font expansion under an engine
194
+ that has none. `--tar` packs it reproducibly, so re-packing an unchanged submission gives an identical
195
+ file.
196
+
197
+ See the [submission guide](https://gpizzorno.github.io/research-helpers/guide/submission.html).
198
+
199
+ ### Logging
200
+
201
+ ```python
202
+ from research_helpers.log import get_logger, setup_logging
203
+
204
+ setup_logging()
205
+ log = get_logger(__name__)
206
+
207
+ log.info('scoring', corpus='perseus', sentences=18_000)
208
+ ```
209
+
210
+ ```console
211
+ [INFO ] 19:41:47 __main__ scoring (corpus=perseus, sentences=18000)
212
+ ```
213
+
214
+ The console and the log file take separate levels. Colour is applied only when the stream is a terminal,
215
+ so no escape codes are included in a redirected log.
216
+
217
+ Details in the [logging guide](https://gpizzorno.github.io/research-helpers/guide/logging.html).
218
+
219
+ ## Documentation
220
+
221
+ The full documentation includes:
222
+
223
+ - **[Configuration](https://gpizzorno.github.io/research-helpers/configuration.html)**: Every section, key, type, and default, plus units and settings resolution
224
+ - **[Guides](https://gpizzorno.github.io/research-helpers/guide/index.html)**: Task-focused walkthroughs
225
+ - [Adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html): Full table pipeline
226
+ - [Figures](https://gpizzorno.github.io/research-helpers/guide/figures.html): Profiles, scoped styling, saving versus rendering
227
+ - [LaTeX tables](https://gpizzorno.github.io/research-helpers/guide/tables.html): Assembling tables and reading them back
228
+ - [Parameter sweeps](https://gpizzorno.github.io/research-helpers/guide/sweeps.html): Planning, running, resuming, and collecting
229
+ - [arXiv and Zenodo](https://gpizzorno.github.io/research-helpers/guide/submission.html): Submission checks and reproducible archives
230
+ - [Logging](https://gpizzorno.github.io/research-helpers/guide/logging.html): Setup, levels, colour, and progress bars
231
+ - **[API Reference](https://gpizzorno.github.io/research-helpers/api/index.html)**: Complete API documentation
232
+
233
+ ## License
234
+
235
+ The project is licensed under the [MIT License](LICENSE), allowing free use, modification, and distribution.