research-helpers 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- research_helpers-1.0.0/CHANGELOG.md +68 -0
- research_helpers-1.0.0/LICENSE +21 -0
- research_helpers-1.0.0/MANIFEST.in +1 -0
- research_helpers-1.0.0/PKG-INFO +293 -0
- research_helpers-1.0.0/README.md +235 -0
- research_helpers-1.0.0/pyproject.toml +224 -0
- research_helpers-1.0.0/setup.cfg +4 -0
- research_helpers-1.0.0/src/research_helpers/__init__.py +17 -0
- research_helpers-1.0.0/src/research_helpers/archive.py +162 -0
- research_helpers-1.0.0/src/research_helpers/arxiv.py +253 -0
- research_helpers-1.0.0/src/research_helpers/build.py +389 -0
- research_helpers-1.0.0/src/research_helpers/cli.py +76 -0
- research_helpers-1.0.0/src/research_helpers/display.py +282 -0
- research_helpers-1.0.0/src/research_helpers/figures.py +299 -0
- research_helpers-1.0.0/src/research_helpers/latex.py +173 -0
- research_helpers-1.0.0/src/research_helpers/log/__init__.py +22 -0
- research_helpers-1.0.0/src/research_helpers/log/config.py +197 -0
- research_helpers-1.0.0/src/research_helpers/log/formatters.py +88 -0
- research_helpers-1.0.0/src/research_helpers/log/handlers.py +72 -0
- research_helpers-1.0.0/src/research_helpers/log/tqdm_integration.py +32 -0
- research_helpers-1.0.0/src/research_helpers/project.py +425 -0
- research_helpers-1.0.0/src/research_helpers/py.typed +0 -0
- research_helpers-1.0.0/src/research_helpers/sweep/__init__.py +30 -0
- research_helpers-1.0.0/src/research_helpers/sweep/collect.py +143 -0
- research_helpers-1.0.0/src/research_helpers/sweep/grid.py +133 -0
- research_helpers-1.0.0/src/research_helpers/sweep/runner.py +150 -0
- research_helpers-1.0.0/src/research_helpers/sweep/sweep.py +339 -0
- research_helpers-1.0.0/src/research_helpers/templates/sweep.sbatch +78 -0
- research_helpers-1.0.0/src/research_helpers.egg-info/PKG-INFO +293 -0
- research_helpers-1.0.0/src/research_helpers.egg-info/SOURCES.txt +43 -0
- research_helpers-1.0.0/src/research_helpers.egg-info/dependency_links.txt +1 -0
- research_helpers-1.0.0/src/research_helpers.egg-info/entry_points.txt +2 -0
- research_helpers-1.0.0/src/research_helpers.egg-info/requires.txt +35 -0
- research_helpers-1.0.0/src/research_helpers.egg-info/top_level.txt +1 -0
- research_helpers-1.0.0/tests/test_archive.py +211 -0
- research_helpers-1.0.0/tests/test_arxiv.py +265 -0
- research_helpers-1.0.0/tests/test_build.py +387 -0
- research_helpers-1.0.0/tests/test_cli.py +261 -0
- research_helpers-1.0.0/tests/test_display.py +256 -0
- research_helpers-1.0.0/tests/test_figures.py +305 -0
- research_helpers-1.0.0/tests/test_latex.py +193 -0
- research_helpers-1.0.0/tests/test_log.py +354 -0
- research_helpers-1.0.0/tests/test_packaging.py +32 -0
- research_helpers-1.0.0/tests/test_project.py +357 -0
- research_helpers-1.0.0/tests/test_sweep.py +401 -0
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. The format follows
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project adheres to
|
|
5
|
+
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [1.0.0] — 2026-09-11
|
|
8
|
+
|
|
9
|
+
First public release.
|
|
10
|
+
|
|
11
|
+
The package was generalized out of three research projects that had each grown their own copy of
|
|
12
|
+
the same utilities.
|
|
13
|
+
|
|
14
|
+
### The idea
|
|
15
|
+
|
|
16
|
+
`research-helpers` reads a project's **wiring**—*i.e., where the paper lives, how wide it is, where
|
|
17
|
+
sweep runs are written—and nothing else. Research parameters stay in the project that owns
|
|
18
|
+
them. This wiring lives in one `[tool.research-helpers]` section of a project's `pyproject.toml`,
|
|
19
|
+
which is also the project-root marker, so finding the configuration and finding the root are one
|
|
20
|
+
operation. Every setting has a working default, and every entry point is callable with plain
|
|
21
|
+
arguments and no project file at all.
|
|
22
|
+
|
|
23
|
+
### Added
|
|
24
|
+
|
|
25
|
+
- **`build`** — a registry of generated artefacts with three-way drift detection: a table the
|
|
26
|
+
paper is behind on, one that is generated but never pulled in, and one typed by hand with no
|
|
27
|
+
emitter behind it. `--check` exits non-zero, so "every number in the paper is generated from
|
|
28
|
+
data in the repository, and is current" becomes a build status.
|
|
29
|
+
- **`latex`** — table assembly with the conventions a paper needs: half-up rounding, escaping
|
|
30
|
+
for text arriving from data, en-dashes for ranges, em-dashes for empty cells.
|
|
31
|
+
- **`display`** — parse a generated `.tex` table back into rows, spans and alignment, for reading
|
|
32
|
+
in a notebook or asserting on in a test.
|
|
33
|
+
- **`figures`** — one matplotlib look across a project, and a `print` profile that sizes a figure
|
|
34
|
+
to the document it is going into, so it renders in the paper at 1:1 with body-text-sized labels.
|
|
35
|
+
- **`arxiv`** — inventory a submission keyed by the paths it takes *inside* the upload, and check
|
|
36
|
+
it against the engine and TeX Live year it will build under: filenames arXiv rejects, a `.bbl`
|
|
37
|
+
that is missing, stale or in the wrong `biblatex` format, and `microtype` font expansion under an
|
|
38
|
+
engine that has none.
|
|
39
|
+
- **`archive`** — byte-reproducible tarballs. Pinned `mtimes`, `uid`, and `gid`, normalised modes,
|
|
40
|
+
sorted members, and an emptied gzip filename field, so a published checksum means something.
|
|
41
|
+
- **`sweep`** — plan a parameter grid once, run it as a scheduler job array, resume what was
|
|
42
|
+
killed, collect the parts into one table. Resumption is per combination rather than per task,
|
|
43
|
+
the manifest is written once so a sparse resubmission reproduces its original slices, and
|
|
44
|
+
planning, running and status need only the standard library.
|
|
45
|
+
- **`log`** — *structlog* over the standard library, with separate console and file levels, colour
|
|
46
|
+
only when the stream is a terminal, and progress bars that do not fight the log.
|
|
47
|
+
- **`research-helpers` CLI** — `doctor` prints every setting, the value in force and where it came
|
|
48
|
+
from; `arxiv` assembles, checks, packs and previews a submission.
|
|
49
|
+
- Lengths may be written in inches, millimetres, centimetres, or TeX points by changing a key's
|
|
50
|
+
suffix: `text-width-pt = 468.0` is what `\showthe\textwidth` prints, copied across with no
|
|
51
|
+
arithmetic in between.
|
|
52
|
+
- Full documentation at <https://gpizzorno.github.io/research-helpers/>.
|
|
53
|
+
|
|
54
|
+
### Notes for packagers
|
|
55
|
+
|
|
56
|
+
- Requires Python 3.11 or newer.
|
|
57
|
+
- **The core package has no third-party dependencies.** Capabilities are opt-in extras:
|
|
58
|
+
`figures` (matplotlib, seaborn), `log` (structlog, colorama, tqdm), `sweep` (pandas, pyarrow),
|
|
59
|
+
and `latex`, which is standard library only and exists, so the capability is discoverable.
|
|
60
|
+
Importing `research_helpers` pulls in none of them.
|
|
61
|
+
- The package ships a `py.typed` marker, so annotations are visible to type checkers.
|
|
62
|
+
|
|
63
|
+
### Verified
|
|
64
|
+
|
|
65
|
+
- 276 tests, 97% coverage, run against Python 3.11, 3.12, and 3.13 in CI.
|
|
66
|
+
- The documentation builds with warnings as errors.
|
|
67
|
+
|
|
68
|
+
[1.0.0]: https://github.com/gpizzorno/research-helpers/releases/tag/v1.0.0
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Gabe Pizzorno
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
include CHANGELOG.md
|
|
@@ -0,0 +1,293 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: research-helpers
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Utilities for code-intensive research projects and the papers they produce
|
|
5
|
+
Author: Gabe Pizzorno
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/gpizzorno/research-helpers
|
|
8
|
+
Project-URL: Documentation, https://gpizzorno.github.io/research-helpers/
|
|
9
|
+
Project-URL: Repository, https://github.com/gpizzorno/research-helpers
|
|
10
|
+
Project-URL: Issues, https://github.com/gpizzorno/research-helpers/issues
|
|
11
|
+
Project-URL: Changelog, https://github.com/gpizzorno/research-helpers/blob/main/CHANGELOG.md
|
|
12
|
+
Keywords: research,latex,arxiv,zenodo,matplotlib,reproducibility,slurm,parameter-sweep
|
|
13
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
14
|
+
Classifier: Intended Audience :: Science/Research
|
|
15
|
+
Classifier: Topic :: Scientific/Engineering
|
|
16
|
+
Classifier: Topic :: Text Processing :: Markup :: LaTeX
|
|
17
|
+
Classifier: Typing :: Typed
|
|
18
|
+
Classifier: Programming Language :: Python
|
|
19
|
+
Classifier: Programming Language :: Python :: 3
|
|
20
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
24
|
+
Classifier: Operating System :: OS Independent
|
|
25
|
+
Requires-Python: >=3.11
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
License-File: LICENSE
|
|
28
|
+
Provides-Extra: figures
|
|
29
|
+
Requires-Dist: matplotlib; extra == "figures"
|
|
30
|
+
Requires-Dist: seaborn; extra == "figures"
|
|
31
|
+
Provides-Extra: log
|
|
32
|
+
Requires-Dist: colorama; extra == "log"
|
|
33
|
+
Requires-Dist: structlog; extra == "log"
|
|
34
|
+
Requires-Dist: tqdm; extra == "log"
|
|
35
|
+
Provides-Extra: latex
|
|
36
|
+
Provides-Extra: sweep
|
|
37
|
+
Requires-Dist: pandas; extra == "sweep"
|
|
38
|
+
Requires-Dist: pyarrow; extra == "sweep"
|
|
39
|
+
Provides-Extra: dev
|
|
40
|
+
Requires-Dist: research-helpers[figures,latex,log,sweep]; extra == "dev"
|
|
41
|
+
Requires-Dist: factory_boy; extra == "dev"
|
|
42
|
+
Requires-Dist: hypothesis; extra == "dev"
|
|
43
|
+
Requires-Dist: pillow; extra == "dev"
|
|
44
|
+
Requires-Dist: pytest-cov; extra == "dev"
|
|
45
|
+
Requires-Dist: pytest-mock; extra == "dev"
|
|
46
|
+
Requires-Dist: pytest; extra == "dev"
|
|
47
|
+
Requires-Dist: ruff; extra == "dev"
|
|
48
|
+
Requires-Dist: ty; extra == "dev"
|
|
49
|
+
Requires-Dist: types-requests; extra == "dev"
|
|
50
|
+
Provides-Extra: docs
|
|
51
|
+
Requires-Dist: research-helpers[figures,latex,log,sweep]; extra == "docs"
|
|
52
|
+
Requires-Dist: sphinx>=7.2.0; extra == "docs"
|
|
53
|
+
Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
|
|
54
|
+
Requires-Dist: sphinx-autodoc-typehints>=1.25.0; extra == "docs"
|
|
55
|
+
Requires-Dist: sphinxcontrib-mermaid>=1.0.0; extra == "docs"
|
|
56
|
+
Requires-Dist: myst-parser>=2.0.0; extra == "docs"
|
|
57
|
+
Dynamic: license-file
|
|
58
|
+
|
|
59
|
+
# Research Helpers
|
|
60
|
+
|
|
61
|
+
[](https://opensource.org/licenses/MIT)
|
|
62
|
+
[](https://www.python.org)
|
|
63
|
+
[](https://github.com/gpizzorno/research-helpers/actions/workflows/tests.yml)
|
|
64
|
+
[](https://gpizzorno.github.io/research-helpers/)
|
|
65
|
+
|
|
66
|
+
**Research Helpers** is a set of utilities for managing code-intensive research projects and their accompanying papers.
|
|
67
|
+
Includes shared figure styling, LaTeX table generation and rendering, a build pipeline that keeps
|
|
68
|
+
a paper's generated tables and figures from drifting out of date, submission packaging for [arXiv](https://arxiv.org)
|
|
69
|
+
and [Zenodo](https://zenodo.org), structured logging, and a parameter-sweep engine for [scheduler](https://slurm.schedmd.com/overview.html) job arrays.
|
|
70
|
+
|
|
71
|
+
[Read the documentation](https://gpizzorno.github.io/research-helpers/)
|
|
72
|
+
|
|
73
|
+
## Features
|
|
74
|
+
|
|
75
|
+
- **Generated Tables and Figures**: Build the paper's tables and figures from the repository's own data
|
|
76
|
+
- **Drift Detection**: Detect when the paper and the data differ
|
|
77
|
+
- **LaTeX Assembly**: Compose tables for LaTeX and also display them in Jupyter
|
|
78
|
+
- **Shared Figure Styling**: Consistent look across LaTeX and Jupyter
|
|
79
|
+
- **Submission Packaging**: Automatic preparation and pre-checking for arXiv and Zenodo
|
|
80
|
+
- **Parameter Sweeps**: Plan a grid, run it as a scheduler job array, resume if necessary, and collect the results
|
|
81
|
+
- **Structured Logging**: Console and file output at separate levels, with progress bars and colour output
|
|
82
|
+
|
|
83
|
+
## Installation
|
|
84
|
+
|
|
85
|
+
The core package has no third-party dependencies. Capabilities are installed as extras:
|
|
86
|
+
|
|
87
|
+
```sh
|
|
88
|
+
pip install research-helpers[figures] # matplotlib, seaborn
|
|
89
|
+
pip install research-helpers[latex] # LaTeX table assembly and rendering (stdlib only)
|
|
90
|
+
pip install research-helpers[log] # structlog, colorama, tqdm
|
|
91
|
+
pip install research-helpers[sweep] # pandas, pyarrow (planning and running need neither)
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Requires Python 3.11 or newer.
|
|
95
|
+
|
|
96
|
+
## Quick Start
|
|
97
|
+
|
|
98
|
+
### Wire up the project
|
|
99
|
+
|
|
100
|
+
A project's wiring—*i.e.*, where the paper lives, how wide it is, where runs are written—goes in the
|
|
101
|
+
`[tool.research-helpers]` section of the `pyproject.toml` file.
|
|
102
|
+
|
|
103
|
+
```toml
|
|
104
|
+
[tool.research-helpers.paper]
|
|
105
|
+
main = "tex/paper.tex"
|
|
106
|
+
tables-dir = "tex/tables"
|
|
107
|
+
figures-dir = "tex/figures"
|
|
108
|
+
text-width-pt = 468.0 # as reported by \showthe\textwidth
|
|
109
|
+
|
|
110
|
+
[tool.research-helpers.figures]
|
|
111
|
+
profile = "print"
|
|
112
|
+
palette = "colorblind"
|
|
113
|
+
dpi = 300
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
`research-helpers doctor` shows every setting, the value in force, and where it came from.
|
|
117
|
+
|
|
118
|
+
Keys and values are described in the [configuration reference](https://gpizzorno.github.io/research-helpers/configuration.html).
|
|
119
|
+
|
|
120
|
+
### Generate a table from data
|
|
121
|
+
|
|
122
|
+
```python
|
|
123
|
+
import json
|
|
124
|
+
|
|
125
|
+
from research_helpers.build import TABLES, Registry
|
|
126
|
+
from research_helpers.latex import half_up, header, table
|
|
127
|
+
|
|
128
|
+
tables = Registry(TABLES)
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
@tables.register('tab:scores')
|
|
132
|
+
def scores() -> str:
|
|
133
|
+
"""Accuracy on the held-out set."""
|
|
134
|
+
data = json.loads(SCORES_PATH.read_text())
|
|
135
|
+
return table(
|
|
136
|
+
spec='lr',
|
|
137
|
+
header_rows=[[header('System'), header('Accuracy')]],
|
|
138
|
+
body_rows=[[name.title(), half_up(v['accuracy'] * 100)] for name, v in data.items()],
|
|
139
|
+
caption='Accuracy on the held-out set.',
|
|
140
|
+
label='tab:scores',
|
|
141
|
+
)
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
if __name__ == '__main__':
|
|
145
|
+
raise SystemExit(tables.main())
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
The label names the file, so this is written to `scores.tex` and the paper reads it with
|
|
149
|
+
`\input{tables/scores}`:
|
|
150
|
+
|
|
151
|
+
```console
|
|
152
|
+
$ python -m demo.tables --install
|
|
153
|
+
wrote build/tables/scores.tex
|
|
154
|
+
installed tex/tables/scores.tex
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
### Check for drift
|
|
158
|
+
|
|
159
|
+
```console
|
|
160
|
+
$ python -m demo.tables --check
|
|
161
|
+
OK: all 1 tables in the paper are current
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
If the data is changed without rebuilding, the check fails:
|
|
165
|
+
|
|
166
|
+
```console
|
|
167
|
+
$ python -m demo.tables --check
|
|
168
|
+
stale, the paper is behind the data: tab:scores
|
|
169
|
+
to fix: rebuild and install the tables
|
|
170
|
+
$ echo $?
|
|
171
|
+
1
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
For a full example, see [adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html).
|
|
175
|
+
|
|
176
|
+
### Style a figure for the page
|
|
177
|
+
|
|
178
|
+
```python
|
|
179
|
+
import matplotlib.pyplot as plt
|
|
180
|
+
|
|
181
|
+
from research_helpers.figures import apply_style, save
|
|
182
|
+
|
|
183
|
+
apply_style(profile='print') # \textwidth across, paper font sizes
|
|
184
|
+
fig, ax = plt.subplots() # the profile's size is already in rcParams
|
|
185
|
+
ax.plot(iterations, accuracy, marker='o')
|
|
186
|
+
save(fig, 'tex/figures/learning-curve.png')
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
See details in the [figures guide](https://gpizzorno.github.io/research-helpers/guide/figures.html).
|
|
190
|
+
|
|
191
|
+
### Run a parameter sweep
|
|
192
|
+
|
|
193
|
+
```python
|
|
194
|
+
from research_helpers.sweep import Sweep
|
|
195
|
+
|
|
196
|
+
sweep = Sweep()
|
|
197
|
+
|
|
198
|
+
|
|
199
|
+
@sweep.context
|
|
200
|
+
def prepare(manifest, run_dir):
|
|
201
|
+
"""Loaded once per array task."""
|
|
202
|
+
return load_corpus(manifest.metadata['corpus'])
|
|
203
|
+
|
|
204
|
+
|
|
205
|
+
@sweep.evaluate
|
|
206
|
+
def evaluate(params, corpus):
|
|
207
|
+
"""One parameter combination."""
|
|
208
|
+
return {'f1': score(corpus, **params)}
|
|
209
|
+
|
|
210
|
+
|
|
211
|
+
if __name__ == '__main__':
|
|
212
|
+
raise SystemExit(sweep.main())
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
```console
|
|
216
|
+
$ python sweep_demo.py plan --config sweep.toml --run-dir runs/demo --tasks 4
|
|
217
|
+
planned 12 combinations over 4 array tasks (~3 per task)
|
|
218
|
+
manifest: runs/demo/manifest.json
|
|
219
|
+
|
|
220
|
+
submit with:
|
|
221
|
+
sbatch --array=1-4 <your sbatch script> runs/demo
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
The manifest is written once and the tasks read it, so re-submitting the two that were killed
|
|
225
|
+
reproduces exactly their original slices instead of repartitioning the grid. Resumption is per
|
|
226
|
+
combination, not per task, so a job that hits its walltime keeps everything it finished.
|
|
227
|
+
`status` and `run` need only the standard library, so nothing needs to be installed to run a
|
|
228
|
+
progress check from a login node.
|
|
229
|
+
|
|
230
|
+
Full details are discussed in the [sweeps guide](https://gpizzorno.github.io/research-helpers/guide/sweeps.html).
|
|
231
|
+
|
|
232
|
+
### Package a submission
|
|
233
|
+
|
|
234
|
+
```console
|
|
235
|
+
$ research-helpers arxiv --dry-run
|
|
236
|
+
paper.tex 43.1 KB
|
|
237
|
+
paper.bbl 82.1 KB
|
|
238
|
+
tables/scores.tex 0.3 KB
|
|
239
|
+
figures/learning-curve.png 66.3 KB
|
|
240
|
+
|
|
241
|
+
4 files, 0.19 MB
|
|
242
|
+
bbl format 3.3 (TeX Live 2025)
|
|
243
|
+
|
|
244
|
+
ready to upload: select xelatex and TeX Live 2025
|
|
245
|
+
|
|
246
|
+
(dry run, nothing written)
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
The target files are keyed by the path they take *inside* the submission, so what resolves locally
|
|
250
|
+
will also resolve at the destination. The checks cover filenames arXiv rejects, a `.bbl` that is missing,
|
|
251
|
+
stale, or in a format the selected TeX Live will not read, and `microtype` font expansion under an engine
|
|
252
|
+
that has none. `--tar` packs it reproducibly, so re-packing an unchanged submission gives an identical
|
|
253
|
+
file.
|
|
254
|
+
|
|
255
|
+
See the [submission guide](https://gpizzorno.github.io/research-helpers/guide/submission.html).
|
|
256
|
+
|
|
257
|
+
### Logging
|
|
258
|
+
|
|
259
|
+
```python
|
|
260
|
+
from research_helpers.log import get_logger, setup_logging
|
|
261
|
+
|
|
262
|
+
setup_logging()
|
|
263
|
+
log = get_logger(__name__)
|
|
264
|
+
|
|
265
|
+
log.info('scoring', corpus='perseus', sentences=18_000)
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
```console
|
|
269
|
+
[INFO ] 19:41:47 __main__ scoring (corpus=perseus, sentences=18000)
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
The console and the log file take separate levels. Colour is applied only when the stream is a terminal,
|
|
273
|
+
so no escape codes are included in a redirected log.
|
|
274
|
+
|
|
275
|
+
Details in the [logging guide](https://gpizzorno.github.io/research-helpers/guide/logging.html).
|
|
276
|
+
|
|
277
|
+
## Documentation
|
|
278
|
+
|
|
279
|
+
The full documentation includes:
|
|
280
|
+
|
|
281
|
+
- **[Configuration](https://gpizzorno.github.io/research-helpers/configuration.html)**: Every section, key, type, and default, plus units and settings resolution
|
|
282
|
+
- **[Guides](https://gpizzorno.github.io/research-helpers/guide/index.html)**: Task-focused walkthroughs
|
|
283
|
+
- [Adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html): Full table pipeline
|
|
284
|
+
- [Figures](https://gpizzorno.github.io/research-helpers/guide/figures.html): Profiles, scoped styling, saving versus rendering
|
|
285
|
+
- [LaTeX tables](https://gpizzorno.github.io/research-helpers/guide/tables.html): Assembling tables and reading them back
|
|
286
|
+
- [Parameter sweeps](https://gpizzorno.github.io/research-helpers/guide/sweeps.html): Planning, running, resuming, and collecting
|
|
287
|
+
- [arXiv and Zenodo](https://gpizzorno.github.io/research-helpers/guide/submission.html): Submission checks and reproducible archives
|
|
288
|
+
- [Logging](https://gpizzorno.github.io/research-helpers/guide/logging.html): Setup, levels, colour, and progress bars
|
|
289
|
+
- **[API Reference](https://gpizzorno.github.io/research-helpers/api/index.html)**: Complete API documentation
|
|
290
|
+
|
|
291
|
+
## License
|
|
292
|
+
|
|
293
|
+
The project is licensed under the [MIT License](LICENSE), allowing free use, modification, and distribution.
|
|
@@ -0,0 +1,235 @@
|
|
|
1
|
+
# Research Helpers
|
|
2
|
+
|
|
3
|
+
[](https://opensource.org/licenses/MIT)
|
|
4
|
+
[](https://www.python.org)
|
|
5
|
+
[](https://github.com/gpizzorno/research-helpers/actions/workflows/tests.yml)
|
|
6
|
+
[](https://gpizzorno.github.io/research-helpers/)
|
|
7
|
+
|
|
8
|
+
**Research Helpers** is a set of utilities for managing code-intensive research projects and their accompanying papers.
|
|
9
|
+
Includes shared figure styling, LaTeX table generation and rendering, a build pipeline that keeps
|
|
10
|
+
a paper's generated tables and figures from drifting out of date, submission packaging for [arXiv](https://arxiv.org)
|
|
11
|
+
and [Zenodo](https://zenodo.org), structured logging, and a parameter-sweep engine for [scheduler](https://slurm.schedmd.com/overview.html) job arrays.
|
|
12
|
+
|
|
13
|
+
[Read the documentation](https://gpizzorno.github.io/research-helpers/)
|
|
14
|
+
|
|
15
|
+
## Features
|
|
16
|
+
|
|
17
|
+
- **Generated Tables and Figures**: Build the paper's tables and figures from the repository's own data
|
|
18
|
+
- **Drift Detection**: Detect when the paper and the data differ
|
|
19
|
+
- **LaTeX Assembly**: Compose tables for LaTeX and also display them in Jupyter
|
|
20
|
+
- **Shared Figure Styling**: Consistent look across LaTeX and Jupyter
|
|
21
|
+
- **Submission Packaging**: Automatic preparation and pre-checking for arXiv and Zenodo
|
|
22
|
+
- **Parameter Sweeps**: Plan a grid, run it as a scheduler job array, resume if necessary, and collect the results
|
|
23
|
+
- **Structured Logging**: Console and file output at separate levels, with progress bars and colour output
|
|
24
|
+
|
|
25
|
+
## Installation
|
|
26
|
+
|
|
27
|
+
The core package has no third-party dependencies. Capabilities are installed as extras:
|
|
28
|
+
|
|
29
|
+
```sh
|
|
30
|
+
pip install research-helpers[figures] # matplotlib, seaborn
|
|
31
|
+
pip install research-helpers[latex] # LaTeX table assembly and rendering (stdlib only)
|
|
32
|
+
pip install research-helpers[log] # structlog, colorama, tqdm
|
|
33
|
+
pip install research-helpers[sweep] # pandas, pyarrow (planning and running need neither)
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Requires Python 3.11 or newer.
|
|
37
|
+
|
|
38
|
+
## Quick Start
|
|
39
|
+
|
|
40
|
+
### Wire up the project
|
|
41
|
+
|
|
42
|
+
A project's wiring—*i.e.*, where the paper lives, how wide it is, where runs are written—goes in the
|
|
43
|
+
`[tool.research-helpers]` section of the `pyproject.toml` file.
|
|
44
|
+
|
|
45
|
+
```toml
|
|
46
|
+
[tool.research-helpers.paper]
|
|
47
|
+
main = "tex/paper.tex"
|
|
48
|
+
tables-dir = "tex/tables"
|
|
49
|
+
figures-dir = "tex/figures"
|
|
50
|
+
text-width-pt = 468.0 # as reported by \showthe\textwidth
|
|
51
|
+
|
|
52
|
+
[tool.research-helpers.figures]
|
|
53
|
+
profile = "print"
|
|
54
|
+
palette = "colorblind"
|
|
55
|
+
dpi = 300
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`research-helpers doctor` shows every setting, the value in force, and where it came from.
|
|
59
|
+
|
|
60
|
+
Keys and values are described in the [configuration reference](https://gpizzorno.github.io/research-helpers/configuration.html).
|
|
61
|
+
|
|
62
|
+
### Generate a table from data
|
|
63
|
+
|
|
64
|
+
```python
|
|
65
|
+
import json
|
|
66
|
+
|
|
67
|
+
from research_helpers.build import TABLES, Registry
|
|
68
|
+
from research_helpers.latex import half_up, header, table
|
|
69
|
+
|
|
70
|
+
tables = Registry(TABLES)
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
@tables.register('tab:scores')
|
|
74
|
+
def scores() -> str:
|
|
75
|
+
"""Accuracy on the held-out set."""
|
|
76
|
+
data = json.loads(SCORES_PATH.read_text())
|
|
77
|
+
return table(
|
|
78
|
+
spec='lr',
|
|
79
|
+
header_rows=[[header('System'), header('Accuracy')]],
|
|
80
|
+
body_rows=[[name.title(), half_up(v['accuracy'] * 100)] for name, v in data.items()],
|
|
81
|
+
caption='Accuracy on the held-out set.',
|
|
82
|
+
label='tab:scores',
|
|
83
|
+
)
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+
if __name__ == '__main__':
|
|
87
|
+
raise SystemExit(tables.main())
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
The label names the file, so this is written to `scores.tex` and the paper reads it with
|
|
91
|
+
`\input{tables/scores}`:
|
|
92
|
+
|
|
93
|
+
```console
|
|
94
|
+
$ python -m demo.tables --install
|
|
95
|
+
wrote build/tables/scores.tex
|
|
96
|
+
installed tex/tables/scores.tex
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### Check for drift
|
|
100
|
+
|
|
101
|
+
```console
|
|
102
|
+
$ python -m demo.tables --check
|
|
103
|
+
OK: all 1 tables in the paper are current
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
If the data is changed without rebuilding, the check fails:
|
|
107
|
+
|
|
108
|
+
```console
|
|
109
|
+
$ python -m demo.tables --check
|
|
110
|
+
stale, the paper is behind the data: tab:scores
|
|
111
|
+
to fix: rebuild and install the tables
|
|
112
|
+
$ echo $?
|
|
113
|
+
1
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
For a full example, see [adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html).
|
|
117
|
+
|
|
118
|
+
### Style a figure for the page
|
|
119
|
+
|
|
120
|
+
```python
|
|
121
|
+
import matplotlib.pyplot as plt
|
|
122
|
+
|
|
123
|
+
from research_helpers.figures import apply_style, save
|
|
124
|
+
|
|
125
|
+
apply_style(profile='print') # \textwidth across, paper font sizes
|
|
126
|
+
fig, ax = plt.subplots() # the profile's size is already in rcParams
|
|
127
|
+
ax.plot(iterations, accuracy, marker='o')
|
|
128
|
+
save(fig, 'tex/figures/learning-curve.png')
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
See details in the [figures guide](https://gpizzorno.github.io/research-helpers/guide/figures.html).
|
|
132
|
+
|
|
133
|
+
### Run a parameter sweep
|
|
134
|
+
|
|
135
|
+
```python
|
|
136
|
+
from research_helpers.sweep import Sweep
|
|
137
|
+
|
|
138
|
+
sweep = Sweep()
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
@sweep.context
|
|
142
|
+
def prepare(manifest, run_dir):
|
|
143
|
+
"""Loaded once per array task."""
|
|
144
|
+
return load_corpus(manifest.metadata['corpus'])
|
|
145
|
+
|
|
146
|
+
|
|
147
|
+
@sweep.evaluate
|
|
148
|
+
def evaluate(params, corpus):
|
|
149
|
+
"""One parameter combination."""
|
|
150
|
+
return {'f1': score(corpus, **params)}
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
if __name__ == '__main__':
|
|
154
|
+
raise SystemExit(sweep.main())
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
```console
|
|
158
|
+
$ python sweep_demo.py plan --config sweep.toml --run-dir runs/demo --tasks 4
|
|
159
|
+
planned 12 combinations over 4 array tasks (~3 per task)
|
|
160
|
+
manifest: runs/demo/manifest.json
|
|
161
|
+
|
|
162
|
+
submit with:
|
|
163
|
+
sbatch --array=1-4 <your sbatch script> runs/demo
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
The manifest is written once and the tasks read it, so re-submitting the two that were killed
|
|
167
|
+
reproduces exactly their original slices instead of repartitioning the grid. Resumption is per
|
|
168
|
+
combination, not per task, so a job that hits its walltime keeps everything it finished.
|
|
169
|
+
`status` and `run` need only the standard library, so nothing needs to be installed to run a
|
|
170
|
+
progress check from a login node.
|
|
171
|
+
|
|
172
|
+
Full details are discussed in the [sweeps guide](https://gpizzorno.github.io/research-helpers/guide/sweeps.html).
|
|
173
|
+
|
|
174
|
+
### Package a submission
|
|
175
|
+
|
|
176
|
+
```console
|
|
177
|
+
$ research-helpers arxiv --dry-run
|
|
178
|
+
paper.tex 43.1 KB
|
|
179
|
+
paper.bbl 82.1 KB
|
|
180
|
+
tables/scores.tex 0.3 KB
|
|
181
|
+
figures/learning-curve.png 66.3 KB
|
|
182
|
+
|
|
183
|
+
4 files, 0.19 MB
|
|
184
|
+
bbl format 3.3 (TeX Live 2025)
|
|
185
|
+
|
|
186
|
+
ready to upload: select xelatex and TeX Live 2025
|
|
187
|
+
|
|
188
|
+
(dry run, nothing written)
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
The target files are keyed by the path they take *inside* the submission, so what resolves locally
|
|
192
|
+
will also resolve at the destination. The checks cover filenames arXiv rejects, a `.bbl` that is missing,
|
|
193
|
+
stale, or in a format the selected TeX Live will not read, and `microtype` font expansion under an engine
|
|
194
|
+
that has none. `--tar` packs it reproducibly, so re-packing an unchanged submission gives an identical
|
|
195
|
+
file.
|
|
196
|
+
|
|
197
|
+
See the [submission guide](https://gpizzorno.github.io/research-helpers/guide/submission.html).
|
|
198
|
+
|
|
199
|
+
### Logging
|
|
200
|
+
|
|
201
|
+
```python
|
|
202
|
+
from research_helpers.log import get_logger, setup_logging
|
|
203
|
+
|
|
204
|
+
setup_logging()
|
|
205
|
+
log = get_logger(__name__)
|
|
206
|
+
|
|
207
|
+
log.info('scoring', corpus='perseus', sentences=18_000)
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
```console
|
|
211
|
+
[INFO ] 19:41:47 __main__ scoring (corpus=perseus, sentences=18000)
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
The console and the log file take separate levels. Colour is applied only when the stream is a terminal,
|
|
215
|
+
so no escape codes are included in a redirected log.
|
|
216
|
+
|
|
217
|
+
Details in the [logging guide](https://gpizzorno.github.io/research-helpers/guide/logging.html).
|
|
218
|
+
|
|
219
|
+
## Documentation
|
|
220
|
+
|
|
221
|
+
The full documentation includes:
|
|
222
|
+
|
|
223
|
+
- **[Configuration](https://gpizzorno.github.io/research-helpers/configuration.html)**: Every section, key, type, and default, plus units and settings resolution
|
|
224
|
+
- **[Guides](https://gpizzorno.github.io/research-helpers/guide/index.html)**: Task-focused walkthroughs
|
|
225
|
+
- [Adding a generated table](https://gpizzorno.github.io/research-helpers/guide/generated-table.html): Full table pipeline
|
|
226
|
+
- [Figures](https://gpizzorno.github.io/research-helpers/guide/figures.html): Profiles, scoped styling, saving versus rendering
|
|
227
|
+
- [LaTeX tables](https://gpizzorno.github.io/research-helpers/guide/tables.html): Assembling tables and reading them back
|
|
228
|
+
- [Parameter sweeps](https://gpizzorno.github.io/research-helpers/guide/sweeps.html): Planning, running, resuming, and collecting
|
|
229
|
+
- [arXiv and Zenodo](https://gpizzorno.github.io/research-helpers/guide/submission.html): Submission checks and reproducible archives
|
|
230
|
+
- [Logging](https://gpizzorno.github.io/research-helpers/guide/logging.html): Setup, levels, colour, and progress bars
|
|
231
|
+
- **[API Reference](https://gpizzorno.github.io/research-helpers/api/index.html)**: Complete API documentation
|
|
232
|
+
|
|
233
|
+
## License
|
|
234
|
+
|
|
235
|
+
The project is licensed under the [MIT License](LICENSE), allowing free use, modification, and distribution.
|