daftar 0.3.2__tar.gz → 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {daftar-0.3.2 → daftar-0.4.0}/CHANGELOG.md +78 -0
- daftar-0.4.0/INSTALL.md +161 -0
- {daftar-0.3.2 → daftar-0.4.0}/MANIFEST.in +4 -2
- {daftar-0.3.2/src/daftar.egg-info → daftar-0.4.0}/PKG-INFO +98 -103
- daftar-0.4.0/PUBLISHING.md +122 -0
- {daftar-0.3.2 → daftar-0.4.0}/README.md +94 -102
- daftar-0.4.0/ROADMAP.md +163 -0
- daftar-0.4.0/TESTING.md +150 -0
- daftar-0.4.0/TROUBLESHOOTING.md +234 -0
- {daftar-0.3.2 → daftar-0.4.0}/examples/adapter_usage.py +44 -1
- {daftar-0.3.2 → daftar-0.4.0}/pyproject.toml +3 -2
- daftar-0.4.0/src/daftar/__version__.py +1 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/adapters/__init__.py +7 -2
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/adapters/brian2_adapter.py +4 -73
- daftar-0.4.0/src/daftar/adapters/mne_adapter.py +372 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/capture.py +2 -1
- {daftar-0.3.2 → daftar-0.4.0/src/daftar.egg-info}/PKG-INFO +98 -103
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar.egg-info/SOURCES.txt +3 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar.egg-info/requires.txt +4 -0
- {daftar-0.3.2 → daftar-0.4.0}/tests/test_adapters_live.py +184 -0
- {daftar-0.3.2 → daftar-0.4.0}/tests/test_core.py +1 -1
- daftar-0.3.2/PUBLISHING.md +0 -134
- daftar-0.3.2/ROADMAP.md +0 -207
- daftar-0.3.2/TROUBLESHOOTING.md +0 -415
- daftar-0.3.2/src/daftar/__version__.py +0 -1
- {daftar-0.3.2 → daftar-0.4.0}/LICENSE +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/examples/daftar_in_notebooks.ipynb +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/examples/demo_end_to_end.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/setup.cfg +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/__init__.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/adapters/base.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/adapters/cpm_adapter.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/adapters/jaxley_adapter.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/adapters/meltingpot_adapter.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/cli.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/diff.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/manifest.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/notebook.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/run.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/store.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar/sweep.py +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar.egg-info/dependency_links.txt +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar.egg-info/entry_points.txt +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/src/daftar.egg-info/top_level.txt +0 -0
- {daftar-0.3.2 → daftar-0.4.0}/tests/test_notebook.py +0 -0
|
@@ -1,5 +1,83 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.0 — MNE-Python
|
|
4
|
+
|
|
5
|
+
MNE preprocessing is largely a sequence of **human decisions that are never
|
|
6
|
+
written down**. Which channels you marked bad, which ICA components you excluded
|
|
7
|
+
after looking at topographies, which epochs were dropped and why: each changes
|
|
8
|
+
every downstream number, and each typically survives only in the analyst's
|
|
9
|
+
memory.
|
|
10
|
+
|
|
11
|
+
**`ica.exclude` is the headline.** It is the most consequential unrecorded
|
|
12
|
+
decision in EEG/MEG analysis. A reviewer asking "which components did you
|
|
13
|
+
remove?" is usually asking a question with no surviving answer six months later.
|
|
14
|
+
The adapter records the exclusion list, the count, and the decomposition it
|
|
15
|
+
indexes into.
|
|
16
|
+
|
|
17
|
+
**Whether ICA converged.** `n_iter_ == max_iter` means FastICA hit the iteration
|
|
18
|
+
limit and stopped, not that it finished. MNE warns at fit time and stores
|
|
19
|
+
nothing you would notice afterwards. Recorded as `result.ica.converged`.
|
|
20
|
+
|
|
21
|
+
**`random_state=None` is flagged loudly.** ICA without a seed is not
|
|
22
|
+
reproducible, and neither are the component indices the exclusions refer to.
|
|
23
|
+
The manifest records `none (NOT REPRODUCIBLE)` rather than a bare `None`.
|
|
24
|
+
|
|
25
|
+
**Filter design, not just the band.** `raw.filter()` updates `info["highpass"]`
|
|
26
|
+
and `info["lowpass"]` and then discards `fir_design`, `phase`, `window` and the
|
|
27
|
+
transition bandwidths. A zero-phase FIR with a wide transition band and a causal
|
|
28
|
+
IIR are different filters, and "1–40 Hz" in a methods section does not
|
|
29
|
+
distinguish them. `filter_raw()` records the design and flags which values were
|
|
30
|
+
MNE defaults you never passed.
|
|
31
|
+
|
|
32
|
+
**`epochs.drop_log`.** How many epochs were rejected and for what reason. An
|
|
33
|
+
evoked average over 40 surviving epochs is a different quantity from one over
|
|
34
|
+
180, and the average alone does not say which you have.
|
|
35
|
+
|
|
36
|
+
### On subject data
|
|
37
|
+
|
|
38
|
+
This adapter runs against human neuroimaging recordings, so it records structure
|
|
39
|
+
and never content. `subject_info` and file paths are **hashed, not stored**, and
|
|
40
|
+
`meas_date` is recorded only as present or absent, because dates of service are
|
|
41
|
+
themselves identifiers. Manifests get committed to public repositories; nothing
|
|
42
|
+
the adapter writes should make that a mistake.
|
|
43
|
+
|
|
44
|
+
This is also why an MNE adapter is possible at all. The original feasibility
|
|
45
|
+
plan deferred fMRI/EEG because human neuroimaging is health data — but that risk
|
|
46
|
+
lived in *hosting* it. daftar is local, offline, and never transmits anything, so
|
|
47
|
+
an adapter can record provenance about an analysis without touching a subject's
|
|
48
|
+
recordings.
|
|
49
|
+
|
|
50
|
+
Six live tests against real MNE objects. MNE requires only `scipy>=1.13`, so it
|
|
51
|
+
shares an environment with cpm's `scipy<1.18` pin.
|
|
52
|
+
|
|
53
|
+
## 0.3.3 — documentation and cleanup
|
|
54
|
+
|
|
55
|
+
No behaviour changes. The four adapters, notebook support and the CLI are all
|
|
56
|
+
verified working; this release makes the documentation match.
|
|
57
|
+
|
|
58
|
+
**Documentation restructured** around the four things a user actually needs to
|
|
59
|
+
do, each in its own file rather than scattered across a README:
|
|
60
|
+
|
|
61
|
+
* `INSTALL.md` — installing daftar, and the per-adapter environment
|
|
62
|
+
requirements. Including the honest part: **the four target frameworks cannot
|
|
63
|
+
currently share one interpreter**, and the two-environment split that works.
|
|
64
|
+
* `TESTING.md` — the three test suites, what each covers, and how to read a
|
|
65
|
+
skip. Skips have three distinct meanings and only one of them is ordinary.
|
|
66
|
+
* `PUBLISHING.md` — the release sequence, trimmed to the steps that matter.
|
|
67
|
+
* `TROUBLESHOOTING.md` — reorganised around what is currently open versus
|
|
68
|
+
already fixed upstream.
|
|
69
|
+
* `ROADMAP.md` — updated for the shipped state.
|
|
70
|
+
|
|
71
|
+
**Removed `diagnose_jaxley.py`.** jaxley 0.14.0 fixed the `jnp.clip(a_max=...)`
|
|
72
|
+
incompatibility, so the script has nothing left to diagnose. `daftar doctor`
|
|
73
|
+
covers the general case.
|
|
74
|
+
|
|
75
|
+
**Removed the unused Brian2 log-capture path.** `capture_methods` and
|
|
76
|
+
`_MethodCapture` were superseded by `resolve_method`, which reproduces Brian2's
|
|
77
|
+
integration-method selection directly and is immune to its caching. Nothing
|
|
78
|
+
called them and no test covered them; unused code that reads private framework
|
|
79
|
+
internals is a maintenance liability, not an escape hatch.
|
|
80
|
+
|
|
3
81
|
## 0.3.2
|
|
4
82
|
|
|
5
83
|
**cpm live tests now skip with an explanation instead of failing** when cpm and
|
daftar-0.4.0/INSTALL.md
ADDED
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Installing daftar
|
|
2
|
+
|
|
3
|
+
## Just the tool
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
pip install daftar
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
That is everything. daftar has **zero runtime dependencies** and works on Python
|
|
10
|
+
3.10 or newer. It needs no account, no server, and makes no network call.
|
|
11
|
+
|
|
12
|
+
Verify:
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
python -c "import daftar; print(daftar.__version__)"
|
|
16
|
+
daftar doctor
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## With the framework adapters
|
|
22
|
+
|
|
23
|
+
Here the honest answer is more complicated than a single command, and it is
|
|
24
|
+
worth understanding why before you start.
|
|
25
|
+
|
|
26
|
+
**The five target frameworks cannot currently share one environment.** Each pins
|
|
27
|
+
its dependencies loosely, and their dependencies remove APIs on schedules the
|
|
28
|
+
frameworks do not track. As of this writing:
|
|
29
|
+
|
|
30
|
+
| Framework | Constraint | Consequence |
|
|
31
|
+
|---|---|---|
|
|
32
|
+
| brian2 ≥ 2.10 | requires **Python ≥ 3.12** | earlier brian2 releases call `ndarray.ptp`, removed in NumPy 2.0 |
|
|
33
|
+
| cpm-toolbox 0.25.6 | needs **scipy < 1.18** | passes `disp=` to `fmin_l_bfgs_b`, removed in SciPy 1.18.0 |
|
|
34
|
+
| dm-meltingpot | needs **dmlab2d**, narrow wheel coverage | weakest on macOS arm64; often needs Python 3.11 or a source build |
|
|
35
|
+
| jaxley ≥ 0.14 | fine on current JAX | 0.13.0 was broken; upgrade rather than pin |
|
|
36
|
+
| mne ≥ 1.12 | `scipy >= 1.13`, Python ≥ 3.10 | compatible with cpm's `scipy<1.18` pin, so they share an environment |
|
|
37
|
+
|
|
38
|
+
This is not anyone's fault and it is not unusual. It is the normal condition of
|
|
39
|
+
a scientific Python environment, and it is a large part of why daftar records
|
|
40
|
+
`env.*` at all.
|
|
41
|
+
|
|
42
|
+
**Do not try to force all four into one environment.** Use two, and let the test
|
|
43
|
+
suite skip whatever is absent in each — both suites go green.
|
|
44
|
+
|
|
45
|
+
### Environment A — brian2, jaxley, cpm, mne (Python 3.12)
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
conda create --name daftar312 python=3.12
|
|
49
|
+
conda activate daftar312
|
|
50
|
+
|
|
51
|
+
pip install daftar
|
|
52
|
+
pip install brian2 jaxley cpm-toolbox mne
|
|
53
|
+
pip install "scipy<1.18" # required by cpm-toolbox 0.25.6; mne is fine with it
|
|
54
|
+
pip install pytest ipython # for the test suite and notebook support
|
|
55
|
+
|
|
56
|
+
daftar doctor
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Expect:
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
daftar 0.3.3
|
|
63
|
+
python 3.12.14 on Darwin arm64
|
|
64
|
+
|
|
65
|
+
brian2 ok
|
|
66
|
+
cpm ok
|
|
67
|
+
jaxley ok
|
|
68
|
+
meltingpot - not installed
|
|
69
|
+
mne ok
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
### Environment B — meltingpot (Python 3.11)
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
conda create --name daftar311 python=3.11
|
|
76
|
+
conda activate daftar311
|
|
77
|
+
|
|
78
|
+
pip install daftar pytest
|
|
79
|
+
pip install dm-meltingpot # or install from a local clone, see below
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
If the wheel will not install on your platform, clone MeltingPot and install it
|
|
83
|
+
editable:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
git clone https://github.com/google-deepmind/meltingpot.git
|
|
87
|
+
cd meltingpot && pip install -e .
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
Note the consequence: every manifest will then record
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
env.dm_meltingpot.source editable:file:///path/to/meltingpot
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
and `daftar replay` will warn that nobody else can fetch that path. That warning
|
|
97
|
+
is correct. If you are producing manifests for other people to act on, install a
|
|
98
|
+
release build rather than a working copy.
|
|
99
|
+
|
|
100
|
+
### All extras at once
|
|
101
|
+
|
|
102
|
+
The extras exist for convenience, but read the constraints above before using
|
|
103
|
+
`[all]`:
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
pip install "daftar[jaxley]"
|
|
107
|
+
pip install "daftar[cpm]"
|
|
108
|
+
pip install "daftar[brian2]"
|
|
109
|
+
pip install "daftar[mne]"
|
|
110
|
+
pip install "daftar[meltingpot]"
|
|
111
|
+
pip install "daftar[all]" # will not resolve cleanly on one interpreter
|
|
112
|
+
pip install "daftar[dev]" # pytest, numpy, ipython
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## For development
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
git clone https://github.com/aslansd/daftar.git
|
|
121
|
+
cd daftar
|
|
122
|
+
pip install -e ".[dev]"
|
|
123
|
+
pytest -q
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
See [TESTING.md](TESTING.md) for what the different suites cover and what a
|
|
127
|
+
skip means.
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Notebooks and Colab
|
|
132
|
+
|
|
133
|
+
Nothing extra is needed — `daftar.track()` detects IPython by itself. IPython is
|
|
134
|
+
never imported when running as a script, so there is no cost to non-notebook
|
|
135
|
+
users.
|
|
136
|
+
|
|
137
|
+
For the `%%daftar` cell magic:
|
|
138
|
+
|
|
139
|
+
```python
|
|
140
|
+
!pip install daftar
|
|
141
|
+
%load_ext daftar
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
On Colab this is the whole setup. See `examples/daftar_in_notebooks.ipynb`.
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
## Checking an installation
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
daftar doctor
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Reports the interpreter, the daftar version, and every adapter as `ok`,
|
|
155
|
+
`not installed`, or **`BROKEN`** with the exception that caused it. Exits 1 if
|
|
156
|
+
anything is broken, so it is usable in CI.
|
|
157
|
+
|
|
158
|
+
**`ok` means the framework imports.** That is not the same as it working — cpm
|
|
159
|
+
under SciPy 1.18 imports cleanly and fails on the first fit. The live adapter
|
|
160
|
+
tests run a real workload and will tell you. See
|
|
161
|
+
[TROUBLESHOOTING.md](TROUBLESHOOTING.md) if something is `BROKEN`.
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
include LICENSE
|
|
2
2
|
include README.md
|
|
3
|
+
include INSTALL.md
|
|
4
|
+
include TESTING.md
|
|
3
5
|
include PUBLISHING.md
|
|
4
|
-
include CHANGELOG.md
|
|
5
|
-
include ROADMAP.md
|
|
6
6
|
include TROUBLESHOOTING.md
|
|
7
|
+
include ROADMAP.md
|
|
8
|
+
include CHANGELOG.md
|
|
7
9
|
recursive-include examples *.py
|
|
8
10
|
recursive-include examples *.ipynb
|
|
9
11
|
recursive-include tests *.py
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: daftar
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.4.0
|
|
4
4
|
Summary: Record what produced each computational result: environment, code, parameters, seeds, inputs.
|
|
5
5
|
Author: Aslan Satary Dizaji
|
|
6
6
|
License-Expression: Apache-2.0
|
|
@@ -23,6 +23,8 @@ Provides-Extra: meltingpot
|
|
|
23
23
|
Requires-Dist: dm-meltingpot; extra == "meltingpot"
|
|
24
24
|
Provides-Extra: brian2
|
|
25
25
|
Requires-Dist: brian2; extra == "brian2"
|
|
26
|
+
Provides-Extra: mne
|
|
27
|
+
Requires-Dist: mne; extra == "mne"
|
|
26
28
|
Provides-Extra: dev
|
|
27
29
|
Requires-Dist: pytest>=7; extra == "dev"
|
|
28
30
|
Requires-Dist: numpy; extra == "dev"
|
|
@@ -34,6 +36,7 @@ Requires-Dist: jaxley; extra == "all"
|
|
|
34
36
|
Requires-Dist: cpm-toolbox; extra == "all"
|
|
35
37
|
Requires-Dist: dm-meltingpot; extra == "all"
|
|
36
38
|
Requires-Dist: brian2; extra == "all"
|
|
39
|
+
Requires-Dist: mne; extra == "all"
|
|
37
40
|
Dynamic: license-file
|
|
38
41
|
|
|
39
42
|
# daftar
|
|
@@ -47,10 +50,10 @@ past run can be rebuilt.
|
|
|
47
50
|
|
|
48
51
|
Experiment trackers solved this for deep learning, where an experiment is
|
|
49
52
|
`model.fit()`. They do not help when the experiment is a Hodgkin–Huxley
|
|
50
|
-
simulation of 302 neurons, a hierarchical fit across 60 participants,
|
|
51
|
-
200-episode multi-agent sweep. Those runs have no epochs, no loss
|
|
52
|
-
checkpoints. They have parameter grids, solver tolerances, random
|
|
53
|
-
derived quantities.
|
|
53
|
+
simulation of 302 neurons, a hierarchical fit across 60 participants, an EEG
|
|
54
|
+
preprocessing pipeline, or a 200-episode multi-agent sweep. Those runs have no epochs, no loss
|
|
55
|
+
curves, and no checkpoints. They have parameter grids, solver tolerances, random
|
|
56
|
+
seeds, and derived quantities.
|
|
54
57
|
|
|
55
58
|
**No dependencies. Runs offline. No account, no server, no network call.**
|
|
56
59
|
|
|
@@ -58,6 +61,13 @@ derived quantities.
|
|
|
58
61
|
pip install daftar
|
|
59
62
|
```
|
|
60
63
|
|
|
64
|
+
Python 3.10+. Apache 2.0.
|
|
65
|
+
|
|
66
|
+
[PyPI](https://pypi.org/project/daftar/) · [INSTALL](INSTALL.md) ·
|
|
67
|
+
[TESTING](TESTING.md) · [PUBLISHING](PUBLISHING.md) ·
|
|
68
|
+
[TROUBLESHOOTING](TROUBLESHOOTING.md) · [ROADMAP](ROADMAP.md) ·
|
|
69
|
+
[CHANGELOG](CHANGELOG.md)
|
|
70
|
+
|
|
61
71
|
---
|
|
62
72
|
|
|
63
73
|
## The thirty-second version
|
|
@@ -106,10 +116,9 @@ platform to save and re-run a computational experiment.
|
|
|
106
116
|
Every one of those tools asked scientists to change how they work today in
|
|
107
117
|
exchange for a benefit arriving in three years. That trade never closes.
|
|
108
118
|
|
|
109
|
-
`daftar` inverts it. `diff` answers a question people ask weekly — *why is
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
the pitch.
|
|
119
|
+
`daftar` inverts it. `diff` answers a question people ask weekly — *why is this
|
|
120
|
+
number different from Tuesday's?* — and provenance arrives as a free side effect
|
|
121
|
+
of a tool they already wanted. Reproducibility is the by-product, not the pitch.
|
|
113
122
|
|
|
114
123
|
Three consequences of taking that seriously:
|
|
115
124
|
|
|
@@ -127,38 +136,31 @@ Three consequences of taking that seriously:
|
|
|
127
136
|
|
|
128
137
|
| Namespace | Contents |
|
|
129
138
|
|---|---|
|
|
130
|
-
| `code.*` | git commit, branch, dirty flag, **hash of uncommitted changes**, entrypoint, argv |
|
|
139
|
+
| `code.*` | git commit, branch, dirty flag, **hash of uncommitted changes**, entrypoint, argv; in notebooks, the **cell hash and session history hash** |
|
|
131
140
|
| `param.*` | everything you chose, including defaults you never passed |
|
|
132
141
|
| `seed.*` | seeds **applied** to `random`, numpy, torch, **Brian2's device RNG**; the JAX root key |
|
|
133
142
|
| `input.*` | sha256 and size of every declared input file or directory |
|
|
134
|
-
| `env.*` | interpreter, OS, versions
|
|
143
|
+
| `env.*` | interpreter, OS, package versions, **and where each package was installed from** |
|
|
135
144
|
| `result.*` | scalar outcomes worth comparing |
|
|
136
145
|
| `output.*` | sha256 of produced files |
|
|
137
146
|
| `cost.*` | wall clock, host, cpu count |
|
|
138
147
|
|
|
139
|
-
|
|
148
|
+
Three of these are less obvious than they look.
|
|
140
149
|
|
|
141
150
|
**Seeds are applied, not merely recorded.** A tool that only writes down the
|
|
142
151
|
seed is close to useless: if the code seeded itself from the clock, recording
|
|
143
152
|
that fact tells you the run is irreproducible but does nothing to fix it.
|
|
144
|
-
`track(seed=...)` sets every RNG it can reach
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
**A version string is not an identity.** jaxley 0.13.0 on PyPI
|
|
149
|
-
current JAX; jaxley `main`
|
|
150
|
-
`env.jaxley = 0.13.0` would call those two environments identical when one
|
|
151
|
-
and one does not. daftar reads PEP 610 `direct_url.json` and adds
|
|
152
|
-
`env.<pkg>.source` for anything not installed from an index
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
env.jaxley 0.13.0
|
|
156
|
-
env.jaxley.source git+https://github.com/jaxleyverse/jaxley.git#2638cca2665e
|
|
157
|
-
```
|
|
158
|
-
|
|
159
|
-
Index installs record nothing, which is the right default: absence means "from
|
|
160
|
-
PyPI". `daftar replay` then emits a `pip install` line that actually works,
|
|
161
|
-
pinning index packages by version and VCS packages by URL and commit.
|
|
153
|
+
`track(seed=...)` sets every RNG it can reach — including Brian2's device RNG,
|
|
154
|
+
which numpy seeding does not touch — then records what it set. If you pass no
|
|
155
|
+
seed, one is generated, applied, and recorded.
|
|
156
|
+
|
|
157
|
+
**A version string is not an identity.** jaxley 0.13.0 on PyPI was broken with
|
|
158
|
+
current JAX; jaxley `main` fixed it and still called itself 0.13.0. Recording
|
|
159
|
+
only `env.jaxley = 0.13.0` would call those two environments identical when one
|
|
160
|
+
works and one does not. daftar reads PEP 610 `direct_url.json` and adds
|
|
161
|
+
`env.<pkg>.source` for anything not installed from an index. `daftar replay`
|
|
162
|
+
then emits a `pip install` line that actually works, pinning index packages by
|
|
163
|
+
version and VCS packages by URL and commit.
|
|
162
164
|
|
|
163
165
|
**Dirty working trees get their diff hashed.** "Dirty" alone tells you there
|
|
164
166
|
were uncommitted edits but not whether they were the *same* edits. Hashing the
|
|
@@ -167,7 +169,7 @@ case during a debugging session.
|
|
|
167
169
|
|
|
168
170
|
---
|
|
169
171
|
|
|
170
|
-
##
|
|
172
|
+
## Commands
|
|
171
173
|
|
|
172
174
|
```bash
|
|
173
175
|
daftar list # what has been run
|
|
@@ -176,11 +178,12 @@ daftar diff r-4f21ab r-88c07e # what changed, and whether it mattered
|
|
|
176
178
|
daftar vary -l my-sweep # which fields differ across many runs
|
|
177
179
|
daftar replay r-4f21ab # what it would take to reproduce this
|
|
178
180
|
daftar export r-4f21ab -o run.zip
|
|
179
|
-
daftar doctor
|
|
181
|
+
daftar doctor # which adapters work here, and why not
|
|
180
182
|
```
|
|
181
183
|
|
|
182
184
|
`diff` exits 0 if the second run reproduces the first and 1 otherwise, so it
|
|
183
|
-
works in CI as a regression check on your own results.
|
|
185
|
+
works in CI as a regression check on your own results. `doctor` exits 1 if any
|
|
186
|
+
adapter's framework is installed but broken.
|
|
184
187
|
|
|
185
188
|
### Verdicts
|
|
186
189
|
|
|
@@ -199,7 +202,7 @@ combination implies:
|
|
|
199
202
|
|
|
200
203
|
That fourth verdict is the valuable one. An unseeded RNG buried three libraries
|
|
201
204
|
deep can survive for years because nobody ever compares two runs precisely
|
|
202
|
-
enough to notice.
|
|
205
|
+
enough to notice. daftar reports it as a finding rather than a glitch.
|
|
203
206
|
|
|
204
207
|
---
|
|
205
208
|
|
|
@@ -220,46 +223,6 @@ point 3 of 40 keeps the first two results.
|
|
|
220
223
|
|
|
221
224
|
---
|
|
222
225
|
|
|
223
|
-
## Replay
|
|
224
|
-
|
|
225
|
-
`daftar replay` prints a plan, and deliberately does **not** execute
|
|
226
|
-
anything. Re-running arbitrary recorded code would mean this package executes
|
|
227
|
-
whatever a manifest tells it to, and it still could not restore your CUDA
|
|
228
|
-
driver. What it does honestly is state the target state, check the current state
|
|
229
|
-
against it, and list every discrepancy:
|
|
230
|
-
|
|
231
|
-
```
|
|
232
|
-
Replay plan for r-4f21ab
|
|
233
|
-
|
|
234
|
-
entrypoint sim/celegans_hh.py::run_network
|
|
235
|
-
commit 9c1d0ae
|
|
236
|
-
seed 42
|
|
237
|
-
|
|
238
|
-
BLOCKERS -- this run cannot be reproduced as recorded:
|
|
239
|
-
- input file changed since the run: data/connectome.csv
|
|
240
|
-
(was a7f39b21, now 3e0c77af)
|
|
241
|
-
|
|
242
|
-
To reproduce:
|
|
243
|
-
git checkout 9c1d0ae
|
|
244
|
-
pip install numpy==2.4.6 "jaxley @ git+https://github.com/jaxleyverse/jaxley.git@2638cca"
|
|
245
|
-
```
|
|
246
|
-
|
|
247
|
-
Packages installed from a local path or in editable mode are listed as comments
|
|
248
|
-
rather than requirements, and raise a warning: nobody else can fetch
|
|
249
|
-
`file:///Users/you/Downloads/thing`, and a version pin would not reproduce it.
|
|
250
|
-
|
|
251
|
-
---
|
|
252
|
-
|
|
253
|
-
## Export
|
|
254
|
-
|
|
255
|
-
`daftar export` writes a zip containing `README.md`, `manifest.json`,
|
|
256
|
-
`fields.tsv`, and the referenced input and output files. The README is generated
|
|
257
|
-
in plain English at the archive root, so a successor learns what they are
|
|
258
|
-
looking at without installing anything. A bundle that needs our tool to be
|
|
259
|
-
understood defeats its own purpose.
|
|
260
|
-
|
|
261
|
-
---
|
|
262
|
-
|
|
263
226
|
## Notebooks and Colab
|
|
264
227
|
|
|
265
228
|
Notebooks are the hardest case: the git commit means little when cells ran in an
|
|
@@ -305,6 +268,46 @@ See `examples/daftar_in_notebooks.ipynb`, which runs on Colab.
|
|
|
305
268
|
|
|
306
269
|
---
|
|
307
270
|
|
|
271
|
+
## Replay
|
|
272
|
+
|
|
273
|
+
`daftar replay` prints a plan, and deliberately does **not** execute anything.
|
|
274
|
+
Re-running arbitrary recorded code would mean this package executes whatever a
|
|
275
|
+
manifest tells it to, and it still could not restore your CUDA driver. What it
|
|
276
|
+
does honestly is state the target state, check the current state against it, and
|
|
277
|
+
list every discrepancy:
|
|
278
|
+
|
|
279
|
+
```
|
|
280
|
+
Replay plan for r-4f21ab
|
|
281
|
+
|
|
282
|
+
entrypoint sim/celegans_hh.py::run_network
|
|
283
|
+
commit 9c1d0ae
|
|
284
|
+
seed 42
|
|
285
|
+
|
|
286
|
+
BLOCKERS -- this run cannot be reproduced as recorded:
|
|
287
|
+
- input file changed since the run: data/connectome.csv
|
|
288
|
+
(was a7f39b21, now 3e0c77af)
|
|
289
|
+
|
|
290
|
+
To reproduce:
|
|
291
|
+
git checkout 9c1d0ae
|
|
292
|
+
pip install numpy==2.4.6 "jaxley @ git+https://github.com/jaxleyverse/jaxley.git@2638cca"
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
Packages installed from a local path or in editable mode are listed as comments
|
|
296
|
+
rather than requirements, and raise a warning: nobody else can fetch
|
|
297
|
+
`file:///Users/you/Downloads/thing`, and a version pin would not reproduce it.
|
|
298
|
+
|
|
299
|
+
---
|
|
300
|
+
|
|
301
|
+
## Export
|
|
302
|
+
|
|
303
|
+
`daftar export` writes a zip containing `README.md`, `manifest.json`,
|
|
304
|
+
`fields.tsv`, and the referenced input and output files — plus `cell.py` for
|
|
305
|
+
notebook runs. The README is generated in plain English at the archive root, so
|
|
306
|
+
a successor learns what they are looking at without installing anything. A
|
|
307
|
+
bundle that needs our tool to be understood defeats its own purpose.
|
|
308
|
+
|
|
309
|
+
---
|
|
310
|
+
|
|
308
311
|
## Framework adapters
|
|
309
312
|
|
|
310
313
|
The core tracks any Python function. An adapter earns its existence only by
|
|
@@ -313,26 +316,34 @@ knowing something a generic tracker cannot infer.
|
|
|
313
316
|
| Adapter | Records what you'd otherwise lose |
|
|
314
317
|
|---|---|
|
|
315
318
|
| `jaxley` | morphology (compartments, branches, channels, synapses), `jx.integrate` defaults you never passed, `jax_enable_x64`, backend |
|
|
316
|
-
| `cpm` | parameter **bounds and priors
|
|
317
|
-
| `brian2` | **the integration method Brian2 actually chose**
|
|
319
|
+
| `cpm` | parameter **bounds and priors** (resolved to the scipy distribution and its arguments), estimator and its scipy settings, restart counts and the initial guesses themselves, per-participant convergence, cohort hash |
|
|
320
|
+
| `brian2` | **the integration method Brian2 actually chose** — its default is a candidate list and the winner is stored nowhere — plus resolved `codegen.target`, network schedule, equation hashes, realised synapse counts |
|
|
321
|
+
| `mne` | **which ICA components were excluded** and whether ICA converged, filter *design* rather than just the band, bad channels, epoch drop counts and reasons — with subject data hashed, never stored |
|
|
318
322
|
| `meltingpot` | resolved substrate ConfigDict hash, roles, episode-length cap, pinned bot checkpoints, per-player returns and Gini |
|
|
319
323
|
|
|
320
|
-
All three are verified against live installs by `tests/test_adapters_live.py`,
|
|
321
|
-
which runs real workloads and skips cleanly when a framework is absent.
|
|
322
|
-
|
|
323
324
|
```python
|
|
324
|
-
from daftar.adapters import
|
|
325
|
+
from daftar.adapters import brian2 as b2a
|
|
325
326
|
|
|
326
|
-
with daftar.track("
|
|
327
|
-
|
|
327
|
+
with daftar.track("balanced-net", seed=42) as run:
|
|
328
|
+
b2a.run_network(net, 1*second, run)
|
|
328
329
|
```
|
|
329
330
|
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
331
|
+
The MNE adapter deserves a note on privacy: it runs against human neuroimaging
|
|
332
|
+
recordings, so it records structure and never content. `subject_info` and file
|
|
333
|
+
paths are hashed rather than stored, and `meas_date` is recorded only as present
|
|
334
|
+
or absent, because dates of service are themselves identifiers. Manifests get
|
|
335
|
+
committed to public repositories; nothing the adapter writes should make that a
|
|
336
|
+
mistake.
|
|
337
|
+
|
|
338
|
+
Adapters never import their framework at module load, so `import daftar` works
|
|
339
|
+
with none of them installed. Every probe is best-effort: a provenance tool that
|
|
340
|
+
crashes a four-hour simulation because a framework renamed an attribute has done
|
|
341
|
+
far more harm than the missing field was worth.
|
|
334
342
|
|
|
335
|
-
|
|
343
|
+
All five are verified against live installs by `tests/test_adapters_live.py`.
|
|
344
|
+
See `examples/adapter_usage.py` for the pattern for each, and
|
|
345
|
+
[INSTALL.md](INSTALL.md) for which environment each needs — they do not all fit
|
|
346
|
+
in one.
|
|
336
347
|
|
|
337
348
|
### Why not Concordia
|
|
338
349
|
|
|
@@ -360,22 +371,6 @@ python examples/demo_end_to_end.py
|
|
|
360
371
|
No frameworks needed. Walks through a clean reproduction, a deliberate parameter
|
|
361
372
|
change, a genuine nondeterminism catch, a sweep, a replay plan, and an export.
|
|
362
373
|
|
|
363
|
-
## Development
|
|
364
|
-
|
|
365
|
-
```bash
|
|
366
|
-
pip install -e ".[dev]"
|
|
367
|
-
pytest -q # core suite, no frameworks needed
|
|
368
|
-
|
|
369
|
-
pip install -e ".[all]" # jaxley, cpm-toolbox, dm-meltingpot
|
|
370
|
-
pytest tests/test_adapters_live.py -v # adapters against real frameworks
|
|
371
|
-
```
|
|
372
|
-
|
|
373
|
-
Adapters fail *silently by design* -- `safe()` turns a renamed attribute into
|
|
374
|
-
`<unavailable>` rather than crashing a four-hour simulation -- so adapter rot
|
|
375
|
-
looks fine until someone reads a manifest. Re-run the live suite after every
|
|
376
|
-
upgrade of a target framework, not only at release. See `TROUBLESHOOTING.md`
|
|
377
|
-
if a framework will not install.
|
|
378
|
-
|
|
379
374
|
## Licence
|
|
380
375
|
|
|
381
376
|
Apache 2.0.
|