useml 0.2.0__tar.gz → 0.3.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. useml-0.3.2/PKG-INFO +471 -0
  2. useml-0.3.2/README.md +454 -0
  3. {useml-0.2.0 → useml-0.3.2}/pyproject.toml +1 -1
  4. {useml-0.2.0 → useml-0.3.2}/tests/test_bundle.py +17 -11
  5. useml-0.3.2/tests/test_level2.py +244 -0
  6. {useml-0.2.0 → useml-0.3.2}/tests/test_session.py +28 -31
  7. {useml-0.2.0 → useml-0.3.2}/tests/test_train_level0.py +22 -14
  8. useml-0.3.2/tests/test_usecases.py +1249 -0
  9. {useml-0.2.0 → useml-0.3.2}/tests/test_vault.py +23 -22
  10. useml-0.3.2/useml/__init__.py +94 -0
  11. {useml-0.2.0 → useml-0.3.2}/useml/template/config.py +19 -12
  12. useml-0.3.2/useml/template/trainer.py +758 -0
  13. {useml-0.2.0 → useml-0.3.2}/useml/vault/code_extractor.py +5 -2
  14. useml-0.3.2/useml/vault/model_registry.py +267 -0
  15. useml-0.3.2/useml/vault/project.py +309 -0
  16. useml-0.3.2/useml/vault/runs.py +814 -0
  17. useml-0.3.2/useml.egg-info/PKG-INFO +471 -0
  18. {useml-0.2.0 → useml-0.3.2}/useml.egg-info/SOURCES.txt +3 -0
  19. useml-0.2.0/PKG-INFO +0 -317
  20. useml-0.2.0/README.md +0 -300
  21. useml-0.2.0/tests/test_usecases.py +0 -308
  22. useml-0.2.0/useml/__init__.py +0 -224
  23. useml-0.2.0/useml/template/trainer.py +0 -296
  24. useml-0.2.0/useml/vault/project.py +0 -168
  25. useml-0.2.0/useml.egg-info/PKG-INFO +0 -317
  26. {useml-0.2.0 → useml-0.3.2}/setup.cfg +0 -0
  27. {useml-0.2.0 → useml-0.3.2}/useml/dataset/__init__.py +0 -0
  28. {useml-0.2.0 → useml-0.3.2}/useml/dataset/bundle.py +0 -0
  29. {useml-0.2.0 → useml-0.3.2}/useml/dataset/loaders.py +0 -0
  30. {useml-0.2.0 → useml-0.3.2}/useml/errors.py +0 -0
  31. {useml-0.2.0 → useml-0.3.2}/useml/imports.py +0 -0
  32. {useml-0.2.0 → useml-0.3.2}/useml/session/__init__.py +0 -0
  33. {useml-0.2.0 → useml-0.3.2}/useml/session/component.py +0 -0
  34. {useml-0.2.0 → useml-0.3.2}/useml/session/manager.py +0 -0
  35. {useml-0.2.0 → useml-0.3.2}/useml/template/__init__.py +0 -0
  36. {useml-0.2.0 → useml-0.3.2}/useml/template/loss.py +0 -0
  37. {useml-0.2.0 → useml-0.3.2}/useml/template/model.py +0 -0
  38. {useml-0.2.0 → useml-0.3.2}/useml/vault/__init__.py +0 -0
  39. {useml-0.2.0 → useml-0.3.2}/useml/vault/core.py +0 -0
  40. {useml-0.2.0 → useml-0.3.2}/useml/vault/snapshot.py +0 -0
  41. {useml-0.2.0 → useml-0.3.2}/useml/workdir/__init__.py +0 -0
  42. {useml-0.2.0 → useml-0.3.2}/useml/workdir/_hook.py +0 -0
  43. {useml-0.2.0 → useml-0.3.2}/useml.egg-info/dependency_links.txt +0 -0
  44. {useml-0.2.0 → useml-0.3.2}/useml.egg-info/requires.txt +0 -0
  45. {useml-0.2.0 → useml-0.3.2}/useml.egg-info/top_level.txt +0 -0
useml-0.3.2/PKG-INFO ADDED
@@ -0,0 +1,471 @@
1
+ Metadata-Version: 2.4
2
+ Name: useml
3
+ Version: 0.3.2
4
+ Summary: Minimalist ML experiment versioning for PyTorch — checkpointing, source archiving, and model isolation.
5
+ Author-email: Enzo-py <85191552+Enzo-py@users.noreply.github.com>
6
+ Project-URL: Homepage, https://github.com/Enzo-py/useml
7
+ Project-URL: Bug Tracker, https://github.com/Enzo-py/useml/issues
8
+ Requires-Python: >=3.8
9
+ Description-Content-Type: text/markdown
10
+ Requires-Dist: torch>=2.0.0
11
+ Requires-Dist: pyyaml>=6.0
12
+ Provides-Extra: dev
13
+ Requires-Dist: pytest>=7.0.0; extra == "dev"
14
+ Requires-Dist: build; extra == "dev"
15
+ Requires-Dist: twine; extra == "dev"
16
+ Requires-Dist: black; extra == "dev"
17
+
18
+ # useml
19
+
20
+ **Stop coding plumbing, start training models.**
21
+
22
+ `useml` is a minimalist ML experiment versioning framework for PyTorch. It handles checkpointing, source archiving, and model isolation so you can focus on the architecture.
23
+
24
+ ## Installation
25
+
26
+ ```bash
27
+ pip install useml
28
+ ```
29
+
30
+ ## Core concepts
31
+
32
+ ```
33
+ vault/
34
+ └── my-experiment/ ← project
35
+ ├── versions.yaml ← model version index
36
+ ├── snap_20240501_.../ ← snapshot (one per checkpoint)
37
+ │ ├── weights/ ← model .pth files
38
+ │ ├── optimizers/ ← optimizer states (for resume)
39
+ │ ├── configs/ ← per-component YAML configs
40
+ │ ├── source/ ← archived source code
41
+ │ ├── manifest.yaml ← component registry
42
+ │ └── metadata.yaml ← message + metrics
43
+ └── snap_20240502_.../
44
+ ```
45
+
46
+ ## Basic usage
47
+
48
+ ```python
49
+ import useml
50
+ from useml import Config
51
+
52
+ useml.init("vault")
53
+ project = useml.new("mnist-cnn") # returns a Project
54
+ # or: project = useml.focus("mnist-cnn") to reopen existing
55
+ ```
56
+
57
+ `useml.*` only exposes setup (`init` / `new` / `focus`) and the core classes
58
+ (`Config`, `Model`, `Loss`, `DataBundle`). Everything else — training, loading,
59
+ comparing — lives on the **project**.
60
+
61
+ ---
62
+
63
+ ## Level-0 — train in two lines
64
+
65
+ ```python
66
+ from useml import Config
67
+
68
+ config = Config(
69
+ epochs=10,
70
+ lr=1e-3,
71
+ loss="cross_entropy",
72
+ checkpoint_strategy="best",
73
+ )
74
+
75
+ run = project.runs.new(ConvNet, "mnist", config=config)
76
+ history = run.train()
77
+ ```
78
+
79
+ Built-in dataset strings: `"mnist"`, `"fashion_mnist"`, `"cifar10"`, `"cifar100"`,
80
+ `"hf:<dataset_name>"` (HuggingFace), or any `torch.utils.data.Dataset`.
81
+
82
+ ### Pass your own loaders
83
+
84
+ ```python
85
+ run = project.runs.new(ConvNet, config=config)
86
+ history = run.train(train_loader, val_loader)
87
+ ```
88
+
89
+ ### Custom step function
90
+
91
+ Override the default `criterion(model(x), y)` without subclassing:
92
+
93
+ ```python
94
+ def ae_step(model, batch, device):
95
+ x, _ = batch
96
+ return F.mse_loss(model(x.to(device)), x.to(device)) # target = input
97
+
98
+ run = project.runs.new(AutoEncoder, "mnist", config=config)
99
+ history = run.train(step_fn=ae_step)
100
+ ```
101
+
102
+ ---
103
+
104
+ ## Inspecting results
105
+
106
+ ```python
107
+ project.show()
108
+ # --- Project: mnist-cnn ---
109
+ # Runs: 5
110
+ # Models: convnet(5)
111
+ # Latest: epoch 10 [snap_20240502_...]
112
+
113
+ # Leaderboard
114
+ print(project.runs.leaderboard("val_loss"))
115
+
116
+ # Best model
117
+ best = project.models["convnet"].best("val_loss")
118
+ model = best.load()
119
+ print(best.metrics) # {"val_loss": 0.07, "epoch": 8}
120
+ ```
121
+
122
+ ---
123
+
124
+ ## Model registry
125
+
126
+ After training, each saved checkpoint becomes a versioned model entry:
127
+
128
+ ```python
129
+ reg = project.models["convnet"] # ModelRegistry
130
+
131
+ reg["v1"].load() # load specific version
132
+ reg["v1"].metrics # {"val_loss": 0.12, "epoch": 5}
133
+ reg["v1"].config # {"lr": 0.001, "epochs": 10, ...}
134
+
135
+ reg.latest.load() # most recent checkpoint
136
+ reg.best("val_loss").load() # best by metric (min by default)
137
+ reg.best("accuracy", mode="max") # best by metric (max)
138
+
139
+ for v in reg:
140
+ print(v.label, v.metrics)
141
+ ```
142
+
143
+ ---
144
+
145
+ ## Runs view
146
+
147
+ ```python
148
+ runs = project.runs
149
+
150
+ runs.latest # most recent RunRecord
151
+ runs.best("val_loss") # best run by metric
152
+ list(runs) # all RunRecords, newest first
153
+ runs.filter(model="convnet") # filter by component name
154
+ runs.leaderboard("val_loss", top=5) # formatted table (str)
155
+ ```
156
+
157
+ Each `RunRecord` lets you reload a model:
158
+
159
+ ```python
160
+ record = project.runs.latest
161
+ model = record.load() # single-model run → no name needed
162
+ model = record.load("encoder", model=Encoder())
163
+ ```
164
+
165
+ ---
166
+
167
+ ## Config
168
+
169
+ `Config` accepts standard fields plus any custom keyword argument.
170
+ Custom fields are serialised to the snapshot YAML and forwarded to the model
171
+ constructor by `ModelVersion.load()`.
172
+
173
+ ```python
174
+ config = Config(
175
+ epochs=30,
176
+ lr=1e-3,
177
+ optimizer="adam", # "adam" | "adamw" | "sgd"
178
+ loss="cross_entropy", # str | nn.Module class/instance | callable
179
+ batch_size=128,
180
+ val_split=0.1,
181
+ seed=42,
182
+ checkpoint_strategy="best", # "every_n" (default) | "best" | "last"
183
+ checkpoint_metric="val_loss",
184
+ checkpoint_mode="min",
185
+ checkpoint_every=5, # used when strategy="every_n"
186
+ early_stop_patience=10,
187
+ device="auto", # "auto" | "cpu" | "cuda" | "mps"
188
+ # Custom fields — stored in YAML and injected into model constructor:
189
+ latent_dim=16,
190
+ kl_weight=1e-3,
191
+ )
192
+
193
+ config.latent_dim # 16
194
+ config.to_dict() # all fields including custom ones
195
+ ```
196
+
197
+ On reload, `useml` reads the snapshot YAML and calls `Model(latent_dim=16)`
198
+ automatically — no hardcoded defaults needed:
199
+
200
+ ```python
201
+ model = project.models["vae"]["v1"].load() # VAE(latent_dim=16) auto-injected
202
+ ```
203
+
204
+ ---
205
+
206
+ ## Level-2 — scheduler, metrics, checkpoint strategies
207
+
208
+ ### LR scheduler
209
+
210
+ ```python
211
+ from torch.optim.lr_scheduler import CosineAnnealingLR
212
+
213
+ run = project.runs.new(Net, dataset, config=config)
214
+ run.scheduler = CosineAnnealingLR(run.optimizer, T_max=config.epochs)
215
+ history = run.train()
216
+ ```
217
+
218
+ Or pass a factory to `train()`:
219
+
220
+ ```python
221
+ history = run.train(
222
+ scheduler=lambda opt: CosineAnnealingLR(opt, T_max=config.epochs)
223
+ )
224
+ ```
225
+
226
+ `ReduceLROnPlateau` is detected and receives the monitored metric automatically.
227
+ Set `run.scheduler_per_batch = True` for per-batch schedulers (`OneCycleLR`).
228
+
229
+ ### Custom metrics
230
+
231
+ ```python
232
+ def accuracy(model, batch, device):
233
+ x, y = batch
234
+ return (model(x.to(device)).argmax(-1) == y.to(device)).float().mean()
235
+
236
+ history = run.train(metrics={"accuracy": accuracy})
237
+ # history keys: train_loss, val_loss, train_accuracy, val_accuracy
238
+ ```
239
+
240
+ Metrics are embedded in every checkpoint's `metadata.yaml`:
241
+
242
+ ```yaml
243
+ metrics:
244
+ val_loss: 0.07
245
+ val_accuracy: 0.94
246
+ epoch: 8
247
+ ```
248
+
249
+ ### Best-val checkpointing
250
+
251
+ ```python
252
+ config = Config(
253
+ checkpoint_strategy="best",
254
+ checkpoint_metric="accuracy",
255
+ checkpoint_mode="max",
256
+ )
257
+ ```
258
+
259
+ `"last"` writes a single snapshot at the very end of training.
260
+
261
+ ### Early stopping
262
+
263
+ ```python
264
+ config = Config(
265
+ early_stop_patience=5,
266
+ early_stop_metric="val_loss",
267
+ early_stop_mode="min",
268
+ )
269
+ ```
270
+
271
+ ---
272
+
273
+ ## Level-2 — Pattern B (custom training loop)
274
+
275
+ For GANs, diffusion, multi-optimizer setups — anything that doesn't fit
276
+ the standard supervised loop:
277
+
278
+ ```python
279
+ config = Config(
280
+ epochs=50,
281
+ checkpoint_strategy="best",
282
+ checkpoint_metric="g_loss",
283
+ checkpoint_mode="min",
284
+ )
285
+
286
+ run = project.runs.new(config=config) # no model yet
287
+ run.register("G", Generator(), optimizer=opt_G)
288
+ run.register("D", Discriminator(), optimizer=opt_D)
289
+
290
+ for epoch in run.epochs(config.epochs):
291
+ for batch in loader:
292
+ # your adversarial step here
293
+ g_loss = ...
294
+ d_loss = ...
295
+ run.update(n=batch_size, g_loss=g_loss, d_loss=d_loss)
296
+ run.epoch_end(epoch) # aggregate + log + checkpoint G and D atomically
297
+ ```
298
+
299
+ `register()` returns `self` for chaining. All registered models are always
300
+ committed **atomically** — no risk of G and D from different epochs.
301
+
302
+ ### on_epoch_end hook
303
+
304
+ ```python
305
+ class MyRun(Run):
306
+ def on_epoch_end(self, epoch, train_metrics, val_metrics):
307
+ if epoch % 10 == 0:
308
+ self.scheduler.base_lrs = [lr * 0.5 for lr in self.scheduler.base_lrs]
309
+ ```
310
+
311
+ ---
312
+
313
+ ## Training resume
314
+
315
+ ```python
316
+ # Pattern A
317
+ run = project.runs.new(Net, dataset, config=Config(epochs=20))
318
+ history = run.train(resume_from="\\latest") # continues from saved epoch
319
+
320
+ # Pattern B
321
+ run = project.runs.new(config=config)
322
+ run.register("G", Generator(), optimizer=opt_G)
323
+ run.resume("\\latest") # loads weights + optimizer state
324
+
325
+ for epoch in run.epochs(config.epochs): # starts from saved_epoch + 1
326
+ ...
327
+ ```
328
+
329
+ ---
330
+
331
+ ## Level-1 — DataBundle (versioned data contracts)
332
+
333
+ `DataBundle` pairs a dataset with a preprocessing transform and hashes both.
334
+ The transform source code is archived verbatim in every snapshot.
335
+
336
+ ```python
337
+ from useml import DataBundle
338
+
339
+ bundle = DataBundle(
340
+ name="polar_clouds",
341
+ source=CartesianDataset(), # raw data — any Dataset or built-in name
342
+ transform=gridify, # per-sample callable: sample -> sample
343
+ version="v1",
344
+ cache=True, # materialise on first run, reload after
345
+ )
346
+
347
+ run = project.runs.new(MyCNN, bundle, config=config)
348
+ history = run.train()
349
+ ```
350
+
351
+ Three hashes are recorded in `metadata.yaml`:
352
+
353
+ | Hash | What it tracks |
354
+ |---|---|
355
+ | `source_hash` | Fingerprint of the raw dataset |
356
+ | `transform_hash` | sha256 of the transform function source + name |
357
+ | `cache_key` | sha256(source_hash + transform_hash) |
358
+
359
+ ### Factory transforms
360
+
361
+ ```python
362
+ def make_gridify(n_r, n_t):
363
+ def gridify(sample): ...
364
+ gridify.__name__ = f"gridify_{n_r}r_{n_t}t"
365
+ return gridify
366
+
367
+ bundle_fine = DataBundle("ds", raw, transform=make_gridify(16, 32))
368
+ bundle_coarse = DataBundle("ds", raw, transform=make_gridify(8, 16))
369
+ # → different transform_hash, different archived .py file
370
+ ```
371
+
372
+ ---
373
+
374
+ ## Code isolation (workdir)
375
+
376
+ ```python
377
+ from useml.session.manager import _session
378
+
379
+ _session.mount("\\latest")
380
+
381
+ from useml.workdir import mymodel # loads archived source from snapshot
382
+ model = mymodel.MyModel() # uses the code version from the checkpoint
383
+
384
+ _session.mount("\\workdir") # unmount, back to live code
385
+ ```
386
+
387
+ Snapshot tags:
388
+
389
+ | Tag | Meaning |
390
+ |---|---|
391
+ | `\latest` | Most recent snapshot |
392
+ | `\head~N` | N snapshots before latest |
393
+ | `\workdir` / `\current` | Unmount |
394
+ | `snap_20240501_...` | Exact folder name |
395
+
396
+ ---
397
+
398
+ ## Custom loss
399
+
400
+ ```python
401
+ Config(loss="cross_entropy") # built-in: "cross_entropy", "mse", "bce", "l1"
402
+ Config(loss=nn.HuberLoss) # class
403
+ Config(loss=nn.HuberLoss(delta=0.5)) # instance
404
+ Config(loss=focal_loss) # callable — source archived automatically
405
+ ```
406
+
407
+ ---
408
+
409
+ ## Error codes
410
+
411
+ All exceptions carry a `[UML-NNN]` code. See [ERRORS.md](ERRORS.md) for the
412
+ full catalog with causes and fixes.
413
+
414
+ ---
415
+
416
+ ## API reference
417
+
418
+ ### Setup
419
+
420
+ | | |
421
+ |---|---|
422
+ | `useml.init(vault_path)` | Connect to a vault directory |
423
+ | `useml.new(name)` → `Project` | Create a new project |
424
+ | `useml.focus(name)` → `Project` | Open an existing project |
425
+
426
+ ### Project
427
+
428
+ | | |
429
+ |---|---|
430
+ | `project.runs` | `RunsView` — all runs, factory for new ones |
431
+ | `project.models` | `ModelsView` — versioned model registry |
432
+ | `project.log()` | List of snapshots, newest first |
433
+ | `project.show()` | Print project summary |
434
+
435
+ ### RunsView / Run
436
+
437
+ | | |
438
+ |---|---|
439
+ | `project.runs.new(Model, dataset?, config=, name=)` | Create a training run |
440
+ | `run.train(loaders?, step_fn?, metrics?, scheduler?, resume_from?)` | Run Pattern A |
441
+ | `run.register(name, model, optimizer?)` | Register a model (Pattern B) |
442
+ | `run.epochs(n)` | Epoch iterator (Pattern B) |
443
+ | `run.update(n=, **metrics)` | Accumulate batch metrics (Pattern B) |
444
+ | `run.epoch_end(epoch)` | Aggregate + checkpoint (Pattern B) |
445
+ | `run.resume(tag)` | Load weights + optimizer states |
446
+ | `project.runs.latest` | Most recent `RunRecord` |
447
+ | `project.runs.best(metric, mode=)` | Best `RunRecord` |
448
+ | `project.runs.filter(model=)` | Filtered `RunsView` |
449
+ | `project.runs.leaderboard(metric, top=)` | Formatted table (str) |
450
+
451
+ ### ModelsView / ModelRegistry / ModelVersion
452
+
453
+ | | |
454
+ |---|---|
455
+ | `project.models["name"]` | `ModelRegistry` for one component |
456
+ | `registry.latest` | Most recent `ModelVersion` |
457
+ | `registry.best(metric, mode=)` | Best `ModelVersion` |
458
+ | `registry["v1"]` | Specific version |
459
+ | `version.load(model=None)` | Restore weights (auto-instantiate if omitted) |
460
+ | `version.metrics` | `{"val_loss": 0.07, "epoch": 8, ...}` |
461
+ | `version.config` | Config dict from snapshot YAML |
462
+
463
+ ### DataBundle
464
+
465
+ | | |
466
+ |---|---|
467
+ | `DataBundle(name, source, transform=, version=, cache=)` | Create a data contract |
468
+ | `bundle.source_hash()` | Raw dataset fingerprint |
469
+ | `bundle.transform_hash()` | Transform source hash |
470
+ | `bundle.cache_key()` | Combined hash |
471
+ | `bundle.to_meta_dict()` | Dict for snapshot `metadata.yaml` |