model-mirror-cli 0.2.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,550 @@
1
+ Metadata-Version: 2.4
2
+ Name: model-mirror-cli
3
+ Version: 0.2.0
4
+ Summary: Mirror and validate Hugging Face archives without leaving payloads in the default cache.
5
+ Project-URL: Repository, https://github.com/xlr8harder/model-mirror
6
+ Project-URL: Issues, https://github.com/xlr8harder/model-mirror/issues
7
+ Author: xlr8harder
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Keywords: huggingface,mirror,model-archive,torrent
11
+ Classifier: Development Status :: 4 - Beta
12
+ Classifier: Environment :: Console
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Operating System :: OS Independent
15
+ Classifier: Programming Language :: Python :: 3
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Topic :: System :: Archiving
20
+ Requires-Python: >=3.11
21
+ Requires-Dist: huggingface-hub>=1.0
22
+ Requires-Dist: pyyaml>=6.0
23
+ Provides-Extra: torrent
24
+ Requires-Dist: libtorrent<2.1,>=2.0.10; extra == 'torrent'
25
+ Description-Content-Type: text/markdown
26
+
27
+ # model-mirror
28
+
29
+ Mirror Hugging Face repositories into local bulk storage and verify that the
30
+ files remain complete.
31
+
32
+ `model-mirror` downloads directly into one archive directory, avoiding payload
33
+ files being left behind in the default Hugging Face cache. It records the exact
34
+ Hub commit mirrored, writes a local hash manifest, and keeps verification state
35
+ beside each model. If you already use `hf auth login`, model-mirror will try to
36
+ find that token automatically.
37
+
38
+ Online operations resolve the requested Hugging Face revision, usually `main`,
39
+ to a specific Hub commit before downloading, verifying, or repairing. The local
40
+ mirror stays tied to that commit so files are not mixed across upstream updates.
41
+ If the Hub repo later moves to a newer commit, `verify` reports the upstream
42
+ change without modifying local files.
43
+
44
+ ## Quick Start
45
+
46
+ Install the `model-mirror-cli` distribution from PyPI with
47
+ [uv](https://docs.astral.sh/uv/) (Python 3.11 or newer):
48
+
49
+ ```bash
50
+ uv tool install model-mirror-cli
51
+
52
+ model-mirror config directory /mnt/big-drive/huggingface
53
+ model-mirror config set hf-xet-reconstruct-write-sequentially true # useful for HDDs
54
+ # Optional if token autodetection does not find your Hugging Face token:
55
+ model-mirror config set token-path ~/.cache/huggingface/token
56
+
57
+ model-mirror mirror org/model
58
+ model-mirror status
59
+ model-mirror verify org/model
60
+ model-mirror repair org/model # if verify reports repair paths
61
+ ```
62
+
63
+ The PyPI distribution is named `model-mirror-cli` to distinguish it from an
64
+ unrelated package; the installed command remains `model-mirror`. To include
65
+ experimental torrent support from the start, install
66
+ `uv tool install 'model-mirror-cli[torrent]'`.
67
+
68
+ Update and inspect an installed release with:
69
+
70
+ ```bash
71
+ uv tool upgrade model-mirror-cli
72
+ model-mirror --version
73
+ ```
74
+
75
+ For development, clone the
76
+ [source repository](https://github.com/xlr8harder/model-mirror), run
77
+ `uv sync --all-extras --dev`, and prefix commands with `uv run`. The rest of
78
+ this README assumes the tool installation above.
79
+
80
+ For periodic maintenance of the whole archive:
81
+
82
+ ```bash
83
+ model-mirror verify --all --max-age 30d || true
84
+ model-mirror repair --all
85
+ ```
86
+
87
+ Mirrors are stored by repo type:
88
+
89
+ ```text
90
+ /mnt/big-drive/huggingface/models/org/model/
91
+ /mnt/big-drive/huggingface/datasets/org/data/
92
+ /mnt/big-drive/huggingface/spaces/org/space/
93
+ /mnt/big-drive/huggingface/.model-mirror/cache/
94
+ /mnt/big-drive/huggingface/.model-mirror/tmp/
95
+ ```
96
+
97
+ The archive has one top-level control directory, `.model-mirror/`. Its `cache/`
98
+ and `tmp/` children contain disposable Hugging Face/Xet state and resumable
99
+ download staging. Each mirrored repository also has its own `.model-mirror/`
100
+ directory for durable commit-scoped metadata. Payload directories never contain
101
+ a model-mirror virtual environment or package-installation cache.
102
+
103
+ Run `model-mirror --help` or `model-mirror COMMAND --help` for the full CLI
104
+ reference. Run `model-mirror config options` for every supported config key.
105
+ Commands exit non-zero for dirty, incomplete, busy, or invalid states where that
106
+ matters; see each subcommand's help for exact exit-status behavior.
107
+
108
+ Without a repository argument, `model-mirror status` prints a compact,
109
+ metadata-only archive table with repository type and ID, recorded file count
110
+ and payload size, abbreviated resolved commit, verification age, and
111
+ exceptional state. A healthy row uses `-` in `EXCEPTIONS` instead of repeating
112
+ `clean`. Torrent and live activity columns appear only when relevant.
113
+ `model-mirror list` is currently an alias for the same output.
114
+
115
+ Status reads model-mirror's existing verification, manifest, snapshot, lock,
116
+ progress, and torrent metadata. It does not recursively scan payload or cache
117
+ directories, read payload bytes, contact the Hub, or update metadata. Missing
118
+ recorded counts or sizes are displayed as unknown; use `verify` to reconcile
119
+ the local filesystem.
120
+
121
+ Pass a repository ID for a detailed, strictly local last-known report without
122
+ contacting the Hub:
123
+
124
+ ```bash
125
+ model-mirror status org/model
126
+ model-mirror list org/model # same detailed view
127
+ model-mirror status --repo-type dataset org/data
128
+ model-mirror status --check-upstream org/model
129
+ model-mirror status --verbose org/model
130
+ model-mirror status --json
131
+ ```
132
+
133
+ The default detail view emphasizes verification state and age, resolved commit,
134
+ recorded payload size, upstream state, torrent publication, exceptional state,
135
+ and useful next actions. Issues and repair paths appear only when present.
136
+ `--verbose` prints all recorded metadata fields.
137
+
138
+ `--check-upstream` performs an advisory live comparison between the locally
139
+ resolved commit and the commit currently resolved by the requested upstream
140
+ revision. It does not write verification state, snapshot metadata, timestamps,
141
+ or locks. The ordinary recorded upstream state is still shown separately from
142
+ the live observation in JSON.
143
+
144
+ `--json` emits the versioned `model-mirror-status` schema with one
145
+ `repositories` array for both single-repository and archive-wide output.
146
+ Numbers remain numeric, timestamps are ISO 8601, and issues and repair paths are
147
+ arrays. `--verbose` is accepted as a no-op when combined with `--json`. A
148
+ mismatch between verification and snapshot commits is reported as
149
+ `snapshot-stale`.
150
+
151
+ ## Verification
152
+
153
+ `mirror` verifies by default. A clean mirror has:
154
+
155
+ - all expected Hub files present
156
+ - expected file sizes
157
+ - local SHA-256 and Git blob SHA-1 hashes in versioned `.manifest`
158
+ - LFS file hashes compared with Hub LFS SHA-256 metadata
159
+ - regular Git files compared with Hub Git blob ids
160
+ - `.verification` with `status: clean`
161
+
162
+ By default, verification ignores unexpected payload files, so they do not make
163
+ an otherwise correct mirror dirty. Use `verify --strict` to report unexpected
164
+ files and make verification fail. Model-mirror's own state and cache paths are
165
+ not treated as payload extras.
166
+
167
+ Useful verification commands:
168
+
169
+ ```bash
170
+ model-mirror verify org/model
171
+ model-mirror verify --cached org/model
172
+ model-mirror verify --offline org/model
173
+ model-mirror verify --strict org/model
174
+ model-mirror verify --all
175
+ model-mirror verify --all --max-age 7d
176
+ ```
177
+
178
+ `--cached` checks presence, sizes, and Hub-provided hashes from current
179
+ `.manifest` rows without rehashing payload files. If cached hash data is missing
180
+ or stale, cached verification exits non-zero and tells you to run full
181
+ verification. `--offline` does not contact the Hub, so it does not detect
182
+ whether the upstream repo has moved to a newer commit. Full offline verification
183
+ requires an existing `.manifest`; `--offline --cached` only reports the current
184
+ `.verification` state. `--max-age` is useful for periodic jobs that should skip
185
+ recently verified clean mirrors.
186
+
187
+ If the upstream repository is unavailable, online verification exits non-zero
188
+ and prints the command to mark the local mirror offline-only:
189
+
190
+ ```bash
191
+ model-mirror offline org/model
192
+ ```
193
+
194
+ Offline-only mirrors use local verification only and do not check whether the
195
+ Hub repo has moved or disappeared. Use `model-mirror online org/model` to
196
+ re-enable Hub checks.
197
+
198
+ If one repo is already locked, a single-repo `verify` exits non-zero. With
199
+ `verify --all`, locked repos are reported as skipped, remaining repos are still
200
+ checked, and the final exit status is non-zero.
201
+
202
+ Verification records missing or corrupt files as repair paths in
203
+ `.verification`. Repairs redownload only those paths:
204
+
205
+ ```bash
206
+ model-mirror verify org/model
207
+ model-mirror repair org/model
208
+ ```
209
+
210
+ `repair org/model` consumes the existing `.verification` state. If no
211
+ verification state exists, it tells you to run `verify` first. It prints how old
212
+ the verification result is, warns after 24 hours, updates manifest rows for
213
+ repaired files, and runs a final cached verification. In a `verify` then
214
+ `repair` workflow, the initial full verify hashes existing files once; repaired
215
+ files are hashed again after download, but unchanged large files are not
216
+ rehashed a second time.
217
+
218
+ Repair also reconciles a pinned snapshot plan whose commit is older than the
219
+ clean verification state. It fetches the recorded commit's Hub metadata and
220
+ reuses current manifest hashes, so payload files are not reread when that cached
221
+ evidence is complete. A successful commit update promotes the new snapshot plan
222
+ before recording the new commit as clean.
223
+
224
+ If repair sees incomplete cached verification data for untouched files, it stops
225
+ before downloading and tells you to run full `verify`. `repair --force-partial`
226
+ overrides that safety check, but may leave the repository inconsistent and still
227
+ exits non-zero if final cached verification cannot prove the result.
228
+
229
+ An offline-only mirror cannot be repaired because there is no upstream source to
230
+ repair against. A direct `repair org/model` exits non-zero with that explanation;
231
+ `repair --all` logs a warning and skips offline-only mirrors.
232
+
233
+ ## Experimental Torrent Publishing And Recovery
234
+
235
+ Torrent support is optional and experimental. The `upgrade` and `torrent`
236
+ commands, their behavior, and the torrent-specific metadata formats may change
237
+ before stabilization. Ordinary mirror, verify, and repair workflows remain
238
+ stable and do not require the torrent extra:
239
+
240
+ ```bash
241
+ # Install or replace the PyPI tool with torrent support:
242
+ uv tool install --force 'model-mirror-cli[torrent]'
243
+
244
+ # Or add the extra to the checkout's development environment:
245
+ uv sync --extra torrent
246
+ ```
247
+
248
+ Every torrent is an immutable publication of one resolved
249
+ `repo@commit`. Publishing first requires a clean, pinned mirror. It completes
250
+ any missing torrent hash coverage, writes an ordinary hybrid v1/v2 `.torrent`,
251
+ prints a standard magnet URI and external-client data location, and creates a
252
+ persistent fence that prevents the canonical archive from moving to another
253
+ commit.
254
+
255
+ ```bash
256
+ model-mirror upgrade org/model # optional explicit precomputation
257
+ model-mirror torrent create org/model # artifacts and fence; no seed intent
258
+ model-mirror torrent publish org/model # artifacts, fence, and durable seed intent
259
+ model-mirror torrent show org/model
260
+ ```
261
+
262
+ Downloads and same-commit repairs accumulate reusable torrent hashes during
263
+ their existing payload pass. `upgrade` is for older archives or incomplete
264
+ coverage; it reads only files still missing hashes for the current publication
265
+ profile:
266
+
267
+ ```bash
268
+ model-mirror upgrade org/model
269
+ model-mirror upgrade --all --dry-run
270
+ model-mirror upgrade --all
271
+ ```
272
+
273
+ The publication profile is the versioned, deterministic set of rules that fixes
274
+ file order, piece sizing, padding, descriptor encoding, and therefore torrent
275
+ identity. The current and only profile is `hybrid-v1-v2-1`; model-mirror selects
276
+ it automatically. There is no user-selectable profile option yet. A future
277
+ identity-affecting algorithm change will use a new profile name rather than
278
+ silently changing an existing swarm.
279
+
280
+ The managed backend is a replaceable long-running process:
281
+
282
+ ```bash
283
+ model-mirror torrent serve
284
+ ```
285
+
286
+ Run it under your normal service manager with restart-on-failure and
287
+ restart-on-boot. Publication intent, verified file fingerprints, and the update
288
+ fence are durable; a restarted backend reconciles desired seeds without an
289
+ explicit reseed command or a payload-wide recheck. `model-mirror status` reports
290
+ desired and observed torrent state. A minimal systemd service uses the same
291
+ configured archive:
292
+
293
+ ```ini
294
+ [Service]
295
+ ExecStart=/path/to/model-mirror --config /path/to/config.yaml torrent serve
296
+ Restart=always
297
+ ```
298
+
299
+ The emitted `.torrent` and magnet are not tied to the managed backend. To use a
300
+ preferred client, either add the printed torrent with its printed data location
301
+ for seeding, or record external ownership explicitly:
302
+
303
+ ```bash
304
+ model-mirror torrent publish --external org/model
305
+ ```
306
+
307
+ Model-mirror does not control or infer the runtime health of an external
308
+ client. Stop that client before modifying payload. `torrent stop` stops managed
309
+ intent (or records that an external seed was stopped) but deliberately retains
310
+ the update fence. Only explicit retirement releases it:
311
+
312
+ ```bash
313
+ model-mirror torrent stop org/model
314
+ model-mirror torrent retire org/model
315
+ model-mirror repair --update org/model
316
+ ```
317
+
318
+ Same-commit `repair` enters maintenance, waits for the managed backend to
319
+ detach, repairs only recorded paths, refreshes their coverage, and resumes the
320
+ same publication when verification succeeds. `repair --update` stays blocked
321
+ until retirement.
322
+
323
+ On another host, model-mirror can download natively or provide an exact
324
+ standard-client handoff:
325
+
326
+ ```bash
327
+ model-mirror torrent join /path/model@commit.torrent
328
+ model-mirror torrent join 'magnet:?xt=...' --seed
329
+
330
+ model-mirror torrent handoff /path/model@commit.torrent
331
+ # Download with any standard client to the printed data location, then run:
332
+ model-mirror torrent import /path/model@commit.torrent /printed/path/model
333
+ ```
334
+
335
+ Import validates hostile paths, the commit-scoped descriptor, sizes, and
336
+ content before atomically moving the staged payload into the normal archive
337
+ layout. Native join reuses libtorrent's verified piece state and reconstructs
338
+ coverage without rereading the payload. External-client import independently
339
+ reads and hashes the downloaded files because model-mirror cannot assume that
340
+ client's runtime state.
341
+
342
+ A trusted torrent proves consistency with the supplied infohash; by itself it
343
+ does not prove that the publisher's bytes were authentic Hugging Face content.
344
+ Imported state therefore records `torrent-verified`,
345
+ `trusted-infohash`, and `not-upstream-verified` separately. It remains usable
346
+ when upstream has disappeared and can later be checked against upstream if it
347
+ becomes available.
348
+
349
+ ## Periodic Jobs
350
+
351
+ For alert-only checks, run verification periodically and let its non-zero exit
352
+ status trigger normal alerting:
353
+
354
+ ```bash
355
+ model-mirror verify --all --max-age 30d
356
+ ```
357
+
358
+ For a repair pass after verification:
359
+
360
+ ```bash
361
+ model-mirror verify --all --max-age 30d || true
362
+ model-mirror repair --all
363
+ ```
364
+
365
+ Do not chain verification to repair with `&&`: `verify` exits non-zero when it
366
+ finds repairable damage. `verify --all` skips recently verified clean mirrors
367
+ when `--max-age` is set. Busy mirrors are reported and skipped, the rest of the
368
+ archive is still checked, and the final exit status is non-zero if any mirror is
369
+ dirty, failed, or busy.
370
+
371
+ ## Upstream Updates
372
+
373
+ Every online operation resolves the requested revision, usually `main`, to a
374
+ specific Hub commit. The local mirror is tied to that commit.
375
+
376
+ If `verify` sees that upstream `main` now points at a different commit, it marks
377
+ the mirror with `upstream_status: changed` but does not overwrite local files.
378
+
379
+ Repair and update are deliberately separate operations:
380
+
381
+ - `repair` restores missing or damaged files from the mirror's currently
382
+ recorded commit. It never moves the mirror to a different commit, even when
383
+ `verify` has detected an upstream change.
384
+ - `repair --update` explicitly moves the mirror to the changed upstream commit
385
+ recorded by `verify`.
386
+
387
+ Updating one or all changed mirrors is therefore explicit:
388
+
389
+ ```bash
390
+ model-mirror repair --update org/model
391
+ model-mirror repair --all --update
392
+ ```
393
+
394
+ Use `model-mirror mirror --commit abc123 org/model` when you want a reproducible
395
+ archive pinned to an exact Hub revision.
396
+
397
+ ## Common Commands
398
+
399
+ ```bash
400
+ model-mirror mirror org/model # download and verify
401
+ model-mirror mirror --no-verify org/model # download without final verification
402
+ model-mirror verify org/model # full verification
403
+ model-mirror verify --cached org/model # use current .manifest hashes
404
+ model-mirror repair org/model # redownload paths from .verification
405
+ model-mirror repair --all # repair all mirrors with recorded repair paths
406
+ model-mirror repair --update org/model # apply a changed upstream commit recorded by verify
407
+ model-mirror offline org/model # local verification only; no Hub checks
408
+ model-mirror online org/model # re-enable Hub checks
409
+ model-mirror list # fast recorded state for every mirror
410
+ model-mirror status # metadata-only archive status
411
+ model-mirror status org/model # concise last-known repository state
412
+ model-mirror status --verbose org/model # full recorded metadata
413
+ model-mirror status --check-upstream org/model # advisory live upstream comparison
414
+ model-mirror status --json # stable machine-readable status
415
+ model-mirror remove org/model # inspect, confirm, and permanently remove one mirror
416
+ model-mirror upgrade org/model # fill missing torrent hash coverage
417
+ model-mirror torrent publish org/model # publish and request durable seeding
418
+ model-mirror torrent join FILE_OR_MAGNET # recover a normal local archive
419
+ ```
420
+
421
+ Datasets and Spaces are supported with `--repo-type dataset` or
422
+ `--repo-type space`.
423
+
424
+ ## Key Configuration
425
+
426
+ ```bash
427
+ model-mirror config show
428
+ model-mirror config options
429
+ model-mirror config directory /mnt/big-drive/huggingface
430
+ model-mirror config set checksum-workers 1
431
+ model-mirror config set hf-xet-reconstruct-write-sequentially true
432
+ model-mirror config set hf-xet-num-concurrent-range-gets 1
433
+ ```
434
+
435
+ `config set` accepts either kebab-case or snake_case keys; the saved YAML and
436
+ `config options` use canonical snake_case. `config directory PATH` is a
437
+ convenience form for the most common setting and creates `PATH` immediately.
438
+ `config set directory PATH` saves the same value without creating the directory
439
+ until it is needed.
440
+
441
+ Important configuration options:
442
+
443
+ - `directory`: archive root
444
+ - `repo_type`: default repo type, usually `model`
445
+ - `revision`: default branch, tag, or commit, usually `main`
446
+ - `checksum`: whether mirror/repair writes local hash manifest records
447
+ - `checksum_workers`: checksum hashing concurrency; `1` is HDD-friendly
448
+ - `verify_after_mirror`: run verification after `mirror`
449
+ - `token_path`: Hugging Face token file path; optional when autodetection finds
450
+ `HF_TOKEN_PATH`, `HF_HOME/token`, `~/.cache/huggingface/token`, or
451
+ `~/.huggingface/token`. If no token is found during Hub access,
452
+ model-mirror warns and prints the config command to set this path. Token
453
+ contents are never printed.
454
+ - `cache_dir`: optional override for the Hugging Face/Xet cache; defaults to
455
+ `DIRECTORY/.model-mirror/cache`
456
+ - `tmp_dir`: optional override for staging and temporary files; defaults to
457
+ `DIRECTORY/.model-mirror/tmp`
458
+ - `hf_xet_reconstruct_write_sequentially`: HDD-friendly Xet reconstruction
459
+ writes; uses the current `HF_XET_RECONSTRUCTION_USE_VECTORED_WRITE=false`
460
+ knob when supported
461
+ - `hf_xet_num_concurrent_range_gets`: Xet internal download concurrency.
462
+ Default `1` is HDD-friendly; increase for SSD/NVMe.
463
+ - `hf_xet_high_performance`: enable Xet high-performance mode. This is off by
464
+ default; use only on high-bandwidth machines with fast disks and ample memory,
465
+ typically 64 GB RAM or more.
466
+
467
+ ## Disk Space During Transfers
468
+
469
+ Model-mirror does not stage a second complete snapshot. Each file is streamed
470
+ through the HTTP or Xet transport into a sibling `NAME.incomplete` file while
471
+ its integrity hashes are accumulated, then atomically renamed to `NAME`. The
472
+ partial file is the eventual payload allocation rather than an additional full
473
+ copy. With the default `download_workers: 1`, only one payload file is
474
+ incomplete at a time. Same-commit repair removes a damaged file before
475
+ redownloading it, so it likewise does not retain both the damaged and repaired
476
+ copy.
477
+
478
+ Allow space for the final snapshot, active partial files, and auxiliary
479
+ transport cache under `DIRECTORY/.model-mirror/tmp/downloads`. The auxiliary
480
+ cache is not specified as a fixed percentage and can vary with the HTTP/Xet
481
+ implementation, but model-mirror does not intentionally reconstruct a second
482
+ full payload there. Interrupted partial files and staging cache are retained
483
+ for resume.
484
+ `model-mirror clean-cache` scans and previews reclaimable cache space before
485
+ `clean-cache --force` removes it without deleting mirrored payloads. It also
486
+ detects legacy archive-root `.cache/` and `.tmp/` directories left by older
487
+ versions. The `.model-mirror/` control directory itself remains in place. The
488
+ metadata-only `status` command reports active work when its progress heartbeat
489
+ is available; do not run forced cache cleanup while it shows an active download
490
+ or repair.
491
+
492
+ ## Locking And Interrupted Commands
493
+
494
+ Repository operations use an advisory kernel `flock` on
495
+ `.verification.lock`. `mirror`, `remove`, `verify`, `repair`, `offline`, `online`,
496
+ `upgrade`, and torrent control-plane transitions take this per-repository lock;
497
+ the managed seeder holds it only during short reconciliation transitions.
498
+ `list` and `status` remain non-blocking and report the command, PID, host, and
499
+ start time for a lock that is actually held.
500
+
501
+ The kernel releases the lock automatically when a process exits or the host
502
+ restarts. The `.verification.lock` file may remain after a crash, but its
503
+ presence alone does not mean the repository is locked; model-mirror probes the
504
+ lock itself and ignores stale contents. Do not delete the file to override a
505
+ genuinely running operation. After a killed download, rerun the same `mirror`
506
+ command to resume its pinned snapshot.
507
+
508
+ ## Removing A Mirror
509
+
510
+ `remove` prints the repository identity, type, path, status, exceptional state,
511
+ resolved commit, last verification time and age, payload file count, and exact
512
+ size. By default it requires typing the full repository ID; `--yes` is available
513
+ for deliberate automation:
514
+
515
+ ```bash
516
+ model-mirror remove org/model
517
+ model-mirror remove --repo-type dataset org/data
518
+ model-mirror remove --yes org/model
519
+ ```
520
+
521
+ Removal is blocked while a torrent publication fence is active. Stop any
522
+ external client first when applicable, then release the local publication:
523
+
524
+ ```bash
525
+ model-mirror torrent stop org/model
526
+ model-mirror torrent retire org/model
527
+ model-mirror remove org/model
528
+ ```
529
+
530
+ After confirmation, model-mirror atomically moves the mirror from its canonical
531
+ path into `.model-mirror-removals/` under the same archive. It removes payload
532
+ files first, ordinary metadata next, and the preserved removal record and lock
533
+ last. If the command or host stops partway through, rerun the same `remove`
534
+ command to resume. A new mirror created at the original path while an older
535
+ removal is pending is never deleted by that resumed cleanup.
536
+
537
+ Torrent retirement releases local managed seed intent and the update fence, but
538
+ cannot revoke torrent metadata already shared with other peers. Removing a
539
+ repository does not remove archive-wide cache; inspect that separately with
540
+ `model-mirror clean-cache`.
541
+
542
+ ## Notes
543
+
544
+ Model-mirror uses Hugging Face metadata and the HTTP/Xet transports, selecting
545
+ Xet automatically when available. It keeps Hugging Face cache and temporary
546
+ directories under the configured archive root so large downloads do not spill
547
+ into the default home cache.
548
+
549
+ See [CONTRIBUTORS.md](CONTRIBUTORS.md) for implementation details, testing, and
550
+ future design notes.
@@ -0,0 +1,24 @@
1
+ model_mirror/__init__.py,sha256=dWnOBXDC-F5PQg3w9HtgEUK6tiI2tzlhLZ_71IW5fOY,165
2
+ model_mirror/audit.py,sha256=jmQZJSj3UYniFjSDe_rV9s8XIl8GoTb1893S-JZ0Oco,7888
3
+ model_mirror/checksums.py,sha256=RE0hPjcsXmPg8sxCW-hmpiZ1AJFNdAU7LeCxqSHinJw,11981
4
+ model_mirror/cli.py,sha256=_ztg97l_RYF47o1ToCfQmW7U8n9O2c_1ULGGIuTHdtc,95786
5
+ model_mirror/config.py,sha256=805bDaVtzsRltrL5bjUCzAE-U00HE6Fc_ZxiiwQar1k,11827
6
+ model_mirror/hub.py,sha256=lWcxP3x0nF5u21hniAtUbuzJwOFbYTjK1-ffbAkj520,30697
7
+ model_mirror/lock.py,sha256=lOS4y_jM9GAtBd7CZJo4mpO6b8d1S640RARYHuilo_o,3485
8
+ model_mirror/mirror.py,sha256=v1ocF6Z28n3Oy9zuGMS3flxwNQcpT5rzOZOf1AMt2T8,6933
9
+ model_mirror/progress.py,sha256=Dozscx_YKZqFN3ELqdo_WTzBOr_upU8FnrCwhNTvV5Y,8420
10
+ model_mirror/removal.py,sha256=rR-S3YVGvhyS_v8Q6VnfQ4cWEVFywSzOdMTw4d-2enE,5230
11
+ model_mirror/repair.py,sha256=96TgXqMfSUGouDFkzNAQ5iHH1mUpE3TBjaXiTkClFQs,12408
12
+ model_mirror/state.py,sha256=v771JrO8VjWgVRh7ztolW47vWtKgbeSVNe-8PNvCHG8,5535
13
+ model_mirror/torrent.py,sha256=gNldH3Ui5nrBq_O0MtxU5QaPrjj57akISssayOiK5cs,16579
14
+ model_mirror/torrent_coverage.py,sha256=CobMsF-ku0FKNV4ppuph-55MjBC6KXCB6DFTfPHnZRA,14928
15
+ model_mirror/torrent_hashes.py,sha256=qb62iU8r7wn6zk3AsoAOxib6Y7vzuBfeUUOeuY5vMbI,6227
16
+ model_mirror/torrent_import.py,sha256=4HWOkg_I6uB4OvK-VtbayPgMZ7xLRSwcqr8tex4e67c,25170
17
+ model_mirror/torrent_publication.py,sha256=yeKaaAWXZsKWr4LQuYm_53smOobeufapWp5W79ijPF4,22853
18
+ model_mirror/torrent_seed.py,sha256=uWNfby7isay5VmWq-R3a3a72m15yzKmPU7AMZLMCCoo,8398
19
+ model_mirror/verify.py,sha256=VJgcXcRRSQe_YVnnPYx7KpMKAHjJXHyTvBCfebpKmRY,4477
20
+ model_mirror_cli-0.2.0.dist-info/METADATA,sha256=PrAOzOAgqAEaNQCeDwfNDSoW0KiFSU0CIp12N013YT0,23967
21
+ model_mirror_cli-0.2.0.dist-info/WHEEL,sha256=lCkmxWfQsSc9CfIClYeavTdQeEX2toPqufh9gI35EQA,87
22
+ model_mirror_cli-0.2.0.dist-info/entry_points.txt,sha256=3TZd_iSp38e0qWULj0iKM39LqeP_N2vb2zRqsOTKFNE,55
23
+ model_mirror_cli-0.2.0.dist-info/licenses/LICENSE,sha256=xXLebneXQDu3LlHRaz2kR5WeEFqvomN4wEZcjDOuHIY,1067
24
+ model_mirror_cli-0.2.0.dist-info/RECORD,,
@@ -0,0 +1,4 @@
1
+ Wheel-Version: 1.0
2
+ Generator: hatchling 1.31.0
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ model-mirror = model_mirror.cli:main
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 xlr8harder
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.