letify 2.8.6__tar.gz → 2.9.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- letify-2.9.0/.github/workflows/publish-crates.yml +89 -0
- letify-2.9.0/.github/workflows/reserve-crates.yml +77 -0
- {letify-2.8.6 → letify-2.9.0}/.gitignore +2 -1
- {letify-2.8.6 → letify-2.9.0}/CLAUDE.md +2 -2
- {letify-2.8.6 → letify-2.9.0}/PKG-INFO +6 -9
- {letify-2.8.6 → letify-2.9.0}/PROJECT.md +25 -1
- {letify-2.8.6 → letify-2.9.0}/README.md +5 -8
- {letify-2.8.6 → letify-2.9.0}/docs/COMPONENT.md +4 -4
- {letify-2.8.6 → letify-2.9.0}/docs/NETWORK.md +1 -21
- {letify-2.8.6 → letify-2.9.0}/docs/SPEC.md +73 -12
- {letify-2.8.6 → letify-2.9.0}/docs/guide/03-execution-modes.md +1 -1
- {letify-2.8.6 → letify-2.9.0}/docs/guide/05-concurrency.md +47 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/06-cost.md +1 -1
- {letify-2.8.6 → letify-2.9.0}/docs/guide/07-troubleshooting.md +1 -1
- {letify-2.8.6 → letify-2.9.0}/docs/guide/08-releasing.md +13 -2
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/03-/354/213/244/355/226/211-/353/260/251/354/213/235.md +1 -1
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/05-/353/217/231/354/213/234-/354/213/244/355/226/211.md +47 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/06-/353/271/204/354/232/251.md +1 -1
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/07-/353/254/270/354/240/234-/355/225/264/352/262/260.md +1 -1
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/08-/353/246/264/353/246/254/354/212/244.md +13 -2
- {letify-2.8.6 → letify-2.9.0}/docs/locales/README_ko.md +4 -7
- {letify-2.8.6 → letify-2.9.0}/letify/__init__.py +22 -1
- {letify-2.8.6 → letify-2.9.0}/letify/cli.py +197 -37
- {letify-2.8.6 → letify-2.9.0}/letify/config/__init__.py +2 -0
- {letify-2.8.6 → letify-2.9.0}/letify/config/schema.py +3 -0
- letify-2.9.0/letify/declare/exit.py +50 -0
- {letify-2.8.6 → letify-2.9.0}/letify/errors.py +23 -0
- {letify-2.8.6 → letify-2.9.0}/letify/launcher.py +1 -0
- {letify-2.8.6 → letify-2.9.0}/letify/paths.py +1 -0
- {letify-2.8.6 → letify-2.9.0}/letify/protocol/codec.py +10 -1
- {letify-2.8.6 → letify-2.9.0}/letify/protocol/wire.py +27 -0
- {letify-2.8.6 → letify-2.9.0}/letify/protocol/worker.py +697 -13
- {letify-2.8.6 → letify-2.9.0}/letify/providers/base.py +20 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/elice.py +5 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/kaggle.py +10 -2
- {letify-2.8.6 → letify-2.9.0}/letify/providers/local.py +13 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/modal.py +6 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/shell.py +39 -15
- letify-2.9.0/letify/remoting/__init__.py +22 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/channel.py +226 -2
- letify-2.9.0/letify/runtime/control.py +536 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/lease.py +10 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/pool.py +28 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/session.py +93 -12
- letify-2.9.0/letify/runtime/takeover.py +173 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/link.py +3 -2
- {letify-2.8.6 → letify-2.9.0}/letify/transport/sshopts.py +30 -0
- {letify-2.8.6 → letify-2.9.0}/letify-core/Cargo.lock +3 -28
- letify-2.9.0/letify-core/Cargo.toml +14 -0
- letify-2.9.0/letify-core/build.py +75 -0
- letify-2.9.0/letify-core/letify/Cargo.toml +14 -0
- letify-2.9.0/letify-core/letify/README.md +21 -0
- letify-2.9.0/letify-core/letify/src/lib.rs +21 -0
- {letify-2.8.6 → letify-2.9.0}/letify-core/letify-quic/Cargo.toml +1 -0
- letify-2.9.0/letify-ext/LICENSE +201 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/package-lock.json +2 -2
- {letify-2.8.6 → letify-2.9.0}/letify-ext/package.json +1 -1
- {letify-2.8.6 → letify-2.9.0}/pyproject.toml +2 -2
- {letify-2.8.6 → letify-2.9.0}/scripts/version.py +1 -1
- {letify-2.8.6 → letify-2.9.0}/tests/test_cli.py +7 -17
- {letify-2.8.6 → letify-2.9.0}/tests/test_cli_output.py +0 -14
- {letify-2.8.6 → letify-2.9.0}/tests/test_paths.py +1 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_providers.py +56 -1
- letify-2.9.0/tests/test_remoting.py +37 -0
- letify-2.9.0/tests/test_run_control.py +716 -0
- letify-2.9.0/tests/test_run_takeover.py +621 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_version.py +25 -0
- letify-2.8.6/letify/remoting/__init__.py +0 -28
- letify-2.8.6/letify/remoting/capability.py +0 -50
- letify-2.8.6/letify/remoting/loader.py +0 -96
- letify-2.8.6/letify/remoting/probe.py +0 -145
- letify-2.8.6/letify-core/Cargo.toml +0 -18
- letify-2.8.6/letify-core/build.py +0 -109
- letify-2.8.6/letify-core/letify-agent/Cargo.toml +0 -13
- letify-2.8.6/letify-core/letify-agent/src/driver.rs +0 -352
- letify-2.8.6/letify-core/letify-agent/src/main.rs +0 -317
- letify-2.8.6/letify-core/letify-driver/Cargo.toml +0 -18
- letify-2.8.6/letify-core/letify-driver/src/client.rs +0 -271
- letify-2.8.6/letify-core/letify-driver/src/exports.rs +0 -464
- letify-2.8.6/letify-core/letify-driver/src/image.rs +0 -106
- letify-2.8.6/letify-core/letify-driver/src/lib.rs +0 -90
- letify-2.8.6/letify-core/letify-driver/src/status.rs +0 -61
- letify-2.8.6/letify-core/letify-driver/src/table.rs +0 -206
- letify-2.8.6/letify-core/letify-wire/Cargo.toml +0 -9
- letify-2.8.6/letify-core/letify-wire/src/lib.rs +0 -957
- letify-2.8.6/letify-core/letify-wire/tests/throughput.rs +0 -133
- letify-2.8.6/tests/test_remoting.py +0 -340
- {letify-2.8.6 → letify-2.9.0}/.github/workflows/ci.yml +0 -0
- {letify-2.8.6 → letify-2.9.0}/.github/workflows/docs.yml +0 -0
- {letify-2.8.6 → letify-2.9.0}/.github/workflows/publish.yml +0 -0
- {letify-2.8.6 → letify-2.9.0}/.researchtree +0 -0
- {letify-2.8.6 → letify-2.9.0}/LICENSE +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/INTENT.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/01-getting-started.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/02-providers.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/04-environments-and-data.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/README.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/01-/354/213/234/354/236/221/355/225/230/352/270/260.md" +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/02-/355/224/204/353/241/234/353/260/224/354/235/264/353/215/224.md" +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/04-/355/231/230/352/262/275/352/263/274-/353/215/260/354/235/264/355/204/260.md" +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/guide/ko/README.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/docs/mkdocs.yml +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/README.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/.letify.example +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/00_smoke.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/01_sweep.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/02_resume.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/03_watch.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/README.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/README_ko.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/common.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/examples/nvfp4-lora/recipe.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/config/inventory.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/config/login.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/config/secrets.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/config/writer.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/cache.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/env.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/function.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/instance.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/reduce.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/declare/replicas.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/install.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/protocol/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/protocol/driver.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/protocol/handle.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/colab.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/colab_files.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/colab_keepalive.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/kaggle_adapter.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/modal_adapter.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/naming.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/tunnel.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/providers/usage.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/client.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/cuda.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/executor.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/frames.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/guard.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/tensor.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/trace.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/remoting/device/value.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/render.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/bootstrap.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/broker.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/runtime/telemetry.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/backends/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/backends/drive.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/backends/filesystem.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/backends/google_auth.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/backends/layout.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/backends/objects.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/cas.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/handoff.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/pathdata.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/sendorder.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/store/volume.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/stubs.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/tools.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/__init__.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/agent.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/announce.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/nat.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/pipeline.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/portmap.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/probe.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/quic.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/rendezvous.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/setup.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify/transport/strategies.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-core/.gitignore +0 -0
- {letify-2.8.6/letify-ext → letify-2.9.0/letify-core/letify}/LICENSE +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-core/letify-quic/src/main.rs +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-core/rust-toolchain.toml +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/.vscodeignore +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/README.md +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/media/icon.svg +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/src/cli.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/src/extension.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/src/history.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/src/model.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/src/poll.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/src/render.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/test/cli.test.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/test/manifest.test.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/test/model.test.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/test/regressions.test.ts +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify-ext/tsconfig.json +0 -0
- {letify-2.8.6 → letify-2.9.0}/letify_providers.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/scripts/sync-main.sh +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/conftest.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/fake_kaggle_adapter.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/fake_modal_adapter.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_colab_keepalive.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_config.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_core.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_data.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_data_streaming.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_declare.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_device.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_distributed.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_drive_handoff.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_drive_store.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_editor_sessions.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_elice.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_frames.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_install.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_kaggle.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_kaggle_cookie.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_kaggle_link.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_link_log.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_notebook.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_parallel_placement.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_portmap.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_protocol.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_quic.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_render.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_runtime.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_ship.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_store.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_strategies.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_stubs.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_transport.py +0 -0
- {letify-2.8.6 → letify-2.9.0}/tests/test_tunnel_setup.py +0 -0
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# Publish the crate letify to crates.io when a version tag is pushed.
|
|
2
|
+
#
|
|
3
|
+
# This runs beside publish.yml on the same tag v<version>. Order inside this file:
|
|
4
|
+
# 1. verify: the tag agrees with every version file, the tagged commit is on develop, the
|
|
5
|
+
# workspace builds and its tests pass, and `cargo publish --dry-run` packages and builds
|
|
6
|
+
# the crate alone.
|
|
7
|
+
# 2. publish-crate: the upload, only after verify succeeds.
|
|
8
|
+
# The upload is last because crates.io never replaces a version. A published number can be
|
|
9
|
+
# yanked but is taken forever, so every check that can fail runs before it.
|
|
10
|
+
#
|
|
11
|
+
# GitHub Release creation stays in publish.yml. Other crates in letify-core set publish = false.
|
|
12
|
+
#
|
|
13
|
+
# One-time setup: add the crates.io API token as the secret CARGO_REGISTRY_TOKEN in the GitHub
|
|
14
|
+
# environment crates-io. See docs/guide/08-releasing.md.
|
|
15
|
+
name: publish-crates
|
|
16
|
+
|
|
17
|
+
on:
|
|
18
|
+
push:
|
|
19
|
+
tags: ["v*"]
|
|
20
|
+
|
|
21
|
+
permissions:
|
|
22
|
+
contents: read
|
|
23
|
+
|
|
24
|
+
concurrency:
|
|
25
|
+
group: publish-crates-${{ github.ref }}
|
|
26
|
+
cancel-in-progress: false
|
|
27
|
+
|
|
28
|
+
jobs:
|
|
29
|
+
verify:
|
|
30
|
+
runs-on: ubuntu-latest
|
|
31
|
+
steps:
|
|
32
|
+
- uses: actions/checkout@v4
|
|
33
|
+
with:
|
|
34
|
+
fetch-depth: 0
|
|
35
|
+
- uses: astral-sh/setup-uv@v6
|
|
36
|
+
- name: Install Rust
|
|
37
|
+
# rustup reads letify-core/rust-toolchain.toml when cargo runs there.
|
|
38
|
+
run: |
|
|
39
|
+
rustup toolchain install stable --profile minimal
|
|
40
|
+
rustup default stable
|
|
41
|
+
- name: Check the tag and every file carry one version
|
|
42
|
+
run: uv run --no-project python scripts/version.py check --tag "$GITHUB_REF_NAME"
|
|
43
|
+
- name: Check the tagged commit is on develop
|
|
44
|
+
# letify tags develop, not main, so develop is the branch a release must come from.
|
|
45
|
+
run: |
|
|
46
|
+
git fetch origin develop
|
|
47
|
+
if ! git merge-base --is-ancestor "$GITHUB_SHA" origin/develop; then
|
|
48
|
+
echo "Tagged commit $GITHUB_SHA is not an ancestor of origin/develop."
|
|
49
|
+
echo "Merge the release commit into develop before tagging."
|
|
50
|
+
exit 1
|
|
51
|
+
fi
|
|
52
|
+
- name: Build the workspace
|
|
53
|
+
# All five crates are pure Rust with no system packages needed on ubuntu-latest.
|
|
54
|
+
working-directory: letify-core
|
|
55
|
+
run: cargo build --workspace --locked
|
|
56
|
+
- name: Test the workspace
|
|
57
|
+
working-directory: letify-core
|
|
58
|
+
run: cargo test --workspace --locked
|
|
59
|
+
- name: Package and build the crate without uploading
|
|
60
|
+
working-directory: letify-core
|
|
61
|
+
run: cargo publish -p letify --dry-run --locked
|
|
62
|
+
|
|
63
|
+
publish-crate:
|
|
64
|
+
needs: verify
|
|
65
|
+
runs-on: ubuntu-latest
|
|
66
|
+
environment: crates-io
|
|
67
|
+
steps:
|
|
68
|
+
- uses: actions/checkout@v4
|
|
69
|
+
- name: Install Rust
|
|
70
|
+
run: |
|
|
71
|
+
rustup toolchain install stable --profile minimal
|
|
72
|
+
rustup default stable
|
|
73
|
+
- name: Publish letify to crates.io
|
|
74
|
+
working-directory: letify-core
|
|
75
|
+
env:
|
|
76
|
+
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
|
77
|
+
run: |
|
|
78
|
+
version="${GITHUB_REF_NAME#v}"
|
|
79
|
+
status=$(curl -s -o /dev/null -w '%{http_code}' -A "letify-release (https://github.com/DarkPyonix/letify)" "https://crates.io/api/v1/crates/letify/${version}")
|
|
80
|
+
if [ "$status" = "200" ]; then
|
|
81
|
+
echo "letify ${version} is already on crates.io, nothing to publish."
|
|
82
|
+
exit 0
|
|
83
|
+
fi
|
|
84
|
+
if [ -z "$CARGO_REGISTRY_TOKEN" ]; then
|
|
85
|
+
echo "CARGO_REGISTRY_TOKEN is empty."
|
|
86
|
+
echo "Add a crates.io API token as the secret CARGO_REGISTRY_TOKEN in the crates-io environment."
|
|
87
|
+
exit 1
|
|
88
|
+
fi
|
|
89
|
+
cargo publish -p letify --locked
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Reserve a letify-* crate name on crates.io before its first real release.
|
|
2
|
+
#
|
|
3
|
+
# Pushing the tag reserve/<crate>, for example reserve/letify-link, publishes version 0.0.0 of
|
|
4
|
+
# that crate: a library with no code whose README says which letify release ships it. The real
|
|
5
|
+
# crate later publishes at letify's own version, as every crate in letify-core does, so 0.0.0
|
|
6
|
+
# stays below every real release.
|
|
7
|
+
#
|
|
8
|
+
# Only names starting with letify- are accepted, and a name crates.io already has is skipped,
|
|
9
|
+
# so pushing the tag again changes nothing. The crate letify itself is not reserved here: it
|
|
10
|
+
# publishes at the release version from publish-crates.yml.
|
|
11
|
+
#
|
|
12
|
+
# The upload runs in the GitHub environment crates-io with the secret CARGO_REGISTRY_TOKEN, as
|
|
13
|
+
# publish-crates.yml does.
|
|
14
|
+
name: reserve-crates
|
|
15
|
+
|
|
16
|
+
on:
|
|
17
|
+
push:
|
|
18
|
+
tags: ["reserve/*"]
|
|
19
|
+
|
|
20
|
+
permissions:
|
|
21
|
+
contents: read
|
|
22
|
+
|
|
23
|
+
concurrency:
|
|
24
|
+
group: reserve-crates-${{ github.ref }}
|
|
25
|
+
cancel-in-progress: false
|
|
26
|
+
|
|
27
|
+
jobs:
|
|
28
|
+
reserve:
|
|
29
|
+
runs-on: ubuntu-latest
|
|
30
|
+
environment: crates-io
|
|
31
|
+
steps:
|
|
32
|
+
- name: Install Rust
|
|
33
|
+
run: |
|
|
34
|
+
rustup toolchain install stable --profile minimal
|
|
35
|
+
rustup default stable
|
|
36
|
+
- name: Publish 0.0.0 unless the name is taken
|
|
37
|
+
env:
|
|
38
|
+
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
|
|
39
|
+
CRATE: ${{ github.ref_name }}
|
|
40
|
+
run: |
|
|
41
|
+
set -euo pipefail
|
|
42
|
+
name="${CRATE#reserve/}"
|
|
43
|
+
if [[ ! "$name" =~ ^letify-[a-z0-9]+(-[a-z0-9]+)*$ ]]; then
|
|
44
|
+
echo "reserve/$name: only names of the form letify-<word> are reserved here" >&2
|
|
45
|
+
exit 1
|
|
46
|
+
fi
|
|
47
|
+
code="$(curl -s -o /dev/null -w '%{http_code}' -A 'letify-reserve (github.com/DarkPyonix/letify)' "https://crates.io/api/v1/crates/$name")"
|
|
48
|
+
if [[ "$code" == "200" ]]; then
|
|
49
|
+
echo "$name is already on crates.io; nothing to reserve"
|
|
50
|
+
exit 0
|
|
51
|
+
fi
|
|
52
|
+
if [[ -z "$CARGO_REGISTRY_TOKEN" ]]; then
|
|
53
|
+
echo "CARGO_REGISTRY_TOKEN is empty. Add it to the crates-io environment." >&2
|
|
54
|
+
exit 1
|
|
55
|
+
fi
|
|
56
|
+
dir="$RUNNER_TEMP/$name"
|
|
57
|
+
mkdir -p "$dir/src"
|
|
58
|
+
cat > "$dir/Cargo.toml" <<TOML
|
|
59
|
+
[package]
|
|
60
|
+
name = "$name"
|
|
61
|
+
version = "0.0.0"
|
|
62
|
+
edition = "2024"
|
|
63
|
+
license = "MIT"
|
|
64
|
+
description = "Reserved for the letify project. The first release ships with letify 3.0."
|
|
65
|
+
repository = "https://github.com/DarkPyonix/letify"
|
|
66
|
+
readme = "README.md"
|
|
67
|
+
TOML
|
|
68
|
+
cat > "$dir/README.md" <<MD
|
|
69
|
+
# $name
|
|
70
|
+
|
|
71
|
+
This name is reserved for the letify project, https://github.com/DarkPyonix/letify.
|
|
72
|
+
Version 0.0.0 holds no code. The first release ships with letify 3.0 and carries
|
|
73
|
+
letify's version, as every letify crate does.
|
|
74
|
+
MD
|
|
75
|
+
echo '#![doc = include_str!("../README.md")]' > "$dir/src/lib.rs"
|
|
76
|
+
cd "$dir"
|
|
77
|
+
cargo publish --allow-dirty
|
|
@@ -253,10 +253,11 @@ __marimo__/
|
|
|
253
253
|
.claude
|
|
254
254
|
|
|
255
255
|
# letify
|
|
256
|
+
.letify-home/
|
|
256
257
|
.letify/
|
|
257
258
|
typings/
|
|
258
259
|
|
|
259
|
-
# letify-
|
|
260
|
+
# The letify-quic binary, copied here by letify-core/build.py
|
|
260
261
|
letify/remoting/lib/
|
|
261
262
|
|
|
262
263
|
# Artifacts the examples write where they run
|
|
@@ -114,7 +114,7 @@ This repository uses [ResearchTree](https://darkpyonix.github.io/researchtree/)
|
|
|
114
114
|
|
|
115
115
|
## Code layout
|
|
116
116
|
|
|
117
|
-
The Python package is pure Python and links no Python extension. The package lives in `letify/` at the repository root. letify-core, the Rust workspace in `letify-core/`, is built in CI and
|
|
117
|
+
The Python package is pure Python and links no Python extension. The package lives in `letify/` at the repository root. letify-core, the Rust workspace in `letify-core/`, is built in CI, and its `letify-quic` binary is shipped in `letify/remoting/lib/` inside platform wheels.
|
|
118
118
|
|
|
119
119
|
```
|
|
120
120
|
letify/
|
|
@@ -129,7 +129,7 @@ letify/
|
|
|
129
129
|
runtime/ Channel, session, pool, lease, bootstrap and telemetry
|
|
130
130
|
store/ Content addressed blob store, volumes and backends
|
|
131
131
|
providers/ Provider base class and one module per provider
|
|
132
|
-
remoting/
|
|
132
|
+
remoting/ PyTorch operator forwarding for host="local" (remoting/device/) and the efficiency arithmetic; also holds the letify-quic binary in lib/
|
|
133
133
|
_vendor/ Vendored third party code
|
|
134
134
|
letify-ext/ VS Code extension (TypeScript) showing quota and GPU activity from the CLI's --json output
|
|
135
135
|
```
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: letify
|
|
3
|
-
Version: 2.
|
|
3
|
+
Version: 2.9.0
|
|
4
4
|
Summary: Declarations that become infrastructure.
|
|
5
5
|
Project-URL: Homepage, https://github.com/DarkPyonix/letify
|
|
6
6
|
Project-URL: Documentation, https://github.com/DarkPyonix/letify/tree/main/docs
|
|
@@ -171,7 +171,7 @@ uv add letify
|
|
|
171
171
|
|
|
172
172
|
That is the whole install, for every provider. letify installs only cloudpickle and blake3. It needs [uv](https://docs.astral.sh/uv/) on the machine, because provider tools such as the Colab CLI and the Modal client run through uv in their own environments, not in your `.venv`.
|
|
173
173
|
|
|
174
|
-
The wheel carries [letify-core](letify-core/), the Rust
|
|
174
|
+
The wheel carries the `letify-quic` binary built from [letify-core](letify-core/), the Rust workspace. Wheels are built for Linux x86_64 and aarch64, Windows x86_64 and arm64, and macOS arm64 and x86_64. On another platform, pip falls back to the source distribution, which has no `letify-quic`, so the QUIC connection strategy is skipped.
|
|
175
175
|
|
|
176
176
|
---
|
|
177
177
|
|
|
@@ -259,9 +259,9 @@ Two words, and neither of them is a mechanism.
|
|
|
259
259
|
|
|
260
260
|
**`device`** carries the provider, the account and the accelerator in one value, because those are one decision. Core count and memory come with the shape the provider registered, so there is nothing to ask for.
|
|
261
261
|
|
|
262
|
-
**`host`** is the CUDA word for the CPU side. `letify.local` keeps Python and the libraries in this process and forwards only
|
|
262
|
+
**`host`** is the CUDA word for the CPU side. `letify.local` keeps Python and the libraries in this process and forwards only PyTorch operators. `letify.remote` ships the function to the machine that holds the device. The strings `"local"` and `"remote"` are the same values.
|
|
263
263
|
|
|
264
|
-
Leave `host` out and you get `letify.local`. When the device is far away, forwarding
|
|
264
|
+
Leave `host` out and you get `letify.local`. When the device is far away, forwarding operators is slow, so the call warns with the expected efficiency and then runs. For a remote GPU, `host=letify.remote` is usually what you want.
|
|
265
265
|
|
|
266
266
|
How long a session lives is not a declaration argument. A call ends its session. `with let.keep_alive():` keeps sessions for the length of a block, so a run of separate calls does not pay session start each time.
|
|
267
267
|
|
|
@@ -283,7 +283,7 @@ Counter-intuitive consequence: **a faster GPU makes forwarding worse**, because
|
|
|
283
283
|
|
|
284
284
|
Decoding is the case that stays bad. Throughput is bounded near `1000 / (k × RTT)` tokens per second, so the card stops mattering. Ship the whole `generate` call instead.
|
|
285
285
|
|
|
286
|
-
Measure your own `k` with `torch.cuda.set_sync_debug_mode("warn")` and your round trip with `
|
|
286
|
+
Measure your own `k` with `torch.cuda.set_sync_debug_mode("warn")` and your round trip with `ping`. See [docs/NETWORK.md](docs/NETWORK.md).
|
|
287
287
|
|
|
288
288
|
</details>
|
|
289
289
|
|
|
@@ -495,7 +495,6 @@ letify status # what is running right now
|
|
|
495
495
|
letify usage # what is left on each account
|
|
496
496
|
letify utilization # how busy each instance's GPU is
|
|
497
497
|
letify check lab # does this machine answer?
|
|
498
|
-
letify probe lab # is host=letify.local worth using here?
|
|
499
498
|
letify efficiency 0.5 3 150 # the formula, from measured terms
|
|
500
499
|
```
|
|
501
500
|
|
|
@@ -513,7 +512,7 @@ Add `--json` to `usage`, `utilization` or `status` for output a program can read
|
|
|
513
512
|
| 📐 [docs/SPEC.md](docs/SPEC.md) | The design as it stands, decision by decision |
|
|
514
513
|
| 🧩 [docs/COMPONENT.md](docs/COMPONENT.md) | Every class, and the vocabulary |
|
|
515
514
|
| 🌐 [docs/NETWORK.md](docs/NETWORK.md) | Transports, latency measurements, tunnel choices |
|
|
516
|
-
| 🦀 [letify-core/](letify-core/) | The Rust workspace
|
|
515
|
+
| 🦀 [letify-core/](letify-core/) | The Rust workspace: the `letify` crate and the `letify-quic` binary |
|
|
517
516
|
| 🧭 [docs/guide/](docs/guide/) | Task-oriented guides |
|
|
518
517
|
| 🇰🇷 [docs/locales/README_ko.md](docs/locales/README_ko.md) | 한국어 |
|
|
519
518
|
|
|
@@ -527,11 +526,9 @@ Alpha, and honest about it. What works today:
|
|
|
527
526
|
✅ Persistent sessions: a session cache keeps values between calls, large arguments travel once
|
|
528
527
|
✅ Content addressed storage, configuration and secrets
|
|
529
528
|
✅ The `Local` and `Colab` providers
|
|
530
|
-
✅ `letify-core`, verified on a real GPU: the agent opens the driver, the local driver forwards an allocation and a copy in both directions, and the bytes match
|
|
531
529
|
|
|
532
530
|
Not finished yet:
|
|
533
531
|
|
|
534
|
-
🚧 `letify-driver` covers the entry points a PyTorch process needs to start up and run one kernel. Anything else names itself and returns `CUDA_ERROR_NOT_SUPPORTED`, so a real run prints the list of what to build next
|
|
535
532
|
🚧 `Modal` and `Elice` follow each published interface but have not been run against the live services
|
|
536
533
|
🚧 Unified memory cannot be forwarded at all, so a paged optimizer needs `host=letify.remote`
|
|
537
534
|
|
|
@@ -190,6 +190,22 @@ cache.fetch_checkpoint("run-1", target)
|
|
|
190
190
|
cache.resume(runtime, "run-1", path) # put the newest one inside a session
|
|
191
191
|
```
|
|
192
192
|
|
|
193
|
+
### Exit hooks
|
|
194
|
+
|
|
195
|
+
```python
|
|
196
|
+
letify.on_exit(fn) # inside a body: run fn if the worker ends before the body does
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
The hook runs on `SIGTERM`, when the lease deadline passes and when the channel closes during the call, newest first, within 120 s in total. Outside a body it raises `UnsupportedMode`. `letify runs`, `interrupt` and `exec` reach a running call from the command line; see [Controlling a running call](docs/SPEC.md#run-control).
|
|
200
|
+
|
|
201
|
+
### Taking over a running call
|
|
202
|
+
|
|
203
|
+
```python
|
|
204
|
+
value = letify.takeover(run) # become the owner of a running call, return its value
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
`letify.takeover(run)` and `letify takeover <run>` make the calling process the owner of a call another process started: it renews the lease, receives the output and the result, writes the call's outputs back under the project the call was sent from, and ends the session. The previous owner, if it is alive, is asked to let go and its waiting call raises `OwnershipMoved`. A provider whose runtime a second process cannot reach raises `UnsupportedMode` naming it; see [Taking over a running call](docs/SPEC.md#run-takeover).
|
|
208
|
+
|
|
193
209
|
### Handoffs
|
|
194
210
|
|
|
195
211
|
```python
|
|
@@ -225,6 +241,7 @@ Both work in the local process and inside a shipped function, on any provider. T
|
|
|
225
241
|
| `ProtocolError` | The remote process died before producing a result | yes |
|
|
226
242
|
| `RemoteError` | The shipped function raised; carries the remote traceback | no |
|
|
227
243
|
| `UnsupportedMode` | The requested mode cannot work here | no |
|
|
244
|
+
| `OwnershipMoved` | The call you were waiting on was taken over by another process, which now receives its result | no |
|
|
228
245
|
| `InsufficientDevices` | The devices a call asks for cannot be allocated; names what holds them | no |
|
|
229
246
|
|
|
230
247
|
### Command line
|
|
@@ -237,8 +254,11 @@ letify devices # accelerators each provider offers
|
|
|
237
254
|
letify status # live sessions and what they are costing
|
|
238
255
|
letify usage [alias] # what is left on each account
|
|
239
256
|
letify utilization [alias] # how busy each instance's accelerator is
|
|
257
|
+
letify runs # calls running from this machine, with their run ids
|
|
258
|
+
letify interrupt <run> # raise KeyboardInterrupt in a running call's body
|
|
259
|
+
letify exec <run> CODE # run source beside a running call (-f FILE, --timeout S)
|
|
260
|
+
letify takeover <run> # become the owner of a running call (--result PATH, --json)
|
|
240
261
|
letify check <alias> # confirm a machine answers
|
|
241
|
-
letify probe <host> # whether host="local" is worth using
|
|
242
262
|
letify efficiency 0.5 3 150 # expected share of a direct run
|
|
243
263
|
letify stubs # write typings/letify_providers.pyi for editor completion
|
|
244
264
|
```
|
|
@@ -298,6 +318,10 @@ That is the only install. `letify` installs cloudpickle and blake3 and nothing e
|
|
|
298
318
|
|
|
299
319
|
`host="local"` additionally needs PyTorch 2.1 or newer in the project, locally and in the runtime's environment, with the same major.minor version on both sides. letify never installs it: torch is the project's own dependency.
|
|
300
320
|
|
|
321
|
+
## Rust crate
|
|
322
|
+
|
|
323
|
+
The Rust crate `letify` is published on crates.io at the same version as the Python package. It will re-export the components meant for use outside letify, first the connection library `letify-link`.
|
|
324
|
+
|
|
301
325
|
## How this project is built
|
|
302
326
|
|
|
303
327
|
Spec driven and test driven. [docs/SPEC.md](docs/SPEC.md) is the source of truth, a change edits it before the code, and every test traces to a section of it. The rules are in [CLAUDE.md](CLAUDE.md).
|
|
@@ -146,7 +146,7 @@ uv add letify
|
|
|
146
146
|
|
|
147
147
|
That is the whole install, for every provider. letify installs only cloudpickle and blake3. It needs [uv](https://docs.astral.sh/uv/) on the machine, because provider tools such as the Colab CLI and the Modal client run through uv in their own environments, not in your `.venv`.
|
|
148
148
|
|
|
149
|
-
The wheel carries [letify-core](letify-core/), the Rust
|
|
149
|
+
The wheel carries the `letify-quic` binary built from [letify-core](letify-core/), the Rust workspace. Wheels are built for Linux x86_64 and aarch64, Windows x86_64 and arm64, and macOS arm64 and x86_64. On another platform, pip falls back to the source distribution, which has no `letify-quic`, so the QUIC connection strategy is skipped.
|
|
150
150
|
|
|
151
151
|
---
|
|
152
152
|
|
|
@@ -234,9 +234,9 @@ Two words, and neither of them is a mechanism.
|
|
|
234
234
|
|
|
235
235
|
**`device`** carries the provider, the account and the accelerator in one value, because those are one decision. Core count and memory come with the shape the provider registered, so there is nothing to ask for.
|
|
236
236
|
|
|
237
|
-
**`host`** is the CUDA word for the CPU side. `letify.local` keeps Python and the libraries in this process and forwards only
|
|
237
|
+
**`host`** is the CUDA word for the CPU side. `letify.local` keeps Python and the libraries in this process and forwards only PyTorch operators. `letify.remote` ships the function to the machine that holds the device. The strings `"local"` and `"remote"` are the same values.
|
|
238
238
|
|
|
239
|
-
Leave `host` out and you get `letify.local`. When the device is far away, forwarding
|
|
239
|
+
Leave `host` out and you get `letify.local`. When the device is far away, forwarding operators is slow, so the call warns with the expected efficiency and then runs. For a remote GPU, `host=letify.remote` is usually what you want.
|
|
240
240
|
|
|
241
241
|
How long a session lives is not a declaration argument. A call ends its session. `with let.keep_alive():` keeps sessions for the length of a block, so a run of separate calls does not pay session start each time.
|
|
242
242
|
|
|
@@ -258,7 +258,7 @@ Counter-intuitive consequence: **a faster GPU makes forwarding worse**, because
|
|
|
258
258
|
|
|
259
259
|
Decoding is the case that stays bad. Throughput is bounded near `1000 / (k × RTT)` tokens per second, so the card stops mattering. Ship the whole `generate` call instead.
|
|
260
260
|
|
|
261
|
-
Measure your own `k` with `torch.cuda.set_sync_debug_mode("warn")` and your round trip with `
|
|
261
|
+
Measure your own `k` with `torch.cuda.set_sync_debug_mode("warn")` and your round trip with `ping`. See [docs/NETWORK.md](docs/NETWORK.md).
|
|
262
262
|
|
|
263
263
|
</details>
|
|
264
264
|
|
|
@@ -470,7 +470,6 @@ letify status # what is running right now
|
|
|
470
470
|
letify usage # what is left on each account
|
|
471
471
|
letify utilization # how busy each instance's GPU is
|
|
472
472
|
letify check lab # does this machine answer?
|
|
473
|
-
letify probe lab # is host=letify.local worth using here?
|
|
474
473
|
letify efficiency 0.5 3 150 # the formula, from measured terms
|
|
475
474
|
```
|
|
476
475
|
|
|
@@ -488,7 +487,7 @@ Add `--json` to `usage`, `utilization` or `status` for output a program can read
|
|
|
488
487
|
| 📐 [docs/SPEC.md](docs/SPEC.md) | The design as it stands, decision by decision |
|
|
489
488
|
| 🧩 [docs/COMPONENT.md](docs/COMPONENT.md) | Every class, and the vocabulary |
|
|
490
489
|
| 🌐 [docs/NETWORK.md](docs/NETWORK.md) | Transports, latency measurements, tunnel choices |
|
|
491
|
-
| 🦀 [letify-core/](letify-core/) | The Rust workspace
|
|
490
|
+
| 🦀 [letify-core/](letify-core/) | The Rust workspace: the `letify` crate and the `letify-quic` binary |
|
|
492
491
|
| 🧭 [docs/guide/](docs/guide/) | Task-oriented guides |
|
|
493
492
|
| 🇰🇷 [docs/locales/README_ko.md](docs/locales/README_ko.md) | 한국어 |
|
|
494
493
|
|
|
@@ -502,11 +501,9 @@ Alpha, and honest about it. What works today:
|
|
|
502
501
|
✅ Persistent sessions: a session cache keeps values between calls, large arguments travel once
|
|
503
502
|
✅ Content addressed storage, configuration and secrets
|
|
504
503
|
✅ The `Local` and `Colab` providers
|
|
505
|
-
✅ `letify-core`, verified on a real GPU: the agent opens the driver, the local driver forwards an allocation and a copy in both directions, and the bytes match
|
|
506
504
|
|
|
507
505
|
Not finished yet:
|
|
508
506
|
|
|
509
|
-
🚧 `letify-driver` covers the entry points a PyTorch process needs to start up and run one kernel. Anything else names itself and returns `CUDA_ERROR_NOT_SUPPORTED`, so a real run prints the list of what to build next
|
|
510
507
|
🚧 `Modal` and `Elice` follow each published interface but have not been run against the live services
|
|
511
508
|
🚧 Unified memory cannot be forwarded at all, so a paged optimizer needs `host=letify.remote`
|
|
512
509
|
|
|
@@ -24,7 +24,7 @@ Read this table first. Most confusion about letify is one of these words meaning
|
|
|
24
24
|
| **persistence** | Whether a provider's storage outlives a session: `persistent` or `ephemeral`. |
|
|
25
25
|
| **RemoteTensor** | Under `host="local"`, the local stand-in for a tensor on the runtime's GPU: a meta tensor for shape and dtype plus a handle. Reports its device as `cuda:0`. |
|
|
26
26
|
| **device worker** | The process on the runtime that runs PyTorch operators on real tensors keyed by handle, one per session. |
|
|
27
|
-
| **letify-core** |
|
|
27
|
+
| **letify-core** | The Rust workspace in `letify-core/`: the `letify` crate and the `letify-quic` binary. The CUDA driver stand-in it once held is removed. |
|
|
28
28
|
|
|
29
29
|
## The shape of the thing
|
|
30
30
|
|
|
@@ -58,7 +58,7 @@ One directory per concern, so the file you need is the one named after the thing
|
|
|
58
58
|
| `runtime/` | The channel, the session, the pool, the lease, bootstrap source |
|
|
59
59
|
| `providers/` | The base class, name normalization, and one module per provider |
|
|
60
60
|
| `store/` | The content addressed store, volumes, and `backends/` |
|
|
61
|
-
| `remoting/` | PyTorch forwarding in `remoting/device/`, and the
|
|
61
|
+
| `remoting/` | PyTorch forwarding in `remoting/device/`, and the efficiency arithmetic |
|
|
62
62
|
| `launcher.py` | `Launcher` itself, which ties the rest together |
|
|
63
63
|
| `errors.py` | The exception hierarchy, which everything imports |
|
|
64
64
|
| `cli.py` | The command line |
|
|
@@ -180,7 +180,7 @@ Three protocol rules exist for performance rather than tidiness. Large arguments
|
|
|
180
180
|
|
|
181
181
|
## remoting
|
|
182
182
|
|
|
183
|
-
PyTorch forwarding, in `remoting/device/`, and the
|
|
183
|
+
PyTorch forwarding, in `remoting/device/`, and the efficiency arithmetic beside it.
|
|
184
184
|
|
|
185
185
|
| Module | Owns |
|
|
186
186
|
|---|---|
|
|
@@ -197,4 +197,4 @@ It refuses rather than degrading when PyTorch is absent or too old. It does not
|
|
|
197
197
|
|
|
198
198
|
## letify-core
|
|
199
199
|
|
|
200
|
-
|
|
200
|
+
The Rust workspace in `letify-core/`. It holds the crate `letify`, published to crates.io, and the binary `letify-quic`, which the wheels carry in `letify/remoting/lib/` and `letify/transport/quic.py` runs. The driver stand-in it once held is removed; the spec's "The driver stand-in is removed" gives the measurement and where the code is.
|
|
@@ -217,26 +217,6 @@ Both sides bound one TCP port, learned the mapping from `stun.nextcloud.com:443`
|
|
|
217
217
|
|
|
218
218
|
`colab console` is not a usable channel: it is a terminal inside tmux, and a single input line of 4000 characters or more arrives truncated or mixed with terminal control sequences. The Colab runtime proxy reaches only port 8080 on the VM; other port prefixes return 404.
|
|
219
219
|
|
|
220
|
-
### letify-core copy throughput
|
|
221
|
-
|
|
222
|
-
A copy to the device through letify-core, with the payload streamed from the caller's buffer into the agent's staging buffer, moves about 2000 MiB/s over loopback TCP. The same payload through an encoded frame, the path before streaming, moves about 520 MiB/s. Both carry one 256 MiB `CopyToDevice` request between a `BufWriter` and a `BufReader` with `TCP_NODELAY`, on an AMD EPYC 7763, and report the median of five runs.
|
|
223
|
-
|
|
224
|
-
| Path | Three invocations |
|
|
225
|
-
|---|---|
|
|
226
|
-
| Encoded frame, before the change | 545, 544, 507 MiB/s |
|
|
227
|
-
| Encoded frame, after the change | 515, 514, 519 MiB/s |
|
|
228
|
-
| Streamed into staging | 2043, 1961, 1873 MiB/s |
|
|
229
|
-
|
|
230
|
-
A copy to the host, with the payload streamed from the agent's staging buffer into the caller's buffer, moves about 3250 MiB/s over loopback TCP. The same payload through an encoded frame and a copy into the caller's buffer, the path before streaming, moves about 520 MiB/s. Both carry one 256 MiB `Payload` reply over the same socket pair and machine and report the median of five runs. The copy on the device is not included.
|
|
231
|
-
|
|
232
|
-
| Path | Three invocations |
|
|
233
|
-
|---|---|
|
|
234
|
-
| Encoded frame, before the change | 522, 514, 519 MiB/s |
|
|
235
|
-
| Encoded frame, after the change | 528, 527, 540 MiB/s |
|
|
236
|
-
| Streamed into the destination | 3234, 3261, 3244 MiB/s |
|
|
237
|
-
|
|
238
|
-
Loopback removes the network, so these numbers bound what the copy path itself costs. On a real link the link rate decides throughput whenever it is below them. To reproduce, run `cargo test --release -p letify-wire --test throughput -- --ignored --nocapture --test-threads=1` in `letify-core/`.
|
|
239
|
-
|
|
240
220
|
## PyTorch forwarding on dept_gpu
|
|
241
221
|
|
|
242
222
|
> Reading the loss once every 50 steps, `host="local"` takes a median 3.45 ms per step against 1.70 ms directly on the server: about twice the direct time, and one round trip per 45 steps. Measured on 2026-09-14.
|
|
@@ -315,7 +295,7 @@ Three checks settle most of what is provider specific.
|
|
|
315
295
|
|
|
316
296
|
```bash
|
|
317
297
|
# Round trip to the machine, which sets the efficiency formula.
|
|
318
|
-
|
|
298
|
+
ping -c 5 gpu.lab.example.edu
|
|
319
299
|
|
|
320
300
|
# Whether a layer 3 tunnel is possible at all.
|
|
321
301
|
ls -l /dev/net/tun
|