launchhelm 0.1.0__tar.gz

launchhelm-0.1.0/.gitignore

@@ -0,0 +1,12 @@
+.coverage
+.coverage.*
+htmlcov/
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.venv/
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/

launchhelm-0.1.0/CHANGELOG.md

@@ -0,0 +1,54 @@
+# Changelog
+
+All notable changes to the `launchhelm` Python SDK are documented here.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
+and this project adheres to pre-1.0 semver: minor releases may break the
+public surface until 1.0.0.
+
+## [0.1.0] - 2026-10-05
+
+Initial public release.
+
+### Added
+
+- `launchhelm.init`/`Run` session API: scalar logging with ordered steps,
+  `flush`/`close` durability levels, overflow modes (`drop_new`, `block`,
+  `raise`), run metadata, config attachment and resumable runs.
+- Cooperative command runtime: `run.on` safe-point handlers, `run.sync`,
+  typed capabilities with schemas, journaled command outcomes (applied,
+  rejected, failed, `outcome_unknown` — never silently retried), and
+  command tombstones under a bounded journal budget.
+- Local durability: SQLite spool with serialized-payload budgets and
+  crash recovery; server-acknowledged flush.
+- Rich values: `Text`/`Image`/`Audio`/`Video`/`Table`/`Histogram` objects
+  inside `run.log`, and the run methods `text`, `image`, `audio`, `video`,
+  `media`, `upload_media`, `media_url`, `rows`, `log_table`, `histogram`,
+  and `histogram_counts`, producing the canonical `rich_text`,
+  `rich_media`, `rich_table`, and `rich_histogram` renderers. Native media
+  uploads run on a background thread through the run's durable
+  content-addressed spool (`media_spool_bytes`, default 256 MiB) before
+  the point is admitted; `flush()` waits for admission. Image sources
+  include paths, bytes, PIL images, and NumPy/torch arrays encoded to PNG.
+- `Client.read_samples` for bounded renderer reads (`latest`, `at`,
+  `spread`, `index` select modes) with a media map of presigned downloads.
+- Artifact capture: content-addressed uploads (multipart for large
+  payloads, 64 MiB part cap), digests verified against the server record.
+- Scoped project opening: `project_id`/`LAUNCHHELM_PROJECT` selects the
+  project document directly for UI-issued project keys.
+- `launchhelm.torch` adapter: `init`/`watch`/`step`/`log`/`log_reduced`,
+  tunables, built-in controls (`lr`, `weight_decay`, `grad_clip`,
+  `dropout`, `checkpoint`, `eval`), `clip_grad_norm_`, checkpoint
+  save/resume via `load_checkpoint("latest")`, and CPU-gloo verified DDP
+  semantics (per-rank participants, rank-0 logging, collective control
+  protocol, `DivergedStateError` on partial failure).
+- `launchhelm.system`: periodic host and per-GPU metrics (psutil + NVML),
+  safe to run without torch.
+- `launchhelm.torch.hf` and `launchhelm.torch.lightning` callbacks.
+- Optional extras: `launchhelm[torch]`, `[system]`, `[hf]`, `[lightning]`.
+  The base package has no torch or numpy import.
+- Typed package: `py.typed` ships with strict-checked source.
+
+### Notes
+
+- Requires Python 3.11+. GPU/NCCL and multi-node training paths are not
+  yet qualified; FSDP checkpoints are refused with a clear error.

launchhelm-0.1.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

launchhelm-0.1.0/PKG-INFO

@@ -0,0 +1,193 @@
+Metadata-Version: 2.4
+Name: launchhelm
+Version: 0.1.0
+Summary: LaunchHelm client and cooperative workload runtime
+Project-URL: Homepage, https://launchhelm.com
+Project-URL: Documentation, https://launchhelm.com/docs/
+License-Expression: Apache-2.0
+License-File: LICENSE
+Keywords: cooperative,launchhelm,runtime,telemetry
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: httpx<1,>=0.28.1
+Requires-Dist: jsonschema<5,>=4.25.1
+Requires-Dist: referencing<1,>=0.36.2
+Provides-Extra: hf
+Requires-Dist: accelerate>=1.1; extra == 'hf'
+Requires-Dist: torch>=2.5; extra == 'hf'
+Requires-Dist: transformers>=4.46; extra == 'hf'
+Provides-Extra: lightning
+Requires-Dist: lightning>=2.4; extra == 'lightning'
+Requires-Dist: torch>=2.5; extra == 'lightning'
+Provides-Extra: system
+Requires-Dist: nvidia-ml-py>=12.560; extra == 'system'
+Requires-Dist: psutil>=6.1; extra == 'system'
+Provides-Extra: torch
+Requires-Dist: torch>=2.5; extra == 'torch'
+Description-Content-Type: text/markdown
+
+# LaunchHelm
+
+Training telemetry and live control for any Python or PyTorch loop — the
+Python SDK for the hosted LaunchHelm service at
+[launchhelm.com](https://launchhelm.com).
+
+Requires Python 3.11 or later on macOS, Linux, or Windows.
+
+## Install
+
+```bash
+pip install launchhelm
+```
+
+| Extra | For |
+| --- | --- |
+| `launchhelm[torch]` | PyTorch adapter: watched trainer, live controls, DDP, checkpoints |
+| `launchhelm[system]` | host metrics (psutil) and per-GPU stats (NVML) |
+| `launchhelm[hf]` | `transformers.Trainer` callback (`launchhelm.torch.hf`) |
+| `launchhelm[lightning]` | PyTorch Lightning callback (`launchhelm.torch.lightning`) |
+
+## Quickstart
+
+In Mission Control, open the organization's **Projects** page and choose
+**New project** — it creates the project and a project key in one step. The
+credential is shown once; save it before leaving:
+
+```bash
+export LAUNCHHELM_API_KEY=lh_...        # shown once — never commit it
+export LAUNCHHELM_ORGANIZATION=...      # organization UUID
+export LAUNCHHELM_PROJECT=...           # project UUID
+# export LAUNCHHELM_URL=https://launchhelm.com   # optional; this is the default
+```
+
+A `.env` file in the working directory or a parent works too; exported
+shell variables take precedence. Then in the training script:
+
+```python
+import launchhelm
+
+with launchhelm.init("quickstart", config={"lr": 3e-4, "batch": 64}) as run:
+    for step in range(100):
+        loss = train_one_step()
+        run.log({"loss": loss}, step=step)
+```
+
+The first argument is the project **name**, which must match the project
+the key belongs to; `LAUNCHHELM_PROJECT` is that project's **UUID**. With
+an organization-wide key, leave `LAUNCHHELM_PROJECT` unset and `init`
+reuses or creates the named project in the organization catalog.
+
+## Rich values
+
+Beyond scalars, a stream can carry rendered values — text, media, tables,
+and histograms — via `run.log` with `Text`/`Image`/`Audio`/`Video`/`Table`/
+`Histogram` objects or the matching run methods:
+
+```python
+run.text("notes", "## epoch 4\nloss **0.25**", format="markdown")
+run.image("frames", "confusion.png")            # background upload + spool
+run.log_table("sweep", [("lr", "number")], [[3e-4]])
+run.histogram("grad", gradient_samples, bins=64)
+
+result = client.read_samples("stream-id", select={"mode": "latest", "limit": 16})
+```
+
+Native media uploads run off the training thread through the run's durable
+spool; `flush()` waits for the artifact to exist before the point is
+admitted. Sample reads return at most one point per axis value — the newest
+write wins — so a text stream keeps one value per step, not an appended log.
+
+## PyTorch
+
+`launchhelm.torch` is an optional adapter (`pip install "launchhelm[torch]"`)
+for arbitrary training loops, including DDP. `import launchhelm` itself
+never imports torch.
+
+```python
+import launchhelm.torch as lht
+
+tr = lht.init("urm-llm", config={"lr": 3e-4, "batch": 64})
+model, optimizer = build()
+tr.watch(model, optimizer, scheduler=sched)
+
+for step in range(steps):
+    loss = train_one_step()
+    tr.clip_grad_norm_()          # current (controllable) max norm
+    optimizer.step(); sched.step()
+    tr.step(loss=loss, samples=batch_size)   # logs + services control; call on every rank
+
+tr.save_checkpoint(extra={"epoch": epoch})
+tr.finish()
+```
+
+`tr.step` logs the built-in series (`train/loss`, `optim/lr/group<i>`,
+`optim/grad_norm`, `perf/step_time_s`, `perf/samples_per_s`,
+`perf/tokens_per_s`, `optim/loss_scale` with a watched `GradScaler`,
+`cuda/max_memory_allocated_bytes` on CUDA) and services the `lht` safe
+point: commands issued from the UI apply only inside `step`, never between
+backward and `optimizer.step`. Registered controls: `lr` (scale or
+absolute, scheduler-aware), `weight_decay`, `grad_clip`, `dropout`,
+`checkpoint`, `eval` when `eval_fn` is passed to `watch`, and
+`tr.tunable(name, ...)` for custom typed knobs.
+
+Distributed: call `lht.init` on every rank — rank 0 opens the run and every
+rank is a `rank-<r>` participant. `step` is a collective; a control applied
+at a step applies on every rank at that step, keeping DDP replicas
+identical, and a rank that fails the apply raises `DivergedStateError` on
+all ranks after the command is journaled as `outcome_unknown`.
+
+`save_checkpoint` writes a `torch.load(weights_only=True)`-safe payload —
+model (DDP-unwrapped), optimizer, scheduler, scaler, step, per-rank RNG
+streams, tunables, control state, `extra` — uploads it, and keeps it so a
+crash-resume finds the latest. `load_checkpoint("latest")` restores
+model/optimizer state identically on every rank plus each rank's own RNG
+stream, and returns `extra`. Honest limits: GPU/NCCL/multi-node operation
+is not yet qualified; FSDP/sharded models are refused with
+`CheckpointError` (configure a full state dict first); NumPy's global RNG
+requires legacy MT19937 state; sampler/data-iterator positions and
+dataloader-worker state are the caller's responsibility — restoring
+weights and global RNG alone does not faithfully reconstruct a shuffled or
+multi-worker data pipeline.
+
+System metrics without torch: `launchhelm.system.start(run, interval=15.0)`
+(`launchhelm[system]`) emits CPU, RSS, disk/network rates (psutil) and
+per-GPU stats (NVML) under `system/...` on a daemon thread that never
+blocks logging.
+
+Framework callbacks: `launchhelm.torch.hf.LaunchHelmCallback`
+(`launchhelm[hf]` extra) and
+`launchhelm.torch.lightning.LaunchHelmCallback` (`launchhelm[lightning]`
+extra) wire the same init/log/control/checkpoint/finish lifecycle through
+each framework's callback API.
+
+## Delivery guarantees
+
+| Call | Guarantee |
+| --- | --- |
+| `run.log(...)` | admitted to volatile memory; acknowledged locally |
+| `run.flush(delivered=False)` | durable in the local SQLite spool — survives process death |
+| `run.flush()` | waits until the server has acknowledged the points (query-visible) |
+| `run.close()` | persists the local queue; does **not** promise server delivery |
+
+Memory and spool limits are serialized-payload budgets; SQLite, journal and
+allocator overhead are separate. Command tombstones are retained under a
+separate bounded journal budget; exhaustion fails closed. Unknown effects
+are never automatically re-executed. On Windows the local spool's file
+permissions come from the user profile's ACL rather than POSIX modes.
+
+## Links
+
+- Documentation: [launchhelm.com/docs](https://launchhelm.com/docs/) —
+  [getting started](https://launchhelm.com/docs/getting-started/),
+  [Python guide](https://launchhelm.com/docs/python/),
+  [concepts](https://launchhelm.com/docs/concepts/)
+- Changelog: `CHANGELOG.md`, shipped in the source distribution

launchhelm-0.1.0/README.md

@@ -0,0 +1,156 @@
+# LaunchHelm
+
+Training telemetry and live control for any Python or PyTorch loop — the
+Python SDK for the hosted LaunchHelm service at
+[launchhelm.com](https://launchhelm.com).
+
+Requires Python 3.11 or later on macOS, Linux, or Windows.
+
+## Install
+
+```bash
+pip install launchhelm
+```
+
+| Extra | For |
+| --- | --- |
+| `launchhelm[torch]` | PyTorch adapter: watched trainer, live controls, DDP, checkpoints |
+| `launchhelm[system]` | host metrics (psutil) and per-GPU stats (NVML) |
+| `launchhelm[hf]` | `transformers.Trainer` callback (`launchhelm.torch.hf`) |
+| `launchhelm[lightning]` | PyTorch Lightning callback (`launchhelm.torch.lightning`) |
+
+## Quickstart
+
+In Mission Control, open the organization's **Projects** page and choose
+**New project** — it creates the project and a project key in one step. The
+credential is shown once; save it before leaving:
+
+```bash
+export LAUNCHHELM_API_KEY=lh_...        # shown once — never commit it
+export LAUNCHHELM_ORGANIZATION=...      # organization UUID
+export LAUNCHHELM_PROJECT=...           # project UUID
+# export LAUNCHHELM_URL=https://launchhelm.com   # optional; this is the default
+```
+
+A `.env` file in the working directory or a parent works too; exported
+shell variables take precedence. Then in the training script:
+
+```python
+import launchhelm
+
+with launchhelm.init("quickstart", config={"lr": 3e-4, "batch": 64}) as run:
+    for step in range(100):
+        loss = train_one_step()
+        run.log({"loss": loss}, step=step)
+```
+
+The first argument is the project **name**, which must match the project
+the key belongs to; `LAUNCHHELM_PROJECT` is that project's **UUID**. With
+an organization-wide key, leave `LAUNCHHELM_PROJECT` unset and `init`
+reuses or creates the named project in the organization catalog.
+
+## Rich values
+
+Beyond scalars, a stream can carry rendered values — text, media, tables,
+and histograms — via `run.log` with `Text`/`Image`/`Audio`/`Video`/`Table`/
+`Histogram` objects or the matching run methods:
+
+```python
+run.text("notes", "## epoch 4\nloss **0.25**", format="markdown")
+run.image("frames", "confusion.png")            # background upload + spool
+run.log_table("sweep", [("lr", "number")], [[3e-4]])
+run.histogram("grad", gradient_samples, bins=64)
+
+result = client.read_samples("stream-id", select={"mode": "latest", "limit": 16})
+```
+
+Native media uploads run off the training thread through the run's durable
+spool; `flush()` waits for the artifact to exist before the point is
+admitted. Sample reads return at most one point per axis value — the newest
+write wins — so a text stream keeps one value per step, not an appended log.
+
+## PyTorch
+
+`launchhelm.torch` is an optional adapter (`pip install "launchhelm[torch]"`)
+for arbitrary training loops, including DDP. `import launchhelm` itself
+never imports torch.
+
+```python
+import launchhelm.torch as lht
+
+tr = lht.init("urm-llm", config={"lr": 3e-4, "batch": 64})
+model, optimizer = build()
+tr.watch(model, optimizer, scheduler=sched)
+
+for step in range(steps):
+    loss = train_one_step()
+    tr.clip_grad_norm_()          # current (controllable) max norm
+    optimizer.step(); sched.step()
+    tr.step(loss=loss, samples=batch_size)   # logs + services control; call on every rank
+
+tr.save_checkpoint(extra={"epoch": epoch})
+tr.finish()
+```
+
+`tr.step` logs the built-in series (`train/loss`, `optim/lr/group<i>`,
+`optim/grad_norm`, `perf/step_time_s`, `perf/samples_per_s`,
+`perf/tokens_per_s`, `optim/loss_scale` with a watched `GradScaler`,
+`cuda/max_memory_allocated_bytes` on CUDA) and services the `lht` safe
+point: commands issued from the UI apply only inside `step`, never between
+backward and `optimizer.step`. Registered controls: `lr` (scale or
+absolute, scheduler-aware), `weight_decay`, `grad_clip`, `dropout`,
+`checkpoint`, `eval` when `eval_fn` is passed to `watch`, and
+`tr.tunable(name, ...)` for custom typed knobs.
+
+Distributed: call `lht.init` on every rank — rank 0 opens the run and every
+rank is a `rank-<r>` participant. `step` is a collective; a control applied
+at a step applies on every rank at that step, keeping DDP replicas
+identical, and a rank that fails the apply raises `DivergedStateError` on
+all ranks after the command is journaled as `outcome_unknown`.
+
+`save_checkpoint` writes a `torch.load(weights_only=True)`-safe payload —
+model (DDP-unwrapped), optimizer, scheduler, scaler, step, per-rank RNG
+streams, tunables, control state, `extra` — uploads it, and keeps it so a
+crash-resume finds the latest. `load_checkpoint("latest")` restores
+model/optimizer state identically on every rank plus each rank's own RNG
+stream, and returns `extra`. Honest limits: GPU/NCCL/multi-node operation
+is not yet qualified; FSDP/sharded models are refused with
+`CheckpointError` (configure a full state dict first); NumPy's global RNG
+requires legacy MT19937 state; sampler/data-iterator positions and
+dataloader-worker state are the caller's responsibility — restoring
+weights and global RNG alone does not faithfully reconstruct a shuffled or
+multi-worker data pipeline.
+
+System metrics without torch: `launchhelm.system.start(run, interval=15.0)`
+(`launchhelm[system]`) emits CPU, RSS, disk/network rates (psutil) and
+per-GPU stats (NVML) under `system/...` on a daemon thread that never
+blocks logging.
+
+Framework callbacks: `launchhelm.torch.hf.LaunchHelmCallback`
+(`launchhelm[hf]` extra) and
+`launchhelm.torch.lightning.LaunchHelmCallback` (`launchhelm[lightning]`
+extra) wire the same init/log/control/checkpoint/finish lifecycle through
+each framework's callback API.
+
+## Delivery guarantees
+
+| Call | Guarantee |
+| --- | --- |
+| `run.log(...)` | admitted to volatile memory; acknowledged locally |
+| `run.flush(delivered=False)` | durable in the local SQLite spool — survives process death |
+| `run.flush()` | waits until the server has acknowledged the points (query-visible) |
+| `run.close()` | persists the local queue; does **not** promise server delivery |
+
+Memory and spool limits are serialized-payload budgets; SQLite, journal and
+allocator overhead are separate. Command tombstones are retained under a
+separate bounded journal budget; exhaustion fails closed. Unknown effects
+are never automatically re-executed. On Windows the local spool's file
+permissions come from the user profile's ACL rather than POSIX modes.
+
+## Links
+
+- Documentation: [launchhelm.com/docs](https://launchhelm.com/docs/) —
+  [getting started](https://launchhelm.com/docs/getting-started/),
+  [Python guide](https://launchhelm.com/docs/python/),
+  [concepts](https://launchhelm.com/docs/concepts/)
+- Changelog: `CHANGELOG.md`, shipped in the source distribution

launchhelm-0.1.0/benchmarks/README.md

@@ -0,0 +1,44 @@
+# LaunchHelm Python SDK benchmarks
+
+## Training-overhead benchmark
+
+From a repository checkout on macOS, Linux, or Windows, with a live server and a private
+credential env file, run:
+
+```sh
+uv sync --frozen --group frameworks
+uv run --frozen --group frameworks python -m benchmarks.training_overhead \
+  --env /path/to/private.env --project quickstart --workload tiny \
+  --pairs 12 --out /tmp/launchhelm-overhead-tiny.json
+uv run --frozen --group frameworks python -m benchmarks.training_overhead \
+  --env /path/to/private.env --project quickstart --workload small \
+  --pairs 12 --out /tmp/launchhelm-overhead-small.json
+```
+
+`--project` is the project name associated with the key, not its UUID. Each
+SDK trial creates and finishes a real run. Trials alternate baseline/SDK order
+and report median and mean-step overhead, a paired bootstrap interval, setup,
+flush and finish times, and per-trial host load and package versions. Every CPU
+metric history must contain its expected steps, the runtime must report zero
+drops, and paired final losses must match. Steps plus warmup cannot exceed
+10,000, the history limit. The driver refuses to start a trial when one-minute
+host load is 15 or higher; stop unrelated workloads yourself or use an idle
+host. Even below that gate, inspect the recorded load and variability before
+interpreting results. These single-rank CPU measurements are not GPU/DDP,
+public-deployment or capacity guarantees.
+
+## Hosted-pilot workload qualification
+
+The maintained cohort harness uses the real SDK init/log/flush/finish path, with independent run/session state per producer. A private JSON manifest contains exactly one entry per producer with `url`, `api_key`, `organization_id`, `project_id` and `project` (the name). Keep this file outside source control with restricted permissions. Accounts are verified through `/v1/me`; shared-account development requires an explicit override and is not 100-user qualification.
+
+```sh
+uv run --frozen python -m benchmarks.pilot_workload \
+  --manifest /path/to/private-cohort.json \
+  --state-dir /path/to/new-cohort-state \
+  --producers 100 --metrics 100 --ticks 1800 --interval 10 \
+  --phase synchronized --out /path/to/new-cohort-report.json
+```
+
+The state directory and report must be new. The five-hour default represents 18 million scheduled points plus one warmup point per stream. `--phase staggered` spreads producer ticks evenly across the interval; synchronized mode exercises a 10,000-point burst. Setup and final full-data verification are outside the logging phase. The fixed schedule records lateness instead of hiding backpressure by lowering the offered rate. Reports include admission latency, receipt-attempt counters, a bounded latency histogram, attempted wire bytes, flush cost, drops and exact raw-history verification for every metric. Warmup/overall ingest counters are recorded separately; overall counters include drain and retries. Duplicate rows, missing ticks, wrong values, broken pagination and nonzero drops fail verification. Synthetic values are deterministic and vary by producer/metric/tick; they are not a promise of real workload compression.
+
+This is SDK delivery/scheduling evidence, not an automatic capacity certificate, chart-freshness percentile, mixed dashboard/control test, or Oracle allocation proof. It creates and finishes real runs without deleting them. No native artifacts are uploaded. Qualification must run from a separate load-generator host and include server resource/disk measurements and recovery tests. Run `uv run --frozen pytest -q tests/test_pilot_workload.py` for harness correctness checks.

launchhelm-0.1.0/benchmarks/__init__.py

@@ -0,0 +1,5 @@
+"""LaunchHelm Python SDK microbenchmarks.
+
+These scripts measure local runtime admission and spooling. They do not
+establish capacity or latency targets for a deployed server.
+"""