runboard 0.2.0__tar.gz

runboard-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mohammad Zbeeb
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

runboard-0.2.0/PKG-INFO

@@ -0,0 +1,362 @@
+Metadata-Version: 2.4
+Name: runboard
+Version: 0.2.0
+Summary: Self-hosted, W&B-style live training dashboards for any cluster or VM, viewable in your browser without SSH.
+Author-email: Mohammad Zbeeb <mhmdzbyb829@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/Moe-Zbeeb/runboard
+Project-URL: Documentation, https://github.com/Moe-Zbeeb/runboard#readme
+Project-URL: Issues, https://github.com/Moe-Zbeeb/runboard/issues
+Project-URL: Changelog, https://github.com/Moe-Zbeeb/runboard/blob/main/CHANGELOG.md
+Keywords: machine-learning,experiment-tracking,training,dashboard,metrics,slurm,cluster,wandb
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Operating System :: MacOS
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+<h1 align="center">runboard</h1>
+
+<p align="center">
+  <b>Your own live experiment dashboard on Cloudflare, from any cluster or VM.</b>
+</p>
+
+<p align="center">
+  <a href="https://github.com/Moe-Zbeeb/runboard/actions/workflows/ci.yml"><img src="https://github.com/Moe-Zbeeb/runboard/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://pypi.org/project/runboard/"><img src="https://img.shields.io/pypi/v/runboard" alt="PyPI"></a>
+  <a href="https://pypi.org/project/runboard/"><img src="https://img.shields.io/pypi/pyversions/runboard" alt="Python versions"></a>
+  <img src="https://img.shields.io/badge/dependencies-0-brightgreen" alt="Zero dependencies">
+  <a href="https://github.com/Moe-Zbeeb/runboard/blob/main/LICENSE"><img src="https://img.shields.io/github/license/Moe-Zbeeb/runboard" alt="License"></a>
+</p>
+
+<p align="center">
+  <a href="https://deploy.workers.cloudflare.com/?url=https://github.com/Moe-Zbeeb/runboard"><img src="https://deploy.workers.cloudflare.com/button" alt="Deploy to Cloudflare"></a>
+</p>
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/Moe-Zbeeb/runboard/main/docs/assets/dashboard-dark.png">
+  <img alt="runboard dashboard comparing three training runs" src="https://raw.githubusercontent.com/Moe-Zbeeb/runboard/main/docs/assets/dashboard-light.png">
+</picture>
+
+runboard is a personal, W&B-style experiment dashboard. Each researcher deploys one small backend to
+their own Cloudflare account. Training jobs send metrics directly over HTTPS, so the cluster does not
+need to host a server and the dashboard keeps the same URL when jobs, login nodes, or laptops restart.
+
+- **Two lines in your training code.** `runboard.init(...)` and `runboard.log({...})`.
+- **Bring your own Cloudflare.** One click creates a Worker, D1 database, and static dashboard
+  in your account. No domain or always-on cluster process is required.
+- **One stable URL.** Jobs and browsers connect to the same `workers.dev` HTTPS endpoint from anywhere.
+- **Zero dependencies.** The client, server, and dashboard use only the Python standard library.
+  `pip install runboard` works on any cluster, even behind a restrictive proxy.
+- **Keeps network and disk work out of training.** `log()` appends metrics in memory; a background
+  thread sends batches, retries outages, and spills excess backlog to disk. Remaining metrics are
+  saved when the run finishes.
+- **A local mode when you need it.** The same package can store plain JSONL files and serve them from a
+  cluster or laptop without Cloudflare storage.
+
+## Contents
+
+- [Quickstart](#quickstart)
+- [How it works](#how-it-works)
+- [Logging API](#logging-api)
+- [Cloudflare deployment](#cloudflare-deployment)
+- [Running on a cluster](#running-on-a-cluster)
+- [The dashboard](#the-dashboard)
+- [Configuration](#configuration)
+- [CLI reference](#cli-reference)
+- [Security](#security)
+- [FAQ](#faq)
+- [Development](#development)
+
+## Quickstart
+
+**1. Deploy your personal backend.** Click the button, sign in to Cloudflare, and choose a long random
+`RUNBOARD_TOKEN` when prompted. Cloudflare creates the Worker, database, and dashboard.
+
+[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/Moe-Zbeeb/runboard)
+
+Cloudflare gives you a URL such as `https://runboard.<account>.workers.dev`.
+
+**2. Install and connect the Python client** on each machine or shared cluster home:
+
+```bash
+pip install runboard
+runboard configure https://runboard.<account>.workers.dev
+```
+
+The command asks for the same token, verifies the deployment, and saves both values in
+`~/.runboard/server.json` with mode `600`.
+
+**3. Log from your training script:**
+
+```python
+import runboard
+
+runboard.init(project="resnet", config={"lr": 3e-4, "batch_size": 256})
+
+for step in range(num_steps):
+    loss = train_step()
+    runboard.log({"train/loss": loss}, step=step)
+
+    if step % 500 == 0:
+        runboard.log({"eval/accuracy": evaluate()}, step=step)
+
+runboard.finish()
+```
+
+**4. Open the URL printed by `runboard configure`.** Charts update live every two seconds.
+
+`runboard.init()` reads the saved endpoint automatically. On clusters with a shared home directory,
+configure it once and every compute node uses it. For containers or separate machines, set
+`RUNBOARD_SERVER` and `RUNBOARD_TOKEN` as environment variables.
+
+## How it works
+
+```text
+  cluster / VM                         your Cloudflare account                 anywhere
+ ┌──────────────────┐    HTTPS    ┌───────────────────────────────┐        ┌───────────┐
+ │ train.py         │ ──────────► │ Worker: auth + REST API       │ ◄────► │  browser  │
+ │   runboard.log() │             │ D1: runs + metric batches     │ HTTPS  │ dashboard │
+ │   retry + spool  │             │ Assets: dashboard             │        └───────────┘
+ └──────────────────┘             │                               │
+                                  └───────────────────────────────┘
+```
+
+1. Your script appends metrics to an in-memory buffer. A background thread sends them to the
+   server once per second, in chunks of up to 5,000 rows.
+2. The Cloudflare Worker authenticates the request and stores run metadata and metric batches in D1.
+   Retried batches have deterministic IDs and are stored once.
+3. The dashboard is served by the same Worker. It polls every two seconds and fetches only batches it
+   has not seen.
+4. If Cloudflare or the network is unavailable, the client retries with backoff and writes remaining
+   data to `~/.runboard/spool/`. `runboard sync` sends it later.
+
+More detail is in [docs/architecture.md](docs/architecture.md).
+
+## Logging API
+
+```python
+import runboard
+
+run = runboard.init(
+    project="my-project",      # groups runs in the dashboard
+    name="baseline-lr3e-4",    # display name (defaults to the run id)
+    config={"lr": 3e-4},       # hyperparameters, shown in run details
+    tags=["baseline"],         # searchable in the sidebar
+    run_id=None,               # pass an existing id to resume a run
+    server=None, token=None,   # override server discovery
+    dir=None,                  # write to a shared directory instead of HTTP
+)
+
+runboard.log({"train/loss": 0.42, "train/lr": 1e-4}, step=100)   # step is optional; auto-increments
+runboard.finish()                                                # also runs automatically at exit
+```
+
+| Behavior | Details |
+|---|---|
+| Values | Python numbers, NumPy scalars, and 0-d torch tensors all work. NaN and inf become gaps; non-numeric values are ignored. |
+| Sections | Metrics named `group/name` are grouped into dashboard sections (`train/…`, `eval/…`). |
+| Status | Runs are marked `running`, `finished`, `crashed` (uncaught exception), or `killed` (Ctrl-C). A run that stops logging for 10 minutes is shown as `stale`. |
+| Metadata | Hostname, command line, and `SLURM_JOB_ID` are recorded automatically. |
+| Multi-GPU | Under `torchrun` or DDP, only rank 0 logs. `init()` and `log()` are no-ops where `RANK` or `LOCAL_RANK` is non-zero, so you can call them unconditionally. Pass `all_ranks=True` to log from every rank. |
+| Context manager | `with runboard.Run(project="p") as run: run.log(...)` marks the run `crashed` if the block raises. |
+
+### When the server is unreachable
+
+Network failures never run inside `log()`. If the server is down, rows are kept in memory and retried
+with backoff; excess backlog is saved in the background. Rows still unsent when the job exits are written to
+`~/.runboard/spool/`. Upload them later with:
+
+```bash
+runboard sync
+```
+
+## Cloudflare deployment
+
+The deploy button is the normal path. It uses [Cloudflare's automatic resource provisioning](https://developers.cloudflare.com/workers/platform/deploy-buttons/),
+so every user receives isolated resources in their own account. A custom domain is optional; the
+generated `workers.dev` address is stable and sufficient.
+
+To deploy from a terminal instead:
+
+```bash
+git clone https://github.com/Moe-Zbeeb/runboard && cd runboard
+npm install
+npx wrangler login
+npm run deploy
+```
+
+Generate a token, then store it as a Worker secret when Wrangler prompts for its value:
+
+```bash
+python -c 'import secrets; print(secrets.token_urlsafe(32))'
+npx wrangler secret put RUNBOARD_TOKEN
+```
+
+The Worker creates its schema on the first authenticated request. The SQL migration is also kept in
+`cloudflare/migrations/` for inspection and future upgrades.
+
+For local Worker development, copy `.dev.vars.example` to `.dev.vars`, replace its value, and run
+`npm run dev`. Wrangler keeps local D1 data under `.wrangler/`.
+
+Cloudflare's free plan is enough for personal use and normal research runs, subject to its current
+[Workers](https://developers.cloudflare.com/workers/platform/pricing/),
+[D1](https://developers.cloudflare.com/d1/platform/pricing/) quotas. Large sweeps or very frequent logging can
+exceed those quotas. Runboard batches writes, but log at a useful interval instead of every inner-loop
+operation.
+
+## Running on a cluster
+
+### Recommended: send directly to Cloudflare
+
+If the cluster home is shared, run `runboard configure` once on a login node. Otherwise inject the
+settings into the job:
+
+```bash
+export RUNBOARD_SERVER="https://runboard.<account>.workers.dev"
+export RUNBOARD_TOKEN="your-token"
+python train.py
+```
+
+Only outbound HTTPS is required. There is no Runboard daemon on the login node and no tunnel to keep
+alive.
+
+### Local server mode
+
+When experiments must remain inside the cluster, run the included local server:
+
+Run it in `tmux` or with `nohup` on the login node:
+
+```bash
+nohup runboard serve --tunnel > ~/runboard.log 2>&1 &
+grep public: ~/runboard.log
+```
+
+If your cluster kills long processes on login nodes, run the server as a low-resource Slurm job
+([examples/slurm/serve.sbatch](examples/slurm/serve.sbatch)):
+
+```bash
+sbatch examples/slurm/serve.sbatch
+grep public: runboard-*.out
+```
+
+### Quick-tunnel URL notifications
+
+The quick-tunnel URL changes whenever the server restarts. Use `--notify` to have each new URL pushed
+to your phone through [ntfy](https://ntfy.sh). Install the app and subscribe to a private,
+hard-to-guess topic:
+
+```bash
+runboard serve --tunnel --notify https://ntfy.sh/my-private-topic-8f3k2
+```
+
+### Local server discovery
+
+Nothing extra is needed. Jobs discover the server through `~/.runboard/server.json`. See
+[examples/slurm/train.sbatch](examples/slurm/train.sbatch) and
+[examples/slurm/sweep.sbatch](examples/slurm/sweep.sbatch) (a job-array learning-rate sweep).
+
+### Compute nodes can't reach the login node?
+
+Some clusters block traffic between nodes. If they share a filesystem, skip HTTP and write straight
+into the server's run directory:
+
+```python
+runboard.init(project="resnet", dir="~/runboard-runs")
+```
+
+or set `export RUNBOARD_DIR=~/runboard-runs` in your job script. The server picks the files up.
+
+## The dashboard
+
+- **Runs sidebar.** Search, project filter, and status badges. Click a run's name to see its config and metadata.
+- **Compare runs.** Select up to 8 runs. Each run keeps its color while it stays selected.
+- **One chart per metric.** Hover for exact values, drag to zoom, double-click to reset. Legends show each run's latest value.
+- **Controls.** EMA smoothing, log-scale y, x axis as step, relative time, or wall clock, a regex metric filter, and system/light/dark themes.
+- **Long runs.** Charts retain peaks and dips while reducing very large series to a display-sized sample. Original metrics remain in the JSONL files.
+- **Summary table.** Latest value of every metric for the selected runs.
+- A responsive layout for phones and desktops.
+
+## Configuration
+
+| Environment variable | Default | Purpose |
+|---|---|---|
+| `RUNBOARD_SERVER` | from `~/.runboard/server.json` | Server URL used by `init()` |
+| `RUNBOARD_TOKEN` | from `~/.runboard/server.json` | Access token used by `init()` |
+| `RUNBOARD_DIR` | unset | If set, `init()` writes files to this directory instead of using HTTP |
+| `RUNBOARD_HOME` | `~/.runboard` | Where the token, server info, and spool files live |
+
+## CLI reference
+
+| Command | Description |
+|---|---|
+| `runboard configure URL` | Verify and save a personal Cloudflare or other hosted endpoint. Prompts securely for its token; `--token` is available for automation. |
+| `runboard serve` | Start the server. Options: `--dir` (default `./runboard-runs`), `--port` (default 8080, falls back to a free port if taken), `--host`, `--tunnel`, `--notify URL`, `--advertise URL` (address jobs should use) |
+| `runboard ls [--dir D]` | List runs with status and last update |
+| `runboard sync [files…]` | Upload spooled offline metrics |
+| `runboard url` | Print the current dashboard URL |
+| `runboard --version` | Print the version |
+
+## Security
+
+- Every request, including the dashboard itself, needs the access token. For Cloudflare, each user
+  chooses it during deployment and stores it as a Worker secret. The local server generates one.
+- Opening `/?token=…` stores the token in an `HttpOnly` cookie and removes it from the address bar.
+- The hosted endpoint uses HTTPS. Run metadata and metric batches are stored in D1 inside the user's
+  Cloudflare account.
+- **Anyone with the URL and the token can read your metrics.** Treat the full URL like a password. To
+  rotate a Cloudflare token, run `npx wrangler secret put RUNBOARD_TOKEN` from the repository and
+  re-run `runboard configure`. For a local server, delete `~/.runboard/token` and restart it.
+
+## FAQ
+
+**How is this different from W&B or TensorBoard?**
+W&B is a hosted service, so your metrics leave your infrastructure and you need an account.
+TensorBoard needs port forwarding or a tunnel you set up yourself, and it reads event files.
+runboard gives every user their own small Cloudflare backend and keeps the Python client dependency-free.
+
+**Does it work on clusters with no internet access?**
+Cloudflare mode needs outbound HTTPS from compute nodes. If that is blocked, use local file mode with
+`RUNBOARD_DIR` and `runboard serve`, then access it over the internal network or an SSH tunnel.
+
+**Can I log from any experiment?**
+Yes. Log numeric metrics with any names, group them with `/`, and include any JSON-compatible
+hyperparameters in the run config. The tracker has no framework dependency, so it can live in a plain
+Python loop or a PyTorch, JAX, or other training script. Each backend retains the run's full metric
+history.
+
+**Can I analyze the data myself?**
+Local mode stores every run as `<dir>/<project>/<run_id>/metrics.jsonl`:
+
+```python
+import pandas as pd
+df = pd.read_json("runboard-runs/resnet/20260924-101500-a1b2c3/metrics.jsonl", lines=True)
+```
+
+## Development
+
+```bash
+git clone https://github.com/Moe-Zbeeb/runboard && cd runboard
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pytest
+npm install
+npm run check:cloud
+```
+
+Try it end to end with `runboard serve` in one terminal and `python examples/quickstart.py` in another.
+See [CONTRIBUTING.md](CONTRIBUTING.md) for details.
+
+## License
+
+[MIT](LICENSE). Bundles [uPlot](https://github.com/leeoniya/uPlot) (MIT).