trainmeter 0.0.2__tar.gz → 0.0.3__tar.gz

{trainmeter-0.0.2 → trainmeter-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: trainmeter
-Version: 0.0.2
+Version: 0.0.3
 Summary: Where your training FLOPs go and how well the GPUs are used: loss, FLOPs, MFU and hardware counters for a wrapped training run.
 Project-URL: Homepage, https://github.com/Almaz-KG/trainmeter
 Project-URL: Repository, https://github.com/Almaz-KG/trainmeter
@@ -31,13 +31,20 @@ trainmeter starts your command, watches the node from outside, and serves a live
 The dashboard shows what a training run needs to be judged: training and validation loss, FLOPs spent and required, MFU with its FLOPs convention spelled out, and how busy the GPUs really are, from `GPU-Util` down to MFU, next to arithmetic intensity.
 Your training code stays as it is.
 
-Status: early scaffold, version 0.0.1.
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/images/dashboard-dark.png">
+  <img alt="The trainmeter dashboard: progress, throughput, MFU and OFU, goodput and the loss curve of a run on four H100s" src="docs/images/dashboard-light.png">
+</picture>
+
+<sub>The dashboard on a synthetic run.</sub>
+
+Status: early scaffold.
 `tm` runs any command untouched, records the run, and turns what a PyTorch loop does into tokens, FLOPs spent and required, ETA, MFU (given a device) and the losses you `emit()`.
 GPU counters (NVML and GPM: the utilization ladder, OFU, observed FLOPs, arithmetic intensity) and `tm doctor` are built and tested against fakes, not yet on a real GPU.
 Series your loop already logs to wandb, TensorBoard or MLflow are picked up without a code change (`--no-tap` turns it off), and DataLoader waits, checkpoint saves and eval spans feed a "where the time goes" view.
 One real step is also counted with `FlopCounterMode` (shown beside MFU as the `counted` convention), and layers and width are read from a model config when there is one.
 Each run ends with a run passport (`passport.md` and `passport.json`) for a model card.
-The dashboard (Cockpit, Efficiency, Hardware, Where the time goes, Plan vs actual and Run passport) is served on localhost while the run goes; it has been checked in a browser on synthetic data, not on a real run.
+The dashboard, one page of progress, tiles and charts with the run details and model card folded away, is served on localhost while the run goes; it has been checked in a browser on synthetic data, not on a real run.
 Not yet used on a real run.
 To try it, `examples/` has a tiny GPT to run under `tm` on a laptop and a run planner that needs no GPU.
 

{trainmeter-0.0.2 → trainmeter-0.0.3}/README.md

@@ -7,13 +7,20 @@ trainmeter starts your command, watches the node from outside, and serves a live
 The dashboard shows what a training run needs to be judged: training and validation loss, FLOPs spent and required, MFU with its FLOPs convention spelled out, and how busy the GPUs really are, from `GPU-Util` down to MFU, next to arithmetic intensity.
 Your training code stays as it is.
 
-Status: early scaffold, version 0.0.1.
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/images/dashboard-dark.png">
+  <img alt="The trainmeter dashboard: progress, throughput, MFU and OFU, goodput and the loss curve of a run on four H100s" src="docs/images/dashboard-light.png">
+</picture>
+
+<sub>The dashboard on a synthetic run.</sub>
+
+Status: early scaffold.
 `tm` runs any command untouched, records the run, and turns what a PyTorch loop does into tokens, FLOPs spent and required, ETA, MFU (given a device) and the losses you `emit()`.
 GPU counters (NVML and GPM: the utilization ladder, OFU, observed FLOPs, arithmetic intensity) and `tm doctor` are built and tested against fakes, not yet on a real GPU.
 Series your loop already logs to wandb, TensorBoard or MLflow are picked up without a code change (`--no-tap` turns it off), and DataLoader waits, checkpoint saves and eval spans feed a "where the time goes" view.
 One real step is also counted with `FlopCounterMode` (shown beside MFU as the `counted` convention), and layers and width are read from a model config when there is one.
 Each run ends with a run passport (`passport.md` and `passport.json`) for a model card.
-The dashboard (Cockpit, Efficiency, Hardware, Where the time goes, Plan vs actual and Run passport) is served on localhost while the run goes; it has been checked in a browser on synthetic data, not on a real run.
+The dashboard, one page of progress, tiles and charts with the run details and model card folded away, is served on localhost while the run goes; it has been checked in a browser on synthetic data, not on a real run.
 Not yet used on a real run.
 To try it, `examples/` has a tiny GPT to run under `tm` on a laptop and a run planner that needs no GPU.
 

{trainmeter-0.0.2 → trainmeter-0.0.3}/docs/architecture.md

@@ -341,42 +341,36 @@ Overlays are computed from the primitives at view time and never stored.
 
 The dashboard is one static page fed by the read-only API of section 10.
 It renders from the catalog, so a new series appears without a page change.
-The views, the rendering rules and the x-axis switch are specified in `docs/spec.md` (Screens).
+The page, the rendering rules and the x-axis switch are specified in `docs/spec.md` (Screens).
 Plotting loss against FLOPs makes runs comparable across batch sizes and sequence lengths.
-Cockpit, Efficiency and Hardware arrive in slice 4, Where the time goes in slice 5, and the run passport and Plan vs actual in slice 7.
+The views of slices 4, 5 and 7 were later folded into this one page without tabs.
 
-Layout sketches, with placeholders instead of numbers:
+Layout sketch, with placeholders instead of numbers:
 
 ```text
 +----------------------------------------------------------------------+
-| <command> | <n> x <GPU> | FLOPs per token <value> (<convention>)     |
-+----------------------------------+-----------------------------------+
-| PROGRESS                         | THROUGHPUT                        |
-| FLOPs spent of FLOPs required    | tokens/s and step time            |
-| tokens seen, ETA                 | data wait share of the step       |
-+----------------------------------+-----------------------------------+
-| TRAIN LOSS and VAL LOSS          | MFU and OFU                       |
-| x axis: step, tokens, FLOPs      | the gap between them              |
-| or wall time                     | latest utilization ladder         |
-+----------------------------------+-----------------------------------+
-| LR, GRAD NORM and every other series the loop logs                   |
+| trainmeter  <n> x <GPU>             [step|tokens|FLOPs|time]  <state>|
++----------------------------------------------------------------------+
+| PROGRESS <percent>     tokens, FLOPs of required, ETA, elapsed       |
++-----------+-----------+-----------+-----------+-----------+----------+
+| tokens/s  | step time | MFU       | OFU       | goodput   | FLOPs/tok|
+| sparkline | sparkline | sparkline | sparkline | sparkline |          |
++-----------+-----------+-----------+-----------+-----------+----------+
+| TRAINING: loss (full width), tokens/s, MFU and OFU, lr, grad norm,   |
+| every other logged series, plan against actual                       |
++----------------------------------------------------------------------+
+| GPU EFFICIENCY: ladder now and over time, roofline, arithmetic       |
+| intensity, FLOPs spent (modeled next to observed)                    |
++----------------------------------------------------------------------+
+| NODE: time split, host CPU and memory                                |
+| GPUS: one small chart per counter, one line per GPU; ranks           |
++----------------------------------------------------------------------+
+| > Run details (collapsed)    > Model card (collapsed)                |
 +----------------------------------------------------------------------+
 ```
 
-```text
-+----------------------------------+-----------------------------------+
-| UTILIZATION LADDER over time     | ROOFLINE                          |
-| GPU-Util, SM active,             | arithmetic intensity against      |
-| tensor active, OFU, MFU          | achieved FLOP/s, with the ridge   |
-|                                  | and the peak of the device        |
-+----------------------------------+-----------------------------------+
-| FLOPs SPENT                      | ARITHMETIC INTENSITY over time    |
-| modeled next to observed         | aggregate over the window         |
-+----------------------------------+-----------------------------------+
-```
-
-The optional price field is the one place where money exists.
-It computes dollars in the browser from `elapsed`, `gpu_hours` and `tokens_seen`, keeps the price in that browser, and never sends it to the supervisor or writes it to the run log.
+The dashboard shows no price and no dollars.
+Money stays an overlay over `elapsed`, `gpu_hours` and `tokens_seen`, and a price never reaches the supervisor or the run log.
 
 ## 8. Run lifecycle
 

{trainmeter-0.0.2 → trainmeter-0.0.3}/docs/spec.md

@@ -73,7 +73,7 @@ Rules:
 
 1. Unknown stays unknown.
    A metric that lacks an input is absent and names the input that would enable it, and a zero is never shown in its place.
-2. An assumed fact carries a visible badge.
+2. An assumed fact is marked `assumed` in the run details, and the dashboard raises a notice for it.
    A declared value that disagrees with a detected one is used, and a warning shows both.
 3. `n_gpus` is the number of GPUs the run uses, taken from the ranks.
    Falling back to the visible devices is an assumed fact with a warning.
@@ -107,7 +107,7 @@ The engine maps the raw key the loop used to a canonical series:
 | `lr` | `lr`, `learning_rate`, `train/lr`, `train/learning_rate` |
 | `grad_norm` | `grad_norm`, `train/grad_norm`, `gradient_norm` |
 
-A mapped series carries a `mapped by name` badge and its original key.
+A mapped series shows its original key, with `mapped by name` in its tooltip.
 `tm.toml` can pin or override a mapping.
 A number that matches no alias still appears as a custom series under its own key.
 
@@ -186,7 +186,7 @@ Across ranks: the spread of step time, for straggler detection.
 Overlays are computed from the primitives at view time and never stored in the run log.
 Dollars are GPU-hours times a price per GPU-hour, or elapsed hours times a price per node-hour.
 GPU energy is the integral of the sampled power.
-A price is typed into the dashboard, stays in that browser, and is never sent to the supervisor.
+The dashboard shows no dollars, and a price is never sent to the supervisor.
 
 ## Run log
 
@@ -257,23 +257,33 @@ The run ends when the program does, with no exit status, because `tm` did not st
 
 ## Screens
 
-| Screen | Shows | Answers |
+The dashboard is one page with no tabs.
+Its header holds only the icon, the rig (GPU count and model), the x-axis switch and the run state.
+Below it, top to bottom:
+
+| Part | Shows | Answers |
 | --- | --- | --- |
-| Cockpit | progress (FLOPs spent of required, tokens, ETA), train and validation loss, tokens/s and step time, MFU and OFU with their gap, lr, grad norm and every other logged series | Is the run on plan? |
-| Efficiency | the utilization ladder over time, MFU against OFU, arithmetic intensity over time, the roofline point, FLOPs spent (modeled next to observed) | Are the GPUs used well, and where does the gap open? |
-| Hardware | per GPU: tensor, DRAM and SM active, power, clocks, memory, temperature, NVLink, stragglers; and the process tree's CPU (cores busy) and resident memory | Which card is slow? |
-| Where the time goes | node time split: boot, startup and compile, data, compute, NCCL, checkpoints, eval, idle | What did the node spend time on without training? |
-| Run passport | architecture, FLOPs per token, hardware, topology, versions, final losses | What goes into the model card? |
-| Plan vs actual | the declared budget and sizing assumptions (MFU, time) next to measured values | How wrong was my plan? |
+| Progress | percent of the token budget, tokens and FLOPs spent of required, ETA, elapsed and GPU-hours | Is the run on plan? |
+| Tiles | tokens/s, step time, MFU, OFU, goodput and FLOPs per token, each a number with a sparkline | How is it going right now? |
+| Training | train and validation loss, tokens/s, MFU against OFU, lr, grad norm, every other logged series, and plan against actual | Is the model learning, and how wrong was my plan? |
+| GPU efficiency | the utilization ladder now and over time, the roofline point, arithmetic intensity over time, FLOPs spent (modeled next to observed) | Are the GPUs used well, and where does the gap open? |
+| Node | node time split (startup and compile, data, checkpoints, eval, compute), and the process tree's CPU (cores busy) and resident memory | What did the node spend time on without training? |
+| GPUs | per GPU: tensor, DRAM and SM active, power, clocks, memory, temperature, NVLink, and step time per rank | Which card is slow? |
+| Run details (collapsed) | the command, host, directory, declared flags, model, FLOPs per token under every convention, hardware and versions | What exactly ran? |
+| Model card (collapsed) | the run passport as Markdown, with a copy button | What goes into the model card? |
+
+A chart with no data is left out, and a part with nothing to draw is one line that names what would fill it.
 
 Rendering rules:
 
-1. Every number shows its convention and provenance, and `assumed` and `mapped by name` carry a visible badge.
-2. A metric that lacks an input names the input that would enable it, such as "budget not declared: pass --budget-tokens", and never shows a zero.
+1. Every MFU and FLOPs number shows its convention next to it in plain text, and a series mapped by name shows its raw key.
+   Provenance (`declared`, `assumed`, `measured`) is listed next to each fact in the run details, and a guessed input also raises a notice at the top of the page.
+   A tile is a label, a number, one short line and a sparkline, with no badges; any longer explanation is in its tooltip.
+2. A metric that lacks an input shows "n/a", names the input that would enable it in its tooltip, such as "budget not declared: pass --budget-tokens", and never shows a zero.
 3. Arithmetic intensity is labeled as an aggregate over the sampling window.
 4. Every time series has an x-axis switch: step, tokens, FLOPs spent (modeled, with its convention shown) or wall time.
    Until a step has been seen, and for a program watched from outside, the charts use wall time, because every point would otherwise sit at zero.
-5. The header has an optional price field that adds dollars spent and dollars per billion tokens, as described under Overlays.
+5. The dashboard shows no price and no dollars.
 6. The dashboard is static files with hand-written SVG charts, no build step and no CDN.
 
 ## MVP

{trainmeter-0.0.2 → trainmeter-0.0.3}/src/trainmeter/__init__.py

@@ -4,7 +4,7 @@ from .emit import emit
 from .flops import CONVENTIONS, ModelShape, all_conventions, flops_per_token
 from .peaks import Peak, lookup
 
-__version__ = "0.0.2"
+__version__ = "0.0.3"
 
 __all__ = [
     "CONVENTIONS",

{trainmeter-0.0.2 → trainmeter-0.0.3}/src/trainmeter/web/server.py

@@ -1,6 +1,6 @@
 """The dashboard server: standard library only, read only, loopback only.
 
-`GET /` and its three static files, and a small JSON API over `LiveState`. There is no endpoint
+`GET /`, its static files and the icon, and a small JSON API over `LiveState`. There is no endpoint
 that changes anything, other methods get 405, and a Host header that is not the loopback name and
 this port is refused, which closes DNS rebinding. Binding beyond loopback is not offered.
 """
@@ -25,6 +25,9 @@ FILES = {
     "/": ("index.html", "text/html; charset=utf-8"),
     "/app.js": ("app.js", "text/javascript; charset=utf-8"),
     "/style.css": ("style.css", "text/css; charset=utf-8"),
+    "/favicon.svg": ("favicon.svg", "image/svg+xml"),
+    # Asked for by clients that ignore the <link>; every current browser renders an SVG here.
+    "/favicon.ico": ("favicon.svg", "image/svg+xml"),
 }
 DEFAULT_PORT = 8765
 STREAM_INTERVAL = 1.0