expr-tracker 0.2.2__tar.gz → 0.2.3__tar.gz

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: expr_tracker
-Version: 0.2.2
+Version: 0.2.3
 Summary: Local-first experiment tracking with queryable history and expression-based alerts on your training metrics
 Project-URL: Homepage, https://hspk.github.io/expr_tracker/
 Project-URL: Documentation, https://hspk.github.io/expr_tracker/
@@ -106,6 +106,8 @@ install command; it never crashes a run.
 | [Alerts](https://hspk.github.io/expr_tracker/guide/alerts/) | An expression DSL with rolling windows, three-valued logic (no false alarms during warm-up), a rule state machine, and a watchdog that catches a hung run. |
 | [Channels](https://hspk.github.io/expr_tracker/guide/alerts/#channels) | Lark, Slack, DingTalk, WeCom, generic webhook, email — with rate limiting, dedup, retries and per-channel routing. |
 | [Artifacts](https://hspk.github.io/expr_tracker/guide/artifacts/) | Versioned file sets, deduplicated by content, shared across a project's runs, with lineage. |
+| [Spans](https://hspk.github.io/expr_tracker/guide/spans/) | Time the parts of a step, and their parts. Each duration becomes a metric, so alerts and queries work on it unchanged; `et trace` exports the timeline for Perfetto. |
+| [Streams](https://hspk.github.io/expr_tracker/guide/streams/) | Independent producers — a data worker and a training loop — each with their own step cursor and file inside one run. |
 | [Distributed](https://hspk.github.io/expr_tracker/guide/distributed/) | Per-rank shards so concurrent appends cannot corrupt step order; only rank 0 alerts by default. |
 | [CLI](https://hspk.github.io/expr_tracker/guide/cli/) | `et history`, `et rules explain`, `et rules test`, `et alert`. |
 
@@ -148,6 +150,9 @@ uv run ruff format src tests
 | `test_hot_paths.py` | contracts and defaults of `et.log` / `et.history` / summary / alerts |
 | `test_value_encoding.py` | numpy, pydantic, datetime, Path, Enum round trips; output types; query bounds |
 | `test_expr_properties.py` | DSL properties: render round-trip stability, precedence, the whole `M` builder |
+| `test_trace.py` | Chrome Trace export: lane layout, stream and step selection, the CLI |
+| `test_spans.py` | nesting, aggregation, decorator and async forms, errors, thread and task isolation |
+| `test_streams.py` | stream naming and validation, isolation, resolution order, backend grouping, two-process runs |
 | `test_distributed.py` | rank shards, `alert_on_rank`, real multi-process runs |
 | `test_wandb.py` | real wandb in offline mode: parameter mapping, step alignment, artifacts |
 | `test_trackio.py` | trackio contract, resume mapping, real end-to-end |

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/README.md

@@ -59,6 +59,8 @@ install command; it never crashes a run.
 | [Alerts](https://hspk.github.io/expr_tracker/guide/alerts/) | An expression DSL with rolling windows, three-valued logic (no false alarms during warm-up), a rule state machine, and a watchdog that catches a hung run. |
 | [Channels](https://hspk.github.io/expr_tracker/guide/alerts/#channels) | Lark, Slack, DingTalk, WeCom, generic webhook, email — with rate limiting, dedup, retries and per-channel routing. |
 | [Artifacts](https://hspk.github.io/expr_tracker/guide/artifacts/) | Versioned file sets, deduplicated by content, shared across a project's runs, with lineage. |
+| [Spans](https://hspk.github.io/expr_tracker/guide/spans/) | Time the parts of a step, and their parts. Each duration becomes a metric, so alerts and queries work on it unchanged; `et trace` exports the timeline for Perfetto. |
+| [Streams](https://hspk.github.io/expr_tracker/guide/streams/) | Independent producers — a data worker and a training loop — each with their own step cursor and file inside one run. |
 | [Distributed](https://hspk.github.io/expr_tracker/guide/distributed/) | Per-rank shards so concurrent appends cannot corrupt step order; only rank 0 alerts by default. |
 | [CLI](https://hspk.github.io/expr_tracker/guide/cli/) | `et history`, `et rules explain`, `et rules test`, `et alert`. |
 
@@ -101,6 +103,9 @@ uv run ruff format src tests
 | `test_hot_paths.py` | contracts and defaults of `et.log` / `et.history` / summary / alerts |
 | `test_value_encoding.py` | numpy, pydantic, datetime, Path, Enum round trips; output types; query bounds |
 | `test_expr_properties.py` | DSL properties: render round-trip stability, precedence, the whole `M` builder |
+| `test_trace.py` | Chrome Trace export: lane layout, stream and step selection, the CLI |
+| `test_spans.py` | nesting, aggregation, decorator and async forms, errors, thread and task isolation |
+| `test_streams.py` | stream naming and validation, isolation, resolution order, backend grouping, two-process runs |
 | `test_distributed.py` | rank shards, `alert_on_rank`, real multi-process runs |
 | `test_wandb.py` | real wandb in offline mode: parameter mapping, step alignment, artifacts |
 | `test_trackio.py` | trackio contract, resume mapping, real end-to-end |

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/docs/design.md

@@ -75,7 +75,48 @@ last line. Patch lines and `step_policy="allow"` both let a file end on a lower
 and continuing from that would reuse an existing one. `max_step` is maintained
 incrementally and persisted in the sidecar.
 
-### A.4 Multiple processes
+### A.4 Streams
+
+Independent producers — a training loop and a data worker — have unrelated step
+semantics, so each gets its own file, cursor and resume state under one run
+directory: `metrics[.stream][.rankN].jsonl`, plus per-stream `summary` and
+`config` sidecars so concurrent processes cannot clobber each other.
+
+The store is instantiated once per stream rather than made multi-tenant, so every
+invariant below holds unchanged within a stream; only the filename differs.
+
+`resolve_run_path()` matches names exactly rather than taking the first glob hit:
+`metrics.data.jsonl` sorts before `metrics.jsonl`, so sort order would silently
+return a stream in place of the default producer.
+
+A stream is forwarded to a backend as its own run, grouped under the run name.
+Neither wandb nor trackio can merge two step axes into one run; wandb's shared
+mode can, but requires a live server and has no trackio equivalent.
+
+### A.5 Spans
+
+`et.span` times a region and its children. A closed span takes the same ingress as
+`log()` — `HistoryStore.ingest()` — so the step policy, the open-row merge and the
+commit rules exist in one place regardless of which API produced the metrics.
+
+Two differences from a log call:
+
+- the merge **accumulates** rather than replaces, because the same span can run
+  many times in one step;
+- the metrics are built here from floats and ints, so they skip `RecordCodec`.
+  That encoder costs 5 µs, which a per-sub-step call cannot afford.
+
+A span never commits a step: durations ride along with whatever `log()` commits,
+so timing a region adds no row. The tree, with timestamps and attributes, goes to
+`spans[.stream][.rankN].jsonl` through a second `JsonlWriter`, enqueued rather
+than appended so a span does not pay for a flush decision of its own.
+
+The nesting stack is a `ContextVar`, not a `threading.local`: a new thread starts
+from the default and each asyncio task gets its own copy, which is exactly the
+scoping spans need. (The run singleton went the other way, from `ContextVar` to a
+locked global, because there the goal was for worker threads to *share* it.)
+
+### A.6 Multiple processes
 
 The supported model is **rank 0 tracks**; there is no cross-rank merging.
 

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/docs/guide/cli.md

@@ -22,6 +22,25 @@ et history tracker/jsonl/demo/run-1 -n -1 --format csv > run.csv
 
 The argument is a run directory or a `metrics.jsonl` file.
 
+## `et trace`
+
+Export recorded [spans](spans.md) as a Chrome Trace, for
+[Perfetto](https://ui.perfetto.dev) or `chrome://tracing`.
+
+```bash
+et trace runs/llm/sft-1                              # every stream -> trace.json
+et trace runs/llm/sft-1 -o pipeline.json
+et trace runs/llm/sft-1 --stream data                # one stream
+et trace runs/llm/sft-1 --stream default,data        # several
+et trace runs/llm/sft-1 --step-range 100:200         # a slice of the run
+```
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `-o`, `--output` | `trace.json` | where to write |
+| `--stream` | every stream | comma separated; `default` is the unnamed one |
+| `--step-range` | all | `start:end`, end exclusive |
+
 ## `et rules explain`
 
 Show how an expression parses, and what it references. Useful when precedence is in

expr_tracker-0.2.3/docs/guide/spans.md

@@ -0,0 +1,169 @@
+# Spans
+
+A step is rarely one thing. `et.span` times the parts, and their parts, and turns
+each duration into an ordinary metric — so `history()`, alert rules and plots all
+work on it with no extra machinery.
+
+```python
+for batch in loader:
+    with et.span("forward"):
+        with et.span("attention"):
+            attn = attention(batch)
+        with et.span("mlp"):
+            out = mlp(attn)
+    with et.span("backward"):
+        loss.backward()
+    et.log({"loss": loss.item()})
+```
+
+The step's row then carries:
+
+```jsonc
+{"_step": 42, "_time": ...,
+ "forward/duration_ms": 31.2,  "forward/count": 1,
+ "forward/attention/duration_ms": 18.4, "forward/attention/count": 1,
+ "forward/mlp/duration_ms": 9.1, "forward/mlp/count": 1,
+ "backward/duration_ms": 44.7, "backward/count": 1,
+ "loss": 0.31}
+```
+
+Nested names join with `/`, so `forward/norm` and `backward/norm` stay distinct,
+and the alert language reads them directly.
+
+## Forms
+
+```python
+with et.span("forward"): ...                # context manager
+
+async with et.span("fetch"): ...            # async
+
+@et.span("preprocess")                      # decorator, sync or async
+def preprocess(batch): ...
+
+span = et.start_span("epoch")               # manual, for crossing scopes
+...
+span.end()
+```
+
+## Repeated spans
+
+The same span several times in one step sums, and counts:
+
+```python
+for layer in layers:            # 32 layers
+    with et.span("layer"):
+        x = layer(x)
+```
+
+```jsonc
+{"layer/duration_ms": 412.8, "layer/count": 32}
+```
+
+The total is usually what you want; divide by the count for the mean.
+
+## A span never commits a step
+
+Durations join whatever `log()` commits, so a span costs no row of its own and
+you can time things before you know what to log:
+
+```python
+with et.span("forward"):
+    ...
+# nothing written yet
+et.log({"loss": loss})     # one row, with the metrics and the durations
+```
+
+## Attributes
+
+Attributes describe an individual span. They reach `spans.jsonl`, not the metrics,
+because they are usually not numbers:
+
+```python
+with et.span("load", batch_size=32) as span:
+    rows = read()
+    span.set(rows=len(rows))
+```
+
+## Alerting on durations
+
+Because a duration is a metric, the [expression language](../reference/expressions.md)
+already handles it:
+
+```python
+et.init(..., alert_rules=[
+    "mean(data/load_ms[50]) > 200 => warning: data loading is slowing down",
+    "forward/duration_ms > 3 * mean(forward/duration_ms[100]) => error: slow step",
+])
+```
+
+## The span file
+
+The full tree is written to `spans.jsonl` beside the metrics, one line per span:
+
+```jsonc
+{"_step": 42, "name": "forward/attention", "depth": 1,
+ "start": 1754323200.123456, "dur_ms": 18.4, "args": {"batch_size": 32}}
+```
+
+Children appear before their parents, because a parent closes last. It follows
+the [stream](streams.md): a data worker writes `spans.data.jsonl`.
+
+Turn it off if you only want the metrics:
+
+```python
+et.init(..., spans=False)
+```
+
+## Viewing the timeline
+
+`et trace` turns the span file into a Chrome Trace, which
+[Perfetto](https://ui.perfetto.dev) and `chrome://tracing` open directly:
+
+```bash
+et trace runs/llm/sft-1 -o trace.json
+et trace runs/llm/sft-1 --stream data --step-range 100:200
+```
+
+Each stream becomes a process and each thread a track, so a data worker and a
+training loop sit on one timeline and the gap where one waited for the other is
+visible. Spans keep their nesting, their step and their attributes.
+
+Exporting a standard format rather than drawing our own view means the result
+can be loaded beside a `torch.profiler` trace, which is usually where the real
+question is: what were the GPUs doing while the loader stalled.
+
+## Errors
+
+An exception is recorded and re-raised — the span never swallows it:
+
+```jsonc
+{"_step": 42, "name": "risky", "dur_ms": 3.1, "error": "ValueError"}
+```
+
+A failure inside the recording itself is logged and ignored: measuring something
+must not break it.
+
+## Threads and tasks
+
+The nesting stack is per thread and per asyncio task, so concurrent work does not
+nest inside unrelated spans:
+
+```python
+async def work(name):
+    async with et.span(name):        # "a" and "b", never "a/b"
+        await asyncio.sleep(1)
+
+await asyncio.gather(work("a"), work("b"))
+```
+
+## Cost
+
+| | per span |
+| --- | --- |
+| default | ~13 µs |
+| `spans=False` | ~6 µs |
+| no active run | ~3 µs |
+
+For comparison, `et.log()` is ~24 µs. Twenty spans on a 100 ms step is 0.26% of
+the step. If your step is closer to a millisecond, set `spans=False` and keep the
+metrics, or time fewer regions.

expr_tracker-0.2.3/docs/guide/streams.md

@@ -0,0 +1,113 @@
+# Streams
+
+A training loop and a data worker measure different things on different clocks:
+training step 100 and data batch 100 are unrelated. A **stream** gives each
+producer its own step cursor and its own file inside one run directory.
+
+```python
+# the training process
+et.init(project="llm", name="sft-1")
+et.log({"train/loss": loss})                    # steps 0, 1, 2, ...
+
+# the data worker, a separate process
+et.init(project="llm", name="sft-1", stream="data")
+et.log({"data/produce_ms": 12.4})               # its own steps 0, 1, 2, ...
+```
+
+Without this, both producers share one cursor: their steps interleave, the merge
+puts unrelated metrics on one row, and `step_policy="monotonic"` drops whichever
+producer happens to fall behind.
+
+## Layout
+
+```
+runs/llm/sft-1/
+├── metrics.jsonl            # the default producer
+├── metrics.data.jsonl       # stream "data"
+├── metrics.meta.json
+├── metrics.data.meta.json
+├── summary.json             # per stream, so processes cannot clobber each other
+├── summary.data.json
+├── config.json
+└── config.data.json
+```
+
+Streams compose with [rank shards](distributed.md): rank 1 of the data worker
+writes `metrics.data.rank1.jsonl`.
+
+Stream names become part of a filename, so they must be letters, digits, `_` or
+`-`, starting with a letter or digit. `rank1` and friends are rejected because
+they already mean a rank shard.
+
+## Reading
+
+```python
+et.history(50)                       # whichever stream this process writes
+et.history(50, stream=None)          # the default producer
+et.history(50, stream="data")        # another stream
+et.history(50, run=path, stream="data")   # offline
+```
+
+Omitting `stream` reads the running stream; passing `stream=None` explicitly means
+the default, unnamed one. Another process's stream is read from its file, so you
+see what it has flushed rather than what it has buffered.
+
+```python
+from expr_tracker.history import list_streams
+list_streams("runs/llm/sft-1")       # [None, "data"]
+```
+
+## Alerts
+
+Each process alerts on what it can see, which is its own stream. That is usually
+what you want — the data worker is the thing that knows its pipeline stalled:
+
+```python
+et.init(
+    project="llm", name="sft-1", stream="data",
+    alert_rules=[
+        "produce_ms > 1000        => warning: data pipeline slow",
+        "no_data(5m)              => error: data worker stopped producing",
+    ],
+)
+```
+
+!!! note
+    A rule cannot span streams that live in different processes, because neither
+    process holds the other's metrics. If you need that, log both from one
+    process, or evaluate the rule downstream against the files.
+
+## Backends
+
+A stream is forwarded as its own backend run, grouped under the run name:
+
+```python
+et.init(project="llm", name="sft-1", stream="data", backends=["wandb"])
+# -> wandb.init(name="sft-1-data", group="sft-1", job_type="data")
+```
+
+Both wandb and trackio understand `group`, and neither can merge two step axes
+into a single run. wandb's shared mode can, but it needs a live server and has no
+trackio equivalent, so grouping is the default.
+
+Override it per backend if you want something else:
+
+```python
+et.init(..., stream="data", backend_kwargs={"wandb": {"group": "my-group"}})
+```
+
+## When you do not need a stream
+
+If the producers are in **one process at different cadences** — an eval loop
+every 100 steps, say — you do not need a stream. Log with the training step and
+the sparse metric simply appears on the steps where you logged it:
+
+```python
+et.log({"train/loss": loss})
+if step % 100 == 0:
+    et.log({"eval/acc": acc}, step=step)
+```
+
+Window functions already work on that, because `eval.acc[20]` counts points of
+that metric, not rows. Reach for a stream when producers are genuinely concurrent
+and their step numbers mean different things.

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/docs/reference/api.md

@@ -23,6 +23,7 @@ et.init(
     backends: Sequence[str] = ("wandb", "jsonl"),
     backend_kwargs: dict[str, dict] | None = None,
     print_to_screen: bool = False,
+    stream: str | None = None,
     alert=None,
     alert_rules: Sequence = (),
     **history_options,
@@ -77,9 +78,13 @@ et.history(
     fill_missing: bool = False,
     dropna: bool = False,
     run: str | Path | None = None,
+    stream: str | None = ...,
 )
 ```
 
+Omit `stream` to read the running one; pass `None` for the default producer. See
+[Streams](../guide/streams.md).
+
 `n=-1` or `None` returns everything. With `run=`, reads that run offline and no
 `init()` is needed. See [Querying history](../guide/history.md).
 
@@ -148,6 +153,27 @@ et.register_backend(kind: str, cls: type[AlertBackend])
 
 Registers a custom channel type.
 
+## Spans
+
+```python
+with et.span(name, **attributes) as span: ...   # also async, also a decorator
+span = et.start_span(name, **attributes)        # ends with span.end()
+span.set(**attributes)
+span.duration_ms
+```
+
+A closed span adds `<path>/duration_ms` and `<path>/count` to the open row and
+appends the full record to `spans.jsonl`. See [Spans](../guide/spans.md).
+
+### Trace export
+
+```python
+from expr_tracker.trace import build_trace, write_trace
+
+write_trace(run, "trace.json", stream="*", step_range=None)   # returns the span count
+build_trace(run, stream="*")                                  # the dict, unwritten
+```
+
 ## Artifacts
 
 ```python

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/docs/reference/configuration.md

@@ -13,6 +13,8 @@ list of valid options rather than being silently ignored.
 | `max_open_seconds` | 60.0 | commit an idle open row after this long; `None` disables |
 | `step_policy` | `"monotonic"` | `"monotonic"` drops backward steps, `"allow"` keeps them |
 | `rank_aware` | `True` | non-zero ranks write their own shard |
+| `stream` | `None` | an independent producer with its own file and step cursor |
+| `spans` | `True` | write the span tree to `spans.jsonl` beside the metrics |
 | `print_to_screen` | `False` | print every committed row |
 | `print_handle` | `print` | where those lines go |
 | `buffer_size` | 50 | flush after this many buffered rows |

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/mkdocs.yml

@@ -34,6 +34,8 @@ nav:
       - Querying history: guide/history.md
       - Alerts: guide/alerts.md
       - Artifacts: guide/artifacts.md
+      - Spans: guide/spans.md
+      - Streams: guide/streams.md
       - Distributed runs: guide/distributed.md
       - Backends: guide/backends.md
       - CLI: guide/cli.md

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/src/expr_tracker/__init__.py

@@ -19,6 +19,7 @@ from .alerts import (
 )
 from .artifacts import Artifact
 from .run import Run
+from .spans import Span, span, start_span
 
 try:
     __version__ = _version("expr_tracker")
@@ -45,6 +46,7 @@ __all__ = [
     "ChannelConfig",
     "M",
     "Run",
+    "Span",
     "WebhookPolicy",
     "__version__",
     "add_alert_rule",
@@ -61,6 +63,8 @@ __all__ = [
     "log_artifact",
     "register_backend",
     "remove_alert_rule",
+    "span",
+    "start_span",
     "summary",
     "use_artifact",
 ]

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/src/expr_tracker/cli.py

@@ -13,6 +13,7 @@ from .alerts.engine import AlertEngine
 from .alerts.expr import EvalContext, compile_condition, parse_rule, validate
 from .alerts.models import AlertConfig, ChannelConfig, WebhookPolicy
 from .history import MetricSeries, read_history
+from .trace import write_trace
 
 
 @click.group()
@@ -54,6 +55,29 @@ def history(run: str, n: int, metrics: str | None, step_range: str | None, fmt:
         click.echo(_to_table(rows))
 
 
+@main.command()
+@click.argument("run", type=click.Path(exists=True))
+@click.option("-o", "--output", default="trace.json", help="Where to write the trace")
+@click.option(
+    "--stream",
+    default=None,
+    help="Comma separated streams; every stream by default. Use 'default' for the "
+    "unnamed one.",
+)
+@click.option("--step-range", default=None, help="start:end (end exclusive)")
+def trace(run: str, output: str, stream: str | None, step_range: str | None):
+    """Export recorded spans as a Chrome Trace.
+
+    Open the result at https://ui.perfetto.dev or chrome://tracing. It can be
+    loaded beside a torch.profiler trace.
+    """
+    wanted: str | list[str] | None = "*"
+    if stream:
+        wanted = [None if s in ("default", "") else s for s in stream.split(",")]
+    count = write_trace(run, output, wanted, step_range=_parse_range(step_range))
+    click.echo(f"wrote {count} span(s) to {output}")
+
+
 @main.group()
 def rules():
     """Alert rule tooling."""

{expr_tracker-0.2.2 → expr_tracker-0.2.3}/src/expr_tracker/history/__init__.py

@@ -1,8 +1,15 @@
 from .codec import RecordCodec
 from .frame import project, to_output
-from .reader import JsonlReader, merge_steps, read_history, resolve_run_path
+from .naming import current_rank, metrics_filename, parse_stream, validate_stream
+from .reader import (
+    JsonlReader,
+    list_streams,
+    merge_steps,
+    read_history,
+    resolve_run_path,
+)
 from .series import MetricSeries
-from .store import HistoryStore, current_rank, resolve_commit
+from .store import HistoryStore, resolve_commit
 from .writer import JsonlWriter
 
 __all__ = [
@@ -12,10 +19,14 @@ __all__ = [
     "MetricSeries",
     "RecordCodec",
     "current_rank",
+    "list_streams",
     "merge_steps",
+    "metrics_filename",
+    "parse_stream",
     "project",
     "read_history",
     "resolve_commit",
     "resolve_run_path",
     "to_output",
+    "validate_stream",
 ]

expr_tracker-0.2.3/src/expr_tracker/history/naming.py

@@ -0,0 +1,74 @@
+"""Run file naming: distributed rank shards and independent producer streams.
+
+Kept separate from the store and the reader because both need it, and the store
+already depends on the reader.
+"""
+
+from __future__ import annotations
+
+import os
+import re
+
+STREAM_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_-]*$")
+RANK_PATTERN = re.compile(r"rank\d+")
+
+
+def current_rank() -> int:
+    """Distributed rank from the usual environment variables, 0 when unset."""
+    rank = os.getenv("RANK") or os.getenv("LOCAL_RANK") or "0"
+    try:
+        return int(rank)
+    except ValueError:
+        return 0
+
+
+def validate_stream(stream: str) -> str:
+    """Check a stream name is safe as a filename component.
+
+    The name becomes part of ``metrics.<stream>[.rankN].jsonl``, so it must not
+    contain a separator, and must not look like a rank shard.
+    """
+    if not STREAM_PATTERN.match(stream):
+        raise ValueError(
+            f"Invalid stream name {stream!r}; use letters, digits, '_' or '-', "
+            "starting with a letter or digit."
+        )
+    if RANK_PATTERN.fullmatch(stream):
+        raise ValueError(
+            f"Stream name {stream!r} collides with the rank shard naming; "
+            "pick another name."
+        )
+    return stream
+
+
+def metrics_filename(stream: str | None, rank_aware: bool = True) -> str:
+    """``metrics[.stream][.rankN].jsonl``.
+
+    Streams separate independent producers; non-zero ranks get their own shard so
+    concurrent appends cannot interleave. The two compose.
+    """
+    rank = current_rank() if rank_aware else 0
+    parts = ["metrics"]
+    if stream:
+        parts.append(stream)
+    if rank > 0:
+        parts.append(f"rank{rank}")
+    return ".".join(parts) + ".jsonl"
+
+
+def spans_filename(stream: str | None, rank_aware: bool = True) -> str:
+    """``spans[.stream][.rankN].jsonl``, alongside the metrics file."""
+    return metrics_filename(stream, rank_aware).replace("metrics", "spans", 1)
+
+
+def sidecar_filename(base: str, stream: str | None, suffix: str) -> str:
+    """``<base>[.stream].<suffix>``, so producers do not overwrite each other."""
+    return f"{base}.{stream}.{suffix}" if stream else f"{base}.{suffix}"
+
+
+def parse_stream(filename: str) -> str | None:
+    """The stream a metrics filename belongs to; ``None`` is the default producer."""
+    parts = filename.split(".")[1:-1]  # drop "metrics" and "jsonl"
+    if parts and RANK_PATTERN.fullmatch(parts[-1]):
+        parts = parts[:-1]
+    return parts[0] if parts else None