expr-tracker 0.1.8__tar.gz → 0.2.0__tar.gz

expr_tracker-0.2.0/.github/workflows/docs.yaml

@@ -0,0 +1,53 @@
+name: Docs
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "docs/**"
+      - "mkdocs.yml"
+      - ".github/workflows/docs.yaml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    name: Build site
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v5
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+        with:
+          enable-cache: true
+
+      - name: Build
+        run: uv run --group docs mkdocs build --strict
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: site
+
+  deploy:
+    name: Deploy to GitHub Pages
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy
+        id: deployment
+        uses: actions/deploy-pages@v4

{expr_tracker-0.1.8 → expr_tracker-0.2.0}/.gitignore

@@ -11,3 +11,6 @@ wandb/
 # Virtual environments
 .venv
 test.ipynb
+.coverage
+htmlcov/
+site/

expr_tracker-0.2.0/PKG-INFO

@@ -0,0 +1,143 @@
+Metadata-Version: 2.4
+Name: expr_tracker
+Version: 0.2.0
+Summary: Add your description here
+Author-email: HSPK <whxway@whu.edu.cn>
+Requires-Python: >=3.10
+Requires-Dist: click>=8.1.0
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: pydantic>=2.0
+Provides-Extra: all
+Requires-Dist: pandas>=1.5; extra == 'all'
+Requires-Dist: polars>=0.20; extra == 'all'
+Requires-Dist: slark>=0.1.28; extra == 'all'
+Requires-Dist: trackio>=0.4.0; extra == 'all'
+Requires-Dist: wandb>=0.21.0; extra == 'all'
+Provides-Extra: lark
+Requires-Dist: slark>=0.1.28; extra == 'lark'
+Provides-Extra: pandas
+Requires-Dist: pandas>=1.5; extra == 'pandas'
+Provides-Extra: polars
+Requires-Dist: polars>=0.20; extra == 'polars'
+Provides-Extra: trackio
+Requires-Dist: trackio>=0.4.0; extra == 'trackio'
+Provides-Extra: wandb
+Requires-Dist: wandb>=0.21.0; extra == 'wandb'
+Description-Content-Type: text/markdown
+
+# Experiment Tracker
+
+A local-first experiment tracker. Metrics land in a JSONL file you own, stay
+queryable while the run is live, and can trigger alerts from an expression language.
+`wandb` and `trackio` are optional mirrors, not requirements.
+
+📖 **Documentation: <https://hspk.github.io/expr_tracker/>**
+
+```python
+import expr_tracker as et
+
+et.init(project="demo", name="run-1", alert_rules=["zscore(loss[50]) > 3 => error: spike"])
+for step in range(1000):
+    et.log({"loss": loss, "lr": lr})
+et.finish()
+
+et.history(50)                    # the last 50 steps, as dicts
+et.history(-1, output_type="pd")  # everything, as a DataFrame
+```
+
+## Why
+
+- **The file is the source of truth.** One JSON object per step, appended to
+  `metrics.jsonl`. No server, no database, no vendor.
+- **History is queryable during the run.** `et.history(n)` answers from an in-memory
+  cache and touches the file only for what it evicted — 227&nbsp;µs for
+  `history(50)` whether the run has 1,000 steps or 100,000.
+- **Alerts are expressions, not callbacks.** `zscore(loss[50]) > 3 or isnan(loss)`
+  is parsed, validated, and evaluated against a rolling window. Rules can be replayed
+  over a finished run to tune thresholds before you trust them.
+- **It stays out of the way.** `log()` costs ~26&nbsp;µs. A failed disk, a dead
+  webhook or an unserialisable value degrades with a warning; none can stop training.
+
+## Install
+
+```bash
+uv add expr_tracker                 # local-first: click, loguru, pydantic only
+uv add "expr_tracker[wandb]"        # mirror to Weights & Biases
+uv add "expr_tracker[trackio]"      # mirror to trackio
+uv add "expr_tracker[lark]"         # Feishu/Lark alert channel
+uv add "expr_tracker[pandas]"       # history(output_type="pandas")
+uv add "expr_tracker[all]"          # everything
+```
+
+Only the JSONL history is built in. A missing extra is reported with the exact
+install command; it never crashes a run.
+
+## Features
+
+| | |
+| --- | --- |
+| [Logging](https://hspk.github.io/expr_tracker/guide/logging/) | One line per step. Several `log()` calls for one step merge into one row, wandb-compatible `step`/`commit` semantics, numpy and pydantic values handled. |
+| [History](https://hspk.github.io/expr_tracker/guide/history/) | `et.history(n)` during or after the run, offline reads of any run directory, dict/pandas/polars output, bounded in-memory cache with observable hit rate. |
+| [Alerts](https://hspk.github.io/expr_tracker/guide/alerts/) | An expression DSL with rolling windows, three-valued logic (no false alarms during warm-up), a rule state machine, and a watchdog that catches a hung run. |
+| [Channels](https://hspk.github.io/expr_tracker/guide/alerts/#channels) | Lark, Slack, DingTalk, WeCom, generic webhook, email — with rate limiting, dedup, retries and per-channel routing. |
+| [Artifacts](https://hspk.github.io/expr_tracker/guide/artifacts/) | Versioned file sets, deduplicated by content, shared across a project's runs, with lineage. |
+| [Distributed](https://hspk.github.io/expr_tracker/guide/distributed/) | Per-rank shards so concurrent appends cannot corrupt step order; only rank 0 alerts by default. |
+| [CLI](https://hspk.github.io/expr_tracker/guide/cli/) | `et history`, `et rules explain`, `et rules test`, `et alert`. |
+
+## wandb compatibility
+
+Migrating an existing script is usually one line:
+
+```python
+# import wandb as et
+import expr_tracker as et
+```
+
+`init`, `log`, `finish`, `alert`, `log_artifact`, `use_artifact`, `Artifact`,
+`define_metric`, `run.summary`, `run.step`, `run.dir` and `run.url` keep their wandb
+names and signatures. See the
+[compatibility table](https://hspk.github.io/expr_tracker/guide/backends/#wandb-compatibility).
+
+## Development
+
+```bash
+uv sync --all-extras
+uv run pytest                                   # everything
+uv run pytest -m "not slow and not benchmark"   # the fast suite
+uv run pytest -m benchmark -s                   # timing and memory report
+uv run pytest --cov=expr_tracker                # coverage
+uv run ruff check src tests
+uv run ruff format src tests
+```
+
+### Test layout
+
+| File | Covers |
+| --- | --- |
+| `test_history`, `test_expr_*`, `test_alert_*`, `test_writer_durability`, … | per-module unit tests |
+| `test_correctness.py` | value and type round trips, randomised commit sequences, ordering invariants |
+| `test_cache.py` | that the cache really serves reads: zero-IO assertions, eviction boundaries, warm/cold parity |
+| `test_failure_modes.py` | degradation: write failures, read-only dirs, encoder blow-ups, dead alert backends |
+| `test_e2e.py` | full runs, resume, crash recovery, offline reads, CLI |
+| `test_scenarios.py` | live cross-process reads, alerts during eviction, out-of-order resume |
+| `test_hot_paths.py` | contracts and defaults of `et.log` / `et.history` / summary / alerts |
+| `test_value_encoding.py` | numpy, pydantic, datetime, Path, Enum round trips; output types; query bounds |
+| `test_expr_properties.py` | DSL properties: render round-trip stability, precedence, the whole `M` builder |
+| `test_distributed.py` | rank shards, `alert_on_rank`, real multi-process runs |
+| `test_wandb.py` | real wandb in offline mode: parameter mapping, step alignment, artifacts |
+| `test_trackio.py` | trackio contract, resume mapping, real end-to-end |
+| `test_lark_live.py` | Lark channel; real delivery when `ET_LARK_TEST_WEBHOOK` is set |
+| `test_stress.py` (`slow`) | 100k-row writes, concurrency, cache thrash, write-failure recovery |
+| `test_benchmark.py` (`benchmark`) | throughput, tail latency, query cost, memory stability |
+
+### Docs
+
+```bash
+uv run --group docs mkdocs serve    # preview at localhost:8000
+uv run --group docs mkdocs build    # build into site/
+```
+
+Published to GitHub Pages by `.github/workflows/docs.yaml` on every push to `main`.
+Internals: [`docs/design.md`](docs/design.md) (data model and key invariants) and
+[`docs/architecture.md`](docs/architecture.md) (module map, read/write paths,
+concurrency model).

expr_tracker-0.2.0/README.md

@@ -0,0 +1,116 @@
+# Experiment Tracker
+
+A local-first experiment tracker. Metrics land in a JSONL file you own, stay
+queryable while the run is live, and can trigger alerts from an expression language.
+`wandb` and `trackio` are optional mirrors, not requirements.
+
+📖 **Documentation: <https://hspk.github.io/expr_tracker/>**
+
+```python
+import expr_tracker as et
+
+et.init(project="demo", name="run-1", alert_rules=["zscore(loss[50]) > 3 => error: spike"])
+for step in range(1000):
+    et.log({"loss": loss, "lr": lr})
+et.finish()
+
+et.history(50)                    # the last 50 steps, as dicts
+et.history(-1, output_type="pd")  # everything, as a DataFrame
+```
+
+## Why
+
+- **The file is the source of truth.** One JSON object per step, appended to
+  `metrics.jsonl`. No server, no database, no vendor.
+- **History is queryable during the run.** `et.history(n)` answers from an in-memory
+  cache and touches the file only for what it evicted — 227&nbsp;µs for
+  `history(50)` whether the run has 1,000 steps or 100,000.
+- **Alerts are expressions, not callbacks.** `zscore(loss[50]) > 3 or isnan(loss)`
+  is parsed, validated, and evaluated against a rolling window. Rules can be replayed
+  over a finished run to tune thresholds before you trust them.
+- **It stays out of the way.** `log()` costs ~26&nbsp;µs. A failed disk, a dead
+  webhook or an unserialisable value degrades with a warning; none can stop training.
+
+## Install
+
+```bash
+uv add expr_tracker                 # local-first: click, loguru, pydantic only
+uv add "expr_tracker[wandb]"        # mirror to Weights & Biases
+uv add "expr_tracker[trackio]"      # mirror to trackio
+uv add "expr_tracker[lark]"         # Feishu/Lark alert channel
+uv add "expr_tracker[pandas]"       # history(output_type="pandas")
+uv add "expr_tracker[all]"          # everything
+```
+
+Only the JSONL history is built in. A missing extra is reported with the exact
+install command; it never crashes a run.
+
+## Features
+
+| | |
+| --- | --- |
+| [Logging](https://hspk.github.io/expr_tracker/guide/logging/) | One line per step. Several `log()` calls for one step merge into one row, wandb-compatible `step`/`commit` semantics, numpy and pydantic values handled. |
+| [History](https://hspk.github.io/expr_tracker/guide/history/) | `et.history(n)` during or after the run, offline reads of any run directory, dict/pandas/polars output, bounded in-memory cache with observable hit rate. |
+| [Alerts](https://hspk.github.io/expr_tracker/guide/alerts/) | An expression DSL with rolling windows, three-valued logic (no false alarms during warm-up), a rule state machine, and a watchdog that catches a hung run. |
+| [Channels](https://hspk.github.io/expr_tracker/guide/alerts/#channels) | Lark, Slack, DingTalk, WeCom, generic webhook, email — with rate limiting, dedup, retries and per-channel routing. |
+| [Artifacts](https://hspk.github.io/expr_tracker/guide/artifacts/) | Versioned file sets, deduplicated by content, shared across a project's runs, with lineage. |
+| [Distributed](https://hspk.github.io/expr_tracker/guide/distributed/) | Per-rank shards so concurrent appends cannot corrupt step order; only rank 0 alerts by default. |
+| [CLI](https://hspk.github.io/expr_tracker/guide/cli/) | `et history`, `et rules explain`, `et rules test`, `et alert`. |
+
+## wandb compatibility
+
+Migrating an existing script is usually one line:
+
+```python
+# import wandb as et
+import expr_tracker as et
+```
+
+`init`, `log`, `finish`, `alert`, `log_artifact`, `use_artifact`, `Artifact`,
+`define_metric`, `run.summary`, `run.step`, `run.dir` and `run.url` keep their wandb
+names and signatures. See the
+[compatibility table](https://hspk.github.io/expr_tracker/guide/backends/#wandb-compatibility).
+
+## Development
+
+```bash
+uv sync --all-extras
+uv run pytest                                   # everything
+uv run pytest -m "not slow and not benchmark"   # the fast suite
+uv run pytest -m benchmark -s                   # timing and memory report
+uv run pytest --cov=expr_tracker                # coverage
+uv run ruff check src tests
+uv run ruff format src tests
+```
+
+### Test layout
+
+| File | Covers |
+| --- | --- |
+| `test_history`, `test_expr_*`, `test_alert_*`, `test_writer_durability`, … | per-module unit tests |
+| `test_correctness.py` | value and type round trips, randomised commit sequences, ordering invariants |
+| `test_cache.py` | that the cache really serves reads: zero-IO assertions, eviction boundaries, warm/cold parity |
+| `test_failure_modes.py` | degradation: write failures, read-only dirs, encoder blow-ups, dead alert backends |
+| `test_e2e.py` | full runs, resume, crash recovery, offline reads, CLI |
+| `test_scenarios.py` | live cross-process reads, alerts during eviction, out-of-order resume |
+| `test_hot_paths.py` | contracts and defaults of `et.log` / `et.history` / summary / alerts |
+| `test_value_encoding.py` | numpy, pydantic, datetime, Path, Enum round trips; output types; query bounds |
+| `test_expr_properties.py` | DSL properties: render round-trip stability, precedence, the whole `M` builder |
+| `test_distributed.py` | rank shards, `alert_on_rank`, real multi-process runs |
+| `test_wandb.py` | real wandb in offline mode: parameter mapping, step alignment, artifacts |
+| `test_trackio.py` | trackio contract, resume mapping, real end-to-end |
+| `test_lark_live.py` | Lark channel; real delivery when `ET_LARK_TEST_WEBHOOK` is set |
+| `test_stress.py` (`slow`) | 100k-row writes, concurrency, cache thrash, write-failure recovery |
+| `test_benchmark.py` (`benchmark`) | throughput, tail latency, query cost, memory stability |
+
+### Docs
+
+```bash
+uv run --group docs mkdocs serve    # preview at localhost:8000
+uv run --group docs mkdocs build    # build into site/
+```
+
+Published to GitHub Pages by `.github/workflows/docs.yaml` on every push to `main`.
+Internals: [`docs/design.md`](docs/design.md) (data model and key invariants) and
+[`docs/architecture.md`](docs/architecture.md) (module map, read/write paths,
+concurrency model).

expr_tracker-0.2.0/docs/architecture.md

@@ -0,0 +1,214 @@
+# Architecture
+
+How the pieces fit together, and which module owns what. Pair this with
+[`design.md`](design.md), which records *why* the data model looks the way it does.
+
+## Module map
+
+```
+expr_tracker/
+├── __init__.py        public names re-exported for `import expr_tracker as et`
+├── tracker.py         the functional API (init/log/history/finish/artifacts/summary)
+├── run.py             Run object + global singleton + backend fan-out
+├── summary.py         run.summary mapping, persisted as summary.json
+├── artifacts.py       Artifact + project-scoped ArtifactStore
+├── encoders.py        JSON coercion (numpy/torch/pydantic/dataclasses/...)
+├── cli.py             `et history` / `et rules` / `et alert`
+├── history/
+│   ├── store.py       HistoryStore: open-row assembly, cache, query planning
+│   ├── writer.py      JsonlWriter: buffered append, sparse index, meta sidecar
+│   ├── reader.py      JsonlReader + the offline read_history entry points
+│   ├── codec.py       RecordCodec: metric values -> JSON lines, warn-once
+│   ├── series.py      MetricSeries: per-metric numeric buffers
+│   └── frame.py       projection + dict/pandas/polars output
+└── alerts/
+    ├── __init__.py    public alert API + config resolution + engine wiring
+    ├── models.py      AlertLevel/Message/ChannelConfig/WebhookPolicy/AlertRule
+    ├── engine.py      rule compilation, state machine, watchdog
+    ├── dispatch.py    routing, rate limiting, dedup, retry, async worker
+    ├── backends/      lark, slack, dingtalk, wecom, webhook, email, callable
+    └── expr/          lexer, parser, AST nodes + builder, functions, evaluator
+```
+
+## Dependency direction
+
+```
+tracker.py ──> run.py ──> history/   (always)
+                    └──> alerts/     (lazily, to avoid an import cycle)
+                    └──> artifacts.py, summary.py
+alerts/expr ──> history/series.py    (read-only: evaluation needs metric windows)
+```
+
+Rules:
+
+* `history/` never imports from `alerts/`. The alert engine reads `MetricSeries`,
+  which lives in `history/` because it is populated on the write path.
+* `alerts/` never imports `run` at module scope; `alerts/__init__` imports it inside
+  functions so `run.py` can import `alerts` lazily without a cycle.
+* `expr/` is self-contained apart from `MetricSeries`, so the DSL can be parsed,
+  validated and replayed without a live run (this is what `et rules test` uses).
+
+## Write path
+
+```
+et.log(data, step, commit)
+        │
+        ▼
+Run.log ──> Summary.observe(data)          # last value per metric
+        └──> HistoryStore.log
+                 │  encode once (jsonable_encoder)
+                 │  merge into the open row for the current step
+                 ▼  commit when the step advances / commit=True / finish / timeout
+             HistoryStore._emit
+                 ├── _store_row(record, line)         (holds the store lock)
+                 │      ├── cache.append((row, step, line))  # ordinal allocated here
+                 │      ├── MetricSeries.add(...)            # feeds alert evaluation
+                 │      └── JsonlWriter.enqueue(step, row, line)
+                 ├── writer.flush() / schedule_timer  (outside the lock)
+                 ├── _evict()
+                 └── _notify(record)                  # screen + alert engine
+                 │
+                 ▼
+             on_commit ──> AlertEngine.on_step(record) ──> Dispatcher (async)
+        └──> other backends (wandb, trackio, custom)
+```
+
+The ordinal allocation and the writer enqueue happen under one lock so that
+row ordinal *N* is always physical line *N*; the query planner depends on it.
+
+`HistoryStore.log` returns the `_step` the metrics landed on, or `None` when the call
+was rejected. `Run.log` forwards that resolved step (and the resolved commit flag) to
+the remote backends, so a backend's row layout matches the local history instead of
+drifting on its own counter — two `log()` calls for one step stay one step everywhere.
+Test the result with `is None`, since step 0 is falsy.
+
+`log()` never branches on the open row directly: `_switch_open_row(step)` points it
+at the requested step (or reuses it when no step is given) and hands back whatever row
+that displaced. `JsonlWriter._track_line` is likewise the single place that folds a line
+into the counters and index, shared by live appends and by resume rescans — so the
+two can never drift apart.
+
+## Read path
+
+```
+et.history(n, ...)
+        │
+        ▼
+HistoryStore.get ──> _collect ──> _collect_tail | _collect_range
+        │                 │              │
+        │                 │              ├─ _view_*   one lock: cache rows + boundary
+        │                 │              └─ _older_*  only when the view is incomplete
+        │                 ├─ _open_rows   the uncommitted row, if wanted and in range
+        │                 └─ _take_steps  merge by step, keep the newest n
+        ▼
+frame.project(...)  ──>  frame.to_output(dict | pandas | polars)
+```
+
+Both query kinds share one shape: take a `_CacheView` under a single lock, and touch
+the disk only when the view reports it cannot answer on its own.
+
+```python
+view = self._view_tail(steps)          # or _view_range(step_range)
+records = view.records()
+if view.complete:
+    return records
+return self._older_tail(view, steps, records) + records
+```
+
+`_CacheView.complete` folds together the three reasons the disk can be skipped: the
+cache holds an older step, nothing was ever evicted, or there is no writer. The
+scans themselves (`_scan_tail`, `_scan_range`, `_nearer_front`, `_newest_steps`) are
+plain functions over a sequence, so they hold no lock and are tested directly.
+
+`et.history(run=...)` bypasses the store entirely and reads through `JsonlReader`
+(`read_history`), which is what makes offline analysis and `et rules test` possible.
+
+### Rows versus steps
+
+A step normally occupies one physical row, but a `max_open_seconds` timeout followed
+by more data for that step writes a *patch line*, so one step can span several rows.
+The two are kept strictly separate in the read path:
+
+* `JsonlReader.tail_rows(n)` returns **physical rows**; `JsonlReader.tail(n)` returns
+  **merged steps** and widens its own read until it has one whole step to spare.
+* `HistoryStore._collect_tail(steps)` works in rows, then `_take_steps` merges and
+  trims by *step*, so `get(n)` never returns a half-merged oldest row.
+* The cache stores each row's step next to its bytes, so `_view_tail` snapshots the
+  exact rows covering *n* steps without parsing any JSON, and stops as soon as one
+  older step proves nothing is truncated.
+* `parse_rows` drops lines that are corrupt or carry no integer `_step`, so every
+  record a reader returns can be ordered and merged by step.
+
+### Query cost
+
+Queries never scan the whole cache:
+
+| query | cost |
+| --- | --- |
+| `history(n)` | O(rows returned), independent of cache size |
+| `history(step_range=...)` | O(distance from the nearer end of the cache) |
+| `history(-1)` | O(run length) — it has to materialise everything |
+
+Disk fallback only happens once rows have been evicted or the run was resumed;
+`stats()["disk_prefix"]` reports whether that is the case.
+
+## Concurrency model
+
+| Lock / thread | Owner | Protects |
+| --- | --- | --- |
+| `HistoryStore._lock` (RLock) | store | open row, cache, series, row ordinal, writer enqueue |
+| `JsonlWriter._lock` (RLock) | writer | buffer, index, meta fields |
+| `JsonlWriter._write_lock` | writer | batch swap + file append (ordering) |
+| open-row `threading.Timer` | store | commits a stale open row; carries a generation token |
+| buffer `threading.Timer` | writer | flushes records that sat in memory too long |
+| `CompiledRule.lock` | engine | one rule's state machine transitions |
+| dispatch worker thread | dispatcher | drains the send queue; drained at exit |
+| watchdog thread | engine | evaluates time-based rules when no logs arrive |
+
+Lock order is always `_write_lock` → `_lock`; nothing acquires them the other way.
+
+## Extension points
+
+| I want to add... | Where it goes |
+| --- | --- |
+| a notification channel | subclass `AlertBackend`, call `register_backend("type", cls)` |
+| an expression function | add to `expr/functions.py` (`WINDOW_FUNCS` / `SCALAR_FUNCS` / `SPECIAL_FUNCS`) |
+| an output format | add a branch in `history/frame.to_output` |
+| a metrics backend | pass any object with `init/log/finish` in `backends=[...]` |
+| an artifact storage mode | extend `ArtifactStore._materialise` (default is `copy`: `link` shares the caller's inode) |
+| a value type to encode | extend `history/codec.py` (and `encoders.py` for the coercion) |
+| a history tunable | add a field to `HistoryOptions`; it is validated and documented automatically |
+
+## Configuration
+
+Every history tunable lives on `HistoryOptions`, a frozen dataclass validated once in
+`HistoryStore.init()`. Unknown names raise `TypeError` listing the valid ones, so a
+typo like `cache_byte=` fails loudly instead of silently keeping the default. Run
+state is initialised in exactly one place, `HistoryStore._reset()`, which both
+`__init__` and `init()` call — re-initialising cannot leave a field behind.
+
+## Invariants
+
+See [`design.md` §G](design.md). The short version: metadata is written last,
+undurable rows are never evicted, the cache/disk boundary is addressed by row
+ordinal, alerts are evaluated once per committed step, and expression evaluation
+degrades to `UNKNOWN` instead of raising.
+
+## Naming
+
+The write path reads as one sentence, so the stages are named after what they do to
+the open row:
+
+```
+_accept_step → _switch_open_row → _update_open_row → _close_open_row → _emit
+```
+
+Two words are reserved and mean exactly one thing each:
+
+| word | meaning |
+| --- | --- |
+| **merge** | combining rows that share a `_step` (`merge_steps`, `_needs_merge`). Never used for folding metrics into a row — that is `_update_open_row`. |
+| **row** | one physical JSONL line. A step may span several rows, so `tail_rows`/`parse_rows` return rows while `tail`/`parse` return merged steps. |
+
+`stats()` distinguishes `rows_on_disk` (lines in the file) from `rows_logged`
+(ordinals this process handed out); they differ only after records are dropped.