nc-gcode-interpreter 0.2.0__tar.gz → 0.2.2__tar.gz

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/.github/workflows/build-and-release.yml

@@ -43,7 +43,14 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v5
+      # Pin the toolchain and explicitly pull in rustfmt so `cargo fmt --check`
+      # does not depend on whatever components the runner image ships with.
+      - uses: dtolnay/rust-toolchain@stable
+        with:
+          components: rustfmt
       - uses: Swatinem/rust-cache@v2
+      - name: Check Rust formatting
+        run: cargo fmt --check
       - name: Run Rust unit and integration tests
         run: cargo test
 

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/CHANGELOG.md

@@ -4,20 +4,58 @@ Notable changes to **nc-gcode-interpreter**. The format loosely follows
 [Keep a Changelog](https://keepachangelog.com/); versions are git tags,
 released to PyPI.
 
-## [v0.2.0] - 2026-07-07
+## [v0.2.2] - 2026-07-07
 
-### Changed
+### Added
 
-- Rust-side polars is gone. The Table -> Python DataFrame handoff no longer
-  builds a polars DataFrame in Rust (via `pyo3-polars`); it builds an Arrow
-  record batch with the minimal `arrow-array`/`arrow-schema`/`arrow-data`
-  crates and hands it to Python zero-copy through the Arrow PyCapsule
-  interface (`__arrow_c_array__`), where `pl.DataFrame(...)` wraps it. The
-  Python API is unchanged (still returns `polars.DataFrame`), needs no
-  `pyarrow`, and performance is unchanged. This drops ~60 crates from the
-  `python`-feature build (127 -> 64), cutting a clean release build ~4x
-  (83s -> 21s), and bumps PyO3 0.28 -> 0.29 (resolving the RUSTSEC pyo3
-  advisories).
+- Variable-change events on the batch path: `nc_to_batches(...,
+  include_variables=True)` exposes, once exhausted, a sparse `variable_events`
+  DataFrame (`row_idx` / `name_id` / `value`) plus a `variable_names` list -
+  the batch-path twin of the per-row `variables` dict `nc_to_rows` already
+  yields. `row_idx` is the output-row index a change is seen at (a change on a
+  variable-only block is attributed to the next output row), so replaying the
+  events reconstructs the symbol table at any row. Off by default (no cost).
+- `string_table` in the interpreter state dict: `DEF STRING` variables now
+  round-trip into `.state` / the `nc_to_dataframe` state tuple as
+  `state["string_table"]` (`dict[str, str]`), alongside the existing `axes` /
+  `symbol_table` / `translation` numeric tables (previously omitted).
+
+## [v0.2.1] - 2026-07-07
+
+### Added
+
+- Optional `line_no` output column on the batch/dataframe path, enabled with
+  `include_line_numbers=True` on `nc_to_dataframe` / `nc_to_batches` (default
+  `False`, so the output schema is unchanged unless you ask for it). When
+  enabled it prepends a leading `Int64` column giving the 1-based source line
+  each output row came from - previously only the streaming `nc_to_rows`
+  exposed it. Loops repeat the value and jumps make it non-monotonic, matching
+  `nc_to_rows` row-for-row; flatten-generated samples keep the originating
+  block's line number; `dataframe_to_nc` ignores it (source provenance, not an
+  emittable word). Concatenating batches reconstructs the same per-row
+  `line_no` as the whole-file dataframe (#45)
+- Structured error locations: parse/interpret failures now raise
+  `nc_gcode_interpreter.NcError` (a `ValueError` subclass, so existing
+  `except ValueError` keeps working) carrying the position as data - `.line`,
+  `.column` (syntax errors), `.context`, and `.line_text` attributes (each an
+  int / str or `None`) - so a caller (e.g. an editor) can locate the offending
+  token without regex-parsing the message. `str(err)` is unchanged
+
+### Fixed
+
+- Arithmetic-function arity: added a regression test pinning that every
+  `SIN`/`COS`/`ATAN2`/`BOUND`/... call already validates its argument count
+  (`check_args`) before indexing `args[..]`, returning
+  `ParsingError::InvalidFunctionArity` instead of panicking on a wrong-arity
+  call (#16).
+
+### Documented
+
+- Pinned iterator-drop -> `ParsingError::StreamClosed` as the supported
+  cancel contract for `nc_to_rows`/`nc_to_batches` (README, docstrings) and
+  noted the release-profile `strip`/`lto` tradeoff in `Development.md` (#49).
+
+## [v0.2.0] - 2026-07-07
 
 ### Added
 
@@ -95,6 +133,16 @@ released to PyPI.
 
 ### Changed
 
+- Rust-side polars is gone. The Table -> Python DataFrame handoff no longer
+  builds a polars DataFrame in Rust (via `pyo3-polars`); it builds an Arrow
+  record batch with the minimal `arrow-array`/`arrow-schema`/`arrow-data`
+  crates and hands it to Python zero-copy through the Arrow PyCapsule
+  interface (`__arrow_c_array__`), where `pl.DataFrame(...)` wraps it. The
+  Python API is unchanged (still returns `polars.DataFrame`), needs no
+  `pyarrow`, and performance is unchanged. This drops ~60 crates from the
+  `python`-feature build (127 -> 64), cutting a clean release build ~4x
+  (83s -> 21s), and bumps PyO3 0.28 -> 0.29 (resolving the RUSTSEC pyo3
+  advisories).
 - **Breaking:** `I`/`J`/`K`/`CR` are now treated as arc interpolation-parameter
   block addresses (output columns), so they can no longer be used as user
   variable names: `I=5` followed by `X=I+1` was a variable read before and is

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/Cargo.lock

@@ -506,7 +506,7 @@ checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
 
 [[package]]
 name = "nc-gcode-interpreter"
-version = "0.2.0"
+version = "0.2.2"
 dependencies = [
  "arrow-array",
  "arrow-data",

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "nc-gcode-interpreter"
-version = "0.2.0"
+version = "0.2.2"
 edition = "2021"
 readme = "README.md"
 

nc_gcode_interpreter-0.2.2/Development.md

@@ -0,0 +1,61 @@
+# Development
+
+
+Use Cargo to build and run the CLi tool:
+
+```
+cargo run -- --help
+```
+
+
+To compile the python module:
+
+```bash
+maturin develop
+```
+
+## Setup python environment
+
+```bash
+uv venv -p 3.12
+uv synv --all-extras
+```
+
+
+
+## Release
+
+```bash
+cargo build --release
+maturin develop --release --uv
+```
+
+### Release profile
+
+`[profile.release]` currently sets no `strip`/`lto` overrides. `strip = true` was
+tried and dropped (`e1ff63a`): cargo runs the *host* `strip` binary even on a
+cross-compiled artifact, so stripping the cross-built macOS x86_64 dylib with
+the arm64 host `strip` produced a wheel PyPI rejected as "not a zipfile" -
+stripping was reverted to unblock the release. It was a secondary size win
+(mostly orthogonal to the polars removal that motivated it) and can be
+revisited via maturin's cross-aware `--strip` flag instead of a Cargo
+profile setting.
+
+`lto` remains an untried, plausible speed knob (doesn't invoke an external
+tool, so it isn't blocked by the cross-compile issue above) - just not yet
+evaluated for its extra build time across the wheel matrix.
+
+
+## Super simple test
+
+There are a bunch of csv files in the examples directory. To test the tool on all of them (use git to check changes)
+
+```bash
+rm **/*.csv && cargo build --release && find examples -name "*.mpf" -type f -print0 | xargs -0 -I {} sh -c './target/release/nc-gcode-interpreter --axis-index-map E:4 --initial_state=examples/defaults.mpf "$1" || echo "Failed to process $1" >&2' sh {}
+```
+
+## python test
+    
+```bash
+maturin develop --release --uv && pytest
+```

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: nc-gcode-interpreter
-Version: 0.2.0
+Version: 0.2.2
 Classifier: Programming Language :: Rust
 Classifier: Programming Language :: Python :: Implementation :: CPython
 Classifier: Programming Language :: Python :: 3.12

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/README.md

@@ -161,6 +161,8 @@ for line_no, row, variables in nc_to_rows(program, include_variables=True):
 
 Rows are typed and forward-filled like the batch DataFrame (disable with `forward_fill=False`); errors raise from `next()` at the offending row; after exhaustion the iterator's `state` attribute holds the final interpreter state. See `python/example/streaming.py` for a runnable version.
 
+**Cancellation.** Dropping the iterator is the supported way to cancel a `nc_to_rows`/`nc_to_batches` run — leaving a `for` loop early or `del`-ing it drops it immediately; relying on garbage collection works too but only whenever the runtime happens to collect it, so it is not prompt on all interpreters: the background interpreter thread's next attempt to send a row hits a closed channel and returns internally via `ParsingError::StreamClosed`, so the thread unwinds promptly instead of running the rest of the program to completion. No explicit `.close()`/`.cancel()` call is needed or provided — drop is the whole contract, and downstreams (e.g. ribweaver's `/sim` live-edit reinterpret) rely on it for latest-wins cancellation.
+
 Additionally, conversion from a Polars DataFrame back to an MPF (NC) program is also supported:
 
 ```bash

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/python/nc_gcode_interpreter/__init__.py

@@ -3,6 +3,7 @@ import os
 import polars as pl
 from ._internal import nc_to_rows as _nc_to_rows
 from ._internal import nc_to_batches as _nc_to_batches
+from ._internal import NcError
 from ._internal import __doc__  # noqa: F401
 import json
 from pathlib import Path
@@ -21,6 +22,7 @@ __all__ = [
     "nc_to_batches",
     "sanitize_dataframe",
     "dataframe_to_nc",
+    "NcError",
 ]
 
 # Batch size nc_to_dataframe uses internally when draining the batch stream it
@@ -60,6 +62,7 @@ def nc_to_dataframe(
     axis_index_map: dict[str, int] | None = None,
     allow_undefined_variables: bool = False,
     flatten_tolerance: float | None = None,
+    include_line_numbers: bool = False,
 ) -> tuple[pl.DataFrame, dict]:
     """
     Parses Sinumerik-flavored NC G-code and converts it into a Polars DataFrame along with the final state.
@@ -93,6 +96,12 @@ def nc_to_dataframe(
         deviation (in path units, i.e. mm) of the true curve. The interpolation
         parameters (I/J/K/CR, PW/SD/PL) are consumed and do not appear in the
         output. Default None (curves pass through untouched).
+    include_line_numbers: bool, optional
+        If True, prepend a `line_no` (Int64) column giving the 1-based source
+        line each output row came from - repeated under a loop, non-monotonic
+        across a jump, matching the streaming `nc_to_rows`. Never
+        forward-filled; `dataframe_to_nc` ignores it. Default False (column
+        absent, output schema unchanged).
 
     Returns:
     --------
@@ -142,6 +151,7 @@ def nc_to_dataframe(
         allow_undefined_variables,
         input_is_path,
         flatten_tolerance,
+        include_line_numbers,
     )
     # pl.DataFrame wraps each Arrow record batch via __arrow_c_array__ (polars
     # >= 1.3), no pyarrow needed. Exhaust the iterator before reading .state.
@@ -245,6 +255,13 @@ def sanitize_dataframe(
     pl.DataFrame
         The sanitized DataFrame ready for analysis or conversion back to G-code.
     """
+    # `line_no` (present only when interpreted with include_line_numbers=True)
+    # is source-provenance metadata, not an emittable G-code word; drop it up
+    # front so it is never cast/forward-filled or written back out by
+    # dataframe_to_nc.
+    if "line_no" in df.columns:
+        df = df.drop("line_no")
+
     modal = [g["short_name"] for g in GGroups.g_groups if g["effectiveness"] == "modal"]
     non_modal = [g["short_name"] for g in GGroups.g_groups if g["effectiveness"] != "modal"]
     known_axes = [
@@ -415,6 +432,13 @@ def nc_to_rows(
     an anonymous iterator drops it; a stored iterator keeps the worker
     alive (parked on the bounded channel) until it is deleted or
     garbage-collected.
+
+    Dropping the iterator is the **supported cancel contract**: the
+    background thread's next attempted send hits the now-closed channel and
+    returns internally via ``ParsingError::StreamClosed``, so it unwinds
+    promptly instead of finishing the rest of the program. There is no
+    separate ``.cancel()``/``.close()`` - drop is the whole mechanism.
+
     Errors raise ``ValueError`` from ``next()`` when reached. After the
     iterator is exhausted, its ``state`` attribute holds the final
     interpreter state (axes, symbol_table, translation).
@@ -453,8 +477,9 @@ class _BatchIterator:
 
     Wraps the Rust batch iterator, which yields each batch as an Arrow record
     batch built column-wise in Rust and handed over the Arrow PyCapsule
-    interface (``__arrow_c_array__``, via pyo3-arrow) - a zero-copy transfer
-    with no Python list of primitives materialized. ``pl.DataFrame`` wraps that
+    interface (``__arrow_c_array__``, exported directly with the minimal
+    arrow-rs crates - no pyo3-arrow/pyo3-polars) - a zero-copy transfer with
+    no Python list of primitives materialized. ``pl.DataFrame`` wraps that
     capsule directly (polars >= 1.3), so no ``pyarrow`` is involved. After
     exhaustion its ``state`` attribute holds the final interpreter state (axes,
     symbol_table, translation), like the iterator returned by :func:`nc_to_rows`.
@@ -473,6 +498,29 @@ class _BatchIterator:
     def state(self) -> dict | None:
         return self._inner.state
 
+    @property
+    def variable_events(self) -> "pl.DataFrame | None":
+        """Sparse variable-change events (``include_variables=True``).
+
+        A :class:`polars.DataFrame` with columns ``row_idx`` (the output-row
+        index the change is seen at), ``name_id`` (index into
+        :attr:`variable_names`) and ``value``. ``None`` unless the iterator was
+        created with ``include_variables=True``. Available once exhausted.
+        Replaying the events in order (decoding ``name_id`` via
+        :attr:`variable_names`) reconstructs the symbol table at any row, the
+        batch-path twin of the per-row ``variables`` dict ``nc_to_rows`` yields.
+        """
+        batch = self._inner.variable_events
+        return None if batch is None else pl.DataFrame(batch)
+
+    @property
+    def variable_names(self) -> list[str] | None:
+        """Variable names ``variable_events.name_id`` indexes into.
+
+        ``None`` unless ``include_variables=True``; available once exhausted.
+        """
+        return self._inner.variable_names
+
 
 def nc_to_batches(
     input: "TextFileLike | str | os.PathLike",
@@ -485,6 +533,8 @@ def nc_to_batches(
     axis_index_map: dict[str, int] | None = None,
     allow_undefined_variables: bool = False,
     flatten_tolerance: float | None = None,
+    include_line_numbers: bool = False,
+    include_variables: bool = False,
 ) -> _BatchIterator:
     """Interpret an NC program into a stream of columnar polars DataFrames.
 
@@ -504,6 +554,23 @@ def nc_to_batches(
     The returned iterator exposes a ``state`` attribute holding the final
     interpreter state once it is exhausted, like :func:`nc_to_rows`.
 
+    Like :func:`nc_to_rows`, this runs on a background thread behind a
+    bounded channel, and dropping the iterator is the supported way to
+    cancel: the next attempted send hits the closed channel and returns
+    internally via ``ParsingError::StreamClosed``, unwinding the thread
+    promptly instead of finishing the rest of the program.
+
+    With ``include_line_numbers=True`` every batch gains a leading ``line_no``
+    (Int64) column: the 1-based source line each row came from (repeated under
+    a loop, non-monotonic across a jump), never forward-filled. Default False
+    leaves the schema unchanged.
+
+    With ``include_variables=True`` the iterator additionally exposes
+    ``variable_events`` (a sparse :class:`polars.DataFrame` of ``row_idx`` /
+    ``name_id`` / ``value``) and ``variable_names`` once exhausted — the
+    batch-path equivalent of the per-row ``variables`` dict :func:`nc_to_rows`
+    yields; replaying the events reconstructs the symbol table at any row.
+
     Example:
     --------
     >>> batches = nc_to_batches("G1 X10\\nX20 Y5", batch_size=1)
@@ -526,5 +593,7 @@ def nc_to_batches(
         allow_undefined_variables,
         input_is_path,
         flatten_tolerance,
+        include_line_numbers,
+        include_variables,
     )
     return _BatchIterator(inner)

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/python/nc_gcode_interpreter/_internal.pyi

@@ -5,6 +5,15 @@ from typing import Any, Iterator, Optional, List, Dict, Tuple
 # wrapper imports are declared here. Signatures mirror the `#[pyo3(signature)]`
 # defaults in src/lib.rs.
 
+class NcError(ValueError):
+    """NC parse/interpret error carrying structured location data. Subclasses
+    ValueError so `except ValueError` keeps working."""
+
+    line: Optional[int]
+    column: Optional[int]
+    context: Optional[str]
+    line_text: Optional[str]
+
 def nc_to_rows(
     input: str,
     initial_state: Optional[str] = None,
@@ -33,8 +42,16 @@ def nc_to_batches(
     allow_undefined_variables: bool = False,
     input_is_path: bool = False,
     flatten_tolerance: Optional[float] = None,
+    include_line_numbers: bool = False,
+    include_variables: bool = False,
 ) -> Any:
-    """Interpret an NC program into an iterator of columnar polars DataFrames."""
+    """Interpret an NC program into an iterator of columnar polars DataFrames.
+
+    The returned iterator exposes ``state`` (dict with ``axes``,
+    ``symbol_table``, ``translation`` and ``string_table``) once exhausted, and
+    - when ``include_variables`` is set - ``variable_events`` (an Arrow batch of
+    ``row_idx`` / ``name_id`` / ``value``) and ``variable_names`` (list[str]).
+    """
     ...
 
 __all__ = ["nc_to_rows", "nc_to_batches"]

nc_gcode_interpreter-0.2.2/python/tests/test_batch_variables.py

@@ -0,0 +1,205 @@
+"""Batch-path variable-change events (#46) + `string_table` in the state dict.
+
+The streaming `nc_to_rows(include_variables=True)` path yields a per-row
+`variables` dict; the batch path (`nc_to_batches` / `nc_to_dataframe`) now
+surfaces the same information as a sparse side-table (`row_idx`, `name_id`,
+`value`) plus a `variable_names` list. These tests pin (a) parity of the
+ordered assignment sequence between the two paths and (b) that `DEF STRING`
+variables round-trip into the returned state dict.
+"""
+
+import polars as pl
+
+from nc_gcode_interpreter import nc_to_batches, nc_to_dataframe, nc_to_rows
+
+
+def _stream_sequence(program: str) -> list[tuple[str, float]]:
+    """Flat, ordered `(name, value)` assignment sequence from the stream."""
+    return [
+        (name, value)
+        for _line, _row, variables in nc_to_rows(program, include_variables=True)
+        for name, value in variables.items()
+    ]
+
+
+def _batch_sequence(it) -> list[tuple[str, float]]:
+    """Flat, ordered `(name, value)` sequence decoded from the batch events."""
+    events = it.variable_events
+    names = it.variable_names
+    return [
+        (names[name_id], value)
+        for name_id, value in zip(events["name_id"], events["value"])
+    ]
+
+
+def test_batch_variable_events_match_stream_simple():
+    program = "\n".join(
+        [
+            "DEF REAL Q=2.5",  # variable-only
+            "R1=0",            # variable-only
+            "X=R1",            # output row, no change
+            "X=Q",             # output row, no change
+        ]
+    )
+    it = nc_to_batches(program, include_variables=True)
+    list(it)  # exhaust to populate variable_events / state
+    assert _batch_sequence(it) == _stream_sequence(program)
+
+
+def test_batch_variable_events_match_stream_while_loop():
+    # A WHILE loop where R1 changes once per iteration: the per-iteration
+    # counter events must appear on the batch path exactly as they do on the
+    # stream, in the same order.
+    program = "\n".join(
+        [
+            "DEF REAL Q=2.5",
+            "R1=0",
+            "WHILE R1<3",
+            "X=R1 Q=Q*2",  # output row that ALSO assigns a variable
+            "R1=R1+1",     # variable-only, per iteration
+            "ENDWHILE",
+            "X=Q",
+        ]
+    )
+    it = nc_to_batches(program, include_variables=True)
+    list(it)
+    assert _batch_sequence(it) == _stream_sequence(program)
+
+
+def test_batch_variable_events_row_idx_reconstructs_symbol_table():
+    # Replaying every event reconstructs the final symbol table (minus the
+    # built-in TRUE/FALSE), mirroring the streaming accumulation invariant.
+    program = "\n".join(
+        [
+            "DEF REAL Q=1",
+            "R1=0",
+            "WHILE R1<3",
+            "X=R1 Q=Q*2",
+            "R1=R1+1",
+            "ENDWHILE",
+        ]
+    )
+    it = nc_to_batches(program, include_variables=True)
+    list(it)
+    accumulated: dict[str, float] = {}
+    for name, value in _batch_sequence(it):
+        accumulated[name] = value
+    symbol_table = dict(it.state["symbol_table"])
+    for builtin in ("TRUE", "FALSE"):
+        symbol_table.pop(builtin)
+    assert accumulated == symbol_table
+
+    # row_idx is monotonic non-decreasing (events arrive in program order) and
+    # bounded by the number of output rows.
+    events = it.variable_events
+    row_idx = events["row_idx"].to_list()
+    assert row_idx == sorted(row_idx)
+
+
+def test_batch_variable_events_row_idx_aligns_with_output_rows():
+    # A change on a variable-only block is attributed to the NEXT output row;
+    # a change on an output row gets that row's own index.
+    program = "\n".join(
+        [
+            "R1=0",   # variable-only, before output row 0
+            "X=R1",   # output row 0
+            "R1=1",   # variable-only, before output row 1
+            "X=R1",   # output row 1
+        ]
+    )
+    it = nc_to_batches(program, include_variables=True)
+    list(it)
+    events = it.variable_events
+    names = it.variable_names
+    decoded = [
+        (row_idx, names[name_id], value)
+        for row_idx, name_id, value in zip(
+            events["row_idx"], events["name_id"], events["value"]
+        )
+    ]
+    assert decoded == [(0, "R1", 0.0), (1, "R1", 1.0)]
+
+
+def test_batch_variable_events_absent_without_flag():
+    it = nc_to_batches("R1=5\nX=R1")
+    list(it)
+    assert it.variable_events is None
+    assert it.variable_names is None
+
+
+def test_batch_variable_events_empty_when_no_assignments():
+    it = nc_to_batches("G1 X10\nX20 Y5", include_variables=True)
+    list(it)
+    assert isinstance(it.variable_events, pl.DataFrame)
+    assert it.variable_events.height == 0
+    assert it.variable_names == []
+
+
+def test_string_table_round_trips_through_state_dict():
+    program = "\n".join(
+        [
+            "DEF STRING[16] MSG",
+            'MSG="HELLO WORLD"',
+            "DEF STRING[8] TAG",
+            'TAG="ABC"',
+            "G1 X1",
+        ]
+    )
+    _df, state = nc_to_dataframe(program)
+    assert state["string_table"] == {"MSG": "HELLO WORLD", "TAG": "ABC"}
+    # The existing numeric sub-tables are untouched by the new key.
+    assert set(state) == {"axes", "symbol_table", "translation", "string_table"}
+    # The string variable never leaks into the numeric symbol table.
+    assert "MSG" not in state["symbol_table"]
+
+
+def test_string_table_on_streaming_and_batch_iterators():
+    program = 'DEF STRING[8] MSG\nMSG="HI"\nG1 X1'
+    rows = nc_to_rows(program)
+    list(rows)
+    assert rows.state["string_table"] == {"MSG": "HI"}
+
+    it = nc_to_batches(program)
+    list(it)
+    assert it.state["string_table"] == {"MSG": "HI"}
+
+
+def test_line_numbers_and_variables_compose_on_the_batch_path():
+    # include_line_numbers (#45) and include_variables (#46) share the batch
+    # path and were merged independently; requesting both at once must give the
+    # leading `line_no` column AND the sparse variable-event side-table, with
+    # `row_idx` still aligned to the emitted (line_no-carrying) output rows.
+    program = "\n".join(
+        [
+            "R1=0",          # variable-only, before output row 0
+            "WHILE R1<3",
+            "X=R1",          # output row (loop body, source line 3)
+            "R1=R1+1",       # variable-only
+            "ENDWHILE",
+            "X9",            # output row, source line 6
+        ]
+    )
+    it = nc_to_batches(
+        program,
+        batch_size=1_000_000,
+        include_line_numbers=True,
+        include_variables=True,
+    )
+    df = pl.concat(list(it), how="diagonal")
+
+    # line_no leads and tracks source lines (loop body repeats line 3).
+    assert df.columns[0] == "line_no"
+    assert df["line_no"].to_list() == [3, 3, 3, 6]
+    assert df["X"].to_list() == [0.0, 1.0, 2.0, 9.0]
+
+    # Variable events still decode independently of the line_no column.
+    names = it.variable_names
+    decoded = [
+        (row_idx, names[name_id], value)
+        for row_idx, name_id, value in zip(
+            it.variable_events["row_idx"],
+            it.variable_events["name_id"],
+            it.variable_events["value"],
+        )
+    ]
+    assert decoded == [(0, "R1", 0.0), (1, "R1", 1.0), (2, "R1", 2.0), (3, "R1", 3.0)]

{nc_gcode_interpreter-0.2.0 → nc_gcode_interpreter-0.2.2}/python/tests/test_batches.py

@@ -155,3 +155,56 @@ def test_path_input_batches(tmp_path):
     df, _ = nc_to_dataframe(program)
     frames = list(nc_to_batches(mpf, batch_size=2))
     assert_frame_equal(pl.concat(frames, how="diagonal").select(df.columns), df)
+
+
+def test_include_line_numbers_is_opt_in():
+    """`line_no` is off by default and, when enabled, is a leading Int64 column
+    that repeats under a loop and is non-monotonic across a jump - matching the
+    streaming `nc_to_rows` row-for-row and reconstructible across batches."""
+    prog = "R1=0\nWHILE R1<2\nX=R1 F100\nR1=R1+1\nENDWHILE\nX9\n"
+
+    # Default off: schema unchanged, no line_no column anywhere.
+    df_off, _ = nc_to_dataframe(prog)
+    assert "line_no" not in df_off.columns
+    off_batches = list(nc_to_batches(prog, batch_size=1))
+    assert all("line_no" not in f.columns for f in off_batches)
+
+    # Opt-in: leading Int64 line_no with the loop's repeated source lines.
+    df_on, _ = nc_to_dataframe(prog, include_line_numbers=True)
+    assert df_on.columns[0] == "line_no"
+    assert df_on["line_no"].dtype == pl.Int64
+    assert df_on["line_no"].to_list() == [3, 3, 6]
+    # Dropping line_no reproduces the default-off frame exactly.
+    assert_frame_equal(df_on.drop("line_no"), df_off)
+
+    # Batches carry it too and concatenate back to the whole-file column.
+    on_batches = list(nc_to_batches(prog, batch_size=1, include_line_numbers=True))
+    assert pl.concat(on_batches, how="diagonal")["line_no"].to_list() == [3, 3, 6]
+
+
+def test_include_line_numbers_matches_streaming_under_jumps():
+    """A GOTO reorders execution; the batch `line_no` column must match what
+    the streaming `nc_to_rows` yields, line for line."""
+    from nc_gcode_interpreter import nc_to_rows
+
+    prog = "N10 X1\nGOTOF 40\nN30 X999\nN40 X4\n"
+    df, _ = nc_to_dataframe(prog, include_line_numbers=True)
+    streamed = [line_no for line_no, _row in nc_to_rows(prog)]
+    assert df["line_no"].to_list() == streamed == [1, 4]
+
+
+def test_include_line_numbers_survives_flattening():
+    """Flatten-generated samples keep the originating block's source line, so a
+    flattened arc/spline block yields many rows all carrying its line_no - and
+    still matches the streaming path row-for-row."""
+    from nc_gcode_interpreter import nc_to_rows
+
+    program = "G1 X0 Y0 F1000\nG2 X100 Y0 I50 J0\nG1 X100 Y50\n"
+    df, _ = nc_to_dataframe(program, flatten_tolerance=0.1, include_line_numbers=True)
+    assert df.columns[0] == "line_no"
+    assert df["line_no"].null_count() == 0
+    per_line = df["line_no"].to_list()
+    # The arc on line 2 expands to many G1 samples, every one tagged line 2.
+    assert per_line.count(2) > 5
+    stream_lines = [line for line, *_ in nc_to_rows(program, flatten_tolerance=0.1)]
+    assert per_line == stream_lines