trace-lanefix 0.1.0__tar.gz

trace_lanefix-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Roman Ralovets
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

trace_lanefix-0.1.0/PKG-INFO

@@ -0,0 +1,105 @@
+Metadata-Version: 2.4
+Name: trace-lanefix
+Version: 0.1.0
+Summary: Fix overlapping Chrome/Kineto trace slices so Perfetto can render them.
+Author: Roman Ralovets
+License-Expression: MIT
+Project-URL: Repository, https://github.com/ralovets/trace-lanefix
+Project-URL: Issues, https://github.com/ralovets/trace-lanefix/issues
+Keywords: chrome-trace,kineto,perfetto,profiling,pytorch
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Debuggers
+Classifier: Topic :: System :: Benchmark
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Dynamic: license-file
+
+# trace-lanefix
+
+`trace-lanefix` rewrites Chrome/Kineto trace JSON so Perfetto can render slices
+that would otherwise be dropped because of crossing overlaps on the same
+`(pid, tid)` track.
+
+It keeps properly nested slices on the same track, snaps tiny timestamp-boundary
+crossings, and moves genuine overlaps to synthetic lanes with collision-free
+thread ids.
+
+## Before / After
+
+Before:
+
+<img src="https://raw.githubusercontent.com/ralovets/trace-lanefix/main/assets/profile-before.png" alt="Profile before lane fix" width="100%">
+
+After:
+
+<img src="https://raw.githubusercontent.com/ralovets/trace-lanefix/main/assets/profile-after.png" alt="Profile after lane fix" width="100%">
+
+In Perfetto's trace overview, the rewritten trace reports `TRACE ERRORS=0`
+instead of `TRACE ERRORS=6`, while `IMPORT ERRORS` and `DATA LOSSES` remain
+at `0`. In this example, the JSON trace size changed from `452.76 KiB` before
+rewriting (`463,629` bytes) to `653.96 KiB` after rewriting (`669,651`
+bytes) because the output is pretty-printed for readability.
+
+## CLI
+
+```bash
+trace-lanefix input.json output.fixed.json
+```
+
+Tiny crossings below `5.0us` are snapped by default. To use a different snap
+threshold:
+
+```bash
+trace-lanefix --snap-threshold-us 2.5 input.json output.fixed.json
+```
+
+Example output:
+
+This run shows the two repair paths: tiny sub-5.0us timestamp crossings are
+snapped in place, while one genuine overlap is moved to a synthetic lane. Flow
+endpoints are then rewritten to follow the adjusted slice starts and lanes.
+
+```text
+trace-lanefix trace.json trace_fixed.json
+processing 3 track(s)
+  pid=7334 tid=7334: 2 lane(s), 1 slice(s) moved, 11 snapped
+  pid=0 tid=7: 1 lane(s), 0 slice(s) moved, 5 snapped
+snapped 16 sub-5.0us crossing(s)
+updated 15 slice(s): total |start shift|=6.195us, total |end shift|=5.240us, total |boundary shift|=11.436us
+kernel timing summary (5 run(s) across 3 kernel name(s)):
+  nvjet_sm120_tst_mma_256x112x64_2_64x56x64_tmaAB_bz_TNNN: runs=2, total delayed=2.848us, total trimmed=0.000us, total boundary shift=2.848us, moved=0, snapped=2
+  nvjet_sm120_tst_mma_96x256x64_2_48x64x64_tmaAB_bz_TNNN: runs=2, total delayed=480ns, total trimmed=0.000us, total boundary shift=480ns, moved=0, snapped=2
+  nvjet_sm120_tst_mma_128x256x64_2_64x64x64_tmaAB_bz_TNNN: runs=1, total delayed=384ns, total trimmed=0.000us, total boundary shift=384ns, moved=0, snapped=1
+moved slices (1):
+  ProfilerStep#5 pid=7334: moved to lane 1 (tid 7334 -> 7334001)
+snapped slices (9):
+  layer0_forward pid=7334 tid=7334: start delayed by 2.483us
+  torch/autograd/profiler.py(843): __exit__ pid=7334 tid=7334: end trimmed by 2.855us
+  torch/_ops.py(1079): __call__ pid=7334 tid=7334: end trimmed by 1.222us
+  <built-in method  of pybind11_builtins.pybind11_detail_function_record_v1_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1 object at 0x719b5a7004b0> pid=7334 tid=7334: end trimmed by 1.162us
+  contextlib.py(141): __exit__ pid=7334 tid=7334: end trimmed by 0.2ns
+  profiling.py(19): trace pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/profiler/profiler.py(912): _transit_action pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/cuda/__init__.py(1161): synchronize pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/cuda/__init__.py(609): __exit__ pid=7334 tid=7334: end trimmed by 0.2ns
+rewrote 5 flow endpoint(s) onto rewritten slice starts/lanes
+wrote trace_fixed.json: 988 slices, 1 new lanes
+```
+
+## Python API
+
+```python
+from trace_lanefix import fix_overlaps
+
+fix_overlaps("input.json", "output.fixed.json")
+fix_overlaps("input.json", "output.fixed.json", snap_threshold_us=2.5)
+```

trace_lanefix-0.1.0/README.md

@@ -0,0 +1,80 @@
+# trace-lanefix
+
+`trace-lanefix` rewrites Chrome/Kineto trace JSON so Perfetto can render slices
+that would otherwise be dropped because of crossing overlaps on the same
+`(pid, tid)` track.
+
+It keeps properly nested slices on the same track, snaps tiny timestamp-boundary
+crossings, and moves genuine overlaps to synthetic lanes with collision-free
+thread ids.
+
+## Before / After
+
+Before:
+
+<img src="https://raw.githubusercontent.com/ralovets/trace-lanefix/main/assets/profile-before.png" alt="Profile before lane fix" width="100%">
+
+After:
+
+<img src="https://raw.githubusercontent.com/ralovets/trace-lanefix/main/assets/profile-after.png" alt="Profile after lane fix" width="100%">
+
+In Perfetto's trace overview, the rewritten trace reports `TRACE ERRORS=0`
+instead of `TRACE ERRORS=6`, while `IMPORT ERRORS` and `DATA LOSSES` remain
+at `0`. In this example, the JSON trace size changed from `452.76 KiB` before
+rewriting (`463,629` bytes) to `653.96 KiB` after rewriting (`669,651`
+bytes) because the output is pretty-printed for readability.
+
+## CLI
+
+```bash
+trace-lanefix input.json output.fixed.json
+```
+
+Tiny crossings below `5.0us` are snapped by default. To use a different snap
+threshold:
+
+```bash
+trace-lanefix --snap-threshold-us 2.5 input.json output.fixed.json
+```
+
+Example output:
+
+This run shows the two repair paths: tiny sub-5.0us timestamp crossings are
+snapped in place, while one genuine overlap is moved to a synthetic lane. Flow
+endpoints are then rewritten to follow the adjusted slice starts and lanes.
+
+```text
+trace-lanefix trace.json trace_fixed.json
+processing 3 track(s)
+  pid=7334 tid=7334: 2 lane(s), 1 slice(s) moved, 11 snapped
+  pid=0 tid=7: 1 lane(s), 0 slice(s) moved, 5 snapped
+snapped 16 sub-5.0us crossing(s)
+updated 15 slice(s): total |start shift|=6.195us, total |end shift|=5.240us, total |boundary shift|=11.436us
+kernel timing summary (5 run(s) across 3 kernel name(s)):
+  nvjet_sm120_tst_mma_256x112x64_2_64x56x64_tmaAB_bz_TNNN: runs=2, total delayed=2.848us, total trimmed=0.000us, total boundary shift=2.848us, moved=0, snapped=2
+  nvjet_sm120_tst_mma_96x256x64_2_48x64x64_tmaAB_bz_TNNN: runs=2, total delayed=480ns, total trimmed=0.000us, total boundary shift=480ns, moved=0, snapped=2
+  nvjet_sm120_tst_mma_128x256x64_2_64x64x64_tmaAB_bz_TNNN: runs=1, total delayed=384ns, total trimmed=0.000us, total boundary shift=384ns, moved=0, snapped=1
+moved slices (1):
+  ProfilerStep#5 pid=7334: moved to lane 1 (tid 7334 -> 7334001)
+snapped slices (9):
+  layer0_forward pid=7334 tid=7334: start delayed by 2.483us
+  torch/autograd/profiler.py(843): __exit__ pid=7334 tid=7334: end trimmed by 2.855us
+  torch/_ops.py(1079): __call__ pid=7334 tid=7334: end trimmed by 1.222us
+  <built-in method  of pybind11_builtins.pybind11_detail_function_record_v1_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1 object at 0x719b5a7004b0> pid=7334 tid=7334: end trimmed by 1.162us
+  contextlib.py(141): __exit__ pid=7334 tid=7334: end trimmed by 0.2ns
+  profiling.py(19): trace pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/profiler/profiler.py(912): _transit_action pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/cuda/__init__.py(1161): synchronize pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/cuda/__init__.py(609): __exit__ pid=7334 tid=7334: end trimmed by 0.2ns
+rewrote 5 flow endpoint(s) onto rewritten slice starts/lanes
+wrote trace_fixed.json: 988 slices, 1 new lanes
+```
+
+## Python API
+
+```python
+from trace_lanefix import fix_overlaps
+
+fix_overlaps("input.json", "output.fixed.json")
+fix_overlaps("input.json", "output.fixed.json", snap_threshold_us=2.5)
+```

trace_lanefix-0.1.0/pyproject.toml

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "trace-lanefix"
+version = "0.1.0"
+description = "Fix overlapping Chrome/Kineto trace slices so Perfetto can render them."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+authors = [
+  {name = "Roman Ralovets"},
+]
+keywords = ["chrome-trace", "kineto", "perfetto", "profiling", "pytorch"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Environment :: Console",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Topic :: Software Development :: Debuggers",
+  "Topic :: System :: Benchmark",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8"]
+
+[project.urls]
+Repository = "https://github.com/ralovets/trace-lanefix"
+Issues = "https://github.com/ralovets/trace-lanefix/issues"
+
+[project.scripts]
+trace-lanefix = "trace_lanefix.cli:main"
+
+[tool.setuptools.packages.find]
+include = ["trace_lanefix*"]

trace_lanefix-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

trace_lanefix-0.1.0/tests/test_fix_overlaps.py

@@ -0,0 +1,213 @@
+import json
+from collections import defaultdict
+
+import pytest
+
+from trace_lanefix import fix_overlaps
+
+
+def _run_fix(trace, tmp_path, **kwargs):
+    in_path = tmp_path / "in.json"
+    out_path = tmp_path / "out.json"
+    in_path.write_text(json.dumps(trace))
+    fix_overlaps(str(in_path), str(out_path), **kwargs)
+    return json.loads(out_path.read_text())["traceEvents"]
+
+
+def _assert_no_crossings(events):
+    slices = [e for e in events if e.get("ph") == "X" and "dur" in e]
+    by_track = defaultdict(list)
+    for e in slices:
+        by_track[(e.get("pid"), e.get("tid"))].append(e)
+
+    for group in by_track.values():
+        group.sort(key=lambda e: (e["ts"], -e["dur"]))
+        stack = []
+        for e in group:
+            start = e["ts"]
+            end = start + e["dur"]
+            while stack and stack[-1][0] <= start:
+                stack.pop()
+            if stack:
+                top_end, top_start = stack[-1]
+                assert top_start <= start and end <= top_end
+            stack.append((end, start))
+
+
+def test_start_snap_rechecks_outer_parent(tmp_path):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "python_function", "name": "outer", "pid": 1, "tid": 1,
+             "ts": 0.0, "dur": 3.405},
+            {"ph": "X", "cat": "python_function", "name": "inner", "pid": 1, "tid": 1,
+             "ts": 0.170, "dur": 3.205},
+            {"ph": "X", "cat": "user_annotation", "name": "layer0_forward", "pid": 1, "tid": 1,
+             "ts": 1.122, "dur": 10.0},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path)
+    _assert_no_crossings(events)
+
+    layer = next(e for e in events if e.get("name") == "layer0_forward")
+    assert layer["tid"] == 1
+    assert layer["ts"] == pytest.approx(3.405)
+    assert layer["dur"] == pytest.approx(7.717)
+
+
+def test_flow_endpoint_tracks_start_snap(tmp_path):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "cuda_runtime", "name": "cudaLaunchKernel", "pid": 11, "tid": 11,
+             "ts": 10.0, "dur": 2.0},
+            {"ph": "s", "id": 1, "pid": 11, "tid": 11, "ts": 10.0, "cat": "ac2g", "name": "ac2g"},
+            {"ph": "X", "cat": "kernel", "name": "kernel_a", "pid": 0, "tid": 7,
+             "ts": 100.0, "dur": 50.0},
+            {"ph": "X", "cat": "kernel", "name": "kernel_b", "pid": 0, "tid": 7,
+             "ts": 149.7, "dur": 10.0},
+            {"ph": "f", "id": 1, "pid": 0, "tid": 7, "ts": 149.7, "cat": "ac2g", "name": "ac2g", "bp": "e"},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path)
+    _assert_no_crossings(events)
+
+    kernel_b = next(e for e in events if e.get("name") == "kernel_b")
+    flow_f = next(e for e in events if e.get("ph") == "f" and e.get("id") == 1)
+
+    assert kernel_b["tid"] == 7
+    assert kernel_b["ts"] == pytest.approx(150.0)
+    assert flow_f["tid"] == 7
+    assert flow_f["ts"] == pytest.approx(150.0)
+
+
+def test_snap_threshold_can_force_synthetic_lane(tmp_path):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "kernel", "name": "kernel_a", "pid": 0, "tid": 7,
+             "ts": 100.0, "dur": 50.0},
+            {"ph": "X", "cat": "kernel", "name": "kernel_b", "pid": 0, "tid": 7,
+             "ts": 149.7, "dur": 10.0},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path, snap_threshold_us=0.1)
+    _assert_no_crossings(events)
+
+    kernel_b = next(e for e in events if e.get("name") == "kernel_b")
+    assert kernel_b["tid"] == 7001
+    assert kernel_b["ts"] == pytest.approx(149.7)
+
+
+def test_child_before_start_snapped_parent_is_rechecked(tmp_path):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "cpu", "name": "previous", "pid": 1, "tid": 1,
+             "ts": 0.0, "dur": 10.0},
+            {"ph": "X", "cat": "cpu", "name": "parent", "pid": 1, "tid": 1,
+             "ts": 9.9, "dur": 10.0},
+            {"ph": "X", "cat": "cpu", "name": "child", "pid": 1, "tid": 1,
+             "ts": 9.95, "dur": 1.0},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path)
+    _assert_no_crossings(events)
+
+    parent = next(e for e in events if e.get("name") == "parent")
+    child = next(e for e in events if e.get("name") == "child")
+    assert parent["tid"] == 1
+    assert parent["ts"] == pytest.approx(10.0)
+    assert child["tid"] == 1
+    assert child["ts"] == pytest.approx(10.0)
+    assert child["dur"] == pytest.approx(0.95)
+
+
+def test_synthetic_lane_tid_avoids_real_track_collision(tmp_path):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "cpu", "name": "real_1001", "pid": 1, "tid": 1001,
+             "ts": 100.0, "dur": 1.0},
+            {"ph": "X", "cat": "cpu", "name": "a", "pid": 1, "tid": 1,
+             "ts": 0.0, "dur": 10.0},
+            {"ph": "X", "cat": "cpu", "name": "b", "pid": 1, "tid": 1,
+             "ts": 5.0, "dur": 10.0},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path)
+    _assert_no_crossings(events)
+
+    moved = next(e for e in events if e.get("name") == "b")
+    real = next(e for e in events if e.get("name") == "real_1001")
+    lane_name = next(
+        e for e in events
+        if e.get("name") == "thread_name" and e.get("args", {}).get("name") == "tid 1 lane 1"
+    )
+    assert real["tid"] == 1001
+    assert moved["tid"] == "1 lane 1"
+    assert lane_name["tid"] == "1 lane 1"
+
+
+def test_string_tid_can_move_to_synthetic_lane(tmp_path):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "cpu", "name": "a", "pid": 1, "tid": "worker",
+             "ts": 0.0, "dur": 10.0},
+            {"ph": "X", "cat": "cpu", "name": "b", "pid": 1, "tid": "worker",
+             "ts": 5.0, "dur": 10.0},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path)
+    _assert_no_crossings(events)
+
+    moved = next(e for e in events if e.get("name") == "b")
+    assert moved["tid"] == "worker lane 1"
+
+
+def test_output_json_is_pretty_printed(tmp_path):
+    trace = {
+        "schemaVersion": 1,
+        "traceEvents": [
+            {"ph": "X", "cat": "cpu", "name": "a", "pid": 1, "tid": 1,
+             "ts": 0.0, "dur": 1.0},
+        ],
+    }
+    in_path = tmp_path / "in.json"
+    out_path = tmp_path / "out.json"
+    in_path.write_text(json.dumps(trace))
+
+    fix_overlaps(str(in_path), str(out_path))
+
+    text = out_path.read_text()
+    assert text.endswith("\n")
+    assert "\n  \"schemaVersion\": 1,\n" in text
+    assert "\n    {\n" in text
+
+
+def test_reports_grouped_kernel_summary_sorted_by_total_shift(tmp_path, capsys):
+    trace = {
+        "traceEvents": [
+            {"ph": "X", "cat": "kernel", "name": "kernel_a", "pid": 0, "tid": 7,
+             "ts": 100.0, "dur": 50.0},
+            {"ph": "X", "cat": "kernel", "name": "kernel_c", "pid": 0, "tid": 7,
+             "ts": 120.0, "dur": 40.0},
+            {"ph": "X", "cat": "kernel", "name": "kernel_b", "pid": 0, "tid": 7,
+             "ts": 149.7, "dur": 10.0},
+            {"ph": "X", "cat": "kernel", "name": "kernel_b", "pid": 0, "tid": 7,
+             "ts": 159.6, "dur": 10.0},
+        ]
+    }
+
+    events = _run_fix(trace, tmp_path)
+    _assert_no_crossings(events)
+
+    out = capsys.readouterr().out
+    assert "updated 3 slice(s)" in out
+    assert "kernel timing summary (3 run(s) across 2 kernel name(s)):" in out
+    assert "  kernel_b: runs=2, total delayed=400ns, total trimmed=0.000us, total boundary shift=400ns, moved=0, snapped=2" in out
+    assert "  kernel_c: runs=1, total delayed=0.000us, total trimmed=0.000us, total boundary shift=0.000us, moved=1, snapped=0" in out
+    assert "moved slices" not in out
+    assert "snapped slices" not in out
+    assert out.index("  kernel_b:") < out.index("  kernel_c:")

trace_lanefix-0.1.0/trace_lanefix/__init__.py

@@ -0,0 +1,5 @@
+"""Utilities for making Chrome/Kineto traces render cleanly in Perfetto."""
+
+from .core import SNAP_THRESHOLD_US, fix_overlaps
+
+__all__ = ["SNAP_THRESHOLD_US", "fix_overlaps"]

trace_lanefix-0.1.0/trace_lanefix/cli.py

@@ -0,0 +1,51 @@
+"""Command-line interface for trace-lanefix."""
+
+import argparse
+
+from .core import SNAP_THRESHOLD_US, fix_overlaps
+
+
+def _non_negative_float(value: str) -> float:
+    parsed = float(value)
+    if parsed < 0:
+        raise argparse.ArgumentTypeError("must be non-negative")
+    return parsed
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="trace-lanefix",
+        description="Rewrite a Chrome/Kineto trace so Perfetto can render overlapping slices.",
+    )
+    parser.add_argument(
+        "input",
+        nargs="?",
+        default="trace.json",
+        help="input Chrome trace JSON path (default: trace.json)",
+    )
+    parser.add_argument(
+        "output",
+        nargs="?",
+        default="trace.fixed.json",
+        help="output fixed trace JSON path (default: trace.fixed.json)",
+    )
+    parser.add_argument(
+        "--snap-threshold-us",
+        type=_non_negative_float,
+        default=SNAP_THRESHOLD_US,
+        help=(
+            "maximum timestamp crossing, in microseconds, to snap in place "
+            f"instead of moving to a synthetic lane (default: {SNAP_THRESHOLD_US})"
+        ),
+    )
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    fix_overlaps(args.input, args.output, snap_threshold_us=args.snap_threshold_us)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

trace_lanefix-0.1.0/trace_lanefix/core.py

@@ -0,0 +1,377 @@
+"""Rewrite a Chrome/Kineto trace so Perfetto can render every slice.
+
+Perfetto's slice model requires strict stack nesting per (pid, tid): two
+slices on the same track must be disjoint or one must fully contain the
+other. Slices that *cross* a boundary get dropped
+(SLICE_DROP_OVERLAPPING_COMPLETE_EVENT).
+
+Strategy — resolve crossings with the minimum visual disturbance:
+
+1. Nesting-aware placement. Each lane keeps a live stack of open frames.
+   An event joins a lane if, after popping closed frames, the lane is
+   empty or the current top strictly contains the event. Nested children
+   stay on their parent's lane.
+
+2. Snap tiny crossings instead of spawning a lane. Two directions are
+   considered for every crossing, and the smaller perturbation wins:
+     - end-snap: clamp the child's end down to its parent's end (nests).
+     - start-snap: push the child's start forward to its parent's end
+       (turns it into a disjoint sibling).
+   The smallest shift is the one that actually represents CUPTI/Kineto
+   timestamp noise; end-snap fits float-epsilon parent-end overruns,
+   start-snap fits the typical GPU-stream case where two kernels appear
+   to overlap by ~100ns at the boundary even though they're back-to-back
+   on the stream. Only shifts below the configured snap threshold are applied, so
+   genuine concurrent slices (ProfilerStep#N, kernels on separate streams
+   sharing a track) still go to a secondary lane.
+
+3. No category gate. Every track runs through the same pass. GPU streams
+   behave identically to the old algorithm (no nesting ⇒ same lane
+   decisions) and CPU threads no longer explode.
+
+4. Collision-free lane ids. Synthetic tids keep the historical numeric
+   `tid * 1000 + lane` form when that id is available, and fall back to a
+   string id when it would collide with a real track.
+"""
+
+import json
+from collections import defaultdict
+
+SNAP_THRESHOLD_US = 5.0  # absolute shift below which we clamp instead of laning.
+# Calibrated against CUPTI boundary noise between consecutive same-stream kernels
+# (observed max ~2.5us on nvjet tensor-core kernels). Kernels on one CUDA stream
+# cannot physically overlap, so any sub-threshold overlap is timing noise.
+
+
+def _format_us(value: float) -> str:
+    if 0 < abs(value) < 1.0:
+        return f"{value * 1000.0:.1f}".rstrip("0").rstrip(".") + "ns"
+    return f"{value:.3f}us"
+
+
+def _describe_change(change: dict) -> str:
+    start_delta = change["new_start"] - change["orig_start"]
+    end_delta = change["new_end"] - change["orig_end"]
+    parts = []
+
+    if change["orig_tid"] != change["new_tid"]:
+        parts.append(f"moved to lane {change['lane_idx']} (tid {change['orig_tid']} -> {change['new_tid']})")
+
+    if start_delta == end_delta and start_delta != 0:
+        parts.append(f"shifted {'later' if start_delta > 0 else 'earlier'} by {_format_us(abs(start_delta))}")
+    else:
+        if start_delta > 0:
+            parts.append(f"start delayed by {_format_us(start_delta)}")
+        elif start_delta < 0:
+            parts.append(f"start moved earlier by {_format_us(-start_delta)}")
+
+        if end_delta > 0:
+            parts.append(f"end extended by {_format_us(end_delta)}")
+        elif end_delta < 0:
+            parts.append(f"end trimmed by {_format_us(-end_delta)}")
+
+    return ", ".join(parts) or "timing unchanged"
+
+
+def _change_shift_stats(change: dict) -> dict:
+    start_delta = change["new_start"] - change["orig_start"]
+    end_delta = change["new_end"] - change["orig_end"]
+    return {
+        "start_delayed": max(0.0, start_delta),
+        "start_earlier": max(0.0, -start_delta),
+        "end_extended": max(0.0, end_delta),
+        "end_trimmed": max(0.0, -end_delta),
+        "boundary_shift": abs(start_delta) + abs(end_delta),
+    }
+
+
+def _summarize_changes_by_name(changes: list[dict]) -> list[dict]:
+    summaries = {}
+    for change in changes:
+        summary = summaries.setdefault(
+            change["name"],
+            {
+                "name": change["name"],
+                "runs": 0,
+                "start_delayed": 0.0,
+                "start_earlier": 0.0,
+                "end_extended": 0.0,
+                "end_trimmed": 0.0,
+                "boundary_shift": 0.0,
+                "max_boundary_shift": 0.0,
+                "moved": 0,
+                "snapped": 0,
+            },
+        )
+        shifts = _change_shift_stats(change)
+        summary["runs"] += 1
+        summary["start_delayed"] += shifts["start_delayed"]
+        summary["start_earlier"] += shifts["start_earlier"]
+        summary["end_extended"] += shifts["end_extended"]
+        summary["end_trimmed"] += shifts["end_trimmed"]
+        summary["boundary_shift"] += shifts["boundary_shift"]
+        summary["max_boundary_shift"] = max(summary["max_boundary_shift"], shifts["boundary_shift"])
+        summary["moved"] += int(change["orig_tid"] != change["new_tid"])
+        summary["snapped"] += int(change["snap_kind"] is not None)
+
+    return sorted(
+        summaries.values(),
+        key=lambda summary: (
+            -summary["boundary_shift"],
+            -summary["runs"],
+            -summary["moved"],
+            -summary["snapped"],
+            -summary["max_boundary_shift"],
+            summary["name"],
+        ),
+    )
+
+
+def _allocate_lane_tid(pid, tid, lane_idx: int, used_tracks: set[tuple]) -> int | str:
+    if type(tid) is int:
+        candidate = tid * 1000 + lane_idx
+        if (pid, candidate) not in used_tracks:
+            used_tracks.add((pid, candidate))
+            return candidate
+
+    base = f"{tid} lane {lane_idx}"
+    candidate = base
+    suffix = 2
+    while (pid, candidate) in used_tracks:
+        candidate = f"{base} #{suffix}"
+        suffix += 1
+
+    used_tracks.add((pid, candidate))
+    return candidate
+
+
+def fix_overlaps(in_path: str, out_path: str, snap_threshold_us: float = SNAP_THRESHOLD_US) -> None:
+    if snap_threshold_us < 0:
+        raise ValueError("snap_threshold_us must be non-negative")
+
+    with open(in_path) as f:
+        trace = json.load(f)
+
+    events = trace.get("traceEvents", trace) if isinstance(trace, dict) else trace
+
+    slices, flows, others = [], [], []
+    for e in events:
+        ph = e.get("ph")
+        if ph == "X" and "dur" in e:
+            slices.append(e)
+        elif ph in ("s", "t", "f"):
+            flows.append(e)
+        else:
+            others.append(e)
+
+    used_tracks = {
+        (e.get("pid"), e.get("tid"))
+        for e in events
+        if "pid" in e and "tid" in e
+    }
+
+    by_track = defaultdict(list)
+    for e in slices:
+        by_track[(e.get("pid"), e.get("tid"))].append(e)
+    print(f"processing {len(by_track)} track(s)")
+
+    lane_names = {}
+    lane_tids = {}
+    endpoint_rewrites = {}
+    snapped_total = 0
+    per_track_report = []
+    changed_slices = []
+
+    for (pid, tid), group in by_track.items():
+        # Outer-first: earliest start, longest dur on ties, so parents are
+        # placed before their children.
+        group.sort(key=lambda e: (e["ts"], -e["dur"]))
+        lanes = []  # each lane is a stack of (end_ts, start_ts)
+        moved = 0
+        snapped = 0
+
+        for e in group:
+            orig_tid = e["tid"]
+            orig_start = e["ts"]
+            orig_dur = e["dur"]
+            orig_end = orig_start + orig_dur
+            start = orig_start
+            end = start + orig_dur
+            placed_lane = None
+            snap_kind = None
+
+            for lane_idx, stack in enumerate(lanes):
+                placement_resolved = False
+                while not placement_resolved:
+                    while stack and stack[-1][0] <= start:
+                        stack.pop()
+
+                    if not stack:
+                        stack.append((end, start))
+                        placed_lane = lane_idx
+                        placement_resolved = True
+                        break
+
+                    top_end, top_start = stack[-1]
+
+                    if lane_idx != 0:
+                        if top_start <= start and end <= top_end:
+                            stack.append((end, start))
+                            placed_lane = lane_idx
+                            placement_resolved = True
+                        break  # Only snap on the primary lane
+
+                    if start < top_start:
+                        start_shift = top_start - start
+                        if start_shift < snap_threshold_us:
+                            start = top_start
+                            e["ts"] = top_start
+                            e["dur"] = max(0.0, end - start)
+                            snapped += 1
+                            snap_kind = "start-snap"
+                            continue
+                        break
+
+                    if end <= top_end:
+                        stack.append((end, start))
+                        placed_lane = lane_idx
+                        placement_resolved = True
+                        break
+
+                    end_shift = end - top_end
+                    start_shift = top_end - start
+
+                    if 0 < end_shift <= start_shift and end_shift < snap_threshold_us:
+                        end = top_end
+                        e["dur"] = max(0.0, end - start)
+                        snapped += 1
+                        snap_kind = "end-snap"
+                    elif 0 < start_shift < snap_threshold_us:
+                        start = top_end
+                        e["ts"] = top_end
+                        e["dur"] = max(0.0, end - start)
+                        snapped += 1
+                        snap_kind = "start-snap"
+                    else:
+                        break  # Cannot snap, try next lane
+
+                if placed_lane is not None:
+                    break
+
+            if placed_lane is None:
+                placed_lane = len(lanes)
+                lanes.append([(end, start)])
+
+            if placed_lane > 0:
+                lane_key = (pid, tid, placed_lane)
+                if lane_key not in lane_tids:
+                    lane_tids[lane_key] = _allocate_lane_tid(pid, tid, placed_lane, used_tracks)
+                new_tid = lane_tids[lane_key]
+                e["tid"] = new_tid
+                lane_names[(pid, new_tid)] = (tid, placed_lane)
+                moved += 1
+
+            new_start = e["ts"]
+            new_dur = e["dur"]
+            new_end = new_start + new_dur
+
+            if e["tid"] != orig_tid or new_start != orig_start or new_dur != orig_dur:
+                changed_slices.append({
+                    "name": e.get("name", "<unnamed>"),
+                    "cat": e.get("cat", ""),
+                    "pid": pid,
+                    "orig_tid": orig_tid,
+                    "new_tid": e["tid"],
+                    "orig_start": orig_start,
+                    "new_start": new_start,
+                    "orig_dur": orig_dur,
+                    "new_dur": new_dur,
+                    "orig_end": orig_end,
+                    "new_end": new_end,
+                    "snap_kind": snap_kind,
+                    "lane_idx": placed_lane,
+                })
+
+            if e["tid"] != orig_tid or new_start != orig_start:
+                endpoint_rewrites[(pid, orig_tid, orig_start)] = (e["tid"], e["ts"])
+
+        snapped_total += snapped
+        if len(lanes) > 1 or snapped:
+            per_track_report.append((pid, tid, len(lanes), moved, snapped))
+
+    for pid, tid, n_lanes, moved, snapped in per_track_report:
+        print(f"  pid={pid} tid={tid}: {n_lanes} lane(s), {moved} slice(s) moved, {snapped} snapped")
+    print(f"snapped {snapped_total} sub-{snap_threshold_us}us crossing(s)")
+
+    if changed_slices:
+        total_start_shift = sum(abs(c["new_start"] - c["orig_start"]) for c in changed_slices)
+        total_end_shift = sum(abs(c["new_end"] - c["orig_end"]) for c in changed_slices)
+
+        print(
+            f"updated {len(changed_slices)} slice(s): "
+            f"total |start shift|={_format_us(total_start_shift)}, "
+            f"total |end shift|={_format_us(total_end_shift)}, "
+            f"total |boundary shift|={_format_us(total_start_shift + total_end_shift)}"
+        )
+
+        kernel_changes = [c for c in changed_slices if c["cat"] == "kernel"]
+        if kernel_changes:
+            kernel_summaries = _summarize_changes_by_name(kernel_changes)
+            print(
+                "kernel timing summary "
+                f"({len(kernel_changes)} run(s) across {len(kernel_summaries)} kernel name(s)):"
+            )
+            for summary in kernel_summaries:
+                print(
+                    f"  {summary['name']}: runs={summary['runs']}, "
+                    f"total delayed={_format_us(summary['start_delayed'])}, "
+                    f"total trimmed={_format_us(summary['end_trimmed'])}, "
+                    f"total boundary shift={_format_us(summary['boundary_shift'])}, "
+                    f"moved={summary['moved']}, snapped={summary['snapped']}"
+                )
+
+        categories = [
+            ("moved slices", lambda c: c["cat"] != "kernel" and c["orig_tid"] != c["new_tid"]),
+            ("snapped slices", lambda c: c["cat"] != "kernel" and c["snap_kind"] is not None),
+        ]
+
+        for title, condition in categories:
+            filtered = [c for c in changed_slices if condition(c)]
+            if filtered:
+                print(f"{title} ({len(filtered)}):")
+                for c in filtered:
+                    tid_info = f" tid={c['new_tid']}" if title == "snapped slices" else ""
+                    print(f"  {c['name']} pid={c['pid']}{tid_info}: {_describe_change(c)}")
+
+    flow_rewrites = 0
+    for fe in flows:
+        key = (fe.get("pid"), fe.get("tid"), fe.get("ts"))
+        if key in endpoint_rewrites:
+            fe["tid"], fe["ts"] = endpoint_rewrites[key]
+            flow_rewrites += 1
+
+    print(f"rewrote {flow_rewrites} flow endpoint(s) onto rewritten slice starts/lanes")
+
+    for (pid, new_tid), (orig_tid, lane_idx) in lane_names.items():
+        others.extend([
+            {
+                "name": "thread_name", "ph": "M", "pid": pid, "tid": new_tid,
+                "args": {"name": f"tid {orig_tid} lane {lane_idx}"},
+            },
+            {
+                "name": "thread_sort_index", "ph": "M", "pid": pid, "tid": new_tid,
+                "args": {"sort_index": lane_idx},
+            }
+        ])
+
+    out_events = others + slices + flows
+    if isinstance(trace, dict):
+        trace["traceEvents"] = out_events
+    else:
+        trace = out_events
+
+    with open(out_path, "w") as f:
+        json.dump(trace, f, indent=2)
+        f.write("\n")
+
+    print(f"wrote {out_path}: {len(slices)} slices, {len(lane_names)} new lanes")

trace_lanefix-0.1.0/trace_lanefix.egg-info/PKG-INFO

@@ -0,0 +1,105 @@
+Metadata-Version: 2.4
+Name: trace-lanefix
+Version: 0.1.0
+Summary: Fix overlapping Chrome/Kineto trace slices so Perfetto can render them.
+Author: Roman Ralovets
+License-Expression: MIT
+Project-URL: Repository, https://github.com/ralovets/trace-lanefix
+Project-URL: Issues, https://github.com/ralovets/trace-lanefix/issues
+Keywords: chrome-trace,kineto,perfetto,profiling,pytorch
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Debuggers
+Classifier: Topic :: System :: Benchmark
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Dynamic: license-file
+
+# trace-lanefix
+
+`trace-lanefix` rewrites Chrome/Kineto trace JSON so Perfetto can render slices
+that would otherwise be dropped because of crossing overlaps on the same
+`(pid, tid)` track.
+
+It keeps properly nested slices on the same track, snaps tiny timestamp-boundary
+crossings, and moves genuine overlaps to synthetic lanes with collision-free
+thread ids.
+
+## Before / After
+
+Before:
+
+<img src="https://raw.githubusercontent.com/ralovets/trace-lanefix/main/assets/profile-before.png" alt="Profile before lane fix" width="100%">
+
+After:
+
+<img src="https://raw.githubusercontent.com/ralovets/trace-lanefix/main/assets/profile-after.png" alt="Profile after lane fix" width="100%">
+
+In Perfetto's trace overview, the rewritten trace reports `TRACE ERRORS=0`
+instead of `TRACE ERRORS=6`, while `IMPORT ERRORS` and `DATA LOSSES` remain
+at `0`. In this example, the JSON trace size changed from `452.76 KiB` before
+rewriting (`463,629` bytes) to `653.96 KiB` after rewriting (`669,651`
+bytes) because the output is pretty-printed for readability.
+
+## CLI
+
+```bash
+trace-lanefix input.json output.fixed.json
+```
+
+Tiny crossings below `5.0us` are snapped by default. To use a different snap
+threshold:
+
+```bash
+trace-lanefix --snap-threshold-us 2.5 input.json output.fixed.json
+```
+
+Example output:
+
+This run shows the two repair paths: tiny sub-5.0us timestamp crossings are
+snapped in place, while one genuine overlap is moved to a synthetic lane. Flow
+endpoints are then rewritten to follow the adjusted slice starts and lanes.
+
+```text
+trace-lanefix trace.json trace_fixed.json
+processing 3 track(s)
+  pid=7334 tid=7334: 2 lane(s), 1 slice(s) moved, 11 snapped
+  pid=0 tid=7: 1 lane(s), 0 slice(s) moved, 5 snapped
+snapped 16 sub-5.0us crossing(s)
+updated 15 slice(s): total |start shift|=6.195us, total |end shift|=5.240us, total |boundary shift|=11.436us
+kernel timing summary (5 run(s) across 3 kernel name(s)):
+  nvjet_sm120_tst_mma_256x112x64_2_64x56x64_tmaAB_bz_TNNN: runs=2, total delayed=2.848us, total trimmed=0.000us, total boundary shift=2.848us, moved=0, snapped=2
+  nvjet_sm120_tst_mma_96x256x64_2_48x64x64_tmaAB_bz_TNNN: runs=2, total delayed=480ns, total trimmed=0.000us, total boundary shift=480ns, moved=0, snapped=2
+  nvjet_sm120_tst_mma_128x256x64_2_64x64x64_tmaAB_bz_TNNN: runs=1, total delayed=384ns, total trimmed=0.000us, total boundary shift=384ns, moved=0, snapped=1
+moved slices (1):
+  ProfilerStep#5 pid=7334: moved to lane 1 (tid 7334 -> 7334001)
+snapped slices (9):
+  layer0_forward pid=7334 tid=7334: start delayed by 2.483us
+  torch/autograd/profiler.py(843): __exit__ pid=7334 tid=7334: end trimmed by 2.855us
+  torch/_ops.py(1079): __call__ pid=7334 tid=7334: end trimmed by 1.222us
+  <built-in method  of pybind11_builtins.pybind11_detail_function_record_v1_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1 object at 0x719b5a7004b0> pid=7334 tid=7334: end trimmed by 1.162us
+  contextlib.py(141): __exit__ pid=7334 tid=7334: end trimmed by 0.2ns
+  profiling.py(19): trace pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/profiler/profiler.py(912): _transit_action pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/cuda/__init__.py(1161): synchronize pid=7334 tid=7334: end trimmed by 0.2ns
+  torch/cuda/__init__.py(609): __exit__ pid=7334 tid=7334: end trimmed by 0.2ns
+rewrote 5 flow endpoint(s) onto rewritten slice starts/lanes
+wrote trace_fixed.json: 988 slices, 1 new lanes
+```
+
+## Python API
+
+```python
+from trace_lanefix import fix_overlaps
+
+fix_overlaps("input.json", "output.fixed.json")
+fix_overlaps("input.json", "output.fixed.json", snap_threshold_us=2.5)
+```

trace_lanefix-0.1.0/trace_lanefix.egg-info/SOURCES.txt

@@ -0,0 +1,13 @@
+LICENSE
+README.md
+pyproject.toml
+tests/test_fix_overlaps.py
+trace_lanefix/__init__.py
+trace_lanefix/cli.py
+trace_lanefix/core.py
+trace_lanefix.egg-info/PKG-INFO
+trace_lanefix.egg-info/SOURCES.txt
+trace_lanefix.egg-info/dependency_links.txt
+trace_lanefix.egg-info/entry_points.txt
+trace_lanefix.egg-info/requires.txt
+trace_lanefix.egg-info/top_level.txt

trace_lanefix-0.1.0/trace_lanefix.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

trace_lanefix-0.1.0/trace_lanefix.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+trace-lanefix = trace_lanefix.cli:main

trace_lanefix-0.1.0/trace_lanefix.egg-info/requires.txt

@@ -0,0 +1,3 @@
+
+[dev]
+pytest>=8

trace_lanefix-0.1.0/trace_lanefix.egg-info/top_level.txt

@@ -0,0 +1 @@
+trace_lanefix