tritonparse 0.3.1.dev20251018071448__tar.gz → 0.3.2.dev20251113071521__tar.gz

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/CHANGELOG.md

@@ -5,6 +5,92 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.3.1] - 2025-11-03
+
+### TritonParse Release Notes (last 24 commits)
+
+- **Date range**: 2025-10-14 — 2025-11-03
+- **Scope**: IR Analysis enhancements (beta), Reproducer template extensions, code viewer improvements, bug fixes.
+
+### Highlights
+
+- **📊 IR Analysis (Beta)**: New analysis capabilities for visualizing Software Pipelining (SWP), BufferOps statistics, and loop schedules in Triton IR. **Note: This is a beta feature.**
+- **🏷️ Variable Location Tracking**: Complete location alias tracking system for mapping IR locations back to source code with frontend visualization.
+- **🔧 TritonBench Template**: New reproducer template for easy TritonBench integration and kernel benchmarking.
+- **🎨 Code Viewer Enhancements**: Full Python source extraction, function highlighting, and performance optimizations.
+- **🔄 Reproducer Refactoring**: AST-based function extraction eliminates code duplication and simplifies template maintenance.
+
+### Changes by area
+
+#### 📊 **IR Analysis (Beta)**
+- **Software Pipelining (SWP) visualization** (PR #189):
+  - Analyzes inner `scf.for` loops and identifies prologue, loop_body, and epilogue stages
+  - Tracks `tt.load` and `tt.dot` operations through TTIR → TTGIR → Python source mappings
+  - Frontend displays simplified source code with SWP stage information
+  - **Limitations**: Does not support Warp Specialization or Blackwell operators yet
+- **BufferOps backend information** (PR #181):
+  - Statistical analysis of buffer operations (tt.load/store, amdgpu.buffer_load/store, global_load/store) at TTGIR and AMDGCN levels
+  - Useful for AMD GPU backend optimization analysis
+- **Web frontend IR Analysis page** (PR #184):
+  - New dedicated page at `/ir-analysis` route with integrated display for loop schedules and BufferOps statistics
+
+#### 🏷️ **Variable Location Tracking**
+Complete three-part implementation (PR #186, #187, #188):
+- Fixed #loc storage key conflict in IR parser
+- Added location alias parsing support in `ir_parser.py` and `trace_processor.py`
+- Frontend visualization with CSS styling and interactive location display in Code Viewer
+
+#### 🔄 **Reproducer System**
+- **TritonBench template support** (commit 3493ac8):
+  - New template: `tritonparse/reproducer/templates/tritonbench.py`
+  - CLI option: `--template tritonbench` for TritonBench-compatible reproducers
+  - Integrates with TritonBench's `BenchmarkOperator` and benchmark harness
+- **AST-based refactoring** (PR #178):
+  - New module: `tritonparse/reproducer/function_extractor.py` using Python AST
+  - Simplified `example.py` template from ~370 lines to ~20 lines
+- **Bug fixes**:
+  - Fixed 1-based to 0-based line number conversion (PR #185)
+  - Corrected output key typo: `repo_*` → `repro_*` (PR #175)
+  - CUDA device normalization to `cuda:0` format (PR #177)
+
+#### 📝 **Callsite Location Support**
+- **TTIR/TTGIR callsite location** (PR #190):
+  - Extended IR parser to extract callsite location information
+  - Better debugging with call graph information and test coverage
+
+#### 💻 **Code Viewer & Frontend**
+- **Full Python source extraction** (commit 2976887):
+  - Enhanced `structured_logging.py` to extract complete Python source files
+- **Full file display with function highlighting** (commit 220d5a4):
+  - CodeViewer now supports displaying entire source files with function-level highlighting
+- **CodeComparisonView performance optimization** (commit c17e584):
+  - Significant rendering performance improvements for large files
+  - Reduced re-renders and improved memory efficiency
+
+#### 🌐 **Website & Maintenance**
+- **Dependency updates** (PR #179): Added automation script `website/scripts/update_deps.sh`
+- **Copyright updates** (PR #183): Updated copyright headers across source files
+
+### Compatibility notes
+
+- **No breaking changes**: All updates are backward compatible with v0.3.0.
+- **IR Analysis (Beta)**: New optional feature accessible through web UI.
+- **TritonBench template**: Optional, does not impact existing reproducer generation.
+
+### Upgrade guidance
+
+1. **Using IR Analysis (Beta)**:
+   - Open web UI and navigate to IR Analysis page after parsing
+   - View SWP stage information (prologue/loop_body/epilogue) and BufferOps statistics
+   - Note: Beta feature with some limitations on advanced pipelining patterns
+
+2. **Generating TritonBench reproducers**:
+   ```bash
+   tritonparseoss reproduce trace.ndjson.gz --line <N> --template tritonbench --out-dir <output>
+   ```
+
+3. **Code viewer enhancements**: Automatically enabled with full source display and function highlighting
+
 ## [0.3.0] - 2025-10-14
 
 ### TritonParse Release Notes (last 44 commits)

{tritonparse-0.3.1.dev20251018071448/tritonparse.egg-info → tritonparse-0.3.2.dev20251113071521}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tritonparse
-Version: 0.3.1.dev20251018071448
+Version: 0.3.2.dev20251113071521
 Summary: TritonParse: A Compiler Tracer, Visualizer, and mini-Reproducer Generator for Triton Kernels
 Author-email: Yueming Hao <yhao@meta.com>
 License-Expression: BSD-3-Clause
@@ -134,7 +134,7 @@ from tritonparse.reproducer.orchestrator import reproduce
 
 result = reproduce(
     input_path="./parsed_output/trace.ndjson.gz",
-    line_index=1,           # Which launch event (1-based)
+    line_index=0,           # 0-based index (first event is 0)
     out_dir="repro_output"
 )
 ```

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/README.md

@@ -116,7 +116,7 @@ from tritonparse.reproducer.orchestrator import reproduce
 
 result = reproduce(
     input_path="./parsed_output/trace.ndjson.gz",
-    line_index=1,           # Which launch event (1-based)
+    line_index=0,           # 0-based index (first event is 0)
     out_dir="repro_output"
 )
 ```

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tests/test_add.py

@@ -12,7 +12,6 @@ import os
 import torch
 import triton
 import triton.language as tl
-
 import tritonparse.structured_logging
 import tritonparse.utils
 

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tests/test_tritonparse.py

@@ -24,7 +24,6 @@ import tritonparse.context_manager
 import tritonparse.structured_logging
 import tritonparse.utils
 from triton import knobs  # @manual=//triton:triton
-
 from triton.compiler import ASTSource, IRSource  # @manual=//triton:triton
 from triton.knobs import CompileTimes  # @manual=//triton:triton
 from tritonparse.common import is_fbcode
@@ -139,6 +138,63 @@ def clear_all_caches(*kernels):
 class TestTritonparseCPU(unittest.TestCase):
     """CPU-only tests (no CUDA required)"""
 
+    def test_callsite_parsing(self):
+        """Test parsing of callsite locations in TTIR/TTGIR"""
+        from tritonparse.ir_parser import extract_loc_definitions
+        from tritonparse.trace_processor import generate_source_mappings
+
+        # Test MLIR callsite location definitions
+        ir_with_callsite = """
+module {
+  #loc7 = loc("/tmp/test.py":1091:8)
+  #loc57 = loc("/tmp/test.py":421:16)
+  #loc58 = loc("/tmp/test.py":853:16)
+  #loc190 = loc(callsite(#loc58 at #loc7))
+  #loc220 = loc(callsite(#loc57 at #loc190))
+  %0 = tt.load %ptr loc(#loc220)
+}
+"""
+        # Extract loc definitions
+        locs = extract_loc_definitions(ir_with_callsite)
+
+        # Verify loc220 (nested callsite)
+        self.assertIn("220", locs)
+        self.assertEqual(locs["220"]["file"], "/tmp/test.py")
+        self.assertEqual(locs["220"]["line"], 421)  # Inherited from callee loc57
+        self.assertEqual(locs["220"]["column"], 16)
+        self.assertTrue(locs["220"].get("is_callsite"))
+        self.assertEqual(locs["220"]["callsite_callee"], "57")
+        self.assertEqual(locs["220"]["callsite_caller"], "190")
+
+        # Verify loc190 (simple callsite)
+        self.assertIn("190", locs)
+        self.assertEqual(locs["190"]["line"], 853)  # Inherited from callee loc58
+        self.assertTrue(locs["190"].get("is_callsite"))
+        self.assertEqual(locs["190"]["callsite_callee"], "58")
+        self.assertEqual(locs["190"]["callsite_caller"], "7")
+
+        # Test source mappings generation
+        mappings = generate_source_mappings(ir_with_callsite, "ttir")
+
+        # Find the line with tt.load
+        line_with_load = None
+        for line_num, content in enumerate(ir_with_callsite.split("\n"), start=1):
+            if "tt.load" in content:
+                line_with_load = str(line_num)
+                break
+
+        self.assertIsNotNone(line_with_load)
+        self.assertIn(line_with_load, mappings)
+
+        mapping = mappings[line_with_load]
+        self.assertEqual(mapping["file"], "/tmp/test.py")
+        self.assertEqual(mapping["line"], 421)  # From loc220 -> loc57
+        self.assertTrue(mapping.get("is_callsite"))
+        self.assertEqual(mapping["callsite_callee"], "57")
+        self.assertEqual(mapping["callsite_caller"], "190")
+
+        print("✓ Callsite parsing tests passed")
+
     def test_convert(self):
         """Test convert function with various data types"""
         # Test with primitive types
@@ -187,6 +243,70 @@ class TestTritonparseCPU(unittest.TestCase):
 
         assert convert(nested_structure) == expected_nested
 
+    def test_loc_alias_parsing(self):
+        """Test parsing of location aliases in TTIR/TTGIR"""
+        from tritonparse.ir_parser import extract_loc_definitions
+
+        # Test case 1: Bare #loc reference (no number)
+        ir_with_bare_loc = """
+module {
+  #loc = loc("/tmp/test.py":10:5)
+  #loc13 = loc("x_ptr"(#loc))
+  func @kernel(%arg0: !tt.ptr<f32> loc(#loc13)) {
+    return loc(#loc)
+  }
+}
+"""
+        locs = extract_loc_definitions(ir_with_bare_loc)
+        # Main #loc should be stored with "" key
+        assert "" in locs, "Main #loc not found"
+        assert locs[""]["file"] == "/tmp/test.py"
+        assert locs[""]["line"] == 10
+        # Alias #loc13 should resolve to same location
+        assert "13" in locs, "#loc13 not found"
+        assert locs["13"]["file"] == "/tmp/test.py"
+        assert locs["13"]["line"] == 10
+        assert locs["13"]["alias_name"] == "x_ptr"
+        assert locs["13"]["alias_of"] == ""
+
+        # Test case 2: Named alias with numbered reference
+        ir_with_numbered_alias = """
+#loc = loc("/tmp/test.py":5:0)
+#loc2 = loc("/tmp/test.py":20:28)
+#loc16 = loc("pid"(#loc2))
+%0 = tt.get_program_id x : i32 loc(#loc16)
+"""
+        locs = extract_loc_definitions(ir_with_numbered_alias)
+        assert "2" in locs
+        assert locs["2"]["line"] == 20
+        assert "16" in locs
+        assert locs["16"]["file"] == "/tmp/test.py"
+        assert locs["16"]["line"] == 20
+        assert locs["16"]["alias_name"] == "pid"
+        assert locs["16"]["alias_of"] == "2"
+
+        # Test case 3: Simple alias (no name)
+        ir_with_simple_alias = """
+#loc = loc("/tmp/test.py":1:1)
+#loc1 = loc("/tmp/test.py":15:10)
+#loc20 = loc(#loc1)
+%1 = arith.constant 0 : i32 loc(#loc20)
+"""
+        locs = extract_loc_definitions(ir_with_simple_alias)
+        assert "1" in locs
+        assert "20" in locs
+        assert locs["20"]["file"] == "/tmp/test.py"
+        assert locs["20"]["line"] == 15
+        assert locs["20"]["alias_of"] == "1"
+        assert "alias_name" not in locs["20"]
+
+        # Test case 4: Definition line tracking
+        assert "def_line" in locs[""]
+        assert "def_line" in locs["1"]
+        assert "def_line" in locs["20"]
+
+        print("✓ All loc alias parsing tests passed")
+
 
 class TestTritonparseCUDA(unittest.TestCase):
     """CUDA tests (require GPU)"""

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tritonparse/__main__.py

@@ -1,3 +1,5 @@
+#  Copyright (c) Meta Platforms, Inc. and affiliates.
+
 from .cli import main
 
 

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tritonparse/cli.py

@@ -68,17 +68,24 @@ def main():
         }
         unified_parse(**parse_args)
     elif args.func == "reproduce":
+        replacer = None
+        if args.use_fbcode:
+            from tritonparse.fb.reproducer.replacer import FBCodePlaceholderReplacer
+
+            replacer = FBCodePlaceholderReplacer()
+            print(f"Using FBCode placeholder replacer for template: {args.template}")
+
         reproduce(
             input_path=args.input,
-            line_index=args.line,
+            line_index=args.line - 1,  # Convert 1-based line number to 0-based index
             out_dir=args.out_dir,
             template=args.template,
             kernel_import=args.kernel_import,
+            replacer=replacer,
         )
     else:
         raise RuntimeError(f"Unknown command: {args.func}")
 
 
 if __name__ == "__main__":
-    # Do not add code here, it won't be run. Add them to the function called below.
     main()  # pragma: no cover

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tritonparse/common.py

@@ -1,7 +1,6 @@
 #  Copyright (c) Meta Platforms, Inc. and affiliates.
 
 import gzip
-
 import importlib
 import importlib.util
 import json
@@ -25,9 +24,9 @@ def is_fbcode():
 
 
 if is_fbcode():
-    from tritonparse.fb.source_type import SourceType
+    from .fb.source_type import SourceType
 else:
-    from tritonparse.source_type import SourceType
+    from .source_type import SourceType
 
 
 class Rank:

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tritonparse/context_manager.py

@@ -1,11 +1,12 @@
+#  Copyright (c) Meta Platforms, Inc. and affiliates.
+
 import os
 import shutil
 import tempfile
 
-from tritonparse.shared_vars import TEST_KEEP_OUTPUT
-
-from tritonparse.structured_logging import clear_logging_config, init
-from tritonparse.utils import unified_parse
+from .shared_vars import TEST_KEEP_OUTPUT
+from .structured_logging import clear_logging_config, init
+from .utils import unified_parse
 
 
 def createUniqueTempDirectory():

{tritonparse-0.3.1.dev20251018071448 → tritonparse-0.3.2.dev20251113071521}/tritonparse/event_diff.py

@@ -1,3 +1,5 @@
+#  Copyright (c) Meta Platforms, Inc. and affiliates.
+
 import json
 from collections import defaultdict
 from typing import Any, Dict, List, Tuple