tree_haver 2.0.0 → 3.0.0

data/README.md

@@ -67,7 +67,11 @@ If you've used [Faraday](https://github.com/lostisland/faraday), [multi_json](ht
 | **multi_xml**  | XML parsing         | Nokogiri, LibXML, Ox                                 |
 | **TreeHaver**  | tree-sitter parsing | ruby_tree_sitter, tree_stump, FFI, Java JARs, Citrus |
 
-**Write once, run anywhere.** Just as Faraday lets you swap HTTP adapters without changing your code, TreeHaver lets you swap tree-sitter backends. Your parsing code remains the same whether you're running on MRI with native C extensions, JRuby with FFI, or TruffleRuby.
+**Write once, run anywhere.** 
+
+**Learn once, write anywhere.**
+
+Just as Faraday lets you swap HTTP adapters without changing your code, TreeHaver lets you swap tree-sitter backends. Your parsing code remains the same whether you're running on MRI with native C extensions, JRuby with FFI, or TruffleRuby.
 
 ```ruby
 # Your code stays the same regardless of backend
@@ -76,7 +80,7 @@ parser.language = TreeHaver::Language.from_library("/path/to/grammar.so")
 tree = parser.parse(source_code)
 
 # TreeHaver automatically picks the best backend:
-# - MRI → ruby_tree_sitter (C extension)
+# - MRI → ruby_tree_sitter (C extensions)
 # - JRuby → FFI (system's libtree-sitter)
 # - TruffleRuby → FFI or MRI backend
 ```
@@ -97,6 +101,74 @@ tree = parser.parse(source_code)
 - **Thread-Safe**: Built-in language registry with thread-safe caching
 - **Minimal API Surface**: Simple, focused API that covers the most common tree-sitter use cases
 
+### Backend Requirements
+
+TreeHaver has minimal dependencies and automatically selects the best backend for your Ruby implementation. Each backend has specific version requirements:
+
+#### MRI Backend (ruby_tree_sitter, C extensions)
+
+**Requires `ruby_tree_sitter` v2.0+**
+
+In ruby_tree_sitter v2.0, all TreeSitter exceptions were changed to inherit from `Exception` (not `StandardError`). This was an intentional breaking change made for thread-safety and signal handling reasons.
+
+**Exception Mapping**: TreeHaver catches `TreeSitter::TreeSitterError` and its subclasses, converting them to `TreeHaver::NotAvailable` while preserving the original error message. This provides a consistent exception API across all backends:
+
+| ruby_tree_sitter Exception          | TreeHaver Exception        | When It Occurs                                 |
+|-------------------------------------|----------------------------|------------------------------------------------|
+| `TreeSitter::ParserNotFoundError`   | `TreeHaver::NotAvailable`  | Parser library file cannot be loaded           |
+| `TreeSitter::LanguageLoadError`     | `TreeHaver::NotAvailable`  | Language symbol loads but returns nothing      |
+| `TreeSitter::SymbolNotFoundError`   | `TreeHaver::NotAvailable`  | Symbol not found in library                    |
+| `TreeSitter::ParserVersionError`    | `TreeHaver::NotAvailable`  | Parser version incompatible with tree-sitter   |
+| `TreeSitter::QueryCreationError`    | `TreeHaver::NotAvailable`  | Query creation fails                           |
+
+```ruby
+# Add to your Gemfile for MRI backend
+gem "ruby_tree_sitter", "~> 2.0"
+```
+
+#### Rust Backend (tree_stump)
+
+Currently requires [pboling's fork](https://github.com/pboling/tree_stump/tree/tree_haver) until upstream PRs are merged.
+
+```ruby
+# Add to your Gemfile for Rust backend
+gem "tree_stump", github: "pboling/tree_stump", branch: "tree_haver"
+```
+
+#### FFI Backend
+
+Requires the `ffi` gem and a system installation of `libtree-sitter`:
+
+```ruby
+# Add to your Gemfile for FFI backend
+gem "ffi", ">= 1.15", "< 2.0"
+```
+
+```bash
+# Install libtree-sitter on your system:
+# macOS
+brew install tree-sitter
+
+# Ubuntu/Debian
+apt-get install libtree-sitter0 libtree-sitter-dev
+
+# Fedora
+dnf install tree-sitter tree-sitter-devel
+```
+
+#### Citrus Backend
+
+Pure Ruby parser with no native dependencies:
+
+```ruby
+# Add to your Gemfile for Citrus backend
+gem "citrus", "~> 3.0"
+```
+
+#### Java Backend (JRuby only)
+
+No additional dependencies required beyond grammar JARs built for java-tree-sitter.
+
 ### Why TreeHaver?
 
 tree-sitter is a powerful parser generator that creates incremental parsers for many programming languages. However, integrating it into Ruby applications can be challenging:
@@ -281,6 +353,108 @@ NOTE: Be prepared to track down certs for signed gems and add them the same way
 
 ## ⚙️ Configuration
 
+### Available Backends
+
+TreeHaver supports multiple parsing backends, each with different trade-offs. The `auto` backend automatically selects the best available option.
+
+| Backend | Description | Performance | Portability | Examples |
+|---------|-------------|-------------|-------------|----------|
+| **Auto** | Auto-selects best backend | Varies | ✅ Universal | [JSON](examples/auto_json.rb) · [JSONC](examples/auto_jsonc.rb) · [Bash](examples/auto_bash.rb) |
+| **MRI** | C extension via ruby_tree_sitter | ⚡ Fastest | MRI only | [JSON](examples/mri_json.rb) · [JSONC](examples/mri_jsonc.rb) · ~~Bash~~* |
+| **Rust** | Precompiled via tree_stump | ⚡ Very Fast | ✅ Good | [JSON](examples/rust_json.rb) · [JSONC](examples/rust_jsonc.rb) · ~~Bash~~* |
+| **FFI** | Dynamic linking via FFI | 🔵 Fast | ✅ Universal | [JSON](examples/ffi_json.rb) · [JSONC](examples/ffi_jsonc.rb) · [Bash](examples/ffi_bash.rb) |
+| **Java** | JNI bindings | ⚡ Very Fast | JRuby only | [JSON](examples/java_json.rb) · [JSONC](examples/java_jsonc.rb) · [Bash](examples/java_bash.rb) |
+| **Citrus** | Pure Ruby parsing | 🟡 Slower | ✅ Universal | [TOML](examples/citrus_toml.rb) · [Finitio](examples/citrus_finitio.rb) · [Dhall](examples/citrus_dhall.rb) |
+
+**Selection Priority (Auto mode):** MRI → Rust → FFI → Java → Citrus
+
+**Known Issues:**
+- *MRI + Bash: ABI incompatibility (use FFI instead)
+- *Rust + Bash: Version mismatch (use FFI instead)
+
+**Backend Requirements:**
+
+```ruby
+# MRI Backend
+gem 'ruby_tree_sitter'
+
+# Rust Backend  
+gem 'tree_stump'
+
+# FFI Backend
+gem 'ffi'
+
+# Citrus Backend
+gem 'citrus'
+# Plus grammar gems: toml-rb, dhall, finitio, etc.
+```
+
+**Force Specific Backend:**
+
+```ruby
+TreeHaver.backend = :ffi    # Force FFI backend
+TreeHaver.backend = :mri    # Force MRI backend
+TreeHaver.backend = :rust   # Force Rust backend
+TreeHaver.backend = :java   # Force Java backend (JRuby)
+TreeHaver.backend = :citrus # Force Citrus backend
+TreeHaver.backend = :auto   # Auto-select (default)
+```
+
+**Block-based Backend Switching:**
+
+Use `with_backend` to temporarily switch backends for a specific block of code.
+This is thread-safe and supports nesting—the previous backend is automatically
+restored when the block exits (even if an exception is raised).
+
+```ruby
+# Temporarily use a specific backend
+TreeHaver.with_backend(:mri) do
+  parser = TreeHaver::Parser.new
+  tree = parser.parse(source)
+  # All operations in this block use the MRI backend
+end
+# Backend is restored to its previous value here
+
+# Nested blocks work correctly
+TreeHaver.with_backend(:rust) do
+  # Uses :rust
+  TreeHaver.with_backend(:citrus) do
+    # Uses :citrus
+    parser = TreeHaver::Parser.new
+  end
+  # Back to :rust
+end
+# Back to original backend
+```
+
+This is particularly useful for:
+
+- **Testing**: Test the same code with different backends
+- **Performance comparison**: Benchmark different backends
+- **Fallback scenarios**: Try one backend, fall back to another
+- **Thread isolation**: Each thread can use a different backend safely
+
+```ruby
+# Example: Testing with multiple backends
+[:mri, :rust, :citrus].each do |backend_name|
+  TreeHaver.with_backend(backend_name) do
+    parser = TreeHaver::Parser.new
+    result = parser.parse(source)
+    puts "#{backend_name}: #{result.root_node.type}"
+  end
+end
+```
+
+**Check Backend Capabilities:**
+
+```ruby
+TreeHaver.backend              # => :ffi
+TreeHaver.backend_module       # => TreeHaver::Backends::FFI
+TreeHaver.capabilities         # => { backend: :ffi, parse: true, query: false, ... }
+```
+
+See [examples/](examples/) directory for 18 complete working examples demonstrating all backends and languages.
+
 ### Security Considerations
 
 **⚠️ Loading shared libraries (.so/.dylib/.dll) executes arbitrary native code.**
@@ -591,6 +765,64 @@ parser = TreeSitter::Parser.new  # Actually creates TreeHaver::Parser
 
 This is safe and idempotent—if the real `TreeSitter` module is already loaded, the shim does nothing.
 
+#### ⚠️ Critical: Exception Hierarchy Incompatibility
+
+**ruby_tree_sitter v2+ exceptions inherit from `Exception` (not `StandardError`).**  
+**TreeHaver exceptions follow Ruby best practices and inherit from `StandardError`.**
+
+This means exception handling behaves **differently** between the two:
+
+| Scenario | ruby_tree_sitter v2+ | TreeHaver Compat Mode |
+|----------|---------------------|----------------------|
+| `rescue => e` | Does NOT catch TreeSitter errors | DOES catch TreeHaver errors |
+| Behavior | Errors propagate (inherit Exception) | Errors caught (inherit StandardError) |
+
+**Example showing the difference:**
+
+```ruby
+# With real ruby_tree_sitter v2+
+begin
+  TreeSitter::Language.load("missing", "/nonexistent.so")
+rescue => e
+  puts "Caught!"  # Never reached - TreeSitter errors inherit Exception
+end
+
+# With TreeHaver compat mode
+require "tree_haver/compat"
+begin
+  TreeSitter::Language.load("missing", "/nonexistent.so")  # Actually TreeHaver
+rescue => e
+  puts "Caught!"  # WILL be reached - TreeHaver errors inherit StandardError
+end
+```
+
+**To write compatible exception handling:**
+
+```ruby
+# Option 1: Catch specific exception (works with both)
+begin
+  TreeSitter::Language.load(...)
+rescue TreeSitter::TreeSitterError => e  # Explicit rescue
+  # Works with both ruby_tree_sitter and TreeHaver compat mode
+end
+
+# Option 2: Use TreeHaver API directly (recommended)
+begin
+  TreeHaver::Language.from_library(...)
+rescue TreeHaver::NotAvailable => e  # TreeHaver's unified exception
+  # Clear and consistent when using TreeHaver
+end
+```
+
+**Why TreeHaver uses StandardError:**
+
+1. **Ruby Best Practice**: The [Ruby style guide](https://rubystyle.guide/#exception-flow-control) recommends inheriting from `StandardError`
+2. **Safety**: Inheriting from `Exception` can catch system signals (`SIGTERM`, `SIGINT`) and `exit`, which is dangerous
+3. **Consistency**: Most Ruby libraries follow this convention
+4. **Testability**: StandardError exceptions are easier to test and mock
+
+See `lib/tree_haver/compat.rb` for detailed documentation.
+
 ## 🔧 Basic Usage
 
 ### Quick Start
@@ -657,6 +889,38 @@ parser.language = toml_language
 tree = parser.parse(toml_source)
 ```
 
+#### Flexible Language Names
+
+The `name` parameter in `register_language` is an arbitrary identifier you choose—it doesn't
+need to match the actual language name. The actual grammar identity comes from the `path`
+and `symbol` parameters (for tree-sitter) or `grammar_module` (for Citrus).
+
+This flexibility is useful for:
+
+- **Aliasing**: Register the same grammar under multiple names
+- **Versioning**: Register different grammar versions (e.g., `:ruby_2`, `:ruby_3`)
+- **Testing**: Use unique names to avoid collisions between tests
+- **Context-specific naming**: Use names that make sense for your application
+
+```ruby
+# Register the same TOML grammar under different names for different purposes
+TreeHaver.register_language(
+  :config_parser,  # Custom name for your app
+  path: "/usr/local/lib/libtree-sitter-toml.so",
+  symbol: "tree_sitter_toml",
+)
+
+TreeHaver.register_language(
+  :toml_v1,  # Version-specific name
+  path: "/usr/local/lib/libtree-sitter-toml.so",
+  symbol: "tree_sitter_toml",
+)
+
+# Use your custom names
+config_lang = TreeHaver::Language.config_parser
+versioned_lang = TreeHaver::Language.toml_v1
+```
+
 ### Parsing Different Languages
 
 TreeHaver works with any tree-sitter grammar:
@@ -875,23 +1139,90 @@ TreeHaver.backend = :mri
 TreeHaver.backend = :citrus
 ```
 
-### Advanced: Testing with Multiple Backends
+### Advanced: Thread-Safe Backend Switching
+
+TreeHaver provides `with_backend` for thread-safe, temporary backend switching. This is
+essential for testing, benchmarking, and applications that need different backends in
+different contexts.
 
-If you're developing a library that uses TreeHaver, you can test against different backends:
+#### Testing with Multiple Backends
+
+Test the same code path with different backends using `with_backend`:
 
 ```ruby
 # In your test setup
 RSpec.describe("MyParser") do
-  before do
-    TreeHaver.reset_backend!(to: :ffi)
+  # Test with each available backend
+  [:mri, :rust, :citrus].each do |backend_name|
+    context "with #{backend_name} backend" do
+      it "parses correctly" do
+        TreeHaver.with_backend(backend_name) do
+          parser = TreeHaver::Parser.new
+          result = parser.parse("x = 42")
+          expect(result.root_node.type).to eq("document")
+        end
+        # Backend automatically restored after block
+      end
+    end
   end
+end
+```
+
+#### Thread Isolation
+
+Each thread can use a different backend safely—`with_backend` uses thread-local storage:
 
-  after do
-    TreeHaver.reset_backend!(to: :auto)
+```ruby
+threads = []
+
+threads << Thread.new do
+  TreeHaver.with_backend(:mri) do
+    # This thread uses MRI backend
+    parser = TreeHaver::Parser.new
+    100.times { parser.parse("x = 1") }
   end
+end
 
-  it "parses correctly with FFI backend" do
-    # Your test code
+threads << Thread.new do
+  TreeHaver.with_backend(:citrus) do
+    # This thread uses Citrus backend simultaneously
+    parser = TreeHaver::Parser.new
+    100.times { parser.parse("x = 1") }
+  end
+end
+
+threads.each(&:join)
+```
+
+#### Nested Blocks
+
+`with_backend` supports nesting—inner blocks override outer blocks:
+
+```ruby
+TreeHaver.with_backend(:rust) do
+  puts TreeHaver.effective_backend  # => :rust
+
+  TreeHaver.with_backend(:citrus) do
+    puts TreeHaver.effective_backend  # => :citrus
+  end
+
+  puts TreeHaver.effective_backend  # => :rust (restored)
+end
+```
+
+#### Fallback Pattern
+
+Try one backend, fall back to another on failure:
+
+```ruby
+def parse_with_fallback(source)
+  TreeHaver.with_backend(:mri) do
+    TreeHaver::Parser.new.tap { |p| p.language = load_language }.parse(source)
+  end
+rescue TreeHaver::NotAvailable
+  # Fall back to Citrus if MRI backend unavailable
+  TreeHaver.with_backend(:citrus) do
+    TreeHaver::Parser.new.tap { |p| p.language = load_language }.parse(source)
   end
 end
 ```
@@ -1292,7 +1623,7 @@ Thanks for RTFM. ☺️
 [📌gitmoji]: https://gitmoji.dev
 [📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
 [🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
-[🧮kloc-img]: https://img.shields.io/badge/KLOC-0.726-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
+[🧮kloc-img]: https://img.shields.io/badge/KLOC-1.067-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
 [🔐security]: SECURITY.md
 [🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
 [📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year

data/lib/tree_haver/backends/citrus.rb

@@ -86,10 +86,16 @@ module TreeHaver
       #   # For TOML, use toml-rb's grammar
       #   language = TreeHaver::Backends::Citrus::Language.new(TomlRB::Document)
       class Language
+        include Comparable
+
         # The Citrus grammar module
         # @return [Module] Citrus grammar module (e.g., TomlRB::Document)
         attr_reader :grammar_module
 
+        # The backend this language is for
+        # @return [Symbol]
+        attr_reader :backend
+
         # @param grammar_module [Module] A Citrus grammar module with a parse method
         def initialize(grammar_module)
           unless grammar_module.respond_to?(:parse)
@@ -98,8 +104,33 @@ module TreeHaver
                 "Expected a Citrus grammar module (e.g., TomlRB::Document)."
           end
           @grammar_module = grammar_module
+          @backend = :citrus
+        end
+
+        # Compare languages for equality
+        #
+        # Citrus languages are equal if they have the same backend and grammar_module.
+        # Grammar module uniquely identifies a Citrus language.
+        #
+        # @param other [Object] object to compare with
+        # @return [Integer, nil] -1, 0, 1, or nil if not comparable
+        def <=>(other)
+          return unless other.is_a?(Language)
+          return unless other.backend == @backend
+
+          # Compare by grammar_module name (modules are compared by object_id by default)
+          @grammar_module.name <=> other.grammar_module.name
+        end
+
+        # Hash value for this language (for use in Sets/Hashes)
+        # @return [Integer]
+        def hash
+          [@backend, @grammar_module.name].hash
         end
 
+        # Alias eql? to ==
+        alias_method :eql?, :==
+
         # Not applicable for Citrus (tree-sitter-specific)
         #
         # Citrus grammars are Ruby modules, not shared libraries.
@@ -131,30 +162,29 @@ module TreeHaver
 
         # Set the grammar for this parser
         #
-        # @param grammar [Language, Module] Citrus grammar module or Language wrapper
-        # @return [Language, Module] the grammar that was set
+        # Note: TreeHaver::Parser unwraps language objects before calling this method.
+        # This backend receives the raw Citrus grammar module (unwrapped), not the Language wrapper.
+        #
+        # @param grammar [Module] Citrus grammar module with a parse method
+        # @return [void]
         # @example
         #   require "toml-rb"
-        #   parser.language = TomlRB::Document  # Pass module directly
-        #   # or
-        #   parser.language = TreeHaver::Backends::Citrus::Language.new(TomlRB::Document)
+        #   # TreeHaver::Parser unwraps Language.new(TomlRB::Document) to just TomlRB::Document
+        #   parser.language = TomlRB::Document  # Backend receives unwrapped module
         def language=(grammar)
-          @grammar = if grammar.respond_to?(:grammar_module)
-            grammar.grammar_module
-          elsif grammar.respond_to?(:parse)
-            grammar
-          else
+          # grammar is already unwrapped by TreeHaver::Parser
+          unless grammar.respond_to?(:parse)
             raise ArgumentError,
-              "Expected Citrus grammar module or Language wrapper, " \
+              "Expected Citrus grammar module with parse method, " \
                 "got #{grammar.class}"
           end
-          grammar
+          @grammar = grammar
         end
 
         # Parse source code
         #
         # @param source [String] the source code to parse
-        # @return [TreeHaver::Tree] wrapped tree
+        # @return [Tree] raw backend tree (wrapping happens in TreeHaver::Parser)
         # @raise [TreeHaver::NotAvailable] if no grammar is set
         # @raise [::Citrus::ParseError] if parsing fails
         def parse(source)
@@ -162,8 +192,8 @@ module TreeHaver
 
           begin
             citrus_match = @grammar.parse(source)
-            inner_tree = Tree.new(citrus_match, source)
-            TreeHaver::Tree.new(inner_tree, source: source)
+            # Return raw Citrus::Tree - TreeHaver::Parser will wrap it
+            Tree.new(citrus_match, source)
           rescue ::Citrus::ParseError => e
             # Re-raise with more context
             raise TreeHaver::Error, "Parse error: #{e.message}"
@@ -176,8 +206,8 @@ module TreeHaver
         #
         # @param old_tree [TreeHaver::Tree, nil] ignored (no incremental parsing support)
         # @param source [String] the source code to parse
-        # @return [TreeHaver::Tree] wrapped tree
-        def parse_string(old_tree, source)
+        # @return [Tree] raw backend tree (wrapping happens in TreeHaver::Parser)
+        def parse_string(old_tree, source) # rubocop:disable Lint/UnusedMethodArgument
           parse(source)  # Citrus doesn't support incremental parsing
         end
       end
@@ -213,6 +243,10 @@ module TreeHaver
       # - matches: child matches
       # - captures: named groups
       #
+      # Language-specific helpers can be mixed in for convenience:
+      #   require "tree_haver/backends/citrus/toml_helpers"
+      #   TreeHaver::Backends::Citrus::Node.include(TreeHaver::Backends::Citrus::TomlHelpers)
+      #
       # @api private
       class Node
         attr_reader :match, :source
@@ -224,17 +258,104 @@ module TreeHaver
 
         # Get node type from Citrus rule name
         #
+        # Uses Citrus grammar introspection to dynamically determine node types.
+        # Works with any Citrus grammar without language-specific knowledge.
+        #
+        # Strategy:
+        # 1. Check if first event has a .name method (returns Symbol) - use that
+        # 2. If first event is a Symbol directly - use that
+        # 3. For compound rules (Repeat, Choice), recurse into first match
+        #
         # @return [String] rule name from grammar
         def type
-          # Citrus stores the rule name in events[0]
           return "unknown" unless @match.respond_to?(:events)
           return "unknown" unless @match.events.is_a?(Array)
           return "unknown" if @match.events.empty?
 
-          first = @match.events.first
-          first.is_a?(Symbol) ? first.to_s : "unknown"
+          extract_type_from_event(@match.events.first)
+        end
+
+        # Check if this node represents a structural element vs a terminal/token
+        #
+        # Uses Citrus grammar's terminal? method to determine if this is
+        # a structural rule (like "table", "keyvalue") vs a terminal token
+        # (like "[", "=", whitespace).
+        #
+        # @return [Boolean] true if this is a structural (non-terminal) node
+        def structural?
+          return false unless @match.respond_to?(:events)
+          return false if @match.events.empty?
+
+          first_event = @match.events.first
+
+          # Check if event has terminal? method (Citrus rule object)
+          if first_event.respond_to?(:terminal?)
+            return !first_event.terminal?
+          end
+
+          # For Symbol events, try to look up in grammar
+          if first_event.is_a?(Symbol) && @match.respond_to?(:grammar)
+            grammar = @match.grammar
+            if grammar.respond_to?(:rules) && grammar.rules.key?(first_event)
+              rule = grammar.rules[first_event]
+              return !rule.terminal? if rule.respond_to?(:terminal?)
+            end
+          end
+
+          # Default: assume structural if not a simple string/regex terminal
+          true
+        end
+
+        private
+
+        # Extract type name from a Citrus event object
+        #
+        # Handles different event types:
+        # - Objects with .name method (Citrus rule objects) -> use .name
+        # - Symbol -> use directly
+        # - Compound rules (Repeat, Choice) -> check string representation
+        #
+        # @param event [Object] Citrus event object
+        # @return [String] type name
+        def extract_type_from_event(event)
+          # Case 1: Event has .name method (returns Symbol)
+          if event.respond_to?(:name)
+            name = event.name
+            return name.to_s if name.is_a?(Symbol)
+          end
+
+          # Case 2: Event is a Symbol directly (most common for child nodes)
+          return event.to_s if event.is_a?(Symbol)
+
+          # Case 3: Event is a String
+          return event if event.is_a?(String)
+
+          # Case 4: For compound rules (Repeat, Choice), try string parsing first
+          # This avoids recursion issues
+          str = event.to_s
+
+          # Try to extract rule name from string representation
+          # Examples: "table", "(comment | table)*", "space?", etc.
+          if str =~ /^([a-z_][a-z0-9_]*)/i
+            return $1
+          end
+
+          # If we have a pattern like "(rule1 | rule2)*", we can't determine
+          # the type without looking at actual matches, but that causes recursion
+          # So just return a generic type based on the pattern
+          if /^\(.*\)\*$/.match?(str)
+            return "repeat"
+          elsif /^\(.*\)\?$/.match?(str)
+            return "optional"
+          elsif /^.*\|.*$/.match?(str)
+            return "choice"
+          end
+
+          "unknown"
         end
 
+        public
+
         def start_byte
           @match.offset
         end