tree_haver 1.0.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +236 -3
- data/CONTRIBUTING.md +100 -0
- data/README.md +470 -85
- data/lib/tree_haver/backends/citrus.rb +423 -0
- data/lib/tree_haver/backends/ffi.rb +405 -150
- data/lib/tree_haver/backends/java.rb +63 -10
- data/lib/tree_haver/backends/mri.rb +154 -27
- data/lib/tree_haver/backends/rust.rb +58 -27
- data/lib/tree_haver/citrus_grammar_finder.rb +170 -0
- data/lib/tree_haver/grammar_finder.rb +42 -7
- data/lib/tree_haver/language_registry.rb +62 -71
- data/lib/tree_haver/node.rb +526 -0
- data/lib/tree_haver/path_validator.rb +47 -27
- data/lib/tree_haver/tree.rb +259 -0
- data/lib/tree_haver/version.rb +2 -2
- data/lib/tree_haver.rb +741 -285
- data/sig/tree_haver/backends.rbs +68 -1
- data/sig/tree_haver/path_validator.rbs +1 -0
- data/sig/tree_haver.rbs +95 -9
- data.tar.gz.sig +0 -0
- metadata +12 -8
- metadata.gz.sig +0 -0
data/README.md
CHANGED
|
@@ -54,20 +54,24 @@
|
|
|
54
54
|
|
|
55
55
|
## 🌻 Synopsis
|
|
56
56
|
|
|
57
|
-
TreeHaver is a cross-Ruby adapter for the [
|
|
57
|
+
TreeHaver is a cross-Ruby adapter for the [tree-sitter](https://tree-sitter.github.io/tree-sitter/) parsing library that works seamlessly across MRI Ruby, JRuby, and TruffleRuby. It provides a unified API for parsing source code using tree-sitter grammars, regardless of your Ruby implementation.
|
|
58
58
|
|
|
59
59
|
### The Adapter Pattern: Like Faraday, but for Parsing
|
|
60
60
|
|
|
61
61
|
If you've used [Faraday](https://github.com/lostisland/faraday), [multi_json](https://github.com/intridea/multi_json), or [multi_xml](https://github.com/sferik/multi_xml), you'll feel right at home with TreeHaver. These gems share a common philosophy:
|
|
62
62
|
|
|
63
|
-
| Gem
|
|
64
|
-
|
|
65
|
-
| **Faraday**
|
|
66
|
-
| **multi_json** | JSON parsing
|
|
67
|
-
| **multi_xml**
|
|
68
|
-
| **TreeHaver**
|
|
63
|
+
| Gem | Unified API for | Backend Examples |
|
|
64
|
+
|----------------|---------------------|------------------------------------------------------|
|
|
65
|
+
| **Faraday** | HTTP requests | Net::HTTP, Typhoeus, Patron, Excon |
|
|
66
|
+
| **multi_json** | JSON parsing | Oj, Yajl, JSON gem |
|
|
67
|
+
| **multi_xml** | XML parsing | Nokogiri, LibXML, Ox |
|
|
68
|
+
| **TreeHaver** | tree-sitter parsing | ruby_tree_sitter, tree_stump, FFI, Java JARs, Citrus |
|
|
69
69
|
|
|
70
|
-
**Write once, run anywhere.**
|
|
70
|
+
**Write once, run anywhere.**
|
|
71
|
+
|
|
72
|
+
**Learn once, write anywhere.**
|
|
73
|
+
|
|
74
|
+
Just as Faraday lets you swap HTTP adapters without changing your code, TreeHaver lets you swap tree-sitter backends. Your parsing code remains the same whether you're running on MRI with native C extensions, JRuby with FFI, or TruffleRuby.
|
|
71
75
|
|
|
72
76
|
```ruby
|
|
73
77
|
# Your code stays the same regardless of backend
|
|
@@ -76,7 +80,7 @@ parser.language = TreeHaver::Language.from_library("/path/to/grammar.so")
|
|
|
76
80
|
tree = parser.parse(source_code)
|
|
77
81
|
|
|
78
82
|
# TreeHaver automatically picks the best backend:
|
|
79
|
-
# - MRI → ruby_tree_sitter (C
|
|
83
|
+
# - MRI → ruby_tree_sitter (C extensions)
|
|
80
84
|
# - JRuby → FFI (system's libtree-sitter)
|
|
81
85
|
# - TruffleRuby → FFI or MRI backend
|
|
82
86
|
```
|
|
@@ -90,15 +94,84 @@ tree = parser.parse(source_code)
|
|
|
90
94
|
- **Note**: Currently requires [pboling's fork](https://github.com/pboling/tree_stump/tree/tree_haver) until PRs [#5](https://github.com/joker1007/tree_stump/pull/5), [#7](https://github.com/joker1007/tree_stump/pull/7), [#11](https://github.com/joker1007/tree_stump/pull/11), and [#13 (inclusive of the others)](https://github.com/joker1007/tree_stump/pull/13) are merged
|
|
91
95
|
- **FFI Backend**: Pure Ruby FFI bindings to `libtree-sitter` (ideal for JRuby)
|
|
92
96
|
- **Java Backend**: Support for JRuby's native Java integration, and native java-tree-sitter grammar JARs
|
|
97
|
+
- **Citrus Backend**: Pure Ruby parser using [`citrus`](https://github.com/mjackson/citrus) gem (no native dependencies, portable)
|
|
93
98
|
- **Automatic Backend Selection**: Intelligently selects the best backend for your Ruby implementation
|
|
94
|
-
- **Language Agnostic**: Load any
|
|
99
|
+
- **Language Agnostic**: Load any tree-sitter grammar dynamically (TOML, JSON, Ruby, JavaScript, etc.)
|
|
95
100
|
- **Grammar Discovery**: Built-in `GrammarFinder` utility for platform-aware grammar library discovery
|
|
96
101
|
- **Thread-Safe**: Built-in language registry with thread-safe caching
|
|
97
|
-
- **Minimal API Surface**: Simple, focused API that covers the most common
|
|
102
|
+
- **Minimal API Surface**: Simple, focused API that covers the most common tree-sitter use cases
|
|
103
|
+
|
|
104
|
+
### Backend Requirements
|
|
105
|
+
|
|
106
|
+
TreeHaver has minimal dependencies and automatically selects the best backend for your Ruby implementation. Each backend has specific version requirements:
|
|
107
|
+
|
|
108
|
+
#### MRI Backend (ruby_tree_sitter, C extensions)
|
|
109
|
+
|
|
110
|
+
**Requires `ruby_tree_sitter` v2.0+**
|
|
111
|
+
|
|
112
|
+
In ruby_tree_sitter v2.0, all TreeSitter exceptions were changed to inherit from `Exception` (not `StandardError`). This was an intentional breaking change made for thread-safety and signal handling reasons.
|
|
113
|
+
|
|
114
|
+
**Exception Mapping**: TreeHaver catches `TreeSitter::TreeSitterError` and its subclasses, converting them to `TreeHaver::NotAvailable` while preserving the original error message. This provides a consistent exception API across all backends:
|
|
115
|
+
|
|
116
|
+
| ruby_tree_sitter Exception | TreeHaver Exception | When It Occurs |
|
|
117
|
+
|-------------------------------------|----------------------------|------------------------------------------------|
|
|
118
|
+
| `TreeSitter::ParserNotFoundError` | `TreeHaver::NotAvailable` | Parser library file cannot be loaded |
|
|
119
|
+
| `TreeSitter::LanguageLoadError` | `TreeHaver::NotAvailable` | Language symbol loads but returns nothing |
|
|
120
|
+
| `TreeSitter::SymbolNotFoundError` | `TreeHaver::NotAvailable` | Symbol not found in library |
|
|
121
|
+
| `TreeSitter::ParserVersionError` | `TreeHaver::NotAvailable` | Parser version incompatible with tree-sitter |
|
|
122
|
+
| `TreeSitter::QueryCreationError` | `TreeHaver::NotAvailable` | Query creation fails |
|
|
123
|
+
|
|
124
|
+
```ruby
|
|
125
|
+
# Add to your Gemfile for MRI backend
|
|
126
|
+
gem "ruby_tree_sitter", "~> 2.0"
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
#### Rust Backend (tree_stump)
|
|
130
|
+
|
|
131
|
+
Currently requires [pboling's fork](https://github.com/pboling/tree_stump/tree/tree_haver) until upstream PRs are merged.
|
|
132
|
+
|
|
133
|
+
```ruby
|
|
134
|
+
# Add to your Gemfile for Rust backend
|
|
135
|
+
gem "tree_stump", github: "pboling/tree_stump", branch: "tree_haver"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
#### FFI Backend
|
|
139
|
+
|
|
140
|
+
Requires the `ffi` gem and a system installation of `libtree-sitter`:
|
|
141
|
+
|
|
142
|
+
```ruby
|
|
143
|
+
# Add to your Gemfile for FFI backend
|
|
144
|
+
gem "ffi", ">= 1.15", "< 2.0"
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
# Install libtree-sitter on your system:
|
|
149
|
+
# macOS
|
|
150
|
+
brew install tree-sitter
|
|
151
|
+
|
|
152
|
+
# Ubuntu/Debian
|
|
153
|
+
apt-get install libtree-sitter0 libtree-sitter-dev
|
|
154
|
+
|
|
155
|
+
# Fedora
|
|
156
|
+
dnf install tree-sitter tree-sitter-devel
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
#### Citrus Backend
|
|
160
|
+
|
|
161
|
+
Pure Ruby parser with no native dependencies:
|
|
162
|
+
|
|
163
|
+
```ruby
|
|
164
|
+
# Add to your Gemfile for Citrus backend
|
|
165
|
+
gem "citrus", "~> 3.0"
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
#### Java Backend (JRuby only)
|
|
169
|
+
|
|
170
|
+
No additional dependencies required beyond grammar JARs built for java-tree-sitter.
|
|
98
171
|
|
|
99
172
|
### Why TreeHaver?
|
|
100
173
|
|
|
101
|
-
|
|
174
|
+
tree-sitter is a powerful parser generator that creates incremental parsers for many programming languages. However, integrating it into Ruby applications can be challenging:
|
|
102
175
|
|
|
103
176
|
- MRI-based C extensions don't work on JRuby
|
|
104
177
|
- FFI-based solutions may not be optimal for MRI
|
|
@@ -106,25 +179,28 @@ Tree-sitter is a powerful parser generator that creates incremental parsers for
|
|
|
106
179
|
|
|
107
180
|
TreeHaver solves these problems by providing a unified API that automatically selects the appropriate backend for your Ruby implementation, allowing you to write code once and run it anywhere.
|
|
108
181
|
|
|
109
|
-
### Comparison with Other Ruby
|
|
110
|
-
|
|
111
|
-
| Feature |
|
|
112
|
-
|
|
113
|
-
| **MRI Ruby** | ✅ Yes
|
|
114
|
-
| **JRuby** | ✅ Yes (FFI or
|
|
115
|
-
| **TruffleRuby** | ✅ Yes (FFI)
|
|
116
|
-
| **Backend** | Multi (MRI C, Rust, FFI, Java) | C extension only | Rust extension |
|
|
117
|
-
| **Incremental Parsing** | ✅ Via MRI C/Rust backend
|
|
118
|
-
| **Query API** | ⚡ Via MRI/Rust backend
|
|
119
|
-
| **Grammar Discovery** | ✅ Built-in `GrammarFinder`
|
|
120
|
-
| **Security Validations** | ✅ `PathValidator`
|
|
121
|
-
| **Language Registration** | ✅ Thread-safe registry
|
|
122
|
-
| **Native Performance** | ⚡ Backend-dependent
|
|
123
|
-
| **Precompiled Binaries** | ⚡ Via Rust backend
|
|
124
|
-
| **
|
|
182
|
+
### Comparison with Other Ruby AST / Parser Bindings
|
|
183
|
+
|
|
184
|
+
| Feature | [tree_haver] (this gem) | [ruby_tree_sitter] | [tree_stump] | [citrus] |
|
|
185
|
+
|---------------------------|----------------------------------------|--------------------|----------------|-------------|
|
|
186
|
+
| **MRI Ruby** | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
|
|
187
|
+
| **JRuby** | ✅ Yes (FFI, Java, or Citrus backend) | ❌ No | ❌ No | ✅ Yes |
|
|
188
|
+
| **TruffleRuby** | ✅ Yes (FFI or Citrus) | ❌ No | ❓ Unknown | ✅ Yes |
|
|
189
|
+
| **Backend** | Multi (MRI C, Rust, FFI, Java, Citrus) | C extension only | Rust extension | Pure Ruby |
|
|
190
|
+
| **Incremental Parsing** | ✅ Via MRI C/Rust/Java backend | ✅ Yes | ✅ Yes | ❌ No |
|
|
191
|
+
| **Query API** | ⚡ Via MRI/Rust/Java backend | ✅ Yes | ✅ Yes | ❌ No |
|
|
192
|
+
| **Grammar Discovery** | ✅ Built-in `GrammarFinder` | ❌ Manual | ❌ Manual | ❌ Manual |
|
|
193
|
+
| **Security Validations** | ✅ `PathValidator` | ❌ No | ❌ No | ❌ No |
|
|
194
|
+
| **Language Registration** | ✅ Thread-safe registry | ❌ No | ❌ No | ❌ No |
|
|
195
|
+
| **Native Performance** | ⚡ Backend-dependent | ✅ Native C | ✅ Native Rust | ❌ Pure Ruby |
|
|
196
|
+
| **Precompiled Binaries** | ⚡ Via Rust backend | ✅ Yes | ✅ Yes | ✅ Pure Ruby |
|
|
197
|
+
| **Zero Native Deps** | ⚡ Via Citrus backend | ❌ No | ❌ No | ✅ Yes |
|
|
198
|
+
| **Minimum Ruby** | 3.2+ | 3.0+ | 3.1+ | 0+ |
|
|
125
199
|
|
|
126
200
|
[ruby_tree_sitter]: https://github.com/Faveod/ruby-tree-sitter
|
|
127
201
|
[tree_stump]: https://github.com/anthropics/tree_stump
|
|
202
|
+
[citrus]: https://github.com/mjackson/citrus
|
|
203
|
+
[tree_haver]: https://github.com/kettle-rb/tree_haver
|
|
128
204
|
|
|
129
205
|
**Note:** Java backend works with grammar JARs built specifically for java-tree-sitter, or grammar .so files that statically link tree-sitter. This is why FFI is recommended for JRuby & TruffleRuby.
|
|
130
206
|
|
|
@@ -135,6 +211,7 @@ TreeHaver solves these problems by providing a unified API that automatically se
|
|
|
135
211
|
#### When to Use Each
|
|
136
212
|
|
|
137
213
|
**Choose TreeHaver when:**
|
|
214
|
+
|
|
138
215
|
- You need JRuby or TruffleRuby support
|
|
139
216
|
- You're building a library that should work across Ruby implementations
|
|
140
217
|
- You want automatic grammar discovery and security validations
|
|
@@ -142,39 +219,48 @@ TreeHaver solves these problems by providing a unified API that automatically se
|
|
|
142
219
|
- You need incremental parsing with a unified API
|
|
143
220
|
|
|
144
221
|
**Choose ruby_tree_sitter directly when:**
|
|
222
|
+
|
|
145
223
|
- You only target MRI Ruby
|
|
146
224
|
- You need the full Query API without abstraction
|
|
147
225
|
- You want the most battle-tested C bindings
|
|
148
226
|
- You don't need TreeHaver's grammar discovery
|
|
149
227
|
|
|
150
228
|
**Choose tree_stump directly when:**
|
|
229
|
+
|
|
151
230
|
- You only target MRI Ruby
|
|
152
231
|
- You prefer Rust-based native extensions
|
|
153
232
|
- You want precompiled binaries without system dependencies
|
|
154
233
|
- You don't need TreeHaver's grammar discovery
|
|
155
234
|
- **Note:** Use [pboling's fork (tree_haver branch)](https://github.com/pboling/tree_stump/tree/tree_haver) until PRs [#5](https://github.com/joker1007/tree_stump/pull/5), [#7](https://github.com/joker1007/tree_stump/pull/7), [#11](https://github.com/joker1007/tree_stump/pull/11), [#13](https://github.com/joker1007/tree_stump/pull/13) are merged
|
|
156
235
|
|
|
236
|
+
**Choose citrus directly when:**
|
|
237
|
+
|
|
238
|
+
- You need zero native dependencies (pure Ruby)
|
|
239
|
+
- You're using a Citrus grammar (not tree-sitter grammars)
|
|
240
|
+
- Performance is less critical than portability
|
|
241
|
+
- You don't need TreeHaver's unified API
|
|
242
|
+
|
|
157
243
|
## 💡 Info you can shake a stick at
|
|
158
244
|
|
|
159
|
-
| Tokens to Remember | [![Gem name][⛳️name-img]][⛳️gem-name] [![Gem namespace][⛳️namespace-img]][⛳️gem-namespace]
|
|
160
|
-
|
|
161
|
-
| Works with JRuby | [![JRuby 10.0 Compat][💎jruby-c-i]][🚎11-c-wf] [![JRuby HEAD Compat][💎jruby-headi]][🚎3-hd-wf]
|
|
162
|
-
| Works with Truffle Ruby | [![Truffle Ruby 23.1 Compat][💎truby-23.1i]][🚎9-t-wf] [![Truffle Ruby 24.1 Compat][💎truby-c-i]][🚎11-c-wf]
|
|
163
|
-
| Works with MRI Ruby 3 | [![Ruby 3.2 Compat][💎ruby-3.2i]][🚎6-s-wf] [![Ruby 3.3 Compat][💎ruby-3.3i]][🚎6-s-wf] [![Ruby 3.4 Compat][💎ruby-c-i]][🚎11-c-wf] [![Ruby HEAD Compat][💎ruby-headi]][🚎3-hd-wf]
|
|
164
|
-
| Support & Community | [![Join Me on Daily.dev's RubyFriends][✉️ruby-friends-img]][✉️ruby-friends] [![Live Chat on Discord][✉️discord-invite-img-ftb]][✉️discord-invite] [![Get help from me on Upwork][👨🏼🏫expsup-upwork-img]][👨🏼🏫expsup-upwork] [![Get help from me on Codementor][👨🏼🏫expsup-codementor-img]][👨🏼🏫expsup-codementor]
|
|
165
|
-
| Source | [![Source on GitLab.com][📜src-gl-img]][📜src-gl] [![Source on CodeBerg.org][📜src-cb-img]][📜src-cb] [![Source on Github.com][📜src-gh-img]][📜src-gh] [![The best SHA: dQw4w9WgXcQ!][🧮kloc-img]][🧮kloc]
|
|
166
|
-
| Documentation | [![Current release on RubyDoc.info][📜docs-cr-rd-img]][🚎yard-current] [![YARD on Galtzo.com][📜docs-head-rd-img]][🚎yard-head] [![Maintainer Blog][🚂maint-blog-img]][🚂maint-blog] [![GitLab Wiki][📜gl-wiki-img]][📜gl-wiki] [![GitHub Wiki][📜gh-wiki-img]][📜gh-wiki]
|
|
245
|
+
| Tokens to Remember | [![Gem name][⛳️name-img]][⛳️gem-name] [![Gem namespace][⛳️namespace-img]][⛳️gem-namespace] |
|
|
246
|
+
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
247
|
+
| Works with JRuby | [![JRuby 10.0 Compat][💎jruby-c-i]][🚎11-c-wf] [![JRuby HEAD Compat][💎jruby-headi]][🚎3-hd-wf] |
|
|
248
|
+
| Works with Truffle Ruby | [![Truffle Ruby 23.1 Compat][💎truby-23.1i]][🚎9-t-wf] [![Truffle Ruby 24.1 Compat][💎truby-c-i]][🚎11-c-wf] |
|
|
249
|
+
| Works with MRI Ruby 3 | [![Ruby 3.2 Compat][💎ruby-3.2i]][🚎6-s-wf] [![Ruby 3.3 Compat][💎ruby-3.3i]][🚎6-s-wf] [![Ruby 3.4 Compat][💎ruby-c-i]][🚎11-c-wf] [![Ruby HEAD Compat][💎ruby-headi]][🚎3-hd-wf] |
|
|
250
|
+
| Support & Community | [![Join Me on Daily.dev's RubyFriends][✉️ruby-friends-img]][✉️ruby-friends] [![Live Chat on Discord][✉️discord-invite-img-ftb]][✉️discord-invite] [![Get help from me on Upwork][👨🏼🏫expsup-upwork-img]][👨🏼🏫expsup-upwork] [![Get help from me on Codementor][👨🏼🏫expsup-codementor-img]][👨🏼🏫expsup-codementor] |
|
|
251
|
+
| Source | [![Source on GitLab.com][📜src-gl-img]][📜src-gl] [![Source on CodeBerg.org][📜src-cb-img]][📜src-cb] [![Source on Github.com][📜src-gh-img]][📜src-gh] [![The best SHA: dQw4w9WgXcQ!][🧮kloc-img]][🧮kloc] |
|
|
252
|
+
| Documentation | [![Current release on RubyDoc.info][📜docs-cr-rd-img]][🚎yard-current] [![YARD on Galtzo.com][📜docs-head-rd-img]][🚎yard-head] [![Maintainer Blog][🚂maint-blog-img]][🚂maint-blog] [![GitLab Wiki][📜gl-wiki-img]][📜gl-wiki] [![GitHub Wiki][📜gh-wiki-img]][📜gh-wiki] |
|
|
167
253
|
| Compliance | [![License: MIT][📄license-img]][📄license-ref] [![Compatible with Apache Software Projects: Verified by SkyWalking Eyes][📄license-compat-img]][📄license-compat] [![📄ilo-declaration-img]][📄ilo-declaration] [![Security Policy][🔐security-img]][🔐security] [![Contributor Covenant 2.1][🪇conduct-img]][🪇conduct] [![SemVer 2.0.0][📌semver-img]][📌semver] |
|
|
168
|
-
| Style | [![Enforced Code Style Linter][💎rlts-img]][💎rlts] [![Keep-A-Changelog 1.0.0][📗keep-changelog-img]][📗keep-changelog] [![Gitmoji Commits][📌gitmoji-img]][📌gitmoji] [![Compatibility appraised by: appraisal2][💎appraisal2-img]][💎appraisal2]
|
|
169
|
-
| Maintainer 🎖️
|
|
170
|
-
| `...` 💖 | [![Find Me on WellFound:][💖✌️wellfound-img]][💖✌️wellfound] [![Find Me on CrunchBase][💖💲crunchbase-img]][💖💲crunchbase] [![My LinkTree][💖🌳linktree-img]][💖🌳linktree] [![More About Me][💖💁🏼♂️aboutme-img]][💖💁🏼♂️aboutme] [🧊][💖🧊berg] [🐙][💖🐙hub]
|
|
254
|
+
| Style | [![Enforced Code Style Linter][💎rlts-img]][💎rlts] [![Keep-A-Changelog 1.0.0][📗keep-changelog-img]][📗keep-changelog] [![Gitmoji Commits][📌gitmoji-img]][📌gitmoji] [![Compatibility appraised by: appraisal2][💎appraisal2-img]][💎appraisal2] |
|
|
255
|
+
| Maintainer 🎖️ | [![Follow Me on LinkedIn][💖🖇linkedin-img]][💖🖇linkedin] [![Follow Me on Ruby.Social][💖🐘ruby-mast-img]][💖🐘ruby-mast] [![Follow Me on Bluesky][💖🦋bluesky-img]][💖🦋bluesky] [![Contact Maintainer][🚂maint-contact-img]][🚂maint-contact] [![My technical writing][💖💁🏼♂️devto-img]][💖💁🏼♂️devto] |
|
|
256
|
+
| `...` 💖 | [![Find Me on WellFound:][💖✌️wellfound-img]][💖✌️wellfound] [![Find Me on CrunchBase][💖💲crunchbase-img]][💖💲crunchbase] [![My LinkTree][💖🌳linktree-img]][💖🌳linktree] [![More About Me][💖💁🏼♂️aboutme-img]][💖💁🏼♂️aboutme] [🧊][💖🧊berg] [🐙][💖🐙hub] [🛖][💖🛖hut] [🧪][💖🧪lab] |
|
|
171
257
|
|
|
172
258
|
### Compatibility
|
|
173
259
|
|
|
174
260
|
Compatible with MRI Ruby 3.2.0+, and concordant releases of JRuby, and TruffleRuby.
|
|
175
261
|
|
|
176
|
-
| 🚚 _Amazing_ test matrix was brought to you by | 🔎 appraisal2 🔎 and the color 💚 green 💚
|
|
177
|
-
|
|
262
|
+
| 🚚 _Amazing_ test matrix was brought to you by | 🔎 appraisal2 🔎 and the color 💚 green 💚 |
|
|
263
|
+
| ---------------------------------------------- | -------------------------------------------------------- |
|
|
178
264
|
| 👟 Check it out! | ✨ [github.com/appraisal-rb/appraisal2][💎appraisal2] ✨ |
|
|
179
265
|
|
|
180
266
|
### Federated DVCS
|
|
@@ -183,9 +269,9 @@ Compatible with MRI Ruby 3.2.0+, and concordant releases of JRuby, and TruffleRu
|
|
|
183
269
|
<summary>Find this repo on federated forges (Coming soon!)</summary>
|
|
184
270
|
|
|
185
271
|
| Federated [DVCS][💎d-in-dvcs] Repository | Status | Issues | PRs | Wiki | CI | Discussions |
|
|
186
|
-
|
|
187
|
-
| 🧪 [kettle-rb/tree_haver on GitLab][📜src-gl] | The Truth | [💚][🤝gl-issues] | [💚][🤝gl-pulls] | [💚][📜gl-wiki] | 🐭 Tiny Matrix | ➖
|
|
188
|
-
| 🧊 [kettle-rb/tree_haver on CodeBerg][📜src-cb] | An Ethical Mirror ([Donate][🤝cb-donate]) | [💚][🤝cb-issues] | [💚][🤝cb-pulls] | ➖
|
|
272
|
+
| ----------------------------------------------- | --------------------------------------------------------------------- | ------------------------- | ------------------------ | ------------------------- | ------------------------ | ---------------------------- |
|
|
273
|
+
| 🧪 [kettle-rb/tree_haver on GitLab][📜src-gl] | The Truth | [💚][🤝gl-issues] | [💚][🤝gl-pulls] | [💚][📜gl-wiki] | 🐭 Tiny Matrix | ➖ |
|
|
274
|
+
| 🧊 [kettle-rb/tree_haver on CodeBerg][📜src-cb] | An Ethical Mirror ([Donate][🤝cb-donate]) | [💚][🤝cb-issues] | [💚][🤝cb-pulls] | ➖ | ⭕️ No Matrix | ➖ |
|
|
189
275
|
| 🐙 [kettle-rb/tree_haver on GitHub][📜src-gh] | Another Mirror | [💚][🤝gh-issues] | [💚][🤝gh-pulls] | [💚][📜gh-wiki] | 💯 Full Matrix | [💚][gh-discussions] |
|
|
190
276
|
| 🎮️ [Discord Server][✉️discord-invite] | [![Live Chat on Discord][✉️discord-invite-img-ftb]][✉️discord-invite] | [Let's][✉️discord-invite] | [talk][✉️discord-invite] | [about][✉️discord-invite] | [this][✉️discord-invite] | [library!][✉️discord-invite] |
|
|
191
277
|
|
|
@@ -206,7 +292,7 @@ The maintainers of this and thousands of other packages are working with Tidelif
|
|
|
206
292
|
|
|
207
293
|
- 💡Subscribe for support guarantees covering _all_ your FLOSS dependencies
|
|
208
294
|
- 💡Tidelift is part of [Sonar][🏙️entsup-tidelift-sonar]
|
|
209
|
-
- 💡Tidelift pays maintainers to maintain the software you depend on!<br/>📊`@`Pointy Haired Boss: An [enterprise support][🏙️entsup-tidelift] subscription is "[never gonna let you down][🧮kloc]", and
|
|
295
|
+
- 💡Tidelift pays maintainers to maintain the software you depend on!<br/>📊`@`Pointy Haired Boss: An [enterprise support][🏙️entsup-tidelift] subscription is "[never gonna let you down][🧮kloc]", and _supports_ open source maintainers
|
|
210
296
|
|
|
211
297
|
Alternatively:
|
|
212
298
|
|
|
@@ -245,7 +331,7 @@ Add my public key (if you haven’t already, expires 2045-04-29) as a trusted ce
|
|
|
245
331
|
gem cert --add <(curl -Ls https://raw.github.com/galtzo-floss/certs/main/pboling.pem)
|
|
246
332
|
```
|
|
247
333
|
|
|
248
|
-
You only need to do that once.
|
|
334
|
+
You only need to do that once. Then proceed to install with:
|
|
249
335
|
|
|
250
336
|
```console
|
|
251
337
|
gem install tree_haver -P HighSecurity
|
|
@@ -267,6 +353,108 @@ NOTE: Be prepared to track down certs for signed gems and add them the same way
|
|
|
267
353
|
|
|
268
354
|
## ⚙️ Configuration
|
|
269
355
|
|
|
356
|
+
### Available Backends
|
|
357
|
+
|
|
358
|
+
TreeHaver supports multiple parsing backends, each with different trade-offs. The `auto` backend automatically selects the best available option.
|
|
359
|
+
|
|
360
|
+
| Backend | Description | Performance | Portability | Examples |
|
|
361
|
+
|---------|-------------|-------------|-------------|----------|
|
|
362
|
+
| **Auto** | Auto-selects best backend | Varies | ✅ Universal | [JSON](examples/auto_json.rb) · [JSONC](examples/auto_jsonc.rb) · [Bash](examples/auto_bash.rb) |
|
|
363
|
+
| **MRI** | C extension via ruby_tree_sitter | ⚡ Fastest | MRI only | [JSON](examples/mri_json.rb) · [JSONC](examples/mri_jsonc.rb) · ~~Bash~~* |
|
|
364
|
+
| **Rust** | Precompiled via tree_stump | ⚡ Very Fast | ✅ Good | [JSON](examples/rust_json.rb) · [JSONC](examples/rust_jsonc.rb) · ~~Bash~~* |
|
|
365
|
+
| **FFI** | Dynamic linking via FFI | 🔵 Fast | ✅ Universal | [JSON](examples/ffi_json.rb) · [JSONC](examples/ffi_jsonc.rb) · [Bash](examples/ffi_bash.rb) |
|
|
366
|
+
| **Java** | JNI bindings | ⚡ Very Fast | JRuby only | [JSON](examples/java_json.rb) · [JSONC](examples/java_jsonc.rb) · [Bash](examples/java_bash.rb) |
|
|
367
|
+
| **Citrus** | Pure Ruby parsing | 🟡 Slower | ✅ Universal | [TOML](examples/citrus_toml.rb) · [Finitio](examples/citrus_finitio.rb) · [Dhall](examples/citrus_dhall.rb) |
|
|
368
|
+
|
|
369
|
+
**Selection Priority (Auto mode):** MRI → Rust → FFI → Java → Citrus
|
|
370
|
+
|
|
371
|
+
**Known Issues:**
|
|
372
|
+
- *MRI + Bash: ABI incompatibility (use FFI instead)
|
|
373
|
+
- *Rust + Bash: Version mismatch (use FFI instead)
|
|
374
|
+
|
|
375
|
+
**Backend Requirements:**
|
|
376
|
+
|
|
377
|
+
```ruby
|
|
378
|
+
# MRI Backend
|
|
379
|
+
gem 'ruby_tree_sitter'
|
|
380
|
+
|
|
381
|
+
# Rust Backend
|
|
382
|
+
gem 'tree_stump'
|
|
383
|
+
|
|
384
|
+
# FFI Backend
|
|
385
|
+
gem 'ffi'
|
|
386
|
+
|
|
387
|
+
# Citrus Backend
|
|
388
|
+
gem 'citrus'
|
|
389
|
+
# Plus grammar gems: toml-rb, dhall, finitio, etc.
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
**Force Specific Backend:**
|
|
393
|
+
|
|
394
|
+
```ruby
|
|
395
|
+
TreeHaver.backend = :ffi # Force FFI backend
|
|
396
|
+
TreeHaver.backend = :mri # Force MRI backend
|
|
397
|
+
TreeHaver.backend = :rust # Force Rust backend
|
|
398
|
+
TreeHaver.backend = :java # Force Java backend (JRuby)
|
|
399
|
+
TreeHaver.backend = :citrus # Force Citrus backend
|
|
400
|
+
TreeHaver.backend = :auto # Auto-select (default)
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
**Block-based Backend Switching:**
|
|
404
|
+
|
|
405
|
+
Use `with_backend` to temporarily switch backends for a specific block of code.
|
|
406
|
+
This is thread-safe and supports nesting—the previous backend is automatically
|
|
407
|
+
restored when the block exits (even if an exception is raised).
|
|
408
|
+
|
|
409
|
+
```ruby
|
|
410
|
+
# Temporarily use a specific backend
|
|
411
|
+
TreeHaver.with_backend(:mri) do
|
|
412
|
+
parser = TreeHaver::Parser.new
|
|
413
|
+
tree = parser.parse(source)
|
|
414
|
+
# All operations in this block use the MRI backend
|
|
415
|
+
end
|
|
416
|
+
# Backend is restored to its previous value here
|
|
417
|
+
|
|
418
|
+
# Nested blocks work correctly
|
|
419
|
+
TreeHaver.with_backend(:rust) do
|
|
420
|
+
# Uses :rust
|
|
421
|
+
TreeHaver.with_backend(:citrus) do
|
|
422
|
+
# Uses :citrus
|
|
423
|
+
parser = TreeHaver::Parser.new
|
|
424
|
+
end
|
|
425
|
+
# Back to :rust
|
|
426
|
+
end
|
|
427
|
+
# Back to original backend
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
This is particularly useful for:
|
|
431
|
+
|
|
432
|
+
- **Testing**: Test the same code with different backends
|
|
433
|
+
- **Performance comparison**: Benchmark different backends
|
|
434
|
+
- **Fallback scenarios**: Try one backend, fall back to another
|
|
435
|
+
- **Thread isolation**: Each thread can use a different backend safely
|
|
436
|
+
|
|
437
|
+
```ruby
|
|
438
|
+
# Example: Testing with multiple backends
|
|
439
|
+
[:mri, :rust, :citrus].each do |backend_name|
|
|
440
|
+
TreeHaver.with_backend(backend_name) do
|
|
441
|
+
parser = TreeHaver::Parser.new
|
|
442
|
+
result = parser.parse(source)
|
|
443
|
+
puts "#{backend_name}: #{result.root_node.type}"
|
|
444
|
+
end
|
|
445
|
+
end
|
|
446
|
+
```
|
|
447
|
+
|
|
448
|
+
**Check Backend Capabilities:**
|
|
449
|
+
|
|
450
|
+
```ruby
|
|
451
|
+
TreeHaver.backend # => :ffi
|
|
452
|
+
TreeHaver.backend_module # => TreeHaver::Backends::FFI
|
|
453
|
+
TreeHaver.capabilities # => { backend: :ffi, parse: true, query: false, ... }
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
See [examples/](examples/) directory for 18 complete working examples demonstrating all backends and languages.
|
|
457
|
+
|
|
270
458
|
### Security Considerations
|
|
271
459
|
|
|
272
460
|
**⚠️ Loading shared libraries (.so/.dylib/.dll) executes arbitrary native code.**
|
|
@@ -359,15 +547,18 @@ TreeHaver automatically selects the best backend for your Ruby implementation, b
|
|
|
359
547
|
TreeHaver.backend = :auto
|
|
360
548
|
|
|
361
549
|
# Force a specific backend
|
|
362
|
-
TreeHaver.backend = :mri
|
|
363
|
-
TreeHaver.backend = :rust
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
TreeHaver.backend = :ffi
|
|
367
|
-
TreeHaver.backend = :java
|
|
550
|
+
TreeHaver.backend = :mri # Use ruby_tree_sitter (MRI only, C extension)
|
|
551
|
+
TreeHaver.backend = :rust # Use tree_stump (MRI, Rust extension with precompiled binaries)
|
|
552
|
+
# Note: Requires pboling's fork until PRs #5, #7, #11, #13 are merged
|
|
553
|
+
# See: https://github.com/pboling/tree_stump/tree/tree_haver
|
|
554
|
+
TreeHaver.backend = :ffi # Use FFI bindings (works on MRI and JRuby)
|
|
555
|
+
TreeHaver.backend = :java # Use Java bindings (JRuby only, coming soon)
|
|
556
|
+
TreeHaver.backend = :citrus # Use Citrus pure Ruby parser
|
|
557
|
+
# NOTE: Portable, all Ruby implementations
|
|
558
|
+
# CAVEAT: few major language grammars, but many esoteric grammars
|
|
368
559
|
```
|
|
369
560
|
|
|
370
|
-
**Auto-selection priority on MRI:** MRI → Rust → FFI
|
|
561
|
+
**Auto-selection priority on MRI:** MRI → Rust → FFI → Citrus
|
|
371
562
|
|
|
372
563
|
You can also set the backend via environment variable:
|
|
373
564
|
|
|
@@ -384,6 +575,7 @@ TreeHaver recognizes several environment variables for configuration:
|
|
|
384
575
|
#### Security Configuration
|
|
385
576
|
|
|
386
577
|
- **`TREE_HAVER_TRUSTED_DIRS`**: Comma-separated list of additional trusted directories for grammar libraries
|
|
578
|
+
|
|
387
579
|
```bash
|
|
388
580
|
# For Homebrew on Linux and luarocks
|
|
389
581
|
export TREE_HAVER_TRUSTED_DIRS="/home/linuxbrew/.linuxbrew/Cellar,~/.local/share/mise/installs/lua"
|
|
@@ -399,11 +591,12 @@ TreeHaver recognizes several environment variables for configuration:
|
|
|
399
591
|
```
|
|
400
592
|
|
|
401
593
|
If not set, TreeHaver tries these names in order:
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
594
|
+
|
|
595
|
+
- `tree-sitter`
|
|
596
|
+
- `libtree-sitter.so.0`
|
|
597
|
+
- `libtree-sitter.so`
|
|
598
|
+
- `libtree-sitter.dylib`
|
|
599
|
+
- `libtree-sitter.dll`
|
|
407
600
|
|
|
408
601
|
#### Language Symbol Resolution
|
|
409
602
|
|
|
@@ -468,7 +661,7 @@ if finder.available?
|
|
|
468
661
|
puts "TOML grammar found at: #{finder.find_library_path}"
|
|
469
662
|
else
|
|
470
663
|
puts finder.not_found_message
|
|
471
|
-
# => "
|
|
664
|
+
# => "tree-sitter toml grammar not found. Searched: /usr/lib/libtree-sitter-toml.so, ..."
|
|
472
665
|
end
|
|
473
666
|
|
|
474
667
|
# Register the language if available
|
|
@@ -482,11 +675,11 @@ language = TreeHaver::Language.toml
|
|
|
482
675
|
|
|
483
676
|
Given just the language name, `GrammarFinder` automatically derives:
|
|
484
677
|
|
|
485
|
-
| Property
|
|
486
|
-
|
|
487
|
-
| ENV var
|
|
678
|
+
| Property | Derived Value (for `:toml`) |
|
|
679
|
+
| ---------------- | ---------------------------------------------------- |
|
|
680
|
+
| ENV var | `TREE_SITTER_TOML_PATH` |
|
|
488
681
|
| Library filename | `libtree-sitter-toml.so` (Linux) or `.dylib` (macOS) |
|
|
489
|
-
| Symbol name
|
|
682
|
+
| Symbol name | `tree_sitter_toml` |
|
|
490
683
|
|
|
491
684
|
#### Search Order
|
|
492
685
|
|
|
@@ -496,7 +689,7 @@ Given just the language name, `GrammarFinder` automatically derives:
|
|
|
496
689
|
2. **Extra paths**: Custom paths provided at initialization
|
|
497
690
|
3. **System paths**: Common installation directories (`/usr/lib`, `/usr/local/lib`, `/opt/homebrew/lib`, etc.)
|
|
498
691
|
|
|
499
|
-
#### Usage in
|
|
692
|
+
#### Usage in \*-merge Gems
|
|
500
693
|
|
|
501
694
|
The `GrammarFinder` pattern enables clean integration in language-specific merge gems:
|
|
502
695
|
|
|
@@ -555,6 +748,8 @@ TreeHaver.capabilities
|
|
|
555
748
|
# => { backend: :mri, query: true, bytes_field: true }
|
|
556
749
|
# or
|
|
557
750
|
# => { backend: :ffi, parse: true, query: false, bytes_field: true }
|
|
751
|
+
# or
|
|
752
|
+
# => { backend: :citrus, parse: true, query: false, bytes_field: false }
|
|
558
753
|
```
|
|
559
754
|
|
|
560
755
|
### Compatibility Mode
|
|
@@ -570,6 +765,64 @@ parser = TreeSitter::Parser.new # Actually creates TreeHaver::Parser
|
|
|
570
765
|
|
|
571
766
|
This is safe and idempotent—if the real `TreeSitter` module is already loaded, the shim does nothing.
|
|
572
767
|
|
|
768
|
+
#### ⚠️ Critical: Exception Hierarchy Incompatibility
|
|
769
|
+
|
|
770
|
+
**ruby_tree_sitter v2+ exceptions inherit from `Exception` (not `StandardError`).**
|
|
771
|
+
**TreeHaver exceptions follow Ruby best practices and inherit from `StandardError`.**
|
|
772
|
+
|
|
773
|
+
This means exception handling behaves **differently** between the two:
|
|
774
|
+
|
|
775
|
+
| Scenario | ruby_tree_sitter v2+ | TreeHaver Compat Mode |
|
|
776
|
+
|----------|---------------------|----------------------|
|
|
777
|
+
| `rescue => e` | Does NOT catch TreeSitter errors | DOES catch TreeHaver errors |
|
|
778
|
+
| Behavior | Errors propagate (inherit Exception) | Errors caught (inherit StandardError) |
|
|
779
|
+
|
|
780
|
+
**Example showing the difference:**
|
|
781
|
+
|
|
782
|
+
```ruby
|
|
783
|
+
# With real ruby_tree_sitter v2+
|
|
784
|
+
begin
|
|
785
|
+
TreeSitter::Language.load("missing", "/nonexistent.so")
|
|
786
|
+
rescue => e
|
|
787
|
+
puts "Caught!" # Never reached - TreeSitter errors inherit Exception
|
|
788
|
+
end
|
|
789
|
+
|
|
790
|
+
# With TreeHaver compat mode
|
|
791
|
+
require "tree_haver/compat"
|
|
792
|
+
begin
|
|
793
|
+
TreeSitter::Language.load("missing", "/nonexistent.so") # Actually TreeHaver
|
|
794
|
+
rescue => e
|
|
795
|
+
puts "Caught!" # WILL be reached - TreeHaver errors inherit StandardError
|
|
796
|
+
end
|
|
797
|
+
```
|
|
798
|
+
|
|
799
|
+
**To write compatible exception handling:**
|
|
800
|
+
|
|
801
|
+
```ruby
|
|
802
|
+
# Option 1: Catch specific exception (works with both)
|
|
803
|
+
begin
|
|
804
|
+
TreeSitter::Language.load(...)
|
|
805
|
+
rescue TreeSitter::TreeSitterError => e # Explicit rescue
|
|
806
|
+
# Works with both ruby_tree_sitter and TreeHaver compat mode
|
|
807
|
+
end
|
|
808
|
+
|
|
809
|
+
# Option 2: Use TreeHaver API directly (recommended)
|
|
810
|
+
begin
|
|
811
|
+
TreeHaver::Language.from_library(...)
|
|
812
|
+
rescue TreeHaver::NotAvailable => e # TreeHaver's unified exception
|
|
813
|
+
# Clear and consistent when using TreeHaver
|
|
814
|
+
end
|
|
815
|
+
```
|
|
816
|
+
|
|
817
|
+
**Why TreeHaver uses StandardError:**
|
|
818
|
+
|
|
819
|
+
1. **Ruby Best Practice**: The [Ruby style guide](https://rubystyle.guide/#exception-flow-control) recommends inheriting from `StandardError`
|
|
820
|
+
2. **Safety**: Inheriting from `Exception` can catch system signals (`SIGTERM`, `SIGINT`) and `exit`, which is dangerous
|
|
821
|
+
3. **Consistency**: Most Ruby libraries follow this convention
|
|
822
|
+
4. **Testability**: StandardError exceptions are easier to test and mock
|
|
823
|
+
|
|
824
|
+
See `lib/tree_haver/compat.rb` for detailed documentation.
|
|
825
|
+
|
|
573
826
|
## 🔧 Basic Usage
|
|
574
827
|
|
|
575
828
|
### Quick Start
|
|
@@ -636,9 +889,41 @@ parser.language = toml_language
|
|
|
636
889
|
tree = parser.parse(toml_source)
|
|
637
890
|
```
|
|
638
891
|
|
|
892
|
+
#### Flexible Language Names
|
|
893
|
+
|
|
894
|
+
The `name` parameter in `register_language` is an arbitrary identifier you choose—it doesn't
|
|
895
|
+
need to match the actual language name. The actual grammar identity comes from the `path`
|
|
896
|
+
and `symbol` parameters (for tree-sitter) or `grammar_module` (for Citrus).
|
|
897
|
+
|
|
898
|
+
This flexibility is useful for:
|
|
899
|
+
|
|
900
|
+
- **Aliasing**: Register the same grammar under multiple names
|
|
901
|
+
- **Versioning**: Register different grammar versions (e.g., `:ruby_2`, `:ruby_3`)
|
|
902
|
+
- **Testing**: Use unique names to avoid collisions between tests
|
|
903
|
+
- **Context-specific naming**: Use names that make sense for your application
|
|
904
|
+
|
|
905
|
+
```ruby
|
|
906
|
+
# Register the same TOML grammar under different names for different purposes
|
|
907
|
+
TreeHaver.register_language(
|
|
908
|
+
:config_parser, # Custom name for your app
|
|
909
|
+
path: "/usr/local/lib/libtree-sitter-toml.so",
|
|
910
|
+
symbol: "tree_sitter_toml",
|
|
911
|
+
)
|
|
912
|
+
|
|
913
|
+
TreeHaver.register_language(
|
|
914
|
+
:toml_v1, # Version-specific name
|
|
915
|
+
path: "/usr/local/lib/libtree-sitter-toml.so",
|
|
916
|
+
symbol: "tree_sitter_toml",
|
|
917
|
+
)
|
|
918
|
+
|
|
919
|
+
# Use your custom names
|
|
920
|
+
config_lang = TreeHaver::Language.config_parser
|
|
921
|
+
versioned_lang = TreeHaver::Language.toml_v1
|
|
922
|
+
```
|
|
923
|
+
|
|
639
924
|
### Parsing Different Languages
|
|
640
925
|
|
|
641
|
-
TreeHaver works with any
|
|
926
|
+
TreeHaver works with any tree-sitter grammar:
|
|
642
927
|
|
|
643
928
|
```ruby
|
|
644
929
|
# Parse Ruby code
|
|
@@ -704,7 +989,7 @@ tree.edit(
|
|
|
704
989
|
new_tree = parser.parse_string(tree, "x = 42")
|
|
705
990
|
```
|
|
706
991
|
|
|
707
|
-
**Note:** Incremental parsing requires the MRI (`ruby_tree_sitter`), Rust (`tree_stump`), or Java (`java-tree-sitter`) backend. The FFI
|
|
992
|
+
**Note:** Incremental parsing requires the MRI (`ruby_tree_sitter`), Rust (`tree_stump`), or Java (`java-tree-sitter`) backend. The FFI and Citrus backends do not currently support incremental parsing. You can check support with:
|
|
708
993
|
|
|
709
994
|
**Note:** `tree_stump` requires [pboling's fork (tree_haver branch)](https://github.com/pboling/tree_stump/tree/tree_haver) until PRs [#5](https://github.com/joker1007/tree_stump/pull/5), [#7](https://github.com/joker1007/tree_stump/pull/7), [#11](https://github.com/joker1007/tree_stump/pull/11), [#13](https://github.com/joker1007/tree_stump/pull/13) are merged.
|
|
710
995
|
|
|
@@ -724,7 +1009,7 @@ end
|
|
|
724
1009
|
# Check if a backend is available
|
|
725
1010
|
if TreeHaver.backend_module.nil?
|
|
726
1011
|
puts "No TreeHaver backend is available!"
|
|
727
|
-
puts "Install ruby_tree_sitter (MRI)
|
|
1012
|
+
puts "Install ruby_tree_sitter (MRI), ffi gem with libtree-sitter, or citrus gem"
|
|
728
1013
|
end
|
|
729
1014
|
```
|
|
730
1015
|
|
|
@@ -745,9 +1030,9 @@ parser = TreeHaver::Parser.new
|
|
|
745
1030
|
|
|
746
1031
|
#### JRuby
|
|
747
1032
|
|
|
748
|
-
On JRuby, TreeHaver can use
|
|
1033
|
+
On JRuby, TreeHaver can use the FFI backend, Java backend, or Citrus backend:
|
|
749
1034
|
|
|
750
|
-
**Option 1: FFI Backend (
|
|
1035
|
+
**Option 1: FFI Backend (recommended for tree-sitter grammars)**
|
|
751
1036
|
|
|
752
1037
|
```ruby
|
|
753
1038
|
# Gemfile
|
|
@@ -809,38 +1094,135 @@ TreeHaver.backend = :ffi
|
|
|
809
1094
|
The FFI backend uses Ruby's FFI gem which relies on the system's dynamic linker, correctly resolving symbol dependencies between `libtree-sitter.so` and grammar libraries.
|
|
810
1095
|
|
|
811
1096
|
The Java backend will work with:
|
|
1097
|
+
|
|
812
1098
|
- Grammar JARs built specifically for java-tree-sitter (self-contained)
|
|
813
1099
|
- Grammar `.so` files that statically link tree-sitter
|
|
814
1100
|
|
|
1101
|
+
**Option 3: Citrus Backend (pure Ruby, portable)**
|
|
1102
|
+
|
|
1103
|
+
```ruby
|
|
1104
|
+
# Gemfile
|
|
1105
|
+
gem "tree_haver"
|
|
1106
|
+
gem "citrus" # Pure Ruby parser, zero native dependencies
|
|
1107
|
+
|
|
1108
|
+
# Code - Force Citrus backend for maximum portability
|
|
1109
|
+
TreeHaver.backend = :citrus
|
|
1110
|
+
|
|
1111
|
+
# Check if Citrus backend is available
|
|
1112
|
+
if TreeHaver::Backends::Citrus.available?
|
|
1113
|
+
puts "Citrus backend is ready!"
|
|
1114
|
+
puts TreeHaver.capabilities
|
|
1115
|
+
# => { backend: :citrus, parse: true, query: false, bytes_field: false }
|
|
1116
|
+
end
|
|
1117
|
+
```
|
|
1118
|
+
|
|
1119
|
+
**⚠️ Citrus Backend Limitations:**
|
|
1120
|
+
|
|
1121
|
+
- Uses Citrus grammars (not tree-sitter grammars)
|
|
1122
|
+
- No incremental parsing support
|
|
1123
|
+
- No query API
|
|
1124
|
+
- Pure Ruby performance (slower than native backends)
|
|
1125
|
+
- Best for: prototyping, environments without native extension support, teaching
|
|
1126
|
+
|
|
815
1127
|
#### TruffleRuby
|
|
816
1128
|
|
|
817
|
-
TruffleRuby can use
|
|
1129
|
+
TruffleRuby can use the MRI, FFI, or Citrus backend:
|
|
818
1130
|
|
|
819
1131
|
```ruby
|
|
820
|
-
# Use FFI backend (recommended)
|
|
1132
|
+
# Use FFI backend (recommended for tree-sitter grammars)
|
|
821
1133
|
TreeHaver.backend = :ffi
|
|
822
1134
|
|
|
823
1135
|
# Or try MRI backend if ruby_tree_sitter compiles on your TruffleRuby version
|
|
824
1136
|
TreeHaver.backend = :mri
|
|
1137
|
+
|
|
1138
|
+
# Or use Citrus backend for zero native dependencies
|
|
1139
|
+
TreeHaver.backend = :citrus
|
|
825
1140
|
```
|
|
826
1141
|
|
|
827
|
-
### Advanced:
|
|
1142
|
+
### Advanced: Thread-Safe Backend Switching
|
|
1143
|
+
|
|
1144
|
+
TreeHaver provides `with_backend` for thread-safe, temporary backend switching. This is
|
|
1145
|
+
essential for testing, benchmarking, and applications that need different backends in
|
|
1146
|
+
different contexts.
|
|
828
1147
|
|
|
829
|
-
|
|
1148
|
+
#### Testing with Multiple Backends
|
|
1149
|
+
|
|
1150
|
+
Test the same code path with different backends using `with_backend`:
|
|
830
1151
|
|
|
831
1152
|
```ruby
|
|
832
1153
|
# In your test setup
|
|
833
1154
|
RSpec.describe("MyParser") do
|
|
834
|
-
|
|
835
|
-
|
|
1155
|
+
# Test with each available backend
|
|
1156
|
+
[:mri, :rust, :citrus].each do |backend_name|
|
|
1157
|
+
context "with #{backend_name} backend" do
|
|
1158
|
+
it "parses correctly" do
|
|
1159
|
+
TreeHaver.with_backend(backend_name) do
|
|
1160
|
+
parser = TreeHaver::Parser.new
|
|
1161
|
+
result = parser.parse("x = 42")
|
|
1162
|
+
expect(result.root_node.type).to eq("document")
|
|
1163
|
+
end
|
|
1164
|
+
# Backend automatically restored after block
|
|
1165
|
+
end
|
|
1166
|
+
end
|
|
1167
|
+
end
|
|
1168
|
+
end
|
|
1169
|
+
```
|
|
1170
|
+
|
|
1171
|
+
#### Thread Isolation
|
|
1172
|
+
|
|
1173
|
+
Each thread can use a different backend safely—`with_backend` uses thread-local storage:
|
|
1174
|
+
|
|
1175
|
+
```ruby
|
|
1176
|
+
threads = []
|
|
1177
|
+
|
|
1178
|
+
threads << Thread.new do
|
|
1179
|
+
TreeHaver.with_backend(:mri) do
|
|
1180
|
+
# This thread uses MRI backend
|
|
1181
|
+
parser = TreeHaver::Parser.new
|
|
1182
|
+
100.times { parser.parse("x = 1") }
|
|
1183
|
+
end
|
|
1184
|
+
end
|
|
1185
|
+
|
|
1186
|
+
threads << Thread.new do
|
|
1187
|
+
TreeHaver.with_backend(:citrus) do
|
|
1188
|
+
# This thread uses Citrus backend simultaneously
|
|
1189
|
+
parser = TreeHaver::Parser.new
|
|
1190
|
+
100.times { parser.parse("x = 1") }
|
|
836
1191
|
end
|
|
1192
|
+
end
|
|
1193
|
+
|
|
1194
|
+
threads.each(&:join)
|
|
1195
|
+
```
|
|
1196
|
+
|
|
1197
|
+
#### Nested Blocks
|
|
837
1198
|
|
|
838
|
-
|
|
839
|
-
|
|
1199
|
+
`with_backend` supports nesting—inner blocks override outer blocks:
|
|
1200
|
+
|
|
1201
|
+
```ruby
|
|
1202
|
+
TreeHaver.with_backend(:rust) do
|
|
1203
|
+
puts TreeHaver.effective_backend # => :rust
|
|
1204
|
+
|
|
1205
|
+
TreeHaver.with_backend(:citrus) do
|
|
1206
|
+
puts TreeHaver.effective_backend # => :citrus
|
|
840
1207
|
end
|
|
841
1208
|
|
|
842
|
-
|
|
843
|
-
|
|
1209
|
+
puts TreeHaver.effective_backend # => :rust (restored)
|
|
1210
|
+
end
|
|
1211
|
+
```
|
|
1212
|
+
|
|
1213
|
+
#### Fallback Pattern
|
|
1214
|
+
|
|
1215
|
+
Try one backend, fall back to another on failure:
|
|
1216
|
+
|
|
1217
|
+
```ruby
|
|
1218
|
+
def parse_with_fallback(source)
|
|
1219
|
+
TreeHaver.with_backend(:mri) do
|
|
1220
|
+
TreeHaver::Parser.new.tap { |p| p.language = load_language }.parse(source)
|
|
1221
|
+
end
|
|
1222
|
+
rescue TreeHaver::NotAvailable
|
|
1223
|
+
# Fall back to Citrus if MRI backend unavailable
|
|
1224
|
+
TreeHaver.with_backend(:citrus) do
|
|
1225
|
+
TreeHaver::Parser.new.tap { |p| p.language = load_language }.parse(source)
|
|
844
1226
|
end
|
|
845
1227
|
end
|
|
846
1228
|
```
|
|
@@ -912,7 +1294,7 @@ You can support the development of kettle-rb tools via
|
|
|
912
1294
|
and [Tidelift][🏙️entsup-tidelift].
|
|
913
1295
|
|
|
914
1296
|
| 📍 NOTE |
|
|
915
|
-
|
|
1297
|
+
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
916
1298
|
| If doing a sponsorship in the form of donation is problematic for your company <br/> from an accounting standpoint, we'd recommend the use of Tidelift, <br/> where you can get a support-like subscription instead. |
|
|
917
1299
|
|
|
918
1300
|
### Open Collective for Individuals
|
|
@@ -922,7 +1304,9 @@ Support us with a monthly donation and help us continue our activities. [[Become
|
|
|
922
1304
|
NOTE: [kettle-readme-backers][kettle-readme-backers] updates this list every day, automatically.
|
|
923
1305
|
|
|
924
1306
|
<!-- OPENCOLLECTIVE-INDIVIDUALS:START -->
|
|
1307
|
+
|
|
925
1308
|
No backers yet. Be the first!
|
|
1309
|
+
|
|
926
1310
|
<!-- OPENCOLLECTIVE-INDIVIDUALS:END -->
|
|
927
1311
|
|
|
928
1312
|
### Open Collective for Organizations
|
|
@@ -932,14 +1316,16 @@ Become a sponsor and get your logo on our README on GitHub with a link to your s
|
|
|
932
1316
|
NOTE: [kettle-readme-backers][kettle-readme-backers] updates this list every day, automatically.
|
|
933
1317
|
|
|
934
1318
|
<!-- OPENCOLLECTIVE-ORGANIZATIONS:START -->
|
|
1319
|
+
|
|
935
1320
|
No sponsors yet. Be the first!
|
|
1321
|
+
|
|
936
1322
|
<!-- OPENCOLLECTIVE-ORGANIZATIONS:END -->
|
|
937
1323
|
|
|
938
1324
|
[kettle-readme-backers]: https://github.com/kettle-rb/tree_haver/blob/main/exe/kettle-readme-backers
|
|
939
1325
|
|
|
940
1326
|
### Another way to support open-source
|
|
941
1327
|
|
|
942
|
-
I’m driven by a passion to foster a thriving open-source community – a space where people can tackle complex problems, no matter how small.
|
|
1328
|
+
I’m driven by a passion to foster a thriving open-source community – a space where people can tackle complex problems, no matter how small. Revitalizing libraries that have fallen into disrepair, and building new libraries focused on solving real-world challenges, are my passions. I was recently affected by layoffs, and the tech jobs market is unwelcoming. I’m reaching out here because your support would significantly aid my efforts to provide for my family, and my farm (11 🐔 chickens, 2 🐶 dogs, 3 🐰 rabbits, 8 🐈 cats).
|
|
943
1329
|
|
|
944
1330
|
If you work at a company that uses my work, please encourage them to support me as a corporate sponsor. My work on gems you use might show up in `bundle fund`.
|
|
945
1331
|
|
|
@@ -1010,7 +1396,7 @@ a new version should be immediately released that restores compatibility.
|
|
|
1010
1396
|
Breaking changes to the public API will only be introduced with new major versions.
|
|
1011
1397
|
|
|
1012
1398
|
> dropping support for a platform is both obviously and objectively a breaking change <br/>
|
|
1013
|
-
|
|
1399
|
+
> —Jordan Harband ([@ljharb](https://github.com/ljharb), maintainer of SemVer) [in SemVer issue 716][📌semver-breaking]
|
|
1014
1400
|
|
|
1015
1401
|
I understand that policy doesn't work universally ("exceptions to every rule!"),
|
|
1016
1402
|
but it is the policy here.
|
|
@@ -1027,7 +1413,7 @@ spec.add_dependency("tree_haver", "~> 1.0")
|
|
|
1027
1413
|
<summary>📌 Is "Platform Support" part of the public API? More details inside.</summary>
|
|
1028
1414
|
|
|
1029
1415
|
SemVer should, IMO, but doesn't explicitly, say that dropping support for specific Platforms
|
|
1030
|
-
is a
|
|
1416
|
+
is a _breaking change_ to an API, and for that reason the bike shedding is endless.
|
|
1031
1417
|
|
|
1032
1418
|
To get a better understanding of how SemVer is intended to work over a project's lifetime,
|
|
1033
1419
|
read this article from the creator of SemVer:
|
|
@@ -1114,7 +1500,6 @@ Thanks for RTFM. ☺️
|
|
|
1114
1500
|
[✉️discord-invite-img-ftb]: https://img.shields.io/discord/1373797679469170758?style=for-the-badge&logo=discord
|
|
1115
1501
|
[✉️ruby-friends-img]: https://img.shields.io/badge/daily.dev-%F0%9F%92%8E_Ruby_Friends-0A0A0A?style=for-the-badge&logo=dailydotdev&logoColor=white
|
|
1116
1502
|
[✉️ruby-friends]: https://app.daily.dev/squads/rubyfriends
|
|
1117
|
-
|
|
1118
1503
|
[✇bundle-group-pattern]: https://gist.github.com/pboling/4564780
|
|
1119
1504
|
[⛳️gem-namespace]: https://github.com/kettle-rb/tree_haver
|
|
1120
1505
|
[⛳️namespace-img]: https://img.shields.io/badge/namespace-TreeHaver-3C2D2D.svg?style=square&logo=ruby&logoColor=white
|
|
@@ -1238,7 +1623,7 @@ Thanks for RTFM. ☺️
|
|
|
1238
1623
|
[📌gitmoji]: https://gitmoji.dev
|
|
1239
1624
|
[📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
|
|
1240
1625
|
[🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
|
1241
|
-
[🧮kloc-img]: https://img.shields.io/badge/KLOC-
|
|
1626
|
+
[🧮kloc-img]: https://img.shields.io/badge/KLOC-1.067-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
|
|
1242
1627
|
[🔐security]: SECURITY.md
|
|
1243
1628
|
[🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
|
|
1244
1629
|
[📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
|