tree_haver 3.2.3 → 3.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +193 -3
- data/README.md +189 -69
- data/lib/tree_haver/backend_api.rb +349 -0
- data/lib/tree_haver/backends/citrus.rb +37 -8
- data/lib/tree_haver/backends/commonmarker.rb +31 -6
- data/lib/tree_haver/backends/java.rb +160 -22
- data/lib/tree_haver/backends/markly.rb +34 -4
- data/lib/tree_haver/backends/prism.rb +21 -8
- data/lib/tree_haver/backends/psych.rb +24 -0
- data/lib/tree_haver/language_registry.rb +57 -3
- data/lib/tree_haver/node.rb +15 -1
- data/lib/tree_haver/rspec/dependency_tags.rb +249 -23
- data/lib/tree_haver/version.rb +1 -1
- data/lib/tree_haver.rb +291 -22
- data.tar.gz.sig +0 -0
- metadata +5 -4
- metadata.gz.sig +0 -0
data/README.md
CHANGED
|
@@ -141,8 +141,8 @@ The Rust backend uses [tree\_stump](https://github.com/joker1007/tree_stump), wh
|
|
|
141
141
|
|
|
142
142
|
- **JRuby**: Cannot load native `.so` extensions (runs on JVM)
|
|
143
143
|
- **TruffleRuby**: magnus/rb-sys are incompatible with TruffleRuby's C API emulation
|
|
144
|
-
NOTE: `tree_stump` currently requires unreleased fixes in the `main` branch.
|
|
145
|
-
|
|
144
|
+
NOTE: `tree_stump` currently requires unreleased fixes in the `main` branch.
|
|
145
|
+
<!-- end list -->
|
|
146
146
|
``` ruby
|
|
147
147
|
# Add to your Gemfile for Rust backend (MRI only)
|
|
148
148
|
gem "tree_stump", github: "joker1007/tree_stump", branch: "main"
|
|
@@ -155,7 +155,6 @@ gem "tree_stump", github: "joker1007/tree_stump", branch: "main"
|
|
|
155
155
|
Requires the `ffi` gem and a system installation of `libtree-sitter`.
|
|
156
156
|
|
|
157
157
|
- **TruffleRuby**: TruffleRuby's FFI implementation doesn't support `STRUCT_BY_VALUE` return types, which tree-sitter's C API uses for functions like `ts_tree_root_node` and `ts_node_child`.
|
|
158
|
-
|
|
159
158
|
<!-- end list -->
|
|
160
159
|
``` ruby
|
|
161
160
|
# Add to your Gemfile for FFI backend (MRI and JRuby)
|
|
@@ -185,25 +184,115 @@ gem "citrus", "~> 3.0"
|
|
|
185
184
|
|
|
186
185
|
#### Java Backend (JRuby only)
|
|
187
186
|
|
|
188
|
-
|
|
187
|
+
**Requires jtreesitter \>= 0.26.0** from Maven Central. Older versions are not supported due to breaking API changes.
|
|
188
|
+
|
|
189
|
+
``` ruby
|
|
190
|
+
# No gem dependency - uses JRuby's built-in Java integration
|
|
191
|
+
# Download the JAR:
|
|
192
|
+
# curl -L -o jtreesitter-0.26.0.jar \
|
|
193
|
+
# "https://repo1.maven.org/maven2/io/github/tree-sitter/jtreesitter/0.26.0/jtreesitter-0.26.0.jar"
|
|
194
|
+
|
|
195
|
+
# Set environment variable:
|
|
196
|
+
# export TREE_SITTER_JAVA_JARS_DIR=/path/to/jars
|
|
197
|
+
```
|
|
189
198
|
|
|
199
|
+
**Also requires**:
|
|
200
|
+
- Tree-sitter runtime library (`libtree-sitter.so`) version 0.26+ (must match jtreesitter version)
|
|
201
|
+
- Grammar `.so` files built against tree-sitter 0.26+ (or rebuilt with `tree-sitter generate`)
|
|
190
202
|
### Backend Platform Compatibility
|
|
191
203
|
|
|
192
204
|
Not all backends work on all Ruby platforms. Here's a complete compatibility matrix:
|
|
193
205
|
|
|
194
|
-
| Backend | MRI | JRuby | TruffleRuby | Notes |
|
|
195
|
-
| --- | :-: | :-: | :-: | --- |
|
|
196
|
-
| **MRI** ([ruby\_tree\_sitter](https://github.com/Faveod/ruby-tree-sitter)) | ✅ | ❌ | ❌ | C extension, MRI only |
|
|
197
|
-
| **Rust** ([tree\_stump](https://github.com/joker1007/tree_stump)) | ✅ | ❌ | ❌ | magnus/rb-sys incompatible with non-MRI |
|
|
198
|
-
| **FFI** | ✅ | ✅ | ❌ | TruffleRuby FFI doesn't support `STRUCT_BY_VALUE` |
|
|
199
|
-
| **Java** ([jtreesitter](https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter)) | ❌ | ✅ | ❌ | JRuby only, requires
|
|
200
|
-
| **Prism** | ✅ | ✅ | ✅ | Ruby parsing, stdlib in Ruby 3.4+ |
|
|
201
|
-
| **Psych** | ✅ | ✅ | ✅ | YAML parsing, stdlib |
|
|
202
|
-
| **Citrus** | ✅ | ✅ | ✅ | Pure Ruby, no native dependencies |
|
|
203
|
-
| **Commonmarker** | ✅ | ❌ | ❓ | Rust extension for Markdown |
|
|
204
|
-
| **Markly** | ✅ | ❌ | ❓ | C extension for Markdown |
|
|
206
|
+
| Backend | MRI | JRuby | TruffleRuby | API Complete | Notes |
|
|
207
|
+
| --- | :-: | :-: | :-: | :-: | --- |
|
|
208
|
+
| **MRI** ([ruby\_tree\_sitter](https://github.com/Faveod/ruby-tree-sitter)) | ✅ | ❌ | ❌ | ✅ | C extension, MRI only |
|
|
209
|
+
| **Rust** ([tree\_stump](https://github.com/joker1007/tree_stump)) | ✅ | ❌ | ❌ | ✅ | magnus/rb-sys incompatible with non-MRI |
|
|
210
|
+
| **FFI** | ✅ | ✅ | ❌ | ⚠️ | TruffleRuby FFI doesn't support `STRUCT_BY_VALUE` |
|
|
211
|
+
| **Java** ([jtreesitter](https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter)) | ❌ | ✅ | ❌ | ✅ | JRuby only, requires jtreesitter \>= 0.26.0 |
|
|
212
|
+
| **Prism** | ✅ | ✅ | ✅ | ✅ | Ruby parsing, stdlib in Ruby 3.4+ |
|
|
213
|
+
| **Psych** | ✅ | ✅ | ✅ | ✅ | YAML parsing, stdlib |
|
|
214
|
+
| **Citrus** | ✅ | ✅ | ✅ | ⚠️ | Pure Ruby, no native dependencies |
|
|
215
|
+
| **Commonmarker** | ✅ | ❌ | ❓ | ✅ | Rust extension for Markdown |
|
|
216
|
+
| **Markly** | ✅ | ❌ | ❓ | ✅ | C extension for Markdown |
|
|
217
|
+
|
|
218
|
+
**Legend**: ✅ = Works / Complete, ❌ = Does not work, ❓ = Untested, ⚠️ = Partial (some optional methods missing)
|
|
219
|
+
|
|
220
|
+
**API Complete** indicates whether the backend implements all optional Node methods (`parent`, `next_sibling`, `prev_sibling`, `named?`, `missing?`, `text`, `child_by_field_name`). Backends marked ⚠️ work but may be missing some advanced traversal methods. Use `TreeHaver::BackendAPI.validate(backend_module)` to check specific backends.
|
|
221
|
+
|
|
222
|
+
### Version Requirements for Tree-Sitter Backends
|
|
223
|
+
|
|
224
|
+
#### tree-sitter Runtime Library
|
|
225
|
+
|
|
226
|
+
All tree-sitter backends (MRI, Rust, FFI, Java) require the tree-sitter runtime library. **Version 0.26+ is required** for the Java backend (to match jtreesitter 0.26.0). Other backends may work with 0.24+, but 0.26+ is recommended for consistency.
|
|
227
|
+
|
|
228
|
+
``` bash
|
|
229
|
+
# Check your tree-sitter version
|
|
230
|
+
tree-sitter --version # Should be 0.26.0 or newer for Java backend
|
|
231
|
+
|
|
232
|
+
# macOS
|
|
233
|
+
brew install tree-sitter
|
|
234
|
+
|
|
235
|
+
# Ubuntu/Debian
|
|
236
|
+
apt-get install libtree-sitter0 libtree-sitter-dev
|
|
237
|
+
|
|
238
|
+
# Fedora
|
|
239
|
+
dnf install tree-sitter tree-sitter-devel
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
#### jtreesitter (Java Backend)
|
|
243
|
+
|
|
244
|
+
**The Java backend requires jtreesitter \>= 0.26.0.** This version introduced breaking API changes:
|
|
245
|
+
|
|
246
|
+
- `Parser.parse()` returns `Optional<Tree>` instead of `Tree`
|
|
247
|
+
- `Tree.getRootNode()` returns `Node` directly (not `Optional<Node>`)
|
|
248
|
+
- `Node.getChild()`, `getParent()`, `getNextSibling()`, `getPrevSibling()` return `Optional<Node>`
|
|
249
|
+
- `Language.load(name)` was removed; use `SymbolLookup` API instead
|
|
250
|
+
Older versions of jtreesitter are **NOT supported**.
|
|
251
|
+
<!-- end list -->
|
|
252
|
+
``` bash
|
|
253
|
+
# Download jtreesitter 0.26.0 from Maven Central
|
|
254
|
+
curl -L -o jtreesitter-0.26.0.jar \
|
|
255
|
+
"https://repo1.maven.org/maven2/io/github/tree-sitter/jtreesitter/0.26.0/jtreesitter-0.26.0.jar"
|
|
256
|
+
|
|
257
|
+
# Or use the provided setup script
|
|
258
|
+
bin/setup-jtreesitter
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
Set the environment variable to point to your JAR directory:
|
|
262
|
+
|
|
263
|
+
``` bash
|
|
264
|
+
export TREE_SITTER_JAVA_JARS_DIR=/path/to/jars
|
|
265
|
+
```
|
|
205
266
|
|
|
206
|
-
|
|
267
|
+
#### Grammar ABI Compatibility
|
|
268
|
+
|
|
269
|
+
**CRITICAL**: Grammars must be built against a compatible tree-sitter version.
|
|
270
|
+
|
|
271
|
+
Tree-sitter 0.24+ changed how language ABI versions are reported (from `ts_language_version()` to `ts_language_abi_version()`). For the Java backend with jtreesitter 0.26.0, grammars must be built against tree-sitter 0.26+. If you get errors like:
|
|
272
|
+
|
|
273
|
+
Failed to load tree_sitter_toml
|
|
274
|
+
Version mismatch detected: The grammar was built against tree-sitter < 0.26
|
|
275
|
+
You need to rebuild the grammar from source:
|
|
276
|
+
|
|
277
|
+
``` bash
|
|
278
|
+
# Use the provided build script
|
|
279
|
+
bin/build-grammar toml
|
|
280
|
+
|
|
281
|
+
# Or manually:
|
|
282
|
+
git clone https://github.com/tree-sitter-grammars/tree-sitter-toml
|
|
283
|
+
cd tree-sitter-toml
|
|
284
|
+
tree-sitter generate # Regenerates parser.c for your tree-sitter version
|
|
285
|
+
cc -shared -fPIC -o libtree-sitter-toml.so src/parser.c src/scanner.c -I src
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
**Grammar sources for common languages:**
|
|
289
|
+
|
|
290
|
+
| Language | Repository |
|
|
291
|
+
| --- | --- |
|
|
292
|
+
| TOML | [tree-sitter-grammars/tree-sitter-toml](https://github.com/tree-sitter-grammars/tree-sitter-toml) |
|
|
293
|
+
| JSON | [tree-sitter/tree-sitter-json](https://github.com/tree-sitter/tree-sitter-json) |
|
|
294
|
+
| JSONC | [WhyNotHugo/tree-sitter-jsonc](https://gitlab.com/WhyNotHugo/tree-sitter-jsonc) |
|
|
295
|
+
| Bash | [tree-sitter/tree-sitter-bash](https://github.com/tree-sitter/tree-sitter-bash) |
|
|
207
296
|
|
|
208
297
|
#### TruffleRuby Limitations
|
|
209
298
|
|
|
@@ -211,20 +300,18 @@ TruffleRuby has **no working tree-sitter backend**:
|
|
|
211
300
|
|
|
212
301
|
- **FFI**: TruffleRuby's FFI doesn't support `STRUCT_BY_VALUE` return types (used by `ts_tree_root_node`, `ts_node_child`, etc.)
|
|
213
302
|
- **MRI/Rust**: C and Rust extensions require MRI's C API internals (`RBasic.flags`, `rb_gc_writebarrier`, etc.) that TruffleRuby doesn't expose
|
|
214
|
-
TruffleRuby users should use: **Prism** (Ruby), **Psych** (YAML), **Citrus** (TOML via toml-rb), or potentially **Commonmarker/Markly** (Markdown).
|
|
215
|
-
|
|
303
|
+
TruffleRuby users should use: **Prism** (Ruby), **Psych** (YAML), **Citrus** (TOML via toml-rb), or potentially **Commonmarker/Markly** (Markdown).
|
|
216
304
|
#### JRuby Limitations
|
|
217
305
|
|
|
218
|
-
JRuby runs on the JVM and **cannot load native `.so` extensions**:
|
|
306
|
+
JRuby runs on the JVM and **cannot load native `.so` extensions via Ruby's C API**:
|
|
219
307
|
|
|
220
308
|
- **MRI/Rust**: C and Rust extensions simply cannot be loaded
|
|
221
309
|
- **FFI**: Works\! JRuby has excellent FFI support
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
[
|
|
225
|
-
[tree_stump
|
|
226
|
-
[jtreesitter]: https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter
|
|
227
|
-
|
|
310
|
+
- **Java**: Works\! The Java backend uses jtreesitter (requires \>= 0.26.0)
|
|
311
|
+
JRuby users should use: **Java backend** (best performance, full API) or **FFI backend** for tree-sitter, plus **Prism**, **Psych**, **Citrus** for other formats.
|
|
312
|
+
[ruby\_tree\_sitter](https://github.com/Faveod/ruby-tree-sitter): https://github.com/Faveod/ruby-tree-sitter
|
|
313
|
+
[tree\_stump](https://github.com/joker1007/tree_stump): https://github.com/joker1007/tree\_stump
|
|
314
|
+
\[jtreesitter\]: https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter
|
|
228
315
|
### Why TreeHaver?
|
|
229
316
|
|
|
230
317
|
tree-sitter is a powerful parser generator that creates incremental parsers for many programming languages. However, integrating it into Ruby applications can be challenging:
|
|
@@ -232,28 +319,27 @@ tree-sitter is a powerful parser generator that creates incremental parsers for
|
|
|
232
319
|
- MRI-based C extensions don't work on JRuby
|
|
233
320
|
- FFI-based solutions may not be optimal for MRI
|
|
234
321
|
- Managing different backends for different Ruby implementations is cumbersome
|
|
235
|
-
TreeHaver solves these problems by providing a unified API that automatically selects the appropriate backend for your Ruby implementation, allowing you to write code once and run it anywhere.
|
|
236
|
-
|
|
322
|
+
TreeHaver solves these problems by providing a unified API that automatically selects the appropriate backend for your Ruby implementation, allowing you to write code once and run it anywhere.
|
|
237
323
|
|
|
238
324
|
### The `*-merge` Gem Family
|
|
239
325
|
|
|
240
326
|
The `*-merge` gem family provides intelligent, AST-based merging for various file formats. At the foundation is [tree_haver][tree_haver], which provides a unified cross-Ruby parsing API that works seamlessly across MRI, JRuby, and TruffleRuby.
|
|
241
327
|
|
|
242
|
-
| Gem | Format
|
|
243
|
-
|
|
244
|
-
| [tree_haver][tree_haver] | Multi
|
|
245
|
-
| [ast-merge][ast-merge] | Text
|
|
246
|
-
| [
|
|
247
|
-
| [
|
|
248
|
-
| [
|
|
249
|
-
| [
|
|
250
|
-
| [
|
|
251
|
-
| [
|
|
252
|
-
| [
|
|
253
|
-
| [
|
|
254
|
-
| [
|
|
255
|
-
| [
|
|
256
|
-
| [
|
|
328
|
+
| Gem | Language<br>/ Format | Parser Backend(s) | Description |
|
|
329
|
+
|------------------------------------------|----------------------|-----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
|
|
330
|
+
| [tree_haver][tree_haver] | Multi | MRI C, Rust, FFI, Java, Prism, Psych, Commonmarker, Markly, Citrus | **Foundation**: Cross-Ruby adapter for parsing libraries (like Faraday for HTTP) |
|
|
331
|
+
| [ast-merge][ast-merge] | Text | internal | **Infrastructure**: Shared base classes and merge logic for all `*-merge` gems |
|
|
332
|
+
| [bash-merge][bash-merge] | Bash | [tree-sitter-bash][ts-bash] (via tree_haver) | Smart merge for Bash scripts |
|
|
333
|
+
| [commonmarker-merge][commonmarker-merge] | Markdown | [Commonmarker][commonmarker] (via tree_haver) | Smart merge for Markdown (CommonMark via comrak Rust) |
|
|
334
|
+
| [dotenv-merge][dotenv-merge] | Dotenv | internal | Smart merge for `.env` files |
|
|
335
|
+
| [json-merge][json-merge] | JSON | [tree-sitter-json][ts-json] (via tree_haver) | Smart merge for JSON files |
|
|
336
|
+
| [jsonc-merge][jsonc-merge] | JSONC | [tree-sitter-jsonc][ts-jsonc] (via tree_haver) | ⚠️ Proof of concept; Smart merge for JSON with Comments |
|
|
337
|
+
| [markdown-merge][markdown-merge] | Markdown | [Commonmarker][commonmarker] / [Markly][markly] (via tree_haver) | **Foundation**: Shared base for Markdown mergers with inner code block merging |
|
|
338
|
+
| [markly-merge][markly-merge] | Markdown | [Markly][markly] (via tree_haver) | Smart merge for Markdown (CommonMark via cmark-gfm C) |
|
|
339
|
+
| [prism-merge][prism-merge] | Ruby | [Prism][prism] (`prism` std lib gem) | Smart merge for Ruby source files |
|
|
340
|
+
| [psych-merge][psych-merge] | YAML | [Psych][psych] (`psych` std lib gem) | Smart merge for YAML files |
|
|
341
|
+
| [rbs-merge][rbs-merge] | RBS | [tree-sitter-bash][ts-rbs] (via tree_haver), [RBS][rbs] (`rbs` std lib gem) | Smart merge for Ruby type signatures |
|
|
342
|
+
| [toml-merge][toml-merge] | TOML | [Citrus + toml-rb][toml-rb] (default, via tree_haver), [tree-sitter-toml][ts-toml] (via tree_haver) | Smart merge for TOML files |
|
|
257
343
|
|
|
258
344
|
**Example implementations** for the gem templating use case:
|
|
259
345
|
|
|
@@ -314,9 +400,9 @@ The `*-merge` gem family provides intelligent, AST-based merging for various fil
|
|
|
314
400
|
[citrus]: https://github.com/mjackson/citrus
|
|
315
401
|
[tree_haver]: https://github.com/kettle-rb/tree_haver
|
|
316
402
|
|
|
317
|
-
**Note:** Java backend works with grammar
|
|
403
|
+
**Note:** Java backend works with grammar `.so` files built against tree-sitter 0.24+. The grammars must be rebuilt with `tree-sitter generate` if they were compiled against older tree-sitter versions. FFI is recommended for JRuby as it's easier to set up.
|
|
318
404
|
|
|
319
|
-
**Note:** TreeHaver can use `ruby_tree_sitter` (MRI) or `tree_stump` (MRI
|
|
405
|
+
**Note:** TreeHaver can use `ruby_tree_sitter` (MRI) or `tree_stump` (MRI) as backends, or `java-tree-sitter` / `jtreesitter` \>= 0.26.0 ([docs](https://tree-sitter.github.io/java-tree-sitter/), [maven](https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter), [source](https://github.com/tree-sitter/java-tree-sitter), JRuby), or FFI on any backend, giving you TreeHaver's unified API, grammar discovery, and security features, plus full access to incremental parsing when using those backends.
|
|
320
406
|
|
|
321
407
|
**Note:** `tree_stump` currently requires unreleased fixes in the `main` branch.
|
|
322
408
|
|
|
@@ -325,28 +411,42 @@ The `*-merge` gem family provides intelligent, AST-based merging for various fil
|
|
|
325
411
|
**Choose TreeHaver when:**
|
|
326
412
|
|
|
327
413
|
- You need JRuby or TruffleRuby support
|
|
414
|
+
|
|
328
415
|
- You're building a library that should work across Ruby implementations
|
|
416
|
+
|
|
329
417
|
- You want automatic grammar discovery and security validations
|
|
418
|
+
|
|
330
419
|
- You want flexibility to switch backends without code changes
|
|
420
|
+
|
|
331
421
|
- You need incremental parsing with a unified API
|
|
332
|
-
**Choose ruby\_tree\_sitter directly when:**
|
|
422
|
+
**Choose ruby\_tree\_sitter directly when:**
|
|
333
423
|
|
|
334
424
|
- You only target MRI Ruby
|
|
425
|
+
|
|
335
426
|
- You need the full Query API without abstraction
|
|
427
|
+
|
|
336
428
|
- You want the most battle-tested C bindings
|
|
429
|
+
|
|
337
430
|
- You don't need TreeHaver's grammar discovery
|
|
338
|
-
**Choose tree\_stump directly when:**
|
|
431
|
+
**Choose tree\_stump directly when:**
|
|
339
432
|
|
|
340
433
|
- You only target MRI Ruby
|
|
434
|
+
|
|
341
435
|
- You prefer Rust-based native extensions
|
|
436
|
+
|
|
342
437
|
- You want precompiled binaries without system dependencies
|
|
438
|
+
|
|
343
439
|
- You don't need TreeHaver's grammar discovery
|
|
440
|
+
|
|
344
441
|
- **Note:** `tree_stump` currently requires unreleased fixes in the `main` branch.
|
|
345
|
-
**Choose citrus directly when:**
|
|
442
|
+
**Choose citrus directly when:**
|
|
346
443
|
|
|
347
444
|
- You need zero native dependencies (pure Ruby)
|
|
445
|
+
|
|
348
446
|
- You're using a Citrus grammar (not tree-sitter grammars)
|
|
447
|
+
|
|
349
448
|
- Performance is less critical than portability
|
|
449
|
+
|
|
350
450
|
- You don't need TreeHaver's unified API
|
|
351
451
|
## 💡 Info you can shake a stick at
|
|
352
452
|
|
|
@@ -399,12 +499,16 @@ The maintainers of this and thousands of other packages are working with Tidelif
|
|
|
399
499
|
[](https://tidelift.com/subscription/pkg/rubygems-tree_haver?utm_source=rubygems-tree_haver&utm_medium=referral&utm_campaign=readme)
|
|
400
500
|
|
|
401
501
|
- 💡Subscribe for support guarantees covering *all* your FLOSS dependencies
|
|
502
|
+
|
|
402
503
|
- 💡Tidelift is part of [Sonar](https://blog.tidelift.com/tidelift-joins-sonar)
|
|
504
|
+
|
|
403
505
|
- 💡Tidelift pays maintainers to maintain the software you depend on\!<br/>📊`@`Pointy Haired Boss: An [enterprise support](https://tidelift.com/subscription/pkg/rubygems-tree_haver?utm_source=rubygems-tree_haver&utm_medium=referral&utm_campaign=readme) subscription is "[never gonna let you down](https://www.youtube.com/watch?v=dQw4w9WgXcQ)", and *supports* open source maintainers
|
|
404
|
-
Alternatively:
|
|
506
|
+
Alternatively:
|
|
405
507
|
|
|
406
508
|
- [](https://discord.gg/3qme4XHNKN)
|
|
509
|
+
|
|
407
510
|
- [](https://www.upwork.com/freelancers/~014942e9b056abdf86?mp_source=share)
|
|
511
|
+
|
|
408
512
|
- [](https://www.codementor.io/peterboling?utm_source=github&utm_medium=button&utm_term=peterboling&utm_campaign=github)
|
|
409
513
|
</details>
|
|
410
514
|
|
|
@@ -471,7 +575,7 @@ TreeHaver supports 10 parsing backends, each with different trade-offs. The `aut
|
|
|
471
575
|
| **MRI** | C extension via ruby\_tree\_sitter | ⚡ Fastest | MRI only | [JSON](examples/mri_json.rb) · [JSONC](examples/mri_jsonc.rb) · \~\~Bash\~\~\* · [TOML](examples/mri_toml.rb) |
|
|
472
576
|
| **Rust** | Precompiled via tree\_stump | ⚡ Very Fast | ✅ Good | [JSON](examples/rust_json.rb) · [JSONC](examples/rust_jsonc.rb) · \~\~Bash\~\~\* · [TOML](examples/rust_toml.rb) |
|
|
473
577
|
| **FFI** | Dynamic linking via FFI | 🔵 Fast | ✅ Universal | [JSON](examples/ffi_json.rb) · [JSONC](examples/ffi_jsonc.rb) · [Bash](examples/ffi_bash.rb) · [TOML](examples/ffi_toml.rb) |
|
|
474
|
-
| **Java** | JNI bindings | ⚡ Very Fast | JRuby only | [JSON](examples/java_json.rb) · [JSONC](examples/java_jsonc.rb) · [Bash](examples/java_bash.rb) · [TOML](examples/java_toml.rb) |
|
|
578
|
+
| **Java** | JNI bindings (jtreesitter \>= 0.26.0) | ⚡ Very Fast | JRuby only | [JSON](examples/java_json.rb) · [JSONC](examples/java_jsonc.rb) · [Bash](examples/java_bash.rb) · [TOML](examples/java_toml.rb) |
|
|
475
579
|
|
|
476
580
|
#### Language-Specific Backends (Native Parser Integration)
|
|
477
581
|
|
|
@@ -488,8 +592,8 @@ TreeHaver supports 10 parsing backends, each with different trade-offs. The `aut
|
|
|
488
592
|
**Known Issues:**
|
|
489
593
|
- \*MRI + Bash: ABI incompatibility (use FFI instead)
|
|
490
594
|
- \*Rust + Bash: Version mismatch (use FFI instead)
|
|
491
|
-
**Backend Requirements:**
|
|
492
|
-
|
|
595
|
+
**Backend Requirements:**
|
|
596
|
+
<!-- end list -->
|
|
493
597
|
``` ruby
|
|
494
598
|
# Tree-sitter backends
|
|
495
599
|
gem "ruby_tree_sitter", "~> 2.0" # MRI backend
|
|
@@ -563,7 +667,6 @@ This is particularly useful for:
|
|
|
563
667
|
- **Performance comparison**: Benchmark different backends
|
|
564
668
|
- **Fallback scenarios**: Try one backend, fall back to another
|
|
565
669
|
- **Thread isolation**: Each thread can use a different backend safely
|
|
566
|
-
|
|
567
670
|
<!-- end list -->
|
|
568
671
|
``` ruby
|
|
569
672
|
# Example: Testing with multiple backends
|
|
@@ -633,10 +736,9 @@ The `find_library_path_safe` method only returns paths in trusted directories.
|
|
|
633
736
|
- `/usr/lib/x86_64-linux-gnu`, `/usr/lib/aarch64-linux-gnu`
|
|
634
737
|
- `/usr/local/lib`
|
|
635
738
|
- `/opt/homebrew/lib`, `/opt/local/lib`
|
|
636
|
-
**Adding custom trusted directories:**
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
739
|
+
**Adding custom trusted directories:**
|
|
740
|
+
For non-standard installations (Homebrew on Linux, luarocks, mise, asdf, etc.), register additional trusted directories:
|
|
741
|
+
<!-- end list -->
|
|
640
742
|
``` ruby
|
|
641
743
|
# Programmatically at application startup
|
|
642
744
|
TreeHaver::PathValidator.add_trusted_directory("/home/linuxbrew/.linuxbrew/Cellar")
|
|
@@ -730,7 +832,6 @@ When loading a language grammar, if you don't specify the `symbol:` parameter, T
|
|
|
730
832
|
1. **`TREE_SITTER_LANG_SYMBOL`**: Explicit symbol override
|
|
731
833
|
2. Guessed from filename (e.g., `libtree-sitter-toml.so` → `tree_sitter_toml`)
|
|
732
834
|
3. Default fallback (`tree_sitter_toml`)
|
|
733
|
-
|
|
734
835
|
<!-- end list -->
|
|
735
836
|
``` bash
|
|
736
837
|
export TREE_SITTER_LANG_SYMBOL=tree_sitter_toml
|
|
@@ -745,12 +846,34 @@ export TREE_SITTER_TOML_PATH=/usr/local/lib/libtree-sitter-toml.so
|
|
|
745
846
|
export TREE_SITTER_JSON_PATH=/usr/local/lib/libtree-sitter-json.so
|
|
746
847
|
```
|
|
747
848
|
|
|
748
|
-
#### JRuby-Specific: Java Backend
|
|
849
|
+
#### JRuby-Specific: Java Backend Configuration
|
|
749
850
|
|
|
750
|
-
For the Java backend on JRuby:
|
|
851
|
+
For the Java backend on JRuby, you need:
|
|
751
852
|
|
|
853
|
+
1. **jtreesitter \>= 0.26.0** JAR from Maven Central
|
|
854
|
+
2. **Tree-sitter runtime library** (`libtree-sitter.so`) version 0.26+
|
|
855
|
+
3. **Grammar `.so` files** built against tree-sitter 0.26+
|
|
856
|
+
<!-- end list -->
|
|
752
857
|
``` bash
|
|
858
|
+
# Download jtreesitter JAR (or use bin/setup-jtreesitter)
|
|
753
859
|
export TREE_SITTER_JAVA_JARS_DIR=/path/to/java-tree-sitter/jars
|
|
860
|
+
|
|
861
|
+
# Point to tree-sitter runtime (must be 0.26+)
|
|
862
|
+
export TREE_SITTER_RUNTIME_LIB=/usr/local/lib/libtree-sitter.so
|
|
863
|
+
|
|
864
|
+
# Point to grammar libraries (must be built for tree-sitter 0.26+)
|
|
865
|
+
export TREE_SITTER_TOML_PATH=/path/to/libtree-sitter-toml.so
|
|
866
|
+
```
|
|
867
|
+
|
|
868
|
+
**Building grammars for Java backend:**
|
|
869
|
+
|
|
870
|
+
If you get "version mismatch" errors, rebuild the grammar:
|
|
871
|
+
|
|
872
|
+
``` bash
|
|
873
|
+
# Use the provided build script
|
|
874
|
+
bin/build-grammar toml
|
|
875
|
+
|
|
876
|
+
# This regenerates parser.c for your tree-sitter version and compiles it
|
|
754
877
|
```
|
|
755
878
|
|
|
756
879
|
For more see [docs](https://tree-sitter.github.io/java-tree-sitter/), [maven](https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter), and [source](https://github.com/tree-sitter/java-tree-sitter).
|
|
@@ -929,7 +1052,6 @@ end
|
|
|
929
1052
|
└── TreeHaver::Error # Base error class
|
|
930
1053
|
├── TreeHaver::NotAvailable # Backend/grammar not available
|
|
931
1054
|
└── TreeHaver::BackendConflict # Backend incompatibility detected
|
|
932
|
-
|
|
933
1055
|
**Compatibility Mode Behavior:**
|
|
934
1056
|
|
|
935
1057
|
The compat mode (`require "tree_haver/compat"`) creates aliases but **does not change the exception hierarchy**:
|
|
@@ -966,14 +1088,13 @@ end
|
|
|
966
1088
|
```
|
|
967
1089
|
|
|
968
1090
|
2. **Never rely on `rescue => e`** to catch TreeHaver errors (it won't work)
|
|
969
|
-
**Why inherit from Exception?**
|
|
970
|
-
|
|
971
|
-
|
|
1091
|
+
**Why inherit from Exception?**
|
|
1092
|
+
Following ruby\_tree\_sitter's reasoning:
|
|
1093
|
+
<!-- end list -->
|
|
972
1094
|
- **Thread safety**: Prevents accidental catching in thread cleanup code
|
|
973
1095
|
- **Signal handling**: Ensures parsing errors don't interfere with SIGTERM/SIGINT
|
|
974
1096
|
- **Intentional handling**: Forces developers to explicitly handle parsing errors
|
|
975
|
-
See `lib/tree_haver/compat.rb` for compatibility layer documentation.
|
|
976
|
-
|
|
1097
|
+
See `lib/tree_haver/compat.rb` for compatibility layer documentation.
|
|
977
1098
|
## 🔧 Basic Usage
|
|
978
1099
|
|
|
979
1100
|
### Quick Start
|
|
@@ -1191,7 +1312,6 @@ This flexibility is useful for:
|
|
|
1191
1312
|
- **Versioning**: Register different grammar versions (e.g., `:ruby_2`, `:ruby_3`)
|
|
1192
1313
|
- **Testing**: Use unique names to avoid collisions between tests
|
|
1193
1314
|
- **Context-specific naming**: Use names that make sense for your application
|
|
1194
|
-
|
|
1195
1315
|
<!-- end list -->
|
|
1196
1316
|
``` ruby
|
|
1197
1317
|
# Register the same TOML grammar under different names for different purposes
|
|
@@ -1388,8 +1508,8 @@ The Java backend will work with:
|
|
|
1388
1508
|
|
|
1389
1509
|
- Grammar JARs built specifically for java-tree-sitter / jtreesitter (self-contained, [docs](https://tree-sitter.github.io/java-tree-sitter/), [maven](https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter), [source](https://github.com/tree-sitter/java-tree-sitter))
|
|
1390
1510
|
- Grammar `.so` files that statically link tree-sitter
|
|
1391
|
-
**Option 3: Citrus Backend (pure Ruby, portable)**
|
|
1392
|
-
|
|
1511
|
+
**Option 3: Citrus Backend (pure Ruby, portable)**
|
|
1512
|
+
<!-- end list -->
|
|
1393
1513
|
``` ruby
|
|
1394
1514
|
# Gemfile
|
|
1395
1515
|
gem "tree_haver"
|
|
@@ -1977,7 +2097,7 @@ Thanks for RTFM. ☺️
|
|
|
1977
2097
|
[📌gitmoji]: https://gitmoji.dev
|
|
1978
2098
|
[📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
|
|
1979
2099
|
[🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
|
|
1980
|
-
[🧮kloc-img]: https://img.shields.io/badge/KLOC-2.
|
|
2100
|
+
[🧮kloc-img]: https://img.shields.io/badge/KLOC-2.422-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
|
|
1981
2101
|
[🔐security]: SECURITY.md
|
|
1982
2102
|
[🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
|
|
1983
2103
|
[📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
|