tree_haver 5.0.1 → 5.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/CHANGELOG.md +26 -1
- data/README.md +132 -137
- data/lib/tree_haver/backend_api.rb +12 -12
- data/lib/tree_haver/grammar_finder.rb +7 -6
- data/lib/tree_haver/rspec/dependency_tags.rb +36 -3
- data/lib/tree_haver/version.rb +1 -1
- data.tar.gz.sig +0 -0
- metadata +4 -4
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4a84c922fc46d5f2832bfba861dcf1927327de29546752189f6d6488f4286795
|
|
4
|
+
data.tar.gz: 526550b1327829aff144d052d3dcf217295f7d2e27f976735b78573350e3b047
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: dce567d8e91b11b01618feab1f3757d4a3651c5282168eea8be88c4704228f2288ab6bf098b8a7bcc7940f3f16af6cf1e544e991e99ecc6de1ee6c1d15ef2d77
|
|
7
|
+
data.tar.gz: c36d6955f70893ec43f247baead4ab70c449508fc744959e0dd485f73316d490c5892c91151527e2e6a714c99b847daeda9b618daca62337b40be5b727a2362e
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
data/CHANGELOG.md
CHANGED
|
@@ -30,6 +30,29 @@ Please file a bug if you notice a violation of semantic versioning.
|
|
|
30
30
|
|
|
31
31
|
### Security
|
|
32
32
|
|
|
33
|
+
## [5.0.2] - 2026-01-13
|
|
34
|
+
|
|
35
|
+
- TAG: [v5.0.2][5.0.2t]
|
|
36
|
+
- COVERAGE: 90.79% -- 2308/2542 lines in 30 files
|
|
37
|
+
- BRANCH COVERAGE: 78.09% -- 930/1191 branches in 30 files
|
|
38
|
+
- 94.78% documented
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
|
|
42
|
+
- More documentation about the Merge Gem Family
|
|
43
|
+
- **`:json_parsing` and `:jsonc_parsing` RSpec dependency tags**: Added missing parsing capability tags
|
|
44
|
+
for JSON and JSONC (JSON with Comments) languages
|
|
45
|
+
- `any_json_backend_available?` - Checks if tree-sitter-json is available
|
|
46
|
+
- `any_jsonc_backend_available?` - Checks if tree-sitter-jsonc is available
|
|
47
|
+
- Tests tagged with `:jsonc_parsing` will now be properly skipped on TruffleRuby and other
|
|
48
|
+
platforms where tree-sitter backends are not available
|
|
49
|
+
- Fixes issue where jsonc-merge specs were running on TruffleRuby and failing because
|
|
50
|
+
the tag was undefined and therefore not excluded
|
|
51
|
+
|
|
52
|
+
### Changed
|
|
53
|
+
|
|
54
|
+
- Restored README.md (was accidentally corrupted during the last release)
|
|
55
|
+
|
|
33
56
|
## [5.0.1] - 2026-01-11
|
|
34
57
|
|
|
35
58
|
- TAG: [v5.0.1][5.0.1t]
|
|
@@ -1259,7 +1282,9 @@ Despite the major version bump to 3.0.0 (following semver due to the breaking `L
|
|
|
1259
1282
|
|
|
1260
1283
|
- Initial release
|
|
1261
1284
|
|
|
1262
|
-
[Unreleased]: https://github.com/kettle-rb/tree_haver/compare/v5.0.
|
|
1285
|
+
[Unreleased]: https://github.com/kettle-rb/tree_haver/compare/v5.0.2...HEAD
|
|
1286
|
+
[5.0.2]: https://github.com/kettle-rb/tree_haver/compare/v5.0.1...v5.0.2
|
|
1287
|
+
[5.0.2t]: https://github.com/kettle-rb/tree_haver/releases/tag/v5.0.2
|
|
1263
1288
|
[5.0.1]: https://github.com/kettle-rb/tree_haver/compare/v5.0.0...v5.0.1
|
|
1264
1289
|
[5.0.1t]: https://github.com/kettle-rb/tree_haver/releases/tag/v5.0.1
|
|
1265
1290
|
[5.0.0]: https://github.com/kettle-rb/tree_haver/compare/v4.0.5...v5.0.0
|
data/README.md
CHANGED
|
@@ -88,28 +88,28 @@ tree = parser.parse(source_code)
|
|
|
88
88
|
|
|
89
89
|
### Key Features
|
|
90
90
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
91
|
+
- **Universal Ruby Support**: Works on MRI Ruby, JRuby, and TruffleRuby
|
|
92
|
+
- **10 Parsing Backends** - Choose the right backend for your needs:
|
|
93
|
+
- **Tree-sitter Backends** (high-performance, incremental parsing):
|
|
94
|
+
- **MRI Backend**: Leverages [`ruby_tree_sitter`][ruby_tree_sitter] gem (C extension, fastest on MRI)
|
|
95
|
+
- **Rust Backend**: Uses [`tree_stump`][tree_stump] gem (Rust with precompiled binaries)
|
|
96
|
+
- **Note**: `tree_stump` currently requires unreleased fixes in the `main` branch.
|
|
97
|
+
- **FFI Backend**: Pure Ruby FFI bindings to `libtree-sitter` (JRuby only; TruffleRuby's FFI doesn't support tree-sitter's struct-by-value returns)
|
|
98
|
+
- **Java Backend**: Native Java integration for JRuby with [`java-tree-sitter`](https://github.com/tree-sitter/java-tree-sitter) / [`jtreesitter`][jtreesitter] grammar JARs
|
|
99
|
+
- **Language-Specific Backends** (native parser integration):
|
|
100
|
+
- **Prism Backend**: Ruby's official parser ([Prism][prism], stdlib in Ruby 3.4+)
|
|
101
|
+
- **Psych Backend**: Ruby's YAML parser ([Psych][psych], stdlib)
|
|
102
|
+
- **Commonmarker Backend**: Fast Markdown parser ([Commonmarker][commonmarker], comrak Rust)
|
|
103
|
+
- **Markly Backend**: GitHub Flavored Markdown ([Markly][markly], cmark-gfm C)
|
|
104
|
+
- **Pure Ruby Fallback**:
|
|
105
|
+
- **Citrus Backend**: Pure Ruby PEG parsing via [`citrus`][citrus] (no native dependencies)
|
|
106
|
+
- **Parslet Backend**: Pure Ruby PEG parsing via [`parslet`][parslet] (no native dependencies)
|
|
107
|
+
- **Automatic Backend Selection**: Intelligently selects the best backend for your Ruby implementation
|
|
108
|
+
- **Language Agnostic**: Parse any language - Ruby, Markdown, YAML, JSON, Bash, TOML, JavaScript, etc.
|
|
109
|
+
- **Grammar Discovery**: Built-in `GrammarFinder` utility for platform-aware grammar library discovery
|
|
110
|
+
- **Unified Position API**: Consistent `start_line`, `end_line`, `source_position` across all backends
|
|
111
|
+
- **Thread-Safe**: Built-in language registry with thread-safe caching
|
|
112
|
+
- **Minimal API Surface**: Simple, focused API that covers the most common use cases
|
|
113
113
|
|
|
114
114
|
### Backend Requirements
|
|
115
115
|
|
|
@@ -142,9 +142,9 @@ gem "ruby_tree_sitter", "~> 2.0"
|
|
|
142
142
|
|
|
143
143
|
The Rust backend uses [tree\_stump][tree_stump], which is a Rust native extension built with [magnus](https://github.com/matsadler/magnus) and [rb-sys](https://github.com/oxidize-rb/rb-sys). These libraries are only compatible with MRI Ruby's C API.
|
|
144
144
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
145
|
+
- **JRuby**: Cannot load native `.so` extensions (runs on JVM)
|
|
146
|
+
- **TruffleRuby**: magnus/rb-sys are incompatible with TruffleRuby's C API emulation
|
|
147
|
+
NOTE: `tree_stump` currently requires unreleased fixes in the `main` branch.
|
|
148
148
|
|
|
149
149
|
```ruby
|
|
150
150
|
# Add to your Gemfile for Rust backend (MRI only)
|
|
@@ -157,7 +157,7 @@ gem "tree_stump", github: "joker1007/tree_stump", branch: "main"
|
|
|
157
157
|
|
|
158
158
|
Requires the `ffi` gem and a system installation of `libtree-sitter`.
|
|
159
159
|
|
|
160
|
-
|
|
160
|
+
- **TruffleRuby**: TruffleRuby's FFI implementation doesn't support `STRUCT_BY_VALUE` return types, which tree-sitter's C API uses for functions like `ts_tree_root_node` and `ts_node_child`.
|
|
161
161
|
|
|
162
162
|
```ruby
|
|
163
163
|
# Add to your Gemfile for FFI backend (MRI and JRuby)
|
|
@@ -210,8 +210,8 @@ gem "parslet", "~> 2.0"
|
|
|
210
210
|
|
|
211
211
|
**Also requires**:
|
|
212
212
|
|
|
213
|
-
|
|
214
|
-
|
|
213
|
+
- Tree-sitter runtime library (`libtree-sitter.so`) version 0.26+ (must match jtreesitter version)
|
|
214
|
+
- Grammar `.so` files built against tree-sitter 0.26+ (or rebuilt with `tree-sitter generate`)
|
|
215
215
|
|
|
216
216
|
### Version Requirements for Tree-Sitter Backends
|
|
217
217
|
|
|
@@ -237,11 +237,11 @@ dnf install tree-sitter tree-sitter-devel
|
|
|
237
237
|
|
|
238
238
|
**The Java backend requires jtreesitter \>= 0.26.0.** This version introduced breaking API changes:
|
|
239
239
|
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
240
|
+
- `Parser.parse()` returns `Optional<Tree>` instead of `Tree`
|
|
241
|
+
- `Tree.getRootNode()` returns `Node` directly (not `Optional<Node>`)
|
|
242
|
+
- `Node.getChild()`, `getParent()`, `getNextSibling()`, `getPrevSibling()` return `Optional<Node>`
|
|
243
|
+
- `Language.load(name)` was removed; use `SymbolLookup` API instead
|
|
244
|
+
Older versions of jtreesitter are **NOT supported**.
|
|
245
245
|
|
|
246
246
|
```bash
|
|
247
247
|
# Download jtreesitter 0.26.0 from Maven Central
|
|
@@ -293,30 +293,27 @@ cc -shared -fPIC -o libtree-sitter-toml.so src/parser.c src/scanner.c -I src
|
|
|
293
293
|
|
|
294
294
|
TruffleRuby has **no working tree-sitter backend**:
|
|
295
295
|
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
296
|
+
- **FFI**: TruffleRuby's FFI doesn't support `STRUCT_BY_VALUE` return types (used by `ts_tree_root_node`, `ts_node_child`, etc.)
|
|
297
|
+
- **MRI/Rust**: C and Rust extensions require MRI's C API internals (`RBasic.flags`, `rb_gc_writebarrier`, etc.) that TruffleRuby doesn't expose
|
|
298
|
+
TruffleRuby users should use: **Prism** (Ruby), **Psych** (YAML), **Citrus/Parslet** (e.g., TOML via toml-rb/toml), or potentially **Commonmarker/Markly** (Markdown).
|
|
299
299
|
|
|
300
300
|
#### JRuby Limitations
|
|
301
301
|
|
|
302
302
|
JRuby runs on the JVM and **cannot load native `.so` extensions via Ruby's C API**:
|
|
303
303
|
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
[ruby\_tree\_sitter][ruby_tree_sitter]: https://github.com/Faveod/ruby-tree-sitter
|
|
309
|
-
[tree\_stump][tree_stump]: https://github.com/joker1007/tree\_stump
|
|
310
|
-
\[jtreesitter\]: https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter
|
|
304
|
+
- **MRI/Rust**: C and Rust extensions simply cannot be loaded
|
|
305
|
+
- **FFI**: Works\! JRuby has excellent FFI support
|
|
306
|
+
- **Java**: Works\! The Java backend uses jtreesitter (requires \>= 0.26.0)
|
|
307
|
+
JRuby users should use: **Java backend** (best performance, full API) or **FFI backend** for tree-sitter, plus **Prism**, **Psych**, **Citrus/Parslet** for other formats.
|
|
311
308
|
|
|
312
309
|
### Why TreeHaver?
|
|
313
310
|
|
|
314
311
|
tree-sitter is a powerful parser generator that creates incremental parsers for many programming languages. However, integrating it into Ruby applications can be challenging:
|
|
315
312
|
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
313
|
+
- MRI-based C extensions don't work on JRuby
|
|
314
|
+
- FFI-based solutions may not be optimal for MRI
|
|
315
|
+
- Managing different backends for different Ruby implementations is cumbersome
|
|
316
|
+
TreeHaver solves these problems by providing a unified API that automatically selects the appropriate backend for your Ruby implementation, allowing you to write code once and run it anywhere.
|
|
320
317
|
|
|
321
318
|
### The `*-merge` Gem Family
|
|
322
319
|
|
|
@@ -324,7 +321,7 @@ The `*-merge` gem family provides intelligent, AST-based merging for various fil
|
|
|
324
321
|
|
|
325
322
|
| Gem | Version | CI | | Language<br>/ Format | Parser Backend(s) | Description |
|
|
326
323
|
|------------------------------------------|----------------------------------------------------------------|--------------------------------------------------------------|----------|-------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|-------------|
|
|
327
|
-
| [tree_haver][tree_haver] | [![Version][tree_haver-gem-i]][tree_haver-gem] | [![Version][tree_haver-ci-i]][tree_haver-ci] | Multi | MRI C, Rust, FFI, Java, Prism, Psych, Commonmarker, Markly, Citrus, Parslet
|
|
324
|
+
| [tree_haver][tree_haver] | [![Version][tree_haver-gem-i]][tree_haver-gem] | [![Version][tree_haver-ci-i]][tree_haver-ci] | Multi | Supported Backends: MRI C, Rust, FFI, Java, Prism, Psych, Commonmarker, Markly, Citrus, Parslet | **Foundation**: Cross-Ruby adapter for parsing libraries (like Faraday for HTTP) |
|
|
328
325
|
| [ast-merge][ast-merge] | [![Version][ast-merge-gem-i]][ast-merge-gem] | [![Version][ast-merge-ci-i]][ast-merge-ci] | Text | internal | **Infrastructure**: Shared base classes and merge logic for all `*-merge` gems |
|
|
329
326
|
| [bash-merge][bash-merge] | [![Version][bash-merge-gem-i]][bash-merge-gem] | [![Version][bash-merge-ci-i]][bash-merge-ci] | Bash | [tree-sitter-bash][ts-bash] (via tree_haver) | Smart merge for Bash scripts |
|
|
330
327
|
| [commonmarker-merge][commonmarker-merge] | [![Version][commonmarker-merge-gem-i]][commonmarker-merge-gem] | [![Version][commonmarker-merge-ci-i]][commonmarker-merge-ci] | Markdown | [Commonmarker][commonmarker] (via tree_haver) | Smart merge for Markdown (CommonMark via comrak Rust) |
|
|
@@ -342,18 +339,18 @@ The `*-merge` gem family provides intelligent, AST-based merging for various fil
|
|
|
342
339
|
|
|
343
340
|
tree_haver supports multiple parsing backends, but not all backends work on all Ruby platforms:
|
|
344
341
|
|
|
345
|
-
| Platform 👉️<br> TreeHaver Backend 👇️
|
|
346
|
-
|
|
347
|
-
| **MRI** ([ruby_tree_sitter][ruby_tree_sitter])
|
|
348
|
-
| **Rust** ([tree_stump][tree_stump])
|
|
349
|
-
| **FFI**
|
|
350
|
-
| **Java** ([jtreesitter][jtreesitter])
|
|
351
|
-
| **Prism**
|
|
352
|
-
| **Psych**
|
|
353
|
-
| **Citrus**
|
|
354
|
-
| **Parslet**
|
|
355
|
-
| **Commonmarker**
|
|
356
|
-
| **Markly**
|
|
342
|
+
| Platform 👉️<br> TreeHaver Backend 👇️ | MRI | JRuby | TruffleRuby | Notes |
|
|
343
|
+
|-------------------------------------------------|:---:|:-----:|:-----------:|----------------------------------------------------------------------------|
|
|
344
|
+
| **MRI** ([ruby_tree_sitter][ruby_tree_sitter]) | ✅ | ❌ | ❌ | C extension, MRI only |
|
|
345
|
+
| **Rust** ([tree_stump][tree_stump]) | ✅ | ❌ | ❌ | Rust extension via magnus/rb-sys, MRI only |
|
|
346
|
+
| **FFI** ([ffi][ffi]) | ✅ | ✅ | ❌ | TruffleRuby's FFI doesn't support `STRUCT_BY_VALUE` |
|
|
347
|
+
| **Java** ([jtreesitter][jtreesitter]) | ❌ | ✅ | ❌ | JRuby only, requires grammar JARs |
|
|
348
|
+
| **Prism** ([prism][prism]) | ✅ | ✅ | ✅ | Ruby parsing, stdlib in Ruby 3.4+ |
|
|
349
|
+
| **Psych** ([psych][psych]) | ✅ | ✅ | ✅ | YAML parsing, stdlib |
|
|
350
|
+
| **Citrus** ([citrus][citrus]) | ✅ | ✅ | ✅ | Pure Ruby PEG parser, no native dependencies |
|
|
351
|
+
| **Parslet** ([parslet][parslet]) | ✅ | ✅ | ✅ | Pure Ruby PEG parser, no native dependencies |
|
|
352
|
+
| **Commonmarker** ([commonmarker][commonmarker]) | ✅ | ❌ | ❓ | Rust extension for Markdown (via [commonmarker-merge][commonmarker-merge]) |
|
|
353
|
+
| **Markly** ([markly][markly]) | ✅ | ❌ | ❓ | C extension for Markdown (via [markly-merge][markly-merge]) |
|
|
357
354
|
|
|
358
355
|
**Legend**: ✅ = Works, ❌ = Does not work, ❓ = Untested
|
|
359
356
|
|
|
@@ -447,6 +444,7 @@ tree_haver supports multiple parsing backends, but not all backends work on all
|
|
|
447
444
|
[kettle-jem-ci]: https://github.com/kettle-rb/kettle-jem/actions/workflows/current.yml
|
|
448
445
|
[prism]: https://github.com/ruby/prism
|
|
449
446
|
[psych]: https://github.com/ruby/psych
|
|
447
|
+
[ffi]: https://github.com/ffi/ffi
|
|
450
448
|
[ts-json]: https://github.com/tree-sitter/tree-sitter-json
|
|
451
449
|
[ts-jsonc]: https://gitlab.com/WhyNotHugo/tree-sitter-jsonc
|
|
452
450
|
[ts-bash]: https://github.com/tree-sitter/tree-sitter-bash
|
|
@@ -461,30 +459,27 @@ tree_haver supports multiple parsing backends, but not all backends work on all
|
|
|
461
459
|
[ruby_tree_sitter]: https://github.com/Faveod/ruby-tree-sitter
|
|
462
460
|
[tree_stump]: https://github.com/joker1007/tree_stump
|
|
463
461
|
[jtreesitter]: https://central.sonatype.com/artifact/io.github.tree-sitter/jtreesitter
|
|
462
|
+
[citrus]: https://github.com/mjackson/citrus
|
|
463
|
+
[parslet]: https://github.com/kschiess/parslet
|
|
464
464
|
|
|
465
465
|
### Comparison with Other Ruby AST / Parser Bindings
|
|
466
466
|
|
|
467
|
-
| Feature | [tree\_haver][📜src-gh] (this gem)
|
|
468
|
-
|
|
469
|
-
| **MRI Ruby** | ✅ Yes
|
|
470
|
-
| **JRuby** | ✅ Yes (FFI, Java, Citrus, or Parslet backend)
|
|
471
|
-
| **TruffleRuby** | ✅ Yes (FFI, Citrus, or Parslet)
|
|
472
|
-
| **Backend** | Multi (MRI C, Rust, FFI, Java, Citrus, Parslet) | C extension only
|
|
473
|
-
| **Incremental Parsing** | ✅ Via MRI C/Rust/Java backend
|
|
474
|
-
| **Query API** | ⚡ Via MRI/Rust/Java backend
|
|
475
|
-
| **Grammar Discovery** | ✅ Built-in `GrammarFinder`
|
|
476
|
-
| **Security Validations** | ✅ `PathValidator`
|
|
477
|
-
| **Language Registration** | ✅ Thread-safe registry
|
|
478
|
-
| **Native Performance** | ⚡ Backend-dependent
|
|
479
|
-
| **Precompiled Binaries** | ⚡ Via Rust backend
|
|
480
|
-
| **Zero Native Deps** | ⚡ Via Citrus/Parslet backend
|
|
481
|
-
| **Minimum Ruby** | 3.2+
|
|
467
|
+
| Feature | [tree\_haver][📜src-gh] (this gem) | [ruby\_tree\_sitter][ruby_tree_sitter] | [tree\_stump][tree_stump] | [citrus][citrus] | [parslet][parslet] |
|
|
468
|
+
|---------------------------|-------------------------------------------------|----------------------------------------|---------------------------|------------------|--------------------|
|
|
469
|
+
| **MRI Ruby** | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
|
|
470
|
+
| **JRuby** | ✅ Yes (FFI, Java, Citrus, or Parslet backend) | ❌ No | ❌ No | ✅ Yes | ✅ Yes |
|
|
471
|
+
| **TruffleRuby** | ✅ Yes (FFI, Citrus, or Parslet) | ❌ No | ❓ Unknown | ✅ Yes | ✅ Yes |
|
|
472
|
+
| **Backend** | Multi (MRI C, Rust, FFI, Java, Citrus, Parslet) | C extension only | Rust extension | Pure Ruby | Pure Ruby |
|
|
473
|
+
| **Incremental Parsing** | ✅ Via MRI C/Rust/Java backend | ✅ Yes | ✅ Yes | ❌ No | ❌ No |
|
|
474
|
+
| **Query API** | ⚡ Via MRI/Rust/Java backend | ✅ Yes | ✅ Yes | ❌ No | ❌ No |
|
|
475
|
+
| **Grammar Discovery** | ✅ Built-in `GrammarFinder` | ❌ Manual | ❌ Manual | ❌ Manual | ❌ Manual |
|
|
476
|
+
| **Security Validations** | ✅ `PathValidator` | ❌ No | ❌ No | ❌ No | ❌ No |
|
|
477
|
+
| **Language Registration** | ✅ Thread-safe registry | ❌ No | ❌ No | ❌ No | ❌ No |
|
|
478
|
+
| **Native Performance** | ⚡ Backend-dependent | ✅ Native C | ✅ Native Rust | ❌ Pure Ruby | ❌ Pure Ruby |
|
|
479
|
+
| **Precompiled Binaries** | ⚡ Via Rust backend | ✅ Yes | ✅ Yes | ✅ Pure Ruby | ✅ Pure Ruby |
|
|
480
|
+
| **Zero Native Deps** | ⚡ Via Citrus/Parslet backend | ❌ No | ❌ No | ✅ Yes | ✅ Yes |
|
|
481
|
+
| **Minimum Ruby** | 3.2+ | 3.0+ | 3.1+ | 0+ | 0+ |
|
|
482
482
|
|
|
483
|
-
[ruby_tree_sitter]: https://github.com/Faveod/ruby-tree-sitter
|
|
484
|
-
[tree_stump]: https://github.com/joker1007/tree_stump
|
|
485
|
-
[citrus]: https://github.com/mjackson/citrus
|
|
486
|
-
[parslet]: https://github.com/kschiess/parslet
|
|
487
|
-
[tree_haver]: https://github.com/kettle-rb/tree_haver
|
|
488
483
|
**Note:** Java backend works with grammar `.so` files built against tree-sitter 0.24+. The grammars must be rebuilt with `tree-sitter generate` if they were compiled against older tree-sitter versions. FFI is recommended for JRuby as it's easier to set up.
|
|
489
484
|
|
|
490
485
|
**Note:** TreeHaver can use `ruby_tree_sitter` (MRI) or `tree_stump` (MRI) as backends, or `java-tree-sitter` / `jtreesitter` \>= 0.26.0 ([docs](https://tree-sitter.github.io/java-tree-sitter/), [maven][jtreesitter], [source](https://github.com/tree-sitter/java-tree-sitter), JRuby), or FFI on any backend, giving you TreeHaver's unified API, grammar discovery, and security features, plus full access to incremental parsing when using those backends.
|
|
@@ -573,18 +568,18 @@ The maintainers of this and thousands of other packages are working with Tidelif
|
|
|
573
568
|
|
|
574
569
|
[![Get help from me on Tidelift][🏙️entsup-tidelift-img]][🏙️entsup-tidelift]
|
|
575
570
|
|
|
576
|
-
|
|
571
|
+
- 💡Subscribe for support guarantees covering *all* your FLOSS dependencies
|
|
577
572
|
|
|
578
|
-
|
|
573
|
+
- 💡Tidelift is part of [Sonar][🏙️entsup-tidelift-sonar]
|
|
579
574
|
|
|
580
|
-
|
|
581
|
-
|
|
575
|
+
- 💡Tidelift pays maintainers to maintain the software you depend on\!<br/>📊`@`Pointy Haired Boss: An [enterprise support][🏙️entsup-tidelift] subscription is "[never gonna let you down][🧮kloc]", and *supports* open source maintainers
|
|
576
|
+
Alternatively:
|
|
582
577
|
|
|
583
|
-
|
|
578
|
+
- [![Live Chat on Discord][✉️discord-invite-img-ftb]][🖼️galtzo-discord]
|
|
584
579
|
|
|
585
|
-
|
|
580
|
+
- [![Get help from me on Upwork][👨🏼🏫expsup-upwork-img]][👨🏼🏫expsup-upwork]
|
|
586
581
|
|
|
587
|
-
|
|
582
|
+
- [![Get help from me on Codementor][👨🏼🏫expsup-codementor-img]][👨🏼🏫expsup-codementor]
|
|
588
583
|
|
|
589
584
|
</details>
|
|
590
585
|
|
|
@@ -668,9 +663,9 @@ TreeHaver supports 10 parsing backends, each with different trade-offs. The `aut
|
|
|
668
663
|
|
|
669
664
|
**Known Issues:**
|
|
670
665
|
|
|
671
|
-
|
|
672
|
-
|
|
673
|
-
|
|
666
|
+
- \*MRI + Bash: ABI incompatibility (use FFI instead)
|
|
667
|
+
- \*Rust + Bash: Version mismatch (use FFI instead)
|
|
668
|
+
**Backend Requirements:**
|
|
674
669
|
|
|
675
670
|
```ruby
|
|
676
671
|
# Tree-sitter backends
|
|
@@ -746,10 +741,10 @@ end
|
|
|
746
741
|
|
|
747
742
|
This is particularly useful for:
|
|
748
743
|
|
|
749
|
-
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
|
|
744
|
+
- **Testing**: Test the same code with different backends
|
|
745
|
+
- **Performance comparison**: Benchmark different backends
|
|
746
|
+
- **Fallback scenarios**: Try one backend, fall back to another
|
|
747
|
+
- **Thread isolation**: Each thread can use a different backend safely
|
|
753
748
|
|
|
754
749
|
```ruby
|
|
755
750
|
# Example: Testing with multiple backends
|
|
@@ -782,13 +777,13 @@ TreeHaver provides defense-in-depth validations, but you should understand the r
|
|
|
782
777
|
|
|
783
778
|
TreeHaver's `PathValidator` module protects against:
|
|
784
779
|
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
790
|
-
|
|
791
|
-
|
|
780
|
+
- **Path traversal**: Paths containing `/../` or `/./` are rejected
|
|
781
|
+
- **Null byte injection**: Paths containing null bytes are rejected
|
|
782
|
+
- **Non-absolute paths**: Relative paths are rejected to prevent CWD-based attacks
|
|
783
|
+
- **Invalid extensions**: Only `.so`, `.dylib`, and `.dll` files are accepted
|
|
784
|
+
- **Malicious filenames**: Filenames must match a safe pattern (alphanumeric, hyphens, underscores)
|
|
785
|
+
- **Invalid language names**: Language names must be lowercase alphanumeric with underscores
|
|
786
|
+
- **Invalid symbol names**: Symbol names must be valid C identifiers
|
|
792
787
|
|
|
793
788
|
#### Secure Usage
|
|
794
789
|
|
|
@@ -816,12 +811,12 @@ The `find_library_path_safe` method only returns paths in trusted directories.
|
|
|
816
811
|
|
|
817
812
|
**Default trusted directories:**
|
|
818
813
|
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
823
|
-
|
|
824
|
-
|
|
814
|
+
- `/usr/lib`, `/usr/lib64`
|
|
815
|
+
- `/usr/lib/x86_64-linux-gnu`, `/usr/lib/aarch64-linux-gnu`
|
|
816
|
+
- `/usr/local/lib`
|
|
817
|
+
- `/opt/homebrew/lib`, `/opt/local/lib`
|
|
818
|
+
**Adding custom trusted directories:**
|
|
819
|
+
For non-standard installations (Homebrew on Linux, luarocks, mise, asdf, etc.), register additional trusted directories:
|
|
825
820
|
|
|
826
821
|
```ruby
|
|
827
822
|
# Programmatically at application startup
|
|
@@ -951,29 +946,29 @@ TreeHaver recognizes several environment variables for configuration:
|
|
|
951
946
|
|
|
952
947
|
#### Security Configuration
|
|
953
948
|
|
|
954
|
-
|
|
949
|
+
- **`TREE_HAVER_TRUSTED_DIRS`**: Comma-separated list of additional trusted directories for grammar libraries
|
|
955
950
|
|
|
956
|
-
|
|
957
|
-
|
|
958
|
-
|
|
959
|
-
|
|
951
|
+
```bash
|
|
952
|
+
# For Homebrew on Linux and luarocks
|
|
953
|
+
export TREE_HAVER_TRUSTED_DIRS="/home/linuxbrew/.linuxbrew/Cellar,~/.local/share/mise/installs/lua"
|
|
954
|
+
```
|
|
960
955
|
|
|
961
|
-
|
|
956
|
+
Tilde (`~`) is expanded to the user's home directory. Directories listed here are considered safe for `find_library_path_safe`.
|
|
962
957
|
|
|
963
958
|
#### Core Runtime Library
|
|
964
959
|
|
|
965
|
-
|
|
966
|
-
|
|
967
|
-
|
|
968
|
-
|
|
960
|
+
- **`TREE_SITTER_RUNTIME_LIB`**: Absolute path to the core `libtree-sitter` shared library
|
|
961
|
+
```bash
|
|
962
|
+
export TREE_SITTER_RUNTIME_LIB=/usr/local/lib/libtree-sitter.so
|
|
963
|
+
```
|
|
969
964
|
|
|
970
965
|
If not set, TreeHaver tries these names in order:
|
|
971
966
|
|
|
972
|
-
|
|
973
|
-
|
|
974
|
-
|
|
975
|
-
|
|
976
|
-
|
|
967
|
+
- `tree-sitter`
|
|
968
|
+
- `libtree-sitter.so.0`
|
|
969
|
+
- `libtree-sitter.so`
|
|
970
|
+
- `libtree-sitter.dylib`
|
|
971
|
+
- `libtree-sitter.dll`
|
|
977
972
|
|
|
978
973
|
#### Language Symbol Resolution
|
|
979
974
|
|
|
@@ -1245,10 +1240,10 @@ end
|
|
|
1245
1240
|
**Why inherit from Exception?**
|
|
1246
1241
|
Following ruby\_tree\_sitter's reasoning:
|
|
1247
1242
|
|
|
1248
|
-
|
|
1249
|
-
|
|
1250
|
-
|
|
1251
|
-
|
|
1243
|
+
- **Thread safety**: Prevents accidental catching in thread cleanup code
|
|
1244
|
+
- **Signal handling**: Ensures parsing errors don't interfere with SIGTERM/SIGINT
|
|
1245
|
+
- **Intentional handling**: Forces developers to explicitly handle parsing errors
|
|
1246
|
+
See `lib/tree_haver/compat.rb` for compatibility layer documentation.
|
|
1252
1247
|
|
|
1253
1248
|
## 🔧 Basic Usage
|
|
1254
1249
|
|
|
@@ -1686,11 +1681,11 @@ end
|
|
|
1686
1681
|
|
|
1687
1682
|
**⚠️ Citrus Backend Limitations:**
|
|
1688
1683
|
|
|
1689
|
-
|
|
1690
|
-
|
|
1691
|
-
|
|
1692
|
-
|
|
1693
|
-
|
|
1684
|
+
- Uses Citrus grammars (not tree-sitter grammars)
|
|
1685
|
+
- No incremental parsing support
|
|
1686
|
+
- No query API
|
|
1687
|
+
- Pure Ruby performance (slower than native backends)
|
|
1688
|
+
- Best for: prototyping, environments without native extension support, teaching
|
|
1694
1689
|
|
|
1695
1690
|
##### Option 4: Parslet Backend (pure Ruby, portable)
|
|
1696
1691
|
|
|
@@ -1712,11 +1707,11 @@ end
|
|
|
1712
1707
|
|
|
1713
1708
|
**⚠️ Parslet Backend Limitations:**
|
|
1714
1709
|
|
|
1715
|
-
|
|
1716
|
-
|
|
1717
|
-
|
|
1718
|
-
|
|
1719
|
-
|
|
1710
|
+
- Uses Parslet grammars (not tree-sitter grammars)
|
|
1711
|
+
- No incremental parsing support
|
|
1712
|
+
- No query API
|
|
1713
|
+
- Pure Ruby performance (slower than native backends)
|
|
1714
|
+
- Best for: prototyping, environments without native extension support, teaching
|
|
1720
1715
|
|
|
1721
1716
|
#### TruffleRuby
|
|
1722
1717
|
|
|
@@ -2118,7 +2113,7 @@ is a *breaking change* to an API, and for that reason the bike shedding is endle
|
|
|
2118
2113
|
To get a better understanding of how SemVer is intended to work over a project's lifetime,
|
|
2119
2114
|
read this article from the creator of SemVer:
|
|
2120
2115
|
|
|
2121
|
-
|
|
2116
|
+
- ["Major Version Numbers are Not Sacred"][📌major-versions-not-sacred]
|
|
2122
2117
|
|
|
2123
2118
|
</details>
|
|
2124
2119
|
|
|
@@ -234,12 +234,12 @@ module TreeHaver
|
|
|
234
234
|
private
|
|
235
235
|
|
|
236
236
|
def validate_module_methods(mod, results)
|
|
237
|
-
unless mod.
|
|
237
|
+
unless mod.singleton_class.method_defined?(:available?)
|
|
238
238
|
results[:errors] << "Missing module method: available?"
|
|
239
239
|
results[:valid] = false
|
|
240
240
|
end
|
|
241
241
|
|
|
242
|
-
unless mod.
|
|
242
|
+
unless mod.singleton_class.method_defined?(:capabilities)
|
|
243
243
|
results[:warnings] << "Missing module method: capabilities"
|
|
244
244
|
end
|
|
245
245
|
end
|
|
@@ -248,26 +248,26 @@ module TreeHaver
|
|
|
248
248
|
# from_library is REQUIRED for all backends
|
|
249
249
|
# Language-specific backends should implement it to ignore path/symbol
|
|
250
250
|
# and return their single language (for API consistency)
|
|
251
|
-
unless klass.
|
|
251
|
+
unless klass.singleton_class.method_defined?(:from_library)
|
|
252
252
|
results[:errors] << "Language missing required class method: from_library"
|
|
253
253
|
results[:valid] = false
|
|
254
254
|
end
|
|
255
255
|
|
|
256
256
|
# Check for optional convenience methods
|
|
257
|
-
optional_methods = LANGUAGE_OPTIONAL_CLASS_METHODS.select { |m| klass.
|
|
257
|
+
optional_methods = LANGUAGE_OPTIONAL_CLASS_METHODS.select { |m| klass.singleton_class.method_defined?(m) }
|
|
258
258
|
if optional_methods.any?
|
|
259
259
|
results[:capabilities][:language_shortcuts] = optional_methods
|
|
260
260
|
end
|
|
261
261
|
|
|
262
262
|
results[:capabilities][:language] = {
|
|
263
|
-
class_methods: LANGUAGE_CLASS_METHODS.select { |m| klass.
|
|
263
|
+
class_methods: LANGUAGE_CLASS_METHODS.select { |m| klass.singleton_class.method_defined?(m) } +
|
|
264
264
|
optional_methods,
|
|
265
265
|
}
|
|
266
266
|
end
|
|
267
267
|
|
|
268
268
|
def validate_parser(klass, results)
|
|
269
269
|
PARSER_CLASS_METHODS.each do |method|
|
|
270
|
-
unless klass.
|
|
270
|
+
unless klass.singleton_class.method_defined?(method)
|
|
271
271
|
results[:errors] << "Parser missing class method: #{method}"
|
|
272
272
|
results[:valid] = false
|
|
273
273
|
end
|
|
@@ -275,14 +275,14 @@ module TreeHaver
|
|
|
275
275
|
|
|
276
276
|
# Check instance methods by inspecting the class
|
|
277
277
|
PARSER_INSTANCE_METHODS.each do |method|
|
|
278
|
-
unless klass.
|
|
278
|
+
unless klass.method_defined?(method) || klass.private_method_defined?(method)
|
|
279
279
|
results[:errors] << "Parser missing instance method: #{method}"
|
|
280
280
|
results[:valid] = false
|
|
281
281
|
end
|
|
282
282
|
end
|
|
283
283
|
|
|
284
284
|
PARSER_OPTIONAL_METHODS.each do |method|
|
|
285
|
-
unless klass.
|
|
285
|
+
unless klass.method_defined?(method)
|
|
286
286
|
results[:warnings] << "Parser missing optional method: #{method}"
|
|
287
287
|
end
|
|
288
288
|
end
|
|
@@ -290,14 +290,14 @@ module TreeHaver
|
|
|
290
290
|
|
|
291
291
|
def validate_tree(klass, results)
|
|
292
292
|
TREE_INSTANCE_METHODS.each do |method|
|
|
293
|
-
unless klass.
|
|
293
|
+
unless klass.method_defined?(method)
|
|
294
294
|
results[:errors] << "Tree missing instance method: #{method}"
|
|
295
295
|
results[:valid] = false
|
|
296
296
|
end
|
|
297
297
|
end
|
|
298
298
|
|
|
299
299
|
TREE_OPTIONAL_METHODS.each do |method|
|
|
300
|
-
unless klass.
|
|
300
|
+
unless klass.method_defined?(method)
|
|
301
301
|
results[:warnings] << "Tree missing optional method: #{method}"
|
|
302
302
|
end
|
|
303
303
|
end
|
|
@@ -330,11 +330,11 @@ module TreeHaver
|
|
|
330
330
|
end
|
|
331
331
|
|
|
332
332
|
def has_method_or_alias?(klass, method)
|
|
333
|
-
return true if klass.
|
|
333
|
+
return true if klass.method_defined?(method)
|
|
334
334
|
|
|
335
335
|
# Check aliases
|
|
336
336
|
aliases = NODE_ALIASES[method] || []
|
|
337
|
-
aliases.any? { |alt| klass.
|
|
337
|
+
aliases.any? { |alt| klass.method_defined?(alt) }
|
|
338
338
|
end
|
|
339
339
|
|
|
340
340
|
def responds_to_with_aliases?(obj, method)
|
|
@@ -264,12 +264,13 @@ module TreeHaver
|
|
|
264
264
|
|
|
265
265
|
# Only tree-sitter backends are relevant here
|
|
266
266
|
# Non-tree-sitter backends (Citrus, Prism, Psych, etc.) don't use grammar files
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
267
|
+
if mod.nil? || !TREE_SITTER_BACKENDS.include?(mod)
|
|
268
|
+
false
|
|
269
|
+
else
|
|
270
|
+
# Try to instantiate a parser - this will fail if runtime isn't available
|
|
271
|
+
mod::Parser.new
|
|
272
|
+
true
|
|
273
|
+
end
|
|
273
274
|
rescue NoMethodError, LoadError, NotAvailable => _e
|
|
274
275
|
# Note: FFI::NotFoundError inherits from LoadError, so it's caught here too
|
|
275
276
|
false
|
|
@@ -167,6 +167,12 @@ require "set"
|
|
|
167
167
|
# [:markdown_parsing]
|
|
168
168
|
# At least one markdown parser (commonmarker OR markly) is available.
|
|
169
169
|
#
|
|
170
|
+
# [:json_parsing]
|
|
171
|
+
# At least one JSON parser (tree-sitter-json) is available.
|
|
172
|
+
#
|
|
173
|
+
# [:jsonc_parsing]
|
|
174
|
+
# At least one JSONC (JSON with Comments) parser (tree-sitter-jsonc) is available.
|
|
175
|
+
#
|
|
170
176
|
# [:rbs_parsing]
|
|
171
177
|
# At least one RBS parser (rbs gem OR tree-sitter-rbs) is available.
|
|
172
178
|
#
|
|
@@ -491,9 +497,12 @@ module TreeHaver
|
|
|
491
497
|
def libtree_sitter_available?
|
|
492
498
|
return @libtree_sitter_available if defined?(@libtree_sitter_available)
|
|
493
499
|
@libtree_sitter_available = begin
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
500
|
+
if !ffi_available?
|
|
501
|
+
false
|
|
502
|
+
else
|
|
503
|
+
TreeHaver::Backends::FFI::Native.try_load!
|
|
504
|
+
true
|
|
505
|
+
end
|
|
497
506
|
rescue TreeHaver::NotAvailable, LoadError
|
|
498
507
|
false
|
|
499
508
|
rescue StandardError
|
|
@@ -748,6 +757,26 @@ module TreeHaver
|
|
|
748
757
|
TreeHaver::BackendRegistry.tag_available?(:commonmarker_backend)
|
|
749
758
|
end
|
|
750
759
|
|
|
760
|
+
# Check if at least one JSON parsing backend is available
|
|
761
|
+
#
|
|
762
|
+
# Currently only tree-sitter-json is supported for JSON parsing.
|
|
763
|
+
# Future backends (e.g., pure-Ruby JSON parsers) can be added here.
|
|
764
|
+
#
|
|
765
|
+
# @return [Boolean] true if any JSON parsing backend works
|
|
766
|
+
def any_json_backend_available?
|
|
767
|
+
tree_sitter_json_available?
|
|
768
|
+
end
|
|
769
|
+
|
|
770
|
+
# Check if at least one JSONC parsing backend is available
|
|
771
|
+
#
|
|
772
|
+
# Currently only tree-sitter-jsonc is supported for JSONC parsing.
|
|
773
|
+
# Future backends (e.g., pure-Ruby JSONC parsers) can be added here.
|
|
774
|
+
#
|
|
775
|
+
# @return [Boolean] true if any JSONC parsing backend works
|
|
776
|
+
def any_jsonc_backend_available?
|
|
777
|
+
tree_sitter_jsonc_available?
|
|
778
|
+
end
|
|
779
|
+
|
|
751
780
|
def any_native_grammar_available?
|
|
752
781
|
libtree_sitter_available? && (
|
|
753
782
|
tree_sitter_bash_available? ||
|
|
@@ -872,6 +901,8 @@ module TreeHaver
|
|
|
872
901
|
# Language parsing capabilities (*_parsing)
|
|
873
902
|
result[:toml_parsing] = any_toml_backend_available?
|
|
874
903
|
result[:markdown_parsing] = any_markdown_backend_available?
|
|
904
|
+
result[:json_parsing] = any_json_backend_available?
|
|
905
|
+
result[:jsonc_parsing] = any_jsonc_backend_available?
|
|
875
906
|
result[:rbs_parsing] = any_rbs_backend_available?
|
|
876
907
|
|
|
877
908
|
# Specific libraries (*_gem)
|
|
@@ -1311,6 +1342,8 @@ RSpec.configure do |config|
|
|
|
1311
1342
|
# Language parsing capabilities
|
|
1312
1343
|
config.filter_run_excluding(not_toml_parsing: true) if deps.any_toml_backend_available?
|
|
1313
1344
|
config.filter_run_excluding(not_markdown_parsing: true) if deps.any_markdown_backend_available?
|
|
1345
|
+
config.filter_run_excluding(not_json_parsing: true) if deps.any_json_backend_available?
|
|
1346
|
+
config.filter_run_excluding(not_jsonc_parsing: true) if deps.any_jsonc_backend_available?
|
|
1314
1347
|
config.filter_run_excluding(not_rbs_parsing: true) if deps.any_rbs_backend_available?
|
|
1315
1348
|
end
|
|
1316
1349
|
|
data/lib/tree_haver/version.rb
CHANGED
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: tree_haver
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 5.0.
|
|
4
|
+
version: 5.0.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Peter H. Boling
|
|
@@ -300,10 +300,10 @@ licenses:
|
|
|
300
300
|
- MIT
|
|
301
301
|
metadata:
|
|
302
302
|
homepage_uri: https://tree-haver.galtzo.com/
|
|
303
|
-
source_code_uri: https://github.com/kettle-rb/tree_haver/tree/v5.0.
|
|
304
|
-
changelog_uri: https://github.com/kettle-rb/tree_haver/blob/v5.0.
|
|
303
|
+
source_code_uri: https://github.com/kettle-rb/tree_haver/tree/v5.0.2
|
|
304
|
+
changelog_uri: https://github.com/kettle-rb/tree_haver/blob/v5.0.2/CHANGELOG.md
|
|
305
305
|
bug_tracker_uri: https://github.com/kettle-rb/tree_haver/issues
|
|
306
|
-
documentation_uri: https://www.rubydoc.info/gems/tree_haver/5.0.
|
|
306
|
+
documentation_uri: https://www.rubydoc.info/gems/tree_haver/5.0.2
|
|
307
307
|
funding_uri: https://github.com/sponsors/pboling
|
|
308
308
|
wiki_uri: https://github.com/kettle-rb/tree_haver/wiki
|
|
309
309
|
news_uri: https://www.railsbling.com/tags/tree_haver
|
metadata.gz.sig
CHANGED
|
Binary file
|