ast-merge 1.0.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. checksums.yaml +4 -4
  2. checksums.yaml.gz.sig +0 -0
  3. data/CHANGELOG.md +194 -1
  4. data/README.md +235 -53
  5. data/exe/ast-merge-recipe +366 -0
  6. data/lib/ast/merge/ast_node.rb +224 -24
  7. data/lib/ast/merge/comment/block.rb +6 -0
  8. data/lib/ast/merge/comment/empty.rb +6 -0
  9. data/lib/ast/merge/comment/line.rb +6 -0
  10. data/lib/ast/merge/comment/parser.rb +9 -7
  11. data/lib/ast/merge/conflict_resolver_base.rb +8 -1
  12. data/lib/ast/merge/content_match_refiner.rb +278 -0
  13. data/lib/ast/merge/debug_logger.rb +6 -1
  14. data/lib/ast/merge/detector/base.rb +193 -0
  15. data/lib/ast/merge/detector/fenced_code_block.rb +227 -0
  16. data/lib/ast/merge/detector/mergeable.rb +369 -0
  17. data/lib/ast/merge/detector/toml_frontmatter.rb +82 -0
  18. data/lib/ast/merge/detector/yaml_frontmatter.rb +82 -0
  19. data/lib/ast/merge/file_analyzable.rb +5 -3
  20. data/lib/ast/merge/freeze_node_base.rb +1 -1
  21. data/lib/ast/merge/match_refiner_base.rb +1 -1
  22. data/lib/ast/merge/match_score_base.rb +1 -1
  23. data/lib/ast/merge/merge_result_base.rb +4 -1
  24. data/lib/ast/merge/merger_config.rb +33 -31
  25. data/lib/ast/merge/navigable_statement.rb +630 -0
  26. data/lib/ast/merge/partial_template_merger.rb +432 -0
  27. data/lib/ast/merge/recipe/config.rb +198 -0
  28. data/lib/ast/merge/recipe/preset.rb +171 -0
  29. data/lib/ast/merge/recipe/runner.rb +254 -0
  30. data/lib/ast/merge/recipe/script_loader.rb +181 -0
  31. data/lib/ast/merge/recipe.rb +26 -0
  32. data/lib/ast/merge/rspec/dependency_tags.rb +252 -0
  33. data/lib/ast/merge/rspec/shared_examples/reproducible_merge.rb +3 -2
  34. data/lib/ast/merge/rspec.rb +33 -2
  35. data/lib/ast/merge/section_typing.rb +52 -50
  36. data/lib/ast/merge/smart_merger_base.rb +86 -3
  37. data/lib/ast/merge/text/line_node.rb +42 -9
  38. data/lib/ast/merge/text/section_splitter.rb +12 -10
  39. data/lib/ast/merge/text/word_node.rb +47 -14
  40. data/lib/ast/merge/version.rb +1 -1
  41. data/lib/ast/merge.rb +10 -6
  42. data/sig/ast/merge.rbs +389 -2
  43. data.tar.gz.sig +0 -0
  44. metadata +76 -12
  45. metadata.gz.sig +0 -0
  46. data/lib/ast/merge/fenced_code_block_detector.rb +0 -211
  47. data/lib/ast/merge/region.rb +0 -124
  48. data/lib/ast/merge/region_detector_base.rb +0 -114
  49. data/lib/ast/merge/region_mergeable.rb +0 -364
  50. data/lib/ast/merge/toml_frontmatter_detector.rb +0 -88
  51. data/lib/ast/merge/yaml_frontmatter_detector.rb +0 -108
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a871b962617929e2c8ed159fd0b971f0f45eb2ed023927fb7d07be2213c26b2f
4
- data.tar.gz: 7dbc474b7d376d4388c00bae81e6bbeb8108faf37e19c6a2c6a87e045f09ad0f
3
+ metadata.gz: 43313fb7a92506c90be16e26fc640ab0bb88d35bc7459ae7435e8196c670e3d1
4
+ data.tar.gz: b2cf98d84d623cb2de6e19729159d4492684bc6e677fa7d0d44df0c3697ecda7
5
5
  SHA512:
6
- metadata.gz: 9f91eb7a729e6c6c1dff895a715a01c6230ff97c4e87ee697ac5d7c01581f0bc94591c619bd00f5f1304209974d5d769336e89b032e8983a6e1c812cb799abe6
7
- data.tar.gz: 48f14b886f7a2fb7fd547e9aee58cc28f6217193957271995b2917e295bab8d6b62faca337793278982c7f9d89e953aea6b6bb21ce6834ef5a20f1053769fd9b
6
+ metadata.gz: 57009048b5e00c49e272356e9762dcd3e36ee3b47d491b6fda0830c995d2b95880f14a92346545454c7351ea54d0a25801117b7f925a543eccf15926c692c997
7
+ data.tar.gz: bbdb268c976c3cfbf171129a7853fbe6f850974ee5dc94486be599cfd147a15c727f193d488f27ea866921c5215e6b1acee0723896434121d1ca8ab957fe5755
checksums.yaml.gz.sig CHANGED
Binary file
data/CHANGELOG.md CHANGED
@@ -30,6 +30,197 @@ Please file a bug if you notice a violation of semantic versioning.
30
30
 
31
31
  ### Security
32
32
 
33
+ ## [2.0.0] - 2025-12-28
34
+
35
+ - TAG: [v2.0.0][2.0.0t]
36
+ - COVERAGE: 88.47% -- 2894/3271 lines in 53 files
37
+ - BRANCH COVERAGE: 67.83% -- 698/1029 branches in 53 files
38
+ - 98.82% documented
39
+
40
+ ### Added
41
+
42
+ - **RSpec Dependency Tags**: Conditional test execution based on available merge gems
43
+ - New `lib/ast/merge/rspec/dependency_tags.rb` provides automatic test filtering
44
+ - Tags for all merge gems: `:markly_merge`, `:prism_merge`, `:json_merge`, `:toml_merge`, etc.
45
+ - Composite tag `:any_markdown_merge` for tests that work with any markdown merger
46
+ - Negated tags (e.g., `:not_prism_merge`) for testing fallback behavior
47
+ - `AST_MERGE_DEBUG=1` environment variable prints dependency summary
48
+ - Eliminates need for `require` statements inside spec files
49
+ - See [lib/ast/merge/rspec/README.md](lib/ast/merge/rspec/README.md) for full documentation
50
+
51
+ - **Recipe::Preset**: Base class for merge configuration presets
52
+ - Provides merge configuration (signature generators, node typing, preferences) without requiring template files
53
+ - `Recipe::Config` now inherits from `Preset`, adding template/target handling
54
+ - `to_h` method converts preset to SmartMerger-compatible options hash
55
+ - Enables kettle-jem and other libraries to define reusable merge presets
56
+ - Supports script loading for signature generators and node typing via `ScriptLoader`
57
+
58
+ - **exe/ast-merge-recipe**: Shipped executable for running merge recipes
59
+ - Uses `bundler/inline` for dependency management
60
+ - Supports `--dry-run`, `--verbose`, `--parser`, `--base-dir` options
61
+ - Uses `optparse` for proper option parsing
62
+ - Loads merge gems from recipe YAML `merge_gems` section
63
+ - Development mode via `KETTLE_RB_DEV=true` or `--dev-root` option
64
+ - All gems sourced from gem.coop
65
+
66
+ - **PartialTemplateMerger**: Section-based merging for partial templates
67
+ - Find and merge specific sections in destination documents
68
+ - Support for both `replace_mode` (full replacement) and merge mode (intelligent merging)
69
+ - Configurable anchor and boundary matchers for section detection
70
+ - Custom `signature_generator` and `node_typing` support for advanced matching
71
+ - `when_missing` behavior: `:skip`, `:append`, `:prepend`
72
+
73
+ - **Recipe**: YAML-based recipe system for defining merge operations
74
+ - Load recipes from YAML files with `Recipe.load(path)`
75
+ - Define template, targets, injection point, merge preferences
76
+ - Support for `when_missing: skip|add|error` behavior
77
+ - Support for `replace_mode` option
78
+ - Automatic path resolution relative to recipe file location
79
+
80
+ - **RecipeRunner**: Execute recipes against multiple target files
81
+ - Uses PartialTemplateMerger for section-based merging
82
+ - Dry-run mode with `--dry-run` flag
83
+ - Results tracking with status, stats, and error reporting
84
+ - TableTennis integration for formatted output
85
+ - Support for different parsers (markly, commonmarker, prism, psych)
86
+
87
+ - **RecipeScriptLoader**: Load Ruby scripts referenced by recipes
88
+ - Convention: scripts in folder matching recipe basename (e.g., `my_recipe/` for `my_recipe.yml`)
89
+ - Support for inline lambda expressions in YAML
90
+ - Script caching for performance
91
+ - Validation that scripts return callable objects
92
+
93
+ - **bin/ast-merge-recipe**: CLI for running merge recipes
94
+ - `bin/ast-merge-recipe RECIPE_FILE [--dry-run] [--verbose]`
95
+ - Color-coded output with status symbols
96
+ - Summary table with counts
97
+
98
+ - **NavigableStatement**: New wrapper class for uniform node navigation (language-agnostic)
99
+ - Provides flat list navigation (`next`, `previous`, `index`) for all nodes
100
+ - Tree depth calculation with `tree_depth` method
101
+ - `same_or_shallower_than?` for level-based boundary detection
102
+ - Language-agnostic section boundaries using tree hierarchy
103
+ - Provides tree navigation (`tree_parent`, `tree_next`, `tree_previous`) for parser-backed nodes
104
+ - `synthetic?` method to detect nodes without tree navigation (GapLineNode, LinkDefinitionNode, etc.)
105
+ - `type?` and `text_matches?` helpers for node matching
106
+ - `node_attribute` for accessing parser-specific attributes with fallbacks
107
+ - `each_following` and `take_until` for sequential traversal
108
+ - `find_matching` and `find_first` class methods for querying statement lists
109
+ - `build_list` class method to create linked statement lists from raw statements
110
+
111
+ - **InjectionPoint**: New class for defining where content can be injected (language-agnostic)
112
+ - Supports positions: `:before`, `:after`, `:first_child`, `:last_child`, `:replace`
113
+ - Optional boundary for replacement ranges
114
+ - `replacement?`, `child_injection?`, `sibling_injection?` predicates
115
+ - `replaced_statements` to get all statements in a replacement range
116
+
117
+ - **InjectionPointFinder**: New class for finding injection points by matching criteria
118
+ - `find` to locate a single injection point by type/text pattern
119
+ - `find_all` to locate all matching injection points
120
+ - Works with any `*-merge` gem (prism-merge, markly-merge, psych-merge, etc.)
121
+
122
+ - **SmartMergerBase**: `add_template_only_nodes` now accepts a callable filter
123
+ - Boolean `true`/`false` still works as before (add all or none)
124
+ - Callable (Proc/Lambda) receives `(node, entry)` and returns truthy to add the node
125
+ - Enables selective addition of template-only nodes based on signature, type, or content
126
+ - Example use case: Add missing link reference definitions while skipping other template content
127
+ - Entry hash includes `:template_node`, `:signature`, `:template_index` for filtering decisions
128
+
129
+ - **ContentMatchRefiner**: New match refiner for fuzzy text content matching
130
+ - Uses Levenshtein distance to pair nodes with similar (but not identical) text
131
+ - Configurable scoring weights for content similarity, length, and position
132
+ - Custom content extractor support for parser-specific text extraction
133
+ - Node type filtering to limit which types are processed
134
+ - Can be combined with other refiners (e.g., TableMatchRefiner)
135
+ - Useful for matching paragraphs, headings, comments with minor edits
136
+
137
+ - **SmartMergerBase**: Added validity check after FileAnalysis creation
138
+ - Checks `valid?` after creating FileAnalysis and raises appropriate parse error if invalid
139
+ - Catches silent failures like grammar not available or parse errors
140
+ - Documented the FileAnalysis error handling pattern for all *-merge gems
141
+
142
+ - **SmartMergerBase**: Added explicit `node_typing` parameter
143
+ - All child SmartMergers were already using `node_typing` via `**format_options`
144
+ - Now explicitly documented and accessible via `attr_reader :node_typing`
145
+ - Validates node_typing configuration via `NodeTyping.validate!` if provided
146
+ - Enables per-node-type merge preferences across all `*-merge` gems
147
+
148
+ - **ConflictResolverBase**: Added `match_refiner` parameter and `**options` for forward compatibility
149
+ - All batch-strategy resolvers were storing `match_refiner` locally
150
+ - Now explicitly accepted in base class with `attr_reader :match_refiner`
151
+ - Added `**options` catch-all for future parameters without breaking child classes
152
+
153
+ - **MergeResultBase**: Added `**options` for forward compatibility
154
+ - Allows subclasses to accept new parameters without modification
155
+ - Maintains backward compatibility with existing no-arg and keyword-arg constructors
156
+
157
+ - **RBS Signatures**: Added comprehensive type signatures for base classes
158
+ - `SmartMergerBase` with all standard options and abstract method declarations
159
+ - `ConflictResolverBase` with strategy-based resolution methods
160
+ - `MergeResultBase` with unified constructor and decision tracking
161
+ - `MatchRefinerBase` with similarity computation interface
162
+ - `RegionMergeable` module for nested content merging
163
+ - `NodeTyping` module with `Wrapper` class for typed nodes
164
+ - Type aliases: `node_typing_callable`, `node_typing_hash`, `preference_type`
165
+
166
+ - **Documentation**: Updated README with comprehensive base class documentation
167
+ - Standard options table with all `SmartMergerBase` parameters
168
+ - Forward compatibility section explaining the `**options` pattern
169
+ - Complete "Creating a New Merge Gem" example with all base classes
170
+ - Base Classes Reference table
171
+
172
+ - **tree_haver Integration**: Major architectural enhancement
173
+ - Added `tree_haver` (~> 3.1) as a runtime dependency
174
+ - `Ast::Merge::AstNode` now implements the TreeHaver::Node protocol for compatibility with tree_haver-based merge operations
175
+ - Adds: `type`, `kind`, `text`, `start_byte`, `end_byte`, `start_point`, `end_point`, `children`, `child_count`, `child(index)`, `each`, `named?`, `structural?`, `has_error?`, `missing?`, `inner_node`
176
+ - Adds `Point` struct compatible with `TreeHaver::Point`
177
+ - Adds `SyntheticNode` alias for clarity (synthetic = not backed by a real parser)
178
+ - `Comment::Line`, `Comment::Block`, `Comment::Empty` now have explicit `type` methods
179
+ - `Text::LineNode` and `Text::WordNode` now inherit from `AstNode`, gaining TreeHaver::Node protocol compliance
180
+ - This enables `*-merge` gems to leverage tree_haver's cross-Ruby parsing capabilities (MRI, JRuby, TruffleRuby)
181
+ - **Documentation**: Comprehensive updates across the gem family
182
+ - Updated all vendor gem READMEs with standardized gem family tables
183
+ - Added `tree_haver` as the foundation layer in architecture documentation
184
+ - Clarified the two-layer architecture: tree_haver (parsing) → ast-merge (merge infrastructure)
185
+ - Added detailed documentation to `FencedCodeBlockDetector` explaining when to use native AST nodes vs text-based detection
186
+ - Updated markdown-merge documentation to highlight inner code block merging capabilities
187
+ - **Example Scripts**: Added comprehensive examples demonstrating inner-merge capabilities
188
+ - `examples/markdown_code_merge.rb` - Shows how markdown-merge delegates to language-specific parsers for semantic merging
189
+ - Documentation proving that language-specific parsers create full ASTs of embedded code blocks
190
+
191
+ ### Changed
192
+
193
+ - **BREAKING - Namespace Reorganization**: Major restructuring for better organization
194
+ - `Ast::Merge::Region` → `Ast::Merge::Detector::Region` (Struct moved into Detector namespace)
195
+ - `Ast::Merge::RegionDetectorBase` → `Ast::Merge::Detector::Base`
196
+ - `Ast::Merge::RegionMergeable` → `Ast::Merge::Detector::Mergeable`
197
+ - `Ast::Merge::FencedCodeBlockDetector` → `Ast::Merge::Detector::FencedCodeBlock`
198
+ - `Ast::Merge::YamlFrontmatterDetector` → `Ast::Merge::Detector::YamlFrontmatter`
199
+ - `Ast::Merge::TomlFrontmatterDetector` → `Ast::Merge::Detector::TomlFrontmatter`
200
+ - `Ast::Merge::Recipe` class → `Ast::Merge::Recipe::Config`
201
+ - `Ast::Merge::RecipeRunner` → `Ast::Merge::Recipe::Runner`
202
+ - `Ast::Merge::RecipeScriptLoader` → `Ast::Merge::Recipe::ScriptLoader`
203
+ - `Ast::Merge::RegionMergeable::RegionConfig` → `Ast::Merge::Detector::Mergeable::Config`
204
+ - `Ast::Merge::RegionMergeable::ExtractedRegion` → `Ast::Merge::Detector::Mergeable::ExtractedRegion`
205
+
206
+ - **Architecture**: Refactored to use tree_haver as the parsing foundation
207
+ - All tree-sitter-based gems (bash-merge, json-merge, jsonc-merge, toml-merge) now use tree_haver
208
+ - Parser-specific gems (prism-merge, psych-merge, markdown-merge, markly-merge, commonmarker-merge) use tree_haver backends
209
+ - Provides unified API across different Ruby implementations and parsing backends
210
+ - **Documentation Structure**: Standardized gem family tables across all 12 vendor gems
211
+ - Changed from 3-column to 4-column format: Gem | Format | Parser Backend(s) | Description
212
+ - All parser backends now annotated with "(via tree_haver)" where applicable
213
+ - ast-merge description updated from "Shared infrastructure" to "**Infrastructure**: Shared base classes and merge logic"
214
+ - markdown-merge description updated to "**Foundation**: Shared base for Markdown mergers with inner code block merging"
215
+ - **Configuration Documentation**: Enhanced backend selection documentation
216
+
217
+ ### Fixed
218
+
219
+ - Fixed gemspec and Appraisals alignment with tree_haver requirements
220
+ - Fixed CI workflow conditions and retry logic
221
+ - Fixed badge rendering in documentation
222
+ - Fixed README structure issues (removed H3 duplicates, standardized gem family tables)
223
+
33
224
  ## [1.0.0] - 2025-12-12
34
225
 
35
226
  - TAG: [v1.0.0][1.0.0t]
@@ -41,6 +232,8 @@ Please file a bug if you notice a violation of semantic versioning.
41
232
 
42
233
  - Initial release
43
234
 
44
- [Unreleased]: https://github.com/kettle-rb/ast-merge/compare/v1.0.0...HEAD
235
+ [Unreleased]: https://github.com/kettle-rb/ast-merge/compare/v2.0.0...HEAD
236
+ [2.0.0]: https://github.com/kettle-rb/ast-merge/compare/v1.0.0...v2.0.0
237
+ [2.0.0t]: https://github.com/kettle-rb/ast-merge/releases/tag/v2.0.0
45
238
  [1.0.0]: https://github.com/kettle-rb/ast-merge/compare/a63a4858cb229530c1706925bb209546695e8b3a...v1.0.0
46
239
  [1.0.0t]: https://github.com/kettle-rb/ast-merge/tags/v1.0.0
data/README.md CHANGED
@@ -42,7 +42,7 @@
42
42
 
43
43
  # ☯️ Ast::Merge
44
44
 
45
- [![Version][👽versioni]][👽version] [![GitHub tag (latest SemVer)][⛳️tag-img]][⛳️tag] [![License: MIT][📄license-img]][📄license-ref] [![Downloads Rank][👽dl-ranki]][👽dl-rank] [![Open Source Helpers][👽oss-helpi]][👽oss-help] [![CodeCov Test Coverage][🏀codecovi]][🏀codecov] [![Coveralls Test Coverage][🏀coveralls-img]][🏀coveralls] [![QLTY Test Coverage][🏀qlty-covi]][🏀qlty-cov] [![QLTY Maintainability][🏀qlty-mnti]][🏀qlty-mnt] [![CI Heads][🚎3-hd-wfi]][🚎3-hd-wf] [![CI Runtime Dependencies @ HEAD][🚎12-crh-wfi]][🚎12-crh-wf] [![CI Current][🚎11-c-wfi]][🚎11-c-wf] [![CI Truffle Ruby][🚎9-t-wfi]][🚎9-t-wf] [![CI JRuby][🚎10-j-wfi]][🚎10-j-wf] [![Deps Locked][🚎13-🔒️-wfi]][🚎13-🔒️-wf] [![Deps Unlocked][🚎14-🔓️-wfi]][🚎14-🔓️-wf] [![CI Supported][🚎6-s-wfi]][🚎6-s-wf] [![CI Legacy][🚎4-lg-wfi]][🚎4-lg-wf] [![CI Unsupported][🚎7-us-wfi]][🚎7-us-wf] [![CI Ancient][🚎1-an-wfi]][🚎1-an-wf] [![CI Test Coverage][🚎2-cov-wfi]][🚎2-cov-wf] [![CI Style][🚎5-st-wfi]][🚎5-st-wf] [![CodeQL][🖐codeQL-img]][🖐codeQL] [![Apache SkyWalking Eyes License Compatibility Check][🚎15-🪪-wfi]][🚎15-🪪-wf]
45
+ [![Version][👽versioni]][👽version] [![GitHub tag (latest SemVer)][⛳️tag-img]][⛳️tag] [![License: MIT][📄license-img]][📄license-ref] [![Downloads Rank][👽dl-ranki]][👽dl-rank] [![Open Source Helpers][👽oss-helpi]][👽oss-help] [![CodeCov Test Coverage][🏀codecovi]][🏀codecov] [![Coveralls Test Coverage][🏀coveralls-img]][🏀coveralls] [![QLTY Test Coverage][🏀qlty-covi]][🏀qlty-cov] [![QLTY Maintainability][🏀qlty-mnti]][🏀qlty-mnt] [![CI Heads][🚎3-hd-wfi]][🚎3-hd-wf] [![CI Runtime Dependencies @ HEAD][🚎12-crh-wfi]][🚎12-crh-wf] [![CI Current][🚎11-c-wfi]][🚎11-c-wf] [![CI Truffle Ruby][🚎9-t-wfi]][🚎9-t-wf] [![Deps Locked][🚎13-🔒️-wfi]][🚎13-🔒️-wf] [![Deps Unlocked][🚎14-🔓️-wfi]][🚎14-🔓️-wf] [![CI Supported][🚎6-s-wfi]][🚎6-s-wf] [![CI Test Coverage][🚎2-cov-wfi]][🚎2-cov-wf] [![CI Style][🚎5-st-wfi]][🚎5-st-wf] [![CodeQL][🖐codeQL-img]][🖐codeQL] [![Apache SkyWalking Eyes License Compatibility Check][🚎15-🪪-wfi]][🚎15-🪪-wf]
46
46
 
47
47
  `if ci_badges.map(&:color).detect { it != "green"}` ☝️ [let me know][🖼️galtzo-discord], as I may have missed the [discord notification][🖼️galtzo-discord].
48
48
 
@@ -58,20 +58,23 @@ Ast::Merge is **not typically used directly** - instead, use one of the format-s
58
58
 
59
59
  ### The `*-merge` Gem Family
60
60
 
61
- | Gem | Format | Parser | Description |
62
- |-----|--------|--------|-------------|
63
- | [ast-merge][ast-merge] | Text | internal | Shared infrastructure for all `*-merge` gems |
64
- | [prism-merge][prism-merge] | Ruby | [Prism][prism] | Smart merge for Ruby source files |
65
- | [psych-merge][psych-merge] | YAML | [Psych][psych] | Smart merge for YAML files |
66
- | [json-merge][json-merge] | JSON | [tree-sitter-json][ts-json] | Smart merge for JSON files |
67
- | [jsonc-merge][jsonc-merge] | JSONC | [tree-sitter-jsonc][ts-jsonc] | ⚠️ Proof of concept; Smart merge for JSON with Comments |
68
- | [bash-merge][bash-merge] | Bash | [tree-sitter-bash][ts-bash] | Smart merge for Bash scripts |
69
- | [rbs-merge][rbs-merge] | RBS | [RBS][rbs] | Smart merge for Ruby type signatures |
70
- | [dotenv-merge][dotenv-merge] | Dotenv | internal ([dotenv][dotenv]) | Smart merge for `.env` files |
71
- | [toml-merge][toml-merge] | TOML | [tree-sitter-toml][ts-toml] | Smart merge for TOML files |
72
- | [markdown-merge][markdown-merge] | Markdown | _base classes_ | Shared foundation for Markdown mergers |
73
- | [markly-merge][markly-merge] | Markdown | [Markly][markly] | Smart merge for Markdown (CommonMark via libcmark-gfm) |
74
- | [commonmarker-merge][commonmarker-merge] | Markdown | [Commonmarker][commonmarker] | Smart merge for Markdown (CommonMark via comrak) |
61
+ The `*-merge` gem family provides intelligent, AST-based merging for various file formats. At the foundation is [tree_haver][tree_haver], which provides a unified cross-Ruby parsing API that works seamlessly across MRI, JRuby, and TruffleRuby.
62
+
63
+ | Gem | Format | Parser Backend(s) | Description |
64
+ |------------------------------------------|----------|-----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
65
+ | [tree_haver][tree_haver] | Multi | MRI C, Rust, FFI, Java, Prism, Psych, Commonmarker, Markly, Citrus | **Foundation**: Cross-Ruby adapter for parsing libraries (like Faraday for HTTP) |
66
+ | [ast-merge][ast-merge] | Text | internal | **Infrastructure**: Shared base classes and merge logic for all `*-merge` gems |
67
+ | [prism-merge][prism-merge] | Ruby | [Prism][prism] | Smart merge for Ruby source files |
68
+ | [psych-merge][psych-merge] | YAML | [Psych][psych] | Smart merge for YAML files |
69
+ | [json-merge][json-merge] | JSON | [tree-sitter-json][ts-json] (via tree_haver) | Smart merge for JSON files |
70
+ | [jsonc-merge][jsonc-merge] | JSONC | [tree-sitter-jsonc][ts-jsonc] (via tree_haver) | ⚠️ Proof of concept; Smart merge for JSON with Comments |
71
+ | [bash-merge][bash-merge] | Bash | [tree-sitter-bash][ts-bash] (via tree_haver) | Smart merge for Bash scripts |
72
+ | [rbs-merge][rbs-merge] | RBS | [RBS][rbs] | Smart merge for Ruby type signatures |
73
+ | [dotenv-merge][dotenv-merge] | Dotenv | internal | Smart merge for `.env` files |
74
+ | [toml-merge][toml-merge] | TOML | [Citrus + toml-rb][toml-rb] (default, via tree_haver), [tree-sitter-toml][ts-toml] (via tree_haver) | Smart merge for TOML files |
75
+ | [markdown-merge][markdown-merge] | Markdown | [Commonmarker][commonmarker] / [Markly][markly] (via tree_haver) | **Foundation**: Shared base for Markdown mergers with inner code block merging |
76
+ | [markly-merge][markly-merge] | Markdown | [Markly][markly] (via tree_haver) | Smart merge for Markdown (CommonMark via cmark-gfm C) |
77
+ | [commonmarker-merge][commonmarker-merge] | Markdown | [Commonmarker][commonmarker] (via tree_haver) | Smart merge for Markdown (CommonMark via comrak Rust) |
75
78
 
76
79
  **Example implementations** for the gem templating use case:
77
80
 
@@ -80,6 +83,7 @@ Ast::Merge is **not typically used directly** - instead, use one of the format-s
80
83
  | [kettle-dev][kettle-dev] | Gem Development | Gem templating tool using `*-merge` gems |
81
84
  | [kettle-jem][kettle-jem] | Gem Templating | Gem template library with smart merge support |
82
85
 
86
+ [tree_haver]: https://github.com/kettle-rb/tree_haver
83
87
  [ast-merge]: https://github.com/kettle-rb/ast-merge
84
88
  [prism-merge]: https://github.com/kettle-rb/prism-merge
85
89
  [psych-merge]: https://github.com/kettle-rb/psych-merge
@@ -97,20 +101,38 @@ Ast::Merge is **not typically used directly** - instead, use one of the format-s
97
101
  [prism]: https://github.com/ruby/prism
98
102
  [psych]: https://github.com/ruby/psych
99
103
  [ts-json]: https://github.com/tree-sitter/tree-sitter-json
100
- [ts-jsonc]: https://gitlab.com/WhyNotHugo/tree-sitter-jsonc
101
104
  [ts-bash]: https://github.com/tree-sitter/tree-sitter-bash
102
105
  [ts-toml]: https://github.com/tree-sitter-grammars/tree-sitter-toml
103
106
  [rbs]: https://github.com/ruby/rbs
104
- [dotenv]: https://github.com/bkeepers/dotenv
105
- [markly]: https://github.com/kivikakk/markly
107
+ [toml-rb]: https://github.com/emancu/toml-rb
108
+ [markly]: https://github.com/ioquatix/markly
106
109
  [commonmarker]: https://github.com/gjtorikian/commonmarker
107
110
 
108
- ### What Ast::Merge Provides
111
+ ### Architecture: tree_haver + ast-merge
112
+
113
+ The `*-merge` gem family is built on a two-layer architecture:
114
+
115
+ #### Layer 1: tree_haver (Parsing Foundation)
116
+
117
+ [tree_haver][tree_haver] provides cross-Ruby parsing capabilities:
118
+
119
+ - **Universal Backend Support**: Automatically selects the best parsing backend for your Ruby implementation (MRI, JRuby, TruffleRuby)
120
+ - **10 Backend Options**: MRI C extensions, Rust bindings, FFI, Java (JRuby), language-specific parsers (Prism, Psych, Commonmarker, Markly), and pure Ruby fallback (Citrus)
121
+ - **Unified API**: Write parsing code once, run on any Ruby implementation
122
+ - **Grammar Discovery**: Built-in `GrammarFinder` for platform-aware grammar library discovery
123
+ - **Thread-Safe**: Language registry with thread-safe caching
124
+
125
+ #### Layer 2: ast-merge (Merge Infrastructure)
126
+
127
+ Ast::Merge builds on tree_haver to provide:
109
128
 
110
129
  - **Base Classes**: `FreezeNode`, `MergeResult` base classes with unified constructors
111
- - **Shared Modules**: `FileAnalysisBase`, `MergerConfig`, `DebugLogger`
112
- - **Freeze Block Support**: Configurable marker patterns for multiple comment syntaxes
130
+ - **Shared Modules**: `FileAnalysisBase`, `FileAnalyzable`, `MergerConfig`, `DebugLogger`
131
+ - **Freeze Block Support**: Configurable marker patterns for multiple comment syntaxes (preserve sections during merge)
132
+ - **Node Typing System**: `NodeTyping` for canonical node type identification across different parsers
133
+ - **Conflict Resolution**: `ConflictResolverBase` with pluggable strategies
113
134
  - **Error Classes**: `ParseError`, `TemplateParseError`, `DestinationParseError`
135
+ - **Region Detection**: `RegionDetectorBase`, `FencedCodeBlockDetector` for text-based analysis
114
136
  - **RSpec Shared Examples**: Test helpers for implementing new merge gems
115
137
 
116
138
  ### Creating a New Merge Gem
@@ -120,34 +142,168 @@ require "ast/merge"
120
142
 
121
143
  module MyFormat
122
144
  module Merge
123
- class FreezeNode < Ast::Merge::FreezeNode
124
- # Override methods as needed for your format
145
+ # Inherit from base classes and pass **options for forward compatibility
146
+
147
+ class SmartMerger < Ast::Merge::SmartMergerBase
148
+ DEFAULT_FREEZE_TOKEN = "myformat-merge"
149
+
150
+ def initialize(template, dest, my_custom_option: nil, **options)
151
+ @my_custom_option = my_custom_option
152
+ super(template, dest, **options)
153
+ end
154
+
155
+ protected
156
+
157
+ def analysis_class
158
+ FileAnalysis
159
+ end
160
+
161
+ def default_freeze_token
162
+ DEFAULT_FREEZE_TOKEN
163
+ end
164
+
165
+ def perform_merge
166
+ # Implement format-specific merge logic
167
+ # Returns a MergeResult
168
+ end
125
169
  end
126
-
127
- class MergeResult < Ast::Merge::MergeResult
128
- # Add format-specific output methods
170
+
171
+ class FileAnalysis
172
+ include Ast::Merge::FileAnalyzable
173
+
174
+ def initialize(source, freeze_token: nil, signature_generator: nil, **options)
175
+ @source = source
176
+ @freeze_token = freeze_token
177
+ @signature_generator = signature_generator
178
+ # Process source...
179
+ end
180
+
181
+ def compute_node_signature(node)
182
+ # Return signature array for node matching
183
+ end
184
+ end
185
+
186
+ class ConflictResolver < Ast::Merge::ConflictResolverBase
187
+ def initialize(template_analysis, dest_analysis, preference: :destination,
188
+ add_template_only_nodes: false, match_refiner: nil, **options)
189
+ super(
190
+ strategy: :batch, # or :node, :boundary
191
+ preference: preference,
192
+ template_analysis: template_analysis,
193
+ dest_analysis: dest_analysis,
194
+ add_template_only_nodes: add_template_only_nodes,
195
+ match_refiner: match_refiner,
196
+ **options
197
+ )
198
+ end
199
+
200
+ protected
201
+
202
+ def resolve_batch(result)
203
+ # Implement batch resolution logic
204
+ end
205
+ end
206
+
207
+ class MergeResult < Ast::Merge::MergeResultBase
208
+ def initialize(**options)
209
+ super(**options)
210
+ @statistics = { merged_count: 0 }
211
+ end
212
+
129
213
  def to_my_format
130
214
  to_s
131
215
  end
132
216
  end
217
+
218
+ class MatchRefiner < Ast::Merge::MatchRefinerBase
219
+ def initialize(threshold: 0.7, node_types: nil, **options)
220
+ super(threshold: threshold, node_types: node_types, **options)
221
+ end
222
+
223
+ def similarity(template_node, dest_node)
224
+ # Return similarity score between 0.0 and 1.0
225
+ end
226
+ end
227
+ end
228
+ end
229
+ ```
133
230
 
134
- class FileAnalysis
135
- include Ast::Merge::FileAnalysisBase
231
+ ### Base Classes Reference
136
232
 
137
- # Implement required methods:
138
- # - compute_node_signature(node)
139
- # - extract_freeze_blocks
140
- end
233
+ | Base Class | Purpose | Key Methods to Implement |
234
+ |------------|---------|-------------------------|
235
+ | `SmartMergerBase` | Main merge orchestration | `analysis_class`, `perform_merge` |
236
+ | `ConflictResolverBase` | Resolve node conflicts | `resolve_batch` or `resolve_node_pair` |
237
+ | `MergeResultBase` | Track merge results | `to_s`, format-specific output |
238
+ | `MatchRefinerBase` | Fuzzy node matching | `similarity` |
239
+ | `ContentMatchRefiner` | Text content fuzzy matching | Ready to use |
240
+ | `FileAnalyzable` | File parsing/analysis | `compute_node_signature` |
141
241
 
142
- class SmartMerger
143
- include Ast::Merge::MergerConfig
242
+ ### ContentMatchRefiner
144
243
 
145
- # Implement merge logic
146
- end
147
- end
148
- end
244
+ `Ast::Merge::ContentMatchRefiner` is a built-in match refiner for fuzzy text content matching using Levenshtein distance. Unlike signature-based matching which requires exact content hashes, this refiner allows matching nodes with similar (but not identical) content.
245
+
246
+ ```ruby
247
+ # Basic usage - match nodes with 70% similarity
248
+ refiner = Ast::Merge::ContentMatchRefiner.new(threshold: 0.7)
249
+
250
+ # Only match specific node types
251
+ refiner = Ast::Merge::ContentMatchRefiner.new(
252
+ threshold: 0.6,
253
+ node_types: [:paragraph, :heading]
254
+ )
255
+
256
+ # Custom weights for scoring
257
+ refiner = Ast::Merge::ContentMatchRefiner.new(
258
+ threshold: 0.7,
259
+ weights: {
260
+ content: 0.8, # Levenshtein similarity (default: 0.7)
261
+ length: 0.1, # Length similarity (default: 0.15)
262
+ position: 0.1 # Position in document (default: 0.15)
263
+ }
264
+ )
265
+
266
+ # Custom content extraction
267
+ refiner = Ast::Merge::ContentMatchRefiner.new(
268
+ threshold: 0.7,
269
+ content_extractor: ->(node) { node.text_content.downcase.strip }
270
+ )
271
+
272
+ # Use with a merger
273
+ merger = MyFormat::SmartMerger.new(
274
+ template,
275
+ destination,
276
+ preference: :template,
277
+ match_refiner: refiner
278
+ )
149
279
  ```
150
280
 
281
+ This is particularly useful for:
282
+ - Paragraphs with minor edits (typos, rewording)
283
+ - Headings with slight changes
284
+ - Comments with updated text
285
+ - Any text-based node that may have been slightly modified
286
+
287
+ ### Namespace Reference
288
+
289
+ The `Ast::Merge` module is organized into several namespaces, each with detailed documentation:
290
+
291
+ | Namespace | Purpose | Documentation |
292
+ |-----------|---------|---------------|
293
+ | `Ast::Merge::Detector` | Region detection and merging | [lib/ast/merge/detector/README.md](lib/ast/merge/detector/README.md) |
294
+ | `Ast::Merge::Recipe` | YAML-based merge recipes | [lib/ast/merge/recipe/README.md](lib/ast/merge/recipe/README.md) |
295
+ | `Ast::Merge::Comment` | Comment parsing and representation | [lib/ast/merge/comment/README.md](lib/ast/merge/comment/README.md) |
296
+ | `Ast::Merge::Text` | Plain text AST parsing | [lib/ast/merge/text/README.md](lib/ast/merge/text/README.md) |
297
+ | `Ast::Merge::RSpec` | Shared RSpec examples | [lib/ast/merge/rspec/README.md](lib/ast/merge/rspec/README.md) |
298
+
299
+ **Key Classes by Namespace:**
300
+
301
+ - **Detector**: `Region`, `Base`, `Mergeable`, `FencedCodeBlock`, `YamlFrontmatter`, `TomlFrontmatter`
302
+ - **Recipe**: `Config`, `Runner`, `ScriptLoader`
303
+ - **Comment**: `Line`, `Block`, `Empty`, `Parser`, `Style`
304
+ - **Text**: `SmartMerger`, `FileAnalysis`, `LineNode`, `WordNode`, `Section`
305
+ - **RSpec**: Shared examples and dependency tags for testing `*-merge` implementations
306
+
151
307
  ## 💡 Info you can shake a stick at
152
308
 
153
309
  | Tokens to Remember | [![Gem name][⛳️name-img]][⛳️gem-name] [![Gem namespace][⛳️namespace-img]][⛳️gem-namespace] |
@@ -274,8 +430,8 @@ merger = SomeFormat::Merge::SmartMerger.new(
274
430
  destination,
275
431
  # When conflicts occur, prefer template or destination values
276
432
  preference: :template, # or :destination (default), or a Hash for per-node-type
277
- # Add nodes that only exist in template
278
- add_template_only_nodes: true, # default: false
433
+ # Add nodes that only exist in template (Boolean or callable filter)
434
+ add_template_only_nodes: true, # default: false, or ->(node, entry) { ... }
279
435
  # Custom node type handling
280
436
  node_typing: {}, # optional, for per-node-type preference
281
437
  )
@@ -293,8 +449,41 @@ Control which source wins when both files have the same structural element:
293
449
 
294
450
  Control whether to add nodes that only exist in the template:
295
451
 
296
- - **`true`** - Add new nodes from template
452
+ - **`true`** - Add all template-only nodes
297
453
  - **`false`** (default) - Skip template-only nodes
454
+ - **Callable** - Filter which template-only nodes to add
455
+
456
+ #### Callable Filter
457
+
458
+ When you need fine-grained control over which template-only nodes are added, pass a callable (Proc/Lambda) that receives `(node, entry)` and returns truthy to add or falsey to skip:
459
+
460
+ ```ruby
461
+ # Only add nodes with gem_family signatures
462
+ merger = SomeFormat::Merge::SmartMerger.new(
463
+ template,
464
+ destination,
465
+ add_template_only_nodes: ->(node, entry) {
466
+ sig = entry[:signature]
467
+ sig.is_a?(Array) && sig.first == :gem_family
468
+ }
469
+ )
470
+
471
+ # Only add link definitions that match a pattern
472
+ merger = Markly::Merge::SmartMerger.new(
473
+ template,
474
+ destination,
475
+ add_template_only_nodes: ->(node, entry) {
476
+ entry[:template_node].type == :link_definition &&
477
+ entry[:signature]&.last&.include?("gem")
478
+ }
479
+ )
480
+ ```
481
+
482
+ The `entry` hash contains:
483
+ - `:template_node` - The node being considered for addition
484
+ - `:signature` - The node's signature (Array or other value)
485
+ - `:template_index` - Index in the template statements
486
+ - `:dest_index` - Always `nil` for template-only nodes
298
487
 
299
488
  ## 🔧 Basic Usage
300
489
 
@@ -462,7 +651,7 @@ The `MergerConfig` class provides factory methods that support all options:
462
651
  # Create config preferring destination
463
652
  config = Ast::Merge::MergerConfig.destination_wins(
464
653
  freeze_token: "my-freeze",
465
- signature_generator: my_generator,
654
+ : my_generator,
466
655
  node_typing: my_typing,
467
656
  )
468
657
 
@@ -765,26 +954,16 @@ Thanks for RTFM. ☺️
765
954
  [🏀coveralls-img]: https://coveralls.io/repos/github/kettle-rb/ast-merge/badge.svg?branch=main
766
955
  [🖐codeQL]: https://github.com/kettle-rb/ast-merge/security/code-scanning
767
956
  [🖐codeQL-img]: https://github.com/kettle-rb/ast-merge/actions/workflows/codeql-analysis.yml/badge.svg
768
- [🚎1-an-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/ancient.yml
769
- [🚎1-an-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/ancient.yml/badge.svg
770
957
  [🚎2-cov-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/coverage.yml
771
958
  [🚎2-cov-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/coverage.yml/badge.svg
772
959
  [🚎3-hd-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/heads.yml
773
960
  [🚎3-hd-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/heads.yml/badge.svg
774
- [🚎4-lg-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/legacy.yml
775
- [🚎4-lg-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/legacy.yml/badge.svg
776
961
  [🚎5-st-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/style.yml
777
962
  [🚎5-st-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/style.yml/badge.svg
778
963
  [🚎6-s-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/supported.yml
779
964
  [🚎6-s-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/supported.yml/badge.svg
780
- [🚎7-us-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/unsupported.yml
781
- [🚎7-us-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/unsupported.yml/badge.svg
782
- [🚎8-ho-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/hoary.yml
783
- [🚎8-ho-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/hoary.yml/badge.svg
784
965
  [🚎9-t-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/truffle.yml
785
966
  [🚎9-t-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/truffle.yml/badge.svg
786
- [🚎10-j-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/jruby.yml
787
- [🚎10-j-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/jruby.yml/badge.svg
788
967
  [🚎11-c-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/current.yml
789
968
  [🚎11-c-wfi]: https://github.com/kettle-rb/ast-merge/actions/workflows/current.yml/badge.svg
790
969
  [🚎12-crh-wf]: https://github.com/kettle-rb/ast-merge/actions/workflows/dep-heads.yml
@@ -830,7 +1009,7 @@ Thanks for RTFM. ☺️
830
1009
  [📌gitmoji]: https://gitmoji.dev
831
1010
  [📌gitmoji-img]: https://img.shields.io/badge/gitmoji_commits-%20%F0%9F%98%9C%20%F0%9F%98%8D-34495e.svg?style=flat-square
832
1011
  [🧮kloc]: https://www.youtube.com/watch?v=dQw4w9WgXcQ
833
- [🧮kloc-img]: https://img.shields.io/badge/KLOC-2.382-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
1012
+ [🧮kloc-img]: https://img.shields.io/badge/KLOC-3.271-FFDD67.svg?style=for-the-badge&logo=YouTube&logoColor=blue
834
1013
  [🔐security]: SECURITY.md
835
1014
  [🔐security-img]: https://img.shields.io/badge/security-policy-259D6C.svg?style=flat
836
1015
  [📄copyright-notice-explainer]: https://opensource.stackexchange.com/questions/5778/why-do-licenses-such-as-the-mit-license-specify-a-single-year
@@ -850,3 +1029,6 @@ Thanks for RTFM. ☺️
850
1029
  [💎appraisal2]: https://github.com/appraisal-rb/appraisal2
851
1030
  [💎appraisal2-img]: https://img.shields.io/badge/appraised_by-appraisal2-34495e.svg?plastic&logo=ruby&logoColor=white
852
1031
  [💎d-in-dvcs]: https://railsbling.com/posts/dvcs/put_the_d_in_dvcs/
1032
+
1033
+ [ts-jsonc]: https://gitlab.com/WhyNotHugo/tree-sitter-jsonc
1034
+ [dotenv]: https://github.com/bkeepers/dotenv