prism 1.9.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (258) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +144 -40
  3. data/CODE_OF_CONDUCT.md +1 -1
  4. data/LICENSE.md +1 -1
  5. data/Makefile +49 -20
  6. data/README.md +38 -98
  7. data/config.yml +174 -161
  8. data/docs/build_system.md +2 -2
  9. data/docs/configuration.md +5 -4
  10. data/docs/encoding.md +2 -2
  11. data/docs/fuzzing.md +9 -16
  12. data/docs/javascript.md +19 -4
  13. data/docs/mapping.md +1 -1
  14. data/docs/releasing.md +77 -5
  15. data/docs/relocation.md +2 -4
  16. data/docs/serialization.md +39 -28
  17. data/docs/testing.md +46 -29
  18. data/ext/prism/api_node.c +1527 -1467
  19. data/ext/prism/extconf.rb +3 -4
  20. data/ext/prism/extension.c +759 -418
  21. data/ext/prism/extension.h +3 -3
  22. data/include/prism/arena.h +45 -0
  23. data/include/prism/arena.hpp +42 -0
  24. data/include/prism/ast.h +2738 -611
  25. data/include/prism/ast.hpp +641 -0
  26. data/include/prism/buffer.h +60 -0
  27. data/include/prism/buffer.hpp +55 -0
  28. data/include/prism/comments.h +72 -0
  29. data/include/prism/compiler/accel.h +26 -0
  30. data/include/prism/compiler/align.h +36 -0
  31. data/include/prism/compiler/assume.h +25 -0
  32. data/include/prism/compiler/branch.h +26 -0
  33. data/include/prism/compiler/bswap.h +46 -0
  34. data/include/prism/compiler/cplusplus.hpp +16 -0
  35. data/include/prism/compiler/endianness.h +34 -0
  36. data/include/prism/compiler/exceptions.hpp +17 -0
  37. data/include/prism/compiler/exported.h +24 -0
  38. data/include/prism/compiler/fallthrough.h +22 -0
  39. data/include/prism/compiler/filesystem.h +32 -0
  40. data/include/prism/compiler/flex_array.h +19 -0
  41. data/include/prism/compiler/force_inline.h +21 -0
  42. data/include/prism/compiler/format.h +27 -0
  43. data/include/prism/compiler/inline.h +17 -0
  44. data/include/prism/compiler/lifetimebound.hpp +25 -0
  45. data/include/prism/compiler/nodiscard.h +22 -0
  46. data/include/prism/compiler/nonnull.h +18 -0
  47. data/include/prism/compiler/unused.h +18 -0
  48. data/include/prism/constant_pool.h +106 -0
  49. data/include/prism/diagnostic.h +55 -402
  50. data/include/prism/errors_format.h +61 -0
  51. data/include/prism/excludes.h +29 -0
  52. data/include/prism/integer.h +41 -0
  53. data/include/prism/internal/allocator.h +68 -0
  54. data/include/prism/internal/allocator_debug.h +88 -0
  55. data/include/prism/internal/arena.h +108 -0
  56. data/include/prism/internal/bit.h +42 -0
  57. data/include/prism/internal/buffer.h +94 -0
  58. data/include/prism/internal/char.h +145 -0
  59. data/include/prism/internal/comments.h +54 -0
  60. data/include/prism/internal/constant_pool.h +152 -0
  61. data/include/prism/internal/diagnostic.h +389 -0
  62. data/include/prism/{encoding.h → internal/encoding.h} +27 -68
  63. data/include/prism/internal/integer.h +68 -0
  64. data/include/prism/internal/isinf.h +16 -0
  65. data/include/prism/internal/line_offset_list.h +34 -0
  66. data/include/prism/{util/pm_list.h → internal/list.h} +11 -52
  67. data/include/prism/internal/magic_comments.h +23 -0
  68. data/include/prism/internal/memchr.h +15 -0
  69. data/include/prism/internal/node.h +42 -0
  70. data/include/prism/internal/options.h +232 -0
  71. data/include/prism/internal/parser.h +1015 -0
  72. data/include/prism/internal/regexp.h +26 -0
  73. data/include/prism/internal/serialize.h +39 -0
  74. data/include/prism/internal/source.h +72 -0
  75. data/include/prism/internal/static_literals.h +60 -0
  76. data/include/prism/internal/stringy.h +30 -0
  77. data/include/prism/{util/pm_strncasecmp.h → internal/strncasecmp.h} +3 -17
  78. data/include/prism/{util/pm_strpbrk.h → internal/strpbrk.h} +7 -20
  79. data/include/prism/internal/tokens.h +11 -0
  80. data/include/prism/internal/xxh3.h +16 -0
  81. data/include/prism/json.h +40 -0
  82. data/include/prism/line_offset_list.h +69 -0
  83. data/include/prism/magic_comments.h +53 -0
  84. data/include/prism/node.h +59 -49
  85. data/include/prism/node.hpp +272 -0
  86. data/include/prism/node_type.hpp +38 -0
  87. data/include/prism/options.h +154 -279
  88. data/include/prism/options.hpp +43 -0
  89. data/include/prism/parser.h +367 -870
  90. data/include/prism/parser.hpp +274 -0
  91. data/include/prism/prettyprint.h +14 -9
  92. data/include/prism/serialize.h +123 -0
  93. data/include/prism/source.h +163 -0
  94. data/include/prism/stream.h +36 -0
  95. data/include/prism/string_query.h +82 -0
  96. data/include/prism/stringy.h +61 -0
  97. data/include/prism/stringy.hpp +31 -0
  98. data/include/prism/version.h +20 -3
  99. data/include/prism.h +35 -320
  100. data/include/prism.hpp +24 -0
  101. data/lib/prism/compiler.rb +319 -305
  102. data/lib/prism/desugar_compiler.rb +105 -34
  103. data/lib/prism/dispatcher.rb +646 -473
  104. data/lib/prism/dot_visitor.rb +592 -511
  105. data/lib/prism/dsl.rb +391 -25
  106. data/lib/prism/ffi.rb +258 -90
  107. data/lib/prism/inspect_visitor.rb +357 -325
  108. data/lib/prism/lex_compat.rb +254 -212
  109. data/lib/prism/mutation_compiler.rb +313 -305
  110. data/lib/prism/node.rb +20966 -6992
  111. data/lib/prism/node_ext.rb +180 -187
  112. data/lib/prism/node_find.rb +215 -0
  113. data/lib/prism/parse_result/errors.rb +7 -1
  114. data/lib/prism/parse_result/newlines.rb +276 -8
  115. data/lib/prism/parse_result.rb +468 -139
  116. data/lib/prism/pattern.rb +65 -20
  117. data/lib/prism/reflection.rb +23 -12
  118. data/lib/prism/relocation.rb +149 -87
  119. data/lib/prism/serialize.rb +2889 -700
  120. data/lib/prism/string_query.rb +24 -1
  121. data/lib/prism/translation/parser/builder.rb +10 -2
  122. data/lib/prism/translation/parser/compiler.rb +44 -52
  123. data/lib/prism/translation/parser/lexer.rb +41 -92
  124. data/lib/prism/translation/parser.rb +1 -1
  125. data/lib/prism/translation/parser_current.rb +1 -1
  126. data/lib/prism/translation/ripper/lexer.rb +8 -10
  127. data/lib/prism/translation/ripper/sexp.rb +2 -10
  128. data/lib/prism/translation/ripper/shim.rb +2 -0
  129. data/lib/prism/translation/ripper.rb +967 -209
  130. data/lib/prism/translation/ruby_parser.rb +73 -316
  131. data/lib/prism/translation.rb +2 -0
  132. data/lib/prism/visitor.rb +632 -5
  133. data/lib/prism.rb +79 -9
  134. data/prism.gemspec +141 -62
  135. data/rbi/generated/prism/compiler.rbi +489 -0
  136. data/rbi/generated/prism/desugar_compiler.rbi +306 -0
  137. data/rbi/generated/prism/dispatcher.rbi +1138 -0
  138. data/rbi/generated/prism/dot_visitor.rbi +618 -0
  139. data/rbi/generated/prism/dsl.rbi +751 -0
  140. data/rbi/generated/prism/inspect_visitor.rbi +503 -0
  141. data/rbi/generated/prism/lex_compat.rbi +162 -0
  142. data/rbi/generated/prism/mutation_compiler.rbi +464 -0
  143. data/rbi/generated/prism/node.rbi +15143 -0
  144. data/rbi/generated/prism/node_ext.rbi +259 -0
  145. data/rbi/generated/prism/node_find.rbi +79 -0
  146. data/rbi/generated/prism/parse_result/errors.rbi +21 -0
  147. data/rbi/generated/prism/parse_result/newlines.rbi +208 -0
  148. data/rbi/generated/prism/parse_result.rbi +776 -0
  149. data/rbi/generated/prism/pattern.rbi +134 -0
  150. data/rbi/generated/prism/reflection.rbi +89 -0
  151. data/rbi/generated/prism/relocation.rbi +329 -0
  152. data/rbi/generated/prism/serialize.rbi +185 -0
  153. data/rbi/generated/prism/string_query.rbi +43 -0
  154. data/rbi/generated/prism/translation.rbi +7 -0
  155. data/rbi/generated/prism/visitor.rbi +650 -0
  156. data/rbi/generated/prism.rbi +86 -0
  157. data/rbi/prism/translation/parser_versions.rbi +5 -0
  158. data/rbi/rubyvm/node_find.rbi +14 -0
  159. data/sig/generated/prism/compiler.rbs +492 -0
  160. data/sig/generated/prism/desugar_compiler.rbs +305 -0
  161. data/sig/generated/prism/dispatcher.rbs +1292 -0
  162. data/sig/generated/prism/dot_visitor.rbs +635 -0
  163. data/sig/generated/prism/dsl.rbs +928 -0
  164. data/sig/generated/prism/inspect_visitor.rbs +509 -0
  165. data/sig/generated/prism/lex_compat.rbs +166 -0
  166. data/sig/{prism → generated/prism}/mutation_compiler.rbs +309 -4
  167. data/sig/generated/prism/node.rbs +21632 -0
  168. data/sig/generated/prism/node_ext.rbs +290 -0
  169. data/sig/generated/prism/node_find.rbs +90 -0
  170. data/sig/generated/prism/parse_result/errors.rbs +22 -0
  171. data/sig/generated/prism/parse_result/newlines.rbs +223 -0
  172. data/sig/generated/prism/parse_result.rbs +871 -0
  173. data/sig/generated/prism/pattern.rbs +156 -0
  174. data/sig/generated/prism/reflection.rbs +101 -0
  175. data/sig/generated/prism/relocation.rbs +428 -0
  176. data/sig/generated/prism/serialize.rbs +200 -0
  177. data/sig/generated/prism/string_query.rbs +43 -0
  178. data/sig/generated/prism/translation.rbs +8 -0
  179. data/sig/generated/prism/visitor.rbs +1111 -0
  180. data/sig/generated/prism.rbs +84 -0
  181. data/sig/manifest.yaml +2 -0
  182. data/src/arena.c +122 -0
  183. data/src/{util/pm_buffer.c → buffer.c} +53 -21
  184. data/src/{util/pm_char.c → char.c} +28 -57
  185. data/src/comments.c +455 -0
  186. data/src/constant_pool.c +406 -0
  187. data/src/diagnostic.c +120 -68
  188. data/src/encoding.c +8 -3
  189. data/src/errors_format.c +519 -0
  190. data/src/{util/pm_integer.c → integer.c} +38 -22
  191. data/src/json.c +5733 -0
  192. data/src/line_offset_list.c +100 -0
  193. data/src/list.c +24 -0
  194. data/src/{util/pm_memchr.c → memchr.c} +8 -6
  195. data/src/node.c +3465 -7787
  196. data/src/options.c +161 -54
  197. data/src/parser.c +371 -0
  198. data/src/prettyprint.c +421 -354
  199. data/src/prism.c +8503 -7280
  200. data/src/regexp.c +960 -45
  201. data/src/serialize.c +2434 -2048
  202. data/src/source.c +524 -0
  203. data/src/static_literals.c +303 -360
  204. data/src/string_query.c +180 -0
  205. data/src/stringy.c +61 -0
  206. data/src/{util/pm_strncasecmp.c → strncasecmp.c} +5 -4
  207. data/src/strpbrk.c +447 -0
  208. data/src/{token_type.c → tokens.c} +39 -18
  209. data/src/xxh3.c +292 -0
  210. metadata +141 -63
  211. data/ext/prism/api_pack.c +0 -276
  212. data/include/prism/defines.h +0 -260
  213. data/include/prism/pack.h +0 -163
  214. data/include/prism/regexp.h +0 -43
  215. data/include/prism/static_literals.h +0 -121
  216. data/include/prism/util/pm_buffer.h +0 -236
  217. data/include/prism/util/pm_char.h +0 -204
  218. data/include/prism/util/pm_constant_pool.h +0 -218
  219. data/include/prism/util/pm_integer.h +0 -130
  220. data/include/prism/util/pm_memchr.h +0 -29
  221. data/include/prism/util/pm_newline_list.h +0 -113
  222. data/include/prism/util/pm_string.h +0 -200
  223. data/lib/prism/pack.rb +0 -230
  224. data/lib/prism/parse_result/comments.rb +0 -188
  225. data/rbi/prism/compiler.rbi +0 -12
  226. data/rbi/prism/dsl.rbi +0 -524
  227. data/rbi/prism/inspect_visitor.rbi +0 -12
  228. data/rbi/prism/node.rbi +0 -8750
  229. data/rbi/prism/node_ext.rbi +0 -107
  230. data/rbi/prism/parse_result.rbi +0 -404
  231. data/rbi/prism/reflection.rbi +0 -58
  232. data/rbi/prism/string_query.rbi +0 -12
  233. data/rbi/prism/visitor.rbi +0 -473
  234. data/rbi/prism.rbi +0 -63
  235. data/sig/prism/compiler.rbs +0 -9
  236. data/sig/prism/dispatcher.rbs +0 -19
  237. data/sig/prism/dot_visitor.rbs +0 -6
  238. data/sig/prism/dsl.rbs +0 -351
  239. data/sig/prism/inspect_visitor.rbs +0 -22
  240. data/sig/prism/lex_compat.rbs +0 -10
  241. data/sig/prism/node.rbs +0 -4042
  242. data/sig/prism/node_ext.rbs +0 -149
  243. data/sig/prism/pack.rbs +0 -43
  244. data/sig/prism/parse_result/comments.rbs +0 -38
  245. data/sig/prism/parse_result.rbs +0 -197
  246. data/sig/prism/pattern.rbs +0 -13
  247. data/sig/prism/reflection.rbs +0 -50
  248. data/sig/prism/relocation.rbs +0 -185
  249. data/sig/prism/serialize.rbs +0 -8
  250. data/sig/prism/string_query.rbs +0 -11
  251. data/sig/prism/visitor.rbs +0 -169
  252. data/sig/prism.rbs +0 -272
  253. data/src/pack.c +0 -509
  254. data/src/util/pm_constant_pool.c +0 -342
  255. data/src/util/pm_list.c +0 -49
  256. data/src/util/pm_newline_list.c +0 -125
  257. data/src/util/pm_string.c +0 -381
  258. data/src/util/pm_strpbrk.c +0 -206
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1f7205a73da36d10903c1495fd475e014a30c875568c932568cac22351eb3059
4
- data.tar.gz: d59eac5f6e8e8955ea9b259b63eec423a0af06c79a748edaa9f64f24463dc874
3
+ metadata.gz: 8e2f8907c10234cbde0a461b0674bc64139201a2d846bc2aa55e7ff416e3bac8
4
+ data.tar.gz: 396fd09bb9efc62dc076783701bb98fc9576b722ff9e2b4e89366eeef02f606e
5
5
  SHA512:
6
- metadata.gz: 59571b78dba19ec01a6f52094cc6d30e5648e633cd24f3abfbbc765145bed778f88fea75eada071d780ecb3dbe77a1bebab563d676289fed0ea6d21e50056290
7
- data.tar.gz: 3c5b40536f98839fefb0afcefd49d5e78a3a31f9a55bd33b78b80665b2ca3bd1c2262c1836f9c6f1d3ddc7ad2d9b7dd12f06b3dff2b4aea9475a7171340d95b4
6
+ metadata.gz: 844a68c1e7f2ebed766ec0a2b9f68ce2849e0205fd8339a04310a5b8725cdb820c1c3d04b077c11f112dff974cd74a1d29a9c7dd1c8f5d3a61efc3cfc1dbc50c
7
+ data.tar.gz: 90ac3995ffabefe8943d3159ac6f6b5a215b214c7d74a3c42c4fbf9abd3a3bfb12af32cee27747b28b1c5467ff0edf956d1346caa065fc81a29b7c3daa24699f
data/CHANGELOG.md CHANGED
@@ -6,6 +6,98 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [2.1.0] - 2026-10-28
10
+
11
+ ### Added
12
+
13
+ - Add `Version::CRuby4_0` to the Rust bindings. `Version::CRuby3_5` remains available but is deprecated in favor of the new name; both select Ruby 4.0 syntax.
14
+ - Let embedders set the allocation size for the first block allocated in an arena via `PM_ARENA_INITIAL_SIZE`.
15
+ - Let embedders set the minimum initial size for the constant pool via `PM_CONSTANT_POOL_MIN_CAPACITY`.
16
+
17
+ ## [2.0.0] - 2026-10-21
18
+
19
+ ### Highlights
20
+
21
+ - Significant performance and memory usage improvements.
22
+ - Precompiled platform-specific variants on rubygems.org.
23
+ - Many documentation improvements, including a new playground.
24
+ - Prism now ships accurate and complete RBS and RBI definitions.
25
+ - The license is now owned by the Ruby developers.
26
+ - Prism now ships an official C++ interface.
27
+
28
+ ### Added
29
+
30
+ - Support for parsing Ruby 4.1 syntax:
31
+ - Add support for `&nil` to declare that a method accepts no block.
32
+ - Thoroughly implement void value expression check.
33
+ - Allow trailing commas in method signatures like `def foo(a,); end`.
34
+ - Reject `END { break }` and `END { next }`.
35
+ - `a /b/` no longer warns about ambiguous `/`.
36
+ - `version: "nearest"` is now supported. It behaves similarly to `version: "current"` but will not raise when the version is unsupported, instead falling back to the nearest syntax version prism currently supports.
37
+ - `raise_error` was added as a valid option. Prism will raise a `SyntaxError` when given syntax-invalid code. When `true` is given, prism decides the formatting (`tty?`, `NO_COLOR`, etc.). If you need a specific style, `:plain`, `:style`, and `:color` are also supported values.
38
+ - Added `Prism::ParseResult#continuable` to check if the parsed source can become valid by appending more code.
39
+ - `Prism.find` takes a `Method`, `UnboundMethod`, `Proc`, or `Thread::Backtrace::Location` and returns the associated prism AST node.
40
+ - `ForwardingSuperNode` now has `keyword_loc`.
41
+ - Allow pattern matching with prism nodes to match on location slices, like `node in Prism::StringNode[content: "foo"]`.
42
+ - `Prism.parse_file` and similar now accept anything responding to `to_path` (like `Pathname`) directly.
43
+ - `attach_comments` was added as a valid option. Each comment is attached to the node that ends latest on the comment's line before it, otherwise to the innermost node that contains it (or to that node's statements, if only whitespace and comments come between the comment and the statements), otherwise to the top-level statements. The attached comments are available through `Node#comments` and through `pm_node_comments_size` and `pm_node_comment` in the C API, as well as `pm_parser_comment_attachment`, `pm_parser_comment_ranges_size`, and `pm_parser_comment_range`, which expose the attachments grouped by node. When the option is set, the comments on the parse result are sorted by start offset.
44
+ - `StringQuery#keyword?` was added to check if the string query matches a keyword.
45
+ - `Source#checksum` was added to retrieve the checksum of the source content.
46
+
47
+ ### Changed
48
+
49
+ Note that for this major version, almost every change listed below is breaking.
50
+
51
+ - Prism now defaults to `version: "nearest"` instead of `version: "latest"`, so code is parsed with the syntax of the running Ruby (or the closest version prism supports). Pass `version: "latest"` to keep the previous behavior.
52
+ - `MissingNode` has been replaced by `ErrorRecoveryNode`. `ErrorRecoveryNode` wraps unexpected nodes, so when walking syntax-invalid code you will now only need to handle this one specific node type.
53
+ - Some fields were renamed for consistency. Previous names are aliased and will continue to work in the Ruby layer.
54
+ - `InNode#{in,then}_loc` -> `InNode#{in,then}_keyword_loc`
55
+ - `MatchPredicateNode#operator_loc` -> `MatchPredicateNode#keyword_loc`
56
+ - `UnlessNode#keyword_loc` -> `UnlessNode#unless_keyword_loc`
57
+ - `UntilNode#{keyword,closing}_loc` -> `UntilNode#{until,end}_keyword_loc`
58
+ - `WhenNode#keyword_loc` -> `WhenNode#when_keyword_loc`
59
+ - `WhileNode#{keyword,closing}_loc` -> `WhileNode#{while,end}_keyword_loc`
60
+ - Optional locations like `StringNode#closing_loc` in `'foo` that were empty for syntax-invalid code will now be `nil` instead.
61
+ - `Prism.lex` now returns an array of tokens instead of an array of `[token, internal_parse_state]`. Previous access like `token[0]` or `token.first` to extract the actual token will continue to work. Additionally the following tokens were changed:
62
+ - `PARENTHESIS_LEFT` was split up into `PARENTHESIS_LEFT` and `PARENTHESIS_LEFT_GROUPING`.
63
+ - `BRACE_LEFT` was split up into `BRACE_LEFT`, `BRACE_LEFT_ARGUMENT`, and `BRACE_LEFT_HASH`.
64
+ - `BACKTICK` was split up into `BACKTICK` and `XSTRING_BACKTICK`.
65
+ - `NEWLINE` was split up into `NEWLINE` and `NEWLINE_TERMINATOR`.
66
+ - `WORDS_SEP` was split up into `WORDS_SEP` and `WORDS_SEP_IMPLICIT`.
67
+
68
+ #### Ruby
69
+
70
+ - Require arguments for `Prism::Source.for`. The second and third positional parameters are no longer optional.
71
+ - Comments now live on the source. `Prism::Source#comments` returns them and `Result#comments` delegates to it, so `ParseResult.new`, `LexResult.new`, and `ParseLexResult.new` no longer take a `comments` argument.
72
+ - Comment attachment now happens during parsing with the `attach_comments` option instead of afterward with `ParseResult#attach_comments!`, which was removed. `Node#comments` now returns the comments attached to the node by that option. `Location#comments`, `Location#leading_comments`, `Location#trailing_comments`, `Node#leading_comments`, `Node#trailing_comments`, and the comment fields of `Prism::Relocation` were removed.
73
+
74
+ #### C
75
+
76
+ - Most structs are now opaque (except for nodes), and many internal functions have been hidden.
77
+ - Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
78
+ - Most allocations are now performed through an arena.
79
+ - You may now build side tables of comment attachments through the `attach_comments` option.
80
+ - `newline_list` was renamed to `list_offset_list`.
81
+ - Visitor callbacks passed to `pm_visit_node` and `pm_visit_child_nodes` now return a `pm_visit_result_t` instead of a `bool`. Returning `PM_VISIT_DESCEND` or `PM_VISIT_SKIP` replaces returning `true` or `false`, and returning `PM_VISIT_STOP` ends the walk. Both functions now return `false` if the walk was stopped.
82
+ - `pm_string_t` is now a view of bytes owned by someone else. It no longer has a `type` field, `pm_string_owned_init` was removed, and `pm_string_constant_init` was renamed to `pm_string_init`, which takes the bytes as `const uint8_t *`.
83
+ - `pm_options_free` no longer frees the names of scope locals. The parser copies them when it is created, so they only need to remain valid until then.
84
+
85
+ #### Rust
86
+
87
+ - Locations are now represented using slices (pairs of 32-bit integers for start offset and length).
88
+ - Most allocations are now performed through an arena.
89
+ - `newline_list` was renamed to `list_offset_list`.
90
+ - `Diagnostic::message` now returns `&[u8]` instead of `&str`.
91
+ - `StringQuery` is now exposed.
92
+
93
+ #### JavaScript
94
+
95
+ - The parse method now only accepts `Uint8Array`, instead of forcing a transcoding to UTF-16.
96
+
97
+ ### Removed
98
+
99
+ - Removed `Prism::Pack`.
100
+
9
101
  ## [1.9.0] - 2026-01-27
10
102
 
11
103
  ### Added
@@ -19,6 +111,15 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
19
111
  - Fixed location of opening tokens when invalid syntax is parsed.
20
112
  - Fix RBI for parsing options.
21
113
 
114
+ ## [1.8.1] - 2026-03-16 (Ruby 4.0.2 maintenance release)
115
+
116
+ ### Changed
117
+
118
+ - Fix `not` binding power in endless methods.
119
+ - Correctly handle `and?` and similar on Ruby 4.0.
120
+ - Fix error message for block/lambda with `...` argument.
121
+ - Fix `in` handling.
122
+
22
123
  ## [1.8.0] - 2026-01-12
23
124
 
24
125
  ### Added
@@ -744,43 +845,46 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
744
845
 
745
846
  - 🎉 Initial release! 🎉
746
847
 
747
- [unreleased]: https://github.com/ruby/prism/compare/v1.9.0...HEAD
748
- [1.9.0]: https://github.com/ruby/prism/compare/v1.8.0...v1.9.0
749
- [1.8.0]: https://github.com/ruby/prism/compare/v1.7.0...v1.8.0
750
- [1.7.0]: https://github.com/ruby/prism/compare/v1.6.0...v1.7.0
751
- [1.6.0]: https://github.com/ruby/prism/compare/v1.5.2...v1.6.0
752
- [1.5.2]: https://github.com/ruby/prism/compare/v1.5.1...v1.5.2
753
- [1.5.1]: https://github.com/ruby/prism/compare/v1.5.0...v1.5.1
754
- [1.5.0]: https://github.com/ruby/prism/compare/v1.4.0...v1.5.0
755
- [1.4.0]: https://github.com/ruby/prism/compare/v1.3.0...v1.4.0
756
- [1.3.0]: https://github.com/ruby/prism/compare/v1.2.0...v1.3.0
757
- [1.2.0]: https://github.com/ruby/prism/compare/v1.1.0...v1.2.0
758
- [1.1.0]: https://github.com/ruby/prism/compare/v1.0.0...v1.1.0
759
- [1.0.0]: https://github.com/ruby/prism/compare/v0.30.0...v1.0.0
760
- [0.30.0]: https://github.com/ruby/prism/compare/v0.29.0...v0.30.0
761
- [0.29.0]: https://github.com/ruby/prism/compare/v0.28.0...v0.29.0
762
- [0.28.0]: https://github.com/ruby/prism/compare/v0.27.0...v0.28.0
763
- [0.27.0]: https://github.com/ruby/prism/compare/v0.26.0...v0.27.0
764
- [0.26.0]: https://github.com/ruby/prism/compare/v0.25.0...v0.26.0
765
- [0.25.0]: https://github.com/ruby/prism/compare/v0.24.0...v0.25.0
766
- [0.24.0]: https://github.com/ruby/prism/compare/v0.23.0...v0.24.0
767
- [0.23.0]: https://github.com/ruby/prism/compare/v0.22.0...v0.23.0
768
- [0.22.0]: https://github.com/ruby/prism/compare/v0.21.0...v0.22.0
769
- [0.21.0]: https://github.com/ruby/prism/compare/v0.20.0...v0.21.0
770
- [0.20.0]: https://github.com/ruby/prism/compare/v0.19.0...v0.20.0
771
- [0.19.0]: https://github.com/ruby/prism/compare/v0.18.0...v0.19.0
772
- [0.18.0]: https://github.com/ruby/prism/compare/v0.17.1...v0.18.0
773
- [0.17.1]: https://github.com/ruby/prism/compare/v0.17.0...v0.17.1
774
- [0.17.0]: https://github.com/ruby/prism/compare/v0.16.0...v0.17.0
775
- [0.16.0]: https://github.com/ruby/prism/compare/v0.15.1...v0.16.0
776
- [0.15.1]: https://github.com/ruby/prism/compare/v0.15.0...v0.15.1
777
- [0.15.0]: https://github.com/ruby/prism/compare/v0.14.0...v0.15.0
778
- [0.14.0]: https://github.com/ruby/prism/compare/v0.13.0...v0.14.0
779
- [0.13.0]: https://github.com/ruby/prism/compare/v0.12.0...v0.13.0
780
- [0.12.0]: https://github.com/ruby/prism/compare/v0.11.0...v0.12.0
781
- [0.11.0]: https://github.com/ruby/prism/compare/v0.10.0...v0.11.0
782
- [0.10.0]: https://github.com/ruby/prism/compare/v0.9.0...v0.10.0
783
- [0.9.0]: https://github.com/ruby/prism/compare/v0.8.0...v0.9.0
784
- [0.8.0]: https://github.com/ruby/prism/compare/v0.7.0...v0.8.0
785
- [0.7.0]: https://github.com/ruby/prism/compare/v0.6.0...v0.7.0
786
- [0.6.0]: https://github.com/ruby/prism/compare/d60531...v0.6.0
848
+ [unreleased]: https://github.com/ruby/prism3/compare/v2.1.0...HEAD
849
+ [2.1.0]: https://github.com/ruby/prism3/compare/v2.0.0...v2.1.0
850
+ [2.0.0]: https://github.com/ruby/prism3/compare/v1.9.0...v2.0.0
851
+ [1.9.0]: https://github.com/ruby/prism3/compare/v1.8.1...v1.9.0
852
+ [1.8.1]: https://github.com/ruby/prism3/compare/v1.8.0...v1.8.1
853
+ [1.8.0]: https://github.com/ruby/prism3/compare/v1.7.0...v1.8.0
854
+ [1.7.0]: https://github.com/ruby/prism3/compare/v1.6.0...v1.7.0
855
+ [1.6.0]: https://github.com/ruby/prism3/compare/v1.5.2...v1.6.0
856
+ [1.5.2]: https://github.com/ruby/prism3/compare/v1.5.1...v1.5.2
857
+ [1.5.1]: https://github.com/ruby/prism3/compare/v1.5.0...v1.5.1
858
+ [1.5.0]: https://github.com/ruby/prism3/compare/v1.4.0...v1.5.0
859
+ [1.4.0]: https://github.com/ruby/prism3/compare/v1.3.0...v1.4.0
860
+ [1.3.0]: https://github.com/ruby/prism3/compare/v1.2.0...v1.3.0
861
+ [1.2.0]: https://github.com/ruby/prism3/compare/v1.1.0...v1.2.0
862
+ [1.1.0]: https://github.com/ruby/prism3/compare/v1.0.0...v1.1.0
863
+ [1.0.0]: https://github.com/ruby/prism3/compare/v0.30.0...v1.0.0
864
+ [0.30.0]: https://github.com/ruby/prism3/compare/v0.29.0...v0.30.0
865
+ [0.29.0]: https://github.com/ruby/prism3/compare/v0.28.0...v0.29.0
866
+ [0.28.0]: https://github.com/ruby/prism3/compare/v0.27.0...v0.28.0
867
+ [0.27.0]: https://github.com/ruby/prism3/compare/v0.26.0...v0.27.0
868
+ [0.26.0]: https://github.com/ruby/prism3/compare/v0.25.0...v0.26.0
869
+ [0.25.0]: https://github.com/ruby/prism3/compare/v0.24.0...v0.25.0
870
+ [0.24.0]: https://github.com/ruby/prism3/compare/v0.23.0...v0.24.0
871
+ [0.23.0]: https://github.com/ruby/prism3/compare/v0.22.0...v0.23.0
872
+ [0.22.0]: https://github.com/ruby/prism3/compare/v0.21.0...v0.22.0
873
+ [0.21.0]: https://github.com/ruby/prism3/compare/v0.20.0...v0.21.0
874
+ [0.20.0]: https://github.com/ruby/prism3/compare/v0.19.0...v0.20.0
875
+ [0.19.0]: https://github.com/ruby/prism3/compare/v0.18.0...v0.19.0
876
+ [0.18.0]: https://github.com/ruby/prism3/compare/v0.17.1...v0.18.0
877
+ [0.17.1]: https://github.com/ruby/prism3/compare/v0.17.0...v0.17.1
878
+ [0.17.0]: https://github.com/ruby/prism3/compare/v0.16.0...v0.17.0
879
+ [0.16.0]: https://github.com/ruby/prism3/compare/v0.15.1...v0.16.0
880
+ [0.15.1]: https://github.com/ruby/prism3/compare/v0.15.0...v0.15.1
881
+ [0.15.0]: https://github.com/ruby/prism3/compare/v0.14.0...v0.15.0
882
+ [0.14.0]: https://github.com/ruby/prism3/compare/v0.13.0...v0.14.0
883
+ [0.13.0]: https://github.com/ruby/prism3/compare/v0.12.0...v0.13.0
884
+ [0.12.0]: https://github.com/ruby/prism3/compare/v0.11.0...v0.12.0
885
+ [0.11.0]: https://github.com/ruby/prism3/compare/v0.10.0...v0.11.0
886
+ [0.10.0]: https://github.com/ruby/prism3/compare/v0.9.0...v0.10.0
887
+ [0.9.0]: https://github.com/ruby/prism3/compare/v0.8.0...v0.9.0
888
+ [0.8.0]: https://github.com/ruby/prism3/compare/v0.7.0...v0.8.0
889
+ [0.7.0]: https://github.com/ruby/prism3/compare/v0.6.0...v0.7.0
890
+ [0.6.0]: https://github.com/ruby/prism3/compare/d60531...v0.6.0
data/CODE_OF_CONDUCT.md CHANGED
@@ -55,7 +55,7 @@ further defined and clarified by project maintainers.
55
55
  ## Enforcement
56
56
 
57
57
  Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
- reported by contacting the project team at opensource@shopify.com. All
58
+ reported by contacting the project team at ruby-core@ruby-lang.org. All
59
59
  complaints will be reviewed and investigated and will result in a response that
60
60
  is deemed necessary and appropriate to the circumstances. The project team is
61
61
  obligated to maintain confidentiality with regard to the reporter of an incident.
data/LICENSE.md CHANGED
@@ -1,4 +1,4 @@
1
- Copyright 2022-present, Shopify Inc.
1
+ Copyright 2022-present, Ruby developers
2
2
 
3
3
  Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
4
4
 
data/Makefile CHANGED
@@ -6,22 +6,36 @@ Q1 = $(V:1=)
6
6
  Q = $(Q1:0=@)
7
7
  ECHO1 = $(V:1=@ :)
8
8
  ECHO = $(ECHO1:0=@ echo)
9
- FUZZ_OUTPUT_DIR = $(CURDIR)/fuzz/output
9
+ FUZZ_OUTPUT_DIR = $(CURDIR)/test/fuzz/output
10
10
 
11
11
  SOEXT ?= $(shell ruby -e 'puts RbConfig::CONFIG["SOEXT"]')
12
12
 
13
13
  CPPFLAGS := -Iinclude $(CPPFLAGS)
14
14
  CFLAGS := -g -O2 -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden -Wimplicit-fallthrough $(CFLAGS)
15
- JAVA_WASM_CFLAGS := -g -Oz -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden -Wimplicit-fallthrough $(JAVA_WASM_CFLAGS)
15
+ JAVA_WASM_CFLAGS := -g0 -O2 -Oz -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fno-sanitize=all -fno-stack-protector -fPIC -fvisibility=hidden -Wimplicit-fallthrough $(JAVA_WASM_CFLAGS)
16
16
  CC ?= cc
17
17
  AR ?= ar
18
18
  ARFLAGS ?= -r$(V0:1=v)
19
19
  WASI_SDK_PATH := /opt/wasi-sdk
20
20
 
21
+ # Exporting a symbol makes it a GC root, so the JavaScript module exports only
22
+ # the symbols javascript/src/parsePrism.js calls. That lets --gc-sections drop
23
+ # the rest of the C API along with the wasi-libc code reachable from it.
24
+ WASM_EXPORTS := -Wl,--export=malloc,--export=free,--export=pm_buffer_new,--export=pm_buffer_value,--export=pm_buffer_length,--export=pm_buffer_free,--export=pm_serialize_parse
25
+
26
+ # Bind the library's own references to its own definitions at link time, so
27
+ # that the functions exported for FFI cannot be interposed by another copy of
28
+ # libprism loaded in the same process (for example the copy built into the
29
+ # interpreter next to the one built by a prism gem, when either copy was loaded
30
+ # with RTLD_GLOBAL). Mach-O and PE do this by default; only ELF needs the flag.
31
+ ifeq ($(SOEXT),so)
32
+ SHARED_LDFLAGS := -Wl,-Bsymbolic
33
+ endif
34
+
21
35
  MAKEDIRS ?= mkdir -p
22
36
  RMALL ?= rm -f -r
23
37
 
24
- HEADERS := $(wildcard include/*.h include/*/*.h include/*/*/*.h')
38
+ HEADERS := $(wildcard include/*.h include/*/*.h include/*/*/*.h)
25
39
  SOURCES := $(wildcard src/*.c src/*/*.c)
26
40
  SHARED_OBJECTS := $(subst src/,build/shared/,$(SOURCES:.c=.o))
27
41
  STATIC_OBJECTS := $(subst src/,build/static/,$(SOURCES:.c=.o))
@@ -31,11 +45,12 @@ all: shared static
31
45
  shared: build/libprism.$(SOEXT)
32
46
  static: build/libprism.a
33
47
  wasm: javascript/src/prism.wasm
34
- java-wasm: java-wasm/src/test/resources/prism.wasm
48
+ java-wasm: java/wasm/src/main/wasm/prism.wasm
49
+ playground: doc/playground
35
50
 
36
51
  build/libprism.$(SOEXT): $(SHARED_OBJECTS)
37
52
  $(ECHO) "linking $@ with $(CC)"
38
- $(Q) $(CC) $(DEBUG_FLAGS) $(CFLAGS) -shared -o $@ $(SHARED_OBJECTS)
53
+ $(Q) $(CC) $(DEBUG_FLAGS) $(CFLAGS) -shared $(SHARED_LDFLAGS) -o $@ $(SHARED_OBJECTS)
39
54
 
40
55
  build/libprism.a: $(STATIC_OBJECTS)
41
56
  $(ECHO) "building $@ with $(AR)"
@@ -45,15 +60,29 @@ javascript/src/prism.wasm: Makefile $(SOURCES) $(HEADERS)
45
60
  $(ECHO) "building $@"
46
61
  $(Q) $(WASI_SDK_PATH)/bin/clang --sysroot=$(WASI_SDK_PATH)/share/wasi-sysroot/ \
47
62
  $(DEBUG_FLAGS) \
48
- -DPRISM_EXPORT_SYMBOLS -DPRISM_EXCLUDE_PRETTYPRINT -DPRISM_EXCLUDE_JSON -DPRISM_EXCLUDE_PACK \
63
+ -DPRISM_EXPORT_SYMBOLS -DPRISM_EXCLUDE_PRETTYPRINT -DPRISM_EXCLUDE_JSON \
49
64
  -D_WASI_EMULATED_MMAN -lwasi-emulated-mman $(CPPFLAGS) $(CFLAGS) \
50
- -Wl,--export-all -Wl,--gc-sections -Wl,--strip-all -Wl,--lto-O3 -Wl,--no-entry -mexec-model=reactor \
65
+ $(WASM_EXPORTS) -Wl,--gc-sections -Wl,--strip-all -Wl,--lto-O3 -Wl,--no-entry -mexec-model=reactor \
51
66
  -Oz -g0 -flto -fdata-sections -ffunction-sections \
52
67
  -o $@ $(SOURCES)
53
68
 
54
- java-wasm/src/test/resources/prism.wasm: Makefile $(SOURCES) $(HEADERS)
69
+ # The playground parses with the JavaScript package built from this checkout, so
70
+ # the page is served the parser the rest of the site was generated from. The
71
+ # JavaScript half of the package is generated by `rake templates`.
72
+ doc/playground: javascript/src/prism.wasm
73
+ $(ECHO) "building $@"
74
+ $(Q) $(MAKEDIRS) $@
75
+ $(Q) cp javascript/src/*.js javascript/src/prism.wasm javascript/package.json $@
76
+
77
+ java/wasm/src/main/wasm/prism.wasm: Makefile $(SOURCES) $(HEADERS)
55
78
  $(ECHO) "building $@"
56
- $(Q) $(WASI_SDK_PATH)/bin/clang $(DEBUG_FLAGS) -DPRISM_EXCLUDE_PRETTYPRINT -DPRISM_EXPORT_SYMBOLS -D_WASI_EMULATED_MMAN -lwasi-emulated-mman $(CPPFLAGS) $(JAVA_WASM_CFLAGS) -Wl,--export-all -Wl,--no-entry -mexec-model=reactor -lc++ -lc++abi -o $@ $(SOURCES)
79
+ $(Q) $(MAKEDIRS) $(@D)
80
+ $(Q) $(WASI_SDK_PATH)/bin/clang \
81
+ $(DEBUG_FLAGS) \
82
+ -DPRISM_EXCLUDE_PRETTYPRINT -DPRISM_EXPORT_SYMBOLS -D_WASI_EMULATED_MMAN \
83
+ -lwasi-emulated-mman $(CPPFLAGS) $(JAVA_WASM_CFLAGS) \
84
+ -Wl,--export-all -Wl,--no-entry -mexec-model=reactor -lc++ -lc++abi \
85
+ -o $@ $(SOURCES)
57
86
 
58
87
  build/shared/%.o: src/%.c Makefile $(HEADERS)
59
88
  $(ECHO) "compiling $@"
@@ -65,24 +94,24 @@ build/static/%.o: src/%.c Makefile $(HEADERS)
65
94
  $(Q) $(MAKEDIRS) $(@D)
66
95
  $(Q) $(CC) $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
67
96
 
68
- build/fuzz.%: $(SOURCES) fuzz/%.c fuzz/fuzz.c
97
+ build/fuzz.%: $(SOURCES) test/fuzz/%.c test/fuzz/fuzz.c
69
98
  $(ECHO) "building $* fuzzer"
70
99
  $(Q) $(MAKEDIRS) $(@D)
71
100
  $(ECHO) "building main fuzz binary"
72
- $(Q) afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize-ignorelist=fuzz/asan.ignore -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@ $^
101
+ $(Q) afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@ $^
73
102
  $(ECHO) "building cmplog binary"
74
- $(Q) AFL_LLVM_CMPLOG=1 afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize-ignorelist=fuzz/asan.ignore -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@.cmplog $^
103
+ $(Q) AFL_LLVM_CMPLOG=1 afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@.cmplog $^
75
104
 
76
- build/fuzz.heisenbug.%: $(SOURCES) fuzz/%.c fuzz/heisenbug.c
77
- $(Q) afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize-ignorelist=fuzz/asan.ignore -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@ $^
105
+ build/fuzz.heisenbug.%: $(SOURCES) test/fuzz/%.c test/fuzz/heisenbug.c
106
+ $(Q) afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@ $^
78
107
 
79
108
  fuzz-debug:
80
109
  $(ECHO) "entering debug shell"
81
- $(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
110
+ $(Q) docker run -it --rm -e HISTFILE=/prism/test/fuzz/output/.bash_history -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
82
111
 
83
- fuzz-docker-build: fuzz/docker/Dockerfile
112
+ fuzz-docker-build: test/fuzz/docker/Dockerfile
84
113
  $(ECHO) "building docker image"
85
- $(Q) docker build -t prism/fuzz fuzz/docker/
114
+ $(Q) docker build -t prism/fuzz test/fuzz/docker/
86
115
 
87
116
  fuzz-run-%: FORCE fuzz-docker-build
88
117
  $(ECHO) "generating templates"
@@ -91,16 +120,16 @@ fuzz-run-%: FORCE fuzz-docker-build
91
120
  $(Q) docker run --rm -v $(CURDIR):/prism prism/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
92
121
  $(ECHO) "starting AFL++ run"
93
122
  $(Q) $(MAKEDIRS) $(FUZZ_OUTPUT_DIR)/$*
94
- $(Q) docker run -it --rm -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
123
+ $(Q) docker run -it --rm -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./test/fuzz/$*.sh /fuzz_output/$*"
95
124
  FORCE:
96
125
 
97
126
  fuzz-clean:
98
- $(Q) $(RMALL) fuzz/output
127
+ $(Q) $(RMALL) test/fuzz/output
99
128
 
100
129
  clean:
101
130
  $(Q) $(RMALL) build
102
131
 
103
- .PHONY: clean fuzz-clean
132
+ .PHONY: clean doc/playground fuzz-clean
104
133
 
105
134
  all-no-debug: DEBUG_FLAGS := -DNDEBUG=1
106
135
  all-no-debug: OPTFLAGS := -O3
data/README.md CHANGED
@@ -1,10 +1,35 @@
1
- <h1 align="center">Prism Ruby parser</h1>
2
- <div align="center">
3
- <img alt="Prism Ruby parser" height="256px" src="https://github.com/ruby/prism/blob/main/doc/images/prism.png?raw=true">
4
- </div>
1
+ <h1 align="center">Prism</h1>
5
2
 
6
3
  This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies.
7
4
 
5
+ ## Getting started
6
+
7
+ To compile the parser, you will need:
8
+
9
+ * C99 compiler
10
+ * GNU make
11
+ * Ruby 2.7.0 or later
12
+
13
+ Once you have these dependencies, run:
14
+
15
+ ```
16
+ bundle install
17
+ ```
18
+
19
+ to fetch the Ruby dependencies. Finally, run:
20
+
21
+ ```
22
+ bundle exec rake compile
23
+ ```
24
+
25
+ to compile the parser. It will be built in the `build` directory. To test that everything is working, run:
26
+
27
+ ```
28
+ bin/parse -e "1 + 2"
29
+ ```
30
+
31
+ to see the syntax tree for the expression `1 + 2`.
32
+
8
33
  ## Overview
9
34
 
10
35
  The repository contains the infrastructure for both a shared library (libprism) and a native CRuby extension. The shared library has no bindings to CRuby itself, and so can be used by other projects. The native CRuby extension links against `ruby.h`, and so is suitable in the context of CRuby.
@@ -12,9 +37,9 @@ The repository contains the infrastructure for both a shared library (libprism)
12
37
  ```
13
38
  .
14
39
  ├── Makefile configuration to compile the shared library and native tests
15
- ├── Rakefile configuration to compile the native extension and run the Ruby tests
40
+ ├── Rakefile configuration to compile the native extension and Ruby tests
16
41
  ├── bin
17
- │   ├── lex runs the lexer on a file or string, prints the tokens, and compares to ripper
42
+ │   ├── lex runs the lexer on a file or string and prints the tokens
18
43
  │   ├── parse runs the parse on a file or string and prints the AST
19
44
  │   └── prism a CLI for development and debugging
20
45
  ├── config.yml specification for tokens and nodes in the tree
@@ -24,13 +49,14 @@ The repository contains the infrastructure for both a shared library (libprism)
24
49
  │   └── prism
25
50
  │   ├── extconf.rb configuration to generate the Makefile for the native extension
26
51
  │   └── extension.c the native extension that interacts with libprism
27
- ├── fuzz files related to fuzz testing
28
52
  ├── gemfiles gemfiles used by different Ruby versions in CI
29
53
  ├── include
30
54
  │   ├── prism header files for the shared library
31
55
  │   └── prism.h main header file for the shared library
32
- ├── java Java bindings for the shared library
33
- ├── java-wasm Java WASM bindings for the shared library
56
+ ├── java
57
+ | ├── api Java API for Prism
58
+ | ├── native Java native bindings for the shared library
59
+ | └── wasm Java WASM bindings for the shared library
34
60
  ├── javascript JavaScript WASM bindings for the shared library
35
61
  ├── lib
36
62
  │   ├── prism Ruby library files
@@ -44,100 +70,14 @@ The repository contains the infrastructure for both a shared library (libprism)
44
70
  │ └── prism Sample code that uses the Ruby API for documentation purposes
45
71
  ├── sig RBS type signatures for the Ruby library
46
72
  ├── src
47
- │   ├── util various utility files
48
73
  │   └── prism.c main entrypoint for the shared library
49
74
  ├── templates contains ERB templates generated by templates/template.rb
50
75
  │   └── template.rb generates code from the nodes and tokens configured by config.yml
51
76
  └── test
77
+ ├── cpp tests for the C++ bindings
78
+ ├── embedded smoke test for bare-metal targets
79
+ ├── fuzz files related to fuzz testing
52
80
  └── prism
53
81
  ├── fixtures Ruby code used for testing
54
82
  └── snapshots snapshots of generated syntax trees corresponding to fixtures
55
83
  ```
56
-
57
- ## Getting started
58
-
59
- To compile the shared library, you will need:
60
-
61
- * C99 compiler
62
- * GNU make
63
- * Ruby 2.7.0 or later
64
-
65
- Once you have these dependencies, run:
66
-
67
- ```
68
- bundle install
69
- ```
70
-
71
- to fetch the Ruby dependencies. Finally, run:
72
-
73
- ```
74
- bundle exec rake compile
75
- ```
76
-
77
- to compile the shared library. It will be built in the `build` directory. To test that everything is working, run:
78
-
79
- ```
80
- bin/parse -e "1 + 2"
81
- ```
82
-
83
- to see the syntax tree for the expression `1 + 2`.
84
-
85
- ## Contributing
86
-
87
- See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additionally have documentation about the overall design of the project as well as various subtopics.
88
-
89
- * [Build system](docs/build_system.md)
90
- * [Configuration](docs/configuration.md)
91
- * [CRuby compilation](docs/cruby_compilation.md)
92
- * [Design](docs/design.md)
93
- * [Encoding](docs/encoding.md)
94
- * [Fuzzing](docs/fuzzing.md)
95
- * [Heredocs](docs/heredocs.md)
96
- * [JavaScript](docs/javascript.md)
97
- * [Local variable depth](docs/local_variable_depth.md)
98
- * [Mapping](docs/mapping.md)
99
- * [Parser translation](docs/parser_translation.md)
100
- * [Parsing rules](docs/parsing_rules.md)
101
- * [Releasing](docs/releasing.md)
102
- * [Ripper translation](docs/ripper_translation.md)
103
- * [Ruby API](docs/ruby_api.md)
104
- * [RubyParser translation](docs/ruby_parser_translation.md)
105
- * [Serialization](docs/serialization.md)
106
- * [Testing](docs/testing.md)
107
-
108
- ## Examples
109
-
110
- Prism has been integrated into the majority of Ruby runtimes, many libraries, and some applications. Below is a list of some of the projects that use Prism:
111
-
112
- ### Runtimes
113
-
114
- * [CRuby](https://github.com/ruby/ruby/pull/7964) (via C)
115
- * [Garnet](https://github.com/camertron/garnet-js) (via WASM)
116
- * [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java)
117
- * [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby)
118
- * [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM)
119
- * [TruffleRuby](https://github.com/truffleruby/truffleruby/issues/3117) (via Java)
120
-
121
- ### Libraries
122
-
123
- * [dispersion](https://github.com/joeldrapper/dispersion)
124
- * [minifyrb](https://github.com/koic/minifyrb)
125
- * [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator)
126
- * [rbi](https://github.com/Shopify/rbi)
127
- * [rails](https://github.com/rails/rails)
128
- * [parsing renders](https://github.com/rails/rails/pull/49438)
129
- * [parsing rdoc](https://github.com/rails/rails/pull/50870)
130
- * [parsing tests](https://github.com/rails/rails/pull/51006)
131
- * [repl_type_completor](https://github.com/ruby/repl_type_completor)
132
- * [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-parser-engine) (via parser translator)
133
- * [ruby-lsp](https://github.com/Shopify/ruby-lsp)
134
- * [smart_todo](https://github.com/Shopify/smart_todo/pull/69)
135
- * [sorbet-eraser](https://github.com/kddnewton/sorbet-eraser/pull/25)
136
- * [synvert](https://github.com/xinminlabs/synvert-core-ruby)
137
- * [typeprof](https://github.com/ruby/typeprof)
138
- * [unparser](https://github.com/mbj/unparser) (via parser translator)
139
-
140
- ### Applications
141
-
142
- * [gem.sh](https://github.com/marcoroth/gem.sh/pull/96)
143
- * [Sorbet](https://github.com/sorbet/sorbet)