prism 0.24.0 → 0.29.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (126) hide show
  1. checksums.yaml +4 -4
  2. data/BSDmakefile +58 -0
  3. data/CHANGELOG.md +132 -1
  4. data/Makefile +25 -18
  5. data/README.md +45 -6
  6. data/config.yml +828 -25
  7. data/docs/build_system.md +31 -0
  8. data/docs/configuration.md +4 -0
  9. data/docs/cruby_compilation.md +1 -1
  10. data/docs/parser_translation.md +14 -9
  11. data/docs/releasing.md +7 -9
  12. data/docs/ripper_translation.md +50 -0
  13. data/docs/ruby_api.md +1 -0
  14. data/docs/serialization.md +26 -5
  15. data/ext/prism/api_node.c +1037 -936
  16. data/ext/prism/api_pack.c +9 -0
  17. data/ext/prism/extconf.rb +62 -18
  18. data/ext/prism/extension.c +351 -71
  19. data/ext/prism/extension.h +5 -4
  20. data/include/prism/ast.h +539 -101
  21. data/include/prism/defines.h +106 -2
  22. data/include/prism/diagnostic.h +168 -74
  23. data/include/prism/encoding.h +22 -4
  24. data/include/prism/node.h +93 -0
  25. data/include/prism/options.h +84 -9
  26. data/include/prism/pack.h +11 -0
  27. data/include/prism/parser.h +213 -54
  28. data/include/prism/prettyprint.h +8 -0
  29. data/include/prism/static_literals.h +120 -0
  30. data/include/prism/util/pm_buffer.h +65 -2
  31. data/include/prism/util/pm_constant_pool.h +18 -1
  32. data/include/prism/util/pm_integer.h +119 -0
  33. data/include/prism/util/pm_list.h +1 -1
  34. data/include/prism/util/pm_newline_list.h +8 -0
  35. data/include/prism/util/pm_string.h +26 -2
  36. data/include/prism/version.h +2 -2
  37. data/include/prism.h +59 -1
  38. data/lib/prism/compiler.rb +8 -1
  39. data/lib/prism/debug.rb +46 -3
  40. data/lib/prism/desugar_compiler.rb +5 -3
  41. data/lib/prism/dispatcher.rb +29 -0
  42. data/lib/prism/dot_visitor.rb +141 -54
  43. data/lib/prism/dsl.rb +48 -36
  44. data/lib/prism/ffi.rb +82 -17
  45. data/lib/prism/inspect_visitor.rb +2156 -0
  46. data/lib/prism/lex_compat.rb +34 -15
  47. data/lib/prism/mutation_compiler.rb +13 -2
  48. data/lib/prism/node.rb +4453 -4459
  49. data/lib/prism/node_ext.rb +249 -30
  50. data/lib/prism/pack.rb +4 -0
  51. data/lib/prism/parse_result/comments.rb +35 -18
  52. data/lib/prism/parse_result/newlines.rb +2 -2
  53. data/lib/prism/parse_result.rb +218 -43
  54. data/lib/prism/pattern.rb +28 -10
  55. data/lib/prism/polyfill/byteindex.rb +13 -0
  56. data/lib/prism/polyfill/unpack1.rb +14 -0
  57. data/lib/prism/reflection.rb +411 -0
  58. data/lib/prism/serialize.rb +480 -112
  59. data/lib/prism/translation/parser/compiler.rb +376 -88
  60. data/lib/prism/translation/parser/lexer.rb +103 -22
  61. data/lib/prism/translation/parser/rubocop.rb +41 -13
  62. data/lib/prism/translation/parser.rb +123 -11
  63. data/lib/prism/translation/parser33.rb +1 -1
  64. data/lib/prism/translation/parser34.rb +1 -1
  65. data/lib/prism/translation/ripper/sexp.rb +125 -0
  66. data/lib/prism/translation/ripper/shim.rb +5 -0
  67. data/lib/prism/translation/ripper.rb +3216 -462
  68. data/lib/prism/translation/ruby_parser.rb +111 -56
  69. data/lib/prism/translation.rb +3 -1
  70. data/lib/prism/visitor.rb +10 -0
  71. data/lib/prism.rb +12 -20
  72. data/prism.gemspec +46 -14
  73. data/rbi/prism/compiler.rbi +12 -0
  74. data/rbi/prism/inspect_visitor.rbi +12 -0
  75. data/rbi/prism/node.rbi +8712 -0
  76. data/rbi/prism/node_ext.rbi +107 -0
  77. data/rbi/prism/parse_result.rbi +358 -0
  78. data/rbi/prism/reflection.rbi +58 -0
  79. data/rbi/prism/translation/parser.rbi +11 -0
  80. data/rbi/prism/translation/parser33.rbi +6 -0
  81. data/rbi/prism/translation/parser34.rbi +6 -0
  82. data/rbi/prism/translation/ripper.rbi +15 -0
  83. data/rbi/prism/visitor.rbi +470 -0
  84. data/rbi/prism.rbi +38 -7748
  85. data/sig/prism/compiler.rbs +9 -0
  86. data/sig/prism/dispatcher.rbs +16 -0
  87. data/sig/prism/dot_visitor.rbs +6 -0
  88. data/sig/prism/dsl.rbs +462 -0
  89. data/sig/prism/inspect_visitor.rbs +22 -0
  90. data/sig/prism/lex_compat.rbs +10 -0
  91. data/sig/prism/mutation_compiler.rbs +158 -0
  92. data/sig/prism/node.rbs +3558 -0
  93. data/sig/prism/node_ext.rbs +82 -0
  94. data/sig/prism/pack.rbs +43 -0
  95. data/sig/prism/parse_result.rbs +160 -0
  96. data/sig/prism/pattern.rbs +13 -0
  97. data/sig/prism/reflection.rbs +50 -0
  98. data/sig/prism/serialize.rbs +6 -0
  99. data/sig/prism/visitor.rbs +168 -0
  100. data/sig/prism.rbs +188 -4767
  101. data/src/diagnostic.c +636 -230
  102. data/src/encoding.c +211 -108
  103. data/src/node.c +7555 -451
  104. data/src/options.c +66 -31
  105. data/src/pack.c +33 -17
  106. data/src/prettyprint.c +1383 -1431
  107. data/src/prism.c +4734 -1310
  108. data/src/regexp.c +17 -2
  109. data/src/serialize.c +68 -46
  110. data/src/static_literals.c +638 -0
  111. data/src/token_type.c +10 -9
  112. data/src/util/pm_buffer.c +147 -20
  113. data/src/util/pm_char.c +4 -4
  114. data/src/util/pm_constant_pool.c +35 -11
  115. data/src/util/pm_integer.c +642 -0
  116. data/src/util/pm_list.c +1 -1
  117. data/src/util/pm_newline_list.c +14 -5
  118. data/src/util/pm_string.c +134 -5
  119. data/src/util/pm_string_list.c +2 -2
  120. metadata +41 -9
  121. data/docs/ripper.md +0 -36
  122. data/include/prism/util/pm_state_stack.h +0 -42
  123. data/lib/prism/node_inspector.rb +0 -68
  124. data/rbi/prism_static.rbi +0 -207
  125. data/sig/prism_static.rbs +0 -201
  126. data/src/util/pm_state_stack.c +0 -25
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 827194fd8b477f9f021677ca386e35fd8300daa86a09fc11fe66a85ee54da97d
4
- data.tar.gz: 2a44a3cf5e31260a949965aaf2aa5fba0cd8b1ac910ae694d22fd8e42f4d6b9b
3
+ metadata.gz: 4ac8167684c0f975fdba449e77206d97756f6c39f22ce871f79a79a61260503f
4
+ data.tar.gz: 6a2dd5c0a47df7c8c575ad2e5b344c789d548e93cfe82e5fa29974d46a52bb3c
5
5
  SHA512:
6
- metadata.gz: 41dab6cd85f9f57bf2e2256e7b7fd7ac44c46883ad6d79c0056bff3b34500b2ecb28a1febeb1deed5a49b862c4f4382856ad2195639f86e770c006d1b6a41f8e
7
- data.tar.gz: 0474d64c9bbd3385f65b00bf83e6b2d3c9543785c85b42d50bce59327e6af7a8fc13d54e5b485b6c930997dabe0fd81c43835243fc96fb28beb3523395ddde71
6
+ metadata.gz: dfa7fe63285b85cb45aa0be681916d6891ee5e5adf2162ab5c18423417f7938afc99f00eca8fd6187fbd0b7168af088d932fbda3c1361d1c73953f39329bed70
7
+ data.tar.gz: 79bc51db60600d74a6bfbeae5eba3a8ec505e9809d0f7ff0b05563f1f7a8cd284cc293233397323a443217a4a5271f90b4c40516168ae3a8be0ddc2d7376cef9
data/BSDmakefile ADDED
@@ -0,0 +1,58 @@
1
+ # GNU makefile proxy script for BSD make
2
+ #
3
+ # Written and maintained by Mahmoud Al-Qudsi <mqudsi@neosmart.net>
4
+ # Copyright NeoSmart Technologies <https://neosmart.net/> 2014-2019
5
+ # Obtain updates from <https://github.com/neosmart/gmake-proxy>
6
+ #
7
+ # Redistribution and use in source and binary forms, with or without
8
+ # modification, are permitted provided that the following conditions are met:
9
+ #
10
+ # 1. Redistributions of source code must retain the above copyright notice, this
11
+ # list of conditions and the following disclaimer.
12
+ #
13
+ # 2. Redistributions in binary form must reproduce the above copyright notice,
14
+ # this list of conditions and the following disclaimer in the documentation
15
+ # and/or other materials provided with the distribution.
16
+ #
17
+ # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
18
+ # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
19
+ # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
20
+ # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
21
+ # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
22
+ # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
23
+ # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
24
+ # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
25
+ # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
26
+ # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
27
+
28
+ JARG =
29
+ GMAKE = "gmake"
30
+ # When gmake is called from another make instance, -w is automatically added
31
+ # which causes extraneous messages about directory changes to be emitted.
32
+ # Running with --no-print-directory silences these messages.
33
+ GARGS = "--no-print-directory"
34
+
35
+ .if "$(.MAKE.JOBS)" != ""
36
+ JARG = -j$(.MAKE.JOBS)
37
+ .endif
38
+
39
+ # bmake prefers out-of-source builds and tries to cd into ./obj (among others)
40
+ # where possible. GNU Make doesn't, so override that value.
41
+ .OBJDIR: ./
42
+
43
+ # The GNU convention is to use the lowercased `prefix` variable/macro to
44
+ # specify the installation directory. Humor them.
45
+ GPREFIX = ""
46
+ .if defined(PREFIX) && ! defined(prefix)
47
+ GPREFIX = 'prefix = "$(PREFIX)"'
48
+ .endif
49
+
50
+ .BEGIN: .SILENT
51
+ which $(GMAKE) || printf "Error: GNU Make is required!\n\n" 1>&2 && false
52
+
53
+ .PHONY: FRC
54
+ $(.TARGETS): FRC
55
+ $(GMAKE) $(GPREFIX) $(GARGS) $(.TARGETS:S,.DONE,,) $(JARG)
56
+
57
+ .DONE .DEFAULT: .SILENT
58
+ $(GMAKE) $(GPREFIX) $(GARGS) $(.TARGETS:S,.DONE,,) $(JARG)
data/CHANGELOG.md CHANGED
@@ -6,6 +6,132 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.29.0] - 2024-05-10
10
+
11
+ ### Added
12
+
13
+ - Added `Prism::CallNode#full_message_loc`, which gives the location including the `=` if there is one.
14
+ - A warning for when `# shareable_constant_value` is not used on its own line.
15
+ - An error for invalid implicit local variable writes.
16
+ - Implicit hash patterns in array patterns are disallowed.
17
+ - We now validate that Unicode escape sequences are not surrogates.
18
+
19
+ ### Changed
20
+
21
+ - All fields named `operator` have been renamed to `binary_operator` for `*OperatorWriteNode` nodes. This is to make it easier to provide C++ support. In the Ruby API, the old fields are aliased to the new fields with a deprecation warning.
22
+ - Many updated error messages to more closely match CRuby.
23
+ - We ensure keyword parameters do not end in `!` or `?`.
24
+ - Fixed some escaping in string literals with control sequences and hex escapes.
25
+ - Fix a bug with RBS types when used outside the `ruby/prism` codebase.
26
+
27
+ ## [0.28.0] - 2024-05-03
28
+
29
+ ### Added
30
+
31
+ - Nested hashes will now warn for duplicated keys, as in: `{ foo: 1, **{ foo: 2 } }`.
32
+ - `Prism::ReturnNode` now has a flag on it to indicate if it is redundant.
33
+ - `Prism::Location#slice_lines` and `Prism::Node#slice_lines` are now provided to slice the source code of a node including the content before the node on the same line that it starts on and the content after the node on the same line that it ends on.
34
+ - Symbols with invalid byte sequences now give errors.
35
+ - You can now pass `"3.3.1"` to the `version:` parameter on all `Prism.*` APIs.
36
+ - `Prism::Source#lines`, `Prism::Location#source_lines`, `Prism::Node#source_lines`, and `Prism::Node#script_lines` are now provided, which will all return the source code of the source as an array of strings.
37
+ - `Prism::ASCIISource` is now provided, which is a subclass of `Prism::Source` but specialized to increase performance when the source is entirely ASCII.
38
+ - Prism now provides errors when parsing Ruby 3.4+ syntax for index expressions with keywords or blocks.
39
+ - Prism now provides an error when `**nil` is used after other keyword parameters.
40
+ - Prism now provides errors when safe navigation is used in call target expressions, e.g., `foo&.bar, = 1`.
41
+ - `Prism::Node#tunnel` is now provided, which returns an array of nodes starting at the current node that contain a given line and column.
42
+
43
+ ### Changed
44
+
45
+ - All translation layers now assume an eval context, which means they will not return errors for invalid jumps like `yield`.
46
+ - `Prism::Node#inspect` now uses a queue instead of recursion to avoid stack overflows.
47
+ - Prism now more closely mirrors CRuby interpolation semantics, which means you could potentially have a static literal string that directly interpolates another static literal string.
48
+ - The shipped RBI sorbet types no longer use generics.
49
+ - `Prism::ConstantPathNode#child` and `Prism::ConstantTargetNode#child` are now deprecated, replaced by two new fields on these nodes: `name` and `name_loc`.
50
+
51
+ ## [0.27.0] - 2024-04-23
52
+
53
+ ### Added
54
+
55
+ - Implemented `===` for each of the nodes, which will check if equality but ignore the specific ranges of locations.
56
+
57
+ ### Changed
58
+
59
+ - Fix translation of `ItParametersNode` for parser translation.
60
+ - Fix translation of `dstr` for ruby_parser translation.
61
+ - Do not allow omitted hash values whose keys end with `!` or `?`.
62
+ - Split up `Prism::ParseResult` into `Prism::Result` with subclasses `Prism::ParseResult`, `Prism::LexResult`, `Prism::ParseLexResult`, and `Prism::LexCompat::Result`.
63
+ - Change reflection classes to have only a single `IntegerField` class and rename `DoubleField` to `FloatField`.
64
+ - Fall back to default `AR` and `CC` in `Makefile`.
65
+ - Use GC-able symbols for the syntax tree to avoid adding to the global symbol table.
66
+ - Fix a bug with karatsuba_multiply that would result in a stack overflow.
67
+ - Fix parser translation when looking for tokens with `srange_find`.
68
+
69
+ ## [0.26.0] - 2024-04-18
70
+
71
+ ### Added
72
+
73
+ - Add `Prism::Node::fields`, which returns a list of `Prism::Reflection::Field` objects representing the fields of the node class. This is useful in metaprogramming contexts.
74
+ - `Prism::Location#chop`, for removing the last byte from a location.
75
+ - The void statement warning is now implemented.
76
+ - The unreachable statement warning is now implemented.
77
+ - A syntax error has been added for block arguments on yields, e.g., `yield(&foo)`.
78
+
79
+ ### Changed
80
+
81
+ - Better fidelity to `parser` when translating heredocs with interpolation.
82
+ - Fixed `RBI` and `RBS` types for `Prism::parse_*` signatures.
83
+ - Remove some incorrect warnings about unused local variables.
84
+ - More closely match CRuby error messages for global variables.
85
+ - Fix an issue with `parser` translation when line continuations are found in string literals.
86
+
87
+ ## [0.25.0] - 2024-04-05
88
+
89
+ ### Added
90
+
91
+ - `Prism::Translation::Ripper` is now able to mirror all of the Ripper APIs.
92
+ - `Prism::Location#leading_comments` and `Prism::Location#trailing_comments` is added.
93
+ - `Prism::Comment#slice` is added.
94
+ - Warn for writing literal values in conditional predicates.
95
+ - Check for `_POSIX_MAPPED_FILES` before using `mmap`.
96
+ - `Prism::ItParametersNode` is added, to support `-> { it }`.
97
+ - Parse integer and float literal values onto the tree.
98
+ - Warn on duplicated hash keys and duplicated when clauses.
99
+ - Ship much improved `RBI` and `RBS` types.
100
+ - Support for the `-p`, `-n`, `-a`, and `-l` command line switches.
101
+ - Warn on integer literals in flip-flops.
102
+ - Support BSD make.
103
+ - Add `Prism::WhenNode#then_keyword_loc`.
104
+ - Support custom allocation functions through the `PRISM_XALLOCATOR` define.
105
+ - Warn for certain keywrods at the end of the line.
106
+ - Provide `pm_visit_node`, a C visitor API.
107
+ - `Prism::parse_stream` is added, which works for any Ruby `IO` object.
108
+ - Provide flags for regular expression literals for their derived encoding.
109
+ - Provide flags for whether or not an interpolated string literal is frozen.
110
+ - Add `Prism::StringNode.mutable?` for when a string is explicitly mutable, to support delineating chilled strings.
111
+ - Warn for incorrect character literal syntax.
112
+ - Warn for chained comparison operators.
113
+ - Warn for `**` interpreted as an argument prefix.
114
+ - Warn for `&` interpreted as an argument prefix.
115
+ - `Prism::ShareableConstantNode` added to support ractors.
116
+ - Warn for frozen string literals found after tokens.
117
+ - Support `PRISM_BUILD_MINIMAL` to provide only the minimal necessary functionality to reduce the binary size.
118
+ - Handle CLRF inside heredocs, strings, and regular expressions.
119
+ - Mark inner strings in interpolated strings as frozen.
120
+ - Support the `-x` command line switch.
121
+ - Error messages now much more closely mirror CRuby.
122
+ - Provide syntax errors for invalid block exits (`break`, `next`, `retry`, and `yield`).
123
+ - Warn on unused local variables.
124
+ - Do not syntax error on default parameter values that only write to the parameter.
125
+
126
+ ### Changed
127
+
128
+ - Many improvements to the compatibility with the `whitequark/parser` translation.
129
+ - Accept newlines before pattern terminators `)` or `]`.
130
+ - `Prism::Node#start_offset` and `Prism::Node#end_offset` are now much more efficient.
131
+ - Read files using `fread` instead of `mmap` when we're going to keep around the source through the Ruby API.
132
+ - Fix `Sexp#line_max` setting in the `seattlerb/ruby_parser` translation layer.
133
+ - Allow spaces before the encoding comment.
134
+
9
135
  ## [0.24.0] - 2024-02-15
10
136
 
11
137
  ### Added
@@ -392,7 +518,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
392
518
 
393
519
  - 🎉 Initial release! 🎉
394
520
 
395
- [unreleased]: https://github.com/ruby/prism/compare/v0.24.0...HEAD
521
+ [unreleased]: https://github.com/ruby/prism/compare/v0.29.0...HEAD
522
+ [0.29.0]: https://github.com/ruby/prism/compare/v0.28.0...v0.29.0
523
+ [0.28.0]: https://github.com/ruby/prism/compare/v0.27.0...v0.28.0
524
+ [0.27.0]: https://github.com/ruby/prism/compare/v0.26.0...v0.27.0
525
+ [0.26.0]: https://github.com/ruby/prism/compare/v0.25.0...v0.26.0
526
+ [0.25.0]: https://github.com/ruby/prism/compare/v0.24.0...v0.25.0
396
527
  [0.24.0]: https://github.com/ruby/prism/compare/v0.23.0...v0.24.0
397
528
  [0.23.0]: https://github.com/ruby/prism/compare/v0.22.0...v0.23.0
398
529
  [0.22.0]: https://github.com/ruby/prism/compare/v0.21.0...v0.22.0
data/Makefile CHANGED
@@ -6,17 +6,21 @@ Q1 = $(V:1=)
6
6
  Q = $(Q1:0=@)
7
7
  ECHO1 = $(V:1=@ :)
8
8
  ECHO = $(ECHO1:0=@ echo)
9
- FUZZ_OUTPUT_DIR = $(shell pwd)/fuzz/output
9
+ FUZZ_OUTPUT_DIR = $(CURDIR)/fuzz/output
10
10
 
11
- SOEXT := $(shell ruby -e 'puts RbConfig::CONFIG["SOEXT"]')
11
+ SOEXT ?= $(shell ruby -e 'puts RbConfig::CONFIG["SOEXT"]')
12
12
 
13
- CPPFLAGS := -Iinclude
14
- CFLAGS := -g -O2 -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden
15
- CC := cc
13
+ CPPFLAGS := -Iinclude $(CPPFLAGS)
14
+ CFLAGS := -g -O2 -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden $(CFLAGS)
15
+ CC ?= cc
16
+ AR ?= ar
16
17
  WASI_SDK_PATH := /opt/wasi-sdk
17
18
 
18
- HEADERS := $(shell find include -name '*.h')
19
- SOURCES := $(shell find src -name '*.c')
19
+ MAKEDIRS ?= mkdir -p
20
+ RMALL ?= rm -f -r
21
+
22
+ HEADERS := $(wildcard include/*.h include/*/*.h include/*/*/*.h')
23
+ SOURCES := $(wildcard src/*.c src/*/*.c)
20
24
  SHARED_OBJECTS := $(subst src/,build/shared/,$(SOURCES:.c=.o))
21
25
  STATIC_OBJECTS := $(subst src/,build/static/,$(SOURCES:.c=.o))
22
26
 
@@ -28,11 +32,11 @@ wasm: javascript/src/prism.wasm
28
32
  java-wasm: java-wasm/src/test/resources/prism.wasm
29
33
 
30
34
  build/libprism.$(SOEXT): $(SHARED_OBJECTS)
31
- $(ECHO) "linking $@"
35
+ $(ECHO) "linking $@ with $(CC)"
32
36
  $(Q) $(CC) $(DEBUG_FLAGS) $(CFLAGS) -shared -o $@ $(SHARED_OBJECTS)
33
37
 
34
38
  build/libprism.a: $(STATIC_OBJECTS)
35
- $(ECHO) "building $@"
39
+ $(ECHO) "building $@ with $(AR)"
36
40
  $(Q) $(AR) $(ARFLAGS) $@ $(STATIC_OBJECTS) $(Q1:0=>/dev/null)
37
41
 
38
42
  javascript/src/prism.wasm: Makefile $(SOURCES) $(HEADERS)
@@ -45,17 +49,17 @@ java-wasm/src/test/resources/prism.wasm: Makefile $(SOURCES) $(HEADERS)
45
49
 
46
50
  build/shared/%.o: src/%.c Makefile $(HEADERS)
47
51
  $(ECHO) "compiling $@"
48
- $(Q) mkdir -p $(@D)
52
+ $(Q) $(MAKEDIRS) $(@D)
49
53
  $(Q) $(CC) $(DEBUG_FLAGS) -DPRISM_EXPORT_SYMBOLS $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
50
54
 
51
55
  build/static/%.o: src/%.c Makefile $(HEADERS)
52
56
  $(ECHO) "compiling $@"
53
- $(Q) mkdir -p $(@D)
57
+ $(Q) $(MAKEDIRS) $(@D)
54
58
  $(Q) $(CC) $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
55
59
 
56
60
  build/fuzz.%: $(SOURCES) fuzz/%.c fuzz/fuzz.c
57
61
  $(ECHO) "building $* fuzzer"
58
- $(Q) mkdir -p $(@D)
62
+ $(Q) $(MAKEDIRS) $(@D)
59
63
  $(ECHO) "building main fuzz binary"
60
64
  $(Q) AFL_HARDEN=1 afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize-ignorelist=fuzz/asan.ignore -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@ $^
61
65
  $(ECHO) "building cmplog binary"
@@ -66,7 +70,7 @@ build/fuzz.heisenbug.%: $(SOURCES) fuzz/%.c fuzz/heisenbug.c
66
70
 
67
71
  fuzz-debug:
68
72
  $(ECHO) "entering debug shell"
69
- $(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(shell pwd):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
73
+ $(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
70
74
 
71
75
  fuzz-docker-build: fuzz/docker/Dockerfile
72
76
  $(ECHO) "building docker image"
@@ -76,17 +80,17 @@ fuzz-run-%: FORCE fuzz-docker-build
76
80
  $(ECHO) "generating templates"
77
81
  $(Q) bundle exec rake templates
78
82
  $(ECHO) "running $* fuzzer"
79
- $(Q) docker run --rm -v $(shell pwd):/prism prism/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
83
+ $(Q) docker run --rm -v $(CURDIR):/prism prism/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
80
84
  $(ECHO) "starting AFL++ run"
81
- $(Q) mkdir -p $(FUZZ_OUTPUT_DIR)/$*
82
- $(Q) docker run -it --rm -v $(shell pwd):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
85
+ $(Q) $(MAKEDIRS) $(FUZZ_OUTPUT_DIR)/$*
86
+ $(Q) docker run -it --rm -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
83
87
  FORCE:
84
88
 
85
89
  fuzz-clean:
86
- $(Q) rm -f -r fuzz/output
90
+ $(Q) $(RMALL) fuzz/output
87
91
 
88
92
  clean:
89
- $(Q) rm -f -r build
93
+ $(Q) $(RMALL) build
90
94
 
91
95
  .PHONY: clean fuzz-clean
92
96
 
@@ -94,6 +98,9 @@ all-no-debug: DEBUG_FLAGS := -DNDEBUG=1
94
98
  all-no-debug: OPTFLAGS := -O3
95
99
  all-no-debug: all
96
100
 
101
+ minimal: CFLAGS := $(CFLAGS) -DPRISM_BUILD_MINIMAL
102
+ minimal: all
103
+
97
104
  run: Makefile $(STATIC_OBJECTS) $(HEADERS) test.c
98
105
  $(ECHO) "compiling test.c"
99
106
  $(Q) $(CC) $(CPPFLAGS) $(CFLAGS) $(STATIC_OBJECTS) test.c
data/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  <img alt="Prism Ruby parser" height="256px" src="https://github.com/ruby/prism/blob/main/doc/images/prism.png?raw=true">
4
4
  </div>
5
5
 
6
- This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies. It is currently being integrated into [CRuby](https://github.com/ruby/ruby), [JRuby](https://github.com/jruby/jruby), [TruffleRuby](https://github.com/oracle/truffleruby), [Sorbet](https://github.com/sorbet/sorbet), and [Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree).
6
+ This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies.
7
7
 
8
8
  ## Overview
9
9
 
@@ -15,27 +15,33 @@ The repository contains the infrastructure for both a shared library (libprism)
15
15
  ├── Rakefile configuration to compile the native extension and run the Ruby tests
16
16
  ├── bin
17
17
  │   ├── lex runs the lexer on a file or string, prints the tokens, and compares to ripper
18
- │   └── parse runs the parser on a file or string and prints the syntax tree
18
+ │   ├── parse runs the parse on a file or string and prints the AST
19
+ │   └── prism a CLI for development and debugging
19
20
  ├── config.yml specification for tokens and nodes in the tree
20
- ├── docs documentation about the project
21
+ ├── doc documentation website
22
+ ├── docs markdown documentation about the project
21
23
  ├── ext
22
24
  │   └── prism
23
25
  │   ├── extconf.rb configuration to generate the Makefile for the native extension
24
26
  │   └── extension.c the native extension that interacts with libprism
25
27
  ├── fuzz files related to fuzz testing
28
+ ├── gemfiles gemfiles used by different Ruby versions in CI
26
29
  ├── include
27
30
  │   ├── prism header files for the shared library
28
31
  │   └── prism.h main header file for the shared library
29
32
  ├── java Java bindings for the shared library
33
+ ├── java-wasm Java WASM bindings for the shared library
34
+ ├── javascript JavaScript WASM bindings for the shared library
30
35
  ├── lib
31
36
  │   ├── prism Ruby library files
32
37
  │   └── prism.rb main entrypoint for the Ruby library
33
38
  ├── rakelib various Rake tasks for the project
39
+ ├── rbi RBI type signatures for the Ruby library
34
40
  ├── rust
35
41
  │   ├── ruby-prism Rustified crate for the shared library
36
42
  │   └── ruby-prism-sys FFI binding for Rust
43
+ ├── sig RBS type signatures for the Ruby library
37
44
  ├── src
38
- │   ├── enc various encoding files
39
45
  │   ├── util various utility files
40
46
  │   └── prism.c main entrypoint for the shared library
41
47
  ├── templates contains ERB templates generated by templates/template.rb
@@ -51,7 +57,7 @@ The repository contains the infrastructure for both a shared library (libprism)
51
57
  To compile the shared library, you will need:
52
58
 
53
59
  * C99 compiler
54
- * make
60
+ * GNU make
55
61
  * Ruby 2.7.0 or later
56
62
 
57
63
  Once you have these dependencies, run:
@@ -91,8 +97,41 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
91
97
  * [Parser translation](docs/parser_translation.md)
92
98
  * [Parsing rules](docs/parsing_rules.md)
93
99
  * [Releasing](docs/releasing.md)
94
- * [Ripper](docs/ripper.md)
100
+ * [Ripper translation](docs/ripper_translation.md)
95
101
  * [Ruby API](docs/ruby_api.md)
96
102
  * [RubyParser translation](docs/ruby_parser_translation.md)
97
103
  * [Serialization](docs/serialization.md)
98
104
  * [Testing](docs/testing.md)
105
+
106
+ ## Examples
107
+
108
+ Prism has been integrated into the majority of Ruby runtimes, many libraries, and some applications. Below is a list of some of the projects that use Prism:
109
+
110
+ ### Runtimes
111
+
112
+ * [CRuby](https://github.com/ruby/ruby/pull/7964) (via C)
113
+ * [Garnet](https://github.com/camertron/garnet-js) (via WASM)
114
+ * [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java)
115
+ * [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby)
116
+ * [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM)
117
+ * [TruffleRuby](https://github.com/oracle/truffleruby/issues/3117) (via Java)
118
+
119
+ ### Libraries
120
+
121
+ * [dispersion](https://github.com/joeldrapper/dispersion)
122
+ * [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator)
123
+ * [rbi](https://github.com/Shopify/rbi)
124
+ * [rails](https://github.com/rails/rails)
125
+ * [parsing renders](https://github.com/rails/rails/pull/49438)
126
+ * [parsing rdoc](https://github.com/rails/rails/pull/50870)
127
+ * [parsing tests](https://github.com/rails/rails/pull/51006)
128
+ * [repl_type_completor](https://github.com/ruby/repl_type_completor)
129
+ * [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-parser-engine) (via parser translator)
130
+ * [ruby-lsp](https://github.com/Shopify/ruby-lsp)
131
+ * [smart_todo](https://github.com/Shopify/smart_todo/pull/69)
132
+ * [sorbet-eraser](https://github.com/kddnewton/sorbet-eraser/pull/25)
133
+ * [synvert](https://github.com/xinminlabs/synvert-core-ruby)
134
+
135
+ ### Applications
136
+
137
+ * [gem.sh](https://github.com/marcoroth/gem.sh/pull/96)