prism 0.24.0 → 0.29.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/BSDmakefile +58 -0
- data/CHANGELOG.md +132 -1
- data/Makefile +25 -18
- data/README.md +45 -6
- data/config.yml +828 -25
- data/docs/build_system.md +31 -0
- data/docs/configuration.md +4 -0
- data/docs/cruby_compilation.md +1 -1
- data/docs/parser_translation.md +14 -9
- data/docs/releasing.md +7 -9
- data/docs/ripper_translation.md +50 -0
- data/docs/ruby_api.md +1 -0
- data/docs/serialization.md +26 -5
- data/ext/prism/api_node.c +1037 -936
- data/ext/prism/api_pack.c +9 -0
- data/ext/prism/extconf.rb +62 -18
- data/ext/prism/extension.c +351 -71
- data/ext/prism/extension.h +5 -4
- data/include/prism/ast.h +539 -101
- data/include/prism/defines.h +106 -2
- data/include/prism/diagnostic.h +168 -74
- data/include/prism/encoding.h +22 -4
- data/include/prism/node.h +93 -0
- data/include/prism/options.h +84 -9
- data/include/prism/pack.h +11 -0
- data/include/prism/parser.h +213 -54
- data/include/prism/prettyprint.h +8 -0
- data/include/prism/static_literals.h +120 -0
- data/include/prism/util/pm_buffer.h +65 -2
- data/include/prism/util/pm_constant_pool.h +18 -1
- data/include/prism/util/pm_integer.h +119 -0
- data/include/prism/util/pm_list.h +1 -1
- data/include/prism/util/pm_newline_list.h +8 -0
- data/include/prism/util/pm_string.h +26 -2
- data/include/prism/version.h +2 -2
- data/include/prism.h +59 -1
- data/lib/prism/compiler.rb +8 -1
- data/lib/prism/debug.rb +46 -3
- data/lib/prism/desugar_compiler.rb +5 -3
- data/lib/prism/dispatcher.rb +29 -0
- data/lib/prism/dot_visitor.rb +141 -54
- data/lib/prism/dsl.rb +48 -36
- data/lib/prism/ffi.rb +82 -17
- data/lib/prism/inspect_visitor.rb +2156 -0
- data/lib/prism/lex_compat.rb +34 -15
- data/lib/prism/mutation_compiler.rb +13 -2
- data/lib/prism/node.rb +4453 -4459
- data/lib/prism/node_ext.rb +249 -30
- data/lib/prism/pack.rb +4 -0
- data/lib/prism/parse_result/comments.rb +35 -18
- data/lib/prism/parse_result/newlines.rb +2 -2
- data/lib/prism/parse_result.rb +218 -43
- data/lib/prism/pattern.rb +28 -10
- data/lib/prism/polyfill/byteindex.rb +13 -0
- data/lib/prism/polyfill/unpack1.rb +14 -0
- data/lib/prism/reflection.rb +411 -0
- data/lib/prism/serialize.rb +480 -112
- data/lib/prism/translation/parser/compiler.rb +376 -88
- data/lib/prism/translation/parser/lexer.rb +103 -22
- data/lib/prism/translation/parser/rubocop.rb +41 -13
- data/lib/prism/translation/parser.rb +123 -11
- data/lib/prism/translation/parser33.rb +1 -1
- data/lib/prism/translation/parser34.rb +1 -1
- data/lib/prism/translation/ripper/sexp.rb +125 -0
- data/lib/prism/translation/ripper/shim.rb +5 -0
- data/lib/prism/translation/ripper.rb +3216 -462
- data/lib/prism/translation/ruby_parser.rb +111 -56
- data/lib/prism/translation.rb +3 -1
- data/lib/prism/visitor.rb +10 -0
- data/lib/prism.rb +12 -20
- data/prism.gemspec +46 -14
- data/rbi/prism/compiler.rbi +12 -0
- data/rbi/prism/inspect_visitor.rbi +12 -0
- data/rbi/prism/node.rbi +8712 -0
- data/rbi/prism/node_ext.rbi +107 -0
- data/rbi/prism/parse_result.rbi +358 -0
- data/rbi/prism/reflection.rbi +58 -0
- data/rbi/prism/translation/parser.rbi +11 -0
- data/rbi/prism/translation/parser33.rbi +6 -0
- data/rbi/prism/translation/parser34.rbi +6 -0
- data/rbi/prism/translation/ripper.rbi +15 -0
- data/rbi/prism/visitor.rbi +470 -0
- data/rbi/prism.rbi +38 -7748
- data/sig/prism/compiler.rbs +9 -0
- data/sig/prism/dispatcher.rbs +16 -0
- data/sig/prism/dot_visitor.rbs +6 -0
- data/sig/prism/dsl.rbs +462 -0
- data/sig/prism/inspect_visitor.rbs +22 -0
- data/sig/prism/lex_compat.rbs +10 -0
- data/sig/prism/mutation_compiler.rbs +158 -0
- data/sig/prism/node.rbs +3558 -0
- data/sig/prism/node_ext.rbs +82 -0
- data/sig/prism/pack.rbs +43 -0
- data/sig/prism/parse_result.rbs +160 -0
- data/sig/prism/pattern.rbs +13 -0
- data/sig/prism/reflection.rbs +50 -0
- data/sig/prism/serialize.rbs +6 -0
- data/sig/prism/visitor.rbs +168 -0
- data/sig/prism.rbs +188 -4767
- data/src/diagnostic.c +636 -230
- data/src/encoding.c +211 -108
- data/src/node.c +7555 -451
- data/src/options.c +66 -31
- data/src/pack.c +33 -17
- data/src/prettyprint.c +1383 -1431
- data/src/prism.c +4734 -1310
- data/src/regexp.c +17 -2
- data/src/serialize.c +68 -46
- data/src/static_literals.c +638 -0
- data/src/token_type.c +10 -9
- data/src/util/pm_buffer.c +147 -20
- data/src/util/pm_char.c +4 -4
- data/src/util/pm_constant_pool.c +35 -11
- data/src/util/pm_integer.c +642 -0
- data/src/util/pm_list.c +1 -1
- data/src/util/pm_newline_list.c +14 -5
- data/src/util/pm_string.c +134 -5
- data/src/util/pm_string_list.c +2 -2
- metadata +41 -9
- data/docs/ripper.md +0 -36
- data/include/prism/util/pm_state_stack.h +0 -42
- data/lib/prism/node_inspector.rb +0 -68
- data/rbi/prism_static.rbi +0 -207
- data/sig/prism_static.rbs +0 -201
- data/src/util/pm_state_stack.c +0 -25
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4ac8167684c0f975fdba449e77206d97756f6c39f22ce871f79a79a61260503f
|
4
|
+
data.tar.gz: 6a2dd5c0a47df7c8c575ad2e5b344c789d548e93cfe82e5fa29974d46a52bb3c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: dfa7fe63285b85cb45aa0be681916d6891ee5e5adf2162ab5c18423417f7938afc99f00eca8fd6187fbd0b7168af088d932fbda3c1361d1c73953f39329bed70
|
7
|
+
data.tar.gz: 79bc51db60600d74a6bfbeae5eba3a8ec505e9809d0f7ff0b05563f1f7a8cd284cc293233397323a443217a4a5271f90b4c40516168ae3a8be0ddc2d7376cef9
|
data/BSDmakefile
ADDED
@@ -0,0 +1,58 @@
|
|
1
|
+
# GNU makefile proxy script for BSD make
|
2
|
+
#
|
3
|
+
# Written and maintained by Mahmoud Al-Qudsi <mqudsi@neosmart.net>
|
4
|
+
# Copyright NeoSmart Technologies <https://neosmart.net/> 2014-2019
|
5
|
+
# Obtain updates from <https://github.com/neosmart/gmake-proxy>
|
6
|
+
#
|
7
|
+
# Redistribution and use in source and binary forms, with or without
|
8
|
+
# modification, are permitted provided that the following conditions are met:
|
9
|
+
#
|
10
|
+
# 1. Redistributions of source code must retain the above copyright notice, this
|
11
|
+
# list of conditions and the following disclaimer.
|
12
|
+
#
|
13
|
+
# 2. Redistributions in binary form must reproduce the above copyright notice,
|
14
|
+
# this list of conditions and the following disclaimer in the documentation
|
15
|
+
# and/or other materials provided with the distribution.
|
16
|
+
#
|
17
|
+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
18
|
+
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
19
|
+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
20
|
+
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
21
|
+
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
22
|
+
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
23
|
+
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
24
|
+
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
25
|
+
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
26
|
+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
27
|
+
|
28
|
+
JARG =
|
29
|
+
GMAKE = "gmake"
|
30
|
+
# When gmake is called from another make instance, -w is automatically added
|
31
|
+
# which causes extraneous messages about directory changes to be emitted.
|
32
|
+
# Running with --no-print-directory silences these messages.
|
33
|
+
GARGS = "--no-print-directory"
|
34
|
+
|
35
|
+
.if "$(.MAKE.JOBS)" != ""
|
36
|
+
JARG = -j$(.MAKE.JOBS)
|
37
|
+
.endif
|
38
|
+
|
39
|
+
# bmake prefers out-of-source builds and tries to cd into ./obj (among others)
|
40
|
+
# where possible. GNU Make doesn't, so override that value.
|
41
|
+
.OBJDIR: ./
|
42
|
+
|
43
|
+
# The GNU convention is to use the lowercased `prefix` variable/macro to
|
44
|
+
# specify the installation directory. Humor them.
|
45
|
+
GPREFIX = ""
|
46
|
+
.if defined(PREFIX) && ! defined(prefix)
|
47
|
+
GPREFIX = 'prefix = "$(PREFIX)"'
|
48
|
+
.endif
|
49
|
+
|
50
|
+
.BEGIN: .SILENT
|
51
|
+
which $(GMAKE) || printf "Error: GNU Make is required!\n\n" 1>&2 && false
|
52
|
+
|
53
|
+
.PHONY: FRC
|
54
|
+
$(.TARGETS): FRC
|
55
|
+
$(GMAKE) $(GPREFIX) $(GARGS) $(.TARGETS:S,.DONE,,) $(JARG)
|
56
|
+
|
57
|
+
.DONE .DEFAULT: .SILENT
|
58
|
+
$(GMAKE) $(GPREFIX) $(GARGS) $(.TARGETS:S,.DONE,,) $(JARG)
|
data/CHANGELOG.md
CHANGED
@@ -6,6 +6,132 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
|
|
6
6
|
|
7
7
|
## [Unreleased]
|
8
8
|
|
9
|
+
## [0.29.0] - 2024-05-10
|
10
|
+
|
11
|
+
### Added
|
12
|
+
|
13
|
+
- Added `Prism::CallNode#full_message_loc`, which gives the location including the `=` if there is one.
|
14
|
+
- A warning for when `# shareable_constant_value` is not used on its own line.
|
15
|
+
- An error for invalid implicit local variable writes.
|
16
|
+
- Implicit hash patterns in array patterns are disallowed.
|
17
|
+
- We now validate that Unicode escape sequences are not surrogates.
|
18
|
+
|
19
|
+
### Changed
|
20
|
+
|
21
|
+
- All fields named `operator` have been renamed to `binary_operator` for `*OperatorWriteNode` nodes. This is to make it easier to provide C++ support. In the Ruby API, the old fields are aliased to the new fields with a deprecation warning.
|
22
|
+
- Many updated error messages to more closely match CRuby.
|
23
|
+
- We ensure keyword parameters do not end in `!` or `?`.
|
24
|
+
- Fixed some escaping in string literals with control sequences and hex escapes.
|
25
|
+
- Fix a bug with RBS types when used outside the `ruby/prism` codebase.
|
26
|
+
|
27
|
+
## [0.28.0] - 2024-05-03
|
28
|
+
|
29
|
+
### Added
|
30
|
+
|
31
|
+
- Nested hashes will now warn for duplicated keys, as in: `{ foo: 1, **{ foo: 2 } }`.
|
32
|
+
- `Prism::ReturnNode` now has a flag on it to indicate if it is redundant.
|
33
|
+
- `Prism::Location#slice_lines` and `Prism::Node#slice_lines` are now provided to slice the source code of a node including the content before the node on the same line that it starts on and the content after the node on the same line that it ends on.
|
34
|
+
- Symbols with invalid byte sequences now give errors.
|
35
|
+
- You can now pass `"3.3.1"` to the `version:` parameter on all `Prism.*` APIs.
|
36
|
+
- `Prism::Source#lines`, `Prism::Location#source_lines`, `Prism::Node#source_lines`, and `Prism::Node#script_lines` are now provided, which will all return the source code of the source as an array of strings.
|
37
|
+
- `Prism::ASCIISource` is now provided, which is a subclass of `Prism::Source` but specialized to increase performance when the source is entirely ASCII.
|
38
|
+
- Prism now provides errors when parsing Ruby 3.4+ syntax for index expressions with keywords or blocks.
|
39
|
+
- Prism now provides an error when `**nil` is used after other keyword parameters.
|
40
|
+
- Prism now provides errors when safe navigation is used in call target expressions, e.g., `foo&.bar, = 1`.
|
41
|
+
- `Prism::Node#tunnel` is now provided, which returns an array of nodes starting at the current node that contain a given line and column.
|
42
|
+
|
43
|
+
### Changed
|
44
|
+
|
45
|
+
- All translation layers now assume an eval context, which means they will not return errors for invalid jumps like `yield`.
|
46
|
+
- `Prism::Node#inspect` now uses a queue instead of recursion to avoid stack overflows.
|
47
|
+
- Prism now more closely mirrors CRuby interpolation semantics, which means you could potentially have a static literal string that directly interpolates another static literal string.
|
48
|
+
- The shipped RBI sorbet types no longer use generics.
|
49
|
+
- `Prism::ConstantPathNode#child` and `Prism::ConstantTargetNode#child` are now deprecated, replaced by two new fields on these nodes: `name` and `name_loc`.
|
50
|
+
|
51
|
+
## [0.27.0] - 2024-04-23
|
52
|
+
|
53
|
+
### Added
|
54
|
+
|
55
|
+
- Implemented `===` for each of the nodes, which will check if equality but ignore the specific ranges of locations.
|
56
|
+
|
57
|
+
### Changed
|
58
|
+
|
59
|
+
- Fix translation of `ItParametersNode` for parser translation.
|
60
|
+
- Fix translation of `dstr` for ruby_parser translation.
|
61
|
+
- Do not allow omitted hash values whose keys end with `!` or `?`.
|
62
|
+
- Split up `Prism::ParseResult` into `Prism::Result` with subclasses `Prism::ParseResult`, `Prism::LexResult`, `Prism::ParseLexResult`, and `Prism::LexCompat::Result`.
|
63
|
+
- Change reflection classes to have only a single `IntegerField` class and rename `DoubleField` to `FloatField`.
|
64
|
+
- Fall back to default `AR` and `CC` in `Makefile`.
|
65
|
+
- Use GC-able symbols for the syntax tree to avoid adding to the global symbol table.
|
66
|
+
- Fix a bug with karatsuba_multiply that would result in a stack overflow.
|
67
|
+
- Fix parser translation when looking for tokens with `srange_find`.
|
68
|
+
|
69
|
+
## [0.26.0] - 2024-04-18
|
70
|
+
|
71
|
+
### Added
|
72
|
+
|
73
|
+
- Add `Prism::Node::fields`, which returns a list of `Prism::Reflection::Field` objects representing the fields of the node class. This is useful in metaprogramming contexts.
|
74
|
+
- `Prism::Location#chop`, for removing the last byte from a location.
|
75
|
+
- The void statement warning is now implemented.
|
76
|
+
- The unreachable statement warning is now implemented.
|
77
|
+
- A syntax error has been added for block arguments on yields, e.g., `yield(&foo)`.
|
78
|
+
|
79
|
+
### Changed
|
80
|
+
|
81
|
+
- Better fidelity to `parser` when translating heredocs with interpolation.
|
82
|
+
- Fixed `RBI` and `RBS` types for `Prism::parse_*` signatures.
|
83
|
+
- Remove some incorrect warnings about unused local variables.
|
84
|
+
- More closely match CRuby error messages for global variables.
|
85
|
+
- Fix an issue with `parser` translation when line continuations are found in string literals.
|
86
|
+
|
87
|
+
## [0.25.0] - 2024-04-05
|
88
|
+
|
89
|
+
### Added
|
90
|
+
|
91
|
+
- `Prism::Translation::Ripper` is now able to mirror all of the Ripper APIs.
|
92
|
+
- `Prism::Location#leading_comments` and `Prism::Location#trailing_comments` is added.
|
93
|
+
- `Prism::Comment#slice` is added.
|
94
|
+
- Warn for writing literal values in conditional predicates.
|
95
|
+
- Check for `_POSIX_MAPPED_FILES` before using `mmap`.
|
96
|
+
- `Prism::ItParametersNode` is added, to support `-> { it }`.
|
97
|
+
- Parse integer and float literal values onto the tree.
|
98
|
+
- Warn on duplicated hash keys and duplicated when clauses.
|
99
|
+
- Ship much improved `RBI` and `RBS` types.
|
100
|
+
- Support for the `-p`, `-n`, `-a`, and `-l` command line switches.
|
101
|
+
- Warn on integer literals in flip-flops.
|
102
|
+
- Support BSD make.
|
103
|
+
- Add `Prism::WhenNode#then_keyword_loc`.
|
104
|
+
- Support custom allocation functions through the `PRISM_XALLOCATOR` define.
|
105
|
+
- Warn for certain keywrods at the end of the line.
|
106
|
+
- Provide `pm_visit_node`, a C visitor API.
|
107
|
+
- `Prism::parse_stream` is added, which works for any Ruby `IO` object.
|
108
|
+
- Provide flags for regular expression literals for their derived encoding.
|
109
|
+
- Provide flags for whether or not an interpolated string literal is frozen.
|
110
|
+
- Add `Prism::StringNode.mutable?` for when a string is explicitly mutable, to support delineating chilled strings.
|
111
|
+
- Warn for incorrect character literal syntax.
|
112
|
+
- Warn for chained comparison operators.
|
113
|
+
- Warn for `**` interpreted as an argument prefix.
|
114
|
+
- Warn for `&` interpreted as an argument prefix.
|
115
|
+
- `Prism::ShareableConstantNode` added to support ractors.
|
116
|
+
- Warn for frozen string literals found after tokens.
|
117
|
+
- Support `PRISM_BUILD_MINIMAL` to provide only the minimal necessary functionality to reduce the binary size.
|
118
|
+
- Handle CLRF inside heredocs, strings, and regular expressions.
|
119
|
+
- Mark inner strings in interpolated strings as frozen.
|
120
|
+
- Support the `-x` command line switch.
|
121
|
+
- Error messages now much more closely mirror CRuby.
|
122
|
+
- Provide syntax errors for invalid block exits (`break`, `next`, `retry`, and `yield`).
|
123
|
+
- Warn on unused local variables.
|
124
|
+
- Do not syntax error on default parameter values that only write to the parameter.
|
125
|
+
|
126
|
+
### Changed
|
127
|
+
|
128
|
+
- Many improvements to the compatibility with the `whitequark/parser` translation.
|
129
|
+
- Accept newlines before pattern terminators `)` or `]`.
|
130
|
+
- `Prism::Node#start_offset` and `Prism::Node#end_offset` are now much more efficient.
|
131
|
+
- Read files using `fread` instead of `mmap` when we're going to keep around the source through the Ruby API.
|
132
|
+
- Fix `Sexp#line_max` setting in the `seattlerb/ruby_parser` translation layer.
|
133
|
+
- Allow spaces before the encoding comment.
|
134
|
+
|
9
135
|
## [0.24.0] - 2024-02-15
|
10
136
|
|
11
137
|
### Added
|
@@ -392,7 +518,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
|
|
392
518
|
|
393
519
|
- 🎉 Initial release! 🎉
|
394
520
|
|
395
|
-
[unreleased]: https://github.com/ruby/prism/compare/v0.
|
521
|
+
[unreleased]: https://github.com/ruby/prism/compare/v0.29.0...HEAD
|
522
|
+
[0.29.0]: https://github.com/ruby/prism/compare/v0.28.0...v0.29.0
|
523
|
+
[0.28.0]: https://github.com/ruby/prism/compare/v0.27.0...v0.28.0
|
524
|
+
[0.27.0]: https://github.com/ruby/prism/compare/v0.26.0...v0.27.0
|
525
|
+
[0.26.0]: https://github.com/ruby/prism/compare/v0.25.0...v0.26.0
|
526
|
+
[0.25.0]: https://github.com/ruby/prism/compare/v0.24.0...v0.25.0
|
396
527
|
[0.24.0]: https://github.com/ruby/prism/compare/v0.23.0...v0.24.0
|
397
528
|
[0.23.0]: https://github.com/ruby/prism/compare/v0.22.0...v0.23.0
|
398
529
|
[0.22.0]: https://github.com/ruby/prism/compare/v0.21.0...v0.22.0
|
data/Makefile
CHANGED
@@ -6,17 +6,21 @@ Q1 = $(V:1=)
|
|
6
6
|
Q = $(Q1:0=@)
|
7
7
|
ECHO1 = $(V:1=@ :)
|
8
8
|
ECHO = $(ECHO1:0=@ echo)
|
9
|
-
FUZZ_OUTPUT_DIR = $(
|
9
|
+
FUZZ_OUTPUT_DIR = $(CURDIR)/fuzz/output
|
10
10
|
|
11
|
-
SOEXT
|
11
|
+
SOEXT ?= $(shell ruby -e 'puts RbConfig::CONFIG["SOEXT"]')
|
12
12
|
|
13
|
-
CPPFLAGS := -Iinclude
|
14
|
-
CFLAGS := -g -O2 -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden
|
15
|
-
CC
|
13
|
+
CPPFLAGS := -Iinclude $(CPPFLAGS)
|
14
|
+
CFLAGS := -g -O2 -std=c99 -Wall -Werror -Wextra -Wpedantic -Wundef -Wconversion -Wno-missing-braces -fPIC -fvisibility=hidden $(CFLAGS)
|
15
|
+
CC ?= cc
|
16
|
+
AR ?= ar
|
16
17
|
WASI_SDK_PATH := /opt/wasi-sdk
|
17
18
|
|
18
|
-
|
19
|
-
|
19
|
+
MAKEDIRS ?= mkdir -p
|
20
|
+
RMALL ?= rm -f -r
|
21
|
+
|
22
|
+
HEADERS := $(wildcard include/*.h include/*/*.h include/*/*/*.h')
|
23
|
+
SOURCES := $(wildcard src/*.c src/*/*.c)
|
20
24
|
SHARED_OBJECTS := $(subst src/,build/shared/,$(SOURCES:.c=.o))
|
21
25
|
STATIC_OBJECTS := $(subst src/,build/static/,$(SOURCES:.c=.o))
|
22
26
|
|
@@ -28,11 +32,11 @@ wasm: javascript/src/prism.wasm
|
|
28
32
|
java-wasm: java-wasm/src/test/resources/prism.wasm
|
29
33
|
|
30
34
|
build/libprism.$(SOEXT): $(SHARED_OBJECTS)
|
31
|
-
$(ECHO) "linking $@"
|
35
|
+
$(ECHO) "linking $@ with $(CC)"
|
32
36
|
$(Q) $(CC) $(DEBUG_FLAGS) $(CFLAGS) -shared -o $@ $(SHARED_OBJECTS)
|
33
37
|
|
34
38
|
build/libprism.a: $(STATIC_OBJECTS)
|
35
|
-
$(ECHO) "building $@"
|
39
|
+
$(ECHO) "building $@ with $(AR)"
|
36
40
|
$(Q) $(AR) $(ARFLAGS) $@ $(STATIC_OBJECTS) $(Q1:0=>/dev/null)
|
37
41
|
|
38
42
|
javascript/src/prism.wasm: Makefile $(SOURCES) $(HEADERS)
|
@@ -45,17 +49,17 @@ java-wasm/src/test/resources/prism.wasm: Makefile $(SOURCES) $(HEADERS)
|
|
45
49
|
|
46
50
|
build/shared/%.o: src/%.c Makefile $(HEADERS)
|
47
51
|
$(ECHO) "compiling $@"
|
48
|
-
$(Q)
|
52
|
+
$(Q) $(MAKEDIRS) $(@D)
|
49
53
|
$(Q) $(CC) $(DEBUG_FLAGS) -DPRISM_EXPORT_SYMBOLS $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
|
50
54
|
|
51
55
|
build/static/%.o: src/%.c Makefile $(HEADERS)
|
52
56
|
$(ECHO) "compiling $@"
|
53
|
-
$(Q)
|
57
|
+
$(Q) $(MAKEDIRS) $(@D)
|
54
58
|
$(Q) $(CC) $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
|
55
59
|
|
56
60
|
build/fuzz.%: $(SOURCES) fuzz/%.c fuzz/fuzz.c
|
57
61
|
$(ECHO) "building $* fuzzer"
|
58
|
-
$(Q)
|
62
|
+
$(Q) $(MAKEDIRS) $(@D)
|
59
63
|
$(ECHO) "building main fuzz binary"
|
60
64
|
$(Q) AFL_HARDEN=1 afl-clang-lto $(DEBUG_FLAGS) $(CPPFLAGS) $(CFLAGS) $(FUZZ_FLAGS) -O0 -fsanitize-ignorelist=fuzz/asan.ignore -fsanitize=fuzzer,address -ggdb3 -std=c99 -Iinclude -o $@ $^
|
61
65
|
$(ECHO) "building cmplog binary"
|
@@ -66,7 +70,7 @@ build/fuzz.heisenbug.%: $(SOURCES) fuzz/%.c fuzz/heisenbug.c
|
|
66
70
|
|
67
71
|
fuzz-debug:
|
68
72
|
$(ECHO) "entering debug shell"
|
69
|
-
$(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(
|
73
|
+
$(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
|
70
74
|
|
71
75
|
fuzz-docker-build: fuzz/docker/Dockerfile
|
72
76
|
$(ECHO) "building docker image"
|
@@ -76,17 +80,17 @@ fuzz-run-%: FORCE fuzz-docker-build
|
|
76
80
|
$(ECHO) "generating templates"
|
77
81
|
$(Q) bundle exec rake templates
|
78
82
|
$(ECHO) "running $* fuzzer"
|
79
|
-
$(Q) docker run --rm -v $(
|
83
|
+
$(Q) docker run --rm -v $(CURDIR):/prism prism/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
|
80
84
|
$(ECHO) "starting AFL++ run"
|
81
|
-
$(Q)
|
82
|
-
$(Q) docker run -it --rm -v $(
|
85
|
+
$(Q) $(MAKEDIRS) $(FUZZ_OUTPUT_DIR)/$*
|
86
|
+
$(Q) docker run -it --rm -v $(CURDIR):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
|
83
87
|
FORCE:
|
84
88
|
|
85
89
|
fuzz-clean:
|
86
|
-
$(Q)
|
90
|
+
$(Q) $(RMALL) fuzz/output
|
87
91
|
|
88
92
|
clean:
|
89
|
-
$(Q)
|
93
|
+
$(Q) $(RMALL) build
|
90
94
|
|
91
95
|
.PHONY: clean fuzz-clean
|
92
96
|
|
@@ -94,6 +98,9 @@ all-no-debug: DEBUG_FLAGS := -DNDEBUG=1
|
|
94
98
|
all-no-debug: OPTFLAGS := -O3
|
95
99
|
all-no-debug: all
|
96
100
|
|
101
|
+
minimal: CFLAGS := $(CFLAGS) -DPRISM_BUILD_MINIMAL
|
102
|
+
minimal: all
|
103
|
+
|
97
104
|
run: Makefile $(STATIC_OBJECTS) $(HEADERS) test.c
|
98
105
|
$(ECHO) "compiling test.c"
|
99
106
|
$(Q) $(CC) $(CPPFLAGS) $(CFLAGS) $(STATIC_OBJECTS) test.c
|
data/README.md
CHANGED
@@ -3,7 +3,7 @@
|
|
3
3
|
<img alt="Prism Ruby parser" height="256px" src="https://github.com/ruby/prism/blob/main/doc/images/prism.png?raw=true">
|
4
4
|
</div>
|
5
5
|
|
6
|
-
This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies.
|
6
|
+
This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies.
|
7
7
|
|
8
8
|
## Overview
|
9
9
|
|
@@ -15,27 +15,33 @@ The repository contains the infrastructure for both a shared library (libprism)
|
|
15
15
|
├── Rakefile configuration to compile the native extension and run the Ruby tests
|
16
16
|
├── bin
|
17
17
|
│ ├── lex runs the lexer on a file or string, prints the tokens, and compares to ripper
|
18
|
-
│
|
18
|
+
│ ├── parse runs the parse on a file or string and prints the AST
|
19
|
+
│ └── prism a CLI for development and debugging
|
19
20
|
├── config.yml specification for tokens and nodes in the tree
|
20
|
-
├──
|
21
|
+
├── doc documentation website
|
22
|
+
├── docs markdown documentation about the project
|
21
23
|
├── ext
|
22
24
|
│ └── prism
|
23
25
|
│ ├── extconf.rb configuration to generate the Makefile for the native extension
|
24
26
|
│ └── extension.c the native extension that interacts with libprism
|
25
27
|
├── fuzz files related to fuzz testing
|
28
|
+
├── gemfiles gemfiles used by different Ruby versions in CI
|
26
29
|
├── include
|
27
30
|
│ ├── prism header files for the shared library
|
28
31
|
│ └── prism.h main header file for the shared library
|
29
32
|
├── java Java bindings for the shared library
|
33
|
+
├── java-wasm Java WASM bindings for the shared library
|
34
|
+
├── javascript JavaScript WASM bindings for the shared library
|
30
35
|
├── lib
|
31
36
|
│ ├── prism Ruby library files
|
32
37
|
│ └── prism.rb main entrypoint for the Ruby library
|
33
38
|
├── rakelib various Rake tasks for the project
|
39
|
+
├── rbi RBI type signatures for the Ruby library
|
34
40
|
├── rust
|
35
41
|
│ ├── ruby-prism Rustified crate for the shared library
|
36
42
|
│ └── ruby-prism-sys FFI binding for Rust
|
43
|
+
├── sig RBS type signatures for the Ruby library
|
37
44
|
├── src
|
38
|
-
│ ├── enc various encoding files
|
39
45
|
│ ├── util various utility files
|
40
46
|
│ └── prism.c main entrypoint for the shared library
|
41
47
|
├── templates contains ERB templates generated by templates/template.rb
|
@@ -51,7 +57,7 @@ The repository contains the infrastructure for both a shared library (libprism)
|
|
51
57
|
To compile the shared library, you will need:
|
52
58
|
|
53
59
|
* C99 compiler
|
54
|
-
* make
|
60
|
+
* GNU make
|
55
61
|
* Ruby 2.7.0 or later
|
56
62
|
|
57
63
|
Once you have these dependencies, run:
|
@@ -91,8 +97,41 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
|
|
91
97
|
* [Parser translation](docs/parser_translation.md)
|
92
98
|
* [Parsing rules](docs/parsing_rules.md)
|
93
99
|
* [Releasing](docs/releasing.md)
|
94
|
-
* [Ripper](docs/
|
100
|
+
* [Ripper translation](docs/ripper_translation.md)
|
95
101
|
* [Ruby API](docs/ruby_api.md)
|
96
102
|
* [RubyParser translation](docs/ruby_parser_translation.md)
|
97
103
|
* [Serialization](docs/serialization.md)
|
98
104
|
* [Testing](docs/testing.md)
|
105
|
+
|
106
|
+
## Examples
|
107
|
+
|
108
|
+
Prism has been integrated into the majority of Ruby runtimes, many libraries, and some applications. Below is a list of some of the projects that use Prism:
|
109
|
+
|
110
|
+
### Runtimes
|
111
|
+
|
112
|
+
* [CRuby](https://github.com/ruby/ruby/pull/7964) (via C)
|
113
|
+
* [Garnet](https://github.com/camertron/garnet-js) (via WASM)
|
114
|
+
* [JRuby](https://github.com/jruby/jruby/pull/8103) (via Java)
|
115
|
+
* [Natalie](https://github.com/natalie-lang/natalie/pull/1213) (via C++ and Ruby)
|
116
|
+
* [Opal](https://github.com/opal/opal/pull/2642) (via Ruby and WASM)
|
117
|
+
* [TruffleRuby](https://github.com/oracle/truffleruby/issues/3117) (via Java)
|
118
|
+
|
119
|
+
### Libraries
|
120
|
+
|
121
|
+
* [dispersion](https://github.com/joeldrapper/dispersion)
|
122
|
+
* [packwerk](https://github.com/Shopify/packwerk/pull/388) (via parser translator)
|
123
|
+
* [rbi](https://github.com/Shopify/rbi)
|
124
|
+
* [rails](https://github.com/rails/rails)
|
125
|
+
* [parsing renders](https://github.com/rails/rails/pull/49438)
|
126
|
+
* [parsing rdoc](https://github.com/rails/rails/pull/50870)
|
127
|
+
* [parsing tests](https://github.com/rails/rails/pull/51006)
|
128
|
+
* [repl_type_completor](https://github.com/ruby/repl_type_completor)
|
129
|
+
* [rubocop](https://docs.rubocop.org/rubocop/configuration.html#setting-the-parser-engine) (via parser translator)
|
130
|
+
* [ruby-lsp](https://github.com/Shopify/ruby-lsp)
|
131
|
+
* [smart_todo](https://github.com/Shopify/smart_todo/pull/69)
|
132
|
+
* [sorbet-eraser](https://github.com/kddnewton/sorbet-eraser/pull/25)
|
133
|
+
* [synvert](https://github.com/xinminlabs/synvert-core-ruby)
|
134
|
+
|
135
|
+
### Applications
|
136
|
+
|
137
|
+
* [gem.sh](https://github.com/marcoroth/gem.sh/pull/96)
|