prism 0.13.0 → 0.15.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 290f2191f3700e2584cc320e6dd99f78cfbd2cfd82006700f0bf977bfdc30424
-  data.tar.gz: 2886b0f5553307c1cd5d5e963ac03ea66330e2fa7d26cc4684d8f556e52520f5
+  metadata.gz: 1d02e58a6abcd72bbe920246599ed073e7c074929dff4f5b2408f87c9a222a51
+  data.tar.gz: 3c89a774748375ff30057d712465678ecff0936db4359d8fef10a819ba410d42
 SHA512:
-  metadata.gz: 24dc8c15f9693b120e8dfcdf8edf2d3a396e6eb636589eded6dac5b651cc62beee1f9c6464d49b1c93b5666e963755bfc4eb041a020ca9585ab7a0bb10172787
-  data.tar.gz: 272d11c30527f924644c3c97ac2895167798774007919d429b03325926f1853422a56463e7272ae02d4855f54dd1f775ce546065d2d2c78b1187192cce722e48
+  metadata.gz: c8b5d4de5b481f30fe65177f04b22d323cc697512d0a16077a44aefe393c821d1527343d369fc0a4a850cae725d7ef5a5f073c65c29d4cb49a558262e5c4decb
+  data.tar.gz: 6a487e4ca950684075ab462ce3cb9e04653cfa0e9af6888280866f08ca05653172ce86dcb89a8dd7bda1d4c5d4bd45f6f8cd106f428cca723ffe9787ff3d19b5

data/CHANGELOG.md

@@ -6,6 +6,40 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.15.0] - 2023-10-18
+
+### Added
+
+- `BackReferenceReadNode#name` is now provided.
+- `Index{Operator,And,Or}WriteNode` are introduced, split out from `Call{Operator,And,Or}WriteNode` when the method is `[]`.
+
+### Changed
+
+- Ensure `PM_NODE_FLAG_COMMON_MASK` into a constant expression to fix compile errors.
+- `super(&arg)` is now fixed.
+- Ensure the last encoding flag on regular expressions wins.
+- Fix the common whitespace calculation when embedded expressions begin on a line.
+- Capture groups in regular expressions now scan the unescaped version to get the correct local variables.
+- `*` and `&` are added to the local table when `...` is found in the parameters of a method definition.
+
+## [0.14.0] - 2023-10-13
+
+### Added
+
+- Syntax errors are added for invalid lambda local semicolon placement.
+- Lambda locals are now checked for duplicate names.
+- Destructured parameters are now checked for duplicate names.
+- `Constant{Read,Path,PathTarget}Node#full_name` and `Constant{Read,Path,PathTarget}Node#full_name_parts` are added to walk constant paths for you to find the full name of the constant.
+- Syntax errors are added when assigning to a numbered parameter.
+- `Node::type` is added, which matches the `Node#type` API.
+- Magic comments are now parsed as part of the parsing process and a new field is added in the form of `ParseResult#magic_comments` to access them.
+
+### Changed
+
+- **BREAKING**: `Call*Node#name` methods now return symbols instead of strings.
+- **BREAKING**: For loops now have their index value considered as part of the body, so depths of local variable assignments will be increased by 1.
+- Tilde heredocs now split up their lines into multiple string nodes to make them easier to dedent.
+
 ## [0.13.0] - 2023-09-29
 
 ### Added
@@ -161,7 +195,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/prism/compare/v0.13.0...HEAD
+[unreleased]: https://github.com/ruby/prism/compare/v0.15.0...HEAD
+[0.15.0]: https://github.com/ruby/prism/compare/v0.14.0...v0.15.0
+[0.14.0]: https://github.com/ruby/prism/compare/v0.13.0...v0.14.0
 [0.13.0]: https://github.com/ruby/prism/compare/v0.12.0...v0.13.0
 [0.12.0]: https://github.com/ruby/prism/compare/v0.11.0...v0.12.0
 [0.11.0]: https://github.com/ruby/prism/compare/v0.10.0...v0.11.0

data/README.md

@@ -1,4 +1,7 @@
-# Prism Ruby parser
+<h1 align="center">Prism Ruby parser</h1>
+<div align="center">
+  <img alt="Prism Ruby parser" height="256px" src="https://github.com/ruby/prism/blob/main/docs/prism.png?raw=true">
+</div>
 
 This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies. It is currently being integrated into [CRuby](https://github.com/ruby/ruby), [JRuby](https://github.com/jruby/jruby), [TruffleRuby](https://github.com/oracle/truffleruby), [Sorbet](https://github.com/sorbet/sorbet), and [Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree).
 

data/config.yml

@@ -361,6 +361,8 @@ flags:
         comment: "x - ignores whitespace and allows comments in regular expressions"
       - name: MULTI_LINE
         comment: "m - allows $ to match the end of lines within strings"
+      - name: ONCE
+        comment: "o - only interpolates values into the regular expression once"
       - name: EUC_JP
         comment: "e - forces the EUC-JP encoding"
       - name: ASCII_8BIT
@@ -369,12 +371,10 @@ flags:
         comment: "s - forces the Windows-31J encoding"
       - name: UTF_8
         comment: "u - forces the UTF-8 encoding"
-      - name: ONCE
-        comment: "o - only interpolates values into the regular expression once"
   - name: StringFlags
     values:
       - name: FROZEN
-        comment: "frozen by virtue of a frozen_string_literal comment"
+        comment: "frozen by virtue of a `frozen_string_literal` comment"
 nodes:
   - name: AliasGlobalVariableNode
     fields:
@@ -507,6 +507,9 @@ nodes:
           { **foo }
             ^^^^^
   - name: BackReferenceReadNode
+    fields:
+      - name: name
+        type: constant
     comment: |
       Represents reading a reference to a field in the previous match.
 
@@ -630,20 +633,13 @@ nodes:
         type: location?
       - name: message_loc
         type: location?
-      - name: opening_loc
-        type: location?
-      - name: arguments
-        type: node?
-        kind: ArgumentsNode
-      - name: closing_loc
-        type: location?
       - name: flags
         type: flags
         kind: CallNodeFlags
       - name: read_name
-        type: string
+        type: constant
       - name: write_name
-        type: string
+        type: constant
       - name: operator_loc
         type: location
       - name: value
@@ -674,7 +670,7 @@ nodes:
         type: flags
         kind: CallNodeFlags
       - name: name
-        type: string
+        type: constant
     comment: |
       Represents a method call, in all of the various forms that can take.
 
@@ -703,20 +699,13 @@ nodes:
         type: location?
       - name: message_loc
         type: location?
-      - name: opening_loc
-        type: location?
-      - name: arguments
-        type: node?
-        kind: ArgumentsNode
-      - name: closing_loc
-        type: location?
       - name: flags
         type: flags
         kind: CallNodeFlags
       - name: read_name
-        type: string
+        type: constant
       - name: write_name
-        type: string
+        type: constant
       - name: operator
         type: constant
       - name: operator_loc
@@ -736,20 +725,13 @@ nodes:
         type: location?
       - name: message_loc
         type: location?
-      - name: opening_loc
-        type: location?
-      - name: arguments
-        type: node?
-        kind: ArgumentsNode
-      - name: closing_loc
-        type: location?
       - name: flags
         type: flags
         kind: CallNodeFlags
       - name: read_name
-        type: string
+        type: constant
       - name: write_name
-        type: string
+        type: constant
       - name: operator_loc
         type: location
       - name: value
@@ -1443,6 +1425,89 @@ nodes:
 
           case a; in b then c end
                   ^^^^^^^^^^^
+  - name: IndexAndWriteNode
+    fields:
+      - name: receiver
+        type: node?
+      - name: call_operator_loc
+        type: location?
+      - name: opening_loc
+        type: location
+      - name: arguments
+        type: node?
+        kind: ArgumentsNode
+      - name: closing_loc
+        type: location
+      - name: block
+        type: node?
+      - name: flags
+        type: flags
+        kind: CallNodeFlags
+      - name: operator_loc
+        type: location
+      - name: value
+        type: node
+    comment: |
+      Represents the use of the `&&=` operator on a call to the `[]` method.
+
+          foo.bar[baz] &&= value
+          ^^^^^^^^^^^^^^^^^^^^^^
+  - name: IndexOperatorWriteNode
+    fields:
+      - name: receiver
+        type: node?
+      - name: call_operator_loc
+        type: location?
+      - name: opening_loc
+        type: location
+      - name: arguments
+        type: node?
+        kind: ArgumentsNode
+      - name: closing_loc
+        type: location
+      - name: block
+        type: node?
+      - name: flags
+        type: flags
+        kind: CallNodeFlags
+      - name: operator
+        type: constant
+      - name: operator_loc
+        type: location
+      - name: value
+        type: node
+    comment: |
+      Represents the use of an assignment operator on a call to `[]`.
+
+          foo.bar[baz] += value
+          ^^^^^^^^^^^^^^^^^^^^^
+  - name: IndexOrWriteNode
+    fields:
+      - name: receiver
+        type: node?
+      - name: call_operator_loc
+        type: location?
+      - name: opening_loc
+        type: location
+      - name: arguments
+        type: node?
+        kind: ArgumentsNode
+      - name: closing_loc
+        type: location
+      - name: block
+        type: node?
+      - name: flags
+        type: flags
+        kind: CallNodeFlags
+      - name: operator_loc
+        type: location
+      - name: value
+        type: node
+    comment: |
+      Represents the use of the `||=` operator on a call to `[]`.
+
+          foo.bar[baz] ||= value
+          ^^^^^^^^^^^^^^^^^^^^^^
   - name: InstanceVariableAndWriteNode
     fields:
       - name: name
@@ -1772,7 +1837,6 @@ nodes:
         type: location
       - name: content_loc
         type: location
-        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location
       - name: unescaped
@@ -2093,7 +2157,6 @@ nodes:
         type: location
       - name: content_loc
         type: location
-        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location
       - name: unescaped
@@ -2287,10 +2350,8 @@ nodes:
         kind: StringFlags
       - name: opening_loc
         type: location?
-        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: content_loc
         type: location
-        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location?
       - name: unescaped

data/docs/fuzzing.md

@@ -6,8 +6,7 @@ We use fuzzing to test the various entrypoints to the library. The fuzzer we use
 fuzz
 ├── corpus
 │   ├── parse             fuzzing corpus for parsing (a symlink to our fixtures)
-│   ├── regexp            fuzzing corpus for regexp
-│   └── unescape          fuzzing corpus for unescaping strings
+│   └── regexp            fuzzing corpus for regexp
 ├── dict                  a AFL++ dictionary containing various tokens
 ├── docker
 │   └── Dockerfile        for building a container with the fuzzer toolchain
@@ -17,11 +16,9 @@ fuzz
 ├── parse.sh              script to run parsing fuzzer
 ├── regexp.c              fuzz handler for regular expression parsing
 ├── regexp.sh             script to run regexp fuzzer
-├── tools
-│   ├── backtrace.sh      generates backtrace files for a crash directory
-│   └── minimize.sh       generates minimized crash or hang files
-├── unescape.c            fuzz handler for unescape functionality
-└── unescape.sh           script to run unescape fuzzer
+└── tools
+    ├── backtrace.sh      generates backtrace files for a crash directory
+    └── minimize.sh       generates minimized crash or hang files
 ```
 
 ## Usage
@@ -30,14 +27,12 @@ There are currently three fuzzing targets
 
 - `pm_parse_serialize` (parse)
 - `pm_regexp_named_capture_group_names` (regexp)
-- `pm_unescape_manipulate_string` (unescape)
 
 Respectively, fuzzing can be performed with
 
 ```
 make fuzz-run-parse
 make fuzz-run-regexp
-make fuzz-run-unescape
 ```
 
 To end a fuzzing job, interrupt with CTRL+C. To enter a container with the fuzzing toolchain and debug utilities, run
@@ -60,7 +55,7 @@ Note, that this may make reproducing bugs difficult as they may depend on memory
 
 ```
 make fuzz-debug # enter the docker container with build tools
-make build/fuzz.heisenbug.parse # or .unescape or .regexp
+make build/fuzz.heisenbug.parse # or .regexp
 ./build/fuzz.heisenbug.parse path-to-problem-input
 ```
 

data/docs/serialization.md

@@ -31,6 +31,7 @@ This drastically cuts down on the size of the serialized string, especially when
 ### comment
 
 The comment type is one of:
+
 * 0=`INLINE` (`# comment`)
 * 1=`EMBEDDED_DOCUMENT` (`=begin`/`=end`)
 * 2=`__END__` (after `__END__`)
@@ -40,6 +41,13 @@ The comment type is one of:
 | `1` | comment type |
 | location | the location in the source of this comment |
 
+### magic comment
+
+| # bytes | field |
+| --- | --- |
+| location | the location of the key of the magic comment |
+| location | the location of the value of the magic comment |
+
 ### diagnostic
 
 | # bytes | field |
@@ -66,6 +74,8 @@ The header is structured like the following table:
 | string | the encoding name |
 | varint | number of comments |
 | comment* | comments |
+| varint | number of magic comments |
+| magic comment* | magic comments |
 | varint | number of errors |
 | diagnostic* | errors |
 | varint | number of warnings |