yarp 0.10.0 → 0.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d7fbd6801f30b7ad185ea9884ad6ab4a0b0fd07494dbf96d90a560d7c5223a3
-  data.tar.gz: a28e10e86eee28950106cea5be18020349ea5181111f4d45d540322677b5eed4
+  metadata.gz: 9afbc3b5f4c070c404f0421f1814d5a1200cb849ef9bf38b2ae23f50ba738fdf
+  data.tar.gz: bed121fb1ac414cf5918a2bcf7df2a9b1318df2f46dff9863cc5b4427ce79409
 SHA512:
-  metadata.gz: c30fdd9590910182706dc3d6f0747ce8c467156ede3c2ed727d8a2efb2aa7f821abd2e5465e80ddfe122ba80c03af23ab1d60c50f7a39348d797106c9c4567d7
-  data.tar.gz: 8c877d31c11834e6e0d22b6acab729258cd87ce4a53e026c8b0ff4b92f98f90a0daa57b21811a29d3c4e2387a65b4a3935d165a4311985d40b3511f95c94f280
+  metadata.gz: 3fd5831ab86ca1ca299e86ca5f2ca184164fa56de353cfa0d9b51d2ece522c2f5bebfadd1896222e4b7e1e984604be414ee8900b1a5bbc0b8eb3b6ee7bb738dc
+  data.tar.gz: 6fa02c77777391a1c4b5dc47445d905fee835213c7c02687cac432a4b566bbae8ead26467609066e83bf030ce19e09f46cf553fc0acdd8f7ef977074568b7061

data/CHANGELOG.md

@@ -6,6 +6,53 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.12.0] - 2023-09-15
+
+### Added
+
+- `RegularExpressionNode#options` and `InterpolatedRegularExpressionNode#options` are now provided. These return integers that match up to the `Regexp#options` API.
+- Greatly improved `Node#inspect` and `Node#pretty_print` APIs.
+- `MatchLastLineNode` and `InterpolatedMatchLastLineNode` are introduced to represent using a regular expression as the predicate of an `if` or `unless` statement.
+- `IntegerNode` now has a base flag on it.
+- Heredocs that were previously `InterpolatedStringNode` and `InterpolatedXStringNode` nodes without any actual interpolation are now `StringNode` and `XStringNode`, respectively.
+- `StringNode` now has a `frozen?` flag on it, which respects the `frozen_string_literal` magic comment.
+- Numbered parameters are now supported, and are properly represented using `LocalVariableReadNode` nodes.
+- `ImplicitNode` is introduced, which wraps implicit calls, local variable reads, or constant reads in omitted hash values.
+- `YARP::Dispatcher` is introduced, which provides a way for multiple objects to listen for certain events on the AST while it is being walked. This is effectively a way to implement a more efficient visitor pattern when you have many different uses for the AST.
+
+### Changed
+
+- **BREAKING**: Flags fields are now marked as private, to ensure we can change their implementation under the hood. Actually querying should be through the accessor methods.
+- **BREAKING**: `AliasNode` is now split into `AliasMethodNode` and `AliasGlobalVariableNode`.
+- Method definitions on local variables is now correctly handled.
+- Unary minus precedence has been fixed.
+- Concatenating character literals with string literals is now fixed.
+- Many more invalid syntaxes are now properly rejected.
+- **BREAKING**: Comments now no longer include their trailing newline.
+
+## [0.11.0] - 2023-09-08
+
+### Added
+
+- `Node#inspect` is much improved.
+- `YARP::Pattern` is introduced, which can construct procs to match against nodes.
+- `BlockLocalVariableNode` is introduced to take the place of the locations array on `BlockParametersNode`.
+- `ParseResult#attach_comments!` is now provided to attach comments to locations in the tree.
+- `MultiTargetNode` is introduced as the target of multi writes and for loops.
+- `Node#comment_targets` is introduced to return the list of objects that can have attached comments.
+
+### Changed
+
+- **BREAKING**: `GlobalVariable*Node#name` now returns a symbol.
+- **BREAKING**: `Constant*Node#name` now returns a symbol.
+- **BREAKING**: `BlockParameterNode`, `KeywordParameterNode`, `KeywordRestParameterNode`, `RestParameterNode`, `DefNode` all have their `name` methods returning symbols now.
+- **BREAKING**: `ClassNode#name` and `ModuleNode#name` now return symbols.
+- **BREAKING**: `Location#end_column` is now exclusive instead of inclusive.
+- `Location#slice` now returns a properly encoded string.
+- `CallNode#operator_loc` is now `CallNode#call_operator_loc`.
+- `CallOperatorAndWriteNode` is renamed to `CallAndWriteNode` and its structure has changed.
+- `CallOperatorOrWriteNode` is renamed to `CallOrWriteNode` and its structure has changed.
+
 ## [0.10.0] - 2023-09-01
 
 ### Added
@@ -94,7 +141,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/yarp/compare/v0.10.0...HEAD
+[unreleased]: https://github.com/ruby/yarp/compare/v0.12.0...HEAD
+[0.12.0]: https://github.com/ruby/yarp/compare/v0.11.0...v0.12.0
+[0.11.0]: https://github.com/ruby/yarp/compare/v0.10.0...v0.11.0
 [0.10.0]: https://github.com/ruby/yarp/compare/v0.9.0...v0.10.0
 [0.9.0]: https://github.com/ruby/yarp/compare/v0.8.0...v0.9.0
 [0.8.0]: https://github.com/ruby/yarp/compare/v0.7.0...v0.8.0

data/CONTRIBUTING.md

@@ -32,6 +32,13 @@ bundle exec rake compile test
 bundle exec rake compile:yarp test
 ```
 
+To test the rust bindings (with caveats about setting up your Rust environment properly first):
+
+``` sh
+bundle exec rake compile test:rust
+```
+
+
 ## Documentation
 
 We could always use more documentation! If you want to contribute documentation, feel free to open a pull request. These will get merged in as soon as possible. Documenting functions or methods is always useful, but we also need more guides and tutorials. If you have an idea for a guide or tutorial, feel free to open an issue and we can discuss it.

data/config.yml

@@ -333,6 +333,16 @@ flags:
         comment: "&. operator"
       - name: VARIABLE_CALL
         comment: "a call that could have been a local variable"
+  - name: IntegerBaseFlags
+    values:
+      - name: BINARY
+        comment: "0b prefix"
+      - name: OCTAL
+        comment: "0o or 0 prefix"
+      - name: DECIMAL
+        comment: "0d or no prefix"
+      - name: HEXADECIMAL
+        comment: "0x prefix"
   - name: LoopFlags
     values:
       - name: BEGIN_MODIFIER
@@ -345,10 +355,10 @@ flags:
     values:
       - name: IGNORE_CASE
         comment: "i - ignores the case of characters when matching"
-      - name: MULTI_LINE
-        comment: "m - allows $ to match the end of lines within strings"
       - name: EXTENDED
         comment: "x - ignores whitespace and allows comments in regular expressions"
+      - name: MULTI_LINE
+        comment: "m - allows $ to match the end of lines within strings"
       - name: EUC_JP
         comment: "e - forces the EUC-JP encoding"
       - name: ASCII_8BIT
@@ -359,8 +369,25 @@ flags:
         comment: "u - forces the UTF-8 encoding"
       - name: ONCE
         comment: "o - only interpolates values into the regular expression once"
+  - name: StringFlags
+    values:
+      - name: FROZEN
+        comment: "frozen by virtue of a frozen_string_literal comment"
 nodes:
-  - name: AliasNode
+  - name: AliasGlobalVariableNode
+    fields:
+      - name: new_name
+        type: node
+      - name: old_name
+        type: node
+      - name: keyword_loc
+        type: location
+    comment: |
+      Represents the use of the `alias` keyword to alias a global variable.
+
+          alias $foo $bar
+          ^^^^^^^^^^^^^^^
+  - name: AliasMethodNode
     fields:
       - name: new_name
         type: node
@@ -369,7 +396,7 @@ nodes:
       - name: keyword_loc
         type: location
     comment: |
-      Represents the use of the `alias` keyword.
+      Represents the use of the `alias` keyword to alias a method.
 
           alias foo bar
           ^^^^^^^^^^^^^
@@ -520,6 +547,15 @@ nodes:
 
           bar(&args)
           ^^^^^^^^^^
+  - name: BlockLocalVariableNode
+    fields:
+      - name: name
+        type: constant
+    comment: |
+      Represents a block local variable.
+
+          a { |; b| }
+                 ^
   - name: BlockNode
     fields:
       - name: locals
@@ -540,6 +576,8 @@ nodes:
                      ^^^^^^^^^^^^^^
   - name: BlockParameterNode
     fields:
+      - name: name
+        type: constant?
       - name: name_loc
         type: location?
       - name: operator_loc
@@ -556,7 +594,7 @@ nodes:
         type: node?
         kind: ParametersNode
       - name: locals
-        type: location[]
+        type: node[]
       - name: opening_loc
         type: location?
       - name: closing_loc
@@ -582,11 +620,42 @@ nodes:
 
           break foo
           ^^^^^^^^^
-  - name: CallNode
+  - name: CallAndWriteNode
     fields:
       - name: receiver
         type: node?
+      - name: call_operator_loc
+        type: location?
+      - name: message_loc
+        type: location?
+      - name: opening_loc
+        type: location?
+      - name: arguments
+        type: node?
+        kind: ArgumentsNode
+      - name: closing_loc
+        type: location?
+      - name: flags
+        type: flags
+        kind: CallNodeFlags
+      - name: read_name
+        type: string
+      - name: write_name
+        type: string
       - name: operator_loc
+        type: location
+      - name: value
+        type: node
+    comment: |
+      Represents the use of the `&&=` operator on a call.
+
+          foo.bar &&= value
+          ^^^^^^^^^^^^^^^^^
+  - name: CallNode
+    fields:
+      - name: receiver
+        type: node?
+      - name: call_operator_loc
         type: location?
       - name: message_loc
         type: location?
@@ -625,50 +694,70 @@ nodes:
 
           foo&.bar
           ^^^^^^^^
-  - name: CallOperatorAndWriteNode
+  - name: CallOperatorWriteNode
     fields:
-      - name: target
-        type: node
-        kind: CallNode
+      - name: receiver
+        type: node?
+      - name: call_operator_loc
+        type: location?
+      - name: message_loc
+        type: location?
+      - name: opening_loc
+        type: location?
+      - name: arguments
+        type: node?
+        kind: ArgumentsNode
+      - name: closing_loc
+        type: location?
+      - name: flags
+        type: flags
+        kind: CallNodeFlags
+      - name: read_name
+        type: string
+      - name: write_name
+        type: string
+      - name: operator
+        type: constant
       - name: operator_loc
         type: location
       - name: value
         type: node
     comment: |
-      Represents the use of the `&&=` operator on a call.
+      Represents the use of an assignment operator on a call.
 
-          foo.bar &&= value
-          ^^^^^^^^^^^^^^^^^
-  - name: CallOperatorOrWriteNode
+          foo.bar += baz
+          ^^^^^^^^^^^^^^
+  - name: CallOrWriteNode
     fields:
-      - name: target
-        type: node
-        kind: CallNode
-      - name: value
-        type: node
+      - name: receiver
+        type: node?
+      - name: call_operator_loc
+        type: location?
+      - name: message_loc
+        type: location?
+      - name: opening_loc
+        type: location?
+      - name: arguments
+        type: node?
+        kind: ArgumentsNode
+      - name: closing_loc
+        type: location?
+      - name: flags
+        type: flags
+        kind: CallNodeFlags
+      - name: read_name
+        type: string
+      - name: write_name
+        type: string
       - name: operator_loc
         type: location
+      - name: value
+        type: node
     comment: |
       Represents the use of the `||=` operator on a call.
 
           foo.bar ||= value
           ^^^^^^^^^^^^^^^^^
-  - name: CallOperatorWriteNode
-    fields:
-      - name: target
-        type: node
-        kind: CallNode
-      - name: operator_loc
-        type: location
-      - name: value
-        type: node
-      - name: operator
-        type: constant
-    comment: |
-      Represents the use of an assignment operator on a call.
-
-          foo.bar += baz
-          ^^^^^^^^^^^^^^
   - name: CapturePatternNode
     fields:
       - name: value
@@ -719,7 +808,7 @@ nodes:
       - name: end_keyword_loc
         type: location
       - name: name
-        type: string
+        type: constant
     comment: |
       Represents a class declaration involving the `class` keyword.
 
@@ -797,7 +886,7 @@ nodes:
       - name: name_loc
         type: location
       - name: value
-        type: node?
+        type: node
       - name: operator_loc
         type: location?
     comment: |
@@ -807,6 +896,8 @@ nodes:
           ^^^^^^^^^
   - name: ConstantAndWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: operator_loc
@@ -820,6 +911,8 @@ nodes:
           ^^^^^^^^^^^^^^^^
   - name: ConstantOperatorWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: operator_loc
@@ -835,6 +928,8 @@ nodes:
           ^^^^^^^^^^^^^^^
   - name: ConstantOrWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: operator_loc
@@ -937,12 +1032,18 @@ nodes:
           ::Foo::Bar = 1
           ^^^^^^^^^^^^^^
   - name: ConstantReadNode
+    fields:
+      - name: name
+        type: constant
     comment: |
       Represents referencing a constant.
 
           Foo
           ^^^
   - name: ConstantTargetNode
+    fields:
+      - name: name
+        type: constant
     comment: |
       Represents writing to a constant in a context that doesn't have an explicit value.
 
@@ -950,6 +1051,8 @@ nodes:
           ^^^  ^^^
   - name: ConstantWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: value
@@ -963,6 +1066,8 @@ nodes:
           ^^^^^^^
   - name: DefNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: receiver
@@ -1166,6 +1271,8 @@ nodes:
           ^^^^^
   - name: GlobalVariableAndWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: operator_loc
@@ -1179,6 +1286,8 @@ nodes:
           ^^^^^^^^^^^^^^^^^
   - name: GlobalVariableOperatorWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: operator_loc
@@ -1194,6 +1303,8 @@ nodes:
           ^^^^^^^^^^^^^^^^
   - name: GlobalVariableOrWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: operator_loc
@@ -1206,12 +1317,18 @@ nodes:
           $target ||= value
           ^^^^^^^^^^^^^^^^^
   - name: GlobalVariableReadNode
+    fields:
+      - name: name
+        type: constant
     comment: |
       Represents referencing a global variable.
 
           $foo
           ^^^^
   - name: GlobalVariableTargetNode
+    fields:
+      - name: name
+        type: constant
     comment: |
       Represents writing to a global variable in a context that doesn't have an explicit value.
 
@@ -1219,6 +1336,8 @@ nodes:
           ^^^^  ^^^^
   - name: GlobalVariableWriteNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: value
@@ -1294,6 +1413,19 @@ nodes:
 
           1.0i
           ^^^^
+  - name: ImplicitNode
+    fields:
+      - name: value
+        type: node
+    comment: |
+      Represents a node that is implicitly being added to the tree but doesn't
+      correspond directly to a node in the source.
+
+          { foo: }
+            ^^^^
+
+          { Foo: }
+            ^^^^
   - name: InNode
     fields:
       - name: pattern
@@ -1391,11 +1523,34 @@ nodes:
           @foo = 1
           ^^^^^^^^
   - name: IntegerNode
+    fields:
+      - name: flags
+        type: flags
+        kind: IntegerBaseFlags
     comment: |
       Represents an integer number literal.
 
           1
           ^
+  - name: InterpolatedMatchLastLineNode
+    fields:
+      - name: opening_loc
+        type: location
+      - name: parts
+        type: node[]
+      - name: closing_loc
+        type: location
+      - name: flags
+        type: flags
+        kind: RegularExpressionFlags
+    newline: parts
+    comment: |
+      Represents a regular expression literal that contains interpolation that
+      is being used in the predicate of a conditional to implicitly match
+      against the last line read by an IO object.
+
+          if /foo #{bar} baz/ then end
+             ^^^^^^^^^^^^^^^^
   - name: InterpolatedRegularExpressionNode
     fields:
       - name: opening_loc
@@ -1466,6 +1621,8 @@ nodes:
               ^^^^
   - name: KeywordParameterNode
     fields:
+      - name: name
+        type: constant
       - name: name_loc
         type: location
       - name: value
@@ -1482,10 +1639,12 @@ nodes:
           end
   - name: KeywordRestParameterNode
     fields:
-      - name: operator_loc
-        type: location
+      - name: name
+        type: constant?
       - name: name_loc
         type: location?
+      - name: operator_loc
+        type: location
     comment: |
       Represents a keyword rest parameter to a method, block, or lambda definition.
 
@@ -1606,6 +1765,26 @@ nodes:
 
           foo = 1
           ^^^^^^^
+  - name: MatchLastLineNode
+    fields:
+      - name: opening_loc
+        type: location
+      - name: content_loc
+        type: location
+      - name: closing_loc
+        type: location
+      - name: unescaped
+        type: string
+      - name: flags
+        type: flags
+        kind: RegularExpressionFlags
+    comment: |
+      Represents a regular expression literal used in the predicate of a
+      conditional to implicitly match against the last line read by an IO
+      object.
+
+          if /foo/i then end
+             ^^^^^^
   - name: MatchPredicateNode
     fields:
       - name: value
@@ -1632,6 +1811,19 @@ nodes:
 
           foo => bar
           ^^^^^^^^^^
+  - name: MatchWriteNode
+    fields:
+      - name: call
+        type: node
+        kind: CallNode
+      - name: locals
+        type: constant[]
+    comment: |
+      Represents writing local variables using a regular expression match with
+      named capture groups.
+
+          /(?<foo>bar)/ =~ baz
+          ^^^^^^^^^^^^^^^^^^^^
   - name: MissingNode
     comment: |
       Represents a node that is missing from the source and results in a syntax
@@ -1649,20 +1841,16 @@ nodes:
       - name: end_keyword_loc
         type: location
       - name: name
-        type: string
+        type: constant
     comment: |
       Represents a module declaration involving the `module` keyword.
 
           module Foo end
           ^^^^^^^^^^^^^^
-  - name: MultiWriteNode
+  - name: MultiTargetNode
     fields:
       - name: targets
         type: node[]
-      - name: operator_loc
-        type: location?
-      - name: value
-        type: node?
       - name: lparen_loc
         type: location?
       - name: rparen_loc
@@ -1670,6 +1858,23 @@ nodes:
     comment: |
       Represents a multi-target expression.
 
+          a, b, c = 1, 2, 3
+          ^^^^^^^
+  - name: MultiWriteNode
+    fields:
+      - name: targets
+        type: node[]
+      - name: lparen_loc
+        type: location?
+      - name: rparen_loc
+        type: location?
+      - name: operator_loc
+        type: location
+      - name: value
+        type: node
+    comment: |
+      Represents a write to a multi-target expression.
+
           a, b, c = 1, 2, 3
           ^^^^^^^^^^^^^^^^^
   - name: NextNode
@@ -1746,11 +1951,11 @@ nodes:
         type: node[]
       - name: optionals
         type: node[]
-      - name: posts
-        type: node[]
       - name: rest
         type: node?
         kind: RestParameterNode
+      - name: posts
+        type: node[]
       - name: keywords
         type: node[]
       - name: keyword_rest
@@ -1965,10 +2170,12 @@ nodes:
       `ex` is in the `exception` field.
   - name: RestParameterNode
     fields:
-      - name: operator_loc
-        type: location
+      - name: name
+        type: constant?
       - name: name_loc
         type: location?
+      - name: operator_loc
+        type: location
     comment: |
       Represents a rest parameter to a method, block, or lambda definition.
 
@@ -2072,6 +2279,9 @@ nodes:
           ^^^^^^^^^^^
   - name: StringNode
     fields:
+      - name: flags
+        type: flags
+        kind: StringFlags
       - name: opening_loc
         type: location?
       - name: content_loc

data/docs/configuration.md

@@ -51,7 +51,6 @@ The available values for `type` are:
 * `constant[]` - A child node that is an array of constants. This is a `yp_constant_id_list_t` in C.
 * `location` - A child node that is a location. This is a `yp_location_t` in C.
 * `location?` - A child node that is a location that is optionally present. This is a `yp_location_t` in C, but if the value is not present then the `start` and `end` fields will be `NULL`.
-* `location[]` - A child node that is an array of locations. This is a `yp_location_list_t` in C.
 * `uint32` - A child node that is a 32-bit unsigned integer. This is a `uint32_t` in C.
 
 If the type is `node` or `node?` then the value also accepts an optional `kind` key (a string). This key is expected to match to the name of another node type within `config.yml`. This changes a couple of places where code is templated out to use the more specific struct name instead of the generic `yp_node_t`. For example, with `kind: StatementsNode` the `yp_node_t *` in C becomes a `yp_statements_node_t *`.