prism 0.16.0 → 0.17.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 498f20248401af377faf45d30b7154d819b8bea560be2747c030d5ae536851c9
-  data.tar.gz: 8d3bb4fc0afb899869d8d6a55a526a40769fb46a582cf465035de750824d72d7
+  metadata.gz: 69cdca044f91ad2a198666562fdffd6035323906da465743ebff2cbd34ccac5d
+  data.tar.gz: 3da1460885f7cabda4b1ae6438274ab64a6e966536587d98eab2b3c764d0b6c0
 SHA512:
-  metadata.gz: 2d71744c4342a671b578ed7ebf3dede7b0146404a61e40ed8b082c544f5d8e9d83dc4e1f0780555cab4c70bd052156e26f9d8b46a2f2012bcedf89bf5b21d54b
-  data.tar.gz: 2a2178b5615fe4c55d7c16156901423fe38dee075e4f60cc893c367630e7c136cc7d6bc19f3fcf037a6958d158b2663f61389656754fe679570271503a5b35a2
+  metadata.gz: c01d8b62728fe1cbce99394d683c28b943b7962e6a1fc82d435b3190286612fc4d7e4aaab93a6bef7a75f8acdfa3b5a597acac3295e489a7dd89aada94b29b23
+  data.tar.gz: ce88826e2a46cb18fe5e89b1a7c18c8fba2387b9c9e5625ec9aac5e5bd1449b21f23624bd0a6b526c4a9fa156c679186f8b89c09d0a003d386663f4a6a33269e

data/CHANGELOG.md

@@ -6,6 +6,20 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.17.0] - 2023-11-03
+
+### Added
+
+- We now properly support forwarding arguments into arrays, like `def foo(*) = [*]`.
+- We now have much better documentation for the C and Ruby APIs.
+- We now properly provide an error message when attempting to assign to numbered parameters from within regular expression named capture groups, as in `/(?<_1>)/ =~ ""`.
+
+### Changed
+
+- **BREAKING**: `KeywordParameterNode` is split into `OptionalKeywordParameterNode` and `RequiredKeywordParameterNode`. `RequiredKeywordParameterNode` has no `value` field.
+- **BREAKING**: Most of the `Prism::` APIs now accept a bunch of keyword options. The options we now support are: `filepath`, `encoding`, `line`, `frozen_string_literal`, `verbose`, and `scopes`. See [the pull request](https://github.com/ruby/prism/pull/1763) for more details.
+- **BREAKING**: Comments are now split into three different classes instead of a single class, and the `type` field has been removed. They are: `InlineComment`, `EmbDocComment`, and `DATAComment`.
+
 ## [0.16.0] - 2023-10-30
 
 ### Added
@@ -219,7 +233,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/prism/compare/v0.16.0...HEAD
+[unreleased]: https://github.com/ruby/prism/compare/v0.17.0...HEAD
+[0.17.0]: https://github.com/ruby/prism/compare/v0.16.0...v0.17.0
 [0.16.0]: https://github.com/ruby/prism/compare/v0.15.1...v0.16.0
 [0.15.1]: https://github.com/ruby/prism/compare/v0.15.0...v0.15.1
 [0.15.0]: https://github.com/ruby/prism/compare/v0.14.0...v0.15.0

data/Makefile

@@ -88,3 +88,9 @@ clean:
 all-no-debug: DEBUG_FLAGS := -DNDEBUG=1
 all-no-debug: OPTFLAGS := -O3
 all-no-debug: all
+
+run: Makefile $(STATIC_OBJECTS) $(HEADERS) test.c
+	$(ECHO) "compiling test.c"
+	$(Q) $(CC) $(CPPFLAGS) $(CFLAGS) $(STATIC_OBJECTS) test.c
+	$(ECHO) "running test.c"
+	$(Q) ./a.out

data/README.md

@@ -1,6 +1,6 @@
 <h1 align="center">Prism Ruby parser</h1>
 <div align="center">
-  <img alt="Prism Ruby parser" height="256px" src="https://github.com/ruby/prism/blob/main/docs/prism.png?raw=true">
+  <img alt="Prism Ruby parser" height="256px" src="https://github.com/ruby/prism/blob/main/doc/images/prism.png?raw=true">
 </div>
 
 This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies. It is currently being integrated into [CRuby](https://github.com/ruby/ruby), [JRuby](https://github.com/jruby/jruby), [TruffleRuby](https://github.com/oracle/truffleruby), [Sorbet](https://github.com/sorbet/sorbet), and [Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree).

data/config.yml

@@ -59,11 +59,11 @@ tokens:
   - name: CONSTANT
     comment: "a constant"
   - name: DOT
-    comment: "."
+    comment: "the . call operator"
   - name: DOT_DOT
-    comment: ".."
+    comment: "the .. range operator"
   - name: DOT_DOT_DOT
-    comment: "..."
+    comment: "the ... range operator or forwarding parameter"
   - name: EMBDOC_BEGIN
     comment: "=begin"
   - name: EMBDOC_END
@@ -311,9 +311,9 @@ tokens:
   - name: UCOLON_COLON
     comment: "unary ::"
   - name: UDOT_DOT
-    comment: "unary .."
+    comment: "unary .. operator"
   - name: UDOT_DOT_DOT
-    comment: "unary ..."
+    comment: "unary ... operator"
   - name: UMINUS
     comment: "-@"
   - name: UMINUS_NUM
@@ -333,12 +333,14 @@ flags:
     values:
       - name: KEYWORD_SPLAT
         comment: "if arguments contain keyword splat"
+    comment: Flags for arguments nodes.
   - name: CallNodeFlags
     values:
       - name: SAFE_NAVIGATION
         comment: "&. operator"
       - name: VARIABLE_CALL
         comment: "a call that could have been a local variable"
+    comment: Flags for call nodes.
   - name: IntegerBaseFlags
     values:
       - name: BINARY
@@ -349,14 +351,17 @@ flags:
         comment: "0d or no prefix"
       - name: HEXADECIMAL
         comment: "0x prefix"
+    comment: Flags for integer nodes that correspond to the base of the integer.
   - name: LoopFlags
     values:
       - name: BEGIN_MODIFIER
         comment: "a loop after a begin statement, so the body is executed first before the condition"
+    comment: Flags for while and until loop nodes.
   - name: RangeFlags
     values:
       - name: EXCLUDE_END
         comment: "... operator"
+    comment: Flags for range and flip-flop nodes.
   - name: RegularExpressionFlags
     values:
       - name: IGNORE_CASE
@@ -375,10 +380,12 @@ flags:
         comment: "s - forces the Windows-31J encoding"
       - name: UTF_8
         comment: "u - forces the UTF-8 encoding"
+    comment: Flags for regular expression and match last line nodes.
   - name: StringFlags
     values:
       - name: FROZEN
         comment: "frozen by virtue of a `frozen_string_literal` comment"
+    comment: Flags for string nodes.
 nodes:
   - name: AliasGlobalVariableNode
     fields:
@@ -777,10 +784,10 @@ nodes:
     comment: |
       Represents the use of a case statement.
 
-      case true
-      ^^^^^^^^^
-      when false
-      end
+          case true
+          when false
+          end
+          ^^^^^^^^^^
   - name: ClassNode
     fields:
       - name: locals
@@ -818,7 +825,7 @@ nodes:
       Represents the use of the `&&=` operator for assignment to a class variable.
 
           @@target &&= value
-          ^^^^^^^^^^^^^^^^
+          ^^^^^^^^^^^^^^^^^^
   - name: ClassVariableOperatorWriteNode
     fields:
       - name: name
@@ -1183,13 +1190,13 @@ nodes:
       Represents a find pattern in pattern matching.
 
           foo in *bar, baz, *qux
-          ^^^^^^^^^^^^^^^^^^^^^^
+                 ^^^^^^^^^^^^^^^
 
           foo in [*bar, baz, *qux]
-          ^^^^^^^^^^^^^^^^^^^^^^^^
+                 ^^^^^^^^^^^^^^^^^
 
           foo in Foo(*bar, baz, *qux)
-          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+                 ^^^^^^^^^^^^^^^^^^^^
   - name: FlipFlopNode
     fields:
       - name: left
@@ -1240,7 +1247,7 @@ nodes:
 
           def foo(...)
             bar(...)
-            ^^^^^^^^
+                ^^^
           end
   - name: ForwardingParameterNode
     comment: |
@@ -1692,24 +1699,6 @@ nodes:
 
           foo(a: b)
               ^^^^
-  - name: KeywordParameterNode
-    fields:
-      - name: name
-        type: constant
-      - name: name_loc
-        type: location
-      - name: value
-        type: node?
-    comment: |
-      Represents a keyword parameter to a method, block, or lambda definition.
-
-          def a(b:)
-                ^^
-          end
-
-          def a(b: 1)
-                ^^^^
-          end
   - name: KeywordRestParameterNode
     fields:
       - name: name
@@ -1997,6 +1986,20 @@ nodes:
 
           $1
           ^^
+  - name: OptionalKeywordParameterNode
+    fields:
+      - name: name
+        type: constant
+      - name: name_loc
+        type: location
+      - name: value
+        type: node
+    comment: |
+      Represents an optional keyword parameter to a method, block, or lambda definition.
+
+          def a(b: 1)
+                ^^^^
+          end
   - name: OptionalParameterNode
     fields:
       - name: name
@@ -2184,6 +2187,18 @@ nodes:
 
           /foo/i
           ^^^^^^
+  - name: RequiredKeywordParameterNode
+    fields:
+      - name: name
+        type: constant
+      - name: name_loc
+        type: location
+    comment: |
+      Represents a required keyword parameter to a method, block, or lambda definition.
+
+          def a(b: )
+                ^^
+          end
   - name: RequiredParameterNode
     fields:
       - name: name
@@ -2206,8 +2221,8 @@ nodes:
     comment: |
       Represents an expression modified with a rescue.
 
-        foo rescue nil
-        ^^^^^^^^^^^^^^
+          foo rescue nil
+          ^^^^^^^^^^^^^^
   - name: RescueNode
     fields:
       - name: keyword_loc
@@ -2229,8 +2244,8 @@ nodes:
 
           begin
           rescue Foo, *splat, Bar => ex
-          ^^^^^^
             foo
+          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           end
 
       `Foo, *splat, Bar` are in the `exceptions` field.

data/docs/fuzzing.md

@@ -25,7 +25,7 @@ fuzz
 
 There are currently three fuzzing targets
 
-- `pm_parse_serialize` (parse)
+- `pm_serialize_parse` (parse)
 - `pm_regexp_named_capture_group_names` (regexp)
 
 Respectively, fuzzing can be performed with

data/docs/serialization.md

@@ -72,6 +72,7 @@ The header is structured like the following table:
 | `1` | patch version number |
 | `1` | 1 indicates only semantics fields were serialized, 0 indicates all fields were serialized (including location fields) |
 | string | the encoding name |
+| varint | the start line |
 | varint | number of comments |
 | comment* | comments |
 | varint | number of magic comments |
@@ -136,56 +137,54 @@ typedef struct {
   size_t capacity;
 } pm_buffer_t;
 
-// Initialize a pm_buffer_t with its default values.
-bool pm_buffer_init(pm_buffer_t *);
-
 // Free the memory associated with the buffer.
 void pm_buffer_free(pm_buffer_t *);
 
 // Parse and serialize the AST represented by the given source to the given
 // buffer.
-void pm_parse_serialize(const uint8_t *source, size_t length, pm_buffer_t *buffer, const char *metadata);
+void pm_serialize_parse(pm_buffer_t *buffer, const uint8_t *source, size_t length, const char *data);
 ```
 
-Typically you would use a stack-allocated `pm_buffer_t` and call `pm_parse_serialize`, as in:
+Typically you would use a stack-allocated `pm_buffer_t` and call `pm_serialize_parse`, as in:
 
 ```c
 void
 serialize(const uint8_t *source, size_t length) {
-  pm_buffer_t buffer;
-  if (!pm_buffer_init(&buffer)) return;
+  pm_buffer_t buffer = { 0 };
+  pm_serialize_parse(&buffer, source, length, NULL);
 
-  pm_parse_serialize(source, length, &buffer, NULL);
   // Do something with the serialized string.
 
   pm_buffer_free(&buffer);
 }
 ```
 
-The final argument to `pm_parse_serialize` controls the metadata of the source.
-This includes the filepath that the source is associated with, and any nested local variables scopes that are necessary to properly parse the file (in the case of parsing an `eval`).
-Note that no `varint` are used here to make it easier to produce the metadata for the caller, and also serialized size is less important here.
-The metadata is a serialized format itself, and is structured as follows:
+The final argument to `pm_serialize_parse` is an optional string that controls the options to the parse function. This includes all of the normal options that could be passed to `pm_parser_init` through a `pm_options_t` struct, but serialized as a string to make it easier for callers through FFI. Note that no `varint` are used here to make it easier to produce the data for the caller, and also serialized size is less important here. The format of the data is structured as follows:
 
-| # bytes | field |
-| --- | --- |
-| `4` | the size of the filepath string |
-| | the filepath string |
-| `4` | the number of local variable scopes |
+| # bytes | field                      |
+| ------- | -------------------------- |
+| `4`     | the length of the filepath |
+| ...     | the filepath bytes         |
+| `4`     | the line number            |
+| `4`     | the length the encoding    |
+| ...     | the encoding bytes         |
+| `1`     | frozen string literal      |
+| `1`     | suppress warnings          |
+| `4`     | the number of scopes       |
+| ...     | the scopes                 |
 
-Then, each local variable scope is encoded as:
+Each scope is layed out as follows:
 
-| # bytes | field |
-| --- | --- |
-| `4` | the number of local variables in the scope |
-| | the local variables |
+| # bytes | field                      |
+| ------- | -------------------------- |
+| `4`     | the number of locals       |
+| ...     | the locals                 |
 
-Each local variable within each scope is encoded as:
+Each local is layed out as follows:
 
-| # bytes | field |
-| --- | --- |
-| `4` | the size of the local variable name |
-| | the local variable name |
+| # bytes | field                      |
+| ------- | -------------------------- |
+| `4`     | the length of the local    |
+| ...     | the local bytes            |
 
-The metadata can be `NULL` (as seen in the example above).
-If it is not null, then a minimal metadata string would be `"\0\0\0\0\0\0\0\0"` which would use 4 bytes to indicate an empty filepath string and 4 bytes to indicate that there were no local variable scopes.
+The data can be `NULL` (as seen in the example above).