yarp 0.7.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 81d6e0266ee0d13563faafa47a0c0d09c5feabe870e2324f32564a7f68299dff
-  data.tar.gz: 771e9fd99caaf99cf7493d897cf9fdd5f93824e90bf09c22a1e5b5ec0e1a4bcf
+  metadata.gz: '028bafce371c205e4f9a845aa0328a8911519b064ec92be9a970c0077088ca5c'
+  data.tar.gz: ed653e7a3734884de2f42d1df304be7eb1532b29d7acf63214c747ee84426ec0
 SHA512:
-  metadata.gz: 16525d9ed87d00d9fd94af5cab5f99b8278ac4a81c7117195e80d3cd3cc6b12358fd0f2255cbe7df15ef1adef609719371bc3a210d8a2273d6e34fbdcb4b1a58
-  data.tar.gz: efbf02ccd9a7cb6a13b6ba56dc8fc3fc6e425d7d779583614fc9c24c74216fe3d89eadc83999f197a7227bfd90ca8ba9446c1b71c128e4874609f44f0869eb72
+  metadata.gz: e33c681424d19626c0ee1d5c21265bcdfce4ec56909a5707d4bf7025ea5356b5ac41b080a1e54a8490124065a1595bd5833af8e488558f0ff01c4f14a92d0430
+  data.tar.gz: d55c0483e944f9a33e76a2baeef11841da8f8bf8755512ff90497d07b0fedf2dc568f6ad8c384831b6b382b1482e34177c6dbd8314d6570542c59faca71ad23e

data/CHANGELOG.md

@@ -6,6 +6,56 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.9.0] - 2023-08-25
+
+### Added
+
+- Regular expressions can now be bound by `\n`, `\r`, and a combination of `\r\n`.
+- Strings delimited by `%`, `%q`, and `%Q` can now be bound by `\n`, `\r`, and a combination of `\r\n`.
+- `IntegerNode#value` now returns the value of the integer as a Ruby `Integer`.
+- `FloatNode#value` now returns the value of the float as a Ruby `Float`.
+- `RationalNode#value` now returns the value of the rational as a Ruby `Rational`.
+- `ImaginaryNode#value` now returns the value of the imaginary as a Ruby `Complex`.
+- `ClassNode#name` is now a string that returns the name of just the class, without the namespace.
+- `ModuleNode#name` is now a string that returns the name of just the module, without the namespace.
+- Regular expressions and strings found after a heredoc declaration but before the heredoc body are now parsed correctly.
+- The serialization API now supports shared strings, which should help reduce the size of the serialized AST.
+- `*Node#copy` is introduced, which returns a copy of the node with the given overrides.
+- `Location#copy` is introduced, which returns a copy of the location with the given overrides.
+- `DesugarVisitor` is introduced, which provides a simpler AST for use in tools that want to process fewer node types.
+- `{ClassVariable,Constant,ConstantPath,GlobalVariable,InstanceVariable,LocalVariable}TargetNode` are introduced. These nodes represent the target of writes in locations where a value cannot be provided, like a multi write or a rescue reference.
+- `UntilNode#closing_loc` and `WhileNode#closing_loc` are now provided.
+- `Location#join` is now provided, which joins two locations together.
+- `YARP::parse_lex` and `YARP::parse_lex_file` are introduced to parse and lex in one result.
+
+### Changed
+
+- When there is a magic encoding comment, the encoding of the first token's source string is now properly reencoded.
+- Constants followed by unary `&` are now properly parsed as a call with a passed block argument.
+- Escaping multi-byte characters in a string literal will now properly escape the entire character.
+- `YARP.lex_compat` now has more accurate behavior when a byte-order mark is present in the file.
+- **BREAKING**: `AndWriteNode`, `OrWriteNode`, and `OperatorWriteNode` have been split back up into their `0.7.0` versions.
+- We now properly support spaces between the `encoding` and `=`/`:` in a magic encoding comment.
+- We now properly parse `-> foo: bar do end`.
+
+## [0.8.0] - 2023-08-18
+
+### Added
+
+- Some performance improvements when converting from the C AST to the Ruby AST.
+- Two rust crates have been added: `yarp-sys` and `yarp`. They are as yet unpublished.
+
+### Changed
+
+- Escaped newlines in strings and heredocs are now handled more correctly.
+- Dedenting heredocs that result in empty string nodes will now drop those string nodes from the list.
+- Beginless and endless ranges in conditional expressions now properly form a flip flop node.
+- `%` at the end of files no longer crashes.
+- Location information has been corrected for `if/elsif` chains that have no `else`.
+- `__END__` at the very end of the file was previously parsed as an identifier, but is now correct.
+- **BREAKING**: Nodes that reference `&&=`, `||=`, and other writing operators have been consolidated. Previously, they were separate individual nodes. Now they are a tree with the target being the left-hand side and the value being the right-hand side with a joining `AndWriteNode`, `OrWriteNode`, or `OperatorWriteNode` in the middle. This impacts all of the nodes that match this pattern: `{ClassVariable,Constant,ConstantPath,GlobalVariable,InstanceVariable,LocalVariable}Operator{And,Or,}WriteNode`.
+- **BREAKING**: `BlockParametersNode`, `ClassNode`, `DefNode`, `LambdaNode`, `ModuleNode`, `ParenthesesNode`, and `SingletonClassNode` have had their `statements` field renamed to `body` to give a hint that it might not be a `StatementsNode` (it could also be a `BeginNode`).
+
 ## [0.7.0] - 2023-08-14
 
 ### Added
@@ -22,7 +72,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - Autotools has been removed from the build system, so when the gem is installed it will no longer need to go through a configure step.
 - The AST for `foo = *bar` has changed to have an explicit array on the right hand side, rather than a splat node. This is more consistent with how other parsers handle this.
-- `RangeNodeFlags` has been renamed to `RangeFlags`.
+- **BREAKING**: `RangeNodeFlags` has been renamed to `RangeFlags`.
 - Unary minus on number literals is now parsed as part of the literal, rather than a call to a unary operator. This is more consistent with how other parsers handle this.
 
 ## [0.6.0] - 2023-08-09
@@ -31,6 +81,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/yarp/compare/v0.7.0...HEAD
+[unreleased]: https://github.com/ruby/yarp/compare/v0.9.0...HEAD
+[0.9.0]: https://github.com/ruby/yarp/compare/v0.8.0...v0.9.0
+[0.8.0]: https://github.com/ruby/yarp/compare/v0.7.0...v0.8.0
 [0.7.0]: https://github.com/ruby/yarp/compare/v0.6.0...v0.7.0
 [0.6.0]: https://github.com/ruby/yarp/compare/d60531...v0.6.0

data/Makefile

@@ -61,6 +61,8 @@ fuzz-docker-build: fuzz/docker/Dockerfile
 	$(Q) docker build -t yarp/fuzz fuzz/docker/
 
 fuzz-run-%: FORCE fuzz-docker-build
+	$(ECHO) "generating templates"
+	$(Q) bundle exec rake templates
 	$(ECHO) "running $* fuzzer"
 	$(Q) docker run --rm -v $(shell pwd):/yarp yarp/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
 	$(ECHO) "starting AFL++ run"

data/README.md

@@ -14,7 +14,6 @@ The repository contains the infrastructure for both a shared library (librubypar
 │   ├── lex               runs the lexer on a file or string, prints the tokens, and compares to ripper
 │   └── parse             runs the parser on a file or string and prints the syntax tree
 ├── config.yml            specification for tokens and nodes in the tree
-├── configure.ac          configuration to generate the Makefile
 ├── docs                  documentation about the project
 ├── ext
 │   └── yarp
@@ -29,6 +28,9 @@ The repository contains the infrastructure for both a shared library (librubypar
 │   ├── yarp              Ruby library files
 │   └── yarp.rb           main entrypoint for the Ruby library
 ├── rakelib               various Rake tasks for the project
+├── rust
+│   ├── yarp              Rustified crate for the shared library
+│   └── yarp-sys          FFI binding for Rust
 ├── src
 │   ├── enc               various encoding files
 │   ├── util              various utility files
@@ -36,8 +38,9 @@ The repository contains the infrastructure for both a shared library (librubypar
 ├── templates             contains ERB templates generated by templates/template.rb
 │   └── template.rb       generates code from the nodes and tokens configured by config.yml
 └── test
-    ├── fixtures          Ruby code used for testing
-    └── snapshots         snapshots of generated syntax trees corresponding to fixtures
+    └── yarp
+        ├── fixtures      Ruby code used for testing
+        └── snapshots     snapshots of generated syntax trees corresponding to fixtures
 ```
 
 ## Getting started
@@ -45,7 +48,7 @@ The repository contains the infrastructure for both a shared library (librubypar
 To compile the shared library, you will need:
 
 * A C99 compiler
-* autotools (autoconf, automake, libtool)
+* autotools autoconf, automake, libtool)
 * make
 * Ruby 3.3.0-preview1 or later
 
@@ -77,10 +80,11 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
 * [Configuration](docs/configuration.md)
 * [Design](docs/design.md)
 * [Encoding](docs/encoding.md)
-* [Extension](docs/extension.md)
 * [Fuzzing](docs/fuzzing.md)
 * [Heredocs](docs/heredocs.md)
 * [Mapping](docs/mapping.md)
 * [Ripper](docs/ripper.md)
+* [Ruby API](docs/ruby_api.md)
 * [Serialization](docs/serialization.md)
 * [Testing](docs/testing.md)
+