prism 0.21.0 → 0.23.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b4d054a1268bf7f8b5947f30ad244c4713c850911e79c1ba469eca0ac36bc47c
-  data.tar.gz: b77e29c93584b79759381d75cfb5ad0753fe8d5f92863cada81895bb67f17572
+  metadata.gz: 21ba5f26f72f4d44a749e9afa3988a7844ae7b13bf9258bd6e9b907aa8a4941c
+  data.tar.gz: 469d952df74bbd51262e5c7945ddee1d9766eafe2091527c4a952e6de2eaba0c
 SHA512:
-  metadata.gz: 00fa781d854c4f9b716b238c392e48f3bd946b52a5ea100c8fa98bd909bd7d2fcd116b80c7877cbfff59bb991d7c78158ded3ff4154d7d3362df3b8c00fd4d08
-  data.tar.gz: cfea37b3aa825f0bb91a0bd19dec1ec72187790aca39a2b8d560a483d83c1f4604346320d071c93cd605f39c1fd975b1b508395d9673d7bf95c16feaeeee52e6
+  metadata.gz: f0a7686666636ac8a379a40415e6e8ce85d292afc8cc6e223d7157bc502ce24cfc635215f1e89623fac857056e399eaa5e8153337f31c2ecf3a770aa8c1db4e2
+  data.tar.gz: d5db770e2d47bbf85f044c461256f6c87eaf234e4c5bb8f11a848ca5a15e55654551879fa8db5a6e655ec506755a1f7cfdda1cfaafa2db0d9977886661e0b1a9

data/CHANGELOG.md

@@ -6,6 +6,43 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.23.0] - 2024-02-14
+
+### Added
+
+- More support for `Prism::RipperCompat` is added.
+- A significantly faster offset cache for `Prism::Translation::Parser` is added for files with multibyte characters.
+- `Prism::Translation::RubyParser` is added.
+- `Prism::ConstantPathTarget#full_name` is added.
+- `version: "3.4.0"` is added as an option that is an alias for `version: "latest"`.
+- Four new APIs are added to `Prism::Location`:
+  - `Prism::Location#start_code_units_offset`
+  - `Prism::Location#end_code_units_offset`
+  - `Prism::Location#start_code_units_column`
+  - `Prism::Location#end_code_units_column`
+- Invalid multibyte characters are now validated within strings, lists, and heredocs.
+
+### Changed
+
+- When defining `def !@`, the `name_loc` was previously only pointing to `!`, but now includes the `@`. The `name` is the same.
+- `Prism::RipperCompat` has been moved to `Prism::Translation::Ripper`.
+- Many of the error messages that prism produces have been changed to match the error messages that CRuby produces.
+
+## [0.22.0] - 2024-02-07
+
+### Added
+
+- More support for `Prism::RipperCompat` is added.
+- Support for Ruby 2.7 has been added, and the minimum Ruby requirement has been lowered to 2.7.
+
+### Changed
+
+- The error for an invalid source encoding has a new `:argument` level to indicate it raises an argument error.
+- `BeginNode` nodes that are used when a class, singleton class, module, method definition, or block have an inline `rescue`/`ensure`/`else` now have their opening locations set to the beginning of the respective keyword.
+- Improved error messages for invalid characters.
+- `Prism.parse_file` and similar APIs will raise more appropriate errors when the file does not exist or cannot be mapped.
+- Correctly handle the `recover` parameter for `Prism::Translation::Parser`.
+
 ## [0.21.0] - 2024-02-05
 
 ### Added
@@ -341,7 +378,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/prism/compare/v0.21.0...HEAD
+[unreleased]: https://github.com/ruby/prism/compare/v0.23.0...HEAD
+[0.23.0]: https://github.com/ruby/prism/compare/v0.22.0...v0.23.0
+[0.22.0]: https://github.com/ruby/prism/compare/v0.21.0...v0.22.0
 [0.21.0]: https://github.com/ruby/prism/compare/v0.20.0...v0.21.0
 [0.20.0]: https://github.com/ruby/prism/compare/v0.19.0...v0.20.0
 [0.19.0]: https://github.com/ruby/prism/compare/v0.18.0...v0.19.0

data/README.md

@@ -52,7 +52,7 @@ To compile the shared library, you will need:
 
 * C99 compiler
 * make
-* Ruby 3.0.0 or later
+* Ruby 2.7.0 or later
 
 Once you have these dependencies, run:
 
@@ -93,5 +93,6 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
 * [Releasing](docs/releasing.md)
 * [Ripper](docs/ripper.md)
 * [Ruby API](docs/ruby_api.md)
+* [RubyParser translation](docs/ruby_parser_translation.md)
 * [Serialization](docs/serialization.md)
 * [Testing](docs/testing.md)

data/docs/releasing.md

@@ -8,23 +8,91 @@ To release a new version of Prism, perform the following steps:
   * Add a new section for the new version at the top of the file.
   * Fill in the relevant changes — it may be easiest to click the link for the `Unreleased` heading to find the commits.
   * Update the links at the bottom of the file.
-* Update the version in the following files:
-  * `prism.gemspec` in the `Gem::Specification#version=` method call
-  * `ext/prism/extension.h` in the `EXPECTED_PRISM_VERSION` macro
-  * `include/prism/version.h` in the version macros
-  * `javascript/package.json` in the `version` field
-  * `rust/ruby-prism-sys/tests/utils_tests.rs` in the `version_test` function
-  * `templates/java/org/prism/Loader.java.erb` in the `load` function
-  * `templates/javascript/src/deserialize.js.erb` in the version constants
-  * `templates/lib/prism/serialize.rb.erb` in the version constants
-* Run `bundle install` to update the `Gemfile.lock` file.
-* Update `rust/ruby-prism-sys/Cargo.toml` to match the new version and run `cargo build`
-* Update `rust/ruby-prism/Cargo.toml` to match the new version and run `cargo build`
-* Commit all of the updated files.
+* Update the version numbers in the various files that reference them:
+
+```sh
+export PRISM_MAJOR="x"
+export PRISM_MINOR="y"
+export PRISM_PATCH="z"
+export PRISM_VERSION="$PRISM_MAJOR.$PRISM_MINOR.$PRISM_PATCH"
+ruby -pi -e 'gsub(/spec\.version = ".+?"/, %Q{spec.version = "#{ENV["PRISM_VERSION"]}"})' prism.gemspec
+ruby -pi -e 'gsub(/EXPECTED_PRISM_VERSION ".+?"/, %Q{EXPECTED_PRISM_VERSION "#{ENV["PRISM_VERSION"]}"})' ext/prism/extension.h
+ruby -pi -e 'gsub(/PRISM_VERSION_MAJOR \d+/, %Q{PRISM_VERSION_MAJOR #{ENV["PRISM_MAJOR"]}})' include/prism/version.h
+ruby -pi -e 'gsub(/PRISM_VERSION_MINOR \d+/, %Q{PRISM_VERSION_MINOR #{ENV["PRISM_MINOR"]}})' include/prism/version.h
+ruby -pi -e 'gsub(/PRISM_VERSION_PATCH \d+/, %Q{PRISM_VERSION_PATCH #{ENV["PRISM_PATCH"]}})' include/prism/version.h
+ruby -pi -e 'gsub(/PRISM_VERSION ".+?"/, %Q{PRISM_VERSION "#{ENV["PRISM_VERSION"]}"})' include/prism/version.h
+ruby -pi -e 'gsub(/"version": ".+?"/, %Q{"version": "#{ENV["PRISM_VERSION"]}"})' javascript/package.json
+ruby -pi -e 'gsub(/lossy\(\), ".+?"/, %Q{lossy(), "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism-sys/tests/utils_tests.rs
+ruby -pi -e 'gsub(/\d+, "prism major/, %Q{#{ENV["PRISM_MAJOR"]}, "prism major})' templates/java/org/prism/Loader.java.erb
+ruby -pi -e 'gsub(/\d+, "prism minor/, %Q{#{ENV["PRISM_MINOR"]}, "prism minor})' templates/java/org/prism/Loader.java.erb
+ruby -pi -e 'gsub(/\d+, "prism patch/, %Q{#{ENV["PRISM_PATCH"]}, "prism patch})' templates/java/org/prism/Loader.java.erb
+ruby -pi -e 'gsub(/MAJOR_VERSION = \d+/, %Q{MAJOR_VERSION = #{ENV["PRISM_MAJOR"]}})' templates/javascript/src/deserialize.js.erb
+ruby -pi -e 'gsub(/MINOR_VERSION = \d+/, %Q{MINOR_VERSION = #{ENV["PRISM_MINOR"]}})' templates/javascript/src/deserialize.js.erb
+ruby -pi -e 'gsub(/PATCH_VERSION = \d+/, %Q{PATCH_VERSION = #{ENV["PRISM_PATCH"]}})' templates/javascript/src/deserialize.js.erb
+ruby -pi -e 'gsub(/MAJOR_VERSION = \d+/, %Q{MAJOR_VERSION = #{ENV["PRISM_MAJOR"]}})' templates/lib/prism/serialize.rb.erb
+ruby -pi -e 'gsub(/MINOR_VERSION = \d+/, %Q{MINOR_VERSION = #{ENV["PRISM_MINOR"]}})' templates/lib/prism/serialize.rb.erb
+ruby -pi -e 'gsub(/PATCH_VERSION = \d+/, %Q{PATCH_VERSION = #{ENV["PRISM_PATCH"]}})' templates/lib/prism/serialize.rb.erb
+ruby -pi -e 'gsub(/^version = ".+?"/, %Q{version = "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism-sys/Cargo.toml
+ruby -pi -e 'gsub(/^version = ".+?"/, %Q{version = "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism/Cargo.toml
+ruby -pi -e 'gsub(/^ruby-prism-sys = { version = ".+?"/, %Q{ruby-prism-sys = { version = "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism/Cargo.toml
+```
+
+* Update the `Gemfile.lock` file:
+
+```sh
+chruby ruby-3.4.0-dev
+bundle install
+```
+
+* Update the version-specific lockfiles:
+
+```sh
+chruby ruby-2.7.8 && BUNDLE_GEMFILE=gemfiles/2.7/Gemfile bundle install
+chruby ruby-3.0.6 && BUNDLE_GEMFILE=gemfiles/3.0/Gemfile bundle install
+chruby ruby-3.1.4 && BUNDLE_GEMFILE=gemfiles/3.1/Gemfile bundle install
+chruby ruby-3.2.3 && BUNDLE_GEMFILE=gemfiles/3.2/Gemfile bundle install
+chruby ruby-3.3.0 && BUNDLE_GEMFILE=gemfiles/3.3/Gemfile bundle install
+chruby ruby-3.4.0-dev && BUNDLE_GEMFILE=gemfiles/3.4/Gemfile bundle install
+chruby jruby-9.4.5.0 && BUNDLE_GEMFILE=gemfiles/jruby/Gemfile bundle install
+chruby truffleruby-23.1.2 && BUNDLE_GEMFILE=gemfiles/truffleruby/Gemfile bundle install
+chruby ruby-3.4.0-dev
+```
+
+* Update the cargo lockfiles:
+
+```sh
+bundle exec rake cargo:build
+```
+
+* Commit all of the updated files:
+
+```sh
+git commit -am "Bump to v$PRISM_VERSION"
+```
 
 ## Publishing
 
 * Update the GitHub release page with a copy of the latest entry in the `CHANGELOG.md` file.
-* Run `bundle exec rake release` to publish the gem to [rubygems.org](rubygems.org). Note that you must have access to the `prism` gem to do this.
-* Either download the `wasm` artifact from GitHub actions or generate it yourself with `make wasm`.
-* Run `npm publish` to publish the JavaScript package to [npmjs.com](npmjs.com). Note that you must have access to the `ruby-prism` package to do this.
+* Publish the gem to [rubygems.org](rubygems.org). Note that you must have access to the `prism` gem to do this.
+
+```sh
+bundle exec rake release
+```
+
+* Generate the `wasm` artifact (or download it from GitHub actions and put it in `javascript/src/prism.wasm`).
+
+```sh
+make wasm
+```
+
+* Publish the JavaScript package to [npmjs.com](npmjs.com). Note that you must have access to the `@ruby/prism` package to do this.
+
+```sh
+npm publish
+```
+
+* Publish the rust crate to [crates.io](crates.io). Note that you must have access to the `ruby-prism-sys` and `ruby-prism` crates to do this.
+
+```sh
+bundle exec rake cargo:publish:real
+```

data/docs/ruby_parser_translation.md

@@ -0,0 +1,19 @@
+# ruby_parser translation
+
+Prism ships with the ability to translate its syntax tree into the syntax tree used by the [seattlerb/ruby_parser](https://github.com/seattlerb/ruby_parser) gem. This allows you to use tools built on top of the `ruby_parser` gem with the `prism` parser.
+
+## Usage
+
+You can call the `parse` and `parse_file` methods on the `Prism::Translation::RubyParser` module:
+
+```ruby
+filepath = "path/to/file.rb"
+Prism::Translation::RubyParser.parse_file(filepath)
+```
+
+This will return to you `Sexp` objects that mirror the result of calling `RubyParser` methods, as in:
+
+```ruby
+filepath = "path/to/file.rb"
+RubyParser.new.parse(File.read(filepath), filepath)
+```

data/docs/serialization.md

@@ -85,6 +85,8 @@ The header is structured like the following table:
 | `1` | 1 indicates only semantics fields were serialized, 0 indicates all fields were serialized (including location fields) |
 | string | the encoding name |
 | varsint | the start line |
+| varuint | number of newline offsets |
+| varuint* | newline offsets |
 | varuint | number of comments |
 | comment* | comments |
 | varuint | number of magic comments |