prism 1.4.0 → 1.7.0

data/docs/parser_translation.md

@@ -6,34 +6,19 @@ Prism ships with the ability to translate its syntax tree into the syntax tree u
 
 The `parser` gem provides multiple parsers to support different versions of the Ruby grammar. This includes all of the Ruby versions going back to 1.8, as well as third-party parsers like MacRuby and RubyMotion. The `prism` gem provides another parser that uses the `prism` parser to build the syntax tree.
 
-You can use the `prism` parser like you would any other. After requiring the parser, you should be able to call any of the regular `Parser::Base` APIs that you would normally use.
+You can use the `prism` parser like you would any other. After requiring `prism`, you should be able to call any of the regular `Parser::Base` APIs that you would normally use.
 
 ```ruby
 require "prism"
 
-Prism::Translation::Parser.parse_file("path/to/file.rb")
-```
-
-### RuboCop
-
-Prism as a parser engine is directly supported since RuboCop 1.62. The class used for parsing is `Prism::Translation::Parser`.
-
-First, specify `prism` in your Gemfile:
-
-```ruby
-gem "prism"
-```
-
-To use Prism with RuboCop, specify `ParserEngine` and `TargetRubyVersion` in your RuboCop configuration file:
+# Same as `Parser::Ruby34`
+Prism::Translation::Parser34.parse_file("path/to/file.rb")
 
-```yaml
-AllCops:
-  ParserEngine: parser_prism
-  TargetRubyVersion: 3.3
+# Same as `Parser::CurrentRuby`
+Prism::Translation::ParserCurrent.parse("puts 'Hello World!'")
 ```
 
-The default value for `ParserEngine` is `parser_whitequark`, which indicates the Parser gem. You need to explicitly switch it to `parser_prism` to indicate Prism. Additionally, the value for `TargetRubyVersion` must be specified as `3.3` or higher, as Prism supports parsing versions of Ruby 3.3 and higher.
-The parser class is determined by the combination of values for `ParserEngine` and `TargetRubyVersion`. For example, if `TargetRubyVersion: 3.3`, parsing is performed by `Prism::Translation::Parser33`, and for `TargetRubyVersion 3.4`, parsing is performed by `Prism::Translation::Parser34`.
+All the parsers are autoloaded, so you don't have to worry about requiring them yourself.
 
-For further information, please refer to the RuboCop documentation:
-https://docs.rubocop.org/rubocop/configuration.html#setting-the-parser-engine
+If you also need to parse Ruby versions below 3.3 (for which the `prism` translation layer does not have explicit support), check out
+[this guide](https://github.com/whitequark/parser/blob/master/doc/PRISM_TRANSLATION.md) from the `parser` gem on how to use both in conjunction.

data/docs/releasing.md

@@ -40,14 +40,17 @@ ruby -pi -e 'gsub(/^ruby-prism-sys = \{ version = ".+?"/, %Q{ruby-prism-sys = \{
 * Update the `Gemfile.lock` file:
 
 ```sh
-chruby ruby-3.5.0-dev
+chruby ruby-4.0.0-dev
 bundle install
 ```
 
 * Update the version-specific lockfiles:
 
 ```sh
-bin/prism bundle install
+for VERSION in "2.7" "3.0" "3.1" "3.2" "3.3" "3.4"; do docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app -e BUNDLE_GEMFILE="gemfiles/$VERSION/Gemfile" "ruby:$VERSION" bundle update; done
+docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app -e BUNDLE_GEMFILE="gemfiles/4.0/Gemfile" ruby:4.0.0-preview2 bundle update
+docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app -e BUNDLE_GEMFILE="gemfiles/jruby/Gemfile" jruby:latest bundle update
+BUNDLE_GEMFILE=gemfiles/truffleruby/Gemfile chruby-exec truffleruby -- bundle update
 ```
 
 * Update the cargo lockfiles:
@@ -71,26 +74,4 @@ git push
 ## Publishing
 
 * Update the GitHub release page with a copy of the latest entry in the `CHANGELOG.md` file.
-* Publish the gem to [rubygems.org](rubygems.org). Note that you must have access to the `prism` gem to do this.
-
-```sh
-bundle exec rake release
-```
-
-* Generate the `wasm` artifact (or download it from GitHub actions and put it in `javascript/src/prism.wasm`).
-
-```sh
-make wasm
-```
-
-* Publish the JavaScript package to [npmjs.com](npmjs.com). Note that you must have access to the `@ruby/prism` package to do this.
-
-```sh
-npm publish
-```
-
-* Publish the rust crate to [crates.io](crates.io). Note that you must have access to the `ruby-prism-sys` and `ruby-prism` crates to do this.
-
-```sh
-bundle exec rake cargo:publish:real
-```
+* Push a new tag to the GitHub repository, following the `vX.Y.Z` format.

data/docs/ripper_translation.md

@@ -55,7 +55,7 @@ It is helpful to understand the differences between the `Ripper` library and the
 
 ### Design
 
-`Ripper` is a streaming parser. This means as it is parsing Ruby code, it dispatches events back to the consumer. This allows quite a bit of flexibility. You can use it to build your own syntax tree or to find specific patterns in the code. `Prism` on the other hand returns to your the completed syntax tree _before_ it allows you to manipulate it. This means the tree that you get back is the only representation that can be generated by the parser _at parse time_ (but of course can be manipulated later).
+`Ripper` is a streaming parser. This means as it is parsing Ruby code, it dispatches events back to the consumer. This allows quite a bit of flexibility. You can use it to build your own syntax tree or to find specific patterns in the code. `Prism` on the other hand returns to you the completed syntax tree _before_ it allows you to manipulate it. This means the tree that you get back is the only representation that can be generated by the parser _at parse time_ (but of course can be manipulated later).
 
 ### Fields
 

data/ext/prism/api_node.c

@@ -1,3 +1,5 @@
+/* :markup: markdown */
+
 /*----------------------------------------------------------------------------*/
 /* This file is generated by the templates/template.rb script and should not  */
 /* be modified manually. See                                                  */
@@ -1787,7 +1789,7 @@ pm_ast_new(const pm_parser_t *parser, const pm_node_t *node, rb_encoding *encodi
 #line 190 "prism/templates/ext/prism/api_node.c.erb"
                 case PM_CALL_NODE: {
                     pm_call_node_t *cast = (pm_call_node_t *) node;
-                    VALUE argv[12];
+                    VALUE argv[13];
 
                     // source
                     argv[0] = source;
@@ -1830,11 +1832,15 @@ pm_ast_new(const pm_parser_t *parser, const pm_node_t *node, rb_encoding *encodi
 #line 243 "prism/templates/ext/prism/api_node.c.erb"
                     argv[10] = cast->closing_loc.start == NULL ? Qnil : pm_location_new(parser, cast->closing_loc.start, cast->closing_loc.end, source, freeze);
 
+                    // equal_loc
+#line 243 "prism/templates/ext/prism/api_node.c.erb"
+                    argv[11] = cast->equal_loc.start == NULL ? Qnil : pm_location_new(parser, cast->equal_loc.start, cast->equal_loc.end, source, freeze);
+
                     // block
 #line 213 "prism/templates/ext/prism/api_node.c.erb"
-                    argv[11] = rb_ary_pop(value_stack);
+                    argv[12] = rb_ary_pop(value_stack);
 
-                    VALUE value = rb_class_new_instance(12, argv, rb_cPrismCallNode);
+                    VALUE value = rb_class_new_instance(13, argv, rb_cPrismCallNode);
                     if (freeze) rb_obj_freeze(value);
 
                     rb_ary_push(value_stack, value);

data/ext/prism/extconf.rb

@@ -37,7 +37,7 @@ end
 def generate_templates
   Dir.chdir(File.expand_path("../..", __dir__)) do
     if !File.exist?("include/prism/ast.h") && Dir.exist?(".git")
-      system("templates/template.rb", exception: true)
+      system(RbConfig.ruby, "templates/template.rb", exception: true)
     end
   end
 end

data/ext/prism/extension.c

@@ -25,6 +25,7 @@ VALUE rb_cPrismLexResult;
 VALUE rb_cPrismParseLexResult;
 VALUE rb_cPrismStringQuery;
 VALUE rb_cPrismScope;
+VALUE rb_cPrismCurrentVersionError;
 
 VALUE rb_cPrismDebugEncoding;
 
@@ -199,7 +200,12 @@ build_options_i(VALUE key, VALUE value, VALUE argument) {
         if (!NIL_P(value)) {
             const char *version = check_string(value);
 
-            if (!pm_options_version_set(options, version, RSTRING_LEN(value))) {
+            if (RSTRING_LEN(value) == 7 && strncmp(version, "current", 7) == 0) {
+                const char *current_version = RSTRING_PTR(rb_const_get(rb_cObject, rb_intern("RUBY_VERSION")));
+                if (!pm_options_version_set(options, current_version, 3)) {
+                    rb_exc_raise(rb_exc_new_cstr(rb_cPrismCurrentVersionError, current_version));
+                }
+            } else if (!pm_options_version_set(options, version, RSTRING_LEN(value))) {
                 rb_raise(rb_eArgError, "invalid version: %" PRIsVALUE, value);
             }
         }
@@ -888,7 +894,7 @@ parse_input(pm_string_t *input, const pm_options_t *options) {
  *       version of Ruby syntax (which you can trigger with `nil` or
  *       `"latest"`). You may also restrict the syntax to a specific version of
  *       Ruby, e.g., with `"3.3.0"`. To parse with the same syntax version that
- *       the current Ruby is running use `version: RUBY_VERSION`. Raises
+ *       the current Ruby is running use `version: "current"`. Raises
  *       ArgumentError if the version is not currently supported by Prism.
  */
 static VALUE
@@ -994,6 +1000,14 @@ profile_file(int argc, VALUE *argv, VALUE self) {
     return Qnil;
 }
 
+static int
+parse_stream_eof(void *stream) {
+    if (rb_funcall((VALUE) stream, rb_intern("eof?"), 0)) {
+        return 1;
+    }
+    return 0;
+}
+
 /**
  * An implementation of fgets that is suitable for use with Ruby IO objects.
  */
@@ -1034,7 +1048,7 @@ parse_stream(int argc, VALUE *argv, VALUE self) {
     pm_parser_t parser;
     pm_buffer_t buffer;
 
-    pm_node_t *node = pm_parse_stream(&parser, &buffer, (void *) stream, parse_stream_fgets, &options);
+    pm_node_t *node = pm_parse_stream(&parser, &buffer, (void *) stream, parse_stream_fgets, parse_stream_eof, &options);
     rb_encoding *encoding = rb_enc_find(parser.encoding->name);
 
     VALUE source = pm_source_new(&parser, encoding, options.freeze);
@@ -1331,6 +1345,11 @@ Init_prism(void) {
         );
     }
 
+#ifdef HAVE_RB_EXT_RACTOR_SAFE
+    // Mark this extension as Ractor-safe.
+    rb_ext_ractor_safe(true);
+#endif
+
     // Grab up references to all of the constants that we're going to need to
     // reference throughout this extension.
     rb_cPrism = rb_define_module("Prism");
@@ -1351,6 +1370,8 @@ Init_prism(void) {
     rb_cPrismStringQuery = rb_define_class_under(rb_cPrism, "StringQuery", rb_cObject);
     rb_cPrismScope = rb_define_class_under(rb_cPrism, "Scope", rb_cObject);
 
+    rb_cPrismCurrentVersionError = rb_const_get(rb_cPrism, rb_intern("CurrentVersionError"));
+
     // Intern all of the IDs eagerly that we support so that we don't have to do
     // it every time we parse.
     rb_id_option_command_line = rb_intern_const("command_line");

data/ext/prism/extension.h

@@ -1,7 +1,7 @@
 #ifndef PRISM_EXT_NODE_H
 #define PRISM_EXT_NODE_H
 
-#define EXPECTED_PRISM_VERSION "1.4.0"
+#define EXPECTED_PRISM_VERSION "1.7.0"
 
 #include <ruby.h>
 #include <ruby/encoding.h>