ripper_ruby_parser 1.10.0 → 1.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a110a987032924880be79cfbb79948e14f90868e8105055a0833284a6f939731
-  data.tar.gz: f38c7fa39cb0f1f6fc78cbe72956c77fe15fe74fe99507a3bd59d057618aed0a
+  metadata.gz: 568f21d8800cba790d212bb45e96554f93a271da3d9bae483fbf101385906d8d
+  data.tar.gz: fa4393288ca29703405077d9da524a51cb0d7575c08b8dcabcc9be3bcbe5d6d0
 SHA512:
-  metadata.gz: 7bf9631eb8941e2ed07ec265519ce8860f9d417cb224b2bbd1bbb8b32277374d76936be126c7893f73bcbcaa1414a25d4ca2280c9e0e701462cdbf5d34c3c77f
-  data.tar.gz: e517b02e7854fc644763932d8935dd5b145e2c7c4546874b419932a1d0f0d53b523cba356949f5b941e1fe6405c5b308cade8c42ba3d02078a8e7abe5df176a6
+  metadata.gz: 5dcfda84f25d0351b0351e06eb231deaab73d6377991a6b1939dc9c4df268c8cfb0960c22ca50a0d3a5dec432a855ef2ddb8a66171b321855ae2732c1f2d5518
+  data.tar.gz: 7c68ec079fe840e6990d1555b1751639b46113f636fcc4e90429af4a3c36382f19ba6fbd6c3c0016271e1082412dd6f9001c7f8fb7c20858aec06d1d88a4af59

data/CHANGELOG.md

@@ -6,6 +6,58 @@ This project adheres to [Semantic Versioning 2.0.0][1].
 
 This document is formatted based on [Keep A CHANGELOG][2].
 
+## 1.11.0 / 2024-01-05
+
+* Support Ruby 3.0 through 3.3, dropping support for Ruby 2.6 and 2.7
+  ([#218], [#219], [#233] and [#246] by [mvz])
+* Target compatibility with `ruby_parser` version 3.20.2
+  ([#199], [#216] and [#230] by [mvz])
+* Support single-line pattern matching ([#188] by [mvz])
+* Support rightward assignment ([#189] by [mvz])
+* Support endless methods with Ruby 3.0 syntax ([#191] by [mvz])
+* Support arguments without parentheses in endless method body ([#193] by [mvz])
+* Add support for pattern matching ([#172] by [mvz])
+* Add tentative support for numbered parameters ([#163] by [mvz])
+* Support several new Ruby 3.1 syntax features ([#196] by [mvz])
+* Support negative rational and imaginary literals ([#197] by [mvz])
+* Correctly handle match operator with regexp literals in parentheses
+  ([#200] by [mvz])
+* Improve operator assignment handling ([#201] by [mvz])
+* Handle literal Ctrl-? (DEL) character correctly ([#202] by [mvz])
+* Handle method names that are keywords in method definitions ([#204] by [mvz])
+* Improve dsym handling compatibility ([#203], [#210] by [mvz])
+* Add support for Ruby 3.2's splat and kwsplat argument forwarding ([#231] by [mvz])
+* Improve handling of the case .. in construction ([#234] by [mvz])
+
+[mvz]: https://github.com/mvz
+
+[#163]: https://github.com/mvz/ripper_ruby_parser/pull/163
+[#172]: https://github.com/mvz/ripper_ruby_parser/pull/172
+[#188]: https://github.com/mvz/ripper_ruby_parser/pull/188
+[#189]: https://github.com/mvz/ripper_ruby_parser/pull/189
+[#191]: https://github.com/mvz/ripper_ruby_parser/pull/191
+[#193]: https://github.com/mvz/ripper_ruby_parser/pull/193
+[#196]: https://github.com/mvz/ripper_ruby_parser/pull/196
+[#197]: https://github.com/mvz/ripper_ruby_parser/pull/197
+[#199]: https://github.com/mvz/ripper_ruby_parser/pull/199
+[#200]: https://github.com/mvz/ripper_ruby_parser/pull/200
+[#201]: https://github.com/mvz/ripper_ruby_parser/pull/201
+[#202]: https://github.com/mvz/ripper_ruby_parser/pull/202
+[#203]: https://github.com/mvz/ripper_ruby_parser/pull/203
+[#204]: https://github.com/mvz/ripper_ruby_parser/pull/204
+[#205]: https://github.com/mvz/ripper_ruby_parser/pull/205
+[#210]: https://github.com/mvz/ripper_ruby_parser/pull/210
+[#216]: https://github.com/mvz/ripper_ruby_parser/pull/216
+[#218]: https://github.com/mvz/ripper_ruby_parser/pull/218
+[#219]: https://github.com/mvz/ripper_ruby_parser/pull/219
+[#230]: https://github.com/mvz/ripper_ruby_parser/pull/230
+[#231]: https://github.com/mvz/ripper_ruby_parser/pull/231
+[#232]: https://github.com/mvz/ripper_ruby_parser/pull/232
+[#233]: https://github.com/mvz/ripper_ruby_parser/pull/233
+[#234]: https://github.com/mvz/ripper_ruby_parser/pull/234
+[#235]: https://github.com/mvz/ripper_ruby_parser/pull/235
+[#246]: https://github.com/mvz/ripper_ruby_parser/pull/246
+
 ## 1.10.0 / 2022-03-13
 
 * Handle shadow arguments ([#161])
@@ -233,6 +285,8 @@ This document is formatted based on [Keep A CHANGELOG][2].
 
 <!-- Pull request links -->
 [#180]: https://github.com/mvz/ripper_ruby_parser/pull/180
+[#172]: https://github.com/mvz/ripper_ruby_parser/pull/172
+[#163]: https://github.com/mvz/ripper_ruby_parser/pull/163
 [#165]: https://github.com/mvz/ripper_ruby_parser/pull/165
 [#161]: https://github.com/mvz/ripper_ruby_parser/pull/161
 [#155]: https://github.com/mvz/ripper_ruby_parser/pull/155

data/README.md

@@ -10,8 +10,8 @@ Parse with Ripper, produce sexps that are compatible with RubyParser.
 
 * Drop-in replacement for RubyParser
 * Should handle 1.9 and later syntax gracefully
-* Requires Ruby 2.6 or higher
-* Compatible with RubyParser 3.18.0
+* Requires Ruby 3.0 or higher
+* Compatible with RubyParser 3.20.2
 
 ## Known incompatibilities
 
@@ -25,7 +25,9 @@ RipperRubyParser has a few incompatibilities with RubyParser.
 
 ## Install
 
-    gem install ripper_ruby_parser
+```bash
+gem install ripper_ruby_parser
+```
 
 ## Synopsis
 
@@ -44,7 +46,7 @@ parser.parse "foo[bar] += baz qux"
 
 ## Requirements
 
-* Ruby 2.5 or higher
+* Ruby 2.7 or higher
 * `sexp_processor`
 
 ## Hacking and contributing
@@ -64,7 +66,7 @@ If you want to send pull requests or patches, please:
 
 (The MIT License)
 
-Copyright (c) 2012, 2014-2022 Matijs van Zuijlen
+Copyright (c) 2012, 2014-2024 Matijs van Zuijlen
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

data/lib/ripper_ruby_parser/commenting_ripper_parser.rb

@@ -8,6 +8,7 @@ module RipperRubyParser
   # Variant of Ripper's SexpBuilder parser class that inserts comments as
   # Sexps into the built parse tree.
   #
+  # rubocop: disable Metrics/ClassLength
   # @api private
   class CommentingRipperParser < Ripper::SexpBuilder
     def initialize(*args)
@@ -35,7 +36,16 @@ module RipperRubyParser
     end
 
     def on_begin(*args)
-      commentize(:begin, super)
+      result = super
+
+      # Some begin blocks are not created by the 'begin' keyword. Skip
+      # commenting for those kinds of blocks.
+      (_, kw,), = @comment_stack.last
+      if kw == "begin"
+        commentize("begin", result)
+      else
+        result
+      end
     end
 
     def on_void_stmt
@@ -72,23 +82,25 @@ module RipperRubyParser
     end
 
     def on_module(*args)
-      commentize(:module, super)
+      commentize("module", super)
     end
 
     def on_class(*args)
-      commentize(:class, super)
+      commentize("class", super)
     end
 
     def on_sclass(*args)
-      commentize(:class, super)
+      commentize("class", super)
     end
 
-    def on_def(*args)
-      commentize(:def, super)
+    def on_def(name, *args)
+      (_, _, loc) = name
+      commentize("def", super, loc)
     end
 
-    def on_defs(*args)
-      commentize(:def, super)
+    def on_defs(receiver, period, name, *rest)
+      (_, _, loc) = name
+      commentize("def", super, loc)
     end
 
     def on_args_new
@@ -254,6 +266,11 @@ module RipperRubyParser
       @seen_space = true
     end
 
+    def on_imaginary(_token)
+      @space_before = @seen_space
+      super
+    end
+
     def on_int(_token)
       @space_before = @seen_space
       super
@@ -264,7 +281,12 @@ module RipperRubyParser
       super
     end
 
-    NUMBER_LITERAL_TYPES = [:@int, :@float].freeze
+    def on_rational(_token)
+      @space_before = @seen_space
+      super
+    end
+
+    NUMBER_LITERAL_TYPES = [:@imaginary, :@int, :@float, :@rational].freeze
 
     def on_unary(operator, value)
       if !@space_before && operator == :-@ && NUMBER_LITERAL_TYPES.include?(value.first)
@@ -300,15 +322,16 @@ module RipperRubyParser
     end
 
     def on_BEGIN(*args)
-      commentize(:BEGIN, super)
+      commentize("BEGIN", super)
     end
 
     def on_END(*args)
-      commentize(:END, super)
+      commentize("END", super)
     end
 
     def on_parse_error(message)
-      raise SyntaxError, message
+      super
+      raise SyntaxError, message if message.start_with?("syntax error,")
     end
 
     def on_class_name_error(message, *)
@@ -327,11 +350,19 @@ module RipperRubyParser
       raise SyntaxError, message
     end
 
-    def commentize(_name, exp)
-      (_, _kw, loc), comment = @comment_stack.pop
+    def commentize(name, exp, target_loc = nil)
+      if target_loc
+        (_, kw, loc), comment = @comment_stack.pop until (loc <=> target_loc) == -1
+      else
+        (_, kw, loc), comment = @comment_stack.pop
+      end
+
+      warn "Comment stack mismatch: expected #{kw} to equal #{name}" unless kw == name
+
       @comment = ""
       exp.push loc
       [:comment, comment, exp]
     end
   end
+  # rubocop: enable Metrics/ClassLength
 end

data/lib/ripper_ruby_parser/sexp_handlers/assignment.rb

@@ -95,50 +95,74 @@ module RipperRubyParser
       def process_opassign(exp)
         _, lvalue, (_, operator,), value = exp.shift 4
 
+        value_type = value.sexp_type
+
         lvalue = process(lvalue)
         value = process(value)
         operator = operator.chop.to_sym
 
-        create_operator_assignment_sub_type lvalue, value, operator
+        case lvalue.sexp_type
+        when :aref_field
+          create_aref_operator_assignment_sub_type(lvalue, value, operator)
+        when :field
+          create_field_operator_assignment_sub_type(lvalue, value, operator, value_type)
+        else
+          create_regular_operator_assignment_sub_type(lvalue, value, operator)
+        end
       end
 
       private
 
       def create_multiple_assignment_sub_types(sexp_list)
         sexp_list.map! do |item|
-          create_valueless_assignment_sub_type item
+          create_valueless_regular_assignment_sub_type item
         end
       end
 
-      def create_valueless_assignment_sub_type(item)
+      def create_valueless_regular_assignment_sub_type(item)
         item = with_line_number(item.line,
                                 create_regular_assignment_sub_type(item, nil))
         item.pop
         item
       end
 
+      def create_valueless_assignment_sub_type(item)
+        item = with_line_number(item.line,
+                                create_assignment_sub_type(item, nil))
+        item.pop
+        item
+      end
+
+      def create_aref_operator_assignment_sub_type(lvalue, value, operator)
+        _, arr, arglist = lvalue
+        arglist.sexp_type = :arglist
+        s(:op_asgn1, arr, arglist, operator, value)
+      end
+
+      def create_field_operator_assignment_sub_type(lvalue, value, operator, value_type)
+        # Structure of lvalue will be something like this:
+        # s(:field, receiver, s(:period, "."), s(:lvar, field))
+        _, receiver, _, (_, field) = lvalue
+        case value_type
+        when :command, :command_call
+          s(:op_asgn, receiver, value, field, operator)
+        else
+          s(:op_asgn2, receiver, :"#{field}=", operator, value)
+        end
+      end
+
       OPERATOR_ASSIGNMENT_MAP = {
         "||": :op_asgn_or,
         "&&": :op_asgn_and
       }.freeze
 
-      def create_operator_assignment_sub_type(lvalue, value, operator)
-        case lvalue.sexp_type
-        when :aref_field
-          _, arr, arglist = lvalue
-          arglist.sexp_type = :arglist
-          s(:op_asgn1, arr, arglist, operator, value)
-        when :field
-          _, obj, _, (_, field) = lvalue
-          s(:op_asgn2, obj, :"#{field}=", operator, value)
+      def create_regular_operator_assignment_sub_type(lvalue, value, operator)
+        value = unwrap_begin(value)
+        if (mapped = OPERATOR_ASSIGNMENT_MAP[operator])
+          s(mapped, lvalue, create_assignment_sub_type(lvalue, value))
         else
-          value = unwrap_begin(value)
-          if (mapped = OPERATOR_ASSIGNMENT_MAP[operator])
-            s(mapped, lvalue, create_assignment_sub_type(lvalue, value))
-          else
-            operator_call = s(:call, lvalue, operator, value)
-            create_assignment_sub_type lvalue, operator_call
-          end
+          operator_call = s(:call, lvalue, operator, value)
+          create_assignment_sub_type lvalue, operator_call
         end
       end
 

data/lib/ripper_ruby_parser/sexp_handlers/blocks.rb

@@ -10,7 +10,7 @@ module RipperRubyParser
         call = process(call)
         args = process(args)
         kwrest = kwrest_param(args) if args
-        stmt = with_kwrest(kwrest) { process(stmt) }
+        stmt = with_new_lvar_scope(kwrest) { process(stmt) }
         make_iter call, args, safe_unwrap_void_stmt(stmt)
       end
 
@@ -153,16 +153,13 @@ module RipperRubyParser
 
         defaults.map do |sym, val|
           s(:lasgn,
-            extract_node_symbol(process(sym)),
+            make_symbol(process(sym)),
             process(val))
         end
       end
 
       def handle_splat(splat)
-        if splat == 0
-          # Only relevant for Ruby < 2.6
-          [s(:excessed_comma)]
-        elsif splat
+        if splat
           [process(splat)]
         else
           []
@@ -173,7 +170,7 @@ module RipperRubyParser
         return [] unless kwargs
 
         kwargs.map do |sym, val|
-          symbol = extract_node_symbol process(sym)
+          symbol = make_symbol process(sym)
           if val
             s(:kwarg, symbol, process(val))
           else
@@ -217,15 +214,25 @@ module RipperRubyParser
         end
       end
 
+      LVAR_MATCHER = Sexp::Matcher.new(:lvar, Sexp._)
+      NUMBERED_PARAMS = (1..9).map { |it| :"_#{it}" }.freeze
+
       def make_iter(call, args, stmt)
         args[-1] = nil if args && args.last == s(:excessed_comma)
-        args ||= 0
+
+        args ||= count_numbered_lvars(stmt)
+
         if stmt.empty?
           s(:iter, call, args)
         else
           s(:iter, call, args, stmt)
         end
       end
+
+      def count_numbered_lvars(stmt)
+        lvar_names = (LVAR_MATCHER / stmt).map { |it| it[1] }
+        (NUMBERED_PARAMS & lvar_names).length
+      end
     end
   end
 end

data/lib/ripper_ruby_parser/sexp_handlers/conditionals.rb

@@ -64,11 +64,78 @@ module RipperRubyParser
           *falsepart)
       end
 
+      def process_in(exp)
+        _, pattern, truepart, falsepart = exp.shift 4
+
+        falsepart = process(falsepart)
+        falsepart = if falsepart.nil?
+                      [nil]
+                    else
+                      falsepart.sexp_body
+                    end
+        pattern = process(pattern)
+        adjust_rightward_assignment_pattern(pattern)
+
+        truepart = process(truepart)
+        truepart = unwrap_nil(truepart) if truepart
+
+        s(:case_body,
+          s(:in, pattern, truepart),
+          *falsepart)
+      end
+
       def process_else(exp)
         _, body = exp.shift 2
         process(body)
       end
 
+      def process_aryptn(exp)
+        _, _, body, rest, = exp.shift 5
+
+        elements = body.map do |elem|
+          if elem.sexp_type == :var_field
+            create_valueless_assignment_sub_type process(elem)
+          else
+            unwrap_begin process(elem)
+          end
+        end
+        if rest
+          rest_var = handle_pattern(rest)
+          elements << convert_marked_argument(s(:splat, rest_var))
+        end
+        s(:array_pat, nil, *elements)
+      end
+
+      def process_hshptn(exp)
+        _, _, body, = exp.shift 4
+
+        elements = body.flat_map do |key, value|
+          if value
+            [process(key), process(value)]
+          else
+            [handle_pattern(key), nil]
+          end
+        end
+        s(:hash_pat, nil, *elements)
+      end
+
+      def process_fndptn(exp)
+        _, wrapper, before, patterns, after = exp.shift 5
+
+        wrapper = process(wrapper)
+        before = make_splat process(before)
+        after = make_splat process(after)
+        patterns = patterns.map do |elem|
+          if elem.sexp_type == :var_field
+            create_valueless_assignment_sub_type process(elem)
+          else
+            unwrap_begin process(elem)
+          end
+        end
+
+        s(:find_pat, wrapper, before, *patterns, after)
+      end
+
       private
 
       def handle_condition(cond)
@@ -88,6 +155,24 @@ module RipperRubyParser
         unwrap_nil process(exp) if exp
       end
 
+      def handle_pattern(exp)
+        pattern = process(exp)
+        case pattern.sexp_type
+        when :lvar, :lit
+          @local_variables << pattern[1]
+        end
+        pattern
+      end
+
+      def adjust_rightward_assignment_pattern(exp)
+        case exp.sexp_type
+        when :lvar
+          exp.sexp_type = :lasgn
+        when :lasgn
+          adjust_rightward_assignment_pattern(exp.sexp_body.last)
+        end
+      end
+
       def construct_conditional(cond, truepart, falsepart)
         if cond.sexp_type == :not
           _, inner = cond
@@ -107,6 +192,10 @@ module RipperRubyParser
           [exp]
         end
       end
+
+      def make_splat(exp)
+        :"*#{exp[1]}"
+      end
     end
   end
 end

data/lib/ripper_ruby_parser/sexp_handlers/helper_methods.rb

@@ -4,12 +4,17 @@ module RipperRubyParser
   module SexpHandlers
     # Utility methods used in several of the sexp handler modules
     module HelperMethods
+      def extract_node_symbol(exp)
+        ident, = extract_node_symbol_with_position(exp)
+        ident
+      end
+
       def extract_node_symbol_with_position(exp)
         _, ident, pos = exp.shift 3
         return ident.to_sym, pos
       end
 
-      def extract_node_symbol(exp)
+      def make_symbol(exp)
         return nil if exp.nil?
         raise "Unexpected number of children: #{exp.length}" if exp.length != 2
 
@@ -36,7 +41,11 @@ module RipperRubyParser
       def generic_add_star(exp)
         _, args, splatarg, *rest = shift_all exp
         items = process args
-        items.push s(:splat, unwrap_begin(process(splatarg)))
+        if splatarg
+          items.push s(:splat, unwrap_begin(process(splatarg)))
+        else
+          items.push s(:splat)
+        end
         items.push(*map_process_list(rest))
       end
 

data/lib/ripper_ruby_parser/sexp_handlers/literals.rb

@@ -36,7 +36,11 @@ module RipperRubyParser
       #   s(:assoc_splat, s(:vcall, s(:@ident, "bar")))
       def process_assoc_splat(exp)
         _, param = exp.shift 2
-        s(:kwsplat, process(param))
+        if param
+          s(:kwsplat, process(param))
+        else
+          s(:kwsplat)
+        end
       end
 
       # number literals

data/lib/ripper_ruby_parser/sexp_handlers/methods.rb

@@ -7,12 +7,12 @@ module RipperRubyParser
       def process_def(exp)
         _, ident, params, body, pos = exp.shift 5
 
-        ident, = extract_node_symbol_with_position ident
+        ident = extract_node_symbol ident
 
         in_method do
           params = convert_arguments(process(params))
           kwrest = kwrest_param(params)
-          body = with_kwrest(kwrest) { method_body(body) }
+          body = with_new_lvar_scope(kwrest) { method_body(body) }
         end
 
         with_position(pos, s(:defn, ident, params, *body))
@@ -21,12 +21,12 @@ module RipperRubyParser
       def process_defs(exp)
         _, receiver, _, ident, params, body, = exp.shift 7
 
-        ident, = extract_node_symbol_with_position ident
+        ident = extract_node_symbol ident
 
         in_method do
           params = convert_arguments(process(params))
           kwrest = kwrest_param(params)
-          body = with_kwrest(kwrest) { method_body(body) }
+          body = with_new_lvar_scope(kwrest) { method_body(body) }
         end
 
         s(:defs, process(receiver), ident, params, *body)
@@ -103,6 +103,8 @@ module RipperRubyParser
       }.freeze
 
       def convert_arguments(args)
+        return s(:args) if args.nil?
+
         args.line ||= args.sexp_body.first&.line
         args.sexp_body = args.sexp_body.map { |item| convert_argument item }
         args
@@ -123,7 +125,7 @@ module RipperRubyParser
 
       def convert_marked_argument(item)
         marker = SPECIAL_ARG_MARKER[item.sexp_type]
-        name = extract_node_symbol item.last
+        name = make_symbol item.last
         :"#{marker}#{name}"
       end
 
@@ -151,15 +153,16 @@ module RipperRubyParser
         Regexp.last_match[1].to_sym if found
       end
 
-      def with_kwrest(kwrest)
-        @kwrest.push kwrest
+      def with_new_lvar_scope(extra_variable)
+        old_lvars = @local_variables.dup
+        @local_variables.push extra_variable if extra_variable
         result = yield
-        @kwrest.pop
+        @local_variables = old_lvars
         result
       end
 
       def kwrest_arg?(method)
-        @kwrest.include?(method)
+        @local_variables.include?(method)
       end
     end
   end

data/lib/ripper_ruby_parser/sexp_handlers/operators.rb

@@ -11,27 +11,28 @@ module RipperRubyParser
         or: :or
       }.freeze
 
+      BINARY_LOGICAL_OPERATORS = BINARY_OPERATOR_MAP.keys.freeze
+
       UNARY_OPERATOR_MAP = {
         not: :!
       }.freeze
 
-      NEGATED_BINARY_OPERATOR_MAP = {
-        "!~": :=~
-      }.freeze
-
       SHIFT_OPERATORS = [:<<, :>>].freeze
 
       def process_binary(exp)
         _, left, op, right = exp.shift 4
 
-        if op == :=~
+        case op
+        when :=~
           make_regexp_match_operator(left, op, right)
-        elsif (mapped = NEGATED_BINARY_OPERATOR_MAP[op])
-          s(:not, make_regexp_match_operator(left, mapped, right))
-        elsif (mapped = BINARY_OPERATOR_MAP[op])
-          make_boolean_operator(left, mapped, right)
-        elsif SHIFT_OPERATORS.include? op
+        when :!~
+          s(:not, make_regexp_match_operator(left, :=~, right))
+        when *BINARY_LOGICAL_OPERATORS
+          make_boolean_operator(left, op, right)
+        when *SHIFT_OPERATORS
           s(:call, unwrap_begin(process(left)), op, unwrap_begin(process(right)))
+        when :"=>"
+          make_rightward_assignment(left, right)
         else
           s(:call, process(left), op, process(right))
         end
@@ -77,20 +78,33 @@ module RipperRubyParser
       private
 
       def make_boolean_operator(left, operator, right)
+        operator = BINARY_OPERATOR_MAP[operator]
         _, left, _, right = rebalance_binary(left, operator, right)
         s(operator, unwrap_begin(process(left)), process(right))
       end
 
       def make_regexp_match_operator(left, operator, right)
-        if left.sexp_type == :regexp_literal
-          s(:match2, process(left), process(right))
-        elsif right.sexp_type == :regexp_literal
-          s(:match3, process(right), process(left))
+        left = process(left)
+        right = process(right)
+
+        if regexp? left
+          s(:match2, left, right)
+        elsif regexp? right
+          s(:match3, right, left)
         else
-          s(:call, process(left), operator, process(right))
+          s(:call, left, operator, right)
         end
       end
 
+      def regexp?(exp)
+        exp.sexp_type == :dregx ||
+          exp.sexp_type == :lit && exp.sexp_body.first.is_a?(Regexp)
+      end
+
+      def make_rightward_assignment(left, right)
+        s(:lasgn, process(right)[1], process(left))
+      end
+
       def rebalance_binary(left, operator, right)
         if BINARY_OPERATOR_MAP[operator] == BINARY_OPERATOR_MAP[left[2]]
           _, left, _, middle = rebalance_binary(*left.sexp_body)

data/lib/ripper_ruby_parser/sexp_handlers/string_literals.rb

@@ -83,7 +83,7 @@ module RipperRubyParser
           return with_line_number(content.line, s(:lit, Regexp.new(content.last, numflags)))
         end
 
-        content.sexp_type = :dregx_once if /o/.match?(flags)
+        content.sexp_type = :dregx_once if flags.include?("o")
         content << numflags unless numflags == 0
         content
       end
@@ -123,7 +123,7 @@ module RipperRubyParser
 
       def process_at_tstring_content(exp)
         _, content, pos, delim = exp.shift 4
-        string = fix_encoding handle_string_unescaping(content, delim)
+        string = handle_string_unescaping(content, delim)
         with_position(pos, s(:str, string))
       end
 
@@ -180,11 +180,11 @@ module RipperRubyParser
       def character_flags_to_numerical(flags)
         numflags = 0
 
-        numflags = Regexp::MULTILINE if /m/.match?(flags)
-        numflags |= Regexp::EXTENDED if /x/.match?(flags)
-        numflags |= Regexp::IGNORECASE if /i/.match?(flags)
+        numflags = Regexp::MULTILINE if flags.include?("m")
+        numflags |= Regexp::EXTENDED if flags.include?("x")
+        numflags |= Regexp::IGNORECASE if flags.include?("i")
 
-        numflags |= Regexp::NOENCODING if /n/.match?(flags)
+        numflags |= Regexp::NOENCODING if flags.include?("n")
         numflags |= Regexp::FIXEDENCODING if /[ues]/.match?(flags)
 
         numflags
@@ -227,22 +227,23 @@ module RipperRubyParser
         end
       end
 
-      INTERPOLATING_HEREDOC = /^<<[-~]?[^-~']/.freeze
-      NON_INTERPOLATING_HEREDOC = /^<<[-~]?'/.freeze
-      INTERPOLATING_STRINGS = ['"', "`", ':"', /^%Q.$/, /^%.$/].freeze
+      INTERPOLATING_HEREDOC = /^<<[-~]?[^-~']/
+      NON_INTERPOLATING_HEREDOC = /^<<[-~]?'/
+      INTERPOLATING_STRINGS = ['"', "`", /^%Q.$/, /^%.$/].freeze
+      INTERPOLATING_DSYM = ':"'
       NON_INTERPOLATING_STRINGS = ["'", ":'", /^%q.$/].freeze
-      INTERPOLATING_WORD_LIST = /^%[WI].$/.freeze
-      NON_INTERPOLATING_WORD_LIST = /^%[wi].$/.freeze
+      INTERPOLATING_WORD_LIST = /^%[WI].$/
+      NON_INTERPOLATING_WORD_LIST = /^%[wi].$/
       REGEXP_LITERALS = ["/", /^%r.$/].freeze
 
       def handle_string_unescaping(content, delim)
         case delim
-        when INTERPOLATING_HEREDOC, *INTERPOLATING_STRINGS
+        when INTERPOLATING_HEREDOC, INTERPOLATING_DSYM, *INTERPOLATING_STRINGS
           unescape(content)
         when INTERPOLATING_WORD_LIST
-          unescape_wordlist_word(content)
+          fix_encoding unescape_wordlist_word(content)
         when *NON_INTERPOLATING_STRINGS
-          simple_unescape(content, delim)
+          fix_encoding simple_unescape(content, delim)
         when *REGEXP_LITERALS
           unescape_regexp(content)
         when NON_INTERPOLATING_WORD_LIST

data/lib/ripper_ruby_parser/sexp_processor.rb

@@ -17,9 +17,7 @@ module RipperRubyParser
       super()
 
       public_methods.each do |name|
-        if name =~ /^process_at_(.*)/
-          @processors["@#{Regexp.last_match(1)}".to_sym] = name.to_sym
-        end
+        @processors[:"@#{Regexp.last_match(1)}"] = name.to_sym if name =~ /^process_at_(.*)/
       end
 
       @filename = filename
@@ -28,8 +26,7 @@ module RipperRubyParser
       @errors = []
 
       @in_method_body = false
-      @kwrest = []
-      @block_kwrest = []
+      @local_variables = []
 
       @kept_comment = nil
     end
@@ -77,7 +74,7 @@ module RipperRubyParser
 
     def process_var_field(exp)
       _, contents = exp.shift 2
-      process(contents)
+      process(contents) || s(:lvar, nil)
     end
 
     def process_var_alias(exp)
@@ -92,7 +89,7 @@ module RipperRubyParser
 
     def process_const_path_ref(exp)
       _, left, right = exp.shift 3
-      s(:colon2, process(left), extract_node_symbol(process(right)))
+      s(:colon2, process(left), make_symbol(process(right)))
     end
 
     def process_const_path_field(exp)
@@ -106,7 +103,7 @@ module RipperRubyParser
 
     def process_top_const_ref(exp)
       _, ref = exp.shift 2
-      s(:colon3, extract_node_symbol(process(ref)))
+      s(:colon3, make_symbol(process(ref)))
     end
 
     def process_top_const_field(exp)

data/lib/ripper_ruby_parser/unescape.rb

@@ -20,7 +20,7 @@ module RipperRubyParser
         M-.                 | # meta
         \n                  | # line break
         .                     # other single character
-      )/x.freeze
+      )/x
 
     SINGLE_LETTER_ESCAPES = {
       "a" => "\a",
@@ -74,7 +74,7 @@ module RipperRubyParser
     def unescape(string)
       string = string.dup if string.frozen?
       string.force_encoding("ASCII-8BIT")
-      string.gsub(ESCAPE_SEQUENCE_REGEXP) do
+      result = string.gsub(ESCAPE_SEQUENCE_REGEXP) do
         bare = Regexp.last_match[1]
         if bare == "\n"
           ""
@@ -82,6 +82,7 @@ module RipperRubyParser
           unescaped_value(bare).force_encoding("ASCII-8BIT")
         end
       end
+      fix_encoding result
     end
 
     def unescape_wordlist_word(string)
@@ -169,6 +170,9 @@ module RipperRubyParser
     end
 
     def control(val)
+      # Special case the \C-? or DEL sequence
+      return 127 if val == 63
+
       val & 0b1001_1111
     end
 

data/lib/ripper_ruby_parser/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RipperRubyParser
-  VERSION = "1.10.0"
+  VERSION = "1.11.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ripper_ruby_parser
 version: !ruby/object:Gem::Version
-  version: 1.10.0
+  version: 1.11.0
 platform: ruby
 authors:
 - Matijs van Zuijlen
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-03-13 00:00:00.000000000 Z
+date: 2024-01-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sexp_processor
@@ -106,70 +106,70 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.25'
+        version: '1.56'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.25'
+        version: '1.56'
 - !ruby/object:Gem::Dependency
   name: rubocop-minitest
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.17.0
+        version: 0.34.1
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.17.0
+        version: 0.34.1
 - !ruby/object:Gem::Dependency
   name: rubocop-packaging
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.5.1
+        version: 0.5.2
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.5.1
+        version: 0.5.2
 - !ruby/object:Gem::Dependency
   name: rubocop-performance
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.13'
+        version: '1.19'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.13'
+        version: '1.19'
 - !ruby/object:Gem::Dependency
   name: ruby_parser
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.18'
+        version: 3.20.2
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '3.18'
+        version: 3.20.2
 - !ruby/object:Gem::Dependency
   name: sexp_processor
   requirement: !ruby/object:Gem::Requirement
@@ -190,14 +190,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.21.0
+        version: 0.22.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.21.0
+        version: 0.22.0
 description: |
   RipperRubyParser is a parser for Ruby based on Ripper that aims to be a
   drop-in replacement for RubyParser.
@@ -246,14 +246,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.6.0
+      version: 3.0.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.33
+rubygems_version: 3.5.3
 signing_key:
 specification_version: 4
 summary: Parse with Ripper, produce sexps that are compatible with RubyParser.