ripper_ruby_parser 1.9.0 → 1.11.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4e4ee66b4ce6abf5b0eeacd55fd137dd7217bf728bc59f1ad6bdf0b197011f86
4
- data.tar.gz: a35a48bfa5178c5a9cec64017842246a4acb8b10a0c09fba2749de9aa650772c
3
+ metadata.gz: 568f21d8800cba790d212bb45e96554f93a271da3d9bae483fbf101385906d8d
4
+ data.tar.gz: fa4393288ca29703405077d9da524a51cb0d7575c08b8dcabcc9be3bcbe5d6d0
5
5
  SHA512:
6
- metadata.gz: 0574addabb87b0f843c2a72a3e5c03b6ffff2461a9f1fa551412ee706841a2baeb354c1decd3ca7c902e1e78c2c2b7c4a346784ebb6c5b1064211b4ca08b8039
7
- data.tar.gz: cffd4a9f013b443e82b3b4d6149719b02119f182aafe86fc2984d81b131bc047bd1143ca5b06cd28dc32e71876cbb67452c9db47979a50d766e28ea7b7f590a2
6
+ metadata.gz: 5dcfda84f25d0351b0351e06eb231deaab73d6377991a6b1939dc9c4df268c8cfb0960c22ca50a0d3a5dec432a855ef2ddb8a66171b321855ae2732c1f2d5518
7
+ data.tar.gz: 7c68ec079fe840e6990d1555b1751639b46113f636fcc4e90429af4a3c36382f19ba6fbd6c3c0016271e1082412dd6f9001c7f8fb7c20858aec06d1d88a4af59
data/CHANGELOG.md CHANGED
@@ -6,6 +6,64 @@ This project adheres to [Semantic Versioning 2.0.0][1].
6
6
 
7
7
  This document is formatted based on [Keep A CHANGELOG][2].
8
8
 
9
+ ## 1.11.0 / 2024-01-05
10
+
11
+ * Support Ruby 3.0 through 3.3, dropping support for Ruby 2.6 and 2.7
12
+ ([#218], [#219], [#233] and [#246] by [mvz])
13
+ * Target compatibility with `ruby_parser` version 3.20.2
14
+ ([#199], [#216] and [#230] by [mvz])
15
+ * Support single-line pattern matching ([#188] by [mvz])
16
+ * Support rightward assignment ([#189] by [mvz])
17
+ * Support endless methods with Ruby 3.0 syntax ([#191] by [mvz])
18
+ * Support arguments without parentheses in endless method body ([#193] by [mvz])
19
+ * Add support for pattern matching ([#172] by [mvz])
20
+ * Add tentative support for numbered parameters ([#163] by [mvz])
21
+ * Support several new Ruby 3.1 syntax features ([#196] by [mvz])
22
+ * Support negative rational and imaginary literals ([#197] by [mvz])
23
+ * Correctly handle match operator with regexp literals in parentheses
24
+ ([#200] by [mvz])
25
+ * Improve operator assignment handling ([#201] by [mvz])
26
+ * Handle literal Ctrl-? (DEL) character correctly ([#202] by [mvz])
27
+ * Handle method names that are keywords in method definitions ([#204] by [mvz])
28
+ * Improve dsym handling compatibility ([#203], [#210] by [mvz])
29
+ * Add support for Ruby 3.2's splat and kwsplat argument forwarding ([#231] by [mvz])
30
+ * Improve handling of the case .. in construction ([#234] by [mvz])
31
+
32
+ [mvz]: https://github.com/mvz
33
+
34
+ [#163]: https://github.com/mvz/ripper_ruby_parser/pull/163
35
+ [#172]: https://github.com/mvz/ripper_ruby_parser/pull/172
36
+ [#188]: https://github.com/mvz/ripper_ruby_parser/pull/188
37
+ [#189]: https://github.com/mvz/ripper_ruby_parser/pull/189
38
+ [#191]: https://github.com/mvz/ripper_ruby_parser/pull/191
39
+ [#193]: https://github.com/mvz/ripper_ruby_parser/pull/193
40
+ [#196]: https://github.com/mvz/ripper_ruby_parser/pull/196
41
+ [#197]: https://github.com/mvz/ripper_ruby_parser/pull/197
42
+ [#199]: https://github.com/mvz/ripper_ruby_parser/pull/199
43
+ [#200]: https://github.com/mvz/ripper_ruby_parser/pull/200
44
+ [#201]: https://github.com/mvz/ripper_ruby_parser/pull/201
45
+ [#202]: https://github.com/mvz/ripper_ruby_parser/pull/202
46
+ [#203]: https://github.com/mvz/ripper_ruby_parser/pull/203
47
+ [#204]: https://github.com/mvz/ripper_ruby_parser/pull/204
48
+ [#205]: https://github.com/mvz/ripper_ruby_parser/pull/205
49
+ [#210]: https://github.com/mvz/ripper_ruby_parser/pull/210
50
+ [#216]: https://github.com/mvz/ripper_ruby_parser/pull/216
51
+ [#218]: https://github.com/mvz/ripper_ruby_parser/pull/218
52
+ [#219]: https://github.com/mvz/ripper_ruby_parser/pull/219
53
+ [#230]: https://github.com/mvz/ripper_ruby_parser/pull/230
54
+ [#231]: https://github.com/mvz/ripper_ruby_parser/pull/231
55
+ [#232]: https://github.com/mvz/ripper_ruby_parser/pull/232
56
+ [#233]: https://github.com/mvz/ripper_ruby_parser/pull/233
57
+ [#234]: https://github.com/mvz/ripper_ruby_parser/pull/234
58
+ [#235]: https://github.com/mvz/ripper_ruby_parser/pull/235
59
+ [#246]: https://github.com/mvz/ripper_ruby_parser/pull/246
60
+
61
+ ## 1.10.0 / 2022-03-13
62
+
63
+ * Handle shadow arguments ([#161])
64
+ * Drop support for Ruby 2.5 ([#165])
65
+ * Support running on Ruby 3.1([#180])
66
+
9
67
  ## 1.9.0 / 2021-08-10
10
68
 
11
69
  * Fix escape sequence handling in non-interpolating strings and word lists
@@ -226,6 +284,11 @@ This document is formatted based on [Keep A CHANGELOG][2].
226
284
  * Initial release
227
285
 
228
286
  <!-- Pull request links -->
287
+ [#180]: https://github.com/mvz/ripper_ruby_parser/pull/180
288
+ [#172]: https://github.com/mvz/ripper_ruby_parser/pull/172
289
+ [#163]: https://github.com/mvz/ripper_ruby_parser/pull/163
290
+ [#165]: https://github.com/mvz/ripper_ruby_parser/pull/165
291
+ [#161]: https://github.com/mvz/ripper_ruby_parser/pull/161
229
292
  [#155]: https://github.com/mvz/ripper_ruby_parser/pull/155
230
293
  [#154]: https://github.com/mvz/ripper_ruby_parser/pull/154
231
294
  [#153]: https://github.com/mvz/ripper_ruby_parser/pull/153
data/README.md CHANGED
@@ -10,8 +10,8 @@ Parse with Ripper, produce sexps that are compatible with RubyParser.
10
10
 
11
11
  * Drop-in replacement for RubyParser
12
12
  * Should handle 1.9 and later syntax gracefully
13
- * Requires Ruby 2.5 or higher
14
- * Compatible with RubyParser 3.17.0
13
+ * Requires Ruby 3.0 or higher
14
+ * Compatible with RubyParser 3.20.2
15
15
 
16
16
  ## Known incompatibilities
17
17
 
@@ -25,7 +25,9 @@ RipperRubyParser has a few incompatibilities with RubyParser.
25
25
 
26
26
  ## Install
27
27
 
28
- gem install ripper_ruby_parser
28
+ ```bash
29
+ gem install ripper_ruby_parser
30
+ ```
29
31
 
30
32
  ## Synopsis
31
33
 
@@ -44,7 +46,7 @@ parser.parse "foo[bar] += baz qux"
44
46
 
45
47
  ## Requirements
46
48
 
47
- * Ruby 2.5 or higher
49
+ * Ruby 2.7 or higher
48
50
  * `sexp_processor`
49
51
 
50
52
  ## Hacking and contributing
@@ -64,7 +66,7 @@ If you want to send pull requests or patches, please:
64
66
 
65
67
  (The MIT License)
66
68
 
67
- Copyright (c) 2012, 2014-2020 Matijs van Zuijlen
69
+ Copyright (c) 2012, 2014-2024 Matijs van Zuijlen
68
70
 
69
71
  Permission is hereby granted, free of charge, to any person obtaining
70
72
  a copy of this software and associated documentation files (the
@@ -8,6 +8,7 @@ module RipperRubyParser
8
8
  # Variant of Ripper's SexpBuilder parser class that inserts comments as
9
9
  # Sexps into the built parse tree.
10
10
  #
11
+ # rubocop: disable Metrics/ClassLength
11
12
  # @api private
12
13
  class CommentingRipperParser < Ripper::SexpBuilder
13
14
  def initialize(*args)
@@ -35,7 +36,16 @@ module RipperRubyParser
35
36
  end
36
37
 
37
38
  def on_begin(*args)
38
- commentize(:begin, super)
39
+ result = super
40
+
41
+ # Some begin blocks are not created by the 'begin' keyword. Skip
42
+ # commenting for those kinds of blocks.
43
+ (_, kw,), = @comment_stack.last
44
+ if kw == "begin"
45
+ commentize("begin", result)
46
+ else
47
+ result
48
+ end
39
49
  end
40
50
 
41
51
  def on_void_stmt
@@ -72,23 +82,25 @@ module RipperRubyParser
72
82
  end
73
83
 
74
84
  def on_module(*args)
75
- commentize(:module, super)
85
+ commentize("module", super)
76
86
  end
77
87
 
78
88
  def on_class(*args)
79
- commentize(:class, super)
89
+ commentize("class", super)
80
90
  end
81
91
 
82
92
  def on_sclass(*args)
83
- commentize(:class, super)
93
+ commentize("class", super)
84
94
  end
85
95
 
86
- def on_def(*args)
87
- commentize(:def, super)
96
+ def on_def(name, *args)
97
+ (_, _, loc) = name
98
+ commentize("def", super, loc)
88
99
  end
89
100
 
90
- def on_defs(*args)
91
- commentize(:def, super)
101
+ def on_defs(receiver, period, name, *rest)
102
+ (_, _, loc) = name
103
+ commentize("def", super, loc)
92
104
  end
93
105
 
94
106
  def on_args_new
@@ -254,6 +266,11 @@ module RipperRubyParser
254
266
  @seen_space = true
255
267
  end
256
268
 
269
+ def on_imaginary(_token)
270
+ @space_before = @seen_space
271
+ super
272
+ end
273
+
257
274
  def on_int(_token)
258
275
  @space_before = @seen_space
259
276
  super
@@ -264,7 +281,12 @@ module RipperRubyParser
264
281
  super
265
282
  end
266
283
 
267
- NUMBER_LITERAL_TYPES = [:@int, :@float].freeze
284
+ def on_rational(_token)
285
+ @space_before = @seen_space
286
+ super
287
+ end
288
+
289
+ NUMBER_LITERAL_TYPES = [:@imaginary, :@int, :@float, :@rational].freeze
268
290
 
269
291
  def on_unary(operator, value)
270
292
  if !@space_before && operator == :-@ && NUMBER_LITERAL_TYPES.include?(value.first)
@@ -300,15 +322,16 @@ module RipperRubyParser
300
322
  end
301
323
 
302
324
  def on_BEGIN(*args)
303
- commentize(:BEGIN, super)
325
+ commentize("BEGIN", super)
304
326
  end
305
327
 
306
328
  def on_END(*args)
307
- commentize(:END, super)
329
+ commentize("END", super)
308
330
  end
309
331
 
310
332
  def on_parse_error(message)
311
- raise SyntaxError, message
333
+ super
334
+ raise SyntaxError, message if message.start_with?("syntax error,")
312
335
  end
313
336
 
314
337
  def on_class_name_error(message, *)
@@ -327,11 +350,19 @@ module RipperRubyParser
327
350
  raise SyntaxError, message
328
351
  end
329
352
 
330
- def commentize(_name, exp)
331
- (_, _kw, loc), comment = @comment_stack.pop
353
+ def commentize(name, exp, target_loc = nil)
354
+ if target_loc
355
+ (_, kw, loc), comment = @comment_stack.pop until (loc <=> target_loc) == -1
356
+ else
357
+ (_, kw, loc), comment = @comment_stack.pop
358
+ end
359
+
360
+ warn "Comment stack mismatch: expected #{kw} to equal #{name}" unless kw == name
361
+
332
362
  @comment = ""
333
363
  exp.push loc
334
364
  [:comment, comment, exp]
335
365
  end
336
366
  end
367
+ # rubocop: enable Metrics/ClassLength
337
368
  end
@@ -95,50 +95,74 @@ module RipperRubyParser
95
95
  def process_opassign(exp)
96
96
  _, lvalue, (_, operator,), value = exp.shift 4
97
97
 
98
+ value_type = value.sexp_type
99
+
98
100
  lvalue = process(lvalue)
99
101
  value = process(value)
100
102
  operator = operator.chop.to_sym
101
103
 
102
- create_operator_assignment_sub_type lvalue, value, operator
104
+ case lvalue.sexp_type
105
+ when :aref_field
106
+ create_aref_operator_assignment_sub_type(lvalue, value, operator)
107
+ when :field
108
+ create_field_operator_assignment_sub_type(lvalue, value, operator, value_type)
109
+ else
110
+ create_regular_operator_assignment_sub_type(lvalue, value, operator)
111
+ end
103
112
  end
104
113
 
105
114
  private
106
115
 
107
116
  def create_multiple_assignment_sub_types(sexp_list)
108
117
  sexp_list.map! do |item|
109
- create_valueless_assignment_sub_type item
118
+ create_valueless_regular_assignment_sub_type item
110
119
  end
111
120
  end
112
121
 
113
- def create_valueless_assignment_sub_type(item)
122
+ def create_valueless_regular_assignment_sub_type(item)
114
123
  item = with_line_number(item.line,
115
124
  create_regular_assignment_sub_type(item, nil))
116
125
  item.pop
117
126
  item
118
127
  end
119
128
 
129
+ def create_valueless_assignment_sub_type(item)
130
+ item = with_line_number(item.line,
131
+ create_assignment_sub_type(item, nil))
132
+ item.pop
133
+ item
134
+ end
135
+
136
+ def create_aref_operator_assignment_sub_type(lvalue, value, operator)
137
+ _, arr, arglist = lvalue
138
+ arglist.sexp_type = :arglist
139
+ s(:op_asgn1, arr, arglist, operator, value)
140
+ end
141
+
142
+ def create_field_operator_assignment_sub_type(lvalue, value, operator, value_type)
143
+ # Structure of lvalue will be something like this:
144
+ # s(:field, receiver, s(:period, "."), s(:lvar, field))
145
+ _, receiver, _, (_, field) = lvalue
146
+ case value_type
147
+ when :command, :command_call
148
+ s(:op_asgn, receiver, value, field, operator)
149
+ else
150
+ s(:op_asgn2, receiver, :"#{field}=", operator, value)
151
+ end
152
+ end
153
+
120
154
  OPERATOR_ASSIGNMENT_MAP = {
121
155
  "||": :op_asgn_or,
122
156
  "&&": :op_asgn_and
123
157
  }.freeze
124
158
 
125
- def create_operator_assignment_sub_type(lvalue, value, operator)
126
- case lvalue.sexp_type
127
- when :aref_field
128
- _, arr, arglist = lvalue
129
- arglist.sexp_type = :arglist
130
- s(:op_asgn1, arr, arglist, operator, value)
131
- when :field
132
- _, obj, _, (_, field) = lvalue
133
- s(:op_asgn2, obj, :"#{field}=", operator, value)
159
+ def create_regular_operator_assignment_sub_type(lvalue, value, operator)
160
+ value = unwrap_begin(value)
161
+ if (mapped = OPERATOR_ASSIGNMENT_MAP[operator])
162
+ s(mapped, lvalue, create_assignment_sub_type(lvalue, value))
134
163
  else
135
- value = unwrap_begin(value)
136
- if (mapped = OPERATOR_ASSIGNMENT_MAP[operator])
137
- s(mapped, lvalue, create_assignment_sub_type(lvalue, value))
138
- else
139
- operator_call = s(:call, lvalue, operator, value)
140
- create_assignment_sub_type lvalue, operator_call
141
- end
164
+ operator_call = s(:call, lvalue, operator, value)
165
+ create_assignment_sub_type lvalue, operator_call
142
166
  end
143
167
  end
144
168
 
@@ -10,10 +10,13 @@ module RipperRubyParser
10
10
  call = process(call)
11
11
  args = process(args)
12
12
  kwrest = kwrest_param(args) if args
13
- stmt = with_kwrest(kwrest) { process(stmt) }
13
+ stmt = with_new_lvar_scope(kwrest) { process(stmt) }
14
14
  make_iter call, args, safe_unwrap_void_stmt(stmt)
15
15
  end
16
16
 
17
+ # NOTE: Argument forwarding is handled differently in Ruby 3.0 and 3.1
18
+ # 3.0: s(:params, nil, nil, s(:args_forward), nil, nil, nil, nil)
19
+ # 3.1: s(:params, nil, nil, nil, nil, nil, s(:args_forward), :&)
17
20
  def process_params(exp)
18
21
  _, normal, defaults, splat, rest, kwargs, doublesplat, block = exp.shift 8
19
22
 
@@ -40,10 +43,15 @@ module RipperRubyParser
40
43
  end
41
44
 
42
45
  def process_block_var(exp)
43
- _, args, = exp.shift 3
46
+ _, args, shadowargs = exp.shift 3
44
47
 
45
48
  names = process(args)
46
49
 
50
+ if shadowargs
51
+ shadowargs = map_process_list(shadowargs).map { |item| item[1] }
52
+ names << s(:shadow, *shadowargs)
53
+ end
54
+
47
55
  convert_arguments names
48
56
  end
49
57
 
@@ -145,16 +153,13 @@ module RipperRubyParser
145
153
 
146
154
  defaults.map do |sym, val|
147
155
  s(:lasgn,
148
- extract_node_symbol(process(sym)),
156
+ make_symbol(process(sym)),
149
157
  process(val))
150
158
  end
151
159
  end
152
160
 
153
161
  def handle_splat(splat)
154
- if splat == 0
155
- # Only relevant for Ruby < 2.6
156
- [s(:excessed_comma)]
157
- elsif splat
162
+ if splat
158
163
  [process(splat)]
159
164
  else
160
165
  []
@@ -165,7 +170,7 @@ module RipperRubyParser
165
170
  return [] unless kwargs
166
171
 
167
172
  kwargs.map do |sym, val|
168
- symbol = extract_node_symbol process(sym)
173
+ symbol = make_symbol process(sym)
169
174
  if val
170
175
  s(:kwarg, symbol, process(val))
171
176
  else
@@ -177,11 +182,18 @@ module RipperRubyParser
177
182
  def handle_double_splat(doublesplat)
178
183
  return [] unless doublesplat
179
184
 
180
- [s(:dsplat, process(doublesplat))]
185
+ contents = process(doublesplat)
186
+ case contents.sexp_type
187
+ when :forward_args # Argument forwarding in Ruby 3.1
188
+ [contents]
189
+ else
190
+ [s(:dsplat, contents)]
191
+ end
181
192
  end
182
193
 
183
194
  def handle_block_argument(block)
184
195
  return [] unless block
196
+ return [] if block == :& # Part of argument forwarding in Ruby 3.1; ignore
185
197
 
186
198
  [process(block)]
187
199
  end
@@ -202,15 +214,25 @@ module RipperRubyParser
202
214
  end
203
215
  end
204
216
 
217
+ LVAR_MATCHER = Sexp::Matcher.new(:lvar, Sexp._)
218
+ NUMBERED_PARAMS = (1..9).map { |it| :"_#{it}" }.freeze
219
+
205
220
  def make_iter(call, args, stmt)
206
221
  args[-1] = nil if args && args.last == s(:excessed_comma)
207
- args ||= 0
222
+
223
+ args ||= count_numbered_lvars(stmt)
224
+
208
225
  if stmt.empty?
209
226
  s(:iter, call, args)
210
227
  else
211
228
  s(:iter, call, args, stmt)
212
229
  end
213
230
  end
231
+
232
+ def count_numbered_lvars(stmt)
233
+ lvar_names = (LVAR_MATCHER / stmt).map { |it| it[1] }
234
+ (NUMBERED_PARAMS & lvar_names).length
235
+ end
214
236
  end
215
237
  end
216
238
  end
@@ -64,11 +64,78 @@ module RipperRubyParser
64
64
  *falsepart)
65
65
  end
66
66
 
67
+ def process_in(exp)
68
+ _, pattern, truepart, falsepart = exp.shift 4
69
+
70
+ falsepart = process(falsepart)
71
+ falsepart = if falsepart.nil?
72
+ [nil]
73
+ else
74
+ falsepart.sexp_body
75
+ end
76
+ pattern = process(pattern)
77
+ adjust_rightward_assignment_pattern(pattern)
78
+
79
+ truepart = process(truepart)
80
+ truepart = unwrap_nil(truepart) if truepart
81
+
82
+ s(:case_body,
83
+ s(:in, pattern, truepart),
84
+ *falsepart)
85
+ end
86
+
67
87
  def process_else(exp)
68
88
  _, body = exp.shift 2
69
89
  process(body)
70
90
  end
71
91
 
92
+ def process_aryptn(exp)
93
+ _, _, body, rest, = exp.shift 5
94
+
95
+ elements = body.map do |elem|
96
+ if elem.sexp_type == :var_field
97
+ create_valueless_assignment_sub_type process(elem)
98
+ else
99
+ unwrap_begin process(elem)
100
+ end
101
+ end
102
+ if rest
103
+ rest_var = handle_pattern(rest)
104
+ elements << convert_marked_argument(s(:splat, rest_var))
105
+ end
106
+ s(:array_pat, nil, *elements)
107
+ end
108
+
109
+ def process_hshptn(exp)
110
+ _, _, body, = exp.shift 4
111
+
112
+ elements = body.flat_map do |key, value|
113
+ if value
114
+ [process(key), process(value)]
115
+ else
116
+ [handle_pattern(key), nil]
117
+ end
118
+ end
119
+ s(:hash_pat, nil, *elements)
120
+ end
121
+
122
+ def process_fndptn(exp)
123
+ _, wrapper, before, patterns, after = exp.shift 5
124
+
125
+ wrapper = process(wrapper)
126
+ before = make_splat process(before)
127
+ after = make_splat process(after)
128
+ patterns = patterns.map do |elem|
129
+ if elem.sexp_type == :var_field
130
+ create_valueless_assignment_sub_type process(elem)
131
+ else
132
+ unwrap_begin process(elem)
133
+ end
134
+ end
135
+
136
+ s(:find_pat, wrapper, before, *patterns, after)
137
+ end
138
+
72
139
  private
73
140
 
74
141
  def handle_condition(cond)
@@ -88,6 +155,24 @@ module RipperRubyParser
88
155
  unwrap_nil process(exp) if exp
89
156
  end
90
157
 
158
+ def handle_pattern(exp)
159
+ pattern = process(exp)
160
+ case pattern.sexp_type
161
+ when :lvar, :lit
162
+ @local_variables << pattern[1]
163
+ end
164
+ pattern
165
+ end
166
+
167
+ def adjust_rightward_assignment_pattern(exp)
168
+ case exp.sexp_type
169
+ when :lvar
170
+ exp.sexp_type = :lasgn
171
+ when :lasgn
172
+ adjust_rightward_assignment_pattern(exp.sexp_body.last)
173
+ end
174
+ end
175
+
91
176
  def construct_conditional(cond, truepart, falsepart)
92
177
  if cond.sexp_type == :not
93
178
  _, inner = cond
@@ -107,6 +192,10 @@ module RipperRubyParser
107
192
  [exp]
108
193
  end
109
194
  end
195
+
196
+ def make_splat(exp)
197
+ :"*#{exp[1]}"
198
+ end
110
199
  end
111
200
  end
112
201
  end
@@ -4,12 +4,17 @@ module RipperRubyParser
4
4
  module SexpHandlers
5
5
  # Utility methods used in several of the sexp handler modules
6
6
  module HelperMethods
7
+ def extract_node_symbol(exp)
8
+ ident, = extract_node_symbol_with_position(exp)
9
+ ident
10
+ end
11
+
7
12
  def extract_node_symbol_with_position(exp)
8
13
  _, ident, pos = exp.shift 3
9
14
  return ident.to_sym, pos
10
15
  end
11
16
 
12
- def extract_node_symbol(exp)
17
+ def make_symbol(exp)
13
18
  return nil if exp.nil?
14
19
  raise "Unexpected number of children: #{exp.length}" if exp.length != 2
15
20
 
@@ -36,7 +41,11 @@ module RipperRubyParser
36
41
  def generic_add_star(exp)
37
42
  _, args, splatarg, *rest = shift_all exp
38
43
  items = process args
39
- items.push s(:splat, unwrap_begin(process(splatarg)))
44
+ if splatarg
45
+ items.push s(:splat, unwrap_begin(process(splatarg)))
46
+ else
47
+ items.push s(:splat)
48
+ end
40
49
  items.push(*map_process_list(rest))
41
50
  end
42
51
 
@@ -7,7 +7,7 @@ module RipperRubyParser
7
7
  # character literals
8
8
  def process_at_CHAR(exp)
9
9
  _, val, pos = exp.shift 3
10
- with_position(pos, s(:str, fix_encoding(unescape(val[1..-1]))))
10
+ with_position(pos, s(:str, fix_encoding(unescape(val[1..]))))
11
11
  end
12
12
 
13
13
  def process_array(exp)
@@ -36,7 +36,11 @@ module RipperRubyParser
36
36
  # s(:assoc_splat, s(:vcall, s(:@ident, "bar")))
37
37
  def process_assoc_splat(exp)
38
38
  _, param = exp.shift 2
39
- s(:kwsplat, process(param))
39
+ if param
40
+ s(:kwsplat, process(param))
41
+ else
42
+ s(:kwsplat)
43
+ end
40
44
  end
41
45
 
42
46
  # number literals
@@ -7,12 +7,12 @@ module RipperRubyParser
7
7
  def process_def(exp)
8
8
  _, ident, params, body, pos = exp.shift 5
9
9
 
10
- ident, = extract_node_symbol_with_position ident
10
+ ident = extract_node_symbol ident
11
11
 
12
12
  in_method do
13
13
  params = convert_arguments(process(params))
14
14
  kwrest = kwrest_param(params)
15
- body = with_kwrest(kwrest) { method_body(body) }
15
+ body = with_new_lvar_scope(kwrest) { method_body(body) }
16
16
  end
17
17
 
18
18
  with_position(pos, s(:defn, ident, params, *body))
@@ -21,12 +21,12 @@ module RipperRubyParser
21
21
  def process_defs(exp)
22
22
  _, receiver, _, ident, params, body, = exp.shift 7
23
23
 
24
- ident, = extract_node_symbol_with_position ident
24
+ ident = extract_node_symbol ident
25
25
 
26
26
  in_method do
27
27
  params = convert_arguments(process(params))
28
28
  kwrest = kwrest_param(params)
29
- body = with_kwrest(kwrest) { method_body(body) }
29
+ body = with_new_lvar_scope(kwrest) { method_body(body) }
30
30
  end
31
31
 
32
32
  s(:defs, process(receiver), ident, params, *body)
@@ -103,6 +103,8 @@ module RipperRubyParser
103
103
  }.freeze
104
104
 
105
105
  def convert_arguments(args)
106
+ return s(:args) if args.nil?
107
+
106
108
  args.line ||= args.sexp_body.first&.line
107
109
  args.sexp_body = args.sexp_body.map { |item| convert_argument item }
108
110
  args
@@ -123,7 +125,7 @@ module RipperRubyParser
123
125
 
124
126
  def convert_marked_argument(item)
125
127
  marker = SPECIAL_ARG_MARKER[item.sexp_type]
126
- name = extract_node_symbol item.last
128
+ name = make_symbol item.last
127
129
  :"#{marker}#{name}"
128
130
  end
129
131
 
@@ -151,15 +153,16 @@ module RipperRubyParser
151
153
  Regexp.last_match[1].to_sym if found
152
154
  end
153
155
 
154
- def with_kwrest(kwrest)
155
- @kwrest.push kwrest
156
+ def with_new_lvar_scope(extra_variable)
157
+ old_lvars = @local_variables.dup
158
+ @local_variables.push extra_variable if extra_variable
156
159
  result = yield
157
- @kwrest.pop
160
+ @local_variables = old_lvars
158
161
  result
159
162
  end
160
163
 
161
164
  def kwrest_arg?(method)
162
- @kwrest.include?(method)
165
+ @local_variables.include?(method)
163
166
  end
164
167
  end
165
168
  end
@@ -11,27 +11,28 @@ module RipperRubyParser
11
11
  or: :or
12
12
  }.freeze
13
13
 
14
+ BINARY_LOGICAL_OPERATORS = BINARY_OPERATOR_MAP.keys.freeze
15
+
14
16
  UNARY_OPERATOR_MAP = {
15
17
  not: :!
16
18
  }.freeze
17
19
 
18
- NEGATED_BINARY_OPERATOR_MAP = {
19
- "!~": :=~
20
- }.freeze
21
-
22
20
  SHIFT_OPERATORS = [:<<, :>>].freeze
23
21
 
24
22
  def process_binary(exp)
25
23
  _, left, op, right = exp.shift 4
26
24
 
27
- if op == :=~
25
+ case op
26
+ when :=~
28
27
  make_regexp_match_operator(left, op, right)
29
- elsif (mapped = NEGATED_BINARY_OPERATOR_MAP[op])
30
- s(:not, make_regexp_match_operator(left, mapped, right))
31
- elsif (mapped = BINARY_OPERATOR_MAP[op])
32
- make_boolean_operator(left, mapped, right)
33
- elsif SHIFT_OPERATORS.include? op
28
+ when :!~
29
+ s(:not, make_regexp_match_operator(left, :=~, right))
30
+ when *BINARY_LOGICAL_OPERATORS
31
+ make_boolean_operator(left, op, right)
32
+ when *SHIFT_OPERATORS
34
33
  s(:call, unwrap_begin(process(left)), op, unwrap_begin(process(right)))
34
+ when :"=>"
35
+ make_rightward_assignment(left, right)
35
36
  else
36
37
  s(:call, process(left), op, process(right))
37
38
  end
@@ -77,20 +78,33 @@ module RipperRubyParser
77
78
  private
78
79
 
79
80
  def make_boolean_operator(left, operator, right)
81
+ operator = BINARY_OPERATOR_MAP[operator]
80
82
  _, left, _, right = rebalance_binary(left, operator, right)
81
83
  s(operator, unwrap_begin(process(left)), process(right))
82
84
  end
83
85
 
84
86
  def make_regexp_match_operator(left, operator, right)
85
- if left.sexp_type == :regexp_literal
86
- s(:match2, process(left), process(right))
87
- elsif right.sexp_type == :regexp_literal
88
- s(:match3, process(right), process(left))
87
+ left = process(left)
88
+ right = process(right)
89
+
90
+ if regexp? left
91
+ s(:match2, left, right)
92
+ elsif regexp? right
93
+ s(:match3, right, left)
89
94
  else
90
- s(:call, process(left), operator, process(right))
95
+ s(:call, left, operator, right)
91
96
  end
92
97
  end
93
98
 
99
+ def regexp?(exp)
100
+ exp.sexp_type == :dregx ||
101
+ exp.sexp_type == :lit && exp.sexp_body.first.is_a?(Regexp)
102
+ end
103
+
104
+ def make_rightward_assignment(left, right)
105
+ s(:lasgn, process(right)[1], process(left))
106
+ end
107
+
94
108
  def rebalance_binary(left, operator, right)
95
109
  if BINARY_OPERATOR_MAP[operator] == BINARY_OPERATOR_MAP[left[2]]
96
110
  _, left, _, middle = rebalance_binary(*left.sexp_body)
@@ -83,7 +83,7 @@ module RipperRubyParser
83
83
  return with_line_number(content.line, s(:lit, Regexp.new(content.last, numflags)))
84
84
  end
85
85
 
86
- content.sexp_type = :dregx_once if /o/.match?(flags)
86
+ content.sexp_type = :dregx_once if flags.include?("o")
87
87
  content << numflags unless numflags == 0
88
88
  content
89
89
  end
@@ -123,7 +123,7 @@ module RipperRubyParser
123
123
 
124
124
  def process_at_tstring_content(exp)
125
125
  _, content, pos, delim = exp.shift 4
126
- string = fix_encoding handle_string_unescaping(content, delim)
126
+ string = handle_string_unescaping(content, delim)
127
127
  with_position(pos, s(:str, string))
128
128
  end
129
129
 
@@ -180,11 +180,11 @@ module RipperRubyParser
180
180
  def character_flags_to_numerical(flags)
181
181
  numflags = 0
182
182
 
183
- numflags = Regexp::MULTILINE if /m/.match?(flags)
184
- numflags |= Regexp::EXTENDED if /x/.match?(flags)
185
- numflags |= Regexp::IGNORECASE if /i/.match?(flags)
183
+ numflags = Regexp::MULTILINE if flags.include?("m")
184
+ numflags |= Regexp::EXTENDED if flags.include?("x")
185
+ numflags |= Regexp::IGNORECASE if flags.include?("i")
186
186
 
187
- numflags |= Regexp::NOENCODING if /n/.match?(flags)
187
+ numflags |= Regexp::NOENCODING if flags.include?("n")
188
188
  numflags |= Regexp::FIXEDENCODING if /[ues]/.match?(flags)
189
189
 
190
190
  numflags
@@ -227,22 +227,23 @@ module RipperRubyParser
227
227
  end
228
228
  end
229
229
 
230
- INTERPOLATING_HEREDOC = /^<<[-~]?[^-~']/.freeze
231
- NON_INTERPOLATING_HEREDOC = /^<<[-~]?'/.freeze
232
- INTERPOLATING_STRINGS = ['"', "`", ':"', /^%Q.$/, /^%.$/].freeze
230
+ INTERPOLATING_HEREDOC = /^<<[-~]?[^-~']/
231
+ NON_INTERPOLATING_HEREDOC = /^<<[-~]?'/
232
+ INTERPOLATING_STRINGS = ['"', "`", /^%Q.$/, /^%.$/].freeze
233
+ INTERPOLATING_DSYM = ':"'
233
234
  NON_INTERPOLATING_STRINGS = ["'", ":'", /^%q.$/].freeze
234
- INTERPOLATING_WORD_LIST = /^%[WI].$/.freeze
235
- NON_INTERPOLATING_WORD_LIST = /^%[wi].$/.freeze
235
+ INTERPOLATING_WORD_LIST = /^%[WI].$/
236
+ NON_INTERPOLATING_WORD_LIST = /^%[wi].$/
236
237
  REGEXP_LITERALS = ["/", /^%r.$/].freeze
237
238
 
238
239
  def handle_string_unescaping(content, delim)
239
240
  case delim
240
- when INTERPOLATING_HEREDOC, *INTERPOLATING_STRINGS
241
+ when INTERPOLATING_HEREDOC, INTERPOLATING_DSYM, *INTERPOLATING_STRINGS
241
242
  unescape(content)
242
243
  when INTERPOLATING_WORD_LIST
243
- unescape_wordlist_word(content)
244
+ fix_encoding unescape_wordlist_word(content)
244
245
  when *NON_INTERPOLATING_STRINGS
245
- simple_unescape(content, delim)
246
+ fix_encoding simple_unescape(content, delim)
246
247
  when *REGEXP_LITERALS
247
248
  unescape_regexp(content)
248
249
  when NON_INTERPOLATING_WORD_LIST
@@ -17,9 +17,7 @@ module RipperRubyParser
17
17
  super()
18
18
 
19
19
  public_methods.each do |name|
20
- if name =~ /^process_at_(.*)/
21
- @processors["@#{Regexp.last_match(1)}".to_sym] = name.to_sym
22
- end
20
+ @processors[:"@#{Regexp.last_match(1)}"] = name.to_sym if name =~ /^process_at_(.*)/
23
21
  end
24
22
 
25
23
  @filename = filename
@@ -28,8 +26,7 @@ module RipperRubyParser
28
26
  @errors = []
29
27
 
30
28
  @in_method_body = false
31
- @kwrest = []
32
- @block_kwrest = []
29
+ @local_variables = []
33
30
 
34
31
  @kept_comment = nil
35
32
  end
@@ -77,7 +74,7 @@ module RipperRubyParser
77
74
 
78
75
  def process_var_field(exp)
79
76
  _, contents = exp.shift 2
80
- process(contents)
77
+ process(contents) || s(:lvar, nil)
81
78
  end
82
79
 
83
80
  def process_var_alias(exp)
@@ -92,7 +89,7 @@ module RipperRubyParser
92
89
 
93
90
  def process_const_path_ref(exp)
94
91
  _, left, right = exp.shift 3
95
- s(:colon2, process(left), extract_node_symbol(process(right)))
92
+ s(:colon2, process(left), make_symbol(process(right)))
96
93
  end
97
94
 
98
95
  def process_const_path_field(exp)
@@ -106,7 +103,7 @@ module RipperRubyParser
106
103
 
107
104
  def process_top_const_ref(exp)
108
105
  _, ref = exp.shift 2
109
- s(:colon3, extract_node_symbol(process(ref)))
106
+ s(:colon3, make_symbol(process(ref)))
110
107
  end
111
108
 
112
109
  def process_top_const_field(exp)
@@ -195,7 +192,7 @@ module RipperRubyParser
195
192
 
196
193
  def process_at_backref(exp)
197
194
  _, str, pos = exp.shift 3
198
- name = str[1..-1]
195
+ name = str[1..]
199
196
  with_position pos do
200
197
  if /[0-9]/.match?(name)
201
198
  s(:nth_ref, name.to_i)
@@ -20,7 +20,7 @@ module RipperRubyParser
20
20
  M-. | # meta
21
21
  \n | # line break
22
22
  . # other single character
23
- )/x.freeze
23
+ )/x
24
24
 
25
25
  SINGLE_LETTER_ESCAPES = {
26
26
  "a" => "\a",
@@ -74,7 +74,7 @@ module RipperRubyParser
74
74
  def unescape(string)
75
75
  string = string.dup if string.frozen?
76
76
  string.force_encoding("ASCII-8BIT")
77
- string.gsub(ESCAPE_SEQUENCE_REGEXP) do
77
+ result = string.gsub(ESCAPE_SEQUENCE_REGEXP) do
78
78
  bare = Regexp.last_match[1]
79
79
  if bare == "\n"
80
80
  ""
@@ -82,6 +82,7 @@ module RipperRubyParser
82
82
  unescaped_value(bare).force_encoding("ASCII-8BIT")
83
83
  end
84
84
  end
85
+ fix_encoding result
85
86
  end
86
87
 
87
88
  def unescape_wordlist_word(string)
@@ -131,7 +132,7 @@ module RipperRubyParser
131
132
  end
132
133
 
133
134
  def unescape_hex_char(bare)
134
- hex_to_char(bare[1..-1])
135
+ hex_to_char(bare[1..])
135
136
  end
136
137
 
137
138
  def unescape_unicode_char(bare)
@@ -169,6 +170,9 @@ module RipperRubyParser
169
170
  end
170
171
 
171
172
  def control(val)
173
+ # Special case the \C-? or DEL sequence
174
+ return 127 if val == 63
175
+
172
176
  val & 0b1001_1111
173
177
  end
174
178
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RipperRubyParser
4
- VERSION = "1.9.0"
4
+ VERSION = "1.11.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ripper_ruby_parser
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.9.0
4
+ version: 1.11.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Matijs van Zuijlen
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-08-10 00:00:00.000000000 Z
11
+ date: 2024-01-05 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: sexp_processor
@@ -43,6 +43,9 @@ dependencies:
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
45
  - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '1.3'
48
+ - - ">="
46
49
  - !ruby/object:Gem::Version
47
50
  version: 1.3.1
48
51
  type: :development
@@ -50,6 +53,9 @@ dependencies:
50
53
  version_requirements: !ruby/object:Gem::Requirement
51
54
  requirements:
52
55
  - - "~>"
56
+ - !ruby/object:Gem::Version
57
+ version: '1.3'
58
+ - - ">="
53
59
  - !ruby/object:Gem::Version
54
60
  version: 1.3.1
55
61
  - !ruby/object:Gem::Dependency
@@ -100,74 +106,82 @@ dependencies:
100
106
  requirements:
101
107
  - - "~>"
102
108
  - !ruby/object:Gem::Version
103
- version: 1.18.0
109
+ version: '1.56'
104
110
  type: :development
105
111
  prerelease: false
106
112
  version_requirements: !ruby/object:Gem::Requirement
107
113
  requirements:
108
114
  - - "~>"
109
115
  - !ruby/object:Gem::Version
110
- version: 1.18.0
116
+ version: '1.56'
111
117
  - !ruby/object:Gem::Dependency
112
118
  name: rubocop-minitest
113
119
  requirement: !ruby/object:Gem::Requirement
114
120
  requirements:
115
121
  - - "~>"
116
122
  - !ruby/object:Gem::Version
117
- version: 0.15.0
123
+ version: 0.34.1
118
124
  type: :development
119
125
  prerelease: false
120
126
  version_requirements: !ruby/object:Gem::Requirement
121
127
  requirements:
122
128
  - - "~>"
123
129
  - !ruby/object:Gem::Version
124
- version: 0.15.0
130
+ version: 0.34.1
131
+ - !ruby/object:Gem::Dependency
132
+ name: rubocop-packaging
133
+ requirement: !ruby/object:Gem::Requirement
134
+ requirements:
135
+ - - "~>"
136
+ - !ruby/object:Gem::Version
137
+ version: 0.5.2
138
+ type: :development
139
+ prerelease: false
140
+ version_requirements: !ruby/object:Gem::Requirement
141
+ requirements:
142
+ - - "~>"
143
+ - !ruby/object:Gem::Version
144
+ version: 0.5.2
125
145
  - !ruby/object:Gem::Dependency
126
146
  name: rubocop-performance
127
147
  requirement: !ruby/object:Gem::Requirement
128
148
  requirements:
129
149
  - - "~>"
130
150
  - !ruby/object:Gem::Version
131
- version: 1.11.0
151
+ version: '1.19'
132
152
  type: :development
133
153
  prerelease: false
134
154
  version_requirements: !ruby/object:Gem::Requirement
135
155
  requirements:
136
156
  - - "~>"
137
157
  - !ruby/object:Gem::Version
138
- version: 1.11.0
158
+ version: '1.19'
139
159
  - !ruby/object:Gem::Dependency
140
160
  name: ruby_parser
141
161
  requirement: !ruby/object:Gem::Requirement
142
162
  requirements:
143
163
  - - "~>"
144
164
  - !ruby/object:Gem::Version
145
- version: 3.17.0
165
+ version: 3.20.2
146
166
  type: :development
147
167
  prerelease: false
148
168
  version_requirements: !ruby/object:Gem::Requirement
149
169
  requirements:
150
170
  - - "~>"
151
171
  - !ruby/object:Gem::Version
152
- version: 3.17.0
172
+ version: 3.20.2
153
173
  - !ruby/object:Gem::Dependency
154
174
  name: sexp_processor
155
175
  requirement: !ruby/object:Gem::Requirement
156
176
  requirements:
157
- - - ">="
158
- - !ruby/object:Gem::Version
159
- version: 4.13.0
160
- - - "<"
177
+ - - "~>"
161
178
  - !ruby/object:Gem::Version
162
179
  version: '4.16'
163
180
  type: :development
164
181
  prerelease: false
165
182
  version_requirements: !ruby/object:Gem::Requirement
166
183
  requirements:
167
- - - ">="
168
- - !ruby/object:Gem::Version
169
- version: 4.13.0
170
- - - "<"
184
+ - - "~>"
171
185
  - !ruby/object:Gem::Version
172
186
  version: '4.16'
173
187
  - !ruby/object:Gem::Dependency
@@ -176,14 +190,14 @@ dependencies:
176
190
  requirements:
177
191
  - - "~>"
178
192
  - !ruby/object:Gem::Version
179
- version: 0.21.0
193
+ version: 0.22.0
180
194
  type: :development
181
195
  prerelease: false
182
196
  version_requirements: !ruby/object:Gem::Requirement
183
197
  requirements:
184
198
  - - "~>"
185
199
  - !ruby/object:Gem::Version
186
- version: 0.21.0
200
+ version: 0.22.0
187
201
  description: |
188
202
  RipperRubyParser is a parser for Ruby based on Ripper that aims to be a
189
203
  drop-in replacement for RubyParser.
@@ -221,6 +235,7 @@ metadata:
221
235
  homepage_uri: http://www.github.com/mvz/ripper_ruby_parser
222
236
  source_code_uri: https://github.com/mvz/ripper_ruby_parser
223
237
  changelog_uri: https://github.com/mvz/ripper_ruby_parser/blob/master/CHANGELOG.md
238
+ rubygems_mfa_required: 'true'
224
239
  post_install_message:
225
240
  rdoc_options:
226
241
  - "--main"
@@ -231,14 +246,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
231
246
  requirements:
232
247
  - - ">="
233
248
  - !ruby/object:Gem::Version
234
- version: 2.5.0
249
+ version: 3.0.0
235
250
  required_rubygems_version: !ruby/object:Gem::Requirement
236
251
  requirements:
237
252
  - - ">="
238
253
  - !ruby/object:Gem::Version
239
254
  version: '0'
240
255
  requirements: []
241
- rubygems_version: 3.2.22
256
+ rubygems_version: 3.5.3
242
257
  signing_key:
243
258
  specification_version: 4
244
259
  summary: Parse with Ripper, produce sexps that are compatible with RubyParser.