ffast 0.0.2 → 0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.travis.yml +2 -2
- data/README.md +42 -18
- data/bin/fast +30 -9
- data/docs/command_line.md +112 -0
- data/docs/experiments.md +147 -0
- data/docs/index.md +356 -0
- data/docs/similarity_tutorial.md +174 -0
- data/docs/syntax.md +370 -0
- data/examples/let_it_be_experiment.rb +11 -0
- data/fast.gemspec +2 -2
- data/lib/fast.rb +30 -4
- data/mkdocs.yml +21 -0
- metadata +11 -4
data/docs/syntax.md
ADDED
@@ -0,0 +1,370 @@
|
|
1
|
+
# Syntax
|
2
|
+
|
3
|
+
The syntax is inspired on [RuboCop Node Pattern](https://github.com/bbatsov/rubocop/blob/master/lib/rubocop/node_pattern.rb).
|
4
|
+
|
5
|
+
You can find a great tutorial about RuboCop node pattern in the
|
6
|
+
[official documentation](https://rubocop.readthedocs.io/en/latest/node_pattern/).
|
7
|
+
|
8
|
+
## Code example
|
9
|
+
|
10
|
+
Let's consider the following `example.rb` code example:
|
11
|
+
|
12
|
+
```ruby
|
13
|
+
class Example
|
14
|
+
ANSWER = 42
|
15
|
+
def magic
|
16
|
+
rand(ANSWER)
|
17
|
+
end
|
18
|
+
def duplicate(value)
|
19
|
+
value * 2
|
20
|
+
end
|
21
|
+
end
|
22
|
+
```
|
23
|
+
|
24
|
+
Looking the AST representation we have:
|
25
|
+
|
26
|
+
$ ruby-parse example.rb
|
27
|
+
|
28
|
+
```
|
29
|
+
(class
|
30
|
+
(const nil :Example) nil
|
31
|
+
(begin
|
32
|
+
(casgn nil :ANSWER
|
33
|
+
(int 42))
|
34
|
+
(def :magic
|
35
|
+
(args)
|
36
|
+
(send nil :rand
|
37
|
+
(const nil :ANSWER)))
|
38
|
+
(def :duplicate
|
39
|
+
(args
|
40
|
+
(arg :value))
|
41
|
+
(send
|
42
|
+
(lvar :value) :*
|
43
|
+
(int 2)))))
|
44
|
+
```
|
45
|
+
|
46
|
+
Now, let's explore all details of the current AST, combining with the syntax
|
47
|
+
operators.
|
48
|
+
|
49
|
+
Fast works with a single word that will be the node type.
|
50
|
+
|
51
|
+
A simple search of `def` nodes can be done and will also print the code.
|
52
|
+
|
53
|
+
$ fast def example.rb
|
54
|
+
|
55
|
+
```ruby
|
56
|
+
# example.rb:3
|
57
|
+
def magic
|
58
|
+
rand(ANSWER)
|
59
|
+
end
|
60
|
+
```
|
61
|
+
|
62
|
+
or check the `casgn` that will show constant assignments:
|
63
|
+
|
64
|
+
$ fast casgn example.rb
|
65
|
+
|
66
|
+
```ruby
|
67
|
+
# example.rb:2
|
68
|
+
ANSWER = 42
|
69
|
+
```
|
70
|
+
|
71
|
+
## `()` to represent a **node** search
|
72
|
+
|
73
|
+
To specify details about a node, the `(` means navigate deeply into a node and
|
74
|
+
go deep into the expression.
|
75
|
+
|
76
|
+
$ fast '(casgn' example.rb
|
77
|
+
|
78
|
+
```ruby
|
79
|
+
# example.rb:2
|
80
|
+
ANSWER = 42
|
81
|
+
```
|
82
|
+
|
83
|
+
Fast matcher never checks the end of the expression and close parens are not
|
84
|
+
necessary. We keep them for the sake of specify more node details but the
|
85
|
+
expression works with incomplete parens.
|
86
|
+
|
87
|
+
$ fast '(casgn)' example.rb
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
# example.rb:2
|
91
|
+
ANSWER = 42
|
92
|
+
```
|
93
|
+
|
94
|
+
Closing extra params also don't have a side effect.
|
95
|
+
|
96
|
+
$ fast '(casgn))' example.rb
|
97
|
+
|
98
|
+
```ruby
|
99
|
+
# example.rb:2
|
100
|
+
ANSWER = 42
|
101
|
+
```
|
102
|
+
|
103
|
+
It also automatically flat parens case you put more levels in the beginning.
|
104
|
+
|
105
|
+
$ fast '((casgn))' example.rb
|
106
|
+
|
107
|
+
```ruby
|
108
|
+
# example.rb:2
|
109
|
+
ANSWER = 42
|
110
|
+
```
|
111
|
+
|
112
|
+
For checking AST details while doing some search, you can use `--ast` in the
|
113
|
+
command line for printing the AST instead of the code:
|
114
|
+
|
115
|
+
$ fast '((casgn ' example.rb --ast
|
116
|
+
|
117
|
+
```ruby
|
118
|
+
# example.rb:2
|
119
|
+
(casgn nil :ANSWER
|
120
|
+
(int 42))
|
121
|
+
```
|
122
|
+
|
123
|
+
## `_` is **something** not nil
|
124
|
+
|
125
|
+
Let's enhance our current expression and specify that we're looking for constant
|
126
|
+
assignments of integers ignoring values and constant names replacing with `_`.
|
127
|
+
|
128
|
+
$ fast '(casgn nil _ (int _))' example.rb
|
129
|
+
|
130
|
+
```ruby
|
131
|
+
# example.rb:2
|
132
|
+
ANSWER = 42
|
133
|
+
```
|
134
|
+
|
135
|
+
Keep in mind that `_` means not nil and `(casgn _ _ (int _))` would not
|
136
|
+
match.
|
137
|
+
|
138
|
+
Let's search for integer nodes:
|
139
|
+
|
140
|
+
$ fast int example.rb
|
141
|
+
```ruby
|
142
|
+
# example.rb:2
|
143
|
+
42
|
144
|
+
# example.rb:7
|
145
|
+
2
|
146
|
+
```
|
147
|
+
|
148
|
+
The current search show the nodes but they are not so useful without understand
|
149
|
+
the expression in their context. We need to check their `parent`.
|
150
|
+
|
151
|
+
## `^` is to get the **parent node** of an expression
|
152
|
+
|
153
|
+
By default, Parser::AST::Node does not have access to parent and for accessing
|
154
|
+
it you can say `^` for reaching the parent.
|
155
|
+
|
156
|
+
$ fast '^int' example.rb
|
157
|
+
|
158
|
+
```ruby
|
159
|
+
# example.rb:2
|
160
|
+
ANSWER = 42
|
161
|
+
# example.rb:7
|
162
|
+
value * 2
|
163
|
+
```
|
164
|
+
|
165
|
+
And using it multiple times will make the node match from levels up:
|
166
|
+
|
167
|
+
$ fast '^^int' example.rb
|
168
|
+
|
169
|
+
```ruby
|
170
|
+
# example.rb:2
|
171
|
+
ANSWER = 42
|
172
|
+
def magic
|
173
|
+
rand(ANSWER)
|
174
|
+
end
|
175
|
+
def duplicate(value)
|
176
|
+
value * 2
|
177
|
+
end
|
178
|
+
```
|
179
|
+
|
180
|
+
## `[]` join conditions
|
181
|
+
|
182
|
+
Let's hunt for integer nodes that the parent is also a method:
|
183
|
+
|
184
|
+
$ fast '[ ^^int def ]' example.rb
|
185
|
+
|
186
|
+
The match will filter only nodes that matches all internal expressions.
|
187
|
+
|
188
|
+
```ruby
|
189
|
+
# example.rb:6
|
190
|
+
def duplicate(value)
|
191
|
+
value * 2
|
192
|
+
end
|
193
|
+
```
|
194
|
+
|
195
|
+
The expression is matching nodes that have a integer granchild and also with
|
196
|
+
type `def`.
|
197
|
+
|
198
|
+
## `...` is a **node** with children
|
199
|
+
|
200
|
+
Looking the method representation we have:
|
201
|
+
|
202
|
+
$ fast def example.rb --ast
|
203
|
+
|
204
|
+
```ruby
|
205
|
+
# example.rb:3
|
206
|
+
(def :magic
|
207
|
+
(args)
|
208
|
+
(send nil :rand
|
209
|
+
(const nil :ANSWER)))
|
210
|
+
# example.rb:6
|
211
|
+
(def :duplicate
|
212
|
+
(args
|
213
|
+
(arg :value))
|
214
|
+
(send
|
215
|
+
(lvar :value) :*
|
216
|
+
(int 2)))
|
217
|
+
```
|
218
|
+
|
219
|
+
And if we want to delimit only methods with arguments:
|
220
|
+
|
221
|
+
$ fast '(def _ ...)' example.rb
|
222
|
+
|
223
|
+
```ruby
|
224
|
+
# example.rb:6
|
225
|
+
def duplicate(value)
|
226
|
+
value * 2
|
227
|
+
end
|
228
|
+
```
|
229
|
+
|
230
|
+
If we use `(def _ _)` instead it will match both methods because `(args)`
|
231
|
+
does not have children but is not nil.
|
232
|
+
|
233
|
+
## `$` is for **capture** current expression
|
234
|
+
|
235
|
+
Now, let's say we want to extract some method name from current classes.
|
236
|
+
|
237
|
+
In such case we don't want to have the node definition but only return the node
|
238
|
+
name.
|
239
|
+
|
240
|
+
```ruby
|
241
|
+
# example.rb:2
|
242
|
+
def magic
|
243
|
+
rand(ANSWER)
|
244
|
+
end
|
245
|
+
# example.rb:
|
246
|
+
magic
|
247
|
+
# example.rb:9
|
248
|
+
def duplicate(value)
|
249
|
+
value * 2
|
250
|
+
end
|
251
|
+
# example.rb:
|
252
|
+
duplicate
|
253
|
+
```
|
254
|
+
One extra method name was printed because of `$` is capturing the element.
|
255
|
+
|
256
|
+
Let's use the `--pry` for inspecting the results.
|
257
|
+
|
258
|
+
$ fast '(def $_)' example.rb --pry
|
259
|
+
|
260
|
+
It will open pry with access to `result` as the first result and
|
261
|
+
`results` with all matching results.
|
262
|
+
|
263
|
+
```
|
264
|
+
From: /Users/jonatasdp/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/ffast-0.0.2/bin/fast @ line 60 :
|
265
|
+
|
266
|
+
55:
|
267
|
+
56: results.each do |result|
|
268
|
+
57: next if result.nil? || result == []
|
269
|
+
58: if pry
|
270
|
+
59: require 'pry'
|
271
|
+
=> 60: binding.pry # rubocop:disable Lint/Debugger
|
272
|
+
61: else
|
273
|
+
62: Fast.report(result, file: file, show_sexp: show_sexp)
|
274
|
+
63: end
|
275
|
+
64: end
|
276
|
+
65: end
|
277
|
+
```
|
278
|
+
|
279
|
+
Inspecting the results you can see that they are mixing AST nodes and the
|
280
|
+
captures.
|
281
|
+
|
282
|
+
```ruby
|
283
|
+
[1] pry(main)> results
|
284
|
+
=> [s(:def, :magic,
|
285
|
+
s(:args),
|
286
|
+
s(:send, nil, :rand,
|
287
|
+
s(:const, nil, :ANSWER))),
|
288
|
+
:magic,
|
289
|
+
s(:def, :duplicate,
|
290
|
+
s(:args,
|
291
|
+
s(:arg, :value)),
|
292
|
+
s(:send,
|
293
|
+
s(:lvar, :value), :*,
|
294
|
+
s(:int, 2))),
|
295
|
+
:duplicate]
|
296
|
+
```
|
297
|
+
|
298
|
+
We can filter the captures to make it easy to analyze.
|
299
|
+
|
300
|
+
```ruby
|
301
|
+
[2] pry(main)> results.grep(Symbol)
|
302
|
+
=> [:magic, :duplicate]
|
303
|
+
```
|
304
|
+
|
305
|
+
## `nil` matches exactly **nil**
|
306
|
+
|
307
|
+
Nil is used in the code as a node type but parser gem also represents empty
|
308
|
+
spaces in expressions with nil.
|
309
|
+
|
310
|
+
Example, a method call from Kernel is a `send` from `nil` calling the method
|
311
|
+
while I can also send a method call from a class.
|
312
|
+
|
313
|
+
```
|
314
|
+
$ ruby-parse -e 'method'
|
315
|
+
(send nil :method)
|
316
|
+
```
|
317
|
+
|
318
|
+
And a method from a object will have the nested target not nil.
|
319
|
+
|
320
|
+
```
|
321
|
+
$ ruby-parse -e 'object.method'
|
322
|
+
(send
|
323
|
+
(send nil :object) :method)
|
324
|
+
```
|
325
|
+
|
326
|
+
Let's build a serch for any calls from `nil`:
|
327
|
+
|
328
|
+
$ fast '(_ nil _)' example.rb
|
329
|
+
|
330
|
+
```ruby
|
331
|
+
# example.rb:3
|
332
|
+
Example
|
333
|
+
# example.rb:4
|
334
|
+
ANSWER = 42
|
335
|
+
# example.rb:6
|
336
|
+
rand(ANSWER)
|
337
|
+
```
|
338
|
+
|
339
|
+
Double check the expressions that have matched printing the AST:
|
340
|
+
|
341
|
+
$ fast '(_ nil _)' example.rb --ast
|
342
|
+
|
343
|
+
```ruby
|
344
|
+
# example.rb:3
|
345
|
+
(const nil :Example)
|
346
|
+
# example.rb:4
|
347
|
+
(casgn nil :ANSWER
|
348
|
+
(int 42))
|
349
|
+
# example.rb:6
|
350
|
+
(send nil :rand
|
351
|
+
(const nil :ANSWER))
|
352
|
+
```
|
353
|
+
|
354
|
+
## `{}` is for **any** matches like **union** conditions with **or** operator
|
355
|
+
|
356
|
+
Let's say we to add check all occurrencies of the constant `ANSWER`.
|
357
|
+
|
358
|
+
We'll need to get both `casgn` and `const` node types. For such cases we can
|
359
|
+
surround the expressions with `{}` and it will return if the node matches with
|
360
|
+
any of the internal expressions.
|
361
|
+
|
362
|
+
$ fast '({casgn const} nil ANSWER)' example.rb
|
363
|
+
|
364
|
+
```
|
365
|
+
# example.rb:4
|
366
|
+
ANSWER = 42
|
367
|
+
# example.rb:6
|
368
|
+
ANSWER
|
369
|
+
```
|
370
|
+
|
@@ -0,0 +1,11 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require '../../fast/lib/fast'
|
4
|
+
|
5
|
+
# For specs using `let(:something) { create ... }` it tries to use `let_it_be` instead
|
6
|
+
Fast.experiment('RSpec/LetItBe') do
|
7
|
+
lookup 'spec/models'
|
8
|
+
search '(block $(send nil let (sym _)) (args) (send nil create))'
|
9
|
+
edit { |_, (let)| replace(let.loc.selector, 'let_it_be') }
|
10
|
+
policy { |new_file| system("bin/spring rspec --fail-fast #{new_file}") }
|
11
|
+
end.run
|
data/fast.gemspec
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
Gem::Specification.new do |spec|
|
4
4
|
spec.name = 'ffast'
|
5
|
-
spec.version = '0.0.
|
5
|
+
spec.version = '0.0.3'
|
6
6
|
spec.required_ruby_version = '>= 2.3'
|
7
7
|
spec.authors = ['Jônatas Davi Paganini']
|
8
8
|
spec.email = ['jonatas.paganini@toptal.com']
|
@@ -23,7 +23,7 @@ Gem::Specification.new do |spec|
|
|
23
23
|
spec.add_development_dependency 'rspec', '~> 3.0'
|
24
24
|
spec.add_dependency 'coderay'
|
25
25
|
spec.add_dependency 'parser'
|
26
|
-
spec.
|
26
|
+
spec.add_dependency 'pry'
|
27
27
|
spec.add_development_dependency 'rubocop'
|
28
28
|
spec.add_development_dependency 'rubocop-rspec'
|
29
29
|
end
|
data/lib/fast.rb
CHANGED
@@ -1,4 +1,3 @@
|
|
1
|
-
|
2
1
|
# frozen_string_literal: true
|
3
2
|
|
4
3
|
# suppress output to avoid parser gem warnings'
|
@@ -20,7 +19,7 @@ end
|
|
20
19
|
|
21
20
|
# Fast is a tool to help you search in the code through the Abstract Syntax Tree
|
22
21
|
module Fast
|
23
|
-
VERSION = '0.
|
22
|
+
VERSION = '0.3.0'
|
24
23
|
LITERAL = {
|
25
24
|
'...' => ->(node) { node&.children&.any? },
|
26
25
|
'_' => ->(node) { !node.nil? },
|
@@ -30,6 +29,8 @@ module Fast
|
|
30
29
|
TOKENIZER = %r/
|
31
30
|
[\+\-\/\*\\!] # operators or negation
|
32
31
|
|
|
32
|
+
===? # == or ===
|
33
|
+
|
|
33
34
|
\d+\.\d* # decimals and floats
|
34
35
|
|
|
35
36
|
"[^"]+" # strings
|
@@ -112,7 +113,8 @@ module Fast
|
|
112
113
|
end
|
113
114
|
|
114
115
|
def ast_from_file(file)
|
115
|
-
|
116
|
+
@cache ||= {}
|
117
|
+
@cache[file] ||= ast(IO.read(file))
|
116
118
|
end
|
117
119
|
|
118
120
|
def highlight(node, show_sexp: false)
|
@@ -125,7 +127,7 @@ module Fast
|
|
125
127
|
CodeRay.scan(output, :ruby).term
|
126
128
|
end
|
127
129
|
|
128
|
-
def report(result, show_sexp
|
130
|
+
def report(result, show_sexp: nil, file: nil)
|
129
131
|
if file
|
130
132
|
line = result.loc.expression.line if result.is_a?(Parser::AST::Node)
|
131
133
|
puts Fast.highlight("# #{file}:#{line}")
|
@@ -153,6 +155,7 @@ module Fast
|
|
153
155
|
|
154
156
|
def debug
|
155
157
|
return yield if debugging
|
158
|
+
|
156
159
|
self.debugging = true
|
157
160
|
result = nil
|
158
161
|
Find.class_eval do
|
@@ -175,6 +178,20 @@ module Fast
|
|
175
178
|
end
|
176
179
|
files
|
177
180
|
end
|
181
|
+
|
182
|
+
def expression_from(node)
|
183
|
+
case node
|
184
|
+
when Parser::AST::Node
|
185
|
+
children_expression = node.children.map(&Fast.method(:expression_from)).join(' ')
|
186
|
+
"(#{node.type}#{' ' + children_expression if node.children.any?})"
|
187
|
+
when nil, 'nil'
|
188
|
+
'nil'
|
189
|
+
when Symbol, String, Integer
|
190
|
+
'_'
|
191
|
+
when Array, Hash
|
192
|
+
'...'
|
193
|
+
end
|
194
|
+
end
|
178
195
|
end
|
179
196
|
|
180
197
|
# Rewriter encapsulates `#match_index` allowing to rewrite only specific matching occurrences
|
@@ -305,6 +322,7 @@ module Fast
|
|
305
322
|
|
306
323
|
def ==(other)
|
307
324
|
return false if other.nil? || !other.respond_to?(:token)
|
325
|
+
|
308
326
|
token == other.token
|
309
327
|
end
|
310
328
|
|
@@ -313,6 +331,7 @@ module Fast
|
|
313
331
|
def valuate(token)
|
314
332
|
if token.is_a?(String)
|
315
333
|
return valuate(LITERAL[token]) if LITERAL.key?(token)
|
334
|
+
|
316
335
|
typecast_value(token)
|
317
336
|
else
|
318
337
|
token
|
@@ -347,6 +366,7 @@ module Fast
|
|
347
366
|
def initialize(token)
|
348
367
|
token = token.token if token.respond_to?(:token)
|
349
368
|
raise 'You must use captures!' unless token
|
369
|
+
|
350
370
|
@capture_index = token.to_i
|
351
371
|
end
|
352
372
|
|
@@ -456,6 +476,7 @@ module Fast
|
|
456
476
|
if tail.empty?
|
457
477
|
return ast == @ast ? find_captures : true # root node
|
458
478
|
end
|
479
|
+
|
459
480
|
child = ast.children
|
460
481
|
tail.each_with_index.all? do |token, i|
|
461
482
|
token.previous_captures = find_captures if token.is_a?(Fast::FindWithCapture)
|
@@ -475,6 +496,7 @@ module Fast
|
|
475
496
|
|
476
497
|
def find_captures(fast = @fast)
|
477
498
|
return true if fast == @fast && !captures?(fast)
|
499
|
+
|
478
500
|
case fast
|
479
501
|
when Capture then fast.captures
|
480
502
|
when Array then fast.flat_map(&method(:find_captures)).compact
|
@@ -560,6 +582,7 @@ module Fast
|
|
560
582
|
def ok_with(combination)
|
561
583
|
@ok_experiments << combination
|
562
584
|
return unless combination.is_a?(Array)
|
585
|
+
|
563
586
|
combination.each do |element|
|
564
587
|
@ok_experiments.delete(element)
|
565
588
|
end
|
@@ -587,6 +610,7 @@ module Fast
|
|
587
610
|
end
|
588
611
|
end
|
589
612
|
return unless new_content
|
613
|
+
|
590
614
|
write_experiment_file(indices, new_content)
|
591
615
|
new_content
|
592
616
|
end
|
@@ -614,6 +638,7 @@ module Fast
|
|
614
638
|
count_executed_combinations = @fail_experiments.size + @ok_experiments.size
|
615
639
|
puts "Done with #{@file} after #{count_executed_combinations}"
|
616
640
|
return unless perfect_combination = @ok_experiments.last # rubocop:disable Lint/AssignmentInCondition
|
641
|
+
|
617
642
|
puts "mv #{experimental_filename(perfect_combination)} #{@file}"
|
618
643
|
`mv #{experimental_filename(perfect_combination)} #{@file}`
|
619
644
|
end
|
@@ -642,6 +667,7 @@ module Fast
|
|
642
667
|
if experimental_file == IO.read(@file)
|
643
668
|
raise 'Returned the same file thinking:'
|
644
669
|
end
|
670
|
+
|
645
671
|
File.open(experimental_file, 'w+') { |f| f.puts content }
|
646
672
|
|
647
673
|
if experiment.ok_if.call(experimental_file)
|