syntax_tree 4.0.1 → 4.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 5d8670e1c7ff4d583f84ee1fc217c714d217c944f36bb2bba5e2527cdd9012c8
4
- data.tar.gz: c00025ed6cfb6031f64c9531282d527ff5113b78fa1228b86774bb84bb834576
3
+ metadata.gz: 61bd75fb26dcf665293adb21c2cf21a5f0f26b56a2077d80ce5ff6806eaf434d
4
+ data.tar.gz: 545306a76376f972805159e8eed622e825011b7fa9ef6b93cb9649c6caf9eccf
5
5
  SHA512:
6
- metadata.gz: bb4cd48f69132f5685e24ebbd29bb020468be6bf2a9a191bc3bba6ebf45dd5d63b2f61669c1695ec1711be4bcb4a5a8a29191fae1051b63d0b7d09bb75e78c0a
7
- data.tar.gz: 5f35a3f095d3208246c013bf18b2d93cd4066ac5375750d5100b1c893806edb8ce240230cd30d70cfb525e6dfae4bbbc1ab108b6d549750a37ab0b16ba07c437
6
+ metadata.gz: 633701480cbe5c34bfcff52c4dd6c48f10cb22cba377b295ac56f2c288ce6a5dfdde57df5e682a3226f3d49ca7c3dd8d02db5d2d6d37acfc54df5c304be8a16f
7
+ data.tar.gz: e4224136ab12c158c45bd6375308e01a4e7075c7da30536f82f77ebecd3d4c426c4798e80fb5eb6fef9a1771036d4ac7e5d709dc49c5b620cbe9a2c3f2dd15a0
data/.rubocop.yml CHANGED
@@ -28,6 +28,9 @@ Lint/InterpolationCheck:
28
28
  Lint/MissingSuper:
29
29
  Enabled: false
30
30
 
31
+ Lint/RedundantRequireStatement:
32
+ Enabled: false
33
+
31
34
  Lint/UnusedMethodArgument:
32
35
  AllowUnusedKeywordArguments: true
33
36
 
data/CHANGELOG.md CHANGED
@@ -6,6 +6,18 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [4.1.0] - 2022-10-24
10
+
11
+ ### Added
12
+
13
+ - [#180](https://github.com/ruby-syntax-tree/syntax_tree/pull/180) - The new `stree search` CLI command and the corresponding `SyntaxTree::Search` class for searching for a pattern against a given syntax tree.
14
+
15
+ ## [4.0.2] - 2022-10-19
16
+
17
+ ### Changed
18
+
19
+ - [#177](https://github.com/ruby-syntax-tree/syntax_tree/pull/177) - Fix up various other issues with the environment visitor addition.
20
+
9
21
  ## [4.0.1] - 2022-10-18
10
22
 
11
23
  ### Changed
@@ -391,7 +403,10 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
391
403
 
392
404
  - 🎉 Initial release! 🎉
393
405
 
394
- [unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.0.0...HEAD
406
+ [unreleased]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.1.0...HEAD
407
+ [4.1.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.0.2...v4.1.0
408
+ [4.0.2]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.0.1...v4.0.2
409
+ [4.0.1]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v4.0.0...v4.0.1
395
410
  [4.0.0]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v3.6.3...v4.0.0
396
411
  [3.6.3]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v3.6.2...v3.6.3
397
412
  [3.6.2]: https://github.com/ruby-syntax-tree/syntax_tree/compare/v3.6.1...v3.6.2
data/Gemfile.lock CHANGED
@@ -1,8 +1,8 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- syntax_tree (4.0.1)
5
- prettier_print (>= 1.0.1)
4
+ syntax_tree (4.1.0)
5
+ prettier_print (>= 1.0.2)
6
6
 
7
7
  GEM
8
8
  remote: https://rubygems.org/
@@ -14,22 +14,22 @@ GEM
14
14
  parallel (1.22.1)
15
15
  parser (3.1.2.1)
16
16
  ast (~> 2.4.1)
17
- prettier_print (1.0.1)
17
+ prettier_print (1.0.2)
18
18
  rainbow (3.1.1)
19
19
  rake (13.0.6)
20
20
  regexp_parser (2.6.0)
21
21
  rexml (3.2.5)
22
- rubocop (1.36.0)
22
+ rubocop (1.37.1)
23
23
  json (~> 2.3)
24
24
  parallel (~> 1.10)
25
25
  parser (>= 3.1.2.1)
26
26
  rainbow (>= 2.2.2, < 4.0)
27
27
  regexp_parser (>= 1.8, < 3.0)
28
28
  rexml (>= 3.2.5, < 4.0)
29
- rubocop-ast (>= 1.20.1, < 2.0)
29
+ rubocop-ast (>= 1.23.0, < 2.0)
30
30
  ruby-progressbar (~> 1.7)
31
31
  unicode-display_width (>= 1.4.0, < 3.0)
32
- rubocop-ast (1.22.0)
32
+ rubocop-ast (1.23.0)
33
33
  parser (>= 3.1.1.0)
34
34
  ruby-progressbar (1.11.0)
35
35
  simplecov (0.21.2)
data/README.md CHANGED
@@ -18,6 +18,7 @@ It is built with only standard library dependencies. It additionally ships with
18
18
  - [format](#format)
19
19
  - [json](#json)
20
20
  - [match](#match)
21
+ - [search](#search)
21
22
  - [write](#write)
22
23
  - [Configuration](#configuration)
23
24
  - [Globbing](#globbing)
@@ -215,6 +216,29 @@ SyntaxTree::Program[
215
216
  ]
216
217
  ```
217
218
 
219
+ ### search
220
+
221
+ This command will search the given filepaths against the specified pattern to find nodes that match. The pattern is a Ruby pattern-matching expression that is matched against each node in the tree. It can optionally be loaded from a file if you specify a filepath as the pattern argument.
222
+
223
+ ```sh
224
+ stree search VarRef path/to/file.rb
225
+ ```
226
+
227
+ For a file that contains `Foo + Bar` you will receive:
228
+
229
+ ```ruby
230
+ path/to/file.rb:1:0: Foo + Bar
231
+ path/to/file.rb:1:6: Foo + Bar
232
+ ```
233
+
234
+ If you put `VarRef` into a file instead (for example, `query.txt`), you would instead run:
235
+
236
+ ```sh
237
+ stree search query.txt path/to/file.rb
238
+ ```
239
+
240
+ Note that the output of the `match` CLI command creates a valid pattern that can be used as the input for this command.
241
+
218
242
  ### write
219
243
 
220
244
  This command will format the listed files and write that formatted version back to the source files. Note that this overwrites the original content, to be sure to be using a version control system.
@@ -212,6 +212,39 @@ module SyntaxTree
212
212
  end
213
213
  end
214
214
 
215
+ # An action of the CLI that searches for the given pattern matching pattern
216
+ # in the given files.
217
+ class Search < Action
218
+ attr_reader :search
219
+
220
+ def initialize(query)
221
+ query = File.read(query) if File.readable?(query)
222
+ @search = SyntaxTree::Search.new(query)
223
+ rescue SyntaxTree::Search::UncompilableError => error
224
+ warn(error.message)
225
+ exit(1)
226
+ end
227
+
228
+ def run(item)
229
+ search.scan(item.handler.parse(item.source)) do |node|
230
+ location = node.location
231
+ line = location.start_line
232
+
233
+ bold_range =
234
+ if line == location.end_line
235
+ location.start_column...location.end_column
236
+ else
237
+ location.start_column..
238
+ end
239
+
240
+ source = item.source.lines[line - 1].chomp
241
+ source[bold_range] = Color.bold(source[bold_range]).to_s
242
+
243
+ puts("#{item.filepath}:#{line}:#{location.start_column}: #{source}")
244
+ end
245
+ end
246
+ end
247
+
215
248
  # An action of the CLI that formats the input source and writes the
216
249
  # formatted output back to the file.
217
250
  class Write < Action
@@ -263,6 +296,9 @@ module SyntaxTree
263
296
  #{Color.bold("stree lsp [--plugins=...] [--print-width=NUMBER]")}
264
297
  Run syntax tree in language server mode
265
298
 
299
+ #{Color.bold("stree search PATTERN [-e SCRIPT] FILE")}
300
+ Search for the given pattern in the given files
301
+
266
302
  #{Color.bold("stree version")}
267
303
  Output the current version of syntax tree
268
304
 
@@ -400,6 +436,8 @@ module SyntaxTree
400
436
  Debug.new(options)
401
437
  when "doc"
402
438
  Doc.new(options)
439
+ when "f", "format"
440
+ Format.new(options)
403
441
  when "help"
404
442
  puts HELP
405
443
  return 0
@@ -411,8 +449,8 @@ module SyntaxTree
411
449
  return 0
412
450
  when "m", "match"
413
451
  Match.new(options)
414
- when "f", "format"
415
- Format.new(options)
452
+ when "s", "search"
453
+ Search.new(arguments.shift)
416
454
  when "version"
417
455
  puts SyntaxTree::VERSION
418
456
  return 0
@@ -434,7 +472,7 @@ module SyntaxTree
434
472
  .glob(pattern)
435
473
  .each do |filepath|
436
474
  if File.readable?(filepath) &&
437
- options.ignore_files.none? { File.fnmatch?(_1, filepath) }
475
+ options.ignore_files.none? { File.fnmatch?(_1, filepath) }
438
476
  queue << FileItem.new(filepath)
439
477
  end
440
478
  end
@@ -1657,12 +1657,12 @@ module SyntaxTree
1657
1657
  # for older Ruby versions.
1658
1658
  unless :+.respond_to?(:name)
1659
1659
  using Module.new {
1660
- refine Symbol do
1661
- def name
1662
- to_s.freeze
1663
- end
1664
- end
1665
- }
1660
+ refine Symbol do
1661
+ def name
1662
+ to_s.freeze
1663
+ end
1664
+ end
1665
+ }
1666
1666
  end
1667
1667
 
1668
1668
  # [untyped] the left-hand side of the expression
@@ -0,0 +1,92 @@
1
+ # frozen_string_literal: true
2
+
3
+ module SyntaxTree
4
+ # Provides an interface for searching for a pattern of nodes against a
5
+ # subtree of an AST.
6
+ class Search
7
+ class UncompilableError < StandardError
8
+ end
9
+
10
+ attr_reader :matcher
11
+
12
+ def initialize(query)
13
+ root = SyntaxTree.parse("case nil\nin #{query}\nend")
14
+ @matcher = compile(root.statements.body.first.consequent.pattern)
15
+ end
16
+
17
+ def scan(root)
18
+ return to_enum(__method__, root) unless block_given?
19
+ queue = [root]
20
+
21
+ until queue.empty?
22
+ node = queue.shift
23
+ next unless node
24
+
25
+ yield node if matcher.call(node)
26
+ queue += node.child_nodes
27
+ end
28
+ end
29
+
30
+ private
31
+
32
+ def compile(pattern)
33
+ case pattern
34
+ in Binary[left:, operator: :|, right:]
35
+ compiled_left = compile(left)
36
+ compiled_right = compile(right)
37
+
38
+ ->(node) { compiled_left.call(node) || compiled_right.call(node) }
39
+ in Const[value:] if SyntaxTree.const_defined?(value)
40
+ clazz = SyntaxTree.const_get(value)
41
+
42
+ ->(node) { node.is_a?(clazz) }
43
+ in Const[value:] if Object.const_defined?(value)
44
+ clazz = Object.const_get(value)
45
+
46
+ ->(node) { node.is_a?(clazz) }
47
+ in ConstPathRef[parent: VarRef[value: Const[value: "SyntaxTree"]]]
48
+ compile(pattern.constant)
49
+ in HshPtn[constant:, keywords:, keyword_rest: nil]
50
+ compiled_constant = compile(constant)
51
+
52
+ preprocessed_keywords =
53
+ keywords.to_h do |keyword, value|
54
+ raise NoMatchingPatternError unless keyword.is_a?(Label)
55
+ [keyword.value.chomp(":").to_sym, compile(value)]
56
+ end
57
+
58
+ compiled_keywords = ->(node) do
59
+ deconstructed = node.deconstruct_keys(preprocessed_keywords.keys)
60
+ preprocessed_keywords.all? do |keyword, matcher|
61
+ matcher.call(deconstructed[keyword])
62
+ end
63
+ end
64
+
65
+ ->(node) do
66
+ compiled_constant.call(node) && compiled_keywords.call(node)
67
+ end
68
+ in RegexpLiteral[parts: [TStringContent[value:]]]
69
+ regexp = /#{value}/
70
+
71
+ ->(attribute) { regexp.match?(attribute) }
72
+ in StringLiteral[parts: [TStringContent[value:]]]
73
+ ->(attribute) { attribute == value }
74
+ in VarRef[value: Const => value]
75
+ compile(value)
76
+ end
77
+ rescue NoMatchingPatternError
78
+ raise UncompilableError, <<~ERROR
79
+ Syntax Tree was unable to compile the pattern you provided to search
80
+ into a usable expression. It failed on the node within the pattern
81
+ matching expression represented by:
82
+
83
+ #{PP.pp(pattern, +"").chomp}
84
+
85
+ Note that not all syntax supported by Ruby's pattern matching syntax is
86
+ also supported by Syntax Tree's code search. If you're using some syntax
87
+ that you believe should be supported, please open an issue on the GitHub
88
+ repository at https://github.com/ruby-syntax-tree/syntax_tree.
89
+ ERROR
90
+ end
91
+ end
92
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SyntaxTree
4
- VERSION = "4.0.1"
4
+ VERSION = "4.1.0"
5
5
  end
@@ -44,8 +44,12 @@ module SyntaxTree
44
44
  with_new_environment { super }
45
45
  end
46
46
 
47
+ # When we find a method invocation with a block, only the code that happens
48
+ # inside of the block needs a fresh environment. The method invocation
49
+ # itself happens in the same environment
47
50
  def visit_method_add_block(node)
48
- with_new_environment { super }
51
+ visit(node.call)
52
+ with_new_environment { visit(node.block) }
49
53
  end
50
54
 
51
55
  def visit_def(node)
@@ -63,9 +67,7 @@ module SyntaxTree
63
67
  # Visit for keeping track of local arguments, such as method and block
64
68
  # arguments
65
69
  def visit_params(node)
66
- node.requireds.each do |param|
67
- current_environment.add_local_definition(param, :argument)
68
- end
70
+ add_argument_definitions(node.requireds)
69
71
 
70
72
  node.posts.each do |param|
71
73
  current_environment.add_local_definition(param, :argument)
@@ -117,13 +119,6 @@ module SyntaxTree
117
119
  alias visit_pinned_var_ref visit_var_field
118
120
 
119
121
  # Visits for keeping track of variable and argument usages
120
- def visit_aref_field(node)
121
- name = node.collection.value
122
- current_environment.add_local_usage(name, :variable) if name
123
-
124
- super
125
- end
126
-
127
122
  def visit_var_ref(node)
128
123
  value = node.value
129
124
 
@@ -137,5 +132,17 @@ module SyntaxTree
137
132
 
138
133
  super
139
134
  end
135
+
136
+ private
137
+
138
+ def add_argument_definitions(list)
139
+ list.each do |param|
140
+ if param.is_a?(SyntaxTree::MLHSParen)
141
+ add_argument_definitions(param.contents.parts)
142
+ else
143
+ current_environment.add_local_definition(param, :argument)
144
+ end
145
+ end
146
+ end
140
147
  end
141
148
  end
data/lib/syntax_tree.rb CHANGED
@@ -21,6 +21,7 @@ require_relative "syntax_tree/visitor/environment"
21
21
  require_relative "syntax_tree/visitor/with_environment"
22
22
 
23
23
  require_relative "syntax_tree/parser"
24
+ require_relative "syntax_tree/search"
24
25
 
25
26
  # Syntax Tree is a suite of tools built on top of the internal CRuby parser. It
26
27
  # provides the ability to generate a syntax tree from source, as well as the
data/syntax_tree.gemspec CHANGED
@@ -25,7 +25,7 @@ Gem::Specification.new do |spec|
25
25
  spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
26
26
  spec.require_paths = %w[lib]
27
27
 
28
- spec.add_dependency "prettier_print", ">= 1.0.1"
28
+ spec.add_dependency "prettier_print", ">= 1.0.2"
29
29
 
30
30
  spec.add_development_dependency "bundler"
31
31
  spec.add_development_dependency "minitest"
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: syntax_tree
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.0.1
4
+ version: 4.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kevin Newton
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2022-10-18 00:00:00.000000000 Z
11
+ date: 2022-10-24 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: prettier_print
@@ -16,14 +16,14 @@ dependencies:
16
16
  requirements:
17
17
  - - ">="
18
18
  - !ruby/object:Gem::Version
19
- version: 1.0.1
19
+ version: 1.0.2
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - ">="
25
25
  - !ruby/object:Gem::Version
26
- version: 1.0.1
26
+ version: 1.0.2
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: bundler
29
29
  requirement: !ruby/object:Gem::Requirement
@@ -135,6 +135,7 @@ files:
135
135
  - lib/syntax_tree/rake/task.rb
136
136
  - lib/syntax_tree/rake/write_task.rb
137
137
  - lib/syntax_tree/rake_tasks.rb
138
+ - lib/syntax_tree/search.rb
138
139
  - lib/syntax_tree/version.rb
139
140
  - lib/syntax_tree/visitor.rb
140
141
  - lib/syntax_tree/visitor/environment.rb