leftovers 0.5.5 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0d697aa7197a87a0e3cc99124031957ab4cc602c89afd6d0369be3fffdef3ba0
-  data.tar.gz: c782e42dc2b5916160213fe22a7eb3c1560930c7e21f6f72489f10d0a03ff612
+  metadata.gz: c2e0c74dad85b7b571608a7207f006fe14d5748e1a155470185112b80a7b80a3
+  data.tar.gz: 68f3d83b310f640e127b961e6ac2ece795a7b9e2a6de961a1b42498916e092d7
 SHA512:
-  metadata.gz: c63edb11d4ed98ab23402f4a7ace623a6a364a5f39218076a5713034879fd98ed90fe024badaca2ac26562e94bbd0fde690f932e185de88a93bff1dca3fb0ce4
-  data.tar.gz: daae5206e9b217ce77201347c504255834aba552a8566c04ecfd224105577197c9d62415f7dc799727842c136d9e2c00e2fe90de8115c95fd4c176068cbcb313
+  metadata.gz: c57c64254cc2092e38b8decfd7bde0a021d66a7c3f563dc3e1f5a68b91c4bfc7b16a8490e5437b3da942071d41d2c35c247e1e51739064e14fdfcc6653b192b3
+  data.tar.gz: b129119436c793987c89b032d5936b10ca935c7fc5398e7a2843665bbfec28ccaacd0a0090c8cd37c5172b6892f4291455aef40df6bae67727d25ac47fd871a3

data/CHANGELOG.md

@@ -1,7 +1,45 @@
-# 0.5.5
+# v0.8.0
+- Allow custom precompilers
+  ```yml
+    require: './path/to/my_precompiler'
+    precompile:
+      - paths: '*.myCustomFormat'
+        format: { custom: 'MyPrecompiler' }
+  ```
+  the built in precompilers have moved into this system too, with `format: haml` etc. `haml_paths` etc are now deprecated
+- a lot of refactoring, which revealed some edge cases i was handling
+  - collect a call to :my_method= with receiver&.my_method ||= (and += etc)
+  - if `add_prefix:` or `add_suffix:` points to another argument for a name that we can't use because it's a variable, it now just don't return the value, rather than returning the value but without the affix
+
+# v0.7.0
+- Rewrite the config parser/validation
+  - to provide clearer error messages with line numbers and everything
+  - to allow removing the json_schemer dependency with its 4 further dependencies
+- Allow `match:` to be used with `has_prefix:` and/or `has_suffix:`. There's not a good use for this but it was easier than encoding 'this can't be used with that' logic especially for it.
+- Add the possibility to quiet the `requires:` config, like:
+  ```yml
+  requires:
+  - 'active_support/inflections'
+  - quiet: './config/initializers/inflections'
+  ```
+- consider all the public methods in rails custom generators to be used, this required some new features:
+  - add filtering methods/constants by public/protected/private
+  - add the possibility to set the privacy of methods and constants with method calls (`set_privacy:`, and `set_default_privacy:`)
+  - add `Method` and `Constant` as options for the `type:` filter
+  - add type filtering to dynamic.
+- check `def self.whatever` as a definition, i didn't realise i wasn't checking this.
+
+# v0.6.0
+- drop ruby 2.4 support, allowing for some performance improvements
+- Add ability to parse JSON and YAML files
+- Add magic comment that points to a particular dynamic rule
+- repeated calls to --write-todo won't have ordering differences
+- fix issue with --write-todo and unused methods defined in test files
+
+# v0.5.5
 - Fix rails resource/resources method signatures
 
-# 0.5.4
+# v0.5.4
 - Add support for slim templates #13 - thanks @veganstraightedge
 - fix the #how-to-resolve link #11 - thanks @veganstraightedge
 

data/README.md

@@ -129,7 +129,7 @@ To do this for all definitions of this name, instead of adding a comment, add th
 
 ### `# leftovers:call`
 _aliases `leftovers:calls`_
-To mark a dynamic call that doesn't use literal values, use `leftovers:call` with the method name listed
+To mark a call that doesn't use literal values, use `leftovers:call` with the method name listed
 ```ruby
 method = [:puts, :warn].sample # leftovers:call puts, warn
 send(method, 'text')
@@ -137,6 +137,26 @@ send(method, 'text')
 
 This would consider `puts` and `warn` to both have been called
 
+### `# leftovers:dynamic:*`
+To mark a dynamic call for literal hash and array values without enumerating everything in the comment again, use `leftovers:dynamic:`  on the same line as the beginning of the array or hash
+
+```ruby
+[ # leftovers:dynamic:call_login
+  :user,
+  :admin
+].each { |method| send("#{method}_login") }
+```
+
+with the following configuration matching the `name: value` to the `leftovers:dynamic:process_name`
+
+```yaml
+dynamic:
+  name: call_login
+  arguments: '*'
+  add_suffix: '_login'
+```
+This would consider `user_login` and `admin_login` to both have been called.
+
 ## Configuration file
 
 The configuration is read from `.leftovers.yml` in your project root.
@@ -146,6 +166,7 @@ Its presence is optional and all of these settings are optional.
 - [`exclude_paths:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#exclude_paths)
 - [`test_paths:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#test_paths)
 - [`requires:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#requires)
+- [`precompile:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#precompile)
 - [`gems:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#gems)
 - [`keep:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#keep)
 - [`test_only:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#test_only)

data/docs/Configuration.md

@@ -6,9 +6,7 @@ Its presence is optional and all of these settings are optional.
 - [`include_paths:`](#include_paths)
 - [`exclude_paths:`](#exclude_paths)
 - [`test_paths:`](#test_paths)
-- [`haml_paths:`](#haml_paths)
-- [`slim_paths:`](#slim_paths)
-- [`erb_paths:`](#erb_paths)
+- [`precompile:`](#precompile)
 - [`requires:`](#requires)
 - [`gems:`](#gems)
 - [`keep:`](#keep)
@@ -57,12 +55,17 @@ List filenames/paths that you want to include
 Unlike other `paths:` configuration, each entry is **not** the gitignore pattern format.
 Instead it is strings that can be passed directly to ruby's `require` method (relative paths should start with `./`).
 
-Missing files/gems will be a warning, but not a LoadError.
+```yml
+require: ./config/initializers/my_other_inflections_file
+```
 
-By default, the default [`gems: active_support`]() config `require: activesupport` and `gems: rails` will `require: ./config/initializers/inflections`
+Missing files/gems will be a warning, but not a LoadError.
+To avoid seeing the warning if the file isn't there use `quiet:`.
 
 ```yml
-require: ./config/initializers/my_other_inflections_file
+  requires:
+  - 'active_support/inflections' # will warn if it's missing
+  - quiet: './config/initializers/inflections' # will say nothing
 ```
 
 Arrays are not necessary for single values
@@ -81,41 +84,78 @@ test_paths:
 
 Arrays are not necessary for single values
 
-## `haml_paths:`
-
-list filenames/paths of test directories that are in the haml format
-Defined using the [.gitignore pattern format](https://git-scm.com/docs/gitignore#_pattern_format)
+## `precompile:`
 
 ```yml
-haml_paths:
-  - '*.haml'
+  require: './path/my_project/my_precompiler'
+  precompile:
+    - paths: '*.myCustomFormat'
+      format: { custom: 'MyProject::MyPrecompiler' }
+    - paths: '*.my.json'
+      format: json
 ```
 
-Arrays are not necessary for single values. `*.haml` is recognized by default
+Define any precompilers and the paths they affect.
 
-## `slim_paths:`
+`paths:` are defined using the [.gitignore pattern format](https://git-scm.com/docs/gitignore#_pattern_format)
 
-list filenames/paths of test directories that are in the slim format
-Defined using the [.gitignore pattern format](https://git-scm.com/docs/gitignore#_pattern_format)
+`format:` must be one of the predefined precompilers (erb, haml, [json](#format-json), slim, [yaml](#format-yaml)), or `custom:` with the name of a [custom precompiler]('../Custom-Precompilers.md) module.
+(use [`require:`](#requires) to have leftovers load its file)
+
+See [Custom precompilers]('../Custom-Precompilers.md) for more details on the custom precompiler class
+
+Arrays are not necessary for single values.
+
+### `format: yaml`
+
+The yaml precompiler considers yaml tags like `!ruby/class 'MyClass'` to be a call to `MyClass`.
+and renders the structure of the yaml document as arguments for the [`document:true`](#document-true) rule.
+
+so you could, e.g. read the class name out of a yaml document like:
 
 ```yml
-slim_paths:
-  - '*.slim'
+class_name: MyClass
 ```
 
-Arrays are not necessary for single values. `*.slim` is recognized by default
+with config like:
 
-## `erb_paths:`
+```yml
+include_paths:
+  - 'config/*.yml'
 
-list filenames/paths of test directories that are in the erb format
-Defined using the [.gitignore pattern format](https://git-scm.com/docs/gitignore#_pattern_format)
+dynamic:
+  document: true
+  path: config/*.yml
+  calls:
+    argument: class_name
+```
+
+[`nested:`](#nested) may be useful for more complex yaml structures
+
+### `format: json`
+
+The json precompiler renders the structure of the json document as arguments for the [`document:true`](#document-true) rule.
+
+so you could, e.g. read the class name out of a json document like:
+
+```json
+{ "class_name": "MyClass" }
+```
+
+with config like:
 
 ```yml
-erb_paths:
-  - '*.erb'
+include_paths:
+  - 'config/*.json'
+
+dynamic:
+  document: true
+  path: config/*.json
+  calls:
+    argument: class_name
 ```
 
-Arrays are not necessary for single values. `*.erb` is recognized by default
+[`nested:`](#nested) may be useful for more complex json structures
 
 ## `gems:`
 _alias `gem:`_
@@ -141,10 +181,12 @@ Each entry can be a string (an exact match for a method, constant, or variable n
   or the properties from `names:`
   - [`has_prefix:`](#has_prefix)
   - [`has_suffix:`](#has_suffix)
-  - [`matches:`](#matches) (can't be used in the same entry as `has_prefix:` or `has_suffix:`)
+  - [`matches:`](#matches)
 - [`paths:`](#paths)
 - [`has_arguments:`](#has_arguments)
 - [`has_receiver:`](#has_receiver)
+- [`type:`](#type)
+- [`privacy:`](#privacy)
 - [`unless`](#unless)
 
 Arrays are not necessary for single values
@@ -170,7 +212,7 @@ Each entry can be a string (an exact match for a method, constant, or variable n
   or the properties from `names:`
   - [`has_prefix:`](#has_prefix)
   - [`has_suffix:`](#has_suffix)
-  - [`matches:`](#matches) (can't be used in the same entry as `has_prefix:` or `has_suffix:`)
+  - [`matches:`](#matches)
 - [`paths:`](#paths)
 - [`has_arguments:`](#has_arguments)
 - [`unless`](#unless)
@@ -202,11 +244,14 @@ Each entry must have at least one of the following properties to restrict which
 - [`paths:`](#paths)
 - [`has_arguments:`](#has_arguments)
 - [`has_receiver:`](#has_receiver)
-- [`unless`](#unless)
+- [`unless:`](#unless)
+- [`document: true`](#document-true)
 
-And must have one or both of
+And must have at least one of
 - ['calls:`](#calls-defines)
 - [`defines:`](#calls-defines)
+- [`set_privacy:](#set-privacy)
+- [`set_default_privacy:`](#set-default-privacy)
 
 Arrays are not necessary for single values.
 
@@ -232,7 +277,7 @@ This is a list of methods/constants/variables, and can be used in [`dynamic:`](#
 Each entry can be a string (an exact match for a method, constant, or variable name that includes the sigil), or have at least one of the following properties:
 - [`has_prefix:`](#has_prefix)
 - [`has_suffix:`](#has_suffix)
-- [`matches:`](#matches) (this can't be used in the same entry as `has_prefix:` or `has_suffix:`)
+- [`matches:`](#matches)
 
 Arrays are not necessary for single values
 
@@ -299,10 +344,37 @@ keep:
     path: /app/helpers
 ```
 
+## `document: true`
+
+Instructs to consider the whole document. this is useful when parsing [YAML](#yaml-paths) or [JSON](#json-paths) config files for various values.
+
+e.g.
+
+```yml
+includes: /config/roles.yml
+dynamic:
+  - document: true
+    path: /config/roles.yml
+    defines:
+      arguments: '*'
+      add_suffix: '?'
+      add_prefix: can_
+```
+
+will parse "config/roles.yml"
+```yml
+- build_house
+- drive_car
+```
+
+and consider it to have created methods like `can_build_house?` and `can_drive_car?`
+
+[`nested:`](#nested) may be useful for more complex data structures
+
 ## `calls:`, `defines:`
 _aliases `call:`, `define:`_
 
-At least one of these must be used in each entry in [`dynamic:`](#dynamic)
+These may be used as entries in [`dynamic:`](#dynamic)
 
 This is a list of values that are called or defined dynamically by the matched method, or eventually after being assigned to the the matched constant or variable.
 
@@ -354,6 +426,43 @@ dynamic:
   calls: 1
 ```
 
+## `set_privacy:`
+
+Set privacy has the same requirements as [`calls:` & `defines:`](#calls-defines).
+
+additional it requires a `to:` with one of either `private`, `public`, or `protected`.
+
+For example, from the ruby config:
+```yml
+dynamic:
+  name: private_class_method
+  has_argument: 0
+  set_privacy:
+    argument: '*'
+    to: private
+```
+
+which sets all methods named by the arguments to the privacy_class_method method to private.
+
+these methods could then be filtered using the [`privacy:`](#privacy) method in another [`dynamic:`](#dynamic) or [`keep:`](#keep) rule.
+
+Leftovers limits this to only affect methods defined in the same file.
+
+## `set_default_privacy:`
+
+This must be one of `public`, `private`, or `protected` and will set all subsequent method definitions in this file to that default privacy (unless its then overridden by [`set_privacy`](#set_privacy))
+
+For example, from the default ruby config:
+```yml
+dynamic:
+  name: private
+  unless:
+    has_argument: 0
+  set_default_privacy: private
+```
+
+these methods could then be filtered using the [`privacy:`](#privacy) method in another [`dynamic:`](#dynamic) or [`keep:`](#keep) rule.
+
 ## `arguments:`
 _alias `argument:`_
 
@@ -366,7 +475,9 @@ It can have any of these properties:
 - [`at:`](#at)
 - [`has_value:`](#has_value_has_receiver)
 
-Arrays are not necessary for single values and if the rule contains only `at:` it can be omitted, and the values moved up a level
+Arrays are not necessary for single values and if the rule contains only `at:` it can be omitted, and the values moved up a level.
+
+Positional arguments are zero indexed
 
 ## `has_arguments:`
 _alias `has_argument:`_
@@ -384,6 +495,8 @@ It can have any of these properties:
 
 Arrays are not necessary for single values and if the rule contains only `at:` it can be omitted, and the values moved up a level
 
+Positional arguments are zero indexed
+
 ## `keywords:`
 When the keyword argument **keywords** are the thing being called.
 
@@ -418,7 +531,7 @@ Each entry can be any of:
 - or have at least one of the following properties to match the keyword/hash key string or symbol:
   - [`has_prefix:`](#has_prefix)
   - [`has_suffix:`](#has_suffix)
-  - [`matches:`](#matches) (this can't be used in the same entry as `has_prefix:` or `has_suffix:`)
+  - [`matches:`](#matches)
 
 Arrays are not necessary for single values
 
@@ -429,10 +542,10 @@ filter [`arguments:`](#arguments), [`has_arguments:`](#has_arguments), and [`key
 Each entry can be one of
 - `true`, `false`, `nil`, or an Integer. matches the literal value
 - a String. matches the literal string or symbol value
-- or have at least one of the following properties to match a string or symbol:
+- or have at least one of the following properties to match the name:
   - [`has_prefix:`](#has_prefix)
   - [`has_suffix:`](#has_suffix)
-  - [`matches:`](#matches) (this can't be used in the same entry as `has_prefix:` or `has_suffix:`)
+  - [`matches:`](#matches)
 - or have at least one of the following properties to match within an array or hash:
   - [`at`](#at)
   - [`has_value`](#has_value_has_receiver)
@@ -441,17 +554,35 @@ Each entry can be one of
 - or have the following property to match the receiver
   - [`has_receiver`](#has_value_has_receiver)
 
+## `privacy:`
+
+filter [`dynamic:`](#dynamic) and [`keep:`](#keep) by method/constant privacy
+
+e.g.
+
+```yml
+keep:
+  - path: '**/generators/**/*_generator.rb'
+    privacy: public
+    type: Method
+```
+
+considers all public methods defined in rails generators to be called.
+
 ## `type:`
 
-Filter [`has_value`](#has_value_has_receiver), by the argument/assigned value type
+Filter by the literal type
 
 Each entry can be one of
-- `'String'`
-- `'Symbol'`
-- `'Integer'`
-- `'Float'`
-- `'Array'`
-- `'Hash'`
+- `'String'` a literal string, defined with "" or '' (not String.new)
+- `'Symbol'` a literal symbol
+- `'Integer'` a literal integer
+- `'Float'` a literal float
+- `'Array'` a literal array defined with [] (not Array.new)
+- `'Hash'` a literal hash, defined with {} (not Hash.new)
+- `'Proc'` a literal proc/lambda, defined with lambda {}, proc {}, or -> {} (not Proc.new)
+- `'Method'` a method call or definition defined with def, (not define_method {})
+- `'Constant'` a constant assignment, or a literal module or class defined with keywords, (not Module/Class.new)
 
 Arrays are not necessary for single values
 

data/docs/Custom-Precompilers.md

@@ -0,0 +1,38 @@
+# Custom Precompilers
+
+In addition to the built in precompilers, it's possible to add a custom precompiler
+
+It must be a class or module with a singleton method `precompile`. take a string of whatever code it likes, and return a string of valid ruby.
+
+```ruby
+require 'not_ruby' # require the gem that does the actual transformation
+
+module MyNotRubyPrecompiler
+  def self.precompile(not_ruby_content)
+    # not_ruby_content is a string of (hopefully) valid precompilable code
+    NotRuby::Parser.parse(not_ruby_content).to_ruby # output a string of valid ruby
+  end
+end
+```
+
+See the [build in precompilers](https://github.com/robotdana/leftovers/tree/main/lib/precompilers) for other examples.
+
+To configure the precompiler to be used by leftovers, add something similar to this to the `.leftovers.yml` file
+
+```yml
+  include_paths:
+    - 'lib/**/*.not_rb'
+  require: './path/to/my_not_ruby_precompiler'
+  precompile:
+    - paths: '*.not_rb'
+      format: { custom: MyNotRubyPrecompiler }
+```
+
+Ensure any necessary extension patterns are added to to [`include_paths:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#include_paths) for those particular files you wish to check.
+
+Require the custom precompiler using [`require:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#requires)
+
+Define which paths use the custom precompiler using [`precompile:`](https://github.com/robotdana/leftovers/tree/main/docs/Configuration.md#precompile),
+reference the name of the precompiler with `format: { custom: MyNotRubyPrecompiler }`
+
+If the `precompile` method raises any errors while precompiling, a warning will be printed to stderr and the file will be skipped

data/leftovers.gemspec

@@ -17,7 +17,9 @@ Gem::Specification.new do |spec|
   spec.metadata['homepage_uri'] = spec.homepage
   spec.metadata['source_code_uri'] = 'http://github.com/robotdana/leftovers'
   spec.metadata['changelog_uri'] = 'http://github.com/robotdana/leftovers/blob/main/CHANGELOG.md'
-  spec.required_ruby_version = '>= 2.4.0'
+
+  spec.metadata['rubygems_mfa_required'] = 'true'
+  spec.required_ruby_version = '>= 2.5.0'
 
   spec.files = Dir.glob('{lib,exe,docs}/**/{*,.*}') + %w{
     CHANGELOG.md
@@ -33,13 +35,15 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'activesupport'
   spec.add_development_dependency 'benchmark-ips'
   spec.add_development_dependency 'bundler', '~> 2.0'
+  spec.add_development_dependency 'did_you_mean'
   spec.add_development_dependency 'haml'
   spec.add_development_dependency 'pry', '~> 0.1'
   spec.add_development_dependency 'rake', '>= 13'
   spec.add_development_dependency 'rspec', '~> 3.0'
-  spec.add_development_dependency 'rubocop', '~> 0.93.1'
-  spec.add_development_dependency 'rubocop-performance', '~> 1.8.1'
-  spec.add_development_dependency 'rubocop-rspec', '~> 1.44.1'
+  spec.add_development_dependency 'rubocop'
+  spec.add_development_dependency 'rubocop-performance'
+  spec.add_development_dependency 'rubocop-rake'
+  spec.add_development_dependency 'rubocop-rspec'
   spec.add_development_dependency 'simplecov', '>= 0.18.5'
   spec.add_development_dependency 'simplecov-console'
   spec.add_development_dependency 'slim'
@@ -48,7 +52,6 @@ Gem::Specification.new do |spec|
 
   spec.add_development_dependency 'spellr', '>= 0.8.1'
   spec.add_dependency 'fast_ignore', '>= 0.15.1'
-  spec.add_dependency 'json_schemer'
   spec.add_dependency 'parallel'
   spec.add_dependency 'parser'
 end

data/lib/config/actionmailer.yml

@@ -9,14 +9,14 @@ keep:
   - delivered_email
 
 dynamic:
-  names:
-    - deliver_now
-    - deliver_later
-  calls:
-    value:
-      deliver
-  names:
-    - before_action
-    - after_action
-    - around_action
-  calls: ['*', if, unless]
+  - names:
+      - deliver_now
+      - deliver_later
+    calls:
+      value:
+        deliver
+  - names:
+      - before_action
+      - after_action
+      - around_action
+    calls: ['*', if, unless]

data/lib/config/actionpack.yml

@@ -7,9 +7,11 @@ include_paths:
   - '*.rjs'
   - '*.rhtml'
 
-erb_paths:
-  - '*.rjs'
-  - '*.rhtml'
+precompile:
+  - paths:
+      - '*.rjs'
+      - '*.rhtml'
+    format: erb
 
 keep:
   - ssl_configured? # ApplicationController

data/lib/config/activesupport.yml

@@ -2,7 +2,7 @@ requires:
   - 'active_support'
   - 'active_support/core_ext/string'
   - 'active_support/inflections'
-  - './config/initializers/inflections'
+  - quiet: './config/initializers/inflections'
 
 # THIS IS INCOMPLETE (you can help by expanding it)
 # rails is _really complicated_ and has a lot of magic which calls methods for you.

data/lib/config/haml.yml

@@ -1,4 +1,6 @@
 include_paths:
   - '*.haml'
-haml_paths:
-  - '*.haml'
+requires: 'haml'
+precompile:
+  format: haml
+  paths: '*.haml'

data/lib/config/rails.yml

@@ -3,7 +3,6 @@
 # some is currently impossible to handle (with_options).
 # Some is just corners of rails I haven't hit yet.
 gems:
-  - actioncable
   - actionmailer
   - actionpack
   - actionview
@@ -11,6 +10,7 @@ gems:
   - activemodel
   - activerecord
   - activesupport
+  - railties
 
 keep:
   - default_url_options # called by url_for, unsure what gem does

data/lib/config/railties.yml

@@ -0,0 +1,11 @@
+keep:
+  - # https://guides.rubyonrails.org/generators.html#generators-lookup
+    path: '**/generators/**/*_generator.rb'
+    # "When a generator is invoked, each public method in the generator is executed sequentially in
+    # the order that it is defined When a generator is invoked, each public method in the generator
+    # is executed sequentially in the order that it is defined"
+    #
+    # https://guides.rubyonrails.org/generators.html
+    privacy: public
+    type: Method
+