keela 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0755e73eaeeb61f9f97ffa109fd8c421316e7725b83ef2502a37c5ba0111858f
4
- data.tar.gz: d66e1092dd578f28ae3d727b179b601fcd7068671550f9186e53c76a543d1763
3
+ metadata.gz: b149f85c049e9f0a669ccdef6c9f70f4bcd627d9bec77a543303fae76592cfde
4
+ data.tar.gz: 3ddae5519560492fbd7fc223065047f787f08bf9c8e99da35efdcbfec0e8e1c7
5
5
  SHA512:
6
- metadata.gz: bf117842ed4d8891671dac92ee34c48b5433f50795f9a90963211c1e650df11b31e72c2d142a6fc489f3dfb040989341764ab6eba4d2ea70432828bd6ed1c1dc
7
- data.tar.gz: ac2d7352393d70240f7ad1541f2ff476e5ed7c8def8b13212721a527df4e7c285b2c31519333ebe712aab31e9a186a997420598e7e7662af181353cf9f8b96ca
6
+ metadata.gz: c02762a44f0cab25a0c3711001bcc1891cd016c489318b55c837b4a6be87973551c7f073a29b3ef97691fadf5e6d5c2b040b873f3b2a37a0f1c58bdcf6fc78d7
7
+ data.tar.gz: 529a21fb0e9e8c1e603f48b6f3565a6b8f85c81a939a7120301476c444ce0478d2aac842eaa2f4689afc80803847f6f1f577e8010842f3517c8a4b20fa6fc8d6
data/CHANGELOG.md CHANGED
@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.6.0] - 2026-10-30
11
+
12
+ ### Added
13
+
14
+ - **Stale exclusions are now reported during a normal run, not just under `--test-exclusions`.** An entry in the excluded YAML whose definition no longer exists (removed, renamed, or its file deleted) was previously only caught by the separate `--test-exclusions` command, so the exclusions file silently rotted. A normal run now flags these inline, elevated to the same behavior as newly unused findings: in baseline mode it fails the run (exit 1), and structured output (`--format json|toon`) carries a `stale_exclusions` section and a `stale_total` summary count. Detection is scoped to the current strategy's section; legacy flat-format entries are skipped to avoid cross-strategy false positives ([#103](https://github.com/kerrizor/keela/issues/103))
15
+ - **New `--normalize-exclusions` flag rewrites the exclusions file in sorted, normalized order.** An explicit write mode (modeled on `--update-baseline`) that deep-sorts the excluded YAML: strategy keys, then file keys within each strategy, then the `name: reason` entries by name. It is reorder-only and non-destructive, it never prunes stale entries, and it preserves every reason verbatim (reasons containing colons stay quoted). Both the strategy-aware and legacy flat formats are handled, output is idempotent, and the file is rewritten with a fixed header (freeform comments are not preserved) ([#93](https://github.com/kerrizor/keela/issues/93))
16
+
17
+ ### Changed
18
+
19
+ - Relaxed the `parallel` dependency from `~> 1.20` to `>= 1.20, < 3`, so keela no longer forces a downgrade in projects already on parallel 2.x, while still supporting 1.x ([#101](https://github.com/kerrizor/keela/pull/101))
20
+
21
+ ### Fixed
22
+
23
+ - **`has_scope :name` is no longer swallowed as a false positive by the `scopes` strategy.** The usage regex excluded any line ending in `scope :` to skip the `scope :name` definition, but `has_scope :name` (common in Rails controllers and ActiveAdmin) also ends in `scope :`, so real usages were rejected and the scope was wrongly reported unused. The lookbehind now requires a whitespace character before `scope`, so an indented definition is still excluded while `has_scope` (preceded by `_`) counts as usage ([#100](https://github.com/kerrizor/keela/issues/100))
24
+ - **A method named only in a comment is no longer counted as used.** Usage detection matched its regex against the whole source, comments included, so a method whose name appeared in a comment was treated as called and genuinely dead code was never flagged. Whole-line Ruby comments (`.rb`, `.rake`, `.builder`) are now dropped from the usage corpus before matching. HAML is left alone (a leading `#` there is an element id), and `#{}` interpolation and trailing comments are kept, since telling a real `#` from one inside a string or regexp needs a tokenizer ([#104](https://github.com/kerrizor/keela/pull/104))
25
+ - Run every selected strategy in baseline mode even when an earlier strategy fails, so text and structured reports include all unused code findings ([#102](https://github.com/kerrizor/keela/pull/102)).
26
+ - **Lazy i18n lookups (`t(".key")`) in mailers, controllers, and `.text.` mailer templates are no longer reported as false-positive unused keys.** The `i18n_keys` strategy only expanded lazy lookups inside `.html.` view templates, so a lazy key used in a mailer/controller Ruby method or a `.text.erb` template was flagged unused. The view-path matcher now also covers `.text.` templates, and a pragmatic line scan of `app/mailers` and `app/controllers` (plus `ee/` variants) resolves each lazy `t(".x")` against its enclosing class and method scope, keeping `_mailer` for mailers and dropping `_controller` for controllers ([#73](https://github.com/kerrizor/keela/issues/73))
27
+ - **Multi-line `delegate` declarations now detect every delegated method, not just the one on the first line.** The `delegations` strategy parsed files line by line, so a `delegate` wrapped across several lines only matched the methods on the opening line; the continuation lines did not match `delegate` and the opening line had no `to:` for target/prefix detection. The strategy now joins each `delegate` statement, across continuation lines while a line ends in a comma (after a trailing comment is stripped), into one logical statement before extracting, so target, `prefix:`, and every method resolve correctly. Single-line delegates and comment skipping are preserved ([#52](https://github.com/kerrizor/keela/issues/52))
28
+
29
+ ## [0.5.1] - 2026-10-06
30
+
31
+ ### Changed
32
+
33
+ - **The `partials` strategy is dramatically faster on large codebases, up to ~27× less CPU.** It now collects every rendered partial in a single pass over the source instead of re-scanning the whole codebase once per partial. Measured against a large reference Rails monolith, partials went from the slowest strategy to effectively free, and no longer meaningfully slows a combined `--type all,partials` run. The report is unchanged ([#96](https://github.com/kerrizor/keela/pull/96))
34
+
10
35
  ## [0.5.0] - 2026-09-21
11
36
 
12
37
  ### Added
data/README.md CHANGED
@@ -63,6 +63,7 @@ If a `.keela_baseline.yml` file exists, Keela compares the current scan against
63
63
  | No changes from baseline | ✅ Pass (silent, exit 0) |
64
64
  | NEW unused code detected | ❌ Fail (shows new items) |
65
65
  | Code REMOVED from baseline | ❌ Fail (prompts to update baseline) |
66
+ | STALE exclusion (its definition is gone) | ❌ Fail (shows stale entries to remove) |
66
67
 
67
68
  This lets you gradually pay down tech debt while preventing new dead code from sneaking in.
68
69
 
@@ -178,6 +179,9 @@ keela --source-location
178
179
  # Validate exclusion file entries (find stale exclusions)
179
180
  keela --test-exclusions
180
181
 
182
+ # Rewrite the exclusion file in sorted, normalized order
183
+ keela --normalize-exclusions
184
+
181
185
  # Show version
182
186
  keela --version
183
187
  ```
@@ -193,8 +197,8 @@ Keela detects several types of unused code:
193
197
  | **constants** | Unused constants | `UNUSED_CONSTANT = 'value'` |
194
198
  | **delegations** | Unused delegate declarations | `delegate :unused, to: :target` |
195
199
  | **attributes** | Unused attr_* declarations | `attr_accessor :unused_attr` |
196
- | **i18n_keys** | Unused translation keys | `en.users.unused_key` in locale YAML |
197
- | **partials** | Unused Rails view partials | `app/views/users/_unused.html.erb` |
200
+ | **i18n_keys** _(beta)_ | Unused translation keys | `en.users.unused_key` in locale YAML |
201
+ | **partials** _(beta)_ | Unused Rails view partials | `app/views/users/_unused.html.erb` |
198
202
 
199
203
  Run all strategies (default) or target specific ones with `--type`.
200
204
 
@@ -508,6 +512,18 @@ Or configure in `keela.yml`:
508
512
  excluded_path: ".keela/excluded.yml"
509
513
  ```
510
514
 
515
+ ### Normalizing the Exclusion File
516
+
517
+ As the exclusion file grows, entries tend to land in whatever order they were added, which makes diffs noisy. Rewrite it in a sorted, canonical order with:
518
+
519
+ ```bash
520
+ keela --normalize-exclusions
521
+ ```
522
+
523
+ This sorts the strategy sections, the files within each section, and the entries within each file, then rewrites the file with a fixed header. It is reorder-only: no entry is removed, and every `name: reason` pair is preserved (use `--test-exclusions` to find stale entries). Running it twice produces identical output.
524
+
525
+ Freeform comments are not preserved: the rewrite regenerates a fixed header, so any hand-written comments in the file are dropped.
526
+
511
527
  ## Ruby API
512
528
 
513
529
  ```ruby
data/exe/keela CHANGED
@@ -94,6 +94,10 @@ OptionParser.new do |opts|
94
94
  options[:test_exclusions] = true
95
95
  end
96
96
 
97
+ opts.on("--normalize-exclusions", "Rewrite the exclusions file in sorted, normalized order") do
98
+ options[:normalize_exclusions] = true
99
+ end
100
+
97
101
  opts.on("-h", "--help", "Show this help") do
98
102
  puts opts
99
103
  exit
@@ -135,6 +139,21 @@ if options[:test_exclusions]
135
139
  exit(success ? 0 : 1)
136
140
  end
137
141
 
142
+ # Handle --normalize-exclusions mode
143
+ if options[:normalize_exclusions]
144
+ excluded_path = Keela.configuration.excluded_path ||
145
+ Keela::Scanner::DEFAULT_EXCLUDED_PATHS.find { |p| File.exist?(p) }
146
+
147
+ unless excluded_path
148
+ warn "Error: No exclusion file found. Specify with --excluded PATH"
149
+ exit 1
150
+ end
151
+
152
+ normalizer = Keela::ExclusionNormalizer.new(excluded_path)
153
+ success = normalizer.normalize
154
+ exit(success ? 0 : 1)
155
+ end
156
+
138
157
  STRATEGY_MAP = {
139
158
  methods: Keela::Strategies::Methods,
140
159
  scopes: Keela::Strategies::Scopes,
@@ -210,6 +229,7 @@ end
210
229
 
211
230
  success = true
212
231
  results = {}
232
+ stale_exclusions = {}
213
233
 
214
234
  structured_output = Keela::Formatters.structured_format?(options[:format])
215
235
  silent_output = structured_output || options[:quiet]
@@ -220,14 +240,16 @@ strategies.each_with_index do |strategy, index|
220
240
  end
221
241
 
222
242
  scanner = Keela::Scanner.new(strategy: strategy, baseline: baseline, source_files: source_files)
223
- success &&= scanner.run(
243
+ strategy_success = scanner.run(
224
244
  force_report: options[:force_report],
225
245
  update_baseline: options[:update_baseline],
226
246
  silent: silent_output
227
247
  )
248
+ success &&= strategy_success
228
249
 
229
250
  # Collect results for structured output formats
230
251
  results[strategy.name] = scanner.unused_collection.transform_values(&:to_a)
252
+ stale_exclusions[strategy.name] = scanner.stale_exclusions.transform_values(&:to_a)
231
253
 
232
254
  unless silent_output
233
255
  puts if strategies.size > 1 && index < strategies.size - 1
@@ -243,7 +265,7 @@ end
243
265
  # Output structured format if requested
244
266
  if structured_output
245
267
  formatter_class = Keela::Formatters.for(options[:format])
246
- formatter = formatter_class.new(results: results, strategies: strategies)
268
+ formatter = formatter_class.new(results: results, strategies: strategies, stale_exclusions: stale_exclusions)
247
269
  puts formatter.format
248
270
  end
249
271
 
@@ -0,0 +1,105 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "yaml"
4
+ require "rainbow"
5
+
6
+ module Keela
7
+ # Rewrites an exclusion file in sorted, normalized order.
8
+ #
9
+ # Reorder-only and non-destructive: it never prunes stale entries, it only
10
+ # deep-sorts the existing data and re-dumps it with a fixed header. Freeform
11
+ # YAML comments are not preserved (Psych drops them on load).
12
+ #
13
+ class ExclusionNormalizer
14
+ STRATEGY_MAP = {
15
+ methods: Strategies::Methods,
16
+ scopes: Strategies::Scopes,
17
+ constants: Strategies::Constants,
18
+ delegations: Strategies::Delegations,
19
+ attributes: Strategies::Attributes,
20
+ i18n_keys: Strategies::I18nKeys
21
+ }.freeze
22
+
23
+ attr_reader :excluded_path
24
+
25
+ def initialize(excluded_path)
26
+ @excluded_path = excluded_path
27
+ end
28
+
29
+ def normalize
30
+ unless File.exist?(excluded_path)
31
+ puts Rainbow("Exclusion file not found: #{excluded_path}").red
32
+ return false
33
+ end
34
+
35
+ data = YAML.load_file(excluded_path, symbolize_names: true) || {}
36
+
37
+ sorted = strategy_aware_format?(data) ? deep_sort_strategy_aware(data) : deep_sort_flat(data)
38
+
39
+ File.write(excluded_path, serialize(sorted))
40
+ puts Rainbow("Normalized #{excluded_path}").green.bright
41
+ true
42
+ end
43
+
44
+ private
45
+
46
+ def strategy_aware_format?(data)
47
+ data.keys.any? { |k| STRATEGY_MAP.key?(k) }
48
+ end
49
+
50
+ def deep_sort_strategy_aware(data)
51
+ data.sort_by { |key, _| key.to_s }.to_h do |strategy_name, files|
52
+ [strategy_name.to_s, sort_files(files || {})]
53
+ end
54
+ end
55
+
56
+ def deep_sort_flat(data)
57
+ sort_files(data)
58
+ end
59
+
60
+ def sort_files(files)
61
+ files.sort_by { |key, _| key.to_s }.to_h do |file_path, entries|
62
+ [file_path.to_s, sort_entries(entries || [])]
63
+ end
64
+ end
65
+
66
+ # Sorts the name:reason entry array by name, converting the symbolized
67
+ # key back to a string so the dumped YAML uses string keys.
68
+ #
69
+ def sort_entries(entries)
70
+ entries
71
+ .map { |entry| { entry.keys.first.to_s => entry.values.first } }
72
+ .sort_by { |entry| entry.keys.first }
73
+ end
74
+
75
+ def serialize(data)
76
+ if data.empty? || data.values.all? { |v| v.respond_to?(:empty?) && v.empty? }
77
+ "#{header}---\n{}\n"
78
+ else
79
+ "#{header}#{format_yaml(data)}"
80
+ end
81
+ end
82
+
83
+ def header
84
+ <<~HEADER
85
+ # Excluded items that Keela should not flag as unused.
86
+ # These are known to be in use (e.g. called via metaprogramming).
87
+ #
88
+ # This file is normalized by `keela --normalize-exclusions`.
89
+ # Freeform comments are not preserved when it is rewritten.
90
+ #
91
+ HEADER
92
+ end
93
+
94
+ def format_yaml(hash)
95
+ indent_yaml_list_items(hash.to_yaml)
96
+ end
97
+
98
+ # Indents YAML list items that are not already indented, matching the
99
+ # style Baseline uses for its output.
100
+ #
101
+ def indent_yaml_list_items(yaml_string)
102
+ yaml_string.gsub(/\n-(\s+\S)/, "\n -\\1")
103
+ end
104
+ end
105
+ end
@@ -3,11 +3,12 @@
3
3
  module Keela
4
4
  module Formatters
5
5
  class Base
6
- attr_reader :results, :strategies
6
+ attr_reader :results, :strategies, :stale_exclusions
7
7
 
8
- def initialize(results:, strategies:)
8
+ def initialize(results:, strategies:, stale_exclusions: {})
9
9
  @results = results
10
10
  @strategies = strategies
11
+ @stale_exclusions = stale_exclusions
11
12
  end
12
13
 
13
14
  def format
@@ -28,6 +29,14 @@ module Keela
28
29
  def non_empty_results
29
30
  results.reject { |_, v| v.empty? }
30
31
  end
32
+
33
+ def non_empty_stale_exclusions
34
+ stale_exclusions.reject { |_, v| v.empty? }
35
+ end
36
+
37
+ def stale_total
38
+ stale_exclusions.values.flat_map(&:values).flatten.size
39
+ end
31
40
  end
32
41
  end
33
42
  end
@@ -9,9 +9,11 @@ module Keela
9
9
  output = {
10
10
  strategies: strategies.map(&:name),
11
11
  unused: non_empty_results,
12
+ stale_exclusions: non_empty_stale_exclusions,
12
13
  summary: {
13
14
  total: total_count,
14
- by_strategy: by_strategy_counts
15
+ by_strategy: by_strategy_counts,
16
+ stale_total: stale_total
15
17
  }
16
18
  }
17
19
 
@@ -9,9 +9,11 @@ module Keela
9
9
  output = {
10
10
  "strategies" => strategies.map(&:name),
11
11
  "unused" => non_empty_results.transform_keys(&:to_s),
12
+ "stale_exclusions" => non_empty_stale_exclusions.transform_keys(&:to_s),
12
13
  "summary" => {
13
14
  "total" => total_count,
14
- "by_strategy" => by_strategy_counts.transform_keys(&:to_s)
15
+ "by_strategy" => by_strategy_counts.transform_keys(&:to_s),
16
+ "stale_total" => stale_total
15
17
  }
16
18
  }
17
19
 
@@ -36,6 +36,20 @@ module Keela
36
36
  print_removed(removed, baseline_path) unless removed.empty?
37
37
  end
38
38
 
39
+ def print_stale_exclusions(stale, excluded_path)
40
+ return if stale.empty?
41
+
42
+ count = stale.values.flatten.size
43
+ message = <<~MESSAGE
44
+ We have detected #{count} stale #{strategy_name} exclusion(s).
45
+
46
+ Their definitions no longer exist; remove them from #{excluded_path}.
47
+ MESSAGE
48
+
49
+ puts Rainbow(message).red.bright
50
+ puts Rainbow(format_yaml(stale)).red.bright
51
+ end
52
+
39
53
  def format_report(collection, source_locations = {})
40
54
  if source_locations.empty?
41
55
  format_yaml(collection)
data/lib/keela/scanner.rb CHANGED
@@ -1,11 +1,12 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "parallel"
4
+ require "set"
4
5
  require "yaml"
5
6
 
6
7
  module Keela
7
8
  class Scanner
8
- attr_reader :strategy, :configuration, :baseline, :source_files, :unused_collection, :source_locations, :new_unused, :removed
9
+ attr_reader :strategy, :configuration, :baseline, :source_files, :unused_collection, :source_locations, :new_unused, :removed, :stale_exclusions
9
10
 
10
11
  DEFAULT_EXCLUDED_PATHS = [".keela/excluded.yml", "keela_excluded.yml"].freeze
11
12
  DEFAULT_BASELINE_PATHS = [".keela/baseline.yml", "keela_baseline.yml"].freeze
@@ -51,6 +52,7 @@ module Keela
51
52
  @source_locations = {} # { "file:name" => line_number }
52
53
  @new_unused = []
53
54
  @removed = []
55
+ @stale_exclusions = Hash.new { |hash, key| hash[key] = [] }
54
56
  end
55
57
 
56
58
  def run(force_report: false, update_baseline: false, silent: false)
@@ -71,7 +73,10 @@ module Keela
71
73
 
72
74
  if report_mode
73
75
  elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
74
- reporter.print_full_report(unused_collection, elapsed, source_locations: source_locations) unless silent
76
+ unless silent
77
+ reporter.print_full_report(unused_collection, elapsed, source_locations: source_locations)
78
+ reporter.print_stale_exclusions(stale_exclusions, resolve_excluded_path) unless stale_exclusions.empty?
79
+ end
75
80
  if update_baseline
76
81
  baseline.set(strategy.name, unused_collection)
77
82
  # Note: caller is responsible for calling baseline.save after all strategies run
@@ -91,7 +96,9 @@ module Keela
91
96
  )
92
97
  end
93
98
 
94
- new_unused.empty? && removed.empty?
99
+ reporter.print_stale_exclusions(stale_exclusions, resolve_excluded_path) unless silent || stale_exclusions.empty?
100
+
101
+ new_unused.empty? && removed.empty? && stale_exclusions.empty?
95
102
  end
96
103
 
97
104
  def file_globs
@@ -181,18 +188,37 @@ module Keela
181
188
  # Support both formats:
182
189
  # 1. Strategy-aware (new): { methods: { "file.rb": [{ name: "reason" }] } }
183
190
  # 2. Flat (legacy): { "file.rb": [{ name: "reason" }] }
184
- excluded = if all_excluded.key?(strategy.name.to_sym)
191
+ strategy_aware = all_excluded.key?(strategy.name.to_sym)
192
+ excluded = if strategy_aware
185
193
  all_excluded[strategy.name.to_sym] || {}
186
194
  else
187
195
  all_excluded # Legacy flat format
188
196
  end
189
197
 
198
+ # Stale detection is only safe for the current strategy's section. A legacy
199
+ # flat entry may belong to another strategy, and keela runs one strategy per
200
+ # Scanner, so flagging it here would be a false positive. Skip flat format.
201
+ #
202
+ detect_stale_exclusions(excluded, definitions) if strategy_aware
203
+
190
204
  definitions.reject do |h|
191
205
  excluded_for_file = excluded[h[:file].to_sym]
192
206
  excluded_for_file&.flat_map(&:keys)&.include?(h[:name].to_sym)
193
207
  end
194
208
  end
195
209
 
210
+ def detect_stale_exclusions(excluded, definitions)
211
+ present = definitions.map { |h| [h[:file].to_sym, h[:name].to_sym] }.to_set
212
+
213
+ excluded.each do |file, entries|
214
+ Array(entries).flat_map(&:keys).each do |name|
215
+ next if present.include?([file, name])
216
+
217
+ @stale_exclusions[file.to_s] << name.to_s
218
+ end
219
+ end
220
+ end
221
+
196
222
  def resolve_excluded_path
197
223
  return configuration.excluded_path if configuration.excluded_path && File.exist?(configuration.excluded_path)
198
224
 
@@ -214,6 +240,17 @@ module Keela
214
240
  # (e.g., I18n lazy lookup)
215
241
  additional_used = strategy.additional_used_names(source_files)
216
242
 
243
+ # A strategy whose #additional_used_names enumerates every used name in one
244
+ # pass (e.g. partials) decides purely by Set membership. Skip building the
245
+ # derived source views and the per-definition #used? scan entirely: that
246
+ # per-definition scan is O(N x whole-codebase) and is the hot path we are
247
+ # avoiding here.
248
+ #
249
+ if strategy.detects_all_usage?
250
+ unused = definitions.reject { |definition| additional_used.include?(definition[:name]) }
251
+ return record_unused(unused)
252
+ end
253
+
217
254
  # Build the source views the strategy needs here, in the parent, so that
218
255
  # the workers Parallel forks below inherit them instead of each building
219
256
  # its own copy of a string the size of the whole codebase.
@@ -228,10 +265,16 @@ module Keela
228
265
  strategy.used?(definition[:name], source) ? [] : definition
229
266
  end
230
267
 
231
- # A single logical definition can be extracted from multiple lines
232
- # (e.g. the same method delegated twice, or a constant declared and
233
- # referenced), so guard against listing the same name twice per file.
234
- #
268
+ record_unused(unused)
269
+ end
270
+
271
+ # Collect the unused definitions into @unused_collection / @source_locations.
272
+ #
273
+ # A single logical definition can be extracted from multiple lines (e.g. the
274
+ # same method delegated twice, or a constant declared and referenced), so
275
+ # guard against listing the same name twice per file.
276
+ #
277
+ def record_unused(unused)
235
278
  unused.each do |unused_def|
236
279
  names = @unused_collection[unused_def[:file]]
237
280
  next if names.include?(unused_def[:name])
data/lib/keela/source.rb CHANGED
@@ -14,6 +14,11 @@ module Keela
14
14
  # control characters the range ends in.
15
15
  ASCII_RANGE = [0x00, 0x7F].map { |code_point| code_point.chr(Encoding::UTF_8) }.join("-").freeze
16
16
 
17
+ # Extensions whose "#" begins a line comment. HAML is excluded on purpose:
18
+ # there "#id" is an element id, not a comment.
19
+ #
20
+ RUBY_EXTENSIONS = %w[.rb .rake .builder].freeze
21
+
17
22
  attr_reader :text
18
23
 
19
24
  def initialize(text)
@@ -29,8 +34,23 @@ module Keela
29
34
  # no single file (issue #83). Empty files contribute nothing.
30
35
  #
31
36
  def self.from_source_files(source_files)
32
- new(source_files.values.map { |lines| terminate(lines.join) }.join)
37
+ new(source_files.map { |filename, lines| terminate(strip_comments(filename, lines).join) }.join)
38
+ end
39
+
40
+ # Drop whole-line Ruby comments from the usage corpus.
41
+ #
42
+ # A method name mentioned in a comment is not a call, but the usage regex
43
+ # matches it anyway and would hide genuinely dead code. Only whole-line
44
+ # comments are dropped: telling a real "#" from one inside a string, a
45
+ # regexp, or "#{" interpolation needs a tokenizer, not worth it for the
46
+ # trailing-comment case.
47
+ #
48
+ def self.strip_comments(filename, lines)
49
+ return lines unless RUBY_EXTENSIONS.include?(File.extname(filename))
50
+
51
+ lines.reject { |line| line.match?(/\A\s*#(?!\{)/) }
33
52
  end
53
+ private_class_method :strip_comments
34
54
 
35
55
  def self.terminate(text)
36
56
  return text if text.empty? || text.end_with?("\n")
@@ -48,6 +48,62 @@ module Keela
48
48
  methods.length == 1 ? methods.first : methods
49
49
  end
50
50
 
51
+ # Own extraction for delegation files so multi-line delegate statements
52
+ # are detected in full. The scanner's default line-by-line path sees only
53
+ # the first line of a wrapped delegate, missing every continuation method.
54
+ # We join each statement into one logical line, then reuse
55
+ # #extract_definition on the result so target/prefix detection is unchanged.
56
+ #
57
+ # Because returning non-nil makes the scanner skip its default path, this
58
+ # must also replicate that path's comment-skipping and its line-gating on
59
+ # configuration.source_location.
60
+ #
61
+ def extract_definitions_from_file(filepath, lines)
62
+ track_lines = Keela.configuration.source_location
63
+ definitions = []
64
+ index = 0
65
+
66
+ while index < lines.length
67
+ line = lines[index]
68
+
69
+ if skip_comments? && line.strip.start_with?("#")
70
+ index += 1
71
+ next
72
+ end
73
+
74
+ unless line =~ /^\s*delegate\s+/
75
+ index += 1
76
+ next
77
+ end
78
+
79
+ start_index = index
80
+ statement = strip_comment(line)
81
+
82
+ # A delegate statement continues onto the next line while the current
83
+ # line, ignoring a trailing comment and trailing whitespace, ends in a
84
+ # comma. It stops at the first line that does not (the options line).
85
+ # Strip inline comments as we join so #extract_definition sees the same
86
+ # clean text the scanner's default (comment-skipping) path would.
87
+ #
88
+ while continuation?(lines[index])
89
+ index += 1
90
+ break if index >= lines.length
91
+
92
+ statement += strip_comment(lines[index])
93
+ end
94
+
95
+ index += 1
96
+
97
+ Array(extract_definition(statement)).compact.each do |name|
98
+ definition = { name: name, file: filepath }
99
+ definition[:line] = start_index + 1 if track_lines
100
+ definitions << definition
101
+ end
102
+ end
103
+
104
+ definitions
105
+ end
106
+
51
107
  def usage_regex(name)
52
108
  # Match usage of the delegated method, but not the delegate declaration
53
109
  # Uses negative lookbehind to avoid matching:
@@ -85,6 +141,25 @@ module Keela
85
141
 
86
142
  private
87
143
 
144
+ # Whether +line+ hands off to a continuation line: true when it ends in a
145
+ # comma after a trailing comment and whitespace are stripped. A `#` inside
146
+ # a delegate statement can only start a comment, since method symbols
147
+ # cannot contain one, so dropping from the first `#` is safe here.
148
+ #
149
+ def continuation?(line)
150
+ return false unless line
151
+
152
+ strip_comment(line).rstrip.end_with?(",")
153
+ end
154
+
155
+ # Drop a trailing line comment, preserving the newline. A `#` inside a
156
+ # delegate statement can only start a comment, since method symbols cannot
157
+ # contain one, so dropping from the first `#` is safe here.
158
+ #
159
+ def strip_comment(line)
160
+ line.sub(/#[^\n]*/, "")
161
+ end
162
+
88
163
  def folded_usage_regex(name)
89
164
  /(?<!:)(?<!delegate\s)(?<![a-z_])#{Regexp.quote(name.downcase(:fold))}(?!\w)/
90
165
  end
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "yaml"
4
+ require "set"
4
5
 
5
6
  module Keela
6
7
  module Strategies
@@ -21,9 +22,17 @@ module Keela
21
22
  # Lazy lookup is supported: t('.title') in app/views/users/show.html.erb
22
23
  # resolves to 'users.show.title'.
23
24
  #
25
+ # Lazy lookup in mailer/controller Ruby methods is also supported, e.g.
26
+ # t('.greeting') in UserMailer#welcome resolves to 'user_mailer.welcome.greeting',
27
+ # and t('.notice') in UsersController#create resolves to 'users.create.notice'.
28
+ #
24
29
  class I18nKeys < Strategy
25
30
  PLURAL_SUFFIXES = %w[zero one two few many other].freeze
26
- VIEW_PATH_REGEX = %r{(?:ee/)?app/views/(.+)\.html\.(?:erb|haml|slim)$}.freeze
31
+ VIEW_PATH_REGEX = %r{(?:ee/)?app/views/(.+)\.(?:html|text)\.(?:erb|haml|slim)$}.freeze
32
+ RUBY_SCOPE_PATH_REGEX = %r{(?:ee/)?app/(?:mailers|controllers)/.+\.rb$}.freeze
33
+ CLASS_DEF_REGEX = /^\s*class\s+([A-Z][\w:]*)/.freeze
34
+ MODULE_DEF_REGEX = /^\s*module\s+([A-Z][\w:]*)/.freeze
35
+ METHOD_DEF_REGEX = /^\s*def\s+(?:self\.)?([a-z_][\w]*[?!=]?)/.freeze
27
36
  LAZY_LOOKUP_REGEX = /(?:I18n\.)?t\s*\(\s*['"](\.[^'"]+)['"]/
28
37
  def name
29
38
  "i18n_keys"
@@ -95,6 +104,7 @@ module Keela
95
104
  # "app/views/users/show.html.erb" -> "users.show"
96
105
  # "app/views/users/_form.html.erb" -> "users.form"
97
106
  # "ee/app/views/users/show.html.erb" -> "users.show"
107
+ # "app/views/user_mailer/welcome.text.erb" -> "user_mailer.welcome"
98
108
  def view_path_to_prefix(path)
99
109
  return nil unless path =~ VIEW_PATH_REGEX
100
110
 
@@ -115,22 +125,21 @@ module Keela
115
125
  content.scan(LAZY_LOOKUP_REGEX).flatten
116
126
  end
117
127
 
118
- # Build a set of expanded lazy lookup keys from view files
119
- # Called by the scanner to detect usage via lazy lookup
128
+ # Build a set of expanded lazy lookup keys from view files,
129
+ # mailers, and controllers. Called by the scanner to detect
130
+ # usage via lazy lookup.
120
131
  def additional_used_names(source_files)
121
132
  expanded = Set.new
122
133
 
123
134
  source_files.each do |filepath, lines|
124
- prefix = view_path_to_prefix(filepath)
125
- next unless prefix
126
-
127
- content = lines.join("\n")
128
- lazy_keys = extract_lazy_keys(content)
129
-
130
- lazy_keys.each do |lazy_key|
131
- # .title -> users.show.title
132
- full_key = "#{prefix}#{lazy_key}"
133
- expanded << full_key
135
+ if (prefix = view_path_to_prefix(filepath))
136
+ content = lines.join("\n")
137
+ extract_lazy_keys(content).each do |lazy_key|
138
+ # .title -> users.show.title
139
+ expanded << "#{prefix}#{lazy_key}"
140
+ end
141
+ elsif filepath =~ RUBY_SCOPE_PATH_REGEX
142
+ expand_ruby_lazy_keys(lines).each { |key| expanded << key }
134
143
  end
135
144
  end
136
145
 
@@ -139,6 +148,70 @@ module Keela
139
148
 
140
149
  private
141
150
 
151
+ # Scan a mailer/controller Ruby file line by line, tracking the current
152
+ # class scope and method, and expand each lazy t('.x') to its full key.
153
+ #
154
+ # This is a pragmatic line scan (no AST) matching keela's regex style.
155
+ # It tracks the last-seen `def` as the enclosing method and does NOT
156
+ # attempt precise `end` matching. A lazy call that appears after a method
157
+ # body but before the next `def` is still attributed to the last method.
158
+ # This over-association can only mark an EXTRA key as used, never cause a
159
+ # false "unused" flag, so it is a safe conservative tradeoff. A lazy call
160
+ # seen before any `def` has no enclosing method and is skipped.
161
+ #
162
+ def expand_ruby_lazy_keys(lines)
163
+ keys = []
164
+ namespace = []
165
+ class_scope = nil
166
+ current_method = nil
167
+
168
+ lines.each do |line|
169
+ if (m = line.match(MODULE_DEF_REGEX))
170
+ namespace << m[1]
171
+ elsif (m = line.match(CLASS_DEF_REGEX))
172
+ class_scope = ruby_class_scope(namespace, m[1])
173
+ current_method = nil
174
+ elsif (m = line.match(METHOD_DEF_REGEX))
175
+ current_method = m[1]
176
+ end
177
+
178
+ next unless class_scope && current_method
179
+
180
+ line.scan(LAZY_LOOKUP_REGEX).flatten.each do |lazy_key|
181
+ # .greeting -> user_mailer.welcome.greeting
182
+ keys << "#{class_scope}.#{current_method}#{lazy_key}"
183
+ end
184
+ end
185
+
186
+ keys
187
+ end
188
+
189
+ # Derive the i18n scope for a class in a mailer/controller file.
190
+ # Combines any enclosing `module` nesting with the class name (which may
191
+ # itself be compact, e.g. `Admin::UserMailer`), underscores the full path,
192
+ # converts "/" to ".", and drops a trailing "_controller" for controllers
193
+ # while keeping "_mailer" for mailers.
194
+ #
195
+ # [], "PaymentOrderMailer" -> "payment_order_mailer"
196
+ # ["Admin"], "UserMailer" -> "admin.user_mailer"
197
+ # [], "Admin::UserMailer" -> "admin.user_mailer"
198
+ # [], "UsersController" -> "users"
199
+ # ["Admin"], "UsersController" -> "admin.users"
200
+ #
201
+ def ruby_class_scope(namespace, class_name)
202
+ full_path = (namespace + class_name.split("::")).map { |part| underscore(part) }.join("/")
203
+ full_path = full_path.delete_suffix("_controller") if full_path.end_with?("_controller")
204
+ full_path.tr("/", ".")
205
+ end
206
+
207
+ # Convert a CamelCase constant segment to snake_case.
208
+ # "PaymentOrderMailer" -> "payment_order_mailer"
209
+ def underscore(word)
210
+ word.gsub(/([A-Z\d]+)([A-Z][a-z])/, '\1_\2')
211
+ .gsub(/([a-z\d])([A-Z])/, '\1_\2')
212
+ .downcase
213
+ end
214
+
142
215
  # Returns the parent key if this is a pluralization key, nil otherwise
143
216
  # "items.count.one" -> "items.count"
144
217
  # "users.show.title" -> nil
@@ -55,6 +55,13 @@ module Keela
55
55
  #
56
56
  CONTROLLER_REGEX = %r{(?:ee/)?app/controllers/(.+)_controller\.rb$}.freeze
57
57
 
58
+ # A view directly under app/views/ with no intermediate directory, e.g.
59
+ # app/views/index.html.erb. Its caller directory is the root (""), so a
60
+ # bareword render "foo" from here resolves to the root-level partial "foo".
61
+ # VIEW_DIR_REGEX requires an intermediate dir and so never matches these.
62
+ #
63
+ ROOT_VIEW_REGEX = %r{(?:ee/)?app/views/[^/]+\.html\.(?:erb|haml|slim)$}.freeze
64
+
58
65
  # The render family we recognize: render, plus render_to_string and
59
66
  # render_to_body, which render partials the same way (Rails/GitLab use both
60
67
  # off-request, e.g. in mailers and background jobs). An optional opening
@@ -123,6 +130,14 @@ module Keela
123
130
  true
124
131
  end
125
132
 
133
+ # #additional_used_names collects every rendered partial name in a single
134
+ # pass, so the scanner decides by Set membership and skips the per-definition
135
+ # #used? scan. See the note on #additional_used_names.
136
+ #
137
+ def detects_all_usage?
138
+ true
139
+ end
140
+
126
141
  # Build the comment-stripped view once in the parent so workers inherit it
127
142
  # copy-on-write instead of each rebuilding a codebase-sized string.
128
143
  #
@@ -136,12 +151,41 @@ module Keela
136
151
  # demand if #prepare was not run for this source (defensive: #used? can be
137
152
  # called directly in tests, or on a different source than #prepare saw).
138
153
  #
154
+ # Retained as the Strategy contract and for direct callers (tests), but the
155
+ # scanner no longer drives detection through it: #additional_used_names now
156
+ # collects every used name in a single pass over the corpus, so the scanner
157
+ # decides via that Set instead of calling #used? per definition. See the
158
+ # note on #additional_used_names.
159
+ #
139
160
  def used?(name, source)
140
161
  usage_regex(name).match?(stripped_text_for(source))
141
162
  end
142
163
 
143
- # Resolve bareword renders against the calling file's directory so that a
144
- # short render marks the fully-qualified partial used.
164
+ # The set of partial names used anywhere in +source_files+, collected in a
165
+ # SINGLE pass over the corpus.
166
+ #
167
+ # This is the whole-run detection path for partials. The scanner checks a
168
+ # definition against this Set and never calls #used? for a name that is in
169
+ # it, so extracting every used name here (not just the bareword/positional
170
+ # ones) turns detection from O(N_partials x codebase) per-definition regex
171
+ # scans into O(codebase + N_partials): we scan each file once, collect the
172
+ # rendered names, then membership-test each definition. This is the fix for
173
+ # the partials strategy being CPU-bound on large codebases.
174
+ #
175
+ # Three render forms are collected, each the single-pass twin of a branch
176
+ # that used to be matched per-definition:
177
+ #
178
+ # 1. Explicit slashed paths (the #usage_regex cases): render "users/form",
179
+ # render partial:/layout: "users/form", render_to_string/_to_body, and
180
+ # configured render_helpers. Captured verbatim as the logical name.
181
+ # 2. Positional-underscore paths: render "shared/notes/_note" names the
182
+ # file directly -> "shared/notes/note" (strip the basename's leading _).
183
+ # 3. Barewords resolved against the caller's directory: render "form" in
184
+ # app/views/users/* or users_controller.rb -> "users/form".
185
+ #
186
+ # All three share the comment-stripped content and the same left word-anchor
187
+ # and render-method rules as #usage_regex, so the collected strings are
188
+ # exactly what the per-name regex would have matched.
145
189
  #
146
190
  def additional_used_names(source_files)
147
191
  used = Set.new
@@ -149,6 +193,14 @@ module Keela
149
193
  source_files.each do |filepath, lines|
150
194
  content = strip_erb_comments(lines.join("\n"))
151
195
 
196
+ # Explicit slashed paths: the single-pass twin of #usage_regex. A
197
+ # basename leading _ is stripped so "users/_form" and "users/form"
198
+ # agree with the positional-underscore form and with definition names.
199
+ #
200
+ content.scan(explicit_path_regex).each do |(path)|
201
+ used << path.sub(%r{(^|/)_([^/]+)$}, '\1\2')
202
+ end
203
+
152
204
  # Positional-underscore paths are absolute logical names, so they are
153
205
  # resolved regardless of the caller's directory. "shared/notes/_note"
154
206
  # -> "shared/notes/note" (strip the leading _ of the basename only).
@@ -161,7 +213,10 @@ module Keela
161
213
  next unless dir
162
214
 
163
215
  content.scan(BAREWORD_RENDER_REGEX).each do |(bareword)|
164
- used << "#{dir}/#{bareword}"
216
+ # dir is "" for a root-level view, so join without a separator to
217
+ # resolve render "foo" -> "foo" rather than "/foo".
218
+ #
219
+ used << (dir.empty? ? bareword : "#{dir}/#{bareword}")
165
220
  end
166
221
  end
167
222
 
@@ -193,17 +248,34 @@ module Keela
193
248
  text.gsub(/<%\s*-?\s*#.*?-?\s*%>/) { |match| " " * match.length }
194
249
  end
195
250
 
196
- # The directory a bareword render resolves against: the view's own dir, or
197
- # the controller-derived dir. Returns nil for files that render nothing
198
- # resolvable this way.
251
+ # The directory a bareword render resolves against: the view's own dir, the
252
+ # controller-derived dir, or "" for a root-level view (render "foo" from
253
+ # app/views/index.html.erb resolves to the root partial "foo"). Returns nil
254
+ # for files that render nothing resolvable this way.
199
255
  #
200
256
  def caller_directory(filepath)
201
257
  return Regexp.last_match(1) if filepath =~ VIEW_DIR_REGEX
202
258
  return Regexp.last_match(1) if filepath =~ CONTROLLER_REGEX
259
+ return "" if filepath =~ ROOT_VIEW_REGEX
203
260
 
204
261
  nil
205
262
  end
206
263
 
264
+ # The single-pass twin of #usage_regex: instead of matching one known name,
265
+ # CAPTURE the rendered path so one scan collects every explicitly-rendered
266
+ # partial. Same left word-anchor, same optional partial:/layout: keyword,
267
+ # same #render_method_pattern (so configured render_helpers are honored),
268
+ # as #usage_regex.
269
+ #
270
+ # The captured path must contain at least one slash: barewords (no slash)
271
+ # are resolved relative to the caller by #additional_used_names via
272
+ # BAREWORD_RENDER_REGEX, exactly as #usage_regex only ever matched slashed
273
+ # logical names. Requiring a slash here keeps the two paths disjoint.
274
+ #
275
+ def explicit_path_regex
276
+ %r{(?<!\w)#{render_method_pattern}(?:(?:partial|layout):\s*)?["']([^"'/]+(?:/[^"'/]+)+)["']}
277
+ end
278
+
207
279
  # A positional string whose basename starts with _ names a partial file
208
280
  # directly: render_to_string("shared/notes/_note") uses shared/notes/note.
209
281
  # There is no partial:/layout: keyword here, so it is handled separately
@@ -18,7 +18,11 @@ module Keela
18
18
  end
19
19
 
20
20
  def usage_regex(name)
21
- /(?<!scope :)(?<!def )#{Regexp.quote(name)}\W/
21
+ # Whitespace-anchoring the lookbehind distinguishes a `scope :name`
22
+ # definition (preceded by indentation whitespace) from `has_scope :name`
23
+ # usage (preceded by `_`), which must count as a real usage.
24
+ #
25
+ /(?<!\sscope :)(?<!def )#{Regexp.quote(name)}\W/
22
26
  end
23
27
 
24
28
  def skip_comments?
@@ -83,6 +83,20 @@ module Keela
83
83
  Set.new
84
84
  end
85
85
 
86
+ # Whether #additional_used_names already collects EVERY used name in a single
87
+ # pass, making the per-definition #used? scan redundant.
88
+ #
89
+ # Defaults to false: the scanner checks #additional_used_names first and then
90
+ # falls back to #used? per definition. A strategy that returns true here is
91
+ # declaring its #additional_used_names is complete, so the scanner skips the
92
+ # per-definition scan entirely. This turns O(N_definitions x codebase) regex
93
+ # scanning into a single O(codebase) pass plus Set membership tests, for
94
+ # strategies whose usage can be enumerated in one sweep (e.g. partials).
95
+ #
96
+ def detects_all_usage?
97
+ false
98
+ end
99
+
86
100
  private
87
101
 
88
102
  def configured_pattern
data/lib/keela/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Keela
4
- VERSION = "0.5.0"
4
+ VERSION = "0.6.0"
5
5
  end
data/lib/keela.rb CHANGED
@@ -16,6 +16,7 @@ require_relative "keela/reporter"
16
16
  require_relative "keela/baseline"
17
17
  require_relative "keela/scanner"
18
18
  require_relative "keela/exclusion_validator"
19
+ require_relative "keela/exclusion_normalizer"
19
20
  require_relative "keela/formatters"
20
21
 
21
22
  module Keela
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: keela
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Kerri Miller
@@ -13,16 +13,22 @@ dependencies:
13
13
  name: parallel
14
14
  requirement: !ruby/object:Gem::Requirement
15
15
  requirements:
16
- - - "~>"
16
+ - - ">="
17
17
  - !ruby/object:Gem::Version
18
18
  version: '1.20'
19
+ - - "<"
20
+ - !ruby/object:Gem::Version
21
+ version: '3'
19
22
  type: :runtime
20
23
  prerelease: false
21
24
  version_requirements: !ruby/object:Gem::Requirement
22
25
  requirements:
23
- - - "~>"
26
+ - - ">="
24
27
  - !ruby/object:Gem::Version
25
28
  version: '1.20'
29
+ - - "<"
30
+ - !ruby/object:Gem::Version
31
+ version: '3'
26
32
  - !ruby/object:Gem::Dependency
27
33
  name: rainbow
28
34
  requirement: !ruby/object:Gem::Requirement
@@ -81,6 +87,7 @@ files:
81
87
  - lib/keela/baseline.rb
82
88
  - lib/keela/config_file.rb
83
89
  - lib/keela/configuration.rb
90
+ - lib/keela/exclusion_normalizer.rb
84
91
  - lib/keela/exclusion_validator.rb
85
92
  - lib/keela/formatters.rb
86
93
  - lib/keela/formatters/base.rb