jitcatch 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 455e6bbed650a966fd2e33df134025b18a30e5b6e210474dbb09690ae2e5e8c2
4
- data.tar.gz: 8fe3fcfebc0f310747b00a77c06c6880269ed3e00e699cabe1ecf677a826337d
3
+ metadata.gz: 883fa4fe8954d93bdce7752d6ffa59ef88b01366cb087a7b3a63e43b3a9b06a2
4
+ data.tar.gz: 0fec80cd0350343eb4d8c29189408f282730553717ec2a183aa0dbfe25998798
5
5
  SHA512:
6
- metadata.gz: 2bffa785cd95c8115963303a28bde798ba8a4e310a5a21427e8565bb9a46edea7fbabffc822fcb734765913d559e299a1df82acf1b3dca5be82c033bc90419f4
7
- data.tar.gz: 22ea5469326808fec9204abb6083db1c5ade9cc47635e1401689b547c97a21223bc0e3cf478e01405f3c5abf8fdc22fe0277cb54b9fcad8f716e089ffc6ffb9d
6
+ metadata.gz: 53a71c2e727f5453d3ed21d310e2d8134a4275c00d64817161d8b569cc3bc81f7d4504b015ad65a610c7c39ea0e047f741bbb74dc5569f544a5aa70ae3a0ae29
7
+ data.tar.gz: c7407444b9c4b9918c6c8ca649e8d3259f827580eee016b7ce08434d04a326a79fe15f8a0d817a1ff7548da5935e3d9d24c31d0b1d42e08a9130306945e33b2a
data/README.ja.md CHANGED
@@ -75,7 +75,7 @@ jitcatch は次を出す。
75
75
  <!-- rails:output -->
76
76
  ```
77
77
  jitcatch run
78
- version 0.1.0
78
+ version 0.2.0
79
79
  base e994a60bff5a85b3bcc92960dd251d598b15835a
80
80
  head c3ef8997c50fc389a24ec4aa3f79caca2d6437b9
81
81
  reruns 2
@@ -172,7 +172,7 @@ jitcatch は次を出す。
172
172
  <!-- quickstart:output -->
173
173
  ```
174
174
  jitcatch run
175
- version 0.1.0
175
+ version 0.2.0
176
176
  base 81b05b22f14f2b5a1e580c048856a5e9d9b25b85
177
177
  head 7c53dd7485d4c994d97bddd3cb7bd5481a337b61
178
178
  reruns 2
@@ -181,6 +181,7 @@ tests:
181
181
  t01 fail_on_base ShippingTest#test_empty_order_has_no_fee shipping_test.rb
182
182
  base: fail Minitest::Assertion Expected: 0
183
183
  Actual: 5
184
+ head: pass
184
185
  t02 reported ShippingTest#test_fifty_ships_free shipping_test.rb m01
185
186
  t03 no_signal ShippingTest#test_small_order_pays_five shipping_test.rb
186
187
  counts generated=3 no_signal=1 fail_on_base=1 candidates=1 flaky=0 discarded=0 reported=1
@@ -232,8 +233,8 @@ jitcatch はそのテストを両方のリビジョンで走らせ、フレー
232
233
  Ruby 3.3 以降。
233
234
  追加の gem はない。
234
235
  `minitest` と `prism` は Ruby に付いている。
235
- `jitcatch --version`(または `-v`)は `jitcatch 0.1.0` を出し、`--help` はコマンドの一覧を出す。
236
- git から入れた gem のように jitcatch 自身の git のチェックアウトから動くときは、`jitcatch 0.1.0 (1fbcd18)` のようにコミットも出す。
236
+ `jitcatch --version`(または `-v`)は `jitcatch 0.2.0` を出し、`--help` はコマンドの一覧を出す。
237
+ git から入れた gem のように jitcatch 自身の git のチェックアウトから動くときは、`jitcatch 0.2.0 (1fbcd18)` のようにコミットも出す。
237
238
  RSpec の候補には、実行する場所に `rspec-core` が要る。
238
239
  プロジェクトの bundle に入れるか、`Gemfile` が無いプロジェクトならその Ruby にインストールしておく。
239
240
  `BASE` は親のリビジョン、`HEAD` は変更である。
@@ -447,7 +448,7 @@ bin/jitcatch assess --json run.json
447
448
  | 区分 | 意味 |
448
449
  | --- | --- |
449
450
  | `no_signal` | base でも head でも通った。 |
450
- | `fail_on_base` | 親がそのテストを満たさなかった。新しい機能だけを書いていることが多い。テキストの報告は、その行の下に base で起きたことを `base: error NoMethodError undefined method 'apply_coupon' for an instance of Shop::Cart` のように出す。 |
451
+ | `fail_on_base` | 親がそのテストを満たさなかった。新しい機能だけを書いていることが多い。テキストの報告は、その行の下に base で起きたことを `base: error NoMethodError undefined method 'apply_coupon' for an instance of Shop::Cart` のように出し、次の行に head で起きたことを出す。変更でテストが通るようになったときは `head: pass`、head が別の失敗をしたときはその失敗、同じ失敗のときは `head: fail, the same as base` となる。 |
451
452
  | `flaky` | base で通り head で落ちたあと、再実行が同じ落ち方をしなかった。捨てる。 |
452
453
  | `discarded` | base で通り head で落ちたあと、ルールが除いた。`assess:` には残る。 |
453
454
  | `reported` | base で通り head で落ち、ルールは除かなかった。この行を判断する。 |
@@ -489,6 +490,11 @@ gem の中から始まったエラーには `touches:changed_method` を付け
489
490
  テストが変更されていない呼び出し元を通してだけ届くメソッドの中では、テストが名指しする入口から最も多くの呼び出しを下ったメソッドを選ぶ。
490
491
  たとえば、リクエストスペックが 1 回下の `LoansController#officer` と 5 回下の `Borrower#approved_cents` に届くときは `Borrower#approved_cents` を選ぶ。
491
492
  `--json` の `caller_depths` がその回数を持つ。
493
+ 変更されたスクリプトを起動するテストは、そのコードが呼ぶ変更メソッドより先に、スクリプトの `exe/tool <main>` を選ぶ。
494
+ 起動するたびにそのコードは動くが、その下のメソッドはそれを選ぶコマンドのときだけ動くからである。
495
+ たとえば、使い方の文言が変わった `quint_driver bogus` のテストは `related guess: exe/quint_driver <main> (m01), ranked first of 3: exe/quint_driver <main> (m01), QuintDriver::Spec#outline (m10), QuintDriver::Targets.report (m14)` となる。
496
+ テストが待つ例外を head が上げなくなった行 (`X expected but nothing was raised.`) では、`raise` の引数だけを変えた編集 (`--json` の `edits_only_raise`) を候補から外す。
497
+ そのメソッドはどの呼び出し元よりも下にあるが、head はもうそこへ届かず、上げなくなった原因はそれを呼ぶ側にあることが多い。
492
498
  上位の二つを id でしか並べられないときは、どれも選ばない。
493
499
  その行は `related guess: none; the test reaches 7 changed methods equally and no frame of head's failure is in one: Book#apply_catalog_record (m02), LoanRules.for (m03), ...` となり、id の位置に `-` が出て、`method_id` は `null` になる。
494
500
  その行は次の一手で終わる。
@@ -504,6 +510,9 @@ id が最も小さいものだけを見るのではない。
504
510
 
505
511
  `context` は 2 つのリビジョンを diff し、ハンクを Prism でメソッドに対応づける。
506
512
  変更メソッドは `m01`、`m02` と付く。
513
+ `exe/` か `bin/` にある拡張子のない実行ファイルで、`#!` の行に `ruby` があるものも読む。
514
+ その中の `def`、クラス、モジュールの外のコードは `exe/tool <main>` という一つのメソッドになるので、使い方の文言の変更も出る。
515
+ その `tests:` はそのパスを書いたテストファイルで、それを起動するテストはそのメソッドへの言及になる。
507
516
  `Gemfile.lock` が変わったときは、`dependencies:` に gem ごとに `gem 旧 -> 新` を出す。
508
517
  変わった定数は `k01` で、その定数を読むだけで本体が変わっていないメソッドは `affected`(`via=k01`)になる。
509
518
  あるファイルから消えたメソッドを、head が別のファイル(テストファイルも含む)で同じ名前で定義していれば、それは `moved` になる。
@@ -557,6 +566,8 @@ Concern の `included do`、`prepended do`、`class_methods do` ブロックの
557
566
  これは `new` のほかにそれを走らせる呼び出しで、Active Record のモデルの `create`、`create!`、`build` と、そのクラスを作る FactoryBot のファクトリーごとの `build(:loan)` と `create(:loan)` である。
558
567
  その `tests:` には、そのどれかか `Owner.new` を呼ぶテストファイルが、呼んでいるもの(`spec/models/loan_spec.rb via create(:loan)`)と一緒に出る。
559
568
  テストが `initialize` をその名前で呼ぶことはないので、コンストラクタには grep の結果を出さない。
569
+ `attr_reader` や `Data.define` のメンバー(`Targets::Hit#code`)の名前は短い語が多いので、grep の結果は、そのクラスかそれを囲む名前空間の名前(`Hit` か `Targets`)も書いたファイルに限る。
570
+ `test/fixture/depot/app/` のように、`test/` や `spec/` の下に置いたフィクスチャのプロジェクトの `app/`、`lib/`、`config/`、`db/` にあるコードはテストではないので、`tests:` に出さない。
560
571
  ファクトリーは `spec/factories` と `test/factories` から、読み込まずにソースとして読む。
561
572
  クラスはまず `class:`(定数、文字列、シンボル)から、次に親(`parent:` か、入れ子の外側のファクトリー)から、最後に名前を camelize したものから決まり、`aliases:` の別名も同じクラスを作る。
562
573
  ファクトリーは、その関連が作るクラスにも数える。
@@ -918,6 +929,7 @@ readers: 1 for 1 changed value
918
929
  メソッドの `callsites:` にすでにある行と、定数の `read_by:` にすでにあるメソッドの中のただの読み出しは出さない。
919
930
  `not seen:` は、見えないものを挙げる。
920
931
  レコードのモデルは変数名かメソッド名(`@branch`、`home_branch`、`branches`)で決めるので、`x = Branch.first; link_to x` は出ない。
932
+ `app/`、`config/routes.rb`、`config/application.rb` のどれもない履歴 (gem など) では、モデルもルートもテンプレートもないので、どの Ruby にもある二つだけを挙げる。
921
933
 
922
934
  ## インストール
923
935
 
@@ -964,6 +976,10 @@ CLI を使わない場合は、`skills/jitcatch` をエージェントの skills
964
976
  そうした呼び出しは、送る名前が変更メソッドの名前か、その `reachable_from` にあるメソッドの名前であれば、その変更メソッドの `dynamic_callers:` に `Emit.looked_up (render)` のように出る。
965
977
  ただし名前が `Controller` で終わるクラスのインスタンスメソッドの名前は除く。
966
978
  コントローラーのアクションはルーターが呼び、アプリのコードがその名前を送ることはないので、`ThingsController#update` が変更に届いても `cipher.update` は出ない。
979
+ ブロックだけを持つローカル変数への呼び出しも出ない。
980
+ `&block` の引数と、メソッドの中のすべての代入が `-> {}`、`lambda {}`、`proc {}`、`Proc.new {}` であるローカル変数がそれにあたる。
981
+ その `call` や `.()` はそのブロックを動かすだけで、リポジトリのメソッドを呼ぶことはない。
982
+ `def sync(job)` の中の `job.call` のような、引数への `call` は出る。
967
983
  クラスが定義していないメソッドを `self` に呼ぶと、その名前を持つ索引上ただ一つのメソッドへ向かう。
968
984
  モジュールやスーパークラスのメソッドのためである。
969
985
  インスタンスメソッドからはインスタンスメソッドだけが対象になる。
@@ -1013,6 +1029,7 @@ CLI を使わない場合は、`skills/jitcatch` をエージェントの skills
1013
1029
  エラークラスは、メッセージを作るために `to_s` を上書きするからである。
1014
1030
  `"exe/tool"` や `"../bin/x"` のような相対パスの形の文字列が head のファイルを指すときは、テストが別のプロセスで起動するスクリプトとみなし、そのファイルの中の名前も言及になる。
1015
1031
  パスは、リポジトリの根から、候補のディレクトリから、または候補の隣で `File.expand_path` が読むとおりに読む。
1032
+ `assert_includes error.backtrace.first, "lib/tool/model.rb"` や `include("lib/x.rb")` のように、アサーションやマッチャーが直接比べる文字列は期待するテキストであり、スクリプトを指さない。
1016
1033
  別のファイルのヘルパーは読まない。
1017
1034
  - `--workers` を渡さなければ、ファイルは 1 つずつ実行する。
1018
1035
  1 つのファイルの base 側と head 側は、それぞれの worktree で同時に走る。
data/README.md CHANGED
@@ -63,7 +63,7 @@ The last commit asks a senior officer from 10000 cents, where it used to ask onl
63
63
  <!-- rails:output -->
64
64
  ```
65
65
  jitcatch run
66
- version 0.1.0
66
+ version 0.2.0
67
67
  base e994a60bff5a85b3bcc92960dd251d598b15835a
68
68
  head c3ef8997c50fc389a24ec4aa3f79caca2d6437b9
69
69
  reruns 2
@@ -145,7 +145,7 @@ jitcatch prints this. The two commit ids are your own.
145
145
  <!-- quickstart:output -->
146
146
  ```
147
147
  jitcatch run
148
- version 0.1.0
148
+ version 0.2.0
149
149
  base 81b05b22f14f2b5a1e580c048856a5e9d9b25b85
150
150
  head 7c53dd7485d4c994d97bddd3cb7bd5481a337b61
151
151
  reruns 2
@@ -154,6 +154,7 @@ tests:
154
154
  t01 fail_on_base ShippingTest#test_empty_order_has_no_fee shipping_test.rb
155
155
  base: fail Minitest::Assertion Expected: 0
156
156
  Actual: 5
157
+ head: pass
157
158
  t02 reported ShippingTest#test_fifty_ships_free shipping_test.rb m01
158
159
  t03 no_signal ShippingTest#test_small_order_pays_five shipping_test.rb
159
160
  counts generated=3 no_signal=1 fail_on_base=1 candidates=1 flaky=0 discarded=0 reported=1
@@ -192,7 +193,7 @@ A coding agent reviewing a change writes tests for that diff. A test that only d
192
193
 
193
194
  The method follows Meta's catching JiT tests ([Harden and Catch](https://arxiv.org/abs/2504.16472); [Just-in-Time Catching Test Generation at Meta](https://arxiv.org/abs/2601.22832)).
194
195
 
195
- Ruby 3.3 or newer. No gems. `minitest` and `prism` ship with Ruby. `jitcatch --version` (or `-v`) prints `jitcatch 0.1.0`, followed by the commit, as in `jitcatch 0.1.0 (1fbcd18)`, when jitcatch runs from its own git checkout, as a gem installed from git does, and `--help` prints the commands. RSpec candidates need `rspec-core` where they run: in the project's bundle, or installed for that Ruby when the project has no `Gemfile`. `BASE` is the parent revision and `HEAD` is the change. Each must resolve to a commit. Uncommitted files are not in the diff.
196
+ Ruby 3.3 or newer. No gems. `minitest` and `prism` ship with Ruby. `jitcatch --version` (or `-v`) prints `jitcatch 0.2.0`, followed by the commit, as in `jitcatch 0.2.0 (1fbcd18)`, when jitcatch runs from its own git checkout, as a gem installed from git does, and `--help` prints the commands. RSpec candidates need `rspec-core` where they run: in the project's bundle, or installed for that Ruby when the project has no `Gemfile`. `BASE` is the parent revision and `HEAD` is the change. Each must resolve to a commit. Uncommitted files are not in the diff.
196
197
 
197
198
  You write the candidate files in a scratch directory. A Minitest file starts with `require "minitest/autorun"`, then requires the library the way the repository loads it. An RSpec file is named `*_spec.rb` and starts with `require "spec_helper"` or `require "rails_helper"`, as the project's own specs do. jitcatch runs those files. You judge each `reported` row against the intent and the diff. The intent in the brief is the commit messages from `BASE` to `HEAD`, without the trailer lines git parses at the end of each message (such as `Co-Authored-By:`). likely-regression means the behavior changed and the intent does not ask for that change. intended means the intent asks for it. unclear means the intent is not enough to tell.
198
199
 
@@ -344,18 +345,18 @@ bin/jitcatch assess --json run.json
344
345
  | Bucket | Meaning |
345
346
  | --- | --- |
346
347
  | `no_signal` | Passed on base and on head. |
347
- | `fail_on_base` | The parent did not satisfy the test. Often the test only describes a new feature. The text report prints what base did under the row, as `base: error NoMethodError undefined method 'apply_coupon' for an instance of Shop::Cart`. |
348
+ | `fail_on_base` | The parent did not satisfy the test. Often the test only describes a new feature. The text report prints what base did under the row, as `base: error NoMethodError undefined method 'apply_coupon' for an instance of Shop::Cart`, and what head did on the next line: `head: pass` when the change makes the test pass, head's own failure, or `head: fail, the same as base`. |
348
349
  | `flaky` | Passed on base, failed on head, then a rerun did not fail the same way. Dropped. |
349
350
  | `discarded` | Passed on base and failed on head, then a rule removed it. Still listed under `assess:`. |
350
351
  | `reported` | Passed on base and failed on head, and no rule removed it. Judge this row. |
351
352
 
352
353
  `generated` counts test methods. `candidates` counts stable pass-on-base / fail-on-head tests before the discard rules. `reported` is what remains.
353
354
 
354
- A tag is a hint, not the judgment. `touches:changed_method` means the test names a method the diff changed. `Owner.new` names the `initialize` of that owner alone (`Mismatch.new` is not `Spec#initialize`), and a changed constructor is the related method only when the test names no other changed method and goes through no changed method's caller. `touches:changed_constant` means it names a constant the diff changed. `touches:changed_dependency` means the error came from inside a gem whose locked version changed. An error that starts inside any gem does not get `touches:changed_method`. `touches:unchanged_caller` means the test goes through an entry point the diff did not edit, and that entry point reaches a changed method. `discard:signature_change`, `discard:load_error`, `discard:timeout`, `discard:no_database`, and `discard:no_table` name the rule that removed a candidate. `discard:signature_change` applies only when the test itself makes the call that a removed method or a new signature broke, or calls `new` of a `Data` or `Struct` class whose members the diff changed, as `Record.new(...)` failing with `missing keyword: :media`. Such a row is not a finding, and its next line says what changed: `candidate is out of date with head: CatalogService::Record (k07) changed its definition; update the call at ../cand/stale_spec.rb:5 to head's code, or drop the test`. `out_of_date` in `--json` holds its `id`, `name`, `change` (`removed`, `parameters`, or `definition`), and `call`. When unchanged code makes it, such as `class B < A` still calling the `A#helper` that head removed, the row is reported with `caller:unchanged:lib/b.rb:5`, the line of that call (`caller:changed:` for a line the diff wrote or a line in a changed method; `changed_lines` in `--json` lists the head lines each file's hunks wrote). The related id (`m05`, or `-` when there is none) is the downstream changed method when several changed methods sit on the path. When the failure's backtrace runs through changed methods, it is the innermost of them, and that pick is not a guess: the outer frames only called it. A frame's line is read as a line of head, so it never hits the method that held that line on base. `related:guess` means the evidence that picked the id points at more than one changed method, as when the test names only an unchanged caller that reaches several; the row's next line names them all with their ids, as `related guess: Shop::Cart#volume_discount_cents (m05), ranked first of 3: Shop::Cart#coupon_discount_cents (m02), Shop::Cart#taxable_cents (m04), Shop::Cart#volume_discount_cents (m05)`, and `method_ids` holds them in `--json`. The pick prefers a method that existed on base over one the diff added, except over an edit that only adds a call to an added method (`forwards_to_added` in `--json`), as `net - volume` becoming `net - volume - coupon(n)` does: the new value comes from the added method. Among methods the test reaches only through an unchanged caller, the pick is the one the most calls below the entry point the test names, so a request spec that reaches `LoansController#officer` one call down and `Borrower#approved_cents` five calls down picks `Borrower#approved_cents`; `caller_depths` in `--json` holds those counts. When only the id would order the best two, the row names no pick: the line reads `related guess: none; the test reaches 7 changed methods equally and no frame of head's failure is in one: Book#apply_catalog_record (m02), LoanRules.for (m03), ...`, and it ends with a next step: find what makes head's value in those methods, or in a changed file that holds no method, such as a view, from the `git diff --stat BASE HEAD` it prints. The row shows `-` in place of the id, and `method_id` is `null`. Such a row gets `touches:changed_method` when the test names any method of `method_ids`, not only the one with the lowest id. `message_varies` means head failed the same way on each rerun with a message that moved, and base passed on each rerun; compare a field the test controls rather than the whole record. `related:dynamic` means no changed method is related, but the test calls a method from `dynamic_callers:`; the row's next line says so, as in `related: none, Emit.looked_up sends render to a receiver the source does not name`.
355
+ A tag is a hint, not the judgment. `touches:changed_method` means the test names a method the diff changed. `Owner.new` names the `initialize` of that owner alone (`Mismatch.new` is not `Spec#initialize`), and a changed constructor is the related method only when the test names no other changed method and goes through no changed method's caller. `touches:changed_constant` means it names a constant the diff changed. `touches:changed_dependency` means the error came from inside a gem whose locked version changed. An error that starts inside any gem does not get `touches:changed_method`. `touches:unchanged_caller` means the test goes through an entry point the diff did not edit, and that entry point reaches a changed method. `discard:signature_change`, `discard:load_error`, `discard:timeout`, `discard:no_database`, and `discard:no_table` name the rule that removed a candidate. `discard:signature_change` applies only when the test itself makes the call that a removed method or a new signature broke, or calls `new` of a `Data` or `Struct` class whose members the diff changed, as `Record.new(...)` failing with `missing keyword: :media`. Such a row is not a finding, and its next line says what changed: `candidate is out of date with head: CatalogService::Record (k07) changed its definition; update the call at ../cand/stale_spec.rb:5 to head's code, or drop the test`. `out_of_date` in `--json` holds its `id`, `name`, `change` (`removed`, `parameters`, or `definition`), and `call`. When unchanged code makes it, such as `class B < A` still calling the `A#helper` that head removed, the row is reported with `caller:unchanged:lib/b.rb:5`, the line of that call (`caller:changed:` for a line the diff wrote or a line in a changed method; `changed_lines` in `--json` lists the head lines each file's hunks wrote). The related id (`m05`, or `-` when there is none) is the downstream changed method when several changed methods sit on the path. When the failure's backtrace runs through changed methods, it is the innermost of them, and that pick is not a guess: the outer frames only called it. A frame's line is read as a line of head, so it never hits the method that held that line on base. `related:guess` means the evidence that picked the id points at more than one changed method, as when the test names only an unchanged caller that reaches several; the row's next line names them all with their ids, as `related guess: Shop::Cart#volume_discount_cents (m05), ranked first of 3: Shop::Cart#coupon_discount_cents (m02), Shop::Cart#taxable_cents (m04), Shop::Cart#volume_discount_cents (m05)`, and `method_ids` holds them in `--json`. The pick prefers a method that existed on base over one the diff added, except over an edit that only adds a call to an added method (`forwards_to_added` in `--json`), as `net - volume` becoming `net - volume - coupon(n)` does: the new value comes from the added method. Among methods the test reaches only through an unchanged caller, the pick is the one the most calls below the entry point the test names, so a request spec that reaches `LoansController#officer` one call down and `Borrower#approved_cents` five calls down picks `Borrower#approved_cents`; `caller_depths` in `--json` holds those counts. A test that starts a changed script picks the script's `exe/tool <main>` ahead of the changed methods that code calls: every start runs it, and each method below it runs only for the command that picks it, so a test of `quint_driver bogus` whose usage text changed reads `related guess: exe/quint_driver <main> (m01), ranked first of 3: exe/quint_driver <main> (m01), QuintDriver::Spec#outline (m10), QuintDriver::Targets.report (m14)`. On a row where head stopped raising the error the test expects (`X expected but nothing was raised.`), the pool leaves out an edit that only changes the arguments of a `raise` (`edits_only_raise` in `--json`): that method sits below every caller, but head no longer reaches it, and the cause is usually in the code that called it. When only the id would order the best two, the row names no pick: the line reads `related guess: none; the test reaches 7 changed methods equally and no frame of head's failure is in one: Book#apply_catalog_record (m02), LoanRules.for (m03), ...`, and it ends with a next step: find what makes head's value in those methods, or in a changed file that holds no method, such as a view, from the `git diff --stat BASE HEAD` it prints. The row shows `-` in place of the id, and `method_id` is `null`. Such a row gets `touches:changed_method` when the test names any method of `method_ids`, not only the one with the lowest id. `message_varies` means head failed the same way on each rerun with a message that moved, and base passed on each rerun; compare a field the test controls rather than the whole record. `related:dynamic` means no changed method is related, but the test calls a method from `dynamic_callers:`; the row's next line says so, as in `related: none, Emit.looked_up sends render to a receiver the source does not name`.
355
356
 
356
357
  ## How it filters
357
358
 
358
- `context` diffs the two revisions and maps hunks onto methods with Prism. Changed methods are `m01`, `m02`, and so on. When `Gemfile.lock` changed, a `dependencies:` block lists each gem as `gem old -> new`. A changed constant is `k01`. A class or module that only one side defines anywhere is a constant too, `added` or `removed`, shown by its header line. A method that did not itself change but reads that constant is `affected` (`via=k01`). A method that leaves one file while head defines the same name in another file, test files included, is `moved`, shown at its new place with `moved_from=` naming the old file; when several files define it, one the diff touched comes first. A constant that moves the same way with the same value is `moved` with `moved_from=` too, and a method that reads it is not `affected`. A constant that moved and took a new value is `modified`, with `moved_from=`. A reader or writer from `attr_reader`, `attr_writer`, `attr_accessor`, or a `Struct.new` or `Data.define` member list is a method, with the one-name form (`attr_reader :seed`) as its source, so a name dropped from such a line is `removed` and the names left on it do not change. A list given as `*FIELDS` counts when the same file set `FIELDS` to a list of symbols before it, as in `FIELDS = %i[x y].freeze`; each member then takes the line of its symbol in that list. Three Active Record macros of a class body define methods too. `scope :recent, -> { ... }` (or `scope(:recent) { ... }`) is the class method `Post.recent`, with the whole statement as its source, and the calls in its body are calls from `Post.recent`. `delegate :name, to: :author` is `Post#name` (`Post#author_name` with `prefix: true`), with the one-name form and the options as its source, so a change to `to:` changes every name on the line. `enum :status, { draft: 0, live: 1 }` gives each value `Post#draft?`, `Post#draft!`, `Post.draft`, and `Post.not_draft`, with `prefix:`, `suffix:`, `scopes: false`, and `instance_methods: false` applied, and its source is that one value (`enum :status, draft: 0`), so changing one value changes only its four methods. The `enum status: {...}` form that Rails 8 removed is not read. A Grape endpoint is a method too: in a class whose superclass is `Grape::API` (or a name that ends in `API` or `Api`, as an app's own base class does), `post "/refresh" do ... end` inside `resource :things do` is `Things#POST /things/refresh`. Its path joins the class's `prefix` and the segments of `resource`, `resources`, `namespace`, `group`, `segment`, and `route_param :id` (`:id`). Its source is the block, with the `desc` and `params` statements just before it, and the calls in the block are calls from the endpoint, so a method it calls shows it under `reachable_from`. A request a test sends, as `post "/api/things/7/refresh"`, names an endpoint with the same verb whose segments end the path, with a `:param` matching any one segment and any segments before them taken as a mount point. A path that interpolates a string constant the file assigns, in the API class (`namespace VERSION`, `post "#{ACTION}/now"`) or in the test (`post "#{API_ROOT}/things"`), reads as that string. In a test, any other interpolation matches any one segment (`"/things/#{thing.id}/refresh"`), or any mount point when the path starts with it (`"#{api_root}/things"`). The `desc` and `params` before a `route_param` or another path block belong to the first endpoint inside it. Such a test is listed under `tests:` and `entry_point_tests:`, and a candidate that sends it gets `touches:changed_method`. An endpoint whose path the API class computes any other way keeps that part as its source in braces, as `Books#GET /books/{Settings.legacy}/list`, matches no request, and gets a `warning:` that says so with its file and line. In a Rails app, `context` and `run` read the routes `config/routes.rb` draws (`resources`, `resource`, `namespace`, `scope` with `module:`, `as:`, or `controller:`, `member`, `collection`, `on:`, `root`, and a verb with `to:`, `=>`, or `controller:` and `action:`), so a request a candidate sends to a path helper (`post things_path`, `patch thing_url(thing)`) or to a path names the controller action that serves it, as `ThingsController#create`. Helpers and controllers take the names Rails gives them with Active Support's inflections and the literal `irregular`, `uncountable`, `singular`, `plural`, and `acronym` calls in `config/initializers/inflections.rb`: `resources :people` draws `person_path`, `resources :news` draws `news_index_path` and `news_path`, and `inflect.acronym "API"` makes `resources :api_keys` reach `APIKeysController`. A rule that file builds at run time is not read. A changed controller action that the routes draw lists under `tests:` each test file whose requests reach it, and each controller test that names the controller and calls the action as `post :create`, with what it sends (`spec/requests/things_spec.rb via POST things_path`); a grep of the action's name matched `create(:thing)` and missed `post things_path`. An action no route draws keeps the grep. A helper drawn any other way, as by a `draw` file, a loop, or a concern, is not read: `run` prints `warning: 1 candidate request names a path helper config/routes.rb does not draw (report_path), so it is tied to no controller action.` A `def` or reader inside a conditional of the class body (`attr_reader :x if cond`) counts. The branches read as if each ran in order, so a bare `private` in a branch carries past the conditional. A definition in a Concern's `included do`, `prepended do`, or `class_methods do` block counts too. Each method block has the old and new source, call sites in the library, the entry points that reach it (`reachable_from`), and existing tests that mention it. A changed `initialize` also shows `built_by:`, the calls besides `new` that run it: `create`, `create!`, and `build` on an Active Record model, and `build(:loan)` and `create(:loan)` for each FactoryBot factory that builds the class. Its `tests:` line lists each test file that calls one of those or `Owner.new`, with the calls it makes (`spec/models/loan_spec.rb via create(:loan)`). No test calls `initialize` by that name, so a constructor gets no grep hits. Factories are read from `spec/factories` and `test/factories` without loading them: the class comes from `class:` (a constant, a string, or a symbol), then from the parent (`parent:`, or the factory it nests in), then from the camelized name, and an alias in `aliases:` builds the same class. A factory also counts for each class its associations build, through their own associations too: a bare `borrower` that names a factory, `association :borrower` (or `factory: :user`), or an attribute block that calls `association`, in the factory or one it inherits from, so `create(:loan)` saves a `Borrower` too. An association in a trait counts only for a call that names the trait: a trait `:cosigned` of `:loan` that builds a cosigner adds `create(:loan, :cosigned)` and `build(:loan, :cosigned)`, not `create(:loan)`, and a test that writes `create(:loan, :cosigned, amount_cents: 5)` mentions both `create(:loan)` and `create(:loan, :cosigned)`. A factory takes the traits of its parent and of the factory it nests in. A trait defined outside a factory, and a trait that includes another trait, are not followed. A factory whose class the source does not settle is not followed and is listed under `factories_skipped:` with the reason: a `class:` computed at run time, a parent that is not a factory read there, or a class that no file defines, which an inflection the app configures can cause (`ApiKey` against `APIKey`). A factory that builds a Ruby value rather than a class of the app (`class: Hash`, as for the shape of an API response, or `Array`, `Data`, `Object`, `OpenStruct`, `Set`, `String`, `Struct`) is left out and not listed. A method that an Active Record model's class body names in a callback or a validation (`after_save :count_save`, `validate :check`, or an `if:` or `unless:` of either, as in `validates :amount, presence: true, if: :checked?`) shows `run_by:`, the calls that run it by Rails's rules, followed by the lines that name it. A save callback lists `Loan.create`, `Loan.create!`, `Loan#save`, `Loan#save!`, `Loan#update`, and `Loan#update!` (a create callback leaves out the updates, an update callback the creates), with `create(:loan)` and `build(:loan)` for each factory of the class. A validation adds `Loan#valid?`, `Loan#invalid?`, `Loan#validate`, and `Loan#validate!`. An `on:` on a validation or on a commit or rollback callback keeps only the calls of that context, and the line shows it: `before_validation :mark_pending, on: :create` lists no update, so a method that calls `update!` does not reach it, and `on: :update` lists no create and no factory. A context of the app's own (`on: :publish`) runs only from a call that takes one, so it lists `Loan#save`, `Loan#save!`, and the four validation calls. An `on:` that is not written as symbols narrows nothing. A destroy callback lists `Loan#destroy` and `Loan#destroy!`. A method that such a callback reaches (`reachable_from`) gets the same line. A method outside the tests that makes one of those calls, as the source writes it (`Loan.create!` in `LoanDesk.open`, or `update!` inside a method of `Loan`), joins `reachable_from` with its own callers, so a test that calls `LoanDesk.open` names the callback. A call on a local or an instance variable the source does not type, such as a parameter, is read as a call on the model Rails names it after, so `loan.save!` in `Teller#renew(loan)` reaches a save callback of `Loan`, and `record.save!` reaches none. Its `tests:` lists each test file that makes one of those calls (`spec/models/loan_spec.rb via Loan.create!`), next to the grep hits, and `run` counts the call as a mention of the method. `Loan#save` counts when the test's source settles the record's class (`loan = Loan.new`). `build(:loan)` does not save, so it counts only in a test that also makes one of the record calls the callback lists, as in `build(:loan).save!` or `expect(build(:loan)).to be_valid`, where `be_valid` calls `valid?`. That call's receiver is not checked, since a local the source does not type holds the built record, so a test that saves another record next to `build(:loan)` counts too. A test that only builds gets no `tests:` hit and no mention in `run`. A callback line in a Concern, one a parent class declares, and a call on a subclass are not followed. When no test mentions it, `entry_point_tests` lists the test files that name the class of a public entry point in `reachable_from` and call a method of that entry point's name. `--coverage` uses the line lists in the JSON file instead of the grep. The brief ends with one `entrypoints <Const>:` section for each class or module that holds a changed method. It lists that class's public methods on head (on base when head has none), changed ones first with their change kind, then the rest as `unchanged`, at most 12, with the ones that reach a changed method first and `(+N more)` for the methods left out. A class that defines `initialize` on either side shows `Owner.new` with the role of `Owner#initialize` in its place, unless the class makes `new` private (`private_class_method :new`, or `private :new` inside `class << self`). These are the methods a test can call, and `run` counts their names as mentions.
359
+ `context` diffs the two revisions and maps hunks onto methods with Prism. Changed methods are `m01`, `m02`, and so on. An executable under `exe/` or `bin/` with no suffix and `ruby` on its `#!` line is read too: its code outside a `def`, a class, or a module is one method, `exe/tool <main>`, so a change to its usage text is listed, its `tests:` are the test files that write its path, and a test that starts it mentions it. When `Gemfile.lock` changed, a `dependencies:` block lists each gem as `gem old -> new`. A changed constant is `k01`. A class or module that only one side defines anywhere is a constant too, `added` or `removed`, shown by its header line. A method that did not itself change but reads that constant is `affected` (`via=k01`). A method that leaves one file while head defines the same name in another file, test files included, is `moved`, shown at its new place with `moved_from=` naming the old file; when several files define it, one the diff touched comes first. A constant that moves the same way with the same value is `moved` with `moved_from=` too, and a method that reads it is not `affected`. A constant that moved and took a new value is `modified`, with `moved_from=`. A reader or writer from `attr_reader`, `attr_writer`, `attr_accessor`, or a `Struct.new` or `Data.define` member list is a method, with the one-name form (`attr_reader :seed`) as its source, so a name dropped from such a line is `removed` and the names left on it do not change. A list given as `*FIELDS` counts when the same file set `FIELDS` to a list of symbols before it, as in `FIELDS = %i[x y].freeze`; each member then takes the line of its symbol in that list. Three Active Record macros of a class body define methods too. `scope :recent, -> { ... }` (or `scope(:recent) { ... }`) is the class method `Post.recent`, with the whole statement as its source, and the calls in its body are calls from `Post.recent`. `delegate :name, to: :author` is `Post#name` (`Post#author_name` with `prefix: true`), with the one-name form and the options as its source, so a change to `to:` changes every name on the line. `enum :status, { draft: 0, live: 1 }` gives each value `Post#draft?`, `Post#draft!`, `Post.draft`, and `Post.not_draft`, with `prefix:`, `suffix:`, `scopes: false`, and `instance_methods: false` applied, and its source is that one value (`enum :status, draft: 0`), so changing one value changes only its four methods. The `enum status: {...}` form that Rails 8 removed is not read. A Grape endpoint is a method too: in a class whose superclass is `Grape::API` (or a name that ends in `API` or `Api`, as an app's own base class does), `post "/refresh" do ... end` inside `resource :things do` is `Things#POST /things/refresh`. Its path joins the class's `prefix` and the segments of `resource`, `resources`, `namespace`, `group`, `segment`, and `route_param :id` (`:id`). Its source is the block, with the `desc` and `params` statements just before it, and the calls in the block are calls from the endpoint, so a method it calls shows it under `reachable_from`. A request a test sends, as `post "/api/things/7/refresh"`, names an endpoint with the same verb whose segments end the path, with a `:param` matching any one segment and any segments before them taken as a mount point. A path that interpolates a string constant the file assigns, in the API class (`namespace VERSION`, `post "#{ACTION}/now"`) or in the test (`post "#{API_ROOT}/things"`), reads as that string. In a test, any other interpolation matches any one segment (`"/things/#{thing.id}/refresh"`), or any mount point when the path starts with it (`"#{api_root}/things"`). The `desc` and `params` before a `route_param` or another path block belong to the first endpoint inside it. Such a test is listed under `tests:` and `entry_point_tests:`, and a candidate that sends it gets `touches:changed_method`. An endpoint whose path the API class computes any other way keeps that part as its source in braces, as `Books#GET /books/{Settings.legacy}/list`, matches no request, and gets a `warning:` that says so with its file and line. In a Rails app, `context` and `run` read the routes `config/routes.rb` draws (`resources`, `resource`, `namespace`, `scope` with `module:`, `as:`, or `controller:`, `member`, `collection`, `on:`, `root`, and a verb with `to:`, `=>`, or `controller:` and `action:`), so a request a candidate sends to a path helper (`post things_path`, `patch thing_url(thing)`) or to a path names the controller action that serves it, as `ThingsController#create`. Helpers and controllers take the names Rails gives them with Active Support's inflections and the literal `irregular`, `uncountable`, `singular`, `plural`, and `acronym` calls in `config/initializers/inflections.rb`: `resources :people` draws `person_path`, `resources :news` draws `news_index_path` and `news_path`, and `inflect.acronym "API"` makes `resources :api_keys` reach `APIKeysController`. A rule that file builds at run time is not read. A changed controller action that the routes draw lists under `tests:` each test file whose requests reach it, and each controller test that names the controller and calls the action as `post :create`, with what it sends (`spec/requests/things_spec.rb via POST things_path`); a grep of the action's name matched `create(:thing)` and missed `post things_path`. An action no route draws keeps the grep. A helper drawn any other way, as by a `draw` file, a loop, or a concern, is not read: `run` prints `warning: 1 candidate request names a path helper config/routes.rb does not draw (report_path), so it is tied to no controller action.` A `def` or reader inside a conditional of the class body (`attr_reader :x if cond`) counts. The branches read as if each ran in order, so a bare `private` in a branch carries past the conditional. A definition in a Concern's `included do`, `prepended do`, or `class_methods do` block counts too. Each method block has the old and new source, call sites in the library, the entry points that reach it (`reachable_from`), and existing tests that mention it. A changed `initialize` also shows `built_by:`, the calls besides `new` that run it: `create`, `create!`, and `build` on an Active Record model, and `build(:loan)` and `create(:loan)` for each FactoryBot factory that builds the class. Its `tests:` line lists each test file that calls one of those or `Owner.new`, with the calls it makes (`spec/models/loan_spec.rb via create(:loan)`). No test calls `initialize` by that name, so a constructor gets no grep hits. A reader or member from `attr_reader` or `Data.define` (`Targets::Hit#code`) is most often a short word, so its grep hits are only the files that also name its class or the namespace around it (`Hit` or `Targets`). The code of a fixture project under `test/` or `spec/`, in its `app/`, `lib/`, `config/`, or `db/` (`test/fixture/depot/app/`), tests nothing and is not listed under `tests:`. Factories are read from `spec/factories` and `test/factories` without loading them: the class comes from `class:` (a constant, a string, or a symbol), then from the parent (`parent:`, or the factory it nests in), then from the camelized name, and an alias in `aliases:` builds the same class. A factory also counts for each class its associations build, through their own associations too: a bare `borrower` that names a factory, `association :borrower` (or `factory: :user`), or an attribute block that calls `association`, in the factory or one it inherits from, so `create(:loan)` saves a `Borrower` too. An association in a trait counts only for a call that names the trait: a trait `:cosigned` of `:loan` that builds a cosigner adds `create(:loan, :cosigned)` and `build(:loan, :cosigned)`, not `create(:loan)`, and a test that writes `create(:loan, :cosigned, amount_cents: 5)` mentions both `create(:loan)` and `create(:loan, :cosigned)`. A factory takes the traits of its parent and of the factory it nests in. A trait defined outside a factory, and a trait that includes another trait, are not followed. A factory whose class the source does not settle is not followed and is listed under `factories_skipped:` with the reason: a `class:` computed at run time, a parent that is not a factory read there, or a class that no file defines, which an inflection the app configures can cause (`ApiKey` against `APIKey`). A factory that builds a Ruby value rather than a class of the app (`class: Hash`, as for the shape of an API response, or `Array`, `Data`, `Object`, `OpenStruct`, `Set`, `String`, `Struct`) is left out and not listed. A method that an Active Record model's class body names in a callback or a validation (`after_save :count_save`, `validate :check`, or an `if:` or `unless:` of either, as in `validates :amount, presence: true, if: :checked?`) shows `run_by:`, the calls that run it by Rails's rules, followed by the lines that name it. A save callback lists `Loan.create`, `Loan.create!`, `Loan#save`, `Loan#save!`, `Loan#update`, and `Loan#update!` (a create callback leaves out the updates, an update callback the creates), with `create(:loan)` and `build(:loan)` for each factory of the class. A validation adds `Loan#valid?`, `Loan#invalid?`, `Loan#validate`, and `Loan#validate!`. An `on:` on a validation or on a commit or rollback callback keeps only the calls of that context, and the line shows it: `before_validation :mark_pending, on: :create` lists no update, so a method that calls `update!` does not reach it, and `on: :update` lists no create and no factory. A context of the app's own (`on: :publish`) runs only from a call that takes one, so it lists `Loan#save`, `Loan#save!`, and the four validation calls. An `on:` that is not written as symbols narrows nothing. A destroy callback lists `Loan#destroy` and `Loan#destroy!`. A method that such a callback reaches (`reachable_from`) gets the same line. A method outside the tests that makes one of those calls, as the source writes it (`Loan.create!` in `LoanDesk.open`, or `update!` inside a method of `Loan`), joins `reachable_from` with its own callers, so a test that calls `LoanDesk.open` names the callback. A call on a local or an instance variable the source does not type, such as a parameter, is read as a call on the model Rails names it after, so `loan.save!` in `Teller#renew(loan)` reaches a save callback of `Loan`, and `record.save!` reaches none. Its `tests:` lists each test file that makes one of those calls (`spec/models/loan_spec.rb via Loan.create!`), next to the grep hits, and `run` counts the call as a mention of the method. `Loan#save` counts when the test's source settles the record's class (`loan = Loan.new`). `build(:loan)` does not save, so it counts only in a test that also makes one of the record calls the callback lists, as in `build(:loan).save!` or `expect(build(:loan)).to be_valid`, where `be_valid` calls `valid?`. That call's receiver is not checked, since a local the source does not type holds the built record, so a test that saves another record next to `build(:loan)` counts too. A test that only builds gets no `tests:` hit and no mention in `run`. A callback line in a Concern, one a parent class declares, and a call on a subclass are not followed. When no test mentions it, `entry_point_tests` lists the test files that name the class of a public entry point in `reachable_from` and call a method of that entry point's name. `--coverage` uses the line lists in the JSON file instead of the grep. The brief ends with one `entrypoints <Const>:` section for each class or module that holds a changed method. It lists that class's public methods on head (on base when head has none), changed ones first with their change kind, then the rest as `unchanged`, at most 12, with the ones that reach a changed method first and `(+N more)` for the methods left out. A class that defines `initialize` on either side shows `Owner.new` with the role of `Owner#initialize` in its place, unless the class makes `new` private (`private_class_method :new`, or `private :new` inside `class << self`). These are the methods a test can call, and `run` counts their names as mentions.
359
360
 
360
361
  `run` adds a git worktree for each revision, copies the scratch directory to `.jitcatch/` in that worktree, and runs each file in its own process:
361
362
 
@@ -470,7 +471,7 @@ readers: 1 for 1 changed value
470
471
  not seen: receivers named unlike their model (`x = Branch.first; link_to x`); dispatch names built at run time from data outside the source (a DB column, a request param); routes drawn by `draw`, loops, or concerns; `to_param` reached through a gem's helper; serializers, jbuilder, and other template languages than ERB; callbacks and validations added by concerns or gems; config read through ENV or a YAML file at boot
471
472
  ```
472
473
 
473
- A row is the place (`commit:path:line`, which `git show` takes), how it reads the value (`mechanism`), its score, `rank:` (why it scores what it does), `why:` (what the source says), `also:` (more places of the same kind for the same value), and `test:` (what a test would check). `note:` marks a row on a line the change itself edits. Rows go from the highest score down, at most 10 in all; the first line counts the rest, and `--all-readers` prints every one, in the text and in `--json`. A row that a method's `callsites:` already lists, or a plain read inside a method that a constant's `read_by:` already lists, is left out. `not seen:` names what the reader cannot see: it names a record's model by the name of its variable or method (`@branch`, `home_branch`, `branches`), so `x = Branch.first; link_to x` is not listed.
474
+ A row is the place (`commit:path:line`, which `git show` takes), how it reads the value (`mechanism`), its score, `rank:` (why it scores what it does), `why:` (what the source says), `also:` (more places of the same kind for the same value), and `test:` (what a test would check). `note:` marks a row on a line the change itself edits. Rows go from the highest score down, at most 10 in all; the first line counts the rest, and `--all-readers` prints every one, in the text and in `--json`. A row that a method's `callsites:` already lists, or a plain read inside a method that a constant's `read_by:` already lists, is left out. `not seen:` names what the reader cannot see: it names a record's model by the name of its variable or method (`@branch`, `home_branch`, `branches`), so `x = Branch.first; link_to x` is not listed. A history with no `app/`, `config/routes.rb`, or `config/application.rb`, such as a gem's, has no models, routes, or templates, so there it names only the two that any Ruby hides.
474
475
 
475
476
  ## Install
476
477
 
@@ -497,7 +498,7 @@ Without the CLI, copy or symlink `skills/jitcatch` into the agent's skills direc
497
498
 
498
499
  ## Limits
499
500
 
500
- - A call site counts only when the source names its receiver's class: a call on `self` (or with no receiver), on a constant (`Cart.build`), on `self.class`, on `new` of one of those, on an instance variable that the class assigns only from one `Const.new`, or on a local variable that every assignment in its method sets from the same `Const.new` (`repo = Repo.new; repo.read`). A value the source does not type goes to the class Rails names it after, when that class defines the method: a parameter or a block parameter (`loan.approve` in `Gate#check(loan)`), a local that no assignment sets from a `Const.new`, and an instance variable that none sets (`@loan = loan`), all as a `Loan`. A reader that `belongs_to :borrower` or `has_one :borrower` declares reads a `Borrower`, or the class its `class_name:` string names, so `borrower.approved_cents` inside `Loan#approve` calls `Borrower#approved_cents`; a polymorphic one reads no class. A name that disagrees with a `Const.new` the method or class assigns to it names no class. A call on what a method of the same class returns, as in `emitter_for(format).render`, goes to each constant that method can return, when it returns only constants and nil (through `case`, `if`, and the ternary). A call on a local assigned two different constants, on a name that names no class, or on any other return value makes no edge, so `reachable_from` can miss a caller. Such a call is listed under `dynamic_callers:` of each changed method whose name, or the name of a method in its `reachable_from`, it sends, as `Emit.looked_up (render)`. An instance method of a class whose name ends in `Controller` is left out of those names, since the router calls a controller action and no code in the app sends its name, so `cipher.update` is not listed when `ThingsController#update` reaches the change. A call on `self` that its class does not define goes to the one indexed method of that name, for a method from a module or a superclass. From an instance method, only an instance method counts. A name the class declares with `attr_reader`, `attr_writer`, or `attr_accessor` goes to that reader or writer. `Const.new`, and `new` called with no receiver in a class method, also goes to that class's `initialize`, or to the nearest one up its superclass chain, so the callers of `Runner.new` reach `Runner#initialize` and what it calls. A superclass is looked up from outside the class it heads, as Ruby does, so `class Error < Error` inside `module Shop` names the outer `Error`. `reachable_from` keeps callers up to four calls away, and the hop from `new` into `initialize` is not one of the four. It keeps the eight nearest. `reached_by` in `--json` holds every caller at any depth, with no cap (`caller_depths` maps each to the fewest calls between them), and `run` reads it for the mentions and the related id, so a test that enters high, such as a request spec through a controller and a service or a CLI test that calls `CLI.start`, still names a changed method five or more calls down. Such a test reaches most changed methods, so its row is usually `related:guess` with that method in the list. `create`, `create!`, and `build` go there too when the class's superclass chain in the repository reaches `ActiveRecord::Base` (`class Post < ApplicationRecord`), since Active Record defines them to call `new`. On any other class they make no edge, since the code that defines them is not in the index. A class method of the same name up the chain takes the call instead.
501
+ - A call site counts only when the source names its receiver's class: a call on `self` (or with no receiver), on a constant (`Cart.build`), on `self.class`, on `new` of one of those, on an instance variable that the class assigns only from one `Const.new`, or on a local variable that every assignment in its method sets from the same `Const.new` (`repo = Repo.new; repo.read`). A value the source does not type goes to the class Rails names it after, when that class defines the method: a parameter or a block parameter (`loan.approve` in `Gate#check(loan)`), a local that no assignment sets from a `Const.new`, and an instance variable that none sets (`@loan = loan`), all as a `Loan`. A reader that `belongs_to :borrower` or `has_one :borrower` declares reads a `Borrower`, or the class its `class_name:` string names, so `borrower.approved_cents` inside `Loan#approve` calls `Borrower#approved_cents`; a polymorphic one reads no class. A name that disagrees with a `Const.new` the method or class assigns to it names no class. A call on what a method of the same class returns, as in `emitter_for(format).render`, goes to each constant that method can return, when it returns only constants and nil (through `case`, `if`, and the ternary). A call on a local assigned two different constants, on a name that names no class, or on any other return value makes no edge, so `reachable_from` can miss a caller. Such a call is listed under `dynamic_callers:` of each changed method whose name, or the name of a method in its `reachable_from`, it sends, as `Emit.looked_up (render)`. An instance method of a class whose name ends in `Controller` is left out of those names, since the router calls a controller action and no code in the app sends its name, so `cipher.update` is not listed when `ThingsController#update` reaches the change. A call on a local that holds only a block is left out too: a `&block` parameter, or a local that every assignment in its method sets to `-> {}`, `lambda {}`, `proc {}`, or `Proc.new {}`. Its `call` or `.()` runs that block, never a method of the repository. A `call` on a parameter, such as `job.call` in `def sync(job)`, is still listed. A call on `self` that its class does not define goes to the one indexed method of that name, for a method from a module or a superclass. From an instance method, only an instance method counts. A name the class declares with `attr_reader`, `attr_writer`, or `attr_accessor` goes to that reader or writer. `Const.new`, and `new` called with no receiver in a class method, also goes to that class's `initialize`, or to the nearest one up its superclass chain, so the callers of `Runner.new` reach `Runner#initialize` and what it calls. A superclass is looked up from outside the class it heads, as Ruby does, so `class Error < Error` inside `module Shop` names the outer `Error`. `reachable_from` keeps callers up to four calls away, and the hop from `new` into `initialize` is not one of the four. It keeps the eight nearest. `reached_by` in `--json` holds every caller at any depth, with no cap (`caller_depths` maps each to the fewest calls between them), and `run` reads it for the mentions and the related id, so a test that enters high, such as a request spec through a controller and a service or a CLI test that calls `CLI.start`, still names a changed method five or more calls down. Such a test reaches most changed methods, so its row is usually `related:guess` with that method in the list. `create`, `create!`, and `build` go there too when the class's superclass chain in the repository reaches `ActiveRecord::Base` (`class Post < ApplicationRecord`), since Active Record defines them to call `new`. On any other class they make no edge, since the code that defines them is not in the index. A class method of the same name up the chain takes the call instead.
501
502
  - `define_method` and `class_eval` are not indexed. A definition with a receiver other than `def self.foo` is coarse.
502
503
  - A rename is a deletion plus an addition (`git diff --no-renames`).
503
504
  - Coverage is not collected. `--coverage` only reads the JSON you pass.
@@ -505,7 +506,7 @@ Without the CLI, copy or symlink `skills/jitcatch` into the agent's skills direc
505
506
  - If the candidate tests never state the old behavior, a regression in that gap stays at `reported` 0.
506
507
  - A test of a new feature fails on base and is not a candidate.
507
508
  - The related id can sit one step away from the line that caused the failure.
508
- - A mention is a method call, a constant, or a symbol in that test method or in a method of the same file that it calls, directly or through other such methods. `Owner.new` mentions the `initialize` of the class whose name ends in `Owner`, and `mentions` lists it as the test writes it. `A::Owner.new` mentions only a class whose name ends in `A::Owner`, so it does not name `B::Owner#initialize`. A call in the changed constructor's `built_by:` mentions it the same way: `Loan.create!` on an Active Record model, or a FactoryBot call (`create(:loan)`, `build(:loan)`, `create_list(:loan, 2)`, `build_stubbed(:loan)`), which `mentions` lists as `create(:loan)` for a call that saves (`create`, `create_list`, `create_pair`) and as `build(:loan)` for one that does not. A file that defines its own method of that name calls that method, not FactoryBot, and `attributes_for` builds no object. A call sent to a class the test names, or to an instance the test builds with `.new`, is also listed with that class, as `Index.parse` or `Cart#total`. Among changed methods of one name, the related id is then the one on that class (or on a class whose name ends in it), unless none is, as for a method a superclass defines. A method name the test sends only to constants the repository does not define, as in `File.write(path, text)` or `Dir.mktmpdir`, is not a mention, so `File.write` does not name a changed method reached from `Emitter.write`. `emitter.send(:write)` sends `write` to `emitter`, so a test that writes both still mentions `write`. A `send`, `__send__`, or `public_send` with a literal name is read as a call of that name in the code too, so `send(:helper)` in a method reaches `helper`. Words in strings and comments are not mentions, except a method name passed as a plain string to `send`, `public_send`, `method`, or `respond_to?`, as in `send("total")`. A symbol that keys a Hash, as in `snap[:total]`, `snap.fetch(:total)`, or `{ total: 1 }`, is not a mention. A call to `message`, `full_message`, or `detailed_message` also mentions `to_s`, which an error class overrides to build its message. A string shaped like a relative path (`"exe/tool"`, `"../bin/x"`) that names a file of head, read from the repository root, from the candidate's directory, or as `File.expand_path` reads it next to the candidate, is a script the test may start in another process: the names in that file count as mentions too. A helper in another file is not read.
509
+ - A mention is a method call, a constant, or a symbol in that test method or in a method of the same file that it calls, directly or through other such methods. `Owner.new` mentions the `initialize` of the class whose name ends in `Owner`, and `mentions` lists it as the test writes it. `A::Owner.new` mentions only a class whose name ends in `A::Owner`, so it does not name `B::Owner#initialize`. A call in the changed constructor's `built_by:` mentions it the same way: `Loan.create!` on an Active Record model, or a FactoryBot call (`create(:loan)`, `build(:loan)`, `create_list(:loan, 2)`, `build_stubbed(:loan)`), which `mentions` lists as `create(:loan)` for a call that saves (`create`, `create_list`, `create_pair`) and as `build(:loan)` for one that does not. A file that defines its own method of that name calls that method, not FactoryBot, and `attributes_for` builds no object. A call sent to a class the test names, or to an instance the test builds with `.new`, is also listed with that class, as `Index.parse` or `Cart#total`. Among changed methods of one name, the related id is then the one on that class (or on a class whose name ends in it), unless none is, as for a method a superclass defines. A method name the test sends only to constants the repository does not define, as in `File.write(path, text)` or `Dir.mktmpdir`, is not a mention, so `File.write` does not name a changed method reached from `Emitter.write`. `emitter.send(:write)` sends `write` to `emitter`, so a test that writes both still mentions `write`. A `send`, `__send__`, or `public_send` with a literal name is read as a call of that name in the code too, so `send(:helper)` in a method reaches `helper`. Words in strings and comments are not mentions, except a method name passed as a plain string to `send`, `public_send`, `method`, or `respond_to?`, as in `send("total")`. A symbol that keys a Hash, as in `snap[:total]`, `snap.fetch(:total)`, or `{ total: 1 }`, is not a mention. A call to `message`, `full_message`, or `detailed_message` also mentions `to_s`, which an error class overrides to build its message. A string shaped like a relative path (`"exe/tool"`, `"../bin/x"`) that names a file of head, read from the repository root, from the candidate's directory, or as `File.expand_path` reads it next to the candidate, is a script the test may start in another process: the names in that file count as mentions too. A string that an assertion or a matcher compares directly, as `assert_includes error.backtrace.first, "lib/tool/model.rb"` or `include("lib/x.rb")`, is expected text and names no script. A helper in another file is not read.
509
510
  - Files run one at a time unless you pass `--workers`. The base run and the head run of one file run at the same time, each in its own worktree, so a test that writes a fixed path outside the worktree or listens on a fixed port can collide with its other side, and with other files under `--workers`. Pass `--serial` for such files. The repository's existing suite is not run. The repository has to be able to add worktrees.
510
511
  - An assertion backtrace usually stops in the test, so a library line is found from names the test mentions.
511
512
  - An RSpec example's mentions come from its own block, the `let`, `subject`, and hook blocks of the groups around it, those groups' arguments, and the same-file helpers they call. A group described as `"#total"` or `".build"` names that method, and `described_class.new` names `Owner.new`. `described_class.parse` is listed as `Owner.parse`. A `let(:result)` or `subject(:result)` name is the spec's own helper, so neither it nor a bare `result` call mentions a changed method `result`. When its block builds one class, as `let(:cart) { Cart.new }` does, `cart.total` is listed as `Cart#total`. An implicit `subject`, a shared example group, and a helper in `spec/support` are not read; an example from a shared group defined in another file reads the whole candidate file.
@@ -314,7 +314,8 @@ module JitCatch
314
314
  # Almost every test builds its object with `.new`, so a changed
315
315
  # constructor is the related method only when nothing else points.
316
316
  constructors, named = mentioned.partition { |item| mention(item).end_with?(".new") }
317
- return guess(named) if named.any?
317
+ stopped = stopped_raising?(test)
318
+ return guess(named, {}, stopped) if named.any?
318
319
 
319
320
  via_caller = @methods.select { |item|
320
321
  upstream = reached_by(item).map { |name| bare(name) }
@@ -323,17 +324,32 @@ module JitCatch
323
324
  sent = via_caller.select { |item| reached_by(item).any? { |name| sent_to?(name, mentions) } }
324
325
  via_caller = sent unless sent.empty?
325
326
  pool = via_caller.empty? ? constructors : via_caller
326
- guess(pool, depth_below_entry(pool, mentions))
327
+ guess(pool, depth_below_entry(pool, mentions), stopped)
327
328
  end
328
329
 
329
330
  # The pick, the pool, and whether only the id ordered the best two.
330
331
  # A tied pick still sets the tags, as it did when the id was shown.
331
- def guess(items, depths = {})
332
+ #
333
+ # On a row where head stopped raising the error the test expects, the
334
+ # method that raises it sits below every caller, so the downstream rule
335
+ # picks it, but head no longer reaches it. A method whose edit only
336
+ # changes the arguments of a raise did not decide that, so it leaves
337
+ # the pool while another method is in it.
338
+ def guess(items, depths = {}, stopped = false)
339
+ deciders = items.reject { |item| item["edits_only_raise"] }
340
+ items = deciders if stopped && deciders.any?
332
341
  ranked = items.sort_by { |item| guess_rank(item, items, depths) }
333
342
  tied = ranked.size > 1 && guess_rank(ranked[0], items, depths)[0...-1] == guess_rank(ranked[1], items, depths)[0...-1]
334
343
  [ranked.first, items, tied]
335
344
  end
336
345
 
346
+ # Minitest's assert_raises and RSpec's raise_error both end their
347
+ # message this way when the block returned.
348
+ def stopped_raising?(test)
349
+ test.dig("base", "status") == "pass" && test.dig("head", "status") == "fail" &&
350
+ test.dig("head", "message").to_s.match?(/but nothing was raised\.?\z/)
351
+ end
352
+
337
353
  # A test that names the class it sends a method to (`Index.parse`,
338
354
  # `described_class.parse`) calls that class's method of the name, not
339
355
  # another class's. When no changed method of the name is on a class
@@ -438,11 +454,18 @@ module JitCatch
438
454
  # a request spec of a Rails app, the method farthest from the entry
439
455
  # point the test calls was the one the failure came from in both, and
440
456
  # the lowest id in one, so depth ranks before the count below.
457
+ #
458
+ # A script's top-level code (`exe/tool <main>`) is in the pool only
459
+ # when the test starts that script, and it runs on every start, while
460
+ # each method it calls runs only for the command that picks it. So it
461
+ # ranks before the downstream rule: a test of `exe/tool bogus` whose
462
+ # usage text changed reached none of the methods below it.
441
463
  def guess_rank(item, items, depths)
442
464
  others = items.reject { |other| other["id"] == item["id"] }
443
465
  covered = others.count { |other| reached_by(item).include?(other["name"]) }
444
466
  below = @methods.count { |other| reached_by(other).include?(item["name"]) }
445
- [preference(item), -covered, -depths.fetch(item["id"], 0), -below, item["id"].to_s]
467
+ started = item["kind"] == "script" ? 0 : 1
468
+ [preference(item), started, -covered, -depths.fetch(item["id"], 0), -below, item["id"].to_s]
446
469
  end
447
470
 
448
471
  # The fewest calls between each item and an entry point the test
@@ -56,6 +56,7 @@ module JitCatch
56
56
  base_edges = edges_for(base_idx)
57
57
  decorate_calls(methods, head_edges, base_edges)
58
58
  mark_forwarders(methods, head_edges, hunks)
59
+ mark_raise_edits(methods)
59
60
  decorate_dynamic_callers(methods, head_idx, base_idx)
60
61
  factories = factories_of(head_idx)
61
62
  builders = builders_for(methods, head_idx, factories)
@@ -168,7 +169,7 @@ module JitCatch
168
169
  map = {}
169
170
  files.each do |file|
170
171
  path = file["new_path"] || file["old_path"]
171
- next unless path&.end_with?(".rb")
172
+ next unless path&.end_with?(".rb") || SourceIndex.script_path?(path.to_s)
172
173
 
173
174
  map[path] = file
174
175
  end
@@ -190,6 +191,9 @@ module JitCatch
190
191
  template_calls = []
191
192
  listed = @git.source_blobs(sha).select { |blob| blob.size <= MAX_BLOB }
192
193
  @texts.merge!(@git.read_blobs(listed.map(&:oid).uniq - @texts.keys))
194
+ scripts, listed = listed.partition { |blob| SourceIndex.script_path?(blob.path) }
195
+ # An executable that is not Ruby, such as a shell `bin/setup`, is not read.
196
+ scripts.select! { |blob| SourceIndex.ruby_script?(@texts.fetch(blob.oid)) }
193
197
  listed, views = listed.partition { |blob| blob.path.end_with?(".rb") }
194
198
  # A template defines no method, so its calls only add call sites, each
195
199
  # made from the template itself.
@@ -198,11 +202,11 @@ module JitCatch
198
202
  index = (@parsed[[blob.path, blob.oid]] ||= SourceIndex.parse(Template.as_ruby(templates[blob.path]), file: blob.path))
199
203
  template_calls.concat(index.calls.map { |call| call.with(enclosing: call.enclosing || blob.path) }) unless index.parse_error
200
204
  end
201
- listed.each do |blob|
205
+ (listed + scripts).sort_by(&:path).each do |blob|
202
206
  file = blob.path
203
207
  sources[file] = @texts.fetch(blob.oid)
204
208
  # A file both sides share is parsed once.
205
- index = (@parsed[[file, blob.oid]] ||= SourceIndex.parse(sources[file], file: file))
209
+ index = (@parsed[[file, blob.oid]] ||= SourceIndex.parse(sources[file], file: file, script: scripts.include?(blob)))
206
210
  indexes[file] = index
207
211
  if index.parse_error
208
212
  errors << { "ref" => label, "file" => file, "message" => index.parse_error }
@@ -405,6 +409,36 @@ module JitCatch
405
409
  end
406
410
  end
407
411
 
412
+ # An edit that changes only the arguments of a `raise` changes what the
413
+ # method raises, not whether it does. A test whose expected error head
414
+ # stopped raising then points at the code that stopped calling it, so
415
+ # the assessor ranks such a method after the others.
416
+ def mark_raise_edits(methods)
417
+ methods.each do |method|
418
+ method["edits_only_raise"] = method["change"] == "modified" &&
419
+ method["old"].source != method["new"].source &&
420
+ without_raises(method["old"].source) == without_raises(method["new"].source)
421
+ end
422
+ end
423
+
424
+ # The source with each receiverless `raise` or `fail` call written as
425
+ # the bare word, so two sources that differ only there compare equal.
426
+ def without_raises(source)
427
+ text = source.b
428
+ pending = [Prism.parse(source).value]
429
+ spans = []
430
+ until pending.empty?
431
+ node = pending.pop
432
+ if node.is_a?(Prism::CallNode) && node.receiver.nil? && %i[raise fail].include?(node.name)
433
+ spans << [node.location.start_offset, node.location.end_offset]
434
+ else
435
+ pending.concat(node.compact_child_nodes)
436
+ end
437
+ end
438
+ spans.sort.reverse_each { |first, last| text = text.byteslice(0, first) + "raise".b + text.byteslice(last..) }
439
+ text
440
+ end
441
+
408
442
  def forwards?(node, added, head_edges, hunks)
409
443
  lines = []
410
444
  removed = 0
@@ -463,10 +497,14 @@ module JitCatch
463
497
  merge_hits(call_hits(test_sources.keys, ["#{node.qualifier}.new", *method["builders"]], "builder", base_idx, head_idx), [])
464
498
  elsif node.kind == "endpoint"
465
499
  request_hits(test_sources, node.bare_name)
500
+ elsif node.kind == "script"
501
+ # A test starts an executable by its path, as `exe/tool` or
502
+ # `../exe/tool`, not by a method name.
503
+ grep_hits(test_sources, node.file)
466
504
  elsif controller_action?(node.qualified) && (routes ||= app_routes(head_idx)).drawn?(node.qualified)
467
505
  route_hits(test_sources, node.qualified, requests ||= test_requests(test_sources, routes, base_idx, head_idx))
468
506
  else
469
- merge_hits(call_hits(test_sources.keys, method["run_by"], "callback", base_idx, head_idx), grep_hits(test_sources, node.bare_name))
507
+ merge_hits(call_hits(test_sources.keys, method["run_by"], "callback", base_idx, head_idx), grep_hits(test_sources, node.bare_name, owner_pattern(node)))
470
508
  end
471
509
  end
472
510
  method["entry_point_tests"] = if @coverage || !method["existing_tests"].empty?
@@ -564,10 +602,17 @@ module JitCatch
564
602
  "edited_by_change" => row.edited, "mechanism" => row.mechanism.to_s, "score" => row.score, "rank" => row.rank,
565
603
  "why" => row.why, "also" => row.also, "test" => row.test }
566
604
  },
567
- "not_seen" => Readers::BLIND
605
+ "not_seen" => rails_shaped?(head_idx) ? Readers::BLIND : Readers::BLIND_OUTSIDE_RAILS
568
606
  }
569
607
  end
570
608
 
609
+ # Models, routes, and templates live in app/ and config/routes.rb, so a
610
+ # history with neither, as a gem's, hides none of what Rails would.
611
+ def rails_shaped?(idx)
612
+ paths = idx["sources"].keys + idx["templates"].keys
613
+ paths.any? { |path| path.start_with?("app/") || %w[config/application.rb config/routes.rb].include?(path) }
614
+ end
615
+
571
616
  # The method or constant entry a value is, by its name and where it is
572
617
  # declared. A setting, a callback, or a default has none.
573
618
  def listed_item(value, methods, constants)
@@ -614,17 +659,32 @@ module JitCatch
614
659
  }
615
660
  end
616
661
 
617
- def grep_hits(test_sources, bare)
662
+ # `owner`, when given, is a pattern the file must match too.
663
+ def grep_hits(test_sources, bare, owner = nil)
618
664
  return [] if bare.nil? || bare.length < 3
619
665
 
620
666
  pattern = /\b#{called(bare)}/
621
667
  test_sources.filter_map { |file, source|
622
- next unless source.match?(pattern)
668
+ next unless source.match?(pattern) && (owner.nil? || source.match?(owner))
623
669
 
624
670
  { "file" => file, "via" => "grep" }
625
671
  }.sort_by { |hit| hit["file"] }.first(TEST_CAP)
626
672
  end
627
673
 
674
+ # A reader or a member (`attr_reader :file`, `Data.define(:code, :line)`)
675
+ # is most often a common word, which a grep finds in files that never
676
+ # touch the class. Such a hit counts only in a file that also names the
677
+ # class or the namespace around it, as `Hit` or `Targets` for
678
+ # `Targets::Hit#code`.
679
+ MEMBER_MACRO = /\A(?:attr_reader|attr_writer|Struct\.new|Data\.define)\z/
680
+
681
+ def owner_pattern(node)
682
+ return nil unless MEMBER_MACRO.match?(node.macro.to_s)
683
+
684
+ words = node.qualifier.split("::").last(2)
685
+ words.empty? ? nil : /\b(?:#{words.map { |word| Regexp.escape(word) }.join("|")})\b/
686
+ end
687
+
628
688
  # A factory that builds a class no file defines may need an inflection
629
689
  # the app configures (`ApiKey` against `APIKey`), so it is not followed.
630
690
  # One that builds a Ruby value, as `class: Hash` does for the shape of an
@@ -767,10 +827,27 @@ module JitCatch
767
827
  end
768
828
 
769
829
  def test_sources_for(base_idx, head_idx)
770
- paths = (base_idx["sources"].keys | head_idx["sources"].keys).select { |path| test_file?(path) }.sort
830
+ paths = (base_idx["sources"].keys | head_idx["sources"].keys).select { |path| test_file?(path) && !fixture_code?(path) }.sort
771
831
  paths.to_h { |path| [path, head_idx["sources"][path] || base_idx["sources"][path]] }
772
832
  end
773
833
 
834
+ # A project kept as a fixture under test/ or spec/ (a small Rails app in
835
+ # `test/fixture/depot`) holds code of its own in app/, lib/, config/,
836
+ # and db/. That code tests nothing, so it is no test a `tests:` line
837
+ # can name; the fixture's own test and spec files still are.
838
+ FIXTURE_DIR = /\Afixtures?\z/
839
+ FIXTURE_CODE = %w[app lib config db].freeze
840
+
841
+ def fixture_code?(path)
842
+ parts = path.split("/")
843
+ at = parts.index { |part| FIXTURE_DIR.match?(part) }
844
+ return false unless at && parts.first(at).any? { |part| %w[test spec].include?(part) }
845
+
846
+ base = parts.last
847
+ inner = parts[(at + 1)...-1]
848
+ !(base.start_with?("test_") || base.end_with?("_test.rb", "_spec.rb")) && inner.any? { |part| FIXTURE_CODE.include?(part) }
849
+ end
850
+
774
851
  def test_file?(path)
775
852
  base = File.basename(path)
776
853
  return false unless base.end_with?(".rb")
@@ -1000,6 +1077,7 @@ module JitCatch
1000
1077
  "entry_point_tests" => item["entry_point_tests"],
1001
1078
  "via_constant" => item["via_constant"],
1002
1079
  "forwards_to_added" => item["forwards_to_added"],
1080
+ "edits_only_raise" => item["edits_only_raise"],
1003
1081
  "moved_from" => item["moved_from"]
1004
1082
  }
1005
1083
  end
@@ -83,7 +83,7 @@ module JitCatch
83
83
  git("ls-tree", "-r", "-l", "-z", "--end-of-options", ref).split("\0").filter_map { |entry|
84
84
  meta, path = entry.split("\t", 2)
85
85
  mode, type, oid, size = meta.split
86
- next unless type == "blob" && mode != SYMLINK_MODE && (path.end_with?(".rb") || Template.template?(path))
86
+ next unless type == "blob" && mode != SYMLINK_MODE && (path.end_with?(".rb") || Template.template?(path) || SourceIndex.script_path?(path))
87
87
 
88
88
  Blob.new(path: path, oid: oid, size: size.to_i)
89
89
  }.sort_by(&:path)
@@ -46,6 +46,13 @@ module JitCatch
46
46
  "config read through ENV or a YAML file at boot"
47
47
  ].freeze
48
48
 
49
+ # What a repository with no `config/application.rb` can still hide:
50
+ # models, routes, and templates are Rails's, and a gem has none.
51
+ BLIND_OUTSIDE_RAILS = [
52
+ "dispatch names built at run time from data outside the source (a file, an argument)",
53
+ "config read through ENV or a YAML file at load time"
54
+ ].freeze
55
+
49
56
  DISPATCH = %i[send public_send __send__ try try! method public_method respond_to? instance_method].freeze
50
57
  SQL_CALLS = %i[where order reorder select joins group having pluck find_by_sql exec_query execute from not count_by_sql].freeze
51
58
  QUERY_CALLS = %i[all find find_by find_by! where first last take count exists? pluck order includes find_each].freeze
@@ -347,7 +347,7 @@ module JitCatch
347
347
 
348
348
  first, *rest = message_lines(base)
349
349
  outcome = [base["status"], base["error_class"], first].reject { |part| part.to_s.empty? }
350
- lines = [" base: #{outcome.join(" ")}", *rest.map { |line| " #{line}" }]
350
+ lines = [" base: #{outcome.join(" ")}", *rest.map { |line| " #{line}" }, *head_outcome_lines(test)]
351
351
  return lines unless Assessor.no_table?(base)
352
352
 
353
353
  databases = base_databases(doc)
@@ -355,6 +355,20 @@ module JitCatch
355
355
  lines << " hint: base found no table, so this row checked nothing; load base's schema into #{into}"
356
356
  end
357
357
 
358
+ # A test written to pin what a change fixes fails on base by design, and
359
+ # head's result is what it checks: `pass`, head's own failure, or that
360
+ # head failed as base did.
361
+ def head_outcome_lines(test)
362
+ head = test["head"]
363
+ return [] unless head
364
+
365
+ same = %w[status error_class message].all? { |key| head[key] == test["base"][key] }
366
+ return [" head: #{head["status"]}, the same as base"] if same && head["status"] != "pass"
367
+
368
+ first, *rest = message_lines(head)
369
+ [" head: #{[head["status"], head["error_class"], first].reject { |part| part.to_s.empty? }.join(" ")}", *rest.map { |line| " #{line}" }]
370
+ end
371
+
358
372
  # Only the numbered databases have names the reader can load a schema
359
373
  # into; the URL itself can hold a password, so it is never printed.
360
374
  def untested_lines(doc)
@@ -783,8 +783,12 @@ module JitCatch
783
783
  [head_dir, File.dirname(candidate), candidate].map { |dir| File.expand_path(path, dir) }
784
784
  .select { |full| full.start_with?(root) && !full.start_with?(own) && File.file?(full) }
785
785
  }.uniq.flat_map { |full|
786
- script = SourceIndex.parse(read_text(full), file: full.delete_prefix(root))
787
- script.parse_error ? [] : script.referenced_names("")
786
+ relative_path = full.delete_prefix(root)
787
+ script = SourceIndex.parse(read_text(full), file: relative_path)
788
+ # Starting an executable runs its top-level code, which the change
789
+ # document lists as one method when the diff edits it.
790
+ main = SourceIndex.script_path?(relative_path) ? [SourceIndex.script_name(relative_path)] : []
791
+ script.parse_error ? main : main + script.referenced_names("")
788
792
  }
789
793
  end
790
794
 
@@ -19,6 +19,9 @@ module JitCatch
19
19
  MESSAGE_CALLS = %i[message full_message detailed_message].freeze
20
20
  # A relative path with a directory part and no spaces: `exe/tool`.
21
21
  PATH_LIKE = %r{\A(?:\.\.?/)*[\w.-]+(?:/[\w.-]+)+\z}
22
+ # Minitest assertions and expectations, RSpec matchers, and the string
23
+ # predicates a test asserts on.
24
+ COMPARING_CALL = /\A(?:assert|refute|must|wont)(?:_\w+)?\z|\A(?:eq|eql|equal|include|match|start_with|end_with|include\?|match\?|start_with\?|end_with\?|==|=~)\z/
22
25
  ATTRIBUTE_MACROS = { attr_reader: [""], attr: [""], attr_writer: ["="], attr_accessor: ["", "="] }.freeze
23
26
  # The constant a member list is given to, and whether its members get writers.
24
27
  MEMBER_BUILDERS = { %i[Struct new] => true, %i[Data define] => false }.freeze
@@ -62,9 +65,13 @@ module JitCatch
62
65
  # does. A local variable is an `instance` when every assignment to it in
63
66
  # its method builds the same constant, and `named` when none builds
64
67
  # one, as for a parameter. A reader that `belongs_to` or `has_one`
65
- # declares is an `instance` of the model it names.
68
+ # declares is an `instance` of the model it names. A local that every
69
+ # assignment in its method sets to a lambda or a proc literal, or that
70
+ # `&block` binds, is a `proc`: its `call` runs that block, never a
71
+ # method the repository defines.
66
72
  Receiver = Data.define(:kind, :const)
67
73
  UNKNOWN_RECEIVER = Receiver.new(kind: "unknown", const: nil)
74
+ PROC_RECEIVER = Receiver.new(kind: "proc", const: nil)
68
75
  # A block given to an RSpec call: a `group` (describe, context), an
69
76
  # `example` (it, specify), or a `setup` (let, subject, before). Code in
70
77
  # an example or a setup block is referred to under `key`, and so are a
@@ -153,8 +160,27 @@ module JitCatch
153
160
  @new_visibility.filter_map { |qualifier, visibility| qualifier unless visibility == "public" }
154
161
  end
155
162
 
156
- def self.parse(source, file:)
157
- new(source, file).parse
163
+ # `script: true` reads the file as an executable, such as `exe/tool`:
164
+ # its top-level code is one method of kind `script`, named by
165
+ # `script_name`, whose calls are that method's.
166
+ def self.parse(source, file:, script: false)
167
+ new(source, file, script).parse
168
+ end
169
+
170
+ # The executables a gem or an app ships, which carry no `.rb` suffix.
171
+ SCRIPT_PATH = %r{\A(?:exe|bin)/[\w-]+\z}
172
+
173
+ def self.script_path?(path)
174
+ SCRIPT_PATH.match?(path)
175
+ end
176
+
177
+ # A file whose first line runs Ruby.
178
+ def self.ruby_script?(text)
179
+ text.to_s.lines.first.to_s.match?(/\A#!.*\bruby\b/)
180
+ end
181
+
182
+ def self.script_name(file)
183
+ "#{file} <main>"
158
184
  end
159
185
 
160
186
  # Every require_relative of a source, wherever it is written. Only a
@@ -180,7 +206,8 @@ module JitCatch
180
206
  parts.all?(Prism::StringNode) && !parts.empty? ? File.join(*parts.map(&:unescaped)) : nil
181
207
  end
182
208
 
183
- def initialize(source, file)
209
+ def initialize(source, file, script = false)
210
+ @script = script
184
211
  @source = source.to_s.gsub("\r\n", "\n")
185
212
  @lines = @source.lines
186
213
  @file = file.tr("\\", "/")
@@ -196,6 +223,7 @@ module JitCatch
196
223
  @references = Hash.new { |hash, key| hash[key] = [] }
197
224
  @ivar_classes = Hash.new { |hash, key| hash[key] = [] }
198
225
  @local_classes = Hash.new { |hash, key| hash[key] = [] }
226
+ @local_procs = Hash.new { |hash, key| hash[key] = [] }
199
227
  @helper_classes = Hash.new { |hash, key| hash[key] = [] }
200
228
  @parse_error = nil
201
229
  @spec_blocks = []
@@ -205,6 +233,7 @@ module JitCatch
205
233
  @association_classes = {}
206
234
  @symbol_lists = {}
207
235
  @paths = Hash.new { |hash, key| hash[key] = [] }
236
+ @compared = {}
208
237
  @factory_calls = Hash.new { |hash, key| hash[key] = [] }
209
238
  @endpoints = {}
210
239
  end
@@ -218,8 +247,9 @@ module JitCatch
218
247
 
219
248
  @string_constants = string_constants(result.value)
220
249
  scan_body(result.value, "", :public, false)
250
+ main = @script ? record_script(result.value) : nil
221
251
  @spec_helpers = spec_helper_names(result.value)
222
- walk(result.value, "", false, nil)
252
+ walk(result.value, "", false, main)
223
253
  type_local_variables
224
254
  type_instance_variables
225
255
  type_spec_helpers
@@ -622,6 +652,25 @@ module JitCatch
622
652
  )
623
653
  end
624
654
 
655
+ # The statements of an executable outside its `def`, class, and module
656
+ # bodies, as one method: a change to its usage text or to the `case`
657
+ # that picks a command is a change to it, and an edit inside a `def`
658
+ # is not. Its source holds only those statements, so `changes` sees
659
+ # an edit to them by the source alone.
660
+ def record_script(program)
661
+ top = statements(program).reject { |stmt| [Prism::DefNode, Prism::ClassNode, Prism::ModuleNode].any? { |kind| stmt.is_a?(kind) } }
662
+ name = self.class.script_name(@file)
663
+ return name if top.empty?
664
+
665
+ @methods << self.class::Method.new(
666
+ file: @file, bare_name: name, qualifier: "", qualified: name, kind: "script", visibility: "public",
667
+ start_line: top.first.location.start_line, end_line: top.last.location.end_line,
668
+ source: top.map { |stmt| extract_lines(stmt.location.start_line, stmt.location.end_line) }.join("\n"),
669
+ params: "", macro: "script"
670
+ )
671
+ name
672
+ end
673
+
625
674
  # A string, a lambda, or a block runs code the line does not name, so
626
675
  # only symbols count.
627
676
  def record_callbacks(stmt, qualifier)
@@ -984,6 +1033,9 @@ module JitCatch
984
1033
  @calls.map! do |call|
985
1034
  next call unless call.receiver.kind == "named"
986
1035
 
1036
+ procs = @local_procs.fetch([call.enclosing, call.receiver.const], [])
1037
+ next call.with(receiver: PROC_RECEIVER) if procs.any? && procs.all?
1038
+
987
1039
  assigned = @local_classes.fetch([call.enclosing, call.receiver.const], [])
988
1040
  next call if assigned.all?(&:empty?)
989
1041
 
@@ -1061,6 +1113,18 @@ module JitCatch
1061
1113
 
1062
1114
  # The constants a value builds with `Const.new`, through `a || b`,
1063
1115
  # `c ? a : b`, and parentheses.
1116
+ # `-> {}`, `lambda {}`, `proc {}`, and `Proc.new {}` make a Proc.
1117
+ def proc_literal?(node)
1118
+ return true if node.is_a?(Prism::LambdaNode)
1119
+ return false unless node.is_a?(Prism::CallNode) && node.block.is_a?(Prism::BlockNode)
1120
+
1121
+ if node.receiver.nil?
1122
+ %i[lambda proc].include?(node.name)
1123
+ else
1124
+ node.name == :new && node.receiver.is_a?(Prism::ConstantReadNode) && node.receiver.name == :Proc
1125
+ end
1126
+ end
1127
+
1064
1128
  def built_constants(node)
1065
1129
  case node
1066
1130
  when Prism::CallNode
@@ -1167,6 +1231,7 @@ module JitCatch
1167
1231
  refer(enclosing, request) if request
1168
1232
  factory = factory_built(node)
1169
1233
  @factory_calls[enclosing] << [node.name.to_s, factory, factory_traits(node)] if factory
1234
+ mark_compared_strings(node)
1170
1235
  end
1171
1236
  node.compact_child_nodes.each do |child|
1172
1237
  children = child.is_a?(Prism::ArgumentsNode) && HASH_KEY_CALLS.include?(node.name) ? child.arguments.grep_v(Prism::SymbolNode) : [child]
@@ -1186,12 +1251,15 @@ module JitCatch
1186
1251
  refer(enclosing, node.unescaped)
1187
1252
  when Prism::StringNode
1188
1253
  text = node.unescaped
1189
- @paths[enclosing] << text if text.valid_encoding? && text.match?(PATH_LIKE)
1254
+ @paths[enclosing] << text if text.valid_encoding? && text.match?(PATH_LIKE) && !@compared.key?(node.location.start_offset)
1190
1255
  when Prism::LocalVariableWriteNode
1191
1256
  @local_classes[[enclosing, node.name.to_s]] << built_constants(node.value).uniq
1257
+ @local_procs[[enclosing, node.name.to_s]] << proc_literal?(node.value)
1192
1258
  node.compact_child_nodes.each { |child| walk(child, qualifier, singleton_context, enclosing) }
1193
1259
  when *LOCAL_BINDINGS
1194
1260
  @local_classes[[enclosing, node.name.to_s]] << [] if node.name
1261
+ # `&block` binds the block given, which is a Proc.
1262
+ @local_procs[[enclosing, node.name.to_s]] << node.is_a?(Prism::BlockParameterNode) if node.name
1195
1263
  node.compact_child_nodes.each { |child| walk(child, qualifier, singleton_context, enclosing) }
1196
1264
  when Prism::InstanceVariableWriteNode, Prism::InstanceVariableOrWriteNode
1197
1265
  @ivar_classes[[qualifier.to_s, node.name.to_s]].concat(built_constants(node.value))
@@ -1201,6 +1269,17 @@ module JitCatch
1201
1269
  end
1202
1270
  end
1203
1271
 
1272
+ # A string an assertion or a matcher compares with a value, as in
1273
+ # `assert_includes error.backtrace.first, "lib/tool/model.rb"`, is
1274
+ # text the test expects, not the path of a script it starts. Only a
1275
+ # direct argument counts, so `assert_equal "1\n", IO.popen(["ruby",
1276
+ # "exe/tool"], &:read)` still names `exe/tool`.
1277
+ def mark_compared_strings(node)
1278
+ return unless COMPARING_CALL.match?(node.name.to_s) && node.arguments
1279
+
1280
+ node.arguments.arguments.grep(Prism::StringNode).each { |arg| @compared[arg.location.start_offset] = true }
1281
+ end
1282
+
1204
1283
  def spec_block?(node)
1205
1284
  return false unless node.block.is_a?(Prism::BlockNode) && SPEC_KINDS.key?(node.name)
1206
1285
 
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module JitCatch
4
- VERSION = "0.1.0"
4
+ VERSION = "0.2.0"
5
5
  end
@@ -13,7 +13,7 @@ Do not commit the candidate tests unless the human asks to promote one.
13
13
 
14
14
  Ruby 3.3 or newer, and git. No gems. `minitest` and `prism` ship with Ruby. RSpec candidates need `rspec-core`: in the project's bundle when it has a `Gemfile`, otherwise installed for the Ruby that runs jitcatch. The subject must be a git repository that can add worktrees. The tool reads commits only. Uncommitted files are not in the diff.
15
15
 
16
- `gem install jitcatch` installs the `jitcatch` command, with no runtime dependency. In an app with a `Gemfile`, such as a Rails app, the gem can sit in the `:development, :test` group instead (`gem "jitcatch", require: false`); run it as `bundle exec jitcatch`. Each side still loads its own bundle, so the line does not need to be in either commit. From a checkout of the jitcatch repository, put its `bin` directory on `PATH`, or run `ruby /path/to/jitcatch/bin/jitcatch`. `jitcatch --help` prints the commands, and `jitcatch --version` prints `jitcatch 0.1.0`, with the commit after it (`jitcatch 0.1.0 (1fbcd18)`) when jitcatch runs from its own git checkout; quote that line with any output you quote. The `version` line of every report and `version` in `--json` hold the number only.
16
+ `gem install jitcatch` installs the `jitcatch` command, with no runtime dependency. In an app with a `Gemfile`, such as a Rails app, the gem can sit in the `:development, :test` group instead (`gem "jitcatch", require: false`); run it as `bundle exec jitcatch`. Each side still loads its own bundle, so the line does not need to be in either commit. From a checkout of the jitcatch repository, put its `bin` directory on `PATH`, or run `ruby /path/to/jitcatch/bin/jitcatch`. `jitcatch --help` prints the commands, and `jitcatch --version` prints `jitcatch 0.2.0`, with the commit after it (`jitcatch 0.2.0 (1fbcd18)`) when jitcatch runs from its own git checkout; quote that line with any output you quote. The `version` line of every report and `version` in `--json` hold the number only.
17
17
 
18
18
  Exit status 0 is success, including when catching tests were found. Exit status 1 is a usage error. Exit status 2 is a git error. Exit status 3 means `run` tested nothing: a side failed its setup check before any test ran, no candidate file loaded on either side, or every row timed out, or found no database or no table, on a side (below).
19
19
 
@@ -38,13 +38,49 @@ jitcatch context --repo DIR --all-readers BASE HEAD
38
38
 
39
39
  Omit `--repo` when the working directory is the repository.
40
40
 
41
- Read `intent:` first. Then the changed files, `dependencies:` (each gem whose `Gemfile.lock` version changed, as `gem old -> new`, shown only when one did), constants (`k01`, …), and methods (`m01`, …). Each method has a change kind, signature, visibility, call sites, `reachable_from`, existing tests, and the old and new source. `reachable_from` goes through constructors: a caller of `Runner.new` reaches `Runner#initialize` and the methods it calls, and that hop does not count toward the four calls `reachable_from` follows. `reachable_from` keeps the eight nearest. `run` reads `reached_by` (`--json`), every caller at any depth, so a test that enters high (a request spec through a controller and a service, or a CLI test of `CLI.start`) still reaches a changed method five or more calls down; its row is usually `related:guess` with that method in the list. On an Active Record model, a class whose superclass chain in the repository reaches `ActiveRecord::Base`, so do `create`, `create!`, and `build`. On another class those three names reach nothing. A method marked `moved` left one file while head defines it in another, test files included; the line shows its new place and `moved_from=` the old file, and a call to it is not a call to a removed method. A constant that moves to another file with the same value is `moved` with `moved_from=` as well, and its readers are not `affected`; with a new value it is `modified`, with `moved_from=`. A method marked `affected` did not change its body. It reads a constant that did (`via=k01`). A class or module that only one side defines anywhere is listed as an `added` or `removed` constant, with its header line as the source. A file that reopens an existing class adds nothing. Methods defined in a block given to a constant (`Point = Struct.new(:x) do`, `Class.new do`, `Data.define do`) are listed under that constant (`Point#norm`), and the constant's source is the text before `do`. A reader or writer that `attr_reader`, `attr_writer`, `attr_accessor`, or a `Struct.new` / `Data.define` member list defines is a method too (`Point#x`, `Point#x=`), with the one-name form as its source (`attr_reader :seed`, `Struct.new(:x)`). On an Active Record model, `scope :recent, -> { ... }` is the class method `Post.recent` with the whole statement as its source, `delegate :name, to: :author` is `Post#name` (`Post#author_name` with `prefix: true`), a Grape endpoint (`post "/refresh" do` inside `resource :things do`, in a class under `Grape::API`) is `Things#POST /things/refresh`, which a request spec reaches by sending that verb to a path ending in those segments (`:id` matches any one, an interpolation in the test matches any one segment, and a string constant the file assigns reads as its string), and a request to a path helper (`post things_path`) reaches the controller action `config/routes.rb` draws for it, with the helper and controller names the app's `config/initializers/inflections.rb` gives (`person_path` for `resources :people`, `APIKeysController` with `inflect.acronym "API"`), so a changed action's `tests:` lists the files whose requests reach it (`via POST things_path`) and the controller tests that call it as `post :create`, not a grep of its name; an endpoint path the source computes otherwise is named with that part in braces and gets a `warning:`, as does a helper the routes file does not draw, and `enum :status, { draft: 0 }` gives each value `Post#draft?`, `Post#draft!`, `Post.draft`, and `Post.not_draft`, with that one value as the source. `Struct.new(*FIELDS)` and `attr_reader(*FIELDS)` count when the same file set `FIELDS` to a list of symbols earlier; a list built at run time or in another file gives no members. A `def` or a reader inside a conditional of the class body (`attr_reader :x if cond`, `if`, `unless`, `else`) counts, read as if every branch ran in order, so a bare `private` in a branch carries past the conditional. A definition counts too in an `ActiveSupport::Concern` block: `included do` and `prepended do` define instance methods of the module, `class_methods do` its class methods. A name dropped from that line is `removed`, the names left on it are unchanged, and a `NoMethodError` on the dropped name, called from the test, is `discard:signature_change`.
41
+ Read `intent:` first. Then the changed files, `dependencies:` (each gem whose `Gemfile.lock` version changed, as `gem old -> new`, shown only when one did), constants (`k01`, …), and methods (`m01`, …). Each method has a change kind, signature, visibility, call sites, `reachable_from`, existing tests, and the old and new source.
42
+
43
+ `reachable_from` goes through constructors: a caller of `Runner.new` reaches `Runner#initialize` and the methods it calls, and that hop does not count toward the four calls `reachable_from` follows. `reachable_from` keeps the eight nearest. `run` reads `reached_by` (`--json`), every caller at any depth, so a test that enters high (a request spec through a controller and a service, or a CLI test of `CLI.start`) still reaches a changed method five or more calls down; its row is usually `related:guess` with that method in the list.
44
+
45
+ Change kinds:
46
+
47
+ - `moved`: the method left one file while head defines it in another, test files included. The line shows its new place and `moved_from=` the old file, and a call to it is not a call to a removed method. A constant that moves to another file with the same value is `moved` with `moved_from=` as well, and its readers are not `affected`; with a new value it is `modified`, with `moved_from=`.
48
+ - `affected`: the method did not change its body. It reads a constant that did (`via=k01`).
49
+ - A class or module that only one side defines anywhere is listed as an `added` or `removed` constant, with its header line as the source. A file that reopens an existing class adds nothing.
50
+
51
+ What counts as a method, in any Ruby:
52
+
53
+ - Methods defined in a block given to a constant (`Point = Struct.new(:x) do`, `Class.new do`, `Data.define do`) are listed under that constant (`Point#norm`), and the constant's source is the text before `do`.
54
+ - A reader or writer that `attr_reader`, `attr_writer`, `attr_accessor`, or a `Struct.new` / `Data.define` member list defines is a method too (`Point#x`, `Point#x=`), with the one-name form as its source (`attr_reader :seed`, `Struct.new(:x)`). A name dropped from that line is `removed`, the names left on it are unchanged, and a `NoMethodError` on the dropped name, called from the test, is `discard:signature_change`. `Struct.new(*FIELDS)` and `attr_reader(*FIELDS)` count when the same file set `FIELDS` to a list of symbols earlier; a list built at run time or in another file gives no members.
55
+ - A `def` or a reader inside a conditional of the class body (`attr_reader :x if cond`, `if`, `unless`, `else`) counts, read as if every branch ran in order, so a bare `private` in a branch carries past the conditional.
56
+
57
+ ### Rails and Grape definitions
58
+
59
+ - On an Active Record model (a class whose superclass chain in the repository reaches `ActiveRecord::Base`), a caller of `create`, `create!`, or `build` reaches `initialize` too, as a caller of `new` does. On another class those three names reach nothing.
60
+ - `scope :recent, -> { ... }` is the class method `Post.recent` with the whole statement as its source.
61
+ - `delegate :name, to: :author` is `Post#name` (`Post#author_name` with `prefix: true`).
62
+ - `enum :status, { draft: 0 }` gives each value `Post#draft?`, `Post#draft!`, `Post.draft`, and `Post.not_draft`, with that one value as the source.
63
+ - A definition counts too in an `ActiveSupport::Concern` block: `included do` and `prepended do` define instance methods of the module, `class_methods do` its class methods.
64
+ - A request to a path helper (`post things_path`) reaches the controller action `config/routes.rb` draws for it, with the helper and controller names the app's `config/initializers/inflections.rb` gives (`person_path` for `resources :people`, `APIKeysController` with `inflect.acronym "API"`). So a changed action's `tests:` lists the files whose requests reach it (`via POST things_path`) and the controller tests that call it as `post :create`, not a grep of its name. A helper the routes file does not draw gets a `warning:`.
65
+ - A Grape endpoint (`post "/refresh" do` inside `resource :things do`, in a class under `Grape::API`) is `Things#POST /things/refresh`. A request spec reaches it by sending that verb to a path ending in those segments: `:id` matches any one, an interpolation in the test matches any one segment, and a string constant the file assigns reads as its string. An endpoint path the source computes otherwise is named with that part in braces and gets a `warning:`.
66
+
67
+ ### Visibility, test files, and executables
42
68
 
43
69
  `visibility=` follows `private`, `protected`, and `public` (bare, with names, or before a `def`), the same inside `class << self`, and `private_class_method` / `public_class_method` (with names or before `def self.x`). A `private` method is not an entry point. Test it through a public caller in `reachable_from`. A method after `module_function`, or named by it, is listed as the public module method `Mod.name`. A method of a class or module that `private_constant` names in the same file, or of anything nested in it, is listed as `private`, because a test cannot name its receiver. Visibility set by `send`, or by `private :name` written above its `def`, is not seen.
44
70
 
45
71
  Methods in test files (under `test/` or `spec/`, or named `test_*.rb` or `*_test.rb`) are not listed, and are not entry points. The changed test files still appear under `files:`.
46
72
 
47
- `tests: (none)` means no test file names that method. Then `entry_point_tests:` lists, when it finds any, the test files that call a public `reachable_from` entry point: the file names the entry point's class (`Cart.new`, not only `Cart::LIMIT`) and calls a method of that name (`.total`). Each file shows the entry points it matched. Read those tests before you write yours. They show how the repository builds the object and which inputs already reach the changed method. Like `tests:`, this line is a grep. It misses tests that drive the code through a subprocess, and it can list a file that calls a same-named method on another object. A `tests:` grep hit on a common name such as `call` can be any file that defines or calls that name. A changed `initialize` shows `built_by:` above `tests:`: `create`, `create!`, and `build` on an Active Record model, and `build(:loan)` and `create(:loan)` for each factory under `spec/factories` or `test/factories` that builds the class. Its `tests:` lists only the test files that call `Owner.new` or one of those, as `spec/models/loan_spec.rb via create(:loan)`, and no grep hit for the word `initialize`. `factories_skipped:` lists each factory whose class the source does not settle (a `class:` computed at run time, an unknown parent, or a class no file defines, as an inflection such as `APIKey` causes). A factory that builds a Ruby value (`class: Hash`, `Array`, `Struct`, `OpenStruct`, and the like) is left out and not listed. A test that builds from one of those names no constructor. A factory also counts for the classes its associations build (`borrower`, `association :borrower`, or `association` in an attribute block), so a changed `Borrower` callback lists `create(:loan)` when `:loan` builds a borrower. An association in a trait lists the call with the trait, `create(:loan, :cosigned)`, and a test that writes `create(:loan, :cosigned)` mentions it; a global trait and a trait that includes another are not followed. A method that a model's class body names in a callback or a validation (`after_save :count_save`, or `if: :checked?` on a `validates`) shows `run_by:`: the calls that run it by Rails's rules (`Loan.create!`, `Loan#save`, a factory of the class), with the lines that name it. An `on: :create` validation lists no update, and an `on: :update` one no create. `build(:loan)` does not save, so it counts only in a test that also saves or validates (`.save!`, `be_valid`). Its `tests:` lists the test files that make those calls, next to the grep hits. A method of the app that makes one of those calls (`LoanDesk.open` calling `Loan.create!`) is in `reachable_from`, with its callers, so a test of `LoanDesk.open` names the callback. A save on a parameter, a local, or an instance variable the source does not type counts for the model it is named after: `loan.save!` for `Loan`, but `record.save!` for none. Any other call on such a value, and on a `belongs_to` or `has_one` reader (`borrower.approved_cents` inside `Loan`), is a call edge to that model's method too, so `reachable_from` follows `@loan.approve` in a service up to its controller. A callback in a Concern or a parent class is not followed.
73
+ An executable under `exe/` or `bin/` with no suffix and a `ruby` line after `#!` is read as Ruby. Its code outside a `def`, a class, or a module is one method, `exe/tool <main>` (`kind: "script"` in `--json`), so a change to its usage text or to the `case` that picks a command is listed with its old and new source. Its `tests:` are the test files that write its path, and a candidate that starts it (`IO.popen(["ruby", "exe/tool"])`) mentions it, so a row can name it as its related id.
74
+
75
+ ### Tests, entry points, and readers
76
+
77
+ `tests: (none)` means no test file names that method. Then `entry_point_tests:` lists, when it finds any, the test files that call a public `reachable_from` entry point: the file names the entry point's class (`Cart.new`, not only `Cart::LIMIT`) and calls a method of that name (`.total`). Each file shows the entry points it matched. Read those tests before you write yours. They show how the repository builds the object and which inputs already reach the changed method. Like `tests:`, this line is a grep. It misses tests that drive the code through a subprocess, and it can list a file that calls a same-named method on another object. A `tests:` grep hit on a common name such as `call` can be any file that defines or calls that name. For a reader or member from `attr_reader` or `Data.define` (`Targets::Hit#code`), a hit counts only in a file that also names its class or the namespace around it (`Hit` or `Targets`). The code of a fixture project under `test/` or `spec/` (`test/fixture/depot/app/`, `lib/`, `config/`, `db/`) is not a test file there.
78
+
79
+ In a Rails app, a changed `initialize` shows `built_by:` above `tests:`: `create`, `create!`, and `build` on an Active Record model, and `build(:loan)` and `create(:loan)` for each factory under `spec/factories` or `test/factories` that builds the class. Its `tests:` lists only the test files that call `Owner.new` or one of those, as `spec/models/loan_spec.rb via create(:loan)`, and no grep hit for the word `initialize`. `factories_skipped:` lists each factory whose class the source does not settle (a `class:` computed at run time, an unknown parent, or a class no file defines, as an inflection such as `APIKey` causes). A factory that builds a Ruby value (`class: Hash`, `Array`, `Struct`, `OpenStruct`, and the like) is left out and not listed. A test that builds from one of those names no constructor. A factory also counts for the classes its associations build (`borrower`, `association :borrower`, or `association` in an attribute block), so a changed `Borrower` callback lists `create(:loan)` when `:loan` builds a borrower. An association in a trait lists the call with the trait, `create(:loan, :cosigned)`, and a test that writes `create(:loan, :cosigned)` mentions it; a global trait and a trait that includes another are not followed.
80
+
81
+ A method that a model's class body names in a callback or a validation (`after_save :count_save`, or `if: :checked?` on a `validates`) shows `run_by:`: the calls that run it by Rails's rules (`Loan.create!`, `Loan#save`, a factory of the class), with the lines that name it. An `on: :create` validation lists no update, and an `on: :update` one no create. `build(:loan)` does not save, so it counts only in a test that also saves or validates (`.save!`, `be_valid`). Its `tests:` lists the test files that make those calls, next to the grep hits. A method of the app that makes one of those calls (`LoanDesk.open` calling `Loan.create!`) is in `reachable_from`, with its callers, so a test of `LoanDesk.open` names the callback.
82
+
83
+ A save on a parameter, a local, or an instance variable the source does not type counts for the model it is named after: `loan.save!` for `Loan`, but `record.save!` for none. Any other call on such a value, and on a `belongs_to` or `has_one` reader (`borrower.approved_cents` inside `Loan`), is a call edge to that model's method too, so `reachable_from` follows `@loan.approve` in a service up to its controller. A callback in a Concern or a parent class is not followed.
48
84
 
49
85
  The brief ends with one `entrypoints <Const>:` section for each class or module that holds a changed method. Each line is a public method of that class on head (on base when head has none) and its role: the change kind (`modified`, `added`, `affected`, `removed`) first, then `unchanged`, at most 12 lines. An unchanged method in a changed method's `reachable_from` comes before the other unchanged ones, and `(+3 more)` counts the methods the cap left out. The cap only shortens the list: a method left out still counts as a mention through `reachable_from`. A class that defines `initialize` on either side lists `Owner.new` first in its group instead, with the role of `Owner#initialize`: `Mismatch.new removed` means head dropped the constructor and `.new` now runs the inherited one. A class that makes `new` private (`private_class_method :new`) shows no `Owner.new`; build it through the public method that calls `new`. Pick the methods your tests call from this list. `run` counts these names as mentions too, so calling one gives a row its `touches:unchanged_caller` tag when it reaches a changed method.
50
86
 
@@ -60,7 +96,7 @@ readers: 1 for 1 changed value
60
96
  not seen: receivers named unlike their model (`x = Branch.first; link_to x`); ...
61
97
  ```
62
98
 
63
- For each row, from the top: open the place (`git show aac7821:app/views/branches/show.html.erb`), read `why:` against the intent, and write the candidate `test:` describes. It states what the old code satisfied, so it passes on base: build the record so the old and the new value differ (a `code` that is not the `id`), and drive the entry point the row names (render the view or send the request, then follow the URL). A row whose effect the intent asks for still gets a test: its `reported` result is then `intended`. `rank:` says why a row scores what it does; a low row (`bare STATUSES is defined nowhere visible`, `in a file the change edits`) is the first to skip. `also:` lists more places of the same kind for the same value, and one test usually covers them. `note: this line is itself part of BASE..HEAD` marks a row the author edited, so it was in front of them. The 10 highest rows print, and the first line counts the rest; `--all-readers` prints every one. A row finds a record's model by the name of the variable that holds it (`@branch`, `home_branch`, `branches`). `not seen:` names what no row can show: a reader there needs a `git grep` of your own. A constant's `read_by:` lists the changed methods that read it by name, and a plain read inside one of those is not repeated under `readers:`.
99
+ For each row, from the top: open the place (`git show aac7821:app/views/branches/show.html.erb`), read `why:` against the intent, and write the candidate `test:` describes. It states what the old code satisfied, so it passes on base: build the record so the old and the new value differ (a `code` that is not the `id`), and drive the entry point the row names (render the view or send the request, then follow the URL). A row whose effect the intent asks for still gets a test: its `reported` result is then `intended`. `rank:` says why a row scores what it does; a low row (`bare STATUSES is defined nowhere visible`, `in a file the change edits`) is the first to skip. `also:` lists more places of the same kind for the same value, and one test usually covers them. `note: this line is itself part of BASE..HEAD` marks a row the author edited, so it was in front of them. The 10 highest rows print, and the first line counts the rest; `--all-readers` prints every one. A row finds a record's model by the name of the variable that holds it (`@branch`, `home_branch`, `branches`). `not seen:` names what no row can show: a reader there needs a `git grep` of your own. In a history with no `app/`, `config/routes.rb`, or `config/application.rb`, such as a gem's, it names only run-time dispatch names and config read at load time. A constant's `read_by:` lists the changed methods that read it by name, and a plain read inside one of those is not repeated under `readers:`.
64
100
 
65
101
  `--coverage FILE` replaces the grep of existing tests. The file is JSON: `"format": "jitcatch-coverage-v1"` and a `tests` array of `{ "id", "file", "lines" }`, where `lines` maps repo paths to line numbers. Without it, the brief says `existing_tests grep`.
66
102
 
@@ -189,7 +225,7 @@ Each candidate row under `assess:` ends with `rerun:`, the command that runs tha
189
225
  | Bucket | Meaning |
190
226
  | --- | --- |
191
227
  | `no_signal` | Passed on both sides. Not evidence. |
192
- | `fail_on_base` | The parent did not satisfy the test. Often a new feature. Not a candidate. The text report prints the base outcome under the row (`base: fail RSpec::Expectations::ExpectationNotMetError expected block to output to stdout, but did not`), with the `got:` or `Actual:` line of a value comparison under it, so a test that does not run on base can be fixed. |
228
+ | `fail_on_base` | The parent did not satisfy the test. Often a new feature. Not a candidate. The text report prints the base outcome under the row (`base: fail RSpec::Expectations::ExpectationNotMetError expected block to output to stdout, but did not`), with the `got:` or `Actual:` line of a value comparison under it, so a test that does not run on base can be fixed. A `head:` line follows: `head: pass` for a test that pins what the change fixes, head's own failure, or `head: fail, the same as base`. |
193
229
  | `flaky` | Passed on base, failed on head, then a rerun disagreed. Dropped. The text prints each rerun under the row (`rerun 1: pass`), and the `--json` row keeps them in `reruns`. |
194
230
  | `discarded` | Stable pass-on-base / fail-on-head, removed by a rule. Still listed under `assess:`. |
195
231
  | `reported` | You must judge this one. |
@@ -206,11 +242,11 @@ Tags are hints, not a verdict. They are sorted:
206
242
  - `caller:unchanged:lib/b.rb:5` — the broken call is on that line, which the diff did not change. For example, `class B < A` still calls the private `A#helper` that head removed. Judge it as a likely regression. `caller:changed:FILE:LINE` names a line the diff wrote, or a line inside a changed method, such as an edited lambda in a constant. A row whose error kept no frames keeps `discard:signature_change`.
207
243
  - `discard:load_error` — `LoadError` on head. `discard:timeout` — the base run of a head that did not pass timed out, or the head run timed out beside a base that took more than half of `--timeout` (a run file from before base kept its time counts as that). Timeouts are not rerun. `discard:no_database` — head raised Active Record's `NoDatabaseError`, `ConnectionNotEstablished`, or `DatabaseConnectionError`: the database of head's checkout does not exist or cannot be reached (Rails apps, below). `discard:no_table` — head raised an `ActiveRecord::StatementInvalid` for a missing table: its database exists but has no schema loaded.
208
244
  - The related id (`m05`, or `-` when none) is the downstream modified method when several changed methods sit on the path. When the failure's backtrace runs through changed methods, it is the innermost of them, by each method's lines on head. Read that method's old and new source before you judge.
209
- - `related:guess` — the evidence that picked the related id points at several changed methods, most often because the test names only an unchanged caller (`build`, `between`) that reaches all of them. The next line names every one with its id (`method_ids` in `--json`): `related guess: Shop::Cart#volume_discount_cents (m05), ranked first of 3: Shop::Cart#coupon_discount_cents (m02), Shop::Cart#taxable_cents (m04), Shop::Cart#volume_discount_cents (m05)`. Read each of them before you judge, not only the pick. The pick prefers a method that existed on base, except an edit whose every added line only calls an added method (`forwards_to_added: true` in `--json`): then the added method is the pick. Among the methods an unchanged caller reaches, the pick is the one farthest below the entry point the test calls (`caller_depths` in `--json`), since a test that enters high passes the plumbing near its entry point on the way to the code it checks. When only the id would order the best two, there is no pick: the row shows `-` and the line reads `related guess: none; the test reaches 7 changed methods equally and no frame of head's failure is in one: Book#apply_catalog_record (m02), LoanRules.for (m03), ...` (`method_id: null`), then gives the next step: find what makes head's value in them, or in a changed file that holds no method, such as a view, from the `git diff --stat BASE HEAD` it prints, and `touches:changed_method` there means the test names one of `method_ids`. An assertion on error plumbing, such as the frames a backtrace keeps, often lands here, because the depth ranking favors the code a value comes from. A test that names the method itself (through `send` when it is private) gets a firm id, and so does an error whose backtrace runs through changed methods: the innermost of them is the id.
245
+ - `related:guess` — the evidence that picked the related id points at several changed methods, most often because the test names only an unchanged caller (`build`, `between`) that reaches all of them. The next line names every one with its id (`method_ids` in `--json`): `related guess: Shop::Cart#volume_discount_cents (m05), ranked first of 3: Shop::Cart#coupon_discount_cents (m02), Shop::Cart#taxable_cents (m04), Shop::Cart#volume_discount_cents (m05)`. Read each of them before you judge, not only the pick. The pick prefers a method that existed on base, except an edit whose every added line only calls an added method (`forwards_to_added: true` in `--json`): then the added method is the pick. Among the methods an unchanged caller reaches, the pick is the one farthest below the entry point the test calls (`caller_depths` in `--json`), since a test that enters high passes the plumbing near its entry point on the way to the code it checks. A test that starts a changed script picks the script's `exe/tool <main>` ahead of the methods that code calls, since every start runs it and each method below it runs only for the command that picks it: `related guess: exe/quint_driver <main> (m01), ranked first of 3: exe/quint_driver <main> (m01), QuintDriver::Spec#outline (m10), QuintDriver::Targets.report (m14)` for a test of `quint_driver bogus` whose usage text changed. Read the methods after it when the test runs a command that calls them. On a row where head stopped raising the error the test expects (`X expected but nothing was raised.`), the pool leaves out an edit that only changes the arguments of a `raise` (`edits_only_raise: true`): head no longer reaches that method, so look for the cause in the code that called it. When only the id would order the best two, there is no pick: the row shows `-` and the line reads `related guess: none; the test reaches 7 changed methods equally and no frame of head's failure is in one: Book#apply_catalog_record (m02), LoanRules.for (m03), ...` (`method_id: null`), then gives the next step: find what makes head's value in them, or in a changed file that holds no method, such as a view, from the `git diff --stat BASE HEAD` it prints, and `touches:changed_method` there means the test names one of `method_ids`. An assertion on error plumbing, such as the frames a backtrace keeps, often lands here, because the depth ranking favors the code a value comes from. A test that names the method itself (through `send` when it is private) gets a firm id, and so does an error whose backtrace runs through changed methods: the innermost of them is the id.
210
246
  - `message_varies` — head failed the same way on every rerun with a message that moved, and base passed on every rerun. The finding holds; the value that moved is noise, so a better test compares a field it controls (`exists?`, one column) rather than a whole printed record.
211
- - `related:dynamic` — no changed method is related, but the test calls a method that sends a changed method's name, or a caller's name, to a receiver the source does not name (`EMITTERS.fetch(format).render`, or `lines.sum(&:total_cents)`, which sends `total_cents` to each element). The next line names it: `related: none, Emit.looked_up sends render to a receiver the source does not name`. The test may reach the change at run time; read the method to judge. `context` lists these under `dynamic_callers:`, leaving out the names of instance methods of a class whose name ends in `Controller`, which the router calls and no code sends.
247
+ - `related:dynamic` — no changed method is related, but the test calls a method that sends a changed method's name, or a caller's name, to a receiver the source does not name (`EMITTERS.fetch(format).render`, or `lines.sum(&:total_cents)`, which sends `total_cents` to each element). The next line names it: `related: none, Emit.looked_up sends render to a receiver the source does not name`. The test may reach the change at run time; read the method to judge. `context` lists these under `dynamic_callers:`, leaving out the names of instance methods of a class whose name ends in `Controller`, which the router calls and no code sends. It also leaves out a `call` or `.()` on a `&block` parameter or on a local that every assignment in its method sets to `-> {}`, `lambda {}`, `proc {}`, or `Proc.new {}`: that call runs the block. A `call` on a parameter (`job.call`), a member, or a method's result stays listed, since a service object with `def call` can receive it.
212
248
  - A call through a name computed at run time is not seen at all: `const_get(kind.capitalize)` for the class, or `send(field)`, `public_send(field)`, and `send("#{field}")` for the method. No edge and no mention reaches the changed method, and the row shows `-` with no `touches:` tag and no `related:dynamic`. For `Shop.read("cart", "total")`, where `read` is `build(kind).public_send(field)` and `build` is `const_get(kind.capitalize).new`, a change to `Cart#total` gave `c01 ReadTest#test_reads_a_field_by_name base=pass head=fail - assertion,report`. When `context` shows `callsites: (none)` for a method that the code reaches by name, grep for `send`, `public_send`, and `const_get`, and judge the row from the method you found. A string literal passed to `send` is still a mention (below).
213
- - A mention is a method call, a constant, or a symbol in the test code. Words in strings and comments are not mentions, so an expected string such as `"jitcatch run\n"` does not point the related id at a method named `run`. A method name passed as a plain string to `send`, `__send__`, `public_send`, `method`, `public_method`, or `respond_to?` is one (`send("total")`). A symbol that keys a Hash is not: an argument of `[]`, `[]=`, `fetch`, `dig`, `key?`, `has_key?`, `values_at`, or `fetch_values`, or a key in a hash literal or keyword argument. So `snap[:discount_cents]` does not point the related id at a method `discount_cents`, and `send(:discount_cents)` does. A call to `message`, `full_message`, or `detailed_message` also mentions `to_s`, so a test that reads `error.message` relates to a changed `Mismatch#to_s`. A test that starts a script (`IO.popen(["ruby", "-Ilib", "exe/tool"])`, `system("bin/x")`, or `File.expand_path("../exe/tool", __dir__)`) mentions the names in that script when the string is a relative path to a file of head, so the row can reach `touches:unchanged_caller`. A path built at run time is not read.
249
+ - A mention is a method call, a constant, or a symbol in the test code. Words in strings and comments are not mentions, so an expected string such as `"jitcatch run\n"` does not point the related id at a method named `run`. A method name passed as a plain string to `send`, `__send__`, `public_send`, `method`, `public_method`, or `respond_to?` is one (`send("total")`). A symbol that keys a Hash is not: an argument of `[]`, `[]=`, `fetch`, `dig`, `key?`, `has_key?`, `values_at`, or `fetch_values`, or a key in a hash literal or keyword argument. So `snap[:discount_cents]` does not point the related id at a method `discount_cents`, and `send(:discount_cents)` does. A call to `message`, `full_message`, or `detailed_message` also mentions `to_s`, so a test that reads `error.message` relates to a changed `Mismatch#to_s`. A test that starts a script (`IO.popen(["ruby", "-Ilib", "exe/tool"])`, `system("bin/x")`, or `File.expand_path("../exe/tool", __dir__)`) mentions the names in that script when the string is a relative path to a file of head, so the row can reach `touches:unchanged_caller`. A path that an assertion or a matcher compares directly (`assert_includes error.backtrace.first, "lib/tool/model.rb"`) is expected text, not a script. A path built at run time is not read.
214
250
 
215
251
  ## Judge
216
252
 
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: jitcatch
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - meganemura