jitcatch 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.ja.md +30 -5
- data/README.md +22 -4
- data/lib/jitcatch/changes.rb +8 -3
- data/lib/jitcatch/failure.rb +8 -4
- data/lib/jitcatch/git_repo.rb +35 -1
- data/lib/jitcatch/render.rb +12 -1
- data/lib/jitcatch/runner.rb +48 -16
- data/lib/jitcatch/version.rb +1 -1
- data/skills/jitcatch/SKILL.md +3 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 246b8f17eec37c3c6693d66fde632340998fc2102245ed422ddca14b44c54fb5
|
|
4
|
+
data.tar.gz: 587e7588d600b8a62e5537d3d59aac1d6df00665b9b8787a0ea7690826ba5bb6
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: de57a2768bc59e4636d1484e1563eb6c4b0d264235c92db6b730b9271da9bee67fc8c4bb006a0b96b1938947808f51ba4f88347ebdf470d0f8f4ce270838c742
|
|
7
|
+
data.tar.gz: fd561550f30e2a11fd4a7eb457ae46568fbc8f5302f3bc651f55bbdef6fc42e7a0bd1df94d8564e6cd306ed09936dad93394bb94a7f6a683d66535b0679f6c83
|
data/README.ja.md
CHANGED
|
@@ -75,7 +75,7 @@ jitcatch は次を出す。
|
|
|
75
75
|
<!-- rails:output -->
|
|
76
76
|
```
|
|
77
77
|
jitcatch run
|
|
78
|
-
version 0.2.
|
|
78
|
+
version 0.2.1
|
|
79
79
|
base e994a60bff5a85b3bcc92960dd251d598b15835a
|
|
80
80
|
head c3ef8997c50fc389a24ec4aa3f79caca2d6437b9
|
|
81
81
|
reruns 2
|
|
@@ -172,7 +172,7 @@ jitcatch は次を出す。
|
|
|
172
172
|
<!-- quickstart:output -->
|
|
173
173
|
```
|
|
174
174
|
jitcatch run
|
|
175
|
-
version 0.2.
|
|
175
|
+
version 0.2.1
|
|
176
176
|
base 81b05b22f14f2b5a1e580c048856a5e9d9b25b85
|
|
177
177
|
head 7c53dd7485d4c994d97bddd3cb7bd5481a337b61
|
|
178
178
|
reruns 2
|
|
@@ -233,8 +233,8 @@ jitcatch はそのテストを両方のリビジョンで走らせ、フレー
|
|
|
233
233
|
Ruby 3.3 以降。
|
|
234
234
|
追加の gem はない。
|
|
235
235
|
`minitest` と `prism` は Ruby に付いている。
|
|
236
|
-
`jitcatch --version`(または `-v`)は `jitcatch 0.2.
|
|
237
|
-
git から入れた gem のように jitcatch 自身の git のチェックアウトから動くときは、`jitcatch 0.2.
|
|
236
|
+
`jitcatch --version`(または `-v`)は `jitcatch 0.2.1` を出し、`--help` はコマンドの一覧を出す。
|
|
237
|
+
git から入れた gem のように jitcatch 自身の git のチェックアウトから動くときは、`jitcatch 0.2.1 (1fbcd18)` のようにコミットも出す。
|
|
238
238
|
RSpec の候補には、実行する場所に `rspec-core` が要る。
|
|
239
239
|
プロジェクトの bundle に入れるか、`Gemfile` が無いプロジェクトならその Ruby にインストールしておく。
|
|
240
240
|
`BASE` は親のリビジョン、`HEAD` は変更である。
|
|
@@ -380,12 +380,37 @@ jitcatch context --repo DIR BASE HEAD
|
|
|
380
380
|
jitcatch run --repo DIR BASE HEAD TESTS_DIR
|
|
381
381
|
```
|
|
382
382
|
|
|
383
|
+
### 大きなリポジトリのディレクトリにある gem
|
|
384
|
+
|
|
385
|
+
gem がリポジトリの中のディレクトリにあり、そこに自分の `lib/`、`Gemfile`、`Gemfile.lock` を持つときは、そのディレクトリから jitcatch を実行する。
|
|
386
|
+
たとえば `ruby/` なら次のようにする。
|
|
387
|
+
|
|
388
|
+
```sh
|
|
389
|
+
cd ruby
|
|
390
|
+
jitcatch run BASE HEAD catch
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
すると、各候補、バンドルの検査、`--setup`、各 `rerun:` 行は、それぞれのチェックアウトの `ruby/` で実行される。
|
|
394
|
+
`-Ilib` は `ruby/lib` を指し、`BUNDLE_GEMFILE` は `ruby/Gemfile` を指し、Bundler の設定は `ruby/.bundle/config` から読む。
|
|
395
|
+
`TESTS_DIR` と再実行の行は `ruby/` からのパスで書くので、再実行の行は jitcatch を実行した場所から動く。
|
|
396
|
+
リポジトリのルートから `--repo ruby` を渡しても同じ実行になる。
|
|
397
|
+
レポートは `head` の下でそれを示す。
|
|
398
|
+
|
|
399
|
+
```text
|
|
400
|
+
head 6af3acfb1a733406d202eaed737b7dabfbece7fa
|
|
401
|
+
directory ruby (candidates, the bundle check, and each rerun line run here; file paths are from the repository root)
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
差分、変更されたメソッド、バックトレースのフレームは、git が書くとおりリポジトリのルートからのパスのままである (`ruby/lib/shelf.rb`)。
|
|
405
|
+
`context` はロックされた gem を `ruby/Gemfile.lock` から読む。
|
|
406
|
+
`context` の Rails の部分 (ルート、モデル、コントローラ、`readers:`) と、`exe/` と `bin/` の下のスクリプトは、リポジトリのルートにあるものだけを読む。
|
|
407
|
+
|
|
383
408
|
### オプション
|
|
384
409
|
|
|
385
410
|
| フラグ | コマンド | 効果 |
|
|
386
411
|
| --- | --- | --- |
|
|
387
412
|
| `--json` | `context`, `run`, `assess` | 同じ内容を JSON で出す。 |
|
|
388
|
-
| `--repo DIR` | `context`, `run` | 読む git
|
|
413
|
+
| `--repo DIR` | `context`, `run` | 読む git リポジトリ、またはその中の実行するディレクトリ (上を参照)。既定は作業ディレクトリ。 |
|
|
389
414
|
| `--all-readers` | `context` | `readers:` の行を、スコアの高い 10 行だけでなく全部出す。 |
|
|
390
415
|
| `--coverage FILE` | `context` | 既存テストの grep の代わりにこのファイルを使う。JSON は `"format": "jitcatch-coverage-v1"` と `tests` 配列を持つ。例は `examples/coverage/jitcatch-coverage-v1.sample.json`。 |
|
|
391
416
|
| `--reruns N` | `run` | base で通り head で落ちたあとの、head での追加実行回数。既定は 2。`0` でフレーク検査をしない。 |
|
data/README.md
CHANGED
|
@@ -63,7 +63,7 @@ The last commit asks a senior officer from 10000 cents, where it used to ask onl
|
|
|
63
63
|
<!-- rails:output -->
|
|
64
64
|
```
|
|
65
65
|
jitcatch run
|
|
66
|
-
version 0.2.
|
|
66
|
+
version 0.2.1
|
|
67
67
|
base e994a60bff5a85b3bcc92960dd251d598b15835a
|
|
68
68
|
head c3ef8997c50fc389a24ec4aa3f79caca2d6437b9
|
|
69
69
|
reruns 2
|
|
@@ -145,7 +145,7 @@ jitcatch prints this. The two commit ids are your own.
|
|
|
145
145
|
<!-- quickstart:output -->
|
|
146
146
|
```
|
|
147
147
|
jitcatch run
|
|
148
|
-
version 0.2.
|
|
148
|
+
version 0.2.1
|
|
149
149
|
base 81b05b22f14f2b5a1e580c048856a5e9d9b25b85
|
|
150
150
|
head 7c53dd7485d4c994d97bddd3cb7bd5481a337b61
|
|
151
151
|
reruns 2
|
|
@@ -193,7 +193,7 @@ A coding agent reviewing a change writes tests for that diff. A test that only d
|
|
|
193
193
|
|
|
194
194
|
The method follows Meta's catching JiT tests ([Harden and Catch](https://arxiv.org/abs/2504.16472); [Just-in-Time Catching Test Generation at Meta](https://arxiv.org/abs/2601.22832)).
|
|
195
195
|
|
|
196
|
-
Ruby 3.3 or newer. No gems. `minitest` and `prism` ship with Ruby. `jitcatch --version` (or `-v`) prints `jitcatch 0.2.
|
|
196
|
+
Ruby 3.3 or newer. No gems. `minitest` and `prism` ship with Ruby. `jitcatch --version` (or `-v`) prints `jitcatch 0.2.1`, followed by the commit, as in `jitcatch 0.2.1 (1fbcd18)`, when jitcatch runs from its own git checkout, as a gem installed from git does, and `--help` prints the commands. RSpec candidates need `rspec-core` where they run: in the project's bundle, or installed for that Ruby when the project has no `Gemfile`. `BASE` is the parent revision and `HEAD` is the change. Each must resolve to a commit. Uncommitted files are not in the diff.
|
|
197
197
|
|
|
198
198
|
You write the candidate files in a scratch directory. A Minitest file starts with `require "minitest/autorun"`, then requires the library the way the repository loads it. An RSpec file is named `*_spec.rb` and starts with `require "spec_helper"` or `require "rails_helper"`, as the project's own specs do. jitcatch runs those files. You judge each `reported` row against the intent and the diff. The intent in the brief is the commit messages from `BASE` to `HEAD`, without the trailer lines git parses at the end of each message (such as `Co-Authored-By:`). likely-regression means the behavior changed and the intent does not ask for that change. intended means the intent asks for it. unclear means the intent is not enough to tell.
|
|
199
199
|
|
|
@@ -299,12 +299,30 @@ jitcatch context --repo DIR BASE HEAD
|
|
|
299
299
|
jitcatch run --repo DIR BASE HEAD TESTS_DIR
|
|
300
300
|
```
|
|
301
301
|
|
|
302
|
+
### A gem in a directory of a larger repository
|
|
303
|
+
|
|
304
|
+
When the gem sits in a directory of the repository, such as `ruby/` with its own `lib/`, `Gemfile`, and `Gemfile.lock`, run jitcatch from that directory:
|
|
305
|
+
|
|
306
|
+
```sh
|
|
307
|
+
cd ruby
|
|
308
|
+
jitcatch run BASE HEAD catch
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Each candidate, the bundle check, `--setup`, and each `rerun:` line then run in `ruby/` of each checkout. `-Ilib` is `ruby/lib`, `BUNDLE_GEMFILE` is `ruby/Gemfile`, and the Bundler settings are read from `ruby/.bundle/config`. `TESTS_DIR` and the rerun line are named from `ruby/`, so the rerun line runs from where you ran jitcatch. `--repo ruby` from the repository root is the same run. The report says so under `head`:
|
|
312
|
+
|
|
313
|
+
```text
|
|
314
|
+
head 6af3acfb1a733406d202eaed737b7dabfbece7fa
|
|
315
|
+
directory ruby (candidates, the bundle check, and each rerun line run here; file paths are from the repository root)
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
The diff, the changed methods, and the backtrace frames keep their paths from the repository root (`ruby/lib/shelf.rb`), as git names them. `context` reads the locked gems from `ruby/Gemfile.lock`. The Rails parts of `context` (routes, models, controllers, and `readers:`) and the scripts under `exe/` and `bin/` are read only at the repository root.
|
|
319
|
+
|
|
302
320
|
### Options
|
|
303
321
|
|
|
304
322
|
| Flag | Commands | Effect |
|
|
305
323
|
| --- | --- | --- |
|
|
306
324
|
| `--json` | `context`, `run`, `assess` | Print the same document as JSON. |
|
|
307
|
-
| `--repo DIR` | `context`, `run` | Git repository to read. The default is the working directory. |
|
|
325
|
+
| `--repo DIR` | `context`, `run` | Git repository to read, or a directory inside it to run from (see above). The default is the working directory. |
|
|
308
326
|
| `--all-readers` | `context` | Print every row under `readers:`, not the 10 that score highest. |
|
|
309
327
|
| `--coverage FILE` | `context` | Replace the grep of existing tests with this file. The JSON has `"format": "jitcatch-coverage-v1"` and a `tests` array. See `examples/coverage/jitcatch-coverage-v1.sample.json`. |
|
|
310
328
|
| `--reruns N` | `run` | Extra runs on head after a pass-on-base / fail-on-head result. The default is 2. `0` skips the flake check. |
|
data/lib/jitcatch/changes.rb
CHANGED
|
@@ -121,15 +121,18 @@ module JitCatch
|
|
|
121
121
|
# a change to it, so a row can fail on a missing column and read as a
|
|
122
122
|
# regression. A helper that calls maintain_test_schema! loads the
|
|
123
123
|
# schema of the checkout it boots in; without one, only --setup can.
|
|
124
|
+
# The app is the directory jitcatch runs in, so its schema and helpers
|
|
125
|
+
# sit under the repository path of that directory.
|
|
124
126
|
def schema_warnings(head, files)
|
|
125
|
-
|
|
127
|
+
prefix = @git.prefix
|
|
128
|
+
changed = files.map { |file| file["file"] }.select { |path| path.start_with?(prefix) && path.delete_prefix(prefix).match?(SCHEMA_FILE) }.sort
|
|
126
129
|
return [] if changed.empty?
|
|
127
130
|
|
|
128
131
|
warning = "head changes the database schema (#{changed.join(", ")}), so each side's test database must hold that side's schema, " \
|
|
129
132
|
"or a row can fail on the database and not on the code."
|
|
130
133
|
return ["#{warning} `--setup` ran `#{@setup}` in each checkout before the candidates."] if @setup
|
|
131
134
|
|
|
132
|
-
helpers = TEST_HELPERS.select { |path| @git.show(head, path) }
|
|
135
|
+
helpers = TEST_HELPERS.map { |path| "#{prefix}#{path}" }.select { |path| @git.show(head, path) }
|
|
133
136
|
return [warning] if helpers.empty? || helpers.any? { |path| @git.show(head, path).include?("maintain_test_schema!") }
|
|
134
137
|
|
|
135
138
|
databases = @databases.each_slice(2).map { |base, head_db| "base #{base}, head #{head_db}" }
|
|
@@ -140,6 +143,8 @@ module JitCatch
|
|
|
140
143
|
|
|
141
144
|
# Each spec line in a lockfile is four spaces, a name, and the version
|
|
142
145
|
# in parentheses. A gem locked for several platforms has several lines.
|
|
146
|
+
# The lockfile is the one beside the Gemfile of the directory jitcatch
|
|
147
|
+
# runs in.
|
|
143
148
|
def dependencies(base, head)
|
|
144
149
|
old = locked_versions(base)
|
|
145
150
|
new = locked_versions(head)
|
|
@@ -151,7 +156,7 @@ module JitCatch
|
|
|
151
156
|
end
|
|
152
157
|
|
|
153
158
|
def locked_versions(sha)
|
|
154
|
-
@git.show(sha, "Gemfile.lock").to_s.scan(/^ (\S+) \(([^)]+)\)$/)
|
|
159
|
+
@git.show(sha, "#{@git.prefix}Gemfile.lock").to_s.scan(/^ (\S+) \(([^)]+)\)$/)
|
|
155
160
|
.group_by(&:first)
|
|
156
161
|
.transform_values { |pairs| pairs.map(&:last).uniq.sort.join(" ") }
|
|
157
162
|
end
|
data/lib/jitcatch/failure.rb
CHANGED
|
@@ -38,14 +38,18 @@ module JitCatch
|
|
|
38
38
|
# +tests+, the tests directory from the repository root, so a message
|
|
39
39
|
# names it after the root, and a frame names it, where either named the
|
|
40
40
|
# copy. An absolute +tests+ is outside every worktree, so it stands alone.
|
|
41
|
-
|
|
41
|
+
# +prefix+ is the directory the run ran in, from the worktree root, as
|
|
42
|
+
# `ruby/`, where the copy sits and from where +tests+ is named.
|
|
43
|
+
def naming_tests(result, tests, prefix = "")
|
|
42
44
|
return nil if result.nil?
|
|
43
45
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
+
copy = prefix.empty? ? TESTS_COPY : /#{Regexp.escape("#{WORKTREE}#{prefix}")}\.jitcatch(?![\w.-])/
|
|
47
|
+
copy_frame = prefix.empty? ? COPY_FRAME : %r{\A(?:#{Regexp.escape(prefix)})?\.jitcatch/}
|
|
48
|
+
rooted = tests.start_with?("/") ? tests : "#{WORKTREE}#{prefix}#{tests}"
|
|
49
|
+
named = ->(text) { text.to_s.gsub(copy) { rooted }.gsub(COPY_PATH) { "#{$1}#{tests}/" } }
|
|
46
50
|
message = named.call(result["message"])
|
|
47
51
|
cause = result["cause"] && result["cause"].merge("message" => named.call(result["cause"]["message"]))
|
|
48
|
-
backtrace = Array(result["backtrace"]).map { |frame| frame.merge("file" => frame["file"].sub(
|
|
52
|
+
backtrace = Array(result["backtrace"]).map { |frame| frame.merge("file" => frame["file"].sub(copy_frame) { "#{tests}/" }) }
|
|
49
53
|
result.merge("message" => message, "message_first_line" => first_line(message), "cause" => cause, "backtrace" => backtrace)
|
|
50
54
|
end
|
|
51
55
|
|
data/lib/jitcatch/git_repo.rb
CHANGED
|
@@ -5,6 +5,10 @@ require "fileutils"
|
|
|
5
5
|
|
|
6
6
|
module JitCatch
|
|
7
7
|
# Thin git wrapper. Every invocation is an argv array, never a shell.
|
|
8
|
+
# `dir` is the directory the reader works in, which can be a directory
|
|
9
|
+
# inside the repository, as a gem's `ruby/` is in a monorepo. git runs
|
|
10
|
+
# from the repository's top, so every path it prints or takes is from
|
|
11
|
+
# there, the way the diff names them.
|
|
8
12
|
class GitRepo
|
|
9
13
|
attr_reader :dir
|
|
10
14
|
|
|
@@ -13,6 +17,19 @@ module JitCatch
|
|
|
13
17
|
raise UsageError, "not a directory: #{path}" unless File.directory?(@dir)
|
|
14
18
|
end
|
|
15
19
|
|
|
20
|
+
# The path of `dir` from the repository's top, with a trailing slash,
|
|
21
|
+
# as `ruby/`, or "" at the top.
|
|
22
|
+
def prefix
|
|
23
|
+
locate
|
|
24
|
+
@prefix
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# The top of the repository, as git prints it.
|
|
28
|
+
def toplevel
|
|
29
|
+
locate
|
|
30
|
+
@toplevel
|
|
31
|
+
end
|
|
32
|
+
|
|
16
33
|
def rev(ref)
|
|
17
34
|
git("rev-parse", "--verify", "--end-of-options", "#{ref}^{commit}").strip
|
|
18
35
|
end
|
|
@@ -163,7 +180,24 @@ module JitCatch
|
|
|
163
180
|
# git quotes a path with non-ASCII bytes by default, and a quoted path
|
|
164
181
|
# names no file in `show` and matches no hunk path.
|
|
165
182
|
def capture(*args, env: nil)
|
|
166
|
-
|
|
183
|
+
capture_in(toplevel, *args, env: env)
|
|
184
|
+
end
|
|
185
|
+
|
|
186
|
+
# From a directory inside the repository, `ls-tree` and `grep` list only
|
|
187
|
+
# that directory, by paths from it, which name no file of the diff.
|
|
188
|
+
def locate
|
|
189
|
+
return if @toplevel
|
|
190
|
+
|
|
191
|
+
stdout, stderr, status = capture_in(@dir, "rev-parse", "--show-toplevel", "--show-prefix")
|
|
192
|
+
raise GitError, "git rev-parse failed: #{stderr.strip}" unless status.success?
|
|
193
|
+
|
|
194
|
+
top, prefix = stdout.lines.map(&:chomp)
|
|
195
|
+
@prefix = prefix.to_s
|
|
196
|
+
@toplevel = top
|
|
197
|
+
end
|
|
198
|
+
|
|
199
|
+
def capture_in(dir, *args, env: nil)
|
|
200
|
+
command = ["git", "--no-pager", "-c", "core.quotePath=false", "-C", dir, *args]
|
|
167
201
|
merged = ENV.to_h.merge(env || {})
|
|
168
202
|
stdout, stderr, status = Open3.capture3(merged, *command)
|
|
169
203
|
[as_utf8(stdout), as_utf8(stderr), status]
|
data/lib/jitcatch/render.rb
CHANGED
|
@@ -96,6 +96,7 @@ module JitCatch
|
|
|
96
96
|
lines << "version #{doc["version"]}"
|
|
97
97
|
lines << "base #{doc["base"]}"
|
|
98
98
|
lines << "head #{doc["head"]}"
|
|
99
|
+
lines.concat(directory_lines(doc))
|
|
99
100
|
lines.concat(warning_lines(doc))
|
|
100
101
|
lines.concat(no_table_warning_lines(doc))
|
|
101
102
|
lines << "reruns #{doc["reruns"]}"
|
|
@@ -140,6 +141,7 @@ module JitCatch
|
|
|
140
141
|
end]
|
|
141
142
|
lines << "base #{doc["base"]}"
|
|
142
143
|
lines << "head #{doc["head"]}"
|
|
144
|
+
lines.concat(directory_lines(doc))
|
|
143
145
|
lines << "ruby #{doc.dig("ruby", "version")} #{doc.dig("ruby", "path")}"
|
|
144
146
|
doc["setup_failed"].each do |failure|
|
|
145
147
|
lines << "#{load ? "load" : "setup"} #{failure["side"]} failed: #{failure["command"]}"
|
|
@@ -153,7 +155,7 @@ module JitCatch
|
|
|
153
155
|
"Run that command in a checkout of that side, with its TEST_ENV_NUMBER and the variables --env passes, fix what it says, and run jitcatch again."
|
|
154
156
|
elsif load
|
|
155
157
|
rails = " A file git ignores, such as config/master.key, is not in the checkouts jitcatch makes; pass its value with --env, such as RAILS_MASTER_KEY." if doc["rails_app"]
|
|
156
|
-
"Run that command from the repository root with that side checked out, fix what it says (often a database, a credential, or a service the app needs at boot), and run jitcatch again.#{rails}"
|
|
158
|
+
"Run that command from #{doc["directory"] ? "#{doc["directory"]}/ in the repository" : "the repository root"} with that side checked out, fix what it says (often a database, a credential, or a service the app needs at boot), and run jitcatch again.#{rails}"
|
|
157
159
|
else
|
|
158
160
|
"Run that command with that Ruby in a checkout of that side, fix what it says (often `bundle install` there; a bundle installed for another Ruby reads as gems not installed), and run jitcatch again."
|
|
159
161
|
end
|
|
@@ -166,12 +168,21 @@ module JitCatch
|
|
|
166
168
|
lines << "version #{doc["version"]}"
|
|
167
169
|
lines << "base #{doc["base"]}"
|
|
168
170
|
lines << "head #{doc["head"]}"
|
|
171
|
+
lines.concat(directory_lines(doc))
|
|
169
172
|
lines.concat(warning_lines(doc))
|
|
170
173
|
lines << counts_line(doc["counts"])
|
|
171
174
|
lines.concat(assess_body(doc))
|
|
172
175
|
finish(lines)
|
|
173
176
|
end
|
|
174
177
|
|
|
178
|
+
# A run from a directory inside the repository says which, since its
|
|
179
|
+
# rerun lines and its tests directory are named from there, while each
|
|
180
|
+
# file of the change is named from the repository root.
|
|
181
|
+
def directory_lines(doc)
|
|
182
|
+
dir = doc["directory"]
|
|
183
|
+
dir ? ["directory #{dir} (candidates, the bundle check, and each rerun line run here; file paths are from the repository root)"] : []
|
|
184
|
+
end
|
|
185
|
+
|
|
175
186
|
def assess_json(doc)
|
|
176
187
|
{
|
|
177
188
|
"command" => "assess",
|
data/lib/jitcatch/runner.rb
CHANGED
|
@@ -138,20 +138,26 @@ module JitCatch
|
|
|
138
138
|
changes = Changes.new(@git, @base_ref, @head_ref, setup: @setup, databases: numbered_databases)
|
|
139
139
|
change = changes.build
|
|
140
140
|
parent = Dir.mktmpdir("jitcatch-")
|
|
141
|
-
|
|
141
|
+
checkouts = (1..@pair_count).map { |n|
|
|
142
142
|
Pair.new(base: File.join(parent, n.to_s, "base"), head: File.join(parent, n.to_s, "head"))
|
|
143
143
|
}
|
|
144
|
+
# Every process runs in the checkout's copy of the directory jitcatch
|
|
145
|
+
# was run from, as `ruby/` of a monorepo, so its lib/ and Gemfile are
|
|
146
|
+
# the ones the reader's own commands would use.
|
|
147
|
+
pairs = checkouts.map { |pair| Pair.new(base: inside(pair.base), head: inside(pair.head)) }
|
|
144
148
|
# The variable parallel_tests sets: a Rails database.yml that appends
|
|
145
149
|
# it to the database name gives each checkout its own database, so base
|
|
146
150
|
# and head never load their schemas into one database at the same time.
|
|
147
151
|
@test_env_numbers = pairs.each_with_index.flat_map { |pair, index| [[pair.base, (2 * index + 1).to_s], [pair.head, (2 * index + 2).to_s]] }.to_h
|
|
148
152
|
begin
|
|
149
153
|
# One at a time: git takes a lock on the repository for each add.
|
|
150
|
-
pairs.each { |pair| check_out(pair, change) }
|
|
154
|
+
checkouts.zip(pairs).each { |checkout, pair| check_out(checkout, pair, change) }
|
|
151
155
|
rails_app = File.file?(File.join(pairs.first.head, "config", "application.rb"))
|
|
152
156
|
# A Rails app boots and loads db/schema.rb in each new checkout.
|
|
153
157
|
@timeout ||= rails_app ? 60 : 10
|
|
154
|
-
|
|
158
|
+
# Only the checkout's top is cut, so a frame keeps the path the diff
|
|
159
|
+
# names, as ruby/lib/gem.rb.
|
|
160
|
+
normalizer = build_normalizer([*checkouts.flat_map { |pair| [pair.base, pair.head] }, @git.toplevel, top_of(@git.dir)])
|
|
155
161
|
first = pairs.first
|
|
156
162
|
frameworks = @frameworks.values.uniq
|
|
157
163
|
base_setup = Thread.new { setup_status("base", change["base"], first.base, normalizer, frameworks) }
|
|
@@ -179,7 +185,7 @@ module JitCatch
|
|
|
179
185
|
unloaded = load_failure_doc(change, rows, bundled, rails_app)
|
|
180
186
|
raise EnvironmentError, unloaded if unloaded
|
|
181
187
|
ensure
|
|
182
|
-
|
|
188
|
+
checkouts.each do |pair|
|
|
183
189
|
@git.worktree_remove(pair.base)
|
|
184
190
|
@git.worktree_remove(pair.head)
|
|
185
191
|
end
|
|
@@ -192,6 +198,7 @@ module JitCatch
|
|
|
192
198
|
"command" => "run",
|
|
193
199
|
"base" => change["base"],
|
|
194
200
|
"head" => change["head"],
|
|
201
|
+
**directory,
|
|
195
202
|
"warnings" => change["warnings"] + require_warnings(files) + helper_warnings + [*(shared_database_warnings(shared) if shared)],
|
|
196
203
|
"reruns" => @reruns,
|
|
197
204
|
"timeout_s" => @timeout,
|
|
@@ -215,19 +222,37 @@ module JitCatch
|
|
|
215
222
|
|
|
216
223
|
private
|
|
217
224
|
|
|
218
|
-
|
|
225
|
+
# A side that has no copy of the directory yet, as the base of the
|
|
226
|
+
# commit that adds it, gets an empty one, as the empty tree does.
|
|
227
|
+
def check_out(checkout, pair, change)
|
|
219
228
|
if @git.empty_tree?(change["base"])
|
|
220
|
-
FileUtils.mkdir_p(
|
|
229
|
+
FileUtils.mkdir_p(checkout.base)
|
|
221
230
|
else
|
|
222
|
-
@git.worktree_add(
|
|
231
|
+
@git.worktree_add(checkout.base, change["base"])
|
|
223
232
|
end
|
|
224
|
-
@git.worktree_add(
|
|
233
|
+
@git.worktree_add(checkout.head, change["head"])
|
|
225
234
|
[pair.base, pair.head].each do |worktree|
|
|
235
|
+
FileUtils.mkdir_p(worktree)
|
|
226
236
|
install(worktree)
|
|
227
237
|
copy_bundler_settings(worktree)
|
|
228
238
|
end
|
|
229
239
|
end
|
|
230
240
|
|
|
241
|
+
def inside(checkout)
|
|
242
|
+
@git.prefix.empty? ? checkout : File.join(checkout, @git.prefix.chomp("/"))
|
|
243
|
+
end
|
|
244
|
+
|
|
245
|
+
# The checkout a directory from `inside` is in.
|
|
246
|
+
def top_of(dir)
|
|
247
|
+
@git.prefix.empty? ? dir : dir.delete_suffix("/#{@git.prefix.chomp("/")}")
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
# Said only from a directory inside the repository, so a run from its
|
|
251
|
+
# top keeps the same bytes.
|
|
252
|
+
def directory
|
|
253
|
+
@git.prefix.empty? ? {} : { "directory" => @git.prefix.chomp("/") }
|
|
254
|
+
end
|
|
255
|
+
|
|
231
256
|
# Each worker takes the next file and runs it in its own pair, so the
|
|
232
257
|
# rows do not depend on which worker ran which file.
|
|
233
258
|
def classify_files(files, pairs, normalizer, known)
|
|
@@ -423,9 +448,9 @@ module JitCatch
|
|
|
423
448
|
|
|
424
449
|
parsed = framework.parse(File.file?(report_path) ? read_text(report_path) : nil, stdout, stderr, normalizer)
|
|
425
450
|
Result.new(
|
|
426
|
-
tests: parsed["tests"].transform_values { |result| Failure.naming_tests(result, tests_from_root) },
|
|
451
|
+
tests: parsed["tests"].transform_values { |result| Failure.naming_tests(result, tests_from_root, @git.prefix) },
|
|
427
452
|
lines: parsed["lines"],
|
|
428
|
-
file_error: Failure.naming_tests(parsed["file_error"], tests_from_root),
|
|
453
|
+
file_error: Failure.naming_tests(parsed["file_error"], tests_from_root, @git.prefix),
|
|
429
454
|
timed_out: false,
|
|
430
455
|
exit_code: status&.exitstatus,
|
|
431
456
|
seconds: seconds
|
|
@@ -636,6 +661,7 @@ module JitCatch
|
|
|
636
661
|
"ruby" => { "version" => RUBY_VERSION, "path" => RbConfig.ruby },
|
|
637
662
|
"base" => change["base"],
|
|
638
663
|
"head" => change["head"],
|
|
664
|
+
**directory,
|
|
639
665
|
"setup_failed" => failed
|
|
640
666
|
}
|
|
641
667
|
end
|
|
@@ -655,7 +681,7 @@ module JitCatch
|
|
|
655
681
|
end
|
|
656
682
|
|
|
657
683
|
def network_library?(head, candidates)
|
|
658
|
-
locked = @git.show(head, "Gemfile.lock").to_s.scan(/^ (\S+) \(/).flatten
|
|
684
|
+
locked = @git.show(head, "#{@git.prefix}Gemfile.lock").to_s.scan(/^ (\S+) \(/).flatten
|
|
659
685
|
(locked & NETWORK_GEMS).any? || [@git.ruby_matches(head, NETWORK_USE_GREP), *candidates].any? { |text| text.match?(NETWORK_USE) }
|
|
660
686
|
end
|
|
661
687
|
|
|
@@ -775,9 +801,11 @@ module JitCatch
|
|
|
775
801
|
# from the candidate's directory, or from the candidate file as
|
|
776
802
|
# `File.expand_path` reads it, names such a script when it is a file of
|
|
777
803
|
# head outside `.jitcatch/`; the names it refers to count as mentions.
|
|
804
|
+
# The test runs in head_dir, and the script is named from the
|
|
805
|
+
# checkout's top, as the diff names it.
|
|
778
806
|
def script_names(paths, relative, head_dir)
|
|
779
807
|
candidate = File.join(head_dir, ".jitcatch", relative)
|
|
780
|
-
root = File.join(head_dir, "")
|
|
808
|
+
root = File.join(top_of(head_dir), "")
|
|
781
809
|
own = File.join(head_dir, ".jitcatch", "")
|
|
782
810
|
paths.flat_map { |path|
|
|
783
811
|
[head_dir, File.dirname(candidate), candidate].map { |dir| File.expand_path(path, dir) }
|
|
@@ -822,7 +850,10 @@ module JitCatch
|
|
|
822
850
|
end
|
|
823
851
|
|
|
824
852
|
path = required.path.end_with?(".rb") ? required.path : "#{required.path}.rb"
|
|
825
|
-
|
|
853
|
+
# The run's directory sits at its place under the checkout's top,
|
|
854
|
+
# so a path that climbs out of it lands where it would in a checkout.
|
|
855
|
+
here = File.join("/", @git.prefix)
|
|
856
|
+
in_run = Pathname(File.expand_path(path, File.dirname(File.join(here, ".jitcatch", relative)))).relative_path_from(Pathname(here)).to_s
|
|
826
857
|
in_rerun = from_root(File.expand_path(path, File.dirname(File.join(tests, relative))))
|
|
827
858
|
next if in_run.start_with?(".jitcatch/") || in_run == in_rerun
|
|
828
859
|
|
|
@@ -849,9 +880,10 @@ module JitCatch
|
|
|
849
880
|
system(env, RbConfig.ruby, "-e", script, chdir: @git.dir, unsetenv_others: true, out: File::NULL, err: File::NULL) == true
|
|
850
881
|
end
|
|
851
882
|
|
|
852
|
-
# The rerun line runs from the
|
|
853
|
-
#
|
|
854
|
-
#
|
|
883
|
+
# The rerun line runs from the directory jitcatch ran in (the repository
|
|
884
|
+
# root, or a directory inside it), so the tests directory is named by
|
|
885
|
+
# its path from there, outside the repository too: a quoted line then
|
|
886
|
+
# names no directory of one machine.
|
|
855
887
|
def tests_from_root
|
|
856
888
|
@tests_from_root ||= from_root(File.realpath(@tests_dir))
|
|
857
889
|
end
|
data/lib/jitcatch/version.rb
CHANGED
data/skills/jitcatch/SKILL.md
CHANGED
|
@@ -13,7 +13,7 @@ Do not commit the candidate tests unless the human asks to promote one.
|
|
|
13
13
|
|
|
14
14
|
Ruby 3.3 or newer, and git. No gems. `minitest` and `prism` ship with Ruby. RSpec candidates need `rspec-core`: in the project's bundle when it has a `Gemfile`, otherwise installed for the Ruby that runs jitcatch. The subject must be a git repository that can add worktrees. The tool reads commits only. Uncommitted files are not in the diff.
|
|
15
15
|
|
|
16
|
-
`gem install jitcatch` installs the `jitcatch` command, with no runtime dependency. In an app with a `Gemfile`, such as a Rails app, the gem can sit in the `:development, :test` group instead (`gem "jitcatch", require: false`); run it as `bundle exec jitcatch`. Each side still loads its own bundle, so the line does not need to be in either commit. From a checkout of the jitcatch repository, put its `bin` directory on `PATH`, or run `ruby /path/to/jitcatch/bin/jitcatch`. `jitcatch --help` prints the commands, and `jitcatch --version` prints `jitcatch 0.2.
|
|
16
|
+
`gem install jitcatch` installs the `jitcatch` command, with no runtime dependency. In an app with a `Gemfile`, such as a Rails app, the gem can sit in the `:development, :test` group instead (`gem "jitcatch", require: false`); run it as `bundle exec jitcatch`. Each side still loads its own bundle, so the line does not need to be in either commit. From a checkout of the jitcatch repository, put its `bin` directory on `PATH`, or run `ruby /path/to/jitcatch/bin/jitcatch`. `jitcatch --help` prints the commands, and `jitcatch --version` prints `jitcatch 0.2.1`, with the commit after it (`jitcatch 0.2.1 (1fbcd18)`) when jitcatch runs from its own git checkout; quote that line with any output you quote. The `version` line of every report and `version` in `--json` hold the number only.
|
|
17
17
|
|
|
18
18
|
Exit status 0 is success, including when catching tests were found. Exit status 1 is a usage error. Exit status 2 is a git error. Exit status 3 means `run` tested nothing: a side failed its setup check before any test ran, no candidate file loaded on either side, or every row timed out, or found no database or no table, on a side (below).
|
|
19
19
|
|
|
@@ -25,6 +25,8 @@ When `BASE` is not an ancestor of `HEAD` (for example, a main branch that moved
|
|
|
25
25
|
|
|
26
26
|
Pass `--repo DIR` when that repository is not the working directory.
|
|
27
27
|
|
|
28
|
+
In a monorepo whose gem sits in a directory with its own `lib/` and `Gemfile`, such as `ruby/`, run every command from that directory (`cd ruby && jitcatch run BASE HEAD catch`), or pass `--repo ruby` from the top. Candidates, the bundle check, `--setup`, and each `rerun:` line then run in `ruby/` of each checkout, with `-Ilib` as `ruby/lib` and `BUNDLE_GEMFILE` as `ruby/Gemfile`, so a candidate needs no `$LOAD_PATH` or `GEM_PATH` workaround. Put `TESTS_DIR` under that directory, and run each rerun line from there. The report prints `directory ruby` under `head`. File paths in the change, the methods, and the frames stay from the repository root (`ruby/lib/shelf.rb`).
|
|
29
|
+
|
|
28
30
|
Commit messages from `BASE` to `HEAD` are the intent the CLI can see (`git log BASE..HEAD`, oldest first, commits separated by `---`). The CLI leaves out the trailer block that git parses at the end of each message (`Co-Authored-By:`, `Signed-off-by:`, and the like). Git counts a last paragraph made only of `Word: text` lines as trailers too, so a closing line such as `Note: callers must refresh.` is also left out. When the intent looks cut short, read `git log BASE..HEAD`. Also read the pull request or the task when you have it. The tool does not.
|
|
29
31
|
|
|
30
32
|
## Context
|