holdfast 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: f19b183542b1eb5619c796e4ed02859386dcf21cc0b381e3b45b84df9eb20c46
+  data.tar.gz: f1f2a7d651cc193820c8a8edaa049568c81842e2b5b1af0b3366526e12d61c7a
+SHA512:
+  metadata.gz: a9071a68c3d371c161db59608ffcc3cfef4462b314a86870747fa9604b89a716c7666777017ca637ef0e8353085a78eabbca80413cc4ba077d122d91548f83ca
+  data.tar.gz: e9f9728bc94ee0bb2bffd4177d218f6ff64b7d53f5f07446711cdc71deeba7701fac230879b5933cc8725b5d1ac0734bfba43b66be9f87245ac584165837fdbb

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 meganemura
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.ja.md

@@ -0,0 +1,235 @@
+# holdfast
+
+holdfast は、Rails アプリについて疑ったことを、読んだだけで終わらせずに実際に走らせます。
+疑い（バグ、二つのアクションの競合、別の領域への影響）は、それぞれ数行の YAML で書きます。
+`holdfast check` はアプリ自身のファクトリで状況を作り、base と head でアクションを走らせます。
+答えは三つです。
+`confirmed` は残すべき失敗するスペックを伴い、`refuted` は観測した値を伴い、`could_not_run` は正確なエラーを伴います。
+`holdfast run` は変更を base と比べて確かめ、何も変わっていないときはアプリの入口をランダムな順序で走らせてバグを探します。
+
+holdfast は、コードを読んで見つけたことを報告するコーディングエージェントのための道具です。
+エージェントがたどるループは [`skills/holdfast/`](skills/holdfast/SKILL.md) のスキルにあります。
+
+## Rails アプリでのクイックスタート
+
+Ruby 3.4 以降、git、そして RSpec と FactoryBot でテスト用データベースに対してスペックが走るアプリが必要です。
+各ステップはアプリのルートで実行します。
+
+### 1. gem を追加する
+
+```ruby
+group :test do
+  gem "holdfast", require: false
+end
+```
+
+続けて `bundle install` を実行します。
+holdfast は hegeltest に依存していて、探索（hunt）がそれを使います。
+
+### 2. 読んで、疑いを書く
+
+以下の例は [`test/fixture/clinic`](test/fixture/clinic) のクリニックアプリで走らせたものです。
+このアプリには予約、来なかった予約を no-show にして料金を請求するジョブ、そして受付のチェックインがあります。
+ジョブを読むと、怪しい点が二つ見つかります。
+確定していない仮予約（tentative）まで対象にしていることと、行を読み込んでから一行ずつロックするので、その間のチェックインが上書きされうることです。
+両方を書き出します。
+
+```yaml
+- says: a passed tentative appointment is charged a no-show fee
+  given: {appointment: {status: tentative, starts_at: -1h}}
+  do: NoShowSweepJob.perform_now
+  expect: no PaymentGateway charge
+- says: check-in between the sweep's load and its lock becomes a no-show
+  given: {appointment: {status: booked, starts_at: -20m}}
+  race: [NoShowSweepJob.perform_now, "CheckIn.new(appointment).call"]
+  expect: appointment.status == "checked_in"
+```
+
+`given` はキーごとに、同じ名前のファクトリからレコードを一つ作ります。
+`do` は Ruby です。
+`race` は、一つ目のアクションを一時停止点で止めて二つ目を走らせ、次にその逆を行い、さらに二つの直列の順序も走らせます。
+`expect` は、走らせた後に成り立つべきことです。
+書き方のすべては [スキル](skills/holdfast/SKILL.md) にあります。
+
+### 3. 確かめる
+
+```sh
+bundle exec holdfast check suspicions.yml
+```
+
+初回の実行で、アプリがテストをどう走らせるかを学び、自分専用のテスト用データベースを作ります。
+クリニックアプリでの出力の全体は次のとおりです。
+
+```text
+# holdfast
+
+check: 2 suspicion(s); no change: checks run on the current tree only (af23e82); app, lib, config, db and the Gemfile match main (a17770f)
+kept specs: spec/holdfast/main/ (branch main); rows: .holdfast/check/main/check.jsonl
+seconds: record 9.6, run 5.5
+
+confirmed=2 could_not_run=0 refuted=0
+
+## Confirmed (the kept spec fails)
+
+- `a passed tentative appointment is charged a no-show fee`: no PaymentGateway charge does not hold (observed [[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]])
+  - ran: NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]
+  - no PaymentGateway charge fails ([[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]])
+  - spec output: suspicion confirmed: no PaymentGateway charge does not hold (observed [[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]])
+  - repro: `DATABASE_URL=postgres://postgres@127.0.0.1:56111/hf_clinic_9f0097_c2 TEST_ENV_NUMBER=c2 bundle exec rspec spec/holdfast/main/s1_a_passed_tentative_appointment_is_charged_a_no_s_spec.rb`
+  - next: keep spec/holdfast/main/s1_a_passed_tentative_appointment_is_charged_a_no_s_spec.rb; it fails until the code is fixed
+- `check-in between the sweep's load and its lock becomes a no-show`: first held at batch_load, second runs (held at Appointment load (1 rows)) ends [[Appointment#1(status: "no_show")], Appointment#1(status: "checked_in"); appointment.status == "checked_in" fails ("no_show")], an end no serial order reaches (first then second ends [[Appointment#1(status: "no_show")], raised CheckIn::Refused; appointment.status == "checked_in" fails ("no_show")]; second then first ends [[], Appointment#1(status: "checked_in"); appointment.status == "checked_in" holds ("checked_in")]); serial order "first then second" also fails expect, so this end is the evidence
+  - pause points tried: batch_load: Appointment load (1 rows)
+  - first held at batch_load, second runs: NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]; CheckIn.new(appointment).call returned Appointment#1(status: "checked_in"); held at Appointment load (1 rows); appointment.status == "checked_in" fails ("no_show")
+  - first then second: NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]; CheckIn.new(appointment).call raised CheckIn::Refused: appointment is no_show; appointment.status == "checked_in" fails ("no_show")
+  - second then first: CheckIn.new(appointment).call returned Appointment#1(status: "checked_in"); NoShowSweepJob.perform_now returned []; appointment.status == "checked_in" holds ("checked_in")
+  - second held at batch_load, first runs: CheckIn.new(appointment).call returned Appointment#1(status: "checked_in"); NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]; held at Appointment load (1 rows); the second waited on a lock; appointment.status == "checked_in" fails ("no_show")
+  - spec output: suspicion confirmed: first held at batch_load, second runs (held at Appointment load (1 rows)) ends [[Appointment#1(status: "no_show")], Appointment#1(status: "checked_in"); appointment.status == "checked_in" fails ("no_show")], an end no serial order reaches (first then second ends [[Appointment#1(status: "no_show")], raised CheckIn::Refused; appointment.status == "checked_in" fails ("no_show")]; second then first ends [[], Appointment#1(status: "checked_in"); appointment.status == "checked_in" holds ("checked_in")]); serial order "first then second" also fails expect, so this end is the evidence
+  - repro: `DATABASE_URL=postgres://postgres@127.0.0.1:56111/hf_clinic_9f0097_c2 TEST_ENV_NUMBER=c2 bundle exec rspec spec/holdfast/main/s2_check_in_between_the_sweep_s_load_and_its_lock_b_spec.rb`
+  - next: keep spec/holdfast/main/s2_check_in_between_the_sweep_s_load_and_its_lock_b_spec.rb; it fails until the code is fixed
+rows: .holdfast/check/main/check.jsonl; kept specs: spec/holdfast/main/
+```
+
+競合が confirmed になるのは、止めた順序が、どの直列の順序でも到達しない状態で終わったからです。
+ある直列の順序でも同じ状態に到達する競合は refuted になります。
+
+### 4. 残す
+
+confirmed の行はそれぞれ `spec/holdfast/<branch>/` の下のスペックを示し、そのスペックはコードを直すまで失敗します。
+コードを直し、スペックを走らせ、残します。
+ジョブが確定済みの予約（booked）だけを選ぶように直すと、一つ目のスペックは通ります。
+
+```sh
+bundle exec rspec spec/holdfast/main/s1_a_passed_tentative_appointment_is_charged_a_no_s_spec.rb
+```
+
+```text
+1 example, 0 failures
+```
+
+終了ステータスは、confirmed の行があれば 2、使い方の誤りなら 1、何も confirmed でなければ 0 です。
+
+## 変更を確かめる
+
+base のブランチと違うブランチの上では（`--base REF` で別の base を指定できます）、疑いはそれぞれ base と作業ツリーの両方で、どちらも head のファクトリで走ります。
+行には、その変更が持ち込んだものかどうかが書かれます。
+ここではブランチ `skip-tentative` がジョブを確定済みの予約だけを選ぶように変え、同じファイルをもう一度走らせています。
+
+```text
+# holdfast
+
+check: 2 suspicion(s); change: each runs on base main (a17770f) and on the working tree (a0a7f53); 1 file(s) differ: app/jobs/no_show_sweep_job.rb
+kept specs: spec/holdfast/skip-tentative/ (branch skip-tentative); rows: .holdfast/check/skip-tentative/check.jsonl
+seconds: record 0.0, run 5.6
+
+confirmed=1 could_not_run=0 refuted=1
+
+## Confirmed (the kept spec fails)
+
+- `check-in between the sweep's load and its lock becomes a no-show`: first held at batch_load, second runs (held at Appointment load (1 rows)) ends [...]
+  - base a17770f: confirmed: [...]; it was there before the change
+  [...]
+
+## Refuted (what was observed)
+
+- `a passed tentative appointment is charged a no-show fee`: no PaymentGateway charge holds (observed [])
+  - base a17770f: confirmed: no PaymentGateway charge does not hold (observed [[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]]); the change removes it
+  - ran: NoShowSweepJob.perform_now returned []
+  - no PaymentGateway charge holds ([])
+rows: .holdfast/check/skip-tentative/check.jsonl; kept specs: spec/holdfast/skip-tentative/
+```
+
+`[...]` は、上の行の繰り返しになる競合の行から切り取った部分です。
+base にないメソッド、カラム、ルートを呼ぶ疑いは head でだけ confirmed になり、行には base では走らせられないと書かれます。
+
+`holdfast run` は、疑いを書かずに変更を確かめます。
+変更された各メソッドに一行を与え、変更された定数やスコープの呼び出し元にも行を与えます。
+各対象の名前を含むテストを base と head で走らせ、各メソッドの引数からピンの下書きを作ります。
+行は `unintended_change`（base で通り head で失敗する）、`intended_change`（意図したと印を付けた違い、または変更で足したテストだけが通る新しい振る舞い）、`unchecked`（誰も引いていない値を示すので、それを疑いとして書く）、`clean`（示した証拠だけを保証する）のいずれかです。
+この部分は [jitcatch](https://github.com/meganemura/jitcatch) と [branchout](https://github.com/meganemura/branchout) を使います。
+どちらも holdfast といっしょに入ります。
+
+## 何も変わっていないときの探索
+
+変更がないときの `holdfast run` は、アプリの書き込み領域（ルート、Grape のエンドポイント、ジョブ）ごとにシーケンスの定義を一つ `.holdfast/sequences/` に下書きします。
+次に、それらのアクションをランダムな順序でテスト用データベースに対して走らせます。
+その際、不変条件、別のタイムゾーン、外部呼び出しでの二つのアクションの割り込み、最初のバッチ読み込みの後で止めたジョブ、受け付けられたリクエストをもう一度送ることを試します。
+失敗は最短のシーケンスまで縮められます（縮める時間は最大 60 秒です）。
+`--budget SECONDS` で実行時間の上限を決めます（既定は 600）。
+クリニックアプリで、上限を 120 秒にした出力です。
+
+```text
+# holdfast
+
+nothing in the app changed against main (27257f9)
+note: hunt: unreached: 1 of 2 actions (refused every time or never drawn); 0 of 1 areas reached under half
+seconds: record 10.1, hunt 16.8
+
+unintended_change=0 bug_suspect=1 unchecked=1 intended_change=0 clean=0
+
+## Suspected bugs
+
+- `sequences: check_ins`: no_show_sweep_job loaded Appointment#1 with status "booked"; check_ins_create set it to "checked_in" while no_show_sweep_job held it; no_show_sweep_job then wrote "no_show", though its own query does not select a row whose status is "checked_in"
+  - failure: no_show_sweep_job loaded Appointment#1 with status "booked"; check_ins_create set it to "checked_in" while no_show_sweep_job held it; no_show_sweep_job then wrote "no_show", though its own query does not select a row whose status is "checked_in"
+  - log: .holdfast/runs/sequences/check_ins.log (the shortest sequence and an `it` block that replays it)
+  - next: paste the `it` block from the log into .holdfast/sequences/check_ins.rb, see it fail, fix the code, run again
+
+## Not checked
+
+- `(hunt)`: unreached: 1 of 2 actions (refused every time or never drawn); 0 of 1 areas reached under half
+  - next: read each area's row below for the refusals, fix the setup or the sign-in in its file, then holdfast run; until then the hunt says little about these areas
+rows: .holdfast/verdicts.jsonl
+```
+
+探索は、疑いを書かなくても、二つ目の疑いと同じ競合を見つけました。
+探索の行は疑いにつながります。
+それを書き出し、`holdfast check` で確かめます。
+`holdfast run --sequences` は変更が触れる領域も探索し、`--observe` は各領域のアクションの順序を base と head で走らせて、違ったステップを一つずつ報告します。
+
+## 使い方
+
+```text
+holdfast check [FILE]... [--suspect YAML]... [--base REF] [--here] [--repo DIR]
+holdfast run [--base REF] [--target NAME]... [--only NAME]... [--intent TEXT] [--learn] [--sequences] [--observe] [--budget SECONDS] [--repo DIR]
+holdfast show
+```
+
+- `--suspect YAML` は疑いを一つ YAML のフローマッピングとして渡し、繰り返して使えます。
+- `--here` は、変更の上でも、作業ツリーで一度だけ確かめます。
+- `--learn` は、失敗したセットアップを直した後に、アプリがテストをどう走らせるかを学び直します。
+- `show` は、最後の `holdfast run` の要約をもう一度表示します。
+
+holdfast はアプリの `.holdfast/` と `spec/holdfast/` の下に書き込みます。
+`.holdfast/` はアプリの `.gitignore` に加えます。
+残すスペックはコミットします。
+
+各アプリは、そのディレクトリから作った自分専用のテスト用データベース名を `DATABASE_URL` で受け取ります。
+そのため、base と head、また一つのサーバー上の二つのアプリが、同じデータベースを共有しません。
+セットアップは `bin/rails db:create db:schema:load` を実行します。
+
+## 制限
+
+- 入力がレコードを一つも作らないピンの下書きは何も確かめませんが、そのピンが変更された行を読み込むと、`holdfast run` はその対象を `clean` と示すことがあります。
+  `clean` の行を信頼する前に、その証拠を読んでください。
+- 競合は、一時停止点 `batch_load`、`outside_call`、`row_lock`、`first_write`、または指定したメソッドで試します。
+  それ以外の割り込み方は試しません。
+- 探索が到達するのは、下書きされたセットアップとサインインが働くアクションだけです。
+  到達しなかった数は最初の行に出ます。
+
+## 開発
+
+```sh
+bundle install
+bundle install --gemfile test/fixture/clinic/Gemfile
+DB_HOST=localhost DB_PORT=5432 ruby -Ilib test/run.rb
+```
+
+`DB_HOST` がなければ、クリニックアプリを走らせるテストはスキップされます。
+詳しくは [AGENTS.md](AGENTS.md) にあります。
+
+## ライセンス
+
+MIT
+
+---
+
+[English](README.md)

data/README.md

@@ -0,0 +1,191 @@
+# holdfast
+
+holdfast runs what you suspect about a Rails app instead of leaving it read but not run. You write each suspicion (a bug, a race between two actions, an effect on another area) as a few lines of YAML. `holdfast check` builds the world with the app's own factories, runs the action on base and head, and answers `confirmed` with a failing spec to keep, `refuted` with what it observed, or `could_not_run` with the exact error. `holdfast run` checks a change against its base, and hunts with random sequences of the app's entry points when nothing changed.
+
+It is made for a coding agent that reads code and has to say what it found. The skill in [`skills/holdfast/`](skills/holdfast/SKILL.md) is the loop an agent follows.
+
+## Quick start in your Rails app
+
+You need Ruby 3.4 or newer, git, and an app whose specs run with RSpec and FactoryBot against a test database. Run each step from the app's root.
+
+### 1. Add the gem
+
+```ruby
+group :test do
+  gem "holdfast", require: false
+end
+```
+
+Then run `bundle install`. holdfast depends on hegeltest, which the hunt uses.
+
+### 2. Read, and write what you suspect
+
+The examples below run against the clinic app in [`test/fixture/clinic`](test/fixture/clinic): appointments, a job that marks missed appointments as no-shows and charges a fee, and a desk check-in. Reading the job, two things look wrong. It also sweeps tentative appointments, which were never confirmed, and it loads its rows before it locks each one, so a check-in in between may be overwritten. Write both down:
+
+```yaml
+- says: a passed tentative appointment is charged a no-show fee
+  given: {appointment: {status: tentative, starts_at: -1h}}
+  do: NoShowSweepJob.perform_now
+  expect: no PaymentGateway charge
+- says: check-in between the sweep's load and its lock becomes a no-show
+  given: {appointment: {status: booked, starts_at: -20m}}
+  race: [NoShowSweepJob.perform_now, "CheckIn.new(appointment).call"]
+  expect: appointment.status == "checked_in"
+```
+
+`given` makes one record per key from the factory of that name. `do` is Ruby. `race` holds the first action at a pause point while the second runs, then the reverse, and also runs both serial orders. `expect` is what must be true afterwards. The [skill](skills/holdfast/SKILL.md) lists every form.
+
+### 3. Check
+
+```sh
+bundle exec holdfast check suspicions.yml
+```
+
+The first run learns how the app runs its tests and makes test databases of its own. This is the whole output on the clinic app:
+
+```text
+# holdfast
+
+check: 2 suspicion(s); no change: checks run on the current tree only (af23e82); app, lib, config, db and the Gemfile match main (a17770f)
+kept specs: spec/holdfast/main/ (branch main); rows: .holdfast/check/main/check.jsonl
+seconds: record 9.6, run 5.5
+
+confirmed=2 could_not_run=0 refuted=0
+
+## Confirmed (the kept spec fails)
+
+- `a passed tentative appointment is charged a no-show fee`: no PaymentGateway charge does not hold (observed [[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]])
+  - ran: NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]
+  - no PaymentGateway charge fails ([[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]])
+  - spec output: suspicion confirmed: no PaymentGateway charge does not hold (observed [[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]])
+  - repro: `DATABASE_URL=postgres://postgres@127.0.0.1:56111/hf_clinic_9f0097_c2 TEST_ENV_NUMBER=c2 bundle exec rspec spec/holdfast/main/s1_a_passed_tentative_appointment_is_charged_a_no_s_spec.rb`
+  - next: keep spec/holdfast/main/s1_a_passed_tentative_appointment_is_charged_a_no_s_spec.rb; it fails until the code is fixed
+- `check-in between the sweep's load and its lock becomes a no-show`: first held at batch_load, second runs (held at Appointment load (1 rows)) ends [[Appointment#1(status: "no_show")], Appointment#1(status: "checked_in"); appointment.status == "checked_in" fails ("no_show")], an end no serial order reaches (first then second ends [[Appointment#1(status: "no_show")], raised CheckIn::Refused; appointment.status == "checked_in" fails ("no_show")]; second then first ends [[], Appointment#1(status: "checked_in"); appointment.status == "checked_in" holds ("checked_in")]); serial order "first then second" also fails expect, so this end is the evidence
+  - pause points tried: batch_load: Appointment load (1 rows)
+  - first held at batch_load, second runs: NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]; CheckIn.new(appointment).call returned Appointment#1(status: "checked_in"); held at Appointment load (1 rows); appointment.status == "checked_in" fails ("no_show")
+  - first then second: NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]; CheckIn.new(appointment).call raised CheckIn::Refused: appointment is no_show; appointment.status == "checked_in" fails ("no_show")
+  - second then first: CheckIn.new(appointment).call returned Appointment#1(status: "checked_in"); NoShowSweepJob.perform_now returned []; appointment.status == "checked_in" holds ("checked_in")
+  - second held at batch_load, first runs: CheckIn.new(appointment).call returned Appointment#1(status: "checked_in"); NoShowSweepJob.perform_now returned [Appointment#1(status: "no_show")]; held at Appointment load (1 rows); the second waited on a lock; appointment.status == "checked_in" fails ("no_show")
+  - spec output: suspicion confirmed: first held at batch_load, second runs (held at Appointment load (1 rows)) ends [[Appointment#1(status: "no_show")], Appointment#1(status: "checked_in"); appointment.status == "checked_in" fails ("no_show")], an end no serial order reaches (first then second ends [[Appointment#1(status: "no_show")], raised CheckIn::Refused; appointment.status == "checked_in" fails ("no_show")]; second then first ends [[], Appointment#1(status: "checked_in"); appointment.status == "checked_in" holds ("checked_in")]); serial order "first then second" also fails expect, so this end is the evidence
+  - repro: `DATABASE_URL=postgres://postgres@127.0.0.1:56111/hf_clinic_9f0097_c2 TEST_ENV_NUMBER=c2 bundle exec rspec spec/holdfast/main/s2_check_in_between_the_sweep_s_load_and_its_lock_b_spec.rb`
+  - next: keep spec/holdfast/main/s2_check_in_between_the_sweep_s_load_and_its_lock_b_spec.rb; it fails until the code is fixed
+rows: .holdfast/check/main/check.jsonl; kept specs: spec/holdfast/main/
+```
+
+The race is confirmed only because the held order ends in a state that no serial order reaches. A race whose end some serial order also reaches is refuted.
+
+### 4. Keep
+
+Each confirmed row names a spec under `spec/holdfast/<branch>/` that fails until the code is fixed. Fix the code, run the spec, and keep it. After the sweep selects only booked appointments, the first spec passes:
+
+```sh
+bundle exec rspec spec/holdfast/main/s1_a_passed_tentative_appointment_is_charged_a_no_s_spec.rb
+```
+
+```text
+1 example, 0 failures
+```
+
+Exit status 2 means a row was confirmed, 1 a usage error, and 0 that nothing was confirmed.
+
+## Check a change
+
+On a branch that differs from the base branch (`--base REF` names another), each suspicion runs on base and on the working tree, both with head's factories, and the row says whether the change brought it in. Here the branch `skip-tentative` changes the sweep to select only booked appointments, and the same file runs again:
+
+```text
+# holdfast
+
+check: 2 suspicion(s); change: each runs on base main (a17770f) and on the working tree (a0a7f53); 1 file(s) differ: app/jobs/no_show_sweep_job.rb
+kept specs: spec/holdfast/skip-tentative/ (branch skip-tentative); rows: .holdfast/check/skip-tentative/check.jsonl
+seconds: record 0.0, run 5.6
+
+confirmed=1 could_not_run=0 refuted=1
+
+## Confirmed (the kept spec fails)
+
+- `check-in between the sweep's load and its lock becomes a no-show`: first held at batch_load, second runs (held at Appointment load (1 rows)) ends [...]
+  - base a17770f: confirmed: [...]; it was there before the change
+  [...]
+
+## Refuted (what was observed)
+
+- `a passed tentative appointment is charged a no-show fee`: no PaymentGateway charge holds (observed [])
+  - base a17770f: confirmed: no PaymentGateway charge does not hold (observed [[{customer: "pat_1", amount_cents: 2500, reference: "no-show-1"}]]); the change removes it
+  - ran: NoShowSweepJob.perform_now returned []
+  - no PaymentGateway charge holds ([])
+rows: .holdfast/check/skip-tentative/check.jsonl; kept specs: spec/holdfast/skip-tentative/
+```
+
+`[...]` marks text cut from the race row, which repeats the row above. A suspicion that calls a method, a column or a route that base does not have is confirmed on head only, and the row says that base cannot run it.
+
+`holdfast run` checks a change without suspicions. It gives each changed method a row, and the callers of a changed constant or scope too, runs the tests that name each target on base and head, and drafts pins from each method's parameters. A row is `unintended_change` (passes on base, fails on head), `intended_change` (a difference you marked as meant, or new behavior that only the change's new specs run), `unchecked` (with the value nobody drew, to write as a suspicion), or `clean` (covering only the evidence it lists). This part runs on [jitcatch](https://github.com/meganemura/jitcatch) and [branchout](https://github.com/meganemura/branchout), which come with holdfast.
+
+## Hunt when nothing changed
+
+`holdfast run` with no change drafts one sequence definition per write area of the app (routes, Grape endpoints, jobs) into `.holdfast/sequences/`. It then runs random orders of their actions against the test database, under invariants, other time zones, two actions interleaved at an outside call, a job held after its first batch load, and an accepted request sent once more. A failure is shrunk to the shortest sequence, for at most 60 seconds. `--budget SECONDS` caps the run (default 600). On the clinic app, with a budget of 120 seconds:
+
+```text
+# holdfast
+
+nothing in the app changed against main (27257f9)
+note: hunt: unreached: 1 of 2 actions (refused every time or never drawn); 0 of 1 areas reached under half
+seconds: record 10.1, hunt 16.8
+
+unintended_change=0 bug_suspect=1 unchecked=1 intended_change=0 clean=0
+
+## Suspected bugs
+
+- `sequences: check_ins`: no_show_sweep_job loaded Appointment#1 with status "booked"; check_ins_create set it to "checked_in" while no_show_sweep_job held it; no_show_sweep_job then wrote "no_show", though its own query does not select a row whose status is "checked_in"
+  - failure: no_show_sweep_job loaded Appointment#1 with status "booked"; check_ins_create set it to "checked_in" while no_show_sweep_job held it; no_show_sweep_job then wrote "no_show", though its own query does not select a row whose status is "checked_in"
+  - log: .holdfast/runs/sequences/check_ins.log (the shortest sequence and an `it` block that replays it)
+  - next: paste the `it` block from the log into .holdfast/sequences/check_ins.rb, see it fail, fix the code, run again
+
+## Not checked
+
+- `(hunt)`: unreached: 1 of 2 actions (refused every time or never drawn); 0 of 1 areas reached under half
+  - next: read each area's row below for the refusals, fix the setup or the sign-in in its file, then holdfast run; until then the hunt says little about these areas
+rows: .holdfast/verdicts.jsonl
+```
+
+The hunt found the same race as the second suspicion, with no suspicion written. A hunt row leads to a suspicion: write it down and confirm it with `holdfast check`. `holdfast run --sequences` also hunts over the areas a change touches, and `--observe` runs each area's action orders on base and head and reports each step that differs.
+
+## Usage
+
+```text
+holdfast check [FILE]... [--suspect YAML]... [--base REF] [--here] [--repo DIR]
+holdfast run [--base REF] [--target NAME]... [--only NAME]... [--intent TEXT] [--learn] [--sequences] [--observe] [--budget SECONDS] [--repo DIR]
+holdfast show
+```
+
+- `--suspect YAML` passes one suspicion as a YAML flow mapping, and repeats.
+- `--here` runs a check once on the working tree, even in a change.
+- `--learn` learns again how the app runs its tests, after you fix a failed setup.
+- `show` prints the last summary of `holdfast run` again.
+
+holdfast writes under `.holdfast/` and `spec/holdfast/` in the app. Add `.holdfast/` to the app's `.gitignore`. Commit the specs you keep.
+
+Each app gets its own test database names through `DATABASE_URL`, made from its directory, so base and head, and two apps on one server, do not share one. The setup runs `bin/rails db:create db:schema:load`.
+
+## Limits
+
+- A drafted pin whose inputs build no record checks nothing, and `holdfast run` can still list its target as `clean` when the pin loads the changed line. Read the evidence of a `clean` row before you trust it.
+- A race is tried at the pause points `batch_load`, `outside_call`, `row_lock`, `first_write`, or a method you name. Other interleavings are not tried.
+- The hunt reaches only the actions whose drafted setup and sign-in work. The first line says how many it did not reach.
+
+## Development
+
+```sh
+bundle install
+bundle install --gemfile test/fixture/clinic/Gemfile
+DB_HOST=localhost DB_PORT=5432 ruby -Ilib test/run.rb
+```
+
+Without `DB_HOST`, the tests that run the clinic app skip. [AGENTS.md](AGENTS.md) has the details.
+
+## License
+
+MIT
+
+---
+
+[Japanese](README.ja.md)

data/bin/holdfast

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+Encoding.default_external = Encoding::UTF_8
+require_relative "../lib/holdfast"
+exit Holdfast::CLI.start(ARGV)

data/lib/holdfast/callers.rb

@@ -0,0 +1,124 @@
+module Holdfast
+  # The unchanged code that reads a changed value through something else: a
+  # scope or a method whose own lines did not change but that reads a
+  # constant the change edits, and the methods that call that scope or
+  # method, one or two calls up. A reader of the diff stops at the diff, so
+  # this is where a break hides; each caller becomes a target with a drafted
+  # pin and ranks first.
+  #
+  # jitcatch's brief names the affected methods and their direct and
+  # dynamic callers. The second hop comes from jitcatch's call graph, which
+  # it does not publish, so this reads it through the brief's own instance.
+  module Callers
+    CAP = 8
+    Value = Data.define(:method, :constant, :model, :column, :values, :outside)
+
+    def self.targets(jc, brief, change)
+      values = brief["methods"].filter_map { value_of(_1, brief, change) }
+      return [] if values.empty?
+
+      head_idx = jc.send(:index_ref, change.head, "head")
+      base_idx = jc.send(:index_ref, change.base, "base")
+      edges = jc.send(:edges_for, head_idx)
+      known = brief["methods"].map { _1["name"] }
+      found = {}
+      values.each do |v|
+        m = v.method
+        hop1 = (Array(m["callsites"]).map { _1["enclosing"] } + Array(m["dynamic_callers"]).map { _1["from"] } +
+                Hash(m["caller_depths"]).select { _2 == 1 }.keys).uniq
+        hop1.each { found[_1] ||= [v, m["name"]] unless skip?(_1, known) }
+        hop1.each do |c|
+          jc.send(:callers_by_depth, c, edges, 1).each_key { found[_1] ||= [v, c] unless skip?(_1, known) }
+        end
+      end
+      found.first(CAP).filter_map { |name, (v, through)| target(name, v, through, head_idx, base_idx, change) }
+    end
+
+    # A value this module walks from: a method the brief marks as affected by
+    # a changed constant, or a changed scope.
+    def self.value_of(m, brief, change)
+      scope = m["new_source"].to_s.lstrip.start_with?("scope ")
+      return nil unless m["change"] == "affected" || (m["change"] == "modified" && scope)
+
+      const = brief["constants"].find { _1["id"] == m["via_constant"] }
+      old_values = strings(const && const["old_source"])
+      new_values = strings(const && const["new_source"])
+      owner = m["name"].split(/[#.]/).first
+      column = scope && const ? where_column(m["new_source"], const["name"].split("::").last) : nil
+      model_file = m["file"] if column
+      outside = model_file ? (other_strings(change, model_file) - old_values - new_values).first : nil
+      Value.new(method: m, constant: const && const["name"], model: (owner if column), column:,
+                values: (old_values | new_values), outside:)
+    end
+
+    def self.skip?(name, known)
+      known.include?(name) || !name.match?(/\A[A-Z][\w:]*[#.][a-z_]\w*[?!=]?\z/) ||
+        name.split(/[#.]/).first.end_with?("Controller") || name.end_with?("#initialize")
+    end
+
+    def self.target(name, v, through, head_idx, base_idx, change)
+      node = head_idx["methods"].find { _1.qualified == name } or return nil
+      old = base_idx["methods"].find { _1.qualified == name }
+      via = v.constant ? "#{v.method["name"]}, which reads #{v.constant}" : v.method["name"]
+      why = through == v.method["name"] ? "calls #{via}" : "calls #{through}, which calls #{via}"
+      reads = { "via" => v.method["name"], "constant" => v.constant, "model" => v.model, "column" => v.column,
+                "values" => v.values, "outside" => v.outside }.compact
+      Target.new(name:, kind: "method", file: node.file, line: node.start_line, why: ["caller", why],
+                 tests: tests_for(change, node), source: node.source, reader: nil, reads:,
+                 span: { "old_start" => old&.start_line, "old_end" => old&.end_line, "new_start" => node.start_line, "new_end" => node.end_line })
+    end
+
+    # Test files at base that name the caller's method, or its class and a
+    # request into it; coverage in the run decides which ones reached it.
+    def self.tests_for(change, node)
+      bare = node.bare_name
+      owner = node.qualified.split(/[#.]/).first.split("::").last
+      out, = Open3.capture2("git", "grep", "-l", "-w", "-e", bare, change.base, "--", "spec", "test", chdir: change.root)
+      named = out.lines.map { _1.strip.sub(/\A#{change.base}:/, "") }
+      out2, = Open3.capture2("git", "grep", "-l", "-w", "-e", owner, change.base, "--", "spec", "test", chdir: change.root)
+      (named & out2.lines.map { _1.strip.sub(/\A#{change.base}:/, "") }).first(6)
+    end
+
+    # The string elements of the first array literal in a source line.
+    def self.strings(source)
+      return [] unless source
+
+      found = []
+      walk(Prism.parse(source).value) do |n|
+        next unless n.is_a?(Prism::ArrayNode) && found.empty?
+
+        found = n.elements.filter_map { |e| e.unescaped if e.is_a?(Prism::StringNode) || e.is_a?(Prism::SymbolNode) }
+      end
+      found
+    end
+
+    # Every string listed in the model file's array constants (STATUSES), so
+    # a draw also holds a value neither side's list holds.
+    def self.other_strings(change, file)
+      text = Open3.capture2("git", "show", "#{change.head}:#{file}", chdir: change.root).first
+      text.scan(/^\s*[A-Z][A-Z0-9_]*\s*=\s*(%w\[[^\]]*\]|\[[^\]]*\])/).flat_map { strings(_1.first) }.uniq
+    end
+
+    # The column a scope compares with the constant: `where(status: OPEN)`.
+    def self.where_column(source, const)
+      col = nil
+      walk(Prism.parse(source.to_s).value) do |n|
+        next unless n.is_a?(Prism::CallNode) && %i[where not].include?(n.name) && n.arguments
+
+        n.arguments.arguments.grep(Prism::KeywordHashNode).flat_map(&:elements).each do |pair|
+          val = pair.value
+          name = val.is_a?(Prism::ConstantReadNode) ? val.name.to_s : val.is_a?(Prism::ConstantPathNode) ? val.name.to_s : nil
+          col ||= pair.key.unescaped if name == const && pair.key.is_a?(Prism::SymbolNode)
+        end
+      end
+      col
+    end
+
+    def self.walk(node, &block)
+      return unless node
+
+      yield node
+      node.compact_child_nodes.each { walk(_1, &block) }
+    end
+  end
+end

data/lib/holdfast/change.rb

@@ -0,0 +1,59 @@
+module Holdfast
+  # What one run looks at: the base whose behavior must survive, the head
+  # (the working tree as it is now, committed or not), and so which mode the
+  # harness runs in. The working tree is snapshotted into a commit object
+  # through a private index, so the caller's index and stash stay untouched.
+  Change = Data.define(:root, :base, :head, :base_label, :mode, :named) do
+    def self.read(root, base_ref:, named:)
+      base_sha = git(root, "merge-base", base_ref, "HEAD")
+      head_sha = snapshot(root)
+      same = git(root, "rev-parse", "#{base_sha}^{tree}") == git(root, "rev-parse", "#{head_sha}^{tree}")
+      # Edits only to tests, factories or docs change no behavior the app
+      # has, so they neither make a change run nor stop a hunt.
+      code = git(root, "diff", "--name-only", base_sha, head_sha, "--", *CODE).lines.map(&:strip).reject(&:empty?)
+      same ||= code.empty?
+      mode = if !same then "change"
+             elsif named.any? then "before"
+             else "hunt"
+             end
+      new(root:, base: base_sha, head: head_sha, base_label: base_ref, mode:, named:)
+    end
+
+    CODE = %w[app lib config db Gemfile Gemfile.lock].freeze
+
+    def self.snapshot(root)
+      dirty = !git(root, "status", "--porcelain", "--untracked-files=all", "--", ".", ":!.holdfast").empty?
+      return git(root, "rev-parse", "HEAD") unless dirty
+
+      index = File.join(Dir.mktmpdir("holdfast-index"), "index")
+      env = { "GIT_INDEX_FILE" => index }
+      git(root, "read-tree", "HEAD", env:)
+      git(root, "add", "-A", env:)
+      git(root, "rm", "-r", "--cached", "--quiet", "--ignore-unmatch", ".holdfast", env:)
+      tree = git(root, "write-tree", env:)
+      # The commit stays private to this run, so it needs no identity from the user and must not fail without one.
+      who = { "GIT_AUTHOR_NAME" => "holdfast", "GIT_AUTHOR_EMAIL" => "holdfast@localhost",
+              "GIT_COMMITTER_NAME" => "holdfast", "GIT_COMMITTER_EMAIL" => "holdfast@localhost" }
+      git(root, "commit-tree", tree, "-p", "HEAD", "-m", "holdfast: working tree", env: env.merge(who))
+    ensure
+      FileUtils.rm_rf(File.dirname(index)) if index
+    end
+
+    def self.git(root, *args, env: {})
+      out, err, status = Open3.capture3(env, "git", *args, chdir: root)
+      raise Holdfast::Error, "git #{args.join(" ")}: #{err.lines.first.to_s.strip}" unless status.success?
+
+      out.strip
+    end
+
+    def changed? = mode == "change"
+    def short(sha) = sha[0, 7]
+    def header
+      case mode
+      when "change" then "change: #{base_label} (#{short(base)}) .. working tree (#{short(head)})"
+      when "before" then "before a change: pinning #{named.join(", ")} at #{short(head)}"
+      else "#{Change.git(root, "diff", "--quiet", base, head).then { "" } rescue "only tests, factories or docs differ from #{base_label}; " }nothing in the app changed against #{base_label} (#{short(base)})"
+      end
+    end
+  end
+end