quint_driver 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,648 @@
1
+ # quint_driver
2
+
3
+ quint_driver holds your Ruby code to a [Quint](https://github.com/quint-co/quint) spec: the spec says what each command must do, and quint_driver runs the same commands on your code and compares the two after every step.
4
+
5
+ ## Quick start in your Rails app
6
+
7
+ ### Add the gem
8
+
9
+ ```ruby
10
+ # Gemfile
11
+ group :test do
12
+ gem "quint_driver", require: false
13
+ end
14
+ ```
15
+
16
+ ```sh
17
+ bundle install
18
+ npm install --global @informalsystems/quint@0.33.0
19
+ ```
20
+
21
+ quint_driver needs Ruby 3.3 or later. It calls the `quint` command that `QUINT` names, so the run below sets `QUINT=quint`; put `export QUINT=quint` in your shell profile to leave it out. Without `QUINT`, the gem runs `npx --yes @informalsystems/quint@0.33.0`, which downloads quint (43 MB) on its first run and starts node once more on every run. The first quint run also downloads quint's Rust evaluator from GitHub into `~/.quint`. When your network blocks that download, the [skill](skills/quint_driver/SKILL.md#install) shows how to fetch it by hand.
22
+
23
+ ### Pick a first model
24
+
25
+ Pick a model with a status column, such as `state` or `status`, and a method that changes it, such as `Order#cancel` or `Loan#approve`. A model whose status a recent commit changed is a good start, because the spec then checks that change: `git log --oneline -- app/models` lists those commits.
26
+
27
+ The steps below use `Loan`. A new loan is `"pending"`, and `Loan#approve(officer)` makes it `"approved"`. Replace `Loan`, the record that the driver's `init` creates, the method, and the state names with yours.
28
+
29
+ ### Write the spec
30
+
31
+ ```quint
32
+ // spec/quint/loan_state.qnt
33
+ // A loan starts pending. Approving it once makes it approved, and
34
+ // approving it again leaves it approved.
35
+ module loan_state {
36
+ var state: str
37
+
38
+ action init = state' = "pending"
39
+ action approve = all { state == "pending", state' = "approved" }
40
+ action approve_refused = all { state == "approved", state' = state }
41
+
42
+ action step = any { approve, approve_refused }
43
+ }
44
+ ```
45
+
46
+ The `var` is the column. `init` is the value a new record has. Every other action is one call of the method, and its first condition says when the call does what the action says: `approve` makes a pending loan approved, and `approve_refused` leaves an approved loan as it is. `step` lists the actions that a trace picks from. `Quint.simulate` runs the module named after the file, so keep the two names the same.
47
+
48
+ ### Write the driver
49
+
50
+ ```ruby
51
+ # spec/quint/loan_state_driver.rb
52
+ # Runs each action of loan_state.qnt on a Loan and reads its state back.
53
+ class LoanStateDriver
54
+ def step(action, _picks)
55
+ case action
56
+ when "init"
57
+ borrower = Borrower.create!(name: "Ada", limit_cents: 20_000)
58
+ @loan = Loan.create!(borrower: borrower, amount_cents: 500)
59
+ when "approve", "approve_refused"
60
+ @loan.approve({name: "Grace", senior: true})
61
+ end
62
+ end
63
+
64
+ def state = {"state" => @loan.reload.state}
65
+ end
66
+ ```
67
+
68
+ `step` runs one action on a real record: `init` creates it, and every other action calls the method. `state` returns the spec's variables, read back from the database.
69
+
70
+ ### Write the example
71
+
72
+ ```ruby
73
+ # spec/quint/loan_state_spec.rb
74
+ require "rails_helper"
75
+ require "quint_driver"
76
+ require_relative "loan_state_driver"
77
+
78
+ RSpec.describe Loan, type: :model do
79
+ # The transactional fixture rolls back once, after the example. Each trace
80
+ # rolls back to a savepoint of its own, so no trace sees the rows of another.
81
+ def rolled_back
82
+ ActiveRecord::Base.transaction(requires_new: true, joinable: false) do
83
+ yield
84
+ raise ActiveRecord::Rollback
85
+ end
86
+ end
87
+
88
+ it "moves its state the way loan_state.qnt says" do
89
+ QuintDriver::Quint.simulate("spec/quint/loan_state.qnt", traces: 20, max_steps: 5, seed: "1") do |trace|
90
+ rolled_back { QuintDriver::Trace.replay(trace, LoanStateDriver.new) }
91
+ end
92
+ end
93
+ end
94
+ ```
95
+
96
+ Quint writes 20 traces of up to 5 steps, and each trace runs on a new loan.
97
+
98
+ ### Run the example
99
+
100
+ ```sh
101
+ QUINT=quint bundle exec rspec spec/quint/loan_state_spec.rb
102
+ ```
103
+
104
+ ## What you will see
105
+
106
+ This output is from a small sample app, a lending desk with a `Borrower` and a `Loan` model.
107
+
108
+ ```text
109
+ .
110
+
111
+ Finished in 0.60298 seconds (files took 0.88104 seconds to load)
112
+ 1 example, 0 failures
113
+ ```
114
+
115
+ Every trace created a loan in the test database, approved it, approved it again, and read the state back after each step. After each step, the loan had the state the spec has.
116
+
117
+ Now a commit makes a new loan a draft:
118
+
119
+ ```diff
120
+ --- a/app/models/loan.rb
121
+ +++ b/app/models/loan.rb
122
+ def mark_pending
123
+ - self.state ||= "pending"
124
+ + self.state ||= "draft"
125
+ end
126
+ ```
127
+
128
+ ```sh
129
+ QUINT=quint bundle exec rspec spec/quint/loan_state_spec.rb
130
+ ```
131
+
132
+ ```text
133
+ F
134
+
135
+ Failures:
136
+
137
+ 1) Loan moves its state the way loan_state.qnt says
138
+ Failure/Error: rolled_back { QuintDriver::Trace.replay(trace, LoanStateDriver.new) }
139
+
140
+ QuintDriver::Mismatch:
141
+ quint: after step 0 `init` {}, the driver differs from the trace
142
+ state: trace "pending", driver "draft"
143
+ steps:
144
+ init {}
145
+ trace: trace_0.itf.json of Quint.simulate("spec/quint/loan_state.qnt", traces: 20, max_steps: 5, seed: "1", step: "step", backend: "rust")
146
+ # ./spec/quint/loan_state_spec.rb:18:in 'block (4 levels) in <top (required)>'
147
+ # ./spec/quint/loan_state_spec.rb:11:in 'block in RSpec::ExampleGroups::Loan#rolled_back'
148
+ # ./spec/quint/loan_state_spec.rb:10:in 'RSpec::ExampleGroups::Loan#rolled_back'
149
+ # ./spec/quint/loan_state_spec.rb:18:in 'block (3 levels) in <top (required)>'
150
+ # ./spec/quint/loan_state_spec.rb:17:in 'block (2 levels) in <top (required)>'
151
+
152
+ Finished in 0.61377 seconds (files took 0.90689 seconds to load)
153
+ 1 example, 1 failure
154
+
155
+ Failed examples:
156
+
157
+ rspec ./spec/quint/loan_state_spec.rb:16 # Loan moves its state the way loan_state.qnt says
158
+ ```
159
+
160
+ The first line names the step where the model and the spec differ: right after `init`, the spec has a pending loan, and the model has a draft. `steps:` lists every step from `init`, so you can run the same commands by hand, and `trace:` names the call that writes the same trace again. When the change is a bug, fix the model. When the change is what you want, change `init` in the spec in the same commit, so the spec keeps saying what the app does.
161
+
162
+ Next, give an action a pick, such as the officer or the amount, and compare a second variable, such as the outcome of each call: [a bigger spec](#a-bigger-spec-refusals-and-picks) does both.
163
+
164
+ ## A bigger spec: refusals and picks
165
+
166
+ The quick start's spec has one record and one variable. This spec of the same app has two loans of one borrower, picks the amount and the officer for each action, and compares the outcome of every call too. The driver keeps the id each loan of the spec got, and reads the rows back as the spec's variables. `Loan#approve(officer)` refuses a loan that is already approved, one over the borrower's limit, and one above 10000 cents when the officer is not senior:
167
+
168
+ ```ruby
169
+ # spec/quint/approval_driver.rb
170
+ class ApprovalDriver
171
+ LOANS = %w[l1 l2].freeze
172
+
173
+ def step(action, picks)
174
+ @outcome = "Done"
175
+ case action.delete_suffix("_refused")
176
+ when "init"
177
+ @borrower = Borrower.create!(name: "Ada", limit_cents: 20_000)
178
+ @ids = {}
179
+ when "open"
180
+ @ids[picks.fetch("l")] = LoanDesk.open(@borrower, picks.fetch("a")).id
181
+ when "approve"
182
+ officer = {name: "Grace", senior: picks.fetch("senior")}
183
+ result = Loan.find(@ids.fetch(picks.fetch("l"))).approve(officer)
184
+ @outcome = ["Refused", result.to_s] unless result == :approved
185
+ end
186
+ end
187
+
188
+ def state
189
+ loans = LOANS.to_h { [_1, @ids[_1] && Loan.find(@ids[_1])] }
190
+ {
191
+ "amount" => loans.transform_values { _1&.amount_cents || 0 },
192
+ "approved" => loans.transform_values { _1&.state == "approved" },
193
+ "outcome" => @outcome
194
+ }
195
+ end
196
+ end
197
+ ```
198
+
199
+ A transactional fixture rolls back once, after the example, but one example replays many traces. Wrap each trace in a savepoint of its own, so no trace reads the rows of another:
200
+
201
+ ```ruby
202
+ # spec/quint/approval_spec.rb
203
+ require "rails_helper"
204
+ require "quint_driver"
205
+ require_relative "approval_driver"
206
+
207
+ RSpec.describe Loan, type: :model do
208
+ def rolled_back
209
+ ActiveRecord::Base.transaction(requires_new: true, joinable: false) do
210
+ yield
211
+ raise ActiveRecord::Rollback
212
+ end
213
+ end
214
+
215
+ it "follows every trace Quint writes for approval.qnt" do
216
+ QuintDriver::Quint.simulate("spec/quint/approval.qnt", traces: 50, max_steps: 8, seed: "1") do |path|
217
+ rolled_back { QuintDriver::Trace.replay(path, ApprovalDriver.new) }
218
+ end
219
+ end
220
+ end
221
+ ```
222
+
223
+ <details>
224
+ <summary>spec/quint/approval.qnt</summary>
225
+
226
+ ```quint
227
+ // Two loans of one borrower with a limit of 20000 cents. A clerk approves
228
+ // up to 10000 cents, and a senior officer approves any amount. approve has
229
+ // a refused twin, so a trace checks the refusals too.
230
+ module approval {
231
+ type Outcome = Done | Refused(str)
232
+
233
+ pure val LOANS = Set("l1", "l2")
234
+ pure val AMOUNTS = Set(500, 10000, 15000)
235
+ pure val LIMIT = 20000
236
+
237
+ // 0 means the loan is not open yet.
238
+ var amount: str -> int
239
+ var approved: str -> bool
240
+ var outcome: Outcome
241
+
242
+ pure def approvedCents(a: str -> int, ap: str -> bool): int =
243
+ LOANS.filter(l => ap.get(l)).fold(0, (sum, l) => sum + a.get(l))
244
+ pure def refusal(a: str -> int, ap: str -> bool, l: str, senior: bool): str =
245
+ if (ap.get(l)) "already"
246
+ else if (approvedCents(a, ap) + a.get(l) > LIMIT) "over_limit"
247
+ else if (a.get(l) > 10000 and not(senior)) "needs_senior"
248
+ else ""
249
+
250
+ action init = all {
251
+ amount' = LOANS.mapBy(_ => 0),
252
+ approved' = LOANS.mapBy(_ => false),
253
+ outcome' = Done,
254
+ }
255
+
256
+ action open(l: str, a: int): bool = all {
257
+ amount.get(l) == 0,
258
+ amount' = amount.put(l, a),
259
+ approved' = approved,
260
+ outcome' = Done,
261
+ }
262
+ action approve(l: str, senior: bool): bool = all {
263
+ amount.get(l) > 0,
264
+ refusal(amount, approved, l, senior) == "",
265
+ amount' = amount,
266
+ approved' = approved.put(l, true),
267
+ outcome' = Done,
268
+ }
269
+ action approve_refused(l: str, senior: bool): bool = all {
270
+ amount.get(l) > 0,
271
+ refusal(amount, approved, l, senior) != "",
272
+ amount' = amount,
273
+ approved' = approved,
274
+ outcome' = Refused(refusal(amount, approved, l, senior)),
275
+ }
276
+
277
+ action step = any {
278
+ nondet l = LOANS.oneOf()
279
+ nondet a = AMOUNTS.oneOf()
280
+ any { open(l, a) },
281
+ nondet l = LOANS.oneOf()
282
+ nondet senior = Set(true, false).oneOf()
283
+ any { approve(l, senior), approve_refused(l, senior) },
284
+ }
285
+
286
+ val within_limit = approvedCents(amount, approved) <= LIMIT
287
+ }
288
+ ```
289
+
290
+ </details>
291
+
292
+ ```sh
293
+ QUINT=quint bundle exec rspec spec/quint/approval_spec.rb
294
+ ```
295
+
296
+ ```text
297
+ .
298
+
299
+ Finished in 1.22 seconds (files took 0.91182 seconds to load)
300
+ 1 example, 0 failures
301
+ ```
302
+
303
+ Each of the 50 traces ran on the real models and the test database, and rolled back. When `Loan#approve` asks for a senior officer at exactly 10000 cents (`amount_cents >= 10_000` in place of `> 10_000`), the same command fails at the first trace that approves such a loan:
304
+
305
+ ```text
306
+ QuintDriver::Mismatch:
307
+ quint: after step 6 `approve` {"l" => "l1", "senior" => false}, the driver differs from the trace
308
+ approved: trace {"l1" => true, "l2" => true}, driver {"l1" => false, "l2" => true}
309
+ outcome: trace "Done", driver ["Refused", "needs_senior"]
310
+ steps:
311
+ init {}
312
+ open {"a" => 500, "l" => "l2"}
313
+ approve {"l" => "l2", "senior" => false}
314
+ open {"a" => 10000, "l" => "l1"}
315
+ approve_refused {"l" => "l2", "senior" => false}
316
+ approve_refused {"l" => "l2", "senior" => true}
317
+ approve {"l" => "l1", "senior" => false}
318
+ trace: trace_1.itf.json of Quint.simulate("spec/quint/approval.qnt", traces: 50, max_steps: 8, seed: "1", step: "step", backend: "rust")
319
+ ```
320
+
321
+ To delete the rows in the driver's `init` instead of a savepoint, and for the reason the savepoint must not be joinable, see the [skill](skills/quint_driver/SKILL.md).
322
+
323
+ ## Try it without an app
324
+
325
+ ### Install
326
+
327
+ ```sh
328
+ gem install quint_driver rspec
329
+ npm install --global @informalsystems/quint@0.33.0
330
+ export QUINT=quint
331
+ ```
332
+
333
+ Here the gems install without a Gemfile. [Add the gem](#add-the-gem) says what `QUINT` does.
334
+
335
+ ### Write a spec, a class, and a test
336
+
337
+ Run this in an empty folder. It writes three files: a spec of a box office that sells at most 3 tickets, the class that implements it, and an RSpec file with the driver that connects the two.
338
+
339
+ ```sh
340
+ cat > tickets.qnt <<'QNT'
341
+ // A box office that sells at most 3 tickets.
342
+ module tickets {
343
+ var sold: int
344
+ var outcome: str
345
+
346
+ action init = all { sold' = 0, outcome' = "ok" }
347
+ action sell = all { sold < 3, sold' = sold + 1, outcome' = "ok" }
348
+ action sell_refused = all { sold >= 3, sold' = sold, outcome' = "sold_out" }
349
+ action refund = all { sold > 0, sold' = sold - 1, outcome' = "ok" }
350
+
351
+ action step = any { sell, sell_refused, refund }
352
+ }
353
+ QNT
354
+
355
+ cat > box_office.rb <<'RUBY'
356
+ class BoxOffice
357
+ LIMIT = 3
358
+
359
+ attr_reader :sold
360
+
361
+ def initialize = @sold = 0
362
+
363
+ def sell
364
+ return :sold_out if @sold >= LIMIT
365
+
366
+ @sold += 1
367
+ :ok
368
+ end
369
+
370
+ def refund
371
+ @sold -= 1
372
+ :ok
373
+ end
374
+ end
375
+ RUBY
376
+
377
+ cat > box_office_spec.rb <<'RUBY'
378
+ require "quint_driver"
379
+ require_relative "box_office"
380
+
381
+ # Runs each action of tickets.qnt on a BoxOffice and reads it back as the spec's variables.
382
+ class BoxOfficeDriver
383
+ def step(action, _picks)
384
+ @office = BoxOffice.new if action == "init"
385
+ @outcome =
386
+ case action
387
+ when "init" then "ok"
388
+ when "sell", "sell_refused" then @office.sell.to_s
389
+ when "refund" then @office.refund.to_s
390
+ end
391
+ end
392
+
393
+ def state = { "sold" => @office.sold, "outcome" => @outcome }
394
+ end
395
+
396
+ RSpec.describe BoxOffice do
397
+ it "sells tickets the way tickets.qnt says" do
398
+ QuintDriver::Quint.simulate("tickets.qnt", traces: 20, max_steps: 10, seed: "1") do |trace|
399
+ QuintDriver::Trace.replay(trace, BoxOfficeDriver.new)
400
+ end
401
+ end
402
+ end
403
+ RUBY
404
+ ```
405
+
406
+ ### Run it
407
+
408
+ ```sh
409
+ rspec box_office_spec.rb
410
+ ```
411
+
412
+ ```text
413
+ .
414
+
415
+ Finished in 0.47699 seconds (files took 0.08795 seconds to load)
416
+ 1 example, 0 failures
417
+ ```
418
+
419
+ Quint wrote 20 traces of up to 10 steps each, and the driver ran every step on a new `BoxOffice`. After each step, the class had the same `sold` and `outcome` as the spec.
420
+
421
+ ### Break it
422
+
423
+ Let the box office sell one ticket too many:
424
+
425
+ ```sh
426
+ ruby -pi -e 'sub("@sold >= LIMIT", "@sold > LIMIT")' box_office.rb
427
+ rspec box_office_spec.rb
428
+ ```
429
+
430
+ ```text
431
+ F
432
+
433
+ Failures:
434
+
435
+ 1) BoxOffice sells tickets the way tickets.qnt says
436
+ Failure/Error: QuintDriver::Trace.replay(trace, BoxOfficeDriver.new)
437
+
438
+ QuintDriver::Mismatch:
439
+ quint: after step 8 `sell_refused` {}, the driver differs from the trace
440
+ sold: trace 3, driver 4
441
+ outcome: trace "sold_out", driver "ok"
442
+ steps:
443
+ init {}
444
+ sell {}
445
+ refund {}
446
+ sell {}
447
+ sell {}
448
+ refund {}
449
+ sell {}
450
+ sell {}
451
+ sell_refused {}
452
+ trace: trace_2.itf.json of Quint.simulate("tickets.qnt", traces: 20, max_steps: 10, seed: "1", step: "step", backend: "rust")
453
+ # ./box_office_spec.rb:22:in 'block (3 levels) in <top (required)>'
454
+ # ./box_office_spec.rb:21:in 'block (2 levels) in <top (required)>'
455
+
456
+ Finished in 0.5298 seconds (files took 0.08509 seconds to load)
457
+ 1 example, 1 failure
458
+
459
+ Failed examples:
460
+
461
+ rspec ./box_office_spec.rb:20 # BoxOffice sells tickets the way tickets.qnt says
462
+ ```
463
+
464
+ The first line names the step where the class and the spec disagree: at step 8, with 3 tickets sold, the spec refuses the sale, and the class sells a 4th. `steps:` lists every step from `init`, so you can run the same commands by hand, and `trace:` names the call that writes the same trace again. Fix the class and run `rspec` again.
465
+
466
+ Next, [write a driver](#write-a-driver) for your own class, or let [Hegel](#run-it-1) choose the steps and shrink a failure to the shortest sequence.
467
+
468
+ ## How it works
469
+
470
+ There are two ways to choose the steps, and both use the same driver:
471
+
472
+ - **Trace replay.** `quint run --mbt` writes traces, and each trace runs on the system step by step. Quint picks the steps, so a trace reaches deep states quickly. The quick start uses this way.
473
+ - **Hegel.** [Hegel](https://github.com/meganemura/hegel-ruby) picks each step from the actions the spec allows now. When the system disagrees with the spec, Hegel shrinks the failure to a short sequence of commands.
474
+
475
+ The spec is read from Quint's JSON IR and evaluated inside the Ruby process, so a Hegel step does not start another process.
476
+
477
+ ## Installation
478
+
479
+ ```ruby
480
+ # Gemfile
481
+ gem "quint_driver"
482
+ gem "hegeltest", "= 0.5.1" # only for the Hegel adapter
483
+ ```
484
+
485
+ Trace replay and the model load with `require "quint_driver"`. They need Ruby 3.3 or later and no other gem.
486
+
487
+ The Hegel adapter loads with `require "quint_driver/hegel"`. It needs hegeltest 0.5.1, which needs Ruby 3.4 or later. The gem does not depend on hegeltest, so add it to your own Gemfile.
488
+
489
+ The gem calls the `quint` command line that `QUINT` names, as the [quick start](#add-the-gem) shows. Without `QUINT`, the gem runs `npx --yes @informalsystems/quint@0.33.0`. When npx cannot fetch quint, for example offline, the gem raises `QuintDriver::QuintFailed` that says so and asks you to set `QUINT`.
490
+
491
+ `Spec.new` compiles a file once per process and quint command, and compiles it again when the text of the file, or of a file it imports, changes. Every module of the file shares that compile, so a `Spec.new` or `Quint.simulate` for another `main:` of the same file starts no node. Threads that ask for the same file at once wait for one compile, so a parallel test run needs no memo of its own.
492
+
493
+ `Quint.simulate` runs Quint's Rust evaluator directly, without node, when it finds one: the copy quint downloaded into `QUINT_HOME/rust-evaluator-v0.7.0/` (`~/.quint` when `QUINT_HOME` is unset). Set `QUINT_EVALUATOR` to the evaluator to use another copy, or to an empty string to run `quint run` instead. The same seed gives the same traces on both ways. The gem reads only the input and output format of evaluator v0.7.0, the one quint 0.33.0 runs; an evaluator that answers in another format raises `QuintDriver::QuintFailed`. Without an evaluator, or with a `step:` that names no action of the module, `Quint.simulate` runs `quint run`.
494
+
495
+ ### The agent skill
496
+
497
+ The [quint_driver skill](skills/quint_driver/SKILL.md) takes a coding agent through the spec shape, the driver, trace replay, the model check, and Hegel. Install it with the [`skills`](https://github.com/vercel-labs/skills) CLI (Node.js 22.20 or later). This repository is private, so these commands need GitHub access.
498
+
499
+ ```sh
500
+ npx skills add meganemura/quint_driver
501
+ ```
502
+
503
+ ```sh
504
+ npx skills add meganemura/quint_driver --skill quint_driver
505
+ ```
506
+
507
+ ```sh
508
+ npx skills add meganemura/quint_driver -g
509
+ ```
510
+
511
+ ```sh
512
+ npx skills add meganemura/quint_driver -a claude-code
513
+ ```
514
+
515
+ `--skill` selects this skill, `-g` installs it for every project, and `-a` targets one agent. Add `-y` to skip prompts.
516
+
517
+ The gem ships the skill and the `examples/loans` spec, so after `bundle install` you can copy or symlink the installed copy instead, with no GitHub access:
518
+
519
+ ```sh
520
+ mkdir -p .agents/skills
521
+ ln -s "$(bundle info --path quint_driver)/skills/quint_driver" .agents/skills/quint_driver
522
+ ```
523
+
524
+ `.claude/skills/` and `.cursor/skills/` work the same way.
525
+
526
+ ## Write a driver
527
+
528
+ A driver runs the spec's actions on your system and reads the system back in the spec's terms.
529
+
530
+ ```ruby
531
+ class LoansDriver
532
+ # Runs one action of the spec. "init" comes first in every sequence.
533
+ # +picks+ holds the nondet values, such as { "m" => "m1", "b" => "b2" }.
534
+ def step(action, picks)
535
+ @outcome = "Done"
536
+ case action.delete_suffix("_refused")
537
+ when "init" then @loans = Loans.new
538
+ when "borrow" then @loans.borrow(picks.fetch("m"), picks.fetch("b"))
539
+ when "give_back" then @loans.give_back(picks.fetch("b"))
540
+ end
541
+ rescue Loans::Refused => refusal
542
+ @outcome = [ "Refused", refusal.reason.to_s ]
543
+ end
544
+
545
+ # Every variable of the spec, as plain Ruby values. One left out fails the step.
546
+ def state
547
+ { "holder" => %w[b1 b2 b3].to_h { [ _1, @loans.holder(_1) || "" ] }, "outcome" => @outcome }
548
+ end
549
+ end
550
+ ```
551
+
552
+ Plain values: a Quint set is a `Set`, a map a `Hash`, a tuple or a list an `Array`, a record a `Hash` with string keys, and a sum type its tag, or `[tag, value]` when it carries a value. A sum type with the cases `None` and `Some`, an Option, is `nil` for `None` and the value itself for `Some`, on a replay and under the machine alike.
553
+
554
+ A variable `state` leaves out fails the step and is named, such as `count: trace 0, but the driver's state leaves count out`, so a variable you forgot cannot pass. A spec can model the world outside your code, such as what a remote service answers, in variables the driver only reads to set up its stubs. Name those in `ignore:`, on `Trace.replay` or on `QuintDriver.machine`, and they are not compared.
555
+
556
+ [examples/loans](examples/loans) has the whole example: the spec, the system, and the driver.
557
+
558
+ ## Run it
559
+
560
+ ```ruby
561
+ require "quint_driver"
562
+
563
+ spec = QuintDriver::Spec.new("loans.qnt", main: "loans")
564
+
565
+ # Trace replay: Quint picks the steps.
566
+ QuintDriver::Quint.simulate("loans.qnt", traces: 50, max_steps: 20, seed: "1") do |path|
567
+ QuintDriver::Trace.replay(path, LoansDriver.new)
568
+ end
569
+
570
+ # Hegel picks the steps and shrinks a failure.
571
+ require "quint_driver/hegel"
572
+ machine = QuintDriver.machine(spec, driver: -> { LoansDriver.new }, invariants: %i[one_book_each])
573
+ Hegel.test(test_cases: 100) { |tc| Hegel::Stateful.run(machine.new, tc) }
574
+ ```
575
+
576
+ Both raise `QuintDriver::Mismatch` with each variable that differs and the steps the driver ran from `init`, so you can run the failure again by hand. Each value prints with a set's elements and a hash's keys sorted, so the spec's side and the driver's side print a record in one key order. The message ends with a `trace:` line. After `Quint.simulate` returns, the file it replayed is gone, so the line names the trace by its file name and the call that writes the same trace again, such as `trace: trace_0.itf.json of Quint.simulate("loans.qnt", traces: 50, max_steps: 20, seed: "1", step: "step", backend: "rust")`. The error reads the same inside the block, and holds those facts as `trace`, `seed`, and `written_by`. A `Mismatch` your own code raises inside the block, such as from a check of your own after the replay, gets the same `trace:` line when it leaves the block. Raise it with a message alone, `raise QuintDriver::Mismatch, "..."`: `Mismatch.new` takes no trace, seed, or call, because the gem names the trace that ran. A replay of a trace file of your own names its path. The backtrace of a `Mismatch` leaves out quint_driver's own frames, and Hegel's under the machine, so RSpec and Minitest print first the line of yours that called the replay or `Hegel.test`, or that raised it, also when your Gemfile loads the gem from a path. A pick the move did not make is absent from `picks`, and a pick it made of the spec's own `None` is present as `nil`, so read a pick an action may go without as `picks["name"]`, which is `nil` then, and a pick it needs as `picks.fetch("name")`. `Trace.replay` returns the steps it ran, each with its `action`, `picks`, and `expected` state, so you can count what the traces reached. To watch a replay, pass `log: $stdout` to `Trace.replay`; it prints each step, such as `step 1 borrow {"b" => "b1", "m" => "m2"}`, before the driver runs it. The code runs the same in a Minitest test and in an RSpec example. In RSpec, include `Hegel::Syntax::Methods` if your own examples draw values; the machine itself does not need it. In a Rails app, transactional fixtures roll back once per example, so every trace and every Hegel test case of one example shares the test database: delete the rows in the driver's `init`, or wrap each trace and each test case in `ActiveRecord::Base.transaction(requires_new: true, joinable: false) { ...; raise ActiveRecord::Rollback }`. The skill shows both and why the savepoint must not be joinable.
577
+
578
+ A trace holds the spec's variables, not its definitions, so a value the spec derives, such as `def approvedSum` over the approved loans, is compared only when you observe it. Pass `observe: %w[approvedSum]` with `spec:` to `Trace.replay`, or `observe:` to `QuintDriver.machine`, and report the value in the driver's `state` under the same name, such as `"approvedSum" => borrower.approved_total`. The driver is held to the value the definition has in each state, and a driver that reports no value for it fails. The model evaluates the definition, also on a replay, where it reads the trace's state, so the failure names the model: `approvedSum: model 0, driver 15000`. A name the driver reports that is in no state fails too, such as `dueCents: driver 0, but dueCents is not in the trace's state, which has amount, approvedBy, outcome`; when the name is a definition of the spec, the line says to name it in `observe:`. A name that is no `val` or `def` without parameters raises `ArgumentError` with the ones the module has.
579
+
580
+ An error a step raises comes out as `QuintDriver::StepRaised`, not as a `Mismatch`, because the spec and the system were never compared. The error can come from the driver's own code, such as a `KeyError` from `picks.fetch("member")` when the pick is named `m`, or from the system beneath it. Its message names the step and the error, such as ``quint: at step 1 `borrow` {"b" => "b2", "m" => "m2"}, the step raised KeyError: key not found: "member"``. A `raised in:` line then names the frame of your project nearest the raise, such as `raised in: loans_driver.rb:6:in 'Hash#fetch'`, with the path relative to the directory the run started in. Frames in a gem, in Ruby's library, and in quint_driver are skipped, so an error that a gem raises names your line that called the gem. When every frame is skipped, the line says so: `raised in: no frame of the project; every frame is in a gem, in Ruby's library, or in quint_driver`. Then read the backtrace. Under Hegel the step number is the one Hegel's report gives the move, such as `Step 2: take`, and `init` is step 0, as on a replay. Then the message lists the steps from `init`, and on a replay ends with the same `trace:` line. Its backtrace is the error's without quint_driver's own frames, and `cause` is the error.
581
+
582
+ `Quint.simulate` runs the module named after the file. Pass `main:` to run another module of that file, the same name `Spec.new` takes. When quint fails or is not installed, `Spec.new` and `Quint.simulate` raise `QuintDriver::QuintFailed` with quint's reason. Its backtrace starts at the line of yours that called the gem, with no frame of quint_driver before it.
583
+
584
+ To test the steps that lead to one state, pass `reach:` the name of a definition that holds there, such as `reach: "two_on_loan"` for `val two_on_loan = BOOKS.filter(b => onLoan(holder, b)).size() == 2`. Each trace then ends at the first state where the definition holds, and `Quint.simulate` yields only the traces that get there. Quint checks `not(two_on_loan)` as the invariant and stops a trace where it breaks. When no trace gets there, it raises `QuintDriver::QuintFailed`, such as `quint: no trace of 20 reached two_on_loan within 1 steps; raise max_steps or traces`. `reach:` takes a predicate, a boolean `val` or `def` without parameters, and `step:` an action without parameters. A name of another kind raises `ArgumentError` before quint runs, such as `quint: reach names borrow, an action with parameters of loans; reach takes a predicate, a boolean val or def without parameters, such as nothing_on_loan, one_book_each, two_on_loan`. A name the module neither declares nor imports raises `ArgumentError` the same way once quint has run, such as `quint: step names borow, which loans neither declares nor imports; step takes an action without parameters, such as step`. Every trace, every model, and every machine starts at the action named `init`, as quint does by default. A module without one raises `ArgumentError`, such as ``quint: begins neither declares nor imports an action named init; quint_driver starts every trace and every model at init, so declare `action init = ...` with what starts begins``. quint_driver takes no other name for it: a trace names its first step after that action, and the driver and the model read that step as `init`. Add `action init = begin` to the module instead. The backtrace of each `ArgumentError` quint_driver raises starts at the line of yours that called it, with no frame of quint_driver before it.
585
+
586
+ `Quint.test(path, "lendTest", seed: "1") { |trace| ... }` writes the trace of a named run, such as `run lendTest = init.then(borrow("m1", "b1")).then(give_back("b1"))`, and yields its file; pass `main:` when the run is in another module of the file. quint test writes no `mbt::actionTaken`, so the spec has to record the action each step took, as a [Choreo](https://github.com/informalsystems/choreo) spec does: a variable that holds a sum type whose tag names the action and whose record holds its picks, such as `taken: Borrow({ m: m, b: b })`. Pass the path to that variable as `action:` to `Trace.replay`. When the state the driver answers for sits inside a variable, such as the `system` field of a Choreo spec's one variable, pass its path as `state:`; the driver's `state` then returns that record or map. `test/recorded.qnt` is a small spec of this shape, and `RecordedDriver` in `test/trace_test.rb` is its driver:
587
+
588
+ ```ruby
589
+ QuintDriver::Quint.test("test/recorded.qnt", "lendTest", seed: "1") do |trace|
590
+ QuintDriver::Trace.replay(trace, RecordedDriver.new, state: %w[s books], action: %w[s taken])
591
+ end
592
+ ```
593
+
594
+ The driver gets `step("Init", {})`, then `step("Borrow", { "m" => "m1", "b" => "b1" })`, and so on. A failure names the trace as `trace: test_0.itf.json of Quint.test("test/recorded.qnt", "lendTest", seed: "1")`. A run that fails in quint itself raises `QuintDriver::QuintFailed` with quint's report, and a name that is no run of the module lists the runs it has, such as `quint: no run is named lend in recorded; it has lendTest, unlentTest`. A trace without the action, such as a quint test trace replayed without `action:`, raises `QuintDriver::Unsupported` that asks for the path.
595
+
596
+ `QuintDriver.machine` takes these options:
597
+
598
+ - `step:` is the action that lists the moves, `"step"` by default. Each move must call a named action. Arguments come from `nondet x = S.oneOf()`. A step that is one `all` around its picks and an `any`, such as `all { nondet x = S.oneOf() any { add(x), sub(x) } }`, moves by the actions of the `any`. A `step:` that is no action of the module raises `ArgumentError` with the actions it has, such as `quint: no action is named steps; the module has counting, init, step`.
599
+ - `invariants:` names `val`s of the spec that must hold after every move.
600
+ - `choose:` is how Hegel draws a move. `:rule` (the default) picks an action and rejects it when the spec disables it. It shrinks to the shortest sequence. `:rotate` passes a disabled action's turn to the next enabled one. It reaches deep states with fewer test cases, but a shrunk sequence can keep extra moves. Hegel's failure report names each draw after the nondet pick it chose, or `rule` and `move` under `:rotate`. A label drawn more than once in a test case is numbered in draw order (`m_1 = "m2"`, `m_2 = "m1"`); a label drawn once prints bare (`m = "m2"`).
601
+
602
+ Each test case starts a new system through the driver's `init` when the machine starts, before Hegel's first step, so a driver that fails at `init` fails with no step in Hegel's report. A sequence of moves that the system already agreed on is not run again, because shrinking asks for the same sequence many times. This assumes that the system answers the same sequence the same way each time. A disagreement always runs on the system again.
603
+
604
+ A Hegel run has no Quint seed. Hegel's own report ends with a blob, such as `reproduce_failure: "AXicXYmhDQAADMJWu/3/70BgQJCmvdEWPwRiKlCBBwzbAGE="`; pass it to `Hegel.test` as `reproduce_failure:` to run that one failing case again, with the same message.
605
+
606
+ ## Check the Ruby model
607
+
608
+ The Hegel way trusts the Ruby evaluation of the spec. The model answers the driver protocol, so you can hold it to Quint's own traces. A module that imports definitions or actions from another module or file reads them as its own, as quint does when it runs the module. It keeps every definition it imports, also one that nothing in the module reaches, and it needs no action named `step`; that costs one more compile for that module:
609
+
610
+ ```ruby
611
+ QuintDriver::Quint.simulate("loans.qnt", traces: 200, max_steps: 30, seed: "7") do |path|
612
+ QuintDriver::Trace.replay(path, spec.model)
613
+ end
614
+ ```
615
+
616
+ The model evaluates the operators a command-per-action spec uses: sets (with `fold` and `i.to(j)`), maps, records, tuples, lists, sum types with `match`, integers, and booleans, with `Bool` as the set of `false` and `true`. `Int` and `Nat` are infinite, so the model cannot list the picks of a move drawn from them, and raises `QuintDriver::Unsupported`; draw from a finite set such as `0.to(9)` instead. On a map it evaluates `Map`, `get`, `keys`, `put`, `mapBy`, `set`, `setBy`, and `setToMap`. A list is a Ruby Array, and the model evaluates `List`, `append`, `concat`, `head`, `tail`, `length`, `nth`, `indices`, `replaceAt`, `slice`, `range`, `select`, `foldl`, and `foldr`. Quint's builtin docs leave `foldr` out, but both of its evaluators fold from the right and pass the element first, `(x, acc) => ...`, and the model does the same. On integers it evaluates `+`, `-`, `*`, unary `-`, and the comparisons, and `/` and `%` only on non-negative operands, because Quint documents no rounding for a negative one. Both of Quint's evaluators truncate toward zero (`-7 % 2` is `-1`), and Ruby floors (`-7 % 2` is `1`), so the model refuses a negative operand rather than answer differently. Where Quint's docs leave an operation undefined, such as `head` of an empty list, `nth` past the end, or `setToMap` of two pairs with one key, the model raises `QuintDriver::Unsupported` instead of picking an answer. `get`, `set`, and `setBy` at a key the map does not have raise it too, because Quint's evaluator stops there; `slice(l, l.length(), l.length())` is `[]`, as both of Quint's evaluators give it. Any other operator, such as `powerset`, raises `QuintDriver::Unsupported`, so a spec that needs more fails loudly. So does a step arm that names no action, and its message names the action that holds the arm, such as `quint: counting assigns n' in an arm that names no action; name every action in step` when `step` opens `counting` in place. The message names the action or definition that holds the operator, such as `quint: no evaluation for operator powerset in subsets`. For an operator in a nondet domain, that is the action that draws it, which can be an action that `step` opens in place.
617
+
618
+ `Spec#model` is the one way to build a model; `Model.new` is private, because the model trusts the shape of quint's compiled IR. A trace gives each pick as a plain value, and the model takes the value of the pick's domain that reads the same, so a pick of a sum type works in `match`.
619
+
620
+ `model.moves(rule)` lists every move a rule of `spec.rules("step")` allows in the model's current state, as `[action, picks, next state]` in a fixed order. `model.enter(next_state)` takes one. The Hegel machine draws its moves this way. `model.value(node, env)` evaluates one IR node, or the definition a string names, in the current state. A refusal names the definition that writes the node, such as the action a domain or an argument of a rule is written in; for a node no definition of the spec holds, the message says the node is `outside any definition`. `rule.fixed` is the set of domain names that are the same set in every state and after every earlier pick: Quint's effect of the domain reads no variable, and the domain names no earlier pick. The Hegel machine under `:rule` evaluates each of them once per machine.
621
+
622
+ ## Development
623
+
624
+ ```sh
625
+ bundle install
626
+ bundle exec bin/test # Minitest
627
+ bundle exec rspec # RSpec
628
+ ```
629
+
630
+ The Hegel tests are under `test/hegel/` and `spec/hegel/`. On Ruby 3.3, leave out hegeltest and run the rest:
631
+
632
+ ```sh
633
+ BUNDLE_WITHOUT=hegel bundle install
634
+ BUNDLE_WITHOUT=hegel bundle exec bin/test test/*_test.rb
635
+ BUNDLE_WITHOUT=hegel bundle exec rspec --exclude-pattern "spec/hegel/**/*_spec.rb"
636
+ ```
637
+
638
+ The tests run `quint`. Set `QUINT` to skip `npx`.
639
+
640
+ `test/readme_test.rb` runs "Try it without an app" in an empty folder. It runs the Rails quick start and "What you will see" only when `QUINT_DRIVER_README_APP` names a Rails app to copy, with rspec-rails, `Borrower`, and `Loan` as in the sample app, and no quint_driver in its Gemfile. The test adds the gem built from this checkout, writes the files, runs the command, applies the diff, and compares each output with the README.
641
+
642
+ `test/skill_test.rb` runs the skill's example "A choice the code leaves to the database" only when `QUINT_DRIVER_SKILL_APP` names a Rails app to copy, with rspec-rails, an `Item` that belongs to a `Batch` and may not be assigned while blocked, the `/session` and `/items/:id/release` routes, and no quint_driver in its Gemfile. The test adds this checkout, writes the skill's files, replays the traces, then removes the wrap and compares the failure with the skill. It uses the app's database, so `DATABASE_URL` runs it on PostgreSQL.
643
+
644
+ ## License
645
+
646
+ MIT
647
+
648
+ [Japanese](README.ja.md)