quint_driver 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE +21 -0
- data/README.ja.md +865 -0
- data/README.md +648 -0
- data/examples/loans/loans.qnt +57 -0
- data/examples/loans/loans.rb +62 -0
- data/lib/quint_driver/hegel.rb +141 -0
- data/lib/quint_driver/model.rb +289 -0
- data/lib/quint_driver/quint.rb +376 -0
- data/lib/quint_driver/spec.rb +110 -0
- data/lib/quint_driver/trace.rb +129 -0
- data/lib/quint_driver/version.rb +3 -0
- data/lib/quint_driver.rb +199 -0
- data/skills/quint_driver/SKILL.md +925 -0
- metadata +55 -0
data/README.md
ADDED
|
@@ -0,0 +1,648 @@
|
|
|
1
|
+
# quint_driver
|
|
2
|
+
|
|
3
|
+
quint_driver holds your Ruby code to a [Quint](https://github.com/quint-co/quint) spec: the spec says what each command must do, and quint_driver runs the same commands on your code and compares the two after every step.
|
|
4
|
+
|
|
5
|
+
## Quick start in your Rails app
|
|
6
|
+
|
|
7
|
+
### Add the gem
|
|
8
|
+
|
|
9
|
+
```ruby
|
|
10
|
+
# Gemfile
|
|
11
|
+
group :test do
|
|
12
|
+
gem "quint_driver", require: false
|
|
13
|
+
end
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
bundle install
|
|
18
|
+
npm install --global @informalsystems/quint@0.33.0
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
quint_driver needs Ruby 3.3 or later. It calls the `quint` command that `QUINT` names, so the run below sets `QUINT=quint`; put `export QUINT=quint` in your shell profile to leave it out. Without `QUINT`, the gem runs `npx --yes @informalsystems/quint@0.33.0`, which downloads quint (43 MB) on its first run and starts node once more on every run. The first quint run also downloads quint's Rust evaluator from GitHub into `~/.quint`. When your network blocks that download, the [skill](skills/quint_driver/SKILL.md#install) shows how to fetch it by hand.
|
|
22
|
+
|
|
23
|
+
### Pick a first model
|
|
24
|
+
|
|
25
|
+
Pick a model with a status column, such as `state` or `status`, and a method that changes it, such as `Order#cancel` or `Loan#approve`. A model whose status a recent commit changed is a good start, because the spec then checks that change: `git log --oneline -- app/models` lists those commits.
|
|
26
|
+
|
|
27
|
+
The steps below use `Loan`. A new loan is `"pending"`, and `Loan#approve(officer)` makes it `"approved"`. Replace `Loan`, the record that the driver's `init` creates, the method, and the state names with yours.
|
|
28
|
+
|
|
29
|
+
### Write the spec
|
|
30
|
+
|
|
31
|
+
```quint
|
|
32
|
+
// spec/quint/loan_state.qnt
|
|
33
|
+
// A loan starts pending. Approving it once makes it approved, and
|
|
34
|
+
// approving it again leaves it approved.
|
|
35
|
+
module loan_state {
|
|
36
|
+
var state: str
|
|
37
|
+
|
|
38
|
+
action init = state' = "pending"
|
|
39
|
+
action approve = all { state == "pending", state' = "approved" }
|
|
40
|
+
action approve_refused = all { state == "approved", state' = state }
|
|
41
|
+
|
|
42
|
+
action step = any { approve, approve_refused }
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
The `var` is the column. `init` is the value a new record has. Every other action is one call of the method, and its first condition says when the call does what the action says: `approve` makes a pending loan approved, and `approve_refused` leaves an approved loan as it is. `step` lists the actions that a trace picks from. `Quint.simulate` runs the module named after the file, so keep the two names the same.
|
|
47
|
+
|
|
48
|
+
### Write the driver
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
# spec/quint/loan_state_driver.rb
|
|
52
|
+
# Runs each action of loan_state.qnt on a Loan and reads its state back.
|
|
53
|
+
class LoanStateDriver
|
|
54
|
+
def step(action, _picks)
|
|
55
|
+
case action
|
|
56
|
+
when "init"
|
|
57
|
+
borrower = Borrower.create!(name: "Ada", limit_cents: 20_000)
|
|
58
|
+
@loan = Loan.create!(borrower: borrower, amount_cents: 500)
|
|
59
|
+
when "approve", "approve_refused"
|
|
60
|
+
@loan.approve({name: "Grace", senior: true})
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def state = {"state" => @loan.reload.state}
|
|
65
|
+
end
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
`step` runs one action on a real record: `init` creates it, and every other action calls the method. `state` returns the spec's variables, read back from the database.
|
|
69
|
+
|
|
70
|
+
### Write the example
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
# spec/quint/loan_state_spec.rb
|
|
74
|
+
require "rails_helper"
|
|
75
|
+
require "quint_driver"
|
|
76
|
+
require_relative "loan_state_driver"
|
|
77
|
+
|
|
78
|
+
RSpec.describe Loan, type: :model do
|
|
79
|
+
# The transactional fixture rolls back once, after the example. Each trace
|
|
80
|
+
# rolls back to a savepoint of its own, so no trace sees the rows of another.
|
|
81
|
+
def rolled_back
|
|
82
|
+
ActiveRecord::Base.transaction(requires_new: true, joinable: false) do
|
|
83
|
+
yield
|
|
84
|
+
raise ActiveRecord::Rollback
|
|
85
|
+
end
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
it "moves its state the way loan_state.qnt says" do
|
|
89
|
+
QuintDriver::Quint.simulate("spec/quint/loan_state.qnt", traces: 20, max_steps: 5, seed: "1") do |trace|
|
|
90
|
+
rolled_back { QuintDriver::Trace.replay(trace, LoanStateDriver.new) }
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Quint writes 20 traces of up to 5 steps, and each trace runs on a new loan.
|
|
97
|
+
|
|
98
|
+
### Run the example
|
|
99
|
+
|
|
100
|
+
```sh
|
|
101
|
+
QUINT=quint bundle exec rspec spec/quint/loan_state_spec.rb
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## What you will see
|
|
105
|
+
|
|
106
|
+
This output is from a small sample app, a lending desk with a `Borrower` and a `Loan` model.
|
|
107
|
+
|
|
108
|
+
```text
|
|
109
|
+
.
|
|
110
|
+
|
|
111
|
+
Finished in 0.60298 seconds (files took 0.88104 seconds to load)
|
|
112
|
+
1 example, 0 failures
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
Every trace created a loan in the test database, approved it, approved it again, and read the state back after each step. After each step, the loan had the state the spec has.
|
|
116
|
+
|
|
117
|
+
Now a commit makes a new loan a draft:
|
|
118
|
+
|
|
119
|
+
```diff
|
|
120
|
+
--- a/app/models/loan.rb
|
|
121
|
+
+++ b/app/models/loan.rb
|
|
122
|
+
def mark_pending
|
|
123
|
+
- self.state ||= "pending"
|
|
124
|
+
+ self.state ||= "draft"
|
|
125
|
+
end
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
```sh
|
|
129
|
+
QUINT=quint bundle exec rspec spec/quint/loan_state_spec.rb
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
```text
|
|
133
|
+
F
|
|
134
|
+
|
|
135
|
+
Failures:
|
|
136
|
+
|
|
137
|
+
1) Loan moves its state the way loan_state.qnt says
|
|
138
|
+
Failure/Error: rolled_back { QuintDriver::Trace.replay(trace, LoanStateDriver.new) }
|
|
139
|
+
|
|
140
|
+
QuintDriver::Mismatch:
|
|
141
|
+
quint: after step 0 `init` {}, the driver differs from the trace
|
|
142
|
+
state: trace "pending", driver "draft"
|
|
143
|
+
steps:
|
|
144
|
+
init {}
|
|
145
|
+
trace: trace_0.itf.json of Quint.simulate("spec/quint/loan_state.qnt", traces: 20, max_steps: 5, seed: "1", step: "step", backend: "rust")
|
|
146
|
+
# ./spec/quint/loan_state_spec.rb:18:in 'block (4 levels) in <top (required)>'
|
|
147
|
+
# ./spec/quint/loan_state_spec.rb:11:in 'block in RSpec::ExampleGroups::Loan#rolled_back'
|
|
148
|
+
# ./spec/quint/loan_state_spec.rb:10:in 'RSpec::ExampleGroups::Loan#rolled_back'
|
|
149
|
+
# ./spec/quint/loan_state_spec.rb:18:in 'block (3 levels) in <top (required)>'
|
|
150
|
+
# ./spec/quint/loan_state_spec.rb:17:in 'block (2 levels) in <top (required)>'
|
|
151
|
+
|
|
152
|
+
Finished in 0.61377 seconds (files took 0.90689 seconds to load)
|
|
153
|
+
1 example, 1 failure
|
|
154
|
+
|
|
155
|
+
Failed examples:
|
|
156
|
+
|
|
157
|
+
rspec ./spec/quint/loan_state_spec.rb:16 # Loan moves its state the way loan_state.qnt says
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
The first line names the step where the model and the spec differ: right after `init`, the spec has a pending loan, and the model has a draft. `steps:` lists every step from `init`, so you can run the same commands by hand, and `trace:` names the call that writes the same trace again. When the change is a bug, fix the model. When the change is what you want, change `init` in the spec in the same commit, so the spec keeps saying what the app does.
|
|
161
|
+
|
|
162
|
+
Next, give an action a pick, such as the officer or the amount, and compare a second variable, such as the outcome of each call: [a bigger spec](#a-bigger-spec-refusals-and-picks) does both.
|
|
163
|
+
|
|
164
|
+
## A bigger spec: refusals and picks
|
|
165
|
+
|
|
166
|
+
The quick start's spec has one record and one variable. This spec of the same app has two loans of one borrower, picks the amount and the officer for each action, and compares the outcome of every call too. The driver keeps the id each loan of the spec got, and reads the rows back as the spec's variables. `Loan#approve(officer)` refuses a loan that is already approved, one over the borrower's limit, and one above 10000 cents when the officer is not senior:
|
|
167
|
+
|
|
168
|
+
```ruby
|
|
169
|
+
# spec/quint/approval_driver.rb
|
|
170
|
+
class ApprovalDriver
|
|
171
|
+
LOANS = %w[l1 l2].freeze
|
|
172
|
+
|
|
173
|
+
def step(action, picks)
|
|
174
|
+
@outcome = "Done"
|
|
175
|
+
case action.delete_suffix("_refused")
|
|
176
|
+
when "init"
|
|
177
|
+
@borrower = Borrower.create!(name: "Ada", limit_cents: 20_000)
|
|
178
|
+
@ids = {}
|
|
179
|
+
when "open"
|
|
180
|
+
@ids[picks.fetch("l")] = LoanDesk.open(@borrower, picks.fetch("a")).id
|
|
181
|
+
when "approve"
|
|
182
|
+
officer = {name: "Grace", senior: picks.fetch("senior")}
|
|
183
|
+
result = Loan.find(@ids.fetch(picks.fetch("l"))).approve(officer)
|
|
184
|
+
@outcome = ["Refused", result.to_s] unless result == :approved
|
|
185
|
+
end
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
def state
|
|
189
|
+
loans = LOANS.to_h { [_1, @ids[_1] && Loan.find(@ids[_1])] }
|
|
190
|
+
{
|
|
191
|
+
"amount" => loans.transform_values { _1&.amount_cents || 0 },
|
|
192
|
+
"approved" => loans.transform_values { _1&.state == "approved" },
|
|
193
|
+
"outcome" => @outcome
|
|
194
|
+
}
|
|
195
|
+
end
|
|
196
|
+
end
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
A transactional fixture rolls back once, after the example, but one example replays many traces. Wrap each trace in a savepoint of its own, so no trace reads the rows of another:
|
|
200
|
+
|
|
201
|
+
```ruby
|
|
202
|
+
# spec/quint/approval_spec.rb
|
|
203
|
+
require "rails_helper"
|
|
204
|
+
require "quint_driver"
|
|
205
|
+
require_relative "approval_driver"
|
|
206
|
+
|
|
207
|
+
RSpec.describe Loan, type: :model do
|
|
208
|
+
def rolled_back
|
|
209
|
+
ActiveRecord::Base.transaction(requires_new: true, joinable: false) do
|
|
210
|
+
yield
|
|
211
|
+
raise ActiveRecord::Rollback
|
|
212
|
+
end
|
|
213
|
+
end
|
|
214
|
+
|
|
215
|
+
it "follows every trace Quint writes for approval.qnt" do
|
|
216
|
+
QuintDriver::Quint.simulate("spec/quint/approval.qnt", traces: 50, max_steps: 8, seed: "1") do |path|
|
|
217
|
+
rolled_back { QuintDriver::Trace.replay(path, ApprovalDriver.new) }
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
end
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
<details>
|
|
224
|
+
<summary>spec/quint/approval.qnt</summary>
|
|
225
|
+
|
|
226
|
+
```quint
|
|
227
|
+
// Two loans of one borrower with a limit of 20000 cents. A clerk approves
|
|
228
|
+
// up to 10000 cents, and a senior officer approves any amount. approve has
|
|
229
|
+
// a refused twin, so a trace checks the refusals too.
|
|
230
|
+
module approval {
|
|
231
|
+
type Outcome = Done | Refused(str)
|
|
232
|
+
|
|
233
|
+
pure val LOANS = Set("l1", "l2")
|
|
234
|
+
pure val AMOUNTS = Set(500, 10000, 15000)
|
|
235
|
+
pure val LIMIT = 20000
|
|
236
|
+
|
|
237
|
+
// 0 means the loan is not open yet.
|
|
238
|
+
var amount: str -> int
|
|
239
|
+
var approved: str -> bool
|
|
240
|
+
var outcome: Outcome
|
|
241
|
+
|
|
242
|
+
pure def approvedCents(a: str -> int, ap: str -> bool): int =
|
|
243
|
+
LOANS.filter(l => ap.get(l)).fold(0, (sum, l) => sum + a.get(l))
|
|
244
|
+
pure def refusal(a: str -> int, ap: str -> bool, l: str, senior: bool): str =
|
|
245
|
+
if (ap.get(l)) "already"
|
|
246
|
+
else if (approvedCents(a, ap) + a.get(l) > LIMIT) "over_limit"
|
|
247
|
+
else if (a.get(l) > 10000 and not(senior)) "needs_senior"
|
|
248
|
+
else ""
|
|
249
|
+
|
|
250
|
+
action init = all {
|
|
251
|
+
amount' = LOANS.mapBy(_ => 0),
|
|
252
|
+
approved' = LOANS.mapBy(_ => false),
|
|
253
|
+
outcome' = Done,
|
|
254
|
+
}
|
|
255
|
+
|
|
256
|
+
action open(l: str, a: int): bool = all {
|
|
257
|
+
amount.get(l) == 0,
|
|
258
|
+
amount' = amount.put(l, a),
|
|
259
|
+
approved' = approved,
|
|
260
|
+
outcome' = Done,
|
|
261
|
+
}
|
|
262
|
+
action approve(l: str, senior: bool): bool = all {
|
|
263
|
+
amount.get(l) > 0,
|
|
264
|
+
refusal(amount, approved, l, senior) == "",
|
|
265
|
+
amount' = amount,
|
|
266
|
+
approved' = approved.put(l, true),
|
|
267
|
+
outcome' = Done,
|
|
268
|
+
}
|
|
269
|
+
action approve_refused(l: str, senior: bool): bool = all {
|
|
270
|
+
amount.get(l) > 0,
|
|
271
|
+
refusal(amount, approved, l, senior) != "",
|
|
272
|
+
amount' = amount,
|
|
273
|
+
approved' = approved,
|
|
274
|
+
outcome' = Refused(refusal(amount, approved, l, senior)),
|
|
275
|
+
}
|
|
276
|
+
|
|
277
|
+
action step = any {
|
|
278
|
+
nondet l = LOANS.oneOf()
|
|
279
|
+
nondet a = AMOUNTS.oneOf()
|
|
280
|
+
any { open(l, a) },
|
|
281
|
+
nondet l = LOANS.oneOf()
|
|
282
|
+
nondet senior = Set(true, false).oneOf()
|
|
283
|
+
any { approve(l, senior), approve_refused(l, senior) },
|
|
284
|
+
}
|
|
285
|
+
|
|
286
|
+
val within_limit = approvedCents(amount, approved) <= LIMIT
|
|
287
|
+
}
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
</details>
|
|
291
|
+
|
|
292
|
+
```sh
|
|
293
|
+
QUINT=quint bundle exec rspec spec/quint/approval_spec.rb
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
```text
|
|
297
|
+
.
|
|
298
|
+
|
|
299
|
+
Finished in 1.22 seconds (files took 0.91182 seconds to load)
|
|
300
|
+
1 example, 0 failures
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
Each of the 50 traces ran on the real models and the test database, and rolled back. When `Loan#approve` asks for a senior officer at exactly 10000 cents (`amount_cents >= 10_000` in place of `> 10_000`), the same command fails at the first trace that approves such a loan:
|
|
304
|
+
|
|
305
|
+
```text
|
|
306
|
+
QuintDriver::Mismatch:
|
|
307
|
+
quint: after step 6 `approve` {"l" => "l1", "senior" => false}, the driver differs from the trace
|
|
308
|
+
approved: trace {"l1" => true, "l2" => true}, driver {"l1" => false, "l2" => true}
|
|
309
|
+
outcome: trace "Done", driver ["Refused", "needs_senior"]
|
|
310
|
+
steps:
|
|
311
|
+
init {}
|
|
312
|
+
open {"a" => 500, "l" => "l2"}
|
|
313
|
+
approve {"l" => "l2", "senior" => false}
|
|
314
|
+
open {"a" => 10000, "l" => "l1"}
|
|
315
|
+
approve_refused {"l" => "l2", "senior" => false}
|
|
316
|
+
approve_refused {"l" => "l2", "senior" => true}
|
|
317
|
+
approve {"l" => "l1", "senior" => false}
|
|
318
|
+
trace: trace_1.itf.json of Quint.simulate("spec/quint/approval.qnt", traces: 50, max_steps: 8, seed: "1", step: "step", backend: "rust")
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
To delete the rows in the driver's `init` instead of a savepoint, and for the reason the savepoint must not be joinable, see the [skill](skills/quint_driver/SKILL.md).
|
|
322
|
+
|
|
323
|
+
## Try it without an app
|
|
324
|
+
|
|
325
|
+
### Install
|
|
326
|
+
|
|
327
|
+
```sh
|
|
328
|
+
gem install quint_driver rspec
|
|
329
|
+
npm install --global @informalsystems/quint@0.33.0
|
|
330
|
+
export QUINT=quint
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
Here the gems install without a Gemfile. [Add the gem](#add-the-gem) says what `QUINT` does.
|
|
334
|
+
|
|
335
|
+
### Write a spec, a class, and a test
|
|
336
|
+
|
|
337
|
+
Run this in an empty folder. It writes three files: a spec of a box office that sells at most 3 tickets, the class that implements it, and an RSpec file with the driver that connects the two.
|
|
338
|
+
|
|
339
|
+
```sh
|
|
340
|
+
cat > tickets.qnt <<'QNT'
|
|
341
|
+
// A box office that sells at most 3 tickets.
|
|
342
|
+
module tickets {
|
|
343
|
+
var sold: int
|
|
344
|
+
var outcome: str
|
|
345
|
+
|
|
346
|
+
action init = all { sold' = 0, outcome' = "ok" }
|
|
347
|
+
action sell = all { sold < 3, sold' = sold + 1, outcome' = "ok" }
|
|
348
|
+
action sell_refused = all { sold >= 3, sold' = sold, outcome' = "sold_out" }
|
|
349
|
+
action refund = all { sold > 0, sold' = sold - 1, outcome' = "ok" }
|
|
350
|
+
|
|
351
|
+
action step = any { sell, sell_refused, refund }
|
|
352
|
+
}
|
|
353
|
+
QNT
|
|
354
|
+
|
|
355
|
+
cat > box_office.rb <<'RUBY'
|
|
356
|
+
class BoxOffice
|
|
357
|
+
LIMIT = 3
|
|
358
|
+
|
|
359
|
+
attr_reader :sold
|
|
360
|
+
|
|
361
|
+
def initialize = @sold = 0
|
|
362
|
+
|
|
363
|
+
def sell
|
|
364
|
+
return :sold_out if @sold >= LIMIT
|
|
365
|
+
|
|
366
|
+
@sold += 1
|
|
367
|
+
:ok
|
|
368
|
+
end
|
|
369
|
+
|
|
370
|
+
def refund
|
|
371
|
+
@sold -= 1
|
|
372
|
+
:ok
|
|
373
|
+
end
|
|
374
|
+
end
|
|
375
|
+
RUBY
|
|
376
|
+
|
|
377
|
+
cat > box_office_spec.rb <<'RUBY'
|
|
378
|
+
require "quint_driver"
|
|
379
|
+
require_relative "box_office"
|
|
380
|
+
|
|
381
|
+
# Runs each action of tickets.qnt on a BoxOffice and reads it back as the spec's variables.
|
|
382
|
+
class BoxOfficeDriver
|
|
383
|
+
def step(action, _picks)
|
|
384
|
+
@office = BoxOffice.new if action == "init"
|
|
385
|
+
@outcome =
|
|
386
|
+
case action
|
|
387
|
+
when "init" then "ok"
|
|
388
|
+
when "sell", "sell_refused" then @office.sell.to_s
|
|
389
|
+
when "refund" then @office.refund.to_s
|
|
390
|
+
end
|
|
391
|
+
end
|
|
392
|
+
|
|
393
|
+
def state = { "sold" => @office.sold, "outcome" => @outcome }
|
|
394
|
+
end
|
|
395
|
+
|
|
396
|
+
RSpec.describe BoxOffice do
|
|
397
|
+
it "sells tickets the way tickets.qnt says" do
|
|
398
|
+
QuintDriver::Quint.simulate("tickets.qnt", traces: 20, max_steps: 10, seed: "1") do |trace|
|
|
399
|
+
QuintDriver::Trace.replay(trace, BoxOfficeDriver.new)
|
|
400
|
+
end
|
|
401
|
+
end
|
|
402
|
+
end
|
|
403
|
+
RUBY
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
### Run it
|
|
407
|
+
|
|
408
|
+
```sh
|
|
409
|
+
rspec box_office_spec.rb
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
```text
|
|
413
|
+
.
|
|
414
|
+
|
|
415
|
+
Finished in 0.47699 seconds (files took 0.08795 seconds to load)
|
|
416
|
+
1 example, 0 failures
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
Quint wrote 20 traces of up to 10 steps each, and the driver ran every step on a new `BoxOffice`. After each step, the class had the same `sold` and `outcome` as the spec.
|
|
420
|
+
|
|
421
|
+
### Break it
|
|
422
|
+
|
|
423
|
+
Let the box office sell one ticket too many:
|
|
424
|
+
|
|
425
|
+
```sh
|
|
426
|
+
ruby -pi -e 'sub("@sold >= LIMIT", "@sold > LIMIT")' box_office.rb
|
|
427
|
+
rspec box_office_spec.rb
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
```text
|
|
431
|
+
F
|
|
432
|
+
|
|
433
|
+
Failures:
|
|
434
|
+
|
|
435
|
+
1) BoxOffice sells tickets the way tickets.qnt says
|
|
436
|
+
Failure/Error: QuintDriver::Trace.replay(trace, BoxOfficeDriver.new)
|
|
437
|
+
|
|
438
|
+
QuintDriver::Mismatch:
|
|
439
|
+
quint: after step 8 `sell_refused` {}, the driver differs from the trace
|
|
440
|
+
sold: trace 3, driver 4
|
|
441
|
+
outcome: trace "sold_out", driver "ok"
|
|
442
|
+
steps:
|
|
443
|
+
init {}
|
|
444
|
+
sell {}
|
|
445
|
+
refund {}
|
|
446
|
+
sell {}
|
|
447
|
+
sell {}
|
|
448
|
+
refund {}
|
|
449
|
+
sell {}
|
|
450
|
+
sell {}
|
|
451
|
+
sell_refused {}
|
|
452
|
+
trace: trace_2.itf.json of Quint.simulate("tickets.qnt", traces: 20, max_steps: 10, seed: "1", step: "step", backend: "rust")
|
|
453
|
+
# ./box_office_spec.rb:22:in 'block (3 levels) in <top (required)>'
|
|
454
|
+
# ./box_office_spec.rb:21:in 'block (2 levels) in <top (required)>'
|
|
455
|
+
|
|
456
|
+
Finished in 0.5298 seconds (files took 0.08509 seconds to load)
|
|
457
|
+
1 example, 1 failure
|
|
458
|
+
|
|
459
|
+
Failed examples:
|
|
460
|
+
|
|
461
|
+
rspec ./box_office_spec.rb:20 # BoxOffice sells tickets the way tickets.qnt says
|
|
462
|
+
```
|
|
463
|
+
|
|
464
|
+
The first line names the step where the class and the spec disagree: at step 8, with 3 tickets sold, the spec refuses the sale, and the class sells a 4th. `steps:` lists every step from `init`, so you can run the same commands by hand, and `trace:` names the call that writes the same trace again. Fix the class and run `rspec` again.
|
|
465
|
+
|
|
466
|
+
Next, [write a driver](#write-a-driver) for your own class, or let [Hegel](#run-it-1) choose the steps and shrink a failure to the shortest sequence.
|
|
467
|
+
|
|
468
|
+
## How it works
|
|
469
|
+
|
|
470
|
+
There are two ways to choose the steps, and both use the same driver:
|
|
471
|
+
|
|
472
|
+
- **Trace replay.** `quint run --mbt` writes traces, and each trace runs on the system step by step. Quint picks the steps, so a trace reaches deep states quickly. The quick start uses this way.
|
|
473
|
+
- **Hegel.** [Hegel](https://github.com/meganemura/hegel-ruby) picks each step from the actions the spec allows now. When the system disagrees with the spec, Hegel shrinks the failure to a short sequence of commands.
|
|
474
|
+
|
|
475
|
+
The spec is read from Quint's JSON IR and evaluated inside the Ruby process, so a Hegel step does not start another process.
|
|
476
|
+
|
|
477
|
+
## Installation
|
|
478
|
+
|
|
479
|
+
```ruby
|
|
480
|
+
# Gemfile
|
|
481
|
+
gem "quint_driver"
|
|
482
|
+
gem "hegeltest", "= 0.5.1" # only for the Hegel adapter
|
|
483
|
+
```
|
|
484
|
+
|
|
485
|
+
Trace replay and the model load with `require "quint_driver"`. They need Ruby 3.3 or later and no other gem.
|
|
486
|
+
|
|
487
|
+
The Hegel adapter loads with `require "quint_driver/hegel"`. It needs hegeltest 0.5.1, which needs Ruby 3.4 or later. The gem does not depend on hegeltest, so add it to your own Gemfile.
|
|
488
|
+
|
|
489
|
+
The gem calls the `quint` command line that `QUINT` names, as the [quick start](#add-the-gem) shows. Without `QUINT`, the gem runs `npx --yes @informalsystems/quint@0.33.0`. When npx cannot fetch quint, for example offline, the gem raises `QuintDriver::QuintFailed` that says so and asks you to set `QUINT`.
|
|
490
|
+
|
|
491
|
+
`Spec.new` compiles a file once per process and quint command, and compiles it again when the text of the file, or of a file it imports, changes. Every module of the file shares that compile, so a `Spec.new` or `Quint.simulate` for another `main:` of the same file starts no node. Threads that ask for the same file at once wait for one compile, so a parallel test run needs no memo of its own.
|
|
492
|
+
|
|
493
|
+
`Quint.simulate` runs Quint's Rust evaluator directly, without node, when it finds one: the copy quint downloaded into `QUINT_HOME/rust-evaluator-v0.7.0/` (`~/.quint` when `QUINT_HOME` is unset). Set `QUINT_EVALUATOR` to the evaluator to use another copy, or to an empty string to run `quint run` instead. The same seed gives the same traces on both ways. The gem reads only the input and output format of evaluator v0.7.0, the one quint 0.33.0 runs; an evaluator that answers in another format raises `QuintDriver::QuintFailed`. Without an evaluator, or with a `step:` that names no action of the module, `Quint.simulate` runs `quint run`.
|
|
494
|
+
|
|
495
|
+
### The agent skill
|
|
496
|
+
|
|
497
|
+
The [quint_driver skill](skills/quint_driver/SKILL.md) takes a coding agent through the spec shape, the driver, trace replay, the model check, and Hegel. Install it with the [`skills`](https://github.com/vercel-labs/skills) CLI (Node.js 22.20 or later). This repository is private, so these commands need GitHub access.
|
|
498
|
+
|
|
499
|
+
```sh
|
|
500
|
+
npx skills add meganemura/quint_driver
|
|
501
|
+
```
|
|
502
|
+
|
|
503
|
+
```sh
|
|
504
|
+
npx skills add meganemura/quint_driver --skill quint_driver
|
|
505
|
+
```
|
|
506
|
+
|
|
507
|
+
```sh
|
|
508
|
+
npx skills add meganemura/quint_driver -g
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
```sh
|
|
512
|
+
npx skills add meganemura/quint_driver -a claude-code
|
|
513
|
+
```
|
|
514
|
+
|
|
515
|
+
`--skill` selects this skill, `-g` installs it for every project, and `-a` targets one agent. Add `-y` to skip prompts.
|
|
516
|
+
|
|
517
|
+
The gem ships the skill and the `examples/loans` spec, so after `bundle install` you can copy or symlink the installed copy instead, with no GitHub access:
|
|
518
|
+
|
|
519
|
+
```sh
|
|
520
|
+
mkdir -p .agents/skills
|
|
521
|
+
ln -s "$(bundle info --path quint_driver)/skills/quint_driver" .agents/skills/quint_driver
|
|
522
|
+
```
|
|
523
|
+
|
|
524
|
+
`.claude/skills/` and `.cursor/skills/` work the same way.
|
|
525
|
+
|
|
526
|
+
## Write a driver
|
|
527
|
+
|
|
528
|
+
A driver runs the spec's actions on your system and reads the system back in the spec's terms.
|
|
529
|
+
|
|
530
|
+
```ruby
|
|
531
|
+
class LoansDriver
|
|
532
|
+
# Runs one action of the spec. "init" comes first in every sequence.
|
|
533
|
+
# +picks+ holds the nondet values, such as { "m" => "m1", "b" => "b2" }.
|
|
534
|
+
def step(action, picks)
|
|
535
|
+
@outcome = "Done"
|
|
536
|
+
case action.delete_suffix("_refused")
|
|
537
|
+
when "init" then @loans = Loans.new
|
|
538
|
+
when "borrow" then @loans.borrow(picks.fetch("m"), picks.fetch("b"))
|
|
539
|
+
when "give_back" then @loans.give_back(picks.fetch("b"))
|
|
540
|
+
end
|
|
541
|
+
rescue Loans::Refused => refusal
|
|
542
|
+
@outcome = [ "Refused", refusal.reason.to_s ]
|
|
543
|
+
end
|
|
544
|
+
|
|
545
|
+
# Every variable of the spec, as plain Ruby values. One left out fails the step.
|
|
546
|
+
def state
|
|
547
|
+
{ "holder" => %w[b1 b2 b3].to_h { [ _1, @loans.holder(_1) || "" ] }, "outcome" => @outcome }
|
|
548
|
+
end
|
|
549
|
+
end
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
Plain values: a Quint set is a `Set`, a map a `Hash`, a tuple or a list an `Array`, a record a `Hash` with string keys, and a sum type its tag, or `[tag, value]` when it carries a value. A sum type with the cases `None` and `Some`, an Option, is `nil` for `None` and the value itself for `Some`, on a replay and under the machine alike.
|
|
553
|
+
|
|
554
|
+
A variable `state` leaves out fails the step and is named, such as `count: trace 0, but the driver's state leaves count out`, so a variable you forgot cannot pass. A spec can model the world outside your code, such as what a remote service answers, in variables the driver only reads to set up its stubs. Name those in `ignore:`, on `Trace.replay` or on `QuintDriver.machine`, and they are not compared.
|
|
555
|
+
|
|
556
|
+
[examples/loans](examples/loans) has the whole example: the spec, the system, and the driver.
|
|
557
|
+
|
|
558
|
+
## Run it
|
|
559
|
+
|
|
560
|
+
```ruby
|
|
561
|
+
require "quint_driver"
|
|
562
|
+
|
|
563
|
+
spec = QuintDriver::Spec.new("loans.qnt", main: "loans")
|
|
564
|
+
|
|
565
|
+
# Trace replay: Quint picks the steps.
|
|
566
|
+
QuintDriver::Quint.simulate("loans.qnt", traces: 50, max_steps: 20, seed: "1") do |path|
|
|
567
|
+
QuintDriver::Trace.replay(path, LoansDriver.new)
|
|
568
|
+
end
|
|
569
|
+
|
|
570
|
+
# Hegel picks the steps and shrinks a failure.
|
|
571
|
+
require "quint_driver/hegel"
|
|
572
|
+
machine = QuintDriver.machine(spec, driver: -> { LoansDriver.new }, invariants: %i[one_book_each])
|
|
573
|
+
Hegel.test(test_cases: 100) { |tc| Hegel::Stateful.run(machine.new, tc) }
|
|
574
|
+
```
|
|
575
|
+
|
|
576
|
+
Both raise `QuintDriver::Mismatch` with each variable that differs and the steps the driver ran from `init`, so you can run the failure again by hand. Each value prints with a set's elements and a hash's keys sorted, so the spec's side and the driver's side print a record in one key order. The message ends with a `trace:` line. After `Quint.simulate` returns, the file it replayed is gone, so the line names the trace by its file name and the call that writes the same trace again, such as `trace: trace_0.itf.json of Quint.simulate("loans.qnt", traces: 50, max_steps: 20, seed: "1", step: "step", backend: "rust")`. The error reads the same inside the block, and holds those facts as `trace`, `seed`, and `written_by`. A `Mismatch` your own code raises inside the block, such as from a check of your own after the replay, gets the same `trace:` line when it leaves the block. Raise it with a message alone, `raise QuintDriver::Mismatch, "..."`: `Mismatch.new` takes no trace, seed, or call, because the gem names the trace that ran. A replay of a trace file of your own names its path. The backtrace of a `Mismatch` leaves out quint_driver's own frames, and Hegel's under the machine, so RSpec and Minitest print first the line of yours that called the replay or `Hegel.test`, or that raised it, also when your Gemfile loads the gem from a path. A pick the move did not make is absent from `picks`, and a pick it made of the spec's own `None` is present as `nil`, so read a pick an action may go without as `picks["name"]`, which is `nil` then, and a pick it needs as `picks.fetch("name")`. `Trace.replay` returns the steps it ran, each with its `action`, `picks`, and `expected` state, so you can count what the traces reached. To watch a replay, pass `log: $stdout` to `Trace.replay`; it prints each step, such as `step 1 borrow {"b" => "b1", "m" => "m2"}`, before the driver runs it. The code runs the same in a Minitest test and in an RSpec example. In RSpec, include `Hegel::Syntax::Methods` if your own examples draw values; the machine itself does not need it. In a Rails app, transactional fixtures roll back once per example, so every trace and every Hegel test case of one example shares the test database: delete the rows in the driver's `init`, or wrap each trace and each test case in `ActiveRecord::Base.transaction(requires_new: true, joinable: false) { ...; raise ActiveRecord::Rollback }`. The skill shows both and why the savepoint must not be joinable.
|
|
577
|
+
|
|
578
|
+
A trace holds the spec's variables, not its definitions, so a value the spec derives, such as `def approvedSum` over the approved loans, is compared only when you observe it. Pass `observe: %w[approvedSum]` with `spec:` to `Trace.replay`, or `observe:` to `QuintDriver.machine`, and report the value in the driver's `state` under the same name, such as `"approvedSum" => borrower.approved_total`. The driver is held to the value the definition has in each state, and a driver that reports no value for it fails. The model evaluates the definition, also on a replay, where it reads the trace's state, so the failure names the model: `approvedSum: model 0, driver 15000`. A name the driver reports that is in no state fails too, such as `dueCents: driver 0, but dueCents is not in the trace's state, which has amount, approvedBy, outcome`; when the name is a definition of the spec, the line says to name it in `observe:`. A name that is no `val` or `def` without parameters raises `ArgumentError` with the ones the module has.
|
|
579
|
+
|
|
580
|
+
An error a step raises comes out as `QuintDriver::StepRaised`, not as a `Mismatch`, because the spec and the system were never compared. The error can come from the driver's own code, such as a `KeyError` from `picks.fetch("member")` when the pick is named `m`, or from the system beneath it. Its message names the step and the error, such as ``quint: at step 1 `borrow` {"b" => "b2", "m" => "m2"}, the step raised KeyError: key not found: "member"``. A `raised in:` line then names the frame of your project nearest the raise, such as `raised in: loans_driver.rb:6:in 'Hash#fetch'`, with the path relative to the directory the run started in. Frames in a gem, in Ruby's library, and in quint_driver are skipped, so an error that a gem raises names your line that called the gem. When every frame is skipped, the line says so: `raised in: no frame of the project; every frame is in a gem, in Ruby's library, or in quint_driver`. Then read the backtrace. Under Hegel the step number is the one Hegel's report gives the move, such as `Step 2: take`, and `init` is step 0, as on a replay. Then the message lists the steps from `init`, and on a replay ends with the same `trace:` line. Its backtrace is the error's without quint_driver's own frames, and `cause` is the error.
|
|
581
|
+
|
|
582
|
+
`Quint.simulate` runs the module named after the file. Pass `main:` to run another module of that file, the same name `Spec.new` takes. When quint fails or is not installed, `Spec.new` and `Quint.simulate` raise `QuintDriver::QuintFailed` with quint's reason. Its backtrace starts at the line of yours that called the gem, with no frame of quint_driver before it.
|
|
583
|
+
|
|
584
|
+
To test the steps that lead to one state, pass `reach:` the name of a definition that holds there, such as `reach: "two_on_loan"` for `val two_on_loan = BOOKS.filter(b => onLoan(holder, b)).size() == 2`. Each trace then ends at the first state where the definition holds, and `Quint.simulate` yields only the traces that get there. Quint checks `not(two_on_loan)` as the invariant and stops a trace where it breaks. When no trace gets there, it raises `QuintDriver::QuintFailed`, such as `quint: no trace of 20 reached two_on_loan within 1 steps; raise max_steps or traces`. `reach:` takes a predicate, a boolean `val` or `def` without parameters, and `step:` an action without parameters. A name of another kind raises `ArgumentError` before quint runs, such as `quint: reach names borrow, an action with parameters of loans; reach takes a predicate, a boolean val or def without parameters, such as nothing_on_loan, one_book_each, two_on_loan`. A name the module neither declares nor imports raises `ArgumentError` the same way once quint has run, such as `quint: step names borow, which loans neither declares nor imports; step takes an action without parameters, such as step`. Every trace, every model, and every machine starts at the action named `init`, as quint does by default. A module without one raises `ArgumentError`, such as ``quint: begins neither declares nor imports an action named init; quint_driver starts every trace and every model at init, so declare `action init = ...` with what starts begins``. quint_driver takes no other name for it: a trace names its first step after that action, and the driver and the model read that step as `init`. Add `action init = begin` to the module instead. The backtrace of each `ArgumentError` quint_driver raises starts at the line of yours that called it, with no frame of quint_driver before it.
|
|
585
|
+
|
|
586
|
+
`Quint.test(path, "lendTest", seed: "1") { |trace| ... }` writes the trace of a named run, such as `run lendTest = init.then(borrow("m1", "b1")).then(give_back("b1"))`, and yields its file; pass `main:` when the run is in another module of the file. quint test writes no `mbt::actionTaken`, so the spec has to record the action each step took, as a [Choreo](https://github.com/informalsystems/choreo) spec does: a variable that holds a sum type whose tag names the action and whose record holds its picks, such as `taken: Borrow({ m: m, b: b })`. Pass the path to that variable as `action:` to `Trace.replay`. When the state the driver answers for sits inside a variable, such as the `system` field of a Choreo spec's one variable, pass its path as `state:`; the driver's `state` then returns that record or map. `test/recorded.qnt` is a small spec of this shape, and `RecordedDriver` in `test/trace_test.rb` is its driver:
|
|
587
|
+
|
|
588
|
+
```ruby
|
|
589
|
+
QuintDriver::Quint.test("test/recorded.qnt", "lendTest", seed: "1") do |trace|
|
|
590
|
+
QuintDriver::Trace.replay(trace, RecordedDriver.new, state: %w[s books], action: %w[s taken])
|
|
591
|
+
end
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
The driver gets `step("Init", {})`, then `step("Borrow", { "m" => "m1", "b" => "b1" })`, and so on. A failure names the trace as `trace: test_0.itf.json of Quint.test("test/recorded.qnt", "lendTest", seed: "1")`. A run that fails in quint itself raises `QuintDriver::QuintFailed` with quint's report, and a name that is no run of the module lists the runs it has, such as `quint: no run is named lend in recorded; it has lendTest, unlentTest`. A trace without the action, such as a quint test trace replayed without `action:`, raises `QuintDriver::Unsupported` that asks for the path.
|
|
595
|
+
|
|
596
|
+
`QuintDriver.machine` takes these options:
|
|
597
|
+
|
|
598
|
+
- `step:` is the action that lists the moves, `"step"` by default. Each move must call a named action. Arguments come from `nondet x = S.oneOf()`. A step that is one `all` around its picks and an `any`, such as `all { nondet x = S.oneOf() any { add(x), sub(x) } }`, moves by the actions of the `any`. A `step:` that is no action of the module raises `ArgumentError` with the actions it has, such as `quint: no action is named steps; the module has counting, init, step`.
|
|
599
|
+
- `invariants:` names `val`s of the spec that must hold after every move.
|
|
600
|
+
- `choose:` is how Hegel draws a move. `:rule` (the default) picks an action and rejects it when the spec disables it. It shrinks to the shortest sequence. `:rotate` passes a disabled action's turn to the next enabled one. It reaches deep states with fewer test cases, but a shrunk sequence can keep extra moves. Hegel's failure report names each draw after the nondet pick it chose, or `rule` and `move` under `:rotate`. A label drawn more than once in a test case is numbered in draw order (`m_1 = "m2"`, `m_2 = "m1"`); a label drawn once prints bare (`m = "m2"`).
|
|
601
|
+
|
|
602
|
+
Each test case starts a new system through the driver's `init` when the machine starts, before Hegel's first step, so a driver that fails at `init` fails with no step in Hegel's report. A sequence of moves that the system already agreed on is not run again, because shrinking asks for the same sequence many times. This assumes that the system answers the same sequence the same way each time. A disagreement always runs on the system again.
|
|
603
|
+
|
|
604
|
+
A Hegel run has no Quint seed. Hegel's own report ends with a blob, such as `reproduce_failure: "AXicXYmhDQAADMJWu/3/70BgQJCmvdEWPwRiKlCBBwzbAGE="`; pass it to `Hegel.test` as `reproduce_failure:` to run that one failing case again, with the same message.
|
|
605
|
+
|
|
606
|
+
## Check the Ruby model
|
|
607
|
+
|
|
608
|
+
The Hegel way trusts the Ruby evaluation of the spec. The model answers the driver protocol, so you can hold it to Quint's own traces. A module that imports definitions or actions from another module or file reads them as its own, as quint does when it runs the module. It keeps every definition it imports, also one that nothing in the module reaches, and it needs no action named `step`; that costs one more compile for that module:
|
|
609
|
+
|
|
610
|
+
```ruby
|
|
611
|
+
QuintDriver::Quint.simulate("loans.qnt", traces: 200, max_steps: 30, seed: "7") do |path|
|
|
612
|
+
QuintDriver::Trace.replay(path, spec.model)
|
|
613
|
+
end
|
|
614
|
+
```
|
|
615
|
+
|
|
616
|
+
The model evaluates the operators a command-per-action spec uses: sets (with `fold` and `i.to(j)`), maps, records, tuples, lists, sum types with `match`, integers, and booleans, with `Bool` as the set of `false` and `true`. `Int` and `Nat` are infinite, so the model cannot list the picks of a move drawn from them, and raises `QuintDriver::Unsupported`; draw from a finite set such as `0.to(9)` instead. On a map it evaluates `Map`, `get`, `keys`, `put`, `mapBy`, `set`, `setBy`, and `setToMap`. A list is a Ruby Array, and the model evaluates `List`, `append`, `concat`, `head`, `tail`, `length`, `nth`, `indices`, `replaceAt`, `slice`, `range`, `select`, `foldl`, and `foldr`. Quint's builtin docs leave `foldr` out, but both of its evaluators fold from the right and pass the element first, `(x, acc) => ...`, and the model does the same. On integers it evaluates `+`, `-`, `*`, unary `-`, and the comparisons, and `/` and `%` only on non-negative operands, because Quint documents no rounding for a negative one. Both of Quint's evaluators truncate toward zero (`-7 % 2` is `-1`), and Ruby floors (`-7 % 2` is `1`), so the model refuses a negative operand rather than answer differently. Where Quint's docs leave an operation undefined, such as `head` of an empty list, `nth` past the end, or `setToMap` of two pairs with one key, the model raises `QuintDriver::Unsupported` instead of picking an answer. `get`, `set`, and `setBy` at a key the map does not have raise it too, because Quint's evaluator stops there; `slice(l, l.length(), l.length())` is `[]`, as both of Quint's evaluators give it. Any other operator, such as `powerset`, raises `QuintDriver::Unsupported`, so a spec that needs more fails loudly. So does a step arm that names no action, and its message names the action that holds the arm, such as `quint: counting assigns n' in an arm that names no action; name every action in step` when `step` opens `counting` in place. The message names the action or definition that holds the operator, such as `quint: no evaluation for operator powerset in subsets`. For an operator in a nondet domain, that is the action that draws it, which can be an action that `step` opens in place.
|
|
617
|
+
|
|
618
|
+
`Spec#model` is the one way to build a model; `Model.new` is private, because the model trusts the shape of quint's compiled IR. A trace gives each pick as a plain value, and the model takes the value of the pick's domain that reads the same, so a pick of a sum type works in `match`.
|
|
619
|
+
|
|
620
|
+
`model.moves(rule)` lists every move a rule of `spec.rules("step")` allows in the model's current state, as `[action, picks, next state]` in a fixed order. `model.enter(next_state)` takes one. The Hegel machine draws its moves this way. `model.value(node, env)` evaluates one IR node, or the definition a string names, in the current state. A refusal names the definition that writes the node, such as the action a domain or an argument of a rule is written in; for a node no definition of the spec holds, the message says the node is `outside any definition`. `rule.fixed` is the set of domain names that are the same set in every state and after every earlier pick: Quint's effect of the domain reads no variable, and the domain names no earlier pick. The Hegel machine under `:rule` evaluates each of them once per machine.
|
|
621
|
+
|
|
622
|
+
## Development
|
|
623
|
+
|
|
624
|
+
```sh
|
|
625
|
+
bundle install
|
|
626
|
+
bundle exec bin/test # Minitest
|
|
627
|
+
bundle exec rspec # RSpec
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
The Hegel tests are under `test/hegel/` and `spec/hegel/`. On Ruby 3.3, leave out hegeltest and run the rest:
|
|
631
|
+
|
|
632
|
+
```sh
|
|
633
|
+
BUNDLE_WITHOUT=hegel bundle install
|
|
634
|
+
BUNDLE_WITHOUT=hegel bundle exec bin/test test/*_test.rb
|
|
635
|
+
BUNDLE_WITHOUT=hegel bundle exec rspec --exclude-pattern "spec/hegel/**/*_spec.rb"
|
|
636
|
+
```
|
|
637
|
+
|
|
638
|
+
The tests run `quint`. Set `QUINT` to skip `npx`.
|
|
639
|
+
|
|
640
|
+
`test/readme_test.rb` runs "Try it without an app" in an empty folder. It runs the Rails quick start and "What you will see" only when `QUINT_DRIVER_README_APP` names a Rails app to copy, with rspec-rails, `Borrower`, and `Loan` as in the sample app, and no quint_driver in its Gemfile. The test adds the gem built from this checkout, writes the files, runs the command, applies the diff, and compares each output with the README.
|
|
641
|
+
|
|
642
|
+
`test/skill_test.rb` runs the skill's example "A choice the code leaves to the database" only when `QUINT_DRIVER_SKILL_APP` names a Rails app to copy, with rspec-rails, an `Item` that belongs to a `Batch` and may not be assigned while blocked, the `/session` and `/items/:id/release` routes, and no quint_driver in its Gemfile. The test adds this checkout, writes the skill's files, replays the traces, then removes the wrap and compares the failure with the skill. It uses the app's database, so `DATABASE_URL` runs it on PostgreSQL.
|
|
643
|
+
|
|
644
|
+
## License
|
|
645
|
+
|
|
646
|
+
MIT
|
|
647
|
+
|
|
648
|
+
[Japanese](README.ja.md)
|