hegeltest 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +22 -0
- data/CODE_OF_CONDUCT.md +10 -0
- data/LICENSE.txt +21 -0
- data/README.md +264 -0
- data/Rakefile +19 -0
- data/docs/README.md +25 -0
- data/docs/adr/0001-bind-libhegel-through-fiddle.md +54 -0
- data/docs/adr/0002-ship-one-prebuilt-engine-per-platform-specific-gem.md +48 -0
- data/docs/adr/0003-publish-as-hegeltest-require-as-hegel.md +39 -0
- data/docs/adr/0004-expose-generators-through-a-mixin-with-keyword-options.md +42 -0
- data/docs/adr/0005-name-drawn-values-from-the-callers-source-with-prism.md +40 -0
- data/docs/adr/0006-verify-the-binding-in-seven-layers-with-full-coverage.md +51 -0
- data/docs/adr/0007-ship-a-thin-ruby-skill-shaped-for-donation.md +56 -0
- data/docs/adr/0008-revisit-the-binding-after-milestone-c-on-measurement.md +81 -0
- data/docs/adr/0009-turn-the-example-database-on-with-a-key.md +89 -0
- data/docs/adr/0010-declare-stateful-rules-with-a-class-macro.md +113 -0
- data/docs/adr/0011-let-the-test-case-own-every-pool-drawn-from-it.md +83 -0
- data/docs/adr/0012-build-a-failure-origin-from-the-callers-own-frame.md +72 -0
- data/docs/adr/0013-bind-libhegel-through-the-ffi-gem.md +102 -0
- data/docs/architecture.md +182 -0
- data/lib/hegel/draw_name.rb +109 -0
- data/lib/hegel/errors.rb +47 -0
- data/lib/hegel/generator.rb +98 -0
- data/lib/hegel/generators.rb +865 -0
- data/lib/hegel/lib_hegel/real.rb +1149 -0
- data/lib/hegel/lib_hegel.rb +269 -0
- data/lib/hegel/libhegel_version.rb +9 -0
- data/lib/hegel/locate.rb +188 -0
- data/lib/hegel/report.rb +87 -0
- data/lib/hegel/runner.rb +464 -0
- data/lib/hegel/settings.rb +164 -0
- data/lib/hegel/state_machine.rb +89 -0
- data/lib/hegel/stateful/pool.rb +111 -0
- data/lib/hegel/stateful.rb +120 -0
- data/lib/hegel/syntax/methods.rb +173 -0
- data/lib/hegel/test_case.rb +523 -0
- data/lib/hegel/version.rb +5 -0
- data/lib/hegel.rb +92 -0
- data/lib/hegeltest.rb +7 -0
- data/lib/tasks/libhegel.rake +112 -0
- data/lib/tasks/platform_gems.rake +111 -0
- data/sig/hegel.rbs +563 -0
- data/skills/hegel-ruby/SKILL.md +30 -0
- data/skills/hegel-ruby/references/ruby/reference.md +1210 -0
- metadata +113 -0
|
@@ -0,0 +1,1210 @@
|
|
|
1
|
+
# Hegel Ruby Reference
|
|
2
|
+
|
|
3
|
+
## Table of Contents
|
|
4
|
+
|
|
5
|
+
- [Setup](#setup)
|
|
6
|
+
- [Test Structure](#test-structure): `Hegel.test`, its block, return value, failure behavior
|
|
7
|
+
- [Settings](#settings): `test_cases:` `seed:` `derandomize:` `verbosity:` `database:` `database_key:` `phases:` `suppress_health_check:` `report_multiple_failures:` `stateful_step_count:` `output:` `reproduce_failure:`
|
|
8
|
+
- [TestCase Methods](#testcase-methods): `draw`, `draw_integer`, `draw_boolean`, `assume`, `reject`, `note`, `target`
|
|
9
|
+
- [Generator Reference](#generator-reference): `booleans`, `integers`, `floats`, `text`, `arrays`, `just`, `sampled_from`, `one_of`, `optional`, `tuples`, `sets`, `hashes`, `characters`, `binary`, `from_regex`, `emails`, `urls`, `domains`, `ip_addresses`, `uuids`, `dates`, `times`, `datetimes`, `composite`, `deferred`
|
|
10
|
+
- [Stateful Testing](#stateful-testing): `Hegel::StateMachine`, `rule`, `invariant`, `Hegel::Stateful.run`, `Hegel::Stateful::Pool`
|
|
11
|
+
- [Combinator Methods](#combinator-methods): `.map`, `.filter`
|
|
12
|
+
- [Gotchas](#gotchas)
|
|
13
|
+
|
|
14
|
+
## Setup
|
|
15
|
+
|
|
16
|
+
Hegel for Ruby is packaged as the `hegeltest` gem, which has not reached
|
|
17
|
+
RubyGems.org yet. Installing it from git works for the Ruby code. A git
|
|
18
|
+
checkout carries no `libhegel` engine, so a run also needs
|
|
19
|
+
`HEGEL_LIBHEGEL_PATH` pointing at a local build:
|
|
20
|
+
|
|
21
|
+
```ruby
|
|
22
|
+
# Gemfile
|
|
23
|
+
gem "hegeltest", git: "https://github.com/meganemura/hegel-ruby"
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Require it, then include the generator methods wherever tests draw values:
|
|
27
|
+
|
|
28
|
+
```ruby
|
|
29
|
+
require "hegel"
|
|
30
|
+
|
|
31
|
+
include Hegel::Syntax::Methods
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`Hegel::Syntax::Methods` makes every generator method (`booleans` through
|
|
35
|
+
`deferred`; see [Generator Reference](#generator-reference)) callable bare,
|
|
36
|
+
the way FactoryBot's own `Syntax::Methods` makes `create` callable bare.
|
|
37
|
+
With Minitest, include it once on the base test class:
|
|
38
|
+
|
|
39
|
+
```ruby
|
|
40
|
+
require "hegel"
|
|
41
|
+
|
|
42
|
+
class Minitest::Test
|
|
43
|
+
include Hegel::Syntax::Methods
|
|
44
|
+
end
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Hegel drives `libhegel`, a native engine that ships separately from the gem.
|
|
48
|
+
Building or testing this repository from source needs it on hand:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
bundle exec rake libhegel:fetch
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
That downloads the pinned build for the host platform into
|
|
55
|
+
`tmp/libhegel/<version>/` and verifies it against its published SHA-256.
|
|
56
|
+
`HEGEL_LIBHEGEL_PATH` overrides the bundled engine with a local build.
|
|
57
|
+
|
|
58
|
+
## Test Structure
|
|
59
|
+
|
|
60
|
+
`Hegel.test` takes a block, runs it as a property, and returns `nil` on a
|
|
61
|
+
passing run:
|
|
62
|
+
|
|
63
|
+
```ruby
|
|
64
|
+
result = Hegel.test(test_cases: 10, verbosity: :quiet) do |tc|
|
|
65
|
+
n = tc.draw(integers)
|
|
66
|
+
raise "not an integer" unless n.is_a?(Integer)
|
|
67
|
+
end
|
|
68
|
+
result # => nil
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
The block receives a `Hegel::TestCase` (`tc` above). Draw values from it,
|
|
72
|
+
run the code under test, and signal a failure by raising. Any uncaught
|
|
73
|
+
exception inside the block is treated as a failing test case. Use whatever
|
|
74
|
+
assertion mechanism the surrounding test framework already provides
|
|
75
|
+
(RSpec's `expect`, Minitest's `assert_equal`, or a plain `raise`).
|
|
76
|
+
|
|
77
|
+
On a failing run, `Hegel.test` writes a failure report to `output:`
|
|
78
|
+
(`$stderr` by default) and re-raises the smallest failing case's own
|
|
79
|
+
exception, class and backtrace intact, so a host test framework reports it
|
|
80
|
+
as its own failure rather than a Hegel-specific one. `seed:` and
|
|
81
|
+
`derandomize: true` (see [Settings](#settings)) are added below only to
|
|
82
|
+
make this example's output reproducible on every run:
|
|
83
|
+
|
|
84
|
+
```ruby
|
|
85
|
+
require "stringio"
|
|
86
|
+
|
|
87
|
+
def my_sort(ls) = ls.sort.uniq # oops: uniq drops duplicates
|
|
88
|
+
|
|
89
|
+
output = StringIO.new
|
|
90
|
+
begin
|
|
91
|
+
Hegel.test(output: output, seed: 1, derandomize: true) do |tc|
|
|
92
|
+
xs = tc.draw(arrays(integers))
|
|
93
|
+
raise "not sorted-equal" unless my_sort(xs) == xs.sort
|
|
94
|
+
end
|
|
95
|
+
rescue => e
|
|
96
|
+
e.class # => RuntimeError
|
|
97
|
+
e.message # => "not sorted-equal"
|
|
98
|
+
end
|
|
99
|
+
output.string
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
`output.string` holds:
|
|
103
|
+
|
|
104
|
+
```
|
|
105
|
+
Falsified after 3 test cases (0 discarded):
|
|
106
|
+
|
|
107
|
+
xs = [0, 0]
|
|
108
|
+
|
|
109
|
+
To reproduce this failure, pass the blob below to Hegel.test:
|
|
110
|
+
reproduce_failure: "AXicY2VgYGBkZOBiZEBhMAAAAd8AIQ=="
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Hegel names each drawn value (`xs` above) by reading the line the `draw`
|
|
114
|
+
call was written on. Pass the blob back through `reproduce_failure:` to
|
|
115
|
+
replay that exact failing case without a full run:
|
|
116
|
+
|
|
117
|
+
```ruby
|
|
118
|
+
Hegel.test(reproduce_failure: "AXicY2VgYGBkZOBiZEBhMAAAAd8AIQ==", verbosity: :quiet) do |tc|
|
|
119
|
+
xs = tc.draw(arrays(integers))
|
|
120
|
+
raise "not sorted-equal: #{xs.inspect}" unless my_sort(xs) == xs.sort
|
|
121
|
+
end
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
## Settings
|
|
125
|
+
|
|
126
|
+
`Hegel.test` takes these keywords, all optional:
|
|
127
|
+
|
|
128
|
+
| Keyword | Type | Default | Purpose |
|
|
129
|
+
|---|---|---|---|
|
|
130
|
+
| `test_cases` | `Integer` or `nil` | `nil` (libhegel's own default) | Number of test cases to run |
|
|
131
|
+
| `seed` | `Integer` or `nil` | `nil` (libhegel picks its own) | Fixed RNG seed; pair with `derandomize: true` for a reproducible sequence |
|
|
132
|
+
| `derandomize` | `true`/`false` or `nil` | `nil` (libhegel's own default) | Derive the seed deterministically from the test itself, instead of drawing a random one |
|
|
133
|
+
| `verbosity` | `Symbol` or `nil` | `nil` (libhegel's own default) | `:quiet`, `:normal`, `:verbose`, or `:debug` |
|
|
134
|
+
| `database` | `String` or `nil` | `nil` | The example database's directory; means something only alongside `database_key:` |
|
|
135
|
+
| `database_key` | `String` or `nil` | `nil` | Turns the example database on, scoped to this key |
|
|
136
|
+
| `phases` | `Array` of `Symbol` or `nil` | `nil` (libhegel's own default: every phase) | Which run phases to enable: `:explicit`, `:reuse`, `:generate`, `:target`, `:shrink` |
|
|
137
|
+
| `suppress_health_check` | `Array` of `Symbol` or `nil` | `nil` (no suppression) | Which health checks to turn off: `:filter_too_much`, `:too_slow`, `:test_cases_too_large`, `:large_initial_test_case` |
|
|
138
|
+
| `report_multiple_failures` | `true`/`false` | `false` | `true` summarizes every distinct failure into one `Hegel::Error` instead of re-raising a single failure's own exception |
|
|
139
|
+
| `stateful_step_count` | `Integer` or `nil` | `nil` (libhegel's own default: 50) | Steps per test case a `Hegel::Stateful.run` call applies, for a stateful test |
|
|
140
|
+
| `output` | `IO` | `$stderr` | Where a failure report is written |
|
|
141
|
+
| `reproduce_failure` | `String` or `nil` | `nil` | Replays the single case the blob encodes, instead of running a full property |
|
|
142
|
+
|
|
143
|
+
`nil` means the same thing for `test_cases`, `seed`, `derandomize`,
|
|
144
|
+
`verbosity`, `phases`, `suppress_health_check`, and `stateful_step_count`: do
|
|
145
|
+
not call the matching libhegel setter, and let the engine's own default
|
|
146
|
+
apply. `database`/`database_key` and `report_multiple_failures` each follow
|
|
147
|
+
their own rule instead, covered below.
|
|
148
|
+
|
|
149
|
+
`database_key:` is the switch that turns libhegel's example database on;
|
|
150
|
+
`database:` only chooses where it writes, and means nothing without a key.
|
|
151
|
+
Passing `database:` alone raises `Hegel::Error` at run time:
|
|
152
|
+
|
|
153
|
+
```ruby
|
|
154
|
+
Hegel.test(database: "/tmp/wherever") { |tc| tc.draw(integers) }
|
|
155
|
+
# raises Hegel::Error, "hegel: database: needs database_key: to scope what
|
|
156
|
+
# it stores and replays; pass database_key: too, or drop database: and pass
|
|
157
|
+
# neither."
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
Give `database_key:` a value unique to the property under test. Two
|
|
161
|
+
`Hegel.test` calls that share a key share one replay scope, so an unrelated
|
|
162
|
+
property can read or overwrite what this one stored. Left at their shared
|
|
163
|
+
default (`nil`, `nil`), a run stores nothing.
|
|
164
|
+
|
|
165
|
+
`phases:` and `suppress_health_check:` each take an `Array` of `Symbol`, and
|
|
166
|
+
each raises `Hegel::Error` at run time for an empty one. An empty `Array`
|
|
167
|
+
has no established meaning against libhegel, unlike dropping the keyword
|
|
168
|
+
entirely:
|
|
169
|
+
|
|
170
|
+
```ruby
|
|
171
|
+
Hegel.test(phases: []) { |tc| tc.draw(integers) }
|
|
172
|
+
# raises Hegel::Error, "hegel: phases expects one or more of [:explicit,
|
|
173
|
+
# :reuse, :generate, :target, :shrink], got an empty Array"
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
`report_multiple_failures:` defaults to `false`, which is not libhegel's own
|
|
177
|
+
default (`true`). With the default `false`, a run stops at its first failing
|
|
178
|
+
example and re-raises that failure's own exception, class and backtrace
|
|
179
|
+
intact, so a host framework reports it as its own. `report_multiple_failures:
|
|
180
|
+
true` keeps generating afterwards to surface other distinct bugs, and then
|
|
181
|
+
raises `Hegel::Error` naming the count instead of any one of the individual
|
|
182
|
+
exceptions:
|
|
183
|
+
|
|
184
|
+
```ruby
|
|
185
|
+
Hegel.test(test_cases: 10, report_multiple_failures: true, verbosity: :quiet) do |tc|
|
|
186
|
+
n = tc.draw_integer(0, 1, label: "n")
|
|
187
|
+
if n.zero?
|
|
188
|
+
raise "boom-zero"
|
|
189
|
+
else
|
|
190
|
+
raise "boom-one"
|
|
191
|
+
end
|
|
192
|
+
end
|
|
193
|
+
# raises Hegel::Error, "Property-based test failed with 2 distinct failures."
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
`stateful_step_count:` bounds how many rules one `Hegel::Stateful.run` call
|
|
197
|
+
applies per test case (see [Stateful Testing](#stateful-testing)); the
|
|
198
|
+
engine documents its own default as 50, and requires the value be at least
|
|
199
|
+
1.
|
|
200
|
+
|
|
201
|
+
`seed:` and `derandomize: true` together make a run reproducible:
|
|
202
|
+
|
|
203
|
+
```ruby
|
|
204
|
+
seen_a = []
|
|
205
|
+
Hegel.test(test_cases: 5, seed: 42, derandomize: true, verbosity: :quiet) do |tc|
|
|
206
|
+
seen_a << tc.draw(integers(min_value: 0, max_value: 100))
|
|
207
|
+
end
|
|
208
|
+
|
|
209
|
+
seen_b = []
|
|
210
|
+
Hegel.test(test_cases: 5, seed: 42, derandomize: true, verbosity: :quiet) do |tc|
|
|
211
|
+
seen_b << tc.draw(integers(min_value: 0, max_value: 100))
|
|
212
|
+
end
|
|
213
|
+
|
|
214
|
+
seen_a == seen_b # => true
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
An unrecognized `verbosity:` raises `Hegel::Error` at run time:
|
|
218
|
+
|
|
219
|
+
```ruby
|
|
220
|
+
Hegel.test(verbosity: :chatty) { |tc| tc.draw(integers) }
|
|
221
|
+
# raises Hegel::Error, "hegel: unknown verbosity :chatty; expected one of
|
|
222
|
+
# [:quiet, :normal, :verbose, :debug]"
|
|
223
|
+
```
|
|
224
|
+
|
|
225
|
+
`verbosity: :quiet` also silences the failure report itself, not just
|
|
226
|
+
libhegel's own progress output. Even when `output:` is given, nothing is
|
|
227
|
+
written to it.
|
|
228
|
+
|
|
229
|
+
## TestCase Methods
|
|
230
|
+
|
|
231
|
+
| Method | Signature | Purpose |
|
|
232
|
+
|---|---|---|
|
|
233
|
+
| `draw` | `draw(generator, label: nil)` | Draw a value from a `Hegel::Generator`; shown in the failure report under `label`, or a name recovered from the caller's own source line |
|
|
234
|
+
| `draw_integer` | `draw_integer(min_value, max_value, label: nil)` | Draw an integer directly, without building an `integers` generator |
|
|
235
|
+
| `draw_boolean` | `draw_boolean(p = 0.5, label: nil)` | Draw a boolean directly, without building a `booleans` generator |
|
|
236
|
+
| `assume` | `assume(condition)` | Discard this test case (see `reject`) unless `condition` holds, read as Ruby truthiness |
|
|
237
|
+
| `reject` | `reject` | Discard this test case unconditionally |
|
|
238
|
+
| `note` | `note(message = nil, &block)` | Record `message` (or the block's return value) for the failure report, interleaved with draws in call order |
|
|
239
|
+
| `target` | `target(value, label: "")` | Feed `value` to libhegel's own hill-climbing search between generation rounds |
|
|
240
|
+
|
|
241
|
+
```ruby
|
|
242
|
+
result = Hegel.test(test_cases: 10, verbosity: :quiet) do |tc|
|
|
243
|
+
a = tc.draw(integers, label: "numerator")
|
|
244
|
+
b = tc.draw_integer(1, 1_000)
|
|
245
|
+
c = tc.draw_boolean(0.5, label: "flag")
|
|
246
|
+
raise "bad draw" unless a.is_a?(Integer) && b.is_a?(Integer) && [true, false].include?(c)
|
|
247
|
+
end
|
|
248
|
+
result # => nil
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
`label:` wins over the name Hegel recovers from the source line, and shows
|
|
252
|
+
up in the failure report:
|
|
253
|
+
|
|
254
|
+
```ruby
|
|
255
|
+
output = StringIO.new
|
|
256
|
+
begin
|
|
257
|
+
Hegel.test(output: output, seed: 1, derandomize: true) do |tc|
|
|
258
|
+
n = tc.draw(integers(min_value: 0, max_value: 10), label: "count")
|
|
259
|
+
raise "boom" if n > 5
|
|
260
|
+
end
|
|
261
|
+
rescue => e
|
|
262
|
+
e.message # => "boom"
|
|
263
|
+
end
|
|
264
|
+
output.string
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
```
|
|
268
|
+
Falsified after 2 test cases (0 discarded):
|
|
269
|
+
|
|
270
|
+
count = 6
|
|
271
|
+
|
|
272
|
+
To reproduce this failure, pass the blob below to Hegel.test:
|
|
273
|
+
reproduce_failure: "AAEAAAAACgEAAAAG"
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
`assume` discards a test case (the same way `reject` does) unless its
|
|
277
|
+
condition holds, read as ordinary Ruby truthiness rather than restricted to
|
|
278
|
+
`true`/`false`, so `tc.assume(hash[:key])` works directly:
|
|
279
|
+
|
|
280
|
+
```ruby
|
|
281
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
282
|
+
n = tc.draw_integer(0, 10)
|
|
283
|
+
tc.assume(n.even?)
|
|
284
|
+
raise "not even" unless n.even?
|
|
285
|
+
end
|
|
286
|
+
result # => nil
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
`.filter` (see [Combinator Methods](#combinator-methods)) discards a test
|
|
290
|
+
case the same way, scoped to one generator's own draw rather than the whole
|
|
291
|
+
body; `assume`/`reject` discard from anywhere in the block.
|
|
292
|
+
|
|
293
|
+
`note` records a message for the eventual failure report, interleaved with
|
|
294
|
+
draws in the order both were called. Pass a message directly, or a block.
|
|
295
|
+
The block form is evaluated only on the one, already-shrunk replay that
|
|
296
|
+
produces the report, so it is the cheaper choice when building the message
|
|
297
|
+
itself costs something. Passing both, or neither, raises `Hegel::Error`:
|
|
298
|
+
|
|
299
|
+
```ruby
|
|
300
|
+
output = StringIO.new
|
|
301
|
+
begin
|
|
302
|
+
Hegel.test(output: output, seed: 1, derandomize: true) do |tc|
|
|
303
|
+
tc.note("starting the queue")
|
|
304
|
+
n = tc.draw_integer(0, 1_000_000, label: "n")
|
|
305
|
+
tc.note("queue was empty") if n > 500
|
|
306
|
+
raise "too big: #{n}" if n > 500
|
|
307
|
+
end
|
|
308
|
+
rescue => e
|
|
309
|
+
e.message # => "too big: 501"
|
|
310
|
+
end
|
|
311
|
+
output.string
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
```
|
|
315
|
+
Falsified after 2 test cases (0 discarded):
|
|
316
|
+
|
|
317
|
+
starting the queue
|
|
318
|
+
n = 501
|
|
319
|
+
queue was empty
|
|
320
|
+
|
|
321
|
+
To reproduce this failure, pass the blob below to Hegel.test:
|
|
322
|
+
reproduce_failure: "AAEAAAAACgIAAAD1AQ=="
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
`target` feeds `value` to libhegel's own hill-climbing search between
|
|
326
|
+
generation rounds, under `label` (default `""`); it does not appear in the
|
|
327
|
+
failure report. The same `label` used twice on one test case raises
|
|
328
|
+
`Hegel::Error`, and so does a non-finite `value` (`Float::NAN` or
|
|
329
|
+
`Float::INFINITY`):
|
|
330
|
+
|
|
331
|
+
```ruby
|
|
332
|
+
Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
333
|
+
n = tc.draw_integer(0, 1000)
|
|
334
|
+
m = tc.draw_integer(0, 1000)
|
|
335
|
+
tc.target(n + m, label: "score")
|
|
336
|
+
end
|
|
337
|
+
```
|
|
338
|
+
|
|
339
|
+
```ruby
|
|
340
|
+
Hegel.test(test_cases: 5, verbosity: :quiet) { |tc| tc.target(1, label: "score"); tc.target(2, label: "score") }
|
|
341
|
+
# raises Hegel::Error, "HEGEL_E_INVALID_ARG (-5): tc.target(2, label=\"score\")
|
|
342
|
+
# would overwrite previous tc.target(_, label=\"score\"); each label can be
|
|
343
|
+
# observed at most once per test case"
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
## Generator Reference
|
|
347
|
+
|
|
348
|
+
Every generator method lives in `Hegel::Syntax::Methods` (bare, once
|
|
349
|
+
included) and as `Hegel::Generators.<name>` (with a module prefix, no
|
|
350
|
+
include needed). Both forms build the same generator. Options are keyword
|
|
351
|
+
arguments; a generator validates them when it is drawn from, not when it is
|
|
352
|
+
built (see [Gotchas](#gotchas)).
|
|
353
|
+
|
|
354
|
+
### `booleans(p: 0.5)`
|
|
355
|
+
|
|
356
|
+
A boolean, true with probability `p`.
|
|
357
|
+
|
|
358
|
+
```ruby
|
|
359
|
+
Hegel.test(test_cases: 10, verbosity: :quiet) do |tc|
|
|
360
|
+
b = tc.draw(booleans)
|
|
361
|
+
raise "not boolean" unless [true, false].include?(b)
|
|
362
|
+
end
|
|
363
|
+
|
|
364
|
+
Hegel.test(test_cases: 10, verbosity: :quiet) do |tc|
|
|
365
|
+
raise "not true" unless tc.draw(booleans(p: 1.0)) == true # p: 1.0 always draws true
|
|
366
|
+
end
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
`p` outside `0.0..1.0` raises `Hegel::Error` at draw time:
|
|
370
|
+
`"booleans: p must be between 0.0 and 1.0, got 1.5"`.
|
|
371
|
+
|
|
372
|
+
### `integers(min_value: nil, max_value: nil)`
|
|
373
|
+
|
|
374
|
+
An integer in `[min_value, max_value]`. Omitting `min_value` or `max_value`
|
|
375
|
+
defaults it to the signed 64-bit boundary (`-2**63` or `2**63 - 1`); an
|
|
376
|
+
explicit bound outside that range draws at arbitrary precision instead.
|
|
377
|
+
|
|
378
|
+
```ruby
|
|
379
|
+
Hegel.test(test_cases: 30, verbosity: :quiet) do |tc|
|
|
380
|
+
n = tc.draw(integers(min_value: 0, max_value: 100))
|
|
381
|
+
raise "out of range" unless n.between?(0, 100)
|
|
382
|
+
end
|
|
383
|
+
|
|
384
|
+
seen = []
|
|
385
|
+
Hegel.test(test_cases: 5, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
386
|
+
seen << tc.draw(integers(min_value: 10**20, max_value: 10**20 + 100))
|
|
387
|
+
end
|
|
388
|
+
seen # => [100000000000000000000, 100000000000000000099, 100000000000000000039, 100000000000000000026, 100000000000000000059]
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
`max_value < min_value` raises `Hegel::Error` at draw time:
|
|
392
|
+
`"integers: max_value < min_value"`. Because an omitted bound still
|
|
393
|
+
defaults to the 64-bit boundary rather than to an unbounded range, this
|
|
394
|
+
also fires for a one-sided big bound: `integers(min_value: 10**30)` raises
|
|
395
|
+
the same error, since `max_value` defaults to `2**63 - 1`, smaller than
|
|
396
|
+
`10**30` (see [Gotchas](#gotchas)). Pass both bounds explicitly to draw
|
|
397
|
+
outside the 64-bit range:
|
|
398
|
+
|
|
399
|
+
```ruby
|
|
400
|
+
Hegel.test(verbosity: :quiet) { |tc| tc.draw(integers(min_value: 10**30, max_value: 10**30 + 100)) }
|
|
401
|
+
# => nil
|
|
402
|
+
```
|
|
403
|
+
|
|
404
|
+
### `floats(min_value: nil, max_value: nil, allow_nan: false, allow_infinity: false, exclude_min: false, exclude_max: false)`
|
|
405
|
+
|
|
406
|
+
A double in `[min_value, max_value]`, unbounded (the full finite range) by
|
|
407
|
+
default. Unlike hegel-rust's and hegel-typescript's `floats()`, `allow_nan`
|
|
408
|
+
and `allow_infinity` both default to **`false`** here even when fully
|
|
409
|
+
unbounded (see [Gotchas](#gotchas)).
|
|
410
|
+
|
|
411
|
+
```ruby
|
|
412
|
+
Hegel.test(test_cases: 30, verbosity: :quiet) do |tc|
|
|
413
|
+
f = tc.draw(floats(min_value: 0.0, max_value: 1.0, exclude_min: true, exclude_max: true))
|
|
414
|
+
raise "out of open interval" unless f > 0.0 && f < 1.0
|
|
415
|
+
end
|
|
416
|
+
|
|
417
|
+
Hegel.test(test_cases: 200, verbosity: :quiet) do |tc|
|
|
418
|
+
tc.draw(floats(allow_nan: true)) # can now draw NaN
|
|
419
|
+
end
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
`max_value < min_value` raises `Hegel::Error` at draw time:
|
|
423
|
+
`"floats: max_value < min_value"`.
|
|
424
|
+
|
|
425
|
+
### `text(min_size: 0, max_size: nil, codec: nil, min_codepoint: nil, max_codepoint: nil)`
|
|
426
|
+
|
|
427
|
+
A Unicode string of `[min_size, max_size]` characters, unbounded above by
|
|
428
|
+
default.
|
|
429
|
+
|
|
430
|
+
```ruby
|
|
431
|
+
Hegel.test(test_cases: 30, verbosity: :quiet) do |tc|
|
|
432
|
+
s = tc.draw(text(min_size: 1, max_size: 10))
|
|
433
|
+
raise "out of size" unless s.length.between?(1, 10)
|
|
434
|
+
end
|
|
435
|
+
|
|
436
|
+
Hegel.test(test_cases: 30, verbosity: :quiet) do |tc|
|
|
437
|
+
s = tc.draw(text(codec: "ascii"))
|
|
438
|
+
raise "not ascii" unless s.ascii_only?
|
|
439
|
+
end
|
|
440
|
+
|
|
441
|
+
Hegel.test(test_cases: 30, verbosity: :quiet) do |tc|
|
|
442
|
+
s = tc.draw(text(min_codepoint: 0x41, max_codepoint: 0x5A))
|
|
443
|
+
raise "out of codepoint range" if s.codepoints.any? { |cp| !cp.between?(0x41, 0x5A) }
|
|
444
|
+
end
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
`max_size < min_size` raises `Hegel::Error` at draw time: `"text: max_size <
|
|
448
|
+
min_size"`. `alphabet:`, `categories:`, `include_characters:`, and
|
|
449
|
+
`exclude_characters:` (character-filtering options other Hegel bindings
|
|
450
|
+
expose) are not available in this binding; `codec:`, `min_codepoint:`, and
|
|
451
|
+
`max_codepoint:` are the alphabet controls it exposes today.
|
|
452
|
+
|
|
453
|
+
### `arrays(elements, min_size: 0, max_size: nil)`
|
|
454
|
+
|
|
455
|
+
An `Array` of values from the `elements` generator, with `[min_size,
|
|
456
|
+
max_size]` entries, unbounded above by default.
|
|
457
|
+
|
|
458
|
+
```ruby
|
|
459
|
+
Hegel.test(test_cases: 30, verbosity: :quiet) do |tc|
|
|
460
|
+
xs = tc.draw(arrays(integers(min_value: 0, max_value: 10), min_size: 1, max_size: 5))
|
|
461
|
+
raise "out of range" unless xs.length.between?(1, 5) && xs.all? { |x| x.between?(0, 10) }
|
|
462
|
+
end
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
`max_size < min_size` raises `Hegel::Error` at draw time: `"arrays:
|
|
466
|
+
max_size < min_size"`. A negative `min_size` also raises at draw time:
|
|
467
|
+
`"arrays: min_size must not be negative"`. `unique:` (deduplicating
|
|
468
|
+
elements) is not available on `arrays`; `sets` (below) draws distinct
|
|
469
|
+
elements instead.
|
|
470
|
+
|
|
471
|
+
### `just(value)`
|
|
472
|
+
|
|
473
|
+
Always `value`, drawing nothing.
|
|
474
|
+
|
|
475
|
+
```ruby
|
|
476
|
+
result = Hegel.test(test_cases: 10, verbosity: :quiet) do |tc|
|
|
477
|
+
v = tc.draw(just(42))
|
|
478
|
+
raise "not 42" unless v == 42
|
|
479
|
+
end
|
|
480
|
+
result # => nil
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
### `sampled_from(collection)`
|
|
484
|
+
|
|
485
|
+
One element of `collection`, picked at random.
|
|
486
|
+
|
|
487
|
+
```ruby
|
|
488
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
489
|
+
v = tc.draw(sampled_from([1, 2, 3]))
|
|
490
|
+
raise "not in collection" unless [1, 2, 3].include?(v)
|
|
491
|
+
end
|
|
492
|
+
result # => nil
|
|
493
|
+
```
|
|
494
|
+
|
|
495
|
+
An empty `collection` raises `Hegel::Error` at draw time:
|
|
496
|
+
`"sampled_from: collection must not be empty"`.
|
|
497
|
+
|
|
498
|
+
### `one_of(*generators)`
|
|
499
|
+
|
|
500
|
+
A value drawn from one of `generators`, picked at random.
|
|
501
|
+
|
|
502
|
+
```ruby
|
|
503
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
504
|
+
v = tc.draw(one_of(just(1), just(2)))
|
|
505
|
+
raise "not in set" unless [1, 2].include?(v)
|
|
506
|
+
end
|
|
507
|
+
result # => nil
|
|
508
|
+
```
|
|
509
|
+
|
|
510
|
+
Calling it with no `generators` raises `Hegel::Error` at draw time:
|
|
511
|
+
`"one_of: at least one generator is required"`.
|
|
512
|
+
|
|
513
|
+
### `optional(generator)`
|
|
514
|
+
|
|
515
|
+
A value drawn from `generator` half the time, `nil` the other half.
|
|
516
|
+
|
|
517
|
+
```ruby
|
|
518
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
519
|
+
v = tc.draw(optional(just(1)))
|
|
520
|
+
raise "wrong value" unless v.nil? || v == 1
|
|
521
|
+
end
|
|
522
|
+
result # => nil
|
|
523
|
+
|
|
524
|
+
seen = []
|
|
525
|
+
Hegel.test(test_cases: 10, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
526
|
+
seen << tc.draw(optional(integers(min_value: 0, max_value: 9)))
|
|
527
|
+
end
|
|
528
|
+
seen # => [nil, 2, 3, 6, 1, 7, 5, 4, 8, 9]
|
|
529
|
+
```
|
|
530
|
+
|
|
531
|
+
### `tuples(*generators)`
|
|
532
|
+
|
|
533
|
+
An `Array` holding one value drawn from each of `generators`, in order
|
|
534
|
+
(Ruby has no tuple type).
|
|
535
|
+
|
|
536
|
+
```ruby
|
|
537
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
538
|
+
v = tc.draw(tuples(integers, text))
|
|
539
|
+
raise "wrong shape" unless v.is_a?(Array) && v.size == 2
|
|
540
|
+
end
|
|
541
|
+
result # => nil
|
|
542
|
+
|
|
543
|
+
seen = []
|
|
544
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
545
|
+
seen << tc.draw(tuples(integers(min_value: 0, max_value: 9), booleans))
|
|
546
|
+
end
|
|
547
|
+
seen # => [[0, false], [5, false], [3, false]]
|
|
548
|
+
```
|
|
549
|
+
|
|
550
|
+
Calling it with no `generators` is a valid, zero-length tuple, not an
|
|
551
|
+
error:
|
|
552
|
+
|
|
553
|
+
```ruby
|
|
554
|
+
value = nil
|
|
555
|
+
Hegel.test(test_cases: 1, verbosity: :quiet) { |tc| value = tc.draw(tuples) }
|
|
556
|
+
value # => []
|
|
557
|
+
```
|
|
558
|
+
|
|
559
|
+
### `sets(elements, min_size: 0, max_size: nil)`
|
|
560
|
+
|
|
561
|
+
A `Set` of values from `elements`, with `[min_size, max_size]` entries,
|
|
562
|
+
unbounded above by default.
|
|
563
|
+
|
|
564
|
+
```ruby
|
|
565
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
566
|
+
v = tc.draw(sets(integers(min_value: 0, max_value: 100), min_size: 1, max_size: 5))
|
|
567
|
+
raise "wrong type" unless v.is_a?(Set) && v.size.between?(1, 5)
|
|
568
|
+
end
|
|
569
|
+
result # => nil
|
|
570
|
+
|
|
571
|
+
seen = nil
|
|
572
|
+
Hegel.test(test_cases: 1, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
573
|
+
seen = tc.draw(sets(integers(min_value: 0, max_value: 9), min_size: 3, max_size: 3))
|
|
574
|
+
end
|
|
575
|
+
seen.to_a # => [5, 2, 6]
|
|
576
|
+
```
|
|
577
|
+
|
|
578
|
+
`max_size < min_size` raises `Hegel::Error` at draw time: `"sets: max_size <
|
|
579
|
+
min_size"`. A negative `min_size` also raises at draw time: `"sets:
|
|
580
|
+
min_size must not be negative"`.
|
|
581
|
+
|
|
582
|
+
### `hashes(keys, values, min_size: 0, max_size: nil)`
|
|
583
|
+
|
|
584
|
+
A `Hash` whose keys are drawn from `keys` and values from `values`, with
|
|
585
|
+
`[min_size, max_size]` entries, unbounded above by default.
|
|
586
|
+
|
|
587
|
+
```ruby
|
|
588
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
589
|
+
v = tc.draw(hashes(integers(min_value: 0, max_value: 100), text, min_size: 1, max_size: 5))
|
|
590
|
+
raise "wrong type" unless v.is_a?(Hash) && v.size.between?(1, 5)
|
|
591
|
+
end
|
|
592
|
+
result # => nil
|
|
593
|
+
|
|
594
|
+
seen = nil
|
|
595
|
+
Hegel.test(test_cases: 1, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
596
|
+
seen = tc.draw(hashes(integers(min_value: 0, max_value: 9), booleans, min_size: 2, max_size: 2))
|
|
597
|
+
end
|
|
598
|
+
seen.to_a # => [[5, false], [3, false]]
|
|
599
|
+
```
|
|
600
|
+
|
|
601
|
+
`max_size < min_size` raises `Hegel::Error` at draw time: `"hashes:
|
|
602
|
+
max_size < min_size"`. A negative `min_size` also raises at draw time:
|
|
603
|
+
`"hashes: min_size must not be negative"`.
|
|
604
|
+
|
|
605
|
+
### `characters(codec: nil, min_codepoint: nil, max_codepoint: nil)`
|
|
606
|
+
|
|
607
|
+
A `String` of exactly one character, sharing `text`'s own alphabet options.
|
|
608
|
+
|
|
609
|
+
```ruby
|
|
610
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
611
|
+
v = tc.draw(characters(codec: "ascii"))
|
|
612
|
+
raise "wrong shape" unless v.is_a?(String) && v.length == 1 && v.ascii_only?
|
|
613
|
+
end
|
|
614
|
+
result # => nil
|
|
615
|
+
|
|
616
|
+
seen = []
|
|
617
|
+
Hegel.test(test_cases: 5, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
618
|
+
seen << tc.draw(characters(min_codepoint: 0x41, max_codepoint: 0x5A))
|
|
619
|
+
end
|
|
620
|
+
seen # => ["A", "Z", "E", "T", "G"]
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
### `binary(min_size: 0, max_size: nil)`
|
|
624
|
+
|
|
625
|
+
A byte `String` of `[min_size, max_size]` bytes, unbounded above by
|
|
626
|
+
default.
|
|
627
|
+
|
|
628
|
+
```ruby
|
|
629
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
630
|
+
v = tc.draw(binary(min_size: 1, max_size: 8))
|
|
631
|
+
raise "wrong encoding" unless v.encoding == Encoding::BINARY && v.bytesize.between?(1, 8)
|
|
632
|
+
end
|
|
633
|
+
result # => nil
|
|
634
|
+
|
|
635
|
+
seen = []
|
|
636
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
637
|
+
seen << tc.draw(binary(min_size: 2, max_size: 2))
|
|
638
|
+
end
|
|
639
|
+
seen # => ["\x00\x00", "\xCA\x95", "\xD5N"]
|
|
640
|
+
```
|
|
641
|
+
|
|
642
|
+
`max_size < min_size` raises `Hegel::Error` at draw time: `"binary:
|
|
643
|
+
max_size < min_size"`.
|
|
644
|
+
|
|
645
|
+
### `from_regex(pattern, fullmatch: false)`
|
|
646
|
+
|
|
647
|
+
A `String` matching `pattern` (Python `re` syntax, not Ruby's `Regexp`
|
|
648
|
+
syntax). `fullmatch` requires the whole string to match, not just contain
|
|
649
|
+
a match.
|
|
650
|
+
|
|
651
|
+
```ruby
|
|
652
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
653
|
+
v = tc.draw(from_regex("[a-z]{3}"))
|
|
654
|
+
raise "no match" unless v.match?(/[a-z]{3}/)
|
|
655
|
+
end
|
|
656
|
+
result # => nil
|
|
657
|
+
|
|
658
|
+
seen = []
|
|
659
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
660
|
+
seen << tc.draw(from_regex("[a-z]{3}", fullmatch: true))
|
|
661
|
+
end
|
|
662
|
+
seen # => ["aaa", "pgq", "iqy"]
|
|
663
|
+
```
|
|
664
|
+
|
|
665
|
+
`pattern` must be a `String`, not a Ruby `Regexp` (the two grammars diverge
|
|
666
|
+
on flags and anchors, so a `Regexp` is rejected rather than silently
|
|
667
|
+
translated): `from_regex(/[a-z]{3}/)` raises `Hegel::Error` at draw time,
|
|
668
|
+
`"from_regex: pattern must be a String in Python re syntax, not a
|
|
669
|
+
Regexp"`. Pass `my_regexp.source` explicitly for a pattern confirmed to
|
|
670
|
+
need no flags and to use only syntax the two grammars share.
|
|
671
|
+
|
|
672
|
+
### `emails`
|
|
673
|
+
|
|
674
|
+
An RFC 5321/5322 email address `String`.
|
|
675
|
+
|
|
676
|
+
```ruby
|
|
677
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
678
|
+
v = tc.draw(emails)
|
|
679
|
+
raise "not a string" unless v.is_a?(String)
|
|
680
|
+
end
|
|
681
|
+
result # => nil
|
|
682
|
+
|
|
683
|
+
seen = []
|
|
684
|
+
Hegel.test(test_cases: 1, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
685
|
+
seen << tc.draw(emails)
|
|
686
|
+
end
|
|
687
|
+
seen # => ["0@a.COM"]
|
|
688
|
+
```
|
|
689
|
+
|
|
690
|
+
### `urls`
|
|
691
|
+
|
|
692
|
+
An RFC 3986 http/https URL `String`.
|
|
693
|
+
|
|
694
|
+
```ruby
|
|
695
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
696
|
+
v = tc.draw(urls)
|
|
697
|
+
raise "not a string" unless v.is_a?(String)
|
|
698
|
+
end
|
|
699
|
+
result # => nil
|
|
700
|
+
|
|
701
|
+
seen = []
|
|
702
|
+
Hegel.test(test_cases: 1, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
703
|
+
seen << tc.draw(urls)
|
|
704
|
+
end
|
|
705
|
+
seen # => ["http://a.COM/"]
|
|
706
|
+
```
|
|
707
|
+
|
|
708
|
+
### `domains(max_length: 255)`
|
|
709
|
+
|
|
710
|
+
A fully-qualified domain name `String` of at most `max_length` characters.
|
|
711
|
+
|
|
712
|
+
```ruby
|
|
713
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
714
|
+
v = tc.draw(domains(max_length: 20))
|
|
715
|
+
raise "too long" unless v.length <= 20
|
|
716
|
+
end
|
|
717
|
+
result # => nil
|
|
718
|
+
|
|
719
|
+
seen = []
|
|
720
|
+
Hegel.test(test_cases: 1, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
721
|
+
seen << tc.draw(domains)
|
|
722
|
+
end
|
|
723
|
+
seen # => ["a.COM"]
|
|
724
|
+
```
|
|
725
|
+
|
|
726
|
+
`max_length` outside libhegel's valid range raises `Hegel::Error` at draw
|
|
727
|
+
time, wrapping the engine's own message:
|
|
728
|
+
|
|
729
|
+
```ruby
|
|
730
|
+
Hegel.test(verbosity: :quiet) { |tc| tc.draw(domains(max_length: 3)) }
|
|
731
|
+
# raises Hegel::Error, "HEGEL_E_INVALID_ARG (-5): domain max_length=3
|
|
732
|
+
# leaves no eligible TLDs"
|
|
733
|
+
```
|
|
734
|
+
|
|
735
|
+
### `ip_addresses(v4: true, v6: true)`
|
|
736
|
+
|
|
737
|
+
An `IPAddr`, v4 or v6 depending on `v4`/`v6`. With both true (the default),
|
|
738
|
+
the family is picked at random for each value.
|
|
739
|
+
|
|
740
|
+
```ruby
|
|
741
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
742
|
+
v = tc.draw(ip_addresses)
|
|
743
|
+
raise "not an ipaddr" unless v.is_a?(IPAddr)
|
|
744
|
+
end
|
|
745
|
+
result # => nil
|
|
746
|
+
|
|
747
|
+
seen = []
|
|
748
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
749
|
+
seen << tc.draw(ip_addresses(v6: false)).to_s
|
|
750
|
+
end
|
|
751
|
+
seen # => ["0.0.0.0", "172.17.134.161", "15.160.245.253"]
|
|
752
|
+
```
|
|
753
|
+
|
|
754
|
+
`v4: false, v6: false` together raise `Hegel::Error` at draw time:
|
|
755
|
+
`"ip_addresses: v4 and v6 must not both be false"`.
|
|
756
|
+
|
|
757
|
+
### `uuids(version: nil)`
|
|
758
|
+
|
|
759
|
+
A UUID `String` in the standard 8-4-4-4-12 hex form. `version: nil` (the
|
|
760
|
+
default) draws uniform random bits except the nil UUID; an explicit
|
|
761
|
+
version forces the RFC 4122 version and variant nibbles.
|
|
762
|
+
|
|
763
|
+
```ruby
|
|
764
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
765
|
+
v = tc.draw(uuids)
|
|
766
|
+
raise "wrong shape" unless v.match?(/\A[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\z/)
|
|
767
|
+
end
|
|
768
|
+
result # => nil
|
|
769
|
+
|
|
770
|
+
seen = []
|
|
771
|
+
Hegel.test(test_cases: 1, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
772
|
+
seen << tc.draw(uuids(version: 4))
|
|
773
|
+
end
|
|
774
|
+
seen # => ["00000000-0000-4000-8000-000000000000"]
|
|
775
|
+
```
|
|
776
|
+
|
|
777
|
+
A `version` outside `0..15` raises `Hegel::Error` at draw time, wrapping
|
|
778
|
+
the engine's own message:
|
|
779
|
+
|
|
780
|
+
```ruby
|
|
781
|
+
Hegel.test(verbosity: :quiet) { |tc| tc.draw(uuids(version: 16)) }
|
|
782
|
+
# raises Hegel::Error, "HEGEL_E_INVALID_ARG (-5): uuid version must be a
|
|
783
|
+
# single hex nibble (0..=15), got 16"
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
### `dates(min_value: nil, max_value: nil)`
|
|
787
|
+
|
|
788
|
+
A proleptic Gregorian calendar `Date` in `[min_value, max_value]`,
|
|
789
|
+
defaulting to year 1 through year 9999.
|
|
790
|
+
|
|
791
|
+
```ruby
|
|
792
|
+
Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
793
|
+
v = tc.draw(dates(min_value: Date.new(2020, 1, 1), max_value: Date.new(2020, 12, 31)))
|
|
794
|
+
raise "out of range" unless v.between?(Date.new(2020, 1, 1), Date.new(2020, 12, 31))
|
|
795
|
+
end
|
|
796
|
+
|
|
797
|
+
seen = []
|
|
798
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
799
|
+
seen << tc.draw(dates(min_value: Date.new(2020, 1, 1), max_value: Date.new(2020, 12, 31))).to_s
|
|
800
|
+
end
|
|
801
|
+
seen # => ["2020-01-01", "2020-07-23", "2020-04-28"]
|
|
802
|
+
```
|
|
803
|
+
|
|
804
|
+
`max_value < min_value` raises `Hegel::Error` at draw time: `"dates:
|
|
805
|
+
max_value < min_value"`.
|
|
806
|
+
|
|
807
|
+
### `times(min_value: nil, max_value: nil)`
|
|
808
|
+
|
|
809
|
+
A time of day `String`, `"HH:MM:SS.ffffff"`, in `[min_value, max_value]`
|
|
810
|
+
(also `"HH:MM:SS.ffffff"` Strings), defaulting to the full day.
|
|
811
|
+
|
|
812
|
+
```ruby
|
|
813
|
+
Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
814
|
+
v = tc.draw(times(min_value: "09:00:00.000000", max_value: "17:00:00.000000"))
|
|
815
|
+
raise "out of range" unless v.between?("09:00:00.000000", "17:00:00.000000")
|
|
816
|
+
end
|
|
817
|
+
|
|
818
|
+
seen = []
|
|
819
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
820
|
+
seen << tc.draw(times(min_value: "09:00:00.000000", max_value: "17:00:00.000000"))
|
|
821
|
+
end
|
|
822
|
+
seen # => ["09:00:00.000000", "09:00:33.554432", "09:00:00.002652"]
|
|
823
|
+
```
|
|
824
|
+
|
|
825
|
+
`max_value < min_value` raises `Hegel::Error` at draw time: `"times:
|
|
826
|
+
max_value < min_value"`. A bound that is not a `"HH:MM:SS.ffffff"` String
|
|
827
|
+
also raises at draw time: `times(min_value: "1:2:3.4")` gives `'times:
|
|
828
|
+
min_value must be "HH:MM:SS.ffffff", got "1:2:3.4"'`.
|
|
829
|
+
|
|
830
|
+
### `datetimes(min_value: nil, max_value: nil)`
|
|
831
|
+
|
|
832
|
+
A naive (no timezone) `Time` in `[min_value, max_value]`, defaulting to
|
|
833
|
+
year 1 through year 9999.
|
|
834
|
+
|
|
835
|
+
```ruby
|
|
836
|
+
Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
837
|
+
v = tc.draw(datetimes(min_value: Time.utc(2020, 1, 1), max_value: Time.utc(2020, 12, 31, 23, 59, 59)))
|
|
838
|
+
raise "not a Time" unless v.is_a?(Time)
|
|
839
|
+
end
|
|
840
|
+
|
|
841
|
+
seen = []
|
|
842
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
843
|
+
seen << tc.draw(datetimes(min_value: Time.utc(2020, 1, 1), max_value: Time.utc(2020, 12, 31, 23, 59, 59))).to_s
|
|
844
|
+
end
|
|
845
|
+
seen # => ["2020-01-01 00:00:00 UTC", "2020-07-23 00:00:00 UTC", "2020-04-28 00:00:00 UTC"]
|
|
846
|
+
```
|
|
847
|
+
|
|
848
|
+
`max_value < min_value` raises `Hegel::Error` at draw time: `"datetimes:
|
|
849
|
+
max_value < min_value"`.
|
|
850
|
+
|
|
851
|
+
### `composite(&block)`
|
|
852
|
+
|
|
853
|
+
A value built from imperative code: `block` receives a draw surface and
|
|
854
|
+
may call `#draw` (or the direct `#draw_integer`/`#draw_boolean`
|
|
855
|
+
primitives) on it any number of times to assemble one value.
|
|
856
|
+
|
|
857
|
+
```ruby
|
|
858
|
+
pair = composite { |dtc| [dtc.draw(integers(min_value: 0, max_value: 10)), dtc.draw(text(max_size: 5, codec: "ascii"))] }
|
|
859
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
860
|
+
v = tc.draw(pair)
|
|
861
|
+
raise "wrong shape" unless v.is_a?(Array) && v.size == 2 && v[0].is_a?(Integer) && v[1].is_a?(String)
|
|
862
|
+
end
|
|
863
|
+
result # => nil
|
|
864
|
+
|
|
865
|
+
seen = []
|
|
866
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
867
|
+
seen << tc.draw(pair)
|
|
868
|
+
end
|
|
869
|
+
seen # => [[0, ""], [6, "11W1"], [3, "z\u007Fz\u0010~"]]
|
|
870
|
+
```
|
|
871
|
+
|
|
872
|
+
```ruby
|
|
873
|
+
generator = composite { |dtc| [dtc.draw_integer(0, 10), dtc.draw_boolean] }
|
|
874
|
+
seen = []
|
|
875
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
876
|
+
seen << tc.draw(generator)
|
|
877
|
+
end
|
|
878
|
+
seen # => [[0, false], [6, false], [3, false]]
|
|
879
|
+
```
|
|
880
|
+
|
|
881
|
+
Calling it with no block raises `Hegel::Error` at draw time: `"composite:
|
|
882
|
+
block is required"`.
|
|
883
|
+
|
|
884
|
+
### `deferred`
|
|
885
|
+
|
|
886
|
+
A forward reference to a generator whose definition is supplied later via
|
|
887
|
+
`#set`, enabling self-recursive and mutually recursive generators:
|
|
888
|
+
|
|
889
|
+
```ruby
|
|
890
|
+
tree = deferred
|
|
891
|
+
tree.set(one_of(integers(min_value: 0, max_value: 10), arrays(tree, max_size: 3)))
|
|
892
|
+
|
|
893
|
+
result = Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
894
|
+
v = tc.draw(tree)
|
|
895
|
+
raise "wrong type" unless v.is_a?(Integer) || v.is_a?(Array)
|
|
896
|
+
end
|
|
897
|
+
result # => nil
|
|
898
|
+
|
|
899
|
+
seen = []
|
|
900
|
+
Hegel.test(test_cases: 3, seed: 1, derandomize: true, verbosity: :quiet) do |tc|
|
|
901
|
+
seen << tc.draw(tree)
|
|
902
|
+
end
|
|
903
|
+
seen # => [0, [0, []], 4]
|
|
904
|
+
```
|
|
905
|
+
|
|
906
|
+
Drawing from it before `#set` raises `Hegel::Error` at draw time:
|
|
907
|
+
`"deferred: draw called before set"`. Calling `#set` a second time raises
|
|
908
|
+
immediately (not at draw time): `"deferred: set called more than once"`.
|
|
909
|
+
|
|
910
|
+
## Stateful Testing
|
|
911
|
+
|
|
912
|
+
A stateful test compares a real object against a simplified model of it,
|
|
913
|
+
across a sequence of actions libhegel chooses and shrinks the same way it
|
|
914
|
+
shrinks any other test case. Declare the actions on a `Hegel::StateMachine`
|
|
915
|
+
subclass, then drive one instance of it from inside an ordinary `Hegel.test`
|
|
916
|
+
block: `Hegel.test { |tc| Hegel::Stateful.run(machine, tc) }` (see
|
|
917
|
+
[A worked example](#a-worked-example) below for a complete one).
|
|
918
|
+
|
|
919
|
+
### Declaring a machine
|
|
920
|
+
|
|
921
|
+
`rule(name, &block)` and `invariant(name, &block)` are class-level
|
|
922
|
+
declarations. `block` runs via `instance_exec` against the machine instance
|
|
923
|
+
under test, and is handed the running `Hegel::TestCase` as its one argument
|
|
924
|
+
(ignored if the block takes none). So it reads and writes the machine's own
|
|
925
|
+
instance variables directly, and calls a generator method (`integers`,
|
|
926
|
+
`arrays`, and so on) bare, the same way a `Hegel.test` block's own
|
|
927
|
+
surrounding class can once it includes `Hegel::Syntax::Methods`;
|
|
928
|
+
`Hegel::StateMachine` already includes that module itself.
|
|
929
|
+
|
|
930
|
+
An invariant runs once before the first rule, and again after every rule
|
|
931
|
+
application that completes without its own assumption failing.
|
|
932
|
+
|
|
933
|
+
A rule or invariant reports a failure the same way any other test-case body
|
|
934
|
+
does: raise. This library brings no assertion methods of its own. Include
|
|
935
|
+
`Minitest::Assertions` (or `RSpec::Matchers`, or plain `raise`) into
|
|
936
|
+
the machine class to use one. `Minitest::Assertions#assert` counts against
|
|
937
|
+
an `assertions` accessor it does not define itself, so a class that includes
|
|
938
|
+
it also needs to supply one, initialized to `0`:
|
|
939
|
+
|
|
940
|
+
```ruby
|
|
941
|
+
class AssertingMachine < Hegel::StateMachine
|
|
942
|
+
include Minitest::Assertions
|
|
943
|
+
attr_accessor :assertions
|
|
944
|
+
|
|
945
|
+
def initialize
|
|
946
|
+
@assertions = 0
|
|
947
|
+
end
|
|
948
|
+
|
|
949
|
+
rule(:step) { |_tc| assert_equal 1, 1 }
|
|
950
|
+
end
|
|
951
|
+
```
|
|
952
|
+
|
|
953
|
+
Declaring the same rule or invariant name twice on one class raises
|
|
954
|
+
`Hegel::Error`; a subclass declaring a name its superclass already declared
|
|
955
|
+
replaces that one instead, keeping its original position in the declared
|
|
956
|
+
order. A class with no rule declared at all raises `Hegel::Error`, before
|
|
957
|
+
`Hegel::Stateful.run` makes any libhegel call:
|
|
958
|
+
|
|
959
|
+
```ruby
|
|
960
|
+
class NoRulesMachine < Hegel::StateMachine
|
|
961
|
+
invariant(:always) {}
|
|
962
|
+
end
|
|
963
|
+
|
|
964
|
+
# nil stands in for a running Hegel::TestCase here: the check above raises
|
|
965
|
+
# before Hegel::Stateful.run reads its second argument at all.
|
|
966
|
+
Hegel::Stateful.run(NoRulesMachine.new, nil)
|
|
967
|
+
# raises Hegel::Error, "hegel: NoRulesMachine has no rules; declare at least
|
|
968
|
+
# one with `rule`"
|
|
969
|
+
```
|
|
970
|
+
|
|
971
|
+
A test case with more than one declared rule enables only a random subset of
|
|
972
|
+
them, and picks each step from that subset. So a machine with two rules can
|
|
973
|
+
run a case where one of them never appears at all.
|
|
974
|
+
|
|
975
|
+
Inside a rule, `tc.assume(false)` means something narrower than it does
|
|
976
|
+
everywhere else in this library: it discards only that one rule application
|
|
977
|
+
(the loop moves on and tries another rule), not the whole test case, unlike
|
|
978
|
+
every other `assume` call, such as one at the top of a `Hegel.test` block,
|
|
979
|
+
or inside a generator's own draw.
|
|
980
|
+
|
|
981
|
+
### A worked example
|
|
982
|
+
|
|
983
|
+
`@real` below never enforces its own capacity. That is the bug a model
|
|
984
|
+
capped at 2 items is built to catch:
|
|
985
|
+
|
|
986
|
+
```ruby
|
|
987
|
+
class BoundedStackModel < Hegel::StateMachine
|
|
988
|
+
CAPACITY = 2
|
|
989
|
+
|
|
990
|
+
def initialize
|
|
991
|
+
@real = []
|
|
992
|
+
@model_size = 0
|
|
993
|
+
end
|
|
994
|
+
|
|
995
|
+
rule(:push) do |tc|
|
|
996
|
+
tc.draw(integers(min_value: 0, max_value: 9))
|
|
997
|
+
@real.push(0) # bug: never checks capacity
|
|
998
|
+
@model_size += 1 if @model_size < CAPACITY
|
|
999
|
+
end
|
|
1000
|
+
|
|
1001
|
+
invariant(:size_matches_capacity) do
|
|
1002
|
+
unless @real.size == @model_size
|
|
1003
|
+
raise "stack has #{@real.size} items, model expects #{@model_size}"
|
|
1004
|
+
end
|
|
1005
|
+
end
|
|
1006
|
+
end
|
|
1007
|
+
|
|
1008
|
+
output = StringIO.new
|
|
1009
|
+
begin
|
|
1010
|
+
Hegel.test(output: output, seed: 1, derandomize: true) do |tc|
|
|
1011
|
+
Hegel::Stateful.run(BoundedStackModel.new, tc)
|
|
1012
|
+
end
|
|
1013
|
+
rescue RuntimeError => e
|
|
1014
|
+
e.message # => "stack has 3 items, model expects 2"
|
|
1015
|
+
end
|
|
1016
|
+
output.string
|
|
1017
|
+
```
|
|
1018
|
+
|
|
1019
|
+
```
|
|
1020
|
+
Falsified after 1 test case (0 discarded):
|
|
1021
|
+
|
|
1022
|
+
Initial invariant check.
|
|
1023
|
+
Step 1: push
|
|
1024
|
+
draw_1 = 0
|
|
1025
|
+
Step 2: push
|
|
1026
|
+
draw_2 = 0
|
|
1027
|
+
Step 3: push
|
|
1028
|
+
draw_3 = 0
|
|
1029
|
+
|
|
1030
|
+
To reproduce this failure, pass the blob below to Hegel.test:
|
|
1031
|
+
reproduce_failure: "AXiclcaxDQAACMMwd0Xi/3dh4QCGKG6E2tzywQAOIgBh"
|
|
1032
|
+
```
|
|
1033
|
+
|
|
1034
|
+
The shrunk report names each step by its rule (`Step 1: push`), and shrinks
|
|
1035
|
+
the number of steps down to the minimum that still breaks the invariant.
|
|
1036
|
+
Here, that is exactly 3 pushes, one past capacity.
|
|
1037
|
+
|
|
1038
|
+
### `Hegel::Stateful::Pool`
|
|
1039
|
+
|
|
1040
|
+
A pool lets one rule's generated value be drawn again by a later rule,
|
|
1041
|
+
such as an allocator's handle freed by a rule that has to name the same
|
|
1042
|
+
handle `alloc` produced. Build one from the running test case, typically in
|
|
1043
|
+
the machine's own constructor:
|
|
1044
|
+
|
|
1045
|
+
```ruby
|
|
1046
|
+
pool = Hegel::Stateful::Pool.new(tc)
|
|
1047
|
+
```
|
|
1048
|
+
|
|
1049
|
+
- `#add(value)` records `value` under a fresh id, and returns `self`.
|
|
1050
|
+
- `#values_reusable` is a `Hegel::Generator` over the pool's values that
|
|
1051
|
+
leaves the chosen value in place, so it can be drawn again.
|
|
1052
|
+
- `#values_consumed` is a `Hegel::Generator` over the pool's values that
|
|
1053
|
+
removes the chosen value, so it is never drawn again.
|
|
1054
|
+
- `#size` and `#empty?` read the pool's own count directly.
|
|
1055
|
+
|
|
1056
|
+
```ruby
|
|
1057
|
+
Hegel.test(test_cases: 5, verbosity: :quiet) do |tc|
|
|
1058
|
+
pool = Hegel::Stateful::Pool.new(tc)
|
|
1059
|
+
raise "a fresh pool must be empty" unless pool.empty?
|
|
1060
|
+
|
|
1061
|
+
pool.add(42)
|
|
1062
|
+
raise "a pool holding a value must not be empty" if pool.empty?
|
|
1063
|
+
|
|
1064
|
+
value = tc.draw(pool.values_reusable)
|
|
1065
|
+
raise "expected 42, got #{value}" unless value == 42
|
|
1066
|
+
end
|
|
1067
|
+
```
|
|
1068
|
+
|
|
1069
|
+
Drawing from an empty pool raises `Hegel::AssumeFailed`: outside a rule, that
|
|
1070
|
+
discards the whole test case, the same as any other failed assumption;
|
|
1071
|
+
inside a rule, it discards only that rule, the same narrower meaning
|
|
1072
|
+
`tc.assume(false)` has there. A caller never frees a pool. The test case
|
|
1073
|
+
that built it owns it, and frees it once that test case is done.
|
|
1074
|
+
|
|
1075
|
+
## Combinator Methods
|
|
1076
|
+
|
|
1077
|
+
Every `Hegel::Generator`, including every generator above, has `.map`
|
|
1078
|
+
and `.filter`, each returning a new `Hegel::Generator`.
|
|
1079
|
+
|
|
1080
|
+
### `.map(&block)`
|
|
1081
|
+
|
|
1082
|
+
Transform drawn values:
|
|
1083
|
+
|
|
1084
|
+
```ruby
|
|
1085
|
+
positive_string = integers(min_value: 1).map { |n| n.to_s }
|
|
1086
|
+
|
|
1087
|
+
Hegel.test(test_cases: 10, verbosity: :quiet) do |tc|
|
|
1088
|
+
s = tc.draw(positive_string)
|
|
1089
|
+
raise "not a digit string" unless s.match?(/\A\d+\z/)
|
|
1090
|
+
end
|
|
1091
|
+
```
|
|
1092
|
+
|
|
1093
|
+
### `.filter(&block)`
|
|
1094
|
+
|
|
1095
|
+
Keep only values matching a predicate:
|
|
1096
|
+
|
|
1097
|
+
```ruby
|
|
1098
|
+
even = integers.filter { |n| n.even? }
|
|
1099
|
+
|
|
1100
|
+
Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
1101
|
+
n = tc.draw(even)
|
|
1102
|
+
raise "not even" unless n.even?
|
|
1103
|
+
end
|
|
1104
|
+
```
|
|
1105
|
+
|
|
1106
|
+
`.filter` retries its source generator up to 3 times per draw, then
|
|
1107
|
+
discards the test case. A predicate that (almost) never holds does not loop
|
|
1108
|
+
forever: libhegel's own health check aborts a run that discards too many
|
|
1109
|
+
test cases in a row, surfacing as `Hegel::Error`:
|
|
1110
|
+
|
|
1111
|
+
```ruby
|
|
1112
|
+
Hegel.test(test_cases: 20, verbosity: :quiet) do |tc|
|
|
1113
|
+
tc.draw(integers.filter { |_n| false })
|
|
1114
|
+
end
|
|
1115
|
+
```
|
|
1116
|
+
|
|
1117
|
+
raises `Hegel::Error`, message starting `"FailedHealthCheck: FilterTooMuch —
|
|
1118
|
+
it looks like this test is filtering out too many inputs."` Pass
|
|
1119
|
+
`suppress_health_check: [:filter_too_much]` (see [Settings](#settings)) to
|
|
1120
|
+
turn that check off for a filter deliberately built to reject often, or
|
|
1121
|
+
restructure the generator instead (prefer `.map` over `.filter`, or draw a
|
|
1122
|
+
value already in the shape you need):
|
|
1123
|
+
|
|
1124
|
+
```ruby
|
|
1125
|
+
result = Hegel.test(test_cases: 20, suppress_health_check: [:filter_too_much], verbosity: :quiet) do |tc|
|
|
1126
|
+
tc.draw(integers.filter { |_n| false })
|
|
1127
|
+
end
|
|
1128
|
+
result # => nil
|
|
1129
|
+
```
|
|
1130
|
+
|
|
1131
|
+
## Gotchas
|
|
1132
|
+
|
|
1133
|
+
1. **`floats` defaults `allow_nan: false, allow_infinity: false`, even when
|
|
1134
|
+
fully unbounded.** hegel-rust's and hegel-typescript's `floats()` default
|
|
1135
|
+
both to `true` when neither bound is set. This binding always starts
|
|
1136
|
+
both off; pass `allow_nan: true` and/or `allow_infinity: true`
|
|
1137
|
+
deliberately if the code under test needs to see them.
|
|
1138
|
+
|
|
1139
|
+
2. **`integers`'s default range, when a bound is omitted, is still the
|
|
1140
|
+
signed 64-bit range (`-2**63..(2**63 - 1)`), even though arbitrary
|
|
1141
|
+
precision is available.** Passing both `min_value` and `max_value`
|
|
1142
|
+
explicitly draws outside that range at arbitrary precision; leaving one
|
|
1143
|
+
bound out still defaults it to `-2**63` or `2**63 - 1`. That default can
|
|
1144
|
+
surprise a caller who supplies only one big bound:
|
|
1145
|
+
`integers(min_value: 10**30)` raises `Hegel::Error`, `"integers:
|
|
1146
|
+
max_value < min_value"`, because `max_value` defaults to `2**63 - 1`,
|
|
1147
|
+
smaller than `10**30`. Pass both bounds explicitly to draw a value
|
|
1148
|
+
outside the 64-bit range.
|
|
1149
|
+
|
|
1150
|
+
3. **Invalid generator options raise at draw time, not at construction
|
|
1151
|
+
time.** `integers(min_value: 5, max_value: 1)` builds without error;
|
|
1152
|
+
only `tc.draw` on it raises `Hegel::Error`. Every generator in this
|
|
1153
|
+
binding follows the same rule, matching hegel-rust's own contract.
|
|
1154
|
+
|
|
1155
|
+
4. **`Hegel.test` returns `nil` on a pass and re-raises the failing case's
|
|
1156
|
+
own exception on a failure**, class and backtrace intact. A host test
|
|
1157
|
+
framework (RSpec, Minitest) reports it as its own failure, not a
|
|
1158
|
+
Hegel-specific one.
|
|
1159
|
+
|
|
1160
|
+
5. **Failures are grouped by the line they are attributed to, and one
|
|
1161
|
+
group is one bug.** The attributed line is the first one in your own
|
|
1162
|
+
code: assertions count as failing where you wrote them, not inside
|
|
1163
|
+
`minitest` or `rspec-expectations` where the exception is raised. So
|
|
1164
|
+
two `assert_equal` failures on different lines are two bugs, and two
|
|
1165
|
+
different failures reaching Hegel through one line are one bug. That
|
|
1166
|
+
holds whether the line is a ternary or the single `raise` inside a
|
|
1167
|
+
helper you wrote. Split the line when two bugs need to stay apart:
|
|
1168
|
+
|
|
1169
|
+
```ruby
|
|
1170
|
+
# one bug: report_multiple_failures: true reports 1 failure
|
|
1171
|
+
n.zero? ? raise("boom-zero") : raise("boom-one")
|
|
1172
|
+
|
|
1173
|
+
# two bugs: report_multiple_failures: true reports 2
|
|
1174
|
+
if n.zero?
|
|
1175
|
+
raise "boom-zero"
|
|
1176
|
+
else
|
|
1177
|
+
raise "boom-one"
|
|
1178
|
+
end
|
|
1179
|
+
```
|
|
1180
|
+
|
|
1181
|
+
The exception's class is left out on purpose: one line raising a
|
|
1182
|
+
`NoMethodError` on one input and a `TypeError` on another is still one
|
|
1183
|
+
bug.
|
|
1184
|
+
|
|
1185
|
+
6. **`verbosity: :quiet` silences the failure report text itself**, not
|
|
1186
|
+
just libhegel's own progress output. Even when `output:` is given,
|
|
1187
|
+
nothing is written to it on a quiet run.
|
|
1188
|
+
|
|
1189
|
+
7. **`database:` without `database_key:` raises `Hegel::Error`
|
|
1190
|
+
immediately.** `database_key:` is the setting that turns the example
|
|
1191
|
+
database on, and `database:` only chooses where it writes. Left at
|
|
1192
|
+
their shared default (`nil`, `nil`), a run stores nothing and leaves no
|
|
1193
|
+
`.hegel/` directory behind.
|
|
1194
|
+
|
|
1195
|
+
8. **`tc.target` is a silent no-op, not an error, when the `:target` phase
|
|
1196
|
+
is disabled.** Dropping `:target` from `phases:` (every other phase
|
|
1197
|
+
kept) still lets every `tc.target` call succeed; it just stops
|
|
1198
|
+
influencing generation.
|
|
1199
|
+
|
|
1200
|
+
9. **Inside a stateful rule, `tc.assume(false)` discards only that one rule
|
|
1201
|
+
application, not the whole test case.** See
|
|
1202
|
+
[Stateful Testing](#stateful-testing). Every other `assume` call in this
|
|
1203
|
+
binding, such as one at the top of a `Hegel.test` block, or inside a
|
|
1204
|
+
generator's own draw, discards the whole test case.
|
|
1205
|
+
|
|
1206
|
+
10. **The suite runs against the real engine on Linux, macOS, and Windows.**
|
|
1207
|
+
Every push to main runs it on Ruby 3.3, 3.4, and 4.0, across Linux x64
|
|
1208
|
+
and arm64, macOS arm64, and Windows x64 and arm64. Windows arm64 starts
|
|
1209
|
+
at Ruby 3.4, the oldest Ruby that RubyInstaller publishes an arm64 build
|
|
1210
|
+
for. Alpine (musl) is the platform this binding has not run on.
|