hegeltest 0.3.4 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +182 -0
  3. data/README.ja.md +408 -0
  4. data/README.md +208 -102
  5. data/Rakefile +6 -3
  6. data/docs/README.md +9 -0
  7. data/docs/adr/0027-report-each-runs-test-location-from-the-callers-frame.md +63 -0
  8. data/docs/adr/0028-expose-settings-profiles-as-a-keyword-and-module-functions.md +91 -0
  9. data/docs/adr/0029-keep-notes-local-and-defer-run-error-kinds.md +72 -0
  10. data/docs/adr/0030-represent-a-duration-as-integer-nanoseconds.md +48 -0
  11. data/docs/adr/0031-take-the-alphabet-options-from-hegel-rust.md +52 -0
  12. data/docs/adr/0032-suppress-span-closes-while-an-abandoned-recursion-attempt-unwinds.md +147 -0
  13. data/docs/adr/0033-cache-compiled-grammars-for-the-length-of-a-run.md +80 -0
  14. data/docs/adr/0034-repeat-a-test-body-with-a-ruby-side-indent.md +120 -0
  15. data/docs/adr/0035-defer-concurrent-state-machines-and-the-engine-printer.md +136 -0
  16. data/docs/architecture.md +16 -1
  17. data/lib/hegel/errors.rb +24 -0
  18. data/lib/hegel/generator.rb +52 -25
  19. data/lib/hegel/generators.rb +715 -45
  20. data/lib/hegel/grammar_cache.rb +36 -0
  21. data/lib/hegel/lib_hegel/real.rb +320 -17
  22. data/lib/hegel/lib_hegel.rb +31 -6
  23. data/lib/hegel/libhegel_version.rb +1 -1
  24. data/lib/hegel/report.rb +13 -10
  25. data/lib/hegel/runner.rb +61 -27
  26. data/lib/hegel/settings.rb +92 -16
  27. data/lib/hegel/state_machine.rb +17 -4
  28. data/lib/hegel/stateful/pool.rb +52 -0
  29. data/lib/hegel/stateful.rb +2 -2
  30. data/lib/hegel/syntax/methods.rb +99 -15
  31. data/lib/hegel/test_case.rb +256 -35
  32. data/lib/hegel/test_location.rb +49 -0
  33. data/lib/hegel/version.rb +1 -1
  34. data/lib/hegel.rb +21 -5
  35. data/sig/hegel.rbs +295 -42
  36. data/skills/hegel-ruby/references/ruby/reference.md +463 -41
  37. metadata +13 -1
data/README.md CHANGED
@@ -27,78 +27,72 @@ failure to a minimal counterexample.
27
27
 
28
28
  This gem drives [libhegel](https://hegel.dev/reference/libhegel), the native
29
29
  engine that the official implementations also drive. The engine runs in the
30
- same process. There is no server and no Python dependency.
30
+ same process, through the [`ffi`](https://github.com/ffi/ffi) gem. There is no
31
+ server and no Python dependency.
31
32
 
32
- ## Status
33
+ ## Installation
33
34
 
34
- **Released as `hegeltest` 0.1.0.** Hegel finds a counterexample, shrinks it,
35
- names the values it drew, and re-raises your own exception.
36
-
37
- Work ran in three stages, and all three are done:
38
-
39
- 1. **The walking skeleton**: the libhegel binding, the run loop, and failure
40
- reports.
41
- 2. **The full generator set**: twenty-five generators, composing through
42
- `map` and `filter`.
43
- 3. **The advanced features**: the example database, targeted testing,
44
- stateful testing, phases, and health checks.
45
-
46
- Install it the way you install any gem:
35
+ The gem is published as [`hegeltest`](https://rubygems.org/gems/hegeltest):
47
36
 
48
37
  ```ruby
49
38
  # Gemfile
50
- gem "hegeltest"
39
+ gem "hegeltest", group: :test
51
40
  ```
52
41
 
53
- Each platform gem carries the matching `libhegel` build, so an install
54
- compiles nothing and needs no engine on the side. The Hegel maintainers
55
- permit a third-party project to redistribute their release binaries
56
- ([hegeldev/hegel-rust#411]).
42
+ The require path is `hegel`, and the namespace is `Hegel`.
43
+ `require "hegeltest"` also works, so Bundler's automatic require finds it.
44
+
45
+ The gem needs Ruby 3.4 or later. Each platform gem carries the matching
46
+ `libhegel` build, so an install compiles nothing and needs no engine on the
47
+ side:
48
+
49
+ | Platform | Gem platform |
50
+ | --- | --- |
51
+ | Linux x64 | `x86_64-linux` |
52
+ | Linux arm64 | `aarch64-linux` |
53
+ | macOS arm64 | `arm64-darwin` |
54
+ | Windows x64 | `x64-mingw-ucrt` |
55
+ | Windows arm64 | `aarch64-mingw-ucrt` |
56
+
57
+ The Hegel maintainers permit a third-party project to redistribute their
58
+ release binaries ([hegeldev/hegel-rust#411]).
57
59
 
58
- A project that already has a `Gemfile.lock` needs one more step. Bundler keeps
59
- the platform list the lock already holds. It adds only the platform of the
60
- machine that runs `bundle install`. A lock therefore names only the platforms
61
- of the machines that have run it. A Linux CI job cannot find the gem when the
62
- lock names no Linux platform. Add each platform your CI uses:
60
+ A project that already has a `Gemfile.lock` needs one more step. Bundler adds
61
+ only the platform of the machine that runs `bundle install`, so a lock made on
62
+ a Mac does not name Linux. A Linux CI job then cannot find the gem. Add each
63
+ platform your CI uses:
63
64
 
64
65
  ```bash
65
66
  bundle lock --add-platform x86_64-linux
66
67
  ```
67
68
 
68
- A project with no lock yet reads the `PLATFORMS` section of the lock its first
69
- `bundle install` writes, and adds whatever its CI needs the same way.
70
-
71
- Every push to main runs the suite on Ruby 3.4 and 4.0, across Linux x64 and
72
- arm64, macOS arm64, and Windows x64 and arm64. A pull request runs Linux x64
73
- alone.
69
+ On a platform with no published `libhegel` build, such as macOS on Intel, build
70
+ the engine yourself and set `HEGEL_LIBHEGEL_PATH` to the library file or to the
71
+ directory that holds it. The same variable makes any platform use a local
72
+ engine build in place of the bundled one.
74
73
 
75
74
  [hegeldev/hegel-rust#411]: https://github.com/hegeldev/hegel-rust/issues/411
76
75
 
77
- ## Design
76
+ ### Which libhegel a release carries
78
77
 
79
- | Decision | Choice |
80
- | --- | --- |
81
- | Gem name | `hegeltest` |
82
- | Require path | `require "hegel"` (`require "hegeltest"` also works) |
83
- | Namespace | `Hegel` |
84
- | Binding | the `ffi` gem, which publishes a prebuilt binary for every platform above |
85
- | Ruby | 3.4 and 4.0 |
86
- | Platforms | Linux amd64/arm64, macOS arm64, Windows amd64/arm64 |
87
- | Engine delivery | One prebuilt `libhegel` per platform-specific gem, built by `rake platform_gems:build` |
78
+ Each gem release pins one libhegel release. To see the pinned version in your
79
+ installed gem:
88
80
 
89
- `HEGEL_LIBHEGEL_PATH` overrides the bundled engine with a local build.
81
+ ```bash
82
+ ruby -rhegel -e 'puts Hegel::LIBHEGEL_VERSION'
83
+ ```
90
84
 
91
- The gem name follows the Rust implementation, whose published crate is
92
- `hegeltest` and whose library is `hegel`. The name `hegel` on RubyGems.org
93
- stays free for whoever publishes an official Ruby implementation.
85
+ The [CHANGELOG](CHANGELOG.md) section of each gem release starts with the
86
+ engine move, when there is one, and links the release notes of that engine
87
+ version. The [GitHub releases](https://github.com/meganemura/hegel-ruby/releases)
88
+ show the same text.
94
89
 
95
- macOS on Intel has no published `libhegel` artifact, so that platform needs
96
- `HEGEL_LIBHEGEL_PATH` and a local build.
90
+ When libhegel publishes a new release, a pull request moves this gem to it.
91
+ The gem version and the engine version are independent: a gem release can
92
+ change the engine, the Ruby code, or both.
97
93
 
98
94
  ## Quickstart
99
95
 
100
- Everything below runs today.
101
-
102
96
  ```ruby
103
97
  # spec/spec_helper.rb
104
98
  require "hegel"
@@ -123,7 +117,7 @@ end
123
117
  ```
124
118
 
125
119
  That test fails, and Hegel shrinks the failure to the smallest input that
126
- still shows the bug:
120
+ still shows the bug. The report looks like this:
127
121
 
128
122
  ```
129
123
  Falsified after 11 test cases (0 discarded):
@@ -134,27 +128,26 @@ To reproduce this failure, pass the blob below to Hegel.test:
134
128
  reproduce_failure: "AXicY2VgYGBkZOBiZEBhMAAAAd8AIQ=="
135
129
  ```
136
130
 
137
- Two duplicates are all it takes. Hegel names the value `xs` by reading the
138
- line the draw was written on, then re-raises RSpec's own expectation failure,
139
- so the framework reports it as its own.
131
+ The count of test cases and the blob change from run to run, and between
132
+ engine releases. The shrunk value is the part to read: two duplicates are all
133
+ it takes. Hegel names the value `xs` by reading the line the draw was written
134
+ on. Then it re-raises RSpec's own expectation failure, so RSpec reports it as
135
+ its own.
140
136
 
141
137
  Including `Hegel::Syntax::Methods` makes the generators available without a
142
- prefix, the way FactoryBot makes `create` available. The same generators stay
143
- reachable as `Hegel::Generators.arrays(...)` without the include.
138
+ prefix, the way FactoryBot makes `create` available. The same generators are
139
+ also available as `Hegel::Generators.arrays(...)` without the include.
144
140
 
145
- An existing RSpec suite can limit the 25 generator names to tagged example
146
- groups. This keeps the names out of other groups that have their own helpers:
141
+ To keep the generator names out of example groups that have their own
142
+ helpers, include the methods only in tagged groups:
147
143
 
148
144
  ```ruby
149
145
  config.include Hegel::Syntax::Methods, :hegel
150
146
  ```
151
147
 
152
- A new project can include the methods in every group, as the Quickstart does.
153
-
154
148
  Hegel draws values imperatively, so an ordinary Ruby method can share a group
155
149
  of draws between properties. Give the method parameter a descriptive name,
156
- which the default RuboCop naming rule accepts where a block parameter `tc`
157
- needs no change:
150
+ which the default RuboCop naming rule accepts:
158
151
 
159
152
  ```ruby
160
153
  def draw_sorted_pair(test_case)
@@ -169,15 +162,9 @@ Hegel.test do |tc|
169
162
  end
170
163
  ```
171
164
 
172
- The generators are `arrays`, `binary`, `booleans`, `characters`, `composite`,
173
- `dates`, `datetimes`, `deferred`, `domains`, `emails`, `floats`, `from_regex`,
174
- `hashes`, `integers`, `ip_addresses`, `just`, `one_of`, `optional`,
175
- `sampled_from`, `sets`, `text`, `times`, `tuples`, `urls`, and `uuids`. Each
176
- one composes through `map` and `filter`.
177
-
178
165
  ### Minitest
179
166
 
180
- Hegel needs nothing from your test framework. Install the generator methods
167
+ Hegel needs nothing from your test framework. Include the generator methods
181
168
  once, and write the property inside an ordinary test:
182
169
 
183
170
  ```ruby
@@ -201,8 +188,7 @@ class MySortTest < Minitest::Test
201
188
  end
202
189
  ```
203
190
 
204
- Minitest reports the shrunk case as its own assertion failure, pointing at
205
- your line:
191
+ Minitest reports the shrunk case as its own assertion failure, at your line:
206
192
 
207
193
  ```
208
194
  1) Failure:
@@ -211,10 +197,28 @@ Expected: [0, 0]
211
197
  Actual: [0]
212
198
  ```
213
199
 
214
- ### Stateful testing
200
+ ## Generators
201
+
202
+ | Kind | Generators |
203
+ | --- | --- |
204
+ | Numbers and booleans | `integers`, `floats`, `booleans` |
205
+ | Text and bytes | `text`, `characters`, `binary`, `from_regex`, `from_lark` |
206
+ | Collections | `arrays`, `sets`, `hashes`, `tuples` |
207
+ | Choice | `just`, `sampled_from`, `permutations`, `subsequences`, `samples`, `one_of`, `optional` |
208
+ | Formats | `emails`, `urls`, `domains`, `ip_addresses`, `uuids` |
209
+ | Time | `dates`, `times`, `datetimes`, `date_strings`, `time_strings`, `datetime_strings`, `durations` |
210
+ | Your own | `composite`, `deferred`, `recursive` |
211
+
212
+ Each one composes through `map` and `filter`. Options are keywords, with the
213
+ names the other Hegel implementations use, such as
214
+ `integers(min_value: 0, max_value: 9)`, `arrays(text, max_size: 5)`, or `arrays(integers, unique: true)`. An
215
+ invalid combination of options raises when the generator is drawn, not when it
216
+ is built.
217
+
218
+ ## Stateful testing
215
219
 
216
220
  Some bugs need a sequence of operations rather than one input. Declare the
217
- operations as rules on a `Hegel::StateMachine`, and Hegel picks which runs
221
+ operations as rules on a `Hegel::StateMachine`. Hegel picks which rule runs
218
222
  next, checks your invariants along the way, and shrinks a failure to the
219
223
  shortest sequence that still shows it:
220
224
 
@@ -245,32 +249,48 @@ Hegel.test { |tc| Hegel::Stateful.run(StackMachine.new, tc) }
245
249
  ```
246
250
 
247
251
  `tc.assume` inside a rule rejects that rule and lets Hegel choose another,
248
- rather than throwing the whole test case away. For a rule that has to act on
249
- something an earlier rule produced, such as freeing a handle that some
250
- `alloc` actually returned, put the value in a `Hegel::Stateful::Pool` and
251
- draw it back out.
252
+ rather than throwing the whole test case away. When a rule must act on a value
253
+ that an earlier rule made, such as freeing a handle that some `alloc` returned,
254
+ put the value in a `Hegel::Stateful::Pool` and draw it back out.
252
255
 
253
256
  Every invariant runs before the first rule and after the last. Between rules,
254
- the engine samples each one, so that it runs about once in a test case
255
- that runs every step.
256
- Declare an invariant with `always_run: true` to check it after every rule
257
- instead. `Hegel::Stateful.run(machine, tc, step_count: 50)` sets the most
258
- rules one test case runs; 50 is the default.
259
-
260
- ### Shaping a run
261
-
262
- `Hegel.test` takes keywords for the rest: `test_cases`, `seed`,
263
- `derandomize`, `verbosity`, `phases`, `suppress_health_check`, and
264
- `report_multiple_failures`. `report_multiple_failures` defaults to `false`.
265
- Each of the others left unset takes its value from the engine's settings
266
- profile. That is libhegel's own default, unless a
267
- `hegel.toml` or a `HEGEL_*` environment variable says otherwise. On a CI
268
- server the engine picks its `ci` profile, which derandomizes the run and
269
- turns the example database off.
270
-
271
- Two more turn on libhegel's example database, which stores a failing case and
272
- replays it first next time. `database_key` is the switch and `database`
273
- chooses the directory:
257
+ the engine samples each invariant. Declare an invariant with
258
+ `always_run: true` to check it after every rule.
259
+ `Hegel::Stateful.run(machine, tc, step_count: 50)` sets the largest number of
260
+ rules that one test case runs; 50 is the default. `rule :name, weight: 2.5`
261
+ asks Hegel to pick a rule more often than the others; the default weight is
262
+ 1.0.
263
+
264
+ ## Configuring a run
265
+
266
+ `Hegel.test` takes these keywords:
267
+
268
+ | Keyword | Effect |
269
+ | --- | --- |
270
+ | `profile` | The settings profile the run starts from; see [Profiles](#profiles) |
271
+ | `test_cases` | How many test cases to generate |
272
+ | `seed` | The seed for the random generation |
273
+ | `derandomize` | Make the generation deterministic |
274
+ | `verbosity` | How much the engine prints; `:quiet` also hides the failure report |
275
+ | `phases` | Which engine phases run, such as generation and shrinking |
276
+ | `suppress_health_check` | Which health checks to turn off |
277
+ | `report_multiple_failures` | Report each distinct failure; `false` by default |
278
+ | `stop_on_first_failure` | `false` spends the whole `test_cases` budget before reporting |
279
+ | `print_blob` | `false` leaves the reproduction line out of the failure report |
280
+ | `show_statistics` | Print what `tc.event` and `tc.event_value` recorded, at the end of the run |
281
+ | `nondeterminism_strictness` | `:quiet`, `:warn`, or `:error` when the test behaves differently on a replay |
282
+ | `backend` | `:default`, or `:urandom` to take fresh entropy on every draw, for Antithesis |
283
+ | `database_key` | Turn the example database on for this property |
284
+ | `database` | The directory of the example database |
285
+ | `reproduce_failure` | Replay one failure from the blob in its report |
286
+
287
+ A keyword that you do not set takes its value from the engine's settings
288
+ profile. That is libhegel's own default, unless a `hegel.toml` file or a
289
+ `HEGEL_*` environment variable changes it. On a CI server the engine uses its
290
+ `ci` profile, which derandomizes the run and turns the example database off.
291
+
292
+ The example database stores a failing case and replays it first next time.
293
+ `database_key` turns it on, and `database` chooses the directory:
274
294
 
275
295
  ```ruby
276
296
  Hegel.test(database_key: "my_sort matches the builtin sort") { |tc| ... }
@@ -278,22 +298,104 @@ Hegel.test(database_key: "my_sort matches the builtin sort") { |tc| ... }
278
298
 
279
299
  Give each property its own key. See
280
300
  [ADR 0009](docs/adr/0009-turn-the-example-database-on-with-a-key.md) for why
281
- the key, rather than the directory, is what turns it on.
301
+ the key, and not the directory, turns it on.
302
+
303
+ Inside a test, `tc.note` records a message that the failure report prints,
304
+ `tc.target` tells Hegel which inputs to search toward, and `tc.repeat` runs a
305
+ block for as many iterations as Hegel chooses.
306
+
307
+ `tc.event` and `tc.event_value` check that a test reaches the cases you care
308
+ about. With `show_statistics: true`, the engine prints how often each event
309
+ happened and a summary of each value. The numbers change from run to run:
310
+
311
+ ```ruby
312
+ Hegel.test(show_statistics: true) do |tc|
313
+ xs = tc.draw(arrays(integers))
314
+ tc.event("empty") if xs.empty?
315
+ tc.event_value("size", xs.size)
316
+ end
317
+ ```
318
+
319
+ ```
320
+ Statistics (over 100 test cases):
321
+ * empty: 6.0% of test cases
322
+ * size: count 100, min 0, median 6, mean 7.99, p90 19, max 20
323
+ ```
324
+
325
+ ## Profiles
326
+
327
+ A profile is a named set of settings. The engine ships `development`, `ci`,
328
+ and `workload`, and a `hegel.toml` file can define more. Pass `profile:` to
329
+ start one run from a profile, and register your own with
330
+ `Hegel::Settings.register_profile`:
331
+
332
+ ```ruby
333
+ Hegel::Settings.register_profile("nightly", profile: "ci", test_cases: 10_000)
334
+
335
+ Hegel.test(profile: "nightly") { |tc| ... }
336
+ ```
337
+
338
+ A keyword passed to `Hegel.test` still beats the profile. A profile you
339
+ name extends the default profile, so what it does not set comes from
340
+ there. `Hegel::Settings.set_default_profile("nightly")` makes a profile the
341
+ default for every run in the process, and `set_default_profile("")` clears
342
+ it. Registration and the default are process-wide, and they do not change
343
+ runs that have already started. `register_profile` takes the keywords of
344
+ `Hegel.test` except `database_key`, which a profile does not carry, and
345
+ `Hegel.test` always passes `report_multiple_failures`, so a profile's value
346
+ for it never applies. An unknown name raises `Hegel::Error`. See
347
+ [ADR 0028](docs/adr/0028-expose-settings-profiles-as-a-keyword-and-module-functions.md).
282
348
 
283
- Inside a test, `tc.note` records a message the failure report prints, and
284
- `tc.target` tells Hegel which inputs to search toward.
349
+ ## Agent skill
350
+
351
+ Two skills work together. The official
352
+ [hegel-skill](https://github.com/hegeldev/hegel-skill) from the Hegel
353
+ maintainers teaches the method: which properties to look for, and how to
354
+ check the tests an agent writes. It works with every Hegel implementation.
355
+ [`skills/hegel-ruby`](skills/hegel-ruby/SKILL.md) adds the Ruby part: this
356
+ gem's API and how to use it with RSpec and Minitest. Install both.
357
+
358
+ In Claude Code, add each repository as a marketplace and install its plugin:
359
+
360
+ ```
361
+ /plugin marketplace add hegeldev/hegel-skill
362
+ /plugin install hegel-skill@hegeldev
363
+ /plugin marketplace add meganemura/hegel-ruby
364
+ /plugin install hegel-ruby@meganemura
365
+ ```
366
+
367
+ For Codex and other agents, see the installation section of the
368
+ [hegel-skill README](https://github.com/hegeldev/hegel-skill#installation).
369
+ For this gem's skill, use the same steps with
370
+ <https://github.com/meganemura/hegel-ruby/tree/main/skills/hegel-ruby>.
285
371
 
286
372
  ## Development
287
373
 
288
374
  ```bash
289
375
  bin/setup # install dependencies
290
376
  bundle exec rake libhegel:fetch # download the pinned engine, checked against its SHA-256
291
- bundle exec rake # run the tests and the linter
377
+ bundle exec rake # run the tests with coverage, and the linter
378
+ bundle exec rake mutation # run mutation testing
292
379
  bin/console # open an interactive prompt
293
380
  ```
294
381
 
295
- `just` recipes call the same Rake tasks, so `just test` and `just lint` work
296
- for anyone who already uses `just` with the other Hegel implementations.
382
+ `just` recipes call the same Rake tasks, so `just check`, `just test`, and
383
+ `just lint` work for anyone who uses `just` with the other Hegel
384
+ implementations.
385
+
386
+ CI runs the suite on each pull request and on each push to main, on every
387
+ platform in the table above, with Ruby 3.4 and 4.0.
388
+
389
+ The design records are in [docs/](docs/README.md).
390
+
391
+ ### Releasing
392
+
393
+ A merge into main that changes `lib/hegel/version.rb` starts a release. The
394
+ Release workflow waits for an approval of the `release` environment. After
395
+ the approval, it runs the tests, publishes the gems to RubyGems, and then
396
+ creates the tag `v<version>` and the GitHub release. The release notes come
397
+ from the version's section of [CHANGELOG.md](CHANGELOG.md). See
398
+ [ADR 0024](docs/adr/0024-release-when-a-merge-changes-the-gem-version.md).
297
399
 
298
400
  ## Contributing
299
401
 
@@ -308,3 +410,7 @@ This gem is available under the [MIT License](LICENSE.txt).
308
410
  Released gems also carry `libhegel`'s own license, which is MIT and copyright
309
411
  Antithesis, LLC. Each platform gem ships `NOTICE-libhegel.txt` next to the
310
412
  engine, and that file holds the text.
413
+
414
+ ---
415
+
416
+ [日本語版 README](README.ja.md)
data/Rakefile CHANGED
@@ -18,8 +18,8 @@ end
18
18
 
19
19
  task default: %i[coverage standard]
20
20
 
21
- # Mutation testing with mutineer, over lib/. Four settings make the suite
22
- # run under it; ADR 0016 records why:
21
+ # Mutation testing with mutineer, over lib/. Five settings make the suite
22
+ # run under it; ADR 0016 records the first four, and ADR 0032 the fifth:
23
23
  # - RUBYOPT gets -Itest, since mutineer puts only lib/ on the load path and
24
24
  # every test file starts with `require "test_helper"`.
25
25
  # - Each test file gets its own --test flag. The flag takes one file, and a
@@ -31,6 +31,9 @@ task default: %i[coverage standard]
31
31
  # reload, loads the whole mutated file by a relative path, which breaks
32
32
  # Hegel::Runner.origin_for and killed three mutants the suite does not
33
33
  # catch.
34
+ # - test_recursive_engine.rb stays out too. Its 30-leaf shrink takes about
35
+ # ten seconds, mutineer's limit for one mutant, so a mutant on a line it
36
+ # covers timed out without a verdict.
34
37
  # MUTINEER_ARGS passes more flags, for example
35
38
  # MUTINEER_ARGS="--since origin/main" to mutate only the lines changed since
36
39
  # origin/main.
@@ -42,7 +45,7 @@ MUTATION_BASELINE = "test/mutation-baseline.json"
42
45
  def run_mutineer(*extra)
43
46
  require "shellwords"
44
47
  tests = Dir["test/*.rb", "test/hegel/*.rb"].sort -
45
- %w[test/test_helper.rb test/hegel/test_conformance.rb]
48
+ %w[test/test_helper.rb test/hegel/test_conformance.rb test/hegel/test_recursive_engine.rb]
46
49
  args = ["lib", "--strategy", "redefine", *tests.flat_map { |t| ["--test", t] }, *extra, *Shellwords.split(ENV.fetch("MUTINEER_ARGS", ""))]
47
50
  rubyopt = ["-Itest", ENV["RUBYOPT"]].compact.join(" ")
48
51
  # mutineer keeps its cache here, and refuses an --output path whose
data/docs/README.md CHANGED
@@ -33,6 +33,15 @@ An index of the design records for `hegel-ruby`.
33
33
  - [0024: Release when a merge changes the gem version](adr/0024-release-when-a-merge-changes-the-gem-version.md)
34
34
  - [0025: Copy each string input into a hegel_string_t](adr/0025-copy-each-string-input-into-a-hegel-string-t.md)
35
35
  - [0026: Select the engine release repository by version](adr/0026-select-the-engine-release-repository-by-version.md)
36
+ - [0027: Report each run's test location from the caller's frame](adr/0027-report-each-runs-test-location-from-the-callers-frame.md)
37
+ - [0028: Expose settings profiles as a keyword and module functions](adr/0028-expose-settings-profiles-as-a-keyword-and-module-functions.md)
38
+ - [0029: Keep notes local, and defer run error kinds](adr/0029-keep-notes-local-and-defer-run-error-kinds.md)
39
+ - [0030: Represent a duration as Integer nanoseconds](adr/0030-represent-a-duration-as-integer-nanoseconds.md)
40
+ - [0031: Take the alphabet options from hegel-rust](adr/0031-take-the-alphabet-options-from-hegel-rust.md)
41
+ - [0032: Suppress span closes while an abandoned recursion attempt unwinds](adr/0032-suppress-span-closes-while-an-abandoned-recursion-attempt-unwinds.md)
42
+ - [0033: Cache compiled grammars for the length of a run](adr/0033-cache-compiled-grammars-for-the-length-of-a-run.md)
43
+ - [0034: Repeat a test body with a Ruby-side indent](adr/0034-repeat-a-test-body-with-a-ruby-side-indent.md)
44
+ - [0035: Defer concurrent state machines and the engine printer](adr/0035-defer-concurrent-state-machines-and-the-engine-printer.md)
36
45
 
37
46
  A new decision gets a new record. A changed decision supersedes the old
38
47
  record instead of editing it, so the history stays readable.
@@ -0,0 +1,63 @@
1
+ # 0027: Report each run's test location from the caller's frame
2
+
3
+ ## Status
4
+
5
+ Accepted.
6
+
7
+ ## Context
8
+
9
+ `hegel_settings_set_test_location` takes the file, the first line, the
10
+ enclosing class, and the name of the test function. Under Antithesis, which
11
+ the engine detects through `ANTITHESIS_OUTPUT_DIR`, the engine appends one
12
+ `always` assertion named `<class>::<function> passes properties` to
13
+ `sdk.jsonl` in that directory, located at `file:begin_line`. Outside
14
+ Antithesis it uses nothing. hegel-rust fills the four values in
15
+ `#[hegel::test]` from `file!()`, `module_path!()`, the function name, and
16
+ `line!()`, and takes no keyword for them.
17
+
18
+ Ruby has no macro that sees the definition. A `Hegel.test` call can read its
19
+ own caller's frame with `caller_locations`, and a frame label in Ruby 3.4
20
+ reads `Class#method` or `Class.method`, with `block in ` or
21
+ `block (2 levels) in ` in front of it inside a block. A call at the top of a
22
+ file, which is where an RSpec example sits, has the label `<main>` or
23
+ `<top (required)>`, and a frame whose method is defined on an anonymous
24
+ class has the bare method name.
25
+
26
+ ## Decision
27
+
28
+ `Hegel.test` builds one `Hegel::TestLocation`, a `Data` of `file`,
29
+ `begin_line`, `class_name`, and `function`, and hands it to `Runner.run`.
30
+ `Runner.run` calls the setter after `Settings.apply` and before it starts
31
+ the run or replays a blob. The location is identity, not a setting, so
32
+ `Settings.apply` does not take it, and `Hegel.test` has no keyword for it.
33
+
34
+ The frame is the first one that `Runner.infrastructure?` does not call
35
+ infrastructure, the same rule that names the origin of a failure (see
36
+ [ADR 0012](0012-build-a-failure-origin-from-the-callers-own-frame.md)). A
37
+ wrapper in this gem, a gem, or the standard library is therefore never named
38
+ as the test.
39
+
40
+ `file` is the frame's absolute path made relative to `Dir.pwd` when it lies
41
+ under it, and absolute otherwise. `begin_line` is the frame's line, the line
42
+ of the `Hegel.test` call.
43
+
44
+ A label that splits as `Class#method` or `Class.method`, after the `block
45
+ ... in ` prefix is stripped, gives `class_name` and `function`. Any other
46
+ label has no name of its own, so `class_name` is the `file` without `.rb`
47
+ and `function` is `line <n>`. Two examples in one spec file then report two
48
+ assertions, where one shared name would merge them into one that fails when
49
+ either fails.
50
+
51
+ ## Consequences
52
+
53
+ The line is that of the call, where hegel-rust's is that of the function's
54
+ definition, so a method with two `Hegel.test` calls reports two assertions
55
+ with the same name at two lines.
56
+
57
+ A caller cannot choose the name. hegel-rust's macro derives it as well, so a
58
+ keyword would be a change to the shared interface, which starts as a proposal
59
+ upstream (see
60
+ [ADR 0015](0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md)).
61
+
62
+ The cost is one `caller_locations` call and one regular expression match for
63
+ each `Hegel.test` call, none for each test case.
@@ -0,0 +1,91 @@
1
+ # 0028: Expose settings profiles as a keyword and module functions
2
+
3
+ ## Status
4
+
5
+ Accepted. Builds on [ADR 0023](0023-leave-unset-settings-to-the-engines-profile.md),
6
+ which left the profile to the engine, and applies
7
+ [ADR 0015](0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md).
8
+
9
+ ## Context
10
+
11
+ libhegel 0.47.2 has three calls for profiles. `hegel_settings_new_for_profile`
12
+ builds a settings handle from a named profile. `hegel_settings_register_profile`
13
+ stores a complete snapshot of a handle under a name, for the whole process,
14
+ replacing an earlier registration of that name. `hegel_set_default_profile`
15
+ names the profile that `hegel_settings_new` resolves, for the whole process,
16
+ and an empty name clears it. A named profile still extends the default
17
+ profile, except `base`. An unknown name, a reserved name, a malformed
18
+ `hegel.toml`, and a malformed `HEGEL_*` variable return `HEGEL_E_INVALID_ARG`
19
+ with a message. The snapshot leaves out the database key, and a registration
20
+ does not change handles that already exist.
21
+
22
+ hegel-rust exposes them on its `Settings` value: `Settings::from_profile`
23
+ and `Settings::try_from_profile` return a `Settings` to adjust with builder
24
+ methods, `Settings::register_profile(name, settings)` registers one, and
25
+ `Settings::set_default_profile(name)` sets the default. Its documentation
26
+ example is `Settings::register_profile("nightly",
27
+ Settings::from_profile("ci").test_cases(10_000))`.
28
+
29
+ Ruby has no `Settings` value. `Hegel.test` takes keywords, and
30
+ `Hegel::Settings` is a module of functions that copy those keywords onto a
31
+ native handle that exists only for the length of one run (ADR 0023).
32
+
33
+ ## Decision
34
+
35
+ **`Hegel.test` takes `profile:`, which stands in for `from_profile`.** A
36
+ `nil` profile calls `hegel_settings_new`, as before. A String calls
37
+ `hegel_settings_new_for_profile`. The keywords apply afterwards in both
38
+ cases, so one passed explicitly still beats the profile's value, and the
39
+ `reproduce_failure:` path builds its settings the same way. This is a
40
+ difference of mechanism under ADR 0015: the value that hegel-rust builds and
41
+ then adjusts is, here, the argument of the call that uses it.
42
+
43
+ **`Hegel::Settings.register_profile(name, profile: nil, **keywords)`
44
+ registers a profile.** It sits on `Hegel::Settings` because hegel-rust puts
45
+ `register_profile` on `Settings`. It opens a context of its own, builds a
46
+ handle from `hegel_settings_new` or from the named `profile:`, applies the
47
+ keywords, registers the handle, and frees the handle and then the context in
48
+ an `ensure`. It takes an `impl:` keyword, defaulting to `Hegel.default_impl`,
49
+ so a test can pass the Fake the way it does to `Hegel.test`.
50
+
51
+ The keywords are those of `Hegel.test`, each `nil` meaning that no setter
52
+ runs, with these exceptions:
53
+
54
+ - `database_key:` is not accepted. The engine leaves the key out of the
55
+ snapshot, so accepting it would promise something the profile cannot keep.
56
+ Ruby's own `ArgumentError` rejects it, as it does any unknown keyword.
57
+ - `database:` goes straight to `hegel_settings_set_database`, a path or `""`
58
+ to turn the database off. The table of ADR 0009 exists to decide what a
59
+ missing key means for a run, and a profile carries no key.
60
+ - `report_multiple_failures:` is `nil` by default here, where `Hegel.test`
61
+ defaults it to `false`.
62
+
63
+ Starting from `hegel_settings_new` matches `Settings::new()`, which resolves
64
+ the default profile. hegel-rust's documented example starts from another
65
+ profile instead, and `profile: "ci"` is the same thing here.
66
+
67
+ **`Hegel::Settings.set_default_profile(name)` sets the default profile,**
68
+ with the same `impl:` keyword. `""` clears it.
69
+
70
+ An error from the engine raises `Hegel::Error` with the engine's message,
71
+ through `LibHegel.check!`. hegel-rust panics with the same message.
72
+
73
+ ## Consequences
74
+
75
+ `Hegel.test` always passes `report_multiple_failures`, as ADR 0023 decided,
76
+ so a profile's value for it is overridden on every run. A profile can carry
77
+ the setting, but it never changes what `Hegel.test` does.
78
+
79
+ Registrations and the default profile are state of the process, held by the
80
+ engine and not by any context. A test that registers a profile leaves it
81
+ registered, and the engine has no call to remove one. A test that sets the
82
+ default profile clears it again with `set_default_profile("")`. Neither
83
+ affects a run that is already in progress.
84
+
85
+ A `HEGEL_*` variable applies over a profile's snapshot, as the header says
86
+ for every handle, so `HEGEL_PRINT_BLOB` beats a `print_blob: false` that the
87
+ profile registered. The tests that observe a profile through the failure
88
+ report clear those variables first.
89
+
90
+ Each call to `register_profile` and `set_default_profile` opens and frees a
91
+ context. Both are meant for setup, run once, not once per test case.