hegeltest 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +103 -0
  3. data/README.md +72 -26
  4. data/Rakefile +81 -0
  5. data/docs/README.md +10 -0
  6. data/docs/adr/0014-name-a-drawn-value-only-when-the-draw-is-the-whole-assigned-value.md +74 -0
  7. data/docs/adr/0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md +101 -0
  8. data/docs/adr/0016-run-mutation-testing-with-mutineer.md +87 -0
  9. data/docs/adr/0017-raise-the-ruby-floor-to-3-4.md +43 -0
  10. data/docs/adr/0018-gate-mutation-testing-on-a-committed-baseline.md +74 -0
  11. data/docs/adr/0019-report-failures-from-the-cases-the-engine-stamps.md +70 -0
  12. data/docs/adr/0020-derive-span-labels-from-generator-names.md +64 -0
  13. data/docs/adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md +59 -0
  14. data/docs/adr/0022-keep-microsecond-times-over-a-nanosecond-engine.md +45 -0
  15. data/docs/adr/0023-leave-unset-settings-to-the-engines-profile.md +50 -0
  16. data/docs/architecture.md +11 -6
  17. data/lib/hegel/draw_name.rb +24 -8
  18. data/lib/hegel/generator.rb +50 -9
  19. data/lib/hegel/generators.rb +134 -102
  20. data/lib/hegel/lib_hegel/real.rb +193 -94
  21. data/lib/hegel/lib_hegel.rb +30 -56
  22. data/lib/hegel/libhegel_version.rb +1 -1
  23. data/lib/hegel/report.rb +17 -8
  24. data/lib/hegel/runner.rb +153 -201
  25. data/lib/hegel/settings.rb +9 -14
  26. data/lib/hegel/state_machine.rb +15 -8
  27. data/lib/hegel/stateful/pool.rb +0 -2
  28. data/lib/hegel/stateful.rb +64 -36
  29. data/lib/hegel/syntax/methods.rb +3 -2
  30. data/lib/hegel/test_case.rb +29 -9
  31. data/lib/hegel/version.rb +1 -1
  32. data/lib/hegel.rb +11 -17
  33. data/lib/tasks/libhegel.rake +9 -2
  34. data/sig/hegel.rbs +60 -61
  35. data/skills/hegel-ruby/references/ruby/reference.md +147 -52
  36. metadata +12 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: facbe6e366b74782ab68e979295856ef15c8da56ea9d9fe9291764cbf1629eb3
4
- data.tar.gz: e78e0364232dc02974f7ca81135a34325a76287c425d59f1d384ef2f2acfb2aa
3
+ metadata.gz: fc58a6595c70e7dd1e873f76049109ceea93f5a7304ff4a6b4d2287072d0a0e6
4
+ data.tar.gz: 42099dfad36fa60a4a15b17d4713062a5224b4926a2ea15f56e84758f56e93f8
5
5
  SHA512:
6
- metadata.gz: c977d4fd78f3439b18b731ae14c6772968aff53271fe1f44a8712f044a6dc0bcdcbd0ece18da677a7e3599a831651806a979c6985d5b942e6d89534c5c6eae14
7
- data.tar.gz: 429fb8e7ce09f224c32368ad0d34bc69d1020f56891bdb2616da02acb86fa0ef5c174f340f27fc5a0cdf4918e8a4f82075781128850656b571e0b4232143abf5
6
+ metadata.gz: eb7cdb67a056b0db5e3803f915bb8b09001ac2ea5918ec1fd3763345f295e56140075aa02260a8efcc3bf3f6d6009b352066ab116a23797f02b42691cc93b018
7
+ data.tar.gz: 06bd1f1bfae6a4918adff338816c0d2f35cfca94fb063017ebcfdbd34c27581fe60b7006d7244fc12266e51a19e704a68053a3e568952aa5e28133703cae871f
data/CHANGELOG.md CHANGED
@@ -1,5 +1,106 @@
1
1
  # Changelog
2
2
 
3
+ ## [Unreleased]
4
+
5
+ ## [0.2.0] - 2026-10-07
6
+
7
+ The floor is Ruby 3.4. A project on Ruby 3.3 stays on 0.1.1. See
8
+ [ADR 0017](docs/adr/0017-raise-the-ruby-floor-to-3-4.md).
9
+
10
+ Each platform gem carries `libhegel` 0.45.0, up from 0.32.5. The engine
11
+ changed its C interface in several places between the two, and these
12
+ bindings follow each change.
13
+
14
+ ### Changes a caller has to make
15
+
16
+ `Hegel.test` no longer takes `stateful_step_count:`. Pass `step_count:` to
17
+ `Hegel::Stateful.run` instead; it defaults to 50, as hegel-rust, hegel-go,
18
+ and hegel-java do. The engine no longer has a default of its own.
19
+
20
+ An invariant no longer runs after every rule. It runs before the first
21
+ rule and after the last, and between rules when the engine samples it,
22
+ about once in a test case that runs every step. Declare it with
23
+ `invariant :name, always_run: true` to keep the check after every rule.
24
+ See [ADR 0021](docs/adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md).
25
+
26
+ ### Failure reports
27
+
28
+ The report comes from the engine's own final replay of each failure, which
29
+ it runs inside the loop before the run ends. The body is no longer run
30
+ again after the run, so a failing run calls it once fewer per failure. A
31
+ `tc.note` block runs on each case the engine stamps for capture, which can
32
+ be more than one per failure. See
33
+ [ADR 0019](docs/adr/0019-report-failures-from-the-cases-the-engine-stamps.md).
34
+
35
+ A body that fails only some of the time is now reported as an ordinary
36
+ failure, with a `note:` line from the engine that says how often it
37
+ reproduced. Before, such a body ended the run in `Hegel::Error`. A failure the engine could not confirm comes
38
+ without a blob, and its report leaves out the reproduction line.
39
+
40
+ `reproduce_failure:` now replays its blob as a run, until a replay fails,
41
+ within the engine's own budget. A blob from a nondeterministic failure
42
+ reproduces this way too. When no replay fails, `Hegel.test` raises
43
+ `Hegel::Error` and names both causes: a fixed bug, or a body whose failure
44
+ did not recur.
45
+
46
+ A body that rejects its input with `tc.assume` or `tc.reject` before it
47
+ draws anything now raises `Hegel::Error` ("Unsatisfiable"). Before, the
48
+ run stopped after that one case and passed.
49
+
50
+ ### Settings
51
+
52
+ A keyword left `nil` now takes its value from the engine's settings
53
+ profile: libhegel's default, a `hegel.toml`, or a `HEGEL_*` environment
54
+ variable such as `HEGEL_TEST_CASES`. On a CI server the engine picks its
55
+ `ci` profile, which derandomizes the run and turns the example database
56
+ off, so a run with `database_key:` and no `database:` stores nothing there.
57
+ A keyword you pass still wins. See
58
+ [ADR 0023](docs/adr/0023-leave-unset-settings-to-the-engines-profile.md).
59
+
60
+ ### Generated values
61
+
62
+ The engine draws integers and floats from new distributions, and shrinks
63
+ further in several cases, so a seeded run draws different values
64
+ than it did on 0.32.5. `from_regex` follows Python's `re` more closely
65
+ under `(?i)` and `(?a)`, and its pattern may now hold a NUL character.
66
+
67
+ `times` and `datetimes` keep microsecond precision. The engine now draws
68
+ nanoseconds, and the binding rounds each one down. See
69
+ [ADR 0022](docs/adr/0022-keep-microsecond-times-over-a-nanosecond-engine.md).
70
+
71
+ A `datetimes` bound with a fraction of a microsecond could produce a value
72
+ outside it, because only the bound's whole microseconds reached the
73
+ engine. The lower bound now rounds up and the upper rounds down.
74
+
75
+ ### Other changes
76
+
77
+ `HEGEL_LIBHEGEL_PATH` pointing at an engine that lacks a function these
78
+ bindings call raises `Hegel::Error` that names the function. Before, FFI
79
+ raised a `TypeError` that named nothing.
80
+
81
+ `rake libhegel:fetch` downloads from hegel-rust's `libhegel-v<version>`
82
+ release tags, which hegel-rust uses for engine releases since 0.42.1.
83
+
84
+ ## [0.1.1] - 2026-08-31
85
+
86
+ The reference that ships inside the gem told a reader to install `hegeltest`
87
+ from git and to point `HEGEL_LIBHEGEL_PATH` at a local build. 0.1.0 shipped
88
+ that text, and 0.1.0 needs neither. A reader who followed it spent their first
89
+ minutes on a checkout and an engine build. The reference now says what the
90
+ README says.
91
+
92
+ A drawn value written inside a larger expression named itself after the
93
+ assignment target, so `term_start_date = Date.new(tc.draw(...), 2, 29)`
94
+ reported `term_start_date` beside a year. Such a draw now takes the generic
95
+ name, and `label:` names it when a name is wanted. See
96
+ [ADR 0014](docs/adr/0014-name-a-drawn-value-only-when-the-draw-is-the-whole-assigned-value.md).
97
+
98
+ The settings table names 100 as libhegel's own default for `test_cases`. The
99
+ README shows how to scope `Hegel::Syntax::Methods` to tagged example groups in
100
+ a suite that already exists, and how to share a group of draws through a plain
101
+ Ruby method. The `text` section says which generators cover the character-set
102
+ options it does not take, measured against libhegel 0.32.5.
103
+
3
104
  ## [0.1.0] - 2026-08-20
4
105
 
5
106
  The first release.
@@ -19,4 +120,6 @@ The engine is called through the `ffi` gem. Each platform gem carries the
19
120
  matching `libhegel` 0.32.5 build. The platform-independent gem carries none,
20
121
  so a run on it needs `HEGEL_LIBHEGEL_PATH` pointing at a local build.
21
122
 
123
+ [0.2.0]: https://github.com/meganemura/hegel-ruby/releases/tag/v0.2.0
124
+ [0.1.1]: https://github.com/meganemura/hegel-ruby/releases/tag/v0.1.1
22
125
  [0.1.0]: https://github.com/meganemura/hegel-ruby/releases/tag/v0.1.0
data/README.md CHANGED
@@ -31,8 +31,7 @@ same process. There is no server and no Python dependency.
31
31
 
32
32
  ## Status
33
33
 
34
- **The first release is prepared, and the version is `0.1.0`.** Everything
35
- described below runs: Hegel finds a counterexample, shrinks it,
34
+ **Released as `hegeltest` 0.1.0.** Hegel finds a counterexample, shrinks it,
36
35
  names the values it drew, and re-raises your own exception.
37
36
 
38
37
  Work ran in three stages, and all three are done:
@@ -44,26 +43,34 @@ Work ran in three stages, and all three are done:
44
43
  3. **The advanced features**: the example database, targeted testing,
45
44
  stateful testing, phases, and health checks.
46
45
 
47
- **Nothing is published to RubyGems yet.** The packaging is built:
48
- `rake platform_gems:build` produces one gem per platform, each carrying the
49
- matching `libhegel` build. The arm64 macOS gem is verified to find that
50
- engine inside itself and run a property with no `HEGEL_LIBHEGEL_PATH` set.
51
- A `v*` tag publishes all six gems from CI. The Hegel maintainers permit a
52
- third-party project to redistribute their release binaries
46
+ Install it the way you install any gem:
47
+
48
+ ```ruby
49
+ # Gemfile
50
+ gem "hegeltest"
51
+ ```
52
+
53
+ Each platform gem carries the matching `libhegel` build, so an install
54
+ compiles nothing and needs no engine on the side. The Hegel maintainers
55
+ permit a third-party project to redistribute their release binaries
53
56
  ([hegeldev/hegel-rust#411]).
54
57
 
55
- So running this today means cloning the repository:
58
+ A project that already has a `Gemfile.lock` needs one more step. Bundler keeps
59
+ the platform list the lock already holds. It adds only the platform of the
60
+ machine that runs `bundle install`. A lock therefore names only the platforms
61
+ of the machines that have run it. A Linux CI job cannot find the gem when the
62
+ lock names no Linux platform. Add each platform your CI uses:
56
63
 
57
64
  ```bash
58
- bin/setup
59
- bundle exec rake libhegel:fetch # downloads the pinned build, checked against its SHA-256
60
- bundle exec rake
65
+ bundle lock --add-platform x86_64-linux
61
66
  ```
62
67
 
63
- Every push to main runs the suite on Ruby 3.3, 3.4, and 4.0, across Linux x64
64
- and arm64, macOS arm64, and Windows x64 and arm64. Windows arm64 starts at
65
- Ruby 3.4, the oldest Ruby that RubyInstaller publishes an arm64 build for. A
66
- pull request runs Linux x64 alone.
68
+ A project with no lock yet reads the `PLATFORMS` section of the lock its first
69
+ `bundle install` writes, and adds whatever its CI needs the same way.
70
+
71
+ Every push to main runs the suite on Ruby 3.4 and 4.0, across Linux x64 and
72
+ arm64, macOS arm64, and Windows x64 and arm64. A pull request runs Linux x64
73
+ alone.
67
74
 
68
75
  [hegeldev/hegel-rust#411]: https://github.com/hegeldev/hegel-rust/issues/411
69
76
 
@@ -75,17 +82,17 @@ pull request runs Linux x64 alone.
75
82
  | Require path | `require "hegel"` (`require "hegeltest"` also works) |
76
83
  | Namespace | `Hegel` |
77
84
  | Binding | the `ffi` gem, which publishes a prebuilt binary for every platform above |
78
- | Ruby | 3.3, 3.4, and 4.0 |
85
+ | Ruby | 3.4 and 4.0 |
79
86
  | Platforms | Linux amd64/arm64, macOS arm64, Windows amd64/arm64 |
80
87
  | Engine delivery | One prebuilt `libhegel` per platform-specific gem, built by `rake platform_gems:build` |
81
88
 
82
- `HEGEL_LIBHEGEL_PATH` will override the bundled engine with a local build.
89
+ `HEGEL_LIBHEGEL_PATH` overrides the bundled engine with a local build.
83
90
 
84
91
  The gem name follows the Rust implementation, whose published crate is
85
92
  `hegeltest` and whose library is `hegel`. The name `hegel` on RubyGems.org
86
93
  stays free for whoever publishes an official Ruby implementation.
87
94
 
88
- macOS on Intel has no published `libhegel` artifact, so that platform will need
95
+ macOS on Intel has no published `libhegel` artifact, so that platform needs
89
96
  `HEGEL_LIBHEGEL_PATH` and a local build.
90
97
 
91
98
  ## Quickstart
@@ -135,6 +142,33 @@ Including `Hegel::Syntax::Methods` makes the generators available without a
135
142
  prefix, the way FactoryBot makes `create` available. The same generators stay
136
143
  reachable as `Hegel::Generators.arrays(...)` without the include.
137
144
 
145
+ An existing RSpec suite can limit the 25 generator names to tagged example
146
+ groups. This keeps the names out of other groups that have their own helpers:
147
+
148
+ ```ruby
149
+ config.include Hegel::Syntax::Methods, :hegel
150
+ ```
151
+
152
+ A new project can include the methods in every group, as the Quickstart does.
153
+
154
+ Hegel draws values imperatively, so an ordinary Ruby method can share a group
155
+ of draws between properties. Give the method parameter a descriptive name,
156
+ which the default RuboCop naming rule accepts where a block parameter `tc`
157
+ needs no change:
158
+
159
+ ```ruby
160
+ def draw_sorted_pair(test_case)
161
+ low = test_case.draw(integers(min_value: 0, max_value: 100))
162
+ high = test_case.draw(integers(min_value: low, max_value: 200))
163
+ [low, high]
164
+ end
165
+
166
+ Hegel.test do |tc|
167
+ low, high = draw_sorted_pair(tc)
168
+ expect(low).to be <= high
169
+ end
170
+ ```
171
+
138
172
  The generators are `arrays`, `binary`, `booleans`, `characters`, `composite`,
139
173
  `dates`, `datetimes`, `deferred`, `domains`, `emails`, `floats`, `from_regex`,
140
174
  `hashes`, `integers`, `ip_addresses`, `just`, `one_of`, `optional`,
@@ -181,7 +215,7 @@ Expected: [0, 0]
181
215
 
182
216
  Some bugs need a sequence of operations rather than one input. Declare the
183
217
  operations as rules on a `Hegel::StateMachine`, and Hegel picks which runs
184
- next, checks your invariants after each one, and shrinks a failure to the
218
+ next, checks your invariants along the way, and shrinks a failure to the
185
219
  shortest sequence that still shows it:
186
220
 
187
221
  ```ruby
@@ -216,12 +250,23 @@ something an earlier rule produced, such as freeing a handle that some
216
250
  `alloc` actually returned, put the value in a `Hegel::Stateful::Pool` and
217
251
  draw it back out.
218
252
 
253
+ Every invariant runs before the first rule and after the last. Between rules,
254
+ the engine samples each one, so that it runs about once in a test case
255
+ that runs every step.
256
+ Declare an invariant with `always_run: true` to check it after every rule
257
+ instead. `Hegel::Stateful.run(machine, tc, step_count: 50)` sets the most
258
+ rules one test case runs; 50 is the default.
259
+
219
260
  ### Shaping a run
220
261
 
221
262
  `Hegel.test` takes keywords for the rest: `test_cases`, `seed`,
222
- `derandomize`, `verbosity`, `phases`, `suppress_health_check`,
223
- `report_multiple_failures`, and `stateful_step_count`. Each one left unset
224
- means the engine's own default.
263
+ `derandomize`, `verbosity`, `phases`, `suppress_health_check`, and
264
+ `report_multiple_failures`. `report_multiple_failures` defaults to `false`.
265
+ Each of the others left unset takes its value from the engine's settings
266
+ profile. That is libhegel's own default, unless a
267
+ `hegel.toml` or a `HEGEL_*` environment variable says otherwise. On a CI
268
+ server the engine picks its `ci` profile, which derandomizes the run and
269
+ turns the example database off.
225
270
 
226
271
  Two more turn on libhegel's example database, which stores a failing case and
227
272
  replays it first next time. `database_key` is the switch and `database`
@@ -241,9 +286,10 @@ Inside a test, `tc.note` records a message the failure report prints, and
241
286
  ## Development
242
287
 
243
288
  ```bash
244
- bin/setup # install dependencies
245
- bundle exec rake # run the tests and the linter
246
- bin/console # open an interactive prompt
289
+ bin/setup # install dependencies
290
+ bundle exec rake libhegel:fetch # download the pinned engine, checked against its SHA-256
291
+ bundle exec rake # run the tests and the linter
292
+ bin/console # open an interactive prompt
247
293
  ```
248
294
 
249
295
  `just` recipes call the same Rake tasks, so `just test` and `just lint` work
data/Rakefile CHANGED
@@ -17,3 +17,84 @@ task :coverage do
17
17
  end
18
18
 
19
19
  task default: %i[coverage standard]
20
+
21
+ # Mutation testing with mutineer, over lib/. Four settings make the suite
22
+ # run under it; ADR 0016 records why:
23
+ # - RUBYOPT gets -Itest, since mutineer puts only lib/ on the load path and
24
+ # every test file starts with `require "test_helper"`.
25
+ # - Each test file gets its own --test flag. The flag takes one file, and a
26
+ # second path after it becomes a source to mutate.
27
+ # - test_conformance.rb stays out. Its capture_subprocess_io reopens
28
+ # $stdout, and mutineer replaces $stdout with a StringIO, so the file fails
29
+ # before any mutant runs.
30
+ # - --strategy redefine reloads only the mutated method. The default,
31
+ # reload, loads the whole mutated file by a relative path, which breaks
32
+ # Hegel::Runner.origin_for and killed three mutants the suite does not
33
+ # catch.
34
+ # MUTINEER_ARGS passes more flags, for example
35
+ # MUTINEER_ARGS="--since origin/main" to mutate only the lines changed since
36
+ # origin/main.
37
+ #
38
+ # `rake mutation` fails on a survivor that the baseline does not list
39
+ # (ADR 0018). `rake mutation:baseline` rewrites the baseline from a full run.
40
+ MUTATION_BASELINE = "test/mutation-baseline.json"
41
+
42
+ def run_mutineer(*extra)
43
+ require "shellwords"
44
+ tests = Dir["test/*.rb", "test/hegel/*.rb"].sort -
45
+ %w[test/test_helper.rb test/hegel/test_conformance.rb]
46
+ args = ["lib", "--strategy", "redefine", *tests.flat_map { |t| ["--test", t] }, *extra, *Shellwords.split(ENV.fetch("MUTINEER_ARGS", ""))]
47
+ rubyopt = ["-Itest", ENV["RUBYOPT"]].compact.join(" ")
48
+ # mutineer keeps its cache here, and refuses an --output path whose
49
+ # directory does not exist yet.
50
+ mkdir_p ".mutineer", verbose: false
51
+ sh({"RUBYOPT" => rubyopt}, "bundle", "exec", "mutineer", "run", *args)
52
+ end
53
+
54
+ # mutineer writes its JSON report on one line, so a change to it reads as one
55
+ # changed line in a pull request. mutineer's --baseline reads only
56
+ # schema_version, summary.score, summary.scoped, and each survivor's id, so
57
+ # the committed file keeps those and writes one survivor per line, sorted.
58
+ # It leaves out line numbers: an edit above a survivor would move every
59
+ # line after it, and the id does not depend on the position. "change" holds
60
+ # the mutated line before and after, so a reviewer can read the survivor.
61
+ def write_mutation_baseline(report_path, baseline_path)
62
+ require "json"
63
+ report = JSON.parse(File.read(report_path))
64
+ # mutineer refuses a --since run as a baseline, since it covers only the
65
+ # changed lines.
66
+ abort "#{report_path} comes from a --since run; rewrite the baseline from a full run" if report.dig("summary", "scoped")
67
+ rows = report.fetch("survivors").map do |survivor|
68
+ change = survivor.fetch("diff").lines.grep(/^[-+] /).map { |line| line.chomp.sub(/^([-+]) +/, '\1 ') }.join(" | ")
69
+ {"id" => survivor.fetch("id"), "file" => survivor.fetch("file"), "subject" => survivor.fetch("subject"),
70
+ "operator" => survivor.fetch("operator"), "change" => change}
71
+ end
72
+ rows.sort_by! { |row| row.values_at("file", "subject", "id") }
73
+ text = +"{\n"
74
+ text << %( "schema_version": #{report.fetch("schema_version").to_json},\n)
75
+ text << %( "summary": #{report.fetch("summary").slice("score", "scoped").to_json},\n)
76
+ text << %( "survivors": [\n#{rows.map { |row| " #{row.to_json}" }.join(",\n")}\n ]\n}\n)
77
+ # The baseline is committed to a public repository, so a path from this
78
+ # machine must not reach it.
79
+ leak = [Dir.pwd, Dir.home].find { |path| text.include?(path) }
80
+ abort "#{baseline_path} would contain #{leak}; not written" if leak
81
+ File.write(baseline_path, text)
82
+ end
83
+
84
+ desc "Run mutation testing over lib/, and fail on a survivor that #{MUTATION_BASELINE} does not list"
85
+ task :mutation do
86
+ # --baseline also fails on a lower score. A mutant that times out on a slow
87
+ # machine leaves the score and can lower it by a fraction of a point, and
88
+ # it can never add a survivor. The epsilon absorbs that, and the survivor
89
+ # ids stay the gate.
90
+ run_mutineer("--baseline", MUTATION_BASELINE, "--baseline-epsilon", "1")
91
+ end
92
+
93
+ namespace :mutation do
94
+ desc "Rewrite #{MUTATION_BASELINE} from a full mutation testing run over lib/"
95
+ task :baseline do
96
+ report = ".mutineer/baseline-run.json"
97
+ run_mutineer("--format", "json", "--output", report)
98
+ write_mutation_baseline(report, MUTATION_BASELINE)
99
+ end
100
+ end
data/docs/README.md CHANGED
@@ -20,6 +20,16 @@ An index of the design records for `hegel-ruby`.
20
20
  - [0011: Let the test case own every pool drawn from it](adr/0011-let-the-test-case-own-every-pool-drawn-from-it.md)
21
21
  - [0012: Build a failure origin from the caller's own frame](adr/0012-build-a-failure-origin-from-the-callers-own-frame.md)
22
22
  - [0013: Bind libhegel through the ffi gem](adr/0013-bind-libhegel-through-the-ffi-gem.md)
23
+ - [0014: Name a drawn value only when the draw is the whole assigned value](adr/0014-name-a-drawn-value-only-when-the-draw-is-the-whole-assigned-value.md)
24
+ - [0015: Follow the hegeldev interface, and take changes upstream first](adr/0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md)
25
+ - [0016: Run mutation testing with mutineer](adr/0016-run-mutation-testing-with-mutineer.md)
26
+ - [0017: Raise the Ruby floor to 3.4](adr/0017-raise-the-ruby-floor-to-3-4.md)
27
+ - [0018: Gate mutation testing on a committed baseline](adr/0018-gate-mutation-testing-on-a-committed-baseline.md)
28
+ - [0019: Report failures from the cases the engine stamps](adr/0019-report-failures-from-the-cases-the-engine-stamps.md)
29
+ - [0020: Derive span labels from generator names](adr/0020-derive-span-labels-from-generator-names.md)
30
+ - [0021: Run a state machine in rounds, with its own step count](adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md)
31
+ - [0022: Keep microsecond times over a nanosecond engine](adr/0022-keep-microsecond-times-over-a-nanosecond-engine.md)
32
+ - [0023: Leave unset settings to the engine's profile](adr/0023-leave-unset-settings-to-the-engines-profile.md)
23
33
 
24
34
  A new decision gets a new record. A changed decision supersedes the old
25
35
  record instead of editing it, so the history stays readable.
@@ -0,0 +1,74 @@
1
+ # 0014: Name a drawn value only when the draw is the whole assigned value
2
+
3
+ ## Status
4
+
5
+ Accepted.
6
+
7
+ ## Context
8
+
9
+ [ADR 0005](0005-name-drawn-values-from-the-callers-source-with-prism.md)
10
+ recovers a drawn value's name by parsing the caller's source and reading the
11
+ assignment that covers the draw's line. Its Consequences section lists three
12
+ cases it had not checked: a heredoc, several draws on one line, and a method
13
+ chain.
14
+
15
+ A first use of the gem outside this repository reached one of them. The
16
+ caller wrote a draw inside a larger expression:
17
+
18
+ ```ruby
19
+ term_start_date = Date.new(tc.draw(integers(min_value: 1904, max_value: 1904)), 2, 29)
20
+ ```
21
+
22
+ Measured against libhegel 0.32.5, the failure report printed:
23
+
24
+ ```
25
+ term_start_date = 1904
26
+ ```
27
+
28
+ The name says "a date". The value is a year. The reader misread that report
29
+ twice before adding a `label:`.
30
+
31
+ `Hegel::DrawName` already carries the rule this breaks. Its own
32
+ documentation states it: a wrong name misdirects a reader of the failure
33
+ report more than a missing one does, so an ambiguous line returns nil rather
34
+ than choosing between candidates. The module applied that rule to two
35
+ assignments written side by side, and applied the opposite rule to a draw
36
+ buried in one assignment's value.
37
+
38
+ A draw pulled out into a plain method is a separate case, and it already
39
+ worked. Measured on the same engine, a helper whose body reads
40
+ `year = tc.draw(integers(min_value: 1900, max_value: 1910))` reported
41
+ `year = 1900`. `Hegel::TestCase::DRAW_CALLER_DEPTH` lands on the line that
42
+ called the draw, which is the helper's own line, and the name there is the
43
+ one the caller wrote.
44
+
45
+ ## Decision
46
+
47
+ Recover a name only when the assignment's value is the draw call itself: the
48
+ value node is a `Prism::CallNode` whose name is one of the methods that reach
49
+ `Hegel::TestCase#record_draw`. Any other value node returns nil, and
50
+ `Hegel::TestCase#name_for` falls back to the generic name.
51
+
52
+ The three method names stay in `Hegel::TestCase`, which defines them, and
53
+ `Hegel::DrawName.for` takes them as an argument. A fourth draw method then
54
+ has one place to be added.
55
+
56
+ ## Consequences
57
+
58
+ A draw written inside a larger expression reports the generic name and its
59
+ value. The reader loses a name and keeps a value that agrees with it. A
60
+ caller who wants the name back passes `label:`, which
61
+ [ADR 0005](0005-name-drawn-values-from-the-callers-source-with-prism.md)
62
+ already gives precedence over the recovered name.
63
+
64
+ A method chain answers the question ADR 0005 left open, and answers it the
65
+ same way: `xs = tc.draw_integer(0, 10).to_s` reports the generic name,
66
+ because `xs` holds a String and the drawn value is an Integer.
67
+
68
+ A draw written inside another draw's arguments shares the enclosing
69
+ assignment's name, because the enclosing assignment's value is a draw call
70
+ and one line carries both draws. `label:` separates them.
71
+
72
+ `Hegel::DrawName` now answers a narrower question than its name suggests on
73
+ its own, so its documentation states the question it answers and the reason
74
+ for the narrowing.
@@ -0,0 +1,101 @@
1
+ # 0015: Follow the hegeldev interface, and take changes upstream first
2
+
3
+ ## Status
4
+
5
+ Accepted.
6
+
7
+ ## Context
8
+
9
+ This gem is one binding of libhegel among several. The Hegel project
10
+ publishes implementations for Rust, Go, C++, TypeScript, Java, and OCaml,
11
+ and each one drives the same engine. A person who writes a property in one
12
+ language and then writes the same property in another expects the same call
13
+ to mean the same thing.
14
+
15
+ `CLAUDE.md` already names hegel-rust as the authority on meaning: what a
16
+ call does, what an error code means, how a value shrinks. That rule settles
17
+ disputes about behaviour. It does not say what to do when this gem's public
18
+ surface is shaped differently from every other binding while each shape
19
+ behaves as documented.
20
+
21
+ `floats` is that case. Three implementations write the same rule for its
22
+ defaults:
23
+
24
+ - hegel-rust, `src/generators/numeric.rs:315`:
25
+ `allow_nan.unwrap_or(!has_min && !has_max)`, and
26
+ `allow_infinity.unwrap_or(!has_min || !has_max)`
27
+ - hegel-go, `primitives.go:136`: the same two expressions, resolved through
28
+ a nil pointer check
29
+ - hegel-typescript, `src/generators/numeric.ts:126`: the same two
30
+ expressions, resolved through `??`
31
+
32
+ This gem instead fixed both to `false`, whatever the bounds. The comment on
33
+ `Hegel::Generators::FloatGenerator` carried the reasoning: a caller who
34
+ never asks for NaN never has to reason about it, a keyword can be added
35
+ later without breaking anyone, and a default flipped from `true` to `false`
36
+ would break someone.
37
+
38
+ That reasoning weighs this gem's own callers. It leaves out what the
39
+ difference costs. `floats` with no bounds is the ordinary way to ask for a
40
+ double. Under the sibling rule that draw yields NaN and both infinities;
41
+ here it yields neither. A property that passes here can fail the moment
42
+ somebody ports it to Rust, and the Ruby test was the weaker one all along,
43
+ because it never drew the values that break floating-point code. The
44
+ difference is invisible at the call site: the same six-word call means two
45
+ things.
46
+
47
+ The skill this repository ships states the difference as a gotcha, so a
48
+ reader who finds it is warned. A reader who does not find it gets a green
49
+ run and a false conclusion.
50
+
51
+ ## Decision
52
+
53
+ **The public surface follows the interface the hegeldev implementations
54
+ share.** Names, arguments, defaults, and the meaning of each. A feature
55
+ that this gem alone has, and a feature that this gem alone lacks, are both
56
+ divergences.
57
+
58
+ **A change to that interface starts as a proposal upstream.** Do not
59
+ implement it here first and wait for the others to follow. When a proposal
60
+ is accepted, this gem takes the name and the semantics that were settled on,
61
+ rather than choosing again.
62
+
63
+ hegel-rust stays the authority. Where the sibling implementations disagree
64
+ with each other, hegel-rust decides, as it already does for behaviour.
65
+
66
+ Three things stay outside this rule, because they are mechanism rather than
67
+ interface. How the engine is reached (the `ffi` gem, ADR 0013). What Ruby
68
+ alone can offer a reader, such as recovering a drawn value's name from the
69
+ caller's source (ADR 0005, ADR 0014). Where a Ruby hazard forces a
70
+ different construction, such as the control exceptions that must inherit
71
+ from `Exception`.
72
+
73
+ This record replaces the reasoning in the comment on
74
+ `Hegel::Generators::FloatGenerator`, which decided the same question for
75
+ `floats` alone and decided it the other way.
76
+
77
+ ## Consequences
78
+
79
+ `floats` changes its defaults to the shared rule, and gains the two checks
80
+ that come with it: `allow_nan: true` with either bound, and
81
+ `allow_infinity: true` with both bounds, each raise `Hegel::Error` at draw
82
+ time. hegel-rust and hegel-go both write those checks; the messages here
83
+ carry the same substring as theirs.
84
+
85
+ This breaks callers. A property that draws unbounded `floats` and passes
86
+ today will see NaN and infinity after the change, and code that does not
87
+ handle them will fail. That failure is the point: the property was not
88
+ testing what it appeared to test. A caller who wants the old behaviour
89
+ passes `allow_nan: false, allow_infinity: false`, and the call then says
90
+ what it does.
91
+
92
+ Growing the surface gets slower. A generator or an option that would be
93
+ useful here has to be worth proposing upstream before it can ship here, and
94
+ a proposal can be rejected. The gem trades the speed of deciding alone for
95
+ callers who can move between languages.
96
+
97
+ The comparison has to be made rather than assumed. Adding an option means
98
+ reading how the other implementations spell it and what they default it to.
99
+ `hegel-rust/src/generators/` holds the generator surface, and hegel-go and
100
+ hegel-typescript hold the two closest readings of it for a language with
101
+ keyword-style options.
@@ -0,0 +1,87 @@
1
+ # 0016: Run mutation testing with mutineer
2
+
3
+ ## Status
4
+
5
+ Accepted. [ADR 0017](0017-raise-the-ruby-floor-to-3-4.md) supersedes the
6
+ Gemfile group `mutation` in the Consequences.
7
+
8
+ ## Context
9
+
10
+ The suite holds line and branch coverage at 100%. Coverage shows that a line
11
+ ran. It does not show that a test would notice a change to that line.
12
+ Mutation testing measures that: it changes the source in small ways, one at
13
+ a time, and runs the tests against each change.
14
+
15
+ mutineer is a mutation testing tool for Ruby. It parses the source with
16
+ Prism and has no runtime dependency. It runs Minitest, and it selects, for
17
+ each mutant, the test files that cover the mutated line.
18
+
19
+ A first run over `lib/` met four problems, all measured on mutineer 1.0.2:
20
+
21
+ - mutineer puts only `lib/` on the load path. Every test file starts with
22
+ `require "test_helper"`, so each one failed to load. The run then reported
23
+ every mutant as uncovered, with a score of N/A and exit status 0. The load
24
+ error appears only with `--verbose`.
25
+ - mutineer pairs a source with `test/<name>_test.rb` only. This suite
26
+ names its tests `test_<name>.rb`, so each test file must be named with
27
+ `--test`. The flag takes one file, and it repeats. A first run gave one
28
+ flag several files, and the files after the first became sources to
29
+ mutate, so the score read 0.7%.
30
+ - mutineer replaces `$stdout` with a `StringIO` before it runs the tests.
31
+ Minitest's `capture_subprocess_io` reopens `$stdout` on a file, so
32
+ `test/hegel/test_conformance.rb` failed before any mutant ran.
33
+ - mutineer runs every mutant's tests in this one working directory. A
34
+ mutant that wrote `./.hegel` left the directory in place, and the
35
+ teardown check in five test classes then failed the tests of every later
36
+ mutant too.
37
+
38
+ ## Decision
39
+
40
+ `bundle exec rake mutation` (and `just mutation`) runs mutineer 1.0.2 over
41
+ `lib/`, with four settings:
42
+
43
+ 1. `RUBYOPT` gets `-Itest`, so `require "test_helper"` resolves.
44
+ 2. Each test file gets its own `--test` flag.
45
+ 3. `test/hegel/test_conformance.rb` stays out of the run. It still runs in
46
+ `rake test`.
47
+ 4. `--strategy redefine`. mutineer's default, `reload`, loads the whole
48
+ mutated file from a tempfile by a relative path, so its backtrace lines
49
+ show a relative path. `Hegel::Runner.origin_for` then takes a line in
50
+ `lib/` for the caller's, and its test fails for every mutant of
51
+ `lib/hegel/test_case.rb`. That alone killed three mutants that the
52
+ suite does not catch: one of them, applied by hand, left all 377 tests
53
+ passing. `redefine` loads only the mutated method.
54
+
55
+ The five teardowns share `HegelDirectoryGuard#refute_new_hegel_directory`
56
+ (`test/test_helper.rb`). When a test leaves a `./.hegel` it did not find at
57
+ the start, the guard removes the directory, then fails the test. The mutant
58
+ that wrote it is still killed, and later mutants no longer fail on the
59
+ leftover. A `./.hegel` that existed before the test stays in place.
60
+
61
+ `mutineer` is a development dependency, pinned exact and never required at
62
+ load time. It needs Ruby 3.4 or later, and the gem supports 3.3, so it sits
63
+ in the Gemfile group `mutation`. CI leaves that group out on every job, and
64
+ a 3.3 install leaves it out with `bundle config set --local without
65
+ mutation`. Mutation testing then runs on 3.4 or later only.
66
+ `.mutineer/` (the cache and reports) and `.hegel/` are ignored by git.
67
+
68
+ ## Consequences
69
+
70
+ The run over `lib/` covers 706 mutants in about a minute: 539 killed, 153
71
+ survived, 8 without coverage, 6 without a verdict, a score of 77.9%. Measured
72
+ with the reload strategy, the guard alone turned 10 former kills into
73
+ survivors, which were false kills before.
74
+
75
+ The survivors are a list of tests to add. A later change works through them
76
+ by file.
77
+
78
+ Settings 1, 3, and 4 work around mutineer itself, and setting 2 follows
79
+ from its pairing rule. Each is reported: the load path as
80
+ [davidteren/mutineer#119](https://github.com/davidteren/mutineer/issues/119),
81
+ the pairing rule as
82
+ [davidteren/mutineer#120](https://github.com/davidteren/mutineer/issues/120),
83
+ the `StringIO` as
84
+ [davidteren/mutineer#121](https://github.com/davidteren/mutineer/issues/121),
85
+ and the relative path as
86
+ [davidteren/mutineer#123](https://github.com/davidteren/mutineer/issues/123).
87
+ When a mutineer release fixes one, drop the setting that it replaces.