hegeltest 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +103 -0
- data/README.md +72 -26
- data/Rakefile +81 -0
- data/docs/README.md +10 -0
- data/docs/adr/0014-name-a-drawn-value-only-when-the-draw-is-the-whole-assigned-value.md +74 -0
- data/docs/adr/0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md +101 -0
- data/docs/adr/0016-run-mutation-testing-with-mutineer.md +87 -0
- data/docs/adr/0017-raise-the-ruby-floor-to-3-4.md +43 -0
- data/docs/adr/0018-gate-mutation-testing-on-a-committed-baseline.md +74 -0
- data/docs/adr/0019-report-failures-from-the-cases-the-engine-stamps.md +70 -0
- data/docs/adr/0020-derive-span-labels-from-generator-names.md +64 -0
- data/docs/adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md +59 -0
- data/docs/adr/0022-keep-microsecond-times-over-a-nanosecond-engine.md +45 -0
- data/docs/adr/0023-leave-unset-settings-to-the-engines-profile.md +50 -0
- data/docs/architecture.md +11 -6
- data/lib/hegel/draw_name.rb +24 -8
- data/lib/hegel/generator.rb +50 -9
- data/lib/hegel/generators.rb +134 -102
- data/lib/hegel/lib_hegel/real.rb +193 -94
- data/lib/hegel/lib_hegel.rb +30 -56
- data/lib/hegel/libhegel_version.rb +1 -1
- data/lib/hegel/report.rb +17 -8
- data/lib/hegel/runner.rb +153 -201
- data/lib/hegel/settings.rb +9 -14
- data/lib/hegel/state_machine.rb +15 -8
- data/lib/hegel/stateful/pool.rb +0 -2
- data/lib/hegel/stateful.rb +64 -36
- data/lib/hegel/syntax/methods.rb +3 -2
- data/lib/hegel/test_case.rb +29 -9
- data/lib/hegel/version.rb +1 -1
- data/lib/hegel.rb +11 -17
- data/lib/tasks/libhegel.rake +9 -2
- data/sig/hegel.rbs +60 -61
- data/skills/hegel-ruby/references/ruby/reference.md +147 -52
- metadata +12 -2
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# 0017: Raise the Ruby floor to 3.4
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted. Supersedes the Gemfile group `mutation` in the Consequences of
|
|
6
|
+
[ADR 0016](0016-run-mutation-testing-with-mutineer.md).
|
|
7
|
+
|
|
8
|
+
## Context
|
|
9
|
+
|
|
10
|
+
The gem supported Ruby 3.3, 3.4, and 4.0. Ruby 3.3 has had security
|
|
11
|
+
maintenance only since 2026-04-01, and its end of life is expected on
|
|
12
|
+
2027-03-31.
|
|
13
|
+
|
|
14
|
+
mutineer 1.0.2, the mutation testing tool of ADR 0016, needs Ruby 3.4 or
|
|
15
|
+
later. With mutineer in the Gemfile, `bundle install` failed on every 3.3 job
|
|
16
|
+
in CI. The first fix put mutineer in a Gemfile group `mutation`. Every CI job
|
|
17
|
+
left that group out, and a developer on 3.3 had to leave it out by hand. That
|
|
18
|
+
fix gave the repository two install paths, and CI never installed a tool that
|
|
19
|
+
the repository runs.
|
|
20
|
+
|
|
21
|
+
## Decision
|
|
22
|
+
|
|
23
|
+
The floor is Ruby 3.4:
|
|
24
|
+
|
|
25
|
+
- `required_ruby_version` is `>= 3.4.0`.
|
|
26
|
+
- CI runs Ruby 3.4 and 4.0.
|
|
27
|
+
- `.standard.yml` targets Ruby 3.4.
|
|
28
|
+
- mutineer is back in the default Gemfile group, and CI installs it.
|
|
29
|
+
|
|
30
|
+
Keeping 3.3 with the group was refused, because of the two install paths above.
|
|
31
|
+
|
|
32
|
+
## Consequences
|
|
33
|
+
|
|
34
|
+
A project on Ruby 3.3 stays on 0.1.1, the last release that installs there.
|
|
35
|
+
The next release says so in the changelog.
|
|
36
|
+
|
|
37
|
+
Every CI job installs the same bundle, and a developer needs one Ruby for the
|
|
38
|
+
gem and its tools. The matrix loses its one exclusion: RubyInstaller has an
|
|
39
|
+
arm64 Windows build for every Ruby in it.
|
|
40
|
+
|
|
41
|
+
ADR 0001, ADR 0005, and ADR 0008 name 3.3 as the floor, which it was when
|
|
42
|
+
they were written. ADR 0005's reason still holds on 3.4: Ruby 3.4 also ships
|
|
43
|
+
Prism as a default gem.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# 0018: Gate mutation testing on a committed baseline
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted. It builds on
|
|
6
|
+
[ADR 0016](0016-run-mutation-testing-with-mutineer.md).
|
|
7
|
+
|
|
8
|
+
## Context
|
|
9
|
+
|
|
10
|
+
ADR 0016 runs mutineer over `lib/`: 706 mutants in about a minute, with 153
|
|
11
|
+
survivors. Nothing checked the result, so a change could add a survivor and
|
|
12
|
+
no one would see it.
|
|
13
|
+
|
|
14
|
+
mutineer has two ways to narrow what fails a run. `--since REF` mutates only
|
|
15
|
+
the lines changed since `REF`. `--baseline FILE` compares the run with an
|
|
16
|
+
earlier JSON report, and fails on a survivor whose stable id is not in the
|
|
17
|
+
report, or on a lower score. The id does not depend on the line number.
|
|
18
|
+
|
|
19
|
+
mutineer writes its JSON report on one line of about 55 KB. Committed as it
|
|
20
|
+
is, a change to the survivors reads as one changed line in a pull request.
|
|
21
|
+
|
|
22
|
+
## Decision
|
|
23
|
+
|
|
24
|
+
`test/mutation-baseline.json` holds the accepted survivors, and CI fails on
|
|
25
|
+
any other survivor:
|
|
26
|
+
|
|
27
|
+
- `rake mutation` runs over all of `lib/` with `--baseline` on that file.
|
|
28
|
+
`rake mutation:baseline` rewrites the file from a full run.
|
|
29
|
+
- The file keeps what `--baseline` reads: `schema_version`, `summary.score`,
|
|
30
|
+
`summary.scoped`, and each survivor's `id`. Each survivor is one line,
|
|
31
|
+
sorted by file, method, and id, with the mutated line before and after. It
|
|
32
|
+
leaves out line numbers.
|
|
33
|
+
- `--baseline-epsilon 1` absorbs a small score drop. A mutant that times out
|
|
34
|
+
on a slow machine leaves the score, and it can never add a survivor.
|
|
35
|
+
- The CI job `mutation` runs `rake mutation` once, on Linux with Ruby 4.0, for
|
|
36
|
+
every push and pull request.
|
|
37
|
+
- `rake mutation:baseline` refuses a `--since` run, which mutineer refuses as
|
|
38
|
+
a baseline too. It also refuses to write a file that contains the working
|
|
39
|
+
directory or the home directory, because the file is public.
|
|
40
|
+
|
|
41
|
+
A full run takes about a minute, so CI runs in full. `--since` stays for a
|
|
42
|
+
local run on a change in progress.
|
|
43
|
+
|
|
44
|
+
Two alternatives were refused:
|
|
45
|
+
|
|
46
|
+
- A baseline made on main by CI and passed to a pull request as an artifact.
|
|
47
|
+
It adds no file to the repository, but a reviewer cannot see a change to the
|
|
48
|
+
survivors, and the workflow needs to find the artifact of the right run.
|
|
49
|
+
- A gate on `--since` alone, which fails on any survivor on a changed line. An
|
|
50
|
+
edit to a line with an old survivor would then fail for a survivor that the
|
|
51
|
+
edit did not add.
|
|
52
|
+
|
|
53
|
+
## Consequences
|
|
54
|
+
|
|
55
|
+
A pull request that adds a survivor fails, until it adds a test or accepts
|
|
56
|
+
the survivor with `rake mutation:baseline`. The accepted survivor then shows
|
|
57
|
+
as one added line in the diff. A pull request that kills a survivor passes
|
|
58
|
+
without a change to the file, and the next rewrite shows the survivor as one
|
|
59
|
+
removed line.
|
|
60
|
+
|
|
61
|
+
A survivor that no test can kill, because the mutant behaves the same as the
|
|
62
|
+
original, gets a `# mutineer:disable-line <operator>` marker on its line,
|
|
63
|
+
with the reason in a comment on the line above. The judgment then moves
|
|
64
|
+
with the code through an edit or a move, which an id list in
|
|
65
|
+
`.mutineer.yml` does not do. The reason cannot follow the marker on the same
|
|
66
|
+
line, because mutineer reads every word after the marker as an operator
|
|
67
|
+
name. The survivor then leaves the baseline too.
|
|
68
|
+
|
|
69
|
+
A marker suppresses every mutant of its operator on its line. When a line
|
|
70
|
+
holds an equivalent mutant and a killed one of the same operator, a marker
|
|
71
|
+
would also hide the killed one. `Runner.classify` has three such lines,
|
|
72
|
+
and `TestCase#generate_boolean` has one. Those equivalent mutants go in
|
|
73
|
+
`.mutineer.yml` under `ignore:`, by their stable id. The reason stays in a
|
|
74
|
+
comment beside the code, and the comment names `.mutineer.yml`.
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# 0019: Report failures from the cases the engine stamps
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
Up to libhegel 0.32.5, a run ended with its failures shrunk, and the
|
|
10
|
+
binding built each report itself. After the run, it read each failure's
|
|
11
|
+
reproduction blob, rebuilt a test case from it with
|
|
12
|
+
`hegel_test_case_from_blob`, and ran the body against that case once more
|
|
13
|
+
while it recorded the drawn values.
|
|
14
|
+
|
|
15
|
+
libhegel 0.44.0 changed both halves. The engine now runs every failure it
|
|
16
|
+
reports once more before the run ends, inside the loop the binding drives.
|
|
17
|
+
It stamps that case, and a few others, through
|
|
18
|
+
`hegel_test_case_should_capture`. The header asks a binding to keep the
|
|
19
|
+
output of a stamped failing case under its origin, and to report each
|
|
20
|
+
failure from its freshest capture instead of replaying its blob after the
|
|
21
|
+
run. A failure can also arrive with no blob: under nondeterministic
|
|
22
|
+
handling, an unconfirmed failure has none, and it carries a caveat
|
|
23
|
+
(`hegel_failure_caveat`) that quotes the engine's replay evidence.
|
|
24
|
+
`hegel_run_start_blob` replays a blob as a run, and the header names it as
|
|
25
|
+
the call a reproduce-failure feature should use.
|
|
26
|
+
|
|
27
|
+
hegel-rust and hegel-java follow this design. hegel-go, hegel-cpp, and
|
|
28
|
+
hegel-typescript still replay the blob after the run; each pins an engine
|
|
29
|
+
from before 0.44.0, or has not moved its report yet.
|
|
30
|
+
|
|
31
|
+
## Decision
|
|
32
|
+
|
|
33
|
+
The runner reads the stamp when each case starts. Only a stamped case
|
|
34
|
+
records its draws. A failing case keeps its exception and its entries
|
|
35
|
+
under its origin. A newer capture replaces an older one, except that an
|
|
36
|
+
unstamped case never replaces a stamped one. hegel-rust ranks its captures
|
|
37
|
+
the same way: a newer capture replaces an older one of the same rank or
|
|
38
|
+
lower.
|
|
39
|
+
|
|
40
|
+
After the run, each failure is reported from the capture under its
|
|
41
|
+
origin. No blob is replayed.
|
|
42
|
+
|
|
43
|
+
A failure with no blob is reported without the reproduction line. A
|
|
44
|
+
failure with a caveat prints it as a `note:` line after the drawn values,
|
|
45
|
+
as hegel-rust does.
|
|
46
|
+
|
|
47
|
+
`Hegel.test(reproduce_failure:)` starts its run with
|
|
48
|
+
`hegel_run_start_blob` and drives it through the same loop. A replay that
|
|
49
|
+
never fails raises `Hegel::Error` that names both causes: a fixed bug, or
|
|
50
|
+
a nondeterministic body whose failure did not recur.
|
|
51
|
+
|
|
52
|
+
Keeping the replay after the run was refused. It ran the body once more
|
|
53
|
+
per failure on top of the engine's own final replay. It also had no blob
|
|
54
|
+
to replay for an unconfirmed failure, so a nondeterministic body would have
|
|
55
|
+
raised an internal error in place of a report.
|
|
56
|
+
|
|
57
|
+
## Consequences
|
|
58
|
+
|
|
59
|
+
A failing run calls the body once fewer per failure. The cost of naming a
|
|
60
|
+
drawn value, a read of the caller's source, falls on the stamped cases
|
|
61
|
+
only. Measured against libhegel 0.45.0, a run that calls its body 60 to
|
|
62
|
+
80 times stamps 5 of them.
|
|
63
|
+
|
|
64
|
+
A `tc.note` block runs on every stamped case, which can be more than one
|
|
65
|
+
per failure: the engine also stamps the first replays that check a newly
|
|
66
|
+
found failure.
|
|
67
|
+
|
|
68
|
+
`hegel_test_case_from_blob` has no caller left, and its binding is gone.
|
|
69
|
+
The sixth layer of verification in ADR 0006 lists "blob replay", which now
|
|
70
|
+
means the replay that `hegel_run_start_blob` drives.
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# 0020: Derive span labels from generator names
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
`hegel_start_span` takes a label. Up to libhegel 0.38, `hegel.h` defined a
|
|
10
|
+
table of `HEGEL_LABEL_*` constants, and this binding passed the matching
|
|
11
|
+
constant for each kind of compound draw: a list, a list element, a map
|
|
12
|
+
entry, and so on.
|
|
13
|
+
|
|
14
|
+
libhegel 0.39.0 removed the table. A label is now an opaque u64 that names
|
|
15
|
+
the generator which opened the span. The engine treats two spans with one
|
|
16
|
+
label as draws of one generator, and swaps, duplicates, or reorders them
|
|
17
|
+
when it shrinks and mutates a test case. The header asks a binding to
|
|
18
|
+
derive labels from names of its own, prefixed with the binding's name to
|
|
19
|
+
keep clear of the engine's own `hegel.<kind>` names. It asks a generator
|
|
20
|
+
built from other generators to combine its own label with its components'
|
|
21
|
+
labels, so that `lists(integers())` and `lists(text())` differ.
|
|
22
|
+
`hegel_label_from_name` is 64-bit FNV-1a over the name's bytes, and the
|
|
23
|
+
header says a binding may compute it ahead of time. hegel-rust computes
|
|
24
|
+
both functions in the language and pins its hash against the engine's.
|
|
25
|
+
|
|
26
|
+
The other implementations differ on names and on combining. hegel-java
|
|
27
|
+
hashes `dev.hegel.*` names in Java, hegel-cpp calls
|
|
28
|
+
`hegel_label_from_name` with `hegel-cpp.*` names, hegel-go hashes the Go
|
|
29
|
+
type, and hegel-typescript passes small integers. Labels are not part of
|
|
30
|
+
any implementation's public interface.
|
|
31
|
+
|
|
32
|
+
## Decision
|
|
33
|
+
|
|
34
|
+
A generator's label comes from its class name with a `hegel-ruby.`
|
|
35
|
+
prefix. A generator built from others combines its own class label with
|
|
36
|
+
the labels of the generators it draws from, in order. Both functions are
|
|
37
|
+
computed in Ruby, with the same hash the header defines, and a test pins
|
|
38
|
+
them against vectors measured from the engine's own functions.
|
|
39
|
+
|
|
40
|
+
A compound generator computes its label once, when it is built, and keeps
|
|
41
|
+
it, as hegel-rust's generators do. Two alternatives were refused. Computing
|
|
42
|
+
the label on every draw slowed a run of nested generators by more than
|
|
43
|
+
twenty times. Computing it on the first draw and keeping it writes to the
|
|
44
|
+
generator at draw time, so a generator the caller froze raised
|
|
45
|
+
`FrozenError` on every draw. A component that is not a generator
|
|
46
|
+
contributes 0 to the label, so building around it does not raise; the
|
|
47
|
+
draw reports the mistake, where a generator validates its arguments.
|
|
48
|
+
|
|
49
|
+
A deferred generator answers with its installed generator's label, or its
|
|
50
|
+
own class label before `set`. A generator built around it before `set`
|
|
51
|
+
keeps that class label, so a self-referential definition has no cycle to
|
|
52
|
+
follow. hegel-rust's deferred generator answers the same way.
|
|
53
|
+
|
|
54
|
+
The span sites stay where they were: around each compound draw, and around
|
|
55
|
+
each element of a collection, labelled with the element generator's own
|
|
56
|
+
label.
|
|
57
|
+
|
|
58
|
+
## Consequences
|
|
59
|
+
|
|
60
|
+
`Hegel::LibHegel` loses the `HEGEL_LABEL_*` constants and gains
|
|
61
|
+
`label_from_name` and `label_combine`. A generator class written outside
|
|
62
|
+
this library gets a label from its own class name with no extra code.
|
|
63
|
+
Renaming a generator class changes its label, which affects nothing past
|
|
64
|
+
the run that uses it.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# 0021: Run a state machine in rounds, with its own step count
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted. Supersedes the `stateful_step_count:` keyword of `Hegel.test`.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
Up to libhegel 0.32.5, a state machine had one call to pick the next rule,
|
|
10
|
+
and the step budget was a run setting,
|
|
11
|
+
`hegel_settings_set_stateful_step_count`, with an engine default of 50.
|
|
12
|
+
This binding exposed it as `Hegel.test(stateful_step_count:)`, and ran
|
|
13
|
+
every invariant after every rule that completed.
|
|
14
|
+
|
|
15
|
+
libhegel 0.33 to 0.43 changed the protocol. A machine runs in rounds:
|
|
16
|
+
`hegel_state_machine_next_group` starts each round, including the first,
|
|
17
|
+
and `hegel_state_machine_next_rule` takes a worker index. The step count
|
|
18
|
+
became an argument of `hegel_new_state_machine`, and the engine has no
|
|
19
|
+
default for it. Between rounds, `hegel_state_machine_should_check_invariant`
|
|
20
|
+
samples each invariant with probability 1 / step_count, so it runs about
|
|
21
|
+
once in a test case of full length; a flag at creation makes an invariant
|
|
22
|
+
run at every round instead. The caller runs every invariant on the initial
|
|
23
|
+
state and on the final state. The machine can also run concurrently, with
|
|
24
|
+
rule groups and weights.
|
|
25
|
+
|
|
26
|
+
hegel-rust (`Machine::steps`), hegel-go (`WithStatefulStepCount`), and
|
|
27
|
+
hegel-java (`Stateful.Options#stepCount`) put the step count on the
|
|
28
|
+
machine runner, with a default of 50. hegel-cpp keeps it in its settings.
|
|
29
|
+
hegel-go, hegel-cpp, and hegel-java sample invariants through the engine.
|
|
30
|
+
hegel-rust and hegel-java name the opt-out `always_run` and `alwaysRun`.
|
|
31
|
+
|
|
32
|
+
## Decision
|
|
33
|
+
|
|
34
|
+
`Hegel::Stateful.run(machine, tc, step_count: 50)` takes the step count,
|
|
35
|
+
and `Hegel.test` no longer takes `stateful_step_count:`. The keyword name
|
|
36
|
+
follows the engine's argument; 50 follows the three implementations that
|
|
37
|
+
moved it.
|
|
38
|
+
|
|
39
|
+
`invariant(name, always_run: true)` checks an invariant at every round.
|
|
40
|
+
Without it, the engine samples the invariant between rounds. Every
|
|
41
|
+
invariant runs before the first rule and after the last.
|
|
42
|
+
|
|
43
|
+
The machine is sequential: one group, equal weights, a concurrency of 1,
|
|
44
|
+
and worker index 0. This binding drives the engine from one thread, and a
|
|
45
|
+
concurrent machine needs one worker thread per drawn level of concurrency.
|
|
46
|
+
Rule weights stay out until a need for them shows up.
|
|
47
|
+
|
|
48
|
+
Keeping `stateful_step_count:` on `Hegel.test` was refused. It would carry
|
|
49
|
+
a value through the runner and the test case to a call that the stateful
|
|
50
|
+
module alone makes, and three of the four implementations that moved it put
|
|
51
|
+
it on the runner.
|
|
52
|
+
|
|
53
|
+
## Consequences
|
|
54
|
+
|
|
55
|
+
A caller who passed `stateful_step_count:` passes `step_count:` to
|
|
56
|
+
`Hegel::Stateful.run` instead. An invariant that a test relied on to run
|
|
57
|
+
after every rule needs `always_run: true`. A machine whose invariants are
|
|
58
|
+
sampled finds a broken invariant at the next sampled round, or at the
|
|
59
|
+
final check, which the shrinker then moves toward the step that broke it.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# 0022: Keep microsecond times over a nanosecond engine
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
libhegel 0.36.0 changed `hegel_time_t` from a microsecond field to a
|
|
10
|
+
nanosecond field, and `hegel_generate_time` and `hegel_generate_datetime`
|
|
11
|
+
now draw whole nanoseconds. The struct kept its layout, so a binding that
|
|
12
|
+
does not change compiles and binds as before, and then reads nanoseconds as
|
|
13
|
+
microseconds.
|
|
14
|
+
|
|
15
|
+
`times` returns a `"HH:MM:SS.ffffff"` String and `datetimes` a `Time`, each
|
|
16
|
+
with microsecond precision. The other implementations disagree on what a
|
|
17
|
+
drawn time shows: hegel-rust prints nine digits whenever the nanosecond is
|
|
18
|
+
not zero, hegel-typescript prints six digits for a whole microsecond and
|
|
19
|
+
nine otherwise, and hegel-cpp truncates to microseconds. hegel-go and
|
|
20
|
+
hegel-java return their languages' typed values.
|
|
21
|
+
|
|
22
|
+
## Decision
|
|
23
|
+
|
|
24
|
+
`times` and `datetimes` keep microsecond precision. A lower bound's
|
|
25
|
+
microsecond becomes its first nanosecond, and an upper bound's its last,
|
|
26
|
+
so the engine draws over every nanosecond inside the bounds. The drawn
|
|
27
|
+
nanosecond rounds down to a microsecond, so every microsecond in range
|
|
28
|
+
stays reachable, both bounds included. The engine does not draw
|
|
29
|
+
nanoseconds uniformly, so the microseconds are not equally likely either:
|
|
30
|
+
measured against libhegel 0.45.0, two runs of 2000 draws over four
|
|
31
|
+
microseconds gave 1437, 284, 162, and 117, and 1428, 296, 153, and 123.
|
|
32
|
+
|
|
33
|
+
A `datetimes` bound with a fraction of a microsecond narrows to the whole
|
|
34
|
+
microseconds inside the range: the lower bound rounds up and the upper
|
|
35
|
+
rounds down.
|
|
36
|
+
|
|
37
|
+
Moving to nanosecond output was refused while the implementations do not
|
|
38
|
+
agree on its format. A change to it belongs upstream first, as ADR 0015
|
|
39
|
+
requires.
|
|
40
|
+
|
|
41
|
+
## Consequences
|
|
42
|
+
|
|
43
|
+
The public format of `times` and the precision of `datetimes` stay as they
|
|
44
|
+
were. The binding layer speaks nanoseconds, and the conversion lives in
|
|
45
|
+
`Hegel::Generators.first_nanosecond` and `last_nanosecond`.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# 0023: Leave unset settings to the engine's profile
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted. Narrows the `nil` rule in the Consequences of
|
|
6
|
+
[ADR 0009](0009-turn-the-example-database-on-with-a-key.md).
|
|
7
|
+
|
|
8
|
+
## Context
|
|
9
|
+
|
|
10
|
+
A `Hegel.test` keyword left `nil` calls no setter, so the engine's value
|
|
11
|
+
applies. Up to libhegel 0.32.5, that value was a fixed default.
|
|
12
|
+
|
|
13
|
+
libhegel 0.40.0 moved defaults into named profiles that the engine
|
|
14
|
+
resolves. `hegel_settings_new` resolves the default profile: one named by
|
|
15
|
+
`hegel_set_default_profile`, `HEGEL_DEFAULT_PROFILE`, or a `hegel.toml`;
|
|
16
|
+
otherwise `ci` on a CI server and `development` elsewhere. libhegel 0.43.4
|
|
17
|
+
then applies `HEGEL_TEST_CASES`, `HEGEL_DATABASE`, `HEGEL_SEED`, and the
|
|
18
|
+
other `HEGEL_*` variables on top. The shipped `ci` profile derandomizes the
|
|
19
|
+
run, turns the example database off, and suppresses the TooSlow health
|
|
20
|
+
check. `hegel_settings_new` can now fail, for a malformed `hegel.toml` or
|
|
21
|
+
`HEGEL_*` variable.
|
|
22
|
+
|
|
23
|
+
hegel-rust builds each run's settings from the reserved `base` profile and
|
|
24
|
+
overwrites every field from its own `Settings` value, which it resolved from
|
|
25
|
+
the default profile earlier. hegel-go, hegel-cpp, hegel-java, and
|
|
26
|
+
hegel-typescript call `hegel_settings_new`.
|
|
27
|
+
|
|
28
|
+
## Decision
|
|
29
|
+
|
|
30
|
+
The binding keeps calling `hegel_settings_new` and keeps the `nil` rule.
|
|
31
|
+
A keyword left `nil` now takes its value from the resolved profile and the
|
|
32
|
+
`HEGEL_*` variables. A keyword the caller passes still wins, because its
|
|
33
|
+
setter runs after the engine resolved the profile. A failure of
|
|
34
|
+
`hegel_settings_new` raises `Hegel::Error` with the engine's message.
|
|
35
|
+
|
|
36
|
+
`report_multiple_failures:` still defaults to `false` and is always
|
|
37
|
+
passed, so a profile cannot turn a re-raised exception into a count.
|
|
38
|
+
|
|
39
|
+
Starting from `base` as hegel-rust does was refused. Ruby has no settings
|
|
40
|
+
object to resolve a profile into, so the binding would have to read every
|
|
41
|
+
field back through the `hegel_settings_get_*` calls to reproduce what
|
|
42
|
+
`hegel_settings_new` already does.
|
|
43
|
+
|
|
44
|
+
## Consequences
|
|
45
|
+
|
|
46
|
+
A project can shape every run through `hegel.toml` or `HEGEL_*` variables
|
|
47
|
+
without changing a call. On a CI server, a run with `database_key:` and no
|
|
48
|
+
`database:` stores nothing, because the `ci` profile turns the database
|
|
49
|
+
off; passing `database:` turns it back on. A run on CI is derandomized
|
|
50
|
+
unless it passes `derandomize: false`.
|
data/docs/architecture.md
CHANGED
|
@@ -15,6 +15,9 @@ wrapping the engine's whole draw surface, and the run loop.
|
|
|
15
15
|
What exists beyond that: the failure report, which names drawn values by
|
|
16
16
|
reading the caller's own source, interleaves them with whatever `tc.note`
|
|
17
17
|
recorded, and prints a blob that `Hegel.test(reproduce_failure:)` replays.
|
|
18
|
+
The report comes from the cases the engine stamps for capture during the
|
|
19
|
+
run; see
|
|
20
|
+
[ADR 0019](adr/0019-report-failures-from-the-cases-the-engine-stamps.md).
|
|
18
21
|
The generator layer covers twenty-five generators composing through `map`
|
|
19
22
|
and `filter`, reachable bare through `Hegel::Syntax::Methods`. A test case
|
|
20
23
|
can discard itself with `tc.assume` or `tc.reject`. A run can be shaped by
|
|
@@ -27,9 +30,11 @@ database keywords.
|
|
|
27
30
|
`Hegel::Stateful.run` drives a `Hegel::StateMachine`'s rules and invariants,
|
|
28
31
|
drawing values an earlier rule produced back out of a
|
|
29
32
|
`Hegel::Stateful::Pool`. [ADR 0010](adr/0010-declare-stateful-rules-with-a-class-macro.md)
|
|
30
|
-
decides how a machine declares its rules
|
|
33
|
+
decides how a machine declares its rules,
|
|
31
34
|
[ADR 0011](adr/0011-let-the-test-case-own-every-pool-drawn-from-it.md) who
|
|
32
|
-
frees a pool
|
|
35
|
+
frees a pool, and
|
|
36
|
+
[ADR 0021](adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md)
|
|
37
|
+
how its rounds, step count, and invariants run.
|
|
33
38
|
|
|
34
39
|
Every feature this binding set out to cover now has a Ruby surface. What
|
|
35
40
|
remains open is not a feature but a measurement, and
|
|
@@ -116,9 +121,9 @@ the other. See
|
|
|
116
121
|
## A rule's own control flow
|
|
117
122
|
|
|
118
123
|
Inside a stateful rule, `tc.assume(false)` means something narrower than it
|
|
119
|
-
does in a test body. `Hegel::Stateful` catches it
|
|
120
|
-
was rejected so
|
|
121
|
-
the test case continues. `Hegel::Runner.classify` never sees it, and no case
|
|
124
|
+
does in a test body. `Hegel::Stateful` catches it and tells libhegel the
|
|
125
|
+
rule was rejected, so its round does not count toward the step budget, and
|
|
126
|
+
the machine starts another round; the test case continues. `Hegel::Runner.classify` never sees it, and no case
|
|
122
127
|
is discarded.
|
|
123
128
|
|
|
124
129
|
A rule is also the one place in this library where a process-ending
|
|
@@ -168,7 +173,7 @@ fields.
|
|
|
168
173
|
|
|
169
174
|
A drawn string arrives as a pointer and a length, is not NUL-terminated,
|
|
170
175
|
and can hold interior NUL bytes, because the drawn alphabet can include
|
|
171
|
-
U+0000. Read by length. Measured against 0.
|
|
176
|
+
U+0000. Read by length. Measured against 0.45.0: an alphabet pinned to
|
|
172
177
|
U+0000 produces a three-byte draw that reads as `""` when taken up to the
|
|
173
178
|
first NUL and as three NUL bytes when taken by its reported length. Every
|
|
174
179
|
string observed reports `valid_encoding?` after `force_encoding(UTF-8)`.
|
data/lib/hegel/draw_name.rb
CHANGED
|
@@ -6,9 +6,10 @@ module Hegel
|
|
|
6
6
|
# Recovers a drawn value's variable name from the caller's own source (see
|
|
7
7
|
# docs/adr/0005), so a failure report can print `n = 501` instead of
|
|
8
8
|
# `draw = 501` when the caller never passed a label:. This module only
|
|
9
|
-
# answers "what name does path:lineno assign
|
|
10
|
-
#
|
|
11
|
-
#
|
|
9
|
+
# answers "what name does path:lineno assign the value a draw call there
|
|
10
|
+
# produced, when that call is the whole assigned value"; deciding whether
|
|
11
|
+
# to call it, what counts as a draw call, and what to fall back to when it
|
|
12
|
+
# answers nil, is Hegel::TestCase#name_for's job, not this one's.
|
|
12
13
|
module DrawName
|
|
13
14
|
# Node types #for treats as "this line names a drawn value". An
|
|
14
15
|
# explicit list, not every Prism::Node subclass whose name ends in
|
|
@@ -23,10 +24,20 @@ module Hegel
|
|
|
23
24
|
|
|
24
25
|
# The name +path+:+lineno+ assigns a drawn value to, or nil when that
|
|
25
26
|
# cannot be answered confidently: +path+ cannot be read, Prism cannot
|
|
26
|
-
# parse it,
|
|
27
|
+
# parse it, the assignments covering +lineno+ do not single one out, or
|
|
28
|
+
# the one they single out does not assign a draw call directly.
|
|
29
|
+
# +call_names+ is the set of method names that count as a draw call
|
|
30
|
+
# (Hegel::TestCase::DRAW_METHOD_NAMES); this module has no opinion of
|
|
31
|
+
# its own about which methods draw.
|
|
32
|
+
#
|
|
27
33
|
# A wrong name would misdirect a reader of the failure report more than
|
|
28
34
|
# a missing one would, so an ambiguous line returns nil rather than
|
|
29
|
-
# guessing between candidates.
|
|
35
|
+
# guessing between candidates. That is also why a singled-out
|
|
36
|
+
# assignment still answers nil when its value is built from a draw
|
|
37
|
+
# rather than being the draw itself, as in `xs = [tc.draw(integers)]`
|
|
38
|
+
# or `n = tc.draw(integers).abs`: the recovered name would describe the
|
|
39
|
+
# Array or the Integer#abs result, not the drawn value the report
|
|
40
|
+
# actually prints next to it.
|
|
30
41
|
#
|
|
31
42
|
# Several assignments can cover one line by nesting rather than by
|
|
32
43
|
# ambiguity. `error = assert_raises do ... n = tc.draw_integer(...) ...
|
|
@@ -35,13 +46,16 @@ module Hegel
|
|
|
35
46
|
# whenever another candidate sits inside it. Two assignments written
|
|
36
47
|
# side by side on one line contain neither the other, nothing singles
|
|
37
48
|
# one out, and the answer is nil.
|
|
38
|
-
def for(path, lineno)
|
|
49
|
+
def for(path, lineno, call_names)
|
|
39
50
|
program = parse(path)
|
|
40
51
|
return nil unless program
|
|
41
52
|
|
|
42
53
|
matches = assignment_nodes(program).select { |node| covers?(node.location, lineno) }
|
|
43
54
|
innermost = matches.reject { |node| matches.any? { |other| encloses?(node, other) } }
|
|
44
|
-
innermost.one?
|
|
55
|
+
return nil unless innermost.one?
|
|
56
|
+
|
|
57
|
+
node = innermost.first
|
|
58
|
+
(node.value.is_a?(Prism::CallNode) && call_names.include?(node.value.name)) ? node.name.to_s : nil
|
|
45
59
|
end
|
|
46
60
|
|
|
47
61
|
# Drops every cached parse. #for's only state; a fresh process would
|
|
@@ -102,7 +116,9 @@ module Hegel
|
|
|
102
116
|
def encloses?(outer, inner)
|
|
103
117
|
return false if outer.equal?(inner)
|
|
104
118
|
|
|
105
|
-
outer
|
|
119
|
+
# An outer assignment's start offset never equals an inner one's, since
|
|
120
|
+
# the outer node begins with the name first.
|
|
121
|
+
outer.location.start_offset <= inner.location.start_offset && # mutineer:disable-line comparison
|
|
106
122
|
inner.location.end_offset <= outer.location.end_offset
|
|
107
123
|
end
|
|
108
124
|
end
|
data/lib/hegel/generator.rb
CHANGED
|
@@ -18,10 +18,43 @@ module Hegel
|
|
|
18
18
|
raise NotImplementedError, "#{self.class} must implement #do_draw"
|
|
19
19
|
end
|
|
20
20
|
|
|
21
|
+
# The span label naming this generator's draws to the engine, which
|
|
22
|
+
# treats two spans with one label as draws of one generator when it
|
|
23
|
+
# shrinks. A generator built from others overrides this and combines
|
|
24
|
+
# its own class label with theirs, so that arrays of integers and
|
|
25
|
+
# arrays of strings do not share a label.
|
|
26
|
+
def label
|
|
27
|
+
self.class.label
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
# The class name, prefixed with this binding's name as hegel.h asks, to
|
|
31
|
+
# keep clear of the engine's own hegel.<kind> labels.
|
|
32
|
+
def self.label
|
|
33
|
+
LibHegel.label_from_name("hegel-ruby.#{name}")
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# Combines this generator's class label with the labels of +parts+, the
|
|
37
|
+
# generators it draws from. A compound generator calls this once, when it
|
|
38
|
+
# is built, and keeps the result: computing it on every draw slowed a
|
|
39
|
+
# run of nested generators by more than twenty times, and keeping it
|
|
40
|
+
# from the first draw would write to a generator a caller may have
|
|
41
|
+
# frozen.
|
|
42
|
+
def combined_label(*parts)
|
|
43
|
+
LibHegel.label_combine([self.class.label, *parts.map { |part| Generator.label_of(part) }])
|
|
44
|
+
end
|
|
45
|
+
private :combined_label
|
|
46
|
+
|
|
47
|
+
# +part+'s label, or 0 for a part that is not a generator. Such a part
|
|
48
|
+
# fails when it is drawn, where a generator validates its arguments, so
|
|
49
|
+
# building the generator around it must not fail first.
|
|
50
|
+
def self.label_of(part)
|
|
51
|
+
part.respond_to?(:label) ? part.label : 0
|
|
52
|
+
end
|
|
53
|
+
|
|
21
54
|
# Returns a new Generator whose #do_draw runs +block+ on the value this
|
|
22
|
-
# generator drew,
|
|
23
|
-
#
|
|
24
|
-
#
|
|
55
|
+
# generator drew, inside one span so the shrinker treats the source draw
|
|
56
|
+
# and the transform as one unit (see hegel-rust's Mapped,
|
|
57
|
+
# src/generators/generators.rs).
|
|
25
58
|
def map(&block)
|
|
26
59
|
Mapped.new(self, block)
|
|
27
60
|
end
|
|
@@ -39,13 +72,15 @@ module Hegel
|
|
|
39
72
|
# generators.rb.
|
|
40
73
|
class Mapped < Generator
|
|
41
74
|
def initialize(source, block)
|
|
42
|
-
super()
|
|
43
75
|
@source = source
|
|
44
76
|
@block = block
|
|
77
|
+
@label = combined_label(@source)
|
|
45
78
|
end
|
|
46
79
|
|
|
80
|
+
attr_reader :label
|
|
81
|
+
|
|
47
82
|
def do_draw(tc)
|
|
48
|
-
tc.start_span(
|
|
83
|
+
tc.start_span(label)
|
|
49
84
|
@block.call(@source.do_draw(tc))
|
|
50
85
|
ensure
|
|
51
86
|
# discard is always false here: a span is only ever marked
|
|
@@ -73,16 +108,22 @@ module Hegel
|
|
|
73
108
|
MAX_ATTEMPTS = 3
|
|
74
109
|
|
|
75
110
|
def initialize(source, block)
|
|
76
|
-
super()
|
|
77
111
|
@source = source
|
|
78
112
|
@block = block
|
|
113
|
+
@label = combined_label(@source)
|
|
79
114
|
end
|
|
80
115
|
|
|
116
|
+
attr_reader :label
|
|
117
|
+
|
|
81
118
|
def do_draw(tc)
|
|
82
119
|
MAX_ATTEMPTS.times do
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
120
|
+
# Each variable is set again before it is read, or stays nil, which
|
|
121
|
+
# acts like false (!nil == !false).
|
|
122
|
+
# The initializer also shows intent: discard the span when an
|
|
123
|
+
# exception skips the reassignment.
|
|
124
|
+
accepted = false # mutineer:disable-line statement_removal
|
|
125
|
+
value = nil # mutineer:disable-line statement_removal, boolean_literal
|
|
126
|
+
tc.start_span(label)
|
|
86
127
|
begin
|
|
87
128
|
value = @source.do_draw(tc)
|
|
88
129
|
accepted = @block.call(value)
|