hegeltest 0.3.1-aarch64-linux → 0.4.0-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +262 -0
- data/README.ja.md +408 -0
- data/README.md +209 -103
- data/Rakefile +6 -3
- data/docs/README.md +11 -0
- data/docs/adr/0025-copy-each-string-input-into-a-hegel-string-t.md +61 -0
- data/docs/adr/0026-select-the-engine-release-repository-by-version.md +47 -0
- data/docs/adr/0027-report-each-runs-test-location-from-the-callers-frame.md +63 -0
- data/docs/adr/0028-expose-settings-profiles-as-a-keyword-and-module-functions.md +91 -0
- data/docs/adr/0029-keep-notes-local-and-defer-run-error-kinds.md +72 -0
- data/docs/adr/0030-represent-a-duration-as-integer-nanoseconds.md +48 -0
- data/docs/adr/0031-take-the-alphabet-options-from-hegel-rust.md +52 -0
- data/docs/adr/0032-suppress-span-closes-while-an-abandoned-recursion-attempt-unwinds.md +147 -0
- data/docs/adr/0033-cache-compiled-grammars-for-the-length-of-a-run.md +80 -0
- data/docs/adr/0034-repeat-a-test-body-with-a-ruby-side-indent.md +120 -0
- data/docs/adr/0035-defer-concurrent-state-machines-and-the-engine-printer.md +136 -0
- data/docs/architecture.md +33 -11
- data/lib/hegel/errors.rb +24 -0
- data/lib/hegel/generator.rb +52 -25
- data/lib/hegel/generators.rb +715 -45
- data/lib/hegel/grammar_cache.rb +36 -0
- data/lib/hegel/lib_hegel/real.rb +455 -120
- data/lib/hegel/lib_hegel.rb +33 -8
- data/lib/hegel/libhegel/libhegel-linux-arm64.so +0 -0
- data/lib/hegel/libhegel_version.rb +1 -1
- data/lib/hegel/locate.rb +10 -10
- data/lib/hegel/report.rb +13 -10
- data/lib/hegel/runner.rb +61 -27
- data/lib/hegel/settings.rb +92 -16
- data/lib/hegel/state_machine.rb +17 -4
- data/lib/hegel/stateful/pool.rb +52 -0
- data/lib/hegel/stateful.rb +2 -2
- data/lib/hegel/syntax/methods.rb +99 -15
- data/lib/hegel/test_case.rb +256 -35
- data/lib/hegel/test_location.rb +49 -0
- data/lib/hegel/version.rb +1 -1
- data/lib/hegel.rb +21 -5
- data/lib/tasks/libhegel.rake +17 -5
- data/lib/tasks/platform_gems.rake +3 -3
- data/sig/hegel.rbs +296 -43
- data/skills/hegel-ruby/references/ruby/reference.md +465 -43
- metadata +15 -1
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# 0028: Expose settings profiles as a keyword and module functions
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted. Builds on [ADR 0023](0023-leave-unset-settings-to-the-engines-profile.md),
|
|
6
|
+
which left the profile to the engine, and applies
|
|
7
|
+
[ADR 0015](0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md).
|
|
8
|
+
|
|
9
|
+
## Context
|
|
10
|
+
|
|
11
|
+
libhegel 0.47.2 has three calls for profiles. `hegel_settings_new_for_profile`
|
|
12
|
+
builds a settings handle from a named profile. `hegel_settings_register_profile`
|
|
13
|
+
stores a complete snapshot of a handle under a name, for the whole process,
|
|
14
|
+
replacing an earlier registration of that name. `hegel_set_default_profile`
|
|
15
|
+
names the profile that `hegel_settings_new` resolves, for the whole process,
|
|
16
|
+
and an empty name clears it. A named profile still extends the default
|
|
17
|
+
profile, except `base`. An unknown name, a reserved name, a malformed
|
|
18
|
+
`hegel.toml`, and a malformed `HEGEL_*` variable return `HEGEL_E_INVALID_ARG`
|
|
19
|
+
with a message. The snapshot leaves out the database key, and a registration
|
|
20
|
+
does not change handles that already exist.
|
|
21
|
+
|
|
22
|
+
hegel-rust exposes them on its `Settings` value: `Settings::from_profile`
|
|
23
|
+
and `Settings::try_from_profile` return a `Settings` to adjust with builder
|
|
24
|
+
methods, `Settings::register_profile(name, settings)` registers one, and
|
|
25
|
+
`Settings::set_default_profile(name)` sets the default. Its documentation
|
|
26
|
+
example is `Settings::register_profile("nightly",
|
|
27
|
+
Settings::from_profile("ci").test_cases(10_000))`.
|
|
28
|
+
|
|
29
|
+
Ruby has no `Settings` value. `Hegel.test` takes keywords, and
|
|
30
|
+
`Hegel::Settings` is a module of functions that copy those keywords onto a
|
|
31
|
+
native handle that exists only for the length of one run (ADR 0023).
|
|
32
|
+
|
|
33
|
+
## Decision
|
|
34
|
+
|
|
35
|
+
**`Hegel.test` takes `profile:`, which stands in for `from_profile`.** A
|
|
36
|
+
`nil` profile calls `hegel_settings_new`, as before. A String calls
|
|
37
|
+
`hegel_settings_new_for_profile`. The keywords apply afterwards in both
|
|
38
|
+
cases, so one passed explicitly still beats the profile's value, and the
|
|
39
|
+
`reproduce_failure:` path builds its settings the same way. This is a
|
|
40
|
+
difference of mechanism under ADR 0015: the value that hegel-rust builds and
|
|
41
|
+
then adjusts is, here, the argument of the call that uses it.
|
|
42
|
+
|
|
43
|
+
**`Hegel::Settings.register_profile(name, profile: nil, **keywords)`
|
|
44
|
+
registers a profile.** It sits on `Hegel::Settings` because hegel-rust puts
|
|
45
|
+
`register_profile` on `Settings`. It opens a context of its own, builds a
|
|
46
|
+
handle from `hegel_settings_new` or from the named `profile:`, applies the
|
|
47
|
+
keywords, registers the handle, and frees the handle and then the context in
|
|
48
|
+
an `ensure`. It takes an `impl:` keyword, defaulting to `Hegel.default_impl`,
|
|
49
|
+
so a test can pass the Fake the way it does to `Hegel.test`.
|
|
50
|
+
|
|
51
|
+
The keywords are those of `Hegel.test`, each `nil` meaning that no setter
|
|
52
|
+
runs, with these exceptions:
|
|
53
|
+
|
|
54
|
+
- `database_key:` is not accepted. The engine leaves the key out of the
|
|
55
|
+
snapshot, so accepting it would promise something the profile cannot keep.
|
|
56
|
+
Ruby's own `ArgumentError` rejects it, as it does any unknown keyword.
|
|
57
|
+
- `database:` goes straight to `hegel_settings_set_database`, a path or `""`
|
|
58
|
+
to turn the database off. The table of ADR 0009 exists to decide what a
|
|
59
|
+
missing key means for a run, and a profile carries no key.
|
|
60
|
+
- `report_multiple_failures:` is `nil` by default here, where `Hegel.test`
|
|
61
|
+
defaults it to `false`.
|
|
62
|
+
|
|
63
|
+
Starting from `hegel_settings_new` matches `Settings::new()`, which resolves
|
|
64
|
+
the default profile. hegel-rust's documented example starts from another
|
|
65
|
+
profile instead, and `profile: "ci"` is the same thing here.
|
|
66
|
+
|
|
67
|
+
**`Hegel::Settings.set_default_profile(name)` sets the default profile,**
|
|
68
|
+
with the same `impl:` keyword. `""` clears it.
|
|
69
|
+
|
|
70
|
+
An error from the engine raises `Hegel::Error` with the engine's message,
|
|
71
|
+
through `LibHegel.check!`. hegel-rust panics with the same message.
|
|
72
|
+
|
|
73
|
+
## Consequences
|
|
74
|
+
|
|
75
|
+
`Hegel.test` always passes `report_multiple_failures`, as ADR 0023 decided,
|
|
76
|
+
so a profile's value for it is overridden on every run. A profile can carry
|
|
77
|
+
the setting, but it never changes what `Hegel.test` does.
|
|
78
|
+
|
|
79
|
+
Registrations and the default profile are state of the process, held by the
|
|
80
|
+
engine and not by any context. A test that registers a profile leaves it
|
|
81
|
+
registered, and the engine has no call to remove one. A test that sets the
|
|
82
|
+
default profile clears it again with `set_default_profile("")`. Neither
|
|
83
|
+
affects a run that is already in progress.
|
|
84
|
+
|
|
85
|
+
A `HEGEL_*` variable applies over a profile's snapshot, as the header says
|
|
86
|
+
for every handle, so `HEGEL_PRINT_BLOB` beats a `print_blob: false` that the
|
|
87
|
+
profile registered. The tests that observe a profile through the failure
|
|
88
|
+
report clear those variables first.
|
|
89
|
+
|
|
90
|
+
Each call to `register_profile` and `set_default_profile` opens and frees a
|
|
91
|
+
context. Both are meant for setup, run once, not once per test case.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# 0029: Keep notes local, and defer run error kinds
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted. It records two calls that libhegel 0.47.2 offers and this gem does
|
|
6
|
+
not bind into its public surface. Each is a decision to wait, with the
|
|
7
|
+
condition that ends the wait.
|
|
8
|
+
|
|
9
|
+
## Context
|
|
10
|
+
|
|
11
|
+
`hegel_note` and `hegel_test_case_printer` both write to the document that
|
|
12
|
+
the engine keeps for each test case. `hegel_test_case_printer` fetches a
|
|
13
|
+
handle onto the test-case handle's print region. `hegel_note` appends a
|
|
14
|
+
string to the same region, one output line for each line of the note, and
|
|
15
|
+
the header says that notes and drawn values from one handle appear in the
|
|
16
|
+
order they were appended. The document stays readable after
|
|
17
|
+
`hegel_mark_complete`, so a client can assemble its output while drawing and
|
|
18
|
+
read it back afterwards.
|
|
19
|
+
|
|
20
|
+
`hegel_run_result_error_kind` reports why a run ended in
|
|
21
|
+
`HEGEL_RUN_STATUS_ERROR`: a failed health check, which it names as a
|
|
22
|
+
`hegel_health_check_t` value, a flaky or nondeterministic test, an
|
|
23
|
+
unsatisfiable one, a misuse of libhegel at run scope, or an internal
|
|
24
|
+
invariant. The header says the message from `hegel_run_result_error` is for
|
|
25
|
+
people, and that a binding should choose which error to raise from the kind
|
|
26
|
+
without parsing the message.
|
|
27
|
+
|
|
28
|
+
`TestCase#note` records `[:note, message]` in the same entry list that
|
|
29
|
+
`TestCase#record_draw` fills with `[:draw, name, value]`, and
|
|
30
|
+
`Hegel::Report` renders both from that list when it assembles the failure
|
|
31
|
+
report. This gem never opens the engine's printer, so the engine's document
|
|
32
|
+
holds nothing.
|
|
33
|
+
|
|
34
|
+
`Hegel::Runner` reads a run's error through `hegel_run_result_error` alone
|
|
35
|
+
and raises `Hegel::Error` with that message. hegel-rust declares
|
|
36
|
+
`hegel_run_result_error_kind` in its generated bindings,
|
|
37
|
+
`frontends/rust/src/ffi/sys/fns.rs` and `frontends/rust/src/ffi/sys/types.rs`,
|
|
38
|
+
and its safe wrapper in `frontends/rust/src/ffi.rs` reads a run's error
|
|
39
|
+
through the message, with `hegel_run_result_error`.
|
|
40
|
+
|
|
41
|
+
## Decision
|
|
42
|
+
|
|
43
|
+
**`TestCase#note` stays on the Ruby side.** `hegel_note` writes into the
|
|
44
|
+
engine's print document, and that document is only worth reading when the
|
|
45
|
+
draws are written there too, through `hegel_test_case_printer`. Moving the
|
|
46
|
+
notes alone would take them out of the entry list that `Hegel::Report`
|
|
47
|
+
renders, so they would vanish from the report. It would also split the
|
|
48
|
+
ordering the header promises, because a note in the engine's document and a
|
|
49
|
+
draw in the Ruby list have no shared sequence. The two move together, when
|
|
50
|
+
this gem adopts the printer for draws.
|
|
51
|
+
|
|
52
|
+
**`hegel_run_result_error_kind` is not bound.** hegel-rust's frontend takes a
|
|
53
|
+
run's error as a message, so a classified `Hegel::Error`, one subclass for a failed
|
|
54
|
+
health check and another for an unsatisfiable run, would be a feature that
|
|
55
|
+
this gem alone has. ADR 0015 calls that a divergence. The route is a
|
|
56
|
+
proposal at hegeldev/hegel: if the shared interface settles on classified
|
|
57
|
+
run errors, this gem takes the names and the semantics that were settled
|
|
58
|
+
on, and binds the call then.
|
|
59
|
+
|
|
60
|
+
## Consequences
|
|
61
|
+
|
|
62
|
+
A note keeps its present cost: one entry in a Ruby array, in the cases that
|
|
63
|
+
record.
|
|
64
|
+
|
|
65
|
+
A caller who needs to tell a failed health check from an unsatisfiable run
|
|
66
|
+
reads the message of the `Hegel::Error`, as before. The message wording may
|
|
67
|
+
change between engine releases, and the header says so; that is the cost of
|
|
68
|
+
waiting for the shared interface.
|
|
69
|
+
|
|
70
|
+
Revisit the first decision together with the adoption of
|
|
71
|
+
`hegel_test_case_printer`, and the second when hegeldev/hegel settles on
|
|
72
|
+
classified run errors.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# 0030: Represent a duration as Integer nanoseconds
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
hegel-rust draws a `std::time::Duration` from `durations()`, by drawing a
|
|
10
|
+
`u64` of nanoseconds and converting it. The largest duration it can
|
|
11
|
+
generate is `u64::MAX` nanoseconds, about 584 years. `max_value` saturates
|
|
12
|
+
at that limit, and a `min_value` above it is a usage error at draw time,
|
|
13
|
+
because no generatable value could satisfy it.
|
|
14
|
+
|
|
15
|
+
Ruby has no duration type in its standard library. The nearest things are
|
|
16
|
+
a `Float` count of seconds, which is what `Time#-` and
|
|
17
|
+
`Process.clock_gettime` return by default, a `Rational` count of seconds,
|
|
18
|
+
and `ActiveSupport::Duration`, which is not part of Ruby. The engine draws
|
|
19
|
+
a nanosecond count as an integer, and `integers` already routes a bound
|
|
20
|
+
above int64 to `hegel_generate_integer_big`.
|
|
21
|
+
|
|
22
|
+
## Decision
|
|
23
|
+
|
|
24
|
+
`durations(min_value: 0, max_value: 2**64 - 1)` returns an `Integer` count
|
|
25
|
+
of nanoseconds, drawn through `integers`. Its bounds are nanoseconds too.
|
|
26
|
+
|
|
27
|
+
An `Integer` is exact over the whole range, where a `Float` of seconds
|
|
28
|
+
loses nanoseconds past about 104 days, and a property that compares two
|
|
29
|
+
durations stays free of rounding. It composes with
|
|
30
|
+
`Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)`, the one
|
|
31
|
+
way Ruby reads elapsed time without converting. Under a `Rational` of
|
|
32
|
+
seconds a caller needs a conversion before every use as a sleep or a
|
|
33
|
+
timeout, and the shrunk value reads as a fraction.
|
|
34
|
+
|
|
35
|
+
The validation follows hegel-rust. `max_value` saturates at `2**64 - 1`.
|
|
36
|
+
A `min_value` above it raises with hegel-rust's wording, and so does
|
|
37
|
+
`max_value < min_value`. A negative `min_value` raises as well. Rust's
|
|
38
|
+
unsigned type cannot hold one, so its message has no Rust counterpart.
|
|
39
|
+
|
|
40
|
+
## Consequences
|
|
41
|
+
|
|
42
|
+
A caller who wants a `Float` of seconds divides by `1_000_000_000.0`. A
|
|
43
|
+
change to the unit or the type belongs upstream first, as
|
|
44
|
+
[ADR 0015](0015-follow-the-hegeldev-interface-and-take-changes-upstream-first.md)
|
|
45
|
+
requires. The unit differs from `times` and `datetimes`, which keep
|
|
46
|
+
microseconds, per [ADR 0022](0022-keep-microsecond-times-over-a-nanosecond-engine.md):
|
|
47
|
+
those return a formatted String or a `Time`, and a duration is a bare
|
|
48
|
+
number with nothing to format.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# 0031: Take the alphabet options from hegel-rust
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
`text` and `characters` took `codec:`, `min_codepoint:`, and
|
|
10
|
+
`max_codepoint:`, and `from_regex` took no alphabet. hegel-rust's
|
|
11
|
+
`text()` and `characters()` also take `categories`, `exclude_categories`,
|
|
12
|
+
`include_characters`, and `exclude_characters`. `text()` takes `alphabet`,
|
|
13
|
+
and `from_regex()` takes `alphabet`. ADR 0015 makes the shared interface
|
|
14
|
+
the target for names, defaults, and meaning.
|
|
15
|
+
|
|
16
|
+
hegel-rust differs from what a Ruby caller might expect in three places.
|
|
17
|
+
|
|
18
|
+
- `alphabet` on `text()` is a string of characters, and `alphabet` on
|
|
19
|
+
`from_regex()` is a `characters()` generator.
|
|
20
|
+
- When `categories` is given, `exclude_categories` is dropped.
|
|
21
|
+
- It rejects the categories `Cs` and `C`, because a Rust string cannot hold
|
|
22
|
+
a surrogate, and it adds `Cs` to the exclusions otherwise. `hegel.h`
|
|
23
|
+
already says that surrogates are always removed from a text generator.
|
|
24
|
+
|
|
25
|
+
## Decision
|
|
26
|
+
|
|
27
|
+
The options carry hegel-rust's names and meanings. A `String` alphabet on
|
|
28
|
+
`text` is what hegel-rust's `alphabet(&str)` is: no categories, with the
|
|
29
|
+
string as `include_characters`. A value that is not a `String` raises at draw
|
|
30
|
+
time, since Ruby has no type to make the case unrepresentable. `from_regex`
|
|
31
|
+
takes a `characters` generator, as hegel-rust does.
|
|
32
|
+
|
|
33
|
+
`categories` wins over `exclude_categories`, as in hegel-rust. `Cs` and `C`
|
|
34
|
+
are rejected in `categories`. `Cs` is not added to the exclusions, since the
|
|
35
|
+
engine removes surrogates itself and a handle is built on every draw.
|
|
36
|
+
|
|
37
|
+
An unknown codec is rejected in Ruby with hegel-rust's message, at draw time
|
|
38
|
+
as ADR 0015 and `CLAUDE.md` require of every validation. The other
|
|
39
|
+
constraints that leave no characters are the engine's to report, as in
|
|
40
|
+
hegel-rust.
|
|
41
|
+
|
|
42
|
+
`from_regex` builds its alphabet as a text handle sized 0 to 0, as hegel-rust
|
|
43
|
+
does, so an empty alphabet builds and the engine decides whether the pattern
|
|
44
|
+
can still match. The handle is built on each draw and freed after the regex
|
|
45
|
+
handle made from it, like the handle `text` builds.
|
|
46
|
+
|
|
47
|
+
## Consequences
|
|
48
|
+
|
|
49
|
+
An alphabet handle is one more native construction on each `from_regex`
|
|
50
|
+
draw. Measured against libhegel 0.47.2 over 5000 draws, `from_regex` took
|
|
51
|
+
about 35 microseconds a draw without an alphabet and 47 with one, and
|
|
52
|
+
`text(max_size: 5)` took about 31 without categories and 51 with two.
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# 0032: Suppress span closes while an abandoned recursion attempt unwinds
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
libhegel 0.47.2 draws a recursively defined value, such as a tree, through a
|
|
10
|
+
recursion handle: `hegel_new_recursion`, then `hegel_recursion_branch` for
|
|
11
|
+
each sub-value, `hegel_recursion_leaf` before each leaf, and
|
|
12
|
+
`hegel_recursion_finish` once the root is complete. The engine owns the
|
|
13
|
+
branch probabilities and the depth and leaf budgets, so a recursive value is
|
|
14
|
+
distributed the same way in every frontend.
|
|
15
|
+
|
|
16
|
+
An attempt can outgrow its leaf budget. `hegel_recursion_leaf` then answers
|
|
17
|
+
`HEGEL_E_RETRY`, and `hegel.h` asks the caller to unwind out of its
|
|
18
|
+
generators "without drawing anything further", back to where it called
|
|
19
|
+
`hegel_new_recursion`, and to call `hegel_recursion_retry`. The retry closes
|
|
20
|
+
the spans the attempt left open and marks them discarded:
|
|
21
|
+
`libhegel/src/native/draws/mod.rs` records the span depth when the
|
|
22
|
+
recursion starts, and `recursion_retry` stops spans with `discard` set until
|
|
23
|
+
the depth is back there. After `RECURSION_MAX_ATTEMPTS`, 9, the retry
|
|
24
|
+
answers `HEGEL_E_ASSUME` and the test case is invalid. `hegel_recursion_finish`
|
|
25
|
+
can answer `HEGEL_E_RETRY` as well, when the finished value was mispriced;
|
|
26
|
+
the engine has then discarded the attempt itself, and the caller draws
|
|
27
|
+
again from the root without calling the retry.
|
|
28
|
+
|
|
29
|
+
hegel-rust unwinds with a panic (`frontends/rust/src/generators/recursive.rs`).
|
|
30
|
+
Its `draw_silent` closes a span only when the draw returns, so a panic
|
|
31
|
+
sends no `stop_span`, and the spans the attempt opened are still open when
|
|
32
|
+
the retry discards them.
|
|
33
|
+
|
|
34
|
+
Every compound generator in this gem closes its span in `ensure`. An
|
|
35
|
+
exception unwinding through them sends `stop_span(discard: false)` for each
|
|
36
|
+
span the abandoned attempt opened, and `discard: true` from a `filter`. By
|
|
37
|
+
the time the retry runs, those spans are closed and kept, and it finds
|
|
38
|
+
nothing above the recursion's depth to discard.
|
|
39
|
+
|
|
40
|
+
## Decision
|
|
41
|
+
|
|
42
|
+
`Hegel::TestCase#recursion_leaf` raises `Hegel::LeafBudgetExceeded` when the
|
|
43
|
+
leaf answers `HEGEL_E_RETRY`, and the test case keeps that exception until
|
|
44
|
+
`#recursion_retry`. While it holds it:
|
|
45
|
+
|
|
46
|
+
- `#stop_span` makes no native call, so the retry discards the spans the
|
|
47
|
+
attempt opened.
|
|
48
|
+
- Every other method that takes the test-case handle raises the same
|
|
49
|
+
exception again before any native call. A body that rescues `Exception`
|
|
50
|
+
and draws on therefore cannot extend an attempt the engine is about to
|
|
51
|
+
discard.
|
|
52
|
+
- A call that takes only the context still runs: `collection_free`,
|
|
53
|
+
`string_generator_free`, and `recursion_free`. hegel-rust's `Drop` runs the
|
|
54
|
+
same frees while its panic unwinds.
|
|
55
|
+
|
|
56
|
+
`LeafBudgetExceeded` descends from `Exception`, as `Hegel::StopTest` does,
|
|
57
|
+
and carries the recursion it belongs to. `RecursiveGenerator` rescues only
|
|
58
|
+
its own, so one raised in an outer recursive draw passes through an inner
|
|
59
|
+
one. After each attempt the generator asks the test case whether that
|
|
60
|
+
attempt was abandoned, rather than relying on having rescued the exception.
|
|
61
|
+
A body that swallowed it and returned a value is caught the same way, and
|
|
62
|
+
the value, built from draws the engine is about to discard, is dropped.
|
|
63
|
+
|
|
64
|
+
`#recursion_retry` reaches the engine with the raw handle. `HEGEL_OK` clears
|
|
65
|
+
the held exception. `HEGEL_E_ASSUME` or `HEGEL_E_STOP_TEST` ends the test
|
|
66
|
+
case, and its exception takes the held one's place, so a later call raises
|
|
67
|
+
that verdict without reaching the engine. hegel-rust's data source
|
|
68
|
+
short-circuits the same way once a test case is aborted
|
|
69
|
+
(`libhegel/src/native/data_source.rs`).
|
|
70
|
+
|
|
71
|
+
The held exception can reach the run loop only when something replaced it
|
|
72
|
+
mid-unwind, such as an `ensure` that raised, and the body rescued that and
|
|
73
|
+
drew again. `Hegel::Runner.classify` then marks the case INVALID. It is the
|
|
74
|
+
library's own control flow, and its attempt never reached the retry, so it
|
|
75
|
+
is not reported as a counterexample.
|
|
76
|
+
|
|
77
|
+
A subtree generator drawn after the recursive draw that made it has ended
|
|
78
|
+
raises `Hegel::Error`, since its recursion handle is already freed.
|
|
79
|
+
|
|
80
|
+
## Alternatives rejected
|
|
81
|
+
|
|
82
|
+
- **Roll the span depth back after `ensure` has closed the spans.** The
|
|
83
|
+
closes have already reached the engine as kept spans, and the ABI has no
|
|
84
|
+
call that reopens a span or marks a closed one discarded.
|
|
85
|
+
- **Close spans only when a draw returns, as hegel-rust does.** That changes
|
|
86
|
+
every exception path through every compound generator: a failed
|
|
87
|
+
assumption, an overrun, a failing assertion, and the span handling of a
|
|
88
|
+
stateful rule, which is written around closing in `ensure`.
|
|
89
|
+
- **Let draws return made-up values while the attempt is abandoned.** The
|
|
90
|
+
caller's code would run on values the engine never drew, could fail or
|
|
91
|
+
loop on them, and the header says to draw nothing further.
|
|
92
|
+
- **Unwind with `throw` and `catch`.** `throw` passes a `rescue Exception`
|
|
93
|
+
by, but it runs `ensure` blocks just as an exception does, so the span
|
|
94
|
+
closes still have to be suppressed. CLAUDE.md requires the library's
|
|
95
|
+
control flow to use exceptions descending from `Exception`, and with the
|
|
96
|
+
held exception in place a swallowed unwind cannot reach the engine anyway,
|
|
97
|
+
so `throw` adds nothing.
|
|
98
|
+
|
|
99
|
+
## Spans
|
|
100
|
+
|
|
101
|
+
hegel-rust draws each leaf and each branch generator through `draw_silent`,
|
|
102
|
+
which wraps it in a span labelled with that generator's label. This gem
|
|
103
|
+
opens a span only around a generator that makes more than one native call
|
|
104
|
+
itself (ADR 0020 and the `new-generator` skill), so the leaf and branch
|
|
105
|
+
spans were settled by measurement. Against libhegel 0.47.2, six
|
|
106
|
+
shrink-quality tests ported from `frontends/rust/tests/test_recursive.rs`
|
|
107
|
+
(shrink to a leaf, the minimal empty branch, the minimal binary branch, the
|
|
108
|
+
deep witness hoisted to the root at `max_depth` 3, the rare one at
|
|
109
|
+
`max_depth` 4, and the saturated budget) each shrank to the expected value
|
|
110
|
+
in 10 of 10 runs both with those spans and without them. The draw opens
|
|
111
|
+
neither.
|
|
112
|
+
|
|
113
|
+
The spans that remain are the recursive generator's own span, and a span
|
|
114
|
+
around every non-root subtree, all labelled with the recursive generator's
|
|
115
|
+
label. hegel-rust gives every depth that label so the shrinker can replace a
|
|
116
|
+
tree with one of its subtrees. `tuples` here opens no span per element, so
|
|
117
|
+
without the subtree's own span a subtree inside a `tuples` branch would
|
|
118
|
+
carry no span with that label. Under `arrays` the element span and the
|
|
119
|
+
subtree span nest with the same label.
|
|
120
|
+
|
|
121
|
+
None of the six tests shows these spans doing their job on this engine.
|
|
122
|
+
With the subtree span removed, with subtrees labelled apart from the
|
|
123
|
+
recursive generator, with the recursive generator's own span removed, and
|
|
124
|
+
with every `start_span` and `stop_span` in the library made a no-op, each
|
|
125
|
+
test still passed 10 of 10. Over eight seeded runs of the rare deep witness,
|
|
126
|
+
the number of test-body calls stayed within 3% of the same total
|
|
127
|
+
(14,488 to 14,899) across all of those variants. The span tests against the
|
|
128
|
+
Fake pin where the spans go, as they do for the other compound generators.
|
|
129
|
+
|
|
130
|
+
## Consequences
|
|
131
|
+
|
|
132
|
+
An attempt that outgrows its leaf budget costs one raised exception and its
|
|
133
|
+
backtrace, and the engine allows at most 9 attempts per draw. Each call
|
|
134
|
+
that takes the test-case handle checks one instance variable first.
|
|
135
|
+
|
|
136
|
+
Against libhegel 0.47.2, the port of hegel-rust's large-tree test, a tree of
|
|
137
|
+
`arrays` branches shrunk to exactly 30 leaves, called the body about 17,000
|
|
138
|
+
to 19,500 times and took 30 to 40 seconds. Each draw made about 240 native
|
|
139
|
+
calls, about 100 of them span opens and closes, and the time inside the
|
|
140
|
+
engine was about a quarter of the total.
|
|
141
|
+
|
|
142
|
+
The real-engine tests of `recursive` live in
|
|
143
|
+
`test/hegel/test_recursive_engine.rb`, which `rake mutation` leaves out.
|
|
144
|
+
With them in, the file took longer than mutineer's 10-second limit for one
|
|
145
|
+
mutant, and 16 of the mutants on the new lines timed out without a verdict.
|
|
146
|
+
The tests against the Fake, in `test/hegel/test_recursive.rb`, pin the
|
|
147
|
+
generator's calls exactly and stay in the mutation run.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# 0033: Cache compiled grammars for the length of a run
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
`from_lark` draws strings of a grammar written in Lark's syntax. The engine
|
|
10
|
+
compiles the grammar into a `hegel_grammar_t` with `hegel_grammar_new`,
|
|
11
|
+
after a `hegel_grammar_builder_t` has gathered the grammar and every file
|
|
12
|
+
its `%import`s name. `hegel.h` calls the compiled grammar immutable, safe to
|
|
13
|
+
share across test cases, and freed with `hegel_grammar_free`, which takes the
|
|
14
|
+
context. Every draw then reads it through `hegel_generate_grammar`.
|
|
15
|
+
|
|
16
|
+
Compiling costs far more than drawing. Measured against libhegel 0.47.2 on
|
|
17
|
+
x86_64 Linux, compiling `start: INT ("," INT)*` with `%import common.INT`,
|
|
18
|
+
which reads the bundled `common.lark`, took about 446 microseconds. With the
|
|
19
|
+
grammar already compiled, a test case that drew one string of it, capped at
|
|
20
|
+
20 characters, took about 38 microseconds, where a case that drew one integer
|
|
21
|
+
took 15. A grammar with no imports compiled in about 46 microseconds.
|
|
22
|
+
|
|
23
|
+
hegel-rust compiles once per generator: `LarkGenerator` keeps the grammar in
|
|
24
|
+
a `OnceLock`, and Rust's `Drop` frees it when the generator goes. Ruby has no
|
|
25
|
+
scope exit to hang that free on, and `CLAUDE.md` asks for every native
|
|
26
|
+
handle to be freed in an `ensure` in the code that owns the run, before the
|
|
27
|
+
context.
|
|
28
|
+
|
|
29
|
+
## Decision
|
|
30
|
+
|
|
31
|
+
`Hegel::Runner.run` creates a `Hegel::GrammarCache` inside the context's
|
|
32
|
+
block and hands it to every `Hegel::TestCase` of the run, the replay of a
|
|
33
|
+
`reproduce_failure:` blob included. An `ensure` frees every grammar in it
|
|
34
|
+
after `hegel_run_free` and before `hegel_context_free`. Freeing carries on
|
|
35
|
+
past a free that raises and raises the first error once every grammar has
|
|
36
|
+
been tried, so one failure does not leave the rest alive when the context
|
|
37
|
+
goes.
|
|
38
|
+
|
|
39
|
+
The cache is keyed by value: the grammar text, the start rule, `max_size`,
|
|
40
|
+
`max_depth`, the alphabet's options, the explicit terminal names in order,
|
|
41
|
+
and the import path as a string. That is everything the compilation reads.
|
|
42
|
+
The explicit generators are not in the key, since only their names reach the
|
|
43
|
+
engine. The generator builds the key when it is built, as ADR 0020 asks of a
|
|
44
|
+
label, so a draw writes nothing to it. Two generators equal by value share
|
|
45
|
+
one grammar, so a `from_lark` built inside the test body compiles once per
|
|
46
|
+
run as well.
|
|
47
|
+
|
|
48
|
+
A compilation that raises stores nothing, and frees what it had built. A
|
|
49
|
+
`Hegel::TestCase` built without a cache, as a unit test builds one, compiles
|
|
50
|
+
for each draw and frees the grammar after it.
|
|
51
|
+
|
|
52
|
+
The new-generator skill's rule that a native handle is never cached on the
|
|
53
|
+
generator still holds. This cache belongs to the run, and is the route a
|
|
54
|
+
handle that is worth keeping across draws takes.
|
|
55
|
+
|
|
56
|
+
Four alternatives were rejected.
|
|
57
|
+
|
|
58
|
+
- **Compile on every draw.** `CLAUDE.md` records a run configured for 20
|
|
59
|
+
test cases calling the body up to 1847 times, and an array of grammar
|
|
60
|
+
strings draws once per element. At 446 microseconds a compilation, that
|
|
61
|
+
dominates the run.
|
|
62
|
+
- **Cache per test case.** It still compiles once per case, so it saves
|
|
63
|
+
only the draws after the first within one case.
|
|
64
|
+
- **Cache on the generator, freed by a finalizer.** `CLAUDE.md` allows a
|
|
65
|
+
finalizer only as a backstop to an `ensure`. The free needs the context
|
|
66
|
+
the grammar came from, which is gone by the time a finalizer runs, and a
|
|
67
|
+
new context can reuse its address. A frozen generator cannot take a cache
|
|
68
|
+
written at draw time either.
|
|
69
|
+
- **Cache for the process on a long-lived context.** No code owns that
|
|
70
|
+
context with an `ensure` around it, and the engine's contexts are
|
|
71
|
+
single-threaded, so two threads running properties would share one.
|
|
72
|
+
|
|
73
|
+
## Consequences
|
|
74
|
+
|
|
75
|
+
An imported file is read once per run for each key, so a change to it shows
|
|
76
|
+
in the next run, not partway through one.
|
|
77
|
+
|
|
78
|
+
A grammar whose text is built from drawn data gets a new key for each
|
|
79
|
+
distinct text, and each stays compiled until the run ends. Such a run holds
|
|
80
|
+
one live grammar per distinct text it drew.
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
# 0034: Repeat a test body with a Ruby-side indent
|
|
2
|
+
|
|
3
|
+
## Status
|
|
4
|
+
|
|
5
|
+
Accepted.
|
|
6
|
+
|
|
7
|
+
## Context
|
|
8
|
+
|
|
9
|
+
hegel-rust's `TestCase::repeat` (`frontends/rust/src/test_case.rs:808` in
|
|
10
|
+
hegeldev/hegel) runs a closure "logically infinitely": until the engine
|
|
11
|
+
ends the loop, or until the closure fails. Under ADR 0015 this gem takes
|
|
12
|
+
the same name and the same meaning. Four parts of the Rust version need a
|
|
13
|
+
Ruby construction, because Ruby reaches the engine and reports a failure
|
|
14
|
+
differently.
|
|
15
|
+
|
|
16
|
+
What the Rust version does:
|
|
17
|
+
|
|
18
|
+
1. It draws the loop's minimum length silently, from `integers` up to
|
|
19
|
+
`max_safe_min_size`, which is `2**40` (`test_case.rs:811`). It opens a
|
|
20
|
+
collection with that minimum and no maximum, and loops while the
|
|
21
|
+
collection wants another element.
|
|
22
|
+
2. Each iteration notes `// Repetition #N` (`test_case.rs:819`) and runs
|
|
23
|
+
the closure under `enter_block(2)`, a handle from
|
|
24
|
+
`hegel_test_case_block`. The doc comment above the function says
|
|
25
|
+
`// Loop iteration N`; the code and the snapshots in
|
|
26
|
+
`frontends/rust/tests/test_loop.rs` say `// Repetition #N`, and this gem
|
|
27
|
+
follows the code.
|
|
28
|
+
3. A closure that panicked with `AssumeFailed` goes on to the next
|
|
29
|
+
iteration. `StopTest`, `InvalidArgument`, and `InternalError` unwind
|
|
30
|
+
unchanged. Any other panic draws one silent boolean and then unwinds.
|
|
31
|
+
4. When the collection says no more, it raises `LoopDone`
|
|
32
|
+
(`test_case.rs:842`). The function's return type is `!`: it never
|
|
33
|
+
returns. `run_lifecycle.rs:318` maps `LoopDone` to a VALID test case.
|
|
34
|
+
|
|
35
|
+
`hegel_test_case_block` (`libhegel/include/hegel.h:1589`) gives a handle
|
|
36
|
+
whose print region is a block nested in the parent's, each line indented
|
|
37
|
+
further. It indents what reaches the engine printer, and nothing else:
|
|
38
|
+
"everything printed or noted through the handle ... lands there". This gem
|
|
39
|
+
builds its failure report in Ruby (`Hegel::Report`, with names from
|
|
40
|
+
ADR 0005), and neither notes nor draws reach the engine printer. A block
|
|
41
|
+
handle would change nothing a caller sees.
|
|
42
|
+
|
|
43
|
+
## Decision
|
|
44
|
+
|
|
45
|
+
**`TestCase#repeat` never returns from the loop's end.** It raises
|
|
46
|
+
`Hegel::LoopDone`, a control exception beside `Hegel::StopTest` and
|
|
47
|
+
`Hegel::AssumeFailed`. It descends from `Exception`, as `CLAUDE.md` requires
|
|
48
|
+
of every control exception, so that a `rescue => e` written around the loop
|
|
49
|
+
cannot catch the end of it. `Hegel::Runner.classify` maps it to VALID, after
|
|
50
|
+
the four fatal exceptions and in the same group as the other control
|
|
51
|
+
exceptions.
|
|
52
|
+
|
|
53
|
+
**One iteration rescues in this order.** First the fatal exceptions
|
|
54
|
+
(`Hegel::FATAL_EXCEPTIONS`), `Hegel::StopTest`, `Hegel::LoopDone`, and
|
|
55
|
+
`Hegel::Error`: each re-raises unchanged. `Hegel::LoopDone` here is the
|
|
56
|
+
end of a nested loop, which ends the outer loop too, as the inner
|
|
57
|
+
`LoopDone` unwinds through the outer loop in Rust. `Hegel::Error` stands
|
|
58
|
+
where `InvalidArgument` and `InternalError` stand, since this gem raises
|
|
59
|
+
both kinds of error as `Hegel::Error`. Next `Hegel::AssumeFailed`, which
|
|
60
|
+
skips to the next iteration. Last, any other `Exception`: one boolean is
|
|
61
|
+
drawn, without recording, and the exception re-raises. The collection is
|
|
62
|
+
freed in an `ensure`, on every path out of the loop.
|
|
63
|
+
|
|
64
|
+
**The indent lives in the recorded entries.** Each entry
|
|
65
|
+
`Hegel::TestCase` records carries an indent, in columns:
|
|
66
|
+
`[:draw, name, value, indent]` and `[:note, message, indent]`.
|
|
67
|
+
`TestCase#repeat` raises the indent by 2 around each iteration's block,
|
|
68
|
+
after the `// Repetition #N` note, and `Hegel::Report` puts that many
|
|
69
|
+
spaces before the line. The rendered report reads:
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
// Repetition #1
|
|
73
|
+
n = 5
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
The indent is the one `enter_block(2)` gives. Nested loops add up, as
|
|
77
|
+
blocks do.
|
|
78
|
+
|
|
79
|
+
**`break` ends the loop.** A Ruby block can `break`, and Rust's closure has
|
|
80
|
+
no way to. The `break` returns from `TestCase#repeat` with its value. The
|
|
81
|
+
collection is freed and the indent restored on the way out, and the test
|
|
82
|
+
body goes on after the loop. Forbidding `break` would turn the ordinary way
|
|
83
|
+
out of a Ruby loop into an error. Taking it as the end of the loop gives it
|
|
84
|
+
the meaning it has in `loop`, `each`, and `while`.
|
|
85
|
+
|
|
86
|
+
Two smaller choices follow this gem's own rules:
|
|
87
|
+
|
|
88
|
+
- The minimum length is drawn with no span around it. hegel-rust's
|
|
89
|
+
`draw_silent` opens a span around every draw. This gem opens one only
|
|
90
|
+
around a generator that makes more than one native call
|
|
91
|
+
(`.claude/skills/new-generator`), and one `hegel_generate_integer` is a
|
|
92
|
+
single call.
|
|
93
|
+
- Draw names are suffixed over the whole test case, as
|
|
94
|
+
`Hegel::Report.assign_names` decides: a name gets a counter only when it
|
|
95
|
+
appears more than once in the report. A draw named `x` that appears once
|
|
96
|
+
stays `x`. hegel-rust's snapshots show `x_1` for the same case, because
|
|
97
|
+
its macro marks every draw inside a nested block as repeatable
|
|
98
|
+
(`frontends/rust/macros/src/common.rs:180`), and a repeatable name always
|
|
99
|
+
takes a counter.
|
|
100
|
+
|
|
101
|
+
## Consequences
|
|
102
|
+
|
|
103
|
+
A test body can hold a sequence of operations whose length the engine
|
|
104
|
+
chooses and shrinks, without a state machine. The tests in
|
|
105
|
+
`test/hegel/test_repeat.rb` port the four snapshots and the two run tests of
|
|
106
|
+
`test_loop.rs`, and pin the rescue order against the Fake.
|
|
107
|
+
|
|
108
|
+
The entry format has an indent in every entry. A later feature that wants
|
|
109
|
+
an indented section, such as a stateful rule's body under its step line,
|
|
110
|
+
can raise the same indent.
|
|
111
|
+
|
|
112
|
+
`break` is a construction this gem has and hegel-rust lacks. ADR 0015 keeps
|
|
113
|
+
mechanism outside its rule, and `break` is the loop exit a Ruby block
|
|
114
|
+
already has, so this record treats it as mechanism. If upstream settles a
|
|
115
|
+
way to end the loop early, this gem takes that and this record gets a
|
|
116
|
+
successor.
|
|
117
|
+
|
|
118
|
+
If this gem moves its report into the engine printer, `TestCase#repeat`
|
|
119
|
+
can open a `hegel_test_case_block` handle for each iteration in place of
|
|
120
|
+
the recorded indent.
|