hegeltest 0.1.1 → 0.2.1

data/docs/adr/0019-report-failures-from-the-cases-the-engine-stamps.md

@@ -0,0 +1,70 @@
+# 0019: Report failures from the cases the engine stamps
+
+## Status
+
+Accepted.
+
+## Context
+
+Up to libhegel 0.32.5, a run ended with its failures shrunk, and the
+binding built each report itself. After the run, it read each failure's
+reproduction blob, rebuilt a test case from it with
+`hegel_test_case_from_blob`, and ran the body against that case once more
+while it recorded the drawn values.
+
+libhegel 0.44.0 changed both halves. The engine now runs every failure it
+reports once more before the run ends, inside the loop the binding drives.
+It stamps that case, and a few others, through
+`hegel_test_case_should_capture`. The header asks a binding to keep the
+output of a stamped failing case under its origin, and to report each
+failure from its freshest capture instead of replaying its blob after the
+run. A failure can also arrive with no blob: under nondeterministic
+handling, an unconfirmed failure has none, and it carries a caveat
+(`hegel_failure_caveat`) that quotes the engine's replay evidence.
+`hegel_run_start_blob` replays a blob as a run, and the header names it as
+the call a reproduce-failure feature should use.
+
+hegel-rust and hegel-java follow this design. hegel-go, hegel-cpp, and
+hegel-typescript still replay the blob after the run; each pins an engine
+from before 0.44.0, or has not moved its report yet.
+
+## Decision
+
+The runner reads the stamp when each case starts. Only a stamped case
+records its draws. A failing case keeps its exception and its entries
+under its origin. A newer capture replaces an older one, except that an
+unstamped case never replaces a stamped one. hegel-rust ranks its captures
+the same way: a newer capture replaces an older one of the same rank or
+lower.
+
+After the run, each failure is reported from the capture under its
+origin. No blob is replayed.
+
+A failure with no blob is reported without the reproduction line. A
+failure with a caveat prints it as a `note:` line after the drawn values,
+as hegel-rust does.
+
+`Hegel.test(reproduce_failure:)` starts its run with
+`hegel_run_start_blob` and drives it through the same loop. A replay that
+never fails raises `Hegel::Error` that names both causes: a fixed bug, or
+a nondeterministic body whose failure did not recur.
+
+Keeping the replay after the run was refused. It ran the body once more
+per failure on top of the engine's own final replay. It also had no blob
+to replay for an unconfirmed failure, so a nondeterministic body would have
+raised an internal error in place of a report.
+
+## Consequences
+
+A failing run calls the body once fewer per failure. The cost of naming a
+drawn value, a read of the caller's source, falls on the stamped cases
+only. Measured against libhegel 0.45.0, a run that calls its body 60 to
+80 times stamps 5 of them.
+
+A `tc.note` block runs on every stamped case, which can be more than one
+per failure: the engine also stamps the first replays that check a newly
+found failure.
+
+`hegel_test_case_from_blob` has no caller left, and its binding is gone.
+The sixth layer of verification in ADR 0006 lists "blob replay", which now
+means the replay that `hegel_run_start_blob` drives.

data/docs/adr/0020-derive-span-labels-from-generator-names.md

@@ -0,0 +1,64 @@
+# 0020: Derive span labels from generator names
+
+## Status
+
+Accepted.
+
+## Context
+
+`hegel_start_span` takes a label. Up to libhegel 0.38, `hegel.h` defined a
+table of `HEGEL_LABEL_*` constants, and this binding passed the matching
+constant for each kind of compound draw: a list, a list element, a map
+entry, and so on.
+
+libhegel 0.39.0 removed the table. A label is now an opaque u64 that names
+the generator which opened the span. The engine treats two spans with one
+label as draws of one generator, and swaps, duplicates, or reorders them
+when it shrinks and mutates a test case. The header asks a binding to
+derive labels from names of its own, prefixed with the binding's name to
+keep clear of the engine's own `hegel.<kind>` names. It asks a generator
+built from other generators to combine its own label with its components'
+labels, so that `lists(integers())` and `lists(text())` differ.
+`hegel_label_from_name` is 64-bit FNV-1a over the name's bytes, and the
+header says a binding may compute it ahead of time. hegel-rust computes
+both functions in the language and pins its hash against the engine's.
+
+The other implementations differ on names and on combining. hegel-java
+hashes `dev.hegel.*` names in Java, hegel-cpp calls
+`hegel_label_from_name` with `hegel-cpp.*` names, hegel-go hashes the Go
+type, and hegel-typescript passes small integers. Labels are not part of
+any implementation's public interface.
+
+## Decision
+
+A generator's label comes from its class name with a `hegel-ruby.`
+prefix. A generator built from others combines its own class label with
+the labels of the generators it draws from, in order. Both functions are
+computed in Ruby, with the same hash the header defines, and a test pins
+them against vectors measured from the engine's own functions.
+
+A compound generator computes its label once, when it is built, and keeps
+it, as hegel-rust's generators do. Two alternatives were refused. Computing
+the label on every draw slowed a run of nested generators by more than
+twenty times. Computing it on the first draw and keeping it writes to the
+generator at draw time, so a generator the caller froze raised
+`FrozenError` on every draw. A component that is not a generator
+contributes 0 to the label, so building around it does not raise; the
+draw reports the mistake, where a generator validates its arguments.
+
+A deferred generator answers with its installed generator's label, or its
+own class label before `set`. A generator built around it before `set`
+keeps that class label, so a self-referential definition has no cycle to
+follow. hegel-rust's deferred generator answers the same way.
+
+The span sites stay where they were: around each compound draw, and around
+each element of a collection, labelled with the element generator's own
+label.
+
+## Consequences
+
+`Hegel::LibHegel` loses the `HEGEL_LABEL_*` constants and gains
+`label_from_name` and `label_combine`. A generator class written outside
+this library gets a label from its own class name with no extra code.
+Renaming a generator class changes its label, which affects nothing past
+the run that uses it.

data/docs/adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md

@@ -0,0 +1,59 @@
+# 0021: Run a state machine in rounds, with its own step count
+
+## Status
+
+Accepted. Supersedes the `stateful_step_count:` keyword of `Hegel.test`.
+
+## Context
+
+Up to libhegel 0.32.5, a state machine had one call to pick the next rule,
+and the step budget was a run setting,
+`hegel_settings_set_stateful_step_count`, with an engine default of 50.
+This binding exposed it as `Hegel.test(stateful_step_count:)`, and ran
+every invariant after every rule that completed.
+
+libhegel 0.33 to 0.43 changed the protocol. A machine runs in rounds:
+`hegel_state_machine_next_group` starts each round, including the first,
+and `hegel_state_machine_next_rule` takes a worker index. The step count
+became an argument of `hegel_new_state_machine`, and the engine has no
+default for it. Between rounds, `hegel_state_machine_should_check_invariant`
+samples each invariant with probability 1 / step_count, so it runs about
+once in a test case of full length; a flag at creation makes an invariant
+run at every round instead. The caller runs every invariant on the initial
+state and on the final state. The machine can also run concurrently, with
+rule groups and weights.
+
+hegel-rust (`Machine::steps`), hegel-go (`WithStatefulStepCount`), and
+hegel-java (`Stateful.Options#stepCount`) put the step count on the
+machine runner, with a default of 50. hegel-cpp keeps it in its settings.
+hegel-go, hegel-cpp, and hegel-java sample invariants through the engine.
+hegel-rust and hegel-java name the opt-out `always_run` and `alwaysRun`.
+
+## Decision
+
+`Hegel::Stateful.run(machine, tc, step_count: 50)` takes the step count,
+and `Hegel.test` no longer takes `stateful_step_count:`. The keyword name
+follows the engine's argument; 50 follows the three implementations that
+moved it.
+
+`invariant(name, always_run: true)` checks an invariant at every round.
+Without it, the engine samples the invariant between rounds. Every
+invariant runs before the first rule and after the last.
+
+The machine is sequential: one group, equal weights, a concurrency of 1,
+and worker index 0. This binding drives the engine from one thread, and a
+concurrent machine needs one worker thread per drawn level of concurrency.
+Rule weights stay out until a need for them shows up.
+
+Keeping `stateful_step_count:` on `Hegel.test` was refused. It would carry
+a value through the runner and the test case to a call that the stateful
+module alone makes, and three of the four implementations that moved it put
+it on the runner.
+
+## Consequences
+
+A caller who passed `stateful_step_count:` passes `step_count:` to
+`Hegel::Stateful.run` instead. An invariant that a test relied on to run
+after every rule needs `always_run: true`. A machine whose invariants are
+sampled finds a broken invariant at the next sampled round, or at the
+final check, which the shrinker then moves toward the step that broke it.

data/docs/adr/0022-keep-microsecond-times-over-a-nanosecond-engine.md

@@ -0,0 +1,45 @@
+# 0022: Keep microsecond times over a nanosecond engine
+
+## Status
+
+Accepted.
+
+## Context
+
+libhegel 0.36.0 changed `hegel_time_t` from a microsecond field to a
+nanosecond field, and `hegel_generate_time` and `hegel_generate_datetime`
+now draw whole nanoseconds. The struct kept its layout, so a binding that
+does not change compiles and binds as before, and then reads nanoseconds as
+microseconds.
+
+`times` returns a `"HH:MM:SS.ffffff"` String and `datetimes` a `Time`, each
+with microsecond precision. The other implementations disagree on what a
+drawn time shows: hegel-rust prints nine digits whenever the nanosecond is
+not zero, hegel-typescript prints six digits for a whole microsecond and
+nine otherwise, and hegel-cpp truncates to microseconds. hegel-go and
+hegel-java return their languages' typed values.
+
+## Decision
+
+`times` and `datetimes` keep microsecond precision. A lower bound's
+microsecond becomes its first nanosecond, and an upper bound's its last,
+so the engine draws over every nanosecond inside the bounds. The drawn
+nanosecond rounds down to a microsecond, so every microsecond in range
+stays reachable, both bounds included. The engine does not draw
+nanoseconds uniformly, so the microseconds are not equally likely either:
+measured against libhegel 0.45.0, two runs of 2000 draws over four
+microseconds gave 1437, 284, 162, and 117, and 1428, 296, 153, and 123.
+
+A `datetimes` bound with a fraction of a microsecond narrows to the whole
+microseconds inside the range: the lower bound rounds up and the upper
+rounds down.
+
+Moving to nanosecond output was refused while the implementations do not
+agree on its format. A change to it belongs upstream first, as ADR 0015
+requires.
+
+## Consequences
+
+The public format of `times` and the precision of `datetimes` stay as they
+were. The binding layer speaks nanoseconds, and the conversion lives in
+`Hegel::Generators.first_nanosecond` and `last_nanosecond`.

data/docs/adr/0023-leave-unset-settings-to-the-engines-profile.md

@@ -0,0 +1,50 @@
+# 0023: Leave unset settings to the engine's profile
+
+## Status
+
+Accepted. Narrows the `nil` rule in the Consequences of
+[ADR 0009](0009-turn-the-example-database-on-with-a-key.md).
+
+## Context
+
+A `Hegel.test` keyword left `nil` calls no setter, so the engine's value
+applies. Up to libhegel 0.32.5, that value was a fixed default.
+
+libhegel 0.40.0 moved defaults into named profiles that the engine
+resolves. `hegel_settings_new` resolves the default profile: one named by
+`hegel_set_default_profile`, `HEGEL_DEFAULT_PROFILE`, or a `hegel.toml`;
+otherwise `ci` on a CI server and `development` elsewhere. libhegel 0.43.4
+then applies `HEGEL_TEST_CASES`, `HEGEL_DATABASE`, `HEGEL_SEED`, and the
+other `HEGEL_*` variables on top. The shipped `ci` profile derandomizes the
+run, turns the example database off, and suppresses the TooSlow health
+check. `hegel_settings_new` can now fail, for a malformed `hegel.toml` or
+`HEGEL_*` variable.
+
+hegel-rust builds each run's settings from the reserved `base` profile and
+overwrites every field from its own `Settings` value, which it resolved from
+the default profile earlier. hegel-go, hegel-cpp, hegel-java, and
+hegel-typescript call `hegel_settings_new`.
+
+## Decision
+
+The binding keeps calling `hegel_settings_new` and keeps the `nil` rule.
+A keyword left `nil` now takes its value from the resolved profile and the
+`HEGEL_*` variables. A keyword the caller passes still wins, because its
+setter runs after the engine resolved the profile. A failure of
+`hegel_settings_new` raises `Hegel::Error` with the engine's message.
+
+`report_multiple_failures:` still defaults to `false` and is always
+passed, so a profile cannot turn a re-raised exception into a count.
+
+Starting from `base` as hegel-rust does was refused. Ruby has no settings
+object to resolve a profile into, so the binding would have to read every
+field back through the `hegel_settings_get_*` calls to reproduce what
+`hegel_settings_new` already does.
+
+## Consequences
+
+A project can shape every run through `hegel.toml` or `HEGEL_*` variables
+without changing a call. On a CI server, a run with `database_key:` and no
+`database:` stores nothing, because the `ci` profile turns the database
+off; passing `database:` turns it back on. A run on CI is derandomized
+unless it passes `derandomize: false`.

data/docs/adr/0024-release-when-a-merge-changes-the-gem-version.md

@@ -0,0 +1,49 @@
+# 0024: Release when a merge changes the gem version
+
+## Status
+
+Accepted. Replaces the tag push that started the Release workflow.
+
+## Context
+
+A release took three steps from the maintainer: merge the pull request that
+prepares the version, push the tag `v<version>` on the merged commit, and
+approve the `release` environment. The tag carried no decision of its own.
+The version in `lib/hegel/version.rb` already said what to release, and the
+approval was the real gate.
+
+A daily routine now opens a pull request for each new libhegel release, with
+a "Prepare <version>" commit. The maintainer's part should be the decision,
+not the steps around it.
+
+GitHub does not start a workflow from a push that a workflow makes with its
+own `GITHUB_TOKEN`. A workflow that pushed the tag after a merge would
+therefore not start a tag-triggered Release workflow.
+
+## Decision
+
+The Release workflow starts on a push to `main` that changes
+`lib/hegel/version.rb`. Its first job reads the version and looks for the tag
+`v<version>`. When the tag exists, as after a revert to a released version,
+the run stops there. Otherwise the publish job waits for the `release`
+environment approval, then builds and pushes the gems from the merged
+commit. The last job creates the tag on that commit and the GitHub release
+together, with `gh release create --target`.
+
+The tag is created last, after the gems are out. A rejected approval then
+leaves no tag behind, and rerunning the workflow run releases the same
+commit later.
+
+Two alternatives were refused. A workflow that only pushed the tag needs a
+personal access token or a GitHub App token to start the tag-triggered
+workflow, which is a long-lived secret for one step. Creating the tag before
+the approval leaves a tag for a version that may never ship, and the check
+job would then skip that version for good.
+
+## Consequences
+
+A release is one merge and one approval. Merge a pull request that changes
+the gem version only when that version should ship.
+
+A tag pushed by hand no longer starts a release. The workflow's own run on
+`main` is the place to retry one.

data/docs/architecture.md

@@ -15,6 +15,9 @@ wrapping the engine's whole draw surface, and the run loop.
 What exists beyond that: the failure report, which names drawn values by
 reading the caller's own source, interleaves them with whatever `tc.note`
 recorded, and prints a blob that `Hegel.test(reproduce_failure:)` replays.
+The report comes from the cases the engine stamps for capture during the
+run; see
+[ADR 0019](adr/0019-report-failures-from-the-cases-the-engine-stamps.md).
 The generator layer covers twenty-five generators composing through `map`
 and `filter`, reachable bare through `Hegel::Syntax::Methods`. A test case
 can discard itself with `tc.assume` or `tc.reject`. A run can be shaped by
@@ -27,9 +30,11 @@ database keywords.
 `Hegel::Stateful.run` drives a `Hegel::StateMachine`'s rules and invariants,
 drawing values an earlier rule produced back out of a
 `Hegel::Stateful::Pool`. [ADR 0010](adr/0010-declare-stateful-rules-with-a-class-macro.md)
-decides how a machine declares its rules and
+decides how a machine declares its rules,
 [ADR 0011](adr/0011-let-the-test-case-own-every-pool-drawn-from-it.md) who
-frees a pool.
+frees a pool, and
+[ADR 0021](adr/0021-run-a-state-machine-in-rounds-with-its-own-step-count.md)
+how its rounds, step count, and invariants run.
 
 Every feature this binding set out to cover now has a Ruby surface. What
 remains open is not a feature but a measurement, and
@@ -116,9 +121,9 @@ the other. See
 ## A rule's own control flow
 
 Inside a stateful rule, `tc.assume(false)` means something narrower than it
-does in a test body. `Hegel::Stateful` catches it, tells libhegel the rule
-was rejected so the attempt does not spend a step, and draws another rule;
-the test case continues. `Hegel::Runner.classify` never sees it, and no case
+does in a test body. `Hegel::Stateful` catches it and tells libhegel the
+rule was rejected, so its round does not count toward the step budget, and
+the machine starts another round; the test case continues. `Hegel::Runner.classify` never sees it, and no case
 is discarded.
 
 A rule is also the one place in this library where a process-ending
@@ -168,7 +173,7 @@ fields.
 
 A drawn string arrives as a pointer and a length, is not NUL-terminated,
 and can hold interior NUL bytes, because the drawn alphabet can include
-U+0000. Read by length. Measured against 0.32.5: an alphabet pinned to
+U+0000. Read by length. Measured against 0.45.0: an alphabet pinned to
 U+0000 produces a three-byte draw that reads as `""` when taken up to the
 first NUL and as three NUL bytes when taken by its reported length. Every
 string observed reports `valid_encoding?` after `force_encoding(UTF-8)`.

data/lib/hegel/draw_name.rb

@@ -116,7 +116,9 @@ module Hegel
     def encloses?(outer, inner)
       return false if outer.equal?(inner)
 
-      outer.location.start_offset <= inner.location.start_offset &&
+      # An outer assignment's start offset never equals an inner one's, since
+      # the outer node begins with the name first.
+      outer.location.start_offset <= inner.location.start_offset && # mutineer:disable-line comparison
         inner.location.end_offset <= outer.location.end_offset
     end
   end

data/lib/hegel/generator.rb

@@ -18,10 +18,43 @@ module Hegel
       raise NotImplementedError, "#{self.class} must implement #do_draw"
     end
 
+    # The span label naming this generator's draws to the engine, which
+    # treats two spans with one label as draws of one generator when it
+    # shrinks. A generator built from others overrides this and combines
+    # its own class label with theirs, so that arrays of integers and
+    # arrays of strings do not share a label.
+    def label
+      self.class.label
+    end
+
+    # The class name, prefixed with this binding's name as hegel.h asks, to
+    # keep clear of the engine's own hegel.<kind> labels.
+    def self.label
+      LibHegel.label_from_name("hegel-ruby.#{name}")
+    end
+
+    # Combines this generator's class label with the labels of +parts+, the
+    # generators it draws from. A compound generator calls this once, when it
+    # is built, and keeps the result: computing it on every draw slowed a
+    # run of nested generators by more than twenty times, and keeping it
+    # from the first draw would write to a generator a caller may have
+    # frozen.
+    def combined_label(*parts)
+      LibHegel.label_combine([self.class.label, *parts.map { |part| Generator.label_of(part) }])
+    end
+    private :combined_label
+
+    # +part+'s label, or 0 for a part that is not a generator. Such a part
+    # fails when it is drawn, where a generator validates its arguments, so
+    # building the generator around it must not fail first.
+    def self.label_of(part)
+      part.respond_to?(:label) ? part.label : 0
+    end
+
     # Returns a new Generator whose #do_draw runs +block+ on the value this
-    # generator drew, spanned with HEGEL_LABEL_MAPPED so the shrinker
-    # treats the source draw and the transform as one unit (see
-    # hegel-rust's Mapped, src/generators/generators.rs).
+    # generator drew, inside one span so the shrinker treats the source draw
+    # and the transform as one unit (see hegel-rust's Mapped,
+    # src/generators/generators.rs).
     def map(&block)
       Mapped.new(self, block)
     end
@@ -39,13 +72,15 @@ module Hegel
     # generators.rb.
     class Mapped < Generator
       def initialize(source, block)
-        super()
         @source = source
         @block = block
+        @label = combined_label(@source)
       end
 
+      attr_reader :label
+
       def do_draw(tc)
-        tc.start_span(LibHegel::HEGEL_LABEL_MAPPED)
+        tc.start_span(label)
         @block.call(@source.do_draw(tc))
       ensure
         # discard is always false here: a span is only ever marked
@@ -73,16 +108,22 @@ module Hegel
       MAX_ATTEMPTS = 3
 
       def initialize(source, block)
-        super()
         @source = source
         @block = block
+        @label = combined_label(@source)
       end
 
+      attr_reader :label
+
       def do_draw(tc)
         MAX_ATTEMPTS.times do
-          accepted = false
-          value = nil
-          tc.start_span(LibHegel::HEGEL_LABEL_FILTER)
+          # Each variable is set again before it is read, or stays nil, which
+          # acts like false (!nil == !false).
+          # The initializer also shows intent: discard the span when an
+          # exception skips the reassignment.
+          accepted = false # mutineer:disable-line statement_removal
+          value = nil # mutineer:disable-line statement_removal, boolean_literal
+          tc.start_span(label)
           begin
             value = @source.do_draw(tc)
             accepted = @block.call(value)