hegeltest 0.3.4-aarch64-linux → 0.4.1-aarch64-linux

data/docs/adr/0036-follow-rubys-unicode-version.md

@@ -0,0 +1,50 @@
+# 0036: Follow Ruby's Unicode version
+
+## Status
+
+Accepted.
+
+## Context
+
+libhegel 0.48.0 adds `hegel_set_unicode_version`. It sets, for the whole
+process, the Unicode version that the text, regex, and grammar generators
+follow: the General Categories an alphabet is built from, and the `\w`,
+`\d`, `\s`, word boundaries, and case-insensitive matching of patterns. The
+engine has tables for Unicode 14.0.0 to 17.0.0. A binding that never calls it
+gets the newest.
+
+Ruby classifies characters by the Unicode version it was built with, which
+`RbConfig::CONFIG["UNICODE_VERSION"]` reports: 15.0.0 on Ruby 3.4. With the
+engine's 17.0.0 tables, `text(categories: ["Lu"])` can draw a character Ruby
+calls unassigned, and `from_regex` can draw a string Ruby's own `Regexp` reads
+differently. U+20C1 is one such character: a currency sign in 17.0.0, and
+unassigned in every earlier version.
+
+The Python frontend in hegeldev/hegel calls the function at import with
+`unicodedata.unidata_version`, for the same reason. The Rust frontend declares
+it in `frontends/rust/src/ffi/sys/fns.rs`; a search of `frontends/rust/src` at
+the `libhegel/0.48.0` tag finds that declaration as its one occurrence, so this
+gem follows the Python frontend here.
+
+## Decision
+
+`LibHegel::Real#initialize` passes `RbConfig::CONFIG["UNICODE_VERSION"]` to
+`hegel_set_unicode_version`, next to the engine version check, so that every
+generator built afterwards follows Ruby's tables. No keyword exposes it. ADR
+0015 leaves mechanism to each binding, and which host's tables the engine
+follows is mechanism: the interface stays the same, and a drawn value now
+means what Ruby takes it to mean.
+
+A version the engine has no tables for raises `Hegel::Error` when the engine
+is loaded. Ruby 3.4, the oldest Ruby this gem supports, reports 15.0.0, and
+the engine answers a version newer than its own with its newest tables, so
+no supported Ruby reaches that error.
+
+## Consequences
+
+On Ruby 3.4, characters assigned in Unicode 15.1.0 to 17.0.0 are drawn as
+unassigned (`Cn`) rather than in their new categories, as Ruby classifies
+them.
+
+The setting is process-wide, so a test that changes it puts back Ruby's
+version afterwards.

data/docs/adr/0037-send-a-maximum-at-the-engine-default-sentinel-as-no-bound.md

@@ -0,0 +1,48 @@
+# 0037: Send a maximum at the engine's default sentinel as no bound
+
+## Status
+
+Accepted.
+
+## Context
+
+libhegel 0.49.0 adds `HEGEL_MAX_SIZE_DEFAULT`, which is `UINT64_MAX - 1`.
+`hegel_new_collection`, `hegel_string_generator_text`, and
+`hegel_generate_bytes` read that `max_size` as "the caller gave no maximum",
+and apply the default of text and bytes: 100, or `min_size + 100` when
+`min_size` is above 100. `UINT64_MAX` continues to mean no upper bound. The
+engine changelog tells a frontend to pass `UINT64_MAX` for a user's explicit
+maximum at or above `HEGEL_MAX_SIZE_DEFAULT`.
+
+Before this change, the gem sent a caller's `max_size` to the engine as
+it is. Measured on linux-amd64 with Ruby 3.4.10 against 0.49.0, without this
+change, 300 cases of `binary(min_size: 200, max_size: 2**64 - 2)` had 200 to
+300 bytes: the engine applied `min_size + 100`. With this change, the same
+draw had 200 to 1465 bytes. A Ruby `Integer` has no upper limit, so a caller
+can also write a maximum above `UINT64_MAX`. Without this change,
+`binary(max_size: 2**70)` raised `RangeError` from the `ffi` gem (measured on
+the same setup).
+
+The Rust frontend declares the constant in
+`frontends/rust/src/ffi/sys/types.rs` at the `libhegel/0.49.0` tag. Its text
+generator, in `frontends/rust/src/generators/strings.rs`, continues to pass
+its own default of 100 (inferred from reading that file).
+
+## Decision
+
+`Hegel::LibHegel.max_size_arg` changes a `max_size` at or above
+`HEGEL_MAX_SIZE_DEFAULT` to `UINT64_MAX`. `Hegel::TestCase#new_collection`,
+`#with_text_generator`, and `#generate_bytes` apply it, so every generator
+that sizes a draw through the engine sends the caller's maximum this way.
+
+The gem continues to compute its own default maximum of text and bytes
+(`Hegel::Generators.default_max_size`) and to send it as a number. It does not
+send the sentinel. The default is the same number, and a reader of the
+generator then sees the default in Ruby.
+
+## Consequences
+
+A maximum at or above `2**64 - 2` means no upper bound. No allocation can
+reach a size of that order, so a caller who writes one gets the meaning
+they wrote. A maximum above `2**64 - 1` also means no upper bound now, where it
+raised `RangeError` before.

data/docs/architecture.md

@@ -18,7 +18,7 @@ recorded, and prints a blob that `Hegel.test(reproduce_failure:)` replays.
 The report comes from the cases the engine stamps for capture during the
 run; see
 [ADR 0019](adr/0019-report-failures-from-the-cases-the-engine-stamps.md).
-The generator layer covers twenty-five generators composing through `map`
+The generator layer covers thirty-four generators composing through `map`
 and `filter`, reachable bare through `Hegel::Syntax::Methods`. A test case
 can discard itself with `tc.assume` or `tc.reject`. A run can be shaped by
 `phases:`, `suppress_health_check:`, `report_multiple_failures:`, and the
@@ -121,6 +121,12 @@ test-case handle, in nested `ensure`s so a failure releasing one cannot skip
 the other. See
 [ADR 0011](adr/0011-let-the-test-case-own-every-pool-drawn-from-it.md).
 
+A compiled grammar outlives the test case that compiled it, since compiling
+costs far more than drawing. `Hegel::Runner.run` keeps the run's grammars
+in a `Hegel::GrammarCache` and frees them after the run and before the
+context. See
+[ADR 0033](adr/0033-cache-compiled-grammars-for-the-length-of-a-run.md).
+
 ## A rule's own control flow
 
 Inside a stateful rule, `tc.assume(false)` means something narrower than it
@@ -135,6 +141,15 @@ exception can be raised inside an open span, which is why
 than beside the run loop: two `rescue Exception` sites need it now, and both
 have to let those four straight through before anything else.
 
+## An abandoned recursive attempt
+
+`recursive` unwinds through open spans on purpose. When an attempt outgrows its leaf budget, `Hegel::TestCase` raises
+`Hegel::LeafBudgetExceeded` and holds it until `hegel_recursion_retry`.
+Meanwhile `stop_span` makes no native call, so the retry can discard the
+spans the attempt opened, and every other call that takes the test-case
+handle raises the held exception again instead of reaching the engine. See
+[ADR 0032](adr/0032-suppress-span-closes-while-an-abandoned-recursion-attempt-unwinds.md).
+
 ## Generator validation happens at draw time
 
 hegel-rust's own contributor documentation states the rule for its

data/lib/hegel/errors.rb

@@ -31,6 +31,30 @@ module Hegel
   # Descends from Exception for the same reason as StopTest above.
   class AssumeFailed < Exception; end
 
+  # Raised for HEGEL_E_RETRY from hegel_recursion_leaf: the attempt at a
+  # recursive value outgrew its leaf budget and must unwind to the
+  # Generators::RecursiveGenerator that owns +recursion+, which retries it.
+  # Internal to this library. See docs/adr/0032.
+  class LeafBudgetExceeded < Exception # :nodoc:
+    attr_reader :recursion
+
+    def initialize(recursion)
+      super("hegel: a recursive draw outgrew its leaf budget; the attempt is retried")
+      @recursion = recursion
+    end
+  end
+
+  # Raised by Hegel::TestCase#repeat when the engine ends its loop: the
+  # repetition ran as long as the engine wanted it to and nothing failed,
+  # so Hegel::Runner.classify reports the case VALID. hegel-rust raises its
+  # own LoopDone at the same point, and its run lifecycle maps it to VALID
+  # the same way.
+  #
+  # Descends from Exception for the same reason as StopTest above: a
+  # `rescue => e` written after the loop must not turn the end of the loop
+  # into ordinary control flow. See docs/adr/0034.
+  class LoopDone < Exception; end
+
   # standard:enable Lint/InheritException
 
   # Say the process is ending, not that a property failed. Every

data/lib/hegel/generator.rb

@@ -18,6 +18,41 @@ module Hegel
       raise NotImplementedError, "#{self.class} must implement #do_draw"
     end
 
+    # The span label of one attempt at a filtered draw, shared by every
+    # generator that runs attempts.
+    FILTER_ATTEMPT_LABEL = LibHegel.label_from_name("hegel-ruby.Hegel::Generator.filter_attempt")
+
+    # Draws one value that +predicate+ accepts, the draw behind #filter, which
+    # calls this on its source generator. The default makes up to
+    # Filtered::MAX_ATTEMPTS attempts, each a #do_draw in a span of its own
+    # that closes as discarded when the predicate rejects the value, then
+    # discards the test case (Hegel::AssumeFailed).
+    #
+    # A generator whose draws have effects a rejected value must not keep
+    # overrides this and runs the attempts itself, as
+    # Hegel::Stateful::Pool::ValuesConsumed does. One that only forwards
+    # #do_draw to another generator forwards this too, so that a filter around
+    # it still reaches the other generator's attempts.
+    def do_draw_filtered(tc, predicate)
+      Filtered::MAX_ATTEMPTS.times do
+        # Each variable is set again before it is read, or stays nil, which
+        # acts like false (!nil == !false).
+        # The initializer also shows intent: discard the span when an
+        # exception skips the reassignment.
+        accepted = false # mutineer:disable-line statement_removal
+        value = nil # mutineer:disable-line statement_removal, boolean_literal
+        tc.start_span(FILTER_ATTEMPT_LABEL)
+        begin
+          value = do_draw(tc)
+          accepted = predicate.call(value)
+        ensure
+          tc.stop_span(discard: !accepted)
+        end
+        return value if accepted
+      end
+      raise Hegel::AssumeFailed
+    end
+
     # The span label naming this generator's draws to the engine, which
     # treats two spans with one label as draws of one generator when it
     # shrinks. A generator built from others overrides this and combines
@@ -30,7 +65,7 @@ module Hegel
     # The class name, prefixed with this binding's name as hegel.h asks, to
     # keep clear of the engine's own hegel.<kind> labels.
     def self.label
-      LibHegel.label_from_name("hegel-ruby.#{name}")
+      @label ||= LibHegel.label_from_name("hegel-ruby.#{name}")
     end
 
     # Combines this generator's class label with the labels of +parts+, the
@@ -92,13 +127,17 @@ module Hegel
       end
     end
 
-    # Generator#filter's result. Retries the source generator, each
-    # attempt in its own span, up to MAX_ATTEMPTS times before discarding
-    # the test case (Hegel::AssumeFailed) -- the same budget hegel-rust's
-    # Filtered spends before calling assume(false)
-    # (src/generators/generators.rs), ported here rather than reinvented so
-    # a predicate that (almost) never holds fails the same way in both
-    # bindings.
+    # Generator#filter's result. Hands its predicate to the source
+    # generator's #do_draw_filtered, which makes up to MAX_ATTEMPTS attempts
+    # before discarding the test case (Hegel::AssumeFailed) -- the same
+    # budget hegel-rust's Filtered spends before calling assume(false)
+    # (frontends/rust/src/generators/generators.rs), ported here rather than
+    # reinvented so a predicate that (almost) never holds fails the same way
+    # in both bindings.
+    #
+    # A filter over a filter combines the two predicates into one draw, so
+    # the pair spends MAX_ATTEMPTS attempts between them, not that many per
+    # level.
     #
     # A predicate that never holds does not loop forever: libhegel's own
     # health check aborts a run that discards too many test cases in a
@@ -116,23 +155,11 @@ module Hegel
       attr_reader :label
 
       def do_draw(tc)
-        MAX_ATTEMPTS.times do
-          # Each variable is set again before it is read, or stays nil, which
-          # acts like false (!nil == !false).
-          # The initializer also shows intent: discard the span when an
-          # exception skips the reassignment.
-          accepted = false # mutineer:disable-line statement_removal
-          value = nil # mutineer:disable-line statement_removal, boolean_literal
-          tc.start_span(label)
-          begin
-            value = @source.do_draw(tc)
-            accepted = @block.call(value)
-          ensure
-            tc.stop_span(discard: !accepted)
-          end
-          return value if accepted
-        end
-        raise Hegel::AssumeFailed
+        @source.do_draw_filtered(tc, @block)
+      end
+
+      def do_draw_filtered(tc, predicate)
+        @source.do_draw_filtered(tc, ->(value) { @block.call(value) && predicate.call(value) })
       end
     end
   end