pbt 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. checksums.yaml +4 -4
  2. data/.standard.yml +1 -1
  3. data/CHANGELOG.md +15 -0
  4. data/CLAUDE.md +2 -2
  5. data/README.md +94 -17
  6. data/benchmark/FIBER.md +97 -0
  7. data/benchmark/fiber_experiment.rb +39 -0
  8. data/benchmark/fiber_scheduler/Gemfile +7 -0
  9. data/benchmark/fiber_scheduler/experiment.rb +131 -0
  10. data/benchmark/fiber_scheduler/spec/experiment_spec.rb +38 -0
  11. data/docs/plans/fiber-experiment.md +12 -0
  12. data/docs/plans/generator-statistics.md +16 -0
  13. data/docs/plans/stateful-pbt-redesign.md +415 -0
  14. data/lib/pbt/arbitrary/arbitrary.rb +63 -10
  15. data/lib/pbt/arbitrary/arbitrary_methods.rb +51 -24
  16. data/lib/pbt/arbitrary/array_arbitrary.rb +45 -11
  17. data/lib/pbt/arbitrary/choose_arbitrary.rb +27 -0
  18. data/lib/pbt/arbitrary/constant.rb +0 -1
  19. data/lib/pbt/arbitrary/constant_arbitrary.rb +5 -0
  20. data/lib/pbt/arbitrary/constant_from_arbitrary.rb +39 -0
  21. data/lib/pbt/arbitrary/filter_arbitrary.rb +40 -2
  22. data/lib/pbt/arbitrary/fixed_hash_arbitrary.rb +16 -0
  23. data/lib/pbt/arbitrary/float_arbitrary.rb +31 -0
  24. data/lib/pbt/arbitrary/frequency_arbitrary.rb +61 -0
  25. data/lib/pbt/arbitrary/integer_arbitrary.rb +11 -0
  26. data/lib/pbt/arbitrary/map_arbitrary.rb +40 -2
  27. data/lib/pbt/arbitrary/one_of_arbitrary.rb +46 -13
  28. data/lib/pbt/arbitrary/sample.rb +97 -0
  29. data/lib/pbt/arbitrary/tuple_arbitrary.rb +22 -0
  30. data/lib/pbt/check/case.rb +1 -1
  31. data/lib/pbt/check/configuration.rb +5 -3
  32. data/lib/pbt/check/property.rb +23 -4
  33. data/lib/pbt/check/runner_iterator.rb +84 -10
  34. data/lib/pbt/check/runner_methods.rb +47 -17
  35. data/lib/pbt/check/tosser.rb +1 -1
  36. data/lib/pbt/reporter/run_details.rb +2 -2
  37. data/lib/pbt/reporter/run_details_reporter.rb +13 -0
  38. data/lib/pbt/reporter/run_execution.rb +47 -4
  39. data/lib/pbt/stateful/property.rb +5 -1
  40. data/lib/pbt/statistics.rb +28 -0
  41. data/lib/pbt/version.rb +1 -1
  42. data/lib/pbt.rb +45 -9
  43. metadata +16 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3275eafb6d07373c24f422feafa7300962213568a387cd21fbcd648c85de4193
4
- data.tar.gz: 7a3b902bb323251050ec023ec8318244d777c0bc2584ad0c8bad2504d93888f0
3
+ metadata.gz: b56a720a974f89209c3237c6b7a57adde668ebccbd77c76a51c882d1b5afd9c3
4
+ data.tar.gz: cae99f6c6d1b0d3b0de2a5ec00ca4544c7899e318c3ccaf8ec42da724d0cdb42
5
5
  SHA512:
6
- metadata.gz: e72d6278580f540b7d0aad1aefb168c2fbc7a3e64d31a90a4929695b979d3d1c29e3dc06fb22fb0fc6d1aadfc121605176cbee7146e298201f9091c0fa2ea647
7
- data.tar.gz: 8967245b883d22dfb00d79b0433acc53502e3e9ecdb07e89bae04002ce413fd67d126f695f4a4e12a965fc7adf171b5cd8e1c9fd3e6d8d28e8ff4444113cbea5
6
+ metadata.gz: 0e366c7d57f1428e240ce4faf92c0e0f302b0423d446af18610b81518a4a8f4528c8bdc0a3b8fc50a9e713979278520ba54cefbf430983185431f00e58cfd6ad
7
+ data.tar.gz: 87835150a4af3bcdbab40fb041d07103fbe3deb5a92d4980631f3485ae3aa89753a983908136d18f7224bf8b85db4e1d9fadede51cacc94471cbf8adb18b285f
data/.standard.yml CHANGED
@@ -1,3 +1,3 @@
1
1
  # For available configuration options, see:
2
2
  # https://github.com/standardrb/standard
3
- ruby_version: 3.1
3
+ ruby_version: 3.3
data/CHANGELOG.md CHANGED
@@ -1,5 +1,20 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.8.0] - 2026-11-22
4
+
5
+ - [Breaking change] Drop support for Ruby 3.2 and earlier (EOL); Ruby 3.3 or later is now required, and Ruby 4.1.0-preview1 is added to CI [#66](https://github.com/ohbarye/pbt/pull/66)
6
+ - [Breaking change] Split literal constant selection into `Pbt.constant_from(*choices)` and change `Pbt.one_of(*arbitraries)` to select uniformly among arbitraries with contextual shrinking [#70](https://github.com/ohbarye/pbt/pull/70)
7
+ - Add `Pbt.frequency` weighted generators with contextual lazy shrink trees (`Arbitrary::Sample`) [#61](https://github.com/ohbarye/pbt/pull/61)
8
+ - Add `Pbt.statistics` for reproducible generator distribution inspection [#62](https://github.com/ohbarye/pbt/pull/62)
9
+ - Allow `Arbitrary#map` without an `unmapper` and with block syntax (`arb.map { |v| ... }`), shrinking via `Sample` provenance [#71](https://github.com/ohbarye/pbt/pull/71)
10
+ - Add 3-valued `Arbitrary#cover?(value)` (including `FloatArbitrary` for `Pbt.float`) and `RunnerIterator` `same_failure?` / `shrink_case` hooks [#72](https://github.com/ohbarye/pbt/pull/72)
11
+ - Document v1 Property DSL contract and fix positional Hash / strict lambda argument passing in `Check::Property#run` [#73](https://github.com/ohbarye/pbt/pull/73)
12
+ - Re-raise fatal exceptions (`SystemExit`, `SignalException`, `NoMemoryError`) in runners without shrinking [#74](https://github.com/ohbarye/pbt/pull/74)
13
+ - Bound `FilterArbitrary` generation retries with `max_attempts:` (raising `Pbt::Arbitrary::FilterExhaustedError`) to prevent infinite loops [#75](https://github.com/ohbarye/pbt/pull/75)
14
+ - Improve array shrinking (continue element shrinking at `min_length` and support non-contiguous element deletion) and fix `Pbt.nat` default `max` [#76](https://github.com/ohbarye/pbt/pull/76)
15
+ - Add `Pbt.assume` precondition helper with `max_skips` guard (`Pbt::TooManySkipsError`) and `RunDetails#num_skips` [#77](https://github.com/ohbarye/pbt/pull/77)
16
+ - Note: the current `Pbt.stateful(model:, sut:)` API is planned to be replaced with a redesigned DSL in v0.9.0 ([#67](https://github.com/ohbarye/pbt/issues/67)).
17
+
3
18
  ## [0.7.0] - 2026-04-04
4
19
 
5
20
  - [Breaking change] Simplify stateful command protocol: `arguments` must now accept `state` parameter, and `applicable?` must now accept both `state` and `args` parameters
data/CLAUDE.md CHANGED
@@ -98,7 +98,7 @@ The codebase uses RSpec for testing and follows these patterns:
98
98
 
99
99
  ### Important Notes
100
100
 
101
- - Ruby 3.1+ is required due to Ractor usage
101
+ - Ruby 3.3+ is required
102
102
  - When using `:ractor` worker, be aware of Ractor limitations (no shared state, limited object sharing)
103
103
  - The gem uses Standard Ruby for code style (configured in `.standard.yml`)
104
- - CI runs tests against Ruby 3.1, 3.2, 3.3, and 3.4
104
+ - CI runs tests against Ruby 3.3, 3.4, 4.0, and 4.1.0-preview1
data/README.md CHANGED
@@ -28,6 +28,8 @@ For a more in-depth understanding of Property-Based Testing, please refer to ext
28
28
 
29
29
  ## Installation
30
30
 
31
+ Ruby 3.3 or later is required. CI tests Ruby 3.3, 3.4, 4.0, and 4.1.0-preview1.
32
+
31
33
  Add this line to your application's Gemfile and run `bundle install`.
32
34
 
33
35
  ```ruby
@@ -77,16 +79,36 @@ The above snippet is very simple but contains the basic components.
77
79
 
78
80
  #### Runner
79
81
 
80
- `Pbt.assert` is the runner. The runner interprets and executes the given property. `Pbt.assert` takes a property and runs it multiple times. If the property fails, it tries to shrink the input that caused the failure.
82
+ `Pbt.assert` is the primary runner. It executes the property returned by the block multiple times, shrinks any failure, and raises `Pbt::PropertyFailure` with reproduction details.
83
+
84
+ If you want to inspect predicate failures without raising `Pbt::PropertyFailure`, use `Pbt.check`, which returns a `Pbt::Reporter::RunDetails` struct (`failed`, `num_runs`, `num_skips`, `num_shrinks`, `seed`, `counterexample`, `counterexample_path`, `error_message`, `error_instance`, `failures`, `verbose`, `execution_summary`, `run_configuration`). Exceptions raised during property construction or value generation still propagate to the caller.
81
85
 
82
- #### Property
86
+ #### Preconditions (`Pbt.assume`)
83
87
 
84
- The snippet above declared a property by calling `Pbt.property`. The property describes the following:
88
+ Use `Pbt.assume(condition)` inside a predicate to discard inputs that do not satisfy a precondition instead of returning early (which would count skipped inputs as passed runs):
89
+
90
+ ```ruby
91
+ Pbt.assert do
92
+ Pbt.property(Pbt.integer, Pbt.integer) do |x, y|
93
+ Pbt.assume(y != 0)
94
+ raise "unexpected" unless (x / y) * y + (x % y) == x
95
+ end
96
+ end
97
+ ```
85
98
 
86
- 1. What the user wants to evaluate. This corresponds to the block (let's call this `predicate`) enclosed by `do` `end`
87
- 2. How to generate inputs for the predicate — using `Arbitrary`
99
+ Discarded cases do not count toward `num_runs` and are skipped during shrinking. If skips exceed `max_skips` (default: `num_runs * 10`), `Pbt.assert` raises `Pbt::TooManySkipsError` (`< Pbt::PropertyFailure`). Prefer narrowing your arbitrary when a precondition rejects most inputs.
88
100
 
89
- The `predicate` block is a function that directly asserts, taking values generated by `Arbitrary` as input.
101
+ #### Property (v1 DSL Contract)
102
+
103
+ `Pbt` intentionally uses explicit runner/property composition (`Pbt.assert` / `Pbt.check` wrapping `Pbt.property` or `Pbt.stateful`):
104
+
105
+ 1. Runner options (`seed`, `num_runs`, `verbose`, `worker`) stay separated from generated inputs.
106
+ 2. Predicates fail by **raising** (including RSpec `expect(...)` and Minitest assertions under default `worker: :none`). Returning `false` alone does not fail a property.
107
+ 3. Argument passing rules:
108
+ - Single arbitrary: passed directly (`Pbt.property(Pbt.integer) { |n| ... }`)
109
+ - Multiple positional arbitraries: wrapped as a tuple (`Pbt.property(Pbt.integer, Pbt.symbol) { |n, sym| ... }`)
110
+ - Keyword arbitraries: wrapped as a fixed hash and passed as keyword arguments under `worker: :none` (`Pbt.property(left: Pbt.integer, right: Pbt.integer) { |left:, right:| ... }`). Under `worker: :ractor`, accept a single hash parameter (`{ |h| ... }`).
111
+ - Mixing positional and keyword arbitraries in one `Pbt.property` call raises `ArgumentError`.
90
112
 
91
113
  #### Arbitrary
92
114
 
@@ -138,17 +160,68 @@ Pbt.tuple(Pbt.symbol, Pbt.integer).generate # => [:atq, 42]
138
160
  Pbt.fixed_hash(x: Pbt.symbol, y: Pbt.integer).generate # => {x: :atq, y: 42}
139
161
  Pbt.hash(Pbt.symbol, Pbt.integer).generate # => {atq: 121, ygab: -1142}
140
162
 
141
- Pbt.one_of(:a, 1, 0.1).generate # => :a or 1 or 0.1
163
+ Pbt.constant_from(:a, 1, 0.1).generate # => :a or 1 or 0.1
164
+ Pbt.one_of(Pbt.integer, Pbt.constant(nil)).generate # => 42 or nil
165
+
166
+ Pbt.integer(min: 0, max: 10).map { |n| n * 2 }.generate # => 8
167
+ Pbt.integer(min: 0, max: 10).filter(&:even?).generate # => 4
142
168
  ````
143
169
 
144
170
  See [ArbitraryMethods](https://github.com/ohbarye/pbt/blob/main/lib/pbt/arbitrary/arbitrary_methods.rb) module for more details.
145
171
 
146
- ## Stateful Testing (Experimental)
172
+ ### Weighted generators
173
+
174
+ Use `Pbt.frequency` to select generators by relative nonnegative integer weights:
147
175
 
148
- `Pbt` also provides an experimental stateful property API for model-based / command-based testing.
176
+ ```ruby
177
+ Pbt.assert(seed: 42) do
178
+ Pbt.property(Pbt.frequency([3, Pbt.integer], [1, Pbt.constant(nil)])) do |value|
179
+ raise "unexpected value" unless value.nil? || value.is_a?(Integer)
180
+ end
181
+ end
182
+ ```
183
+
184
+ At least one weight must be positive. Zero-weight generators are never selected.
185
+ Normal property execution retains the selected generator's shrink tree, including
186
+ inside arrays, tuples, fixed hashes, maps and filters. Shrinking stays within that
187
+ branch. Generated values and reported counterexamples remain ordinary Ruby values.
188
+
189
+ For manual shrinking, retain a sample: `sample = arb.sample(Random.new(42))`,
190
+ then inspect `sample.value` and `sample.shrink.map(&:value)`. Calling
191
+ `arb.shrink(value)` directly on `Pbt.frequency`, `Pbt.one_of`, or `Arbitrary#map`
192
+ without an `unmapper` returns no candidates because a bare value cannot identify its
193
+ source branch or preimage. The current `Pbt.stateful` command arguments use the
194
+ plain-value protocol, so those contextual generators do not shrink command arguments there.
195
+
196
+ ### Generator statistics
197
+
198
+ Inspect generated values before running a property:
199
+
200
+ ```ruby
201
+ stats = Pbt.statistics(Pbt.integer, num_runs: 1000, seed: 42) do |number|
202
+ number.negative? ? :negative : :nonnegative
203
+ end
204
+ stats.counts # => counts grouped by label
205
+ stats.percentages # => unrounded percentages grouped by label
206
+ stats.seed # => 42; reuse this seed to replay the sample
207
+ ```
208
+
209
+ Without a block, actual generated values are counted. Each call generates exactly
210
+ `num_runs` values using its own seeded RNG; it does not execute a property or shrink
211
+ values. The default is 100 runs with a fresh seed, retained in the result. Zero runs
212
+ return empty counts and percentages. Counts are a mutable Hash local to the result;
213
+ values or labels used as its keys must remain stable. For generators that reuse mutable
214
+ objects, return a stable label from the block (for example, a symbol or integer).
215
+ Generator and classifier errors propagate to the caller.
216
+
217
+ ## Stateful Testing
218
+
219
+ `Pbt` also provides a stateful property API for model-based / command-based testing.
149
220
  It is designed as a property-compatible object (`generate`, `shrink`, `run`) so it works with the existing runner (`Pbt.assert` / `Pbt.check`) without changing the runner API.
150
221
 
151
- This API is still experimental. Expect interface refinements and behavior changes in future releases.
222
+ > **Note:** The current `Pbt.stateful(model:, sut:)` protocol is planned to be replaced with a cleaner yield-builder DSL and improved shrinking engine in v0.9.0 before v1.0.0. See [#67](https://github.com/ohbarye/pbt/issues/67) and [docs/plans/stateful-pbt-redesign.md](docs/plans/stateful-pbt-redesign.md).
223
+
224
+ Commands currently implement `arguments(state)` and `applicable?(state, args)`; the earlier zero-argument / state-only forms are no longer supported.
152
225
 
153
226
  ### Minimal usage
154
227
 
@@ -229,12 +302,12 @@ end
229
302
  - `command.run!(sut, args)` -> command result
230
303
  - `command.verify!(before_state:, after_state:, args:, result:, sut:)`
231
304
 
232
- ### Current limitations (MVP)
305
+ ### Current limitations
233
306
 
234
307
  - `Pbt.stateful` runs sequentially by default, even if the global worker is `:ractor`.
235
308
  - You can still pass `worker: :none` explicitly if you want to make that choice obvious in a test.
236
309
  - `worker: :ractor` is currently unsupported and raises `Pbt::InvalidConfiguration`.
237
- - Shrinking supports shorter prefixes and command-argument shrinking (using `command.arguments.shrink(args)`).
310
+ - Shrinking supports shorter prefixes and command-argument shrinking (using `command.arguments(state).shrink(args)`).
238
311
 
239
312
  ## What if property-based tests fail?
240
313
 
@@ -327,6 +400,10 @@ Pbt.configure do |config|
327
400
  # Whether to report exceptions in threads.
328
401
  # It's useful to suppress error logs on Ractor that reports many errors. Default is `false`.
329
402
  config.thread_report_on_exception = false
403
+
404
+ # Maximum number of skipped cases allowed via `Pbt.assume`.
405
+ # Defaults to `num_runs * 10` when `nil`.
406
+ config.max_skips = nil
330
407
  end
331
408
  ```
332
409
 
@@ -420,11 +497,11 @@ Once this project finishes the following, we will release v1.0.0.
420
497
  - [x] Rich report by verbose mode
421
498
  - [x] (Partially) Allow to use expectations and matchers provided by test framework in Ractor. (dropped)
422
499
  - It'd be so hard to pass assertions like `expect`, `assert` to a Ractor.
423
- - [ ] Implement frequency arbitrary
424
- - [ ] Statistics feature to aggregate generated values
425
- - [ ] Decide DSL
426
- - [ ] Try Fiber
427
- - [x] Stateful property-based testing (experimental, via `Pbt.stateful`)
500
+ - [x] Implement frequency arbitrary
501
+ - [x] Statistics feature to aggregate generated values
502
+ - [x] Decide DSL (explicit `Pbt.assert` / `Pbt.check` + `Pbt.property` / `Pbt.stateful` composition)
503
+ - [x] Try Fiber ([experiment and v1 decision](benchmark/FIBER.md))
504
+ - [x] Stateful property-based testing (via `Pbt.stateful`)
428
505
 
429
506
  ## Development
430
507
 
@@ -0,0 +1,97 @@
1
+ # Fiber predicate experiment
2
+
3
+ Run from the repository root:
4
+
5
+ ```sh
6
+ bundle exec ruby benchmark/fiber_experiment.rb
7
+ NUM_RUNS=1000 bundle exec ruby benchmark/fiber_experiment.rb
8
+ ```
9
+
10
+ This compares the normal sequential runner with the same runner wrapping each
11
+ predicate call in a newly created Fiber. Both use seed 42, the same number of
12
+ cases, and the same CPU / blocking-sleep predicates. The script checks successful
13
+ case counts and measures wall time with a monotonic clock. It installs no scheduler.
14
+
15
+ ## Decision for v1
16
+
17
+ Keep `worker: :none` as the default and `:ractor` as the existing opt-in worker.
18
+ The bare-Fiber experiment is complete; it does not justify a new public worker.
19
+ A Fiber per case adds allocation/context-switching work, while blocking sleep
20
+ still blocks its thread. This is a baseline experiment, not a benchmark of a
21
+ scheduler-backed asynchronous worker or evidence against all Fiber designs.
22
+
23
+ A local smoke run on Ruby 4.0.9 (arm64 macOS), 100 cases, seed 42 produced:
24
+
25
+ ```text
26
+ cpu sequential 0.001758 seconds
27
+ cpu fiber 0.002105 seconds
28
+ blocking_sleep sequential 0.126800 seconds
29
+ blocking_sleep fiber 0.128960 seconds
30
+ ```
31
+
32
+ These short timings are illustrative and noisy, not performance guarantees.
33
+ Re-run on the target Ruby/platform with representative workloads before drawing
34
+ performance conclusions. CPU parallelism is not provided by these same-thread Fibers.
35
+
36
+ ## Scheduler-backed TCP IO experiment
37
+
38
+ The separate experiment in `benchmark/fiber_scheduler/experiment.rb` uses Async's
39
+ scheduler. Async is an optional benchmark dependency, not a dependency of pbt:
40
+
41
+ ```sh
42
+ BUNDLE_GEMFILE=benchmark/fiber_scheduler/Gemfile bundle install
43
+ BUNDLE_GEMFILE=benchmark/fiber_scheduler/Gemfile bundle exec rspec benchmark/fiber_scheduler/spec
44
+ BUNDLE_GEMFILE=benchmark/fiber_scheduler/Gemfile bundle exec ruby benchmark/fiber_scheduler/experiment.rb
45
+ ```
46
+
47
+ Parameters are `NUM_RUNS` (100), `CONCURRENCY` (10), `IO_DELAY` (0.01 seconds),
48
+ and `REPEATS` (5; use an odd number). Each mode gets a separate warmup. The timer
49
+ includes client socket creation, connection, reads, property calls and scheduler
50
+ startup/cleanup, but excludes gem loading, input generation and server startup.
51
+
52
+ A dedicated server thread accepts loopback TCP connections and independently delays
53
+ each `ok` response by 10ms. Clients perform a real blocking `Socket#read` through
54
+ `TCPSocket`, which Async makes cooperative. The three modes use the same seed-42
55
+ integer inputs and identical protocol: serial requests, serial requests wrapped in
56
+ bare Fibers, and batches of at most 10 concurrent Async tasks. Batches wait for all
57
+ members before starting the next batch; this is not a continuously filled pool.
58
+ The server records its peak number of pending responses to verify that IO waits
59
+ actually overlap; timing thresholds are deliberately absent from tests.
60
+
61
+ Local measurement: Ruby 4.0.9, Async 2.46.0, arm64 macOS, 100 cases, seed 42,
62
+ 10ms per response, maximum concurrency 10, 5 repetitions after warmup:
63
+
64
+ ```text
65
+ mode elapsed seconds (all 5 samples) median seconds
66
+ sequential 1.232761 1.246074 1.233465 1.250332 1.226240 1.233465
67
+ bare Fiber 1.232813 1.231938 1.224650 1.241142 1.213324 1.231938
68
+ scheduler 0.124735 0.129789 0.130015 0.130826 0.128622 0.129789
69
+ ```
70
+
71
+ Every sample completed 100 requests and produced the same input checksum, 47379.
72
+ Server peak pending responses were 1 for serial/bare Fiber and 10 for Scheduler
73
+ in every repetition. The Scheduler median was about 9.5 times faster than serial
74
+ for this intentionally wait-dominated workload. This demonstrates overlapping IO
75
+ waits, rather than CPU parallelism. It is not a general throughput guarantee;
76
+ loopback, synthetic delay, small payloads, batch size and machine load all matter.
77
+
78
+ The optional tests cover identical inputs and counts, bounded pending requests,
79
+ exception propagation, server-thread cleanup and refusing to replace an existing
80
+ caller-owned scheduler. Client and server sockets are closed in ensure blocks;
81
+ remaining batch tasks are stopped after failure. Async may log the expected failure
82
+ in the cleanup test. The benchmark owns its scheduler only when none is installed.
83
+
84
+ ### Decision after both experiments
85
+
86
+ Keep the existing public workers for v1. The scheduled experiment calls generated
87
+ properties directly; it does not integrate with `Pbt.check`, its early-failure
88
+ ordering, cancellation policy, shrinking, reports or Ractor worker. Concurrent failure/shrink/report parity is outside this experiment.
89
+
90
+ Scheduler-backed execution can help IO-heavy properties, but adoption needs a
91
+ separate design for in-flight case cleanup, failure ordering, exception propagation,
92
+ shared fixture state, concurrency limits and scheduler ownership. A caller-installed
93
+ scheduler must not be replaced or closed by pbt. No public API or runtime dependency
94
+ is added by this experiment.
95
+
96
+ References: [Ruby Fiber scheduler interface](https://docs.ruby-lang.org/en/master/Fiber/Scheduler.html),
97
+ [Async](https://github.com/socketry/async).
@@ -0,0 +1,39 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pbt"
4
+ require "delegate"
5
+
6
+ # Standalone experiment, not a supported worker implementation.
7
+ module FiberExperiment
8
+ class Property < SimpleDelegator
9
+ def run(value)
10
+ Fiber.new { __getobj__.run(value) }.resume
11
+ end
12
+ end
13
+
14
+ def self.check(fiber:, num_runs:, &predicate)
15
+ property = Pbt.property(Pbt.integer, &predicate)
16
+ property = Property.new(property) if fiber
17
+ Pbt.check(seed: 42, num_runs: num_runs, worker: :none) { property }
18
+ end
19
+
20
+ def self.run(num_runs: 100)
21
+ workloads = {
22
+ cpu: ->(n) { 1000.times { n * n } },
23
+ blocking_sleep: ->(_n) { sleep(0.001) }
24
+ }
25
+ puts "Ruby #{RUBY_VERSION}; #{num_runs} cases; seed 42; no scheduler installed"
26
+ workloads.each do |name, predicate|
27
+ [false, true].each do |fiber|
28
+ started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
29
+ result = check(fiber: fiber, num_runs: num_runs, &predicate)
30
+ elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started
31
+ raise "incorrect execution" if result.failed || result.num_runs != num_runs
32
+
33
+ puts format("%-26s %.6f seconds", "#{name} #{fiber ? "fiber" : "sequential"}", elapsed)
34
+ end
35
+ end
36
+ end
37
+ end
38
+
39
+ FiberExperiment.run(num_runs: Integer(ENV.fetch("NUM_RUNS", "100"))) if $PROGRAM_NAME == __FILE__
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ source "https://rubygems.org"
4
+
5
+ gem "pbt", path: "../.."
6
+ gem "async", "~> 2.0"
7
+ gem "rspec", "~> 3.13"
@@ -0,0 +1,131 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pbt"
4
+ require "async"
5
+ require "socket"
6
+
7
+ # Benchmark only: does not execute Pbt.check or implement a public worker.
8
+ module SchedulerExperiment
9
+ # One server thread delays each response independently. It never uses a Fiber
10
+ # scheduler, so all three client modes see the same delayed TCP protocol.
11
+ class Server
12
+ attr_reader :port
13
+
14
+ def initialize(num_runs:, delay:)
15
+ @listener = TCPServer.new("127.0.0.1", 0)
16
+ @port = @listener.addr[1]
17
+ @thread = Thread.new { serve(num_runs, delay) }
18
+ end
19
+
20
+ def serve(num_runs, delay)
21
+ pending = {}
22
+ completed = 0
23
+ peak = 0
24
+ until completed == num_runs
25
+ timeout = pending.empty? ? nil : [pending.values.min - SchedulerExperiment.now, 0].max
26
+ if IO.select([@listener], nil, nil, timeout)
27
+ loop do
28
+ socket = @listener.accept_nonblock(exception: false)
29
+ break if socket == :wait_readable
30
+ pending[socket] = SchedulerExperiment.now + delay
31
+ peak = [peak, pending.size].max
32
+ end
33
+ end
34
+ pending.keys.each do |socket|
35
+ next if pending.fetch(socket) > SchedulerExperiment.now
36
+ socket.write("ok")
37
+ socket.close
38
+ pending.delete(socket)
39
+ completed += 1
40
+ end
41
+ end
42
+ peak
43
+ ensure
44
+ pending&.each_key { |socket| socket.close unless socket.closed? }
45
+ end
46
+
47
+ def peak
48
+ @thread.value
49
+ end
50
+
51
+ def close
52
+ @thread.kill if @thread.alive?
53
+ @thread.join
54
+ @listener.close
55
+ end
56
+ end
57
+
58
+ def self.now
59
+ Process.clock_gettime(Process::CLOCK_MONOTONIC)
60
+ end
61
+
62
+ def self.measure(mode:, num_runs:, concurrency:, delay:)
63
+ raise ArgumentError, "existing scheduler must not be replaced" if Fiber.scheduler
64
+ raise ArgumentError, "unknown mode" unless [:sequential, :fiber, :scheduler].include?(mode)
65
+ raise ArgumentError, "positive runs and concurrency required" unless num_runs.positive? && concurrency.positive?
66
+
67
+ rng = Random.new(42)
68
+ values = Array.new(num_runs) { Pbt.integer(min: 0, max: 1000).generate(rng) }
69
+ property = Pbt.property(Pbt.integer) { |value| yield(value) if block_given? }
70
+ server = Server.new(num_runs: num_runs, delay: delay)
71
+ observed = Array.new(num_runs)
72
+ request = lambda do |index|
73
+ socket = TCPSocket.new("127.0.0.1", server.port)
74
+ raise "incorrect response" unless socket.read(2) == "ok"
75
+ property.run(values.fetch(index))
76
+ observed[index] = values.fetch(index)
77
+ ensure
78
+ socket&.close
79
+ end
80
+
81
+ started = now
82
+ if mode == :scheduler
83
+ Async do |parent|
84
+ (0...num_runs).each_slice(concurrency) do |batch|
85
+ tasks = []
86
+ begin
87
+ batch.each { |index| tasks << parent.async { request.call(index) } }
88
+ tasks.each(&:wait)
89
+ ensure
90
+ tasks.each(&:stop)
91
+ end
92
+ end
93
+ end.wait
94
+ else
95
+ num_runs.times do |index|
96
+ if mode == :fiber
97
+ Fiber.new { request.call(index) }.resume
98
+ else
99
+ request.call(index)
100
+ end
101
+ end
102
+ end
103
+ elapsed = now - started
104
+ raise "incorrect generated inputs" unless observed == values
105
+ {elapsed: elapsed, values: observed, completed: observed.size, max_pending: server.peak}
106
+ ensure
107
+ server&.close
108
+ end
109
+
110
+ def self.run(num_runs: 100, concurrency: 10, delay: 0.01, repeats: 5)
111
+ raise ArgumentError, "repeats must be a positive odd number" unless repeats.positive? && repeats.odd?
112
+
113
+ puts "Ruby #{RUBY_VERSION}; async #{Async::VERSION}; #{num_runs} cases; seed 42; delay #{delay}s; concurrency #{concurrency}; #{repeats} repeats"
114
+ [:sequential, :fiber, :scheduler].each do |mode|
115
+ # Warm up separately; exclude startup/gem loading from each timed request loop.
116
+ measure(mode: mode, num_runs: concurrency, concurrency: concurrency, delay: delay)
117
+ results = Array.new(repeats) { measure(mode: mode, num_runs: num_runs, concurrency: concurrency, delay: delay) }
118
+ times = results.map { |result| result.fetch(:elapsed) }
119
+ puts "#{mode}: seconds=#{times.map { |time| format("%.6f", time) }.join(",")}; median=#{format("%.6f", times.sort.fetch(repeats / 2))}; completed=#{results.map { |r| r.fetch(:completed) }.uniq.inspect}; max_pending=#{results.map { |r| r.fetch(:max_pending) }.inspect}; checksum=#{results.first.fetch(:values).sum}"
120
+ end
121
+ end
122
+ end
123
+
124
+ if $PROGRAM_NAME == __FILE__
125
+ SchedulerExperiment.run(
126
+ num_runs: Integer(ENV.fetch("NUM_RUNS", "100")),
127
+ concurrency: Integer(ENV.fetch("CONCURRENCY", "10")),
128
+ delay: Float(ENV.fetch("IO_DELAY", "0.01")),
129
+ repeats: Integer(ENV.fetch("REPEATS", "5"))
130
+ )
131
+ end
@@ -0,0 +1,38 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "rspec"
4
+ require_relative "../experiment"
5
+
6
+ RSpec.describe SchedulerExperiment do
7
+ it "preserves the seeded inputs and bounds concurrent socket requests" do
8
+ results = [:sequential, :fiber, :scheduler].map do |mode|
9
+ described_class.measure(mode: mode, num_runs: 12, concurrency: 4, delay: 0.01)
10
+ end
11
+ expect(results.map { |result| result.fetch(:values) }.uniq.size).to eq(1)
12
+ expect(results.map { |result| result.fetch(:completed) }).to eq([12, 12, 12])
13
+ expect(results.first(2).map { |result| result.fetch(:max_pending) }).to eq([1, 1])
14
+ expect(results.last.fetch(:max_pending)).to be_between(1, 4)
15
+ expect(Fiber.scheduler).to be_nil
16
+ end
17
+
18
+ it "propagates predicate failures and cleans up the server and scheduler" do
19
+ server_threads = Thread.list
20
+ expect {
21
+ described_class.measure(mode: :scheduler, num_runs: 8, concurrency: 4, delay: 0.001) do
22
+ raise "predicate failure"
23
+ end
24
+ }.to raise_error("predicate failure")
25
+ expect(Fiber.scheduler).to be_nil
26
+ expect(Thread.list - server_threads).to be_empty
27
+ end
28
+
29
+ it "does not replace a caller's scheduler" do
30
+ Async do
31
+ scheduler = Fiber.scheduler
32
+ expect {
33
+ described_class.measure(mode: :scheduler, num_runs: 1, concurrency: 1, delay: 0)
34
+ }.to raise_error(ArgumentError, /existing scheduler/)
35
+ expect(Fiber.scheduler).to equal(scheduler)
36
+ end.wait
37
+ end
38
+ end
@@ -0,0 +1,12 @@
1
+ Fiber experiment: add reproducible benchmark comparing ordinary sequential calls with
2
+ creating/resuming one Fiber per case, same generated inputs and predicates. Include CPU
3
+ and blocking sleep workloads; configurable small runs. No production worker yet.
4
+ Document that bare Fibers do not overlap blocking work without a scheduler; using an
5
+ external scheduler introduces dependency/lifecycle/cancellation decisions. Validate same
6
+ successful count and deterministic failure+shrinking by running each predicate inside a Fiber.
7
+ Decision: keep :none/:ractor for v1; revisit scheduler-backed worker for concrete IO use case.
8
+
9
+ Completed: red specs committed a439561; adapter and benchmark added.
10
+ The existing halving strategy reaches 7 (not global minimum 5) with seed42;
11
+ regression expectation corrected to the serial baseline. Full rake: 183 examples,
12
+ 0 failures; Standard passes. Smoke benchmark recorded in benchmark/FIBER.md.
@@ -0,0 +1,16 @@
1
+ # Generator statistics
2
+
3
+ Goal: inspect generated value distributions independently of property execution.
4
+ API: `Pbt.statistics(arb, num_runs: 100, seed: Random.new_seed) { |value| label }`.
5
+ Return seed, num_runs, counts and unrounded percentages; no block counts values.
6
+ Runs must be a nonnegative integer. Zero generates/classifies nothing. Use a local
7
+ seeded RNG, propagate errors and keep each invocation independent. Values or classifier
8
+ labels used as Hash keys must be stable (not mutated by later generation).
9
+
10
+ - [x] Write tests, run red (12 failures, missing API), review contract/tests.
11
+ - [x] Commit tests before implementation.
12
+ - [x] Add statistics class and Pbt entry point, docs and full suite/lint.
13
+ - [x] Independent review, fix/re-review, commit, PR referencing #26.
14
+
15
+ Verification: 13 red tests committed 04fb022; full suite 194 examples, 0 failures;
16
+ Standard passes. Independent review clean, including global RNG isolation regression.