pbt 0.4.1 → 0.5.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c1a5d4c94783e479474b521113dcad84959db11c6139780031cf52aff6748e61
4
- data.tar.gz: eb112f231d4417963341fb36b585c9b0d4fa8b14f0ac726c51b80c7424542cb1
3
+ metadata.gz: 2b93d8a06cbff5946bab644527868f42601ccd7521d4d11e3746fa00c279cd62
4
+ data.tar.gz: ab19c9a123d4408cb49e7a24225bca930d5bd4134000e98a3efc9c233ce97f9c
5
5
  SHA512:
6
- metadata.gz: e40bde143e84c6ed1fa76cf69f3ed93261bcdae70a229359ea996190c63c24ff9b7155f9974a0d4a39c8430fb17e935539631e463a072af30439b5805c939021
7
- data.tar.gz: d082a2078753d09bf19a1bfc41931258a19db3842590c7f366a1f5fece9097b3a6668db096baa9a679ab0026df05e45ac1f03f14ceca0f1b7aeb635bc1f82c83
6
+ metadata.gz: cd5fd29fb5e5d3afb5817fd24b0c8967037e2158dd3cc57e4d4ab83eb12c7a31c1dfff376a88b96173ca14853cd3d56b89de212c3630bb6cffe4bb4f04b106d6
7
+ data.tar.gz: b0d37a7198cd0d1c504170f00041b5decab223e73c8896ebd1d357f4482503665ec0ca200cf5557bbed6c04f775cb3c7264eb61b1a42e6b6982f30e90941155e
data/CHANGELOG.md CHANGED
@@ -1,5 +1,14 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.5.0] - 2024-12-30
4
+
5
+ - [Breaking change] Drop `:process` and `:thread` workers since there are no concrete use cases.
6
+ - [Breaking change] Drop `:experimental_ractor_rspec_integration` option since there are no concrete use cases.
7
+
8
+ ## [0.4.2] - 2024-05-23
9
+
10
+ - Fix Prism `LoadError` message [#27](https://github.com/ohbarye/pbt/pull/27) by @sambostock
11
+
3
12
  ## [0.4.1] - 2024-05-10
4
13
 
5
14
  - Fix a bug for experimental_ractor_rspec_integration mode. When a test file name starts with a number, it can't be a constant name.
data/README.md CHANGED
@@ -8,6 +8,8 @@ A property-based testing tool for Ruby with experimental features that allow you
8
8
 
9
9
  PBT stands for Property-Based Testing.
10
10
 
11
+ As for the results of the parallelization experiment, please refer the talk at RubyKaigi 2024: [Unlocking Potential of Property Based Testing with Ractor](https://rubykaigi.org/2024/presentations/ohbarye.html).
12
+
11
13
  ## What's Property-Based Testing?
12
14
 
13
15
  Property-Based Testing is a testing methodology that focuses on the properties a system should always satisfy, rather than checking individual examples. Instead of writing tests for predefined inputs and outputs, PBT allows you to specify the general characteristics that your code should adhere to and then automatically generates a wide range of inputs to verify these properties.
@@ -39,29 +41,34 @@ Off course you can install with `gem intstall pbt`.
39
41
  ### Simple property
40
42
 
41
43
  ```ruby
42
- # Let's say you have a method that returns just a multiplicative inverse.
43
- def multiplicative_inverse(number)
44
- Rational(1, number)
44
+ # Let's say you have your own sort method.
45
+ def sort(array)
46
+ return array if array.size <= 2 # Here's a bug! It should be 1.
47
+ pivot, *rest = array
48
+ left, right = rest.partition { |n| n <= pivot }
49
+ sort(left) + [pivot] + sort(right)
45
50
  end
46
51
 
47
52
  Pbt.assert do
48
- # The given block is executed 100 times with different random numbers.
49
- # Besides, the block runs in parallel by Ractor.
50
- Pbt.property(Pbt.integer) do |number|
51
- result = multiplicative_inverse(number)
52
- raise "Result should be the multiplicative inverse of the number" if result * number != 1
53
+ # The given block is executed 100 times with different arrays with random numbers.
54
+ # Besides, if you set `worker: :ractor` option to `assert` method, it runs in parallel using Ractor.
55
+ Pbt.property(Pbt.array(Pbt.integer)) do |numbers|
56
+ result = sort(numbers)
57
+ result.each_cons(2) do |x, y|
58
+ raise "Sort algorithm is wrong." unless x <= y
59
+ end
53
60
  end
54
61
  end
55
62
 
56
- # If the function has a bug, the test fails with a counterexample.
57
- # For example, the multiplicative_inverse method doesn't work for 0 regardless of the behavior is intended or not.
63
+ # If the method has a bug, the test fails and it reports a minimum counterexample.
64
+ # For example, the sort method doesn't work for [0, -1].
58
65
  #
59
66
  # Pbt::PropertyFailure:
60
67
  # Property failed after 23 test(s)
61
- # { seed: 11001296583699917659214176011685741769 }
62
- # Counterexample: 0
63
- # Shrunk 3 time(s)
64
- # Got ZeroDivisionError: divided by 0
68
+ # seed: 43738985293126714007411539287084402325
69
+ # counterexample: [0, -1]
70
+ # Shrunk 40 time(s)
71
+ # Got RuntimeError: Sort algorithm is wrong.
65
72
  ```
66
73
 
67
74
  ### Explain The Snippet
@@ -92,7 +99,7 @@ Here, we used only one type of arbitrary, `Pbt.integer`. There are many other bu
92
99
  In PBT, If a test fails, it attempts to shrink the case that caused the failure into a form that is easier for humans to understand.
93
100
  In other words, instead of stopping the test itself the first time it fails and reporting the failed value, it tries to find the minimal value that causes the error.
94
101
 
95
- When there is a test that fails when given an even number, a counterexample of `2` is simpler and easier to understand than `432743417662`.
102
+ When there is a test that fails when given an even number, a counterexample of `[0, -1]` is simpler and easier to understand than any complex example like `[-897860, -930517, 577817, -16302, 310864, 856411, -304517, 86613, -78231]`.
96
103
 
97
104
  ### Arbitrary
98
105
 
@@ -103,16 +110,16 @@ There are many built-in arbitraries in `Pbt`. You can use them to generate rando
103
110
  ```ruby
104
111
  rng = Random.new
105
112
 
106
- Pbt.integer.generate(rng) # => 42
113
+ Pbt.integer.generate(rng) # => 42
107
114
  Pbt.integer(min: -1, max: 8).generate(rng) # => Integer between -1 and 8
108
115
 
109
- Pbt.symbol.generate(rng) # => :atq
116
+ Pbt.symbol.generate(rng) # => :atq
110
117
 
111
- Pbt.ascii_char.generate(rng) # => "a"
112
- Pbt.ascii_string.generate(rng) # => "aagjZfao"
118
+ Pbt.ascii_char.generate(rng) # => "a"
119
+ Pbt.ascii_string.generate(rng) # => "aagjZfao"
113
120
 
114
- Pbt.boolean.generate(rng) # => true or false
115
- Pbt.constant(42).generate(rng) # => 42 always
121
+ Pbt.boolean.generate(rng) # => true or false
122
+ Pbt.constant(42).generate(rng) # => 42 always
116
123
  ```
117
124
 
118
125
  #### Composites
@@ -120,15 +127,15 @@ Pbt.constant(42).generate(rng) # => 42 always
120
127
  ```ruby
121
128
  rng = Random.new
122
129
 
123
- Pbt.array(Pbt.integer).generate(rng) # => [121, -13141, 9825]
124
- Pbt.array(Pbt.integer, max: 1, empty: true).generate(rng) # => [] or [42] etc.
130
+ Pbt.array(Pbt.integer).generate(rng) # => [121, -13141, 9825]
131
+ Pbt.array(Pbt.integer, max: 1, empty: true).generate(rng) # => [] or [42] etc.
125
132
 
126
- Pbt.tuple(Pbt.symbol, Pbt.integer).generate(rng) # => [:atq, 42]
133
+ Pbt.tuple(Pbt.symbol, Pbt.integer).generate(rng) # => [:atq, 42]
127
134
 
128
135
  Pbt.fixed_hash(x: Pbt.symbol, y: Pbt.integer).generate(rng) # => {x: :atq, y: 42}
129
- Pbt.hash(Pbt.symbol, Pbt.integer).generate(rng) # => {atq: 121, ygab: -1142}
136
+ Pbt.hash(Pbt.symbol, Pbt.integer).generate(rng) # => {atq: 121, ygab: -1142}
130
137
 
131
- Pbt.one_of(:a, 1, 0.1).generate(rng) # => :a or 1 or 0.1
138
+ Pbt.one_of(:a, 1, 0.1).generate(rng) # => :a or 1 or 0.1
132
139
  ````
133
140
 
134
141
  See [ArbitraryMethods](https://github.com/ohbarye/pbt/blob/main/lib/pbt/arbitrary/arbitrary_methods.rb) module for more details.
@@ -144,18 +151,18 @@ When a test fails, you'll see a message like below.
144
151
  ```text
145
152
  Pbt::PropertyFailure:
146
153
  Property failed after 23 test(s)
147
- { seed: 11001296583699917659214176011685741769 }
148
- Counterexample: 0
149
- Shrunk 3 time(s)
150
- Got ZeroDivisionError: divided by 0
154
+ seed: 43738985293126714007411539287084402325
155
+ counterexample: [0, -1]
156
+ Shrunk 40 time(s)
157
+ Got RuntimeError: Sort algorithm is wrong.
151
158
  # and backtraces
152
159
  ```
153
160
 
154
161
  You can reproduce the failure by passing the seed to `Pbt.assert`.
155
162
 
156
163
  ```ruby
157
- Pbt.assert(seed: 11001296583699917659214176011685741769) do
158
- Pbt.property(Pbt.integer) do |number|
164
+ Pbt.assert(seed: 43738985293126714007411539287084402325) do
165
+ Pbt.property(Pbt.array(Pbt.integer)) do |number|
159
166
  # your test
160
167
  end
161
168
  end
@@ -177,25 +184,29 @@ The verbose mode prints the results of each tested values.
177
184
 
178
185
  ```text
179
186
  Encountered failures were:
180
- - [152477, 666997, -531468, -92182, 623948, 425913, 656138, 856463, -64529]
181
- - [76239, 666997, -531468, -92182, 623948]
182
- - [76239, 666997, -531468]
187
+ - [-897860, -930517, 577817, -16302, 310864, 856411, -304517, 86613, -78231]
188
+ - [310864, 856411, -304517, 86613, -78231]
189
+ - [-304517, 86613, -78231]
183
190
  (snipped for README)
184
- - [2, 163]
185
- - [2, 11]
191
+ - [0, -3]
192
+ - [0, -2]
193
+ - [0, -1]
186
194
 
187
195
  Execution summary:
188
- . × [152477, 666997, -531468, -92182, 623948, 425913, 656138, 856463, -64529]
189
- . . √ [152477, 666997, -531468, -92182, 623948]
190
- . . √ [-64529]
191
- . . × [76239, 666997, -531468, -92182, 623948, 425913, 656138, 856463, -64529]
192
- . . . × [76239, 666997, -531468, -92182, 623948]
196
+ . × [-897860, -930517, 577817, -16302, 310864, 856411, -304517, 86613, -78231]
197
+ . . √ [-897860, -930517, 577817, -16302, 310864]
198
+ . . √ [-930517, 577817, -16302, 310864, 856411]
199
+ . . [577817, -16302, 310864, 856411, -304517]
200
+ . . [-16302, 310864, 856411, -304517, 86613]
201
+ . . × [310864, 856411, -304517, 86613, -78231]
193
202
  (snipped for README)
194
- . . . . . . . . . . . . . . . . . √ [2, 21]
195
- . . . . . . . . . . . . . . . . . × [2, 11]
196
- . . . . . . . . . . . . . . . . . . []
197
- . . . . . . . . . . . . . . . . . . √ [2, 1]
198
- . . . . . . . . . . . . . . . . . . √ [2, 0]
203
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . √ [-2]
204
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . √ []
205
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . × [0, -1]
206
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . √ [0]
207
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . √ [-1]
208
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . √ []
209
+ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . √ [0, 0]
199
210
  ```
200
211
 
201
212
  ## Configuration
@@ -207,7 +218,7 @@ Pbt.configure do |config|
207
218
  # Whether to print verbose output. Default is `false`.
208
219
  config.verbose = false
209
220
 
210
- # The concurrency method to use. `:ractor`, `:thread`, `:process` and `:none` are supported. Default is `:none`.
221
+ # The concurrency method to use. `:ractor` and `:none` are supported. Default is `:none`.
211
222
  config.worker = :none
212
223
 
213
224
  # The number of runs to perform. Default is `100`.
@@ -220,9 +231,6 @@ Pbt.configure do |config|
220
231
  # Whether to report exceptions in threads.
221
232
  # It's useful to suppress error logs on Ractor that reports many errors. Default is `false`.
222
233
  config.thread_report_on_exception = false
223
-
224
- # Whether to allow RSpec expectation and matchers in Ractor. It's quite experimental! Default is `false`.
225
- config.experimental_ractor_rspec_integration = false
226
234
  end
227
235
  ```
228
236
 
@@ -238,15 +246,13 @@ end
238
246
 
239
247
  One of the key features of `Pbt` is its ability to rapidly execute test cases in parallel or concurrently, using a large number of values (by default, `100`) generated by `Arbitrary`.
240
248
 
241
- For concurrent processing, you can specify any of the three workers—`:ractor`, `:process`, or `:thread`—using the `worker` option. Alternatively, choose `:none` for serial execution.
242
-
243
- `Pbt` supports 3 concurrency methods and 1 sequential one. You can choose one of them by setting the `worker` option.
249
+ For concurrent processing, you can specify `:ractor` using the `worker` option. Alternatively, choose `:none` for serial execution.
244
250
 
245
251
  Be aware that the performance of each method depends on the test subject. For example, if the test subject is CPU-bound, `:ractor` may be the best choice. Otherwise, `:none` shall be the best choice for most cases. See [benchmarks](benchmark/README.md).
246
252
 
247
253
  ### Ractor
248
254
 
249
- `:ractor` worker is useful for test cases that are CPU-bound. But it's experimental and has some limitations as described below. If you encounter any issues due to those limitations, consider using `:process` as workers whose benchmark is the most similar to `:ractor`.
255
+ `:ractor` worker is useful for test cases that are CPU-bound. But it's experimental and has some limitations as described below. If you encounter any issues due to those limitations, consider falling back to `:none`.
250
256
 
251
257
  ```ruby
252
258
  Pbt.assert(worker: :ractor) do
@@ -286,53 +292,9 @@ it do
286
292
  end
287
293
  ```
288
294
 
289
- If you're a challenger, you can enable the experimental feature to allow using RSpec expectations and matchers in Ractor. It works but it's quite experimental and could cause unexpected behaviors.
290
-
291
- Please note that this feature depends on [prism](https://ruby.github.io/prism/) gem. If you use Ruby 3.2 or prior, you need to install the gem by yourself.
292
-
293
- ```ruby
294
- it do
295
- Pbt.assert(worker: :ractor, experimental_ractor_rspec_integration: true) do
296
- Pbt.property(Pbt.integer) do |n|
297
- # Some RSpec expectations and matchers are available in Ractor by hack.
298
- # Other features like `let`, `subject`, `before`, `after` that access out of block are still not available.
299
- expect(n).to be_an(Integer)
300
- end
301
- end
302
- end
303
- ```
304
-
305
- ### Process
306
-
307
- If you'd like to run test cases that are CPU-bound and `:ractor` is not available, `:process` becomes a good choice.
308
-
309
- ```ruby
310
- Pbt.assert(worker: :process) do
311
- Pbt.property(Pbt.integer) do |n|
312
- # ...
313
- end
314
- end
315
- ```
316
-
317
- If you want to use `:process`, you need to install the [parallel](https://github.com/grosser/parallel) gem.
318
-
319
- ### Thread
320
-
321
- You may not need to run test cases with multi-threads.
322
-
323
- ```ruby
324
- Pbt.assert(worker: :thread) do
325
- Pbt.property(Pbt.integer) do |n|
326
- # ...
327
- end
328
- end
329
- ```
330
-
331
- If you want to use `:thread`, you need to install the [parallel](https://github.com/grosser/parallel) gem.
332
-
333
295
  ### None
334
296
 
335
- For most cases, `:none` is the best choice. It runs tests sequentially (without parallelism) but most test cases finishes within a reasonable time.
297
+ For most cases, `:none` is the best choice. It runs tests sequentially but most test cases finishes within a reasonable time.
336
298
 
337
299
  ```ruby
338
300
  Pbt.assert(worker: :none) do
@@ -351,8 +313,8 @@ Once this project finishes the following, we will release v1.0.0.
351
313
  - [x] Support shrinking
352
314
  - [x] Support multiple concurrency methods
353
315
  - [x] Ractor
354
- - [x] Process
355
- - [x] Thread
316
+ - [x] Process (dropped)
317
+ - [x] Thread (dropped)
356
318
  - [x] None (Run tests sequentially)
357
319
  - [x] Documentation
358
320
  - [x] Add better examples
@@ -360,11 +322,13 @@ Once this project finishes the following, we will release v1.0.0.
360
322
  - [x] Configuration
361
323
  - [x] Benchmark
362
324
  - [x] Rich report by verbose mode
363
- - [x] (Partially) Allow to use expectations and matchers provided by test framework in Ractor if possible.
325
+ - [x] (Partially) Allow to use expectations and matchers provided by test framework in Ractor. (dropped)
364
326
  - It'd be so hard to pass assertions like `expect`, `assert` to a Ractor.
365
327
  - [ ] Implement frequency arbitrary
366
328
  - [ ] Statistics feature to aggregate generated values
367
329
  - [ ] Decide DSL
330
+ - [ ] Try Fiber
331
+ - [ ] Stateful property-based testing
368
332
 
369
333
  ## Development
370
334
 
@@ -17,26 +17,6 @@ Benchmark.ips do |x|
17
17
  # noop
18
18
  end
19
19
 
20
- x.report("process") do
21
- Pbt.assert(worker: :process, seed:, num_runs: 100) do
22
- Pbt.property(Pbt.integer) do |x|
23
- task(x)
24
- end
25
- end
26
- rescue Pbt::PropertyFailure
27
- # noop
28
- end
29
-
30
- x.report("thread") do
31
- Pbt.assert(worker: :thread, seed:, num_runs: 100) do
32
- Pbt.property(Pbt.integer) do |x|
33
- task(x)
34
- end
35
- end
36
- rescue Pbt::PropertyFailure
37
- # noop
38
- end
39
-
40
20
  x.report("none") do
41
21
  Pbt.assert(worker: :none, seed:, num_runs: 100) do
42
22
  Pbt.property(Pbt.integer) do |x|
@@ -25,22 +25,6 @@ Benchmark.ips do |x|
25
25
  end
26
26
  end
27
27
 
28
- x.report("process") do
29
- Pbt.assert(worker: :process, num_runs: 100) do
30
- Pbt.property(Pbt.constant([a, b, c])) do |x, y, z|
31
- task(x, y, z)
32
- end
33
- end
34
- end
35
-
36
- x.report("thread") do
37
- Pbt.assert(worker: :thread, num_runs: 100) do
38
- Pbt.property(Pbt.constant([a, b, c])) do |x, y, z|
39
- task(x, y, z)
40
- end
41
- end
42
- end
43
-
44
28
  x.report("none") do
45
29
  Pbt.assert(worker: :none, num_runs: 100) do
46
30
  Pbt.property(Pbt.constant([a, b, c])) do |x, y, z|
@@ -18,22 +18,6 @@ Benchmark.ips do |x|
18
18
  end
19
19
  end
20
20
 
21
- x.report("process") do
22
- Pbt.assert(worker: :process, seed:, num_runs: 100) do
23
- Pbt.property(Pbt.ascii_string) do |str|
24
- task(str)
25
- end
26
- end
27
- end
28
-
29
- x.report("thread") do
30
- Pbt.assert(worker: :thread, seed:, num_runs: 100) do
31
- Pbt.property(Pbt.ascii_string) do |str|
32
- task(str)
33
- end
34
- end
35
- end
36
-
37
21
  x.report("none") do
38
22
  Pbt.assert(worker: :none, seed:, num_runs: 100) do
39
23
  Pbt.property(Pbt.ascii_string) do |str|
@@ -14,22 +14,6 @@ Benchmark.ips do |x|
14
14
  end
15
15
  end
16
16
 
17
- x.report("process") do
18
- Pbt.assert(worker: :process, seed:, num_runs: 100) do
19
- Pbt.property(Pbt.integer) do |x|
20
- task(x)
21
- end
22
- end
23
- end
24
-
25
- x.report("thread") do
26
- Pbt.assert(worker: :thread, seed:, num_runs: 100) do
27
- Pbt.property(Pbt.integer) do |x|
28
- task(x)
29
- end
30
- end
31
- end
32
-
33
17
  x.report("none") do
34
18
  Pbt.assert(worker: :none, seed:, num_runs: 100) do
35
19
  Pbt.property(Pbt.integer) do |x|
@@ -92,7 +92,7 @@ module Pbt
92
92
  one_of(*HEXA_CHARS)
93
93
  end
94
94
 
95
- # For lowercase hexadecimal stings.
95
+ # For lowercase hexadecimal strings.
96
96
  #
97
97
  # @see Pbt.array
98
98
  # @param kwargs [Hash] Options for ArrayArbitrary. See `.array` for more information.
@@ -9,22 +9,19 @@ module Pbt
9
9
  :num_runs,
10
10
  :seed,
11
11
  :thread_report_on_exception,
12
- :experimental_ractor_rspec_integration,
13
12
  keyword_init: true
14
13
  ) do
15
14
  # @param verbose [Boolean] Whether to print verbose output. Default is `false`.
16
- # @param worker [Symbol] The concurrency method to use. :ractor`, `:thread`, `:process` and `:none` are supported. Default is `:none`.
15
+ # @param worker [Symbol] The concurrency method to use. :ractor` and `:none` are supported. Default is `:none`.
17
16
  # @param num_runs [Integer] The number of runs to perform. Default is `100`.
18
17
  # @param seed [Integer] The seed to use for random number generation. It's useful to reproduce failed test with the seed you'd pick up from failure messages. Default is a random seed.
19
18
  # @param thread_report_on_exception [Boolean] Whether to report exceptions in threads. It's useful to suppress error logs on Ractor that reports many errors. Default is `false`.
20
- # @param experimental_ractor_rspec_integration [Boolean] Whether to allow RSpec expectation and matchers in Ractor. It's quite experimental! Default is `false`.
21
19
  def initialize(
22
20
  verbose: false,
23
21
  worker: :none,
24
22
  num_runs: 100,
25
- seed: Random.new.seed,
26
- thread_report_on_exception: false,
27
- experimental_ractor_rspec_integration: false
23
+ seed: Random.new_seed,
24
+ thread_report_on_exception: false
28
25
  )
29
26
  super
30
27
  end
@@ -54,8 +54,6 @@ module Pbt
54
54
  #
55
55
  # - `:thread_report_on_exception`
56
56
  # So many exception reports happen in Ractor and a console gets too messy. Suppress them to avoid that.
57
- # - `:experimental_ractor_rspec_integration`
58
- # Allow to use Ractor with RSpec. This is an experimental feature and it's not stable.
59
57
  #
60
58
  # @param config [Hash] Configuration parameters.
61
59
  # @param property [Property]
@@ -64,21 +62,12 @@ module Pbt
64
62
  if config[:worker] == :ractor
65
63
  original_report_on_exception = Thread.report_on_exception
66
64
  Thread.report_on_exception = config[:thread_report_on_exception]
67
-
68
- if config[:experimental_ractor_rspec_integration]
69
- require "pbt/check/rspec_adapter/integration"
70
- class << property
71
- include Pbt::Check::RSpecAdapter::PropertyExtension
72
- end
73
- property.setup_rspec_integration
74
- end
75
65
  end
76
66
 
77
67
  yield
78
68
  ensure
79
69
  if config[:worker] == :ractor
80
70
  Thread.report_on_exception = original_report_on_exception
81
- property.teardown_rspec_integration if config[:experimental_ractor_rspec_integration]
82
71
  end
83
72
  end
84
73
 
@@ -96,10 +85,6 @@ module Pbt
96
85
  run_it_in_sequential(property, runner)
97
86
  in :ractor
98
87
  run_it_in_ractors(property, runner)
99
- in :process
100
- run_it_in_processes(property, runner)
101
- in :thread
102
- run_it_in_threads(property, runner)
103
88
  end
104
89
  end
105
90
  runner.run_execution
@@ -133,80 +118,11 @@ module Pbt
133
118
  c.ractor.take
134
119
  runner.handle_result(c)
135
120
  rescue => e
136
- handle_ractor_error(e.cause, c)
121
+ c.exception = e.cause # Ractor error is wrapped in a Ractor::RemoteError. We need to get the cause.
137
122
  runner.handle_result(c)
138
123
  break # Ignore the rest of the cases. Just pick up the first failure.
139
124
  end
140
125
  end
141
-
142
- def handle_ractor_error(cause, c)
143
- # Ractor error is wrapped in a Ractor::RemoteError. We need to get the cause.
144
- unless defined?(Pbt::Check::RSpecAdapter) && cause.is_a?(Pbt::Check::RSpecAdapter::ExpectationNotMet) # Unknown error.
145
- c.exception = cause
146
- return
147
- end
148
-
149
- # Convert Pbt's custom error to RSpec's error.
150
- begin
151
- RSpec::Expectations::ExpectationHelper.handle_failure(cause.matcher, cause.custom_message, cause.failure_message_method)
152
- rescue RSpec::Expectations::ExpectationNotMetError => e # The class inherits Exception, not StandardError.
153
- c.exception = e
154
- end
155
- end
156
-
157
- # @param property [Property] Property to test.
158
- # @param runner [RunnerIterator]
159
- # @return [void]
160
- def run_it_in_threads(property, runner)
161
- require_parallel
162
-
163
- Parallel.map_with_index(runner, in_threads: Parallel.processor_count) do |val, index|
164
- Case.new(val:, index:).tap do |c|
165
- property.run(val)
166
- # Catch all exceptions including RSpec's ExpectationNotMet (It inherits Exception).
167
- rescue Exception => e # standard:disable Lint/RescueException:
168
- c.exception = e
169
- # It's possible to break this loop here by raising `Parallel::Break`.
170
- # But if it raises, we cannot fetch all cases' result. So this loop continues until the end.
171
- end
172
- end.each do |c|
173
- runner.handle_result(c)
174
- break if c.exception
175
- # Ignore the rest of the cases. Just pick up the first failure.
176
- end
177
- end
178
-
179
- # @param property [Property] Property to test.
180
- # @param runner [RunnerIterator]
181
- # @return [void]
182
- def run_it_in_processes(property, runner)
183
- require_parallel
184
-
185
- Parallel.map_with_index(runner, in_processes: Parallel.processor_count) do |val, index|
186
- Case.new(val:, index:).tap do |c|
187
- property.run(val)
188
- # Catch all exceptions including RSpec's ExpectationNotMet (It inherits Exception).
189
- rescue Exception => e # standard:disable Lint/RescueException:
190
- c.exception = e
191
- # It's possible to break this loop here by raising `Parallel::Break`.
192
- # But if it raises, we cannot fetch all cases' result. So this loop continues until the end.
193
- end
194
- end.each do |c|
195
- runner.handle_result(c)
196
- break if c.exception
197
- # Ignore the rest of the cases. Just pick up the first failure.
198
- end
199
- end
200
-
201
- # Load Parallel gem. If it's not installed, raise an error.
202
- # @see https://github.com/grosser/parallel
203
- # @raise [InvalidConfiguration]
204
- def require_parallel
205
- require "parallel"
206
- rescue LoadError
207
- raise InvalidConfiguration,
208
- "Parallel gem (https://github.com/grosser/parallel) is required to use worker `:process` or `:thread`. Please add `gem 'parallel'` to your Gemfile."
209
- end
210
126
  end
211
127
  end
212
128
  end
data/lib/pbt/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pbt
4
- VERSION = "0.4.1"
4
+ VERSION = "0.5.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pbt
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.1
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - ohbarye
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2024-05-09 00:00:00.000000000 Z
11
+ date: 2024-12-30 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description:
14
14
  email:
@@ -46,9 +46,6 @@ files:
46
46
  - lib/pbt/check/case.rb
47
47
  - lib/pbt/check/configuration.rb
48
48
  - lib/pbt/check/property.rb
49
- - lib/pbt/check/rspec_adapter/integration.rb
50
- - lib/pbt/check/rspec_adapter/predicate_block_inspector.rb
51
- - lib/pbt/check/rspec_adapter/property_extension.rb
52
49
  - lib/pbt/check/runner_iterator.rb
53
50
  - lib/pbt/check/runner_methods.rb
54
51
  - lib/pbt/check/tosser.rb
@@ -79,7 +76,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
79
76
  - !ruby/object:Gem::Version
80
77
  version: '0'
81
78
  requirements: []
82
- rubygems_version: 3.5.3
79
+ rubygems_version: 3.5.21
83
80
  signing_key:
84
81
  specification_version: 4
85
82
  summary: Property-Based Testing tool for Ruby, utilizing Ractor for parallelizing
@@ -1,64 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- unless defined?(RSpec)
4
- raise InvalidConfigurationError, "You configured `experimental_ractor_rspec_integration: true` but RSpec is not loaded. Please use RSpec or set the config `false`."
5
- end
6
-
7
- require "pbt/check/rspec_adapter/predicate_block_inspector"
8
- require "pbt/check/rspec_adapter/property_extension"
9
-
10
- module Pbt
11
- module Check
12
- # @private
13
- module RSpecAdapter
14
- # This custom error contains RSpec matcher and message to handle Pbt's runner.
15
- # @private
16
- class ExpectationNotMet < StandardError
17
- attr_reader :matcher, :custom_message, :failure_message_method
18
-
19
- def initialize(msg, matcher, custom_message, failure_message_method)
20
- super(msg)
21
- @matcher = matcher
22
- @custom_message = custom_message
23
- @failure_message_method = failure_message_method
24
- end
25
- end
26
- end
27
- end
28
- end
29
-
30
- # `autoload` is not allowed in Ractor but RSpec uses autoload for matchers.
31
- # We need to load them in advance in order to be able to use them in Ractor.
32
- #
33
- # e.g. Ractor raises... `be_a_kind_of': require by autoload on non-main Ractor is not supported (BeAKindOf) (Ractor::UnsafeError)
34
- RSpec::Matchers::BuiltIn.constants.each { |c| Object.const_get("RSpec::Matchers::BuiltIn::#{c}") }
35
-
36
- # TODO: preload more helpers like aggregate_failures.
37
- # RSpec::Expectations.constants.each { |c| Object.const_get("RSpec::Expectations::#{c}") }
38
- # The code above is not enough. Even if we run this code in advance, Ractor raises...
39
- # in `failure_notifier': can not access non-shareable objects in constant RSpec::Support::DEFAULT_FAILURE_NOTIFIER by non-main ractor. (Ractor::IsolationError)
40
-
41
- # CAUTION: This is a dirty hack! We need to override the original method to make it Ractor-safe.
42
- RSpec::Expectations::ExpectationHelper.singleton_class.prepend(Module.new do
43
- def with_matcher(handler, matcher, message)
44
- check_message(message)
45
- matcher = modern_matcher_from(matcher)
46
- yield matcher
47
- ensure
48
- # The original method is not Ractor-safe unless stopping assigning these class variables.
49
- if Ractor.current == Ractor.main
50
- ::RSpec::Matchers.last_expectation_handler = handler
51
- ::RSpec::Matchers.last_matcher = matcher
52
- end
53
- end
54
-
55
- def handle_failure(matcher, message, failure_message_method)
56
- # This method is not Ractor-safe. RSpec::Support::ObjectFormatter.default_instance assigns class variables.
57
- # If this method is called in non-main-Ractor, it raises a custom error and let it handle in the main Ractor.
58
- if Ractor.current != Ractor.main
59
- raise Pbt::Check::RSpecAdapter::ExpectationNotMet.new(nil, matcher, message, failure_message_method)
60
- end
61
-
62
- super
63
- end
64
- end)
@@ -1,47 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- begin
4
- # Use prism to get user-defined block code.
5
- require "prism"
6
- rescue LoadError
7
- raise InvalidConfiguration,
8
- "Prism gem (https://github.com/grosser/parallel) is required to use worker `:rator` and `:experimental_ractor_rspec_integration`. Please add `gem 'parallel'` to your Gemfile."
9
- end
10
-
11
- module Pbt
12
- module Check
13
- module RSpecAdapter
14
- # This class is used to get user-defined block code.
15
- # If a user defines code like below:
16
- #
17
- # Pbt.property(Pbt.integer, Pbt.integer) do |x, y|
18
- # x > 0 && y > 0
19
- # end
20
- #
21
- # inspector.method_params #=> "x, y"
22
- # inspector.method_body #=> "x > 0 && y > 0"
23
- #
24
- # @private
25
- # @!attribute [r] method_body
26
- # @!attribute [r] method_params
27
- class PredicateBlockInspector < Prism::Visitor
28
- attr_reader :method_body, :method_params
29
-
30
- def initialize(line)
31
- @line = line
32
- @method_body = nil
33
- super()
34
- end
35
-
36
- def visit_call_node(node)
37
- if node.name == :property && node.block.opening_loc.start_line == @line
38
- @method_params = node.block.parameters.parameters.slice
39
- @method_body = node.block.body.slice
40
- end
41
-
42
- super
43
- end
44
- end
45
- end
46
- end
47
- end
@@ -1,74 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Pbt
4
- module Check
5
- module RSpecAdapter
6
- # @private
7
- module PropertyExtension
8
- # Define an original class to be called in Ractor.
9
- #
10
- # @return [void]
11
- def setup_rspec_integration
12
- filepath, line = @predicate.source_location
13
- basename = File.basename(filepath, ".rb")
14
- @class_name = "Test" + basename.split("_").map(&:capitalize).join + line.to_s
15
- @method_name = "predicate_#{basename}_#{line}"
16
- define_ractor_callable_class
17
- end
18
-
19
- # Clean up an original class to be called in Ractor to avoid any persisted namespace pollution.
20
- #
21
- # @return [void]
22
- def teardown_rspec_integration
23
- RSpecAdapter.__send__(:remove_const, @class_name) if RSpecAdapter.const_defined?(@class_name)
24
- end
25
-
26
- # Run the predicate with the generated `val`.
27
- # This overrides the original `Property#run_in_ractor`.
28
- #
29
- # @param val [Object]
30
- # @return [Ractor]
31
- def run_in_ractor(val)
32
- Ractor.new(@class_name, @method_name, @predicate.parameters.size, val) do |class_name, method_name, param_size, val|
33
- klass = RSpecAdapter.const_get(class_name)
34
- if val.is_a?(Hash)
35
- klass.new.send(method_name, **val)
36
- elsif param_size >= 2
37
- klass.new.send(method_name, *val)
38
- else
39
- klass.new.send(method_name, val)
40
- end
41
- end
42
- end
43
-
44
- private
45
-
46
- # @return [void]
47
- def define_ractor_callable_class
48
- # The @method_name is invisible in the Class.new block, so we need to assign it to a local variable.
49
- method_name = @method_name
50
-
51
- inspector = extract_predicate_source_code
52
-
53
- RSpecAdapter.const_set(@class_name, Class.new do
54
- include ::RSpec::Matchers
55
- class_eval <<-RUBY, __FILE__, __LINE__ + 1
56
- def #{method_name}(#{inspector.method_params})
57
- #{inspector.method_body}
58
- end
59
- RUBY
60
- end)
61
- end
62
-
63
- # @return [PredicateBlockInspector]
64
- def extract_predicate_source_code
65
- filepath, line = @predicate.source_location
66
- PredicateBlockInspector.new(line).tap do |inspector|
67
- res = Prism.parse_file(filepath)
68
- res.value.statements.accept(inspector)
69
- end
70
- end
71
- end
72
- end
73
- end
74
- end