propr 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- data/NOTES.md +62 -0
- data/README.md +553 -0
- data/Rakefile +83 -0
- data/TODO.md +64 -0
- data/lib/propr.rb +123 -0
- data/lib/propr/dsl.rb +6 -0
- data/lib/propr/dsl/check.rb +49 -0
- data/lib/propr/dsl/property.rb +62 -0
- data/lib/propr/property.rb +23 -0
- data/lib/propr/random.rb +143 -0
- data/lib/propr/random/array.rb +19 -0
- data/lib/propr/random/bigdecimal.rb +43 -0
- data/lib/propr/random/boolean.rb +7 -0
- data/lib/propr/random/complex.rb +0 -0
- data/lib/propr/random/date.rb +17 -0
- data/lib/propr/random/float.rb +60 -0
- data/lib/propr/random/hash.rb +55 -0
- data/lib/propr/random/integer.rb +38 -0
- data/lib/propr/random/maybe.rb +0 -0
- data/lib/propr/random/nil.rb +8 -0
- data/lib/propr/random/range.rb +32 -0
- data/lib/propr/random/rational.rb +0 -0
- data/lib/propr/random/set.rb +22 -0
- data/lib/propr/random/string.rb +41 -0
- data/lib/propr/random/symbol.rb +13 -0
- data/lib/propr/random/time.rb +14 -0
- data/lib/propr/rspec.rb +97 -0
- data/lib/propr/runner.rb +53 -0
- data/lib/propr/shrink/array.rb +16 -0
- data/lib/propr/shrink/bigdecimal.rb +17 -0
- data/lib/propr/shrink/boolean.rb +11 -0
- data/lib/propr/shrink/complex.rb +0 -0
- data/lib/propr/shrink/date.rb +12 -0
- data/lib/propr/shrink/float.rb +17 -0
- data/lib/propr/shrink/hash.rb +18 -0
- data/lib/propr/shrink/integer.rb +10 -0
- data/lib/propr/shrink/maybe.rb +11 -0
- data/lib/propr/shrink/nil.rb +5 -0
- data/lib/propr/shrink/object.rb +5 -0
- data/lib/propr/shrink/range.rb +4 -0
- data/lib/propr/shrink/rational.rb +4 -0
- data/lib/propr/shrink/set.rb +18 -0
- data/lib/propr/shrink/string.rb +19 -0
- data/lib/propr/shrink/symbol.rb +5 -0
- data/lib/propr/shrink/time.rb +9 -0
- data/spec/examples/choose/array.example +12 -0
- data/spec/examples/choose/hash.example +12 -0
- data/spec/examples/choose/range.example +13 -0
- data/spec/examples/choose/set.example +12 -0
- data/spec/examples/guard.example +38 -0
- data/spec/examples/random/array.example +38 -0
- data/spec/examples/random/hash.example +18 -0
- data/spec/examples/random/integer.example +23 -0
- data/spec/examples/random/range.example +43 -0
- data/spec/examples/scale.example +17 -0
- data/spec/examples/shrink/array.example +20 -0
- data/spec/examples/shrink/bigdecimal.example +20 -0
- data/spec/examples/shrink/float.example +20 -0
- data/spec/examples/shrink/hash.example +20 -0
- data/spec/examples/shrink/integer.example +21 -0
- data/spec/examples/shrink/maybe.example +24 -0
- data/spec/examples/shrink/set.example +21 -0
- data/spec/examples/shrink/string.example +17 -0
- data/spec/issues/003.example +9 -0
- data/spec/spec_helper.rb +24 -0
- metadata +143 -0
data/NOTES.md
ADDED
@@ -0,0 +1,62 @@
|
|
1
|
+
## Property DSL
|
2
|
+
|
3
|
+
Properties are basically just functions. They look like this:
|
4
|
+
|
5
|
+
lambda{|a,...| ... }
|
6
|
+
|
7
|
+
## Property DSL
|
8
|
+
|
9
|
+
* `error?(ex) { ... }`
|
10
|
+
True if code block throws an exception
|
11
|
+
|
12
|
+
* `guard(cond)`
|
13
|
+
Skip test for inputs that don't meet the condition
|
14
|
+
|
15
|
+
* `label(str)`
|
16
|
+
Classify each invocation of the property
|
17
|
+
|
18
|
+
## Random DSL
|
19
|
+
|
20
|
+
* `scale(n, zero)`
|
21
|
+
Scale the numeric value closer to zero, using current factor 0..1
|
22
|
+
|
23
|
+
* `guard(cond)`
|
24
|
+
Supress the generated value unless a condition is met
|
25
|
+
|
26
|
+
* `rand(limit)`
|
27
|
+
Generate a random number using `Kernel.rand`
|
28
|
+
|
29
|
+
## Wiring
|
30
|
+
|
31
|
+
Define boolean property
|
32
|
+
|
33
|
+
>> f = lambda{|a,b,c| a + (b + c) == (a + b) + c }
|
34
|
+
=> #<Proc:...>
|
35
|
+
|
36
|
+
>> p = Propr::Property.new("assoc", Propr::Dsl::Property.new(f))
|
37
|
+
=> #<Propr::Property:...>
|
38
|
+
|
39
|
+
Define generator of random data
|
40
|
+
|
41
|
+
>> c = lambda do
|
42
|
+
bind(Integer.random) do |a|
|
43
|
+
bind(Integer.random) do |b|
|
44
|
+
bind(Integer.random) do |c|
|
45
|
+
unit([a, b, c]);
|
46
|
+
end; end; end; end
|
47
|
+
=> #<Proc:...>
|
48
|
+
|
49
|
+
>> c = Propr::Dsl::Check.wrap(r)
|
50
|
+
=> #<Proc:...>
|
51
|
+
|
52
|
+
Generating a random input
|
53
|
+
|
54
|
+
>> Propr::Random.eval r
|
55
|
+
=> [-1473273057635678493, 3003717222078111739, -4075345202237457298]
|
56
|
+
|
57
|
+
Simple API for testing the property with random data
|
58
|
+
|
59
|
+
>> p.check { Propr::Random.eval c }
|
60
|
+
=> true
|
61
|
+
|
62
|
+
|
data/README.md
ADDED
@@ -0,0 +1,553 @@
|
|
1
|
+
## Introduction
|
2
|
+
|
3
|
+
The usual approach to testing software is to describe a set of test inputs
|
4
|
+
and their corresponding expected outputs. The program is run with these
|
5
|
+
inputs and the actual outputs are compared with the expected outputs to
|
6
|
+
ensure the program behaves as expected. This methodology is simple to
|
7
|
+
implement and automate, but suffers from problems like:
|
8
|
+
|
9
|
+
* Writing test cases is tedious.
|
10
|
+
* Non-obvious edge cases aren't tested.
|
11
|
+
* Code coverage tools alone don't provide much assurance.
|
12
|
+
|
13
|
+
Property-based testing is an alternative, and complementary, approach in
|
14
|
+
which the general relationships between program inputs and desired output
|
15
|
+
are expressed, rather than enumerating particular inputs and outputs. The
|
16
|
+
properties specify things like, "assuming the program is correct, when its
|
17
|
+
run with any valid inputs, the inputs and the program output are related by
|
18
|
+
`f(input, output)`". The test framework produces random (valid) inputs,
|
19
|
+
searching for a counterexample.
|
20
|
+
|
21
|
+
## Installation
|
22
|
+
|
23
|
+
There are a few things I'd like to fix before publishing this as a gem. Until
|
24
|
+
then, you can install directly from the git repo using Bundler, with this in
|
25
|
+
your Gemfile:
|
26
|
+
|
27
|
+
gem "propr", git: "git@github.com:kputnam/propr.git", branch: "rewrite"
|
28
|
+
|
29
|
+
You'll probably want to specify the current tag, also (eg, `..., tag: "v0.1.0"`)
|
30
|
+
|
31
|
+
## Properties
|
32
|
+
|
33
|
+
The following example demonstrates testing a property with a specific input,
|
34
|
+
then generalizing the test for any input.
|
35
|
+
|
36
|
+
```ruby
|
37
|
+
describe Array do
|
38
|
+
include Propr::RSpec
|
39
|
+
|
40
|
+
describe "#+" do
|
41
|
+
# Traditional unit test
|
42
|
+
it "sums lengths" do
|
43
|
+
xs = [100, 200, 300]
|
44
|
+
ys = [400, 500]
|
45
|
+
(xs + ys).length.should == xs.length + ys.length
|
46
|
+
end
|
47
|
+
|
48
|
+
# Property-based test
|
49
|
+
property("sums lengths"){|xs, ys| (xs + ys).length == xs.length + ys.length }
|
50
|
+
.check([100, 200, 300], [500, 200])
|
51
|
+
.check{ sequence [Array.random { Integer.random }, Array.random { Integer.random }] }
|
52
|
+
end
|
53
|
+
end
|
54
|
+
```
|
55
|
+
|
56
|
+
The following example is similar, but contains an error in specification
|
57
|
+
|
58
|
+
```ruby
|
59
|
+
describe Array do
|
60
|
+
include Propr::RSpec
|
61
|
+
|
62
|
+
describe "#|" do
|
63
|
+
# Traditional unit test
|
64
|
+
it "sums lengths" do
|
65
|
+
xs = [100, 200, 300]
|
66
|
+
ys = [400, 500]
|
67
|
+
(xs | ys).length.should == xs.length + ys.length
|
68
|
+
end
|
69
|
+
|
70
|
+
# Property-based test
|
71
|
+
property("sums lengths"){|xs, ys| (xs | ys).length == xs.length + ys.length }
|
72
|
+
.check([100, 200, 300], [400, 500])
|
73
|
+
.check{ sequence [Array.random{Integer.random(min:0, max:50)}]*2 }
|
74
|
+
end
|
75
|
+
end
|
76
|
+
```
|
77
|
+
|
78
|
+
When this specification is executed, the following error is reported.
|
79
|
+
|
80
|
+
$ rake spec
|
81
|
+
..F
|
82
|
+
|
83
|
+
Failures:
|
84
|
+
|
85
|
+
1) Array#| sums lengths
|
86
|
+
Failure/Error: raise Falsifiable.new(counterex, m.shrink(counterex), passed, skipped)
|
87
|
+
Propr::Falsifiable:
|
88
|
+
input: [25, 24], [24, 27]
|
89
|
+
shrunken: [], [0, 0]
|
90
|
+
after: 49 passed, 0 skipped
|
91
|
+
# ./lib/propr/rspec.rb:29:in `block in check'
|
92
|
+
|
93
|
+
Finished in 0.22829 seconds
|
94
|
+
3 examples, 1 failure
|
95
|
+
|
96
|
+
You may have figured out the error is that `|` removes duplicate elements
|
97
|
+
from the result. We might not have caught the mistake by writing individual
|
98
|
+
test cases. The output indicates Propr generated 49 sets of input before
|
99
|
+
finding one that failed.
|
100
|
+
|
101
|
+
Now that a failing test case has been identified, you might write another
|
102
|
+
`check` with those specific inputs to prevent regressions.
|
103
|
+
|
104
|
+
You could also print the initial random seed like this and when a test fails,
|
105
|
+
explicitly set the random seed to regenerate the same inputs for the entire
|
106
|
+
test suite:
|
107
|
+
|
108
|
+
$ cat spec/spec_helper.rb
|
109
|
+
RSpec.configure do |config|
|
110
|
+
srand.tap{|seed| puts "Random seed is #{seed}"; srand seed }
|
111
|
+
end
|
112
|
+
|
113
|
+
$ rake spec
|
114
|
+
Random seed is 146211424375622429406889408197139382303
|
115
|
+
..F
|
116
|
+
|
117
|
+
Failures:
|
118
|
+
|
119
|
+
1) Array#| sums lengths
|
120
|
+
Failure/Error: raise Falsifiable.new(counterex, m.shrink(counterex), passed, skipped)
|
121
|
+
Propr::Falsifiable:
|
122
|
+
input: [25, 24], [24, 27]
|
123
|
+
shrunken: [], [0, 0]
|
124
|
+
after: 49 passed, 0 skipped
|
125
|
+
|
126
|
+
Finished in 0.22829 seconds
|
127
|
+
3 examples, 1 failure
|
128
|
+
|
129
|
+
Now change spec\_helper.rb to explicitly set the random seed:
|
130
|
+
|
131
|
+
$ cat spec/spec_helper.rb
|
132
|
+
RSpec.configure do |config|
|
133
|
+
srand 146211424375622429406889408197139382303
|
134
|
+
srand.tap{|seed| puts "Random seed is #{seed}"; srand seed }
|
135
|
+
end
|
136
|
+
|
137
|
+
$ rake spec
|
138
|
+
Random seed is 146211424375622429406889408197139382303
|
139
|
+
|
140
|
+
The remaining output should be identical every time you run the suite, so
|
141
|
+
long as specs are in the same order each time.
|
142
|
+
|
143
|
+
### Just Plain Functions
|
144
|
+
|
145
|
+
Properties are basically just functions, they should return `true` or `false`.
|
146
|
+
|
147
|
+
p = Propr::Property.new("name", lambda{|a,b| a + b == b + a })
|
148
|
+
|
149
|
+
You can invoke a property using `#check`. Like lambdas and procs, you can also
|
150
|
+
invoke them using `#call` or `#[]`.
|
151
|
+
|
152
|
+
p.check(3, 4) #=> true
|
153
|
+
p.check("x", "y") #=> false
|
154
|
+
|
155
|
+
But you can also invoke them by yielding a function that generates random inputs.
|
156
|
+
|
157
|
+
m = Propr::Random
|
158
|
+
p.check { m.eval(m.sequence [Integer.random, Float.random]) } #=> true
|
159
|
+
p.check { m.eval(m.sequence [String.random , String.random]) } #=> false
|
160
|
+
|
161
|
+
When invoked with a block, `check` will run `p` with 100 random inputs by
|
162
|
+
default, but you can also pass an argument to `check` indicating how many
|
163
|
+
examples `p` should be tested against.
|
164
|
+
|
165
|
+
## Using Propr + Test Frameworks
|
166
|
+
|
167
|
+
Mixing in a module magically defines the `property` singleton method, so
|
168
|
+
you can use it to generate test cases.
|
169
|
+
|
170
|
+
```ruby
|
171
|
+
describe "foo" do
|
172
|
+
include Propr::RSpec
|
173
|
+
|
174
|
+
# This defines three test cases, one per each `check`
|
175
|
+
property("length"){|a| a.length >= 0 }
|
176
|
+
check("abc").
|
177
|
+
check("xyz").
|
178
|
+
check{ String.random }
|
179
|
+
end
|
180
|
+
```
|
181
|
+
|
182
|
+
Note your property should still return `true` or `false`. You should *not* use
|
183
|
+
`#should` or `#assert`, because the test generator will generate the assertion
|
184
|
+
for you. This also reduces visual clutter.
|
185
|
+
|
186
|
+
Alternatively, to use Propr with all specification, you can add this to your
|
187
|
+
`spec_helper.rb`
|
188
|
+
|
189
|
+
```ruby
|
190
|
+
RSpec.configure do |config|
|
191
|
+
include Propr::RSpec
|
192
|
+
end
|
193
|
+
```
|
194
|
+
|
195
|
+
### Property DSL
|
196
|
+
|
197
|
+
The code block inside `property { ... }` has an extended scope that defines
|
198
|
+
a few helpful methods:
|
199
|
+
|
200
|
+
* __guard__: Skip this iteration unless all the given conditions are met. This
|
201
|
+
can be used, for instance, to define a property only on even integers.
|
202
|
+
`property{|x| guard(x.even?); x & 1 == 0 }`
|
203
|
+
|
204
|
+
* __error?__: True if the code block throws an exception of the given type.
|
205
|
+
`property{|x| error? { x / 0 }}`
|
206
|
+
|
207
|
+
* __m__: Short alias for `Propr::Random`, used to generate random data as described
|
208
|
+
below.
|
209
|
+
`property{|x| m.eval(m.sequence([m.unit 0] * x)).length == x }`
|
210
|
+
|
211
|
+
### Check DSL
|
212
|
+
|
213
|
+
The code block inside `check { ... }` should return a generator value. The code
|
214
|
+
block's scope is extended with a few combinators to compose generators.
|
215
|
+
|
216
|
+
* __unit__: Create a generator that returns the given value. For instance, to yield
|
217
|
+
`3` as an argument to the property,
|
218
|
+
`check { unit(3) }`
|
219
|
+
|
220
|
+
* __bind__: Chain the value yielded by one generator into another. For instance, to
|
221
|
+
yield two integers as arguments to a property,
|
222
|
+
`check { bind(Integer.random){|a| bind(Integer.random){|b| unit([a,b]) }}}`
|
223
|
+
|
224
|
+
* __guard__: Short-circuit the chain if the given condition is false. The entire chain
|
225
|
+
will be re-run until the guard passes. For instance, to generate two distinct numbers,
|
226
|
+
`check { bind(Integer.random){|a| bind(Integer.random){|b| guard(a != b){ unit([a,b]) }}}}`
|
227
|
+
|
228
|
+
* __join__: Remove one level of generator nesting. If you have a generator `x` that
|
229
|
+
*yields* a number generator, then `join x` is a number generator. For instance, to yield
|
230
|
+
either a number or a string,
|
231
|
+
`check { join([Integer.random, String.random].random) }`
|
232
|
+
|
233
|
+
* __sequence__: Convert a list of generator values to a list generator. For instance, to
|
234
|
+
yield three integers to a property,
|
235
|
+
`check { sequence [Integer.random]*3 }`
|
236
|
+
|
237
|
+
## Generating Random Values
|
238
|
+
|
239
|
+
Propr defines a `random` method that returns a generator for most standard
|
240
|
+
Ruby types. You can run the generator using the `Propr::Random.eval` method.
|
241
|
+
|
242
|
+
>> m = Propr::Random
|
243
|
+
=> ...
|
244
|
+
|
245
|
+
>> m.eval(Boolean.random)
|
246
|
+
=> false
|
247
|
+
|
248
|
+
### Boolean
|
249
|
+
|
250
|
+
>> m.eval Boolean.random
|
251
|
+
=> true
|
252
|
+
|
253
|
+
### Date
|
254
|
+
|
255
|
+
>> m.eval(Date.random(min: Date.today - 10, max: Date.today + 10)).to_s
|
256
|
+
=> "2012-03-01"
|
257
|
+
|
258
|
+
Options
|
259
|
+
|
260
|
+
* `min:` minimum value, defaults to 0001-01-01
|
261
|
+
* `max:` maximum value, defaults to 9999-12-31
|
262
|
+
* `center:` defaults to the midpoint between min and max
|
263
|
+
|
264
|
+
### Time
|
265
|
+
|
266
|
+
>> m.eval Time.random(min: Time.now, max: Time.now + 3600)
|
267
|
+
=> 2012-02-20 13:47:57 -0600
|
268
|
+
|
269
|
+
Options
|
270
|
+
|
271
|
+
* `min:` minimum value, defaults to 1000-01-01 00:00:00 UTC
|
272
|
+
* `max:` maximum value, defaults to 9999-12-31 12:59:59 UTC
|
273
|
+
* `center:` defaults to the midpoint between min and max
|
274
|
+
|
275
|
+
### String
|
276
|
+
|
277
|
+
>> m.eval String.random(min: 5, max: 10, charset: :lower)
|
278
|
+
=> "rqyhw"
|
279
|
+
|
280
|
+
Options
|
281
|
+
|
282
|
+
* `min:` minimum size, defaults to 0
|
283
|
+
* `max:` maximum size, defaults to 10
|
284
|
+
* `center:` defaults to the midpoint between min and max
|
285
|
+
* `charset:` regular expression character class, defaults to `/[[:print]]/`
|
286
|
+
|
287
|
+
### Numbers
|
288
|
+
|
289
|
+
#### Integer.random
|
290
|
+
|
291
|
+
>> m.eval Integer.random(min: -500, max: 500)
|
292
|
+
=> -382
|
293
|
+
|
294
|
+
Options
|
295
|
+
|
296
|
+
* `min:` minimum value, defaults to Integer::MIN
|
297
|
+
* `max:` maximum value, defaults to Integer::MAX
|
298
|
+
* `center:` defaults to the midpoint between min and max.
|
299
|
+
|
300
|
+
#### Float.random
|
301
|
+
|
302
|
+
>> m.eval Float.random(min: -500, max: 500)
|
303
|
+
=> 48.252030464134364
|
304
|
+
|
305
|
+
Options
|
306
|
+
|
307
|
+
* `min:` minimum value, defaults to -Float::MAX
|
308
|
+
* `max:` maximum value, defaults to Float::MAX
|
309
|
+
* `center:` defaults to the midpoint between min and max.
|
310
|
+
|
311
|
+
#### Rational.random
|
312
|
+
|
313
|
+
>> m.eval m.bind(m.sequence [Integer.random]*2){|a,b| unit Rational(a,b) }
|
314
|
+
=> (300421843/443649464)
|
315
|
+
|
316
|
+
Not implemented, as there isn't a nice way to ensure a `min` works. Instead,
|
317
|
+
generate two numeric values and combine them:
|
318
|
+
|
319
|
+
#### BigDecimal.random
|
320
|
+
|
321
|
+
>> m.eval(BigDecimal.random(min: 10, max: 20)).to_s("F")
|
322
|
+
=> "14.934854011762374703280016489856414847259220844969789892"
|
323
|
+
|
324
|
+
Options
|
325
|
+
|
326
|
+
* `min:` minimum value, defaults to -Float::MAX
|
327
|
+
* `max:` maximum value, defaults to Float::MAX
|
328
|
+
* `center:` defaults to the midpoint between min and max
|
329
|
+
|
330
|
+
#### Bignum.random
|
331
|
+
|
332
|
+
>> m.eval Integer.random(min: Integer::MAX, max: Integer::MAX * 2)
|
333
|
+
=> 2015151263
|
334
|
+
|
335
|
+
There's no constructor specifically for Bignum. You can use `Integer.random`
|
336
|
+
and specify `min: Integer::MAX + 1` and some even larger `max` value. Ruby
|
337
|
+
will automatically handle Integer overflow by coercing to Bignum.
|
338
|
+
|
339
|
+
#### Complex.random
|
340
|
+
|
341
|
+
>> m.eval(m.bind(m.sequence [Float.random(min:-10, max:10)]*2){|a,b| m.unit Complex(a,b) })
|
342
|
+
=> (9.806161068637833+7.523520738439842i)
|
343
|
+
|
344
|
+
Not implemented, as there's no simple way to implement min and max, nor the types
|
345
|
+
of the components. Instead, generate two numeric values and combine them:
|
346
|
+
|
347
|
+
### Collections
|
348
|
+
|
349
|
+
The class method `random` returns a generator to construct a collection of
|
350
|
+
elements, while the `#random` instance method returns a generator which returns
|
351
|
+
an element from the collection.
|
352
|
+
|
353
|
+
#### Array.random
|
354
|
+
|
355
|
+
Expects a block parameter that yields a generator for elements.
|
356
|
+
|
357
|
+
>> m.eval Array.random(min:4, max:4) { String.random(min:4, max:4) }
|
358
|
+
=> ["2n #", "UZ1d", "0vF,", "cV_{"]
|
359
|
+
|
360
|
+
Options
|
361
|
+
|
362
|
+
* `min:` minimum size, defaults to 0
|
363
|
+
* `max:` maximum size, defaults to 10
|
364
|
+
* `center:` defaults to the midpoint between min and max
|
365
|
+
|
366
|
+
#### Hash.random
|
367
|
+
|
368
|
+
Expects a block parameter that yields generator of [key, value] pairs.
|
369
|
+
|
370
|
+
>> m.eval Hash.random(min:2, max:4) { m.sequence [Integer.random, m.unit(nil)] }
|
371
|
+
=> {564854752=>nil, -1065292239=>nil, 830081146=>nil}
|
372
|
+
|
373
|
+
Options
|
374
|
+
|
375
|
+
* `min:` minimum size, defaults to 0
|
376
|
+
* `max:` maximum size, defaults to 10
|
377
|
+
* `center:` defaults to the midpoint between min and max
|
378
|
+
|
379
|
+
#### Hash.random_vals
|
380
|
+
|
381
|
+
Expects a hash whose keys are ordinary values, and whose values are
|
382
|
+
generators.
|
383
|
+
|
384
|
+
>> m.eval Hash.random_vals(a: String.random, b: Integer.random)
|
385
|
+
=> {:a=>"Fi?p`g", :b=>4551738453396095365}
|
386
|
+
|
387
|
+
Doesn't accept any options.
|
388
|
+
|
389
|
+
#### Set.random
|
390
|
+
|
391
|
+
Expects a block parameter that yields a generator for elements.
|
392
|
+
|
393
|
+
>> m.eval Set.random(min:4, max:4) { String.random(min:4, max:4) }
|
394
|
+
=> #<Set: {"2n #", "UZ1d", "0vF,", "cV_{"}>
|
395
|
+
|
396
|
+
Options
|
397
|
+
|
398
|
+
* `min:` minimum size, defaults to 0
|
399
|
+
* `max:` maximum size, defaults to 10
|
400
|
+
* `center:` defaults to the midpoint between min and max
|
401
|
+
|
402
|
+
#### Range.random
|
403
|
+
|
404
|
+
Expects __either__ a block parameter __or__ one or both of min and max.
|
405
|
+
|
406
|
+
>> m.eval Range.random(min: 0, max: 100)
|
407
|
+
=> 81..58
|
408
|
+
|
409
|
+
>> m.eval Range.random { Integer.random(min: 0, max: 100) }
|
410
|
+
=> 9..80
|
411
|
+
|
412
|
+
Options
|
413
|
+
|
414
|
+
* `min:` minimum element
|
415
|
+
* `max:` maximum element
|
416
|
+
* `inclusive?:` defaults to true, meaning Range includes max element
|
417
|
+
|
418
|
+
#### Elements from a collection
|
419
|
+
|
420
|
+
The `#random` instance method is defined on the above types. It takes no parameters.
|
421
|
+
|
422
|
+
>> m.eval([1,2,3,4,5].random)
|
423
|
+
=> 4
|
424
|
+
|
425
|
+
>> m.eval({a: 1, b: 2, c: 3, d: 4}.random)
|
426
|
+
=> [:b, 2]
|
427
|
+
|
428
|
+
>> m.eval((0..100).random)
|
429
|
+
=> 12
|
430
|
+
|
431
|
+
>> m.eval(Set.new([1,2,3,4]).random)
|
432
|
+
=> 4
|
433
|
+
|
434
|
+
## Attenuation (limiting the search space for counter examples)
|
435
|
+
|
436
|
+
The `m.eval` method has a second parameter that serves to exponentially reduce
|
437
|
+
the domain for generators, specified with `min:` and `max:` parameters. The scale
|
438
|
+
value may range from `0` to `1`, where `1` causes no change.
|
439
|
+
|
440
|
+
When scale is `0`, the domain is reduced to a single value, which is specified by
|
441
|
+
the `center:` parameter. Usually this defaults to the midpoint between `min:` and
|
442
|
+
`max:`. Any value between `min:` and `max:` can be given for `center:`, in addition
|
443
|
+
to the three symbolic values, `:min`, `:mid`, and `:max`.
|
444
|
+
|
445
|
+
Scale values beteween `0` and `1` adjust the domain exponentially, so a domain with
|
446
|
+
10,000 elements when `scale = 1` will have 1,000 elements when `scale = 0.5` and
|
447
|
+
only 100 when `scale = 0.25`.
|
448
|
+
|
449
|
+
With `scale = 0`, the domain contains at most `10000^0 = 1` elements:
|
450
|
+
|
451
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :min), 0
|
452
|
+
== m.eval Integer.random(min: 0, max: 0)
|
453
|
+
|
454
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :mid), 0
|
455
|
+
== m.eval Integer.random(min: 5000, max: 5000)
|
456
|
+
|
457
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :max), 0
|
458
|
+
== m.eval Integer.random(min: 10000, max: 10000)
|
459
|
+
|
460
|
+
With `scale = 0.25`, the domain contains at most `10000^0.25 = 10` elements:
|
461
|
+
|
462
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :min), 0.25
|
463
|
+
== m.eval Integer.random(min: 0, max: 9)
|
464
|
+
|
465
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :mid), 0.25
|
466
|
+
== m.eval Integer.random(min: 4996, max: 5004)
|
467
|
+
|
468
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :max), 0.25
|
469
|
+
== m.eval Integer.random(min: 9991, max: 10000)
|
470
|
+
|
471
|
+
With `scale = 0.50`, the domain contains at most `10000^0.5 = 100` elements:
|
472
|
+
|
473
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :min), 0.5
|
474
|
+
== m.eval Integer.random(min: 0, max: 99)
|
475
|
+
|
476
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :mid), 0.5
|
477
|
+
== m.eval Integer.random(min: 4951, max: 5048)
|
478
|
+
|
479
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :max), 0.5
|
480
|
+
== m.eval Integer.random(min: 9901, max: 10000)
|
481
|
+
|
482
|
+
With `scale = 0.75`, the domain contains at most `10000^0.75 = 1000` elements:
|
483
|
+
|
484
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :min), 0.75
|
485
|
+
== m.eval Integer.random(min: 0, max: 998)
|
486
|
+
|
487
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :mid), 0.75
|
488
|
+
== m.eval Integer.random(min: 4507, max: 5499)
|
489
|
+
|
490
|
+
>> m.eval Integer.random(min: 0, max: 10000, center: :max), 0.75
|
491
|
+
== m.eval Integer.random(min: 9002, max: 10000)
|
492
|
+
|
493
|
+
### Deepening of the Search Space
|
494
|
+
|
495
|
+
By default, the test framework adapters increase the scale linearly (causing
|
496
|
+
an exponential increase of the domain size) each time the property is tested.
|
497
|
+
|
498
|
+
That is, when running 100 iterations, scale values will be 0.00, 0.01, 0.02,
|
499
|
+
0.03, 0.04, etc. This is intended to test the simplest counterexamples first,
|
500
|
+
and increase the complexity of generated inputs exponentially.
|
501
|
+
|
502
|
+
### Simplification of Counterexamples
|
503
|
+
|
504
|
+
Once a random input has been classified as a counterexample, Propr will
|
505
|
+
search for a simpler counterexample. This is done by iteratively calling
|
506
|
+
`#shrink` on each successively smaller counterexample.
|
507
|
+
|
508
|
+
$ cat shrink.example
|
509
|
+
require "rspec"
|
510
|
+
require "propr"
|
511
|
+
|
512
|
+
RSpec.configure do |config|
|
513
|
+
include Propr::RSpec
|
514
|
+
|
515
|
+
srand 146211424375622429406889408197139382303
|
516
|
+
srand.tap{|seed| puts "Random seed is #{seed}"; srand seed }
|
517
|
+
end
|
518
|
+
|
519
|
+
describe Float do
|
520
|
+
property("assoc"){|x,y,z| (x + y) + z == x + (y + z) }
|
521
|
+
.check(-382863.98514407175, 224121.21177705095, 276118.77134001954)
|
522
|
+
end
|
523
|
+
|
524
|
+
$ rspec shrink.example
|
525
|
+
Random seed is 146211424375622429406889408197139382303
|
526
|
+
F
|
527
|
+
|
528
|
+
Failures:
|
529
|
+
|
530
|
+
1) Float assoc
|
531
|
+
Propr::Falsifiable:
|
532
|
+
input: -382863.98514407175, 224121.21177705095, 276118.77134001954
|
533
|
+
shrunken: -0.007740960460133677, 0.011895728563551701, 3.9765678826328424e-05
|
534
|
+
after: 0 passed, 0 skipped
|
535
|
+
# ./lib/propr/rspec.rb:36:in `block in check'
|
536
|
+
|
537
|
+
Finished in 10.52 seconds
|
538
|
+
1 example, 1 failure
|
539
|
+
|
540
|
+
Notice the output shows a "simpler" counterexample than the inputs we explicitly
|
541
|
+
tested. This becomes useful when testing with more complex data like trees, where
|
542
|
+
it can be difficult to understand which aspect of the counterexample is relevant.
|
543
|
+
|
544
|
+
## More Reading
|
545
|
+
|
546
|
+
* [Presentation at KC Ruby Meetup Group](https://github.com/kputnam/presentations/raw/master/Property-Based-Testing.pdf)
|
547
|
+
|
548
|
+
## Related Projects
|
549
|
+
|
550
|
+
* [Rantly](https://github.com/hayeah/rantly)
|
551
|
+
* [PropER](https://github.com/manopapad/proper)
|
552
|
+
* [QuviQ](http://www.quviq.com/documents/QuviqFlyer.pdf)
|
553
|
+
* [QuickCheck](http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck)
|