hypothesis-specs 0.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: ca2e609a02788991ddea6552d8dde5660b918308
4
+ data.tar.gz: f0c02fb905b411d91a55fece2d5e5117a130d984
5
+ SHA512:
6
+ metadata.gz: 7d3750c2b0a756f38ec82525157b8471d1dab93c0bcf139260da68b37a57d8d9fea3bb8faf9ef947c36534dbb51658563e3d4ff1714e9f9617ac98d6a516b5a2
7
+ data.tar.gz: 732da786e9e63999fd709338edfae86466b807288a787e8f8e0cfed8b667c911eef15424611b2f8c3990fc95f1cf6d97a6ea3a7b45f1c04ce603e128cc4e56ea
data/CHANGELOG.md ADDED
@@ -0,0 +1,10 @@
1
+ ## Hypothesis for Ruby 0.0.3 (2018-02-19)
2
+
3
+ This is an initial developer preview of Hypothesis for Ruby.
4
+ It's ready to use, but isn't yet stable and has significant
5
+ limitations. It is mostly released so that people can easily give
6
+ feedback on the API and implementation, and is likely to change
7
+ substantially before a stable release.
8
+
9
+ Note that while there were some earlier release numbers internally,
10
+ these were pulled. This is the first official release.
data/Cargo.toml ADDED
@@ -0,0 +1,11 @@
1
+ [package]
2
+ name = "hypothesis-ruby-core"
3
+ version = "0.1.0"
4
+ authors = ["David R. MacIver <david@drmaciver.com>"]
5
+
6
+ [lib]
7
+ crate-type = ["cdylib"]
8
+
9
+ [dependencies]
10
+ helix = '0.7.2'
11
+ rand = '0.3'
data/LICENSE.txt ADDED
@@ -0,0 +1,8 @@
1
+ Copyright (c) 2018, David R. MacIver
2
+
3
+ All code in this repository except where explicitly noted otherwise is released
4
+ under the Mozilla Public License v 2.0. You can obtain a copy at http://mozilla.org/MPL/2.0/.
5
+
6
+ Some code in this repository may come from other projects. Where applicable, the
7
+ original copyright and license are noted and any modifications made are released
8
+ dual licensed with the original license.
data/README.markdown ADDED
@@ -0,0 +1,86 @@
1
+ # Hypothesis for Ruby
2
+
3
+ Hypothesis is a powerful, flexible, and easy to use library for *property-based testing*.
4
+
5
+ In property-based testing,
6
+ in contrast to traditional *example-based testing*,
7
+ a test is written not against a single example but as a statement that should hold for any of a range of possible values.
8
+
9
+ ## Usage
10
+
11
+ In Hypothesis for Ruby, a test looks something like this:
12
+
13
+ ```ruby
14
+ require "hypothesis"
15
+
16
+ RSpec.configure do |config|
17
+ config.include(Hypothesis)
18
+ config.include(Hypothesis::Possibilities)
19
+ end
20
+
21
+ RSpec.describe "removing an element from a list" do
22
+ it "results in the element no longer being in the list" do
23
+ hypothesis do
24
+ # Or lists(of: integers, min_size: 1), but this lets us
25
+ # demonstrate assume.
26
+ values = any array(of: integers)
27
+
28
+ # If this is not true then the test will stop here.
29
+ assume values.size > 0
30
+
31
+ to_remove = any element_of(values)
32
+
33
+ values.delete_at(values.index(to_remove))
34
+
35
+ # Will fail if the value ws duplicated in the list.
36
+ expect(values.include?(to_remove)).to be false
37
+
38
+ end
39
+ end
40
+ end
41
+ ```
42
+
43
+ This would then fail with:
44
+
45
+ ```
46
+ 1) removing an element from a list results in the element no longer being in the list
47
+ Failure/Error: expect(values.include?(to_remove)).to be false
48
+
49
+ Given #1: [0, 0]
50
+ Given #2: 0
51
+
52
+ expected false
53
+ got true
54
+ ```
55
+
56
+ The use of RSpec here is incidental:
57
+ Hypothesis for Ruby works just as well with minitest,
58
+ and should work with anything else you care to use.
59
+
60
+ ## Getting Started
61
+
62
+ Hypothesis is not available on rubygems.org as a developer preview.
63
+ If you want to try it today you can use the current development branch by adding the following to your Gemfile:
64
+
65
+ ```ruby
66
+ gem 'hypothesis-specs'
67
+ ```
68
+
69
+ The API is still in flux, so be warned that you should expect it to break on upgrades!
70
+ Right now this is really more to allow you to try it out and provide feedback than something you should expect to rely on.
71
+ The more feedback we get, the sooner it will get here!
72
+
73
+ Note that in order to use Hypothesis for Ruby, you will need a rust toolchain installed.
74
+ Please go to [https://www.rustup.rs](https://www.rustup.rs) and follow the instructions if you do not already have one.
75
+
76
+ ## Project Status
77
+
78
+ Hypothesis for Ruby is currently in an *early alpha* stage.
79
+ It works, and has a solid core set of features, but you should expect to find rough edges,
80
+ it is far from feature complete, and the API makes no promises of backwards compatibility.
81
+
82
+ Right now you should consider it to be more in the spirit of a developer preview.
83
+ You can and should try it out, and hopefully you will find all sorts of interesting bugs in your code by doing so!
84
+ But you'll probably find interesting bugs in Hypothesis too,
85
+ and we'd appreciate you reporting them,
86
+ as well as any just general usability issues or points of confusion you have.
data/Rakefile ADDED
@@ -0,0 +1,145 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'rubygems'
4
+ require 'helix_runtime/build_task'
5
+
6
+ begin
7
+ require 'rspec/core/rake_task'
8
+ RSpec::Core::RakeTask.new(:spec)
9
+
10
+ require 'rake/testtask'
11
+
12
+ Rake::TestTask.new(minitests: :build) do |t|
13
+ t.test_files = FileList['minitests/**/test_*.rb']
14
+ t.verbose = true
15
+ end
16
+
17
+ task test: %i[build spec minitests]
18
+ rescue LoadError
19
+ end
20
+
21
+ # Monkeypatch build to fail on error.
22
+ # See https://github.com/tildeio/helix/issues/133
23
+ module HelixRuntime
24
+ class Project
25
+ alias original_build cargo_build
26
+
27
+ def cargo_build
28
+ raise 'Build failed' unless original_build
29
+ end
30
+ end
31
+ end
32
+
33
+ HelixRuntime::BuildTask.new
34
+
35
+ task :format do
36
+ sh 'bundle exec rubocop -a lib spec minitests ' \
37
+ 'Rakefile hypothesis-specs.gemspec'
38
+ end
39
+
40
+ begin
41
+ require 'yard'
42
+
43
+ YARD::Rake::YardocTask.new(:runyard) do |t|
44
+ t.files = [
45
+ 'lib/hypothesis.rb', 'lib/hypothesis/errors.rb',
46
+ 'lib/hypothesis/possible.rb'
47
+ ]
48
+ t.options = ['--markup=markdown', '--no-private']
49
+ end
50
+
51
+ task doc: :runyard do
52
+ YARD::Registry.load
53
+
54
+ objs = YARD::Registry.select do |o|
55
+ is_private = false
56
+ t = o
57
+ until t.root?
58
+ if t.visibility != :public
59
+ is_private = true
60
+ break
61
+ end
62
+ t = t.parent
63
+ end
64
+
65
+ !is_private && o.docstring.blank?
66
+ end
67
+
68
+ objs.sort_by! { |o| o.name.to_s }
69
+
70
+ unless objs.empty?
71
+ abort "Undocumented objects: #{objs.map(&:name).join(', ')}"
72
+ end
73
+ end
74
+ rescue LoadError
75
+ end
76
+
77
+ task :gem do
78
+ uncommitted = `git ls-files lib/ --others --exclude-standard`.split
79
+ uncommitted_ruby = uncommitted.grep(/\.rb$/)
80
+ uncommitted_ruby.sort!
81
+ unless uncommitted_ruby.empty?
82
+ abort 'Cannot build gem with uncomitted Ruby '\
83
+ "files #{uncommitted_ruby.join(', ')}"
84
+ end
85
+
86
+ sh 'rm -rf hypothesis-specs*.gem'
87
+ sh 'git clean -fdx lib'
88
+ sh 'gem build hypothesis-specs.gemspec'
89
+ end
90
+
91
+ def git(*args)
92
+ sh 'git', *args
93
+ end
94
+
95
+ file 'secrets.tar.enc' => 'secrets' do
96
+ sh 'rm -f secrets.tar secrets.tar.enc'
97
+ sh 'tar -cf secrets.tar secrets'
98
+ sh 'travis encrypt-file secrets.tar'
99
+ end
100
+
101
+ task deploy: :gem do
102
+ on_master = system("git merge-base --is-ancestor HEAD origin/master")
103
+
104
+ unless on_master
105
+ puts 'Not on master, so no deploy'
106
+ next
107
+ end
108
+
109
+ spec = Gem::Specification.load('hypothesis-specs.gemspec')
110
+
111
+ succeeded = system('git', 'tag', spec.version.to_s)
112
+
113
+ unless succeeded
114
+ puts "Looks like we've already done this release."
115
+ next
116
+ end
117
+
118
+ unless File.directory? 'secrets'
119
+ sh 'rm -rf secrets'
120
+ sh 'openssl aes-256-cbc -K $encrypted_b0055249143b_key -iv ' \
121
+ '$encrypted_b0055249143b_iv -in secrets.tar.enc -out secrets.tar -d'
122
+
123
+ sh 'tar -xvf secrets.tar'
124
+ end
125
+
126
+ git('config', 'user.name', 'Travis CI on behalf of David R. MacIver')
127
+ git('config', 'user.email', 'david@drmaciver.com')
128
+ git('config', 'core.sshCommand', 'ssh -i secrets/deploy_key')
129
+ git(
130
+ 'remote', 'add', 'ssh-origin',
131
+ 'git@github.com:HypothesisWorks/hypothesis-ruby.git'
132
+ )
133
+
134
+ sh(
135
+ 'ssh-agent', 'sh', '-c',
136
+ 'chmod 0600 secrets/deploy_key && ssh-add secrets/deploy_key && ' \
137
+ 'git push ssh-origin --tags'
138
+ )
139
+
140
+ sh 'rm -f ~/.gem/credentials'
141
+ sh 'mkdir -p ~/.gem'
142
+ sh 'ln -s $(pwd)/secrets/api_key.yaml ~/.gem/credentials'
143
+ sh 'chmod 0600 ~/.gem/credentials'
144
+ sh 'gem push hypothesis-specs*.gem'
145
+ end
data/ext/Makefile ADDED
@@ -0,0 +1,7 @@
1
+ all:
2
+ cd ..
3
+ rake build
4
+ clean:
5
+ rm -rf ../target
6
+
7
+ install: ;
data/ext/extconf.rb ADDED
@@ -0,0 +1,5 @@
1
+ if !system('cargo --version')
2
+ raise 'Hypothesis requires cargo to be installed (https://www.rust-lang.org/)'
3
+ end
4
+
5
+ require 'rake'
data/lib/hypothesis.rb ADDED
@@ -0,0 +1,223 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'hypothesis/errors'
4
+ require 'hypothesis/possible'
5
+ require 'hypothesis/testcase'
6
+ require 'hypothesis/engine'
7
+ require 'hypothesis/world'
8
+
9
+ # This is the main module for using Hypothesis.
10
+ # It is expected that you will include this in your
11
+ # tests, but its methods are also available on the
12
+ # module itself.
13
+ #
14
+ # The main entry point for using this is the
15
+ # {Hypothesis#hypothesis} method. All of the other
16
+ # methods make sense only inside blocks passed to
17
+ # it.
18
+ module Hypothesis
19
+ # @!visibility private
20
+ HYPOTHESIS_LOCATION = File.dirname(__FILE__)
21
+
22
+ # @!visibility private
23
+ def hypothesis_stable_identifier
24
+ # Attempt to get a "stable identifier" for any any
25
+ # call into hypothesis. We use these to create
26
+ # database keys (or will when we have a database) that
27
+ # are stable across runs, so that when a test that
28
+ # previously failed is rerun, we can fetch and reuse
29
+ # the previous examples.
30
+
31
+ # Note that essentially any answer to this method is
32
+ # "fine" in that the failure mode is that sometiems we
33
+ # just won't run the same test, but it's nice to keep
34
+ # this as stable as possible if the code isn't changing.
35
+
36
+ # Minitest makes it nice and easy to create a stable
37
+ # test identifier, because it follows the classic xunit
38
+ # pattern where a test is just a method invocation on a
39
+ # fresh test class instance and it's easy to find out
40
+ # which invocation that was.
41
+ return "#{self.class.name}::#{@NAME}" if defined? @NAME
42
+
43
+ # If we are running in an rspec example then, sadly,
44
+ # rspec take the entirely unreasonable stance that
45
+ # the correct way to pass data to a test is by passing
46
+ # it as a function argument. Honestly, what is this,
47
+ # Haskell? Ahem. Perfectly reasonable design decisions
48
+ # on rspec's part, this creates some annoying difficulties
49
+ # for us. We solve this through brute force and ignorance
50
+ # by relying on the information we want being in the
51
+ # inspect for the Example object, even if it's just there
52
+ # as a string.
53
+ begin
54
+ is_rspec = is_a? RSpec::Core::ExampleGroup
55
+ # We do our coverage testing inside rspec, so this will
56
+ # never trigger! Though we also don't currently have a
57
+ # test that covers it outside of rspec...
58
+ # :nocov:
59
+ rescue NameError
60
+ is_rspec = false
61
+ end
62
+ # :nocov:
63
+
64
+ if is_rspec
65
+ return [
66
+ self.class.description,
67
+ inspect.match(/"([^"]+)"/)[1]
68
+ ].join(' ')
69
+ end
70
+
71
+ # Fallback time! We just walk the stack until we find the
72
+ # entry point into code we control. This will typically be
73
+ # where "hypothesis" was called.
74
+ Thread.current.backtrace.each do |line|
75
+ return line unless line.include?(Hypothesis::HYPOTHESIS_LOCATION)
76
+ end
77
+ # This should never happen unless something very strange is
78
+ # going on.
79
+ # :nocov:
80
+ raise 'BUG: Somehow we have no caller!'
81
+ # :nocov:
82
+ end
83
+
84
+ # Run a test using Hypothesis.
85
+ #
86
+ # For example:
87
+ #
88
+ # ```ruby
89
+ # hypothesis do
90
+ # x = any integer
91
+ # y = any integer(min: x)
92
+ # expect(y).to be >= x
93
+ # end
94
+ # ```
95
+ #
96
+ # The arguments to `any` are `Possible` instances which
97
+ # specify the range of value values for it to return.
98
+ #
99
+ # Typically you would include this inside some test in your
100
+ # normal testing framework - e.g. in an rspec it block or a
101
+ # minitest test method.
102
+ #
103
+ # This will run the block many times with integer values for
104
+ # x and y, and each time it will pass because we specified that
105
+ # y had a minimum value of x.
106
+ # If we changed it to `expect(y).to be > x` we would see output
107
+ # like the following:
108
+ #
109
+ # ```
110
+ # Failure/Error: expect(y).to be > x
111
+ #
112
+ # Given #1: 0
113
+ # Given #2: 0
114
+ # expected: > 0
115
+ # got: 0
116
+ # ```
117
+ #
118
+ # In more detail:
119
+ #
120
+ # hypothesis calls its provided block many times. Each invocation
121
+ # of the block is a *test case*.
122
+ # A test case has three important features:
123
+ #
124
+ # * *givens* are the result of a call to self.given, and are the
125
+ # values that make up the test case. These might be values such
126
+ # as strings, integers, etc. or they might be values specific to
127
+ # your application such as a User object.
128
+ # * *assumptions*, where you call `self.assume(some_condition)`. If
129
+ # an assumption fails (`some_condition` is false), then the test
130
+ # case is considered invalid, and is discarded.
131
+ # * *assertions* are anything that will raise an error if the test
132
+ # case should be considered a failure. These could be e.g. RSpec
133
+ # expectations or minitest matchers, but anything that throws an
134
+ # exception will be treated as a failed assertion.
135
+ #
136
+ # A test case which satisfies all of its assumptions and assertions
137
+ # is *valid*. A test-case which satisfies all of its assumptions but
138
+ # fails one of its assertions is *failing*.
139
+ #
140
+ # A call to hypothesis does the following:
141
+ #
142
+ # 1. It tries to *generate* a failing test case.
143
+ # 2. If it succeeded then it will *shrink* that failing test case.
144
+ # 3. Finally, it will *display* the shrunk failing test case by
145
+ # the error from its failing assertion, modified to show the
146
+ # givens of the test case.
147
+ #
148
+ # Generation consists of randomly trying test cases until one of
149
+ # three things has happened:
150
+ #
151
+ # 1. It has found a failing test case. At this point it will start
152
+ # *shrinking* the test case (see below).
153
+ # 2. It has found enough valid test cases. At this point it will
154
+ # silently stop.
155
+ # 3. It has found so many invalid test cases that it seems unlikely
156
+ # that it will find any more valid ones in a reasonable amount of
157
+ # time. At this point it will either silently stop or raise
158
+ # `Hypothesis::Unsatisfiable` depending on how many valid
159
+ # examples it found.
160
+ #
161
+ # *Shrinking* is when Hypothesis takes a failing test case and tries
162
+ # to make it easier to understand. It does this by replacing the givens
163
+ # in the test case with smaller and simpler values. These givens will
164
+ # still come from the possible values, and will obey all the usual
165
+ # constraints.
166
+ # In general, shrinking is automatic and you shouldn't need to care
167
+ # about the details of it. If the test case you're shown at the end
168
+ # is messy or needlessly large, please file a bug explaining the problem!
169
+ #
170
+ # @param max_valid_test_cases [Integer] The maximum number of valid test
171
+ # cases to run without finding a failing test case before stopping.
172
+ def hypothesis(max_valid_test_cases: 200, &block)
173
+ unless World.current_engine.nil?
174
+ raise UsageError, 'Cannot nest hypothesis calls'
175
+ end
176
+ begin
177
+ World.current_engine = Engine.new(
178
+ max_examples: max_valid_test_cases
179
+ )
180
+ World.current_engine.run(&block)
181
+ ensure
182
+ World.current_engine = nil
183
+ end
184
+ end
185
+
186
+ # Supplies a value to be used in your hypothesis.
187
+ # @note It is invalid to call this method outside of a hypothesis block.
188
+ # @return [Object] A value provided by the possible argument.
189
+ # @param possible [Possible] A possible that specifies the possible values
190
+ # to return.
191
+ # @param name [String, nil] An optional name to show next to the result on
192
+ # failure. This can be helpful if you have a lot of givens in your
193
+ # hypothesis, as it makes it easier to keep track of which is which.
194
+ def any(possible, name: nil, &block)
195
+ if World.current_engine.nil?
196
+ raise UsageError, 'Cannot call any outside of a hypothesis block'
197
+ end
198
+
199
+ World.current_engine.current_source.any(
200
+ possible, name: name, &block
201
+ )
202
+ end
203
+
204
+ # Specify an assumption of your test case. Only test cases which satisfy
205
+ # their assumptions will treated as valid, and all others will be
206
+ # discarded.
207
+ # @note It is invalid to call this method outside of a hypothesis block.
208
+ # @note Try to use this only with "easy" conditions. If the condition is
209
+ # too hard to satisfy this can make your testing much worse, because
210
+ # Hypothesis will have to retry the test many times and will struggle
211
+ # to find "interesting" test cases. For example `assume(x != y)` is
212
+ # typically fine, and `assume(x == y)` is rarely a good idea.
213
+ # @param condition [Boolean] The condition to assume. If this is false,
214
+ # the current test case will be treated as invalid and the block will
215
+ # exit by throwing an exception. The next test case will then be run
216
+ # as normal.
217
+ def assume(condition)
218
+ if World.current_engine.nil?
219
+ raise UsageError, 'Cannot call assume outside of a hypothesis block'
220
+ end
221
+ World.current_engine.current_source.assume(condition)
222
+ end
223
+ end