hypothesis-specs 0.0.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: ca2e609a02788991ddea6552d8dde5660b918308
4
+ data.tar.gz: f0c02fb905b411d91a55fece2d5e5117a130d984
5
+ SHA512:
6
+ metadata.gz: 7d3750c2b0a756f38ec82525157b8471d1dab93c0bcf139260da68b37a57d8d9fea3bb8faf9ef947c36534dbb51658563e3d4ff1714e9f9617ac98d6a516b5a2
7
+ data.tar.gz: 732da786e9e63999fd709338edfae86466b807288a787e8f8e0cfed8b667c911eef15424611b2f8c3990fc95f1cf6d97a6ea3a7b45f1c04ce603e128cc4e56ea
data/CHANGELOG.md ADDED
@@ -0,0 +1,10 @@
1
+ ## Hypothesis for Ruby 0.0.3 (2018-02-19)
2
+
3
+ This is an initial developer preview of Hypothesis for Ruby.
4
+ It's ready to use, but isn't yet stable and has significant
5
+ limitations. It is mostly released so that people can easily give
6
+ feedback on the API and implementation, and is likely to change
7
+ substantially before a stable release.
8
+
9
+ Note that while there were some earlier release numbers internally,
10
+ these were pulled. This is the first official release.
data/Cargo.toml ADDED
@@ -0,0 +1,11 @@
1
+ [package]
2
+ name = "hypothesis-ruby-core"
3
+ version = "0.1.0"
4
+ authors = ["David R. MacIver <david@drmaciver.com>"]
5
+
6
+ [lib]
7
+ crate-type = ["cdylib"]
8
+
9
+ [dependencies]
10
+ helix = '0.7.2'
11
+ rand = '0.3'
data/LICENSE.txt ADDED
@@ -0,0 +1,8 @@
1
+ Copyright (c) 2018, David R. MacIver
2
+
3
+ All code in this repository except where explicitly noted otherwise is released
4
+ under the Mozilla Public License v 2.0. You can obtain a copy at http://mozilla.org/MPL/2.0/.
5
+
6
+ Some code in this repository may come from other projects. Where applicable, the
7
+ original copyright and license are noted and any modifications made are released
8
+ dual licensed with the original license.
data/README.markdown ADDED
@@ -0,0 +1,86 @@
1
+ # Hypothesis for Ruby
2
+
3
+ Hypothesis is a powerful, flexible, and easy to use library for *property-based testing*.
4
+
5
+ In property-based testing,
6
+ in contrast to traditional *example-based testing*,
7
+ a test is written not against a single example but as a statement that should hold for any of a range of possible values.
8
+
9
+ ## Usage
10
+
11
+ In Hypothesis for Ruby, a test looks something like this:
12
+
13
+ ```ruby
14
+ require "hypothesis"
15
+
16
+ RSpec.configure do |config|
17
+ config.include(Hypothesis)
18
+ config.include(Hypothesis::Possibilities)
19
+ end
20
+
21
+ RSpec.describe "removing an element from a list" do
22
+ it "results in the element no longer being in the list" do
23
+ hypothesis do
24
+ # Or lists(of: integers, min_size: 1), but this lets us
25
+ # demonstrate assume.
26
+ values = any array(of: integers)
27
+
28
+ # If this is not true then the test will stop here.
29
+ assume values.size > 0
30
+
31
+ to_remove = any element_of(values)
32
+
33
+ values.delete_at(values.index(to_remove))
34
+
35
+ # Will fail if the value ws duplicated in the list.
36
+ expect(values.include?(to_remove)).to be false
37
+
38
+ end
39
+ end
40
+ end
41
+ ```
42
+
43
+ This would then fail with:
44
+
45
+ ```
46
+ 1) removing an element from a list results in the element no longer being in the list
47
+ Failure/Error: expect(values.include?(to_remove)).to be false
48
+
49
+ Given #1: [0, 0]
50
+ Given #2: 0
51
+
52
+ expected false
53
+ got true
54
+ ```
55
+
56
+ The use of RSpec here is incidental:
57
+ Hypothesis for Ruby works just as well with minitest,
58
+ and should work with anything else you care to use.
59
+
60
+ ## Getting Started
61
+
62
+ Hypothesis is not available on rubygems.org as a developer preview.
63
+ If you want to try it today you can use the current development branch by adding the following to your Gemfile:
64
+
65
+ ```ruby
66
+ gem 'hypothesis-specs'
67
+ ```
68
+
69
+ The API is still in flux, so be warned that you should expect it to break on upgrades!
70
+ Right now this is really more to allow you to try it out and provide feedback than something you should expect to rely on.
71
+ The more feedback we get, the sooner it will get here!
72
+
73
+ Note that in order to use Hypothesis for Ruby, you will need a rust toolchain installed.
74
+ Please go to [https://www.rustup.rs](https://www.rustup.rs) and follow the instructions if you do not already have one.
75
+
76
+ ## Project Status
77
+
78
+ Hypothesis for Ruby is currently in an *early alpha* stage.
79
+ It works, and has a solid core set of features, but you should expect to find rough edges,
80
+ it is far from feature complete, and the API makes no promises of backwards compatibility.
81
+
82
+ Right now you should consider it to be more in the spirit of a developer preview.
83
+ You can and should try it out, and hopefully you will find all sorts of interesting bugs in your code by doing so!
84
+ But you'll probably find interesting bugs in Hypothesis too,
85
+ and we'd appreciate you reporting them,
86
+ as well as any just general usability issues or points of confusion you have.
data/Rakefile ADDED
@@ -0,0 +1,145 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'rubygems'
4
+ require 'helix_runtime/build_task'
5
+
6
+ begin
7
+ require 'rspec/core/rake_task'
8
+ RSpec::Core::RakeTask.new(:spec)
9
+
10
+ require 'rake/testtask'
11
+
12
+ Rake::TestTask.new(minitests: :build) do |t|
13
+ t.test_files = FileList['minitests/**/test_*.rb']
14
+ t.verbose = true
15
+ end
16
+
17
+ task test: %i[build spec minitests]
18
+ rescue LoadError
19
+ end
20
+
21
+ # Monkeypatch build to fail on error.
22
+ # See https://github.com/tildeio/helix/issues/133
23
+ module HelixRuntime
24
+ class Project
25
+ alias original_build cargo_build
26
+
27
+ def cargo_build
28
+ raise 'Build failed' unless original_build
29
+ end
30
+ end
31
+ end
32
+
33
+ HelixRuntime::BuildTask.new
34
+
35
+ task :format do
36
+ sh 'bundle exec rubocop -a lib spec minitests ' \
37
+ 'Rakefile hypothesis-specs.gemspec'
38
+ end
39
+
40
+ begin
41
+ require 'yard'
42
+
43
+ YARD::Rake::YardocTask.new(:runyard) do |t|
44
+ t.files = [
45
+ 'lib/hypothesis.rb', 'lib/hypothesis/errors.rb',
46
+ 'lib/hypothesis/possible.rb'
47
+ ]
48
+ t.options = ['--markup=markdown', '--no-private']
49
+ end
50
+
51
+ task doc: :runyard do
52
+ YARD::Registry.load
53
+
54
+ objs = YARD::Registry.select do |o|
55
+ is_private = false
56
+ t = o
57
+ until t.root?
58
+ if t.visibility != :public
59
+ is_private = true
60
+ break
61
+ end
62
+ t = t.parent
63
+ end
64
+
65
+ !is_private && o.docstring.blank?
66
+ end
67
+
68
+ objs.sort_by! { |o| o.name.to_s }
69
+
70
+ unless objs.empty?
71
+ abort "Undocumented objects: #{objs.map(&:name).join(', ')}"
72
+ end
73
+ end
74
+ rescue LoadError
75
+ end
76
+
77
+ task :gem do
78
+ uncommitted = `git ls-files lib/ --others --exclude-standard`.split
79
+ uncommitted_ruby = uncommitted.grep(/\.rb$/)
80
+ uncommitted_ruby.sort!
81
+ unless uncommitted_ruby.empty?
82
+ abort 'Cannot build gem with uncomitted Ruby '\
83
+ "files #{uncommitted_ruby.join(', ')}"
84
+ end
85
+
86
+ sh 'rm -rf hypothesis-specs*.gem'
87
+ sh 'git clean -fdx lib'
88
+ sh 'gem build hypothesis-specs.gemspec'
89
+ end
90
+
91
+ def git(*args)
92
+ sh 'git', *args
93
+ end
94
+
95
+ file 'secrets.tar.enc' => 'secrets' do
96
+ sh 'rm -f secrets.tar secrets.tar.enc'
97
+ sh 'tar -cf secrets.tar secrets'
98
+ sh 'travis encrypt-file secrets.tar'
99
+ end
100
+
101
+ task deploy: :gem do
102
+ on_master = system("git merge-base --is-ancestor HEAD origin/master")
103
+
104
+ unless on_master
105
+ puts 'Not on master, so no deploy'
106
+ next
107
+ end
108
+
109
+ spec = Gem::Specification.load('hypothesis-specs.gemspec')
110
+
111
+ succeeded = system('git', 'tag', spec.version.to_s)
112
+
113
+ unless succeeded
114
+ puts "Looks like we've already done this release."
115
+ next
116
+ end
117
+
118
+ unless File.directory? 'secrets'
119
+ sh 'rm -rf secrets'
120
+ sh 'openssl aes-256-cbc -K $encrypted_b0055249143b_key -iv ' \
121
+ '$encrypted_b0055249143b_iv -in secrets.tar.enc -out secrets.tar -d'
122
+
123
+ sh 'tar -xvf secrets.tar'
124
+ end
125
+
126
+ git('config', 'user.name', 'Travis CI on behalf of David R. MacIver')
127
+ git('config', 'user.email', 'david@drmaciver.com')
128
+ git('config', 'core.sshCommand', 'ssh -i secrets/deploy_key')
129
+ git(
130
+ 'remote', 'add', 'ssh-origin',
131
+ 'git@github.com:HypothesisWorks/hypothesis-ruby.git'
132
+ )
133
+
134
+ sh(
135
+ 'ssh-agent', 'sh', '-c',
136
+ 'chmod 0600 secrets/deploy_key && ssh-add secrets/deploy_key && ' \
137
+ 'git push ssh-origin --tags'
138
+ )
139
+
140
+ sh 'rm -f ~/.gem/credentials'
141
+ sh 'mkdir -p ~/.gem'
142
+ sh 'ln -s $(pwd)/secrets/api_key.yaml ~/.gem/credentials'
143
+ sh 'chmod 0600 ~/.gem/credentials'
144
+ sh 'gem push hypothesis-specs*.gem'
145
+ end
data/ext/Makefile ADDED
@@ -0,0 +1,7 @@
1
+ all:
2
+ cd ..
3
+ rake build
4
+ clean:
5
+ rm -rf ../target
6
+
7
+ install: ;
data/ext/extconf.rb ADDED
@@ -0,0 +1,5 @@
1
+ if !system('cargo --version')
2
+ raise 'Hypothesis requires cargo to be installed (https://www.rust-lang.org/)'
3
+ end
4
+
5
+ require 'rake'
data/lib/hypothesis.rb ADDED
@@ -0,0 +1,223 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'hypothesis/errors'
4
+ require 'hypothesis/possible'
5
+ require 'hypothesis/testcase'
6
+ require 'hypothesis/engine'
7
+ require 'hypothesis/world'
8
+
9
+ # This is the main module for using Hypothesis.
10
+ # It is expected that you will include this in your
11
+ # tests, but its methods are also available on the
12
+ # module itself.
13
+ #
14
+ # The main entry point for using this is the
15
+ # {Hypothesis#hypothesis} method. All of the other
16
+ # methods make sense only inside blocks passed to
17
+ # it.
18
+ module Hypothesis
19
+ # @!visibility private
20
+ HYPOTHESIS_LOCATION = File.dirname(__FILE__)
21
+
22
+ # @!visibility private
23
+ def hypothesis_stable_identifier
24
+ # Attempt to get a "stable identifier" for any any
25
+ # call into hypothesis. We use these to create
26
+ # database keys (or will when we have a database) that
27
+ # are stable across runs, so that when a test that
28
+ # previously failed is rerun, we can fetch and reuse
29
+ # the previous examples.
30
+
31
+ # Note that essentially any answer to this method is
32
+ # "fine" in that the failure mode is that sometiems we
33
+ # just won't run the same test, but it's nice to keep
34
+ # this as stable as possible if the code isn't changing.
35
+
36
+ # Minitest makes it nice and easy to create a stable
37
+ # test identifier, because it follows the classic xunit
38
+ # pattern where a test is just a method invocation on a
39
+ # fresh test class instance and it's easy to find out
40
+ # which invocation that was.
41
+ return "#{self.class.name}::#{@NAME}" if defined? @NAME
42
+
43
+ # If we are running in an rspec example then, sadly,
44
+ # rspec take the entirely unreasonable stance that
45
+ # the correct way to pass data to a test is by passing
46
+ # it as a function argument. Honestly, what is this,
47
+ # Haskell? Ahem. Perfectly reasonable design decisions
48
+ # on rspec's part, this creates some annoying difficulties
49
+ # for us. We solve this through brute force and ignorance
50
+ # by relying on the information we want being in the
51
+ # inspect for the Example object, even if it's just there
52
+ # as a string.
53
+ begin
54
+ is_rspec = is_a? RSpec::Core::ExampleGroup
55
+ # We do our coverage testing inside rspec, so this will
56
+ # never trigger! Though we also don't currently have a
57
+ # test that covers it outside of rspec...
58
+ # :nocov:
59
+ rescue NameError
60
+ is_rspec = false
61
+ end
62
+ # :nocov:
63
+
64
+ if is_rspec
65
+ return [
66
+ self.class.description,
67
+ inspect.match(/"([^"]+)"/)[1]
68
+ ].join(' ')
69
+ end
70
+
71
+ # Fallback time! We just walk the stack until we find the
72
+ # entry point into code we control. This will typically be
73
+ # where "hypothesis" was called.
74
+ Thread.current.backtrace.each do |line|
75
+ return line unless line.include?(Hypothesis::HYPOTHESIS_LOCATION)
76
+ end
77
+ # This should never happen unless something very strange is
78
+ # going on.
79
+ # :nocov:
80
+ raise 'BUG: Somehow we have no caller!'
81
+ # :nocov:
82
+ end
83
+
84
+ # Run a test using Hypothesis.
85
+ #
86
+ # For example:
87
+ #
88
+ # ```ruby
89
+ # hypothesis do
90
+ # x = any integer
91
+ # y = any integer(min: x)
92
+ # expect(y).to be >= x
93
+ # end
94
+ # ```
95
+ #
96
+ # The arguments to `any` are `Possible` instances which
97
+ # specify the range of value values for it to return.
98
+ #
99
+ # Typically you would include this inside some test in your
100
+ # normal testing framework - e.g. in an rspec it block or a
101
+ # minitest test method.
102
+ #
103
+ # This will run the block many times with integer values for
104
+ # x and y, and each time it will pass because we specified that
105
+ # y had a minimum value of x.
106
+ # If we changed it to `expect(y).to be > x` we would see output
107
+ # like the following:
108
+ #
109
+ # ```
110
+ # Failure/Error: expect(y).to be > x
111
+ #
112
+ # Given #1: 0
113
+ # Given #2: 0
114
+ # expected: > 0
115
+ # got: 0
116
+ # ```
117
+ #
118
+ # In more detail:
119
+ #
120
+ # hypothesis calls its provided block many times. Each invocation
121
+ # of the block is a *test case*.
122
+ # A test case has three important features:
123
+ #
124
+ # * *givens* are the result of a call to self.given, and are the
125
+ # values that make up the test case. These might be values such
126
+ # as strings, integers, etc. or they might be values specific to
127
+ # your application such as a User object.
128
+ # * *assumptions*, where you call `self.assume(some_condition)`. If
129
+ # an assumption fails (`some_condition` is false), then the test
130
+ # case is considered invalid, and is discarded.
131
+ # * *assertions* are anything that will raise an error if the test
132
+ # case should be considered a failure. These could be e.g. RSpec
133
+ # expectations or minitest matchers, but anything that throws an
134
+ # exception will be treated as a failed assertion.
135
+ #
136
+ # A test case which satisfies all of its assumptions and assertions
137
+ # is *valid*. A test-case which satisfies all of its assumptions but
138
+ # fails one of its assertions is *failing*.
139
+ #
140
+ # A call to hypothesis does the following:
141
+ #
142
+ # 1. It tries to *generate* a failing test case.
143
+ # 2. If it succeeded then it will *shrink* that failing test case.
144
+ # 3. Finally, it will *display* the shrunk failing test case by
145
+ # the error from its failing assertion, modified to show the
146
+ # givens of the test case.
147
+ #
148
+ # Generation consists of randomly trying test cases until one of
149
+ # three things has happened:
150
+ #
151
+ # 1. It has found a failing test case. At this point it will start
152
+ # *shrinking* the test case (see below).
153
+ # 2. It has found enough valid test cases. At this point it will
154
+ # silently stop.
155
+ # 3. It has found so many invalid test cases that it seems unlikely
156
+ # that it will find any more valid ones in a reasonable amount of
157
+ # time. At this point it will either silently stop or raise
158
+ # `Hypothesis::Unsatisfiable` depending on how many valid
159
+ # examples it found.
160
+ #
161
+ # *Shrinking* is when Hypothesis takes a failing test case and tries
162
+ # to make it easier to understand. It does this by replacing the givens
163
+ # in the test case with smaller and simpler values. These givens will
164
+ # still come from the possible values, and will obey all the usual
165
+ # constraints.
166
+ # In general, shrinking is automatic and you shouldn't need to care
167
+ # about the details of it. If the test case you're shown at the end
168
+ # is messy or needlessly large, please file a bug explaining the problem!
169
+ #
170
+ # @param max_valid_test_cases [Integer] The maximum number of valid test
171
+ # cases to run without finding a failing test case before stopping.
172
+ def hypothesis(max_valid_test_cases: 200, &block)
173
+ unless World.current_engine.nil?
174
+ raise UsageError, 'Cannot nest hypothesis calls'
175
+ end
176
+ begin
177
+ World.current_engine = Engine.new(
178
+ max_examples: max_valid_test_cases
179
+ )
180
+ World.current_engine.run(&block)
181
+ ensure
182
+ World.current_engine = nil
183
+ end
184
+ end
185
+
186
+ # Supplies a value to be used in your hypothesis.
187
+ # @note It is invalid to call this method outside of a hypothesis block.
188
+ # @return [Object] A value provided by the possible argument.
189
+ # @param possible [Possible] A possible that specifies the possible values
190
+ # to return.
191
+ # @param name [String, nil] An optional name to show next to the result on
192
+ # failure. This can be helpful if you have a lot of givens in your
193
+ # hypothesis, as it makes it easier to keep track of which is which.
194
+ def any(possible, name: nil, &block)
195
+ if World.current_engine.nil?
196
+ raise UsageError, 'Cannot call any outside of a hypothesis block'
197
+ end
198
+
199
+ World.current_engine.current_source.any(
200
+ possible, name: name, &block
201
+ )
202
+ end
203
+
204
+ # Specify an assumption of your test case. Only test cases which satisfy
205
+ # their assumptions will treated as valid, and all others will be
206
+ # discarded.
207
+ # @note It is invalid to call this method outside of a hypothesis block.
208
+ # @note Try to use this only with "easy" conditions. If the condition is
209
+ # too hard to satisfy this can make your testing much worse, because
210
+ # Hypothesis will have to retry the test many times and will struggle
211
+ # to find "interesting" test cases. For example `assume(x != y)` is
212
+ # typically fine, and `assume(x == y)` is rarely a good idea.
213
+ # @param condition [Boolean] The condition to assume. If this is false,
214
+ # the current test case will be treated as invalid and the block will
215
+ # exit by throwing an exception. The next test case will then be run
216
+ # as normal.
217
+ def assume(condition)
218
+ if World.current_engine.nil?
219
+ raise UsageError, 'Cannot call assume outside of a hypothesis block'
220
+ end
221
+ World.current_engine.current_source.assume(condition)
222
+ end
223
+ end