fuzzbert 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +104 -51
- data/lib/fuzzbert/autorun.rb +62 -29
- data/lib/fuzzbert/dsl.rb +2 -0
- data/lib/fuzzbert/error_handler.rb +53 -5
- data/lib/fuzzbert/executor.rb +108 -94
- data/lib/fuzzbert/generators.rb +35 -37
- data/lib/fuzzbert/rake_task.rb +13 -12
- data/lib/fuzzbert/template.rb +95 -93
- data/lib/fuzzbert/test_suite.rb +4 -1
- data/spec/autorun_spec.rb +30 -16
- data/spec/dsl_spec.rb +43 -0
- data/spec/executor_spec.rb +26 -8
- data/spec/generator_spec.rb +11 -3
- data/spec/mutator_spec.rb +6 -4
- data/spec/template_spec.rb +20 -3
- metadata +20 -21
- data/lib/fuzzbert/autorun.rb~ +0 -33
data/README.md
CHANGED
@@ -22,21 +22,7 @@ For further information on random testing, here are two excellent starting point
|
|
22
22
|
## Defining a random test
|
23
23
|
|
24
24
|
FuzzBert defines an RSpec-like DSL that can be used to define different fuzzing
|
25
|
-
scenarios. The DSL uses three words: `fuzz`, `deploy` and `data`.
|
26
|
-
thought of as defining a new scenario, such as "fuzz this command line tool",
|
27
|
-
"fuzz this particular URL of my web app" or "fuzz this library method taking
|
28
|
-
external input".
|
29
|
-
|
30
|
-
Within a `fuzz` block, there must be one occurence of `deploy` and one or several
|
31
|
-
occurences of `data`. The `deploy` block is the spot where we deliver the random
|
32
|
-
payload that has been generated. It is agnostic about the actual target in order to
|
33
|
-
leave you free to fuzz whatever you require in your particular case. The `data`
|
34
|
-
blocks define the shape of the random data being generated. There can be more than
|
35
|
-
one such block because it is often beneficial to not only shoot completely random
|
36
|
-
data at the target - you often want to deliver more structured data as well, trying
|
37
|
-
to find the edge cases deeper within your code. Good random test suites make use
|
38
|
-
of both - totally random data as well as structured data - in order to cover as
|
39
|
-
much "code surface" as possible.
|
25
|
+
scenarios. The DSL uses three words: `fuzz`, `deploy` and `data`.
|
40
26
|
|
41
27
|
Here is a quick example that fuzzes `JSON.parse`:
|
42
28
|
|
@@ -78,6 +64,21 @@ fuzz "JSON.parse" do
|
|
78
64
|
end
|
79
65
|
```
|
80
66
|
|
67
|
+
`fuzz` can be thought of as defining a new scenario, such as "fuzz this command
|
68
|
+
line tool", "fuzz this particular URL of my web app" or "fuzz this library method
|
69
|
+
taking external input".
|
70
|
+
|
71
|
+
Within a `fuzz` block, there must be one occurence of `deploy` and one or several
|
72
|
+
occurences of `data`. The `deploy` block is the spot where we deliver the random
|
73
|
+
payload that has been generated. It is agnostic about the actual target in order to
|
74
|
+
leave you free to fuzz whatever you require in your particular case. The `data`
|
75
|
+
blocks define the shape of the random data being generated. There can be more than
|
76
|
+
one such block because it is often beneficial to not only shoot completely random
|
77
|
+
data at the target - you often want to deliver more structured data as well, trying
|
78
|
+
to find the edge cases deeper within your code. Good random test suites make use
|
79
|
+
of both - totally random data as well as structured data - in order to cover as
|
80
|
+
much "code surface" as possible.
|
81
|
+
|
81
82
|
The `deploy` block takes the generated data as a parameter. The block itself is
|
82
83
|
responsible of deploying the payload. An execution is considered successful if
|
83
84
|
the `deploy` block passes with no uncaught error being raised. If an error slips
|
@@ -88,14 +89,91 @@ considered as a failure.
|
|
88
89
|
choose completely custom lambdas of your own or use those predefined for you in
|
89
90
|
`FuzzBert::Generators`.
|
90
91
|
|
92
|
+
## Running a random test
|
93
|
+
|
94
|
+
Once the FuzzBert files are set up, you may run your tests similar to how you
|
95
|
+
would run unit tests:
|
96
|
+
|
97
|
+
fuzzbert "fuzz/**/fuzz_*.rb"
|
98
|
+
|
99
|
+
If your FuzzBert files are already in a directory named 'fuzz' and each of them
|
100
|
+
begins with 'fuzz_', you may omit the pattern altogether.
|
101
|
+
|
102
|
+
Each `fuzz` block defines a `TestSuite`. These are executed in a round-robin manner.
|
103
|
+
Each individual `TestSuite` will then apply the `deploy` block with a sample of
|
104
|
+
data generated successively by each one of the `data` blocks. Once all `data` blocks
|
105
|
+
are used up, the next `TestSuite` will be executed etc. By default, a FuzzBert
|
106
|
+
fuzzing session runs forever, until the process is either killed or by manually hitting
|
107
|
+
`CTRL+C` for example. This was a deliberate design choice since random testing suites
|
108
|
+
need to be run for quite some time to be effective. It's something you want to run over
|
109
|
+
the weekend rather than for a couple of minutes. Still, it can make sense to explicitly
|
110
|
+
limit the number of runs, for example when integrating FuzzBert with a CI server or
|
111
|
+
with Travis. You can do so by passing the `--limit` parameter to the `fuzzbert`
|
112
|
+
executable:
|
113
|
+
|
114
|
+
fuzzbert --limit 1000000 "fuzz/**/fuzz_*.rb"
|
115
|
+
|
116
|
+
Every single execution of `deploy` is run in a separate process. The main reason for
|
117
|
+
this is that we typically want to detect hard crashes when a C extension or even Ruby
|
118
|
+
itself encounters an input it can't handle. Besides being able to cope with these cases,
|
119
|
+
running in separate processes proves beneficial otherwise as well: by default, FuzzBert
|
120
|
+
runs the tests in four separate processes at once, therefore utilizing your CPU's cores
|
121
|
+
effectively. You can tweak that setting with `--pool-size` to set this number to 1
|
122
|
+
(for completely sequential runs) or to the exact number of cores your CPU offers.
|
123
|
+
|
124
|
+
fuzzbert --pool-size 1 my/fuzzbert/file
|
125
|
+
|
126
|
+
## What happens if we encounter a bug?
|
127
|
+
|
128
|
+
If a test should end up failing (either the process crashed completely or caused an
|
129
|
+
uncaught error), FuzzBert will output the failing test on your terminal and tell you
|
130
|
+
where it stored the data that caused this. This conveniently allows you to run FuzzBert
|
131
|
+
over the weekend and when you return on Monday, the troubleshooters will sit there all
|
132
|
+
lined up for you to go through and filter. By using the `--console` command line switch
|
133
|
+
you can tell FuzzBert to not explicitly store the data, but echoing the data that
|
134
|
+
caused the crash to the terminal instead.
|
135
|
+
|
136
|
+
fuzzbert --console "fuzz/**/fuzz_*.rb"
|
137
|
+
|
138
|
+
If you don't want to litter your current working directory with the files generated
|
139
|
+
by FuzzBert, you can also specify a specific path to where they should be saved
|
140
|
+
instead:
|
141
|
+
|
142
|
+
fuzzbert --bug-dir bugs "fuzz/**/fuzz_*.rb"
|
143
|
+
|
144
|
+
This is still not quite what you want to happen in case a test crashes? There's
|
145
|
+
also the possibility to define a handler of your own:
|
146
|
+
|
147
|
+
```ruby
|
148
|
+
require 'fuzzbert'
|
149
|
+
|
150
|
+
class MyHandler
|
151
|
+
def handle(error_data)
|
152
|
+
#create an issue in the bug tracker
|
153
|
+
puts error_data[:id]
|
154
|
+
p error_data[:data]
|
155
|
+
puts error_data[:pid]
|
156
|
+
puts error_data[:status]
|
157
|
+
end
|
158
|
+
end
|
159
|
+
|
160
|
+
fuzz "Define here as usual" do
|
161
|
+
...
|
162
|
+
end
|
163
|
+
```
|
164
|
+
|
165
|
+
Then tell fuzzbert by telling it about the custom handler:
|
166
|
+
|
167
|
+
fuzzbert --handler MyHandler my/fuzzbert/file
|
168
|
+
|
91
169
|
## Templates
|
92
170
|
|
93
171
|
Using the approach described so far is most useful for binary protocols, but as
|
94
|
-
soon as
|
95
|
-
actually want in these situations is some sort of template mechanism
|
96
|
-
with mostly fixed data and only replaces a few selected parts with
|
97
|
-
generated data. This, too, is possible with FuzzBert, it comes with a
|
98
|
-
templating language:
|
172
|
+
soon as you work with mainly String-based data this can quickly become a chore.
|
173
|
+
What you actually want in these situations is some sort of template mechanism
|
174
|
+
that comes with mostly fixed data and only replaces a few selected parts with
|
175
|
+
randomly generated data. This, too, is possible with FuzzBert, it comes with a
|
176
|
+
minimal templating language:
|
99
177
|
|
100
178
|
```ruby
|
101
179
|
require 'fuzzbert'
|
@@ -119,6 +197,10 @@ fuzz "My Web App" do
|
|
119
197
|
end
|
120
198
|
```
|
121
199
|
|
200
|
+
Simply specify your template variables using `${..}` and assign a callback for
|
201
|
+
them via `set`. Of course you may escape the dollar sign with a back slash as
|
202
|
+
usual.
|
203
|
+
|
122
204
|
## Mutators
|
123
205
|
|
124
206
|
Mutation is the principle used in "Babysitting an Army of Monkeys". The basis for
|
@@ -141,37 +223,8 @@ fuzz "Web App" do
|
|
141
223
|
end
|
142
224
|
```
|
143
225
|
|
144
|
-
|
145
|
-
|
146
|
-
Each `fuzz` block defines a `TestSuite`. These are executed in a round-robin manner.
|
147
|
-
Each individual `TestSuite` will then apply the `deploy` block with a sample of
|
148
|
-
data generated successively by each one of the `data` blocks. Once all `data` blocks
|
149
|
-
were used, the next `TestSuite` will be executed and so on... By default, a FuzzBert
|
150
|
-
fuzzing session runs forever, until the process is either killed or by manually hitting
|
151
|
-
`CTRL+C` for example. This was a deliberate design choice since random testing suites
|
152
|
-
need to be run for quite some time to be effective. It's something you want to run over
|
153
|
-
the weekend rather than for a couple of minutes. Still, it can make sense to explicitly
|
154
|
-
limit the number of runs, for example when integrating FuzzBert with a CI server or
|
155
|
-
with Travis. You can do so by passing the `--limit` parameter to the `fuzzbert`
|
156
|
-
executable.
|
157
|
-
|
158
|
-
Every single execution of `deploy` is run in a separate process. The main reason for
|
159
|
-
this is that we typically want to detect hard crashes when a C extension or even Ruby
|
160
|
-
itself encounters an input it can't handle. Besides being able to cope with these cases,
|
161
|
-
running in separate processes proves beneficial otherwise as well: by default, FuzzBert
|
162
|
-
runs the tests in four separate processes at once, therefore utilizing your CPU's cores
|
163
|
-
effectively. You can tweak that setting with `--pool-size` to set this number to 1
|
164
|
-
(for completely sequential runs) or to the exact number of cores your CPU offers.
|
165
|
-
|
166
|
-
## What happens if we encounter a bug?
|
167
|
-
|
168
|
-
If a test should end up failing (either the process crashed completely or caused an
|
169
|
-
uncaught error), FuzzBert will output the failing test on your terminal and tell you
|
170
|
-
where it stored the data that caused this. This conveniently allows you to run FuzzBert
|
171
|
-
over the weekend and when you return on Monday, the troubleshooters will sit there all
|
172
|
-
lined up for you to go through and filter. By using the `--console` command line switch
|
173
|
-
you can tell FuzzBert to not explicitly store the data, but echoing the data that
|
174
|
-
caused the crash to the terminal instead.
|
226
|
+
This will take the original JSON data and modify one byte each time data is being
|
227
|
+
generated.
|
175
228
|
|
176
229
|
## Rake integration
|
177
230
|
|
data/lib/fuzzbert/autorun.rb
CHANGED
@@ -17,49 +17,82 @@ module FuzzBert::AutoRun
|
|
17
17
|
end
|
18
18
|
|
19
19
|
def run(options=nil)
|
20
|
-
|
21
|
-
|
20
|
+
raise RuntimeError.new "No test cases were found" if TEST_CASES.empty?
|
21
|
+
FuzzBert::Executor.new(TEST_CASES, options).run
|
22
22
|
end
|
23
23
|
|
24
24
|
private; module_function
|
25
25
|
|
26
|
-
|
27
|
-
|
28
|
-
|
26
|
+
def load_files(files)
|
27
|
+
files.each do |pattern|
|
28
|
+
Dir.glob(pattern).each { |f| load File.expand_path(f) }
|
29
|
+
end
|
29
30
|
end
|
30
|
-
end
|
31
31
|
|
32
|
-
|
33
|
-
|
34
|
-
|
32
|
+
def process_args(args = [])
|
33
|
+
options = {}
|
34
|
+
orig_args = args.dup
|
35
35
|
|
36
|
-
|
37
|
-
|
38
|
-
|
36
|
+
OptionParser.new do |opts|
|
37
|
+
opts.banner = 'Usage: fuzzbert [OPTIONS] PATTERN [PATTERNS]'
|
38
|
+
opts.separator <<-EOS
|
39
39
|
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
40
|
+
Run your random tests by pointing fuzzbert to a single or many explicit files
|
41
|
+
or by providing a pattern. The default pattern is 'fuzz/**/fuzz_*.rb, assuming
|
42
|
+
that your FuzzBert files (files beginning with 'fuzz_') live in a directory
|
43
|
+
named fuzz located under the current directory that you are in.
|
44
44
|
|
45
|
-
|
46
|
-
|
47
|
-
end
|
45
|
+
By default, fuzzbert will run the tests you specify forever, be sure to hit
|
46
|
+
CTRL+C when you are done or specify a limit with '--limit'.
|
48
47
|
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
-
|
48
|
+
EOS
|
49
|
+
|
50
|
+
opts.version = FuzzBert::VERSION
|
51
|
+
|
52
|
+
opts.on '-h', '--help', 'Run ' do
|
53
|
+
puts opts
|
54
|
+
exit
|
55
|
+
end
|
56
|
+
|
57
|
+
opts.on '--pool-size SIZE', Integer, "Sets the number of concurrently running processes to SIZE. Default is 4." do |n|
|
58
|
+
options[:pool_size] = n.to_i
|
59
|
+
end
|
53
60
|
|
54
|
-
|
55
|
-
|
61
|
+
opts.on '--limit LIMIT', Integer, "Instead of running permanently, fuzzing will be stopped after running LIMIT of instances." do |n|
|
62
|
+
options[:limit] = n.to_i
|
63
|
+
end
|
64
|
+
|
65
|
+
opts.on '--console', "Output the failing cases including data on the console instead of saving them in a file." do
|
66
|
+
options[:handler] = FuzzBert::Handler::Console.new
|
67
|
+
end
|
68
|
+
|
69
|
+
opts.on '--sleep-delay SECONDS', Float, "Specify the number of SECONDS that the main process sleeps before checking that the limit has been reached. Default is 1." do |f|
|
70
|
+
options[:sleep_delay] = f.to_f
|
71
|
+
end
|
72
|
+
|
73
|
+
opts.on '--handler CLASS', String, "Specify the full path to a CLASS that will serve as your Handler." do |path|
|
74
|
+
#lazy initialization: the Handler must be defined in one of the fuzzer files
|
75
|
+
options[:handler] = Class.new do
|
76
|
+
@@path = path
|
77
|
+
|
78
|
+
def handle(id, data, pid, status)
|
79
|
+
@inner ||= Object.const_get(@@path).new
|
80
|
+
@inner.handle(id, data, pid, status)
|
81
|
+
end
|
82
|
+
end.new
|
83
|
+
end
|
84
|
+
|
85
|
+
opts.on '--bug-dir DIRECTORY', String, "The DIRECTORY where the resulting bug files will be stored. Default is the current directory." do |dir|
|
86
|
+
raise ArgumentError.new "#{dir} is not a directory" unless Dir.exists?(dir)
|
87
|
+
options[:handler] = FuzzBert::Handler::FileOutput.new(dir)
|
88
|
+
end
|
89
|
+
|
90
|
+
opts.parse! args
|
56
91
|
end
|
57
92
|
|
58
|
-
|
93
|
+
raise ArgumentError.new("No file pattern was given") if args.empty?
|
94
|
+
[options, args]
|
59
95
|
end
|
60
96
|
|
61
|
-
[options, args]
|
62
|
-
end
|
63
|
-
|
64
97
|
end
|
65
98
|
|
data/lib/fuzzbert/dsl.rb
CHANGED
@@ -1,6 +1,8 @@
|
|
1
1
|
module FuzzBert::DSL
|
2
2
|
def fuzz(*args, &blk)
|
3
3
|
suite = FuzzBert::TestSuite.create(*args, &blk)
|
4
|
+
raise RuntimeError.new "No 'deploy' block was given" unless suite.test
|
5
|
+
raise RuntimeError.new "No 'data' blocks were given" unless suite.generators
|
4
6
|
FuzzBert::AutoRun.register(suite)
|
5
7
|
end
|
6
8
|
end
|
@@ -1,19 +1,67 @@
|
|
1
1
|
|
2
2
|
module FuzzBert::Handler
|
3
|
+
|
4
|
+
module ConsoleHelper
|
5
|
+
def info(error_data)
|
6
|
+
id = error_data[:id]
|
7
|
+
status = error_data[:status]
|
8
|
+
|
9
|
+
crashed = status.termsig
|
10
|
+
|
11
|
+
if crashed
|
12
|
+
puts "The data caused a hard crash."
|
13
|
+
else
|
14
|
+
puts "The data caused an uncaught error."
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
3
19
|
class FileOutput
|
4
|
-
|
5
|
-
|
6
|
-
|
20
|
+
include FuzzBert::Handler::ConsoleHelper
|
21
|
+
|
22
|
+
def initialize(dir=nil)
|
23
|
+
@dir = dir
|
24
|
+
if @dir && !@dir.end_with?("/")
|
25
|
+
@dir << "/"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
def handle(error_data)
|
30
|
+
id = error_data[:id]
|
31
|
+
data = error_data[:data]
|
32
|
+
status = error_data[:status]
|
33
|
+
pid = error_data[:pid]
|
34
|
+
|
35
|
+
crashed = status.termsig
|
36
|
+
prefix = crashed ? "crash" : "bug"
|
37
|
+
|
38
|
+
filename = "#{dir_prefix}#{prefix}#{pid}"
|
39
|
+
while File.exists?(filename)
|
40
|
+
filename << ('a'..'z').to_a.sample
|
41
|
+
end
|
7
42
|
File.open(filename, "wb") { |f| f.print(data) }
|
43
|
+
|
8
44
|
puts "#{id} failed. Data was saved as #{filename}."
|
45
|
+
info(error_data)
|
9
46
|
end
|
47
|
+
|
48
|
+
private
|
49
|
+
|
50
|
+
def dir_prefix
|
51
|
+
return "./" unless @dir
|
52
|
+
@dir
|
53
|
+
end
|
10
54
|
end
|
11
55
|
|
12
56
|
class Console
|
13
|
-
|
14
|
-
|
57
|
+
include FuzzBert::Handler::ConsoleHelper
|
58
|
+
|
59
|
+
def handle(error_data)
|
60
|
+
puts "#{error_data[:id]} failed. Data: #{error_data[:data].inspect}"
|
61
|
+
info(error_data)
|
15
62
|
end
|
16
63
|
end
|
64
|
+
|
17
65
|
end
|
18
66
|
|
19
67
|
|
data/lib/fuzzbert/executor.rb
CHANGED
@@ -1,19 +1,27 @@
|
|
1
1
|
class FuzzBert::Executor
|
2
2
|
|
3
|
-
attr_reader :pool_size, :limit, :handler
|
3
|
+
attr_reader :pool_size, :limit, :handler, :sleep_delay
|
4
4
|
|
5
5
|
DEFAULT_POOL_SIZE = 4
|
6
6
|
DEFAULT_LIMIT = -1
|
7
7
|
DEFAULT_HANDLER = FuzzBert::Handler::FileOutput
|
8
|
+
DEFAULT_SLEEP_DELAY = 1
|
8
9
|
|
9
|
-
|
10
|
+
DEFAULT_ARGS = {
|
10
11
|
pool_size: DEFAULT_POOL_SIZE,
|
11
12
|
limit: DEFAULT_LIMIT,
|
12
|
-
handler: DEFAULT_HANDLER.new
|
13
|
-
|
13
|
+
handler: DEFAULT_HANDLER.new,
|
14
|
+
sleep_delay: DEFAULT_SLEEP_DELAY
|
15
|
+
}
|
16
|
+
|
17
|
+
def initialize(suites, args = DEFAULT_ARGS)
|
18
|
+
raise ArgumentError.new("No test cases were passed") unless suites
|
19
|
+
|
20
|
+
args ||= DEFAULT_ARGS
|
14
21
|
@pool_size = args[:pool_size] || DEFAULT_POOL_SIZE
|
15
22
|
@limit = args[:limit] || DEFAULT_LIMIT
|
16
23
|
@handler = args[:handler] || DEFAULT_HANDLER.new
|
24
|
+
@sleep_delay = args[:sleep_delay] || DEFAULT_SLEEP_DELAY
|
17
25
|
@data_cache = {}
|
18
26
|
@n = 0
|
19
27
|
@exiting = false
|
@@ -31,123 +39,129 @@ class FuzzBert::Executor
|
|
31
39
|
|
32
40
|
private
|
33
41
|
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
42
|
+
def run_instance(description, test, generator)
|
43
|
+
data = generator.to_data
|
44
|
+
pid = fork do
|
45
|
+
begin
|
46
|
+
test.run(data)
|
47
|
+
rescue StandardError
|
48
|
+
abort
|
49
|
+
end
|
41
50
|
end
|
51
|
+
id = "#{description}/#{generator.description}"
|
52
|
+
@data_cache[pid] = [id, data]
|
42
53
|
end
|
43
|
-
id = "#{description}/#{generator.description}"
|
44
|
-
@data_cache[pid] = [id, data]
|
45
|
-
end
|
46
54
|
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
55
|
+
def trap_child_exit
|
56
|
+
trap(:CHLD) do
|
57
|
+
begin
|
58
|
+
while exitval = Process.wait2(-1, Process::WNOHANG)
|
59
|
+
pid = exitval[0]
|
60
|
+
status = exitval[1]
|
61
|
+
data_ary = @data_cache.delete(pid)
|
62
|
+
unless status.success?
|
63
|
+
handle({
|
64
|
+
id: data_ary[0],
|
65
|
+
data: data_ary[1],
|
66
|
+
pid: pid,
|
67
|
+
status: status
|
68
|
+
}) unless interrupted(status)
|
69
|
+
end
|
70
|
+
@n += 1
|
71
|
+
if @limit == -1 || @n < @limit
|
72
|
+
run_instance(*@producer.next)
|
73
|
+
else
|
74
|
+
@running = false
|
75
|
+
end
|
62
76
|
end
|
77
|
+
rescue Errno::ECHILD
|
63
78
|
end
|
64
|
-
rescue Errno::ECHILD
|
65
79
|
end
|
66
80
|
end
|
67
|
-
end
|
68
81
|
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
82
|
+
def trap_interrupt
|
83
|
+
trap(:INT) do
|
84
|
+
exit! (1) if @exiting
|
85
|
+
@exiting = true
|
86
|
+
graceful_exit
|
87
|
+
end
|
74
88
|
end
|
75
|
-
end
|
76
89
|
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
90
|
+
def graceful_exit
|
91
|
+
puts "\nExiting...Interrupt again to exit immediately"
|
92
|
+
begin
|
93
|
+
while Process.wait; end
|
94
|
+
rescue Errno::ECHILD
|
95
|
+
end
|
96
|
+
exit 0
|
82
97
|
end
|
83
|
-
exit 0
|
84
|
-
end
|
85
|
-
|
86
|
-
def handle(id, data, pid, status)
|
87
|
-
@handler.handle(id, data, pid, status)
|
88
|
-
end
|
89
98
|
|
90
|
-
|
91
|
-
|
92
|
-
|
93
|
-
end
|
94
|
-
|
95
|
-
def conditional_sleep
|
96
|
-
sleep 0.1 until @running == false
|
97
|
-
end
|
99
|
+
def handle(error_data)
|
100
|
+
@handler.handle(error_data)
|
101
|
+
end
|
98
102
|
|
99
|
-
|
100
|
-
|
101
|
-
|
102
|
-
update
|
103
|
+
def interrupted(status)
|
104
|
+
return false if status.exited?
|
105
|
+
return true if status.termsig == nil || status.termsig == 2
|
103
106
|
end
|
104
107
|
|
105
|
-
def
|
106
|
-
@
|
107
|
-
@gen_iter = ProcessSafeEnumerator.new(@suite.generators)
|
108
|
+
def conditional_sleep
|
109
|
+
sleep @sleep_delay until @running == false
|
108
110
|
end
|
109
111
|
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
|
114
|
-
gen = @gen_iter.next
|
115
|
-
rescue StopIteration
|
116
|
-
update
|
117
|
-
end
|
112
|
+
class DataProducer
|
113
|
+
def initialize(suites)
|
114
|
+
@ring = Ring.new(suites)
|
115
|
+
update
|
118
116
|
end
|
119
|
-
|
120
|
-
|
121
|
-
|
122
|
-
|
123
|
-
def initialize(objs)
|
124
|
-
@i = 0
|
125
|
-
objs = [objs] unless objs.respond_to?(:each)
|
126
|
-
@objs = objs.to_a
|
117
|
+
|
118
|
+
def update
|
119
|
+
@suite = @ring.next
|
120
|
+
@gen_iter = ProcessSafeEnumerator.new(@suite.generators)
|
127
121
|
end
|
128
122
|
|
129
123
|
def next
|
130
|
-
|
131
|
-
|
132
|
-
|
124
|
+
gen = nil
|
125
|
+
until gen
|
126
|
+
begin
|
127
|
+
gen = @gen_iter.next
|
128
|
+
rescue StopIteration
|
129
|
+
update
|
130
|
+
end
|
131
|
+
end
|
132
|
+
[@suite.description, @suite.test, gen]
|
133
133
|
end
|
134
|
-
|
134
|
+
|
135
|
+
class Ring
|
136
|
+
def initialize(objs)
|
137
|
+
@i = 0
|
138
|
+
objs = [objs] unless objs.respond_to?(:each)
|
139
|
+
@objs = objs.to_a
|
140
|
+
raise ArgumentError.new("No test cases found") if @objs.empty?
|
141
|
+
end
|
135
142
|
|
136
|
-
|
137
|
-
|
138
|
-
|
139
|
-
|
140
|
-
|
143
|
+
def next
|
144
|
+
obj = @objs[@i]
|
145
|
+
@i = (@i + 1) % @objs.size
|
146
|
+
obj
|
147
|
+
end
|
141
148
|
end
|
142
149
|
|
143
|
-
|
144
|
-
|
145
|
-
|
146
|
-
|
147
|
-
|
150
|
+
#needed because the Fiber used for normal Enumerators has race conditions
|
151
|
+
class ProcessSafeEnumerator
|
152
|
+
def initialize(ary)
|
153
|
+
@i = 0
|
154
|
+
@ary = ary.to_a
|
155
|
+
end
|
156
|
+
|
157
|
+
def next
|
158
|
+
obj = @ary[@i]
|
159
|
+
raise StopIteration unless obj
|
160
|
+
@i += 1
|
161
|
+
obj
|
162
|
+
end
|
148
163
|
end
|
149
164
|
end
|
150
|
-
end
|
151
165
|
|
152
166
|
end
|
153
167
|
|