benchmark-ips 1.2.0 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/History.txt +18 -0
- data/Manifest.txt +3 -2
- data/{README.txt → README.md} +54 -10
- data/Rakefile +6 -0
- data/lib/benchmark/compare.rb +32 -29
- data/lib/benchmark/helpers.rb +3 -3
- data/lib/benchmark/ips.rb +28 -241
- data/lib/benchmark/ips/job.rb +243 -0
- data/lib/benchmark/ips/report.rb +64 -0
- data/test/test_benchmark_ips.rb +52 -6
- metadata +73 -77
- data/bin/benchmark_ips +0 -3
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 21b099f60a4072356c1870798deab178dc6fe7f5
|
4
|
+
data.tar.gz: 173c42ebe9730b48a3cf019b8f92c093066cd497
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: cd342a80b701d34c2034462a3ee765ac92ef809e2345fac3be3a05673fa34662c1db4ad618065eafd74c50078ab77081ccf3f70efdd63045efc4156a88dd88cb
|
7
|
+
data.tar.gz: 09e2598b1fb76dde4abe095703d3ffea4f5f8dcd30b53492168e69b437d04ca59198d003da417637cc09beecc2f87d025754522fe30915e499606389865fe85a
|
data/History.txt
CHANGED
@@ -1,3 +1,21 @@
|
|
1
|
+
=== 2.0.0 / 2014-06-18
|
2
|
+
|
3
|
+
* The 'Davy Stevenson' release!
|
4
|
+
* Codename: Springtime Hummingbird Dance
|
5
|
+
|
6
|
+
* Big API refactoring so the internal bits are easier to use
|
7
|
+
* Bump to 2.0 because return types changed to make the API better
|
8
|
+
|
9
|
+
* Contributors added:
|
10
|
+
* Davy Stevenson
|
11
|
+
* Juanito Fatas
|
12
|
+
* Benoit Daloze
|
13
|
+
* Matias
|
14
|
+
* Tony Arcieri
|
15
|
+
* Vipul A M
|
16
|
+
* Zachary Scott
|
17
|
+
* schneems (Richard Schneeman)
|
18
|
+
|
1
19
|
=== 1.0.0 / 2012-03-23
|
2
20
|
|
3
21
|
* 1 major enhancement
|
data/Manifest.txt
CHANGED
@@ -1,11 +1,12 @@
|
|
1
1
|
.autotest
|
2
2
|
History.txt
|
3
3
|
Manifest.txt
|
4
|
-
README.
|
4
|
+
README.md
|
5
5
|
Rakefile
|
6
|
-
bin/benchmark_ips
|
7
6
|
lib/benchmark/compare.rb
|
8
7
|
lib/benchmark/helpers.rb
|
9
8
|
lib/benchmark/ips.rb
|
9
|
+
lib/benchmark/ips/report.rb
|
10
|
+
lib/benchmark/ips/job.rb
|
10
11
|
lib/benchmark/timing.rb
|
11
12
|
test/test_benchmark_ips.rb
|
data/{README.txt → README.md}
RENAMED
@@ -1,22 +1,33 @@
|
|
1
|
-
|
1
|
+
[![Build Status](https://secure.travis-ci.org/evanphx/benchmark-ips.png)](http://travis-ci.org/evanphx/benchmark-ips)
|
2
|
+
|
3
|
+
# benchmark-ips
|
2
4
|
|
3
5
|
* http://github.com/evanphx/benchmark-ips
|
4
6
|
|
5
|
-
|
7
|
+
## DESCRIPTION:
|
6
8
|
|
7
9
|
A iterations per second enhancement to Benchmark
|
8
10
|
|
9
|
-
|
11
|
+
## FEATURES/PROBLEMS:
|
10
12
|
|
11
13
|
* benchmark/ips - benchmarks a blocks iterations/second. For short snippits
|
12
14
|
of code, ips automatically figures out how many times to run the code
|
13
15
|
to get interesting data. No more guessing at random iteration counts!
|
14
16
|
|
15
|
-
|
17
|
+
## SYNOPSIS:
|
16
18
|
|
19
|
+
```ruby
|
17
20
|
require 'benchmark/ips'
|
18
21
|
|
19
22
|
Benchmark.ips do |x|
|
23
|
+
# Configure the number of seconds used during
|
24
|
+
# the warmup phase and calculation phase
|
25
|
+
x.config(:time => 5, :warmup => 2)
|
26
|
+
|
27
|
+
# These parameters can also be configured this way
|
28
|
+
x.time = 5
|
29
|
+
x.warmup = 2
|
30
|
+
|
20
31
|
# Typical mode, runs the block as many times as it can
|
21
32
|
x.report("addition") { 1 + 2 }
|
22
33
|
|
@@ -28,6 +39,7 @@ Benchmark.ips do |x|
|
|
28
39
|
i = 0
|
29
40
|
while i < times
|
30
41
|
1 + 2
|
42
|
+
i += 1
|
31
43
|
end
|
32
44
|
end
|
33
45
|
|
@@ -35,26 +47,58 @@ Benchmark.ips do |x|
|
|
35
47
|
# the loop that performs the iterations internally to reduce
|
36
48
|
# overhead. Typically not needed, use the |times| form instead.
|
37
49
|
x.report("addition3", "1 + 2")
|
50
|
+
|
51
|
+
# Really long labels should be formatted correctly
|
52
|
+
x.report("addition-test-long-label") { 1 + 2 }
|
53
|
+
|
54
|
+
# Compare the iterations per second of the various reports!
|
55
|
+
x.compare!
|
38
56
|
end
|
57
|
+
```
|
39
58
|
|
40
|
-
|
59
|
+
This will generate the following report:
|
60
|
+
|
61
|
+
```
|
62
|
+
Calculating -------------------------------------
|
63
|
+
addition 147625 i/100ms
|
64
|
+
addition2 151046 i/100ms
|
65
|
+
addition3 172914 i/100ms
|
66
|
+
-------------------------------------------------
|
67
|
+
addition 9247039.5 (±13.2%) i/s - 45320875 in 5.009003s
|
68
|
+
addition2 26436533.8 (±22.1%) i/s - 124461904 in 4.994989s
|
69
|
+
addition3 32227427.9 (±13.0%) i/s - 157524654 in 5.002928s
|
70
|
+
```
|
71
|
+
|
72
|
+
Benchmark/ips will report the number of iterations per second for a given block
|
73
|
+
of code. When analyzing the results, notice the percent of [standard
|
74
|
+
deviation](http://en.wikipedia.org/wiki/Standard\_deviation) which tells us how
|
75
|
+
spread out our measurements are from the average. A high standard deviation
|
76
|
+
could indicate the results having too much variability.
|
77
|
+
|
78
|
+
One benefit to using this method is benchmark-ips automatically determines the
|
79
|
+
data points for testing our code, so we can focus on the results instead of
|
80
|
+
guessing iteration counts as we do with the traditional Benchmark library.
|
81
|
+
|
82
|
+
## REQUIREMENTS:
|
41
83
|
|
42
84
|
* None!
|
43
85
|
|
44
|
-
|
86
|
+
## INSTALL:
|
45
87
|
|
46
|
-
|
88
|
+
$ gem install benchmark-ips
|
47
89
|
|
48
|
-
|
90
|
+
## DEVELOPERS:
|
49
91
|
|
50
92
|
After checking out the source, run:
|
51
93
|
|
52
|
-
|
94
|
+
$ rake newb
|
53
95
|
|
54
96
|
This task will install any missing dependencies, run the tests/specs,
|
55
97
|
and generate the RDoc.
|
56
98
|
|
57
|
-
|
99
|
+
Run the example file `bin/benchmark_ips` to test that everything is working properly.
|
100
|
+
|
101
|
+
## LICENSE:
|
58
102
|
|
59
103
|
(The MIT License)
|
60
104
|
|
data/Rakefile
CHANGED
data/lib/benchmark/compare.rb
CHANGED
@@ -1,41 +1,44 @@
|
|
1
1
|
module Benchmark
|
2
|
-
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
2
|
+
module Compare
|
3
|
+
|
4
|
+
def compare(*reports)
|
5
|
+
return if reports.size < 2
|
6
|
+
|
7
|
+
iter = false
|
8
|
+
sorted = reports.sort do |a,b|
|
9
|
+
if a.respond_to? :ips
|
10
|
+
iter = true
|
11
|
+
b.ips <=> a.ips
|
12
|
+
else
|
13
|
+
a.runtime <=> b.runtime
|
14
|
+
end
|
12
15
|
end
|
13
|
-
end
|
14
|
-
|
15
|
-
best = sorted.shift
|
16
16
|
|
17
|
-
|
18
|
-
|
19
|
-
if iter
|
20
|
-
STDOUT.printf "%20s: %10.1f i/s\n", best.label, best.ips
|
21
|
-
else
|
22
|
-
STDOUT.puts "#{best.rjust(20)}: #{best.runtime}s"
|
23
|
-
end
|
17
|
+
best = sorted.shift
|
24
18
|
|
25
|
-
|
26
|
-
name = report.label
|
19
|
+
STDOUT.puts "\nComparison:"
|
27
20
|
|
28
21
|
if iter
|
29
|
-
|
30
|
-
STDOUT.printf "%20s: %10.1f i/s - %.2fx slower\n", name, report.ips, x
|
22
|
+
STDOUT.printf "%20s: %10.1f i/s\n", best.label, best.ips
|
31
23
|
else
|
32
|
-
|
33
|
-
STDOUT.puts "#{name.rjust(20)}: #{report.runtime}s - #{x}x slower"
|
24
|
+
STDOUT.puts "#{best.rjust(20)}: #{best.runtime}s"
|
34
25
|
end
|
35
|
-
end
|
36
26
|
|
37
|
-
|
27
|
+
sorted.each do |report|
|
28
|
+
name = report.label
|
29
|
+
|
30
|
+
if iter
|
31
|
+
x = (best.ips.to_f / report.ips.to_f)
|
32
|
+
STDOUT.printf "%20s: %10.1f i/s - %.2fx slower\n", name, report.ips, x
|
33
|
+
else
|
34
|
+
x = "%.2f" % (report.ips.to_f / best.ips.to_f)
|
35
|
+
STDOUT.puts "#{name.rjust(20)}: #{report.runtime}s - #{x}x slower"
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
STDOUT.puts
|
40
|
+
end
|
38
41
|
end
|
39
42
|
|
40
|
-
|
43
|
+
extend Benchmark::Compare
|
41
44
|
end
|
data/lib/benchmark/helpers.rb
CHANGED
@@ -20,7 +20,7 @@ module Benchmark
|
|
20
20
|
module_function :fixnum_max
|
21
21
|
|
22
22
|
def fixnum_min
|
23
|
-
|
23
|
+
if Object.const_defined?(:RUBY_ENGINE)
|
24
24
|
case RUBY_ENGINE
|
25
25
|
when "ruby"
|
26
26
|
- 2 ** (wordsize - 2)
|
@@ -29,12 +29,12 @@ module Benchmark
|
|
29
29
|
when "jruby"
|
30
30
|
-9223372036854775808
|
31
31
|
else
|
32
|
-
raise "
|
32
|
+
raise "Minimum Fixnum size now known yet for #{RUBY_ENGINE}"
|
33
33
|
end
|
34
34
|
else
|
35
35
|
- 2 ** (wordsize - 2)
|
36
36
|
end
|
37
|
-
|
37
|
+
end
|
38
38
|
module_function :fixnum_min
|
39
39
|
|
40
40
|
def wordsize
|
data/lib/benchmark/ips.rb
CHANGED
@@ -1,267 +1,54 @@
|
|
1
1
|
# encoding: utf-8
|
2
2
|
require 'benchmark/timing'
|
3
3
|
require 'benchmark/compare'
|
4
|
+
require 'benchmark/ips/report'
|
5
|
+
require 'benchmark/ips/job'
|
4
6
|
|
5
7
|
module Benchmark
|
8
|
+
module IPS
|
9
|
+
VERSION = "2.0.0"
|
10
|
+
CODENAME = "Springtime Hummingbird Dance"
|
6
11
|
|
7
|
-
|
8
|
-
|
12
|
+
def ips(time=nil, warmup=nil)
|
13
|
+
suite = nil
|
9
14
|
|
10
|
-
|
11
|
-
@label = label
|
12
|
-
@microseconds = us
|
13
|
-
@iterations = iters
|
14
|
-
@ips = ips
|
15
|
-
@ips_sd = ips_sd
|
16
|
-
@measurement_cycle = cycles
|
17
|
-
end
|
18
|
-
|
19
|
-
attr_reader :label, :microseconds, :iterations, :ips, :ips_sd, :measurement_cycle
|
20
|
-
|
21
|
-
def seconds
|
22
|
-
@microseconds.to_f / 1_000_000.0
|
23
|
-
end
|
24
|
-
|
25
|
-
def stddev_percentage
|
26
|
-
100.0 * (@ips_sd.to_f / @ips.to_f)
|
27
|
-
end
|
28
|
-
|
29
|
-
alias_method :runtime, :seconds
|
30
|
-
|
31
|
-
def body
|
32
|
-
left = "%10.1f (±%.1f%%) i/s" % [ips, stddev_percentage]
|
33
|
-
left.ljust(20) + (" - %10d in %10.6fs" % [@iterations, runtime])
|
34
|
-
end
|
35
|
-
|
36
|
-
def header
|
37
|
-
@label.rjust(20)
|
38
|
-
end
|
15
|
+
sync, $stdout.sync = $stdout.sync, true
|
39
16
|
|
40
|
-
|
41
|
-
|
42
|
-
end
|
43
|
-
|
44
|
-
def display
|
45
|
-
$stdout.puts to_s
|
46
|
-
end
|
47
|
-
end
|
48
|
-
|
49
|
-
class IPSJob
|
50
|
-
class Entry
|
51
|
-
def initialize(label, action)
|
52
|
-
@label = label
|
53
|
-
|
54
|
-
if action.kind_of? String
|
55
|
-
compile action
|
56
|
-
@action = self
|
57
|
-
@as_action = true
|
58
|
-
else
|
59
|
-
unless action.respond_to? :call
|
60
|
-
raise ArgumentError, "invalid action, must respond to #call"
|
61
|
-
end
|
62
|
-
|
63
|
-
@action = action
|
64
|
-
|
65
|
-
if action.respond_to? :arity and action.arity > 0
|
66
|
-
@call_loop = true
|
67
|
-
else
|
68
|
-
@call_loop = false
|
69
|
-
end
|
70
|
-
|
71
|
-
@as_action = false
|
72
|
-
end
|
17
|
+
if defined? Benchmark::Suite and Suite.current
|
18
|
+
suite = Benchmark::Suite.current
|
73
19
|
end
|
74
20
|
|
75
|
-
|
21
|
+
quiet = suite && !suite.quiet?
|
76
22
|
|
77
|
-
|
78
|
-
|
79
|
-
|
23
|
+
job = Job.new({:suite => suite,
|
24
|
+
:quiet => quiet
|
25
|
+
})
|
80
26
|
|
81
|
-
|
82
|
-
|
27
|
+
job_opts = {}
|
28
|
+
job_opts[:time] = time unless time.nil?
|
29
|
+
job_opts[:warmup] = warmup unless warmup.nil?
|
83
30
|
|
84
|
-
|
31
|
+
job.config job_opts
|
85
32
|
|
86
|
-
|
87
|
-
while i < times
|
88
|
-
act.call
|
89
|
-
i += 1
|
90
|
-
end
|
91
|
-
end
|
92
|
-
|
93
|
-
def compile(str)
|
94
|
-
m = (class << self; self; end)
|
95
|
-
code = <<-CODE
|
96
|
-
def call_times(__total);
|
97
|
-
__i = 0
|
98
|
-
while __i < __total
|
99
|
-
#{str};
|
100
|
-
__i += 1
|
101
|
-
end
|
102
|
-
end
|
103
|
-
CODE
|
104
|
-
m.class_eval code
|
105
|
-
end
|
106
|
-
end
|
107
|
-
|
108
|
-
def initialize
|
109
|
-
@list = []
|
110
|
-
@compare = false
|
111
|
-
end
|
112
|
-
|
113
|
-
attr_reader :compare
|
114
|
-
|
115
|
-
def compare!
|
116
|
-
@compare = true
|
117
|
-
end
|
118
|
-
|
119
|
-
#
|
120
|
-
# Registers the given label and block pair in the job list.
|
121
|
-
#
|
122
|
-
def item(label="", str=nil, &blk) # :yield:
|
123
|
-
if blk and str
|
124
|
-
raise ArgmentError, "specify a block and a str, but not both"
|
125
|
-
end
|
126
|
-
|
127
|
-
action = str || blk
|
128
|
-
raise ArgmentError, "no block or string" unless action
|
129
|
-
|
130
|
-
@list.push Entry.new(label, action)
|
131
|
-
self
|
132
|
-
end
|
133
|
-
|
134
|
-
alias_method :report, :item
|
33
|
+
yield job
|
135
34
|
|
136
|
-
|
137
|
-
attr_reader :list
|
138
|
-
end
|
139
|
-
|
140
|
-
def ips(time=5, warmup=2)
|
141
|
-
suite = nil
|
142
|
-
|
143
|
-
sync, $stdout.sync = $stdout.sync, true
|
144
|
-
|
145
|
-
if defined? Benchmark::Suite and Suite.current
|
146
|
-
suite = Benchmark::Suite.current
|
147
|
-
end
|
35
|
+
$stdout.puts "Calculating -------------------------------------" unless quiet
|
148
36
|
|
149
|
-
|
37
|
+
job.run_warmup
|
150
38
|
|
151
|
-
|
152
|
-
yield job
|
39
|
+
$stdout.puts "-------------------------------------------------" unless quiet
|
153
40
|
|
154
|
-
|
41
|
+
job.run
|
155
42
|
|
156
|
-
|
43
|
+
$stdout.sync = sync
|
157
44
|
|
158
|
-
|
159
|
-
|
160
|
-
job.list.each do |item|
|
161
|
-
suite.warming item.label, warmup if suite
|
162
|
-
|
163
|
-
Timing.clean_env
|
164
|
-
|
165
|
-
unless quiet
|
166
|
-
if item.label.size > 20
|
167
|
-
$stdout.print "#{item.label}\n#{' ' * 20}"
|
168
|
-
else
|
169
|
-
$stdout.print item.label.rjust(20)
|
170
|
-
end
|
171
|
-
end
|
172
|
-
|
173
|
-
before = Time.now
|
174
|
-
target = Time.now + warmup
|
175
|
-
|
176
|
-
warmup_iter = 0
|
177
|
-
|
178
|
-
while Time.now < target
|
179
|
-
item.call_times(1)
|
180
|
-
warmup_iter += 1
|
181
|
-
end
|
182
|
-
|
183
|
-
after = Time.now
|
184
|
-
|
185
|
-
warmup_time = (after.to_f - before.to_f) * 1_000_000.0
|
186
|
-
|
187
|
-
# calculate the time to run approx 100ms
|
188
|
-
|
189
|
-
cycles_per_100ms = ((100_000 / warmup_time) * warmup_iter).to_i
|
190
|
-
cycles_per_100ms = 1 if cycles_per_100ms <= 0
|
191
|
-
|
192
|
-
timing[item] = cycles_per_100ms
|
193
|
-
|
194
|
-
$stdout.printf "%10d i/100ms\n", cycles_per_100ms unless quiet
|
195
|
-
|
196
|
-
suite.warmup_stats warmup_time, cycles_per_100ms if suite
|
197
|
-
end
|
198
|
-
|
199
|
-
$stdout.puts "-------------------------------------------------" unless quiet
|
200
|
-
|
201
|
-
job.list.each do |item|
|
202
|
-
unless quiet
|
203
|
-
if item.label.size > 20
|
204
|
-
$stdout.print "#{item.label}\n#{' ' * 20}"
|
205
|
-
else
|
206
|
-
$stdout.print item.label.rjust(20)
|
207
|
-
end
|
208
|
-
end
|
209
|
-
|
210
|
-
Timing.clean_env
|
211
|
-
|
212
|
-
suite.running item.label, time if suite
|
213
|
-
|
214
|
-
iter = 0
|
215
|
-
|
216
|
-
target = Time.now + time
|
217
|
-
|
218
|
-
measurements = []
|
219
|
-
|
220
|
-
cycles_per_100ms = timing[item]
|
221
|
-
|
222
|
-
while Time.now < target
|
223
|
-
before = Time.now
|
224
|
-
item.call_times cycles_per_100ms
|
225
|
-
after = Time.now
|
226
|
-
|
227
|
-
# If for some reason the timing said this too no time (O_o)
|
228
|
-
# then ignore the iteration entirely and start another.
|
229
|
-
#
|
230
|
-
m = ((after.to_f - before.to_f) * 1_000_000.0)
|
231
|
-
next if m <= 0.0
|
232
|
-
|
233
|
-
iter += cycles_per_100ms
|
234
|
-
|
235
|
-
measurements << m
|
45
|
+
if job.compare?
|
46
|
+
job.run_comparison
|
236
47
|
end
|
237
48
|
|
238
|
-
|
239
|
-
|
240
|
-
seconds = measured_us.to_f / 1_000_000.0
|
241
|
-
|
242
|
-
all_ips = measurements.map { |i| cycles_per_100ms.to_f / (i.to_f / 1_000_000) }
|
243
|
-
|
244
|
-
avg_ips = Timing.mean(all_ips)
|
245
|
-
sd_ips = Timing.stddev(all_ips).round
|
246
|
-
|
247
|
-
rep = IPSReport.new(item.label, measured_us, iter, avg_ips, sd_ips, cycles_per_100ms)
|
248
|
-
|
249
|
-
$stdout.puts " #{rep.body}" unless quiet
|
250
|
-
|
251
|
-
suite.add_report rep, caller(1).first if suite
|
252
|
-
|
253
|
-
reports << rep
|
49
|
+
return job.full_report
|
254
50
|
end
|
255
|
-
|
256
|
-
$stdout.sync = sync
|
257
|
-
|
258
|
-
if job.compare
|
259
|
-
Benchmark.compare(*reports)
|
260
|
-
end
|
261
|
-
|
262
|
-
return reports
|
263
51
|
end
|
264
52
|
|
265
|
-
|
266
|
-
module_function :ips
|
53
|
+
extend Benchmark::IPS
|
267
54
|
end
|
@@ -0,0 +1,243 @@
|
|
1
|
+
module Benchmark
|
2
|
+
module IPS
|
3
|
+
class Job
|
4
|
+
|
5
|
+
MICROSECONDS_PER_100MS = 100_000
|
6
|
+
MICROSECONDS_PER_SECOND = 1_000_000
|
7
|
+
|
8
|
+
class Entry
|
9
|
+
def initialize(label, action)
|
10
|
+
@label = label
|
11
|
+
|
12
|
+
if action.kind_of? String
|
13
|
+
compile action
|
14
|
+
@action = self
|
15
|
+
@as_action = true
|
16
|
+
else
|
17
|
+
unless action.respond_to? :call
|
18
|
+
raise ArgumentError, "invalid action, must respond to #call"
|
19
|
+
end
|
20
|
+
|
21
|
+
@action = action
|
22
|
+
|
23
|
+
if action.respond_to? :arity and action.arity > 0
|
24
|
+
@call_loop = true
|
25
|
+
else
|
26
|
+
@call_loop = false
|
27
|
+
end
|
28
|
+
|
29
|
+
@as_action = false
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
attr_reader :label, :action
|
34
|
+
|
35
|
+
def label_rjust
|
36
|
+
if @label.size > 20
|
37
|
+
"#{@label}\n#{' ' * 20}"
|
38
|
+
else
|
39
|
+
@label.rjust(20)
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def as_action?
|
44
|
+
@as_action
|
45
|
+
end
|
46
|
+
|
47
|
+
def call_times(times)
|
48
|
+
return @action.call(times) if @call_loop
|
49
|
+
|
50
|
+
act = @action
|
51
|
+
|
52
|
+
i = 0
|
53
|
+
while i < times
|
54
|
+
act.call
|
55
|
+
i += 1
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
def compile(str)
|
60
|
+
m = (class << self; self; end)
|
61
|
+
code = <<-CODE
|
62
|
+
def call_times(__total);
|
63
|
+
__i = 0
|
64
|
+
while __i < __total
|
65
|
+
#{str};
|
66
|
+
__i += 1
|
67
|
+
end
|
68
|
+
end
|
69
|
+
CODE
|
70
|
+
m.class_eval code
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
# An array of 2-element arrays, consisting of label and block pairs.
|
75
|
+
attr_reader :list
|
76
|
+
|
77
|
+
# Boolean determining whether to run comparison utility
|
78
|
+
attr_reader :compare
|
79
|
+
|
80
|
+
# Report object containing information about the run
|
81
|
+
attr_reader :full_report
|
82
|
+
|
83
|
+
attr_accessor :warmup, :time
|
84
|
+
attr_reader :timing
|
85
|
+
|
86
|
+
def initialize opts={}
|
87
|
+
@suite = opts[:suite] || nil
|
88
|
+
@quiet = opts[:quiet] || false
|
89
|
+
@list = []
|
90
|
+
@compare = false
|
91
|
+
|
92
|
+
@timing = {}
|
93
|
+
@full_report = Report.new
|
94
|
+
|
95
|
+
# defaults
|
96
|
+
@warmup = 2
|
97
|
+
@time = 5
|
98
|
+
end
|
99
|
+
|
100
|
+
def config opts
|
101
|
+
@warmup = opts[:warmup] if opts[:warmup]
|
102
|
+
@time = opts[:time] if opts[:time]
|
103
|
+
end
|
104
|
+
|
105
|
+
def compare?
|
106
|
+
@compare
|
107
|
+
end
|
108
|
+
|
109
|
+
def compare!
|
110
|
+
@compare = true
|
111
|
+
end
|
112
|
+
|
113
|
+
#
|
114
|
+
# Registers the given label and block pair in the job list.
|
115
|
+
#
|
116
|
+
def item(label="", str=nil, &blk) # :yield:
|
117
|
+
if blk and str
|
118
|
+
raise ArgumentError, "specify a block and a str, but not both"
|
119
|
+
end
|
120
|
+
|
121
|
+
action = str || blk
|
122
|
+
raise ArgumentError, "no block or string" unless action
|
123
|
+
|
124
|
+
@list.push Entry.new(label, action)
|
125
|
+
self
|
126
|
+
end
|
127
|
+
alias_method :report, :item
|
128
|
+
|
129
|
+
# calculate the cycles needed to run for approx 100ms
|
130
|
+
# given the number of iterations to run the given time
|
131
|
+
def cycles_per_100ms time_msec, iters
|
132
|
+
cycles = ((MICROSECONDS_PER_100MS / time_msec) * iters).to_i
|
133
|
+
cycles = 1 if cycles <= 0
|
134
|
+
cycles
|
135
|
+
end
|
136
|
+
|
137
|
+
# calculate the difference in microseconds between
|
138
|
+
# before and after
|
139
|
+
def time_us before, after
|
140
|
+
(after.to_f - before.to_f) * MICROSECONDS_PER_SECOND
|
141
|
+
end
|
142
|
+
|
143
|
+
# calculate the interations per second given the number
|
144
|
+
# of cycles run and the time in microseconds that elapsed
|
145
|
+
def iterations_per_sec cycles, time_us
|
146
|
+
MICROSECONDS_PER_SECOND * (cycles.to_f / time_us.to_f)
|
147
|
+
end
|
148
|
+
|
149
|
+
def run_warmup
|
150
|
+
@list.each do |item|
|
151
|
+
@suite.warming item.label, @warmup if @suite
|
152
|
+
|
153
|
+
unless @quiet
|
154
|
+
$stdout.printf item.label_rjust
|
155
|
+
end
|
156
|
+
|
157
|
+
Timing.clean_env
|
158
|
+
|
159
|
+
before = Time.now
|
160
|
+
target = Time.now + @warmup
|
161
|
+
|
162
|
+
warmup_iter = 0
|
163
|
+
|
164
|
+
while Time.now < target
|
165
|
+
item.call_times(1)
|
166
|
+
warmup_iter += 1
|
167
|
+
end
|
168
|
+
|
169
|
+
after = Time.now
|
170
|
+
|
171
|
+
warmup_time_us = time_us before, after
|
172
|
+
|
173
|
+
@timing[item] = cycles_per_100ms warmup_time_us, warmup_iter
|
174
|
+
|
175
|
+
$stdout.printf "%10d i/100ms\n", @timing[item] unless @quiet
|
176
|
+
|
177
|
+
@suite.warmup_stats warmup_time_us, @timing[item] if @suite
|
178
|
+
end
|
179
|
+
end
|
180
|
+
|
181
|
+
def run
|
182
|
+
@list.each do |item|
|
183
|
+
@suite.running item.label, @time if @suite
|
184
|
+
|
185
|
+
unless @quiet
|
186
|
+
$stdout.print item.label_rjust
|
187
|
+
end
|
188
|
+
|
189
|
+
Timing.clean_env
|
190
|
+
|
191
|
+
iter = 0
|
192
|
+
|
193
|
+
target = Time.now + @time
|
194
|
+
|
195
|
+
measurements_us = []
|
196
|
+
|
197
|
+
# running this number of cycles should take around 100ms
|
198
|
+
cycles = @timing[item]
|
199
|
+
|
200
|
+
while Time.now < target
|
201
|
+
before = Time.now
|
202
|
+
item.call_times cycles
|
203
|
+
after = Time.now
|
204
|
+
|
205
|
+
# If for some reason the timing said this took no time (O_o)
|
206
|
+
# then ignore the iteration entirely and start another.
|
207
|
+
#
|
208
|
+
iter_us = time_us before, after
|
209
|
+
next if iter_us <= 0.0
|
210
|
+
|
211
|
+
iter += cycles
|
212
|
+
|
213
|
+
measurements_us << iter_us
|
214
|
+
end
|
215
|
+
|
216
|
+
measured_us = measurements_us.inject(0) { |a,i| a + i }
|
217
|
+
|
218
|
+
all_ips = measurements_us.map { |time_us|
|
219
|
+
iterations_per_sec cycles, time_us
|
220
|
+
}
|
221
|
+
|
222
|
+
avg_ips = Timing.mean(all_ips)
|
223
|
+
sd_ips = Timing.stddev(all_ips).round
|
224
|
+
|
225
|
+
rep = create_report(item, measured_us, iter, avg_ips, sd_ips, cycles)
|
226
|
+
|
227
|
+
$stdout.puts " #{rep.body}" unless @quiet
|
228
|
+
|
229
|
+
@suite.add_report rep, caller(1).first if @suite
|
230
|
+
end
|
231
|
+
end
|
232
|
+
|
233
|
+
def run_comparison
|
234
|
+
@full_report.run_comparison
|
235
|
+
end
|
236
|
+
|
237
|
+
def create_report(item, measured_us, iter, avg_ips, sd_ips, cycles)
|
238
|
+
@full_report.add_entry item.label, measured_us, iter, avg_ips, sd_ips, cycles
|
239
|
+
end
|
240
|
+
|
241
|
+
end
|
242
|
+
end
|
243
|
+
end
|
@@ -0,0 +1,64 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
module Benchmark
|
3
|
+
module IPS
|
4
|
+
class Report
|
5
|
+
|
6
|
+
class Entry
|
7
|
+
|
8
|
+
def initialize(label, us, iters, ips, ips_sd, cycles)
|
9
|
+
@label = label
|
10
|
+
@microseconds = us
|
11
|
+
@iterations = iters
|
12
|
+
@ips = ips
|
13
|
+
@ips_sd = ips_sd
|
14
|
+
@measurement_cycle = cycles
|
15
|
+
end
|
16
|
+
|
17
|
+
attr_reader :label, :microseconds, :iterations, :ips, :ips_sd, :measurement_cycle
|
18
|
+
|
19
|
+
def seconds
|
20
|
+
@microseconds.to_f / 1_000_000.0
|
21
|
+
end
|
22
|
+
|
23
|
+
def stddev_percentage
|
24
|
+
100.0 * (@ips_sd.to_f / @ips.to_f)
|
25
|
+
end
|
26
|
+
|
27
|
+
alias_method :runtime, :seconds
|
28
|
+
|
29
|
+
def body
|
30
|
+
left = "%10.1f (±%.1f%%) i/s" % [ips, stddev_percentage]
|
31
|
+
left.ljust(20) + (" - %10d in %10.6fs" % [@iterations, runtime])
|
32
|
+
end
|
33
|
+
|
34
|
+
def header
|
35
|
+
@label.rjust(20)
|
36
|
+
end
|
37
|
+
|
38
|
+
def to_s
|
39
|
+
"#{header} #{body}"
|
40
|
+
end
|
41
|
+
|
42
|
+
def display
|
43
|
+
$stdout.puts to_s
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
attr_reader :entries
|
48
|
+
|
49
|
+
def initialize
|
50
|
+
@entries = []
|
51
|
+
end
|
52
|
+
|
53
|
+
def add_entry label, microseconds, iters, ips, ips_sd, measurement_cycle
|
54
|
+
@entries << Entry.new(label, microseconds, iters, ips, ips_sd, measurement_cycle)
|
55
|
+
@entries.last
|
56
|
+
end
|
57
|
+
|
58
|
+
def run_comparison
|
59
|
+
Benchmark.compare(*@entries)
|
60
|
+
end
|
61
|
+
|
62
|
+
end
|
63
|
+
end
|
64
|
+
end
|
data/test/test_benchmark_ips.rb
CHANGED
@@ -1,8 +1,8 @@
|
|
1
|
-
require "
|
1
|
+
require "minitest/autorun"
|
2
2
|
require "benchmark/ips"
|
3
3
|
require "stringio"
|
4
4
|
|
5
|
-
class TestBenchmarkIPS < Test
|
5
|
+
class TestBenchmarkIPS < Minitest::Test
|
6
6
|
def setup
|
7
7
|
@old_stdout = $stdout
|
8
8
|
$stdout = StringIO.new
|
@@ -13,14 +13,60 @@ class TestBenchmarkIPS < Test::Unit::TestCase
|
|
13
13
|
end
|
14
14
|
|
15
15
|
def test_ips
|
16
|
-
|
17
|
-
x.
|
16
|
+
report = Benchmark.ips do |x|
|
17
|
+
x.config(:time => 1, :warmup => 1)
|
18
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
19
|
+
x.report("sleep 0.05") { sleep(0.05) }
|
20
|
+
x.compare!
|
18
21
|
end
|
19
22
|
|
20
|
-
|
23
|
+
rep1 = report.entries[0]
|
24
|
+
rep2 = report.entries[1]
|
21
25
|
|
22
|
-
assert_equal "sleep",
|
26
|
+
assert_equal "sleep 0.25", rep1.label
|
27
|
+
assert_equal 4, rep1.iterations
|
28
|
+
assert_in_delta 4.0, rep1.ips, 0.2
|
29
|
+
|
30
|
+
assert_equal "sleep 0.05", rep2.label
|
31
|
+
assert_equal 20, rep2.iterations
|
32
|
+
assert_in_delta 20.0, rep2.ips, 1.0
|
33
|
+
end
|
34
|
+
|
35
|
+
def test_ips_alternate_config
|
36
|
+
report = Benchmark.ips do |x|
|
37
|
+
x.time = 1
|
38
|
+
x.warmup = 1
|
39
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
40
|
+
end
|
41
|
+
|
42
|
+
rep = report.entries.first
|
43
|
+
|
44
|
+
assert_equal "sleep 0.25", rep.label
|
23
45
|
assert_equal 4, rep.iterations
|
24
46
|
assert_in_delta 4.0, rep.ips, 0.2
|
25
47
|
end
|
48
|
+
|
49
|
+
def test_ips_old_config
|
50
|
+
report = Benchmark.ips(1,1) do |x|
|
51
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
52
|
+
end
|
53
|
+
|
54
|
+
rep = report.entries.first
|
55
|
+
|
56
|
+
assert_equal "sleep 0.25", rep.label
|
57
|
+
assert_equal 4, rep.iterations
|
58
|
+
assert_in_delta 4.0, rep.ips, 0.2
|
59
|
+
end
|
60
|
+
|
61
|
+
def test_ips_defaults
|
62
|
+
report = Benchmark.ips do |x|
|
63
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
64
|
+
end
|
65
|
+
|
66
|
+
rep = report.entries.first
|
67
|
+
|
68
|
+
assert_equal "sleep 0.25", rep.label
|
69
|
+
assert_equal 4*5, rep.iterations
|
70
|
+
assert_in_delta 4.0, rep.ips, 0.2
|
71
|
+
end
|
26
72
|
end
|
metadata
CHANGED
@@ -1,109 +1,105 @@
|
|
1
|
-
--- !ruby/object:Gem::Specification
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
2
|
name: benchmark-ips
|
3
|
-
version: !ruby/object:Gem::Version
|
4
|
-
|
5
|
-
prerelease:
|
6
|
-
segments:
|
7
|
-
- 1
|
8
|
-
- 2
|
9
|
-
- 0
|
10
|
-
version: 1.2.0
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 2.0.0
|
11
5
|
platform: ruby
|
12
|
-
authors:
|
6
|
+
authors:
|
13
7
|
- Evan Phoenix
|
14
8
|
autorequire:
|
15
9
|
bindir: bin
|
16
10
|
cert_chain: []
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
11
|
+
date: 2014-06-18 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: minitest
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '5.3'
|
20
|
+
type: :development
|
22
21
|
prerelease: false
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '5.3'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rdoc
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '4.0'
|
33
34
|
type: :development
|
34
|
-
version_requirements: *id001
|
35
|
-
- !ruby/object:Gem::Dependency
|
36
|
-
name: hoe
|
37
35
|
prerelease: false
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '4.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: hoe
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '3.12'
|
48
48
|
type: :development
|
49
|
-
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '3.12'
|
50
55
|
description: A iterations per second enhancement to Benchmark
|
51
|
-
email:
|
56
|
+
email:
|
52
57
|
- evan@phx.io
|
53
|
-
executables:
|
54
|
-
- benchmark_ips
|
58
|
+
executables: []
|
55
59
|
extensions: []
|
56
|
-
|
57
|
-
extra_rdoc_files:
|
60
|
+
extra_rdoc_files:
|
58
61
|
- History.txt
|
59
62
|
- Manifest.txt
|
60
|
-
- README.
|
61
|
-
files:
|
62
|
-
- .autotest
|
63
|
+
- README.md
|
64
|
+
files:
|
65
|
+
- ".autotest"
|
66
|
+
- ".gemtest"
|
63
67
|
- History.txt
|
64
68
|
- Manifest.txt
|
65
|
-
- README.
|
69
|
+
- README.md
|
66
70
|
- Rakefile
|
67
|
-
- bin/benchmark_ips
|
68
71
|
- lib/benchmark/compare.rb
|
69
72
|
- lib/benchmark/helpers.rb
|
70
73
|
- lib/benchmark/ips.rb
|
74
|
+
- lib/benchmark/ips/job.rb
|
75
|
+
- lib/benchmark/ips/report.rb
|
71
76
|
- lib/benchmark/timing.rb
|
72
77
|
- test/test_benchmark_ips.rb
|
73
|
-
- .gemtest
|
74
78
|
homepage: http://github.com/evanphx/benchmark-ips
|
75
|
-
licenses:
|
76
|
-
|
79
|
+
licenses:
|
80
|
+
- MIT
|
81
|
+
metadata: {}
|
77
82
|
post_install_message:
|
78
|
-
rdoc_options:
|
79
|
-
- --main
|
80
|
-
- README.
|
81
|
-
require_paths:
|
83
|
+
rdoc_options:
|
84
|
+
- "--main"
|
85
|
+
- README.md
|
86
|
+
require_paths:
|
82
87
|
- lib
|
83
|
-
required_ruby_version: !ruby/object:Gem::Requirement
|
84
|
-
|
85
|
-
requirements:
|
88
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
89
|
+
requirements:
|
86
90
|
- - ">="
|
87
|
-
- !ruby/object:Gem::Version
|
88
|
-
|
89
|
-
|
90
|
-
|
91
|
-
version: "0"
|
92
|
-
required_rubygems_version: !ruby/object:Gem::Requirement
|
93
|
-
none: false
|
94
|
-
requirements:
|
91
|
+
- !ruby/object:Gem::Version
|
92
|
+
version: '0'
|
93
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
94
|
+
requirements:
|
95
95
|
- - ">="
|
96
|
-
- !ruby/object:Gem::Version
|
97
|
-
|
98
|
-
segments:
|
99
|
-
- 0
|
100
|
-
version: "0"
|
96
|
+
- !ruby/object:Gem::Version
|
97
|
+
version: '0'
|
101
98
|
requirements: []
|
102
|
-
|
103
|
-
|
104
|
-
rubygems_version: 1.8.21
|
99
|
+
rubyforge_project:
|
100
|
+
rubygems_version: 2.2.2
|
105
101
|
signing_key:
|
106
|
-
specification_version:
|
102
|
+
specification_version: 4
|
107
103
|
summary: A iterations per second enhancement to Benchmark
|
108
|
-
test_files:
|
104
|
+
test_files:
|
109
105
|
- test/test_benchmark_ips.rb
|
data/bin/benchmark_ips
DELETED