benchmark-ips 1.2.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/History.txt +18 -0
- data/Manifest.txt +3 -2
- data/{README.txt → README.md} +54 -10
- data/Rakefile +6 -0
- data/lib/benchmark/compare.rb +32 -29
- data/lib/benchmark/helpers.rb +3 -3
- data/lib/benchmark/ips.rb +28 -241
- data/lib/benchmark/ips/job.rb +243 -0
- data/lib/benchmark/ips/report.rb +64 -0
- data/test/test_benchmark_ips.rb +52 -6
- metadata +73 -77
- data/bin/benchmark_ips +0 -3
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 21b099f60a4072356c1870798deab178dc6fe7f5
|
4
|
+
data.tar.gz: 173c42ebe9730b48a3cf019b8f92c093066cd497
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: cd342a80b701d34c2034462a3ee765ac92ef809e2345fac3be3a05673fa34662c1db4ad618065eafd74c50078ab77081ccf3f70efdd63045efc4156a88dd88cb
|
7
|
+
data.tar.gz: 09e2598b1fb76dde4abe095703d3ffea4f5f8dcd30b53492168e69b437d04ca59198d003da417637cc09beecc2f87d025754522fe30915e499606389865fe85a
|
data/History.txt
CHANGED
@@ -1,3 +1,21 @@
|
|
1
|
+
=== 2.0.0 / 2014-06-18
|
2
|
+
|
3
|
+
* The 'Davy Stevenson' release!
|
4
|
+
* Codename: Springtime Hummingbird Dance
|
5
|
+
|
6
|
+
* Big API refactoring so the internal bits are easier to use
|
7
|
+
* Bump to 2.0 because return types changed to make the API better
|
8
|
+
|
9
|
+
* Contributors added:
|
10
|
+
* Davy Stevenson
|
11
|
+
* Juanito Fatas
|
12
|
+
* Benoit Daloze
|
13
|
+
* Matias
|
14
|
+
* Tony Arcieri
|
15
|
+
* Vipul A M
|
16
|
+
* Zachary Scott
|
17
|
+
* schneems (Richard Schneeman)
|
18
|
+
|
1
19
|
=== 1.0.0 / 2012-03-23
|
2
20
|
|
3
21
|
* 1 major enhancement
|
data/Manifest.txt
CHANGED
@@ -1,11 +1,12 @@
|
|
1
1
|
.autotest
|
2
2
|
History.txt
|
3
3
|
Manifest.txt
|
4
|
-
README.
|
4
|
+
README.md
|
5
5
|
Rakefile
|
6
|
-
bin/benchmark_ips
|
7
6
|
lib/benchmark/compare.rb
|
8
7
|
lib/benchmark/helpers.rb
|
9
8
|
lib/benchmark/ips.rb
|
9
|
+
lib/benchmark/ips/report.rb
|
10
|
+
lib/benchmark/ips/job.rb
|
10
11
|
lib/benchmark/timing.rb
|
11
12
|
test/test_benchmark_ips.rb
|
data/{README.txt → README.md}
RENAMED
@@ -1,22 +1,33 @@
|
|
1
|
-
|
1
|
+
[](http://travis-ci.org/evanphx/benchmark-ips)
|
2
|
+
|
3
|
+
# benchmark-ips
|
2
4
|
|
3
5
|
* http://github.com/evanphx/benchmark-ips
|
4
6
|
|
5
|
-
|
7
|
+
## DESCRIPTION:
|
6
8
|
|
7
9
|
A iterations per second enhancement to Benchmark
|
8
10
|
|
9
|
-
|
11
|
+
## FEATURES/PROBLEMS:
|
10
12
|
|
11
13
|
* benchmark/ips - benchmarks a blocks iterations/second. For short snippits
|
12
14
|
of code, ips automatically figures out how many times to run the code
|
13
15
|
to get interesting data. No more guessing at random iteration counts!
|
14
16
|
|
15
|
-
|
17
|
+
## SYNOPSIS:
|
16
18
|
|
19
|
+
```ruby
|
17
20
|
require 'benchmark/ips'
|
18
21
|
|
19
22
|
Benchmark.ips do |x|
|
23
|
+
# Configure the number of seconds used during
|
24
|
+
# the warmup phase and calculation phase
|
25
|
+
x.config(:time => 5, :warmup => 2)
|
26
|
+
|
27
|
+
# These parameters can also be configured this way
|
28
|
+
x.time = 5
|
29
|
+
x.warmup = 2
|
30
|
+
|
20
31
|
# Typical mode, runs the block as many times as it can
|
21
32
|
x.report("addition") { 1 + 2 }
|
22
33
|
|
@@ -28,6 +39,7 @@ Benchmark.ips do |x|
|
|
28
39
|
i = 0
|
29
40
|
while i < times
|
30
41
|
1 + 2
|
42
|
+
i += 1
|
31
43
|
end
|
32
44
|
end
|
33
45
|
|
@@ -35,26 +47,58 @@ Benchmark.ips do |x|
|
|
35
47
|
# the loop that performs the iterations internally to reduce
|
36
48
|
# overhead. Typically not needed, use the |times| form instead.
|
37
49
|
x.report("addition3", "1 + 2")
|
50
|
+
|
51
|
+
# Really long labels should be formatted correctly
|
52
|
+
x.report("addition-test-long-label") { 1 + 2 }
|
53
|
+
|
54
|
+
# Compare the iterations per second of the various reports!
|
55
|
+
x.compare!
|
38
56
|
end
|
57
|
+
```
|
39
58
|
|
40
|
-
|
59
|
+
This will generate the following report:
|
60
|
+
|
61
|
+
```
|
62
|
+
Calculating -------------------------------------
|
63
|
+
addition 147625 i/100ms
|
64
|
+
addition2 151046 i/100ms
|
65
|
+
addition3 172914 i/100ms
|
66
|
+
-------------------------------------------------
|
67
|
+
addition 9247039.5 (±13.2%) i/s - 45320875 in 5.009003s
|
68
|
+
addition2 26436533.8 (±22.1%) i/s - 124461904 in 4.994989s
|
69
|
+
addition3 32227427.9 (±13.0%) i/s - 157524654 in 5.002928s
|
70
|
+
```
|
71
|
+
|
72
|
+
Benchmark/ips will report the number of iterations per second for a given block
|
73
|
+
of code. When analyzing the results, notice the percent of [standard
|
74
|
+
deviation](http://en.wikipedia.org/wiki/Standard\_deviation) which tells us how
|
75
|
+
spread out our measurements are from the average. A high standard deviation
|
76
|
+
could indicate the results having too much variability.
|
77
|
+
|
78
|
+
One benefit to using this method is benchmark-ips automatically determines the
|
79
|
+
data points for testing our code, so we can focus on the results instead of
|
80
|
+
guessing iteration counts as we do with the traditional Benchmark library.
|
81
|
+
|
82
|
+
## REQUIREMENTS:
|
41
83
|
|
42
84
|
* None!
|
43
85
|
|
44
|
-
|
86
|
+
## INSTALL:
|
45
87
|
|
46
|
-
|
88
|
+
$ gem install benchmark-ips
|
47
89
|
|
48
|
-
|
90
|
+
## DEVELOPERS:
|
49
91
|
|
50
92
|
After checking out the source, run:
|
51
93
|
|
52
|
-
|
94
|
+
$ rake newb
|
53
95
|
|
54
96
|
This task will install any missing dependencies, run the tests/specs,
|
55
97
|
and generate the RDoc.
|
56
98
|
|
57
|
-
|
99
|
+
Run the example file `bin/benchmark_ips` to test that everything is working properly.
|
100
|
+
|
101
|
+
## LICENSE:
|
58
102
|
|
59
103
|
(The MIT License)
|
60
104
|
|
data/Rakefile
CHANGED
data/lib/benchmark/compare.rb
CHANGED
@@ -1,41 +1,44 @@
|
|
1
1
|
module Benchmark
|
2
|
-
|
3
|
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
2
|
+
module Compare
|
3
|
+
|
4
|
+
def compare(*reports)
|
5
|
+
return if reports.size < 2
|
6
|
+
|
7
|
+
iter = false
|
8
|
+
sorted = reports.sort do |a,b|
|
9
|
+
if a.respond_to? :ips
|
10
|
+
iter = true
|
11
|
+
b.ips <=> a.ips
|
12
|
+
else
|
13
|
+
a.runtime <=> b.runtime
|
14
|
+
end
|
12
15
|
end
|
13
|
-
end
|
14
|
-
|
15
|
-
best = sorted.shift
|
16
16
|
|
17
|
-
|
18
|
-
|
19
|
-
if iter
|
20
|
-
STDOUT.printf "%20s: %10.1f i/s\n", best.label, best.ips
|
21
|
-
else
|
22
|
-
STDOUT.puts "#{best.rjust(20)}: #{best.runtime}s"
|
23
|
-
end
|
17
|
+
best = sorted.shift
|
24
18
|
|
25
|
-
|
26
|
-
name = report.label
|
19
|
+
STDOUT.puts "\nComparison:"
|
27
20
|
|
28
21
|
if iter
|
29
|
-
|
30
|
-
STDOUT.printf "%20s: %10.1f i/s - %.2fx slower\n", name, report.ips, x
|
22
|
+
STDOUT.printf "%20s: %10.1f i/s\n", best.label, best.ips
|
31
23
|
else
|
32
|
-
|
33
|
-
STDOUT.puts "#{name.rjust(20)}: #{report.runtime}s - #{x}x slower"
|
24
|
+
STDOUT.puts "#{best.rjust(20)}: #{best.runtime}s"
|
34
25
|
end
|
35
|
-
end
|
36
26
|
|
37
|
-
|
27
|
+
sorted.each do |report|
|
28
|
+
name = report.label
|
29
|
+
|
30
|
+
if iter
|
31
|
+
x = (best.ips.to_f / report.ips.to_f)
|
32
|
+
STDOUT.printf "%20s: %10.1f i/s - %.2fx slower\n", name, report.ips, x
|
33
|
+
else
|
34
|
+
x = "%.2f" % (report.ips.to_f / best.ips.to_f)
|
35
|
+
STDOUT.puts "#{name.rjust(20)}: #{report.runtime}s - #{x}x slower"
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
STDOUT.puts
|
40
|
+
end
|
38
41
|
end
|
39
42
|
|
40
|
-
|
43
|
+
extend Benchmark::Compare
|
41
44
|
end
|
data/lib/benchmark/helpers.rb
CHANGED
@@ -20,7 +20,7 @@ module Benchmark
|
|
20
20
|
module_function :fixnum_max
|
21
21
|
|
22
22
|
def fixnum_min
|
23
|
-
|
23
|
+
if Object.const_defined?(:RUBY_ENGINE)
|
24
24
|
case RUBY_ENGINE
|
25
25
|
when "ruby"
|
26
26
|
- 2 ** (wordsize - 2)
|
@@ -29,12 +29,12 @@ module Benchmark
|
|
29
29
|
when "jruby"
|
30
30
|
-9223372036854775808
|
31
31
|
else
|
32
|
-
raise "
|
32
|
+
raise "Minimum Fixnum size now known yet for #{RUBY_ENGINE}"
|
33
33
|
end
|
34
34
|
else
|
35
35
|
- 2 ** (wordsize - 2)
|
36
36
|
end
|
37
|
-
|
37
|
+
end
|
38
38
|
module_function :fixnum_min
|
39
39
|
|
40
40
|
def wordsize
|
data/lib/benchmark/ips.rb
CHANGED
@@ -1,267 +1,54 @@
|
|
1
1
|
# encoding: utf-8
|
2
2
|
require 'benchmark/timing'
|
3
3
|
require 'benchmark/compare'
|
4
|
+
require 'benchmark/ips/report'
|
5
|
+
require 'benchmark/ips/job'
|
4
6
|
|
5
7
|
module Benchmark
|
8
|
+
module IPS
|
9
|
+
VERSION = "2.0.0"
|
10
|
+
CODENAME = "Springtime Hummingbird Dance"
|
6
11
|
|
7
|
-
|
8
|
-
|
12
|
+
def ips(time=nil, warmup=nil)
|
13
|
+
suite = nil
|
9
14
|
|
10
|
-
|
11
|
-
@label = label
|
12
|
-
@microseconds = us
|
13
|
-
@iterations = iters
|
14
|
-
@ips = ips
|
15
|
-
@ips_sd = ips_sd
|
16
|
-
@measurement_cycle = cycles
|
17
|
-
end
|
18
|
-
|
19
|
-
attr_reader :label, :microseconds, :iterations, :ips, :ips_sd, :measurement_cycle
|
20
|
-
|
21
|
-
def seconds
|
22
|
-
@microseconds.to_f / 1_000_000.0
|
23
|
-
end
|
24
|
-
|
25
|
-
def stddev_percentage
|
26
|
-
100.0 * (@ips_sd.to_f / @ips.to_f)
|
27
|
-
end
|
28
|
-
|
29
|
-
alias_method :runtime, :seconds
|
30
|
-
|
31
|
-
def body
|
32
|
-
left = "%10.1f (±%.1f%%) i/s" % [ips, stddev_percentage]
|
33
|
-
left.ljust(20) + (" - %10d in %10.6fs" % [@iterations, runtime])
|
34
|
-
end
|
35
|
-
|
36
|
-
def header
|
37
|
-
@label.rjust(20)
|
38
|
-
end
|
15
|
+
sync, $stdout.sync = $stdout.sync, true
|
39
16
|
|
40
|
-
|
41
|
-
|
42
|
-
end
|
43
|
-
|
44
|
-
def display
|
45
|
-
$stdout.puts to_s
|
46
|
-
end
|
47
|
-
end
|
48
|
-
|
49
|
-
class IPSJob
|
50
|
-
class Entry
|
51
|
-
def initialize(label, action)
|
52
|
-
@label = label
|
53
|
-
|
54
|
-
if action.kind_of? String
|
55
|
-
compile action
|
56
|
-
@action = self
|
57
|
-
@as_action = true
|
58
|
-
else
|
59
|
-
unless action.respond_to? :call
|
60
|
-
raise ArgumentError, "invalid action, must respond to #call"
|
61
|
-
end
|
62
|
-
|
63
|
-
@action = action
|
64
|
-
|
65
|
-
if action.respond_to? :arity and action.arity > 0
|
66
|
-
@call_loop = true
|
67
|
-
else
|
68
|
-
@call_loop = false
|
69
|
-
end
|
70
|
-
|
71
|
-
@as_action = false
|
72
|
-
end
|
17
|
+
if defined? Benchmark::Suite and Suite.current
|
18
|
+
suite = Benchmark::Suite.current
|
73
19
|
end
|
74
20
|
|
75
|
-
|
21
|
+
quiet = suite && !suite.quiet?
|
76
22
|
|
77
|
-
|
78
|
-
|
79
|
-
|
23
|
+
job = Job.new({:suite => suite,
|
24
|
+
:quiet => quiet
|
25
|
+
})
|
80
26
|
|
81
|
-
|
82
|
-
|
27
|
+
job_opts = {}
|
28
|
+
job_opts[:time] = time unless time.nil?
|
29
|
+
job_opts[:warmup] = warmup unless warmup.nil?
|
83
30
|
|
84
|
-
|
31
|
+
job.config job_opts
|
85
32
|
|
86
|
-
|
87
|
-
while i < times
|
88
|
-
act.call
|
89
|
-
i += 1
|
90
|
-
end
|
91
|
-
end
|
92
|
-
|
93
|
-
def compile(str)
|
94
|
-
m = (class << self; self; end)
|
95
|
-
code = <<-CODE
|
96
|
-
def call_times(__total);
|
97
|
-
__i = 0
|
98
|
-
while __i < __total
|
99
|
-
#{str};
|
100
|
-
__i += 1
|
101
|
-
end
|
102
|
-
end
|
103
|
-
CODE
|
104
|
-
m.class_eval code
|
105
|
-
end
|
106
|
-
end
|
107
|
-
|
108
|
-
def initialize
|
109
|
-
@list = []
|
110
|
-
@compare = false
|
111
|
-
end
|
112
|
-
|
113
|
-
attr_reader :compare
|
114
|
-
|
115
|
-
def compare!
|
116
|
-
@compare = true
|
117
|
-
end
|
118
|
-
|
119
|
-
#
|
120
|
-
# Registers the given label and block pair in the job list.
|
121
|
-
#
|
122
|
-
def item(label="", str=nil, &blk) # :yield:
|
123
|
-
if blk and str
|
124
|
-
raise ArgmentError, "specify a block and a str, but not both"
|
125
|
-
end
|
126
|
-
|
127
|
-
action = str || blk
|
128
|
-
raise ArgmentError, "no block or string" unless action
|
129
|
-
|
130
|
-
@list.push Entry.new(label, action)
|
131
|
-
self
|
132
|
-
end
|
133
|
-
|
134
|
-
alias_method :report, :item
|
33
|
+
yield job
|
135
34
|
|
136
|
-
|
137
|
-
attr_reader :list
|
138
|
-
end
|
139
|
-
|
140
|
-
def ips(time=5, warmup=2)
|
141
|
-
suite = nil
|
142
|
-
|
143
|
-
sync, $stdout.sync = $stdout.sync, true
|
144
|
-
|
145
|
-
if defined? Benchmark::Suite and Suite.current
|
146
|
-
suite = Benchmark::Suite.current
|
147
|
-
end
|
35
|
+
$stdout.puts "Calculating -------------------------------------" unless quiet
|
148
36
|
|
149
|
-
|
37
|
+
job.run_warmup
|
150
38
|
|
151
|
-
|
152
|
-
yield job
|
39
|
+
$stdout.puts "-------------------------------------------------" unless quiet
|
153
40
|
|
154
|
-
|
41
|
+
job.run
|
155
42
|
|
156
|
-
|
43
|
+
$stdout.sync = sync
|
157
44
|
|
158
|
-
|
159
|
-
|
160
|
-
job.list.each do |item|
|
161
|
-
suite.warming item.label, warmup if suite
|
162
|
-
|
163
|
-
Timing.clean_env
|
164
|
-
|
165
|
-
unless quiet
|
166
|
-
if item.label.size > 20
|
167
|
-
$stdout.print "#{item.label}\n#{' ' * 20}"
|
168
|
-
else
|
169
|
-
$stdout.print item.label.rjust(20)
|
170
|
-
end
|
171
|
-
end
|
172
|
-
|
173
|
-
before = Time.now
|
174
|
-
target = Time.now + warmup
|
175
|
-
|
176
|
-
warmup_iter = 0
|
177
|
-
|
178
|
-
while Time.now < target
|
179
|
-
item.call_times(1)
|
180
|
-
warmup_iter += 1
|
181
|
-
end
|
182
|
-
|
183
|
-
after = Time.now
|
184
|
-
|
185
|
-
warmup_time = (after.to_f - before.to_f) * 1_000_000.0
|
186
|
-
|
187
|
-
# calculate the time to run approx 100ms
|
188
|
-
|
189
|
-
cycles_per_100ms = ((100_000 / warmup_time) * warmup_iter).to_i
|
190
|
-
cycles_per_100ms = 1 if cycles_per_100ms <= 0
|
191
|
-
|
192
|
-
timing[item] = cycles_per_100ms
|
193
|
-
|
194
|
-
$stdout.printf "%10d i/100ms\n", cycles_per_100ms unless quiet
|
195
|
-
|
196
|
-
suite.warmup_stats warmup_time, cycles_per_100ms if suite
|
197
|
-
end
|
198
|
-
|
199
|
-
$stdout.puts "-------------------------------------------------" unless quiet
|
200
|
-
|
201
|
-
job.list.each do |item|
|
202
|
-
unless quiet
|
203
|
-
if item.label.size > 20
|
204
|
-
$stdout.print "#{item.label}\n#{' ' * 20}"
|
205
|
-
else
|
206
|
-
$stdout.print item.label.rjust(20)
|
207
|
-
end
|
208
|
-
end
|
209
|
-
|
210
|
-
Timing.clean_env
|
211
|
-
|
212
|
-
suite.running item.label, time if suite
|
213
|
-
|
214
|
-
iter = 0
|
215
|
-
|
216
|
-
target = Time.now + time
|
217
|
-
|
218
|
-
measurements = []
|
219
|
-
|
220
|
-
cycles_per_100ms = timing[item]
|
221
|
-
|
222
|
-
while Time.now < target
|
223
|
-
before = Time.now
|
224
|
-
item.call_times cycles_per_100ms
|
225
|
-
after = Time.now
|
226
|
-
|
227
|
-
# If for some reason the timing said this too no time (O_o)
|
228
|
-
# then ignore the iteration entirely and start another.
|
229
|
-
#
|
230
|
-
m = ((after.to_f - before.to_f) * 1_000_000.0)
|
231
|
-
next if m <= 0.0
|
232
|
-
|
233
|
-
iter += cycles_per_100ms
|
234
|
-
|
235
|
-
measurements << m
|
45
|
+
if job.compare?
|
46
|
+
job.run_comparison
|
236
47
|
end
|
237
48
|
|
238
|
-
|
239
|
-
|
240
|
-
seconds = measured_us.to_f / 1_000_000.0
|
241
|
-
|
242
|
-
all_ips = measurements.map { |i| cycles_per_100ms.to_f / (i.to_f / 1_000_000) }
|
243
|
-
|
244
|
-
avg_ips = Timing.mean(all_ips)
|
245
|
-
sd_ips = Timing.stddev(all_ips).round
|
246
|
-
|
247
|
-
rep = IPSReport.new(item.label, measured_us, iter, avg_ips, sd_ips, cycles_per_100ms)
|
248
|
-
|
249
|
-
$stdout.puts " #{rep.body}" unless quiet
|
250
|
-
|
251
|
-
suite.add_report rep, caller(1).first if suite
|
252
|
-
|
253
|
-
reports << rep
|
49
|
+
return job.full_report
|
254
50
|
end
|
255
|
-
|
256
|
-
$stdout.sync = sync
|
257
|
-
|
258
|
-
if job.compare
|
259
|
-
Benchmark.compare(*reports)
|
260
|
-
end
|
261
|
-
|
262
|
-
return reports
|
263
51
|
end
|
264
52
|
|
265
|
-
|
266
|
-
module_function :ips
|
53
|
+
extend Benchmark::IPS
|
267
54
|
end
|
@@ -0,0 +1,243 @@
|
|
1
|
+
module Benchmark
|
2
|
+
module IPS
|
3
|
+
class Job
|
4
|
+
|
5
|
+
MICROSECONDS_PER_100MS = 100_000
|
6
|
+
MICROSECONDS_PER_SECOND = 1_000_000
|
7
|
+
|
8
|
+
class Entry
|
9
|
+
def initialize(label, action)
|
10
|
+
@label = label
|
11
|
+
|
12
|
+
if action.kind_of? String
|
13
|
+
compile action
|
14
|
+
@action = self
|
15
|
+
@as_action = true
|
16
|
+
else
|
17
|
+
unless action.respond_to? :call
|
18
|
+
raise ArgumentError, "invalid action, must respond to #call"
|
19
|
+
end
|
20
|
+
|
21
|
+
@action = action
|
22
|
+
|
23
|
+
if action.respond_to? :arity and action.arity > 0
|
24
|
+
@call_loop = true
|
25
|
+
else
|
26
|
+
@call_loop = false
|
27
|
+
end
|
28
|
+
|
29
|
+
@as_action = false
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
attr_reader :label, :action
|
34
|
+
|
35
|
+
def label_rjust
|
36
|
+
if @label.size > 20
|
37
|
+
"#{@label}\n#{' ' * 20}"
|
38
|
+
else
|
39
|
+
@label.rjust(20)
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def as_action?
|
44
|
+
@as_action
|
45
|
+
end
|
46
|
+
|
47
|
+
def call_times(times)
|
48
|
+
return @action.call(times) if @call_loop
|
49
|
+
|
50
|
+
act = @action
|
51
|
+
|
52
|
+
i = 0
|
53
|
+
while i < times
|
54
|
+
act.call
|
55
|
+
i += 1
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
def compile(str)
|
60
|
+
m = (class << self; self; end)
|
61
|
+
code = <<-CODE
|
62
|
+
def call_times(__total);
|
63
|
+
__i = 0
|
64
|
+
while __i < __total
|
65
|
+
#{str};
|
66
|
+
__i += 1
|
67
|
+
end
|
68
|
+
end
|
69
|
+
CODE
|
70
|
+
m.class_eval code
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
# An array of 2-element arrays, consisting of label and block pairs.
|
75
|
+
attr_reader :list
|
76
|
+
|
77
|
+
# Boolean determining whether to run comparison utility
|
78
|
+
attr_reader :compare
|
79
|
+
|
80
|
+
# Report object containing information about the run
|
81
|
+
attr_reader :full_report
|
82
|
+
|
83
|
+
attr_accessor :warmup, :time
|
84
|
+
attr_reader :timing
|
85
|
+
|
86
|
+
def initialize opts={}
|
87
|
+
@suite = opts[:suite] || nil
|
88
|
+
@quiet = opts[:quiet] || false
|
89
|
+
@list = []
|
90
|
+
@compare = false
|
91
|
+
|
92
|
+
@timing = {}
|
93
|
+
@full_report = Report.new
|
94
|
+
|
95
|
+
# defaults
|
96
|
+
@warmup = 2
|
97
|
+
@time = 5
|
98
|
+
end
|
99
|
+
|
100
|
+
def config opts
|
101
|
+
@warmup = opts[:warmup] if opts[:warmup]
|
102
|
+
@time = opts[:time] if opts[:time]
|
103
|
+
end
|
104
|
+
|
105
|
+
def compare?
|
106
|
+
@compare
|
107
|
+
end
|
108
|
+
|
109
|
+
def compare!
|
110
|
+
@compare = true
|
111
|
+
end
|
112
|
+
|
113
|
+
#
|
114
|
+
# Registers the given label and block pair in the job list.
|
115
|
+
#
|
116
|
+
def item(label="", str=nil, &blk) # :yield:
|
117
|
+
if blk and str
|
118
|
+
raise ArgumentError, "specify a block and a str, but not both"
|
119
|
+
end
|
120
|
+
|
121
|
+
action = str || blk
|
122
|
+
raise ArgumentError, "no block or string" unless action
|
123
|
+
|
124
|
+
@list.push Entry.new(label, action)
|
125
|
+
self
|
126
|
+
end
|
127
|
+
alias_method :report, :item
|
128
|
+
|
129
|
+
# calculate the cycles needed to run for approx 100ms
|
130
|
+
# given the number of iterations to run the given time
|
131
|
+
def cycles_per_100ms time_msec, iters
|
132
|
+
cycles = ((MICROSECONDS_PER_100MS / time_msec) * iters).to_i
|
133
|
+
cycles = 1 if cycles <= 0
|
134
|
+
cycles
|
135
|
+
end
|
136
|
+
|
137
|
+
# calculate the difference in microseconds between
|
138
|
+
# before and after
|
139
|
+
def time_us before, after
|
140
|
+
(after.to_f - before.to_f) * MICROSECONDS_PER_SECOND
|
141
|
+
end
|
142
|
+
|
143
|
+
# calculate the interations per second given the number
|
144
|
+
# of cycles run and the time in microseconds that elapsed
|
145
|
+
def iterations_per_sec cycles, time_us
|
146
|
+
MICROSECONDS_PER_SECOND * (cycles.to_f / time_us.to_f)
|
147
|
+
end
|
148
|
+
|
149
|
+
def run_warmup
|
150
|
+
@list.each do |item|
|
151
|
+
@suite.warming item.label, @warmup if @suite
|
152
|
+
|
153
|
+
unless @quiet
|
154
|
+
$stdout.printf item.label_rjust
|
155
|
+
end
|
156
|
+
|
157
|
+
Timing.clean_env
|
158
|
+
|
159
|
+
before = Time.now
|
160
|
+
target = Time.now + @warmup
|
161
|
+
|
162
|
+
warmup_iter = 0
|
163
|
+
|
164
|
+
while Time.now < target
|
165
|
+
item.call_times(1)
|
166
|
+
warmup_iter += 1
|
167
|
+
end
|
168
|
+
|
169
|
+
after = Time.now
|
170
|
+
|
171
|
+
warmup_time_us = time_us before, after
|
172
|
+
|
173
|
+
@timing[item] = cycles_per_100ms warmup_time_us, warmup_iter
|
174
|
+
|
175
|
+
$stdout.printf "%10d i/100ms\n", @timing[item] unless @quiet
|
176
|
+
|
177
|
+
@suite.warmup_stats warmup_time_us, @timing[item] if @suite
|
178
|
+
end
|
179
|
+
end
|
180
|
+
|
181
|
+
def run
|
182
|
+
@list.each do |item|
|
183
|
+
@suite.running item.label, @time if @suite
|
184
|
+
|
185
|
+
unless @quiet
|
186
|
+
$stdout.print item.label_rjust
|
187
|
+
end
|
188
|
+
|
189
|
+
Timing.clean_env
|
190
|
+
|
191
|
+
iter = 0
|
192
|
+
|
193
|
+
target = Time.now + @time
|
194
|
+
|
195
|
+
measurements_us = []
|
196
|
+
|
197
|
+
# running this number of cycles should take around 100ms
|
198
|
+
cycles = @timing[item]
|
199
|
+
|
200
|
+
while Time.now < target
|
201
|
+
before = Time.now
|
202
|
+
item.call_times cycles
|
203
|
+
after = Time.now
|
204
|
+
|
205
|
+
# If for some reason the timing said this took no time (O_o)
|
206
|
+
# then ignore the iteration entirely and start another.
|
207
|
+
#
|
208
|
+
iter_us = time_us before, after
|
209
|
+
next if iter_us <= 0.0
|
210
|
+
|
211
|
+
iter += cycles
|
212
|
+
|
213
|
+
measurements_us << iter_us
|
214
|
+
end
|
215
|
+
|
216
|
+
measured_us = measurements_us.inject(0) { |a,i| a + i }
|
217
|
+
|
218
|
+
all_ips = measurements_us.map { |time_us|
|
219
|
+
iterations_per_sec cycles, time_us
|
220
|
+
}
|
221
|
+
|
222
|
+
avg_ips = Timing.mean(all_ips)
|
223
|
+
sd_ips = Timing.stddev(all_ips).round
|
224
|
+
|
225
|
+
rep = create_report(item, measured_us, iter, avg_ips, sd_ips, cycles)
|
226
|
+
|
227
|
+
$stdout.puts " #{rep.body}" unless @quiet
|
228
|
+
|
229
|
+
@suite.add_report rep, caller(1).first if @suite
|
230
|
+
end
|
231
|
+
end
|
232
|
+
|
233
|
+
def run_comparison
|
234
|
+
@full_report.run_comparison
|
235
|
+
end
|
236
|
+
|
237
|
+
def create_report(item, measured_us, iter, avg_ips, sd_ips, cycles)
|
238
|
+
@full_report.add_entry item.label, measured_us, iter, avg_ips, sd_ips, cycles
|
239
|
+
end
|
240
|
+
|
241
|
+
end
|
242
|
+
end
|
243
|
+
end
|
@@ -0,0 +1,64 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
module Benchmark
|
3
|
+
module IPS
|
4
|
+
class Report
|
5
|
+
|
6
|
+
class Entry
|
7
|
+
|
8
|
+
def initialize(label, us, iters, ips, ips_sd, cycles)
|
9
|
+
@label = label
|
10
|
+
@microseconds = us
|
11
|
+
@iterations = iters
|
12
|
+
@ips = ips
|
13
|
+
@ips_sd = ips_sd
|
14
|
+
@measurement_cycle = cycles
|
15
|
+
end
|
16
|
+
|
17
|
+
attr_reader :label, :microseconds, :iterations, :ips, :ips_sd, :measurement_cycle
|
18
|
+
|
19
|
+
def seconds
|
20
|
+
@microseconds.to_f / 1_000_000.0
|
21
|
+
end
|
22
|
+
|
23
|
+
def stddev_percentage
|
24
|
+
100.0 * (@ips_sd.to_f / @ips.to_f)
|
25
|
+
end
|
26
|
+
|
27
|
+
alias_method :runtime, :seconds
|
28
|
+
|
29
|
+
def body
|
30
|
+
left = "%10.1f (±%.1f%%) i/s" % [ips, stddev_percentage]
|
31
|
+
left.ljust(20) + (" - %10d in %10.6fs" % [@iterations, runtime])
|
32
|
+
end
|
33
|
+
|
34
|
+
def header
|
35
|
+
@label.rjust(20)
|
36
|
+
end
|
37
|
+
|
38
|
+
def to_s
|
39
|
+
"#{header} #{body}"
|
40
|
+
end
|
41
|
+
|
42
|
+
def display
|
43
|
+
$stdout.puts to_s
|
44
|
+
end
|
45
|
+
end
|
46
|
+
|
47
|
+
attr_reader :entries
|
48
|
+
|
49
|
+
def initialize
|
50
|
+
@entries = []
|
51
|
+
end
|
52
|
+
|
53
|
+
def add_entry label, microseconds, iters, ips, ips_sd, measurement_cycle
|
54
|
+
@entries << Entry.new(label, microseconds, iters, ips, ips_sd, measurement_cycle)
|
55
|
+
@entries.last
|
56
|
+
end
|
57
|
+
|
58
|
+
def run_comparison
|
59
|
+
Benchmark.compare(*@entries)
|
60
|
+
end
|
61
|
+
|
62
|
+
end
|
63
|
+
end
|
64
|
+
end
|
data/test/test_benchmark_ips.rb
CHANGED
@@ -1,8 +1,8 @@
|
|
1
|
-
require "
|
1
|
+
require "minitest/autorun"
|
2
2
|
require "benchmark/ips"
|
3
3
|
require "stringio"
|
4
4
|
|
5
|
-
class TestBenchmarkIPS < Test
|
5
|
+
class TestBenchmarkIPS < Minitest::Test
|
6
6
|
def setup
|
7
7
|
@old_stdout = $stdout
|
8
8
|
$stdout = StringIO.new
|
@@ -13,14 +13,60 @@ class TestBenchmarkIPS < Test::Unit::TestCase
|
|
13
13
|
end
|
14
14
|
|
15
15
|
def test_ips
|
16
|
-
|
17
|
-
x.
|
16
|
+
report = Benchmark.ips do |x|
|
17
|
+
x.config(:time => 1, :warmup => 1)
|
18
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
19
|
+
x.report("sleep 0.05") { sleep(0.05) }
|
20
|
+
x.compare!
|
18
21
|
end
|
19
22
|
|
20
|
-
|
23
|
+
rep1 = report.entries[0]
|
24
|
+
rep2 = report.entries[1]
|
21
25
|
|
22
|
-
assert_equal "sleep",
|
26
|
+
assert_equal "sleep 0.25", rep1.label
|
27
|
+
assert_equal 4, rep1.iterations
|
28
|
+
assert_in_delta 4.0, rep1.ips, 0.2
|
29
|
+
|
30
|
+
assert_equal "sleep 0.05", rep2.label
|
31
|
+
assert_equal 20, rep2.iterations
|
32
|
+
assert_in_delta 20.0, rep2.ips, 1.0
|
33
|
+
end
|
34
|
+
|
35
|
+
def test_ips_alternate_config
|
36
|
+
report = Benchmark.ips do |x|
|
37
|
+
x.time = 1
|
38
|
+
x.warmup = 1
|
39
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
40
|
+
end
|
41
|
+
|
42
|
+
rep = report.entries.first
|
43
|
+
|
44
|
+
assert_equal "sleep 0.25", rep.label
|
23
45
|
assert_equal 4, rep.iterations
|
24
46
|
assert_in_delta 4.0, rep.ips, 0.2
|
25
47
|
end
|
48
|
+
|
49
|
+
def test_ips_old_config
|
50
|
+
report = Benchmark.ips(1,1) do |x|
|
51
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
52
|
+
end
|
53
|
+
|
54
|
+
rep = report.entries.first
|
55
|
+
|
56
|
+
assert_equal "sleep 0.25", rep.label
|
57
|
+
assert_equal 4, rep.iterations
|
58
|
+
assert_in_delta 4.0, rep.ips, 0.2
|
59
|
+
end
|
60
|
+
|
61
|
+
def test_ips_defaults
|
62
|
+
report = Benchmark.ips do |x|
|
63
|
+
x.report("sleep 0.25") { sleep(0.25) }
|
64
|
+
end
|
65
|
+
|
66
|
+
rep = report.entries.first
|
67
|
+
|
68
|
+
assert_equal "sleep 0.25", rep.label
|
69
|
+
assert_equal 4*5, rep.iterations
|
70
|
+
assert_in_delta 4.0, rep.ips, 0.2
|
71
|
+
end
|
26
72
|
end
|
metadata
CHANGED
@@ -1,109 +1,105 @@
|
|
1
|
-
--- !ruby/object:Gem::Specification
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
2
|
name: benchmark-ips
|
3
|
-
version: !ruby/object:Gem::Version
|
4
|
-
|
5
|
-
prerelease:
|
6
|
-
segments:
|
7
|
-
- 1
|
8
|
-
- 2
|
9
|
-
- 0
|
10
|
-
version: 1.2.0
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 2.0.0
|
11
5
|
platform: ruby
|
12
|
-
authors:
|
6
|
+
authors:
|
13
7
|
- Evan Phoenix
|
14
8
|
autorequire:
|
15
9
|
bindir: bin
|
16
10
|
cert_chain: []
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
11
|
+
date: 2014-06-18 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: minitest
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '5.3'
|
20
|
+
type: :development
|
22
21
|
prerelease: false
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '5.3'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rdoc
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '4.0'
|
33
34
|
type: :development
|
34
|
-
version_requirements: *id001
|
35
|
-
- !ruby/object:Gem::Dependency
|
36
|
-
name: hoe
|
37
35
|
prerelease: false
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '4.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: hoe
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '3.12'
|
48
48
|
type: :development
|
49
|
-
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '3.12'
|
50
55
|
description: A iterations per second enhancement to Benchmark
|
51
|
-
email:
|
56
|
+
email:
|
52
57
|
- evan@phx.io
|
53
|
-
executables:
|
54
|
-
- benchmark_ips
|
58
|
+
executables: []
|
55
59
|
extensions: []
|
56
|
-
|
57
|
-
extra_rdoc_files:
|
60
|
+
extra_rdoc_files:
|
58
61
|
- History.txt
|
59
62
|
- Manifest.txt
|
60
|
-
- README.
|
61
|
-
files:
|
62
|
-
- .autotest
|
63
|
+
- README.md
|
64
|
+
files:
|
65
|
+
- ".autotest"
|
66
|
+
- ".gemtest"
|
63
67
|
- History.txt
|
64
68
|
- Manifest.txt
|
65
|
-
- README.
|
69
|
+
- README.md
|
66
70
|
- Rakefile
|
67
|
-
- bin/benchmark_ips
|
68
71
|
- lib/benchmark/compare.rb
|
69
72
|
- lib/benchmark/helpers.rb
|
70
73
|
- lib/benchmark/ips.rb
|
74
|
+
- lib/benchmark/ips/job.rb
|
75
|
+
- lib/benchmark/ips/report.rb
|
71
76
|
- lib/benchmark/timing.rb
|
72
77
|
- test/test_benchmark_ips.rb
|
73
|
-
- .gemtest
|
74
78
|
homepage: http://github.com/evanphx/benchmark-ips
|
75
|
-
licenses:
|
76
|
-
|
79
|
+
licenses:
|
80
|
+
- MIT
|
81
|
+
metadata: {}
|
77
82
|
post_install_message:
|
78
|
-
rdoc_options:
|
79
|
-
- --main
|
80
|
-
- README.
|
81
|
-
require_paths:
|
83
|
+
rdoc_options:
|
84
|
+
- "--main"
|
85
|
+
- README.md
|
86
|
+
require_paths:
|
82
87
|
- lib
|
83
|
-
required_ruby_version: !ruby/object:Gem::Requirement
|
84
|
-
|
85
|
-
requirements:
|
88
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
89
|
+
requirements:
|
86
90
|
- - ">="
|
87
|
-
- !ruby/object:Gem::Version
|
88
|
-
|
89
|
-
|
90
|
-
|
91
|
-
version: "0"
|
92
|
-
required_rubygems_version: !ruby/object:Gem::Requirement
|
93
|
-
none: false
|
94
|
-
requirements:
|
91
|
+
- !ruby/object:Gem::Version
|
92
|
+
version: '0'
|
93
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
94
|
+
requirements:
|
95
95
|
- - ">="
|
96
|
-
- !ruby/object:Gem::Version
|
97
|
-
|
98
|
-
segments:
|
99
|
-
- 0
|
100
|
-
version: "0"
|
96
|
+
- !ruby/object:Gem::Version
|
97
|
+
version: '0'
|
101
98
|
requirements: []
|
102
|
-
|
103
|
-
|
104
|
-
rubygems_version: 1.8.21
|
99
|
+
rubyforge_project:
|
100
|
+
rubygems_version: 2.2.2
|
105
101
|
signing_key:
|
106
|
-
specification_version:
|
102
|
+
specification_version: 4
|
107
103
|
summary: A iterations per second enhancement to Benchmark
|
108
|
-
test_files:
|
104
|
+
test_files:
|
109
105
|
- test/test_benchmark_ips.rb
|
data/bin/benchmark_ips
DELETED