rspec-benchmark 0.5.1 → 0.6.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 5d86ffbdf2e75f6789264a4206e3caf39f32927f5ad751eb6f02b69fc72743dc
4
- data.tar.gz: '096f74a7d6ac5b64b347c38f19757ddd7b870e9dc80996e2dcb09ad13bdacfe2'
3
+ metadata.gz: 907b4211a79dbed8db1edb3b2c6a694536302ef576111d6cab8e16ebe0c628f3
4
+ data.tar.gz: 1ed51ed22ab19a877fafb4c074161b2b4d5c9f2cfc87ac840c552a1b33046825
5
5
  SHA512:
6
- metadata.gz: 673740a08e95e8d0790a9200e30400bb0f2fc0b1ec5f12a4665b709363f0687af9a35a767b336d24aba13d21b25fea334e68011d8572b8c2a0de9e90e845c3e3
7
- data.tar.gz: e5a4fd2ffa45d39980edfd900d385f8df95083ef7d6741cea8fc1a885ce34008feec1d13ad602123861f07acb8a5b1fb1fe1f41f12b150cfe037c2d6aa2e5ffe
6
+ metadata.gz: e312397def220c32daa2a2b3f385560cf12852096722434c508279ffcb41d38b2110ca2ce8179c73a83c8d5abb1faeac3b20b4bc2a6af0cf83aed4781489cb61
7
+ data.tar.gz: a2a1a4f205ee93369b88b354edc6742651a5bd4d0cd8773993c33769227ef1b8e7e62053a05719f8818ef3db96da05af3b1c4ae8ce03ae9bdb8466f7ce68a334
@@ -1,5 +1,22 @@
1
1
  # Change log
2
2
 
3
+ ## [v0.6.0] - 2020-03-09
4
+
5
+ ### Added
6
+ * Add Formatter#format_unit for making numbers more readable
7
+ * Add :format configuration option
8
+
9
+ ### Changed
10
+ * Change TimingMatcher to use new #cpu interface
11
+ * Change IterationMatcher to use new #ips interface
12
+ * Change IterationMatcher to format iterations with units
13
+ * Change gemspec to add metadata, remove test artefacts
14
+ * Change benchmark-perf, benchmark-trend & benchmark-malloc versions
15
+
16
+ ### Fixed
17
+ * Fix Ruby 2.7 warnings converting hash into keywowrd arguments
18
+ * Fix IterationMatcher to stop raising FloatDomainError
19
+
3
20
  ## [v0.5.1] - 2019-09-11
4
21
 
5
22
  ### Fixed
@@ -54,6 +71,7 @@
54
71
 
55
72
  Initial release
56
73
 
74
+ [v0.6.0]: https://github.com/peter-murach/rspec-benchmark/compare/v0.5.1...v0.6.0
57
75
  [v0.5.1]: https://github.com/peter-murach/rspec-benchmark/compare/v0.5.0...v0.5.1
58
76
  [v0.5.0]: https://github.com/peter-murach/rspec-benchmark/compare/v0.4.0...v0.5.0
59
77
  [v0.4.0]: https://github.com/peter-murach/rspec-benchmark/compare/v0.3.0...v0.4.0
data/README.md CHANGED
@@ -41,6 +41,7 @@ If you are new to performance testing you may find [Caveats](#5-caveats) section
41
41
  * [3.1 :disable_gc](#31-disable_gc)
42
42
  * [3.2 :run_in_subprocess](#32-run_in_subprocess)
43
43
  * [3.3 :samples](#33-samples)
44
+ * [3.4 :format](#34-format)
44
45
  * [4. Filtering](#4-filtering)
45
46
  * [5. Caveats](#5-caveats)
46
47
 
@@ -269,7 +270,9 @@ expect { |n, i|
269
270
 
270
271
  The `perform_allocation` matcher checks how much memory or objects have been allocated during a piece of Ruby code execution.
271
272
 
272
- By default the matcher verify the number of object allocations. You can also check for memory allocation using the `bytes` matcher.
273
+ By default the matcher verifies the number of object allocations. The specified number serves as the _upper limit_ of allocations, so your tests won't become brittle as different Ruby versions change internally how many objects are allocated for some operations.
274
+
275
+ Note that you can also check for memory allocation using the `bytes` matcher.
273
276
 
274
277
  To check number of objects allocated do:
275
278
 
@@ -356,6 +359,16 @@ RSpec::Benchmark.configure do |config|
356
359
  end
357
360
  ```
358
361
 
362
+ ### 3.4 `:format`
363
+
364
+ The `perform_at_least` matcher uses the `:format` option to format the number of iterations when a failure message gets displayed. By default, the `:human` values is used to make numbers more readable. For example, the `12300 i/s` gets turned into `12.3k i/s`. If you rather have an exact numbers presented do:
365
+
366
+ ```ruby
367
+ RSpec::Benchmark.configure do |config|
368
+ config.format = :raw
369
+ end
370
+ ```
371
+
359
372
  ## 4. Filtering
360
373
 
361
374
  Usually performance tests are best left for CI or occasional runs that do not affect TDD/BDD cycle.
@@ -1 +1 @@
1
- require_relative 'rspec/benchmark'
1
+ require_relative "rspec/benchmark"
@@ -1,8 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative 'benchmark/configuration'
4
- require_relative 'benchmark/matchers'
5
- require_relative 'benchmark/version'
3
+ require_relative "benchmark/configuration"
4
+ require_relative "benchmark/matchers"
5
+ require_relative "benchmark/version"
6
6
 
7
7
  module RSpec
8
8
  module Benchmark
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'benchmark-malloc'
3
+ require "benchmark-malloc"
4
4
 
5
5
  module RSpec
6
6
  module Benchmark
@@ -31,7 +31,7 @@ module RSpec
31
31
  # @api private
32
32
  def matches?(block)
33
33
  @block = block
34
- alloc_stats = @bench.run(&block)
34
+ alloc_stats = @bench.trace(&block)
35
35
  @actual = nil
36
36
  @actual_retained = nil
37
37
 
@@ -150,10 +150,10 @@ module RSpec
150
150
 
151
151
  def objects_to_s(value)
152
152
  if value.respond_to?(:to_hash)
153
- value.
154
- sort_by { |k,v| k.to_s }.
155
- map { |key, val| "#{val} #{key}" if @objects.keys.include?(key) }.
156
- compact.join(" and ")
153
+ value
154
+ .sort_by { |k, v| k.to_s }
155
+ .map { |key, val| "#{val} #{key}" if @objects.keys.include?(key) }
156
+ .compact.join(" and ")
157
157
  else
158
158
  value
159
159
  end
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'benchmark-perf'
3
+ require "benchmark-perf"
4
4
 
5
5
  module RSpec
6
6
  module Benchmark
@@ -152,9 +152,9 @@ module RSpec
152
152
  return "was not a block" unless @actual.is_a?(Proc)
153
153
 
154
154
  if @ratio < 1
155
- "performed slower by #{format('%.2f', (@ratio**-1))} times"
155
+ "performed slower by #{format("%.2f", (@ratio**-1))} times"
156
156
  elsif @ratio > 1
157
- "performed faster by #{format('%.2f', @ratio)} times"
157
+ "performed faster by #{format("%.2f", @ratio)} times"
158
158
  else
159
159
  "performed by the same time"
160
160
  end
@@ -264,7 +264,7 @@ module RSpec
264
264
 
265
265
  def check_comparison(type)
266
266
  [:slower, :faster].include?(type) ||
267
- (raise ArgumentError, 'comparison_type must be ' \
267
+ (raise ArgumentError, "comparison_type must be " \
268
268
  ":faster or :slower, not `:#{type}`")
269
269
  end
270
270
  end # Matcher
@@ -28,11 +28,19 @@ module RSpec
28
28
  # @api public
29
29
  attr_accessor :fit_quality
30
30
 
31
+ # The formatting for number of iterations
32
+ #
33
+ # @return [String]
34
+ #
35
+ # @api public
36
+ attr_accessor :format
37
+
31
38
  # @api private
32
39
  def initialize
33
40
  @disable_gc = false
34
41
  @samples = 1
35
42
  @fit_quality = 0.9
43
+ @format = :human
36
44
  @run_in_subprocess = false
37
45
  end
38
46
  end # Configuration
@@ -0,0 +1,51 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RSpec
4
+ module Benchmark
5
+ module Formatter
6
+ # Format time for easy matcher reporting
7
+ #
8
+ # @param [Float] time
9
+ # the time to format
10
+ #
11
+ # @return [String]
12
+ # the human readable time value
13
+ #
14
+ # @api public
15
+ def format_time(time)
16
+ if time >= 100.0
17
+ "%.0f sec" % [time]
18
+ elsif time >= 1.0
19
+ "%.3g sec" % [time]
20
+ elsif time >= 1e-3
21
+ "%.3g ms" % [time * 1e3]
22
+ elsif time >= 1e-6
23
+ "%.3g μs" % [time * 1e6]
24
+ else
25
+ "%.3g ns" % [time * 1e9]
26
+ end
27
+ end
28
+ module_function :format_time
29
+
30
+ UNITS = ([""] + %w[k M B T Q]).freeze
31
+
32
+ # Format large numbers and replace thousands with a unit
33
+ # for increased readability
34
+ #
35
+ # @param [Numeric] number
36
+ # the number to format
37
+ #
38
+ # @return [String]
39
+ #
40
+ # @api pubic
41
+ def format_unit(number)
42
+ scale = (Math.log10(number) / 3).to_i
43
+ scale = 0 if scale > 5
44
+ suffix = UNITS[scale]
45
+
46
+ "%.3g#{suffix}" % [number.to_f / (1000 ** scale)]
47
+ end
48
+ module_function :format_unit
49
+ end
50
+ end # Benchmark
51
+ end # RSpec
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'benchmark-perf'
3
+ require "benchmark-perf"
4
4
 
5
5
  module RSpec
6
6
  module Benchmark
@@ -13,7 +13,18 @@ module RSpec
13
13
  @iterations = iterations
14
14
  @time = options.fetch(:time) { 0.2 }
15
15
  @warmup = options.fetch(:warmup) { 0.1 }
16
- @bench = ::Benchmark::Perf::Iteration
16
+ @format = options.fetch(:format) {
17
+ RSpec::Benchmark.configuration.format }
18
+ @bench = ::Benchmark::Perf
19
+ end
20
+
21
+ # Check if human format should be used
22
+ #
23
+ # @return [Boolean]
24
+ #
25
+ # @api private
26
+ def human_format?
27
+ @format == :human
17
28
  end
18
29
 
19
30
  # Indicates this matcher matches against a block
@@ -29,7 +40,7 @@ module RSpec
29
40
  #
30
41
  # @api private
31
42
  def matches?(block)
32
- @average, @stddev, = @bench.run(time: @time, warmup: @warmup, &block)
43
+ @average, @stddev, = @bench.ips(time: @time, warmup: @warmup, &block)
33
44
  @iterations <= (@average + 3 * @stddev)
34
45
  end
35
46
 
@@ -70,11 +81,13 @@ module RSpec
70
81
  end
71
82
 
72
83
  def description
73
- "perform at least #{@iterations} i/s"
84
+ iters = human_format? ? Formatter.format_unit(@iterations) : @iterations
85
+ "perform at least #{iters} i/s"
74
86
  end
75
87
 
76
88
  def actual
77
- "%d %d%%) i/s" % [@average, (@stddev / @average.to_f) * 100]
89
+ avg = human_format? ? Formatter.format_unit(@average) : @average
90
+ "%s (± %d%%) i/s" % [avg, (@stddev / @average.to_f) * 100]
78
91
  end
79
92
 
80
93
  def positive_failure_reason
@@ -1,10 +1,10 @@
1
- # frozen_string_literal
1
+ # frozen_string_literal: true
2
2
 
3
- require_relative 'allocation_matcher'
4
- require_relative 'comparison_matcher'
5
- require_relative 'complexity_matcher'
6
- require_relative 'iteration_matcher'
7
- require_relative 'timing_matcher'
3
+ require_relative "allocation_matcher"
4
+ require_relative "comparison_matcher"
5
+ require_relative "complexity_matcher"
6
+ require_relative "iteration_matcher"
7
+ require_relative "timing_matcher"
8
8
 
9
9
  module RSpec
10
10
  module Benchmark
@@ -35,7 +35,7 @@ module RSpec
35
35
  #
36
36
  # @api public
37
37
  def perform_allocation(objects, **options)
38
- AllocationMatcher::Matcher.new(objects, options)
38
+ AllocationMatcher::Matcher.new(objects, **options)
39
39
  end
40
40
 
41
41
  # Passes if code block performs at least iterations
@@ -48,7 +48,7 @@ module RSpec
48
48
  #
49
49
  # @api public
50
50
  def perform_at_least(iterations, **options)
51
- IterationMatcher::Matcher.new(iterations, options)
51
+ IterationMatcher::Matcher.new(iterations, **options)
52
52
  end
53
53
 
54
54
  # Passes if code block performs under threshold
@@ -62,7 +62,7 @@ module RSpec
62
62
  #
63
63
  # @api public
64
64
  def perform_under(threshold, **options)
65
- TimingMatcher::Matcher.new(threshold, options)
65
+ TimingMatcher::Matcher.new(threshold, **options)
66
66
  end
67
67
 
68
68
  # Passes if code block performs faster than sample block
@@ -75,7 +75,7 @@ module RSpec
75
75
  #
76
76
  # @api public
77
77
  def perform_faster_than(**options, &sample)
78
- ComparisonMatcher::Matcher.new(sample, :faster, options)
78
+ ComparisonMatcher::Matcher.new(sample, :faster, **options)
79
79
  end
80
80
 
81
81
  # Passes if code block performs slower than sample block
@@ -88,7 +88,7 @@ module RSpec
88
88
  #
89
89
  # @api public
90
90
  def perform_slower_than(**options, &sample)
91
- ComparisonMatcher::Matcher.new(sample, :slower, options)
91
+ ComparisonMatcher::Matcher.new(sample, :slower, **options)
92
92
  end
93
93
 
94
94
  # Pass if code block performs constant
@@ -100,7 +100,7 @@ module RSpec
100
100
  #
101
101
  # @api public
102
102
  def perform_constant(**options)
103
- ComplexityMatcher::Matcher.new(:constant, options)
103
+ ComplexityMatcher::Matcher.new(:constant, **options)
104
104
  end
105
105
 
106
106
  # Pass if code block performs logarithmic
@@ -113,7 +113,7 @@ module RSpec
113
113
  #
114
114
  # @api public
115
115
  def perform_logarithmic(**options)
116
- ComplexityMatcher::Matcher.new(:logarithmic, options)
116
+ ComplexityMatcher::Matcher.new(:logarithmic, **options)
117
117
  end
118
118
  alias perform_log perform_logarithmic
119
119
 
@@ -126,7 +126,7 @@ module RSpec
126
126
  #
127
127
  # @api public
128
128
  def perform_linear(**options)
129
- ComplexityMatcher::Matcher.new(:linear, options)
129
+ ComplexityMatcher::Matcher.new(:linear, **options)
130
130
  end
131
131
 
132
132
  # Pass if code block performs power
@@ -138,7 +138,7 @@ module RSpec
138
138
  #
139
139
  # @api public
140
140
  def perform_power(**options)
141
- ComplexityMatcher::Matcher.new(:power, options)
141
+ ComplexityMatcher::Matcher.new(:power, **options)
142
142
  end
143
143
 
144
144
  # Pass if code block performs exponential
@@ -150,7 +150,7 @@ module RSpec
150
150
  #
151
151
  # @api public
152
152
  def perform_exponential(**options)
153
- ComplexityMatcher::Matcher.new(:exponential, options)
153
+ ComplexityMatcher::Matcher.new(:exponential, **options)
154
154
  end
155
155
  alias perform_exp perform_exponential
156
156
 
@@ -1,8 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'benchmark-perf'
3
+ require "benchmark-perf"
4
4
 
5
- require_relative 'format_time'
5
+ require_relative "formatter"
6
6
 
7
7
  module RSpec
8
8
  module Benchmark
@@ -18,13 +18,15 @@ module RSpec
18
18
  def initialize(threshold, **options)
19
19
  @threshold = threshold
20
20
  @samples = options.fetch(:samples) {
21
- RSpec::Benchmark.configuration.samples }
21
+ RSpec::Benchmark.configuration.samples
22
+ }
22
23
  @warmup = options.fetch(:warmup) { 1 }
23
24
  @subprocess = options.fetch(:subprocess) {
24
- RSpec::Benchmark.configuration.run_in_subprocess }
25
+ RSpec::Benchmark.configuration.run_in_subprocess
26
+ }
25
27
  @scale = threshold.to_s.split(/\./).last.size
26
28
  @block = nil
27
- @bench = ::Benchmark::Perf::ExecutionTime
29
+ @bench = ::Benchmark::Perf
28
30
  end
29
31
 
30
32
  # Indicates this matcher matches against a block
@@ -42,10 +44,8 @@ module RSpec
42
44
  def matches?(block)
43
45
  @block = block
44
46
  return false unless block.is_a?(Proc)
45
- @average, @stddev = @bench.run(repeat: @samples,
46
- warmup: @warmup,
47
- subprocess: @subprocess,
48
- &block)
47
+ @average, @stddev = @bench.cpu(repeat: @samples, warmup: @warmup,
48
+ subprocess: @subprocess, &block)
49
49
  @average <= @threshold
50
50
  end
51
51
 
@@ -116,20 +116,20 @@ module RSpec
116
116
  end
117
117
 
118
118
  def description
119
- "perform under #{format_time(@threshold)}"
119
+ "perform under #{Formatter.format_time(@threshold)}"
120
120
  end
121
121
 
122
122
  def actual
123
- "#{format_time(@average)} (± #{format_time(@stddev)})"
123
+ "#{Formatter.format_time(@average)} (± #{Formatter.format_time(@stddev)})"
124
124
  end
125
125
 
126
126
  def positive_failure_reason
127
- return 'was not a block' unless @block.is_a?(Proc)
127
+ return "was not a block" unless @block.is_a?(Proc)
128
128
  "performed above #{actual} "
129
129
  end
130
130
 
131
131
  def negative_failure_reason
132
- return 'was not a block' unless @block.is_a?(Proc)
132
+ return "was not a block" unless @block.is_a?(Proc)
133
133
  "performed #{actual} under"
134
134
  end
135
135
  end # Matcher