ips 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6b0cebe0c91bffd744baf3e8f52b8215799699ba4ca56d0856af1802dfc3df13
4
- data.tar.gz: 43f27237a0abb534d951ac7da7c96fe0f8e3f053a1910a0534fcbfafaaf4dabf
3
+ metadata.gz: e1f6fa1d4276f6a2e4490785949a2f9d776fd22deefd7bf8b4a32a1373e1ace2
4
+ data.tar.gz: dc254907783de35d83da05e504aff4b94479fcbb0b18e5fd227fc9af97864a6e
5
5
  SHA512:
6
- metadata.gz: 5bbe4651a8879db139bdd2be7b551d4f5787a42cb22ab145056b92883e02acc7818144b61689f94e70f3218ea9f7566f8e337a97ef0a6ba9b280174a508cafb4
7
- data.tar.gz: 3665d4bf59457dcfb246908fcc7f4d0eba583790d9171995f25dc02dad786fb6f5ab44ac4e32ee573b4775a87f4dfbbeb46e4389b1b3ad77ce2d994da5a51d3b
6
+ metadata.gz: aaabca2ef626e4c0856f7ba3786fc0f91613b8a52a5681351dcd5dfd563918188dc62f2ffd2b1e25b6940ddabe447da85b9608002c63d01373a3de95548c990c
7
+ data.tar.gz: e9ba746e1183237048aff5b35f41865d7886ee6e57b77be7c238f1d9ae09f2e707599adf7aa32c98bfee964ab3ae7a909cf80d5d1602b05dd989b28396d9b0dd
data/README.md CHANGED
@@ -1,47 +1,54 @@
1
1
  # IPS
2
2
 
3
- A Ruby benchmarking tool that measures iterations per second with statistical analysis, automatic warmup, and comparison summaries.
3
+ ![demo](demo.gif)
4
4
 
5
5
  ## Installation
6
6
 
7
- Install the gem and add to the application's Gemfile by executing:
8
-
9
- ```bash
10
- bundle add ips
11
- ```
12
-
13
- If bundler is not being used to manage dependencies, install the gem by executing:
14
-
15
7
  ```bash
16
8
  gem install ips
17
9
  ```
18
10
 
19
11
  ## Usage
20
12
 
13
+ ### Library
14
+
21
15
  ```ruby
22
16
  require "ips"
23
17
 
24
18
  IPS.run do |x|
25
- x.report("addition") { 1 + 1 }
26
- x.report("multiplication") { 2 * 3 }
27
- x.report("wtf") { eval("#{1.to_s} + #{1.to_s}") }
19
+ x.report("split/join") { "hello world".split(" ").join("-") }
20
+ x.report("gsub") { "hello world".gsub(" ", "-") }
28
21
  end
29
22
  ```
30
23
 
31
- Output:
24
+ ### CLI
25
+
26
+ ```
27
+ $ ips -e '"hello world".split(" ").join("-")' -e '"hello world".gsub(" ", "-")'
28
+ ```
29
+
30
+ Compare across Ruby versions:
32
31
 
33
32
  ```
34
- addition: 45.759M i/s 1.2%)
35
- multiplication: 45.593M i/s (± 0.2%)
36
- wtf: 463.420k i/s (± 0.9%, GC 6.3%)
33
+ $ ips --ruby 3.4 --ruby 4.0 -e 'Object.new'
34
+ ```
37
35
 
38
- Summary
39
- addition ran
40
- 1.00 ± 0.01 times faster than multiplication
41
- 98.74 ± 1.49 times faster than wtf
36
+ Compare across Ruby versions, arguments, and commands:
37
+
38
+ ```
39
+ $ ips --ruby '3.4' --ruby '3.4 --yjit' --ruby '4.0' --ruby '4.0 --yjit' -e 'Object.new' -e 'Object.allocate'
42
40
  ```
43
41
 
44
- When two or more items are reported, a comparison summary is automatically displayed.
42
+ Options:
43
+
44
+ - `--ruby VERSION` — run against a specific Ruby (repeatable)
45
+ - `-e CODE` — inline benchmark expression (repeatable)
46
+ - `-t SECONDS` — benchmark time (default: 5)
47
+ - `-w SECONDS` — warmup time (default: 20% of benchmark time)
48
+ - `--save PATH` — save results to JSON (appends to existing file)
49
+ - `--debug` — show compiled source, cycle counts, and per-batch timing
50
+ - `--yjit`, `--zjit`, `--disable-gems` — passed through to Ruby
51
+ - `-r LIB` — require a library before running
45
52
 
46
53
  ## Contributing
47
54
 
data/demo.gif ADDED
Binary file
data/demo.tape ADDED
@@ -0,0 +1,16 @@
1
+ # https://github.com/charmbracelet/vhs
2
+ Output demo.gif
3
+
4
+ Set Width 960
5
+ Set Height 500
6
+ Set FontSize 14
7
+ Set FontFamily "Menlo"
8
+ Set Theme "Catppuccin Mocha"
9
+ Set TypingSpeed 10ms
10
+ Set Padding 20
11
+
12
+ Type "ips -t 2 --ruby 3.4 --ruby 4.0 --ruby '4.0 --yjit' -e 'Object.new' -e 'Object.allocate'"
13
+ Sleep 250ms
14
+ Enter
15
+
16
+ Sleep 25s
data/exe/ips ADDED
@@ -0,0 +1,156 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ versions = []
5
+ inline_code = []
6
+ ruby_flags = []
7
+ save_path = nil
8
+ bench_time = nil
9
+ warmup_time = nil
10
+ debug = false
11
+ frozen_string_literal = true
12
+ args = ARGV.dup
13
+ while args.first
14
+ case args.first
15
+ when "--ruby"
16
+ args.shift
17
+ versions << args.shift
18
+ when "-e"
19
+ args.shift
20
+ inline_code << args.shift
21
+ when "-r"
22
+ args.shift
23
+ ruby_flags.push("-r", args.shift)
24
+ when "--disable-gems", /\A--yjit/, /\A--zjit/
25
+ ruby_flags << args.shift
26
+ when "--save"
27
+ args.shift
28
+ save_path = args.shift
29
+ when "-t"
30
+ args.shift
31
+ bench_time = args.shift.to_f
32
+ when "-w"
33
+ args.shift
34
+ warmup_time = args.shift.to_f
35
+ when "--no-frozen-string-literal"
36
+ args.shift
37
+ frozen_string_literal = false
38
+ when "--debug"
39
+ args.shift
40
+ debug = true
41
+ when "-v", "--verbose"
42
+ args.shift
43
+ verbose = true
44
+ when /\A-/
45
+ $stderr.puts "Unknown option: #{args.first}"
46
+ exit 1
47
+ else
48
+ break
49
+ end
50
+ end
51
+ script = args.shift
52
+
53
+ if script.nil? && inline_code.empty?
54
+ $stderr.puts "Usage: ips [--ruby VERSION ...] [-e CODE ...] [<script>]"
55
+ exit 1
56
+ end
57
+
58
+ if inline_code.any?
59
+ run_opts = []
60
+ run_opts << "summary: false" if versions.size > 1
61
+ run_opts << "time: #{bench_time}" if bench_time
62
+ run_opts << "warmup: #{warmup_time}" if warmup_time
63
+ run_opts << "debug: true" if debug
64
+ run_opts << "frozen_string_literal: false" unless frozen_string_literal
65
+ run_opts_str = run_opts.any? ? "(#{run_opts.join(", ")})" : ""
66
+ script_source = +""
67
+ script_source << "# frozen_string_literal: true\n" if frozen_string_literal
68
+ script_source << "require 'ips'\nresult = IPS.run#{run_opts_str} do |x|\n"
69
+ inline_code.each do |code|
70
+ script_source << " x.report(#{code.inspect}, #{code.inspect})\n"
71
+ end
72
+ script_source << "end\n"
73
+ script_source << "IO.open(3, 'w') { |f| Marshal.dump(result, f) }\n"
74
+ ruby_args = ["-e", script_source]
75
+ else
76
+ ruby_args = [script]
77
+ end
78
+
79
+ # Shell-escape for display purposes only, not for security.
80
+ def shell_quote(arg)
81
+ return arg if arg =~ /\A[A-Za-z0-9_\-.,:+\/@]+\z/
82
+ "'" + arg.gsub("'", "'\\\\''") + "'"
83
+ end
84
+
85
+ RUBY_DIRS = [File.expand_path("~/.rubies"), "/opt/rubies"]
86
+ LIB_DIR = File.expand_path("../lib", __dir__)
87
+
88
+ def find_ruby(version)
89
+ RUBY_DIRS.each do |dir|
90
+ next unless Dir.exist?(dir)
91
+ Dir.children(dir).sort.each do |name|
92
+ bare = name.delete_prefix("ruby-")
93
+ if bare.start_with?(version)
94
+ ruby = File.join(dir, name, "bin", "ruby")
95
+ return ruby if File.executable?(ruby)
96
+ end
97
+ end
98
+ end
99
+ nil
100
+ end
101
+
102
+ def resolve_ruby(str)
103
+ version, *extra_flags = str.split
104
+ if File.executable?(version)
105
+ ruby = version
106
+ else
107
+ ruby = find_ruby(version) or abort "No ruby found matching #{version.inspect}"
108
+ end
109
+ [ruby, extra_flags]
110
+ end
111
+
112
+ if versions.empty?
113
+ rubies = [[nil, nil, []]]
114
+ else
115
+ rubies = versions.map { |v|
116
+ ruby, extra_flags = resolve_ruby(v)
117
+ [v, ruby, extra_flags]
118
+ }
119
+ end
120
+
121
+ $LOAD_PATH.unshift(LIB_DIR)
122
+ require "ips"
123
+
124
+ results = []
125
+
126
+ rubies.each do |version_str, ruby, extra_flags|
127
+ if ruby
128
+ env = { "GEM_HOME" => nil, "GEM_PATH" => nil }
129
+ else
130
+ ruby = "ruby"
131
+ env = {}
132
+ end
133
+
134
+ cmd = [ruby, *extra_flags, *ruby_flags, "-v", "-I", LIB_DIR, *ruby_args]
135
+ $stderr.puts cmd.map { |a| shell_quote(a) }.join(" ") if verbose
136
+ rd, wr = IO.pipe
137
+ pid = spawn(env, *cmd, 3 => wr)
138
+ wr.close
139
+ data = rd.read
140
+ rd.close
141
+ Process.wait(pid)
142
+ raise "Child process failed" unless $?.success?
143
+ unless data.empty?
144
+ result = Marshal.load(data)
145
+ result.ruby_executable = ruby
146
+ result.run_label = "ruby #{version_str}" if version_str
147
+ results << result
148
+ end
149
+ puts
150
+ end
151
+
152
+ IPS::Result.compare(results)
153
+
154
+ if save_path && results.any?
155
+ IPS::Result.save!(save_path, results)
156
+ end
data/lib/ips/display.rb CHANGED
@@ -4,11 +4,11 @@ module IPS
4
4
  class Display
5
5
  BAR_WIDTH = 30
6
6
 
7
- def initialize(labels, out: $stdout)
7
+ def initialize(labels, out: $stdout, debug: false)
8
8
  @out = out
9
9
  @max_label = labels.map(&:size).max
10
10
  @max_label = 20 if @max_label < 20
11
- @tty = @out.tty?
11
+ @tty = @out.tty? && !debug
12
12
  end
13
13
 
14
14
  def start_item(label, total_ns)
@@ -42,22 +42,32 @@ module IPS
42
42
 
43
43
  def summary(results)
44
44
  return if results.size < 2
45
+ pairs = results.map { |r| [r.label, r] }
46
+ Display.compare(pairs, out: @out)
47
+ end
48
+
49
+ def self.compare(pairs, out: $stdout)
50
+ return if pairs.size < 2
45
51
 
46
- sorted = results.sort_by { |r| -r.ips }
47
- best = sorted.first
52
+ sorted = pairs.sort_by { |_, e| -e.ips }
53
+ best_label, best = sorted.first
48
54
 
49
- @out.puts "\nSummary"
50
- @out.puts " #{best.label} ran"
55
+ out.puts "\nSummary"
56
+ out.puts " #{best_label} ran"
51
57
 
52
- sorted[1..].each do |r|
53
- ratio = best.ips / r.ips
54
- ratio_error = ratio * Math.sqrt((best.stddev / best.ips)**2 + (r.stddev / r.ips)**2)
55
- @out.printf " %.2f ± %.2f times faster than %s\n",
56
- ratio, ratio_error, r.label
58
+ sorted[1..].each do |label, entry|
59
+ ratio = best.ips / entry.ips
60
+ ratio_error = ratio * Math.sqrt((best.stddev / best.ips)**2 + (entry.stddev / entry.ips)**2)
61
+ out.printf " %.2f ± %.2f times faster than %s\n",
62
+ ratio, ratio_error, label
57
63
  end
58
64
  end
59
65
 
60
66
  def format_ips(ips)
67
+ Display.format_ips(ips)
68
+ end
69
+
70
+ def self.format_ips(ips)
61
71
  if ips >= 1_000_000_000
62
72
  "%.3fB" % (ips / 1_000_000_000.0)
63
73
  elsif ips >= 1_000_000
data/lib/ips/job/entry.rb CHANGED
@@ -3,23 +3,40 @@
3
3
  module IPS
4
4
  class Job
5
5
  class Entry
6
- attr_reader :label
6
+ attr_reader :label, :source
7
7
 
8
- def initialize(label, action)
8
+ def initialize(label, action, frozen_string_literal:)
9
9
  @label = label
10
- @action = action
10
+ @source = nil
11
+ @frozen_string_literal = frozen_string_literal
11
12
 
12
- if action.arity > 0
13
- define_call_times_manual_loop
13
+ if action.kind_of?(String)
14
+ compile_string(action)
15
+ elsif action.arity > 0
16
+ define_call_times_manual_loop(action)
14
17
  else
15
- define_call_times_block
18
+ define_call_times_block(action)
16
19
  end
17
20
  end
18
21
 
19
22
  private
20
23
 
21
- def define_call_times_block
22
- act = @action
24
+ def compile_string(str)
25
+ @source = <<~RUBY
26
+ def call_times(__total)
27
+ __i = 0
28
+ while __i < __total
29
+ #{str}
30
+ __i += 1
31
+ end
32
+ end
33
+ RUBY
34
+ eval_source = "# frozen_string_literal: #{@frozen_string_literal}\n#{@source}"
35
+ m = (class << self; self; end)
36
+ m.class_eval(eval_source)
37
+ end
38
+
39
+ def define_call_times_block(act)
23
40
  define_singleton_method(:call_times) do |times|
24
41
  i = 0
25
42
  while i < times
@@ -29,8 +46,7 @@ module IPS
29
46
  end
30
47
  end
31
48
 
32
- def define_call_times_manual_loop
33
- act = @action
49
+ def define_call_times_manual_loop(act)
34
50
  define_singleton_method(:call_times) do |times|
35
51
  act.call(times)
36
52
  end
data/lib/ips/job.rb CHANGED
@@ -10,24 +10,35 @@ module IPS
10
10
 
11
11
  attr_accessor :warmup, :time
12
12
 
13
- def initialize(time: 5, warmup: 2, out: $stdout)
13
+ def initialize(time: 5, warmup: time * 0.2, summary: true, debug: false, frozen_string_literal: true, out: $stdout)
14
14
  @list = []
15
15
  @time = time
16
16
  @warmup = warmup
17
+ @summary = summary
18
+ @debug = debug
19
+ @frozen_string_literal = frozen_string_literal
17
20
  @timing = {}
18
21
  @out = out
19
22
  end
20
23
 
21
- def report(label, &block)
22
- @list.push Entry.new(label, block)
24
+ def report(label, action = nil, &block)
25
+ raise ArgumentError, "cannot specify both action and block" if action && block
26
+ action ||= block || label
27
+ @list.push Entry.new(label, action, frozen_string_literal: @frozen_string_literal)
23
28
  self
24
29
  end
25
30
 
26
31
  def run
27
- display = Display.new(@list.map(&:label), out: @out)
32
+ display = Display.new(@list.map(&:label), out: @out, debug: @debug)
28
33
  total_ns = ((@warmup + @time) * Timing::NANOSECONDS_PER_SECOND).to_i
29
34
 
30
35
  results = @list.map do |item|
36
+ if @debug
37
+ source = item.source
38
+ @out.puts "--- #{item.label} ---"
39
+ @out.puts source if source
40
+ @out.puts
41
+ end
31
42
  display.start_item(item.label, total_ns)
32
43
  warmup_item(item, display)
33
44
  result = measure_item(item, display)
@@ -35,7 +46,8 @@ module IPS
35
46
  result
36
47
  end
37
48
 
38
- display.summary(results)
49
+ display.summary(results) if @summary
50
+ Result.build(results)
39
51
  end
40
52
 
41
53
  private
@@ -69,6 +81,7 @@ module IPS
69
81
  per_100ms = cycles_per_100ms(warmup_ns, warmup_iter)
70
82
  cycles = per_100ms > MAX_ITERATIONS ? MAX_ITERATIONS : per_100ms
71
83
  @timing[item] = cycles
84
+ @out.puts " cycles: #{cycles}" if @debug
72
85
 
73
86
  target = Timing.add_second(before, @warmup)
74
87
  while Timing.now + Timing::NANOSECONDS_PER_100MS < target
@@ -104,11 +117,18 @@ module IPS
104
117
  times << t0 << t1
105
118
  gc_times << gc0 << gc1
106
119
 
120
+ if @debug
121
+ batch_ips = Timing::NANOSECONDS_PER_SECOND * (cycles.to_f / elapsed_ns)
122
+ gc_ns = gc1 - gc0
123
+ @out.printf " batch: %.3fms ips: %s gc: %.3fms\n",
124
+ elapsed_ns / 1_000_000.0, Display.format_ips(batch_ips), gc_ns / 1_000_000.0
125
+ end
126
+
107
127
  total_ns = t1 - times[0]
108
128
  display.progress(estimate: Timing::NANOSECONDS_PER_SECOND * (iter.to_f / total_ns))
109
129
  end while Timing.now < target
110
130
 
111
- Result.new(item.label, cycles, times, gc_times)
131
+ Result::Entry.new(item.label, cycles, times, gc_times)
112
132
  end
113
133
  end
114
134
  end
data/lib/ips/result.rb CHANGED
@@ -2,90 +2,206 @@
2
2
 
3
3
  module IPS
4
4
  class Result
5
- attr_reader :label, :cycles, :times, :gc_times
6
-
7
- # times: flat array [start0, end0, start1, end1, ...]
8
- # gc_times: flat array [gc_before0, gc_after0, gc_before1, gc_after1, ...]
9
- def initialize(label, cycles, times, gc_times)
10
- @label = label
11
- @cycles = cycles
12
- @times = times
13
- @gc_times = gc_times
5
+ attr_accessor :uuid, :run_label, :ruby_version, :ruby_description, :ruby_executable, :pid, :yjit_enabled, :entries
6
+
7
+ def initialize(entries, uuid:, ruby_version:, ruby_description:, ruby_executable:, pid:, yjit_enabled:)
8
+ @entries = entries
9
+ @uuid = uuid
10
+ @ruby_version = ruby_version
11
+ @ruby_description = ruby_description
12
+ @ruby_executable = ruby_executable
13
+ @pid = pid
14
+ @yjit_enabled = yjit_enabled
15
+ end
16
+
17
+ def self.build(entries)
18
+ new(entries,
19
+ uuid: generate_uuid,
20
+ ruby_version: RUBY_VERSION,
21
+ ruby_description: RUBY_DESCRIPTION,
22
+ ruby_executable: nil,
23
+ pid: Process.pid,
24
+ yjit_enabled: defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled?)
14
25
  end
15
26
 
16
- def sample_count
17
- @times.size / 2
27
+ def self.generate_uuid
28
+ random = Random.urandom(16)
29
+ bytes = random.unpack("NnnnnN")
30
+ bytes[2] = (bytes[2] & 0x0fff) | 0x4000
31
+ bytes[3] = (bytes[3] & 0x3fff) | 0x8000
32
+ "%08x-%04x-%04x-%04x-%04x%08x" % bytes
18
33
  end
19
34
 
20
- def iterations
21
- sample_count * @cycles
35
+ def to_h
36
+ {
37
+ "uuid" => @uuid,
38
+ "ruby_version" => @ruby_version,
39
+ "ruby_description" => @ruby_description,
40
+ "ruby_executable" => @ruby_executable,
41
+ "pid" => @pid,
42
+ "yjit_enabled" => @yjit_enabled,
43
+ "results" => @entries.map(&:to_h)
44
+ }
22
45
  end
23
46
 
24
- # Total wall time from first batch start to last batch end
25
- def total_ns
26
- @times[-1] - @times[0]
47
+ def self.from_h(hash)
48
+ entries = hash["results"].map { |h| Entry.from_h(h) }
49
+ new(entries,
50
+ uuid: hash["uuid"],
51
+ ruby_version: hash["ruby_version"],
52
+ ruby_description: hash["ruby_description"],
53
+ ruby_executable: hash["ruby_executable"],
54
+ pid: hash["pid"],
55
+ yjit_enabled: hash["yjit_enabled"])
27
56
  end
28
57
 
29
- # Time spent actually running batches
30
- def busy_ns
31
- total = 0
32
- i = 0
33
- while i < @times.size
34
- total += @times[i + 1] - @times[i]
35
- i += 2
58
+ def self.save!(path, results)
59
+ require "json"
60
+
61
+ data = if File.exist?(path)
62
+ JSON.parse(File.read(path))
63
+ else
64
+ { "runs" => [] }
65
+ end
66
+
67
+ Array(results).each do |result|
68
+ data["runs"] << result.to_h
36
69
  end
37
- total
70
+
71
+ File.write(path, JSON.pretty_generate(data) + "\n")
38
72
  end
39
73
 
40
- # Time between batches (measurement overhead, GC, scheduling)
41
- def overhead_ns
42
- total_ns - busy_ns
74
+ def self.load(path)
75
+ require "json"
76
+ data = JSON.parse(File.read(path))
77
+ data["runs"].map { |h| from_h(h) }
43
78
  end
44
79
 
45
- # Total GC time during measurement
46
- def gc_ns
47
- total = 0
48
- i = 0
49
- while i < @gc_times.size
50
- total += @gc_times[i + 1] - @gc_times[i]
51
- i += 2
52
- end
53
- total
80
+ def self.compare(results, out: $stdout)
81
+ return if results.size < 2
82
+
83
+ run_labels = distinguish(results)
84
+ one_entry = results.all? { |r| r.entries.size == 1 }
85
+
86
+ pairs = results.flat_map { |r|
87
+ r.entries.map { |e|
88
+ label = one_entry ? run_labels[r.uuid] : "#{e.label} (#{run_labels[r.uuid]})"
89
+ [label, e]
90
+ }
91
+ }
92
+
93
+ Display.compare(pairs, out: out)
54
94
  end
55
95
 
56
- def ips
57
- Timing::NANOSECONDS_PER_SECOND * (iterations.to_f / total_ns)
96
+ def self.distinguish(results)
97
+ [:run_label, :ruby_version, :ruby_description, :ruby_executable, :uuid].each do |field|
98
+ values = results.map { |r| r.send(field) }
99
+ next if values.any?(&:nil?)
100
+ if values.uniq.size == results.size
101
+ return results.each_with_object({}) { |r, h| h[r.uuid] = r.send(field).to_s }
102
+ end
103
+ end
58
104
  end
59
105
 
60
- # Per-batch IPS samples for error estimation
61
- def samples
62
- @samples ||= begin
63
- s = Array.new(sample_count)
106
+ class Entry
107
+ attr_reader :label, :cycles, :times, :gc_times
108
+
109
+ # times: flat array [start0, end0, start1, end1, ...]
110
+ # gc_times: flat array [gc_before0, gc_after0, gc_before1, gc_after1, ...]
111
+ def initialize(label, cycles, times, gc_times)
112
+ @label = label
113
+ @cycles = cycles
114
+ @times = times
115
+ @gc_times = gc_times
116
+ end
117
+
118
+ def sample_count
119
+ @times.size / 2
120
+ end
121
+
122
+ def iterations
123
+ sample_count * @cycles
124
+ end
125
+
126
+ # Total wall time from first batch start to last batch end
127
+ def total_ns
128
+ @times[-1] - @times[0]
129
+ end
130
+
131
+ # Time spent actually running batches
132
+ def busy_ns
133
+ total = 0
64
134
  i = 0
65
135
  while i < @times.size
66
- s[i / 2] = Timing::NANOSECONDS_PER_SECOND * (@cycles.to_f / (@times[i + 1] - @times[i]))
136
+ total += @times[i + 1] - @times[i]
67
137
  i += 2
68
138
  end
69
- s
139
+ total
70
140
  end
71
- end
72
141
 
73
- def sample_mean
74
- @sample_mean ||= samples.sum / samples.size
75
- end
142
+ # Time between batches (measurement overhead, GC, scheduling)
143
+ def overhead_ns
144
+ total_ns - busy_ns
145
+ end
76
146
 
77
- def stddev
78
- variance = samples.sum { |s| (s - sample_mean) ** 2 } / samples.size
79
- Math.sqrt(variance)
80
- end
147
+ # Total GC time during measurement
148
+ def gc_ns
149
+ total = 0
150
+ i = 0
151
+ while i < @gc_times.size
152
+ total += @gc_times[i + 1] - @gc_times[i]
153
+ i += 2
154
+ end
155
+ total
156
+ end
81
157
 
82
- def error_pct
83
- (stddev / ips) * 100.0
84
- end
158
+ def ips
159
+ Timing::NANOSECONDS_PER_SECOND * (iterations.to_f / total_ns)
160
+ end
161
+
162
+ # Per-batch IPS samples for error estimation
163
+ def samples
164
+ @samples ||= begin
165
+ s = Array.new(sample_count)
166
+ i = 0
167
+ while i < @times.size
168
+ s[i / 2] = Timing::NANOSECONDS_PER_SECOND * (@cycles.to_f / (@times[i + 1] - @times[i]))
169
+ i += 2
170
+ end
171
+ s
172
+ end
173
+ end
85
174
 
86
- def gc_pct
87
- total = total_ns
88
- total > 0 ? (gc_ns.to_f / total) * 100.0 : 0.0
175
+ def sample_mean
176
+ @sample_mean ||= samples.sum / samples.size
177
+ end
178
+
179
+ def stddev
180
+ variance = samples.sum { |s| (s - sample_mean) ** 2 } / samples.size
181
+ Math.sqrt(variance)
182
+ end
183
+
184
+ def error_pct
185
+ (stddev / ips) * 100.0
186
+ end
187
+
188
+ def gc_pct
189
+ total = total_ns
190
+ total > 0 ? (gc_ns.to_f / total) * 100.0 : 0.0
191
+ end
192
+
193
+ def to_h
194
+ {
195
+ "label" => @label,
196
+ "cycles" => @cycles,
197
+ "times" => @times,
198
+ "gc_times" => @gc_times
199
+ }
200
+ end
201
+
202
+ def self.from_h(hash)
203
+ new(hash["label"], hash["cycles"], hash["times"], hash["gc_times"])
204
+ end
89
205
  end
90
206
  end
91
207
  end
data/lib/ips/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module IPS
4
- VERSION = "0.1.0"
4
+ VERSION = "0.2.0"
5
5
  end
data/lib/ips.rb CHANGED
@@ -6,11 +6,23 @@ require "ips/display"
6
6
  require "ips/job"
7
7
 
8
8
  module IPS
9
- def self.run(time: 5, warmup: 2, out: $stdout)
10
- job = Job.new(time: time, warmup: warmup, out: out)
9
+ def self.run(time: 5, warmup: time * 0.2, summary: true, debug: false, frozen_string_literal: true, out: $stdout)
10
+ job = Job.new(time: time, warmup: warmup, summary: summary, debug: debug, frozen_string_literal: frozen_string_literal, out: out)
11
11
  yield job
12
12
  job.run
13
13
  end
14
+
15
+ def self.report(label = nil, action = nil, &block)
16
+ action ||= block || label || raise(ArgumentError, "no action or block given")
17
+ label ||= action.is_a?(String) ? action : "block"
18
+ run do |x|
19
+ x.report(label, action)
20
+ end
21
+ end
22
+
23
+ class << self
24
+ alias_method :ips, :run
25
+ end
14
26
  end
15
27
 
16
28
  Ips = IPS
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ips
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - John Hawthorn
@@ -13,7 +13,8 @@ description: A Ruby benchmarking tool that measures iterations per second with a
13
13
  warmup, statistical error estimation, GC reporting, and comparison summaries.
14
14
  email:
15
15
  - john@hawthorn.email
16
- executables: []
16
+ executables:
17
+ - ips
17
18
  extensions: []
18
19
  extra_rdoc_files: []
19
20
  files:
@@ -21,6 +22,8 @@ files:
21
22
  - LICENSE.txt
22
23
  - README.md
23
24
  - Rakefile
25
+ - demo.gif
26
+ - demo.tape
24
27
  - examples/basic.rb
25
28
  - examples/gc_pressure.rb
26
29
  - examples/hash_vs_struct.rb
@@ -28,6 +31,7 @@ files:
28
31
  - examples/outlier.rb
29
32
  - examples/singleton_class.rb
30
33
  - examples/string_matching.rb
34
+ - exe/ips
31
35
  - lib/ips.rb
32
36
  - lib/ips/display.rb
33
37
  - lib/ips/job.rb
@@ -55,7 +59,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
55
59
  - !ruby/object:Gem::Version
56
60
  version: '0'
57
61
  requirements: []
58
- rubygems_version: 4.0.3
62
+ rubygems_version: 3.6.9
59
63
  specification_version: 4
60
64
  summary: Iterations per second benchmarking with statistical analysis
61
65
  test_files: []