winr 1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: b5b9f7f96814ddc74f0cf997f96355ca43be031355fa06c58de54421ec395ee9
+  data.tar.gz: 073a1747bac59f25a0eaa161fdff7ef491eca92eb917ef36094bd6cf12077f35
+SHA512:
+  metadata.gz: 99584b189c83cc9bac6558a6e379b70f074c59f9e73f95425096d0ad557ef5b239eb2feaad9ad55c90e60b99ae03c9f23d523fb73edc0d0d37a2af2286ae6176
+  data.tar.gz: 4ee9ad210345e670ed3bcae76a5d08ced2dd47a1c1a61fb417d457960c73fc17e89b68a63d6f02ed71a5949b03339b6d62ac812801024b4f7290baa8fcf6528c

data/Gemfile

@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+
+gemspec

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Steve Shreeve
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,152 @@
+# winr
+
+A quick and lightweight benchmarking tool for Ruby
+
+## Goals
+
+1. Provide a simple way to benchmark code
+2. Easy to configure and compare results
+
+## Overview
+
+There are several tools for benchmarking Ruby code. Unfortunately, it's hard
+to remember the way to set up the benchmarks, what format to use to represent
+what to test, and how to perform the testing. One of the most popular tools
+used for benchmarking Ruby code is `benchmark-driver`, which was used heavily
+by the Ruby 3x3 team. It uses three levels of abstraction and is powerful,
+but there is a lot of inconsistency in naming conventions and using the tool.
+
+The idea of `winr` is to offer an easy to configure benchmarking tool that is
+easy to use. There are three levels of abstraction:
+
+* environments - An environment is an optional top-level container for running
+  code. It can represent different versions Ruby, or different setup options
+  that will be shared by any nested contexts or tasks.
+* contexts - A context represents an optional mid-level container for running
+  code. Any context configuration is shared by any contained tasks.
+* tasks - A task represents a unit of work to be tested, within its optional
+  containing context and optional containing environment.
+
+Within an environment, context, or task, the following can be defined:
+
+* name - The name of the element, to use when displaying output
+* begin - Code that should be run before any contained elements
+* end - Code that should run after any contained elements
+* script - Code that is contained within a task
+* loops - The number of times that a task's script should be executed
+* warmup - An optional top-level setting that is used when the number of
+  loops has not been defined. This value is the number of seconds to run
+  the task before determining the number of loops that have run. The
+  default value is 3 seconds, meaning that the number of loops is not
+  defined, the task will be allowed to run for 3 seconds and the number
+  of iterations during this time will be used.
+
+## Examples
+
+Configuration information is stored in a Ruby file, like the following:
+
+```ruby
+$ cat test/sum.rb
+
+{
+  environments: [
+    {
+      name: "Shiny new Ruby",
+      command: "/opt/ruby-3.2.0/bin/ruby",
+    },
+    {
+      name: "Old trusted Ruby",
+      command: "/opt/ruby-2.7.6/bin/ruby",
+    },
+  ],
+  contexts: [
+    {
+      begin: <<~"|",
+        require "digest/md5"
+
+        max = 1e5.to_i
+      |
+    },
+  ],
+  tasks: [
+    {
+      name: "array splat",
+      script: <<~"|",
+        ary = [*1..max]
+        sum = ary.sum
+      |
+    },
+    {
+      name: "times",
+      script: <<~"|",
+        sum = 0
+        max.to_i.times {|n| sum += n }
+      |
+    },
+  ],
+  warmup: 3,
+}
+```
+
+The benchmark is run as follows:
+
+```
+$ winr test/sum.rb
+
+┌──────────────────┬───────────────────────────────────────┐
+│ Shiny new Ruby   │                Results                │
+├──────────────────┼───────────┬─────────────┬─────────────┤
+│ array splat      │   2.99  s │ 768.19  i/s │   1.30 ms/i │
+│ times            │   7.86  s │ 292.33  i/s │   3.42 ms/i │
+└──────────────────┴───────────┴─────────────┴─────────────┘
+
+┌──────────────────┬───────────────────────────────────────┐
+│ Old trusted Ruby │                Results                │
+├──────────────────┼───────────┬─────────────┬─────────────┤
+│ array splat      │   2.96  s │ 777.46  i/s │   1.29 ms/i │
+│ times            │   7.73  s │ 297.19  i/s │   3.36 ms/i │
+└──────────────────┴───────────┴─────────────┴─────────────┘
+
+┌──────────────────┬─────────────────────────────────────┐
+│ Rank             │             Performance             │
+├──────────────────┼─────────────┬──────────────┬────────┤
+│ array splat      │ 777.46  i/s │   fastest    │ 2/1/1  │
+│ array splat      │ 768.19  i/s │ 1.01x slower │ 1/1/1  │
+│ times            │ 297.19  i/s │ 2.62x slower │ 2/1/2  │
+│ times            │ 292.33  i/s │ 2.66x slower │ 1/1/2  │
+└──────────────────┴─────────────┴──────────────┴────────┘
+```
+
+## Install
+
+Install via `rubygems` with:
+
+```
+gem install winr
+```
+
+## Options
+
+```
+$ winr -h
+
+usage: winr [options] <dir ...>
+    -c, --[no-]color                 Enable color output (default is true)
+    -d, --debug                      Enable debug mode
+    -h, --help                       Show help and command usage
+    -i, --iterations <count>         Force the number of iterations for each task
+    -r, --reverse                    Show contexts vertically and tasks horizontally
+    -s, --stats                      Comma-separated list of stats (loops, time, ips, spi)
+    -v, --verbose                    Show command, version details, and markdown backticks
+
+Available statistics:
+
+  ips      iterations per second
+  loops    number of iterations
+  spi      seconds for iteration
+  time     time to run all iterations
+```
+
+## License
+
+This software is licensed under terms of the MIT License.

data/bin/winr

@@ -0,0 +1,314 @@
+#!/usr/bin/env ruby
+
+# ============================================================================
+# winr - A quick and lightweight benchmarking tool for Ruby
+#
+# Author: Steve Shreeve (steve.shreeve@gmail.com)
+#   Date: Feb 14, 2023
+# ============================================================================
+# GOALS:
+# 1. Provide a simple way to benchmark code
+# 2. Easy to configure and compare results
+# 3. Accurately measure times, see http://bit.ly/3ltE7MP
+#
+# TODO:
+# 1. Enable YAML config files
+# ============================================================================
+
+trap("INT" ) { abort "\n" }
+
+require "erb"
+require "optparse"
+require "shellwords"
+require "tempfile"
+
+OptionParser.new.instance_eval do
+  @banner  = "usage: #{program_name} [options] <dir ...>"
+
+  on "-c"               , "--[no-]color", "Enable color output (default is true)", TrueClass
+  on "-d"               , "--debug"     , "Enable debug mode", TrueClass
+  on "-h"               , "--help"      , "Show help and command usage" do Kernel.abort to_s; end
+  on "-i <count>"       , "--iterations", "Force the number of iterations for each task", Integer
+  on "-r"               , "--reverse"   , "Show contexts vertically and tasks horizontally", TrueClass
+  on "-s <time,ips,spi>", "--stats "    , "Comma-separated list of stats (loops, time, ips, spi)"
+  on "-v"               , "--verbose"   , "Show command, version details, and markdown backticks", TrueClass
+
+  separator <<~"end"
+
+      Available statistics:
+
+        ips      iterations per second
+        loops    number of iterations
+        spi      seconds for iteration
+        time     time to run all iterations
+  end
+
+  self
+end.parse!(into: opts={}) rescue abort($!.message)
+
+# option munging
+ansi =  opts[:color]; ansi = true if ansi.nil?
+hack =  opts[:debug]
+hush = !opts[:verbose]
+runs =  opts[:iterations]; abort "invalid number of runs" if runs && runs < 1
+show =  opts[:stats] || "time,ips,spi"
+show =  show.downcase.scan(/[a-z]+/i).uniq & %w[ ips loops spi time ]
+swap = !opts[:reverse]
+
+# option errors
+show.empty? and abort "invalid list of statistics #{opts[:stats].inspect}"
+
+# ==[ Define some constants, ansi codes, and make hashes more flexible ]==
+
+Infinity = 1.0 / 0
+Overflow = "\n\nERROR: numeric overflow"
+
+module Ansi
+  refine String do
+    $ansi = <<~"".scan(/(\d+)(?:\/(\d+))?=(\w+)/).inject({}) do |ansi, (code, undo, name)|
+      0=reset 1/22=bold 2/22=dim 3/23=italic 4/24=under 5/25=blink 7/27=inverse 9/29=strike
+      30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white 39=default
+      40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white 49=default
+
+      ansi[name +                       "_off"     ] = undo if undo
+      ansi[name + (code[0,2] =~ /4\d/ ? "_bg" : "")] = code
+      ansi
+    end
+    def ansi (*list); list.map {|code| "\e[#{$ansi[code.to_s] || 0}m"}.join + self; end
+    def ansi!(*list); ansi(*list) + "\e[0m"; end
+  end
+end
+
+using Ansi
+
+module FlexHash
+  refine Hash do
+    alias_method :default_lookup, :[]
+
+    def [](key, miss=nil) # method_missing calls this with key as a symbol
+      key?(key) and return default_lookup(key) || miss
+
+      ary = key.to_s.split(/(?:[.\/\[]|\][.\/]?)/)
+      val = ary.inject(self) do |obj, sub|
+        if    obj == self        then default_lookup(sub.to_sym)
+        elsif obj == nil         then break
+        elsif sub =~ /\A-?\d*\z/ then obj[sub.to_i]
+        else                          obj[sub.to_sym]
+        end
+      end or miss
+    end
+  end
+end
+
+using FlexHash
+
+class Hash
+  def method_missing(name, *args)
+    name =~ /=$/ ? send(:[]=, $`.to_sym, *args) : send(:[], name, *args)
+  end
+end
+
+# ==[ Templates ]==
+
+def compile(task, path)
+  <<~"|".strip
+    #{ task.begin }
+
+    # number of loops requested
+    __winr_iters = #{ task.loops.to_i }
+
+    # calculate loops if not supplied
+    if __winr_iters == 0
+      __winr_until = __winr_timer + #{ $config.warmup(3) }
+      while __winr_timer < __winr_until
+        #{ task.script&.strip }
+        __winr_iters += 1
+      end
+    end
+
+    # calculate time wasted on loop overhead
+    __winr_waste = 0
+    __winr_loops = 0
+    __winr_begin = __winr_timer
+    while __winr_loops < __winr_iters
+      __winr_loops += 1
+    end
+    __winr_waste = __winr_timer - __winr_begin
+
+    # calculate time spent running our task
+    __winr_loops = 0
+    __winr_begin = __winr_timer
+    while __winr_loops < __winr_iters
+      #{ task.script&.strip }
+      __winr_loops += 1
+    end
+    __winr_delay = __winr_timer - __winr_begin
+
+    File.write(#{ path.inspect }, [__winr_loops, __winr_delay - __winr_waste].inspect)
+
+    #{ task.end }
+  |
+end
+
+# ==[ Helpers ]==
+
+def boxlines(main, cols, runs=1)
+  [ "┌┬──┐",
+    "├┼┬─┤",
+    "└┴┴─┘" ]
+  .map do |str|
+    list = [main, *(cols * runs)]
+    list.map.with_index do |col, i|
+      chr = str[i < 2 ? i : (i - 1) % cols.size == 0 ? 1 : 2]
+      chr + str[3] * (col + 2)
+    end.join + str[-1]
+  end
+end
+
+def execute(command, path)
+  IO.popen(["ruby", path].join(" "), &:read)
+  $?.success? or raise
+  eval(File.read(path))
+end
+
+def scale(show, unit)
+  slot = 3
+  span = ["G", "M", "K", " ", "m", "µ", "p"]
+  [0, Infinity].include?(show) and abort Overflow
+  show *= 1000.0 and slot += 1 while show > 0 && show < 1.0
+  show /= 1000.0 and slot -= 1 while show >= 1000.0
+  slot.between?(0, 6) or abort Overflow
+  "%6.2f %s%s" % [show, span[slot], unit]
+end
+
+def stats(list, scope=nil)
+  list.map do |item|
+    pair = case item
+    when "loops" then ["runs"     , "times"]
+    when "time"  then ["time"     , "s"    ]
+    when "ips"   then ["runs/time", "i/s"  ]
+    when "spi"   then ["time/runs", "s/i"  ]
+    else abort "unknown statistic #{item.inspect}"
+    end
+    scope ? eval(pair[0], scope) : pair[1]
+  end
+end
+
+def write(file, data)
+  file.puts(data)
+  file.close
+  yield file.path
+end
+
+# ==[ Workflow ]==
+
+# read the winr script
+winr = ARGV.first or abort "missing winr script"
+tmpl = ERB.new(DATA.read)
+
+# grok the config
+$config = eval(File.read(winr))
+es = $config.environments || [{}]
+cs = $config.contexts     || [{}]
+ts = $config.tasks        || [{}]
+
+# box drawing
+cols = stats(show)
+full = cols.map(&:size).sum + cols.size * 11 - 3
+wide = [*es.map {|e| e.name("").size}, *ts.map {|t| t.name("").size}].max
+rank = []
+
+# row: top, middle, bottom
+rt, rm, rb = boxlines(wide, cols.map {|e| e.size + 8 }, (swap ? cs : ts).size)
+
+# begin output
+puts "```", [$0, *ARGV].shelljoin, "" unless hush
+
+# loop over environment(s)
+es.each_with_index do |e, ei|
+  puts IO.popen(["ruby", "-v"].join(" "), &:read) unless hush
+  puts rt
+
+  command = ["/usr/bin/env ruby"]
+
+  # loop over context(s) and task(s)
+  ys, xs = swap ? [ts, cs] : [cs, ts]
+
+  # row: content, header
+  rc = "Task" # or "Context"
+  rh = "│ %-*.*s │" % [wide, wide, e.name(es.size > 1 ? "Env ##{ei + 1}" : rc)]
+  rh = xs.inject(rh) {|s, x| s << " %-*.*s │" % [full, full, x.name("Results").center(full)] }
+  puts rh, rm
+
+  ys.each_with_index do |y, yi|
+    print "│ %-*.*s │" % [wide, wide, y.name("Results")]
+    xs.each_with_index do |x, xi|
+    t, ti, c, ci = swap ? [y, yi, x, xi] : [x, xi, y, yi]
+      delay = Tempfile.open(['winr-', '.rb']) do |file|
+        t.loops = runs if runs
+        code = tmpl.result(binding).rstrip + "\n"
+        write(file, code) do |path|
+          runs, time = execute(command, path)
+          t.loops ||= runs
+          vals = stats(show, binding)
+          rank << [runs/time, ei, ci, ti]
+          print vals.zip(cols).map {|pair| " %s │" % scale(*pair) }.join
+        end
+        puts "", code, "=" * 78 if hack
+      end
+    end
+    print "\n"
+  end
+  puts rb, ""
+end
+
+# show the comparison
+rank.sort! {|a, b| b[0] <=> a[0] }
+fast = rank.first[0]
+slow = rank.last[0]
+pict = "%.2fx slower"
+room = (pict % [fast / slow]).size
+cols = [11, room, 6]
+cols.pop if (es.size == 1) && (cs.size == 1)
+full = cols.sum + (cols.size - 1) * 3
+rt, rm, rb = boxlines(wide, cols)
+rh = "│ %-*.*s │" % [wide, wide, "Rank"]
+rh << " %-*.*s │" % [full, full, "Performance".center(full)]
+
+puts rt, rh, rm
+flip = cs.size > 1 && ts.size == 1
+rank.each do |ips, ei, ci, ti|
+  name = (flip ? cs[ci] : ts[ti]).name
+  print "│ %-*.*s │ %s │ " % [wide, wide, name, scale(ips, "i/s")]
+  if ips.round(2) == fast.round(2)
+    text = "fastest".center(room)
+    print ansi ? text.ansi!(:green, :bold) : text
+  else
+    print  "%*.*s" % [room, room, pict % [fast/ips]]
+  end
+  print " │ %-6s" % ([ei+1,ci+1,ti+1] * "/") if cols.size > 2
+  print " │\n"
+end
+puts rb
+puts "```" unless hush
+
+__END__
+
+# ============================================================================
+# Environment <%= ei + 1 %>: <%= e.name %>
+#     Context <%= ci + 1 %>: <%= c.name %>
+#        Task <%= ti + 1 %>: <%= t.name %>
+# ============================================================================
+
+trap("INT") { exit }
+
+# def __winr_timer; Process.clock_gettime(Process::CLOCK_MONOTONIC); end
+def __winr_timer; Time.now.to_f; end
+
+<%= e.begin %>
+<%= c.begin %>
+
+<%= compile(t, file.path) %>
+
+<%= c.end %>
+<%= e.end %>

data/test/sum.rb

@@ -0,0 +1,38 @@
+{
+  environments: [
+    {
+      name: "Shiny new Ruby",
+      command: "/opt/ruby-3.2.0/bin/ruby",
+    },
+    {
+      name: "Old trusted Ruby",
+      command: "/opt/ruby-2.7.6/bin/ruby",
+    },
+  ],
+  contexts: [
+    {
+      begin: <<~"|",
+        require "digest/md5"
+
+        max = 1e5.to_i
+      |
+    },
+  ],
+  tasks: [
+    {
+      name: "array splat",
+      script: <<~"|",
+        ary = [*1..max]
+        sum = ary.sum
+      |
+    },
+    {
+      name: "times",
+      script: <<~"|",
+        sum = 0
+        max.to_i.times {|n| sum += n }
+      |
+    },
+  ],
+  warmup: 3,
+}

data/winr.gemspec

@@ -0,0 +1,14 @@
+# encoding: utf-8
+
+Gem::Specification.new do |s|
+  s.name        = "winr"
+  s.version     = "1.0"
+  s.author      = "Steve Shreeve"
+  s.email       = "steve.shreeve@gmail.com"
+  s.summary     =
+  s.description = "A quick and lightweight benchmarking tool for Ruby"
+  s.homepage    = "https://github.com/shreeve/winr"
+  s.license     = "MIT"
+  s.files       = `git ls-files`.split("\n") - %w[.gitignore]
+  s.executables = `cd bin && git ls-files .`.split("\n")
+end

metadata

@@ -0,0 +1,49 @@
+--- !ruby/object:Gem::Specification
+name: winr
+version: !ruby/object:Gem::Version
+  version: '1.0'
+platform: ruby
+authors:
+- Steve Shreeve
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2023-02-14 00:00:00.000000000 Z
+dependencies: []
+description: A quick and lightweight benchmarking tool for Ruby
+email: steve.shreeve@gmail.com
+executables:
+- winr
+extensions: []
+extra_rdoc_files: []
+files:
+- Gemfile
+- LICENSE
+- README.md
+- bin/winr
+- test/sum.rb
+- winr.gemspec
+homepage: https://github.com/shreeve/winr
+licenses:
+- MIT
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.4.6
+signing_key:
+specification_version: 4
+summary: A quick and lightweight benchmarking tool for Ruby
+test_files: []