benchmark-swap 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 300c84291632dc4b0f85e22e83dde631f6baffeaee7d61911adbb62ecceaa288
+  data.tar.gz: 7c5fd3e887f3af1d60012ae3f62b6be0f04e9f425ebb4e69fdd2cc663ad6f1fd
+SHA512:
+  metadata.gz: 12661ba83db0e3e3f861768b2f90cf096c3acd273df8c9099b7870def338d8f0698cc0cb5f9db37dcbe02a0f6052e365b592513c1bba8d19e1e21ee1aed3cd86
+  data.tar.gz: 29ee23f65939cf66613bc324c3acb8048c9093b59889d9652208340d2f9acbaad5691865df2ee43474a332cbb13f0cf3b447363c53f88244a327186ca2d33d3c

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+# Changelog
+
+## 0.1.0
+
+- First release.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Mehmet Emin INAC
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,113 @@
+# Benchmark::Swap
+
+Compare two implementations of a method with benchmark-ips, without lifting the method out of its call chain into two standalone lambdas. That lift is slow to do and easy to get wrong when the method sits deep inside real code. `Benchmark::Swap` keeps the method where it lives and swaps its body in place.
+
+Built for exploring performance changes inside a large Rails app from the Rails console.
+
+## Installation
+
+Add this line to your Gemfile:
+
+```ruby
+gem "benchmark-swap"
+```
+
+Then run `bundle install`.
+
+Requires Ruby 3.2 or newer. Depends on `benchmark-ips`.
+
+## Usage
+
+Keep the original method. Add a second implementation next to it, with the same name plus a `_perf` suffix:
+
+```ruby
+class Pow
+  def initialize(number)
+    @number = number
+  end
+
+  def pow
+    do_pow
+  end
+
+  private
+
+  def do_pow
+    @number**2
+  end
+
+  def do_pow_perf
+    @number * @number
+  end
+end
+
+Benchmark.swap { Pow.new(2).pow }
+```
+
+`Benchmark.swap` is a shortcut for `Benchmark::Swap.test`. Both take the same options.
+
+Sample output:
+
+```
+benchmark-swap: swapping 1 method
+  Pow#do_pow -> do_pow_perf
+
+ruby 3.3.11 (2026-03-26 revision 1f2d15125a) [arm64-darwin24]
+Warming up --------------------------------------
+            original   848.389k i/100ms
+Calculating -------------------------------------
+            original      8.480M (± 0.3%) i/s  (117.93 ns/i) -      8.484M in   1.000511s
+ruby 3.3.11 (2026-03-26 revision 1f2d15125a) [arm64-darwin24]
+Warming up --------------------------------------
+             swapped   905.345k i/100ms
+Calculating -------------------------------------
+             swapped      9.083M (± 0.2%) i/s  (110.09 ns/i) -      9.959M in   1.096372s
+
+Comparison:
+original:  8479556.9 i/s
+ swapped:  9083408.7 i/s - 1.07x  faster
+```
+
+Two "Warming up / Calculating" blocks and one "Comparison:" block are expected. Each side gets its own benchmark-ips run, then the two are compared with the original as the baseline.
+
+## How it works
+
+1. **Discovery.** The block runs once under a Ruby `TracePoint` on the `:call` event. It collects every method that was actually called and has a twin with the suffix defined on the same owner. Only calls from the current thread count. A method whose twin lives on a different class or module (for example the original on a parent class, the twin on the child) is skipped. Frozen owners are skipped too.
+2. **Verification.** The block runs once per side, and the two results are compared with `==`. A mismatch prints a warning, but the benchmark still runs: this step never raises. If both sides raise the same error class and message, that also counts as a match. Turn it off with `verify: false`.
+3. **Benchmark.** Each side gets its own benchmark-ips run, then `Benchmark.compare` reports the two with the original as the baseline.
+
+All discovered twins are swapped together. There is no one-at-a-time mode in this version.
+
+The swap itself copies the twin's `UnboundMethod` body onto the original method name with `define_method`, keeping the original body aside, and puts it back afterwards. It does not delegate through a wrapper method, so both sides run at the same call depth and the numbers describe the method bodies, not the swap. A spec proves this by comparing `caller.size` on both sides. Method visibility (public, protected, private) is preserved and restored. Originals are restored even when the block raises. When a module is prepended to the owner, the swap targets the definition the owner itself holds, so a prepended override that calls `super` keeps working and its own body is left untouched.
+
+## Options
+
+| Option | Default | Description |
+| --- | --- | --- |
+| `suffix:` | `"_perf"` | Suffix used to find the twin method |
+| `verify:` | `true` | Compare both sides once before benchmarking |
+| `output:` | `$stdout` | Where the gem's own report lines go |
+| anything else | | Passed to benchmark-ips, for example `time:` and `warmup:` |
+
+The call returns a `Runner::Result` struct with `candidates`, `verification`, `original`, and `swapped`. `original` and `swapped` are benchmark-ips `Report` objects. It returns `nil` when no twin was called.
+
+## Caveats
+
+- The block runs several times: once for discovery, twice for verification, then many times per benchmark side. Side effects add up, so build fresh objects inside the block instead of reusing a memoised one.
+- If the block raises during discovery, the error propagates.
+- Only methods defined in Ruby are found. The TracePoint `:call` event does not fire for methods implemented in C, so an `attr_reader` original, or a method from a C extension, is skipped even when a twin exists next to it. Write the original in Ruby if you want to measure it.
+- While a side is being measured, the swap is visible to the whole process, not just the calling thread. Discovery only watches the current thread, but the swapped body is what every thread sees. So do not run this on a process that is serving real traffic.
+- Remember to delete the `_perf` method before you commit.
+
+## Development
+
+```
+bundle install
+bundle exec rspec
+bundle exec rubocop
+ruby examples/pow.rb
+```
+
+## License
+
+MIT.

data/lib/benchmark/swap/candidate.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module Benchmark
+  module Swap
+    # A method that has a suffixed twin, plus the owner both are defined on.
+    Candidate = Struct.new(:owner, :name, :perf_name) do
+      def label
+        if owner.singleton_class?
+          "#{owner.attached_object}.#{name}"
+        else
+          "#{owner}##{name}"
+        end
+      rescue TypeError
+        "#{owner}##{name}"
+      end
+
+      def to_s
+        "#{label} -> #{perf_name}"
+      end
+    end
+  end
+end

data/lib/benchmark/swap/discovery.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+module Benchmark
+  module Swap
+    # Runs the block once under a TracePoint and keeps every method call that
+    # has a twin with the configured suffix on the same owner.
+    class Discovery
+      def initialize(suffix:)
+        @suffix = suffix
+      end
+
+      def call(&block)
+        seen = Set.new
+        found = []
+        thread = Thread.current
+
+        trace = TracePoint.new(:call) do |trace_point|
+          next unless Thread.current.equal?(thread)
+
+          owner = trace_point.defined_class
+          name = trace_point.method_id
+          key = [owner.object_id, name]
+
+          next unless seen.add?(key)
+
+          candidate = candidate_for(owner, name)
+
+          found << candidate if candidate
+        end
+
+        trace.enable(&block)
+
+        found
+      end
+
+      private
+
+      def candidate_for(owner, name)
+        return unless owner.is_a?(Module)
+        return if owner.frozen?
+        return if name.nil? || name.to_s.end_with?(@suffix)
+
+        perf_name = :"#{name}#{@suffix}"
+        return unless own_method?(owner, name) && own_method?(owner, perf_name)
+
+        Candidate.new(owner, name, perf_name)
+      end
+
+      def own_method?(owner, name)
+        owner.public_method_defined?(name, false) ||
+          owner.private_method_defined?(name, false) ||
+          owner.protected_method_defined?(name, false)
+      end
+    end
+  end
+end

data/lib/benchmark/swap/runner.rb

@@ -0,0 +1,93 @@
+# frozen_string_literal: true
+
+module Benchmark
+  module Swap
+    # Ties the three steps together:
+    #   - find the twins
+    #   - check both sides agree
+    #   - benchmark each side
+    #
+    # Each side gets its own benchmark-ips run, because a single run executes
+    # every report block after the setup block returns, which leaves no point
+    # to turn the swap on between them.
+    class Runner
+      ORIGINAL_LABEL = "original"
+      SWAPPED_LABEL = "swapped"
+      PREFIX = "benchmark-swap:"
+
+      Result = Struct.new(:candidates, :verification, :original, :swapped)
+
+      def initialize(suffix:, verify:, output:, ips_options:)
+        @suffix = suffix
+        @verify = verify
+        @output = output
+        @ips_options = ips_options
+      end
+
+      def call(&block)
+        candidates = Discovery.new(suffix: @suffix).call(&block)
+
+        return nothing_found if candidates.empty?
+
+        announce(candidates)
+
+        swapper = Swapper.new(candidates)
+        verification = verify(swapper, &block)
+        original, swapped = benchmark(swapper, &block)
+
+        Result.new(candidates, verification, original, swapped)
+      end
+
+      private
+
+      def nothing_found
+        say "#{PREFIX} no *#{@suffix} twin was called by this block, nothing to compare."
+
+        nil
+      end
+
+      def announce(candidates)
+        count = candidates.size
+        say "#{PREFIX} swapping #{count} method#{"s" if count > 1}"
+        candidates.each { |candidate| say "  #{candidate}" }
+        say ""
+      end
+
+      def verify(swapper, &block)
+        return unless @verify
+
+        outcome = Verifier.new.call(swapper, &block)
+        return outcome if outcome.match?
+
+        say "#{PREFIX} WARNING both sides returned a different result"
+        say "  #{ORIGINAL_LABEL}: #{outcome.original}"
+        say "  #{SWAPPED_LABEL}: #{outcome.swapped}"
+        say ""
+
+        outcome
+      end
+
+      def benchmark(swapper, &block)
+        original = ips(ORIGINAL_LABEL, &block)
+        swapped = swapper.swapped { ips(SWAPPED_LABEL, &block) }
+
+        Benchmark.compare(*original.entries, *swapped.entries, order: :baseline)
+
+        [original, swapped]
+      end
+
+      def ips(label, &block)
+        options = @ips_options
+
+        Benchmark.ips do |job|
+          job.config(**options) unless options.empty?
+          job.report(label, &block)
+        end
+      end
+
+      def say(line)
+        @output.puts(line)
+      end
+    end
+  end
+end

data/lib/benchmark/swap/swapper.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+module Benchmark
+  module Swap
+    # Copies each twin's body onto the original method name, and puts the
+    # original body back afterwards.
+    #
+    # define_method with an UnboundMethod copies the body instead of
+    # delegating to it, so both sides of the benchmark run at the same call
+    # depth and the numbers describe the bodies, not the swap.
+    class Swapper
+      def initialize(candidates)
+        @candidates = candidates
+        @saved = []
+      end
+
+      def swapped
+        enable
+        yield
+      ensure
+        disable
+      end
+
+      def enable
+        @candidates.each do |candidate|
+          owner = candidate.owner
+          original = own_instance_method(owner, candidate.name)
+          visibility = visibility_of(owner, candidate.name)
+
+          define(owner, candidate.name, own_instance_method(owner, candidate.perf_name), visibility)
+          @saved << [owner, candidate.name, original, visibility]
+        end
+      end
+
+      def disable
+        while (entry = @saved.pop)
+          define(*entry)
+        end
+      end
+
+      private
+
+      # A module prepended to the owner shadows the owner's own definition, so
+      # Module#instance_method can hand back a body the owner does not own.
+      # Saving that body and writing it back on restore would overwrite the
+      # real one for good, so walk down to the definition the owner itself
+      # holds.
+      def own_instance_method(owner, name)
+        method = owner.instance_method(name)
+        method = method.super_method until method.nil? || method.owner.equal?(owner)
+
+        method || raise(ArgumentError, "#{owner} does not define #{name}, it only inherits it")
+      end
+
+      def define(owner, name, body, visibility)
+        owner.send(:define_method, name, body)
+        owner.send(visibility, name)
+      end
+
+      def visibility_of(owner, name)
+        if owner.private_method_defined?(name, false)
+          :private
+        elsif owner.protected_method_defined?(name, false)
+          :protected
+        else
+          :public
+        end
+      end
+    end
+  end
+end

data/lib/benchmark/swap/verifier.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module Benchmark
+  module Swap
+    # Runs the block once per side and compares the outcomes. A mismatch is a
+    # warning, never an error: the block may return something that does not
+    # compare with ==, and that is the caller's call to make.
+    class Verifier
+      # An inspect of a real object graph can fill the console, and this is
+      # only here to show the reader what differed.
+      MAX_LENGTH = 200
+
+      Result = Struct.new(:value, :error) do
+        def self.capture
+          new(yield, nil)
+        rescue StandardError, ScriptError => e
+          new(nil, e)
+        end
+
+        def match?(other)
+          return value == other.value unless error || other.error
+
+          other.error.instance_of?(error.class) && other.error.message == error.message
+        end
+
+        def to_s
+          return "#{error.class}: #{error.message}" if error
+
+          text = value.inspect
+          text.length > MAX_LENGTH ? "#{text[0, MAX_LENGTH]}..." : text
+        end
+      end
+
+      Outcome = Struct.new(:original, :swapped) do
+        def match?
+          original.match?(swapped)
+        end
+      end
+
+      def call(swapper, &block)
+        original = Result.capture(&block)
+        swapped = swapper.swapped { Result.capture(&block) }
+
+        Outcome.new(original, swapped)
+      end
+    end
+  end
+end

data/lib/benchmark/swap/version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module Benchmark
+  module Swap
+    VERSION = "0.1.0"
+  end
+end

data/lib/benchmark/swap.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+require "benchmark/ips"
+
+require_relative "swap/version"
+require_relative "swap/candidate"
+require_relative "swap/discovery"
+require_relative "swap/swapper"
+require_relative "swap/verifier"
+require_relative "swap/runner"
+
+module Benchmark
+  # Benchmarks a second implementation of a method where it already runs.
+  #
+  #   class Pow
+  #     def pow
+  #       do_pow
+  #     end
+  #
+  #     private
+  #
+  #     def do_pow
+  #       @number**2
+  #     end
+  #
+  #     def do_pow_perf
+  #       @number * @number
+  #     end
+  #   end
+  #
+  #   Benchmark.swap { Pow.new(2).pow }
+  module Swap
+    DEFAULT_SUFFIX = "_perf"
+
+    class << self
+      # @param suffix [String] the twin's name suffix
+      # @param verify [Boolean] compare both sides once before benchmarking
+      # @param output [IO] where the swap report goes
+      # @param ips_options [Hash] passed to benchmark-ips, e.g. time:, warmup:
+      # @return [Runner::Result, nil] nil when no twin was called
+      def test(suffix: DEFAULT_SUFFIX, verify: true, output: $stdout, **ips_options, &block)
+        raise ArgumentError, "a block is required" unless block
+
+        Runner.new(suffix: suffix, verify: verify, output: output, ips_options: ips_options).call(&block)
+      end
+    end
+  end
+
+  def self.swap(...)
+    Swap.test(...)
+  end
+end

metadata

@@ -0,0 +1,71 @@
+--- !ruby/object:Gem::Specification
+name: benchmark-swap
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Mehmet Emin INAC
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: benchmark-ips
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.13'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.13'
+description: |
+  Write your alternative implementation next to the original one, with a
+  _perf suffix on the name. Benchmark.swap runs your block twice, once with
+  the original methods and once with the suffixed twins in their place, and
+  compares the two. No need to lift the method out of its call chain first.
+email:
+- mehmetemininac@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- lib/benchmark/swap.rb
+- lib/benchmark/swap/candidate.rb
+- lib/benchmark/swap/discovery.rb
+- lib/benchmark/swap/runner.rb
+- lib/benchmark/swap/swapper.rb
+- lib/benchmark/swap/verifier.rb
+- lib/benchmark/swap/version.rb
+homepage: https://github.com/meinac/benchmark-swap
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/meinac/benchmark-swap
+  changelog_uri: https://github.com/meinac/benchmark-swap/blob/master/CHANGELOG.md
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.10
+specification_version: 4
+summary: Benchmark a second implementation of a method where it already runs.
+test_files: []