derailed_benchmarks 2.0.1 → 2.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: cd9db4d2438152b5594f932fa35b162859a60024b4d74d00f3d4c7d5ada3f09f
-  data.tar.gz: f77ae05368c875fee001102adbd9652c1ef8001d2160064260c8b59dd455b00b
+  metadata.gz: 1004e69d7a94d86e514586324fc21ce1871ca9f8690c989eafcf5283fc69e293
+  data.tar.gz: 0bfbb09013d068279ee58e1361ef44f122c944662cc590941299b12d80151a80
 SHA512:
-  metadata.gz: f97fc2da5011cad2e69a6269776ceb7f4c769c3a7bd8bc5813848fc367f6f11ae6fdef1306139f3b8a3d5ead9cc93f4ce6d199688643111a794964392dd2d5d2
-  data.tar.gz: 39597ebde572f3ea60dcf0224c5986a6480055e5a50b4998f277c2a7fe70503b36d288e0ea146f02bf99a15edafb7c06d6d3be1ed1ba1cee81c3dfc2c0c165f7
+  metadata.gz: 48dee96d301b69c53f17467bd19918c9ea788568b5d630fe3888e2c0659345dfdef66be0b911ed2624af1cc11c9507a3700760b40f261d846f092b5e49743b79
+  data.tar.gz: d69df370bb2cd4aebc48d1b077eb0d81b9da5dc659209da091d15b49df76af196f32c3b4daa0d402db2fa1aaf968067fc8eb9e3c690a9fce1c2c4458087256aa

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 ## HEAD
 
+## 2.1.1
+
+- Fix Thor's deprecation warning by implementing `exit_on_failure?` (https://github.com/schneems/derailed_benchmarks/pull/195)
+
+## 2.1.0
+
+- Add `perf:heap_diff` tool (https://github.com/schneems/derailed_benchmarks/pull/193)
+
 ## 2.0.1
 
 - `rack-test` dependency added (https://github.com/schneems/derailed_benchmarks/pull/187)

data/README.md

@@ -198,12 +198,17 @@ You can run commands against your app by running `$ derailed exec`. There are se
 ```
 $ bundle exec derailed exec --help
   $ derailed exec perf:allocated_objects  # outputs allocated object diff after app is called TEST_COUNT times
+  $ derailed exec perf:app  # runs the performance test against two most recent commits of the current app
   $ derailed exec perf:gc  # outputs GC::Profiler.report data while app is called TEST_COUNT times
+  $ derailed exec perf:heap  # heap analyzer
   $ derailed exec perf:ips  # iterations per second
+  $ derailed exec perf:library  # runs the same test against two different branches for statistical comparison
   $ derailed exec perf:mem  # show memory usage caused by invoking require per gem
-  $ derailed exec perf:objects  # profiles ruby allocation
   $ derailed exec perf:mem_over_time  # outputs memory usage over time
+  $ derailed exec perf:objects  # profiles ruby allocation
+  $ derailed exec perf:stackprof  # stackprof
   $ derailed exec perf:test  # hits the url TEST_COUNT times
+  $ derailed exec perf:heap_diff  # three heaps generation for comparison
 ```
 
 Instead of going over each command we'll look at common problems and which commands are best used to diagnose them. Later on we'll cover all of the environment variables you can use to configure derailed benchmarks in it's own section.
@@ -271,7 +276,7 @@ This is similar to `$ bundle exec derailed bundle:objects` however it includes o
 
 ## I want a Heap Dump
 
-If you're still struggling with runtime memory you can generate a heap dump that can later be analyzed using [heap_inspect](https://github.com/schneems/heapy).
+If you're still struggling with runtime memory you can generate a heap dump that can later be analyzed using [heapy](https://github.com/schneems/heapy).
 
 ```
 $ bundle exec derailed exec perf:heap
@@ -295,6 +300,40 @@ For more help on getting data from a heap dump see
 $ heapy --help
 ```
 
+### I want more heap dumps
+
+When searching for a leak, you can use heap dumps for comparison to see what is
+retained. See [Analyzing memory heaps](https://medium.com/klaxit-techblog/tracking-a-ruby-memory-leak-in-2021-9eb56575f731#875b)
+(inspired from [SamSaffron's original idea](https://speakerdeck.com/samsaffron/why-ruby-2-dot-1-excites-me?slide=27))
+for a clear example. You can generate 3 dumps (one every `TEST_COUNT` calls) using the
+next command:
+
+```
+$ bundle exec derailed exec perf:heap_diff
+Endpoint: "/"
+Running 1000 times
+Heap file generated: "tmp/2021-05-06T15:19:26+02:00-heap-0.ndjson"
+Running 1000 times
+Heap file generated: "tmp/2021-05-06T15:19:26+02:00-heap-1.ndjson"
+Running 1000 times
+Heap file generated: "tmp/2021-05-06T15:19:26+02:00-heap-2.ndjson"
+
+Diff
+====
+Retained STRING 90 objects of size 4790/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/utils.rb:461
+Retained ICLASS 20 objects of size 800/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/sinatra-contrib-2.0.8.1/lib/sinatra/namespace.rb:198
+Retained DATA 20 objects of size 1360/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/2.7.0/monitor.rb:238
+Retained STRING 20 objects of size 800/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/rack-protection-2.0.8.1/lib/rack/protection/xss_header.rb:20
+Retained STRING 10 objects of size 880/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/newrelic_rpm-5.4.0.347/lib/new_relic/agent/transaction.rb:890
+Retained CLASS 10 objects of size 4640/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/sinatra-contrib-2.0.8.1/lib/sinatra/namespace.rb:198
+Retained IMEMO 10 objects of size 480/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/sinatra-2.0.8.1/lib/sinatra/base.rb:1017
+...
+
+Run `$ heapy --help` for more options
+
+Also read https://medium.com/klaxit-techblog/tracking-a-ruby-memory-leak-in-2021-9eb56575f731#875b to understand better what you are reading.
+```
+
 ### Memory Is large at boot.
 
 Ruby memory typically goes in one direction, up. If your memory is large when you boot the application it will likely only increase. In addition to debugging memory retained from dependencies obtained while running `$ derailed bundle:mem` you'll likely want to see how your own files contribute to memory use.

data/bin/derailed

@@ -20,6 +20,9 @@ Bundler.setup
 require 'thor'
 
 class DerailedBenchmarkCLI < Thor
+  def self.exit_on_failure?
+    true
+  end
 
   desc "exec", "executes given derailed benchmark"
   def exec(task = nil)

data/lib/derailed_benchmarks/tasks.rb

@@ -247,6 +247,42 @@ namespace :perf do
     puts "Also try uploading #{file_name.inspect} to http://tenderlove.github.io/heap-analyzer/"
   end
 
+  desc "three heaps generation for comparison."
+  task :heap_diff => [:setup] do
+    require 'objspace'
+
+    launch_time = Time.now.iso8601
+    FileUtils.mkdir_p("tmp")
+    ObjectSpace.trace_object_allocations_start
+    3.times do |i|
+      file_name = "tmp/#{launch_time}-heap-#{i}.ndjson"
+      puts "Running #{ TEST_COUNT } times"
+      TEST_COUNT.times {
+        call_app
+      }
+      GC.start
+
+      puts "Heap file generated: #{ file_name.inspect }"
+      ObjectSpace.dump_all(output: File.open(file_name, 'w'))
+    end
+
+    require 'heapy'
+
+    puts ""
+    puts "Diff"
+    puts "===="
+    Heapy::Diff.new(
+      before: "tmp/#{launch_time}-heap-0.ndjson",
+      after: "tmp/#{launch_time}-heap-1.ndjson",
+      retained: "tmp/#{launch_time}-heap-2.ndjson"
+    ).call
+
+    puts ""
+    puts "Run `$ heapy --help` for more options"
+    puts ""
+    puts "Also read https://medium.com/klaxit-techblog/tracking-a-ruby-memory-leak-in-2021-9eb56575f731#875b to understand better what you are reading."
+  end
+
   def run!(cmd)
     out = `#{cmd}`
     raise "Error while running #{cmd.inspect}: #{out}" unless $?.success?

data/lib/derailed_benchmarks/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module DerailedBenchmarks
-  VERSION = "2.0.1"
+  VERSION = "2.1.1"
 end

data/test/derailed_test.rb

@@ -11,4 +11,19 @@ class DerailedBenchmarksTest < ActiveSupport::TestCase
     assert DerailedBenchmarks.gem_is_bundled?("rack")
     refute DerailedBenchmarks.gem_is_bundled?("wicked")
   end
+
+  test "readme contains correct output" do
+    readme_path = File.join(__dir__, "..", "README.md")
+    lines = File.foreach(readme_path)
+    lineno = 1
+    expected = lines.lazy.drop_while { |line|
+      lineno += 1
+      line != "$ bundle exec derailed exec --help\n"
+    }.drop(1).take_while { |line| line != "```\n" }.force.join
+    assert_equal(
+      expected,
+      `bundle exec derailed exec --help`,
+      "Please update README.md:#{lineno}"
+    )
+  end
 end

data/test/integration/tasks_test.rb

@@ -153,6 +153,10 @@ class TasksTest < ActiveSupport::TestCase
   end
 
   test 'ips' do
-    rake "perf:mem_over_time"
+    rake "perf:ips"
+  end
+
+  test 'heap_diff' do
+    rake "perf:heap_diff", env: { "TEST_COUNT" => 5 }
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: derailed_benchmarks
 version: !ruby/object:Gem::Version
-  version: 2.0.1
+  version: 2.1.1
 platform: ruby
 authors:
 - Richard Schneeman
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-02-09 00:00:00.000000000 Z
+date: 2021-07-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: heapy
@@ -387,7 +387,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.3
+rubygems_version: 3.2.22
 signing_key:
 specification_version: 4
 summary: Benchmarks designed to performance test your ENTIRE site