ruby_memcheck 0.3.0 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 149d43766feab55baa7f6822767ec4d68ae465668fc6183c8ad9cca7b4bd7955
4
- data.tar.gz: 1d97e5ac70355a3b14f14e7e7acbba925b8d4e43a1907e5ceb0af7de29a60732
3
+ metadata.gz: 751718a23aee213973e93656dba653e4e253fd3f83618007c39e809827273ac5
4
+ data.tar.gz: 55d6196fb48e0c18fabb5c87536c2c4114d7a1c1a2b04128ec2c707389934de0
5
5
  SHA512:
6
- metadata.gz: f1953d6feaba3701b695d0c8d84086efa586ce05e2da518b4e68e8e8b5f8a615e35360ce36a32e5f61a489bc3bb88517b48353a644a72cb2f902edb6ff34e633
7
- data.tar.gz: a95a1a75f17ddafb64a2e767d0a32ce5aa0b941e5bd4522a9ab5b3bbe5fc647c49579014fe09571f27b64e543521d6327b3dbde4e066044005696edc13442a39
6
+ metadata.gz: 861d0639b3d06d124f4020cb372472954018ad113ba4864b9367be6c6d167a338297d18c7c4adbcae767ba4a30e8925402bd7faa5a97841a513914ce41e10997
7
+ data.tar.gz: e0dc1e236947746baef93281f00ec8129f651b7e560e56d2dffc5c188f14e5b32b39adc204ecea2da51c1e9ed341f1c00ff55b4b82521bb2b7b54085d5bfddfc
data/README.md CHANGED
@@ -10,10 +10,13 @@ This gem provides a sane way to use Valgrind's memcheck on your native extension
10
10
  1. [Limitations](#limitations)
11
11
  1. [Installation](#installation)
12
12
  1. [Setup](#setup)
13
+ 1. [Configuration](#configuration)
14
+ 1. [Suppression files](#suppression-files)
15
+ 1. [License](#license)
13
16
 
14
17
  ## What is this gem?
15
18
 
16
- Valgrind's memcheck is a great tool to find and debug memory issues (e.g. memory leak, use-after-free, etc.). However, it doesn't work well on Ruby because Ruby does not free all of the memory it allocates during shutdown. This results in Valgrind reporting thousands (or more) false-positives, making it very difficult for Valgrind to actually be useful. This gem solves the problem by using heuristics to filter out false-positives.
19
+ Valgrind's memcheck is a great tool to find and debug memory issues (e.g. memory leak, use-after-free, etc.). However, it doesn't work well on Ruby because Ruby does not free all of the memory it allocates during shutdown. This results in Valgrind reporting thousands (or more) false positives, making it very difficult for Valgrind to actually be useful. This gem solves the problem by using heuristics to filter out false positives.
17
20
 
18
21
  ### Who should use this gem?
19
22
 
@@ -21,11 +24,11 @@ Only gems with native extensions can use this gem. If your gem is written in pla
21
24
 
22
25
  ### How does it work?
23
26
 
24
- This gem runs Valgrind with the `--xml` option to generate a XML of all the errors. It will then parse the XML and use various heuristics based on the type of the error and the stack trace to filter out errors that are false-positives.
27
+ This gem runs Valgrind with the `--xml` option to generate an XML of all the errors. It will then parse the XML and use various heuristics based on the type of the error and the stack trace to filter out errors that are false positives.
25
28
 
26
29
  ### Limitations
27
30
 
28
- Because of the aggressive heuristics used to filter out false-positives, there are various limitations of what this gem can detect.
31
+ Because of the aggressive heuristics used to filter out false positives, there are various limitations of what this gem can detect.
29
32
 
30
33
  1. This gem is only expected to work on Linux.
31
34
  1. It will not find memory leaks in Ruby. It filters out everything in Ruby.
@@ -115,7 +118,47 @@ The easiest way to use this gem is to use it on your test suite (minitest or RSp
115
118
  ```
116
119
 
117
120
  1. You're ready to run your test suite with Valgrind using `rake test:valgrind` or `rake spec:valgrind`! Note that this will take a while to run because Valgrind will make Ruby significantly slower.
118
- 1. (Optional) If you find false-positives in the output, you can create suppression files in a `suppressions` directory in the root directory of your gem. In this directory, you can create [Valgrind suppression files](https://wiki.wxwidgets.org/Valgrind_Suppression_File_Howto). The most basic suppression file is `your_binary_name_ruby.supp`. If you want some suppressions for only specific versions of Ruby, you can add the Ruby version to the filename. For example, `your_binary_name_ruby-3.supp` will suppress for any Rubies with a major version of 3 (e.g. 3.0.0, 3.1.1, etc.), while suppression file `your_binary_name_ruby-3.1.supp` will only be used for Ruby with a major and minor version of 3.1 (e.g. 3.1.0, 3.1.1, etc.).
121
+ 1. (Optional) If you find false positives in the output, you can create Valgrind suppression files. See the [`Suppression files`](#suppression-files) section for more details.
122
+
123
+ ## Configuration
124
+
125
+ When you run `RubyMemcheck.config`, you are creating a default `RubyMemcheck::Configuration`. By default, the Rake tasks for minitest and RSpec will use this configuration. You can also manually pass in a `Configuration` object as the first argument to the constructor of `RubyMemcheck::TestTask` or `RubyMemcheck::RSpec::RakeTask` to use a different `Configuration` object rather than the default one.
126
+
127
+ `RubyMemcheck::Configuration` accepts a variety of keyword arguments. Here are all the arguments:
128
+
129
+ - `binary_name`: Required. The binary name of your native extension gem. This is the same value you passed into `create_makefile` in your `extconf.rb` file.
130
+ - `ruby`: Optional. The command to run to invoke Ruby. Defaults to the Ruby that is currently being used.
131
+ - `valgrind`: Optional. The command to run to invoke Valgrind. Defaults to the string `"valgrind"`.
132
+ - `valgrind_options`: Optional. Array of options to pass into Valgrind. This is only present as an escape hatch, so avoid using it. This may be deprecated or removed in future versions.
133
+ - `valgrind_suppressions_dir`: Optional. The string path of the directory that stores suppression files for Valgrind. See the [`Suppression files`](#suppression-files) section for more details. Defaults to `suppressions`.
134
+ - `valgrind_generate_suppressions`: Optional. Whether suppressions should also be outputted along with the errors. the [`Suppression files`](#suppression-files) section for more details. Defaults to `false`.
135
+ - `skipped_ruby_functions`: Optional. Ruby functions that are ignored because they are considered a call back into Ruby. This is only present as an escape hatch, so avoid using it. If you find another Ruby function that is a false positive because it calls back into Ruby, please send a patch into this repo. Otherwise, use a Valgrind suppression file.
136
+ - `valgrind_xml_dir`: Optional. The directory to store temporary XML files for Valgrind. It defaults to a temporary directory. This is present for development debugging, so you shouldn't have to use it.
137
+ - `output_io`: Optional. The `IO` object to output Valgrind errors to. Defaults to standard error.
138
+
139
+ ## Suppression files
140
+
141
+ If you find false positives in the output, you can create suppression files in a `suppressions` directory in the root directory of your gem. In this directory, you can create [Valgrind suppression files](https://wiki.wxwidgets.org/Valgrind_Suppression_File_Howto).
142
+
143
+ The most basic suppression file is `your_binary_name_ruby.supp`. If you want some suppressions for only specific versions of Ruby, you can add the Ruby version to the filename. For example, `your_binary_name_ruby-3.supp` will suppress for any Rubies with a major version of 3 (e.g. 3.0.0, 3.1.1, etc.), while suppression file `your_binary_name_ruby-3.1.supp` will only be used for Ruby with a major and minor version of 3.1 (e.g. 3.1.0, 3.1.1, etc.).
144
+
145
+ ## Success stories
146
+
147
+ Let's celebrate wins from this gem! If this gem was useful for you, please share your story below too!
148
+
149
+ - [`liquid-c`](https://github.com/Shopify/liquid-c):
150
+ - Found 2 memory leaks: [#157](https://github.com/Shopify/liquid-c/pull/157), [#161](https://github.com/Shopify/liquid-c/pull/161)
151
+ - Running on CI: [#162](https://github.com/Shopify/liquid-c/pull/162)
152
+ - [`nokogiri`](https://github.com/sparklemotion/nokogiri):
153
+ - Found 5 memory leaks: [4 in #2345](https://github.com/sparklemotion/nokogiri/pull/2345), [#2347](https://github.com/sparklemotion/nokogiri/pull/2347)
154
+ - CI is WIP: [#2344](https://github.com/sparklemotion/nokogiri/pull/2344)
155
+ - [`rotoscope`](https://github.com/Shopify/rotoscope):
156
+ - Found a [memory leak in Ruby TracePoint](https://bugs.ruby-lang.org/issues/18264)
157
+ - Running on CI: [#89](https://github.com/Shopify/rotoscope/pull/89)
158
+ - [`protobuf`](https://github.com/protocolbuffers/protobuf):
159
+ - Found 1 memory leak: [#9150](https://github.com/protocolbuffers/protobuf/pull/9150)
160
+ - [`gRPC`](https://github.com/grpc/grpc):
161
+ - Found 1 memory leak: [#27900](https://github.com/grpc/grpc/pull/27900)
119
162
 
120
163
  ## License
121
164
 
@@ -16,7 +16,7 @@ module RubyMemcheck
16
16
  /\Aeval_string_with_cref\z/,
17
17
  /\Arb_add_method_cfunc\z/,
18
18
  /\Arb_check_funcall/,
19
- /\Arb_class_boot\z/,
19
+ /\Arb_class_boot\z/, # Called for all the different ways to create a Class
20
20
  /\Arb_enc_raise\z/,
21
21
  /\Arb_exc_raise\z/,
22
22
  /\Arb_extend_object\z/,
@@ -27,6 +27,7 @@ module RubyMemcheck
27
27
  /\Arb_raise\z/,
28
28
  /\Arb_rescue/,
29
29
  /\Arb_respond_to\z/,
30
+ /\Arb_thread_create\z/, # Threads are relased to a cache, so they may be reported as a leak
30
31
  /\Arb_yield/,
31
32
  ].freeze
32
33
 
@@ -79,42 +80,6 @@ module RubyMemcheck
79
80
  ].flatten.join(" ")
80
81
  end
81
82
 
82
- def skip_stack?(stack)
83
- in_binary = false
84
-
85
- stack.frames.each do |frame|
86
- fn = frame.fn
87
-
88
- if frame_in_ruby?(frame) # in ruby
89
- unless in_binary
90
- # Skip this stack because it was called from Ruby
91
- return true if skipped_ruby_functions.any? { |r| r.match?(fn) }
92
- end
93
- elsif frame_in_binary?(frame) # in binary
94
- in_binary = true
95
-
96
- # Skip the Init function
97
- return true if fn == "Init_#{binary_name}"
98
- end
99
- end
100
-
101
- !in_binary
102
- end
103
-
104
- def frame_in_ruby?(frame)
105
- frame.obj == ruby ||
106
- # Hack to fix Ruby built with --enabled-shared
107
- File.basename(frame.obj) == "libruby.so.#{RUBY_VERSION}"
108
- end
109
-
110
- def frame_in_binary?(frame)
111
- if frame.obj
112
- File.basename(frame.obj, ".*") == binary_name
113
- else
114
- false
115
- end
116
- end
117
-
118
83
  private
119
84
 
120
85
  def get_valgrind_suppression_files(dir)
@@ -2,17 +2,29 @@
2
2
 
3
3
  module RubyMemcheck
4
4
  class Frame
5
- attr_reader :fn, :obj, :file, :line, :in_binary
6
- alias_method :in_binary?, :in_binary
5
+ attr_reader :configuration, :fn, :obj, :file, :line
7
6
 
8
7
  def initialize(configuration, frame_xml)
8
+ @configuration = configuration
9
9
  @fn = frame_xml.at_xpath("fn")&.content
10
10
  @obj = frame_xml.at_xpath("obj")&.content
11
11
  # file and line may not be available
12
12
  @file = frame_xml.at_xpath("file")&.content
13
13
  @line = frame_xml.at_xpath("line")&.content
14
+ end
15
+
16
+ def in_ruby?
17
+ obj == configuration.ruby ||
18
+ # Hack to fix Ruby built with --enabled-shared
19
+ File.basename(obj) == "libruby.so.#{RUBY_VERSION}"
20
+ end
14
21
 
15
- @in_binary = configuration.frame_in_binary?(self)
22
+ def in_binary?
23
+ if obj
24
+ File.basename(obj, ".*") == configuration.binary_name
25
+ else
26
+ false
27
+ end
16
28
  end
17
29
 
18
30
  def to_s
@@ -2,10 +2,38 @@
2
2
 
3
3
  module RubyMemcheck
4
4
  class Stack
5
- attr_reader :frames
5
+ attr_reader :configuration, :frames
6
6
 
7
7
  def initialize(configuration, stack_xml)
8
+ @configuration = configuration
8
9
  @frames = stack_xml.xpath("frame").map { |frame| Frame.new(configuration, frame) }
9
10
  end
11
+
12
+ def skip?
13
+ in_binary = false
14
+
15
+ frames.each do |frame|
16
+ fn = frame.fn
17
+
18
+ if frame.in_ruby?
19
+ # If a stack from from the binary was encountered first, then this
20
+ # memory leak did not occur from Ruby
21
+ unless in_binary
22
+ # Skip this stack because it was called from Ruby
23
+ return true if configuration.skipped_ruby_functions.any? { |r| r.match?(fn) }
24
+ end
25
+ elsif frame.in_binary?
26
+ in_binary = true
27
+
28
+ # Skip the Init function because it is only ever called once, so
29
+ # leaks in it cannot cause memory bloat
30
+ return true if fn == "Init_#{configuration.binary_name}"
31
+ end
32
+ end
33
+
34
+ # Skip if the stack was never in the binary because it is very likely
35
+ # not a leak in the native gem
36
+ !in_binary
37
+ end
10
38
  end
11
39
  end
@@ -27,11 +27,7 @@ module RubyMemcheck
27
27
  end
28
28
 
29
29
  def skip?
30
- if should_filter?
31
- @configuration.skip_stack?(stack)
32
- else
33
- false
34
- end
30
+ should_filter? && stack.skip?
35
31
  end
36
32
 
37
33
  def to_s
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RubyMemcheck
4
- VERSION = "0.3.0"
4
+ VERSION = "1.0.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ruby_memcheck
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.0
4
+ version: 1.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Peter Zhu
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2021-11-01 00:00:00.000000000 Z
11
+ date: 2021-11-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: nokogiri