memtf 0.0.2 → 0.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/README.md +42 -12
- data/lib/memtf.rb +12 -2
- data/lib/memtf/analyzer.rb +37 -14
- data/lib/memtf/analyzer/memory.rb +21 -0
- data/lib/memtf/persistance.rb +11 -4
- data/lib/memtf/reporter.rb +24 -3
- data/lib/memtf/runner.rb +8 -1
- data/lib/memtf/utilities.rb +2 -1
- data/lib/memtf/utilities/array.rb +6 -1
- data/lib/memtf/version.rb +1 -1
- data/memtf.gemspec +2 -1
- data/spec/lib/memtf/analyzer_spec.rb +167 -0
- data/spec/lib/memtf/persistance_spec.rb +69 -0
- data/spec/lib/memtf/reporter_spec.rb +71 -0
- data/spec/lib/memtf_spec.rb +1 -1
- data/spec/spec_helper.rb +28 -0
- metadata +52 -31
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6c248874dd670cf32b12d58d1ffb8c10f799cfa8
|
4
|
+
data.tar.gz: a9c0bd0d27092285884f10abe2191a5672b7ac2f
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 69e7d5eff222b8860da1f6821f418358d80087782ab6f0fb02a7b8b0d5e164945a91f358c126a735085e7301e4336b8d14441a09ef4af5bb7586b2fd6825df64
|
7
|
+
data.tar.gz: d683378a0345795576cb00db1145a60f4ed095a4ffcd5d22fd3a8141bb2be52536aa7a072e2a6765e92b33f640e441d96cec552383a72259ffdd80c515fe3eb1
|
data/.gitignore
CHANGED
data/README.md
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
[](http://badge.fury.io/rb/memtf)
|
1
2
|
[](http://travis-ci.org/dresselm/memtf)
|
2
3
|
|
3
4
|
# Memtf
|
@@ -6,6 +7,22 @@ A simple utility to help isolate memory leaks in your ruby applications.
|
|
6
7
|
|
7
8
|
## Why do we need another 'memory profiler'?
|
8
9
|
|
10
|
+
Simplicity and focus. This utility is limited in its features, but should help you quickly isolate the
|
11
|
+
class that is causing the bloat. This alone may be enough. No patches to ruby, no complicated setup and
|
12
|
+
no confusing output.
|
13
|
+
|
14
|
+
Future releases will support a second pass to drill into a given class. Hopefully this 2-stroke approach
|
15
|
+
will be sufficient for most memory leaks.
|
16
|
+
|
17
|
+
In the meantime, here are some other memory utilities that may help:
|
18
|
+
|
19
|
+
* [perftools](https://github.com/tmm1/perftools.rb) - Aman Gupta, enough said - low-level sampling profiler
|
20
|
+
* [memprof](https://github.com/ice799/memprof) - Joe Damato, not sure which versions of ruby are supported
|
21
|
+
* [rubymass](https://github.com/archan937/ruby-mass) - pure ruby and easy to use
|
22
|
+
* [memlog](https://rubygems.org/gems/memlog) - pure ruby and extremely limited, but may help you get started
|
23
|
+
|
24
|
+
Let me know if there are others that I should add to the above list.
|
25
|
+
|
9
26
|
## Installation
|
10
27
|
|
11
28
|
Add this line to your application's Gemfile:
|
@@ -46,21 +63,34 @@ The APIs used by the gem require ruby 1.9.3+.
|
|
46
63
|
> 500000.times { |i| leaky_array << "#{i % 2}-#{Time.now.to_i}" }
|
47
64
|
> end
|
48
65
|
|
49
|
-
|
50
|
-
| Class |
|
51
|
-
|
52
|
-
| Array |
|
53
|
-
| RubyVM::InstructionSequence |
|
54
|
-
| Module |
|
55
|
-
| Class |
|
56
|
-
| String |
|
57
|
-
| Regexp |
|
58
|
-
| Hash |
|
59
|
-
| Thread | 0
|
60
|
-
|
66
|
+
+-----------------------------+--------+---------+---------+---------+---------+
|
67
|
+
| Class | Impact | Leakage | Change | Objects | Change |
|
68
|
+
+-----------------------------+--------+---------+---------+---------+---------+
|
69
|
+
| Array | 96.85% | 4.972MB | 4.972MB | 2189 | 1985 |
|
70
|
+
| RubyVM::InstructionSequence | 2.47% | 0.127MB | 0.000MB | 99 | 0 |
|
71
|
+
| Module | 0.33% | 0.017MB | 0.002MB | 18 | 0 |
|
72
|
+
| Class | 0.20% | 0.010MB | 0.001MB | 13 | 0 |
|
73
|
+
| String | 0.12% | 0.006MB | 0.001MB | 663007 | 123650 |
|
74
|
+
| Regexp | 0.02% | 0.001MB | 0.000MB | 2 | 0 |
|
75
|
+
| Hash | 0.02% | 0.001MB | 0.001MB | 9 | 2 |
|
76
|
+
| Thread | 0.00% | 0.000MB | 0.000MB | 0 | 0 |
|
77
|
+
+-----------------------------+--------+---------+---------+---------+---------+
|
61
78
|
|
62
79
|
## What should I do with these results?
|
63
80
|
|
81
|
+
If there is an obvious Class that is impacting the overall memory footprint asymmetrically, then
|
82
|
+
you should focus further efforts on that and identifying where it is referenced and what it
|
83
|
+
references.
|
84
|
+
|
85
|
+
If there is no obvious Class, then either you need to better reproduce the leak (more iterations,
|
86
|
+
execute different paths, etc), or multiple Classes are leaking. If the latter, then try to create
|
87
|
+
a focused test that narrows down the leak.
|
88
|
+
|
89
|
+
If the highlighted Class is a generic container, like Array or Hash, the leak is probably one or more
|
90
|
+
Classes referenced within the container. I am working on additional functionality to expose the most
|
91
|
+
likely candidates. While I build out this and other additional functionality, please use the tools
|
92
|
+
mentioned above to help isolate the leak.
|
93
|
+
|
64
94
|
## Contributing
|
65
95
|
|
66
96
|
1. Fork it
|
data/lib/memtf.rb
CHANGED
@@ -1,18 +1,25 @@
|
|
1
|
+
# A simple utility to help isolate memory leaks. Two
|
2
|
+
# memory snapshots are compared to determine which
|
3
|
+
# classes, if any, are leaking.
|
1
4
|
module Memtf
|
2
|
-
# Represents the starting
|
5
|
+
# Represents the starting memory snapshot
|
3
6
|
START = :start
|
4
|
-
# Represents the
|
7
|
+
# Represents the ending memory snapshot
|
5
8
|
STOP = :stop
|
6
9
|
|
7
10
|
class << self
|
8
11
|
attr_accessor :runner
|
9
12
|
|
13
|
+
# Generate an initial memory snapshot.
|
14
|
+
#
|
10
15
|
# @param [Hash] options
|
11
16
|
# @return [Runner]
|
12
17
|
def start(options={})
|
13
18
|
self.runner = Runner.run(START, options)
|
14
19
|
end
|
15
20
|
|
21
|
+
# Generate a final memory snapshot.
|
22
|
+
#
|
16
23
|
# @param [Hash] options
|
17
24
|
def stop(options={})
|
18
25
|
default_group = self.runner.group
|
@@ -21,6 +28,9 @@ module Memtf
|
|
21
28
|
self.runner = nil
|
22
29
|
end
|
23
30
|
|
31
|
+
# Generate an initial memory snapshot, execute
|
32
|
+
# the block, then generate the final memory snapshot.
|
33
|
+
#
|
24
34
|
# @param [Hash] options
|
25
35
|
def around(options={}, &block)
|
26
36
|
start(options)
|
data/lib/memtf/analyzer.rb
CHANGED
@@ -1,22 +1,32 @@
|
|
1
|
-
|
2
|
-
|
1
|
+
# Encapsulates logic that measures the memory footprint of all
|
2
|
+
# objects at a given point in time and compares the memory footprints
|
3
|
+
# of two points in time.
|
3
4
|
class Memtf::Analyzer
|
4
5
|
|
5
|
-
attr_reader :threshold, :filter
|
6
|
+
attr_reader :threshold, :filter, :memory_tracker
|
6
7
|
|
8
|
+
# The threshold of total memory consumption required
|
9
|
+
# to be included in the output
|
7
10
|
DEFAULT_THRESHOLD = 0.005
|
11
|
+
# Represents 1 million bytes
|
8
12
|
MB = 1024.0**2
|
9
13
|
|
14
|
+
# Determine the memory footprint of each class and filter out classes
|
15
|
+
# that do not meet the configured threshold.
|
16
|
+
#
|
10
17
|
# @param [Hash] options
|
11
18
|
def self.analyze(options={})
|
12
19
|
new(options).analyze
|
13
20
|
end
|
14
21
|
|
22
|
+
# Compare the memory footprints for the start and end memory snapshots
|
23
|
+
# within the same snapshot group.
|
24
|
+
#
|
15
25
|
# @param [String] group
|
16
26
|
# @return [Hash]
|
17
27
|
def self.analyze_group(group)
|
18
|
-
start_analysis = Memtf::Persistance.load(Memtf::START,
|
19
|
-
end_analysis = Memtf::Persistance.load(Memtf::STOP,
|
28
|
+
start_analysis = Memtf::Persistance.load(Memtf::START, group)
|
29
|
+
end_analysis = Memtf::Persistance.load(Memtf::STOP, group)
|
20
30
|
|
21
31
|
comparison = {}
|
22
32
|
total_memsize = 0
|
@@ -29,32 +39,41 @@ class Memtf::Analyzer
|
|
29
39
|
start_val = start_stats.nil? ? 0 : start_stats[stat_key]
|
30
40
|
end_val = end_stats[stat_key]
|
31
41
|
delta = end_val - start_val
|
32
|
-
comparison[clazz][stat_key] = delta
|
33
42
|
|
34
|
-
|
35
|
-
|
43
|
+
comparison[clazz][stat_key] = end_val
|
44
|
+
comparison[clazz]["#{stat_key}_delta"] = delta
|
45
|
+
|
46
|
+
total_memsize += end_val if stat_key == 'size'
|
36
47
|
end
|
37
48
|
end
|
38
49
|
|
39
50
|
# Determine the relative memory impact of each class
|
51
|
+
# TODO look into object count impact via ObjectSpace.count_objects
|
40
52
|
comparison.keys.each do |klazz|
|
41
53
|
stats = comparison[klazz]
|
42
|
-
stats['impact'] = stats['size'] / total_memsize
|
54
|
+
stats['impact'] = (stats['size']*1.0) / total_memsize
|
43
55
|
end
|
44
56
|
|
45
57
|
comparison
|
46
58
|
end
|
47
59
|
|
48
60
|
def initialize(options={})
|
49
|
-
@filter
|
50
|
-
@threshold
|
61
|
+
@filter = options[:filter]
|
62
|
+
@threshold = options.fetch(:threshold, DEFAULT_THRESHOLD)
|
63
|
+
@memory_tracker = options.fetch(:memory_tracker, Memtf::Analyzer::Memory)
|
51
64
|
end
|
52
65
|
|
66
|
+
# Determine the memory footprint of each class and filter out classes
|
67
|
+
# that do not meet the configured threshold.
|
68
|
+
#
|
53
69
|
# @return [Hash]
|
54
70
|
def analyze
|
71
|
+
# Signal a new GC to attempt to clear out non-leaked memory
|
72
|
+
# TODO investigate ObjectSpace.garbage_collect
|
55
73
|
GC.start
|
56
74
|
|
57
75
|
classes_stats = {}
|
76
|
+
# TODO investigate ObjectSpace.count_objects_size[:TOTAL]
|
58
77
|
total_memsize = 0
|
59
78
|
|
60
79
|
# Track the memory footprint of each class
|
@@ -69,13 +88,15 @@ class Memtf::Analyzer
|
|
69
88
|
# 'String' => [2,1]
|
70
89
|
# }
|
71
90
|
#
|
72
|
-
|
91
|
+
memory_tracker.iterate do |obj|
|
73
92
|
if (clazz = obj.class).respond_to?(:name)
|
74
93
|
class_name = clazz.name
|
75
94
|
class_stats = (classes_stats[class_name] ||= [])
|
76
95
|
|
77
|
-
obj_memsize =
|
96
|
+
obj_memsize = memory_tracker.size_of(obj)
|
78
97
|
class_stats << obj_memsize
|
98
|
+
|
99
|
+
# Note: could also use ObjectSpace.memsize_of_all(clazz)
|
79
100
|
total_memsize += obj_memsize
|
80
101
|
end
|
81
102
|
end
|
@@ -143,4 +164,6 @@ class Memtf::Analyzer
|
|
143
164
|
smh_hash
|
144
165
|
end
|
145
166
|
|
146
|
-
end
|
167
|
+
end
|
168
|
+
|
169
|
+
require 'memtf/analyzer/memory'
|
@@ -0,0 +1,21 @@
|
|
1
|
+
require 'objspace'
|
2
|
+
|
3
|
+
# Encapsulate implementation of object memory tracking
|
4
|
+
class Memtf::Analyzer::Memory
|
5
|
+
class << self
|
6
|
+
# Iterate over each object on the heap
|
7
|
+
def iterate(&block)
|
8
|
+
ObjectSpace.each_object do |obj|
|
9
|
+
block.call(obj)
|
10
|
+
end
|
11
|
+
end
|
12
|
+
|
13
|
+
# Calculate the memory allocated to a given Object in bytes
|
14
|
+
#
|
15
|
+
# @param [Object] object
|
16
|
+
# @return [Number]
|
17
|
+
def size_of(object)
|
18
|
+
ObjectSpace.memsize_of(object)
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
data/lib/memtf/persistance.rb
CHANGED
@@ -1,10 +1,15 @@
|
|
1
1
|
require 'fileutils'
|
2
2
|
require 'multi_json'
|
3
3
|
|
4
|
+
# Encapsulates the logic and mechanism for saving
|
5
|
+
# and loading raw memory data.
|
4
6
|
class Memtf::Persistance
|
7
|
+
# The directory where raw data is stored
|
5
8
|
OUTPUT_DIR = "tmp/memtf"
|
6
9
|
|
7
10
|
class << self
|
11
|
+
# Serialize group data to the filesystem.
|
12
|
+
#
|
8
13
|
# @param [String] name
|
9
14
|
# @param [String] group
|
10
15
|
# @param [Object] payload
|
@@ -12,17 +17,19 @@ class Memtf::Persistance
|
|
12
17
|
group_directory = group_dir(group)
|
13
18
|
FileUtils.mkdir_p("#{group_directory}")
|
14
19
|
|
15
|
-
save_file = "#{group_directory}/#{name}.json"
|
20
|
+
save_file = "#{group_directory}/#{name}-#{Process.pid}.json"
|
16
21
|
File.open(save_file, 'w+') do |f|
|
17
|
-
f.
|
22
|
+
f.print ::MultiJson.encode(payload)
|
18
23
|
end
|
19
24
|
end
|
20
25
|
|
26
|
+
# De-serialize group data from the filesystem.
|
27
|
+
#
|
21
28
|
# @param [String] name
|
22
29
|
# @param [String] group
|
23
30
|
# @return [Object]
|
24
31
|
def load(name, group)
|
25
|
-
load_file = "#{group_dir(group)}/#{name}.json"
|
32
|
+
load_file = "#{group_dir(group)}/#{name}-#{Process.pid}.json"
|
26
33
|
::MultiJson.decode File.read(load_file)
|
27
34
|
end
|
28
35
|
|
@@ -34,4 +41,4 @@ class Memtf::Persistance
|
|
34
41
|
"#{OUTPUT_DIR}/#{group}"
|
35
42
|
end
|
36
43
|
end
|
37
|
-
end
|
44
|
+
end
|
data/lib/memtf/reporter.rb
CHANGED
@@ -1,9 +1,23 @@
|
|
1
1
|
require 'terminal-table'
|
2
2
|
|
3
|
+
# Encapsulates the formatting and output of Memtf analysis.
|
4
|
+
#
|
5
|
+
# Example Report:
|
6
|
+
#
|
7
|
+
# +-----------------------------+--------+---------+---------+---------+---------+
|
8
|
+
# | Class | Impact | Leakage | Change | Objects | Change |
|
9
|
+
# +-----------------------------+--------+---------+---------+---------+---------+
|
10
|
+
# | Array | 96.85% | 4.972MB | 4.972MB | 2189 | 1985 |
|
11
|
+
# ...
|
12
|
+
#
|
3
13
|
class Memtf::Reporter
|
14
|
+
# The report table headers
|
15
|
+
HEADERS = ['Class', 'Impact', 'Leakage', 'Change', 'Objects', 'Change']
|
4
16
|
|
5
17
|
attr_reader :group, :options
|
6
18
|
|
19
|
+
# Print the analysis in a concise tabular format.
|
20
|
+
#
|
7
21
|
# @param [String] group
|
8
22
|
def self.report(group)
|
9
23
|
new(group).report
|
@@ -14,12 +28,19 @@ class Memtf::Reporter
|
|
14
28
|
@options = options
|
15
29
|
end
|
16
30
|
|
31
|
+
# Print the analysis in a concise tabular format.
|
32
|
+
#
|
17
33
|
# @return [Terminal::Table]
|
18
34
|
def report
|
19
|
-
Terminal::Table.new(:headings =>
|
35
|
+
Terminal::Table.new(:headings => HEADERS) do |t|
|
20
36
|
group_analysis = Memtf::Analyzer.analyze_group(group)
|
21
37
|
group_analysis.sort_by { |k,v| -v['impact'] }.each do |k,v|
|
22
|
-
t << [k,
|
38
|
+
t << [k,
|
39
|
+
to_pct(v['impact']),
|
40
|
+
to_MB(v['size']),
|
41
|
+
to_MB(v['size_delta']),
|
42
|
+
v['count'],
|
43
|
+
v['count_delta']]
|
23
44
|
end
|
24
45
|
end
|
25
46
|
end
|
@@ -35,4 +56,4 @@ class Memtf::Reporter
|
|
35
56
|
def to_pct(num)
|
36
57
|
"%.2f%" % [num * 100]
|
37
58
|
end
|
38
|
-
end
|
59
|
+
end
|
data/lib/memtf/runner.rb
CHANGED
@@ -1,6 +1,10 @@
|
|
1
|
+
# Encapsulates the multiple steps required to accomplish
|
2
|
+
# Memtf analysis and reporting.
|
1
3
|
class Memtf::Runner
|
2
4
|
attr_reader :group, :options, :report
|
3
5
|
|
6
|
+
# Run the Memtf analysis and reporting.
|
7
|
+
#
|
4
8
|
# @param [String] stage
|
5
9
|
# @param [Hash] options
|
6
10
|
def self.run(stage, options={})
|
@@ -12,7 +16,10 @@ class Memtf::Runner
|
|
12
16
|
@options = options
|
13
17
|
end
|
14
18
|
|
19
|
+
# Run the Memtf analysis and reporting.
|
20
|
+
#
|
15
21
|
# @param [String] stage
|
22
|
+
# @return [Memtf::Runner]
|
16
23
|
def run(stage)
|
17
24
|
analysis = Memtf::Analyzer.analyze(options)
|
18
25
|
Memtf::Persistance.save(stage, group, analysis)
|
@@ -25,4 +32,4 @@ class Memtf::Runner
|
|
25
32
|
|
26
33
|
self
|
27
34
|
end
|
28
|
-
end
|
35
|
+
end
|
data/lib/memtf/utilities.rb
CHANGED
@@ -1,10 +1,15 @@
|
|
1
|
+
# Monkeypatch Array with various utilities.
|
1
2
|
module Memtf::Utilities::Array
|
3
|
+
|
4
|
+
# Add up all array elements.
|
5
|
+
#
|
2
6
|
# @return [Number]
|
3
7
|
def sum
|
4
8
|
inject(0) { |sum, elem| sum + elem }
|
5
9
|
end
|
6
10
|
end
|
7
11
|
|
12
|
+
# Only monkeypatch if the method has not been defined already.
|
8
13
|
unless Array.method_defined? :sum
|
9
14
|
Array.send :include, Memtf::Utilities::Array
|
10
|
-
end
|
15
|
+
end
|
data/lib/memtf/version.rb
CHANGED
data/memtf.gemspec
CHANGED
@@ -22,10 +22,11 @@ Gem::Specification.new do |spec|
|
|
22
22
|
spec.add_dependency 'terminal-table'
|
23
23
|
|
24
24
|
spec.add_development_dependency "bundler", "~> 1.3"
|
25
|
+
spec.add_development_dependency "fakefs"
|
26
|
+
spec.add_development_dependency "guard-rspec"
|
25
27
|
spec.add_development_dependency "pry"
|
26
28
|
spec.add_development_dependency "rake"
|
27
29
|
spec.add_development_dependency "rspec"
|
28
|
-
spec.add_development_dependency "guard-rspec"
|
29
30
|
spec.add_development_dependency "simplecov"
|
30
31
|
spec.add_development_dependency "yard"
|
31
32
|
end
|
@@ -0,0 +1,167 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Memtf::Analyzer do
|
4
|
+
let(:options) { {} }
|
5
|
+
|
6
|
+
describe '.analyze' do
|
7
|
+
it 'should create a new Analyzer' do
|
8
|
+
mock_analyzer = double(described_class)
|
9
|
+
mock_analyzer.stub(:analyze)
|
10
|
+
described_class.should_receive(:new).with(options).and_return(mock_analyzer)
|
11
|
+
|
12
|
+
described_class.analyze(options)
|
13
|
+
end
|
14
|
+
|
15
|
+
it 'should delegate to analyze' do
|
16
|
+
mock_analyzer = double(described_class)
|
17
|
+
described_class.stub(:new).with(options).and_return(mock_analyzer)
|
18
|
+
mock_analyzer.should_receive(:analyze).and_return({})
|
19
|
+
|
20
|
+
described_class.analyze(options)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
describe '.analyze_group' do
|
25
|
+
let(:start_data) {
|
26
|
+
{
|
27
|
+
'Array' => {'count' => 1, 'size' => 10},
|
28
|
+
'Hash' => {'count' => 2, 'size' => 5},
|
29
|
+
'Fixnum' => {'count' => 2, 'size' => 10},
|
30
|
+
'Others*' => {'count' => 3, 'size' => 4}
|
31
|
+
}
|
32
|
+
}
|
33
|
+
let(:end_data) {
|
34
|
+
{
|
35
|
+
'Array' => {'count' => 3, 'size' => 50},
|
36
|
+
'Hash' => {'count' => 4, 'size' => 20},
|
37
|
+
'Fixnum' => {'count' => 8, 'size' => 20},
|
38
|
+
'Others*' => {'count' => 6, 'size' => 10}
|
39
|
+
}
|
40
|
+
}
|
41
|
+
let(:group) { 'test_group' }
|
42
|
+
|
43
|
+
it 'should load the start data' do
|
44
|
+
Memtf::Persistance.should_receive(:load).with(Memtf::START,group).and_return(start_data)
|
45
|
+
Memtf::Persistance.stub(:load).with(Memtf::STOP,group).and_return(end_data)
|
46
|
+
|
47
|
+
described_class.analyze_group(group)
|
48
|
+
end
|
49
|
+
|
50
|
+
it 'should load the end data' do
|
51
|
+
Memtf::Persistance.stub(:load).with(Memtf::START,group).and_return(start_data)
|
52
|
+
Memtf::Persistance.should_receive(:load).with(Memtf::STOP,group).and_return(end_data)
|
53
|
+
|
54
|
+
described_class.analyze_group(group)
|
55
|
+
end
|
56
|
+
|
57
|
+
it 'should compare the start and end object counts' do
|
58
|
+
Memtf::Persistance.stub(:load).with(Memtf::START,group).and_return(start_data)
|
59
|
+
Memtf::Persistance.stub(:load).with(Memtf::STOP,group).and_return(end_data)
|
60
|
+
|
61
|
+
output = described_class.analyze_group(group)
|
62
|
+
count_deltas = output.values.map { |o| o['count_delta']}
|
63
|
+
count_deltas.should_not be_empty
|
64
|
+
count_deltas.size.should == 4
|
65
|
+
count_deltas.should == [(3-1),(4-2),(8-2),(6-3)]
|
66
|
+
end
|
67
|
+
|
68
|
+
it 'should compare the start and end memory sizes' do
|
69
|
+
Memtf::Persistance.stub(:load).with(Memtf::START,group).and_return(start_data)
|
70
|
+
Memtf::Persistance.stub(:load).with(Memtf::STOP,group).and_return(end_data)
|
71
|
+
|
72
|
+
output = described_class.analyze_group(group)
|
73
|
+
size_deltas = output.values.map { |o| o['size_delta']}
|
74
|
+
size_deltas.should_not be_empty
|
75
|
+
size_deltas.size.should == 4
|
76
|
+
size_deltas.should == [(50-10),(20-5),(20-10),(10-4)]
|
77
|
+
end
|
78
|
+
|
79
|
+
it 'should generate an impact value' do
|
80
|
+
Memtf::Persistance.stub(:load).with(Memtf::START,group).and_return(start_data)
|
81
|
+
Memtf::Persistance.stub(:load).with(Memtf::STOP,group).and_return(end_data)
|
82
|
+
|
83
|
+
output = described_class.analyze_group(group)
|
84
|
+
impacts = output.values.map { |o| o['impact'] }
|
85
|
+
impacts.should_not be_empty
|
86
|
+
impacts.size.should == 4
|
87
|
+
# The impact is calculates as memory size of class
|
88
|
+
# compared to the total memory size
|
89
|
+
impacts.should == [0.5,0.2,0.2,0.1]
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
describe '#analyze' do
|
94
|
+
class StubbedMemoryTracker
|
95
|
+
STRING1 = 'some_string'
|
96
|
+
STRING2 = 'some_other_string'
|
97
|
+
ARRAY1 = [1,2,3,4,5,6]
|
98
|
+
HASH1 = {'foo' => 'bar'}
|
99
|
+
|
100
|
+
def self.iterate(&block)
|
101
|
+
[STRING1, STRING2, ARRAY1, HASH1].each do |obj|
|
102
|
+
block.call(obj)
|
103
|
+
end
|
104
|
+
end
|
105
|
+
|
106
|
+
def self.size_of(obj)
|
107
|
+
case obj
|
108
|
+
when STRING1 then convert_to_mb(10)
|
109
|
+
when STRING2 then convert_to_mb(20)
|
110
|
+
when ARRAY1 then convert_to_mb(90)
|
111
|
+
when HASH1 then convert_to_mb(0.005)
|
112
|
+
else
|
113
|
+
0
|
114
|
+
end
|
115
|
+
end
|
116
|
+
|
117
|
+
def self.convert_to_mb(integer)
|
118
|
+
integer * 1024**2
|
119
|
+
end
|
120
|
+
end
|
121
|
+
|
122
|
+
let(:options) {
|
123
|
+
{
|
124
|
+
:memory_tracker => StubbedMemoryTracker,
|
125
|
+
:threshold => 0.05
|
126
|
+
}
|
127
|
+
}
|
128
|
+
let(:analyzer) { described_class.new(options) }
|
129
|
+
|
130
|
+
it 'should try to initiate garbage collection' do
|
131
|
+
GC.should_receive(:start)
|
132
|
+
|
133
|
+
analyzer.analyze
|
134
|
+
end
|
135
|
+
|
136
|
+
it 'should calculate the total memory allocated to each class' do
|
137
|
+
analysis = analyzer.analyze
|
138
|
+
|
139
|
+
analysis.should_not be_nil
|
140
|
+
analysis['Array'][:size].should == 90.0
|
141
|
+
analysis['String'][:size].should == 20.0 + 10.0
|
142
|
+
end
|
143
|
+
|
144
|
+
it 'should calculate the number of objects of each class in memory' do
|
145
|
+
analysis = analyzer.analyze
|
146
|
+
|
147
|
+
analysis.should_not be_nil
|
148
|
+
analysis['Array'][:count].should == 1
|
149
|
+
analysis['String'][:count].should == 2
|
150
|
+
end
|
151
|
+
|
152
|
+
it 'should isolate classes that exceed a given memory threshold' do
|
153
|
+
analysis = analyzer.analyze
|
154
|
+
|
155
|
+
analysis.should_not be_nil
|
156
|
+
analysis['Hash'].should be_nil # doesn't meet threshold
|
157
|
+
end
|
158
|
+
|
159
|
+
it 'should generate an aggregated Others* row' do
|
160
|
+
analysis = analyzer.analyze
|
161
|
+
|
162
|
+
analysis.should_not be_nil
|
163
|
+
analysis['Others*'][:size].should == 0.005
|
164
|
+
analysis['Others*'][:count].should == 1
|
165
|
+
end
|
166
|
+
end
|
167
|
+
end
|
@@ -0,0 +1,69 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Memtf::Persistance do
|
4
|
+
include FakeFS::SpecHelpers
|
5
|
+
|
6
|
+
let(:name) { 'test_name' }
|
7
|
+
let(:group) { 'test_group' }
|
8
|
+
let(:date) { '2013-12-26' }
|
9
|
+
let(:pid) { 9 }
|
10
|
+
|
11
|
+
let(:expected_dir) { "tmp/memtf/#{group}" }
|
12
|
+
let(:expected_file_path) { "#{expected_dir}/#{name}-#{pid}.json" }
|
13
|
+
|
14
|
+
before(:each) do
|
15
|
+
Process.stub(:pid).and_return(pid)
|
16
|
+
end
|
17
|
+
|
18
|
+
describe '.save' do
|
19
|
+
let(:payload) {{ :test => :payload }}
|
20
|
+
|
21
|
+
context 'when the group directory does not exist' do
|
22
|
+
it 'should create the directory' do
|
23
|
+
Dir.exist?(expected_dir).should == false
|
24
|
+
described_class.save(name, group, payload)
|
25
|
+
Dir.exist?(expected_dir).should == true
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
it 'should save the payload' do
|
30
|
+
expected_file_path = "#{expected_dir}/#{name}-#{pid}.json"
|
31
|
+
File.exist?(expected_file_path).should == false
|
32
|
+
described_class.save(name, group, payload)
|
33
|
+
File.exist?(expected_file_path).should == true
|
34
|
+
end
|
35
|
+
|
36
|
+
it 'should encode the payload' do
|
37
|
+
described_class.save(name, group, payload)
|
38
|
+
f = File.read(expected_file_path)
|
39
|
+
expect(f).to eq "{\"test\":\"payload\"}"
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
describe '.load' do
|
44
|
+
before(:each) do
|
45
|
+
FileUtils.mkdir_p(expected_dir)
|
46
|
+
File.open(expected_file_path, 'w+') do |f|
|
47
|
+
f.print("{\"test\":\"payload\"}")
|
48
|
+
end
|
49
|
+
end
|
50
|
+
|
51
|
+
it 'should load the payload' do
|
52
|
+
payload = described_class.load(name, group)
|
53
|
+
expect(payload).not_to be_nil
|
54
|
+
end
|
55
|
+
|
56
|
+
it 'should decode the payload' do
|
57
|
+
payload = described_class.load(name, group)
|
58
|
+
expect(payload).to eq({ 'test' => 'payload' })
|
59
|
+
end
|
60
|
+
|
61
|
+
context 'when the file does not exist' do
|
62
|
+
it 'should raise an Errno::ENOENT error' do
|
63
|
+
expect {
|
64
|
+
described_class.load("bogus_name", group)
|
65
|
+
}.to raise_error(Errno::ENOENT)
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
69
|
+
end
|
@@ -0,0 +1,71 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Memtf::Reporter do
|
4
|
+
|
5
|
+
let(:group) { 'test_group' }
|
6
|
+
let(:reporter) { described_class.new(group) }
|
7
|
+
|
8
|
+
describe '.report' do
|
9
|
+
|
10
|
+
it 'should initialize a new reporter' do
|
11
|
+
reporter.stub(:report)
|
12
|
+
described_class.should_receive(:new).with(group).and_return(reporter)
|
13
|
+
|
14
|
+
described_class.report(group)
|
15
|
+
end
|
16
|
+
|
17
|
+
it 'should delegate to report' do
|
18
|
+
described_class.stub(:new).and_return(reporter)
|
19
|
+
reporter.should_receive(:report)
|
20
|
+
|
21
|
+
described_class.report(group)
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
describe '#report' do
|
26
|
+
let(:analysis) {
|
27
|
+
{
|
28
|
+
'key1' => {'size' => 2, 'size_delta' => 0, 'count' => 1000, 'count_delta' => 50, 'impact' => 0.02},
|
29
|
+
'key2' => {'size' => 78, 'size_delta' => 76, 'count' => 600, 'count_delta' => 500, 'impact' => 0.78},
|
30
|
+
'key3' => {'size' => 20, 'size_delta' => 15, 'count' => 50, 'count_delta' => 25, 'impact' => 0.20}
|
31
|
+
}
|
32
|
+
}
|
33
|
+
|
34
|
+
it 'should delegate to Memtf::Analyzer.analyze_group' do
|
35
|
+
Memtf::Analyzer.should_receive(:analyze_group).with(group).and_return(analysis)
|
36
|
+
|
37
|
+
reporter.report
|
38
|
+
end
|
39
|
+
|
40
|
+
it 'should sort the analysis by impact' do
|
41
|
+
Memtf::Analyzer.stub(:analyze_group).with(group).and_return(analysis)
|
42
|
+
|
43
|
+
table = reporter.report
|
44
|
+
table.rows.map {|r| r.cells.first.value }.should == ['key2', 'key3', 'key1']
|
45
|
+
end
|
46
|
+
|
47
|
+
it 'should print headers' do
|
48
|
+
Memtf::Analyzer.stub(:analyze_group).with(group).and_return(analysis)
|
49
|
+
|
50
|
+
table = reporter.report
|
51
|
+
table.headings.cells.map(&:value).should == ["Class", "Impact", "Leakage", "Change", "Objects", "Change"]
|
52
|
+
end
|
53
|
+
|
54
|
+
it 'should convert impact to a human readable percentage' do
|
55
|
+
Memtf::Analyzer.stub(:analyze_group).with(group).and_return(analysis)
|
56
|
+
|
57
|
+
table = reporter.report
|
58
|
+
table.rows.map {|r| r.cells[1].value }.should == ['78.00%','20.00%','2.00%']
|
59
|
+
end
|
60
|
+
|
61
|
+
it 'should convert sizes to human readable MB' do
|
62
|
+
Memtf::Analyzer.stub(:analyze_group).with(group).and_return(analysis)
|
63
|
+
|
64
|
+
table = reporter.report
|
65
|
+
table.rows.map {|r| r.cells[2].value }.should == ['78.000MB','20.000MB','2.000MB']
|
66
|
+
table.rows.map {|r| r.cells[3].value }.should == ['76.000MB','15.000MB','0.000MB']
|
67
|
+
end
|
68
|
+
|
69
|
+
end
|
70
|
+
|
71
|
+
end
|
data/spec/lib/memtf_spec.rb
CHANGED
data/spec/spec_helper.rb
CHANGED
@@ -1,2 +1,30 @@
|
|
1
|
+
require 'fakefs/spec_helpers'
|
1
2
|
require 'simplecov'
|
3
|
+
|
2
4
|
require 'memtf'
|
5
|
+
|
6
|
+
# Silence stdout:
|
7
|
+
# https://gist.github.com/adamstegman/926858
|
8
|
+
|
9
|
+
# Redirects stderr and stdout to /dev/null.
|
10
|
+
def silence_output
|
11
|
+
@orig_stderr = $stderr
|
12
|
+
@orig_stdout = $stdout
|
13
|
+
|
14
|
+
# redirect stderr and stdout to /dev/null
|
15
|
+
$stderr = File.new('/dev/null', 'w')
|
16
|
+
$stdout = File.new('/dev/null', 'w')
|
17
|
+
end
|
18
|
+
|
19
|
+
# Replace stdout and stderr so anything else is output correctly.
|
20
|
+
def enable_output
|
21
|
+
$stderr = @orig_stderr
|
22
|
+
$stdout = @orig_stdout
|
23
|
+
@orig_stderr = nil
|
24
|
+
@orig_stdout = nil
|
25
|
+
end
|
26
|
+
|
27
|
+
RSpec.configure do |config|
|
28
|
+
config.before(:all) { silence_output }
|
29
|
+
config.after(:all) { enable_output }
|
30
|
+
end
|
metadata
CHANGED
@@ -1,139 +1,153 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: memtf
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthew Dressel
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2013-12-
|
11
|
+
date: 2013-12-30 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: multi_json
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- -
|
17
|
+
- - '>='
|
18
18
|
- !ruby/object:Gem::Version
|
19
19
|
version: '0'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- -
|
24
|
+
- - '>='
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '0'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: terminal-table
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
|
-
- -
|
31
|
+
- - '>='
|
32
32
|
- !ruby/object:Gem::Version
|
33
33
|
version: '0'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
|
-
- -
|
38
|
+
- - '>='
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '0'
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: bundler
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
44
44
|
requirements:
|
45
|
-
- -
|
45
|
+
- - ~>
|
46
46
|
- !ruby/object:Gem::Version
|
47
47
|
version: '1.3'
|
48
48
|
type: :development
|
49
49
|
prerelease: false
|
50
50
|
version_requirements: !ruby/object:Gem::Requirement
|
51
51
|
requirements:
|
52
|
-
- -
|
52
|
+
- - ~>
|
53
53
|
- !ruby/object:Gem::Version
|
54
54
|
version: '1.3'
|
55
55
|
- !ruby/object:Gem::Dependency
|
56
|
-
name:
|
56
|
+
name: fakefs
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
58
58
|
requirements:
|
59
|
-
- -
|
59
|
+
- - '>='
|
60
60
|
- !ruby/object:Gem::Version
|
61
61
|
version: '0'
|
62
62
|
type: :development
|
63
63
|
prerelease: false
|
64
64
|
version_requirements: !ruby/object:Gem::Requirement
|
65
65
|
requirements:
|
66
|
-
- -
|
66
|
+
- - '>='
|
67
67
|
- !ruby/object:Gem::Version
|
68
68
|
version: '0'
|
69
69
|
- !ruby/object:Gem::Dependency
|
70
|
-
name:
|
70
|
+
name: guard-rspec
|
71
71
|
requirement: !ruby/object:Gem::Requirement
|
72
72
|
requirements:
|
73
|
-
- -
|
73
|
+
- - '>='
|
74
74
|
- !ruby/object:Gem::Version
|
75
75
|
version: '0'
|
76
76
|
type: :development
|
77
77
|
prerelease: false
|
78
78
|
version_requirements: !ruby/object:Gem::Requirement
|
79
79
|
requirements:
|
80
|
-
- -
|
80
|
+
- - '>='
|
81
81
|
- !ruby/object:Gem::Version
|
82
82
|
version: '0'
|
83
83
|
- !ruby/object:Gem::Dependency
|
84
|
-
name:
|
84
|
+
name: pry
|
85
85
|
requirement: !ruby/object:Gem::Requirement
|
86
86
|
requirements:
|
87
|
-
- -
|
87
|
+
- - '>='
|
88
88
|
- !ruby/object:Gem::Version
|
89
89
|
version: '0'
|
90
90
|
type: :development
|
91
91
|
prerelease: false
|
92
92
|
version_requirements: !ruby/object:Gem::Requirement
|
93
93
|
requirements:
|
94
|
-
- -
|
94
|
+
- - '>='
|
95
95
|
- !ruby/object:Gem::Version
|
96
96
|
version: '0'
|
97
97
|
- !ruby/object:Gem::Dependency
|
98
|
-
name:
|
98
|
+
name: rake
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - '>='
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '0'
|
104
|
+
type: :development
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - '>='
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '0'
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: rspec
|
99
113
|
requirement: !ruby/object:Gem::Requirement
|
100
114
|
requirements:
|
101
|
-
- -
|
115
|
+
- - '>='
|
102
116
|
- !ruby/object:Gem::Version
|
103
117
|
version: '0'
|
104
118
|
type: :development
|
105
119
|
prerelease: false
|
106
120
|
version_requirements: !ruby/object:Gem::Requirement
|
107
121
|
requirements:
|
108
|
-
- -
|
122
|
+
- - '>='
|
109
123
|
- !ruby/object:Gem::Version
|
110
124
|
version: '0'
|
111
125
|
- !ruby/object:Gem::Dependency
|
112
126
|
name: simplecov
|
113
127
|
requirement: !ruby/object:Gem::Requirement
|
114
128
|
requirements:
|
115
|
-
- -
|
129
|
+
- - '>='
|
116
130
|
- !ruby/object:Gem::Version
|
117
131
|
version: '0'
|
118
132
|
type: :development
|
119
133
|
prerelease: false
|
120
134
|
version_requirements: !ruby/object:Gem::Requirement
|
121
135
|
requirements:
|
122
|
-
- -
|
136
|
+
- - '>='
|
123
137
|
- !ruby/object:Gem::Version
|
124
138
|
version: '0'
|
125
139
|
- !ruby/object:Gem::Dependency
|
126
140
|
name: yard
|
127
141
|
requirement: !ruby/object:Gem::Requirement
|
128
142
|
requirements:
|
129
|
-
- -
|
143
|
+
- - '>='
|
130
144
|
- !ruby/object:Gem::Version
|
131
145
|
version: '0'
|
132
146
|
type: :development
|
133
147
|
prerelease: false
|
134
148
|
version_requirements: !ruby/object:Gem::Requirement
|
135
149
|
requirements:
|
136
|
-
- -
|
150
|
+
- - '>='
|
137
151
|
- !ruby/object:Gem::Version
|
138
152
|
version: '0'
|
139
153
|
description: A simple utility to help you isolate the little bastards that are stealing
|
@@ -144,10 +158,10 @@ executables: []
|
|
144
158
|
extensions: []
|
145
159
|
extra_rdoc_files: []
|
146
160
|
files:
|
147
|
-
-
|
148
|
-
-
|
149
|
-
-
|
150
|
-
-
|
161
|
+
- .gitignore
|
162
|
+
- .ruby-gemset
|
163
|
+
- .ruby-version
|
164
|
+
- .travis.yml
|
151
165
|
- CHANGELOG.md
|
152
166
|
- Gemfile
|
153
167
|
- Guardfile
|
@@ -156,6 +170,7 @@ files:
|
|
156
170
|
- Rakefile
|
157
171
|
- lib/memtf.rb
|
158
172
|
- lib/memtf/analyzer.rb
|
173
|
+
- lib/memtf/analyzer/memory.rb
|
159
174
|
- lib/memtf/persistance.rb
|
160
175
|
- lib/memtf/reporter.rb
|
161
176
|
- lib/memtf/runner.rb
|
@@ -164,6 +179,9 @@ files:
|
|
164
179
|
- lib/memtf/version.rb
|
165
180
|
- memtf.gemspec
|
166
181
|
- spec/integration/memtf_spec.rb
|
182
|
+
- spec/lib/memtf/analyzer_spec.rb
|
183
|
+
- spec/lib/memtf/persistance_spec.rb
|
184
|
+
- spec/lib/memtf/reporter_spec.rb
|
167
185
|
- spec/lib/memtf_spec.rb
|
168
186
|
- spec/spec_helper.rb
|
169
187
|
homepage: http://github.com/dresselm/memtf
|
@@ -176,22 +194,25 @@ require_paths:
|
|
176
194
|
- lib
|
177
195
|
required_ruby_version: !ruby/object:Gem::Requirement
|
178
196
|
requirements:
|
179
|
-
- -
|
197
|
+
- - '>='
|
180
198
|
- !ruby/object:Gem::Version
|
181
199
|
version: '0'
|
182
200
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
183
201
|
requirements:
|
184
|
-
- -
|
202
|
+
- - '>='
|
185
203
|
- !ruby/object:Gem::Version
|
186
204
|
version: '0'
|
187
205
|
requirements: []
|
188
206
|
rubyforge_project:
|
189
|
-
rubygems_version: 2.
|
207
|
+
rubygems_version: 2.0.14
|
190
208
|
signing_key:
|
191
209
|
specification_version: 4
|
192
210
|
summary: Leaking memory like a sieve? Cursing? Memtf is here to help.
|
193
211
|
test_files:
|
194
212
|
- spec/integration/memtf_spec.rb
|
213
|
+
- spec/lib/memtf/analyzer_spec.rb
|
214
|
+
- spec/lib/memtf/persistance_spec.rb
|
215
|
+
- spec/lib/memtf/reporter_spec.rb
|
195
216
|
- spec/lib/memtf_spec.rb
|
196
217
|
- spec/spec_helper.rb
|
197
218
|
has_rdoc:
|