bong 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (7) hide show
  1. data/History.txt +8 -0
  2. data/Manifest.txt +6 -0
  3. data/README.txt +101 -0
  4. data/Rakefile +16 -0
  5. data/bin/bong +95 -0
  6. data/lib/bong.rb +183 -0
  7. metadata +62 -0
@@ -0,0 +1,8 @@
1
+ == 0.0.1 / 2007-11-07
2
+
3
+ * First release. See README.txt for the details.
4
+
5
+ == TODO
6
+
7
+ * Compare multiple benchmarking runs and generate a report.
8
+ * Generate Gruff graphs or an HTML report.
@@ -0,0 +1,6 @@
1
+ History.txt
2
+ Manifest.txt
3
+ README.txt
4
+ Rakefile
5
+ bin/bong
6
+ lib/bong.rb
@@ -0,0 +1,101 @@
1
+ bong
2
+ by Geoffrey Grosenbach
3
+ boss@topfunky.com
4
+ http://topfunky.com
5
+
6
+ == DESCRIPTION:
7
+
8
+ Hit your website with bong. Uses httperf to run a suite of benchmarking tests against specified urls on your site.
9
+
10
+ Graphical output and multi-test comparisons are planned. Apache ab support may be added in the future.
11
+
12
+ == USAGE
13
+
14
+ See all options with:
15
+
16
+ % bong --help
17
+
18
+ Generate a config file (writes to config/httperf.yml by default):
19
+
20
+ % bong --generate
21
+
22
+ Edit the config file with a list of servers, urls, and your preferred sample size. NOTE: Don't run this against servers you don't own!
23
+
24
+ ---
25
+ uris:
26
+ - /
27
+ - /pages/about
28
+ samples: 2
29
+ servers:
30
+ - localhost:3000
31
+
32
+ Run the benchmarking suite, label it 'baseline', and output the raw data to a file (writes output to log/httperf-report.yml by default):
33
+
34
+ % bong baseline
35
+
36
+ A report will be printed to the screen and the raw data will be saved to log/httperf-report.yml (change with the --out option). It's a good idea to use a label for each test so you can compare them later and find out what the fastest implementation was. Examples: 'baseline', 'memcached-optimization', 'sql-queries', etc.
37
+
38
+ baseline
39
+ localhost
40
+ / 37-56 req/sec
41
+ /products.rss 395-403 req/sec
42
+ example.com
43
+ / 35-58 req/sec
44
+ /products.rss 400-407 req/sec
45
+
46
+ View the saved report again with:
47
+
48
+ % bong baseline -r log/httperf-report.yml
49
+
50
+ Lather, rinse, repeat.
51
+
52
+ == LIMITATIONS
53
+
54
+ * Can't access pages that require login.
55
+ * HTTP GET only.
56
+
57
+ == REQUIREMENTS
58
+
59
+ The httperf command-line tool must be installed. Get it here:
60
+
61
+ http://www.hpl.hp.com/research/linux/httperf/download.php
62
+
63
+ You must start a webserver (or just Mongrel). Ideally, this would be on a different machine over a fast network. You can also run bong against the local machine or (less ideally) against a remote machine over a slow network, but these will give different performance numbers and may not be accurate.
64
+
65
+ Internally, bong will
66
+
67
+ * Run a short series of 5 hits against a URL.
68
+ * Calculate the number of hits needed to run for 10 seconds, or 2 samples. You can change this in the config file, but 2 is the minimum for getting a meaningful standard deviation out of the report.
69
+ * A test will be run again.
70
+ * A short report will be displayed and the raw data will be saved for later comparison.
71
+
72
+ See http://peepcode.com/products/benchmarking-with-httperf for a full tutorial on httperf (produced by Geoffrey Grosenbach, technical editing by Zed Shaw).
73
+
74
+ == INSTALL:
75
+
76
+ * sudo gem install bong
77
+
78
+ == LICENSE:
79
+
80
+ (The MIT License)
81
+
82
+ Copyright (c) 2007 Topfunky Corporation
83
+
84
+ Permission is hereby granted, free of charge, to any person obtaining
85
+ a copy of this software and associated documentation files (the
86
+ 'Software'), to deal in the Software without restriction, including
87
+ without limitation the rights to use, copy, modify, merge, publish,
88
+ distribute, sublicense, and/or sell copies of the Software, and to
89
+ permit persons to whom the Software is furnished to do so, subject to
90
+ the following conditions:
91
+
92
+ The above copyright notice and this permission notice shall be
93
+ included in all copies or substantial portions of the Software.
94
+
95
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
96
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
97
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
98
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
99
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
100
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
101
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,16 @@
1
+ require 'rubygems'
2
+
3
+ require 'hoe'
4
+ require './lib/bong.rb'
5
+
6
+ Hoe.new('bong', Bong::VERSION) do |p|
7
+ p.rubyforge_name = 'bong'
8
+ p.author = 'Geoffrey Grosenbach'
9
+ p.email = 'boss@topfunky.com'
10
+ p.summary = 'Website benchmarking helper.'
11
+ p.description = p.paragraphs_of('README.txt', 1..2).join("\n\n")
12
+ p.url = "http://rubyforge.org/projects/bong"
13
+ p.changes = p.paragraphs_of('History.txt', 0..1).join("\n\n")
14
+
15
+ p.remote_rdoc_dir = '' # Release to root
16
+ end
@@ -0,0 +1,95 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ ##
4
+ # Created on 2007-11-6.
5
+ # Copyright (c) 2007 Topfunky Corporation
6
+ #
7
+ # Permission is hereby granted, free of charge, to any person obtaining
8
+ # a copy of this software and associated documentation files (the
9
+ # "Software"), to deal in the Software without restriction, including
10
+ # without limitation the rights to use, copy, modify, merge, publish,
11
+ # distribute, sublicense, and/or sell copies of the Software, and to
12
+ # permit persons to whom the Software is furnished to do so, subject to
13
+ # the following conditions:
14
+ #
15
+ # The above copyright notice and this permission notice shall be
16
+ # included in all copies or substantial portions of the Software.
17
+ #
18
+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
19
+ # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
20
+ # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
21
+ # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
22
+ # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
23
+ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
24
+ # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
25
+
26
+ begin
27
+ require "rubygems"
28
+ require "bong"
29
+ rescue LoadError
30
+ require File.dirname(__FILE__) + "/../lib/bong"
31
+ end
32
+
33
+ require 'optparse'
34
+
35
+ OPTIONS = {
36
+ :config => 'config/httperf.yml',
37
+ :generate => 'config/httperf.yml',
38
+ :out => "log/httperf-report.yml"
39
+ }
40
+ MANDATORY_OPTIONS = %w( )
41
+
42
+ parser = OptionParser.new do |opts|
43
+ opts.banner = <<-BANNER
44
+ A benchmarking tool for staging hits on your website. Uses httperf.
45
+
46
+ Usage: #{File.basename($0)} [label] [options]
47
+
48
+ label is a name for this benchmarking run.
49
+ Examples: 'baseline', 'with-optimizations', 'custom-sql-queries'
50
+
51
+ Other options are:
52
+ BANNER
53
+
54
+ opts.separator ""
55
+
56
+ opts.on("-c", "--config [PATH]", String, "Path to config file.", "Default: config/httperf.yml") do |v|
57
+ OPTIONS[:config] = v || OPTIONS[:config]
58
+ end
59
+
60
+ opts.on("-g", "--generate [PATH]", String, "Generate a config file.", "Default: config/httperf.yml") do |v|
61
+ Bong.generate(v || OPTIONS[:generate]); exit
62
+ end
63
+
64
+ opts.on("-o", "--out [PATH]", String, "Write output to a file.", "Default: log/httperf-report.yml") do |v|
65
+ OPTIONS[:out] = v || OPTIONS[:out]
66
+ end
67
+
68
+ opts.on("-r", "--report PATH", String, "Display the report for a data file.") do |v|
69
+ OPTIONS[:report] = v
70
+ end
71
+
72
+ opts.on("-h", "--help", "Show this help message.") { puts opts; exit }
73
+
74
+ opts.parse!(ARGV)
75
+
76
+ if MANDATORY_OPTIONS && MANDATORY_OPTIONS.find { |option| OPTIONS[option.to_sym].nil? }
77
+ puts opts; exit
78
+ end
79
+
80
+ OPTIONS[:label] = ARGV.first || "benchmark-#{Time.now.to_i}"
81
+
82
+ end
83
+
84
+ # Finally, run the app.
85
+
86
+ bong = Bong.new(OPTIONS[:config], OPTIONS[:label])
87
+
88
+ if OPTIONS[:report]
89
+ bong.load_report(OPTIONS[:report])
90
+ else
91
+ bong.run
92
+ bong.save_report(OPTIONS[:out])
93
+ end
94
+
95
+ puts bong.report
@@ -0,0 +1,183 @@
1
+ require 'yaml'
2
+ require 'logger'
3
+
4
+ ##
5
+ # A tool for running httperf against a website. Documentation coming soon.
6
+
7
+ class Bong
8
+ VERSION = '0.0.1'
9
+
10
+ ##
11
+ # Generate a sample config file.
12
+
13
+ def self.generate(config_yml_path)
14
+ config_data = {
15
+ 'servers' => ['localhost:3000'],
16
+ 'uris' => ['/', '/pages/about'],
17
+ 'samples' => 2
18
+ }
19
+
20
+ if File.exist?(config_yml_path)
21
+ puts("A config file already exists at '#{config_yml_path}'.")
22
+ exit
23
+ end
24
+
25
+ File.open(config_yml_path, 'w') do |f|
26
+ f.write config_data.to_yaml
27
+ end
28
+ end
29
+
30
+ def initialize(config_yml_path, label)
31
+ unless File.exist?(config_yml_path)
32
+ puts <<-MESSAGE
33
+
34
+ A config file could not be found at '#{config_yml_path}'.
35
+
36
+ Please generate one with the -g option.
37
+
38
+ MESSAGE
39
+ exit
40
+ end
41
+
42
+ @config = YAML.load(File.read(config_yml_path))
43
+ @label = label
44
+ @stats = {}
45
+
46
+ @logger = Logger.new(STDOUT)
47
+ @logger.level = Logger::DEBUG
48
+
49
+ @logger.info "Running suite for label '#{@label}'"
50
+ end
51
+
52
+ def run
53
+ servers.each do |server, port|
54
+ port ||= 80
55
+ @stats[server] = {}
56
+ uris.each do |uri|
57
+ run_benchmark(server, port, uri)
58
+ end
59
+ end
60
+ end
61
+
62
+ def load_report(report_yml_path, label=nil)
63
+ @report = YAML.load(File.read(report_yml_path))
64
+ label = label || @label || @report.keys.first
65
+ @stats = @report[label]
66
+ end
67
+
68
+ def report
69
+ length_of_longest_uri = uris.inject(0) { |length, uri| uri_length = uri.length; (uri_length > length ? uri_length : length) }
70
+
71
+ output = ["\n#{@label}"]
72
+ servers.each do |server, port|
73
+ output << " #{server}"
74
+ uris.each do |uri|
75
+ output << " #{format_string(uri, length_of_longest_uri)} #{format_rounded(@stats[server][uri]['avg_low'])}-#{format_rounded(@stats[server][uri]['avg_high'])} req/sec"
76
+ end
77
+ end
78
+ output.join("\n")
79
+ end
80
+
81
+ def save_report(path)
82
+ @all_stats = {}
83
+ if File.exist?(path)
84
+ @all_stats = YAML.load(File.read(path))
85
+ end
86
+
87
+ @all_stats[@label] = @stats
88
+
89
+ File.open(path, 'w') do |f|
90
+ f.write @all_stats.to_yaml
91
+ end
92
+ end
93
+
94
+ protected
95
+
96
+ ##
97
+ # A list of servers and ports from the config file.
98
+
99
+ def servers
100
+ @config['servers'].map { |server| server.split(/:/) }
101
+ end
102
+
103
+ def uris
104
+ @config['uris']
105
+ end
106
+
107
+ def run_benchmark(server, port, uri)
108
+ until (sufficient_sample_size?(server, uri) && no_errors?)
109
+ increase_num_conns(server, uri)
110
+ exec_command(server, port, uri, @num_conns)
111
+ @stats[server][uri] = parse_results
112
+ end
113
+ end
114
+
115
+ def exec_command(server, port, uri, num_conns)
116
+ @logger.info "Sending #{@num_conns} hits to #{server}:#{port}#{uri}"
117
+ cmd = "httperf --server #{server} --port #{port} --uri #{uri} --num-conns #{num_conns}"
118
+ @output = `#{cmd}`
119
+ end
120
+
121
+ def parse_results
122
+ stat = {}
123
+
124
+ # Total: connections 5 requests 5 replies 5 test-duration 0.013 s
125
+ stat['duration'] = @output.scan(/test-duration ([\d.]+)/).flatten.first.to_f
126
+
127
+ # Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
128
+ (stat['min'], stat['avg'], stat['max'], stat['stddev'], stat['samples']) = @output.scan(/Reply rate \[replies\/s\]: min ([\d.]+) avg ([\d.]+) max ([\d.]+) stddev ([\d.]+) \((\d+) samples\)/).flatten.map { |i| i.to_f }
129
+
130
+ # Reply status: 1xx=0 2xx=5 3xx=0 4xx=0 5xx=0
131
+ (stat['1xx'], stat['2xx'], stat['3xx'], stat['4xx'], stat['5xx']) = @output.scan(/Reply status: 1xx=(\d+) 2xx=(\d+) 3xx=(\d+) 4xx=(\d+) 5xx=(\d+)/).flatten.map { |i| i.to_f }
132
+
133
+ stat['avg_low'] = stat['avg'].to_f - 2.0 * stat['stddev'].to_f
134
+ stat['avg_high'] = stat['avg'].to_f + 2.0 * stat['stddev'].to_f
135
+
136
+ stat
137
+ end
138
+
139
+ def sufficient_sample_size?(server, uri)
140
+ @stats[server][uri]['samples'] >= @config['samples'].to_f
141
+ rescue
142
+ false
143
+ end
144
+
145
+ def no_errors?
146
+ # TODO
147
+ true
148
+ end
149
+
150
+ def increase_num_conns(server, uri)
151
+ samples = @stats[server][uri]['samples']
152
+ duration = @stats[server][uri]['duration']
153
+ target_samples = @config['samples']
154
+
155
+ seconds_per_request = (duration / @num_conns.to_f) # 0.02
156
+ adjusted_conns = (target_samples * 5.0) / seconds_per_request # 500
157
+
158
+ # Increase the connections by the factor and a bit more.
159
+ @num_conns = (adjusted_conns * 1.2).to_i
160
+ rescue
161
+ @num_conns = 5
162
+ nil
163
+ end
164
+
165
+ ##
166
+ # Return a string with rounding for display.
167
+ #
168
+ # Small numbers will have a decimal point. Larger numbers will be shown
169
+ # as plain integers.
170
+
171
+ def format_rounded(number)
172
+ if number > 20
173
+ number.to_i
174
+ else
175
+ sprintf('%0.1f', number)
176
+ end
177
+ end
178
+
179
+ def format_string(string, length)
180
+ sprintf "%-#{length + 2}s", string
181
+ end
182
+
183
+ end
metadata ADDED
@@ -0,0 +1,62 @@
1
+ --- !ruby/object:Gem::Specification
2
+ rubygems_version: 0.9.0
3
+ specification_version: 1
4
+ name: bong
5
+ version: !ruby/object:Gem::Version
6
+ version: 0.0.1
7
+ date: 2007-11-07 00:00:00 -08:00
8
+ summary: Website benchmarking helper.
9
+ require_paths:
10
+ - lib
11
+ email: boss@topfunky.com
12
+ homepage: http://rubyforge.org/projects/bong
13
+ rubyforge_project: bong
14
+ description: "== DESCRIPTION: Hit your website with bong. Uses httperf to run a suite of benchmarking tests against specified urls on your site. Graphical output and multi-test comparisons are planned. Apache ab support may be added in the future."
15
+ autorequire:
16
+ default_executable:
17
+ bindir: bin
18
+ has_rdoc: true
19
+ required_ruby_version: !ruby/object:Gem::Version::Requirement
20
+ requirements:
21
+ - - ">"
22
+ - !ruby/object:Gem::Version
23
+ version: 0.0.0
24
+ version:
25
+ platform: ruby
26
+ signing_key:
27
+ cert_chain:
28
+ post_install_message:
29
+ authors:
30
+ - Geoffrey Grosenbach
31
+ files:
32
+ - History.txt
33
+ - Manifest.txt
34
+ - README.txt
35
+ - Rakefile
36
+ - bin/bong
37
+ - lib/bong.rb
38
+ test_files: []
39
+
40
+ rdoc_options:
41
+ - --main
42
+ - README.txt
43
+ extra_rdoc_files:
44
+ - History.txt
45
+ - Manifest.txt
46
+ - README.txt
47
+ executables:
48
+ - bong
49
+ extensions: []
50
+
51
+ requirements: []
52
+
53
+ dependencies:
54
+ - !ruby/object:Gem::Dependency
55
+ name: hoe
56
+ version_requirement:
57
+ version_requirements: !ruby/object:Gem::Version::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: 1.3.0
62
+ version: