bong 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. data/History.txt +8 -0
  2. data/Manifest.txt +6 -0
  3. data/README.txt +101 -0
  4. data/Rakefile +16 -0
  5. data/bin/bong +95 -0
  6. data/lib/bong.rb +183 -0
  7. metadata +62 -0
@@ -0,0 +1,8 @@
1
+ == 0.0.1 / 2007-11-07
2
+
3
+ * First release. See README.txt for the details.
4
+
5
+ == TODO
6
+
7
+ * Compare multiple benchmarking runs and generate a report.
8
+ * Generate Gruff graphs or an HTML report.
@@ -0,0 +1,6 @@
1
+ History.txt
2
+ Manifest.txt
3
+ README.txt
4
+ Rakefile
5
+ bin/bong
6
+ lib/bong.rb
@@ -0,0 +1,101 @@
1
+ bong
2
+ by Geoffrey Grosenbach
3
+ boss@topfunky.com
4
+ http://topfunky.com
5
+
6
+ == DESCRIPTION:
7
+
8
+ Hit your website with bong. Uses httperf to run a suite of benchmarking tests against specified urls on your site.
9
+
10
+ Graphical output and multi-test comparisons are planned. Apache ab support may be added in the future.
11
+
12
+ == USAGE
13
+
14
+ See all options with:
15
+
16
+ % bong --help
17
+
18
+ Generate a config file (writes to config/httperf.yml by default):
19
+
20
+ % bong --generate
21
+
22
+ Edit the config file with a list of servers, urls, and your preferred sample size. NOTE: Don't run this against servers you don't own!
23
+
24
+ ---
25
+ uris:
26
+ - /
27
+ - /pages/about
28
+ samples: 2
29
+ servers:
30
+ - localhost:3000
31
+
32
+ Run the benchmarking suite, label it 'baseline', and output the raw data to a file (writes output to log/httperf-report.yml by default):
33
+
34
+ % bong baseline
35
+
36
+ A report will be printed to the screen and the raw data will be saved to log/httperf-report.yml (change with the --out option). It's a good idea to use a label for each test so you can compare them later and find out what the fastest implementation was. Examples: 'baseline', 'memcached-optimization', 'sql-queries', etc.
37
+
38
+ baseline
39
+ localhost
40
+ / 37-56 req/sec
41
+ /products.rss 395-403 req/sec
42
+ example.com
43
+ / 35-58 req/sec
44
+ /products.rss 400-407 req/sec
45
+
46
+ View the saved report again with:
47
+
48
+ % bong baseline -r log/httperf-report.yml
49
+
50
+ Lather, rinse, repeat.
51
+
52
+ == LIMITATIONS
53
+
54
+ * Can't access pages that require login.
55
+ * HTTP GET only.
56
+
57
+ == REQUIREMENTS
58
+
59
+ The httperf command-line tool must be installed. Get it here:
60
+
61
+ http://www.hpl.hp.com/research/linux/httperf/download.php
62
+
63
+ You must start a webserver (or just Mongrel). Ideally, this would be on a different machine over a fast network. You can also run bong against the local machine or (less ideally) against a remote machine over a slow network, but these will give different performance numbers and may not be accurate.
64
+
65
+ Internally, bong will
66
+
67
+ * Run a short series of 5 hits against a URL.
68
+ * Calculate the number of hits needed to run for 10 seconds, or 2 samples. You can change this in the config file, but 2 is the minimum for getting a meaningful standard deviation out of the report.
69
+ * A test will be run again.
70
+ * A short report will be displayed and the raw data will be saved for later comparison.
71
+
72
+ See http://peepcode.com/products/benchmarking-with-httperf for a full tutorial on httperf (produced by Geoffrey Grosenbach, technical editing by Zed Shaw).
73
+
74
+ == INSTALL:
75
+
76
+ * sudo gem install bong
77
+
78
+ == LICENSE:
79
+
80
+ (The MIT License)
81
+
82
+ Copyright (c) 2007 Topfunky Corporation
83
+
84
+ Permission is hereby granted, free of charge, to any person obtaining
85
+ a copy of this software and associated documentation files (the
86
+ 'Software'), to deal in the Software without restriction, including
87
+ without limitation the rights to use, copy, modify, merge, publish,
88
+ distribute, sublicense, and/or sell copies of the Software, and to
89
+ permit persons to whom the Software is furnished to do so, subject to
90
+ the following conditions:
91
+
92
+ The above copyright notice and this permission notice shall be
93
+ included in all copies or substantial portions of the Software.
94
+
95
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
96
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
97
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
98
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
99
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
100
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
101
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,16 @@
1
+ require 'rubygems'
2
+
3
+ require 'hoe'
4
+ require './lib/bong.rb'
5
+
6
+ Hoe.new('bong', Bong::VERSION) do |p|
7
+ p.rubyforge_name = 'bong'
8
+ p.author = 'Geoffrey Grosenbach'
9
+ p.email = 'boss@topfunky.com'
10
+ p.summary = 'Website benchmarking helper.'
11
+ p.description = p.paragraphs_of('README.txt', 1..2).join("\n\n")
12
+ p.url = "http://rubyforge.org/projects/bong"
13
+ p.changes = p.paragraphs_of('History.txt', 0..1).join("\n\n")
14
+
15
+ p.remote_rdoc_dir = '' # Release to root
16
+ end
@@ -0,0 +1,95 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ ##
4
+ # Created on 2007-11-6.
5
+ # Copyright (c) 2007 Topfunky Corporation
6
+ #
7
+ # Permission is hereby granted, free of charge, to any person obtaining
8
+ # a copy of this software and associated documentation files (the
9
+ # "Software"), to deal in the Software without restriction, including
10
+ # without limitation the rights to use, copy, modify, merge, publish,
11
+ # distribute, sublicense, and/or sell copies of the Software, and to
12
+ # permit persons to whom the Software is furnished to do so, subject to
13
+ # the following conditions:
14
+ #
15
+ # The above copyright notice and this permission notice shall be
16
+ # included in all copies or substantial portions of the Software.
17
+ #
18
+ # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
19
+ # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
20
+ # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
21
+ # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
22
+ # LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
23
+ # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
24
+ # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
25
+
26
+ begin
27
+ require "rubygems"
28
+ require "bong"
29
+ rescue LoadError
30
+ require File.dirname(__FILE__) + "/../lib/bong"
31
+ end
32
+
33
+ require 'optparse'
34
+
35
+ OPTIONS = {
36
+ :config => 'config/httperf.yml',
37
+ :generate => 'config/httperf.yml',
38
+ :out => "log/httperf-report.yml"
39
+ }
40
+ MANDATORY_OPTIONS = %w( )
41
+
42
+ parser = OptionParser.new do |opts|
43
+ opts.banner = <<-BANNER
44
+ A benchmarking tool for staging hits on your website. Uses httperf.
45
+
46
+ Usage: #{File.basename($0)} [label] [options]
47
+
48
+ label is a name for this benchmarking run.
49
+ Examples: 'baseline', 'with-optimizations', 'custom-sql-queries'
50
+
51
+ Other options are:
52
+ BANNER
53
+
54
+ opts.separator ""
55
+
56
+ opts.on("-c", "--config [PATH]", String, "Path to config file.", "Default: config/httperf.yml") do |v|
57
+ OPTIONS[:config] = v || OPTIONS[:config]
58
+ end
59
+
60
+ opts.on("-g", "--generate [PATH]", String, "Generate a config file.", "Default: config/httperf.yml") do |v|
61
+ Bong.generate(v || OPTIONS[:generate]); exit
62
+ end
63
+
64
+ opts.on("-o", "--out [PATH]", String, "Write output to a file.", "Default: log/httperf-report.yml") do |v|
65
+ OPTIONS[:out] = v || OPTIONS[:out]
66
+ end
67
+
68
+ opts.on("-r", "--report PATH", String, "Display the report for a data file.") do |v|
69
+ OPTIONS[:report] = v
70
+ end
71
+
72
+ opts.on("-h", "--help", "Show this help message.") { puts opts; exit }
73
+
74
+ opts.parse!(ARGV)
75
+
76
+ if MANDATORY_OPTIONS && MANDATORY_OPTIONS.find { |option| OPTIONS[option.to_sym].nil? }
77
+ puts opts; exit
78
+ end
79
+
80
+ OPTIONS[:label] = ARGV.first || "benchmark-#{Time.now.to_i}"
81
+
82
+ end
83
+
84
+ # Finally, run the app.
85
+
86
+ bong = Bong.new(OPTIONS[:config], OPTIONS[:label])
87
+
88
+ if OPTIONS[:report]
89
+ bong.load_report(OPTIONS[:report])
90
+ else
91
+ bong.run
92
+ bong.save_report(OPTIONS[:out])
93
+ end
94
+
95
+ puts bong.report
@@ -0,0 +1,183 @@
1
+ require 'yaml'
2
+ require 'logger'
3
+
4
+ ##
5
+ # A tool for running httperf against a website. Documentation coming soon.
6
+
7
+ class Bong
8
+ VERSION = '0.0.1'
9
+
10
+ ##
11
+ # Generate a sample config file.
12
+
13
+ def self.generate(config_yml_path)
14
+ config_data = {
15
+ 'servers' => ['localhost:3000'],
16
+ 'uris' => ['/', '/pages/about'],
17
+ 'samples' => 2
18
+ }
19
+
20
+ if File.exist?(config_yml_path)
21
+ puts("A config file already exists at '#{config_yml_path}'.")
22
+ exit
23
+ end
24
+
25
+ File.open(config_yml_path, 'w') do |f|
26
+ f.write config_data.to_yaml
27
+ end
28
+ end
29
+
30
+ def initialize(config_yml_path, label)
31
+ unless File.exist?(config_yml_path)
32
+ puts <<-MESSAGE
33
+
34
+ A config file could not be found at '#{config_yml_path}'.
35
+
36
+ Please generate one with the -g option.
37
+
38
+ MESSAGE
39
+ exit
40
+ end
41
+
42
+ @config = YAML.load(File.read(config_yml_path))
43
+ @label = label
44
+ @stats = {}
45
+
46
+ @logger = Logger.new(STDOUT)
47
+ @logger.level = Logger::DEBUG
48
+
49
+ @logger.info "Running suite for label '#{@label}'"
50
+ end
51
+
52
+ def run
53
+ servers.each do |server, port|
54
+ port ||= 80
55
+ @stats[server] = {}
56
+ uris.each do |uri|
57
+ run_benchmark(server, port, uri)
58
+ end
59
+ end
60
+ end
61
+
62
+ def load_report(report_yml_path, label=nil)
63
+ @report = YAML.load(File.read(report_yml_path))
64
+ label = label || @label || @report.keys.first
65
+ @stats = @report[label]
66
+ end
67
+
68
+ def report
69
+ length_of_longest_uri = uris.inject(0) { |length, uri| uri_length = uri.length; (uri_length > length ? uri_length : length) }
70
+
71
+ output = ["\n#{@label}"]
72
+ servers.each do |server, port|
73
+ output << " #{server}"
74
+ uris.each do |uri|
75
+ output << " #{format_string(uri, length_of_longest_uri)} #{format_rounded(@stats[server][uri]['avg_low'])}-#{format_rounded(@stats[server][uri]['avg_high'])} req/sec"
76
+ end
77
+ end
78
+ output.join("\n")
79
+ end
80
+
81
+ def save_report(path)
82
+ @all_stats = {}
83
+ if File.exist?(path)
84
+ @all_stats = YAML.load(File.read(path))
85
+ end
86
+
87
+ @all_stats[@label] = @stats
88
+
89
+ File.open(path, 'w') do |f|
90
+ f.write @all_stats.to_yaml
91
+ end
92
+ end
93
+
94
+ protected
95
+
96
+ ##
97
+ # A list of servers and ports from the config file.
98
+
99
+ def servers
100
+ @config['servers'].map { |server| server.split(/:/) }
101
+ end
102
+
103
+ def uris
104
+ @config['uris']
105
+ end
106
+
107
+ def run_benchmark(server, port, uri)
108
+ until (sufficient_sample_size?(server, uri) && no_errors?)
109
+ increase_num_conns(server, uri)
110
+ exec_command(server, port, uri, @num_conns)
111
+ @stats[server][uri] = parse_results
112
+ end
113
+ end
114
+
115
+ def exec_command(server, port, uri, num_conns)
116
+ @logger.info "Sending #{@num_conns} hits to #{server}:#{port}#{uri}"
117
+ cmd = "httperf --server #{server} --port #{port} --uri #{uri} --num-conns #{num_conns}"
118
+ @output = `#{cmd}`
119
+ end
120
+
121
+ def parse_results
122
+ stat = {}
123
+
124
+ # Total: connections 5 requests 5 replies 5 test-duration 0.013 s
125
+ stat['duration'] = @output.scan(/test-duration ([\d.]+)/).flatten.first.to_f
126
+
127
+ # Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
128
+ (stat['min'], stat['avg'], stat['max'], stat['stddev'], stat['samples']) = @output.scan(/Reply rate \[replies\/s\]: min ([\d.]+) avg ([\d.]+) max ([\d.]+) stddev ([\d.]+) \((\d+) samples\)/).flatten.map { |i| i.to_f }
129
+
130
+ # Reply status: 1xx=0 2xx=5 3xx=0 4xx=0 5xx=0
131
+ (stat['1xx'], stat['2xx'], stat['3xx'], stat['4xx'], stat['5xx']) = @output.scan(/Reply status: 1xx=(\d+) 2xx=(\d+) 3xx=(\d+) 4xx=(\d+) 5xx=(\d+)/).flatten.map { |i| i.to_f }
132
+
133
+ stat['avg_low'] = stat['avg'].to_f - 2.0 * stat['stddev'].to_f
134
+ stat['avg_high'] = stat['avg'].to_f + 2.0 * stat['stddev'].to_f
135
+
136
+ stat
137
+ end
138
+
139
+ def sufficient_sample_size?(server, uri)
140
+ @stats[server][uri]['samples'] >= @config['samples'].to_f
141
+ rescue
142
+ false
143
+ end
144
+
145
+ def no_errors?
146
+ # TODO
147
+ true
148
+ end
149
+
150
+ def increase_num_conns(server, uri)
151
+ samples = @stats[server][uri]['samples']
152
+ duration = @stats[server][uri]['duration']
153
+ target_samples = @config['samples']
154
+
155
+ seconds_per_request = (duration / @num_conns.to_f) # 0.02
156
+ adjusted_conns = (target_samples * 5.0) / seconds_per_request # 500
157
+
158
+ # Increase the connections by the factor and a bit more.
159
+ @num_conns = (adjusted_conns * 1.2).to_i
160
+ rescue
161
+ @num_conns = 5
162
+ nil
163
+ end
164
+
165
+ ##
166
+ # Return a string with rounding for display.
167
+ #
168
+ # Small numbers will have a decimal point. Larger numbers will be shown
169
+ # as plain integers.
170
+
171
+ def format_rounded(number)
172
+ if number > 20
173
+ number.to_i
174
+ else
175
+ sprintf('%0.1f', number)
176
+ end
177
+ end
178
+
179
+ def format_string(string, length)
180
+ sprintf "%-#{length + 2}s", string
181
+ end
182
+
183
+ end
metadata ADDED
@@ -0,0 +1,62 @@
1
+ --- !ruby/object:Gem::Specification
2
+ rubygems_version: 0.9.0
3
+ specification_version: 1
4
+ name: bong
5
+ version: !ruby/object:Gem::Version
6
+ version: 0.0.1
7
+ date: 2007-11-07 00:00:00 -08:00
8
+ summary: Website benchmarking helper.
9
+ require_paths:
10
+ - lib
11
+ email: boss@topfunky.com
12
+ homepage: http://rubyforge.org/projects/bong
13
+ rubyforge_project: bong
14
+ description: "== DESCRIPTION: Hit your website with bong. Uses httperf to run a suite of benchmarking tests against specified urls on your site. Graphical output and multi-test comparisons are planned. Apache ab support may be added in the future."
15
+ autorequire:
16
+ default_executable:
17
+ bindir: bin
18
+ has_rdoc: true
19
+ required_ruby_version: !ruby/object:Gem::Version::Requirement
20
+ requirements:
21
+ - - ">"
22
+ - !ruby/object:Gem::Version
23
+ version: 0.0.0
24
+ version:
25
+ platform: ruby
26
+ signing_key:
27
+ cert_chain:
28
+ post_install_message:
29
+ authors:
30
+ - Geoffrey Grosenbach
31
+ files:
32
+ - History.txt
33
+ - Manifest.txt
34
+ - README.txt
35
+ - Rakefile
36
+ - bin/bong
37
+ - lib/bong.rb
38
+ test_files: []
39
+
40
+ rdoc_options:
41
+ - --main
42
+ - README.txt
43
+ extra_rdoc_files:
44
+ - History.txt
45
+ - Manifest.txt
46
+ - README.txt
47
+ executables:
48
+ - bong
49
+ extensions: []
50
+
51
+ requirements: []
52
+
53
+ dependencies:
54
+ - !ruby/object:Gem::Dependency
55
+ name: hoe
56
+ version_requirement:
57
+ version_requirements: !ruby/object:Gem::Version::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: 1.3.0
62
+ version: