stella 0.5.3 → 0.5.4

data/{README.txt → README.textile}

@@ -1,66 +1,86 @@
-Stella - Your Performance Testing Friend
+h1. Stella - Your Performance Testing Friend
 
-Release: 0.5.3-preview (2008-12-23)
+Release: 0.5.4-alpha (2009-01-07)
 
-This is a PREVIEW release. Don't trust and double verify!
+This is a BETA release. Proceed in double verify mode!
 
 
-== Prerequisites
+h2. Prerequisites
 
-* Linux, *BSD, Solaris
-* Ruby 1.8.x or 1.9.x
+* Linux, BSD, Solaris
+* Ruby 1.8.x, 1.9.x, or JRuby 1.1
+* Native Libraries (c or c++) 
+** Pcap (optional, for packet sniffing)
 * Ruby Libraries
-  * fastthread
-	* mongrel
-	* rspec
-	* rdoc
-	
+** rspec
+** net-dns 
+** Ruby-Pcap (optional, for packet sniffing)
+
 * One of:
-  * Apache Bench
-  * Siege
-  * Httperf
+** "Apache Bench":http://httpd.apache.org/docs/2.0/programs/ab.html
+** "Siege":http://www.joedog.org/index/siege-home
+** "Httperf":http://www.hpl.hp.com/research/linux/httperf/
 
 
-== Installation
+h2. Installation
 
 Get it in one of the following ways:
+
 * RubyForge: http://stella.rubyforge.org/
-* gem install stella 
-  
+** gem install stella 
+* git clone git://github.com/solutious/stella.git
+** git submodule init
+** git submodule update
+ 
 Use ab, siege, and httperf like you normally would with the addition of stella at the beginning (examples are below).
 
-=== Debian (and derivatives)
+h3. Debian (and derivatives)
 
 Debian and its derivative (Ubunutu) handling packing a bit differently (see: http://pkg-ruby-extras.alioth.debian.org/rubygems.html). There are a couple errors to watch out for during the installation. The solutions are below:
 
-"no such file to load -- mkmf (LoadError)"
-  
-	apt-get install ruby1.8-dev
+  "no such file to load -- mkmf (LoadError)"
+	$ apt-get install ruby1.8-dev
 
-"ERROR: RDoc documentation generator not installed!"
-  
-	apt-get install rdoc
+  "ERROR: RDoc documentation generator not installed!"
+	$ apt-get install rdoc
 
 
-== Examples
+h3. Installing libpcap and Ruby-Pcap
 
-Run Apache Bench with a warmup and rampup from 100 to 300 virtual users in increments of 25
+*Stella currently supports packet filtering with Native Ruby (no JRuby) on Linux/Unix/OS X only.* 
+*Ruby-Pcap does not compile with Ruby 1.9.1-RC1*
 
-  stella --warmup=0.5 --rampup=25,300 ab -c 100 -n 10000 http://stellaaahhhh.com/search?term=trooper
-    
+libpcap and Ruby-Pcap are used to by Stella's watch command to filter packets off the network. This allows you to view HTTP requests and response (including headers and content) without having to configure a proxy. Both need to be compiled and installed separately:
+* "libpcap":http://www.tcpdump.org/release/
+* "Ruby-Pcap":http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap-e.html
+
+On Ruby, 1.8.6, Ruby-Pcap produces the following warnings at runtime: "pcap.bundle: warning: do not use Fixnums as Symbols". See: support/ruby-pcap-takuma-notes.txt. 
+  
 
-Run Siege, repeat the test 5 times. Automatically creates a summary averages and standard deviations. 
+h2. Usage Examples
 
-  stella --agent=ff-3-osx --testruns=5  siege -c 100 -r 100 -b http://stellaaahhhh.com/search?term=flock+of+seagulls
+  Run Apache Bench with a warmup and rampup from 100 to 300 virtual users in increments of 25
+  $ stella --warmup=0.5 --rampup=25,300 ab -c 100 -n 10000 http://stellaaahhhh.com/search?term=trooper
     
+  Run Siege, repeat the test 5 times. Automatically creates a summary averages and standard deviations. 
+  $ stella --agent=ff-3-osx --testruns=5  siege -c 100 -r 100 -b http://stellaaahhhh.com/search?term=flock+of+seagulls
+    
+  Run Httperf like you normally would (but all the test data will be collected for you)
+  $ stella httperf --hog --client=0/1 --server=127.0.0.1 --port=5600 --uri=/ --rate=50  --num-conns=3000 --timeout=5
 
-Run Httperf like you normally would (but all the test data will be collected for you)
-
-  stella httperf --hog --client=0/1 --server=127.0.0.1 --port=5600 --uri=/ --rate=50  --num-conns=3000 --timeout=5
+  Monitor DNS traffic using Pcap
+  $ stella watch -C dns
+	
+  Monitor HTTP traffic using Pcap
+  $ stella watch -C http
+	
+  Monitor HTTP traffic using a proxy (don't forget to configure your browser)
+  $ stella watch -P
 
 
-== Sample Output
+h3. Sample Output
 
+<pre><code>
   $ stella -f csv -x 5 -w 0.75 -r 25,125 -m "httpd 2.2.9-prefork" siege -c 75 -r 10 -b http://stella:5600/
 	Writing test data to: stella/testruns/2008-12-23/test-054
 
@@ -93,7 +113,7 @@ Run Httperf like you normally would (but all the test data will be collected for
 	 -------------------------------------------------------------------
 	   Total:      90000@98     100%    378.58/s    0.259s    0.035MB/s    8.250MB  236.030s
 	 Std Dev:                            18.09/s    0.042s    0.002MB/s               4.225s
-
+</pre></code>
 
 All test data is collected under ./stella (this can be changed with the parameter --datapath):
 
@@ -113,23 +133,26 @@ Each run directory contains all associated data, including the command and confi
     COMMAND.txt    STDOUT.txt    siege.log    STDERR.txt    SUMMARY.csv    siegerc
     
   
-== Known Issues
+h2. Known Issues
 
 * The output for the REQ@VU/s columns is a work in progress. It's not aligned across tools and it will likely change in the next release. 
 * The summary data has not been audited. Don't trust and double verify!
 * httperf is functional but needs a lot more testing (most dev was done with ab and siege).
 * The Ruby API has not been finalized. It's functional but there's no example because it is subject to change. 
 * There are no specs. 
+* There is an issue with parsing ipv6 ip addresses with Ruby-Pcap or Net-DNS. This affects only dns sniffing with pcap. Reproduce by running "stella watch -P dns" and requesting "apple.com"
 
-== Report an issue
+
+h2. Report an issue
 
 Email issues and bugs to stella@solutious.com
 
 
-== Even More Information
+h2. More Information
+
+"Stellaaahhhh":http://www.youtube.com/watch?v=wmq-JDonTpc
 
-http://www.youtube.com/watch?v=wmq-JDonTpc
 
-== License
+h2. License
 
 See LICENSE.txt

data/Rakefile

@@ -42,17 +42,19 @@ name = "stella"
 spec = Gem::Specification.new do |s|
 	s.name = name
 	s.version = version
-	s.summary = "A friend in performance testing."
+	s.summary = "Your friend in performance testing."
 	s.description = "Run Apache Bench, Siege, or httperf tests in batches and aggregate results."
 	s.author = "Delano Mandelbaum"
 	s.email = "delano@solutious.com"
 	s.homepage = "http://stella.solutious.com/"
-	s.executables = [ "stella" ]
+	s.executables = [ "stella", "stella.bat" ]
 	s.rubyforge_project = "stella"
-  s.extra_rdoc_files  = ['README.txt']
-
-	s.add_dependency 'mongrel'
+  s.extra_rdoc_files  = ['README.textile']
+  
+  # NOTE: how to make optional dependencies?
+	#s.add_dependency 'mongrel'
 	s.add_dependency 'rspec'
+	s.add_dependency 'net-dns'
 	#s.add_dependency 'session'
 
 	s.platform = Gem::Platform::RUBY

data/bin/stella

@@ -1,7 +1,7 @@
 #!/usr/bin/env ruby 
 
 STELLA_HOME = File.expand_path(File.join(File.dirname(__FILE__), '..'))
-$: << File.join(STELLA_HOME, 'lib')
+$:.unshift(File.join(STELLA_HOME, 'lib')) # Make sure our local lib is first in line
 
 require 'stella'
 require 'stella/cli'

data/bin/stella.bat

@@ -0,0 +1,12 @@
+@echo off
+
+rem Check for funkiness when called from another batch script.
+rem We want FULL_PATH to contain the full path to the stella bin directory. 
+IF EXIST "%~dp0stella.bat" (set FULL_PATH=%~dp0) ELSE (set FULL_PATH=%~dp$PATH:0)
+
+rem Check for JRuby, otherwise use Ruby.
+rem We want EXECUTABLE to contain either "jruby" or "ruby"
+IF EXIST "%JRUBY_HOME%" (set EXECUTABLE=jruby) ELSE (set EXECUTABLE=ruby)
+
+rem Call the Ruby script, passing it all the arguments.
+@%EXECUTABLE% %FULL_PATH%stella %*

data/lib/pcaplet.rb

@@ -0,0 +1,180 @@
+require 'pcap'
+
+
+# Pcaplet
+#
+# Adapted from Ruby portion of Ruby-Pcap:
+# http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap-e.html
+# The lib/pcaplet.rb and lib/pcap_misc.rb files are in merge into this file
+# and cleaned up. The Ruby-Pcap C extension was modified to apply several fixes
+# so it would compile on OS X and to remove several warning messages. With 
+# help from: http://d.hatena.ne.jp/takuma104/20080210/1202638583  
+# We specifically removed the dependency on ARGV and OptParse. It was messy
+# and required ARGV to be specifically modified before requiring this package. 
+# Manual: http://www.goto.info.waseda.ac.jp/~fukusima/ruby/pcap/doc/index.html
+class Pcaplet
+
+  attr_accessor :debug, :verbose
+    # do not convert address to name
+  attr_accessor :convert
+  attr_accessor :device, :rfile, :count, :snaplen, :filter
+
+  def initialize(args = {})
+  
+    @debug   = args[:debug]   || false
+    @verbose = args[:verbose] || false
+    @device = args[:device] || guess_device
+    @rfile = args[:rfile]
+    @convert = args[:convert] || false
+    @count   = args[:count].to_i || 1000
+    @snaplen = args[:snaplen].to_i > 0 ? args[:snaplen] : 1500
+    @filter = args[:filter] || ''
+  
+    Pcap.convert = @convert
+    
+    # check option consistency
+    usage(1) if @device && @rfile
+    if !@device and !@rfile
+      @device = Pcap.lookupdev
+    end
+
+    # open
+    begin
+      if @device
+        @capture = Pcap::Capture.open_live(@device, @snaplen)
+      elsif @rfile
+        if @rfile !~ /\.gz$/
+          @capture = Capture.open_offline(@rfile)
+        else
+          $stdin = IO.popen("gzip -dc < #@rfile", 'r')
+          @capture = Capture.open_offline('-')
+        end
+      end
+      @capture.setfilter(@filter) if @filter
+      rescue Pcap::PcapError, ArgumentError
+        $stdout.flush
+        $stderr.puts $!
+        exit(1)
+      end
+    end
+
+  attr('capture')
+
+
+  def add_filter(f)
+    if @filter == nil || @filter =~ /^\s*$/  # if empty
+      #puts "filter is nil or empty"
+      @filter = f
+    else
+      #puts "filter is #{@filter}"
+      f = f.source if f.is_a? Filter
+      @filter = "( #{@filter} ) and ( #{f} )"
+    end
+    #puts "filter being set #{@filter}"
+    @capture.setfilter(@filter)
+  end
+
+  def each_packet(&block)
+    
+    begin
+      duplicated = (RUBY_PLATFORM =~ /linux/ && @device == "lo")
+      unless duplicated
+        @capture.loop(@count, &block)
+      else
+        flip = true
+        @capture.loop(@count) do |pkt|
+          flip = (! flip)
+          next if flip
+          block.call pkt
+        end
+      end
+    end
+  end
+
+  alias each each_packet
+
+  def close
+    @capture.close
+  end
+end
+
+
+
+module Pcap
+  class Packet
+    def to_s
+      'Some packet'
+    end
+
+    def inspect
+      "#<#{type}: #{self}>"
+    end
+  end
+
+  class IPPacket
+    def to_s
+      "#{ip_src} > #{ip_dst}"
+    end
+  end
+
+  class TCPPacket
+    def tcp_data_len
+      ip_len - 4 * (ip_hlen + tcp_hlen)
+    end
+    
+    # The 3-way handshake that initiates a request
+    # ....S. --->
+    # .A..S. <---
+    # .A.... --->
+    #
+    # The teardown procedure at the end of the request
+    # .A...F --->
+    # .A.... <---
+    # .A...F --->
+    # .A.... <...
+    def tcp_flags_s
+      return \
+	      (tcp_urg? ? 'U' : '.') +  # Urgent
+	      (tcp_ack? ? 'A' : '.') +  # ACKnowledgement: Successful transfer
+	      (tcp_psh? ? 'P' : '.') +  # Push
+	      (tcp_rst? ? 'R' : '.') +  # Reset
+	      (tcp_syn? ? 'S' : '.') +  # SYNchronization: this is the first segment in a new transaction
+        (tcp_fin? ? 'F' : '.')    # FINal: final transaction
+    end
+
+    def to_s
+      "#{src}:#{sport} > #{dst}:#{dport} #{tcp_flags_s}"
+    end
+  end
+
+  class UDPPacket
+    def to_s
+      "#{src}:#{sport} > #{dst}:#{dport} len #{udp_len} sum #{udp_sum}"
+    end
+  end
+
+  class ICMPPacket
+    def to_s
+      "#{src} > #{dst}: icmp: #{icmp_typestr}"
+    end
+  end
+
+  #
+  # Backword compatibility
+  #
+  IpPacket = IPPacket unless defined? IpPacket
+  IpAddress = IPAddress unless defined? IpAddress
+  TcpPacket = TCPPacket unless defined? TcpPacket
+  UdpPacket = UDPPacket unless defined? UdpPacket
+
+end
+
+class Time
+  # tcpdump style format
+  def tcpdump
+    sprintf "%0.2d:%0.2d:%0.2d.%0.6d", hour, min, sec, tv_usec
+  end
+end
+
+
+

data/lib/stella/adapter/ab.rb

@@ -40,8 +40,8 @@ module Stella
     #    -f protocol     Specify SSL/TLS protocol (SSL2, SSL3, TLS1, or ALL)
     class ApacheBench < Stella::Adapter::Base
       
-      
-      attr_accessor :n, :c, :t, :b, :p, :T, :v, :w, :i, :x, :z, :y
+      attr_writer :n, :c
+      attr_accessor :t, :b, :p, :T, :v, :w, :i, :x, :z, :y
       attr_accessor :C, :H, :A, :P, :X, :V, :k, :d, :S, :e, :g, :r, :h, :Z, :f
       
       def initialize(options={}, arguments=[])
@@ -53,13 +53,14 @@ module Stella
         @load_factor = 1
       end
       
-
+      def error
+        (File.exists? stderr_path) ? FileUtil.read_file(stderr_path) : "Unknown error"
+      end
+      
       def version
         vsn = 0
-        text = ""
-        Open3.popen3("#{@name} -V") do |stdin, stdout, stderr| 
-          text = stdout.readlines.join
-          text.scan(/Version (\d+?\.\d+)/) { |v| vsn = v[0] }
+        Stella::Util.capture_output("#{@name} -V") do |stdout, stderr|
+           stdout.join.scan(/Version (\d+?\.\d+)/) { |v| vsn = v[0] }
         end
         vsn
       end
@@ -73,13 +74,14 @@ module Stella
         @g = File.expand_path(@g)
 
       end
+      
       def command
         raise CommandNotReady.new(self.class.to_s) unless ready?
 
         command = "#{@name} "
 
         instance_variables.each do |name|
-          canon = name.tr('@', '')        # instance_variables returns '@name'
+          canon = name.to_s.tr('@', '')        # instance_variables returns '@name'
           next if @private_variables.member?(canon)
 
           # It's important that we take the value from the getter method
@@ -93,7 +95,7 @@ module Stella
 
         end
 
-        command << (@arguments.map { |uri| "'#{uri}'" }).join(' ') unless @arguments.empty?
+        command << (@arguments.map { |uri| "#{uri}" }).join(' ') unless @arguments.empty?
         command
       end
       # loadtest
@@ -112,36 +114,58 @@ module Stella
       def process_options(arguments)
         options = OpenStruct.new
         opts = OptionParser.new 
-        opts.on('-v') do |v| options.v = true end
-        opts.on('-w') do |v| options.w = true end # TODO: Print a note that we don't parse the HTML results
-        opts.on('-i') do |v| options.i = true end
-        opts.on('-V') do |v| options.V = true end
-        opts.on('-k') do |v| options.k = true end
-        opts.on('-d') do |v| options.d = true end
-        opts.on('-S') do |v| options.S = true end
-        opts.on('-r') do |v| options.r = true end
-        opts.on('-h') do |v| options.h = true end
-        opts.on('-e S', String) do |v| options.e = v end
-        opts.on('-g S', String) do |v| options.g = v end
-        opts.on('-p S', String) do |v| options.p = v end
-        opts.on('-T S', String) do |v| options.t = v end
-        opts.on('-x S', String) do |v| options.x = v end
-        opts.on('-y S', String) do |v| options.y = v end
-        opts.on('-z S', String) do |v| options.z = v end
-        opts.on('-P S', String) do |v| options.P = v end
-        opts.on('-Z S', String) do |v| options.Z = v end
-        opts.on('-f S', String) do |v| options.f = v end
-        opts.on('-c N', Integer) do |v| options.c = v end
-        opts.on('-n N', Integer) do |v| options.n = v end
-        opts.on('-t N', Integer) do |v| options.t = v end
-        opts.on('-b N', Integer) do |v| options.b = v end
+        
+        # TODO: Print a note for w that we don't parse the HTML results
+        %w{v w i V k d S r h}.each do |n|
+          opts.on("-#{n}") do |v| options.send("#{n}=", true) end
+        end
+        
+        %w{e g p T x y z P Z f A}.each do |n|
+          opts.on("-#{n} S", String) do |v| options.send("#{n}=", v)  end
+        end
+        
+        %w{c n t b}.each do |n|
+          opts.on("-#{n} S", Integer) do |v| options.send("#{n}=", v) end
+        end
+        
         opts.on('-H S', String) do |v| options.H ||= []; options.H << v; end
         opts.on('-C S', String) do |v| options.C ||= []; options.C << v; end
         
+        opts.on('-b') do |v| 
+          Stella::LOGGER.warn("-b is not an ab option. I'll pretend it's not there.")
+        end
+        
+        opts.on('-r N',Integer) do |v| 
+          Stella::LOGGER.error("-r is not an ab parameter. You probably want -n.")
+          exit 1
+        end
+        
         # NOTE: parse! removes the options it finds in @arguments. It will leave
         # all unnamed arguments and throw a fit about unknown ones. 
         opts.parse!(arguments)
         
+        if arguments.empty?
+          Stella::LOGGER.error("You need to provide a URI")
+          exit 1
+        elsif arguments.size > 1
+          Stella::LOGGER.warn("ab can handle only one URI. The others will be ignored.")
+          arguments = arguments.first
+        else
+          # Let's make sure the URI has a path (at least a trailing slash). Otherwise
+          # ab gives a cryptic error. 
+          begin
+            uri = URI.parse(arguments.first)
+            if !uri || uri.path.empty?
+              Stella::LOGGER.error("ab requires a trailing slash for #{uri.to_s}") 
+              exit 1
+            end
+          rescue => ex
+            Stella::LOGGER.error("Bad URI: #{arguments.first}") 
+            exit 1
+          end
+        end
+        
+        
         options
       rescue OptionParser::InvalidOption => ex
         # We want to replace this text so we grab just the name of the argument
@@ -234,7 +258,7 @@ module Stella
       
       # Apache bench writes the summary to STDOUT
       def stats_file
-        File.new(stdout_path)
+        File.new(stdout_path) if File.exists?(stdout_path)
       end
 
       def stats

data/lib/stella/adapter/base.rb

@@ -10,7 +10,8 @@ module Stella::Adapter
   class Base
     
     
-    attr_accessor :arguments, :load_factor, :working_directory, :stats
+    attr_accessor :working_directory, :stats
+    attr_reader :load_factor, :arguments
     
     def initialize(options={}, arguments=[])
       self.arguments = arguments
@@ -36,10 +37,16 @@ module Stella::Adapter
     #
     # This method must be overridden by the implementing class. This is intended
     # for processing the command-specific command-line arguments
+    # TODO: Rename to process_arguments
     def process_options
       raise Stella::TEXT.msg(:error_class_must_override, 'process_options')
     end
     
+    # options=
+    #
+    # Takes a hash or OpenStruct and applies the values to the instance variables. 
+    # The keys should conincide with with the command line argument names. 
+    # i.e. The key for --help should be :help 
     def options=(options={})
       options = options.marshal_dump if options.is_a? OpenStruct
       unless options.nil? || options.empty?
@@ -82,6 +89,9 @@ module Stella::Adapter
       raise Stella::TEXT.msg(:error_class_must_override, 'user_agent=')
     end
     
+    private
+    
+    
   end
 end
   

data/lib/stella/adapter/httperf.rb

@@ -21,11 +21,12 @@ module Stella
       
       
       attr_accessor :hog, :server, :uri, :num_conns, :num_calls, :rate, :timeout, :think_timeout, :port
-      attr_accessor :add_header, :burst_length, :client, :close_with_reset, :debug, :failure_status
+      attr_accessor :burst_length, :client, :close_with_reset, :debug, :failure_status
       attr_accessor :help, :http_version, :max_connections, :max_piped_calls, :method, :no_host_hdr
       attr_accessor :period, :print_reply, :print_request, :recv_buffer, :retry_on_failure, :send_buffer
-      attr_accessor :server_name, :session_cookies, :ssl, :ssl_ciphers, :ssl_no_reuse, :verbose
-      attr_accessor :version, :wlog, :wsess, :wsesslog, :wset
+      attr_accessor :server_name, :session_cookies, :ssl, :ssl_ciphers, :ssl_no_reuse, :verbose 
+      
+      attr_writer :version, :add_header, :wlog, :wsess, :wsesslog, :wset
       
       def initialize(options={}, arguments=[])
         super(options, arguments)
@@ -36,7 +37,11 @@ module Stella
       end
       
       
-
+      
+      def error
+        (File.exists? stderr_path) ? FileUtil.read_file(stderr_path) : "Unknown error"
+      end
+      
       # Before calling run
       def before
 
@@ -141,10 +146,8 @@ module Stella
       
       def version
         vsn = 0
-        text = ""
-        Open3.popen3("#{@name} --version") do |stdin, stdout, stderr| 
-          text = stdout.readlines.join
-          text.scan(/httperf\-([\d\.]+)\s/) { |v| vsn = v[0] }
+        Stella::Util.capture_output("#{@name} --version") do |stdout, stderr|
+           stdout.join.scan(/httperf\-([\d\.]+)\s/) { |v| vsn = v[0] }
         end
         vsn
       end
@@ -222,11 +225,11 @@ module Stella
 
       # Siege writes the summary to STDERR
       def stats_file
-        File.new(stdout_path)
+        File.new(stdout_path) if File.exists?(stdout_path)
       end
       
       def rc_file
-        File.join(@working_directory, "siegerc")
+        File.join(@working_directory, "siegerc") 
       end
       
       def log_file