ebb 0.0.4 → 0.1.0

data/README

@@ -1,91 +1,82 @@
 # A Web Server Called *Ebb*
 
 Ebb aims to be a small and fast web server specifically for hosting 
-web frameworks like Rails, Merb, and in the future Django.
+dynamic web applications. It is not meant to be a full featured web server
+like Lighttpd, Apache, or Nginx. Rather it should be used in multiplicity
+behind a load balancer and a front-end server. It is not meant to serve static files in production.
 
-It is not meant to be a full featured web server like Lighttpd, Apache, or
-Nginx. Rather it should be used in multiplicity behind a 
-[load balancer](http://brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel) 
-and a front-end server. It is not meant to serve static files in production.
+At one level Ebb is a minimalist C library that ties together the 
+[Mongrel state machine](http://mongrel.rubyforge.org/browser/tags/rel_1-0-1/ext/http11/http11_parser.rl) 
+and [libev](http://software.schmorp.de/pkg/libev.html) event loop. One can use
+this library to drive a web application written in C. (Perhaps for embedded 
+devices?) However, most people will be interested in the binding of this
+library to the Ruby programming language. The binding provides a
+[Rack](http://rack.rubyforge.org/) server interface that allows it to host
+Rails, Merb, or other frameworks.
 
-## Design
+A Python-WSGI binding is under development.
 
-The design is similar to the [Evented
-Mongrel](http://swiftiply.swiftcore.org/mongrel.html) web server; except
-instead of using [EventMachine](http://rubyeventmachine.com/) to drive
-network interactions, the Ebb web server handles sockets directly in C and
-employs the use of the [libev](http://software.schmorp.de/pkg/libev.html)
-event loop.
-
-Connections are processed as follows:
-
-1. libev loops and waits for incoming connections.
-
-2. When Ebb receives a connection, it passes the request into the
-   [mongrel state machine](http://mongrel.rubyforge.org/browser/tags/rel_1-0-1/ext/http11/http11_parser.rl) 
-   which securely parses the headers.
-
-3. When the request is complete, Ebb passes the information to a user
-   supplied callback.
-
-4. The Ruby binding supplying this callback transforms the
-   request into a [Rack](http://rack.rubyforge.org/) compatible `env` hash 
-   and passes it on a Rack adapter.
-
-Because Ebb is written mostly in C, other language bindings can be added to
-make it useful to Non-Ruby frameworks. For example, a Python WSGI interface is
-forthcoming.
-
-## Download
+## Install
 
 The Ruby binding is available as a Ruby Gem. It can be install by executing
 
-  `gem install ebb`
+    gem install ebb
 
-Ebb depends on having glib2 headers and libraries installed. (Easily available
-on any UNIX system.) Manual download can be done at
+Ebb depends on having glib2 headers and libraries installed. For example, in
+Macintosh if one is using Darwin ports then the following should do the trick
+  
+    port install glib2
+  
+Downloads are available at
 the [RubyForge project page](http://rubyforge.org/frs/?group_id=5640).
 
-## Why?
+## Running
 
-Because by building the server in C one is able to side-step the 
-limitations on speed of many scripting languages. Inefficiencies are okay
-for quick and beautiful code, but for production web servers that might handle
-thousands of requests a second, an attempt should be made to be as efficient
-as possible in processing connections.
+Using the executable `ebb_rails` one can start Ebb with a Rails project. Use
+`ebb_rails -h` to see all of the options but to start one can try
 
-Following are some benchmarks. Please take these measurements with a grain
-of salt. Benchmarks like these are notorious for presenting an inaccurate
-or highly slanted view of how software performs. 
-The code for these can be found in the `benchmark` directory.
+    cd my_rails_project/
+    ebb_rails start
 
-![Response Size](http://s3.amazonaws.com/four.livejournal/20080227/response_size.png)
+When using `ebb_rails` from monit, the monitrc entry might look like this:
 
-This shows how the web servers perform with respect to throughput (using a 
-simple Rack application). Concurrency is at 50 clients.
+    check process myApp4000
+      with pidfile /home/webuser/myApp/current/tmp/ebb.4000.pid
+      start program = "/usr/bin/ruby /usr/bin/ebb_rails start -d -e production -p 4000 -P /home/webuser/myApp/current/tmp/ebb.4000.pid -c /home/webuser/myApp/current" as uid webuser and gid webuser
+      stop program = "/usr/bin/ruby /usr/bin/ebb_rails stop -P /home/webuser/myApp/current/tmp/ebb.4000.pid" as uid webuser and gid webuser
+      if totalmem > 120.0 MB for 2 cycles then restart
+      if loadavg(5min) greater than 10 for 8 cycles then restart
+      group myApp
 
-![Concurrency](http://s3.amazonaws.com/four.livejournal/20080227/concurrency.png)
+To use Ebb with a different framework you will have to do a small amount of
+hacking at the moment! :)
 
-A simple concurrent clients benchmark serving a *hello world* page.
+## Speed
 
-![Uploads](http://s3.amazonaws.com/four.livejournal/20080227/post_size.png)
+Because Ebb-Ruby handles most of the processing in C, it is able to do work
+often times more efficiently than other Ruby language web servers.
 
-Ebb processes uploads before handing it over to the web application. This 
-allows Ebb to continue to process other clients while the upload is in 
-progress. The cliff at 40k here is because Ebb's internal request
-buffer is set at 40 kilobytes before it writes to file.
+![Benchmark](http://s3.amazonaws.com/four.livejournal/20080311/ebb.png)
+
+Ebb-Ruby can handle threaded processing better than the other 'evented' 
+servers. This won't be of any benefit to Rails applications because Rails
+places a lock around each request that wouldn't allow concurrent processing
+anyway. In Merb, for example, Ebb's thread handling will allow Ebb instances
+to handle larger loads. [More](http://four.livejournal.com/848525.html)
 
 ## Contributions
 
-Contributions (patches, criticism, advice) are very welcome! The source code
-is hosted at [repo.or.cz](http://repo.or.cz/w/ebb.git). It can be retrieved 
+Contributions (patches, criticism, advice) are very welcome! 
+Please send all to to 
+[the mailing list](http://groups.google.com/group/ebbebb).
+
+The source code
+is hosted [github](http://github.com/ry/ebb/tree/master). It can be retrieved 
 by executing
 
-`git clone http://repo.or.cz/r/ebb.git`
+    git clone git://github.com/ry/ebb.git
 
-I intend to keep the C code base very small, so do email me before writing any
-large additions. Here are some features that I would like to add:
-* Streaming responses!
+Here are some features that I would like to add:
 * HTTP 1.1 Expect/Continue (RFC 2616, sections 8.2.3 and 10.1.1)
 * A parser for multipart/form-data
 * Optimize and clean up upload handling

data/benchmark/application.rb

@@ -35,7 +35,7 @@ class SimpleApp
       status = 200
     
     elsif commands.include?('wait')
-      n = commands.last.to_i
+      n = commands.last.to_f
       raise "wait called with n <= 0" if n <= 0
       wait(n)
       body = "waited about #{n} seconds"
@@ -74,11 +74,5 @@ end
 
 if $0 == __FILE__
   require DIR + '/../ruby_lib/ebb'
-  require 'rubygems'
-  require 'ruby-debug'
-  Debugger.start
-  
-  server = Ebb::Server.new(SimpleApp.new, :port => 4001)
-  puts "Ebb started on http://0.0.0.0:4001/"
-  server.start
-end
+  server = Ebb::start_server(SimpleApp.new, :port => 4001)
+end

data/benchmark/bench_results.rb

@@ -4,29 +4,40 @@ require 'google_chart'
 require 'server_test'
 
 class Array
-  def avg
-    sum.to_f / length
+  def max
+    inject(first) { |m, i| i > m ? i : m } 
   end
-  def sum
-    inject(0) { |i, s| s += i }
+  
+  def min
+    inject(first) { |m, i| i < m ? i : m }
   end
 end
 
 
 
-colors = %w{F74343 444130 7DA478 E4AC3D}
-max_x = 0
-max_y = 0
+colors = %w{F74343 444130 7DA478 E4AC3D 1F479E}
+data_x = []
+data_y = []
 results = ServerTestResults.open(ARGV[0])
-all_m = []
-response_chart = GoogleChart::LineChart.new('400x300', Time.now.strftime('%Y.%m.%d'), true)
-results.servers.each do |server|
+
+response_chart = GoogleChart::LineChart.new('500x300', Time.now.strftime('%Y.%m.%d'), true)
+servers = results.servers.sort_by do |x,y|
+  results.data(x).map { |d| d[1] }.mean
+end.reverse
+
+cmap = {}
+results.servers.sort.each { |x| cmap[x] = colors.shift }
+
+servers.each do |server|
+  data = results.data(server).sort
+  data_x += data.map { |d| d[0] }
+  data_y += data.map { |d| d[1] }
+end
+
+servers.each do |server|
   data = results.data(server).sort
-  response_chart.data(server, data, colors.shift)
-  x = data.map { |d| d[0] }.max
-  y = data.map { |d| d[1] }.max
-  max_x = x if x > max_x
-  max_y = y if y > max_y
+  data.map! { |d| [d[0]-data_x.min, d[1]-data_y.min]}
+  response_chart.data(server, data, cmap[server])
 end
 
 label = case results.benchmark
@@ -36,10 +47,12 @@ when "wait_fib", "concurrency"
   "concurrency"
 when "post_size"
   "kilobytes uploaded"
+when "wait", "wait_fib"
+  "seconds waited every 10 requests"
 end
 
-response_chart.axis(:y, :range => [0,max_y])
+response_chart.axis(:y, :range => [data_y.min,data_y.max])
 response_chart.axis(:y, :labels => ['req/s'], :positions => [50])
-response_chart.axis(:x, :range => [0,max_x])
+response_chart.axis(:x, :range => [data_x.min,data_x.max])
 response_chart.axis(:x, :labels => [label], :positions => [50])
 puts response_chart.to_url

data/benchmark/server_test.rb

@@ -6,12 +6,35 @@ require 'application'
 
 
 class Array
-  def avg
-    sum.to_f / length
+  def mean
+    @mean ||= sum / length
+  end
+  
+  def variance
+    @variance ||= map { |x| (mean - x)**2 }.sum / length
+  end
+  
+  def stdd
+    @stdd ||= Math.sqrt(variance)
+  end
+  
+  def errors
+    map { |x| (mean - x).abs }
+  end
+  
+  def center_avg
+    return mean if stdd < 10
+    acceptable_error = 0
+    good_points = []
+    while good_points.empty?
+      acceptable_error += stdd
+      good_points = select { |x| (mean - x).abs < acceptable_error }
+    end
+    good_points.mean
   end
   
   def sum
-    inject(0) { |i, s| s += i }
+    inject(0.to_f) { |i, s| s += i }
   end
   
   def rand_each(&block)
@@ -27,7 +50,7 @@ class ServerTestResults
       new
     end
   end
-
+  attr_reader :results
   def initialize(results = [])
     @results = results
   end
@@ -61,7 +84,7 @@ class ServerTestResults
     datas = []
     ticks.each do |c|
       measurements = server_data.find_all { |d| d[:input] == c }.map { |d| d[:rps] }
-      datas << [c, measurements.avg]
+      datas << [c, measurements.mean] unless measurements.empty?
     end
     datas
   end
@@ -80,7 +103,7 @@ class ServerTest
   end
   
   def kill
-    Process.kill('KILL', @pid)
+    Process.kill('KILL', @pid) if @pid
   end
   
   def running?
@@ -92,12 +115,16 @@ class ServerTest
     case name
     when 'emongrel'
       @pid = fork { start_emongrel }
-    when 'ebb'
-      @pid = fork { start_ebb }
+    when 'ebb_threaded'
+      @pid = fork { start_ebb_threaded }
+    when 'ebb_sequential'
+      @pid = fork { start_ebb_sequential }
     when 'mongrel'
       @pid = fork { start_mongrel }
     when 'thin'
       @pid = fork { start_thin }
+    when 'fcgi'
+      @pid = fork { start_fcgi }
     end
   end
   
@@ -112,9 +139,14 @@ class ServerTest
     Rack::Handler::Mongrel.run(app, :Host => '0.0.0.0', :Port => @port.to_i)
   end
   
-  def start_ebb
+  def start_ebb_threaded
     require File.dirname(__FILE__) + '/../ruby_lib/ebb'
-    server = Ebb::Server.run(app, :port => @port)
+    server = Ebb::start_server(app, :port => @port, :threaded_processing => true)
+  end
+
+  def start_ebb_sequential
+    require File.dirname(__FILE__) + '/../ruby_lib/ebb'
+    server = Ebb::start_server(app, :port => @port, :threaded_processing => false)
   end
   
   def start_mongrel
@@ -128,6 +160,10 @@ class ServerTest
     Rack::Handler::Thin.run(app, :Port => @port)
   end
   
+  def start_fcgi
+    Rack::Handler::FastCGI.run(app, :Port => @port)
+  end
+  
   def trial(ab_cmd)
     cmd = ab_cmd.sub('PORT', @port.to_s)
     
@@ -140,13 +176,19 @@ class ServerTest
     if r =~ /Complete requests:\s*(\d+)/
       requests_completed = $1.to_i
     end
-    puts "   #{rps} req/sec (#{requests_completed} completed)"
+    if r =~ /Failed requests:\s*(\d+)/     
+      failed_requests =  $1.to_i
+    else
+      raise "didn't get how many failed requests from ab"
+    end
+    puts "   #{rps} req/sec (#{requests_completed} completed, #{failed_requests} failed)"
     
     {
       :server => @name,
       :rps => rps,
       :requests_completed => requests_completed,
+      :requests_failed => failed_requests,
       :ab_cmd => cmd
     }
   end
-end
+end

data/bin/ebb_rails

@@ -1,79 +1,4 @@
 #!/usr/bin/env ruby
 require File.dirname(__FILE__) + '/../ruby_lib/ebb'
-require 'optparse'
-require 'rubygems'
-require 'rack'
 
-module Rack
-  module Adapter
-    autoload :Rails, Ebb::LIBDIR + '/rack/adapter/rails'
-  end
-end
-
-options = {
-  :root     => Dir.pwd,
-  :env      => 'development',
-  :hort     => '0.0.0.0',
-  :port     => 3000,
-  :timeout  => 60
-}
-
-
-opts = OptionParser.new do |opts|
-  opts.banner = "Usage: ebb_rails [options] start|stop"
-
-  opts.separator ""
-  opts.separator "Server options:"
-
-#  opts.on("-s", "--socket SOCKET", "listen on socket")                   { |socket| options[:socket] = socket }
-  opts.on("-p", "--port PORT", "use PORT (default: 3000)")               { |port| options[:port] = port }
-  opts.on("-e", "--env ENV", "Rails environment (default: development)") { |env| options[:env] = env }
-  opts.on("-c", "--chdir PATH", "Rails root dir (default: current dir)") { |dir| options[:root] = dir }
-  opts.on("-d", "--daemonize", "Daemonize")                              { options[:daemonize] = true }
-  opts.on("-l", "--log-file FILE", "File to redirect output")            { |file| options[:log_file] = file }
-  opts.on("-P", "--pid-file FILE", "File to store PID")                  { |file| options[:pid_file] = file }
-  opts.on("-t", "--timeout SEC", "Request or command timeout in sec",    
-                                 "(default: #{options[:timeout]})")      { |sec| options[:timeout] = sec }
- 
-  opts.separator ""
-  opts.separator "Common options:"
-
-  opts.on_tail("-h", "--help", "Show this message") do
-    puts opts
-    exit
-  end
-
-  opts.on_tail('-v', '--version', "Show version") do
-    puts "Ebb #{Ebb::VERSION}"
-    exit
-  end
-
-  opts.parse! ARGV
-end
-
-case ARGV[0]
-  
-when 'start'
-  app = Rack::Adapter::Rails.new(options)
-  server = Ebb::Server.new(app, options)
-  
-  server.pid_file = options[:pid_file]
-  server.log_file = options[:log_file]
-  server.timeout  = options[:timeout]
-  
-  server.daemonize if options[:daemonize]
-  
-  server.start
-
-when 'stop'
-  Ebb::Server.kill options[:pid_file], options[:timeout]
-
-when nil
-  puts "Command required"
-  puts opts
-  exit 1
-  
-else
-  abort "Invalid command : #{ARGV[0]}"
-  
-end
+Ebb::Runner::Rails.new.run(ARGV)