goliath 0.9.0

data/.gitignore

@@ -0,0 +1,15 @@
+Makefile
+mkmf.log
+*.o
+*.bundle
+doc/
+pkg/
+examples/log
+*.swp
+Gemfile.lock
+.yardoc
+.livereload
+*.watchr
+*.rbc
+.rvmrc
+_site

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--format progress

data/.yardopts

@@ -0,0 +1,2 @@
+--main README.md
+--no-private

data/Gemfile

@@ -0,0 +1,3 @@
+source "http://rubygems.org"
+
+gemspec

data/LICENSE

@@ -0,0 +1,66 @@
+Copyright (c) 2011 PostRank Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to
+deal in the Software without restriction, including without limitation the
+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+
+
+Portions of this code are from the thin project (https://github.com/macournoyer/thin) and under the following license:
+Copyright (c) 2008 Marc-Andre Cournoyer
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to
+deal in the Software without restriction, including without limitation the
+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+
+
+Portions of this code are from the Sinatra project (https://github.com/bmizerany/sinatra) and under the following license:
+Copyright (c) 2007, 2008, 2009 Blake Mizerany
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,86 @@
+# Goliath
+
+Goliath is an open source version of the non-blocking (asynchronous) Ruby web server framework powering PostRank. It is a lightweight framework designed to meet the following goals: bare metal performance, Rack API and middleware support, simple configuration, fully asynchronous processing, and readable and maintainable code (read: no callbacks).
+
+The framework is powered by an EventMachine reactor, a high-performance HTTP parser and Ruby 1.9 runtime. The one major advantage Goliath has over other asynchronous frameworks is the fact that by leveraging Ruby fibers introduced in Ruby 1.9+, it can untangle the complicated callback-based code into a format we are all familiar and comfortable with: linear execution, which leads to more maintainable and  readable code.
+
+Each HTTP request within Goliath is executed in its own Ruby fiber and all asynchronous I/O operations can transparently suspend and later resume the processing without requiring the developer to write any additional code. Both request processing and response processing can be done in fully asynchronous fashion: streaming uploads, firehose API's, request/response, and so on.
+
+## Installation & Prerequisites
+
+* Install Ruby 1.9 (via RVM or natively)
+
+        $> gem install rvm
+        $> rvm install 1.9.2
+        $> rvm use 1.9.2
+
+* Install Goliath:
+
+        $> gem install goliath
+
+## Getting Started: Hello World
+
+    require 'goliath'
+
+    class Hello < Goliath::API
+      # reload code on every request in dev environment
+      use ::Rack::Reloader, 0 if Goliath.dev?
+
+      def response(env)
+        [200, {}, "Hello World"]
+      end
+    end
+
+    > ruby hello.rb -sv
+    > [97570:INFO] 2011-02-15 00:33:51 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.
+
+See examples directory for more, hands-on examples of building Goliath powered web-services.
+
+## Performance: MRI, JRuby, Rubinius
+
+Goliath is not tied to a single Ruby runtime - it is able to run on MRI Ruby, JRuby and Rubinius today. Depending on which platform you are working with, you will see different performance characteristics. At the moment, we recommend MRI Ruby 1.9.2 as the best performing VM: a roundtrip through the full Goliath stack on MRI 1.9.2p136 takes ~0.33ms (~3000 req/s).
+
+JRuby performance (with 1.9 mode enabled) is currently much worse than MRI Ruby 1.9.2, due to the fact that all JRuby fibers are mapped to native Java threads. However, there is [very promising](http://classparser.blogspot.com/2010/04/jruby-coroutines-really-fast.html), existing work that promises to make JRuby fibers even faster than those of MRI Ruby. Once this functionality is built into JRuby ([JRUBY-5461](http://jira.codehaus.org/browse/JRUBY-5461)), JRuby may well take the performance crown. At the moment, without the MLVM support, a request through the full Goliath stack takes ~6ms (166 req/s).
+
+Rubinius + Goliath performance is tough to pin down - there is a lot of room for optimization within the Rubinius VM. Currently, requests can take as little as 0.2ms and later spike to 50ms+. Stay tuned!
+
+Goliath has been in production at PostRank for over a year, serving a sustained 500 requests/s for internal and external applications. Many of the Goliath processes have been running for months at a time (read: no memory leaks) and have served hundreds of gigabytes of data without restarts. To scale up and provide failover and redundancy, our individual Goliath servers at PostRank are usually deployed behind a reverse proxy (such as HAProxy).
+
+## FAQ
+
+* How does Goliath compare to other Ruby async app-servers like Thin?
+    * They are similar (both use Eventmachine reactor), but also very different. Goliath is able to run on different Ruby platforms (see above), uses a different HTTP parser, supports HTTP keepalive & pipelining, and offers a fully asynchronous API for both request and response processing.
+
+* How does Goliath compare to Mongrel, Passenger, Unicorn?
+    * Mongrel is a threaded web-server, and both Passenger and Unicorn fork an entire VM to isolate each request from each other. By contrast, Goliath builds a single instance of the Rack app and runs all requests in parallel through a single VM, which leads to a much smaller memory footprint and less overhead.
+
+* How do I deploy Goliath in production?
+    * We recommend deploying Goliath behind a reverse proxy such as HAProxy, Nginx or equivalent. Using one of the above, you can easily run multiple instances of the same application and load balance between then within the reverse proxy.
+
+## Guides
+
+* [Server Options](https://github.com/postrank-labs/goliath/wiki/Server)
+* [Middleware](https://github.com/postrank-labs/goliath/wiki/Middleware)
+* [Configuration](https://github.com/postrank-labs/goliath/wiki/Configuration)
+* [Plugins](https://github.com/postrank-labs/goliath/wiki/Plugins)
+
+### Hands-on applications:
+
+* [Asynchronous HTTP, MySQL, etc](https://github.com/postrank-labs/goliath/wiki/Asynchronous-Processing)
+* [Response streaming with Goliath](https://github.com/postrank-labs/goliath/wiki/Streaming)
+* [Examples](https://github.com/postrank-labs/goliath/tree/master/examples)
+
+## Coverage
+
+* [Goliath: Non-blocking, Ruby 1.9 Web Server](http://www.igvita.com/2011/03/08/goliath-non-blocking-ruby-19-web-server)
+* [Stage left: Enter Goliath - HTTP Proxy + MongoDB](http://everburning.com/news/stage-left-enter-goliath/)
+
+## Discussion and Support
+
+* [Source](https://github.com/postrank-labs/goliath)
+* [Issues](https://github.com/postrank-labs/goliath/issues)
+* [Mailing List](http://groups.google.com/group/goliath-io)
+
+## License & Acknowledgments
+
+Goliath is distributed under the MIT license, for full details please see the LICENSE file.

data/Rakefile

@@ -0,0 +1,18 @@
+require 'bundler'
+Bundler::GemHelper.install_tasks
+
+require 'yard'
+require 'rspec/core/rake_task'
+
+task :default => [:spec]
+
+desc "run spec tests"
+RSpec::Core::RakeTask.new('spec') do |t|
+  t.pattern = 'spec/**/*_spec.rb'
+end
+
+desc 'Generate documentation'
+YARD::Rake::YardocTask.new do |t|
+  t.files   = ['lib/**/*.rb', '-', 'LICENSE']
+  t.options = ['--main', 'README.md', '--no-private']
+end

data/examples/activerecord/config/srv.rb

@@ -0,0 +1,7 @@
+require 'mysql2'
+
+ActiveRecord::Base.establish_connection(:adapter  => 'em_mysql2',
+                                        :database => 'goliath_test',
+                                        :username => 'goliath',
+                                        :password => 'goliath',
+                                        :host     => 'localhost')

data/examples/activerecord/srv.rb

@@ -0,0 +1,37 @@
+#!/usr/bin/env ruby
+
+# gem install activerecord
+# gem install mysql2
+
+# create database goliath_test
+# create user 'goliath'@'localhost' identified by 'goliath'
+# grant all on goliath_test.* to 'goliath'@'localhost'
+# create table users (id int not null auto_increment primary key, name varchar(255), email varchar(255));
+# insert into users (name, email) values ('dan', 'dj2@everyburning.com'), ('Ilya', 'ilya@igvita.com');
+
+$: << "../../lib" << "./lib"
+
+require 'goliath'
+require 'active_record'
+require 'yajl'
+
+class User < ActiveRecord::Base
+end
+
+class Srv < Goliath::API
+  use ::Rack::Reloader, 0 if Goliath.dev?
+
+  use Goliath::Rack::Params
+  use Goliath::Rack::DefaultMimeType
+  use Goliath::Rack::Formatters::JSON
+  use Goliath::Rack::Render
+  use Goliath::Rack::ValidationError
+
+  use Goliath::Rack::Validation::RequiredParam, {:key => 'id', :type => 'ID'}
+  use Goliath::Rack::Validation::NumericRange, {:key => 'id', :min => 1}
+
+  def response(env)
+    User.find_by_sql("SELECT SLEEP(10)")
+    [200, {}, User.find(params['id'])]
+  end
+end

data/examples/async_upload.rb

@@ -0,0 +1,34 @@
+#!/usr/bin/env ruby
+$:<< '../lib' << 'lib'
+
+require 'goliath'
+require 'yajl'
+
+class AsyncUpload < Goliath::API
+
+  # reload code on every request in dev environment
+  use ::Rack::Reloader, 0 if Goliath.dev?
+
+  use Goliath::Rack::Params             # parse & merge query and body parameters
+  use Goliath::Rack::DefaultMimeType    # cleanup accepted media types
+  use Goliath::Rack::Formatters::JSON   # JSON output formatter
+  use Goliath::Rack::Render             # auto-negotiate response format
+
+  def on_headers(env, headers)
+    env.logger.info 'received headers: ' + headers.inspect
+    env['async-headers'] = headers
+  end
+
+  def on_body(env, data)
+    env.logger.info 'received data: ' + data
+    (env['async-body'] ||= '') << data
+  end
+
+  def on_close(env)
+    env.logger.info 'closing connection'
+  end
+
+  def response(env)
+    [200, {}, {body: env['async-body'], head: env['async-headers']}]
+  end
+end

data/examples/conf_test.rb

@@ -0,0 +1,27 @@
+#!/usr/bin/env ruby
+$:<< '../lib' << 'lib'
+
+#
+# Example of using "config/conf_test.rb" during server initialization
+#  - options parser allows us to extend the command line parameters for API
+#  - config file shows loading of environment specific, global, and shared variables
+#
+
+require 'goliath'
+
+class ConfTest < Goliath::API
+
+  use Goliath::Rack::Params
+  use Goliath::Rack::DefaultMimeType
+  use Goliath::Rack::Formatters::JSON
+  use Goliath::Rack::Render
+
+  def options_parser(opts, options)
+    options[:test] = 0
+    opts.on('-t', '--test NUM', "The test number") { |val| options[:test] = val.to_i }
+  end
+
+  def response(env)
+    [200, {}, {:response => env.config}]
+  end
+end

data/examples/config/conf_test.rb

@@ -0,0 +1,12 @@
+config['global'] = 'my global'
+config['test'] = options[:test]
+
+import('shared')
+
+environment :production do
+  config['prod'] = 'my prod'
+end
+
+environment :development do
+  config['dev'] = 'my dev'
+end

data/examples/config/echo.rb

@@ -0,0 +1 @@
+import('shared')

data/examples/config/http_log.rb

@@ -0,0 +1,7 @@
+config['forwarder'] = 'http://localhost:8080'
+
+config['mongo'] = EventMachine::Synchrony::ConnectionPool.new(size: 20) do
+  # Need to deal with this just never connecting ... ?
+  conn = EM::Mongo::Connection.new('localhost', 27017, 1, {:reconnect_in => 1})
+  conn.db('http_log').collection('aggregators')
+end

data/examples/config/shared.rb

@@ -0,0 +1,5 @@
+config['shared'] = 'shared'
+
+environment :development do
+  # shared development only configuration
+end

data/examples/custom_server.rb

@@ -0,0 +1,57 @@
+#!/usr/bin/env ruby
+$:<< '../lib' << 'lib'
+
+require 'goliath/api'
+require 'goliath/runner'
+
+# Example demonstrating how to use a custom rack builder with the
+# Goliath server and mixing Goliath APIs with normal Rack end points.
+#
+# Note, that the same routing behavior is supported by Goliath, loost at
+# the rack_routes.rb example to see how to define custom routes.
+
+# Our custom Goliath API
+class HelloWorld < Goliath::API
+  def response(env)
+    [200, {}, "hello world!"]
+  end
+end
+
+# Another Goliath API
+class Bonjour < Goliath::API
+  def response(env)
+    [200, {}, "bonjour!"]
+  end
+end
+
+# Rack builder acting as a router
+router = Rack::Builder.new do
+  # Rack end point
+  map '/version' do
+    use ::Rack::ContentLength
+    run Proc.new {|env| [200, {"Content-Type" => "text/html"}, ["Version 0.1"]] }
+  end
+
+  # map the /hello_world uri to our Goliath API
+  map "/hello_world" do
+    use ::Rack::ContentLength
+    run HelloWorld.new
+  end
+
+  # map the /bonjour uri to our other Goliath API
+  map "/bonjour" do
+    use ::Rack::ContentLength
+    run Bonjour.new
+  end
+
+  # catch the root route and return a 404
+  map "/" do
+    use ::Rack::ContentLength
+    run Proc.new {|env| [404, {"Content-Type" => "text/html"}, ["Try /version /hello_world or /bonjour"]] }
+  end
+end
+
+runner = Goliath::Runner.new(ARGV, nil)
+runner.app = router
+runner.run
+

data/examples/echo.rb

@@ -0,0 +1,37 @@
+#!/usr/bin/env ruby
+$:<< '../lib' << 'lib'
+
+require 'goliath'
+require 'goliath/plugins/latency'
+
+# Goliath uses multi-jon, so pick your favorite JSON serializer
+# require 'json'
+require 'yajl'
+
+class Echo < Goliath::API
+
+  # reload code on every request in dev environment
+  use ::Rack::Reloader, 0 if Goliath.dev?
+
+  use Goliath::Rack::Params             # parse & merge query and body parameters
+  use Goliath::Rack::DefaultMimeType    # cleanup accepted media types
+  use Goliath::Rack::Formatters::JSON   # JSON output formatter
+  use Goliath::Rack::Render             # auto-negotiate response format
+  use Goliath::Rack::Heartbeat          # respond to /status with 200, OK (monitoring, etc)
+  use Goliath::Rack::ValidationError    # catch and render validation errors
+
+  use Goliath::Rack::Validation::RequestMethod, %w(GET)           # allow GET requests only
+  use Goliath::Rack::Validation::RequiredParam, {:key => 'echo'}  # must provide ?echo= query or body param
+
+  plugin Goliath::Plugin::Latency       # output reactor latency every second
+
+  def process_request
+    logger.info "Processing request"
+
+    {response: env.params['echo']}
+  end
+
+  def response(env)
+    [200, {}, process_request]
+  end
+end

data/examples/gziped.rb

@@ -0,0 +1,40 @@
+#!/usr/bin/env ruby
+$:<< '../lib' << 'lib'
+
+#
+# Example of using the rack/deflater middleware to automatically GZIP your response
+# if the client indicated that it will accept gzipped data.
+#
+# Note that we're also using Rack::Rewrite to alter incoming request prior to it
+# being parsed by the Rack::Params middleware. This allows us to transparently
+# rewrite incoming requests before any processing is done on it.
+#
+# curl -s -H "Accept-Encoding: gzip,deflate" -H "Connection: close" localhost:9000?gziped=test | gunzip
+#
+
+require 'rack/deflater'
+require 'rack/rewrite'
+require 'goliath'
+require 'yajl'
+
+class Gziped < Goliath::API
+  # if client requested, compress the response
+  use ::Rack::Deflater
+
+  # example of using rack rewriting to rewrite the param gziped to echo
+  use ::Rack::Rewrite do
+    rewrite %r{^(.*?)\??gziped=(.*)$}, lambda { |match, env| "#{match[1]}?echo=#{match[2]}" }
+  end
+
+  use Goliath::Rack::Params             # parse & merge query and body parameters
+  use Goliath::Rack::Formatters::JSON   # JSON output formatter
+  use Goliath::Rack::Render             # auto-negotiate response format
+  use Goliath::Rack::ValidationError    # catch and render validation errors
+
+  use Goliath::Rack::Validation::RequestMethod, %w(GET)           # allow GET requests only
+  use Goliath::Rack::Validation::RequiredParam, {:key => 'echo'}  # must provide ?echo= query or body param
+
+  def response(env)
+    [200, {}, {response: env.params['echo']}]
+  end
+end