raptor-io 0.0.1

checksums.yaml

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    NjQ5NDlkZTFhNjE3YTUwMGYzMDIzMTcyYTgxOWUxZmM2MTkwMjc1MQ==
+  data.tar.gz: !binary |-
+    NTFiOGZjZjYxNzRjZmJmNjgwZTIwOGVkYzMwNTUxNDMxZTdkMzRiNQ==
+SHA512:
+  metadata.gz: !binary |-
+    ODU0Y2ZhYjQzMjBkNzEzNzgyNmFiMjBmMTZhMTVhNWNhZDUzOWFlYjAwMDFk
+    YTZkNjQ3ZTk5NzZlOTZlZTE3MTQ3ZWYzZGEzYjVmYTI2MWY1ZTRmNjNiM2E5
+    ZTYxMDJlMDg1NDEyYTA5OTc3OTUyMGY3OGU5ZTI2ZGE4MjI5M2Y=
+  data.tar.gz: !binary |-
+    YjIzMjI0NGZmYjY3MzgwOTVlN2YyNmFkMTZkOGYxM2RhMDMwZTEzOTc0MzVj
+    NjlkZWI3YWY2OGY1NWRmMWE5MGQ4NjhjNjM1MTFlZWU0OGZhNzJlOTBhMWU5
+    NmZkZDg2NmY2MTg4ZTI5ZjFmYTkyNjc4MGViMDAyOWIwMjE4YjQ=

data/LICENSE

@@ -0,0 +1,30 @@
+```
+Copyright (c) 2013 Rapid7, Inc.
+
+License: BSD-3-clause
+
+ Redistribution and use in source and binary forms, with or without modification,
+ are permitted provided that the following conditions are met:
+ .
+     * Redistributions of source code must retain the above copyright notice,
+     this list of conditions and the following disclaimer.
+ .
+     * Redistributions in binary form must reproduce the above copyright notice,
+     this list of conditions and the following disclaimer in the documentation
+     and/or other materials provided with the distribution.
+ .
+     * Neither the name of Rapid7 LLC nor the names of its contributors
+     may be used to endorse or promote products derived from this software
+     without specific prior written permission.
+ .
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+```

data/README.md

@@ -0,0 +1,51 @@
+# RaptorIO
+
+Lighter, faster and smarter than REX,  RaptorIO is the next evolutionary step in Metasploit's basic network IO functionality.
+
+
+
+
+## FAQ
+
+1. **What is RaptorIO?**  RaptorIO is the next generation of the core networking libraries used by Metasploit.  This includes the core Socket and Switchboard implementations as well as clients and servers for lots of protocols, both open source and proprietary.  It is intended as a general-purpose library for doing network exploitation, research, and reverse engineering, but you can also use it to power the server hosting your collection of cat GIFs.
+2. **Should I use it?**   Sure!  Just remember that it's not 1.0 yet and won't be for awhile.  We are open-sourcing it now because the critical early stages have been completed and we want the Ruby community to be able to have a look and help us expand it.
+3. **What are the next goals?** We want to continue building out client/server pairs for each of multiple protocols, starting with the SMB family (DCE/RPC, etc).  Goals on the horizon include a less client-centric replacement for Net::SSH as well.
+4. **I know network programming and Ruby - how can I help?**  Pull requests are welcome! Implementations of client/server pairs must be fully spec'd (ideally from the protocol's RFC) and follow the One True Rule of RaptorIO: "*an implementation must follow the spec by default, but allow tinkering/abuse of the protocol at every level.*"  Abstractions should be for convenience, but not lock users out of dealing with lower-level constructs.
+5. **When will RaptorIO be live as the networking layer in Metasploit Framework?** We hesitate to put a firm date on it, but we are working toward that goal as fast as possible.  There's lots of work to be done, and we hope you can help!
+6. **Why RaptorIO::Socket? Why RaptorIO::Switchboard?** As the networking layer for an exploitation toolkit, RaptorIO has an an explicit design goal the ability to "pivot" new network traffic through established network connections.  The Switchboard, Socket and Comm constructs exist to  facilitate this capability.
+
+## Installation
+
+### From source
+
+    gem install bundler
+    git clone git@github.com:rapid7/raptor.git
+    bundle install
+    
+### From RubyGems
+*submission to RubyGems anticipated mid-February 2014*    
+
+## Usage
+
+* [HTTP client and server operations](https://github.com/rapid7/raptor-io/wiki/HTTP-examples) with asychronous, synchronous, and manually queued interactions
+* [Arbitrary network connections](https://github.com/rapid7/raptor-io/wiki/Raptor-Socket-Requirements) using the Comm(connection establishment) and Switchboard(routing) abstractions.
+
+
+## Immediate Goals
+* ~~Implement improved Switchboard (in-memory router)~~
+* ~~Implement improved Socket (built on Ruby's TCPSocket)~~
+* ~~Create basic Comm classes (for multi-channel communication abstraction)~~
+* * ~~local~~
+* * ~~SOCKS~~
+* ~~HTTP client and server functionality~~
+* SMB client and server functionality
+* (Implement all protocol libraries supported by REX) 
+
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/lib/rack/handler/raptor-io.rb

@@ -0,0 +1,130 @@
+require 'raptor-io'
+require 'rack'
+require 'stringio'
+require 'rack/content_length'
+
+# Rack handler for {RaptorIO::Protocol::HTTP::Server}.
+class Rack::Handler::RaptorIO
+
+  Rack::Handler.register self.to_s.split( ':' ).last.downcase, self
+
+  # Starts the server and runs the `app`.
+  #
+  # @param  [#call] app   Rack Application to run.
+  # @param  [Hash]  options Rack options.
+  def self.run( app, options = {} )
+    return false if @server
+
+    options[:address]  = options.delete(:Host) || default_host
+    options[:port]   ||= options.delete(:Port) || 8080
+
+    @app    = app
+    @server = ::RaptorIO::Protocol::HTTP::Server.new( options ) do |response|
+      service response
+    end
+    yield @server if block_given?
+    @server.run
+
+    true
+  end
+
+  # Shuts down the server.
+  def self.shutdown
+    return false if !@server
+
+    @server.stop
+    @server = nil
+
+    true
+  end
+
+  private
+
+  def self.valid_options
+    {
+        'Host=HOST' => "Hostname to listen on (default: #{default_host})",
+        'Port=PORT' => 'Port to listen on (default: 8080)'
+    }
+  end
+
+  def self.default_host
+    (ENV['RACK_ENV'] || 'development') == 'development' ? 'localhost' : '0.0.0.0'
+  end
+
+  def self.service( response )
+    request      = response.request
+    path         = request.effective_url.path
+    http_version = "HTTP/#{request.version}"
+
+    query_string = request.effective_url.to_s.split( '?' ).last.to_s
+    query_string = '' if query_string == '/'
+
+    environment = {
+        'REQUEST_METHOD'  => request.http_method.to_s.upcase,
+        'SCRIPT_NAME'     => '',
+        'PATH_INFO'       => path,
+        'REQUEST_PATH'    => path,
+        'QUERY_STRING'    => query_string,
+        'SERVER_NAME'     => @server.address,
+        'SERVER_PORT'     => @server.port.to_s,
+        'HTTP_VERSION'    => http_version,
+        'REMOTE_ADDR'     => request.client_address
+    }
+
+    request.headers.each do |k, v|
+      environment["HTTP_#{k.upcase.gsub( '-', '_' )}"] = v
+    end
+
+    if environment['HTTP_CONTENT_TYPE']
+      environment['CONTENT_TYPE'] = environment.delete( 'HTTP_CONTENT_TYPE' )
+    end
+
+    if environment['HTTP_CONTENT_LENGTH']
+      environment['CONTENT_LENGTH'] = environment.delete( 'HTTP_CONTENT_LENGTH' )
+    end
+
+    environment['SERVER_PROTOCOL'] = environment['HTTP_VERSION']
+
+    rack_input = StringIO.new( request.body.to_s )
+    rack_input.set_encoding( Encoding::BINARY ) if rack_input.respond_to?( :set_encoding )
+
+    environment.update(
+        'rack.version'      => Rack::VERSION,
+        'rack.input'        => rack_input,
+        'rack.errors'       => $stderr,
+        'rack.multithread'  => true,
+        'rack.multiprocess' => false,
+        'rack.run_once'     => false,
+        'rack.url_scheme'   => 'http',
+        'rack.hijack?'      => false,
+        'raptor.request'    => request
+    )
+
+    begin
+      status, headers, body = @app.call( environment )
+      body = '' if !body
+
+      response.code = status
+
+      if body.is_a? String
+        response.body = body
+      else
+        body.each { |part| (response.body ||= '') << part }
+      end
+
+      response.headers.merge! headers
+    rescue RuntimeError => e
+      response.code = 501
+      response.body = "#{e} (#{e.class})"
+
+      environment['rack.errors'].puts "#{e} (#{e.class})"
+      e.backtrace.each do |line|
+        environment['rack.errors'].puts line
+      end
+
+      response.headers['content-type'] = 'text/plain'
+    end
+  ensure
+    body.close if body.respond_to? :close
+  end
+end

data/lib/raptor-io.rb

@@ -0,0 +1,11 @@
+require 'rubygems'
+require 'bundler/setup'
+
+#Smaller, more agile, and it can open doors.
+module RaptorIO
+end
+
+require 'raptor-io/version'
+require 'raptor-io/error'
+require 'raptor-io/ruby'
+require 'raptor-io/protocol'

data/lib/raptor-io/error.rb

@@ -0,0 +1,19 @@
+
+module RaptorIO
+
+#
+# Represents a RaptorIO error base-class and also provides the namespace for all
+# RaptorIO errors.
+#
+# @author Tasos Laskos <tasos_laskos@rapid7.com>
+#
+class Error < StandardError
+
+# {RaptorIO} timeout error.
+#
+# @author Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
+class Timeout < Error
+end
+
+end
+end

data/lib/raptor-io/protocol.rb

@@ -0,0 +1,6 @@
+# A collection of useful protocols
+module RaptorIO::Protocol
+end
+
+require 'raptor-io/protocol/error'
+require 'raptor-io/protocol/http'

data/lib/raptor-io/protocol/error.rb

@@ -0,0 +1,10 @@
+#
+# {Protocol} error namespace.
+#
+# All {Protocol} errors inherit from and live under it.
+#
+# @author Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
+#
+class RaptorIO::Protocol::Error < RaptorIO::Error
+
+end

data/lib/raptor-io/protocol/http.rb

@@ -0,0 +1,34 @@
+require 'thread'
+require 'raptor-io/socket'
+require 'base64'
+require 'cgi'
+require 'uri'
+
+#
+# HTTP protocol implementation.
+#
+# @author Tasos Laskos <tasos_laskos@rapid7.com>
+#
+module RaptorIO::Protocol::HTTP
+
+  # Matches line separator characters for HTTP messages.
+  CRLF_PATTERN             = /\r?\n/
+
+  # CRLF character sequence.
+  CRLF                     = "\r\n"
+
+  # Matches sequence used to separate headers from the body.
+  HEADER_SEPARATOR_PATTERN = /\r?\n\r?\n/
+
+  # Header separator character sequence.
+  HEADER_SEPARATOR         = "\r\n\r\n"
+
+end
+
+require 'raptor-io/protocol/http/error'
+require 'raptor-io/protocol/http/headers'
+require 'raptor-io/protocol/http/message'
+require 'raptor-io/protocol/http/request'
+require 'raptor-io/protocol/http/response'
+require 'raptor-io/protocol/http/server'
+require 'raptor-io/protocol/http/client'

data/lib/raptor-io/protocol/http/client.rb

@@ -0,0 +1,685 @@
+require 'thread'
+require 'base64'
+
+module RaptorIO
+module Protocol::HTTP
+
+#
+# HTTP Client class.
+#
+# @author Tasos Laskos <tasos_laskos@rapid7.com>
+#
+class Client
+
+  # @return [Integer, Float] Timeout in seconds.
+  attr_accessor :timeout
+
+  # @return [Integer] Maximum open sockets.
+  attr_accessor :concurrency
+
+  # @return [String]  User-agent string to use.
+  attr_accessor :user_agent
+
+  # @return [Hash{Symbol=>Hash}]
+  #   Request manipulators, and their options, to be run against each
+  #   {#queue queued} request.
+  attr_accessor :manipulators
+
+  # @return [Hash]  Persistent storage for the manipulators..
+  attr_accessor :datastore
+
+  # @return [Symbol]  SSL version.
+  attr_accessor :ssl_version
+
+  # @return [Constant]  Peer verification mode.
+  attr_accessor :ssl_verify_mode
+
+  # @return [OpenSSL::SSL::SSLContext]  SSL context to use.
+  attr_accessor :ssl_context
+
+  # @return [SwitchBoard] The routing table from which this {Client}
+  #   will {Socket::SwitchBoard#create_tcp make new TCP connections}.
+  attr_reader :switch_board
+
+  # Default client options.
+  DEFAULT_OPTIONS = {
+      concurrency:      20,
+      user_agent:       "RaptorIO::HTTP/#{RaptorIO::VERSION}",
+      timeout:          10,
+      manipulators:     {},
+      ssl_version:      :TLSv1,
+      ssl_verify_mode:  OpenSSL::SSL::VERIFY_NONE,
+      ssl_context:      nil
+  }.freeze
+
+  # @param  [Hash]  options Request options.
+  # @option options [Integer] :concurrency (20)
+  #   Amount of open sockets at any given time.
+  # @option options [String] :user_agent ('RaptorIO::HTTP/<RaptorIO::VERSION>')
+  #   User-agent string to include in the requests.
+  # @option options [Integer, Float] :timeout (10)
+  #   Timeout in seconds.
+  # @option options [Hash{Symbol=>Hash}] :manipulators
+  #   Request manipulators and their options.
+  # @option options [Symbol]  :ssl_version (:TLSv1)
+  # @option options [Constant]  :ssl_verify_mode (OpenSSL::SSL::VERIFY_NONE)
+  #   Peer verification mode.
+  # @option options [OpenSSL::SSL::SSLContext]  :ssl_context (nil)
+  #   SSL context to use.
+  # @option options [Socket::SwitchBoard] :switch_board The switch board
+  #   from which we can create new connections.
+  #
+  # @raise  RaptorIO::Protocol::HTTP::Request::Manipulator::Error::InvalidOptions
+  #   On invalid manipulator options.
+  def initialize( options = {} )
+    @switch_board = options.delete(:switch_board)
+    unless @switch_board.respond_to?(:create_tcp)
+      raise ArgumentError, 'Must provide a :switch_board'
+    end
+
+    DEFAULT_OPTIONS.merge( options ).each do |k, v|
+      begin
+        send( "#{k}=", try_dup( v ) )
+      rescue NoMethodError
+        instance_variable_set( "@#{k}".to_sym, try_dup( v ) )
+      end
+    end
+
+    validate_manipulators!( manipulators )
+
+    # Holds Request objects.
+    @queue = []
+
+    # Persistent storage for request manipulators.
+    @datastore = Hash.new { |h, k| h[k] = {} }
+
+    reset_sockets
+    reset_pending_responses
+  end
+
+  #
+  # Updates the client {#manipulators} and perform and sanity check on their
+  # options.
+  #
+  # @param  [Hash{String=>Hash}]  manipulators
+  #   Manipulators and their options.
+  #
+  # @raise  RaptorIO::Protocol::HTTP::Request::Manipulator::Error::InvalidOptions
+  def update_manipulators( manipulators )
+    validate_manipulators!( manipulators )
+    @manipulators.merge!( manipulators )
+  end
+
+  #
+  # Creates and {#queue queues} a {Request}.
+  #
+  # @param  [String]  url URL of the remote resource.
+  # @param  [Hash]  options {Request} options with the following extras:
+  # @option options [Symbol]  :mode (:async)
+  #   Mode to use for the request, available options are:
+  #
+  #   * `:async`  -- Adds the request to the queue.
+  #   * `:sync`  -- Performs the request in a blocking manner and returns the
+  #     {Response}.
+  #
+  # @option options [Hash, String]  :cookies
+  #   Cookies as name=>pair values -- should already be escaped.
+  #
+  # @option options [Hash{Symbol=>Hash}]  :manipulators
+  #   Manipulator names for keys and their options as their values.
+  #
+  # @param  [Block] block Callback to be passed the {Response}.
+  #
+  # @return [Request, Response]
+  #   Queued {Request} when in `:async` `:mode`, {Response} when in `:sync`
+  #   `:mode`.
+  #
+  # @see Request#initialize
+  # @see Request#on_complete
+  # @see #queue
+  #
+  def request( url, options = {}, &block )
+    options = options.dup
+    options[:timeout] ||= @timeout
+
+    req = Request.new( options.merge( url: url ) )
+
+    req.headers['User-Agent'] = @user_agent if !@user_agent.to_s.empty?
+
+    case options[:cookies]
+      when Hash
+        req.headers['Cookie'] =
+            options[:cookies].map { |k, v| "#{k}=#{v}" }.join( ';' )
+      when String
+        req.headers['Cookie'] = options[:cookies]
+    end
+
+    return sync_request( req, options[:manipulators] || {} ) if options[:mode] == :sync
+
+    req.on_complete( &block ) if block_given?
+
+    queue( req, options[:manipulators] || {} )
+    req
+  end
+
+  #
+  # Creates and {#queue queues} a GET {Request}.
+  #
+  # @param (see #request)
+  # @return (see #request)
+  #
+  def get( url, options = {}, &block )
+    request( url, options.merge( http_method: :get ), &block )
+  end
+
+  #
+  # Creates and {#queue queues} a POST {Request}.
+  #
+  # @param (see #request)
+  # @return (see #request)
+  #
+  def post( url, options = {}, &block )
+    request( url, options.merge( http_method: :post ), &block )
+  end
+
+  # @return [Integer] The amount of {#queue queued} requests.
+  def queue_size
+    @queue.size
+  end
+
+  #
+  # Queues a {Request}.
+  #
+  # @param  [Request] request
+  # @param  [Hash{Symbol=>Hash}]  manipulators
+  #   Manipulator names for keys and their options as their values.
+  # @return [Request] `request`
+  #
+  def queue( request, manipulators = {} )
+    validate_manipulators!( manipulators )
+
+    request.timeout ||= timeout
+
+    @manipulators.merge( manipulators ).each do |manipulator, options|
+      Request::Manipulators.process( manipulator, self, request, options )
+    end
+
+    @queue << request
+    request
+  end
+  alias :<< :queue
+
+  # Runs the {#queue queued} {Request}.
+  def run
+    while @queue.any?
+      # Get us some seeds.
+      consume_requests
+
+      while @sockets[:done].size != @sockets[:lookup_request].size
+
+        if @sockets[:reads].any?
+          # Use the lowest available timeout for #select.
+          lowest_timeout =
+              @sockets[:reads].map { |socket| @pending_responses[socket][:timeout] }.sort.first
+
+          clock = Time.now
+          res = select( @sockets[:reads], nil, @sockets[:reads], lowest_timeout )
+          waiting_time = Time.now - clock
+
+          # Adjust the timeouts for *all* sockets since they all benefited from
+          # the #select waiting period which just elapsed.
+          #
+          # And this is the whole reason for keeping track of timeouts externally.
+          @pending_responses.each do |_, pending_response|
+            pending_response[:timeout] -= waiting_time
+            pending_response[:timeout]  = 0 if pending_response[:timeout] < 0
+          end
+
+          # #select timed out, go digging.
+          if !res
+            # Find and handle the sockets which timed out.
+            @sockets[:reads].each do |socket|
+              if waiting_time >= @pending_responses[socket][:timeout]
+                error = RaptorIO::Error::Timeout.new( 'Request timed-out.' )
+                error.set_backtrace( caller )
+                handle_error( @sockets[:lookup_request][socket], error, socket )
+              end
+
+              # Fill the available pool space.
+              consume_requests
+            end
+
+          # #select didn't time out, yay!
+          else
+
+            # Handle sockets with errors -- like reset connections.
+            if res[2].any?
+              res[2].each do |socket|
+                handle_error( @sockets[:lookup_request][socket], nil, socket )
+
+                # Fill the available pool space.
+                consume_requests
+              end
+            end
+
+            # Handle sockets which are ready to be read.
+            if res[0].any?
+              res[0].each do |socket|
+                # Buffer/handle the response for the given socket.
+                read( socket )
+
+                # Fill the available pool space.
+                consume_requests
+              end
+            end
+          end
+        end
+
+        next if @sockets[:writes].empty?
+
+        _, writes, errors = select( nil, @sockets[:writes], @sockets[:writes] )
+
+        errors.each do |socket|
+          handle_error( @sockets[:lookup_request][socket], nil, socket )
+        end
+
+        writes.each do |socket|
+          # Send the request for the given socket.
+          write( socket )
+
+          # Fully utilize our socket allowance.
+          consume_requests
+        end
+
+      end
+    end
+
+    reset_sockets
+    reset_pending_responses
+
+    nil
+  end
+
+  # @return [Integer] Amount of open sockets.
+  def open_socket_count
+    open_sockets.size
+  end
+
+  private
+
+  # @param  [Request] request Request to perform in blocking mode.
+  # @return [Response]  HTTP response.
+  def sync_request( request, manipulators = {} )
+    client = self.class.new(
+        switch_board:     @switch_board,
+        user_agent:       user_agent,
+        timeout:          timeout
+    )
+
+    # The normal and sync clients should share these structures, that's why
+    # we're not passing them via the initializer.
+    client.manipulators = @manipulators
+    client.datastore    = @datastore
+
+    res = nil
+    request.on_complete { |r| res = r }
+    client.queue( request, manipulators )
+    client.run
+
+    raise res.error if res.error
+
+    res
+  end
+
+  # @return [Array<Socket>] Sockets currently open.
+  def open_sockets
+    @sockets[:writes] + @sockets[:reads]
+  end
+
+  #
+  # Writes the associated {Request} to `socket`.
+  #
+  # @param  [#write]  socket  Writable IO object.
+  #
+  def write( socket, retry_on_fail = true )
+    request        = @sockets[:lookup_request][socket]
+    request_string = request.to_s.repack
+
+    # Send out the request, **all** of it.
+    loop do
+      begin
+        bytes_written = socket.write( request_string )
+      # All hope is lost.
+      rescue RaptorIO::Socket::Error::ConnectionError => error
+        handle_error( request, error, socket )
+        return
+
+      # The connection has been closed so retry but only if the request is
+      # idempotent.
+      rescue RaptorIO::Socket::Error::BrokenPipe => e
+        if request.idempotent? && retry_on_fail
+          @sockets[:writes].delete( socket )
+
+          fresh_socket = refresh_connection( socket )
+          @sockets[:writes] << fresh_socket
+          return write( fresh_socket, false )
+        else
+          error = Protocol::Error::BrokenPipe.new( e.to_s )
+          error.set_backtrace( e.backtrace )
+          handle_error( request, error, socket )
+          return
+        end
+      end
+
+      break if bytes_written == request_string.size
+
+      request_string = request_string[bytes_written..-1]
+    end
+
+    # Move it to the read list.
+    @sockets[:reads] << @sockets[:writes].delete( socket )
+
+    true
+  end
+
+  #
+  # Reads/buffers a response from `socket` and calls the callback of the
+  # associated request once the full response is received -- at which point it
+  # also closes the socket.
+  #
+  # @param  [#gets] socket  Readable IO object.
+  #
+  # @return [true, nil]
+  #   `true` if the response finished being buffered, `nil` otherwise.
+  #
+  def read( socket )
+    reset_timeout( socket )
+
+    response = @pending_responses[socket]
+
+    if response[:has_full_headers]
+      headers = response[:parsed_headers]
+
+      if headers['Transfer-Encoding'].to_s.downcase == 'chunked'
+        read_size = socket.gets.to_s[0...-CRLF.size]
+        return if read_size.empty?
+
+        if (read_size = read_size.to_i( 16 )) > 0
+          response[:body] << socket.read( read_size + CRLF.size ).to_s[0...read_size]
+          return
+        end
+      else
+        # A Content-Type is not strictly necessary, the end of the response body
+        # can also be signaled by the server closing the connection. That's why
+        # the following code is so ugly.
+
+        read_size = nil
+        if (content_length = headers['Content-length'].to_i) > 0
+          read_size = content_length - response[:body].size
+        end
+
+        has_body = headers['Content-length'] != '0'
+
+        closed = false
+        if has_body
+          begin
+            line = (read_size ? socket.read(read_size) : socket.gets)
+            if line
+              response[:body] << line
+            else
+              raise RaptorIO::Socket::Error::BrokenPipe
+            end
+          rescue RaptorIO::Socket::Error::BrokenPipe
+            closed = true
+            response[:force_no_keep_alive] = true
+          end
+        end
+
+        # Return back to the #select loop if there's more data to be read
+        # and wait for our next turn.
+        return if has_body && ((!read_size && !closed) || (response[:body].size < content_length))
+      end
+
+      handle_success( socket )
+      return true
+    end
+
+    response[:headers] << socket.gets.to_s
+
+    # Keep going until we get all the headers.
+    return if !(response[:headers] =~ HEADER_SEPARATOR_PATTERN)
+    response[:has_full_headers] = true
+
+    # Perform some preliminary parsing to make our lives easier.
+    response[:partial_response] = Response.parse( response[:headers] )
+    response[:parsed_headers]   = response[:partial_response].headers
+
+    # Some of the body may have gotten into the headers' buffer, sort them out.
+    response[:headers], response[:body] = response[:headers].split( HEADER_SEPARATOR_PATTERN, 2 )
+
+    # If there is no body to expect handle the response now.
+    if response[:partial_response].headers['Content-length'] == '0' ||
+        status_without_body?( response[:partial_response].code )
+      handle_success( socket )
+      return true
+    end
+
+    nil
+  end
+
+  def reset_timeout( socket )
+    @pending_responses[socket][:timeout] = @sockets[:lookup_request][socket].timeout
+  end
+
+  #
+  # @note Respects the {#concurrency} limit.
+  #
+  # Consumes requests from the queue and adds write-sockets for them.
+  #
+  # @return [Integer] Amount of requests consumed.
+  #
+  def consume_requests
+    added = 0
+
+    (@concurrency - open_socket_count).times do
+      return added if @queue.empty?
+      q_request = @queue.pop
+
+      socket = connection_for_request( q_request )
+      next if !socket
+
+      @sockets[:lookup_request][socket] = q_request
+      @sockets[:writes] << socket
+
+      added += 1
+    end
+
+    added
+  end
+
+  # @param  [Request] request
+  def connection_for_request( request )
+    # If the request is idempotent grab a pool connection as we can risk a retry
+    # in case it has been closed...
+    socket =  if request.idempotent?
+                # If there's an idling connection to that server, use it instead of opening
+                # a new one.
+                if connection_pool[request.connection_id].empty?
+                  connect( request )
+                else
+                  connection_pool[request.connection_id].pop
+                end
+
+              # ...otherwise establish and use a new connection (and make room
+              # in the queue).
+              else
+                if !connection_pool[request.connection_id].empty?
+                  connection_pool[request.connection_id].pop.close
+                end
+                connect( request )
+              end
+
+    @sockets[:done].delete( socket )
+    socket
+  end
+
+  def self.reset
+    connection_pool.each do |_, q|
+      q.pop.close while !q.empty?
+    end
+    nil
+  end
+
+  def self.connection_pool
+    @connection_pool ||= Hash.new do |h, k|
+      h[k] = Queue.new
+    end
+  end
+  def connection_pool
+    self.class.connection_pool
+  end
+
+  # Opens up an non-blocking socket for the given `request`.
+  #
+  # @param  [Request] request
+  def connect( request )
+    @addresses ||= {}
+
+    host = request.parsed_url.host
+    port = request.parsed_url.port
+    cid  = request.connection_id
+
+    begin
+      socket = @switch_board.create_tcp(
+        peer_host:       (@addresses[cid] ||= Socket.getaddrinfo( host, nil ))[0][3],
+        peer_port:       port,
+        connect_timeout: @timeout
+      )
+
+      if request.parsed_url.scheme.to_s == 'https'
+        socket = socket.to_ssl(
+            ssl_context:     @ssl_context,
+            ssl_version:     @ssl_version,
+            ssl_verify_mode: @ssl_verify_mode
+        )
+      end
+
+      socket
+    rescue RaptorIO::Socket::Error => e
+      handle_error( request, e )
+      nil
+    end
+  end
+
+  def handle_success( socket )
+    response_data = @pending_responses.delete( socket )
+    @sockets[:done] << @sockets[:reads].delete( socket )
+    response_text = "#{response_data[:headers]}#{HEADER_SEPARATOR}#{response_data[:body]}"
+    response = Response.parse( response_text )
+    request  = @sockets[:lookup_request][socket]
+
+    if response.keep_alive? && !response_data[:force_no_keep_alive]
+      connection_pool[request.connection_id] << socket
+    else
+      socket.close
+    end
+
+    if response.code == 100 && request.continue?
+      request = request.dup
+      request.headers.delete 'expect'
+      queue( request.dup )
+      return
+    end
+
+    request.handle_response response
+  end
+
+  def handle_error( request, error = nil, socket = nil )
+    if socket
+      socket.close
+      [:reads, :writes].each { |state| @sockets[state].delete( socket ) }
+      @sockets[:done] << socket
+    end
+
+    response = Response.new( error: error )
+    request.handle_response response
+  end
+
+  def reset_sockets
+    # A socket starts in `:writes`, once the request is written it gets moved
+    # to `:reads`, at which point it stays there while the response is being
+    # buffered. Once a full response is received, and keep-alive is disabled, the
+    # socket is closed and moved to `:done`.
+    #
+    # If keep-alive is enabled, the connection is moved to `:done` without
+    # first being closed and will be reused appropriately, at which point it
+    # will be removed from `:done` and start all over.
+    @sockets = {
+        # Socket => HTTP Request lookup
+        lookup_request: {},
+
+        # Sockets ready to read from.
+        reads:          [],
+
+        # Sockets ready to write to.
+        writes:         [],
+
+        # Sockets with errors.
+        errors:         [],
+
+        # Sockets which have finished (or errored).
+        done:           []
+    }
+  end
+
+  def reset_pending_responses
+    # Response buffer.
+    @pending_responses = Hash.new do |h, socket|
+      h[socket] = {
+          # Do we have full headers?
+          has_full_headers: false,
+
+          # HTTP headers buffer.
+          headers: '',
+
+          # Response body buffer.
+          body: '',
+
+          # We use this to keep track of individual socket timeouts.
+          timeout: @sockets[:lookup_request][socket].timeout
+      }
+    end
+  end
+
+  def refresh_connection( socket )
+    request      = @sockets[:lookup_request].delete( socket )
+    fresh_socket = connection_for_request( request )
+
+    @sockets[:lookup_request][fresh_socket] = request
+    fresh_socket
+  ensure
+    socket.close
+    @pending_responses.delete( socket )
+    [:reads, :writes].each { |state| @sockets[state].delete( socket ) }
+  end
+
+  def status_without_body?( status_code )
+    status_code.to_s.start_with?( '1' ) || [204, 304].include?( status_code.to_i )
+  end
+
+  def validate_manipulators!( manipulators )
+    Request::Manipulators.validate_batch_options!( manipulators, self )
+  end
+
+  def validate_manipulators( manipulators )
+    Request::Manipulators.validate_batch_options( manipulators, self )
+  end
+
+  def try_dup( value )
+    value.dup rescue value
+  end
+
+end
+
+end
+end