rack-streaming-proxy 1.0.3 → 2.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 053ec0676586f5914af3a502686ef50783f25034
+  data.tar.gz: 6ee9e037cab39fec94571ff1dee67aa014f8d1cf
+SHA512:
+  metadata.gz: 589c2c12792a03f1329b2b7fb49b7095cd3faebe0c53c8cf1e5ebf4c21c76fe055720c3ee8d727e93e4209fa295aa2f9fe72bbe6d9710ff6507d9ddefea5ba62
+  data.tar.gz: 25e30cb68919cbfc459b67095d35cebac93c510b9bb493ce7736eef73bc4ac70490ffc43794356cf61b9d5a08429f37e8be3d974e896c784523b3ee43c3717ad

data/.gitignore

@@ -1,18 +1,17 @@
-# The list of files that should be ignored by Mr Bones.
-# Lines that start with '#' are comments.
-#
-# A .gitignore file can be used instead by setting it as the ignore
-# file in your Rakefile:
-#
-#   Bones {
-#     ignore_file  '.gitignore'
-#   }
-#
-# For a project with a C extension, the following would be a good set of
-# exclude patterns (uncomment them if you want to use them):
-# *.[oa]
-# *~
-announcement.txt
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
 coverage
-doc
+doc/
+lib/bundler/man
 pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in rack-streaming_proxy.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2009-2013 Fred Ngo, Nathan Witmer
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,95 @@
+# Rack::StreamingProxy
+
+A transparent streaming proxy to be used as rack middleware.
+
+* Streams the response from the downstream server to minimize memory usage
+* Handles chunked encoding if used
+* Proxies GET/PUT/POST/DELETE, XHR, and cookies
+
+Now updated to be compatible with Rails 3 and 4, and fixes major concurrency issues that were present in 1.0.
+
+Use Rack::StreamingProxy when you need to have the response streamed back to the client, for example when handling large file requests that could be proxied directly but need to be authenticated against the rest of your middleware stack.
+
+Note that this will not work well with EventMachine. EM buffers the entire rack response before sending it to the client. When testing, try Unicorn or Passenger rather than the EM-based Thin (See [discussion](http://groups.google.com/group/thin-ruby/browse_thread/thread/4762f8f851b965f6)).
+
+A simple streamer app has been included for testing and development.
+
+## Usage
+
+To use inside a Rails app, add a `config/initializers/streaming_proxy.rb` initialization file, and place in it:
+
+```ruby
+require 'rack/streaming_proxy'
+
+YourRailsApp::Application.configure do
+  config.streaming_proxy.logger             = Rails.logger                          # stdout by default
+  config.streaming_proxy.log_verbosity      = Rails.env.production? ? :low : :high  # :low or :high, :low by default
+  config.streaming_proxy.num_retries_on_5xx = 5                                     # 0 by default
+  config.streaming_proxy.raise_on_5xx       = true                                  # false by default
+
+  # Will be inserted at the end of the middleware stack by default.
+  config.middleware.use Rack::StreamingProxy::Proxy do |request|
+
+    # Inside the request block, return the full URI to redirect the request to,
+    # or nil/false if the request should continue on down the middleware stack.
+    if request.path.start_with?('/search')
+      "http://www.some-other-service.com/search?#{request.query}"
+    end
+  end
+end
+```
+
+To use as a Rack app:
+
+```ruby
+require 'rack/streaming_proxy'
+
+use Rack::StreamingProxy::Proxy do |request|
+  if request.path.start_with?('/proxy')
+    # You probably want to get rid of the '/proxy' in the path, when requesting from the destination.
+    proxy_path = request.path.sub %r{^/proxy}, ''
+    "http://www.another-server.com#{proxy_path}"
+  end
+end
+```
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'rack-streaming-proxy'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install rack-streaming-proxy
+
+## Requirements
+
+* Ruby = 1.9.3
+* rack >= 1.4
+* servolux ~> 0.10
+
+These requirements (other than Ruby) will be automatically installed via Bundler.
+
+This gem has not been tested with versions lower than those indicated.
+
+This gem works with Ubuntu 10.04. It has not been tested with later versions of Ubuntu or other Linuxes, but it should work just fine. It has not been tested with OS X but should work as well. However, I doubt it will work on any version of Windows, as it does process-based stuff. You are most welcome to try it and report back.
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Implement your changes, and make sure to add tests!
+4. Commit your changes (`git commit -am 'Add some feature'`)
+5. Push to the branch (`git push origin my-new-feature`)
+6. Create new Pull Request
+
+## Thanks To
+
+* [Nathan Witmer](http://github.com/zerowidth) for the 1.0 implementation of [Rack::StreamingProxy](http://github.com/zerowidth/rack-streaming-proxy)
+* [Tom Lea](http://github.com/cwninja) for [Rack::Proxy](http://gist.github.com/207938), which inspired Rack::StreamingProxy.
+* [Tim Pease](http://github.com/TwP) for [Servolux](https://github.com/Twp/servolux)

data/README.txt

@@ -1,6 +1,6 @@
 rack-streaming-proxy
     by Nathan Witmer <nwitmer@gmail.com>
-    http://github.com/aniero/rack-streaming-proxy
+    http://github.com/zerowidth/rack-streaming-proxy
 
 == DESCRIPTION:
 

data/Rakefile

@@ -1,28 +1 @@
-begin
-  require 'bones'
-rescue LoadError
-  abort '### Please install the "bones" gem ###'
-end
-
-ensure_in_path 'lib'
-require 'rack/streaming_proxy'
-
-task :default => 'spec:specdoc'
-task 'gem:release' => 'spec:specdoc'
-
-Bones {
-  name  'rack-streaming-proxy'
-  authors  'Nathan Witmer'
-  email  'nwitmer@gmail.com'
-  url  'http://github.com/aniero/rack-streaming-proxy'
-  version  Rack::StreamingProxy::VERSION
-  ignore_file  '.gitignore'
-  depend_on "rack", :version => ">= 1.0"
-  depend_on "servolux", :version => "~> 0.8.1"
-  depend_on "rack-test", :version => "~> 0.5.1", :development => true
-  spec {
-    opts ["--colour",  "--loadby mtime", "--reverse", "--diff unified"]
-  }
-  enable_sudo
-}
-
+require "bundler/gem_tasks"

data/dev/client.rb

@@ -0,0 +1,24 @@
+#!/usr/bin/env ruby
+# coding: utf-8
+# cf. https://gist.github.com/sonots/7751554
+
+require 'net/http'
+require 'uri'
+
+# unicorn spec/app.ru -p 4321
+# unicorn spec/proxy.ru -p 4322
+PORT = ARGV[0] || 8080
+
+http = Net::HTTP.new "localhost", PORT
+request = Net::HTTP::Get.new "/slow_stream"
+#request['Transfer-Encoding'] = 'chunked'
+request['Connection'] = 'keep-alive'
+http.request(request){|response|
+  puts "content-length: #{response.content_length}"
+  body = []
+  response.read_body{|x|
+    body << Time.now
+    puts "read_block: #{body.length}, #{x.size}byte(s)"
+  }
+  puts body
+}

data/lib/rack/streaming_proxy.rb

@@ -1,97 +1,3 @@
-module Rack
-  class StreamingProxy
-
-    class Error < StandardError; end
-
-    # :stopdoc:
-    VERSION = '1.0.3'
-    LIBPATH = ::File.expand_path(::File.dirname(__FILE__)) + ::File::SEPARATOR
-    PATH = ::File.expand_path(::File.join(::File.dirname(__FILE__), "..", "..")) + ::File::SEPARATOR
-    # :startdoc:
-
-    # Returns the version string for the library.
-    #
-    def self.version
-      VERSION
-    end
-
-    # Returns the library path for the module. If any arguments are given,
-    # they will be joined to the end of the libray path using
-    # <tt>File.join</tt>.
-    #
-    def self.libpath( *args )
-      args.empty? ? LIBPATH : ::File.join(LIBPATH, args.flatten)
-    end
-
-    # Returns the lpath for the module. If any arguments are given,
-    # they will be joined to the end of the path using
-    # <tt>File.join</tt>.
-    #
-    def self.path( *args )
-      args.empty? ? PATH : ::File.join(PATH, args.flatten)
-    end
-
-    # Utility method used to require all files ending in .rb that lie in the
-    # directory below this file that has the same name as the filename passed
-    # in. Optionally, a specific _directory_ name can be passed in such that
-    # the _filename_ does not have to be equivalent to the directory.
-    #
-    def self.require_all_libs_relative_to( fname, dir = nil )
-      dir ||= ::File.basename(fname, '.*')
-      search_me = ::File.expand_path(
-        ::File.join(::File.dirname(fname), dir, '**', '*.rb'))
-
-        Dir.glob(search_me).sort.each {|rb| require rb}
-    end
-
-    # The block provided to the initializer is given a Rack::Request
-    # and should return:
-    #
-    #   * nil/false to skip the proxy and continue down the stack
-    #   * a complete uri (with query string if applicable) to proxy to
-    #
-    # E.g.
-    #
-    #   use Rack::StreamingProxy do |req|
-    #     if req.path.start_with?("/search")
-    #       "http://some_other_service/search?#{req.query}"
-    #     end
-    #   end
-    #
-    # Most headers, request body, and HTTP method are preserved.
-    #
-    def initialize(app, &block)
-      @request_uri = block
-      @app = app
-    end
-
-    def call(env)
-      req = Rack::Request.new(env)
-      return app.call(env) unless uri = request_uri.call(req)
-      begin # only want to catch proxy errors, not app errors
-        proxy = ProxyRequest.new(req, uri)
-        [proxy.status, proxy.headers, proxy]
-      rescue => e
-        msg = "Proxy error when proxying to #{uri}: #{e.class}: #{e.message}"
-        env["rack.errors"].puts msg
-        env["rack.errors"].puts e.backtrace.map { |l| "\t" + l }
-        env["rack.errors"].flush
-        raise Error, msg
-      end
-    end
-
-    protected
-
-    attr_reader :request_uri, :app
-
-  end
-
-end
-
-require "rack"
-require "servolux"
-require "net/http"
-require "uri"
-
-Rack::StreamingProxy.require_all_libs_relative_to(__FILE__)
-
+require 'rack/streaming_proxy/version'
+require 'rack/streaming_proxy/proxy'
+require 'rack/streaming_proxy/railtie' if defined? ::Rails::Railtie

data/lib/rack/streaming_proxy/errors.rb

@@ -0,0 +1,5 @@
+module Rack::StreamingProxy
+  class Error           < RuntimeError; end
+  class UnknownError    < Error;        end
+  class HttpServerError < Error;        end
+end

data/lib/rack/streaming_proxy/proxy.rb

@@ -0,0 +1,115 @@
+require 'rack'
+require 'logger'
+require 'rack/streaming_proxy/session'
+require 'rack/streaming_proxy/request'
+require 'rack/streaming_proxy/response'
+
+class Rack::StreamingProxy::Proxy
+
+  class << self
+    attr_accessor :logger, :log_verbosity, :num_retries_on_5xx, :raise_on_5xx
+
+    def set_default_configuration
+      # Logs to stdout by default unless configured with another logger via Railtie.
+      @logger ||= Logger.new(STDOUT)
+
+      # At :low verbosity by default -- will not output :debug level messages.
+      # :high verbosity outputs :debug level messages.
+      # This is independent of the Logger's log_level, as set in Rails, for example,
+      # although the Logger's level can override this setting.
+      @log_verbosity ||= :low
+
+      # No retries are performed by default.
+      @num_retries_on_5xx ||= 0
+
+      # If the proxy cannot recover from 5xx's through retries (see num_retries_on_5xx),
+      # then it by default passes through the content from the destination
+      # e.g. the Apache error page. If you want an exception to be raised instead so
+      # you can handle it yourself (i.e. display your own error page), set raise_on_5xx to true.
+      @raise_on_5xx ||= false
+    end
+
+    def log(level, message)
+      unless log_verbosity == :low && level == :debug
+        @logger.send level, "[Rack::StreamingProxy] #{message}"
+      end
+    end
+
+  end
+
+  # The block provided to the initializer is given a Rack::Request
+  # and should return:
+  #
+  #   * nil/false to skip the proxy and continue down the stack
+  #   * a complete uri (with query string if applicable) to proxy to
+  #
+  # Example:
+  #
+  #   use Rack::StreamingProxy::Proxy do |req|
+  #     if req.path.start_with?('/search')
+  #       "http://some_other_service/search?#{req.query}"
+  #     end
+  #   end
+  #
+  # Most headers, request body, and HTTP method are preserved.
+  #
+  def initialize(app, &block)
+    self.class.set_default_configuration
+    @app   = app
+    @block = block
+  end
+
+  def call(env)
+    current_request = Rack::Request.new(env)
+
+    # Decide whether this request should be proxied.
+    if destination_uri = @block.call(current_request)
+      self.class.log :info, "Starting proxy request to: #{destination_uri}"
+
+      request  = Rack::StreamingProxy::Request.new(destination_uri, current_request)
+      begin
+        response = Rack::StreamingProxy::Session.new(request).start
+      rescue Exception => e # Rescuing only for the purpose of logging to rack.errors
+        log_rack_error(env, e)
+        raise e
+      end
+
+      # Notify client http version to the instance of Response class.
+      response.client_http_version = env['HTTP_VERSION'].sub(/HTTP\//, '') if env.has_key?('HTTP_VERSION')
+      # Ideally, both a Content-Length header field and a Transfer-Encoding 
+      # header field are not expected to be present from servers which 
+      # are compliant with RFC2616. However, irresponsible servers may send 
+      # both to rack-streaming-proxy.
+      # RFC2616 says if a message is received with both a Transfer-Encoding 
+      # header field and a Content-Length header field, the latter MUST be 
+      # ignored. So I deleted a Content-Length header here.
+      #
+      # Though there is a case that rack-streaming-proxy deletes both a 
+      # Content-Length and a Transfer-Encoding, a client can acknowledge the 
+      # end of body by closing the connection when the entire response has 
+      # been sent without a Content-Length header. So a Content-Length header 
+      # does not have to be required here in our understaing.
+      response.headers.delete('Content-Length') if response.headers.has_key?('Transfer-Encoding')
+      if env.has_key?('HTTP_VERSION') && env['HTTP_VERSION'] < 'HTTP/1.1'
+        # Be compliant with RFC2146
+        response.headers.delete('Transfer-Encoding')
+      end
+
+      self.class.log :info, "Finishing proxy request to: #{destination_uri}"
+      [response.status, response.headers, response]
+
+    # Continue down the middleware stack if the request is not to be proxied.
+    else
+      @app.call(env)
+    end
+  end
+
+private
+
+  def log_rack_error(env, e)
+    env['rack.errors'].puts e.message
+    env['rack.errors'].puts e.backtrace #.collect { |line| "\t" + line }
+    env['rack.errors'].flush
+  end
+
+end