cap_proxy 0.1.0

data/README.md

@@ -0,0 +1,194 @@
+# CapProxy
+
+[![Build Status](https://travis-ci.org/ayosec/cap_proxy.svg)](https://travis-ci.org/ayosec/cap_proxy)
+
+HTTP proxy with the ability to capture requests and generate a fake response. It is *not* intended to use in production environments, but only in integration tests.
+
+It is tested in Ruby MRI (1.9, 2.0, 2.1), JRuby and Rubinius. Check out the [Travis page](https://travis-ci.org/ayosec/cap_proxy) to see the current status on every platform.
+
+## Usage
+
+### Installation
+
+Install the gem with
+
+    $ gem install cap_proxy
+
+Or add it to your `Gemfile`
+
+```ruby
+group :test do
+  gem "cap_proxy"
+end
+```
+
+Finally, load it into your application
+
+```ruby
+require "cap_proxy/server"
+```
+
+### Create a proxy instance
+
+The main class to use the proxy is `CapProxy::Server`. You have to indicate the address where the proxy will be listening, and the address of the HTTP server which receives every non-captured request.
+
+When the proxy is initialized, you have to call to the `#run!` method to start the server. This method has to be invoked when [EventMachine](http://eventmachine.rubyforge.org/) is running.
+
+```ruby
+proxy = CapProxy::Server.new("localhost:1234", "localhost:5678")
+
+EM.run do
+  proxy.run!
+end
+```
+
+You can use a logger with the proxy:
+
+```ruby
+logger = Logger.new(STDOUT)
+proxy = CapProxy::Server.new("localhost:1234", "localhost:5678", logger)
+
+EM.run do
+  proxy.run!
+end
+```
+
+Now, every connection to `http://localhost:1234` will be forwarded to `http://localhost:5678`.
+
+### Testing
+
+For your convenience, if you are using RSpec you can use `CapProxy::TestWrapper` to wrap every example.
+
+```ruby
+require "cap_proxy/testkit"
+
+# ...
+
+around :each do |example|
+  CapProxy::TestWrapper.run(example, "localhost:4001", "localhost:3000") do |proxy|
+    @proxy = proxy
+  end
+end
+```
+
+`CapProxy::TestWrapper` will initialize EventMachine, creates a proxy running in it, and launch the example in a different thread. When the example is finished, the EventMachine is stopped.
+
+### Capturing and manipulating requests
+
+With `#capture` you can capture any request, and generate a dummy response for it. You have to define a filter, and a block to generate the response.
+
+```ruby
+@proxy.capture(method: "get", path: "/users") do |client, request|
+  client.respond 404, {}, "Dummy response"
+end
+```
+
+Details about how to capture requests, and to generate a response for it, are in the section *Capturing requests*.
+
+## Capturing requests
+
+The first step is to define a filter. If a request matches multiple filters, it will be managed by the oldest one. If no filter matches the request, it will be forwarded to the default HTTP server.
+
+### Defining a filter
+
+The easiest way to define a filter is with a hash, which can contain any of the following items:
+
+* `:method`
+* `:path`
+* `:headers`
+
+`:method` can accept any string to represent a HTTP method (`"get"` and `"GET"` are equivalent).
+
+```ruby
+@proxy.capture(method: "get") { ... }
+```
+
+`:path` can be defined with a string (full URI has to match), or with a regular expression (matched with the `=~` operator).
+
+```ruby
+@proxy.capture(path: "/users") { ... }
+
+@proxy.capture(method: "post", path: %r{/users/\d+}) { ... }
+```
+
+`:headers` expects a hash with the headers to be matched. The value of every header can be a string or a regular expression.
+
+```ruby
+@proxy.capture(path: "/", headers: { "content-type" => /json/ }) do
+  ...
+end
+```
+
+### Custom filters
+
+If you need more control to filter the request, you can create your own filter
+
+```ruby
+class MyFilter < CapProxy::Filter
+
+  def apply?(request)
+    # Conditions
+  end
+
+end
+```
+
+The `#apply?` method receives an instance of `Http::Parser`, from the [http_parser.rb gem](https://github.com/tmm1/http_parser.rb). You can use methods like `http_method`, `request_url` or `headers` to query info about the request.
+
+If `#apply?` returns `false` or `nil`, the filter will skip this request.
+
+To use your filter, create an instance of it:
+
+```ruby
+@proxy.capture(MyFilter.new) { ... }
+```
+
+### Generating responses
+
+The block of the `capture` method is invoked with two arguments: the first one is an instance of `CapProxy::Client`. The last one is the instance of `HTTP::Parser`.
+
+The simplest way to generate a response is calling to `client.respond(status, headers, body)`.
+
+```ruby
+@proxy.capture(path: "/users", method: "post") do |client, request|
+  client.respond 201, {"Content-Type" => "application/json"}, %q[{"id": 123}]
+end
+```
+
+### Chunked responses
+
+You can generate a response in [multiple chunks](http://en.wikipedia.org/wiki/Chunked_transfer_encoding).
+
+First, you have to call to `client.chunks_start(status, headers)`, to initialize the chunked response. Then, for every chunk, call to `client.chunks_send(data)`. Finally, finish the connection with `client.chunks_finish`
+
+```ruby
+@proxy.capture(path: "/chunks") do |client, request|
+  client.chunks_start 200, "content-type" => "text/plain"
+  EM.add_timer(0.4) { client.chunks_send("Cap") }
+  EM.add_timer(0.8) { client.chunks_send("Proxy\n") }
+  EM.add_timer(1.2) { client.chunks_finish }
+end
+```
+
+## Example
+
+```ruby
+require "cap_proxy/server"
+require "cap_proxy/testkit"
+
+describe MyApp do
+
+  around :each do |test|
+    CapProxy::TestWrapper.run(test, "localhost:4001", "localhost:3000") do |proxy|
+      @proxy = proxy
+    end
+  end
+
+  it "should do it" do
+    @proxy.capture(method: "get", path: "/users") do |client, request|
+      request.request_url.should include("foo_id")
+      client.respond 404, {}, "Dummy response"
+    end
+  end
+end
+```

data/lib/cap_proxy/client.rb

@@ -0,0 +1,90 @@
+require "eventmachine"
+require "http/parser"
+require_relative "remote_connection"
+require_relative "http_codes"
+
+module CapProxy
+  class Client < EM::Connection
+    attr_reader :server, :head
+
+    def initialize(server)
+      @server = server
+      @remote = nil
+      @data = nil
+      @http_parser = HTTP::Parser.new
+
+      @http_parser.on_headers_complete = proc do
+        verify_headers!
+      end
+    end
+
+    def unbind
+      if @remote
+        @remote.close_connection_after_writing
+      end
+    end
+
+    def write_head(status, headers)
+      head = [ "HTTP/1.1 #{status} #{HTTPCodes[status]}\r\n" ]
+
+      if headers
+        headers.each_pair do |key, value|
+          head << "#{key}: #{value}\r\n"
+        end
+      end
+
+      head << "\r\n"
+      send_data head.join
+    end
+
+    def respond(status, headers, body = nil)
+      write_head(status, headers)
+      send_data(body) if body
+      close_connection_after_writing
+    end
+
+    def verify_headers!
+      parser = @http_parser
+      filter = server.filters.find {|f| f[:filter].apply?(parser) }
+
+      server.log.info "#{parser.http_method} #{parser.request_url} - #{filter ? "filtered" : "raw"}" if server.log
+      if filter
+        filter[:handler].call self, parser
+      else
+        @remote = EM.connect(server.target_host, server.target_port, RemoteConnection, self)
+        @remote.send_data @data
+        @data = nil
+        @http_parser = nil
+      end
+    end
+
+    def receive_data(data)
+      if @remote
+        @remote.send_data data
+      else
+        if @data
+          @data << data
+        else
+          @data = data
+        end
+        @http_parser << data
+      end
+    end
+
+    def chunks_start(status, headers = {})
+      write_head(status, headers.merge("Transfer-Encoding" => "chunked"))
+    end
+
+    def chunks_send(data)
+      send_data "#{data.bytesize.to_s(16)}\r\n"
+      send_data data
+      send_data "\r\n"
+    end
+
+    def chunks_finish
+      send_data "0\r\n\r\n"
+      close_connection_after_writing
+    end
+
+  end
+end

data/lib/cap_proxy/filter.rb

@@ -0,0 +1,75 @@
+module CapProxy
+
+  class InvalidFilterParam < RuntimeError; end
+  class InvalidRule < RuntimeError; end
+
+  class Filter
+    def self.from_hash(hash)
+      RulesEngine.new(
+        hash.each_pair.map do |param, value|
+          case param
+          when :method
+            [ [ :method, value.upcase ] ]
+          when :path
+            case value
+            when Regexp
+              [ [ :path_regexp, value ] ]
+            when String
+              [ [ :path, value ] ]
+            else
+              raise InvalidFilterParam.new("Invalid value #{value.inspect} for :path")
+            end
+          when :headers
+            value.each_pair.map do |header, value|
+              case value
+              when String
+                [ :header, header.downcase, value ]
+              when Regexp
+                [ :header_regexp, header.downcase, value ]
+              else
+                raise InvalidRule.new("Invalid header rule #{value.inspect}")
+              end
+            end
+          else
+            raise InvalidFilterParam.new("Invalid item #{param.inspect}")
+          end
+        end.inject {|a, b| a.concat(b)}
+      )
+    end
+
+    def apply?(request)
+      raise NotImplementedError.new("apply? has to be implemented by inherited classes")
+    end
+  end
+
+  class RulesEngine < Filter
+    def initialize(rules)
+      raise InvalidRule.new("At least one rule is required") if rules.empty?
+      @rules = rules
+    end
+
+    def apply?(request)
+      @rules.all? do |rule|
+        case rule[0]
+        when :method
+          rule[1] == request.http_method
+        when :path_regexp
+          request.request_url =~ rule[1]
+        when :path
+          request.request_url == rule[1]
+        when :header_regexp
+          request.headers.each_pair.any? do |rh, rv|
+            rh.downcase == rule[1] && rv =~ rule[2]
+          end
+        when :header
+          request.headers.each_pair.any? do |rh, rv|
+            rh.downcase == rule[1] && rv == rule[2]
+          end
+        else
+          raise InvalidRule.new("Invalid rule #{rule.inspect}")
+        end
+      end
+    end
+  end
+
+end

data/lib/cap_proxy/http_codes.rb

@@ -0,0 +1,48 @@
+module CapProxy
+
+  HTTPCodes = {
+
+    100 => "Continue",
+    101 => "Switch Protocol",
+    200 => "OK",
+    201 => "Created",
+    202 => "Accepted",
+    203 => "Non Authoritative Information",
+    204 => "No Content",
+    205 => "Reset Content",
+    206 => "Partial Content",
+    300 => "Multiple Choice",
+    301 => "Moved Permanently",
+    302 => "Found",
+    303 => "See Other",
+    304 => "Not Modified",
+    305 => "Use Proxy",
+    307 => "Temporary Redirect",
+    400 => "Bad Request",
+    401 => "Unauthorized",
+    402 => "Payment Required",
+    403 => "Forbidden",
+    404 => "Not Found",
+    405 => "Method Not Allowed",
+    406 => "Not Acceptable",
+    407 => "Proxy Authentication Required",
+    408 => "Request Time Out",
+    409 => "Conflict",
+    410 => "Gone",
+    411 => "Length Required",
+    412 => "Precondition Failed",
+    413 => "Request Entity Too Large",
+    414 => "Request URIToo Long",
+    415 => "Unsupported Media Type",
+    416 => "Requested Range Not Satisfiable",
+    417 => "Expectation Failed",
+    500 => "Internal Server Error",
+    501 => "Not Implemented",
+    502 => "Bad Gateway",
+    503 => "Service Unavailable",
+    504 => "Gateway Time Out",
+    505 => "Version Not Supported"
+
+  }
+
+end

data/lib/cap_proxy/remote_connection.rb

@@ -0,0 +1,21 @@
+require "eventmachine"
+
+module CapProxy
+  class RemoteConnection < EM::Connection
+    attr_reader :proxy_connection
+
+    def initialize(proxy_connection)
+      @proxy_connection = proxy_connection
+    end
+
+    def receive_data(data)
+      log = proxy_connection.server.log
+      log.debug("Closing #{proxy_connection.head}") if log
+      proxy_connection.send_data data
+    end
+
+    def unbind
+      proxy_connection.close_connection_after_writing
+    end
+  end
+end

data/lib/cap_proxy/server.rb

@@ -0,0 +1,49 @@
+require "eventmachine"
+require "thread_safe"
+require_relative "client"
+require_relative "filter"
+
+module CapProxy
+  class Server
+
+    attr_reader :proxy_port, :proxy_host, :target_host, :target_port, :log, :filters
+
+    def initialize(bind_address, target, log = nil)
+
+      proxy_host, proxy_port = bind_address.split(":", 2)
+      target_host, target_port = target.split(":", 2)
+
+      @log = log
+      @proxy_port = proxy_port
+      @proxy_host = proxy_host
+      @target_host = target_host
+      @target_port = target_port
+      reset_filters!
+    end
+
+    def reset_filters!
+      @filters = ThreadSafe::Array.new
+    end
+
+    def capture(filter_param, &handler)
+      filter =
+        case filter_param
+        when Hash
+          Filter.from_hash(filter_param)
+        when Filter
+          filter_param
+        else
+          raise RuntimeError("#capture requires a hash or a Filter object")
+        end
+
+      @filters.push filter: filter, handler: handler
+      filter
+    end
+
+    def run!
+      log.info "CapProxy: Listening on #{proxy_host}:#{proxy_port}" if log
+      EM.start_server proxy_host, proxy_port, Client, self
+    end
+
+  end
+end

data/lib/cap_proxy/testkit.rb

@@ -0,0 +1 @@
+require_relative "testkit/em_wrapper"

data/lib/cap_proxy/testkit/em_wrapper.rb

@@ -0,0 +1,45 @@
+require "eventmachine"
+require "uri"
+require "logger"
+
+module CapProxy
+
+  module TestWrapper
+
+    class SimpleResponder < EM::Connection
+      def receive_data(_)
+        send_data "TCPServer\n"
+        close_connection_after_writing
+      end
+    end
+
+    def self.run(test, bind_address, target)
+
+      EM.run do
+
+        proxy = nil
+
+        EM.error_handler do |error|
+          STDERR.puts error
+          STDERR.puts error.backtrace.map {|l| "\t#{l}" }
+        end
+
+        logger = Logger.new(STDERR)
+        logger.level = Logger::ERROR
+        proxy = Server.new(bind_address, target, logger)
+        proxy.run!
+
+        if block_given?
+          yield proxy
+        end
+
+        Thread.new do
+          test.run
+        end
+        EM.stop_event_loop
+      end
+
+    end
+  end
+
+end

data/spec/filters_spec.rb

@@ -0,0 +1,72 @@
+require "spec_helper"
+
+describe CapProxy::Filter do
+
+  context "Filters from hashes" do
+
+    def hash_filter?(hash, request)
+      parser = HTTP::Parser.new
+      parser << request
+
+      filter = CapProxy::Filter.from_hash(hash)
+      filter.apply?(parser)
+    end
+
+    it "filter by method and path" do
+      hash_filter?(
+        {method: "get", path: "/foo/bar"},
+        %[GET /foo/bar HTTP/1.0\r\n\r\n]
+      ).should be_true
+
+      hash_filter?(
+        {method: "post", path: "/foo/bar"},
+        %[GET /foo/bar HTTP/1.0\r\n\r\n]
+      ).should be_false
+
+      hash_filter?(
+        {method: "post", path: %r{/foo/bar/(\d+)}},
+        %[GET /foo/bar/100 HTTP/1.0\r\n\r\n]
+      ).should be_false
+    end
+
+    it "filter by path and headers" do
+
+      hash_filter?(
+        {
+          path: "/",
+          headers: {
+            "content-type" => /json/
+          }
+        },
+        %[GET / HTTP/1.0\r\nContent-Type: application/json\r\n\r\n]
+      ).should be_true
+
+      hash_filter?(
+        {
+          path: "/",
+          headers: {
+            "content-type" => /json/,
+            "user-agent" => "none"
+          }
+        },
+        %[GET / HTTP/1.0\r\nContent-Type: application/json\r\n\r\n]
+      ).should be_false
+
+      hash_filter?(
+        {
+          path: "/",
+          headers: {
+            "user-agent" => "none",
+            "Accept" => "*"
+          }
+        },
+        %[GET / HTTP/1.0\r\n] +
+        %[User-Agent: none\r\n] +
+        %[accept: *\r\n] +
+        %[\r\n]
+      ).should be_true
+    end
+
+  end
+
+end

data/spec/server_spec.rb

@@ -0,0 +1,80 @@
+# coding: utf-8
+
+require "spec_helper"
+require "net/http"
+
+describe CapProxy::Server do
+
+  around :each do |test|
+    CapProxy::TestWrapper.run(test, "localhost:50300", "localhost:50301") do |proxy|
+      EM.start_server "localhost", 50301, CapProxy::TestWrapper::SimpleResponder
+      @proxy = proxy
+    end
+  end
+
+  def proxy_req!(packet)
+    conn = TCPSocket.new("localhost", 50300)
+    conn.write(packet)
+    response = conn.read(512)
+    conn.close
+    response
+  end
+
+  it "should proxy basic HTTP requests" do
+    proxy_req!("GET / HTTP/1.0\r\n\r\n").should == "TCPServer\n"
+  end
+
+  it "should capture requests" do
+    @proxy.capture(method: "get", path: /x/) do |client, request|
+      request.request_url.should include("x")
+      client.respond 404, {}, "foobar"
+    end
+
+    proxy_req!("GET /a/x/ HTTP/1.0\r\n\r\n").should =~ %r[\AHTTP/1.1 404 Not Found\r\n.*foobar\Z]m
+    proxy_req!("POST /a/x/ HTTP/1.0\r\n\r\n").should == "TCPServer\n"
+  end
+
+  it "should respond with custom headers" do
+    @proxy.capture(method: "put") do |client, request|
+      request.http_method.should == "PUT"
+      client.respond 200, {"x-foo" => "that"}, "."
+    end
+
+    resp = proxy_req!("PUT / HTTP/1.0\r\n\r\n")
+    resp.should =~ %r[\AHTTP/1.1 200 OK\r\n]
+    resp.should include("x-foo: that\r\n")
+  end
+
+  it "should use a custom filter" do
+    class CaptureAll < CapProxy::Filter
+      def apply?(request)
+        true
+      end
+    end
+
+    @proxy.capture(CaptureAll.new) do |client, request|
+      client.respond 200, {}, "-captured-"
+    end
+
+    proxy_req!("PUT / HTTP/1.0\r\n\r\n").should include("-captured-")
+    proxy_req!("GET /foo HTTP/1.0\r\n\r\n").should include("-captured-")
+  end
+
+  it "should manage chunked responsed" do
+    @proxy.capture(method: "get") do |client, request|
+      client.chunks_start 201, "content-type" => "text/foo"
+      EM.add_timer(0.1) { client.chunks_send([-30, -100, -108, 97].pack("c*")) }
+      EM.add_timer(0.3) { client.chunks_send("abc") }
+      EM.add_timer(0.5) { client.chunks_finish }
+    end
+
+    start_time = Time.now.to_f
+    resp = Net::HTTP.get_response(URI("http://localhost:50300"))
+    resp.code.should == "201"
+    resp["Content-Type"].should == "text/foo"
+    resp.body.bytes.to_a.should == "✔aabc".bytes.to_a
+
+    (Time.now.to_f - start_time).should >= 0.5
+  end
+
+end

data/spec/spec_helper.rb

@@ -0,0 +1,3 @@
+require "rspec"
+require_relative "../lib/cap_proxy/server"
+require_relative "../lib/cap_proxy/testkit"

metadata

@@ -0,0 +1,138 @@
+--- !ruby/object:Gem::Specification
+name: cap_proxy
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+  prerelease: 
+platform: ruby
+authors:
+- Ayose Cazorla
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2014-04-23 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: eventmachine
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.0.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 1.0.3
+- !ruby/object:Gem::Dependency
+  name: http_parser.rb
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: thread_safe
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 10.3.0
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ~>
+      - !ruby/object:Gem::Version
+        version: 10.3.0
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: HTTP proxy with the ability to capture requests and generate a fake response.
+  It is *not* intended to use in production environments, but only in integration
+  tests.
+email: ayosec@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- lib/cap_proxy/client.rb
+- lib/cap_proxy/filter.rb
+- lib/cap_proxy/http_codes.rb
+- lib/cap_proxy/remote_connection.rb
+- lib/cap_proxy/server.rb
+- lib/cap_proxy/testkit.rb
+- lib/cap_proxy/testkit/em_wrapper.rb
+- spec/filters_spec.rb
+- spec/server_spec.rb
+- spec/spec_helper.rb
+homepage: https://github.com/ayosec/cap_proxy
+licenses:
+- MIT
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.23
+signing_key: 
+specification_version: 3
+summary: HTTP Proxy with ability to capture and manipulate requests.
+test_files: []