plum 0.1.3 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: df4cac7083913b918a193ac157d4fd88bd1c0022
-  data.tar.gz: ef49c0a591dbcb1fa671b9f206705531f006e025
+  metadata.gz: 4194e9fa59a660fc839ea356770a05a627dba684
+  data.tar.gz: 5b4f22bd734f7d7df5dca01667716ecc3dc4e51e
 SHA512:
-  metadata.gz: 635b191791c1f6465d0a8e8f3d84c763506f9449c5e685c3ce4c80ed1b5bd22f1fc906d0f1f4c74a47e52e1d44dbea6245282575f88eff61af784fae8be1dbb2
-  data.tar.gz: 30cdfdd10ff68e47521f6d5928f920300706f9722bf32002844c18b2ced3d7215e2a24165a37abdcd9a53009173d5efc578ef5ddee61a766c2ca90af60cc9707
+  metadata.gz: 987380963b7702bd5a6aa0f4ab817ae515f34cbd1b74ba48ea2b46f5e27169173c80821af49f353bc61363a634107290bc4a43abe599a21707cbaadbc6446f0f
+  data.tar.gz: bd645808edbcf763c896e00085ab3d69d89639f44b1de09c3ac39f7ed2c79d6c5f313f2791dad57c43d31d68eeb9b59e745ba3e4040835e5670543f9284c905e

data/README.md

@@ -1,5 +1,9 @@
-# Plum [![Build Status](https://travis-ci.org/rhenium/plum.png?branch=master)](https://travis-ci.org/rhenium/plum) [![Code Climate](https://codeclimate.com/github/rhenium/plum/badges/gpa.svg)](https://codeclimate.com/github/rhenium/plum) [![Test Coverage](https://codeclimate.com/github/rhenium/plum/badges/coverage.svg)](https://codeclimate.com/github/rhenium/plum/coverage)
-A minimal pure Ruby implementation of HTTP/2 library / server.
+# Plum: An HTTP/2 Library for Ruby
+A pure Ruby HTTP/2 server and client implementation.
+
+WARNING: Plum is currently under heavy development. You *will* encounter bugs when using it.
+
+[![Circle CI](https://circleci.com/gh/rhenium/plum.svg?style=svg)](https://circleci.com/gh/rhenium/plum) [![Build Status](https://travis-ci.org/rhenium/plum.png?branch=master)](https://travis-ci.org/rhenium/plum) [![Code Climate](https://codeclimate.com/github/rhenium/plum/badges/gpa.svg)](https://codeclimate.com/github/rhenium/plum) [![Test Coverage](https://codeclimate.com/github/rhenium/plum/badges/coverage.svg)](https://codeclimate.com/github/rhenium/plum/coverage)
 
 ## Requirements
 * Ruby
@@ -7,18 +11,21 @@ A minimal pure Ruby implementation of HTTP/2 library / server.
   * or latest Ruby 2.3.0-dev
 * OpenSSL 1.0.2 or newer (HTTP/2 requires ALPN)
 * Optional:
-  * [http-parser.rb gem](https://rubygems.org/gems/http_parser.rb) (HTTP/1.1 parser; if you use "http" URI scheme)
-  * [rack gem](https://rubygems.org/gems/rack) if you use Plum as Rack server.
+  * [http_parser.rb gem](https://rubygems.org/gems/http_parser.rb) (HTTP/1.x parser; if you use "http" URI scheme)
+  * [rack gem](https://rubygems.org/gems/rack) (if you use Plum as Rack server)
+
+## Installation
+```sh
+gem install plum
+```
 
 ## Usage
-### As a library
-* See documentation: http://www.rubydoc.info/gems/plum
-* See examples in `examples/`
-* [rhenium/plum-server](https://github.com/rhenium/plum-server) - A static-file server for https://rhe.jp and http://rhe.jp.
+* Documentation: http://www.rubydoc.info/gems/plum
+* Some examples are in `examples/`
 
 ### As a Rack-compatible server
 
-Most existing Rack-based applications (plum doesn't support Rack hijack API) should work without modification.
+Most existing Rack-based applications should work without modification.
 
 ```ruby
 # config.ru
@@ -26,7 +33,7 @@ App = -> env {
   [
     200,
     { "Content-Type" => "text/plain" },
-    [" request: #{env["REQUEST_METHOD"]} #{env["PATH_INFO"]}"]
+    ["request: #{env["REQUEST_METHOD"]} #{env["PATH_INFO"]}"]
   ]
 }
 
@@ -36,13 +43,78 @@ run App
 You can run it:
 
 ```sh
-% plum -e production -p 8080 --https config.ru
+% plum -e production -p 8080 --https --cert server.crt --key server.key config.ru
 ```
 
-By default, Plum generates a dummy server certificate if `--cert` and `--key` options are not specified.
+NOTE: If `--cert` and `--key` are omitted, a temporary dummy certificate will be generated.
+
+### As a HTTP/2 (HTTP/1.x) client library
+If the server does't support HTTP/2, `Plum::Client` tries to use HTTP/1.x instead.
+
+```
+   +-----------------+
+   |:http2 option    | false
+   |(default: true)  |-------> HTTP/1.x
+   +-----------------+
+            v true
+   +-----------------+
+   |:scheme option   | "http"
+   |(default:"https")|-------> Try Upgrade from HTTP/1.1
+   +-----------------+
+            v "https"
+   +-----------------+
+   | ALPN(/NPN)      | failed
+   | negotiation     |-------> HTTP/1.x
+   +-----------------+
+            | "h2"
+            v
+          HTTP/2
+```
+
+##### Sequential request
+```ruby
+client = Plum::Client.start("http2.rhe.jp", 443, user_agent: "nyaan")
+res1 = client.get!("/", headers: { "accept" => "*/*" })
+puts res1.body # => "..."
+res2 = client.post!("/post", "data")
+puts res2.body # => "..."
+
+client.close
+```
+
+##### Parallel request
+```ruby
+res1 = res2 = nil
+Plum::Client.start("rhe.jp", 443, http2_settings: { max_frame_size: 32768 }) { |client|
+  res1 = client.get("/")
+  res2 = client.post("/post", "data")
+  # res1.status == nil ; because it's async request
+} # wait for response(s) and close
+
+p res1.status # => "200"
+```
+
+##### Download a large file
+```ruby
+Plum::Client.start("http2.rhe.jp", 443, hostname: "assets.rhe.jp") { |client|
+  client.get("/large") do |res| # called when received response headers
+    p res.status # => "200"
+    File.open("/tmp/large.file", "wb") { |file|
+      res.on_chunk do |chunk| # called when each chunk of response body arrived
+        file << chunk
+      end
+    }
+  end
+}
+```
 
 ## TODO
 * **Better API**
+* Plum::Client
+  * PING frame handling
+  * Server Push support
+  * Stream Priority support
+  * Better HTTP/1.x support
 
 ## License
 MIT License

data/circle.yml

@@ -0,0 +1,27 @@
+machine:
+  environment:
+    CODECLIMATE_REPO_TOKEN: f5092ab344fac7f2de9d7332e00597642a4d24e3d560f7d7f329172a2e5a2def
+dependencies:
+  pre:
+    - echo openssl_url=https://www.openssl.org/source >> $rvm_path/user/db
+    - echo openssl_version=1.0.2d >> $rvm_path/user/db
+    - rvm pkg install openssl
+    - >
+      case $CIRCLE_NODE_INDEX in
+        0)
+          rvm install 2.2.3-alpn --with-openssl-dir=$rvm_path/usr --patch https://gist.githubusercontent.com/rhenium/b1711edcc903e8887a51/raw/2309e469f5a3ba15917d804ac61b19e62b3d8faf/ruby-openssl-alpn-no-tests-and-docs.patch
+          rvm use 2.2.3-alpn --default
+          ;;
+        1)
+          rvm install ruby-head --with-openssl-dir=$rvm_path/usr
+          rvm use ruby-head --default
+          ;;
+      esac
+  override:
+    - gem install bundler
+    - bundle install
+test:
+  override:
+    - $rvm_path/usr/bin/openssl version
+    - ruby -v
+    - bundle exec rake test

data/examples/client/large.rb

@@ -0,0 +1,20 @@
+# -*- frozen-string-literal: true -*-
+# client/large.rb: download 3 large files in parallel
+$LOAD_PATH.unshift File.expand_path("../../../lib", __FILE__)
+require "plum"
+
+def log(s)
+  puts "[#{Time.now.strftime("%Y/%m/%d %H:%M:%S.%L")}] #{s}"
+end
+
+Plum::Client.start("http2.golang.org", 443, http2_settings: { max_frame_size: 32768 }) { |rest|
+  3.times { |i|
+    rest.get("/file/go.src.tar.gz",
+             headers: { "accept-encoding" => "identity;q=1" }) { |res|
+      log "#{i}: #{res.status}"
+      res.on_chunk { |chunk|
+        log "#{i}: chunk: #{chunk.size}"
+      }
+    }
+  }
+}

data/examples/client/twitter.rb

@@ -0,0 +1,51 @@
+# -*- frozen-string-literal: true -*-
+# client/twitter.rb
+# Twitter の User stream に（現在はサーバーが非対応のため）HTTP/1.1 を使用して接続する。
+# 「にゃーん」を含むツイートを受信したら、REST API で HTTP/2 を使用して返信する。
+$LOAD_PATH.unshift File.expand_path("../../../lib", __FILE__)
+require "plum"
+require "json"
+require "cgi"
+require "simple_oauth"
+
+credentials = { consumer_key: "",
+                consumer_secret: "",
+                token: "",
+                token_secret: "" }
+
+rest = Plum::Client.start("api.twitter.com", 443)
+Plum::Client.start("userstream.twitter.com", 443) { |streaming|
+  streaming.get("/1.1/user.json",
+                headers: { "authorization" => SimpleOAuth::Header.new(:get, "https://userstream.twitter.com/1.1/user.json", {}, credentials).to_s,
+                           "accept-encoding" => "identity;q=1" }) { |res| # plum doesn't have built-in gzip/deflate decoder
+    if res.status != "200"
+      puts "failed userstream"
+      exit
+    end
+
+    buf = String.new
+    res.on_chunk { |chunk| # when received DATA frame
+      buf << chunk
+      *msgs, buf = buf.split("\r\n", -1)
+
+      msgs.each do |msg|
+        next if msg.empty?
+
+        json = JSON.parse(msg)
+        next unless json["user"] # unless it is a tweet
+
+        puts "@#{json["user"]["screen_name"]}: #{json["text"]}"
+
+        if /にゃーん/ =~ json["text"]
+          args = { "status" => "@#{json["user"]["screen_name"]} にゃーん",
+                   "in_reply_to_status_id" => json["id"].to_s }
+          rest.post!("/1.1/statuses/update.json",
+                     args.map { |k, v| "#{k}=#{CGI.escape(v)}" }.join("&"),
+                     headers: { "authorization" => SimpleOAuth::Header.new(:post, "https://api.twitter.com/1.1/statuses/update.json", args, credentials).to_s,
+                                "content-type" => "application/x-www-form-urlencoded" })
+        end
+      end
+    }
+  }
+}
+rest.close

data/examples/non_tls_server.rb

@@ -25,7 +25,7 @@ loop do
     next
   end
 
-  plum = Plum::HTTPConnection.new(sock)
+  plum = Plum::HTTPServerConnection.new(sock)
 
   plum.on(:frame) do |frame|
     log(id, frame.stream_id, "recv: #{frame.inspect}")
@@ -77,42 +77,48 @@ loop do
         <input type=submit>
         </form>
         EOF
-        stream.respond({
+        stream.send_headers({
           ":status": "200",
           "server": "plum",
           "content-type": "text/html",
           "content-length": body.bytesize
-        }, body)
+        }, end_stream: false)
+        stream.send_data(body, end_stream: true)
       when ["POST", "/post.page"]
         body = "Posted value is: #{CGI.unescape(data).gsub("<", "&lt;").gsub(">", "&gt;")}<br> <a href=/>Back to top page</a>"
-        stream.respond({
+        stream.send_headers({
           ":status": "200",
           "server": "plum",
           "content-type": "text/html",
           "content-length": body.bytesize
-        }, body)
+        }, end_stream: false)
+        stream.send_data(body, end_stream: true)
       else
         body = "Page not found! <a href=/>Back to top page</a>"
-        stream.respond({
+        stream.send_headers({
           ":status": "404",
           "server": "plum",
           "content-type": "text/html",
           "content-length": body.bytesize
-        }, body)
+        }, end_stream: false)
+        stream.send_data(body, end_stream: true)
       end
     end
   end
 
   Thread.new {
     begin
-      plum.run
+      while !sock.closed? && !sock.eof?
+        plum << sock.readpartial(1024)
+      end
     rescue Plum::LegacyHTTPError
+      data = "Use modern web browser with HTTP/2 support."
       resp = "HTTP/1.1 505 HTTP Version Not Supported\r\n"
              "Content-Type: text/plain\r\n"
              "Content-Length: #{data.bytesize}\r\n"
              "Server: plum/#{Plum::VERSION}\r\n"
              "\r\n"
-             "Use modern web browser with HTTP/2 support."
+             "#{data}"
 
       sock.write(resp)
     rescue

data/examples/static_server.rb

@@ -43,7 +43,7 @@ loop do
     next
   end
 
-  plum = Plum::HTTPSConnection.new(sock)
+  plum = Plum::HTTPSServerConnection.new(sock)
 
   plum.on(:frame) do |frame|
     log(id, frame.stream_id, "recv: #{frame.inspect}")
@@ -95,59 +95,66 @@ loop do
         <input type=submit>
         </form>
         EOF
-        stream.respond({
+        stream.send_headers({
           ":status": "200",
           "server": "plum",
           "content-type": "text/html",
-          "content-length": body.size
-        }, body)
+          "content-length": body.bytesize
+        }, end_stream: false)
+        stream.send_data(body, end_stream: true)
       when ["GET", "/abc.html"]
         body = "ABC! <a href=/>Back to top page</a><br><img src=/image.nyan>"
+        stream.send_headers({
+          ":status": "200",
+          "server": "plum",
+          "content-type": "text/html",
+          "content-length": body.bytesize
+        }, end_stream: false)
         i_stream = stream.promise({
           ":authority": "localhost:40443",
           ":method": "GET",
           ":scheme": "https",
           ":path": "/image.nyan"
         })
-        stream.respond({
-          ":status": "200",
-          "server": "plum",
-          "content-type": "text/html",
-          "content-length": body.size
-        }, body)
-        image = ("iVBORw0KGgoAAAANSUhEUgAAAEAAAABAAgMAAADXB5lNAAAACVBMVEX///93o0jG/4mTMy20AAAA"
-                 "bklEQVQ4y2NgoAoIRQJkCoSimIdTgJGBBU1ABE1A1AVdBQuaACu6gCALhhZ0axlZCDgMWYAB6ilU"
-                 "35IoADEMxWyyBDD45AhQCFahM0kXWIVu3sAJrILzyBcgytoFeATABBcXWohhCEC14BCgGAAAX1ZQ"
+        stream.send_data(body, end_stream: true)
+        image = ("iVBORw0KGgoAAAANSUhEUgAAAEAAAABAAgMAAADXB5lNAAAACVBMVEX///93o0jG/4mTMy20AAAA" \
+                 "bklEQVQ4y2NgoAoIRQJkCoSimIdTgJGBBU1ABE1A1AVdBQuaACu6gCALhhZ0axlZCDgMWYAB6ilU" \
+                 "35IoADEMxWyyBDD45AhQCFahM0kXWIVu3sAJrILzyBcgytoFeATABBcXWohhCEC14BCgGAAAX1ZQ" \
                  "ZtJp0zAAAAAASUVORK5CYII=").unpack("m")[0]
-        i_stream.respond({
+        i_stream.send_headers({
           ":status": "200",
           "server": "plum",
           "content-type": "image/png",
-          "content-length": image.size
-        }, image)
+          "content-length": image.bytesize
+        }, end_stream: false)
+        i_stream.send_data(image, end_stream: true)
       when ["POST", "/post.page"]
         body = "Posted value is: #{CGI.unescape(data).gsub("<", "&lt;").gsub(">", "&gt;")}<br> <a href=/>Back to top page</a>"
-        stream.respond({
+        stream.send_headers({
           ":status": "200",
           "server": "plum",
           "content-type": "text/html",
-          "content-length": body.size
-        }, body)
+          "content-length": body.bytesize
+        }, end_stream: false)
+        stream.send_data(body, end_stream: true)
       else
         body = "Page not found! <a href=/>Back to top page</a>"
-        stream.respond({
+        stream.send_headers({
           ":status": "404",
           "server": "plum",
           "content-type": "text/html",
-          "content-length": body.size
-        }, body)
+          "content-length": body.bytesize
+        }, end_stream: false)
+        stream.send_data(body, end_stream: true)
       end
     end
   end
 
   Thread.new {
     begin
-      plum.run
+      while !sock.closed? && !sock.eof?
+        plum << sock.readpartial(1024)
+      end
     rescue
       puts $!
       puts $!.backtrace

data/lib/plum.rb

@@ -17,7 +17,14 @@ require "plum/frame"
 require "plum/flow_control"
 require "plum/connection_utils"
 require "plum/connection"
-require "plum/https_connection"
-require "plum/http_connection"
 require "plum/stream_utils"
 require "plum/stream"
+require "plum/server/connection"
+require "plum/server/https_connection"
+require "plum/server/http_connection"
+require "plum/client"
+require "plum/client/response"
+require "plum/client/connection"
+require "plum/client/client_session"
+require "plum/client/legacy_client_session"
+require "plum/client/upgrade_client_session"

data/lib/plum/client.rb

@@ -0,0 +1,198 @@
+# -*- frozen-string-literal: true -*-
+module Plum
+  class Client
+    DEFAULT_CONFIG = {
+      http2: true,
+      scheme: "https",
+      hostname: nil,
+      verify_mode: OpenSSL::SSL::VERIFY_PEER,
+      ssl_context: nil,
+      http2_settings: {},
+      user_agent: "plum/#{Plum::VERSION}",
+    }.freeze
+
+    attr_reader :host, :port, :config
+    attr_reader :socket, :session
+
+    # Creates a new HTTP client and starts communication.
+    # A shorthand for `Plum::Client.new(args).start(&block)`
+    def self.start(host, port = nil, config = {}, &block)
+      client = self.new(host, port, config)
+      client.start(&block)
+    end
+
+    # Creates a new HTTP client.
+    # @param host [String | IO] the host to connect, or IO object.
+    # @param port [Integer] the port number to connect
+    # @param config [Hash<Symbol, Object>] the client configuration
+    def initialize(host, port = nil, config = {})
+      if host.is_a?(IO)
+        @socket = host
+      else
+        @host = host
+        @port = port || (config[:scheme] == "https" ? 443 : 80)
+      end
+      @config = DEFAULT_CONFIG.merge(hostname: host).merge(config)
+      @started = false
+    end
+
+    # Starts communication.
+    # If block passed, waits for asynchronous requests and closes the connection after calling the block.
+    def start(&block)
+      raise IOError, "Session already started" if @started
+      _start
+      if block_given?
+        begin
+          ret = yield(self)
+          resume
+          return ret
+        ensure
+          close
+        end
+      end
+      self
+    end
+
+    # Resume communication with the server, until the specified (or all running) requests are complete.
+    # @param response [Response] if specified, waits only for the response
+    # @return [Response] if parameter response is specified
+    def resume(response = nil)
+      if response
+        @session.succ until response.failed? || response.finished?
+        response
+      else
+        @session.succ until @session.empty?
+      end
+    end
+
+    # Closes the connection immediately.
+    def close
+      @session.close if @session
+    ensure
+      @socket.close if @socket
+    end
+
+    # Creates a new HTTP request.
+    # @param headers [Hash<String, String>] the request headers
+    # @param body [String] the request body
+    # @param options [Hash<Symbol, Object>] request options
+    # @param block [Proc] if passed, it will be called when received response headers.
+    def request(headers, body, options = {}, &block)
+      raise ArgumentError, ":method and :path headers are required" unless headers[":method"] && headers[":path"]
+      @session.request(headers, body, options, &block)
+    end
+
+    # @!method get!
+    # @!method head!
+    # @!method delete!
+    # @param path [String] the absolute path to request (translated into :path header)
+    # @param options [Hash<Symbol, Object>] the request options
+    # @param block [Proc] if specified, calls the block when finished
+    # Shorthand method for `Client#resume(Client#request(*args))`
+
+    # @!method get
+    # @!method head
+    # @!method delete
+    # @param path [String] the absolute path to request (translated into :path header)
+    # @param options [Hash<Symbol, Object>] the request options
+    # @param block [Proc] if specified, calls the block when finished
+    # Shorthand method for `#request`
+    %w(GET HEAD DELETE).each { |method|
+      define_method(:"#{method.downcase}!") do |path, options = {}, &block|
+        resume _request_helper(method, path, nil, options, &block)
+      end
+      define_method(:"#{method.downcase}") do |path, options = {}, &block|
+        _request_helper(method, path, nil, options, &block)
+      end
+    }
+    # @!method post!
+    # @!method put!
+    # @param path [String] the absolute path to request (translated into :path header)
+    # @param body [String] the request body
+    # @param options [Hash<Symbol, Object>] the request options
+    # @param block [Proc] if specified, calls the block when finished
+    # Shorthand method for `Client#resume(Client#request(*args))`
+
+    # @!method post
+    # @!method put
+    # @param path [String] the absolute path to request (translated into :path header)
+    # @param body [String] the request body
+    # @param options [Hash<Symbol, Object>] the request options
+    # @param block [Proc] if specified, calls the block when finished
+    # Shorthand method for `#request`
+    %w(POST PUT).each { |method|
+      define_method(:"#{method.downcase}!") do |path, body, options = {}, &block|
+        resume _request_helper(method, path, body, options, &block)
+      end
+      define_method(:"#{method.downcase}") do |path, body, options = {}, &block|
+        _request_helper(method, path, body, options, &block)
+      end
+    }
+
+    private
+    # @return [Boolean] http2 nego?
+    def _connect
+      @socket = TCPSocket.open(@host, @port)
+
+      if @config[:scheme] == "https"
+        ctx = @config[:ssl_context] || new_ssl_ctx
+        @socket = OpenSSL::SSL::SSLSocket.new(@socket, ctx)
+        @socket.hostname = @config[:hostname] if @socket.respond_to?(:hostname=)
+        @socket.sync_close = true
+        @socket.connect
+        @socket.post_connection_check(@config[:hostname]) if ctx.verify_mode != OpenSSL::SSL::VERIFY_NONE
+
+        @socket.respond_to?(:alpn_protocol) && @socket.alpn_protocol == "h2" ||
+          @socket.respond_to?(:npn_protocol) && @socket.npn_protocol == "h2"
+      end
+    end
+
+    def _start
+      @started = true
+
+      klass = @config[:http2] ? ClientSession : LegacyClientSession
+      nego = @socket || _connect
+
+      if @config[:http2]
+        if @config[:scheme] == "https"
+          klass = nego ? ClientSession : LegacyClientSession
+        else
+          klass = UpgradeClientSession
+        end
+      else
+        klass = LegacyClientSession
+      end
+
+      @session = klass.new(@socket, @config)
+    end
+
+    def new_ssl_ctx
+      ctx = OpenSSL::SSL::SSLContext.new
+      ctx.ssl_version = :TLSv1_2
+      ctx.verify_mode = @config[:verify_mode]
+      cert_store = OpenSSL::X509::Store.new
+      cert_store.set_default_paths
+      ctx.cert_store = cert_store
+      if @config[:http2]
+        ctx.ciphers = "ALL:!" + HTTPSServerConnection::CIPHER_BLACKLIST.join(":!")
+        if ctx.respond_to?(:alpn_protocols)
+          ctx.alpn_protocols = ["h2", "http/1.1"]
+        end
+        if ctx.respond_to?(:npn_select_cb) # TODO: RFC 7540 does not define protocol negotiation with NPN
+          ctx.npn_select_cb = -> protocols {
+            protocols.include?("h2") ? "h2" : protocols.first
+         }
+        end
+      end
+      ctx
+    end
+
+    def _request_helper(method, path, body, options, &block)
+      base = { ":method" => method,
+               ":path" => path,
+               "user-agent" => @config[:user_agent] }
+      base.merge!(options[:headers]) if options[:headers]
+      request(base, body, options, &block)
+    end
+  end
+end