twirp 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: f3e14392e7046ad1e01f88323f3966e612dacc23
-  data.tar.gz: a12d8cc09c357bfd1501cb12594b83eacf2b0779
+  metadata.gz: de95a607dde7addc708877a41f8b0f99453f1653
+  data.tar.gz: 7e84e7a36ad28d2d79c8d1ed793194fb3b549552
 SHA512:
-  metadata.gz: cdf630358e9d36782f10f5069aa389fb97c588092cb6b2af89a72f5763b8d63f95d70d49d576bcf294de8ec495a71eb3f1ccbdd9cf53032d84108dccfdd5a7f1
-  data.tar.gz: c33a3b5aee715484475bffef8f325753d001a7c20ea69fbb180ffd4eb80db1123a6dd4c8f815905bf45c2a60fc3394b2965fdaa16a3d28696d1408b9d57a4104
+  metadata.gz: 7e553cb895062292854559ece9f472213cd66a4fd61bd81b671f9645d51d65057078692c4c6ef2ebf96697265efd0cd3d212b1d069cb08a588937e4a64770760
+  data.tar.gz: 2ad9fcca77b96fa7e5740be188c18d5f08e514acfb08c91bba1b785a5061f3cd2b325b074466076273ac94ae1b627ca0dd5f9dc247138fb7019205692a6304ed

data/README.md

@@ -1,32 +1,41 @@
-# Ruby Twirp
+# Twirp
 
-Twirp services and clients in Ruby.
+Ruby implementation for for [Twirp](https://github.com/twitchtv/twirp). It includes:
 
-### Installation
-Install the `twirp` gem:
+  * [twirp gem](https://rubygems.org/gems/twirp) with classes for:
+    * Twirp::Error
+    * Twirp::Service
+    * Twirp::Client
+  * `protoc-gen-twirp_ruby` protoc plugin for code generation.
+
+
+## Installation
+
+Add `gem "twirp"` to your Gemfile, or install with:
 
 ```sh
 ➜ gem install twirp
 ```
 
-Use `go get` to install the ruby_twirp protoc plugin:
+For code generation, Make sure that you have the [protobuf compiler](https://github.com/golang/protobuf) (install version 3+).
+And then use `go get` to install the ruby_twirp protoc plugin:
 
 ```sh
-➜ go get github.com/cyrusaf/ruby-twirp/protoc-gen-twirp_ruby
+➜ go get -u github.com/cyrusaf/ruby-twirp/protoc-gen-twirp_ruby
 ```
 
-You will also need:
 
- - [protoc](https://github.com/golang/protobuf), the protobuf compiler. You need
-   version 3+.
+## Usage
 
-### HelloWorld Example
+Let's make a `HelloWorld` service in Twirp.
 
-See the `example/` folder for the final product.
+### Code Generation
 
-First create a basic `.proto` file:
+Starting with a [Protobuf](https://developers.google.com/protocol-buffers/docs/proto3) file:
 
 ```protobuf
+// hello_world/service.proto
+
 syntax = "proto3";
 package example;
 
@@ -43,58 +52,142 @@ message HelloResponse {
 }
 ```
 
-Run the `protoc` binary to auto-generate `helloworld_pb.rb` and `haberdasher_twirp.rb` files:
+Run the `protoc` binary with the `twirp_ruby` plugin to auto-generate code:
 
 ```sh
-➜ protoc --proto_path=. ./haberdasher.proto --ruby_out=gen --twirp_ruby_out=gen
+➜ protoc --proto_path=. ./hello_world/service.proto --ruby_out=. --twirp_ruby_out=.
 ```
 
-Write a handler for the auto-generated service, this is your implementation:
+This will generate `/hello_world/service_pb.rb` and `hello_world/service_twirp.rb`. The generated code looks something like this:
+
+```ruby
+# Code generated by protoc-gen-twirp_ruby, DO NOT EDIT.
+require 'twirp'
+
+module Example
+  class HelloWorldService < Twirp::Service
+    package "example"
+    service "HelloWorld"
+    rpc :Hello, HelloRequest, HelloResponse, :ruby_method => :hello
+  end
+
+  class HelloWorldClient < Twirp::Client
+    client_for HelloWorldService
+  end
+end
+```
+
+If you don't have Proto files, or don't like the code-generation step, you can always define your service and/or client using the DSL.
+
+
+#### Implement the Service Handler
+
+The Service Handler is a simple class that has one method to handle each rpc call.
+For each method, the `intput` is an instance of the protobuf request message. The Twirp `env`
+contains metadata related to the request, and other fields that could have been set from before
+hooks (e.g. `env[:user_id]` from authentication).
 
 ```ruby
-class HellowWorldHandler
+class HelloWorldHandler
   def hello(input, env)
     {message: "Hello #{input.name}"}
   end
 end
 ```
 
-Initialize the service with your handler and mount it as a Rack app:
+### Mount the service to receive HTTP requests
+
+The service is a Rack app instantiated with your handler impementation.
 
 ```ruby
 require 'rack'
-require_relative 'gen/haberdasher_pb.rb'
-require_relative 'gen/haberdasher_twirp.rb'
 
-handler = HellowWorldHandler.new()
-service = Example::HelloWorld.new(handler)
+handler = HelloWorldHandler.new() # your implementation
+service = Example::HelloWorldService.new(handler) # twirp-generated
+
 Rack::Handler::WEBrick.run service
 ```
 
-You can also mount onto a rails app:
+Since it is a Rack app, it can easily be mounted onto a Rails route with `mount service, at: service.full_name`.
+
+Now you can start the server and `curl` with JSON to test if everything works:
+
+```sh
+➜ curl --request POST \
+  --url http://localhost:8080/example.HelloWorld/Hello \
+  --header 'Content-Type: application/json' \
+  --data '{"name": "World"}'
+```
+
+### Unit testing the Service Handler
+
+Twirp already takes care of HTTP routing and serialization, you don't really need to build fake HTTP requests in your tests.
+Instead, you should focus on testing your Service Handler. For convenience, the Twirp Service has the method
+`.call_rpc(rpc_method, attrs={}, env={})` to call the handler with a fake Twirp env and making sure that the handler output is valid.
 
 ```ruby
-App::Application.routes.draw do
-  mount service, at: service.full_name
+require 'minitest/autorun'
+
+class HelloWorldHandlerTest < Minitest::Test
+  def test_hello_responds_with_name
+    service = Example::HelloWorld.new(HelloWorldHandler.new())
+    out = service.call_rpc :Hello, name: "World"
+    assert_equal "Hello World", out.message
+  end
 end
 ```
 
-Twirp services accept both Protobuf and JSON messages. It is easy to `curl` your service to get a response:
 
-```sh
-curl --request POST \
-  --url http://localhost:8080/example.HelloWorld/Hello \
-  --header 'content-type: application/json' \
-  --data '{"name":"World"}'
+## Clients
+
+Generated clients implement the methods defined in the proto file. The response object contains `data` with an instance of the response class if successfull,
+or an `error` with an instance of `Twirp::Error` if there was a problem. For example, with the HelloWorld generated client:
+
+```ruby
+c = Example::HelloWorldClient.new("http://localhost:3000")
+resp = c.hello(name: "World")
+if resp.error
+  puts resp.error #=> <Twirp::Error code:... msg:"..." meta:{...}>
+else
+  puts resp.data #=> <Example::HelloResponse: message:"Hello World">
+end
 ```
 
+You can also use the DSL to define your own client if you don't have easy access to the proto file or generated code:
+
+```ruby
+class MyClient < Twirp::Client
+  package "example"
+  service "MyService"
+  rpc :MyMethod, ReqClass, RespClass, :ruby_method => :my_method
+end
+
+c = MyClient.new("http://localhost:3000")
+resp = c.my_method(ReqClass.new())
+```
+
+
+### Configure client with Faraday
 
-## Hooks
+A Twirp client takes care of routing, serialization and error handling.
+
+Other advanced HTTP options can be configured with [Faraday](https://github.com/lostisland/faraday) middleware. For example:
+
+```ruby
+c = MyClient.new(Faraday.new(:url => 'http://localhost:3000') do |c|
+  c.use Faraday::Request::Retry # configure retries
+  c.use Faraday::Request::BasicAuthentication, 'login', 'pass'
+  c.use Faraday::Response::Logger # log to STDOUT
+  c.use Faraday::Adapter::NetHttp # multiple adapters for different HTTP libraries
+end)
+```
+
+## Server Hooks
 
 In the lifecycle of a request, the Twirp service starts by routing the request to a valid
-RPC method. If routing fails, the `on_error` hook is called with a bad_route error. 
-If routing succeeds, the `before` hook is called before calling the RPC method handler, 
-and then either `on_success` or `on_error` depending if the response is a Twirp error or not. 
+RPC method. If routing fails, the `on_error` hook is called with a bad_route error.
+If routing succeeds, the `before` hook is called before calling the RPC method handler,
+and then either `on_success` or `on_error` depending if the response is a Twirp error or not.
 
 ```
 routing -> before -> handler -> on_success
@@ -114,54 +207,40 @@ routing -> before -> handler
                      ! exception_raised -> on_error
 ```
 
-Example code with hooks:
-
+Hooks are setup in the service instance:
 
 ```ruby
-class HaberdasherHandler
-  def make_hat(size, env)
-    return {}
-  end
-end
-
-handler = HaberdasherHandler.new
-svc = Example::Haberdasher.new(handler)
-
+svc = Example::HelloWorld.new(handler)
 
 svc.before do |rack_env, env|
   # Runs if properly routed to an rpc method, but before calling the method handler.
-  # This is the only place to read the Rack Env to access http request and middleware data.
+  # This is the only place to read the Rack env to access http request and middleware data.
   # The Twirp env has the same routing info as in the handler method, e.g. :rpc_method, :input and :input_class.
-  # If it returns a Twirp::Error, the handler is not called and this error is returned instead.
-  # If an exception is raised, the exception_raised hook will be called and then on_error with the internal error.
+  # Returning a Twirp::Error here cancels the request, and the error is returned instead.
+  # If an exception is raised, the exception_raised hook will be called followed by on_error.
+  env[:user_id] = authenticate(rack_env)
 end
 
 svc.on_success do |env|
   # Runs after the rpc method is handled, if it didn't return Twirp errors or raised exceptions.
-  # The env[:output] contains the serialized message of class env[:ouput_class]
-  # Returned values are ignored (even if it returns Twirp::Error).
-  # Exceptions should not happen here, but if an exception is raised the exception_raised hook will be
-  # called, however on_error will not (either on_success or on_error are called once per request).
+  # The env[:output] contains the serialized message of class env[:ouput_class].
+  # If an exception is raised, the exception_raised hook will be called.
+  success_count += 1
 end
 
 svc.on_error do |twerr, env|
   # Runs on error responses, that is:
-  #  * routing errors (env does not have routing info here)
+  #  * bad_route errors
   #  * before filters returning Twirp errors or raising exceptions.
   #  * hander methods returning Twirp errors or raising exceptions.
   # Raised exceptions are wrapped with Twirp::Error.internal_with(e).
-  # Returned values are ignored (even if it returns Twirp::Error).
-  # Exceptions should not happen here, but if an exception is raised the exception_raised hook will be
-  # called without calling on_error again later.
+  # If an exception is raised here, the exception_raised hook will be called.
+  error_count += 1
 end
 
 svc.exception_raised do |e, env|
   # Runs if an exception was raised from the handler or any of the hooks.
-  environment = (ENV['APP_ENV'] || ENV['RACK_ENV'] || :development).to_sym
-  case environment
-    when :development raise e
-    when :test
-      puts "[Error] #{e}\n#{e.backtrace.join("\n")}"
-  end
+  puts "[Error] #{e}\n#{e.backtrace.join("\n")}"
 end
 ```
+

data/lib/twirp.rb

@@ -1,3 +1,4 @@
 require_relative 'twirp/version'
 require_relative 'twirp/error'
 require_relative 'twirp/service'
+require_relative 'twirp/client'

data/lib/twirp/client.rb

@@ -0,0 +1,147 @@
+require 'faraday'
+require 'json'
+
+require_relative "error"
+require_relative "service_dsl"
+
+module Twirp
+
+  class Client
+
+    # DSL to define a client with package, service and rpcs.
+    extend ServiceDSL
+
+    # DSL (alternative) to define a client from a Service class.
+    def self.client_for(svclass)
+      package svclass.package_name
+      service svclass.service_name
+      svclass.rpcs.each do |rpc_method, rpcdef|
+        rpc rpc_method, rpcdef[:input_class], rpcdef[:output_class], ruby_method: rpcdef[:ruby_method]
+      end
+    end
+
+    # Hook for ServiceDSL#rpc to define a new method client.<ruby_method>(input, opts).
+    def self.rpc_define_method(rpcdef)
+      define_method rpcdef[:ruby_method] do |input|
+        call_rpc(rpcdef[:rpc_method], input)
+      end
+    end
+
+    # Init with a Faraday connection.
+    def initialize(conn)
+      @conn = case conn
+      when String then Faraday.new(url: conn) # init with hostname
+      when Faraday::Connection then conn # inith with connection
+      else raise ArgumentError.new("Expected hostname String or Faraday::Connection")
+      end
+    end
+
+    def service_full_name; self.class.service_full_name; end
+
+    def rpc_path(rpc_method)
+      "/#{service_full_name}/#{rpc_method}"
+    end
+
+    def call_rpc(rpc_method, input)
+      rpcdef = self.class.rpcs[rpc_method.to_s]
+      if !rpcdef
+        return ClientResp.new(nil, Twirp::Error.bad_route("rpc not defined on this client"))
+      end
+
+      input = rpcdef[:input_class].new(input) if input.is_a? Hash
+      body = rpcdef[:input_class].encode(input)
+
+      resp = @conn.post do |r|
+        r.url rpc_path(rpc_method)
+        r.headers['Content-Type'] = 'application/protobuf'
+        r.body = body
+      end
+
+      if resp.status != 200
+        return ClientResp.new(nil, error_from_response(resp))
+      end
+
+      if resp.headers['Content-Type'] != 'application/protobuf'
+        return ClientResp.new(nil, Twirp::Error.internal("Expected response Content-Type \"application/protobuf\" but found #{resp.headers['Content-Type'].inspect}"))
+      end
+
+      data = rpcdef[:output_class].decode(resp.body)
+      return ClientResp.new(data, nil)
+    end
+
+    def error_from_response(resp)
+      status = resp.status
+
+      if is_http_redirect? status
+        return twirp_redirect_error(status, resp.headers['Location'])
+      end
+
+      err_attrs = nil
+      begin
+        err_attrs = JSON.parse(resp.body)
+      rescue JSON::ParserError => e
+        return twirp_error_from_intermediary(status, "Response is not JSON", resp.body)
+      end
+
+      code = err_attrs["code"]
+      if code.to_s.empty?
+        return twirp_error_from_intermediary(status, "Response is JSON but it has no \"code\" attribute", resp.body)
+      end
+      code = code.to_s.to_sym
+      if !Twirp::Error.valid_code?(code)
+        return Twirp::Error.internal("Invalid Twirp error code: #{code}", invalid_code: code.to_s, body: resp.body)
+      end
+
+      Twirp::Error.new(code, err_attrs["msg"], err_attrs["meta"])
+    end
+
+    # Error that was caused by an intermediary proxy like a load balancer.
+    # The HTTP errors code from non-twirp sources is mapped to equivalent twirp errors.
+    # The mapping is similar to gRPC: https://github.com/grpc/grpc/blob/master/doc/http-grpc-status-mapping.md.
+    # Returned twirp Errors have some additional metadata for inspection.
+    def twirp_error_from_intermediary(status, reason, body)
+      code = case status
+        when 400 then :internal
+        when 401 then :unauthenticated
+        when 403 then :permission_denied
+        when 404 then :bad_route
+        when 429, 502, 503, 504 then :unavailable
+        else :unknown
+      end
+
+      twerr = Twirp::Error.new(code, code.to_s, {
+        http_error_from_intermediary: "true",
+        not_a_twirp_error_because: reason,
+        status_code: status.to_s,
+        body: body.to_s,
+      })
+    end
+
+    # Twirp clients should not follow redirects automatically, Twirp only handles
+    # POST requests, redirects should only happen on GET and HEAD requests.
+    def twirp_redirect_error(status, location)
+      msg = "Unexpected HTTP Redirect from location=#{location}"
+      Twirp::Error.new(:internal, msg, {
+        http_error_from_intermediary: "true",
+        not_a_twirp_error_because: "Redirects not allowed on Twirp requests",
+        status_code: status.to_s,
+        location: location.to_s,
+      })
+    end
+
+    def is_http_redirect?(status)
+      status >= 300 && status <= 399
+    end
+
+  end
+
+  class ClientResp
+    attr_accessor :data
+    attr_accessor :error
+
+    def initialize(data, error)
+      @data = data
+      @error = error
+    end
+  end
+end