chimera_http_client 1.2.0 → 1.2.1.beta

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5a1e1ddc7ca2263c940a81d57b6fbd4a94cb013ba040cf47a10e9428b4ad2fee
-  data.tar.gz: 2314f9fac4ec7b9514d19fe50d96051f47e9a9fce4cc588c3c2f5381bf53b8d2
+  metadata.gz: d5bd262f2489439bd64af6657b8a929217fbbf7d89e410192e7470512b329ea2
+  data.tar.gz: bcc303549d4a428bc2589c558881e63330f88e080e475daeeb28d0d1a9a84ddb
 SHA512:
-  metadata.gz: 3074c920d64e9b4ce923590c320e1db14ede59d63527bbf29070b8625edfc3738a9354fa00ca21dcf61635882f97d6a755be10fa0c0776e567c13264a5f7358c
-  data.tar.gz: 2530bee55981a60ce9beaf1959b8af1b7ca333f1d16655de7782bd994bf75d6aff094beced3a8ebe31ea8dc3c2109e1bc9159058bf45932cb006e305925f8567
+  metadata.gz: '09047e533a15ce537f83839a652de962c1a6931aabd895e4f2ae25386e047874732f163f4c151f2483ddaafd381cfbee58ab390d346826c228564d2d81671027'
+  data.tar.gz: ea9a533ac8ea2307cc0d2c3e074d41cc663d4d18ba9fffde5f78bcf36e43e69609a6b69b816194ad61e84e73a1e2288730ed9ad97a3c99f61eb1e770a1252a52

data/README.markdown

@@ -66,18 +66,28 @@ The basic usage looks like this:
 
 ```ruby
 connection = ChimeraHttpClient::Connection.new(base_url: 'http://localhost/namespace')
+
 response = connection.get!(endpoint, params: params)
 ```
 
 ### Initialization
 
-`connection = ChimeraHttpClient::Connection.new(base_url: 'http://localhost:3000/v1', logger: logger, cache: cache)`
+```ruby
+connection = ChimeraHttpClient::Connection.new(
+  base_url: 'http://localhost:3000/v1',
+  cache: cache,
+  deserializer: { deserializer: { error: HtmlParser, response: XMLParser }},
+  logger: logger,
+  monitor: monitor,
+  timeout:  2
+)
+```
 
 #### Mandatory initialization parameter `base_url`
 
 The mandatory parameter is **base_url** which should include the host, port and base path to the API endpoints you want to call, e.g. `'http://localhost:3000/v1'`.
 
-Setting the `base_url` is meant to be a comfort feature, as you can then pass short endpoints to each request like `/users`. You could set an empty string `''` as `base_url` and then pass full qualified URLs as endpoint of the requests.
+Setting the `base_url` is meant to be a comfort feature, as you can then pass short endpoints to each request like `users` (yes, feel free to omit leading or trailing slashes). You could set an empty string `''` as `base_url` and then pass full qualified URLs as endpoint of the requests.
 
 #### Optional initialization parameters
 
@@ -91,34 +101,7 @@ The optional parameters are:
 * `user_agent` - if you would like your calls to identify with a specific user agent
 * `verbose` - the default is `false`, set it to true while debugging issues
 
-##### Custom deserializers
-
-In case the API you are connecting to does not return JSON, you can pass custom deserializers to `Connection.new` or `Queue.new`:
-
-    deserializers: { error: your_error_deserializer, response: your_response_deserializer }
-
-A Deserializer has to be an object on which the method `call` with the parameter `body` can be called:
-
-    custom_deserializer.call(body)
-
-where `body` is the response body (in the default case a JSON object). The class `Deserializer` contains the default objects that are used. They might help you creating your own. Don't forget to make requests with another header than the default `"Content-Type" => "application/json"`, when the API you connect to does not support JSON.
-
-##### Monitoring, metrics, instrumentation
-
-Pass an object as `:monitor` to a connection that defines the method `call` and accepts a hash as parameter.
-
-    monitor.call({...})
-
-It will receive information about every request as soon as it finished. What you do with this information is up for you to implement.
-
-| Field          | Description                                                           |
-|:---------------|:----------------------------------------------------------------------|
-| `url`          | URL of the endpoint that was called                                   |
-| `method`       | HTTP method: get, post, ...                                           |
-| `status`       | HTTP status code: 200, ...                                            |
-| `runtime`      | the time in seconds it took the request to finish                     |
-| `completed_at` | Time.now.utc.iso8601(3)                                               |
-| `context`      | Whatever you pass as `monitoring_context` to the options of a request |
+> Detailed information about every parameter can be found below.
 
 ### Request methods
 
@@ -126,9 +109,12 @@ The available methods are:
 
 * `get` / `get!`
 * `post` / `post!`
-* `put` / `put`
+* `put` / `put!`
 * `patch` / `patch!`
 * `delete` / `delete!`
+* `head` / `head!`
+* `options` / `options!`
+* `trace` / `trace!`
 
 where the methods ending on a _bang!_ will raise an error (which you should handle in your application) while the others will return an error object.
 
@@ -143,7 +129,7 @@ connection.get("users/#{id}")
 connection.get("/users/#{id}")
 ```
 
-All forms above ave valid and will make a request to the same URL.
+All forms above are valid and will make a request to the same URL.
 
 * Please take note that _the endpoint can be given as a String, a Symbol, or an Array._  
 * While they do no harm, there is _no need to pass leading or trailing `/` in endpoints._  
@@ -162,68 +148,20 @@ All request methods expect a mandatory `endpoint` and an optional hash as parame
 * `cache` - optionally overwrite the cache store set in `Connection` in any request
 * `monitoring_context` - pass additional information you want to collect with your instrumentation `monitor`
 
-Example:
+> Detailed information about every parameter can be found below.
+
+### Example usage
 
 ```ruby
-connection.post(
+response = connection.post(
   :users,
   body: { name: "Andy" },
   params: { origin: `Twitter`},
   headers: { "Authorization" => "Bearer #{token}" },
-  timeout: 10,
-  cache: nil
+  timeout: 5
 )
 ```
 
-#### Basic auth
-
-In case you need to use an API that is protected by **basic_auth** just pass the credentials as optional parameters:  
-`username: 'admin', password: 'secret'`
-
-#### Timeout duration
-
-The default timeout duration is **3 seconds**.
-
-If you want to use a different timeout, you can pass the key `timeout` when initializing the `Connection`. You can also overwrite it on every call.
-
-#### Custom logger
-
-By default no logging is happening. If you need request logging, you can pass your custom Logger to the key `logger` when initializing the `Connection`. It will write to `logger.info` when starting and when completing a request.
-
-#### Caching responses
-
-To cache all the reponses of a connection, just pass the optional parameter `cache` to its initializer. You can also overwrite the connection's cache configuration by passing the parameter `cache` to any `get` call.
-
-It could be an instance of an implementation as simple as this:
-
-```ruby
-class Cache
-  def initialize
-    @memory = {}
-  end
-
-  def get(request)
-    @memory[request]
-  end
-
-  def set(request, response)
-    @memory[request] = response
-  end
-end
-```
-
-Or use an adapter for Dalli, Redis, or Rails cache that also support an optional time-to-live `default_ttl` parameter. If you use `Rails.cache` with the adapter `:memory_store` or `:mem_cache_store`, the object you would have to pass looks like this:
-
-```ruby
-require "typhoeus/cache/rails"
-
-cache: Typhoeus::Cache::Rails.new(Rails.cache, default_ttl: 600) # 600 seconds
-```
-
-Read more about how to use it: https://github.com/typhoeus/typhoeus#caching
-
-### Example usage
-
 To use the gem, it is recommended to write wrapper classes for the endpoints used. While it would be possible to use the `get, get!, post, post!, put, put!, patch, patch!, delete, delete!` or also the bare `request.run` methods directly, wrapper classes will unify the usage pattern and be very convenient to use by veterans and newcomers to the team. A wrapper class could look like this:
 
 ```ruby
@@ -296,6 +234,117 @@ To create and fetch a user from a remote service with the `Users` wrapper listed
   user.name # == "Andy"
 ```
 
+### Connection parameter details
+
+#### base_url
+
+#### Cache
+
+To cache the responses of a connection, just pass the optional parameter `cache` to its initializer. You can also overwrite the connection's cache configuration by passing the parameter `cache` to any `get` call.
+
+It could be an instance of an implementation as simple as this:
+
+```ruby
+class Cache
+  def initialize
+    @memory = {}
+  end
+
+  def get(request)
+    @memory[request]
+  end
+
+  def set(request, response)
+    @memory[request] = response
+  end
+end
+```
+
+Or use an adapter for Dalli, Redis, or Rails cache that also support an optional time-to-live `default_ttl` parameter. If you use `Rails.cache` with the adapter `:memory_store` or `:mem_cache_store`, the object you would have to pass looks like this:
+
+```ruby
+require "typhoeus/cache/rails"
+
+cache: Typhoeus::Cache::Rails.new(Rails.cache, default_ttl: 600) # 600 seconds
+```
+
+Read more about how to use it: https://github.com/typhoeus/typhoeus#caching
+
+#### Custom deserializers
+
+In case the API you are connecting to does not return JSON, you can pass custom deserializers to `Connection.new` or `Queue.new`:
+
+    deserializers: { error: your_error_deserializer, response: your_response_deserializer }
+
+A Deserializer has to be an object on which the method `call` with the parameter `body` can be called:
+
+    custom_deserializer.call(body)
+
+where `body` is the response body (in the default case a JSON object). The class `Deserializer` contains the default objects that are used. They might help you creating your own. Don't forget to make requests with another header than the default `"Content-Type" => "application/json"`, when the API you connect to does not support JSON.
+
+If you don't want that Chimera deserialize the response body, pass a Proc that does nothing:
+
+    deserializer: { response: proc { |body| body }, response: proc { |body| body } }
+
+#### Logger
+
+By default no logging is happening. If you need request logging, you can pass your custom Logger as option `:logger` when initializing the `Connection`. Chimera will write to `logger.info` when starting and when completing a request.
+
+#### Monitoring, metrics, instrumentation
+
+Pass an object as `:monitor` to a connection that defines the method `call` and accepts a hash as parameter.
+
+    monitor.call({...})
+
+It will receive information about every request as soon as it finished. What you do with this information is up for you to implement.
+
+| Field          | Description                                                           |
+|:---------------|:----------------------------------------------------------------------|
+| `url`          | URL of the endpoint that was called                                   |
+| `method`       | HTTP method: get, post, ...                                           |
+| `status`       | HTTP status code: 200, ...                                            |
+| `runtime`      | the time in seconds it took the request to finish                     |
+| `completed_at` | Time.now.utc.iso8601(3)                                               |
+| `context`      | Whatever you pass as `monitoring_context` to the options of a request |
+
+#### Timeout
+
+The default timeout duration is **3 seconds**.
+
+If you want to use a different timeout, you can pass the option `timeout` when initializing the `Connection`. Give the timeout in seconds (it can be below 1 second, just pass `0.5`). You can also overwrite the time out on every call.
+
+#### User Agent
+
+#### verbose
+
+### Request parameter details
+
+#### endpoint
+
+#### Body
+
+#### Headers
+
+#### Params
+
+#### Basic auth / `username:password`
+
+In case you need to use an API that is protected by **basic_auth** just pass the credentials as optional parameters:  
+`username: 'admin', password: 'secret'`
+
+#### Timeout duration
+
+The default timeout duration is **3 seconds**. You can pass the option `:timeout` to overwrite the Connection default or its custom setting it on every call.
+
+#### Caching responses
+
+You inject a caching adapter on a per request basis, this is also possible when an adapter has been set for the Connection already. Please keep in mind that not all HTTP calls support caching.
+<!-- # TODO: list examples -->
+
+#### monitoring_context
+
+<!-- # TODO: list examples -->
+
 ## The Request class
 
 Usually it does not have to be used directly. It is the class that executes the `Typhoeus::Requests`, raises `Errors` on failing and returns `Response` objects on successful calls.
@@ -308,10 +357,10 @@ The `ChimeraHttpClient::Response` objects have the following interface:
 
     * body             (content the call returns)
     * code             (http code, should be 200 or 2xx)
-    * time             (for monitoring)
-    * response         (the full response object, including the request)
     * error?           (returns false)
-    * parsed_body      (returns the result of `deserializer[:response].call(body)`)
+    * parsed_body      (returns the result of `deserializer[:response].call(body)` by default it deserializes JSON)
+    * response         (the full response object, including the request)
+    * time             (for monitoring)
 
 If your API does not use JSON, but a different format e.g. XML, you can pass a custom deserializer to the Connection.
 
@@ -319,14 +368,14 @@ If your API does not use JSON, but a different format e.g. XML, you can pass a c
 
 All errors inherit from `ChimeraHttpClient::Error` and therefore offer the same attributes:
 
-    * code             (http error code)
     * body             (alias => message)
-    * time             (for monitoring)
-    * response         (the full response object, including the request)
+    * code             (http error code)
     * error?           (returns true)
     * error_class      (e.g. ChimeraHttpClient::NotFoundError)
-    * to_s             (information for logging / respects ENV['CHIMERA_HTTP_CLIENT_LOG_REQUESTS'])
+    * response         (the full response object, including the request)
+    * time             (runtime of the request for monitoring)
     * to_json          (information to return to the API consumer / respects ENV['CHIMERA_HTTP_CLIENT_LOG_REQUESTS'])
+    * to_s             (information for logging / respects ENV['CHIMERA_HTTP_CLIENT_LOG_REQUESTS'])
 
 The error classes and their corresponding http error codes:
 
@@ -345,7 +394,10 @@ The error classes and their corresponding http error codes:
 
 ## The Queue class
 
-Instead of making single requests immediately, the ChimeraHttpClient allows to queue requests and run them in **parallel**.
+Instead of making single requests immediately, the ChimeraHttpClient allows to queue requests and run them in **parallel**. The two big benefits are:
+
+* only one HTTP connection has to be created (and closed) for all the requests (if the server supports this)
+* no need for more asynchronicity in your code, just use the aggregated result set
 
 The number of parallel requests is limited by your system. There is a hard limit for 200 concurrent requests. You will have to measure yourself where the sweet spot for optimal performance is - and when things start to get flaky. I recommend to queue not much more than 20 requests before running them.
 
@@ -370,6 +422,42 @@ The only difference is that a parameter to set the HTTP method has to prepended.
 * `:put` / `'put'` / `'PUT'`
 * `:patch` / `'patch'` / `'PATCH'`
 * `:delete` / `'delete'` / `'DELETE'`
+* `:head` / `'head'` / `'HEAD'`
+* `:options` / `'options'` / `'OPTIONS'`
+* `:trace` / `'trace'` / `'TRACE'`
+
+#### Request chaining
+
+In version _1.2.1.beta_ `queue.add` accepts the parameter `success_handler` which takes a proc or lambda that accepts the **two parameters** `queue` and `response`. It will be executed once the given request returns a successful response. In it another request can be queued, which can use parts of the response of the previous successful request.
+
+You could for example first queue a user object to receive some related _id_ and then use this _id_ to fetch this object.
+
+```ruby
+success_handler =
+  proc do |queue, response|
+    neighbour_id = response.parsed_body[:neighbour_id]
+
+    queue.add(:get, "users/#{neighbour_id}")
+
+    response
+  end
+```
+
+It is important to let your success_handler _always_ return the `response` of the original request (unless you are not interested in the response).
+
+Calls defined in a success_handler will be run after all previously queued requests.
+
+```ruby
+queue.add(method, endpoint, success_handler: success_handler) # first
+queue.add(method, other_endpoint) # second
+
+responses = queue.execute
+
+responses[0] # first
+responses[1] # second
+responses[2] # third, the call given in the success_handler
+```
+
 
 ### Executing requests in parallel
 

data/TODO.markdown

@@ -15,6 +15,7 @@ _none known_
 * [x] ~~add (example) to README~~
 * [ ] add logger.warn / .error for error cases (?)
 * [ ] streamline log message
+* [ ] refactor to `monitor` concept, pass in proc, not a logger object
 
 ### ~~Custom De-serializer~~
 
@@ -26,8 +27,9 @@ _none known_
 
 * [x] ~~allow to queue multiple requests~~
 * [x] ~~execute (up to 200) requests in parallel~~
+* [x] allow to pass one proc / block to request to use as on_complete handler
+* [ ] ensure to check timeouts and connection errors are not re-run (conditionally?)
 * [ ] allow to pass one proc / block (for all requests) to use as on_complete handler for each request
-* [ ] allow to pass one proc / block per requests) to use as on_complete handler
 * [ ] add example to README
 
 ### Add server for testing

data/chimera_http_client.gemspec

@@ -16,7 +16,7 @@ Gem::Specification.new do |spec|
     It is lightweight, fast and enables you to queue HTTP requests to run them in parallel
     for better performance and simple aggregating of distributed data. Despite it's simple
     interface it allows for advanced features like using custom deserializers, loggers,
-    caching requests individiually, and instrumentation support (soon to be implemented).
+    caching requests individiually, and custom instrumentation support.
   DESCRIPTION
 
   spec.homepage      = "https://github.com/mediafinger/chimera_http_client"

data/lib/chimera_http_client/queue.rb

@@ -20,7 +20,20 @@ module ChimeraHttpClient
 
       hydra.run
 
-      responses = queued_requests.map { |request| request.result }
+      responses = queued_requests.map do |request|
+        if request.result.nil?
+          options = request.send(:instance_variable_get, "@options")
+          response = request.request.run
+
+          if response.success?
+            ::ChimeraHttpClient::Response.new(response, options)
+          else
+            ::ChimeraHttpClient::Error.new(response, options)
+          end
+        else
+          request.result
+        end
+      end
 
       empty
 
@@ -42,6 +55,8 @@ module ChimeraHttpClient
         deserializer: @deserializer,
         logger: @logger,
         monitor: @monitor,
+        success_handler: options[:success_handler],
+        queue: self,
       }
 
       Request.new(instance_options).create(

data/lib/chimera_http_client/request.rb

@@ -58,7 +58,11 @@ module ChimeraHttpClient
     private
 
     def on_complete_handler(response)
-      return Response.new(response, @options) if response.success?
+      if response.success?
+        return Response.new(response, @options) unless @options[:success_handler]
+
+        return @options[:success_handler].call(@options[:queue], Response.new(response, @options))
+      end
 
       exception_for(response)
     end

data/lib/chimera_http_client/version.rb

@@ -1,3 +1,3 @@
 module ChimeraHttpClient
-  VERSION = "1.2.0".freeze
+  VERSION = "1.2.1.beta".freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: chimera_http_client
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.2.1.beta
 platform: ruby
 authors:
 - Andreas Finger
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2019-06-15 00:00:00.000000000 Z
+date: 2019-06-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: typhoeus
@@ -225,7 +225,7 @@ description: |
   It is lightweight, fast and enables you to queue HTTP requests to run them in parallel
   for better performance and simple aggregating of distributed data. Despite it's simple
   interface it allows for advanced features like using custom deserializers, loggers,
-  caching requests individiually, and instrumentation support (soon to be implemented).
+  caching requests individiually, and custom instrumentation support.
 email:
 - webmaster@mediafinger.com
 executables: []
@@ -267,9 +267,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: 2.5.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubygems_version: 3.0.3
 signing_key: