gruf 2.4.1 → 2.4.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b1289ff979cec29b1c475fc7fb9292d3eab752229e5147b59f18c107fb34531d
-  data.tar.gz: 55acd4dee980cd8274edbbc7308edc63e09b6a4b19e4a87a6d1d1e72b11dd46f
+  metadata.gz: bef9466a137b8d52bc76a4d24c65fef26b0701cff21eefb2ae9811e020dd619c
+  data.tar.gz: f3209bfefd0e3a7fe7ee03b6bcc7a417974593b924966521346d095d87de6d5d
 SHA512:
-  metadata.gz: 99959b15a10222393883ac16b80c8550ba35190aa9a9858764687e4349d04db9f696d37fe9ed7fd7bf4db0df8e1841791760eb49de7633392720fe9f8175d733
-  data.tar.gz: 22f6f6f2b9db9bea6768b1f3089c6502d02323e012efdad113becd430a94af4136d8d6f216d4e73079e66d59490ee379c6ba962d1c3f9f9649243fa0b382b57d
+  metadata.gz: 906fa455028407ea0ac54dceeb89c54b032ddf9e1090e85d14dabf8496aad91b22d19056fdf8e84d9836399518198ed7afffa50c1cc69ac9d8a4e5a43e53f40b
+  data.tar.gz: c3ec65443321ff6ac00b839bf3b6a9fed748e5f7ab2ec73ffd4476d1b4b7590ccd2d865ac4f5c6703a59c5ae89746b2964f58ee43e56c62a3df62e8bddae9271

data/CHANGELOG.md

@@ -2,6 +2,20 @@ Changelog for the gruf gem. This includes internal history before the gem was ma
 
 ### Pending release
 
+### 2.4.2
+
+- Added error handling for GRPC::Core::CallError, a low-level error in the grpc library that does not inherit
+  from StandardError. [#59]
+- Removed `Thread.abort\_on\_exception = true`. Exceptions should be handled by gruf or the application,
+  and should not cause the server process to crash. [#59]
+- Added guard for size of trailing metadata attached to grpc call. The default max for http2 trailing metadata
+  in the gRPC C library is 8kb. If we go over that limit (either through custom metadata attached to the
+  error by the application, or via the error payload encoded by the error serializer), the gRPC library
+  will throw RESOURCE\_EXHAUSTED. Gruf now detects this case, and attempts to prevent it by logging the
+  original error and substituting it with an internal error indicating that the metadata was too large. [#60]
+- Truncate stack trace in error payload to help avoid overflowing the trailing metadata. Added backtrace\_limit
+  configuration parameter, which defaults to 10.[#60]
+
 ### 2.4.1
 
 - Safer configuration of GRPC::RpcServer. From now on, use `Gruf.rpc_server_options` for the params
@@ -12,8 +26,8 @@ Changelog for the gruf gem. This includes internal history before the gem was ma
 
 ### 2.4.0
 
-- Added a hash of error log levels to RequestLogging interceptor, mapping error code to level of logging to use. To 
-override the level of logging per error response, provide a map of codes to log level in options, key :log_levels. 
+- Added a hash of error log levels to RequestLogging interceptor, mapping error code to level of logging to use. To
+override the level of logging per error response, provide a map of codes to log level in options, key :log_levels.
 The default is :error log level.
 
 ### 2.3.0
@@ -28,7 +42,7 @@ The default is :error log level.
 
 ### 2.2.1
 
-- Now changes proc title once server is ready to process incoming requests [#44] 
+- Now changes proc title once server is ready to process incoming requests [#44]
 - Gruf now requires gRPC 1.10.x+ due to various fixes and improvements in the gRPC core libraries
 
 ### 2.2.0
@@ -37,13 +51,13 @@ The default is :error log level.
 
 ### 2.1.1
 
-- Add ability to pass in client stub options into Gruf::Client 
+- Add ability to pass in client stub options into Gruf::Client
 
 ### 2.1.0
 
 - Add ability to list, clear, insert before, insert after, and remove to a server's interceptor
 registry
-- Ensure interceptors and services cannot be adjusted on the server after it starts to 
+- Ensure interceptors and services cannot be adjusted on the server after it starts to
 prevent threading issues
 - [#36], [#37] Adds `response_class`, `request_class`, and `service` accessors to controller request
 
@@ -106,22 +120,22 @@ Gruf 2.0 is a major shift from Gruf 1.0. See [UPGRADING.md](UPGRADING.md) for de
 - Instrumentation hooks now execute similarly to outer_around hooks; they can
   now instrument failures
 - Instrumentation hooks now pass a `RequestContext` object that contains information
-  about the incoming request, instead of relying on instance variables 
+  about the incoming request, instead of relying on instance variables
 - StatsD hook now sends success/failure metrics for endpoints
 - Add ability to turn off sending exception message on uncaught exception.
 - Add configuration to set the error message when an uncaught exception is
   handled by gruf.
-- Add a request logging hook for Rails-style request logging, with optional 
-  parameter logging, blacklists, and formatter support 
+- Add a request logging hook for Rails-style request logging, with optional
+  parameter logging, blacklists, and formatter support
 - Optimizations around Symbol casting within service calls
 
 ### 1.1.0
 
-- Add the ability for call options to the client, which enables deadline setting 
+- Add the ability for call options to the client, which enables deadline setting
 
 ### 1.0.0
 
-- Bump gRPC to 1.4 
+- Bump gRPC to 1.4
 
 ### 0.14.2
 
@@ -150,7 +164,7 @@ Gruf 2.0 is a major shift from Gruf 1.0. See [UPGRADING.md](UPGRADING.md) for de
 ### 0.12.0
 
 - Add ability to run multiple around hooks
-- Fix bug with error handling that caused error messages to repeat across streams 
+- Fix bug with error handling that caused error messages to repeat across streams
 
 ### 0.11.5
 

data/README.md

@@ -87,9 +87,9 @@ end
 
 ::Gruf::Client.new(
   service: ::Demo::ThingService,
-  client_options: [
+  client_options: {
     interceptors: [MyInterceptor.new]
-  ])
+  })
 ```
 
 The `interceptors` option in `client_options` can accept either a `GRPC::ClientInterceptor` class or a
@@ -239,9 +239,11 @@ For the client, you'll need to point to the public certificate:
 ```ruby
 ::Gruf::Client.new(
   service: Demo::ThingService,
-  ssl_certificate: 'x509 public certificate here',
-  # OR
-  ssl_certificate_file: '/path/to/my.crt'
+  options: {
+    ssl_certificate: 'x509 public certificate here',
+    # OR
+    ssl_certificate_file: '/path/to/my.crt'
+  }
 )
 ```
 
@@ -485,6 +487,8 @@ gruf that you can use today:
 
 * [gruf-zipkin](https://github.com/bigcommerce/gruf-zipkin) - Provides a [Zipkin](https://zipkin.io)
 integration
+* [gruf-lightstep](https://github.com/bigcommerce/gruf-lightstep) - Provides a seamless 
+[LightStep](https://lightstep.com) integration
 * [gruf-circuit-breaker](https://github.com/bigcommerce/gruf-circuit-breaker) - Circuit breaker
 support for services
 * [gruf-profiler](https://github.com/bigcommerce/gruf-profiler) - Profiles and provides memory usage

data/lib/gruf/configuration.rb

@@ -36,6 +36,7 @@ module Gruf
       append_server_errors_to_trailing_metadata: true,
       use_default_interceptors: true,
       backtrace_on_error: false,
+      backtrace_limit: 10,
       use_exception_message: true,
       internal_error_message: 'Internal Server Error',
       event_listener_proc: nil,

data/lib/gruf/controllers/base.rb

@@ -71,8 +71,8 @@ module Gruf
           send(method_key, &block)
         end
       rescue GRPC::BadStatus
-        raise # passthrough
-      rescue StandardError => e
+        raise # passthrough, to be caught by Gruf::Interceptors::Timer
+      rescue GRPC::Core::CallError, StandardError => e # CallError is not a StandardError
         set_debug_info(e.message, e.backtrace) if Gruf.backtrace_on_error
         error_message = Gruf.use_exception_message ? e.message : Gruf.internal_error_message
         fail!(:internal, :unknown, error_message)

data/lib/gruf/error.rb

@@ -50,6 +50,13 @@ module Gruf
       data_loss: GRPC::DataLoss
     }.freeze
 
+    # Default limit on trailing metadata is 8KB. We need to be careful
+    # not to overflow this limit, or the response message will never
+    # be sent. Instead, resource_exhausted will be thrown.
+    MAX_METADATA_SIZE = 7.5 * 1_024
+    METADATA_SIZE_EXCEEDED_CODE = 'metadata_size_exceeded'.freeze
+    METADATA_SIZE_EXCEEDED_MSG = 'Metadata too long, risks exceeding http2 trailing metadata limit.'.freeze
+
     # @return [Symbol] The given internal gRPC code for the error
     attr_accessor :code
     # @return [Symbol] An arbitrary application code that can be used for logical processing of the error by the client
@@ -58,8 +65,8 @@ module Gruf
     attr_accessor :message
     # @return [Array] An array of field errors that can be returned by the server
     attr_accessor :field_errors
-    # @return [Object] A hash of debugging information, such as a stack trace and exception name, that can be used to
-    # debug an given error response. This is sent by the server over the trailing metadata.
+    # @return [Errors::DebugInfo] A object containing debugging information, such as a stack trace and exception name,
+    # that can be used to debug an given error response. This is sent by the server over the trailing metadata.
     attr_accessor :debug_info
     # @return [GRPC::BadStatus] The gRPC BadStatus error object that was generated
     attr_writer :grpc_error
@@ -72,10 +79,11 @@ module Gruf
     # @param [Hash] args (Optional) An optional hash of arguments that will set fields on the error object
     #
     def initialize(args = {})
+      @field_errors = []
+      @metadata = {}
       args.each do |k, v|
         send("#{k}=", v) if respond_to?(k)
       end
-      @field_errors = []
     end
 
     ##
@@ -130,16 +138,30 @@ module Gruf
     end
 
     ##
-    # Append any appropriate errors to the gRPC call and properly update the output metadata
+    # Update the trailing metadata on the given gRPC call, including the error payload if configured
+    # to do so.
     #
     # @param [GRPC::ActiveCall] active_call The marshalled gRPC call
-    # @return [Error] Return the error itself with the GRPC::ActiveCall attached and error metadata appended
+    # @return [Error] Return the error itself after updating metadata on the given gRPC call.
+    #                 In the case of a metadata overflow error, we replace the current error with
+    #                 a new one that won't cause a low-level http2 error.
     #
     def attach_to_call(active_call)
       metadata[Gruf.error_metadata_key.to_sym] = serialize if Gruf.append_server_errors_to_trailing_metadata
-      if !metadata.empty? && active_call && active_call.respond_to?(:output_metadata)
-        active_call.output_metadata.update(metadata)
+      return self if metadata.empty? || !active_call || !active_call.respond_to?(:output_metadata)
+
+      # Check if we've overflown the maximum size of output metadata. If so,
+      # log a warning and replace the metadata with something smaller to avoid
+      # resource exhausted errors.
+      if metadata.inspect.size > MAX_METADATA_SIZE
+        code = METADATA_SIZE_EXCEEDED_CODE
+        msg = METADATA_SIZE_EXCEEDED_MSG
+        logger.warn "#{code}: #{msg} Original error: #{to_h.inspect}"
+        err = Gruf::Error.new(code: :internal, app_code: code, message: msg)
+        return err.attach_to_call(active_call)
       end
+
+      active_call.output_metadata.update(metadata)
       self
     end
 

data/lib/gruf/errors/debug_info.rb

@@ -30,7 +30,12 @@ module Gruf
       #
       def initialize(detail, stack_trace = [])
         @detail = detail
-        @stack_trace = stack_trace.is_a?(String) ? stack_trace.split("\n") : stack_trace
+        @stack_trace = (stack_trace.is_a?(String) ? stack_trace.split("\n") : stack_trace)
+
+        # Limit the size of the stack trace to reduce risk of overflowing metadata
+        stack_trace_limit = Gruf.backtrace_limit.to_i
+        stack_trace_limit = 10 if stack_trace_limit < 0
+        @stack_trace = @stack_trace[0..stack_trace_limit] if stack_trace_limit > 0
       end
 
       ##

data/lib/gruf/server.rb

@@ -203,8 +203,6 @@ module Gruf
     #
     # :nocov:
     def setup_signal_handlers
-      Thread.abort_on_exception = true
-
       Signal.trap('INT') do
         @stop_server = true
         @stop_server_cv.broadcast

data/lib/gruf/version.rb

@@ -14,5 +14,5 @@
 # OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 #
 module Gruf
-  VERSION = '2.4.1'.freeze
+  VERSION = '2.4.2'.freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: gruf
 version: !ruby/object:Gem::Version
-  version: 2.4.1
+  version: 2.4.2
 platform: ruby
 authors:
 - Shaun McCormick
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-08-07 00:00:00.000000000 Z
+date: 2018-09-07 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler