lhc 12.3.0 → 13.4.0.pre.pro1766.1

data/.ruby-version

@@ -1 +1 @@
-ruby-2.6.5
+ruby-2.7.2

data/Gemfile.activesupport5

@@ -1,4 +1,4 @@
 source 'https://rubygems.org/'
 
 gemspec
-gem 'activesupport', '~> 5.0.0'
+gem 'activesupport', '~> 5.2'

data/Gemfile.activesupport6

@@ -1,4 +1,4 @@
 source 'https://rubygems.org/'
 
 gemspec
-gem 'activesupport', '~> 6.0.0'
+gem 'activesupport', '~> 6.0'

data/README.md

@@ -51,6 +51,7 @@ use it like:
   * [Configuration](#configuration)
      * [Configuring endpoints](#configuring-endpoints)
      * [Configuring placeholders](#configuring-placeholders)
+     * [Configuring scrubs](#configuring-scrubs)
   * [Interceptors](#interceptors)
      * [Quick start: Configure/Enable Interceptors](#quick-start-configureenable-interceptors)
      * [Interceptors on local request level](#interceptors-on-local-request-level)
@@ -73,6 +74,10 @@ use it like:
            * [Installation](#installation-1)
            * [Environment](#environment)
            * [What it tracks](#what-it-tracks)
+              * [Before and after request tracking](#before-and-after-request-tracking)
+              * [Response tracking](#response-tracking)
+              * [Timeout tracking](#timeout-tracking)
+              * [Caching tracking](#caching-tracking)
            * [Configure](#configure-1)
         * [Prometheus Interceptor](#prometheus-interceptor)
         * [Retry Interceptor](#retry-interceptor)
@@ -95,6 +100,7 @@ use it like:
 
 
 
+
 ## Basic methods
 
 Available are `get`, `post`, `put` & `delete`.
@@ -486,6 +492,50 @@ You can configure global placeholders, that are used when generating urls from u
   LHC.get(:feedbacks) # http://datastore/v2/feedbacks
 ```
 
+### Configuring scrubs
+
+You can filter out sensitive request data from your log files and rollbar by appending them to `LHS.config.scrubs`. These values will be marked `[FILTERED]` in the log and on rollbar. Also nested parameters are being filtered.
+The scrubbing configuration affects all request done by LHC independent of the endpoint. You can scrub any attribute within `params`, `headers` or `body`. For auth you either can choose `:bearer` or `:auth` (default is both).
+
+LHS scrubs per default:
+- Bearer Token within the request header
+- Basic Auth username and password within the request header
+- password and password_confirmation within the request body
+
+Enhance the default scrubbing by pushing the name of the parameter, which should get scrubbed, as string to the existing configuration.
+You can also add multiple parameters at once by pushing multiple strings.
+
+Example:
+```ruby
+  LHC.configure do |c|
+    c.scrubs[:params] << 'api_key'
+    c.scrubs[:body].push('user_token', 'secret_key')
+  end
+```
+
+For disabling scrubbing, add following configuration:
+```ruby
+  LHC.configure do |c|
+    c.scrubs = {}
+  end
+```
+
+If you want to turn off `:bearer` or `:auth` scrubbing, then just overwrite the auth configuration.
+
+Example:
+```ruby
+  LHC.configure do |c|
+    c.scrubs[:auth] = [:bearer]
+  end
+```
+
+If your app has a different authentication strategy than Bearer Authentication or Basic Authentication then you can filter the data by scrubbing the whole header:
+```ruby
+  LHC.configure do |c|
+    c.scrubs[:headers] << 'Authorization'
+  end
+```
+
 ## Interceptors
 
 To monitor and manipulate the HTTP communication done with LHC, you can define interceptors that follow the (Inteceptor Pattern)[https://en.wikipedia.org/wiki/Interceptor_pattern].
@@ -601,7 +651,6 @@ You can configure your own cache (default Rails.cache) and logger (default Rails
 
 ```ruby
   LHC::Caching.cache = ActiveSupport::Cache::MemoryStore.new
-  LHC::Caching.logger = Logger.new(STDOUT)
 ```
 
 Caching is not enabled by default, although you added it to your basic set of interceptors.
@@ -632,6 +681,18 @@ Responses served from cache are marked as served from cache:
   response.from_cache? # true
 ```
 
+You can also use a central http cache to be used by the `LHC::Caching` interceptor.
+
+If you configure a local and a central cache, LHC will perform multi-level-caching.
+LHC will try to retrieve cached information first from the central, in case of a miss from the local cache, while writing back into both.
+
+```ruby
+  LHC::Caching.central = {
+    read: 'redis://$PASSWORD@central-http-cache-replica.namespace:6379/0',
+    write: 'redis://$PASSWORD@central-http-cache-master.namespace:6379/0'
+  }
+```
+
 ##### Options
 
 ```ruby
@@ -644,7 +705,7 @@ Responses served from cache are marked as served from cache:
 
 `race_condition_ttl` - very useful in situations where a cache entry is used very frequently and is under heavy load.
 If a cache expires and due to heavy load several different processes will try to read data natively and then they all will try to write to cache.
-To avoid that case the first process to find an expired cache entry will bump the cache expiration time by the value set in `cache_race_condition_ttl`.
+To avoid that case the first process to find an expired cache entry will bump the cache expiration time by the value set in `race_condition_ttl`.
 
 `use` - Set an explicit cache to be used for this request. If this option is missing `LHC::Caching.cache` is used.
 
@@ -733,11 +794,15 @@ It tracks request attempts with `before_request` and `after_request` (counts).
 In case your workers/processes are getting killed due limited time constraints,
 you are able to detect deltas with relying on "before_request", and "after_request" counts:
 
+###### Before and after request tracking
+
 ```ruby
   "lhc.<app_name>.<env>.<host>.<http_method>.before_request", 1
   "lhc.<app_name>.<env>.<host>.<http_method>.after_request", 1
 ```
 
+###### Response tracking
+
 In case of a successful response it reports the response code with a count and the response time with a gauge value.
 
 ```ruby
@@ -748,6 +813,17 @@ In case of a successful response it reports the response code with a count and t
   "lhc.<app_name>.<env>.<host>.<http_method>.time", 43
 ```
 
+In case of a unsuccessful response it reports the response code with a count but no time:
+
+```ruby
+  LHC.get('http://local.ch')
+
+  "lhc.<app_name>.<env>.<host>.<http_method>.count", 1
+  "lhc.<app_name>.<env>.<host>.<http_method>.500", 1
+```
+
+###### Timeout tracking
+
 Timeouts are also reported:
 
 ```ruby
@@ -756,6 +832,30 @@ Timeouts are also reported:
 
 All the dots in the host are getting replaced with underscore, because dot is the default separator in graphite.
 
+###### Caching tracking
+
+When you want to track caching stats please make sure you have enabled the `LHC::Caching` and the `LHC::Monitoring` interceptor.
+
+Make sure that the `LHC::Caching` is listed before `LHC::Monitoring` interceptor when configuring interceptors:
+
+```ruby
+LHC.configure do |c|
+  c.interceptors = [LHC::Caching, LHC::Monitoring]
+end
+```
+
+If a response was served from cache it tracks:
+
+```ruby
+  "lhc.<app_name>.<env>.<host>.<http_method>.cache.hit", 1
+```
+
+If a response was not served from cache it tracks:
+
+```ruby
+  "lhc.<app_name>.<env>.<host>.<http_method>.cache.miss", 1
+```
+
 ##### Configure
 
 It is possible to set the key for Monitoring Interceptor on per request basis:

data/Rakefile

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 begin
   require 'bundler/setup'
 rescue LoadError
@@ -17,9 +19,7 @@ end
 begin
   require 'rspec/core/rake_task'
   RSpec::Core::RakeTask.new(:spec)
-  task :default => :spec
-rescue LoadError
-  # no rspec available
+  task default: :spec
 end
 
 Bundler::GemHelper.install_tasks

data/lhc.gemspec

@@ -20,17 +20,20 @@ Gem::Specification.new do |s|
   s.require_paths = ['lib']
 
   s.requirements << 'Ruby >= 2.0.0'
+  s.required_ruby_version = '>= 2.7' # Needed for rubocop
 
-  s.add_dependency 'activesupport', '>= 4.2'
+  s.add_dependency 'activesupport', '>= 5.2'
   s.add_dependency 'addressable'
   s.add_dependency 'typhoeus', '>= 0.11'
 
   s.add_development_dependency 'geminabox'
   s.add_development_dependency 'prometheus-client', '~> 0.7.1'
   s.add_development_dependency 'pry'
-  s.add_development_dependency 'rails', '>= 4.2'
+  s.add_development_dependency 'rails', '>= 5.2'
+  s.add_development_dependency 'redis'
   s.add_development_dependency 'rspec-rails', '>= 3.0.0'
-  s.add_development_dependency 'rubocop', '~> 0.57.1'
+  s.add_development_dependency 'rubocop', '~> 1.0'
+  s.add_development_dependency 'rubocop-performance', '~> 1.0'
   s.add_development_dependency 'rubocop-rspec', '~> 1.26.0'
   s.add_development_dependency 'timecop'
   s.add_development_dependency 'webmock'

data/lib/lhc.rb

@@ -6,131 +6,142 @@ require 'active_support/core_ext/hash/keys'
 
 module LHC
   autoload :BasicMethodsConcern,
-    'lhc/concerns/lhc/basic_methods_concern'
+           'lhc/concerns/lhc/basic_methods_concern'
   autoload :ConfigurationConcern,
-    'lhc/concerns/lhc/configuration_concern'
+           'lhc/concerns/lhc/configuration_concern'
   autoload :FixInvalidEncodingConcern,
-    'lhc/concerns/lhc/fix_invalid_encoding_concern'
+           'lhc/concerns/lhc/fix_invalid_encoding_concern'
   autoload :FormatsConcern,
-    'lhc/concerns/lhc/formats_concern'
+           'lhc/concerns/lhc/formats_concern'
 
   include BasicMethodsConcern
   include ConfigurationConcern
   include FormatsConcern
 
   autoload :Auth,
-    'lhc/interceptors/auth'
+           'lhc/interceptors/auth'
   autoload :Caching,
-    'lhc/interceptors/caching'
+           'lhc/interceptors/caching'
   autoload :DefaultTimeout,
-    'lhc/interceptors/default_timeout'
+           'lhc/interceptors/default_timeout'
   autoload :Logging,
-    'lhc/interceptors/logging'
+           'lhc/interceptors/logging'
   autoload :Prometheus,
-    'lhc/interceptors/prometheus'
+           'lhc/interceptors/prometheus'
   autoload :Retry,
-    'lhc/interceptors/retry'
+           'lhc/interceptors/retry'
   autoload :Throttle,
-    'lhc/interceptors/throttle'
+           'lhc/interceptors/throttle'
 
   autoload :Config,
-    'lhc/config'
+           'lhc/config'
   autoload :Endpoint,
-    'lhc/endpoint'
+           'lhc/endpoint'
 
   autoload :Error,
-    'lhc/error'
+           'lhc/error'
   autoload :ClientError,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :BadRequest,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :Unauthorized,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :PaymentRequired,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :Forbidden,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :Forbidden,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :NotFound,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :MethodNotAllowed,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :NotAcceptable,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :ProxyAuthenticationRequired,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :RequestTimeout,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :Conflict,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :Gone,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :LengthRequired,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :PreconditionFailed,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :RequestEntityTooLarge,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :RequestUriToLong,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :UnsupportedMediaType,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :RequestedRangeNotSatisfiable,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :ExpectationFailed,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :UnprocessableEntity,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :Locked,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :FailedDependency,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :UpgradeRequired,
-    'lhc/errors/client_error'
+           'lhc/errors/client_error'
   autoload :ParserError,
-    'lhc/errors/parser_error'
+           'lhc/errors/parser_error'
   autoload :ServerError,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :InternalServerError,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :NotImplemented,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :BadGateway,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :ServiceUnavailable,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :GatewayTimeout,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :HttpVersionNotSupported,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :InsufficientStorage,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :NotExtended,
-    'lhc/errors/server_error'
+           'lhc/errors/server_error'
   autoload :Timeout,
-    'lhc/errors/timeout'
+           'lhc/errors/timeout'
   autoload :UnknownError,
-    'lhc/errors/unknown_error'
+           'lhc/errors/unknown_error'
+
+  autoload :Scrubber,
+           'lhc/scrubber'
+  autoload :AuthScrubber,
+           'lhc/scrubbers/auth_scrubber'
+  autoload :BodyScrubber,
+           'lhc/scrubbers/body_scrubber'
+  autoload :HeadersScrubber,
+           'lhc/scrubbers/headers_scrubber'
+  autoload :ParamsScrubber,
+           'lhc/scrubbers/params_scrubber'
 
   autoload :Interceptor,
-    'lhc/interceptor'
+           'lhc/interceptor'
   autoload :Interceptors,
-    'lhc/interceptors'
+           'lhc/interceptors'
   autoload :Formats,
-    'lhc/formats'
+           'lhc/formats'
   autoload :Format,
-    'lhc/format'
+           'lhc/format'
   autoload :Monitoring,
-    'lhc/interceptors/monitoring'
+           'lhc/interceptors/monitoring'
   autoload :Request,
-    'lhc/request'
+           'lhc/request'
   autoload :Response,
-    'lhc/response'
+           'lhc/response'
   autoload :Rollbar,
-    'lhc/interceptors/rollbar'
+           'lhc/interceptors/rollbar'
   autoload :Zipkin,
-    'lhc/interceptors/zipkin'
+           'lhc/interceptors/zipkin'
 
   require 'lhc/railtie' if defined?(Rails)
 end

data/lib/lhc/concerns/lhc/fix_invalid_encoding_concern.rb

@@ -12,6 +12,7 @@ module LHC
       # an empty string is returned instead
       def fix_invalid_encoding(string)
         return string unless string.is_a?(String)
+
         result = string.dup
 
         # we assume it's ISO-8859-1 first

data/lib/lhc/config.rb

@@ -5,14 +5,18 @@ require 'singleton'
 class LHC::Config
   include Singleton
 
+  attr_accessor :scrubs
+
   def initialize
     @endpoints = {}
     @placeholders = {}
+    @scrubs = default_scrubs
   end
 
   def endpoint(name, url, options = {})
     name = name.to_sym
     raise 'Endpoint already exists for that name' if @endpoints[name]
+
     @endpoints[name] = LHC::Endpoint.new(url, options)
   end
 
@@ -23,6 +27,7 @@ class LHC::Config
   def placeholder(name, value)
     name = name.to_sym
     raise 'Placeholder already exists for that name' if @placeholders[name]
+
     @placeholders[name] = value
   end
 
@@ -36,12 +41,23 @@ class LHC::Config
 
   def interceptors=(interceptors)
     raise 'Default interceptors already set and can only be set once' if @interceptors
+
     @interceptors = interceptors
   end
 
+  def default_scrubs
+    {
+      auth: [:bearer, :basic],
+      params: [],
+      headers: [],
+      body: ['password', 'password_confirmation']
+    }
+  end
+
   def reset
     @endpoints = {}
     @placeholders = {}
     @interceptors = nil
+    @scrubs = default_scrubs
   end
 end