kiev 2.7.3

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 4993218a36a9d91204d3af2a87327a3394494e3f
+  data.tar.gz: f446c0d1c31a6aacf5d12d47488cafb19756dabd
+SHA512:
+  metadata.gz: 66768446011943507c8741451239becbbaa9397b1d4cf96214f3bc0bc31e524d407821d55566ea998953092ae5bcf869c8e3a74622ad06f9a0cca446f1fcf0d2
+  data.tar.gz: d11e2427f18cb51feb4ccddcb270b840638f37531819685e9913c6a1825a0cc8dc2d5fd6e44cfd2799a27b47caaa94178edabad127b135cbc6d5c40161c288c5

data/.gitignore

@@ -0,0 +1,11 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/gemfiles/*.gemfile.lock
+/pkg/
+/spec/reports/
+/tmp/
+.rubocop-http*

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--require spec_helper

data/.rubocop.yml

@@ -0,0 +1,25 @@
+inherit_from:
+  - https://raw.githubusercontent.com/blacklane/rubocop/master/rubocop.yml
+
+Lint/HandleExceptions:
+  Exclude:
+    - test/rails_integration_test.rb
+    - test/sinatra_integration_test.rb
+Lint/RescueException:
+  Exclude:
+    - lib/kiev/request_body_filter/json.rb
+    - lib/kiev/sidekiq/request_logger.rb
+    - lib/kiev/rack/request_logger.rb
+    - lib/kiev/json.rb
+    - test/sidekiq_test.rb
+Style/GlobalVars:
+  Exclude:
+    - test/helper.rb
+Style/GuardClause:
+  Exclude:
+    - lib/kiev/logger.rb
+Style/NestedParenthesizedCalls:
+  Exclude:
+  - spec/lib/kiev/json_spec.rb
+Style/BlockDelimiters:
+  EnforcedStyle: line_count_based

data/.ruby-version

@@ -0,0 +1 @@
+2.3.3

data/.travis.yml

@@ -0,0 +1,27 @@
+sudo: false
+branches:
+  only:
+    - master
+cache:
+  - bundler
+language: ruby
+rvm:
+  - 2.3.3
+  - 2.2.3
+addons:
+  postgresql: "9.4"
+services:
+  - postgresql
+  - redis-server
+before_script:
+  - psql -c 'create database que_test;' -U postgres
+gemfile:
+  - gemfiles/que_0.12.2.gemfile
+  - gemfiles/que_0.12.3.gemfile
+  - gemfiles/rails_4.1.gemfile
+  - gemfiles/rails_4.2.gemfile
+  - gemfiles/sidekiq_4.2.gemfile
+  - gemfiles/sinatra_1.4.gemfile
+  - gemfiles/sinatra_2.0.gemfile
+matrix:
+  fast_finish: true

data/Gemfile

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+eval_gemfile(File.join(File.dirname(__FILE__), "gemfiles/rails_4.1.gemfile"))
+
+gem "wwtd"

data/LICENSE.md

@@ -0,0 +1,7 @@
+Copyright 2017 Blacklane
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,461 @@
+# Kiev [![Build Status](https://travis-ci.com/blacklane/kiev.svg?token=j5tcq3Fz9ERKZ2HhzC8T&branch=master)](https://travis-ci.com/blacklane/kiev)
+
+Kiev is a comprehensive logging library aimed at covering a wide range of frameworks and tools from the Ruby ecosystem:
+
+- Rails
+- Sinatra
+- Rack and other Rack-based frameworks
+- Sidekiq
+- Que
+- Her and other Faraday-based libraries
+- HTTParty
+
+The main goal of Kiev is consistent logging across distributed systems, like **tracking HTTP requests across various Ruby micro-services**. Kiev will generate and propagate request IDs and make it easy for you to identify service calls and branching requests, **including background jobs triggered by these requests**.
+
+Aside from web requests and background jobs, which are tracked out of the box, Kiev makes it easy to append additional information or introduce **custom events**.
+
+Kiev produces structured logs in the **JSON format**, which are ready to be ingested by ElasticSearch or other similar JSON-driven data stores. It eliminates the need for Logstash in a typical ELK stack.
+
+In **development mode**, Kiev can print human-readable logs - pretty much like the default Rails logger, but including all the additional information that you've provided via Kiev events.
+
+## Install
+
+Add the gem to your `Gemfile`:
+
+```ruby
+gem "kiev"
+```
+
+Don't forget to `bundle install`.
+
+## Configure
+
+### Rails
+
+Place your configuration under `config/initializers/kiev.rb`:
+
+```ruby
+require "kiev"
+
+Kiev.configure do |config|
+  config.app = :my_app
+  config.development_mode = Rails.env.development?
+  config.log_path = Rails.root.join("log", "structured.log") unless Rails.env.development? || $stdout.isatty
+end
+```
+
+The middleware stack is included automatically via a *Railtie*.
+
+### Sinatra
+
+Somewhere in your code, ideally before the server configuration, add the following lines:
+
+```ruby
+require "kiev"
+
+Kiev.configure do |config|
+  config.app = :my_app
+  config.log_path = File.join("log", "structured.log")
+end
+```
+
+Within your `Sinatra::Base` implementation, include the `Kiev::Rack` module, in order to register the middleware stack:
+
+```ruby
+require "kiev"
+require "sinatra/base"
+
+class MyController < Sinatra::Base
+  include Kiev::Rack
+
+  use SomeOtherMiddleware
+
+  get "/hello" do
+    "world"
+  end
+end
+```
+
+### Rack
+
+Somewhere in your code, ideally before the server configuration, add the following lines:
+
+```ruby
+require "kiev"
+
+Kiev.configure do |config|
+  config.app = :my_app
+  config.log_path = File.join("log", "structured.log")
+end
+```
+
+Within your `Rack::Builder` implementation, include the `Kiev::Rack` module, in order to register the middleware stack:
+
+```ruby
+require "kiev"
+require "rack"
+
+app = Rack::Builder.new do
+  include Kiev::Rack
+
+  use SomeOtherMiddleware
+
+  run labmda { |env| [ 200, {}, [ "hello world" ] ] }
+end
+
+run(app)
+```
+
+### Sidekiq
+
+Add the following lines to your initializer code:
+
+```ruby
+Kiev::Sidekiq.enable
+```
+
+### Que
+
+Add the following lines to your initializer code:
+
+```ruby
+require "kiev/que/job"
+
+class MyJob < Kiev::Que::Job
+  ...
+end
+```
+
+### Her
+
+Add the following lines to your initializer code:
+
+```ruby
+Her::API.setup(url: "https://api.example.com") do |c|
+  c.use Kiev::HerExt::ClientRequestId
+  # other middleware
+end
+```
+
+## Loading only the required parts
+
+You can load only parts of the gem, if you don't want to use all features:
+
+```ruby
+require "kiev/her_ext/client_request_id"
+```
+
+## Logging
+
+### Requests
+
+For web requests the Kiev middleware will log the following information by default:
+
+```json
+{
+  "application":"my_app",
+  "event":"request_finished",
+  "level":"INFO",
+  "timestamp":"2017-01-27T16:11:44.123Z",
+  "host":"localhost",
+  "verb":"GET",
+  "path":"/",
+  "params":"{\"hello\":\"world\",\"password\":\"[FILTERED]\"}",
+  "ip":"127.0.0.1",
+  "request_id":"UUID",
+  "request_depth":0,
+  "route":"RootController#index",
+  "user_agent":"curl/7.50.1",
+  "status":200,
+  "request_duration":62.3773,
+  "body":"See #log_response_body_condition",
+  "error_message": "...",
+  "error_class": "...",
+  "error_backtrace": "...",
+  "tree_path": "ACE",
+  "tree_leaf": true
+}
+```
+
+The `params` attribute will store both query parameters and request body fields (as long as they are parseable). Sensitive fields will be filtered out - see the `#filtered_params` option.
+
+The `request_id` is the correlation ID and will be the same across all requests within a chain of requests. It's represented as a UUID (version 4).
+
+The `request_depth` represents the position of the current request within a chain of requests. It starts with 0.
+
+The `route` attribute will be set to either the Rails route (`RootController#index`) or Sinatra route (`/`) or the path, depending on the context.
+
+The `request_duration` is measured in miliseconds.
+
+The `body` attribute coresponds to the response body and will be logged depending on the `#log_response_body_condition` option.
+
+The `tree_path` attribute can be used to follow the branching of requests within a chain of requests. It's a lexicographically sortable string.
+
+The `tree_leaf` points out that this request is a leaf in the request chain tree structure.
+
+### Background jobs
+
+For background jobs, Kiev will log the following information by default:
+
+```json
+{
+  "application":"my_app",
+  "event":"job_finished",
+  "level":"INFO",
+  "timestamp":"2017-01-27T16:11:44.123Z",
+  "job_name":"name",
+  "params": "...",
+  "jid":123,
+  "request_id":"UUID",
+  "request_depth":0,
+  "request_duration":0.000623773,
+  "error_message": "...",
+  "error_class": "...",
+  "error_backtrace": "...",
+  "tree_path": "BDF",
+  "tree_leaf": true
+}
+```
+
+### Appending data to the request log entry
+
+You can also append **arbitrary data** to the request log by calling:
+
+```ruby
+# Append structured data (will be merged)
+Kiev.payload(first_name: "john", last_name: "smith")
+
+# Same thing
+Kiev[:first_name] = "john"
+Kiev[:last_name] = "smith"
+```
+
+### Other events
+
+Kiev allows you to log custom events as well.
+
+The recommended way to do this is by using the `#event` method:
+
+```ruby
+# Log event without any data
+Kiev.event(:my_event)
+
+# Log structured data (will be merged)
+Kiev.event(:my_event, { some_array: [1, 2, 3] })
+
+# Log other data types (will be available under the `message` key)
+Kiev.event(:my_event, "hello world")
+```
+
+However, `Kiev.logger` implements the Ruby `Logger` class, so all the other methods are available as well:
+
+```ruby
+Kiev.logger.info("hello world")
+Kiev.logger.debug({ first_name: "john", last_name: "smith" })
+```
+
+Note that, even when logging custom events, Kiev **will try to append request information** to the entries: the HTTP `verb` and `path` for web request or `job_name` and `jid` for background jobs. The payload, however, will be logged only for the `request_finished` or `job_finished` events. If you want to add a payload to a custom event, use the second argument of the `event` method.
+
+## Advanced configuration
+
+### development_mode
+
+Kiev offers human-readable logging for development purposes. You can enable it via the `development_mode` option:
+
+```ruby
+Kiev.configure do |config|
+  config.development_mode = Rails.env.development?
+end
+```
+
+### filtered_params
+
+By default, Kiev filters out the values for the following parameters:
+
+- client_secret
+- token
+- password,
+- password_confirmation
+- old_password
+- credit_card_number
+- credit_card_cvv
+
+You can override this behaviour via the `filtered_params` option:
+
+```ruby
+Kiev.configure do |config|
+  config.filtered_params = %w(email first_name last_name)
+end
+```
+
+### ignored_params
+
+By default, Kiev ignores the following parameters:
+
+- controller
+- action
+- format
+- authenticity_token
+- utf8
+
+You can override this behaviour via the `ignored_params` option:
+
+```ruby
+Kiev.configure do |config|
+  config.ignored_params = %w(some_field some_other_field)
+end
+```
+
+### log_request_condition
+
+By default, Kiev doesn't log requests to `/ping` or `/health` or requests to assets.
+
+You can override this behaviour via the `log_request_condition` option, which should be a `proc` returning a `boolean`:
+
+```ruby
+Kiev.configure do |config|
+  config.log_request_condition = proc do |request, response|
+    !%r{(^(/ping|/health))|(\.(js|css|png|jpg|gif)$)}.match(request.path)
+  end
+end
+```
+
+### log_request_error_condition
+
+Kiev logs Ruby exceptions. By default, it won't log the exceptions produced by 404s.
+
+You can override this behaviour via the `log_request_error_condition` option, which should be a `proc` returning a `boolean`:
+
+```ruby
+Kiev.configure do |config|
+  config.log_request_error_condition = proc do |request, response|
+    response.status != 404
+  end
+end
+```
+
+### log_response_body_condition
+
+Kiev can log the response body. By default, it will only log the response body when the status code is in the 4xx range and the content type is JSON or XML.
+
+You can override this behaviour via the `log_response_body_condition` option, which should be a `proc` returning a `boolean`:
+
+```ruby
+Kiev.configure do |config|
+  config.log_response_body_condition = proc do |request, response|
+    response.status >= 400 && response.status < 500 && response.content_type =~ /(json|xml)/
+  end
+end
+```
+
+### persistent_log_fields
+
+If you need to log some data for every event in the session (e.g. the user ID), you can do this via the `persistent_log_fields` option.
+
+```ruby
+Kiev.configure do |config|
+  config.persistent_log_fields = [:user_id]
+end
+
+# Somewhere in application
+before do
+  Kiev[:user_id] = current_user.id
+end
+
+get "/" do
+  "hello world"
+end
+```
+
+## nginx
+
+If you want to log 499 and 50x errors in nginx, which will not be captured by Ruby application, consider adding this to your nginx configuration:
+
+```
+log_format kiev '{"application":"app_name", "event":"request_finished",'
+  '"timestamp":"$time_iso8601", "request_id":"$http_x_request_id",'
+  '"user_agent":"$http_user_agent", "status":$status,'
+  '"request_duration_seconds":$request_time, "host":"$host",'
+  '"verb":"$request_method", "path":"$request_uri", "tree_path": "$http_x_tree_path"}';
+
+log_format simple_log '$remote_addr - $remote_user [$time_local] '
+                       '"$request" $status $bytes_sent '
+                       '"$http_referer" "$http_user_agent"';
+
+map $status $not_loggable {
+  ~(499) 0;
+  default 1;
+}
+
+map $status $loggable {
+  ~(499) 1;
+  default 0;
+}
+
+server {
+  access_log /var/log/nginx/access.kiev.log kiev if=$loggable;
+  access_log /var/log/nginx/access.log simple_log if=$not_loggable;
+
+  location = /50x.html {
+    access_log /var/log/nginx/access.kiev.log kiev;
+  }
+}
+```
+
+If you'd like to measure nginx queue latency, add the following to your nginx configuration:
+
+```
+server {
+  ...
+  proxy_set_header X-Request-Start "${msec}";
+  ...
+}
+```
+
+Other libs/technologies using `X-Request-Start` are [rack-timeout](https://github.com/heroku/rack-timeout) and [NewRelic](https://docs.newrelic.com/docs/apm/applications-menu/features/request-queue-server-configuration-examples). There's no [support for ELB](https://forums.aws.amazon.com/message.jspa?messageID=396283) :(
+
+## Logstash, Logrotate, Filebeat
+
+Kiev does not provide facilities to log directly to ElasticSearch. This is done for simplicity. Instead we recommend using [Filebeat](https://www.elastic.co/products/beats/filebeat) to deliver logs to ElasticSearch.
+
+When storing logs on disk, we recommend using Logrotate in truncate mode.
+
+You can use [jq](https://stedolan.github.io/jq/) to traverse JSON log files, when you're not running Kiev in *development mode*.
+
+## Alternatives
+
+### Logging
+
+- [semantic_logger](http://rocketjob.github.io/semantic_logger/)
+- [lograge](https://github.com/roidrage/lograge)
+- [logging](https://github.com/TwP/logging)
+
+### Request-Id
+
+- [Pliny::Middleware::RequestID](https://github.com/interagent/pliny/blob/master/lib/pliny/middleware/request_id.rb)
+- [ActionDispatch::RequestId](http://api.rubyonrails.org/classes/ActionDispatch/RequestId.html)
+- [request_id](https://github.com/remind101/request_id)
+
+## Development
+
+Pull the code:
+
+```
+git clone git@github.com:blacklane/kiev.git
+```
+
+Run tests:
+
+```sh
+bundle exec rake
+```
+
+Run tests for different rubies, frameworks and framework versions:
+
+```sh
+# Create a Postgres test database for Que
+createdb que_test
+
+# Run the tests (replace myuser with your username)
+DATABASE_URL=postgres://myuser:@localhost/que_test bundle exec wwtd
+```