lograge_rails_request_queuing 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: adaca376be4396d88e5bdf9f25826e587eb78c88b0d6bdd1a8ba3972841d4aa8
+  data.tar.gz: 02e2b2be34c7568f1e7848212fc96f7c12edc3d061ff6e23b312dbeca033a901
+SHA512:
+  metadata.gz: a80c1b66d9328d9b25865332ee7d2f57b10c69f7c828fcc4bb16987ffaf4cc6a9fd2f5457e2c08ff4bef86c8bb905bb237e75857facc31047e92689961b65d3b
+  data.tar.gz: 979df62a426b344dfbd3322d009141b333e805581bc40bbf920db5021cd349dbd172126dfb3ca0161c3d1a545511a7fcd97c438d1a0571c6e68f94f198c724fb

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at laust@object.io. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2018 Laust Rud Jacobsen
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,115 @@
+# LogrageRailsRequestQueuing
+
+[![Build Status](https://travis-ci.org/rud/lograge_rails_request_queuing.svg?branch=master)](https://travis-ci.org/rud/lograge_rails_request_queuing)
+
+[Lograge](https://github.com/roidrage/lograge) makes Rails logging output a lot more more useful.
+Using the logstash formatter, the log output for a request will look something like this:
+
+```
+status=200 duration=58.33 view=40.43 db=15.26 controller=WelcomeController action=show
+```
+
+This gem adds another field with how long the request spent in the request queue in nginx in milliseconds, the `rq` value:
+
+```
+status=200 duration=58.33 view=40.43 db=15.26 rq=3.14 [...]
+```
+
+Request queueing time is the time that passes between a request is received in Nginx, and until it hits the Rails stack in a web worker.
+Under normal load in production this value will be in the order of a handful milliseconds.
+However, if all Rails web-processes are busy, the number will quickly climb as individual requests are queued and waiting to be served.
+It's one of those numbers that are good to keep an eye on in monitoring and is very helpful to include when graphing response times over time.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'lograge_rails_request_queuing'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install lograge_rails_request_queuing
+
+Then add it to your lograge initializer, typically in `config/initializers/lograge.rb`:
+
+``` ruby
+Rails.application.configure do
+  config.lograge.enabled = true
+
+  # Keep emitting the verbose logging for easier debug
+  config.lograge.keep_original_rails_log = !Rails.env.production?
+
+  config.lograge.custom_options = lambda do |event|
+    custom_options = {}
+
+    queued_ms = RequestStore[:lograge_request_queueing].queued_ms
+    custom_options[:rq] = queued_ms.round(2) if queued_ms
+
+    custom_options
+  end
+end
+```
+
+In your nginx config, add:
+```
+proxy_set_header X-Request-Start "t=${msec}";`
+```
+
+This adds a new header to the incoming request, with current time in milliseconds as the value. 
+
+After this is deployed, you now get the `"rq=.."` value added to the output when the value is available.
+If you do not see the `"rq=.."` value in logging out, double check you have added the new header in the nginx config.
+  
+
+### Bonus feature: compact exception logging
+
+Add the following to your `config/initializers/lograge.rb` file:
+
+``` ruby
+  config.lograge.custom_options = lambda do |event|
+    LogrageRailsRequestQueuing::ExceptionDetails.add_any_exception!(
+      event, custom_options
+    )
+
+    # ... see above ...
+  end
+
+  # Adding this removes the verbose exception output in the Rails log:
+  ActionDispatch::DebugExceptions.prepend(
+    LogrageRailsRequestQueuing::SilenceExceptionLogging
+  )
+```
+
+Exceptions logged look like this, with a few newlines added for readability here:
+
+```
+I, [2018-06-29T18:44:26.267292 #4]  INFO -- : [29f3a9f6-1848-4858-93be-6ad76a6b9389] 
+method=GET path=/ format=html controller=EchoController action=index status=500 
+error='ArgumentError: Insufficient mittens' duration=0.27 view=0.00 rq=4.12ms 
+exception=["ArgumentError", "Insufficient mittens"] 
+backtrace=["/app/app/controllers/echo_controller.rb:4:in `index'", "/app/vendor/bundle/ruby/2.5.0/gems/actionpack-5.2.0/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'", ...SNIP...]
+```
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/rud/lograge_rails_request_queuing. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the LogrageRailsRequestQueuing project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/rud/lograge_rails_request_queuing/blob/master/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+require 'rdoc/task'
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'LogrageRailsRequestQueuing'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.md')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+require 'bundler/gem_tasks'
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.test_files = FileList['test/**/*_test.rb']
+  t.verbose = true
+end
+
+desc 'Run all tests'
+task ci: :test
+
+task default: :test

data/lib/lograge_rails_request_queuing.rb

@@ -0,0 +1,4 @@
+# frozen_string_literal: true
+
+require 'lograge'
+require 'lograge_rails_request_queuing/engine'

data/lib/lograge_rails_request_queuing/engine.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require 'rails/engine'
+
+module LogrageRailsRequestQueuing
+  # Hook into Rails loading
+  class Engine < ::Rails::Engine
+    isolate_namespace LogrageRailsRequestQueuing
+
+    config.eager_load_paths += %W[#{config.root}/lib]
+
+    initializer 'lograge_rails_request_queuing.add_timing_middleware' do |app|
+      app.middleware.use LogrageRailsRequestQueuing::RequestStartedMiddleware
+    end
+  end
+end

data/lib/lograge_rails_request_queuing/exception_details.rb

@@ -0,0 +1,13 @@
+module LogrageRailsRequestQueuing
+  class ExceptionDetails
+    # If the current request has an exception attached, add the exception details
+    # and backtrace to the logging custom options
+    def self.add_any_exception!(event, log_custom_options)
+      if event.payload[:exception_object]
+        log_custom_options[:exception] = event.payload[:exception]
+        log_custom_options[:backtrace] = Array(event.payload[:exception_object].backtrace)
+      end
+      nil
+    end
+  end
+end

data/lib/lograge_rails_request_queuing/request_queuing.rb

@@ -0,0 +1,56 @@
+# frozen_string_literal: true
+
+module LogrageRailsRequestQueuing
+  class RequestQueueing
+    # How long was the request queued for, in milliseconds
+    attr_reader :queued_ms
+    attr_reader :request_started_at, :request_queued_raw
+
+    REQUEST_START_HEADER = 'HTTP_X_REQUEST_START'
+    EARLIEST_REQUEST_DATE = Time.new(2000)
+
+    def initialize(env, request_started_at = Time.zone.now.to_f)
+      @request_queued_raw = request_start_header(env)
+      @request_started_at = request_started_at
+      @queued_ms = calculate_queued_ms
+    end
+
+    def request_queued_at
+      return if request_queued_float.blank?
+      @request_queued_at ||= begin
+        [1000, 1].each do |divisor|
+          adjusted = Time.zone.at(request_queued_float / divisor)
+          return adjusted if adjusted > EARLIEST_REQUEST_DATE
+        end
+      end
+    end
+
+    private
+
+    def calculate_queued_ms
+      return if request_queued_at.blank?
+
+      waiting_interval_secs = request_started_at - request_queued_at.to_f
+      return if waiting_interval_secs < 0 # clocks out of alignment
+
+      waiting_interval_secs * 1000
+    end
+
+    def request_queued_float
+      return nil if request_queued_raw.blank?
+
+      # convert values of the form:
+      #   "t=1529578997.145"
+      #     "1529578997145"
+      # to a Float:
+
+      if request_queued_raw =~ /(t=)?([.\d+]+)/
+        Float(Regexp.last_match(2))
+      end
+    end
+
+    def request_start_header(env)
+      env[REQUEST_START_HEADER]
+    end
+  end
+end

data/lib/lograge_rails_request_queuing/request_started_middleware.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module LogrageRailsRequestQueuing
+  # Track at what time the request handling starts
+  class RequestStartedMiddleware
+    def initialize(app)
+      @app = app
+    end
+
+    def call(env)
+      require 'lograge_rails_request_queuing/request_queuing'
+
+      RequestStore[:lograge_request_queueing] = RequestQueueing.new(env)
+      @app.call env
+    end
+  end
+end

data/lib/lograge_rails_request_queuing/silence_exception_logging.rb

@@ -0,0 +1,15 @@
+module LogrageRailsRequestQueuing
+  # Meant to be prepended in ActionDispatch::DebugExceptions like this:
+  #
+  #   ActionDispatch::DebugExceptions.prepend(
+  #     LogrageRailsRequestQueuing::SilenceExceptionLogging
+  #   )
+  module SilenceExceptionLogging
+
+    # Normally ActionDispatch::DebugExceptions will print exception details
+    # and a full stacktrace to the Rails log. This stops that behavior:
+    def log_error *_args
+      # pass
+    end
+  end
+end

data/lib/lograge_rails_request_queuing/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module LogrageRailsRequestQueuing
+  VERSION = '0.2.0'
+end

data/lib/tasks/lograge_rails_request_queuing_tasks.rake

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+# desc "Explaining what the task does"
+# task :lograge_rails_request_queuing do
+#   # Task goes here
+# end

metadata

@@ -0,0 +1,112 @@
+--- !ruby/object:Gem::Specification
+name: lograge_rails_request_queuing
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- Laust Rud Jacobsen
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2018-06-29 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: lograge
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 5.2.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 5.2.0
+- !ruby/object:Gem::Dependency
+  name: request_store
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pry
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: 
+email:
+- laust@valuestream.io
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CODE_OF_CONDUCT.md
+- MIT-LICENSE
+- README.md
+- Rakefile
+- lib/lograge_rails_request_queuing.rb
+- lib/lograge_rails_request_queuing/engine.rb
+- lib/lograge_rails_request_queuing/exception_details.rb
+- lib/lograge_rails_request_queuing/request_queuing.rb
+- lib/lograge_rails_request_queuing/request_started_middleware.rb
+- lib/lograge_rails_request_queuing/silence_exception_logging.rb
+- lib/lograge_rails_request_queuing/version.rb
+- lib/tasks/lograge_rails_request_queuing_tasks.rake
+homepage: https://github.com/rud/lograge_rails_request_queuing
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.7.6
+signing_key: 
+specification_version: 4
+summary: Add request queueing time to lograge output.
+test_files: []