roo_on_rails 1.7.0 → 1.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8adca9db27e5e65ab629420b4d02f4900a9c9629
-  data.tar.gz: e50c1756717e4f484684ae31de1653527ee1b081
+  metadata.gz: 3c14df25a7b6424101a9e98b40db865b5481b507
+  data.tar.gz: dfff2f9bea33530d06533183c2d31be578d0aa96
 SHA512:
-  metadata.gz: fc9a1b2dfc35fff8b74f8a749e6f2858ac8c41d7a8d20b012d2d75d107581bc2643ec44ab2c01ec125f1419c92af06cbad55dca42b9eaa08f8487e554c0d62d9
-  data.tar.gz: 3d8545de35da3827a95ba6ad2d74367d662e4d09e5f0b900aaa567cf5f200b61ea0a7ed6f5110e64ccd0e81c82dbc59160ce877010d55a03a38d0540909093b7
+  metadata.gz: 8c8811e21f02b5eaf1ce83a6653e5e0329c8c0a80f9ef9cf80e407576409c57818ed90662d4ced4b1f44bc6ac51a46dff57376af82a6d8f905d66069acfa6205
+  data.tar.gz: 5e5b53d7ca2df0137cc3326c13231242f30d2ae9743024e34131ad6b9ad167aa429f746d205484e9995b472a2317e235ea528d47de73af14864ca7989dda72ae

data/.rubocop.yml

@@ -28,7 +28,7 @@ Metrics/LineLength:
   Max: 100
   Exclude:
     - Gemfile
-  
+
 Style/Documentation:
   Enabled: false
 
@@ -39,3 +39,8 @@ Style/DotPosition:
   Enabled: false
 Metrics/AbcSize:
   Max: 30
+
+Rubocop/Metrics/MethodLength:
+  Max: 20
+Rubocop/Style/TrailingCommaInLiteral:
+  Enabled: false

data/CHANGELOG.md

@@ -1,3 +1,23 @@
+# v1.8.0 (2017-07-26)
+
+Bug fixes:
+
+- Do not consider 'quiet' workers in the SLA sidekiq metric (#51)
+
+Features:
+
+- Provides a `PLAYBOOK.md` template when detected missing (#42)
+- Adds pre-baked Google OAuth support (#44, #49)
+- Reports Sidekiq metrics only for queues defined in process (#50)
+- Finer-grained Sidekiq configuration (#46)
+    - adds 'default' to list of default Sidekiq queues
+    - accepts custom Sidekiq queue names and permitted latency values
+    - allows environment-specific application checks
+
+Other:
+
+- Fixes the test harness (#48)
+
 # v1.7.0 (2017-07-18)
 
 Features:

data/README.google_oauth2.md

@@ -0,0 +1,84 @@
+## Using the Google OAuth feature
+
+This feature is only supported when using Rails 5+.
+
+`roo_on_rails` provides a pre-baked Omniauth setup to help protect your app with
+Google authentication. Read the main `README` first to set things up; you'll
+need at least `GOOGLE_AUTH_ENABLED=YES`, and `GOOGLE_AUTH_CLIENT_ID` and
+`GOOGLE_AUTH_CLIENT_SECRET` configured.
+
+Let's build a tiny app that has just a homepage, prompts you to sign in, and
+show your email once you have.
+
+We add the landing page route:
+
+```ruby
+# config/routes.rb
+Rails.application.routes.draw do
+  root to: 'landing#index'
+end
+```
+
+And a controller/view:
+
+```ruby
+# app/controllers/landing_controller.rb
+class LandingController < ApplicationController
+  def index
+    if session[:email]
+      render inline: %{
+        You are logged in as <%= session[:email] %>! <br/>
+        <%= link_to 'Logout', auth_logout_path %>
+      }
+    else
+      render inline: %{
+        You are not logged in <br/>
+        <%= link_to 'Login', auth_google_oauth2_path %>
+      }
+    end
+  end
+end
+```
+
+The authentication routes get added by `roo_on_rails`; we need to implement at
+least session creation, destruction, and handling of failure:
+
+```ruby
+# app/controllers/sessions_controller.rb
+
+class SessionsController < ApplicationController
+  def create
+    auth_data = request.env['omniauth.auth']
+    session[:email] = auth_data.info.email.downcase
+    redirect_to root_path
+  end
+
+  def destroy
+    session.clear
+    redirect_to root_path
+  end
+
+  def failure
+    @error = env['omniauth.error']
+    render inline: %{
+      Authentication failed: <br/>
+      <%= @error.class.name %> <br/>
+      <%= @error.message %>
+    }
+  end
+end
+```
+
+And that's it. If you want to blanket-protect a controller, an idiomatic way
+would be to:
+
+```ruby
+before_filter { redirect_to auth_google_oauth2_path unless session[:email] }
+```
+
+If you dislike the name `SessionsController`, you can update
+`GOOGLE_AUTH_CONTROLLER` to point to a different controller.
+
+You can also change the `/auth` path prefix used by this feature; in this case
+you'll want to update the example above. For instance, if you change `/auth` to
+`/prefix`, `auth_google_oauth2_path` becomes `prefix_google_oauth2_path`.

data/README.md

@@ -103,6 +103,11 @@ statement timeouts directly in the database._
 
 ### Sidekiq
 
+Deliveroo services implement Sidekiq with an _urgency_ pattern. By only having
+time-based [SLA](https://en.wikipedia.org/wiki/Service-level_agreement) queue
+names (eg. `within5minutes`) we can automatically create incident alerting for
+queues which take longer than the time the application needs them to be processed.
+
 When `SIDEKIQ_ENABLED` is set we'll:
 
 - check for the existence of a worker line in your Procfile;
@@ -112,11 +117,16 @@ When `SIDEKIQ_ENABLED` is set we'll:
 The following ENV are available:
 
 - `SIDEKIQ_ENABLED`
+- `SIDEKIQ_QUEUES` - comma-separated custom queue names; if not specified, default queues are processed which are defined [here](./lib/roo_on_rails/sidekiq/settings.rb). For accurate health reporting and scaling for your custom queue names, you can specify their permitted latency within this variable e.g. `within5seconds,queue-one:10seconds,queue-two:20minutes,queue-three:3hours,queue-four:1day,default`. For non-default queue names that don't follow the `withinXunit` pattern, you will **need** to specify the permitted latency otherwise the initialization will error.
 - `SIDEKIQ_THREADS` (default: 25) - Sets sidekiq concurrency value
 - `SIDEKIQ_DATABASE_REAPING_FREQUENCY` (default: 10) - For sidekiq processes the
   amount of time in seconds rails will wait before attempting to find and
   recover connections from dead threads
 
+NB. If you are migrating to SLA-based queue names, do not set `SIDEKIQ_ENABLED`
+to `true` before your old queues have finished processing (this will prevent
+Sidekiq from seeing the old queues at all).
+
 ### HireFire (for Sidekiq workers)
 
 When `HIREFIRE_TOKEN` is set an endpoint will be mounted at `/hirefire` that
@@ -173,25 +183,21 @@ logger.with(a: 1, b: 2).info('Stuff')
 See the [class documentation](lib/roo_on_rails/context_logging.rb) for further
 details.
 
-### Google Oauth
+### Google OAuth authentication
+
+When `GOOGLE_AUTH_ENABLED` is set to true we inject a `Omniauth` Rack middleware
+with a pre-configured strategy for Google Oauth2.
 
-When `GOOGLE_AUTH_ENABLED` is set to true we'll:
+Parameters:
 
-* Inject a `Omniauth` Rack middleware with a pre-configured strategy for Google
-  Oauth2.
-* Onject custom Rack middleare to handle Oauth callback requests.
-* Generate the `config/initializers/google_oauth.rb` file that contains some
-  examples of how to wire in your authentication logic.
+- `GOOGLE_AUTH_CLIENT_ID` and `GOOGLE_AUTH_CLIENT_SECRET` (mandatory)
+- `GOOGLE_AUTH_PATH_PREFIX` (optional, defaults to `/auth`)
+- `GOOGLE_AUTH_CONTROLLER` (optional, defaults to `sessions`)
 
-To use this functionality, you must:
+This feature is bring-your-own-controller — it won't magically protect your
+application.
 
-* Obtain the Oauth2 credentials from Google and configure them in
-  `GOOGLE_AUTH_CLIENT_ID` and `GOOGLE_AUTH_CLIENT_SECRET`.
-* Provide in `GOOGLE_AUTH_ALLOWED_DOMAINS` a comma-separated list of domains, to
-  whitelist the allowed email addresses.
-* Customize the code in the generated Rails initializer to hook into your
-  application's authentication logic.
-* Update your Rails controllers to require authentication, when necessary.
+A simple but secure example is detailed in `README.google_oauth2.md`.
 
 
 ## Command features
@@ -204,9 +210,23 @@ Run the following from your app's top-level directory:
 bundle exec roo_on_rails
 ```
 
-You will (currently) need to have admin privileges on the
-`roo-dd-bridge-production` application for this to work. This will be addressed
-eventually.
+That command will sequentially run a number of checks. For it to run successfully, you will need:
+
+- a GitHub API token that can read your GitHub repository's settings placed in `~/.roo_on_rails/github-token`
+- the Heroku toolbelt installed and logged in
+- admin privileges on the `roo-dd-bridge-production` (this will be addressed eventually)
+- checks are run sequentially for staging and then for production. The process halts at any non-fixable failing check. To process only specific environments, you can set a config variable while running the command, like so:
+
+```
+# the default behaviour:
+ROO_ON_RAILS_ENVIRONMENTS=staging,production bundle exec roo_on_rails
+
+# run checks only on staging:
+ROO_ON_RAILS_ENVIRONMENTS=staging bundle exec roo_on_rails
+
+# run checks only on production:
+ROO_ON_RAILS_ENVIRONMENTS=production bundle exec roo_on_rails
+```
 
 
 ### Description

data/gemfiles/rails_3.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    roo_on_rails (1.7.0)
+    roo_on_rails (1.8.0)
       dogstatsd-ruby
       dotenv-rails (~> 2.1)
       faraday
@@ -74,7 +74,7 @@ GEM
       railties (>= 3.2, < 5.2)
     erubis (2.7.0)
     excon (0.57.1)
-    faraday (0.12.1)
+    faraday (0.12.2)
       multipart-post (>= 1.2, < 3)
     faraday_middleware (0.11.0.1)
       faraday (>= 0.7.4, < 1.0)
@@ -137,7 +137,7 @@ GEM
     omniauth (1.4.2)
       hashie (>= 1.2, < 4)
       rack (>= 1.0, < 3)
-    omniauth-google-oauth2 (0.5.0)
+    omniauth-google-oauth2 (0.5.1)
       jwt (~> 1.5)
       multi_json (~> 1.3)
       omniauth (>= 1.1.1)

data/gemfiles/rails_4.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    roo_on_rails (1.7.0)
+    roo_on_rails (1.8.0)
       dogstatsd-ruby
       dotenv-rails (~> 2.1)
       faraday
@@ -81,7 +81,7 @@ GEM
       railties (>= 3.2, < 5.2)
     erubis (2.7.0)
     excon (0.57.1)
-    faraday (0.12.1)
+    faraday (0.12.2)
       multipart-post (>= 1.2, < 3)
     faraday_middleware (0.11.0.1)
       faraday (>= 0.7.4, < 1.0)
@@ -151,7 +151,7 @@ GEM
     omniauth (1.6.1)
       hashie (>= 3.4.6, < 3.6.0)
       rack (>= 1.6.2, < 3)
-    omniauth-google-oauth2 (0.5.0)
+    omniauth-google-oauth2 (0.5.1)
       jwt (~> 1.5)
       multi_json (~> 1.3)
       omniauth (>= 1.1.1)

data/gemfiles/rails_5.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    roo_on_rails (1.7.0)
+    roo_on_rails (1.8.0)
       dogstatsd-ruby
       dotenv-rails (~> 2.1)
       faraday
@@ -84,7 +84,7 @@ GEM
       railties (>= 3.2, < 5.2)
     erubis (2.7.0)
     excon (0.57.1)
-    faraday (0.12.1)
+    faraday (0.12.2)
       multipart-post (>= 1.2, < 3)
     faraday_middleware (0.11.0.1)
       faraday (>= 0.7.4, < 1.0)
@@ -155,7 +155,7 @@ GEM
     omniauth (1.6.1)
       hashie (>= 3.4.6, < 3.6.0)
       rack (>= 1.6.2, < 3)
-    omniauth-google-oauth2 (0.5.0)
+    omniauth-google-oauth2 (0.5.1)
       jwt (~> 1.5)
       multi_json (~> 1.3)
       omniauth (>= 1.1.1)

data/gemfiles/rails_5_1.gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: ..
   specs:
-    roo_on_rails (1.7.0)
+    roo_on_rails (1.8.0)
       dogstatsd-ruby
       dotenv-rails (~> 2.1)
       faraday
@@ -85,7 +85,7 @@ GEM
     erubi (1.6.0)
     erubis (2.7.0)
     excon (0.57.1)
-    faraday (0.12.1)
+    faraday (0.12.2)
       multipart-post (>= 1.2, < 3)
     faraday_middleware (0.11.0.1)
       faraday (>= 0.7.4, < 1.0)
@@ -156,7 +156,7 @@ GEM
     omniauth (1.6.1)
       hashie (>= 3.4.6, < 3.6.0)
       rack (>= 1.6.2, < 3)
-    omniauth-google-oauth2 (0.5.0)
+    omniauth-google-oauth2 (0.5.1)
       jwt (~> 1.5)
       multi_json (~> 1.3)
       omniauth (>= 1.1.1)

data/lib/roo_on_rails/checks/base.rb

@@ -20,10 +20,12 @@ module RooOnRails
         @fix     = @options.delete(:fix) { false }
         @context = @options.delete(:context) { Hashie::Mash.new }
         @shell   = @options.delete(:shell) { Shell.new }
+        @dry_run = options.fetch(:dry_run, false)
       end
 
       def run
         resolve dependencies
+        return true if @dry_run
         say intro
         call
         true

data/lib/roo_on_rails/checks/documentation/playbook.rb

@@ -1,4 +1,5 @@
 require 'roo_on_rails/checks/base'
+require 'fileutils'
 
 module RooOnRails
   module Checks
@@ -11,11 +12,32 @@ module RooOnRails
         end
 
         def call
-          if File.exist?(LOCATION)
-            pass 'playbook found, legion on-call engineers thank you.'
-          else
-            fail! "please create a playbook at #{LOCATION}."
+          fail! "no playbook at #{LOCATION}." if playbook_missing?
+          fail! 'playbook still contains FIXME template sections' if playbook_unfinished?
+          pass  'playbook found, legion on-call engineers thank you.'
+        end
+
+        def fix
+          if !playbook_missing? && playbook_unfinished?
+            fail! 'please add detail to your playbook, removing FIXME sections'
           end
+
+          FileUtils.cp(
+            File.join(__dir__, 'playbook_template.md'),
+            LOCATION
+          )
+        end
+
+        private
+
+        def playbook_unfinished?
+          # The regexp is so that you can still refer to strings saying FIXME in your readme
+          # if you need to, by putting the phrase in backticks: `FIXME`
+          !File.read(LOCATION).match(/FIXME(?!`)/).nil?
+        end
+
+        def playbook_missing?
+          !File.exist?(LOCATION)
         end
       end
     end

data/lib/roo_on_rails/checks/documentation/playbook_template.md

@@ -0,0 +1,29 @@
+# Playbook
+
+This document details the failure modes of this service and mitigating steps that can be taken.
+
+_FIXME: go through this document and fill out the sections marked FIXME._
+
+## Responsible humans
+
+As a last resort, in an emergency, the following people have experience working with this service and have claimed responsibility for the running of this service. **If you have executed the plays below** and there is still a clear and present SEV-1 or greater issue, use the links below to get in touch:
+
+- Responsible McPerson ([PagerDuty](https://deliveroo.pagerduty.com/users/FIXME), [Slack](https://deliveroo.slack.com/team/FIXME))
+
+## Owned models
+
+- _FIXME_ - very brief description of the model, to aid in assessing what knock-on effects trouble with this service might have.
+
+## Failure modes
+
+### FIXME: an ability/functional area of the service
+
+FIXME: A very brief high-level explanation of what this functional area does
+
+#### 🚨 FIXME: a New Relic alert which might be raised - named so it can be searched for
+
+FIXME: A description of the New Relic alert, what it means for the service, and what actions should be taken to remedy the situation.
+
+#### 🚨 FIXME: an action which might need to be taken during an incident
+
+FIXME: A description of how to perform this action, and any side-effects that the on-call engineer should be aware of.

data/lib/roo_on_rails/checks/environment.rb

@@ -3,10 +3,7 @@ require 'roo_on_rails/checks/github/branch_protection'
 require 'roo_on_rails/checks/heroku/app_exists'
 require 'roo_on_rails/checks/heroku/preboot_enabled'
 require 'roo_on_rails/checks/heroku/app_exists'
-require 'roo_on_rails/checks/sidekiq/settings'
 require 'roo_on_rails/checks/heroku/drains_metrics'
-require 'roo_on_rails/checks/documentation/playbook'
-require 'roo_on_rails/checks/google_oauth/initializer'
 require 'roo_on_rails/checks/papertrail/all'
 
 module RooOnRails
@@ -15,9 +12,6 @@ module RooOnRails
       requires GitHub::BranchProtection
       requires Heroku::DrainsMetrics
       requires Heroku::PrebootEnabled
-      requires Sidekiq::Settings
-      requires Documentation::Playbook
-      requires GoogleOauth::Initializer
       requires Papertrail::All
 
       def call

data/lib/roo_on_rails/checks/sidekiq/settings.rb

@@ -12,7 +12,14 @@ module RooOnRails
         end
 
         def call
-          fail! 'Custom Sidekiq settings found.' if File.exist?('config/sidekiq.yml')
+          if File.exist?('config/sidekiq.yml')
+            message = [
+              'Custom Sidekiq settings found.',
+              '  Please see the Roo On Rails readme for more information.'
+            ].join("\n")
+
+            fail! message
+          end
         end
       end
     end

data/lib/roo_on_rails/checks/sidekiq/sidekiq.rb

@@ -1,5 +1,4 @@
 require 'roo_on_rails/checks/base'
-require 'thor'
 
 module RooOnRails
   module Checks
@@ -20,11 +19,8 @@ module RooOnRails
         end
 
         def fix
-          if File.exist?('Procfile')
-            append_to_file 'Procfile', "\n#{WORKER_PROCFILE_LINE}"
-          else
-            create_file 'Procfile', WORKER_PROCFILE_LINE
-          end
+          output = File.exist?('Procfile') ? "\n#{WORKER_PROCFILE_LINE}" : WORKER_PROCFILE_LINE
+          File.open('Procfile', 'a') { |f| f.write(output) }
         end
 
         def check_for_procfile

data/lib/roo_on_rails/config.rb

@@ -17,6 +17,14 @@ module RooOnRails
         enabled? 'GOOGLE_AUTH_ENABLED', default: false
       end
 
+      def google_auth_path_prefix
+        ENV.fetch('GOOGLE_AUTH_PATH_PREFIX')
+      end
+
+      def google_auth_controller
+        ENV.fetch('GOOGLE_AUTH_CONTROLLER')
+      end
+
       private
 
       ENABLED_PATTERN = /\A(YES|TRUE|ON|1)\Z/i

data/lib/roo_on_rails/default.env

@@ -1,7 +1,9 @@
-GOOGLE_AUTH_ENABLED=true
+GOOGLE_AUTH_ENABLED=false
 GOOGLE_AUTH_CLIENT_ID=''
 GOOGLE_AUTH_CLIENT_SECRET=''
 GOOGLE_AUTH_ALLOWED_DOMAINS=''
+GOOGLE_AUTH_PATH_PREFIX=/auth
+GOOGLE_AUTH_CONTROLLER=sessions
 NEW_RELIC_LOG=stdout
 NEW_RELIC_AGENT_ENABLED=true
 NEW_RELIC_MONITOR_MODE=true

data/lib/roo_on_rails/harness.rb

@@ -2,21 +2,29 @@ require 'thor'
 require 'hashie'
 require 'roo_on_rails/checks/environment'
 require 'roo_on_rails/environment'
+require 'roo_on_rails/checks/sidekiq/settings'
+require 'roo_on_rails/checks/documentation/playbook'
 
 module RooOnRails
   class Harness
     include Thor::Shell
 
-    def initialize(try_fix: false, context: nil)
+    def initialize(try_fix: false, context: Hashie::Mash.new, dry_run: false)
       @try_fix = try_fix
-      @context = context || Hashie::Mash.new
+      @context = context
+      @dry_run = dry_run
     end
 
     def run
-      [
-        Checks::Environment.new(env: 'staging',    fix: @try_fix, context: @context),
-        Checks::Environment.new(env: 'production', fix: @try_fix, context: @context),
-      ].each(&:run)
+      checks = [
+        Checks::Sidekiq::Settings.new(fix: @try_fix, context: @context, dry_run: @dry_run),
+        Checks::Documentation::Playbook.new(fix: @try_fix, context: @context, dry_run: @dry_run),
+      ]
+      environments.each do |env|
+        checks << Checks::Environment.new(env: env.strip, fix: @try_fix, context: @context, dry_run: @dry_run)
+      end
+
+      checks.each(&:run)
       self
     rescue Shell::CommandFailed
       say 'A command failed to run, aborting', %i[bold red]
@@ -25,5 +33,11 @@ module RooOnRails
       say 'A check failed, exiting', %i[bold red]
       exit 1
     end
+
+    private
+
+    def environments
+      ENV.fetch('ROO_ON_RAILS_ENVIRONMENTS', 'staging,production').split(',')
+    end
   end
 end

data/lib/roo_on_rails/railties/google_oauth.rb

@@ -0,0 +1,75 @@
+module RooOnRails
+  module Railties
+    class GoogleOAuth < Rails::Railtie
+      initializer 'roo_on_rails.google_auth.middleware' do |app|
+        _if_enabled do
+          _add_middleware(app)
+        end
+      end
+
+      initializer 'roo_on_rails.google_auth.routes', after: :set_routes_reloader_hook do |app|
+        _if_enabled do
+          _add_routes(app)
+          # support development mode on route changes (only works in Rails 5)
+          app.reloader.to_prepare { _add_routes(app) }
+        end
+      end
+
+      private
+
+      def _if_enabled
+        return unless Config.google_auth_enabled?
+        if Rails::VERSION::MAJOR < 5
+          Rails.logger.warn 'The Google OAuth feature is not supported with Rails < 5'
+          return
+        end
+
+        yield
+      end
+
+      def _add_middleware(app)
+        $stderr.puts 'initializer roo_on_rails.google_auth.middleware'
+
+        require 'roo_on_rails/config'
+        require 'omniauth'
+        require 'omniauth-google-oauth2'
+        require 'active_support/core_ext/object/blank'
+
+        options = {
+          path_prefix: Config.google_auth_path_prefix,
+          prompt: 'consent',
+          # https://stackoverflow.com/questions/45271730/jwtinvalidissuererror-invalid-issuer-expected-accounts-google-com-received
+          # https://github.com/zquestz/omniauth-google-oauth2/issues/197
+          skip_jwt: true,
+        }
+
+        domain_list = ENV.fetch('GOOGLE_AUTH_ALLOWED_DOMAINS', '').split(',').reject(&:blank?)
+        options[:hd] = domain_list if domain_list.any?
+
+        app.config.middleware.use ::OmniAuth::Builder do
+          provider :google_oauth2,
+            ENV.fetch('GOOGLE_AUTH_CLIENT_ID'),
+            ENV.fetch('GOOGLE_AUTH_CLIENT_SECRET'),
+            options
+        end
+      end
+
+      def _add_routes(app)
+        $stderr.puts 'initializer roo_on_rails.google_auth.routes'
+
+        prefix = Config.google_auth_path_prefix
+        ctrl   = Config.google_auth_controller
+
+        app.routes.disable_clear_and_finalize = true
+        app.routes.draw do
+          get  "#{prefix}/google_oauth2",           controller: ctrl, action: 'failure'
+          get  "#{prefix}/google_oauth2/callback",  controller: ctrl, action: 'create'
+          post "#{prefix}/google_oauth2/callback",  controller: ctrl, action: 'create'
+          get  "#{prefix}/failure",                 controller: ctrl, action: 'failure'
+          get  "#{prefix}/logout",                  controller: ctrl, action: 'destroy'
+        end
+        app.routes.disable_clear_and_finalize = false
+      end
+    end
+  end
+end

data/lib/roo_on_rails/sidekiq/metrics_worker.rb

@@ -1,8 +1,14 @@
 require 'sidekiq/api'
 require 'roo_on_rails/sidekiq/queue_latency'
 require 'roo_on_rails/sidekiq/process_scaling'
+require 'roo_on_rails/sidekiq/settings'
 require 'roo_on_rails/statsd'
 
+# Reports Sidekiq queue metrics for queues configured within the current Sidekiq process
+# i.e. queues returned by `RooOnRails::Sidekiq::Settings.queues`
+# To enable reporting for custom queues, ensure your process is running the monitoring
+# queue e.g. `SIDEKIQ_QUEUES="new-queue:5seconds,monitoring" bundle exec sidekiq`
+
 module RooOnRails
   module Sidekiq
     class MetricsWorker
@@ -12,7 +18,7 @@ module RooOnRails
 
       def perform
         RooOnRails.statsd.batch do |stats|
-          queues = ::Sidekiq::Queue.all.map { |q| QueueLatency.new(q) }
+          queues = QueueLatency.queues
           queue_stats(stats, queues)
           process_stats(stats, queues)
         end

data/lib/roo_on_rails/sidekiq/process_scaling.rb

@@ -3,37 +3,38 @@ require 'sidekiq/api'
 module RooOnRails
   module Sidekiq
     class ProcessScaling
+      WORKER_INCREASE_THRESHOLD = ENV.fetch('WORKER_INCREASE_THRESHOLD', 0.5).to_f
+      WORKER_DECREASE_THRESHOLD = ENV.fetch('WORKER_DECREASE_THRESHOLD', 0.1).to_f
+      private_constant :WORKER_INCREASE_THRESHOLD
+      private_constant :WORKER_DECREASE_THRESHOLD
+
       def initialize(queue_latencies)
         @queue_latencies = queue_latencies
+        @queue_names = @queue_latencies.map(&:name)
       end
 
       def current_processes
-        ::Sidekiq::ProcessSet.new.count
+        ::Sidekiq::ProcessSet.new.count do |process|
+          process['quiet'] == 'false' &&
+          @queue_names.any? do |queue_name|
+            process['queues'].include?(queue_name)
+          end
+        end
       end
 
       def max_normalised_latency
-        @queue_latencies.any? ? @queue_latencies.map(&:normalised_latency).max : 0
+          @queue_latencies.any? ? @queue_latencies.map(&:normalised_latency).max.to_f : 0.0
       end
 
       def requested_processes
-        if max_normalised_latency > increasing_latency
+        if max_normalised_latency > WORKER_INCREASE_THRESHOLD
           current_processes + 1
-        elsif max_normalised_latency < decreasing_latency
+        elsif max_normalised_latency < WORKER_DECREASE_THRESHOLD
           [current_processes - 1, 1].max
         else
           current_processes
         end
       end
-
-      protected
-
-      def increasing_latency
-        ENV.fetch('WORKER_INCREASE_THRESHOLD', 0.5).to_f
-      end
-
-      def decreasing_latency
-        ENV.fetch('WORKER_DECREASE_THRESHOLD', 0.1).to_f
-      end
     end
   end
 end

data/lib/roo_on_rails/sidekiq/queue_latency.rb

@@ -1,4 +1,6 @@
+require 'active_support'
 require 'active_support/core_ext/numeric'
+require 'roo_on_rails/sidekiq/settings'
 
 module RooOnRails
   module Sidekiq
@@ -8,23 +10,20 @@ module RooOnRails
       def_delegators :@queue, :size, :latency, :name
       attr_reader :queue
 
+      def self.queues
+        ::Sidekiq::Queue.all.each_with_object([]) do |q, array|
+          array << new(q) if Settings.queues.include?(q.name.to_s)
+        end
+      end
+
       def initialize(queue)
         @queue = queue
       end
 
       def normalised_latency
-        metric = queue.latency.to_f / permitted_latency
-        metric.round(3)
-      end
-
-      def permitted_latency
-        prefix, number, unit = queue.name.partition(/[0-9]+/)
-        case prefix
-        when 'monitoring', 'realtime' then 10.seconds.to_i
-        when 'default' then 1.day.to_i
-        when 'within' then number.to_i.public_send(unit.to_sym).to_i
-        else raise "Cannot determine permitted latency for queue #{queue.name}"
-        end
+        permitted_latency = Settings.permitted_latency_values[queue.name]
+        return queue.latency.fdiv(permitted_latency).round(3) if permitted_latency
+        raise("Cannot determine permitted latency for queue #{queue.name}")
       end
     end
   end

data/lib/roo_on_rails/sidekiq/settings.rb

@@ -1,20 +1,54 @@
+require_relative './queue_latency'
+
 module RooOnRails
   module Sidekiq
     class Settings
-      def self.queues
-        %w(
-          monitoring
-          realtime
-          within1minute
-          within5minutes
-          within30minutes
-          within1hour
-          within1day
-        ).freeze
-      end
+      DEFAULT_QUEUE_LATENCY_VALUES = {
+        'monitoring' => 10.seconds.to_i,
+        'realtime' => 10.seconds.to_i,
+        'within1minute' => 1.minute.to_i,
+        'within5minutes' => 5.minutes.to_i,
+        'within30minutes' => 30.minutes.to_i,
+        'within1hour' => 1.hour.to_i,
+        'within1day' => 1.day.to_i,
+        'default' => 1.day.to_i
+      }.freeze
+
+      class << self
+        def queues
+          @queues ||= permitted_latency_values.sort_by(&:last).map(&:first).freeze
+        end
+
+        def concurrency
+          ENV.fetch('SIDEKIQ_THREADS', 25)
+        end
+
+        def permitted_latency_values
+          @permitted_latency_values ||= ENV.key?('SIDEKIQ_QUEUES') ? env_queue_latency_values.freeze : DEFAULT_QUEUE_LATENCY_VALUES
+        end
+
+        private
 
-      def self.concurrency
-        ENV.fetch('SIDEKIQ_THREADS', 25)
+        def env_queue_latency_values
+          {}.tap do |hash|
+            ENV['SIDEKIQ_QUEUES'].split(',').each do |entry|
+              queue_entry = entry.strip
+              if DEFAULT_QUEUE_LATENCY_VALUES.key?(queue_entry)
+                queue_name = queue_entry
+                hash[queue_name] = DEFAULT_QUEUE_LATENCY_VALUES[queue_entry]
+              elsif queue_entry.match(/\Awithin\d+.+\z/)
+                _, number, unit = queue_entry.partition(/\d+/)
+                hash[queue_entry] = number.to_i.public_send(unit.strip).to_i
+              elsif queue_entry.include?(':')
+                queue_name, latency_info = queue_entry.split(':')
+                _, number, unit = latency_info.partition(/\d+/)
+                hash[queue_name] = number.to_i.public_send(unit.strip).to_i
+              else
+                hash[queue_entry] = nil
+              end
+            end
+          end
+        end
       end
     end
   end

data/lib/roo_on_rails/sidekiq/sla_metric.rb

@@ -6,10 +6,10 @@ module RooOnRails
   module Sidekiq
     # Returns stats on the current SLA performance of queues in a Sidekiq instance.
     #
-    # Assumes workers are not bound to queues.
+    # Only returns stats for queues being processed by current Sidekiq process
     class SlaMetric
       def self.queue
-        queues = ::Sidekiq::Queue.all.map { |q| QueueLatency.new(q) }
+        queues = QueueLatency.queues
         global_stats = ProcessScaling.new(queues)
         global_stats.requested_processes
       end

data/lib/roo_on_rails/version.rb

@@ -1,3 +1,3 @@
 module RooOnRails
-  VERSION = '1.7.0'.freeze
+  VERSION = '1.8.0'.freeze
 end

data/lib/roo_on_rails.rb

@@ -11,5 +11,5 @@ if defined?(Rails)
   require 'roo_on_rails/railties/http'
   require 'roo_on_rails/railties/sidekiq'
   require 'roo_on_rails/railties/rake_tasks'
-  require 'roo_on_rails/railties/google_auth'
+  require 'roo_on_rails/railties/google_oauth'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: roo_on_rails
 version: !ruby/object:Gem::Version
-  version: 1.7.0
+  version: 1.8.0
 platform: ruby
 authors:
 - Julien Letessier
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-07-18 00:00:00.000000000 Z
+date: 2017-07-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: dotenv-rails
@@ -358,6 +358,7 @@ files:
 - Gemfile
 - Guardfile
 - LICENSE.txt
+- README.google_oauth2.md
 - README.md
 - Rakefile
 - appraise
@@ -377,13 +378,12 @@ files:
 - lib/roo_on_rails/checks.rb
 - lib/roo_on_rails/checks/base.rb
 - lib/roo_on_rails/checks/documentation/playbook.rb
+- lib/roo_on_rails/checks/documentation/playbook_template.md
 - lib/roo_on_rails/checks/env_specific.rb
 - lib/roo_on_rails/checks/environment.rb
 - lib/roo_on_rails/checks/git/origin.rb
 - lib/roo_on_rails/checks/github/branch_protection.rb
 - lib/roo_on_rails/checks/github/token.rb
-- lib/roo_on_rails/checks/google_oauth/_template.rb
-- lib/roo_on_rails/checks/google_oauth/initializer.rb
 - lib/roo_on_rails/checks/helpers.rb
 - lib/roo_on_rails/checks/heroku/app_exists.rb
 - lib/roo_on_rails/checks/heroku/drains_metrics.rb
@@ -408,11 +408,10 @@ files:
 - lib/roo_on_rails/harness.rb
 - lib/roo_on_rails/logfmt.rb
 - lib/roo_on_rails/papertrail_client.rb
-- lib/roo_on_rails/rack/google_oauth.rb
 - lib/roo_on_rails/rack/safe_timeouts.rb
 - lib/roo_on_rails/railtie.rb
 - lib/roo_on_rails/railties/database.rb
-- lib/roo_on_rails/railties/google_auth.rb
+- lib/roo_on_rails/railties/google_oauth.rb
 - lib/roo_on_rails/railties/http.rb
 - lib/roo_on_rails/railties/new_relic.rb
 - lib/roo_on_rails/railties/rake_tasks.rb

data/lib/roo_on_rails/checks/google_oauth/_template.rb

@@ -1,49 +0,0 @@
-# Google Oauth initializer, generated by RooOnRails
-
-require 'roo_on_rails/rack/google_oauth'
-
-Rails.application.config.middleware.use RooOnRails::Rack::GoogleOauth do |env|
-  # This is your auth strategy.
-  # Here you're supposed to do something with the OAuth payload and
-  # return a valid Rack response.
-
-  # A simple and insecure example:
-  #
-  require 'digest/md5'
-  auth_data = env['omniauth.auth']
-  naive_token = Digest::MD5.hexdigest(auth_data.info.email.downcase)
-  expires_in = Time.current + 60 * 60 * 24
-  headers = { 'Location' => '/' }
-  Rack::Utils.set_cookie_header!(headers, 'naive_auth_cookie', {
-    value: naive_token, expires: expires_in, path: '/'
-  })
-  [302, headers, ['You are being redirecred to /']]
-
-  # You can also hand it over to a Rails controller action, where the
-  # OAuth payload will be available in `request.env['omniauth.auth']`.
-  # If you do this, the controller will take care of returning a valid
-  # response for Rack.
-  #
-  # This is the recommenced approach as it makes it easier to use
-  # Rails encrypted cookies and other security features.
-  #
-  # For example:
-  # MyAuthController.action(:login).call(env)
-end
-
-# What to do in case of failure.
-# Must be a 302 redirect.
-# It can invoke a Rails controller action
-#
-OmniAuth.config.on_failure = proc do |env|
-  error = env['omniauth.error']           # e.g. #<OmniAuth::Strategies::OAuth2::CallbackError: OmniAuth::Strategies::OAuth2::CallbackError>
-  details = error.message                 # e.g. "invalid_hd | Invalid Hosted Domain"
-  error_type = env['omniauth.error.type'] # e.g. :invalid_credentials
-
-  Rails.logger.info("[RooOnRails] Login failed (#{error_type}): #{details}")
-
-  # To use a rails controller;
-  # MyAuthController.action(:login_failed).call(env)
-
-  [302, { 'Location' => '/' }, ['']]
-end

data/lib/roo_on_rails/checks/google_oauth/initializer.rb

@@ -1,43 +0,0 @@
-require 'roo_on_rails/config'
-require 'roo_on_rails/checks/base'
-require 'fileutils'
-
-module RooOnRails
-  module Checks
-    module GoogleOauth
-      class Initializer < Base
-        LOCATION = 'config/initializers/google_oauth.rb'.freeze
-
-        def intro
-          'Google Oauth protection'
-        end
-
-        def call
-          if RooOnRails::Config.google_auth_enabled?
-            check_initializer
-          else
-            pass 'Google Oauth is not enabled. Doing nothing'
-          end
-        end
-
-        def fix
-          FileUtils.cp(template, LOCATION)
-        end
-
-        private
-
-        def check_initializer
-          if File.exist? LOCATION
-            pass 'Google Oauth initializer is present. Doing nothing.'
-          else
-            fail! 'Google Oauth is enabled but the initializer is missing.'
-          end
-        end
-
-        def template
-          File.join(__dir__, '_template.rb')
-        end
-      end
-    end
-  end
-end

data/lib/roo_on_rails/rack/google_oauth.rb

@@ -1,34 +0,0 @@
-require 'rack'
-
-module RooOnRails
-  module Rack
-    class GoogleOauth
-      OAUTH_CALLBACK = '/auth/google_oauth2/callback'.freeze
-
-      def initialize(app, *args, &block)
-        @app = app
-        @args = args
-        @strategy = block
-      end
-
-      def call(env)
-        if is_oauth_callback?(env)
-          @strategy.call(env)
-        else
-          send_to_upstream(env)
-        end
-      end
-
-      private
-
-      def send_to_upstream(env)
-        @app.call(env)
-      end
-
-      def is_oauth_callback?(env)
-        request = ::Rack::Request.new(env)
-        request.fullpath.start_with?(OAUTH_CALLBACK)
-      end
-    end
-  end
-end

data/lib/roo_on_rails/railties/google_auth.rb

@@ -1,36 +0,0 @@
-require 'roo_on_rails/config'
-require 'omniauth'
-require 'omniauth-google-oauth2'
-require 'active_support/core_ext/object/blank'
-
-module RooOnRails
-  module Railties
-    class GoogleAuth < Rails::Railtie
-      initializer 'roo_on_rails.google_auth' do |app|
-        if RooOnRails::Config.google_auth_enabled?
-          $stderr.puts 'initializer roo_on_rails.google_auth'
-
-          google_oauth2_client_id = ENV.fetch('GOOGLE_AUTH_CLIENT_ID')
-          google_oauth2_client_secret = ENV.fetch('GOOGLE_AUTH_CLIENT_SECRET')
-
-          options = {
-            path_prefix: '/auth',
-            prompt: 'consent',
-          }
-
-          domain_list = ENV.fetch('GOOGLE_AUTH_ALLOWED_DOMAINS', '').split(',').reject(&:blank?)
-          options[:hd] = domain_list if domain_list.any?
-
-          app.config.middleware.use ::OmniAuth::Builder do
-            provider :google_oauth2,
-              google_oauth2_client_id,
-              google_oauth2_client_secret,
-              options
-          end
-        else
-          $stderr.puts 'skipping initializer roo_on_rails.google_auth'
-        end
-      end
-    end
-  end
-end