ndr_dev_support 2.1.2 → 3.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: b462909165b10c8d687b0748b757ee8a797766bc
-  data.tar.gz: bfaa3539a90a7b010f81e12d31c2852a25f1a6e4
+SHA256:
+  metadata.gz: dbcfdf282fc498259213e7c7961b199abf2fab0b20ffd2b3e7ef38d8222722d6
+  data.tar.gz: 3403335c3c26478a99ff0b2de557d0d4c7899d8325073073d12a84d78e36211d
 SHA512:
-  metadata.gz: 202df7fbda561139125efae9eeb3ba6e1753c0252c8e3097a6febb0668a81e866bb2daf1ba8f6480dafb6ecd37cabd1123d7f1a798869ef825a21ea7336f428f
-  data.tar.gz: abf58184c149a5053dc394ef41184e54b733a4fe5aaa01be61b3455f11765c5d1aabf91af8a927efdff74d83a3dbcfcbafeac72dc1ceb9ee00fb8cc7216fb26e
+  metadata.gz: '02974ba29c5f09941a52fb3fcc7af3aeb01a9edb9d80608ac500959dd19fb39290b35e5b33c55e55f833f552f0313a86c0ca370d333531b6e182f5db4ec8a2a3'
+  data.tar.gz: ec95966fbb3dcf4826e54e1eb1c14b981541243d49daf1c83662e2b109f4b36a9973a5320f9b6615e6fe13de53c8d817966d596bc834914a66af10c270032aca

data/.rubocop.yml

@@ -13,10 +13,23 @@ AllCops:
   - 'tmp/**/*'
   - 'vendor/**/*'
 
+# Once supported by RuboCop, this will prevent cop-specific Excludes from
+# overwriting the AllCops defaults above:
+#
+#   inherit_mode:
+#     merge:
+#       - Exclude
+
 # Run the Rails cops by default (-R/--rails not required):
 Rails:
   Enabled: true
 
+##################### Layout #################################
+
+Layout/DotPosition:
+  # Multi-line method chaining should be done with trailing dots.
+  EnforcedStyle: trailing
+
 ##################### Style ##################################
 
 # We make use of block comments, e.g. for validation documentation.
@@ -27,10 +40,6 @@ Style/Documentation:
   Exclude:
   - 'test/**/*.rb'
 
-Style/DotPosition:
-  # Multi-line method chaining should be done with trailing dots.
-  EnforcedStyle: trailing
-
 Style/FrozenStringLiteralComment:
   # We're not confident enough to make this recommendation everywhere
   Enabled: false
@@ -46,6 +55,10 @@ Style/NumericLiterals:
   Exclude:
   - 'test/**/*.rb'
 
+Style/YodaCondition:
+  # Disagree; literals as first argument can guard against accidental assignment.
+  Enabled: false
+
 Style/SingleLineBlockParams:
   # Prefer readability of contextually-named variables.
   Enabled: false
@@ -101,3 +114,9 @@ Rails/ActionFilter:
   # projects will want to override this configuration to use 'filter' instead.
   EnforcedStyle: action
 
+Rails/SkipsModelValidations:
+  # Methods like 'update_column' exist for a reason, and it is the developer's
+  # responsibilty to understand the behaviour of the code they write; blanket
+  # avoiding them is not helpful/practical.
+  Enabled: false
+

data/README.md

@@ -85,54 +85,47 @@ $ find . -iregex .*\.rake$ | xargs rake rubocop:diff:file
 
 ### Integration test environment
 
-ndr_dev_support bundles a configured Rails integration testing environment. It uses `capybara` and `poltergeist` to drive a PhantomJS headless browser, and includes some sensible configuration.
+ndr_dev_support bundles a configured Rails integration testing environment.
 
-If an integration test errors or fails, `capybara-screenshot` is used to automatically retrieve a full-height screenshot from PhantomJS, which is then stored in `tmp/`.
+By default, it uses `capybara` and `poltergeist` to drive a PhantomJS headless browser, and includes some sensible configuration.
 
-Beyond standard Capybara testing DSL, ndr_dev_support bundles some additional functionality:
-
-* `clear_headless_session!` - causes PhantomJS to reset, simulating a browser restart.
-* `delete_all_cookies!` - causes PhantomJS to delete all cookies. Helpful for testing AJAX logouts.
-* `within_screenshot_compatible_window` – similar to `within_window`, but allows failure screenshots to be taken of the failing child window, rather than the spawning parent.
-
-To use, ensure `phantomjs` is installed, and add the following to your application's `test_helper.rb`
+To use, simply add the following to your application's `test_helper.rb`
 
 ```ruby
 require 'ndr_dev_support/integration_testing'
 ```
 
-When using `capybara` with PhantomJS, the test database must be consistent between the test runner and the application being tested. With transactional tests in operation, this means that both must share a connection. Doing so is error-prone, and can introduce race conditions. However, some projects have had success with the approach, so it is available within `ndr_dev_support` with the following additional require statement:
+#### Other drivers
+
+Other drivers are also supported; `chrome` / `chrome_headless` / `firefox` are all powered by selenium, and can either be explicitly used with:
 
 ```ruby
-# WARNING: can result in race conditions within the test suite
-require 'ndr_dev_support/integration_testing/connection_sharing'
+Capybara.default_driver    = :chrome_headless
+Capybara.javascript_driver = :chrome_headless
 ```
 
-The slower, more reliable, alternative is to use the `database_cleaner` gem. `ndr_dev_support` provides no built-in support for this approach, as configuration can be quite project-specific. However, as a starting point:
-
-Add to the `Gemfile`:
+...or, assuming no driver has been explicitly set, can be selected at runtime:
 
-```ruby
-group :test do
-  gem 'database_cleaner'
-end
+```
+$ INTEGRATION_DRIVER=chrome_headless bin/rake test
 ```
 
-Add to `test_helper.rb`:
+#### Screenshots
 
-```ruby
-require 'database_cleaner'
-DatabaseCleaner.strategy = :deletion # anecdotally, faster than :truncation for our projects
+If an integration test errors or fails, `capybara-screenshot` is used to automatically retrieve a full-height screenshot from the headless browser, which is then stored in `tmp/`.
 
-class ActionDispatch::IntegrationTest
-  # Don't wrap each test case in a transaction:
-  self.use_transactional_tests = false
+#### DSL extensions
 
-  # Instead, insert fixtures afresh between each test:
-  setup    { DatabaseCleaner.start }
-  teardown { DatabaseCleaner.clean }
-end
-```
+Beyond standard Capybara testing DSL, ndr_dev_support bundles some additional functionality:
+
+* `clear_headless_session!` - causes the headless browser to reset, simulating a browser restart.
+* `delete_all_cookies!` - causes the headless browser to delete all cookies. Helpful for testing AJAX logouts.
+* `within_screenshot_compatible_window` – similar to `within_window`, but allows failure screenshots to be taken of the failing child window, rather than the spawning parent.
+* `within_modal` - scope capybara to only interact within a modal, and (by default) expect the modal to disappear when done.
+
+#### Database synchronisation
+
+When using a headless browser for integration tests, the test database must be consistent between the test runner and the application being tested. With transactional tests in operation, this means that both must share a connection. It is up to the individual project to provide this facility; as of Rails 5.1, it is built in to the framework directly.
 
 ## Development
 

data/code_safety.yml

@@ -11,7 +11,7 @@ file safety:
   ".rubocop.yml":
     comments: 
     reviewed_by: timgentry
-    safe_revision: ac89be1deb3d47e9fb00ac2de424116593012074
+    safe_revision: 3a4fc58f43e28c32e4fea25fa56a1eaf4a887d84
   ".travis.yml":
     comments: 
     reviewed_by: josh.pencheon
@@ -30,8 +30,8 @@ file safety:
     safe_revision: c59a45986f8b6d087c8c21b1e889f31f7346da17
   README.md:
     comments: 
-    reviewed_by: timgentry
-    safe_revision: 04a0d70cc467dad5a380744a6c678693429e0213
+    reviewed_by: josh.pencheon
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
   Rakefile:
     comments: 
     reviewed_by: josh.pencheon
@@ -48,26 +48,54 @@ file safety:
     comments: 
     reviewed_by: timgentry
     safe_revision: c59a45986f8b6d087c8c21b1e889f31f7346da17
+  lib/ndr_dev_support/daemon/ci_server.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 3d81c21c0085acb44aedadeaa66313ac280eb326
+  lib/ndr_dev_support/daemon/stoppable.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 1bdb008829c0c19d7167f9d6aea9b97db15a5403
   lib/ndr_dev_support/integration_testing.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 5b238d73cfb0e506ced8b4e474cc2b3392838925
-  lib/ndr_dev_support/integration_testing/capybara.rb:
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
+  lib/ndr_dev_support/integration_testing/drivers/chrome.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
+  lib/ndr_dev_support/integration_testing/drivers/chrome_headless.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
+  lib/ndr_dev_support/integration_testing/drivers/firefox.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
+  lib/ndr_dev_support/integration_testing/drivers/poltergeist.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 88ac9269eb8f0b44c5920a946708e4715421bcad
-  lib/ndr_dev_support/integration_testing/connection_sharing.rb:
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
+  lib/ndr_dev_support/integration_testing/drivers/switchable.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 6cf4a9cab6f8f028d45e05e256b63d9fe9d1b7fc
-  lib/ndr_dev_support/integration_testing/poltergeist.rb:
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
+  lib/ndr_dev_support/integration_testing/dsl.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 88ac9269eb8f0b44c5920a946708e4715421bcad
-  lib/ndr_dev_support/integration_testing/screenshot.rb:
+    safe_revision: 12d9c51ec2bc946ee02eee63700e905b8c0c01c1
+  lib/ndr_dev_support/rake_ci/brakeman_helper.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 88ac9269eb8f0b44c5920a946708e4715421bcad
+    safe_revision: 49caa4ef8ad9926843354d36f4610d40603ec632
+  lib/ndr_dev_support/rake_ci/concerns/commit_metadata_persistable.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 49caa4ef8ad9926843354d36f4610d40603ec632
+  lib/ndr_dev_support/rake_ci/simple_cov_helper.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 6240a84dc859af0623328f8667b93299c8eae257
   lib/ndr_dev_support/rubocop/executor.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -84,27 +112,79 @@ file safety:
     comments: 
     reviewed_by: josh.pencheon
     safe_revision: 41cf1558f567928faaa1e670a36d941c03c78044
+  lib/ndr_dev_support/slack_message_publisher.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: ad38e92c6e56b9d81fdab10681d8f2924eeadf5a
   lib/ndr_dev_support/tasks.rb:
     comments: 
-    reviewed_by: joshpencheon
-    safe_revision: a55c971cb770a8e5dc4be7ed66663dd5d695fcd1
+    reviewed_by: josh.pencheon
+    safe_revision: 6240a84dc859af0623328f8667b93299c8eae257
   lib/ndr_dev_support/version.rb:
     comments: 
-    reviewed_by: timgentry
-    safe_revision: fde5bc348b5393ab95b8f0e8d5cae9df29a85f34
+    reviewed_by: josh.pencheon
+    safe_revision: b725560e101395a6dd1a9ece106462edb6ebcc74
   lib/tasks/audit_code.rake:
     comments: Identical to the version reviewed by josh.pencheon when contained within
       ndr_support
+    reviewed_by: josh.pencheon
+    safe_revision: 73030d56a06d05ae2d8716713b01647d76b7d904
+  lib/tasks/ci/brakeman.rake:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 49caa4ef8ad9926843354d36f4610d40603ec632
+  lib/tasks/ci/bundle_audit.rake:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: ddcf7de0bf1f5fb17cc28e2460009e162ff8917c
+  lib/tasks/ci/bundle_install.rake:
+    comments: 
+    reviewed_by: timgentry
+    safe_revision: cf22c7ab8b2eb67b501b5ebd1309c832db4fdf6c
+  lib/tasks/ci/housekeep.rake:
+    comments: 
+    reviewed_by: timgentry
+    safe_revision: 3dfcdb6c76fbccc44a331986673d63282f87f2fa
+  lib/tasks/ci/linguist.rake:
+    comments: 
+    reviewed_by: timgentry
+    safe_revision: 00cd6c2f71612a467ee88ad53c21e08f073871d5
+  lib/tasks/ci/notes.rake:
+    comments: 
+    reviewed_by: timgentry
+    safe_revision: f828113894a16581d0aa181504c799e661f8401d
+  lib/tasks/ci/prometheus.rake:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: a10b6f3f5c9261135fae534e5785e2342430dde2
+  lib/tasks/ci/rugged.rake:
+    comments: 
+    reviewed_by: timgentry
+    safe_revision: 7ab7061b257f916eb43bc8d184aa425f1f08b739
+  lib/tasks/ci/server.rake:
+    comments: 
+    reviewed_by: timgentry
+    safe_revision: e581239a77666500a3108bcb3d480e370a82adb8
+  lib/tasks/ci/simplecov.rake:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 6240a84dc859af0623328f8667b93299c8eae257
+  lib/tasks/ci/slack.rake:
+    comments: 
     reviewed_by: timgentry
-    safe_revision: 4813e92fc951a3c033c08c848a9e07e8d5867133
+    safe_revision: ad38e92c6e56b9d81fdab10681d8f2924eeadf5a
+  lib/tasks/ci/stats.rake:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: 137be2205f283d444286c47502387487bb1b7015
   lib/tasks/rubocop.rake:
     comments: 
     reviewed_by: josh.pencheon
     safe_revision: 41cf1558f567928faaa1e670a36d941c03c78044
   ndr_dev_support.gemspec:
     comments: 
-    reviewed_by: timgentry
-    safe_revision: ac89be1deb3d47e9fb00ac2de424116593012074
+    reviewed_by: josh.pencheon
+    safe_revision: e0b3871607c649565330d6e62740ffd860930338
   test/ndr_dev_support_test.rb:
     comments: 
     reviewed_by: timgentry

data/lib/ndr_dev_support/daemon/ci_server.rb

@@ -0,0 +1,94 @@
+require_relative 'stoppable'
+require 'rugged'
+
+module NdrDevSupport
+  module Daemon
+    # Wrapper around Rake based CI testing loop
+    class CIServer
+      include Stoppable
+
+      GIT_SVN_REMOTE_BRANCH_NAME = 'git-svn'.freeze
+      MASTER_BRANCH_NAME = 'master'.freeze
+      ORIGIN_MASTER_BRANCH_NAME = 'origin/master'.freeze
+
+      attr_reader :repo
+
+      def self.from_args(env)
+        name = env['WORKER_NAME'].to_s
+
+        new(name: name)
+      end
+
+      def initialize(name:)
+        super
+
+        # Worker name can be used for clear logging:
+        @name = name
+        raise ArgumentError, 'No WORKER_NAME specified!' if name.blank?
+
+        @repo = Rugged::Repository.new('.')
+      end
+
+      def self.friendly_revision_name(commit)
+        if (matchdata = commit.message.match(/\bgit-svn-id: [^@]+@(\d+)\s/))
+          matchdata[1]
+        else
+          commit.oid[0, 7]
+        end
+      end
+
+      private
+
+      def run_once
+        git_fetch
+        git_checkout(MASTER_BRANCH_NAME)
+
+        objectids_between_master_and_remote.each do |oid|
+          `git rebase #{oid}`
+
+          Rake::Task['ci:all'].invoke
+        end
+      end
+
+      def git_fetch
+        svn_remote? ? `git svn fetch` : `git fetch`
+      end
+
+      def git_checkout(oid)
+        `git checkout #{oid}`
+      end
+
+      def svn_remote?
+        GIT_SVN_REMOTE_BRANCH_NAME == remote_branch
+      end
+
+      def remote_branch
+        return @remote_branch if @remote_branch
+
+        remote_branches = repo.branches.each_name(:remote)
+        if remote_branches.count == 1
+          @remote_branch = remote_branches.first
+        elsif remote_branches.include?(ORIGIN_MASTER_BRANCH_NAME)
+          @remote_branch = ORIGIN_MASTER_BRANCH_NAME
+        else
+          raise "One remote branch expected (#{remote_branches.to_a.inspect})"
+        end
+      end
+
+      def objectids_between_master_and_remote
+        walker = Rugged::Walker.new(@repo)
+        walker.push(repo.branches[remote_branch].target_id)
+        current_target_id = repo.branches[MASTER_BRANCH_NAME].target_id
+
+        revisions = []
+        # walk backwards from the most recent commit, breaking at the current one
+        walker.each do |commit|
+          break if commit.oid == current_target_id
+          revisions << commit.oid
+        end
+
+        revisions.reverse
+      end
+    end
+  end
+end

data/lib/ndr_dev_support/daemon/stoppable.rb

@@ -0,0 +1,111 @@
+require 'active_support/all'
+require 'active_support/core_ext/numeric/bytes'
+
+module NdrDevSupport
+  module Daemon
+    # Behaviour that allows daemons to be restarted, and stopped by god.
+    # To use, you need to call `super` in the initialize method, if defined.
+    module Stoppable
+      extend ActiveSupport::Concern
+
+      # touch this file to trigger graceful exit
+      # TODO: Consider supporting Rails
+      # RESTART_FILENAME = Rails.root.join('tmp', 'restart.txt')
+      RESTART_FILENAME = 'restart.txt'
+
+      MAX_MEMORY = 3.gigabytes # restart between jobs if memory consumption exceeds this
+      MAX_UPTIME = 2.hours     # restart between jobs if have been up this long
+
+      # how long the daemon waits when it runs out of things to do:
+      # BIG_SLEEP = Rails.env.development? ? 10.seconds : 1.minute
+      BIG_SLEEP = 1.minute
+
+      # when idle, how long the daemon between making restart checks?
+      LITTLE_SLEEP = 5.seconds
+
+      included do
+        attr_reader :name, :start_time
+      end
+
+      def initialize(*)
+        setup_signals
+
+        @start_time = Time.current
+      end
+
+      def stop
+        @should_stop = true
+      end
+
+      def should_stop?
+        @should_stop ||= restart_file_touched? || excessive_memory? || been_up_a_while?
+      end
+
+      def run(exit_when_done: false)
+        loop do
+          run_once
+
+          # we've done all we can for the time being; either exit now, or
+          # have a sleep and loop round for another go:
+          break if exit_when_done
+          snooze(BIG_SLEEP)
+          # Our snooze may have come to an abrupt end:
+          break if should_stop?
+        end
+
+        if should_stop?
+          # An instruction to stop has been received:
+          log('Stopping')
+          return :stopped
+        else
+          # Processing has come to a natural end:
+          log('Done, exiting')
+          return :exiting
+        end
+      end
+
+      def log(message, level = :info)
+        tags    = "[#{Time.current.to_s(:db)}] [#{level.upcase}] [daemon: #{name} (#{Process.pid})]"
+        message = "#{tags} #{message}"
+
+        $stdout.puts(message) unless Rails.env.test? || !$stdout.isatty
+        Rails.logger.send(level, message)
+      end
+
+      private
+
+      def setup_signals
+        Signal.trap('TERM') { stop }
+      end
+
+      def restart_file_touched?
+        File.exist?(RESTART_FILENAME) && File.mtime(RESTART_FILENAME) > start_time
+      end
+
+      def excessive_memory?
+        (`ps -o rss= -p #{$$}`.to_i.kilobytes) > MAX_MEMORY
+      rescue
+        false
+      end
+
+      def been_up_a_while?
+        start_time < MAX_UPTIME.ago
+      end
+
+      # sleeps for `duration`, but wakes up periodically to
+      # see if the daemon has been asked to restart. If so,
+      # returns immediately.
+      def snooze(duration)
+        number_of_mini_sleeps = duration / LITTLE_SLEEP
+        initial_sleep_length  = duration % LITTLE_SLEEP
+
+        sleep(initial_sleep_length)
+
+        number_of_mini_sleeps.times do
+          return if should_stop?
+          sleep(LITTLE_SLEEP)
+        end
+      end
+    end
+  end
+end