ndr_dev_support 5.5.0 → 5.8.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 266a4e4e0fe8d5aaf551ce205b4d699aec39922d634c34a922cdf825b2e38f0d
-  data.tar.gz: 1f07112de5ce4ecb8e18aa1b9a2013bd71d6b99e4442aeb27b8a239928242d0b
+  metadata.gz: 37a848fce50262a1d6c4b211e1e638317723bc9e45bbd4f8561bce0ce9e4ff1b
+  data.tar.gz: 2cc041c4aa592b91c8daea31b946f2909dcf46292d966e6a82e57cdf0b177403
 SHA512:
-  metadata.gz: d458bf7e248657f21ae473e6809288f91fd0b71dfa660b8706a199c3f4e74f06a02f850d5a3b3008ec5c7ae03545a3720a3a0917ca84f6c69793c629fe1a65bb
-  data.tar.gz: b6d391962ebe0e2c5522c4e60186f8a3c0dce42df2e11c8534db28f8e6838d1390c0910adf67007ffc419dfbbe8ee2f346e02d8c77b49366b950661d571659bb
+  metadata.gz: be1ff5ae22a75022f0cabebb425e7d6dbfd34cf75e61856eaf40920446d38af62d9d39eba689f5aaa0cd815f11d42385842c97a6160c272f6b9cb7850921fe3b
+  data.tar.gz: f1a8251774d6917daf5d86ad22aa63f6afe2139ad4a106389a842dcd105fb0fe946ae490e0929fd73de4a442be9bac414150c49f1420cfbeeb947b60d6e14556

data/CHANGELOG.md

@@ -1,7 +1,35 @@
 ## [Unreleased]
 *no unreleased changes*
 
+## 5.8.1 / 2020-07-10
+### Fixed
+* Fix issue running `brakeman:fingerprint_details` task
+
+## 5.8.0 / 2020-04-07
+### Added
+* Ability to select git CI branch by exporting `RAKE_CI_BRANCH_NAME`
+
+## 5.7.1 / 2020-04-07
+### Fixed
+* Address issue updating redmine using commit message tags
+
+## 5.7.0 / 2020-03-26
+### Added
+* Add `ci:brakeman:strict` alternative CI task. (#77)
+* Send `brakeman_filtered_warnings` metrics. (#78)
+* Allow redmine tickets to be updated (but not resolved) when the build fails (#73)
+
+### Fixed
+* Stop including asset compilation caches in the deployment archive.
+* Ensure brakeman alerts aren't sent to Slack if they've been reviewed and filtered out
+
+## 5.6.0 / 2020-02-14
+### Added
+* Add `flakey_test` to the minitest DSL, to allow sporadic failures to be retried
+* CI: include minitest seed in slack output
+
 ## 5.5.0 / 2020-01-27
+### Added
 * bundle master RuboCop config, and allow it to be `required`
 
 ## 5.4.8 / 2020-01-24

data/README.md

@@ -1,4 +1,4 @@
-## NdrDevSupport [![Build Status](https://travis-ci.org/PublicHealthEngland/ndr_dev_support.svg?branch=master)](https://travis-ci.org/PublicHealthEngland/ndr_dev_support) [![Gem Version](https://badge.fury.io/rb/ndr_dev_support.svg)](https://badge.fury.io/rb/ndr_dev_support)
+## NdrDevSupport [![Maintainability](https://api.codeclimate.com/v1/badges/2b2a644964f2aa930f81/maintainability)](https://codeclimate.com/github/PublicHealthEngland/ndr_dev_support/maintainability) [![Build Status](https://travis-ci.org/PublicHealthEngland/ndr_dev_support.svg?branch=master)](https://travis-ci.org/PublicHealthEngland/ndr_dev_support) [![Gem Version](https://badge.fury.io/rb/ndr_dev_support.svg)](https://badge.fury.io/rb/ndr_dev_support)
 
 This is the Public Health England (PHE) National Disease Registers (NDR) Developer Support ruby gem,
 providing:
@@ -28,7 +28,7 @@ Or install it yourself as:
 To add development support tasks (see below) to your project, add this line to your application's `Rakefile`:
 
 ```ruby
-require 'ndr_dev_support/tasks'
+require 'ndr_dev_support/tasks' if Rails.env.development? || Rails.env.test?
 ```
 
 ## Usage
@@ -135,6 +135,27 @@ Beyond standard Capybara testing DSL, ndr_dev_support bundles some additional fu
 
 When using a headless browser for integration tests, the test database must be consistent between the test runner and the application being tested. With transactional tests in operation, this means that both must share a connection. It is up to the individual project to provide this facility; as of Rails 5.1, it is built in to the framework directly.
 
+#### Flakey Tests
+
+It is an unfortunate reality that sometimes tests are written that will fail sporadically. Whilst in such cases the test methodology should be addressed, investigations can be time-consuming.
+Therefore, `ndr_dev_support` grudgingly provides "flakey test" support, to minimise disruption to CI pipelines whilst root causes are investigated.
+
+```ruby
+test 'thing that always passes' do
+  # reliable test
+end
+
+flakey_test 'thing that occassionally needs a second or third attempt' do
+  # less reliable test
+end
+
+flakey_test 'thing that often needs multiple attempts', attempts: 10 do
+  # really unreliable test (gulp...)
+end
+```
+
+If tests still fail, they'll fail as normal. If tests pass after flakey failure, they'll be flagged to the RakeCI server, and rendered in purple on Slack.
+
 ### Deployment support
 
 There are various capistrano plugins in the `ndr_dev_support/capistrano` directory - see each one for details.

data/code_safety.yml

@@ -19,7 +19,7 @@ file safety:
   CHANGELOG.md:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: aa0d11190daa8842da406b933d61c4043823ef9a
+    safe_revision: b32e7980e8854b3021e513b2803f7424e78e28d9
   CODE_OF_CONDUCT.md:
     comments: 
     reviewed_by: timgentry
@@ -35,7 +35,7 @@ file safety:
   README.md:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 2cf2b6e3ce2f054289699527c13cd7d795347ce3
+    safe_revision: e389930b98976f20e3fc94c771e5c456b729f8cb
   Rakefile:
     comments: 
     reviewed_by: josh.pencheon
@@ -67,7 +67,7 @@ file safety:
   lib/minitest/rake_ci_plugin.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 26efe45f35f211e7cb7a04967c51c0b6d116321e
+    safe_revision: 958ef09c5cf50bb50348e6bff949e3e5989254f9
   lib/ndr_dev_support.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -99,7 +99,7 @@ file safety:
   lib/ndr_dev_support/capistrano/svn_cache.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 123d0cd32bdf9e0940fac8945985aa9f467616c1
+    safe_revision: 2baa360b06290775a961a7573fe08edd18f8312e
   lib/ndr_dev_support/capistrano/sysadmin_scripts.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -107,7 +107,7 @@ file safety:
   lib/ndr_dev_support/daemon/ci_server.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 5075331fb3f33cd214c2ca76b199f27ddd743170
+    safe_revision: 3fdf010a91bd9927ef34e3df66b8a4bbbd20315a
   lib/ndr_dev_support/daemon/stoppable.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -115,7 +115,7 @@ file safety:
   lib/ndr_dev_support/integration_testing.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 1ecf54219c77969066a142a42b694e805d69f8b4
+    safe_revision: a2f178853da640112cf3063ca1d640157c9edb9f
   lib/ndr_dev_support/integration_testing/drivers/chrome.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -140,10 +140,14 @@ file safety:
     comments: 
     reviewed_by: josh.pencheon
     safe_revision: f1a32b1f2d1851b87a883dbf8620aa0e921e436c
+  lib/ndr_dev_support/integration_testing/flakey_tests.rb:
+    comments: 
+    reviewed_by: josh.pencheon
+    safe_revision: a2f178853da640112cf3063ca1d640157c9edb9f
   lib/ndr_dev_support/rake_ci/brakeman_helper.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: d3d238a863c0b016d2d782b1696964fa3f40738b
+    safe_revision: 053c9834ca5d402a1f8dc8d09257dc7075a5ec06
   lib/ndr_dev_support/rake_ci/commit_cop.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -175,7 +179,7 @@ file safety:
   lib/ndr_dev_support/rake_ci/redmine/ticket_resolver.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 2154aa7f32e731933ff6091b8f42b2b014028a6a
+    safe_revision: 788b8fc31952a6fb640101df96a370b21d269249
   lib/ndr_dev_support/rake_ci/simple_cov_helper.rb:
     comments: 
     reviewed_by: timgentry
@@ -211,7 +215,7 @@ file safety:
   lib/ndr_dev_support/version.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: aa0d11190daa8842da406b933d61c4043823ef9a
+    safe_revision: b32e7980e8854b3021e513b2803f7424e78e28d9
   lib/tasks/audit_code.rake:
     comments: Identical to the version reviewed by josh.pencheon when contained within
       ndr_support
@@ -220,7 +224,7 @@ file safety:
   lib/tasks/ci/brakeman.rake:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 908c04663ca6949259f41fe38ee705f03774cbdf
+    safe_revision: fe7dfc7f1775e8b85ae6968e8f475b9e930addf7
   lib/tasks/ci/bundle_audit.rake:
     comments: 
     reviewed_by: josh.pencheon
@@ -244,7 +248,7 @@ file safety:
   lib/tasks/ci/minitest.rake:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 29b7dc86673f4f876667db95e79a57b20766077e
+    safe_revision: 12aad1254186ee9ce0fde263bf8dccf46cdfebe2
   lib/tasks/ci/notes.rake:
     comments: 
     reviewed_by: timgentry
@@ -256,7 +260,7 @@ file safety:
   lib/tasks/ci/redmine.rake:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 3c24b66ea4beb348de34330984302f1fe25faefd
+    safe_revision: 12aad1254186ee9ce0fde263bf8dccf46cdfebe2
   lib/tasks/ci/rugged.rake:
     comments: 
     reviewed_by: josh.pencheon
@@ -284,7 +288,7 @@ file safety:
   ndr_dev_support.gemspec:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 83097be1002f924a37fcbd6d9099e08e9ed65621
+    safe_revision: f4c1ea57d3eb817783fdc47a16169d215f9788a6
   test/daemon/ci_server_test.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -312,7 +316,7 @@ file safety:
   test/rake_ci/redmine/ticket_resolver_test.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 2154aa7f32e731933ff6091b8f42b2b014028a6a
+    safe_revision: 12aad1254186ee9ce0fde263bf8dccf46cdfebe2
   test/test_helper.rb:
     comments: 
     reviewed_by: josh.pencheon

data/lib/minitest/rake_ci_plugin.rb

@@ -5,12 +5,35 @@ require 'ndr_dev_support/rake_ci/concerns/commit_metadata_persistable'
 
 # The plugin needs to extend Minitest
 module Minitest
-  def self.plugin_rake_ci_init(_options)
-    reporter << RakeCIReporter.new if RakeCIReporter.enabled?
+  def self.plugin_rake_ci_init(options)
+    reporter << RakeCIReporter.new(options[:io], options) if RakeCIReporter.enabled?
+  end
+
+  # Intermediate Reporter than can also track flakey failures
+  class FlakeyStatisticsReporter < StatisticsReporter
+    attr_accessor :flakey_results
+
+    def initialize(*)
+      super
+
+      self.flakey_results = []
+    end
+
+    def record(result)
+      super
+
+      return unless result.respond_to?(:flakes)
+
+      flakey_results << result if result.flakes.any?
+    end
+
+    def flakes
+      flakey_results.sum { |result| result.flakes.length }
+    end
   end
 
   # RakeCI Minitest Reporter
-  class RakeCIReporter < StatisticsReporter
+  class RakeCIReporter < FlakeyStatisticsReporter
     def self.enable!
       @enabled = true
     end
@@ -55,6 +78,10 @@ module Minitest
       snippets_for results.reject(&:skipped?).reject(&:error?)
     end
 
+    def flake_snippets
+      snippets_for flakey_results
+    end
+
     # Adapted from Rails' TestUnit reporter
     def snippets_for(results, limit = 5)
       executable = defined?(Rails) ? 'bin/rails test ' : 'bundle exec rake test TEST='
@@ -78,7 +105,7 @@ module Minitest
     def current_statistics
       @current_statistics ||= {
         total_time: total_time, runs: count, assertions: assertions, failures: failures,
-        errors: errors, skips: skips
+        errors: errors, skips: skips, flakes: flakes
       }
     end
 
@@ -106,6 +133,7 @@ module Minitest
       @current_attachments << failures_attachment if failures.positive?
       @current_attachments << errors_attachment if errors.positive?
       @current_attachments << pass_attachment if passing?
+      @current_attachments << flakes_attachment if flakes.positive?
       @current_attachments
     end
 
@@ -113,7 +141,15 @@ module Minitest
       {
         color: 'danger',
         text: 'test failure'.pluralize(failures) + failure_snippets,
-        footer: 'bundle exec rake ci:minitest'
+        footer: footer
+      }
+    end
+
+    def flakes_attachment
+      {
+        color: '#bb44ff',
+        text: 'flakey test'.pluralize(flakes) + flake_snippets,
+        footer: footer
       }
     end
 
@@ -121,7 +157,7 @@ module Minitest
       {
         color: 'warning',
         text: 'test error'.pluralize(errors) + error_snippets,
-        footer: 'bundle exec rake ci:minitest'
+        footer: footer
       }
     end
 
@@ -129,7 +165,7 @@ module Minitest
       {
         color: 'good',
         text: newly_passing? ? 'Tests now pass! :tada:' : 'Tests passed',
-        footer: 'bundle exec rake ci:minitest'
+        footer: footer
       }
     end
 
@@ -150,5 +186,9 @@ module Minitest
     def name
       'minitest'
     end
+
+    def footer
+      "bundle exec rake ci:minitest --seed #{options[:seed]}"
+    end
   end
 end

data/lib/ndr_dev_support/capistrano/svn_cache.rb

@@ -17,7 +17,13 @@ Capistrano::Configuration.instance(:must_exist).load do
         system("rm -rf #{cache}")
       end
     end
+
+    desc 'Ensures compilation artefacts are removed from the compressed archive sent to the server'
+    task :augment_copy_exclude do
+      set :copy_exclude, (fetch(:copy_exclude) || []) + %w[node_modules tmp/*]
+    end
   end
 
+  before 'deploy:update_code', 'ndr_dev_support:augment_copy_exclude'
   before 'deploy:update_code', 'ndr_dev_support:remove_svn_cache_if_needed'
 end

data/lib/ndr_dev_support/daemon/ci_server.rb

@@ -48,7 +48,7 @@ module NdrDevSupport
 
         git_fetch
         git_discard_changes
-        git_checkout(MASTER_BRANCH_NAME)
+        git_checkout(git_branch_name)
 
         objectids_between_master_and_remote.each do |oid|
           log("testing #{oid}...")
@@ -72,6 +72,10 @@ module NdrDevSupport
         raise exception
       end
 
+      def git_branch_name
+        ENV.fetch('RAKE_CI_BRANCH_NAME', MASTER_BRANCH_NAME)
+      end
+
       def git_fetch
         system(svn_remote? ? 'git svn fetch' : 'git fetch')
       end
@@ -117,7 +121,7 @@ module NdrDevSupport
       def objectids_between_master_and_remote
         walker = Rugged::Walker.new(@repo)
         walker.push(repo.branches[remote_branch].target_id)
-        current_target_id = repo.branches[MASTER_BRANCH_NAME].target_id
+        current_target_id = repo.branches[git_branch_name].target_id
 
         revisions = []
         # walk backwards from the most recent commit, breaking at the current one

data/lib/ndr_dev_support/integration_testing.rb

@@ -18,6 +18,9 @@ end
 # Include our custom DSL extensions, that also cover screenshotting:
 require 'ndr_dev_support/integration_testing/dsl'
 
+# Include support for retrying tests that sporadically fail:
+require 'ndr_dev_support/integration_testing/flakey_tests'
+
 # Keeps the selenium webdrivers automatically updated:
 require 'webdrivers'
 Webdrivers.cache_time = 24.hours

data/lib/ndr_dev_support/integration_testing/flakey_tests.rb

@@ -0,0 +1,55 @@
+module NdrDevSupport
+  module IntegrationTesting
+    # Grudging handling of flakey integration tests. Allows tests to be declared
+    # with `flakey_test`. Our CI reporter gathers information on flakey failures.
+    module FlakeyTests
+      extend ActiveSupport::Concern
+
+      included do
+        class_attribute :attempts_per_test, default: {}
+      end
+
+      class_methods do
+        def flakey_test(description, attempts: 3, &block)
+          test(description, &block).tap do |test_name|
+            self.attempts_per_test = attempts_per_test.merge(test_name.to_s => attempts)
+          end
+        end
+      end
+
+      def flakes
+        @flakes ||= []
+      end
+
+      def run
+        attempts_remaining = attempts_per_test[name]
+        return super unless attempts_remaining
+
+        previous_failure = failures.last
+        failed_attempts = []
+
+        loop do
+          break if attempts_remaining < 1
+
+          super
+
+          # No failure was added; we passed!
+          break if failures.last == previous_failure
+
+          # Ran out of attempts:
+          break if (attempts_remaining -= 1) < 1
+
+          # Loop round and have another go:
+          failed_attempts << failures.pop
+        end
+
+        # Attempts were only flakey if we eventually passed:
+        flakes.concat(failed_attempts) if failures.last == previous_failure
+
+        self
+      end
+    end
+  end
+end
+
+ActionDispatch::IntegrationTest.include(NdrDevSupport::IntegrationTesting::FlakeyTests)

data/lib/ndr_dev_support/rake_ci/brakeman_helper.rb

@@ -10,7 +10,9 @@ module NdrDevSupport
 
       attr_reader :new_fingerprints, :old_fingerprints, :tracker
 
-      def run
+      def run(strict:)
+        @strict = strict
+
         @tracker = ::Brakeman.run(app_path: '.')
 
         last_commit_fingerprints = load_last_commit_data
@@ -22,10 +24,16 @@ module NdrDevSupport
         end
       end
 
+      # All warnings (including those we've flagged as false positives)
       def warnings
         @tracker.warnings
       end
 
+      # Only the warnings we haven't flagged as false positives (i.e. the outstanding ones)
+      def filtered_warnings
+        @tracker.filtered_warnings
+      end
+
       def warning_counts_by_confidence
         return @warning_counts_by_confidence if @warning_counts_by_confidence
 
@@ -36,13 +44,90 @@ module NdrDevSupport
         @warning_counts_by_confidence
       end
 
+      def filtered_warning_counts_by_confidence
+        return @filtered_warning_counts_by_confidence if @filtered_warning_counts_by_confidence
+
+        @filtered_warning_counts_by_confidence = {}
+        filtered_warnings.group_by(&:confidence).each do |confidence, grouped_warnings|
+          @filtered_warning_counts_by_confidence[confidence] = grouped_warnings.count
+        end
+        @filtered_warning_counts_by_confidence
+      end
+
       def current_fingerprints
-        @current_fingerprints ||= warnings.map(&:fingerprint).to_set
+        @current_fingerprints ||= filtered_warnings.map(&:fingerprint).to_set
       end
 
       def save_current_fingerprints
         save_current_commit_data(current_fingerprints)
       end
+
+      def metrics
+        metrics = []
+
+        ::Brakeman::Warning::TEXT_CONFIDENCE.each do |confidence, text|
+          overall_metric = {
+            name: 'brakeman_warnings',
+            type: :gauge,
+            label_set: { confidence: text },
+            value: warning_counts_by_confidence[confidence] || 0
+          }
+          filtered_metric = {
+            name: 'brakeman_filtered_warnings',
+            type: :gauge,
+            label_set: { confidence: text },
+            value: filtered_warning_counts_by_confidence[confidence] || 0
+          }
+          metrics << overall_metric << filtered_metric
+          puts overall_metric.inspect
+          puts filtered_metric.inspect
+        end
+
+        metrics
+      end
+
+      def attachments
+        attachments = []
+
+        if @strict && current_fingerprints.any?
+          # all warnings found
+          attachment = {
+            color: 'danger',
+            title: "#{current_fingerprints.size} Brakeman warning(s) :rotating_light:",
+            text: '_Brakeman_ warning fingerprint(s):' \
+            "```#{current_fingerprints.to_a.join("\n")}```",
+            footer: 'bundle exec rake ci:brakeman:fingerprint_details FINGERPRINTS=...',
+            mrkdwn_in: ['text']
+          }
+          attachments << attachment
+          puts attachment.inspect
+        elsif new_fingerprints.any?
+          # new warnings found
+          attachment = {
+            color: 'danger',
+            title: "#{new_fingerprints.size} new Brakeman warning(s) :rotating_light:",
+            text: '_Brakeman_ warning fingerprint(s):' \
+            "```#{new_fingerprints.to_a.join("\n")}```",
+            footer: 'bundle exec rake ci:brakeman:fingerprint_details FINGERPRINTS=...',
+            mrkdwn_in: ['text']
+          }
+          attachments << attachment
+          puts attachment.inspect
+        end
+
+        unless old_fingerprints.empty?
+          # old warnings missing
+          attachment = {
+            color: 'good',
+            title: "#{old_fingerprints.size} Brakeman warning(s) resolved :+1:",
+            footer: 'bundle exec rake ci:brakeman'
+          }
+          attachments << attachment
+          puts attachment.inspect
+        end
+
+        attachments
+      end
     end
   end
 end

data/lib/ndr_dev_support/rake_ci/redmine/ticket_resolver.rb

@@ -36,13 +36,13 @@ module NdrDevSupport
           @hostname = hostname
         end
 
-        def process_commit(user, revision, message)
+        def process_commit(user, revision, message, tests_passed)
           resolved_tickets = []
 
           each_ticket_from(message) do |ticket, resolved|
-            update_ticket(message, user, revision, ticket, resolved)
+            update_ticket(message, user, revision, ticket, resolved, tests_passed)
 
-            resolved_tickets << ticket if resolved
+            resolved_tickets << ticket if resolved && tests_passed
           end
 
           resolved_tickets
@@ -62,13 +62,17 @@ module NdrDevSupport
           end
         end
 
-        def update_payload(message, user, revision, ticket_closed, resolved)
+        def update_payload(message, user, revision, ticket_closed, resolved, tests_passed)
+          if resolved && !ticket_closed && !tests_passed
+            message += "\n\n*Automated tests did not pass successfully, so ticket status left unchanged.*"
+          end
+
           payload = {
             notes: "_#{resolved ? 'Resolved' : 'Referenced'} by #{user} in #{revision}_:" \
                    "#{resolved ? message.gsub(CLOSE_REGEX, '+\1+') : message}"
           }
 
-          payload[:status_id] = 3 if resolved && !ticket_closed
+          payload[:status_id] = 3 if resolved && !ticket_closed && tests_passed
           payload
         end
 
@@ -82,8 +86,8 @@ module NdrDevSupport
           @http
         end
 
-        def update_ticket(message, user, revision, ticket, resolved)
-          payload = update_payload(message, user, revision, ticket_closed?(ticket), resolved)
+        def update_ticket(message, user, revision, ticket, resolved, tests_passed)
+          payload = update_payload(message, user, revision, ticket_closed?(ticket), resolved, tests_passed)
 
           http.send_request('PUT',
                             "/issues/#{ticket.to_i}.json",

data/lib/ndr_dev_support/version.rb

@@ -2,5 +2,5 @@
 # This defines the NdrDevSupport version. If you change it, rebuild and commit the gem.
 # Use "rake build" to build the gem, see rake -T for all bundler rake tasks (and our own).
 module NdrDevSupport
-  VERSION = '5.5.0'.freeze
+  VERSION = '5.8.1'.freeze
 end

data/lib/tasks/ci/brakeman.rake

@@ -11,48 +11,35 @@ namespace :ci do
 
     brakeman = NdrDevSupport::RakeCI::BrakemanHelper.new
     brakeman.commit = @commit
-    brakeman.run
-
-    Brakeman::Warning::TEXT_CONFIDENCE.each do |confidence, text|
-      metric = {
-        name: 'brakeman_warnings',
-        type: :gauge,
-        label_set: { confidence: text },
-        value: brakeman.warning_counts_by_confidence[confidence] || 0
-      }
-      @metrics << metric
-      puts metric.inspect
-    end
-
-    unless brakeman.new_fingerprints.empty?
-      # new warnings found
-      attachment = {
-        color: 'danger',
-        title: "#{brakeman.new_fingerprints.size} new Brakeman warning(s) :rotating_light:",
-        text: '_Brakeman_ warning fingerprint(s):' \
-              "```#{brakeman.new_fingerprints.to_a.join("\n")}```",
-        footer: 'bundle exec rake ci:brakeman:fingerprint_details FINGERPRINTS=...',
-        mrkdwn_in: ['text']
-      }
-      @attachments << attachment
-      puts attachment.inspect
-    end
+    brakeman.run(strict: false)
 
-    unless brakeman.old_fingerprints.empty?
-      # old warnings missing
-      attachment = {
-        color: 'good',
-        title: "#{brakeman.old_fingerprints.size} Brakeman warning(s) resolved :+1:",
-        footer: 'bundle exec rake ci:brakeman'
-      }
-      @attachments << attachment
-      puts attachment.inspect
-    end
+    @metrics.concat(brakeman.metrics)
+    @attachments.concat(brakeman.attachments)
 
     brakeman.save_current_fingerprints
   end
 
   namespace :brakeman do
+    desc "Brakeman (strict mode - all issues must be reviewed by Brakeman's interactive mode)"
+    task strict: 'ci:rugged:setup' do
+      next unless defined?(Rails)
+
+      require 'ndr_dev_support/rake_ci/brakeman_helper'
+      # Usage: bundle exec rake ci:brakeman:strict
+
+      @metrics ||= []
+      @attachments ||= []
+
+      brakeman = NdrDevSupport::RakeCI::BrakemanHelper.new
+      brakeman.commit = @commit
+      brakeman.run(strict: true)
+
+      @metrics.concat(brakeman.metrics)
+      @attachments.concat(brakeman.attachments)
+
+      brakeman.save_current_fingerprints
+    end
+
     desc 'Brakeman fingerprint details'
     task fingerprint_details: 'ci:rugged:setup' do
       # Usage: bundle exec rake ci:brakeman:fingerprint_details FINGERPRINTS=fp1,fp2,...
@@ -70,7 +57,7 @@ namespace :ci do
 
       brakeman = NdrDevSupport::RakeCI::BrakemanHelper.new
       brakeman.commit = @commit
-      brakeman.run
+      brakeman.run(strict: false)
 
       text_reporter = Brakeman::Report::Text.new(brakeman.tracker)
 

data/lib/tasks/ci/minitest.rake

@@ -47,10 +47,9 @@ namespace :ci do
       # Test(s) ran
       Rake::Task['ci:simplecov:process'].invoke
 
-      if hash[:statistics][:failures].zero? && hash[:statistics][:errors].zero? &&
-         Rake::Task.task_defined?('ci:redmine:update_tickets')
-        # Test(s) passing
-        Rake::Task['ci:redmine:update_tickets'].invoke
+      if Rake::Task.task_defined?('ci:redmine:update_tickets')
+        tests_passed = hash[:statistics][:failures].zero? && hash[:statistics][:errors].zero?
+        Rake::Task['ci:redmine:update_tickets'].invoke(tests_passed)
       end
 
       @attachments.concat(hash[:attachments])

data/lib/tasks/ci/redmine.rake

@@ -12,7 +12,7 @@ namespace :ci do
     end
 
     desc 'Update Redmine tickets'
-    task update_tickets: ['ci:rugged:setup', 'ci:redmine:setup'] do
+    task :update_tickets, [:tests_passed] => ['ci:rugged:setup', 'ci:redmine:setup'] do |task, args|
       api_key = ENV['REDMINE_API_KEY']
       hostname = ENV['REDMINE_HOSTNAME']
       next if api_key.nil? || hostname.nil?
@@ -27,7 +27,7 @@ namespace :ci do
         ticket_resolver = NdrDevSupport::RakeCI::Redmine::TicketResolver.new(api_key, hostname)
         resolved_tickets = ticket_resolver.process_commit(@commit.author[:name],
                                                           @friendly_revision_name,
-                                                          @commit.message)
+                                                          @commit.message, args.tests_passed)
       rescue
         @attachments << {
           color: 'danger',
@@ -41,7 +41,8 @@ namespace :ci do
       resolved_tickets.map! { |ticket_id| "https://#{hostname}/issues/#{ticket_id}" }
 
       old_attachment = @attachments.detect { |att| att[:text] =~ /Tests( now)? pass/ }
-      basic_text     = old_attachment.try(:[], :text) || 'Tests passed'
+      basic_text = old_attachment.try(:[], :text) || 'Tests passed'
+      footer = old_attachment.try(:[], :footer) || 'bundle exec rake ci:minitest'
 
       @attachments.delete(old_attachment) if old_attachment
 
@@ -51,7 +52,7 @@ namespace :ci do
         title: "#{issue_s.capitalize} Resolved",
         text: "#{basic_text}, so #{issue_s} #{resolved_tickets.join(', ')}" \
               " #{resolved_tickets.count == 1 ? 'has' : 'have'} been resolved",
-        footer: 'bundle exec rake ci:minitest',
+        footer: footer,
         mrkdwn_in: ['text']
       }
       @attachments << attachment

data/ndr_dev_support.gemspec

@@ -36,6 +36,7 @@ Gem::Specification.new do |spec|
   # Integration test dependencies:
   spec.add_dependency 'capybara', '>= 3.20'
   spec.add_dependency 'capybara-screenshot'
+  spec.add_dependency 'minitest', '~> 5.0'
   spec.add_dependency 'poltergeist', '>= 1.8.0'
   spec.add_dependency 'selenium-webdriver'
   spec.add_dependency 'show_me_the_cookies'
@@ -55,7 +56,6 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'capistrano', '~> 2.15'
 
   spec.add_development_dependency 'bundler'
-  spec.add_development_dependency 'minitest', '~> 5.0'
   spec.add_development_dependency 'mocha'
-  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rake', '~> 12.3', '>= 12.3.3'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ndr_dev_support
 version: !ruby/object:Gem::Version
-  version: 5.5.0
+  version: 5.8.1
 platform: ruby
 authors:
 - NCRS Development Team
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-01-27 00:00:00.000000000 Z
+date: 2020-07-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: pry
@@ -136,6 +136,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
 - !ruby/object:Gem::Dependency
   name: poltergeist
   requirement: !ruby/object:Gem::Requirement
@@ -332,20 +346,6 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: minitest
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '5.0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '5.0'
 - !ruby/object:Gem::Dependency
   name: mocha
   requirement: !ruby/object:Gem::Requirement
@@ -366,14 +366,20 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: '12.3'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 12.3.3
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '10.0'
+        version: '12.3'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 12.3.3
 description: Provides support to developers of NDR projects
 email: []
 executables: []
@@ -415,6 +421,7 @@ files:
 - lib/ndr_dev_support/integration_testing/drivers/poltergeist.rb
 - lib/ndr_dev_support/integration_testing/drivers/switchable.rb
 - lib/ndr_dev_support/integration_testing/dsl.rb
+- lib/ndr_dev_support/integration_testing/flakey_tests.rb
 - lib/ndr_dev_support/rake_ci/brakeman_helper.rb
 - lib/ndr_dev_support/rake_ci/commit_cop.rb
 - lib/ndr_dev_support/rake_ci/commit_cop/concerns/deputisable.rb