ndr_dev_support 4.2.0 → 4.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 87c153a7110ac07b2d6a99db00e1767cb99c6cbe0ff24e0a9a3ae02a6a0e8a49
-  data.tar.gz: 81338a061ce5554213835bde60ce17a7a76603c356efcac1d2d5dce92a34f6d1
+  metadata.gz: 07b63ea8d81ee459134b9050fe9de402b50f79b0ad32139d73d90e6e26b7053c
+  data.tar.gz: 3479a5ccbd481bae7528036b8ba0cf20aba3676c99ca0f4c4b68f20920df35d0
 SHA512:
-  metadata.gz: 7fc483a76029f95b49ba8718be0346616296b9b636e4ff693fda738e54803fce8d67d28d7d4b324b04340581c8e8c134566227e9a7983d8412b43a3aab604456
-  data.tar.gz: b44a5b0eaa442d0ff383e90c8418b17a9014c113040e7c93d8e02e4e0b768b473fcedd2dabd5d35bc5029b7bfcc7ff67c5c058591e6fe5f2b6f4d5c4a388beac
+  metadata.gz: feee00d50b38a0e07f7f2817b580d9236b177a85ff949f18662e5dcd90488b8a85b8d3851655a922356f35f8e78f0fcab0aeea0a9d9a8c99c9cc89c7d2b71e51
+  data.tar.gz: 1f12d9c909139011e27e6c185289d97189d63c7bf8949e09b49767aeee3edef302f33385528a8a301775361799308fcc73d7d7b0193015d493b5b2eae2dde40c

data/CHANGELOG.md

@@ -1,6 +1,10 @@
 ## [Unreleased]
 *no unreleased changes*
 
+## 4.2.1 / 2018-12-18
+### Fixed
+* ci_server: improve logging and error handling (#44)
+
 ## 4.2.0 / 2018-11-28
 ### Added
 * Improve `Stoppable` integration within a Rails project, adding configurable logger.

data/README.md

@@ -7,6 +7,7 @@ providing:
 2. rake tasks to limit Rubocop's output to changed (and related) code
 3. integration testing support, which can be required from a project's `test_helper.rb`
 4. Deployment support, through Capistrano.
+5. a rake task based Continuous Integration (CI) server.
 
 ## Installation
 
@@ -140,6 +141,71 @@ require 'ndr_dev_support/capistrano/ndr_model'
 
 This will pull in the majority of behaviour needed to deploy in our preferred style.
 
+## Rake CI server
+
+ndr_dev_support provides a rake based continuous integration server that runs on a `git` or `git svn` working copy of your application.
+It polls for changes to the respository and, unlike some CI servers, it checks out and tests every commit; enabling full and comparative analysis of code quality and other statistical trends.
+
+Out of the box it does nothing, but does provide a number of rake tasks that you can opt to use.
+Those rake tasks utilise the concepts of metrics and attachments (messages) and tasks tend to either generate them or publish them.
+
+NOTE: As the way tests are run across applications differs, the `:default` rake task must be able to run your full suite of tests.
+
+CI rake tasks have been written for:
+
+* `ci:brakeman` - [brakeman](https://brakemanscanner.org/) vulnerability scanner metrics are generated for warning counts and "danger" messages for new warnings and "good" messages for fixed warnings.
+* `ci:bundle_audit` - generates "danger" messages for high criticality [bundle audit](https://github.com/rubysec/bundler-audit) advisories and "warning" messages for all others.
+* `ci:commit_cop` - Runs a number of commit "Cops" which create messages when common commit mistakes occur. Current cops look for a Rails migration added without a structure dump file, modified Rails migrations and renamed Rails migrations.
+* `ci:dependencies:process` - generates a line of pipe delimited markup showing system dependencies (that could be used in a wiki page on Redmine)
+* `ci:housekeep` - runs `rake log:clear` and `rake tmp:clear` if defined
+* `ci:linguist` - generates project programming language metrics for languages over 1% of codebase.
+* `ci:minitest` - sets up Minitest and SimpleCov to capture metrics and messages and runs the `default` rake task and `ci:simplecov:process` before running `ci:redmine:update_tickets` if all tests pass.
+* `ci:notes` - runs the Rails `rake notes` task (if using Rails) and converts annotation counts into metrics.
+* `ci:prometheus:publish` - sends all metrics to specified [Prometheus](https://prometheus.io/) push gateway.
+* `ci:redmine:update_tickets` - if all tests pass, this will parse the commit message and resolve associated [Redmine](https://www.redmine.org/) tickets.
+* `ci:rugged:commit_details` - if there are messages, then it prepends message list with commit details.
+* `ci:simplecov:process` - generates metrics for [SimpleCov](https://github.com/colszowka/simplecov) measured test covered lines, test coverage percentage and total lines of code.
+* `ci:slack:publish` - sends all messages to specified [Slack](https://slack.com/) channel.
+* `ci:stats` - runs the Rails `rake stats` task (if using Rails) and converts counts into metrics
+
+To start the server, `cd` to the working copy and execute:
+
+    $ rake ci:server
+
+Configuration is managed within your application by implementing the `ci:all` rake task. When a new commit is detected, it checks it out and runs `rake ci:all`.
+
+An example Rails application rake task might look like:
+
+```ruby
+namespace :ci do
+  desc 'Setup CI stack, integrations, etc up front'
+  task setup: [
+    'ci:rugged:setup',
+    'ci:slack:setup',
+    'ci:prometheus:setup'
+  ]
+
+  desc 'all'
+  task all: [
+    # Setup
+    'ci:setup',
+    'ci:housekeep',
+    'db:migrate',
+    # Test and Analyse
+    'ci:minitest',
+    'ci:brakeman',
+    'ci:bundle_audit',
+    'ci:linguist',
+    'ci:notes',
+    'ci:stats',
+    # Report
+    'ci:publish'
+  ]
+end
+```
+
+NOTE: Defining the `ci:setup` rake tasks up front is not necessary, but will prompt for missing server credentials at the start of the first CI run.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/code_safety.yml

@@ -19,7 +19,7 @@ file safety:
   CHANGELOG.md:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 261906b5303610ac97278d8e653d5394428625d1
+    safe_revision: a73717bdeb9f48c257aec4fa41a19735e99555c8
   CODE_OF_CONDUCT.md:
     comments: 
     reviewed_by: timgentry
@@ -35,7 +35,7 @@ file safety:
   README.md:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: a2b7c20eb58572c213f77677b36aa2f7e6db747e
+    safe_revision: dd220d1cd30d878bf8a35495efe474f08c5c0971
   Rakefile:
     comments: 
     reviewed_by: josh.pencheon
@@ -95,7 +95,7 @@ file safety:
   lib/ndr_dev_support/daemon/ci_server.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 66ce36e2646d392e3a67e19fcb9322ed8c5a6c16
+    safe_revision: 0677afc93b0a64d56383e80fa1247e5be64bfb24
   lib/ndr_dev_support/daemon/stoppable.rb:
     comments: 
     reviewed_by: josh.pencheon
@@ -191,7 +191,7 @@ file safety:
   lib/ndr_dev_support/version.rb:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 261906b5303610ac97278d8e653d5394428625d1
+    safe_revision: a73717bdeb9f48c257aec4fa41a19735e99555c8
   lib/tasks/audit_code.rake:
     comments: Identical to the version reviewed by josh.pencheon when contained within
       ndr_support
@@ -224,7 +224,7 @@ file safety:
   lib/tasks/ci/minitest.rake:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 3cadf2dc33034cfe7258f6cf4b199d3101b2a7ba
+    safe_revision: 0677afc93b0a64d56383e80fa1247e5be64bfb24
   lib/tasks/ci/notes.rake:
     comments: 
     reviewed_by: timgentry
@@ -236,7 +236,7 @@ file safety:
   lib/tasks/ci/redmine.rake:
     comments: 
     reviewed_by: josh.pencheon
-    safe_revision: 2154aa7f32e731933ff6091b8f42b2b014028a6a
+    safe_revision: ba171e9460b57d2d9932f6c0c0bdc3a5b282af80
   lib/tasks/ci/rugged.rake:
     comments: 
     reviewed_by: timgentry

data/lib/ndr_dev_support/daemon/ci_server.rb

@@ -44,39 +44,49 @@ module NdrDevSupport
       private
 
       def run_once
+        log('running once...')
+
         git_fetch
         git_checkout(MASTER_BRANCH_NAME)
 
         objectids_between_master_and_remote.each do |oid|
+          log("testing #{oid}...")
           git_rebase(oid)
 
           WithCleanRbenv.with_clean_rbenv do
             # TODO: rbenv_install
             bundle_install
-            `rbenv exec bundle exec rake ci:all`
+            system('rbenv exec bundle exec rake ci:all')
             git_discard_changes
           end
         end
+
+        log('completed single run.')
+      rescue => exception
+        log(<<~MSG)
+          Unhandled exception! #{exception.class}: #{exception.message}
+          #{(exception.backtrace || []).join("\n")}
+        MSG
+
+        raise exception
       end
 
       def git_fetch
-        svn_remote? ? `git svn fetch` : `git fetch`
+        system(svn_remote? ? 'git svn fetch' : 'git fetch')
       end
 
       def git_checkout(oid)
         Open3.popen3('git', 'checkout', oid) do |_stdin, _stdout, stderr, wait_thr|
           msg = stderr.read.strip
-          # TODO: Once https://github.com/PublicHealthEngland/ndr_dev_support/issues/27
-          # has been resolved, use logging instead of puts
-          puts msg unless msg == "Already on '#{oid}'"
+          log(msg) unless msg == "Already on '#{oid}'"
 
           process_status = wait_thr.value
-          raise stderr.read unless process_status.exited?
+          raise msg unless process_status.exited?
         end
       end
 
       def git_rebase(oid)
-        `git rebase #{Shellwords.escape(oid)}`
+        system("git rebase #{Shellwords.escape(oid)}") || raise('Unable to rebase!')
       end
 
       def git_discard_changes
@@ -122,7 +132,7 @@ module NdrDevSupport
         return unless File.file?('Gemfile')
         return if system('bundle check')
 
-        `rbenv exec bundle install --local --jobs=3`
+        system('rbenv exec bundle install --local --jobs=3')
         return if $CHILD_STATUS.exitstatus.zero? || ENV['SLACK_WEBHOOK_URL'].nil?
 
         slack_publisher = NdrDevSupport::SlackMessagePublisher.new(ENV['SLACK_WEBHOOK_URL'],
@@ -138,6 +148,8 @@ module NdrDevSupport
         }
 
         slack_publisher.post(attachments: [attachment])
+
+        raise 'Failure running `bundle install --local`'
       end
     end
   end

data/lib/ndr_dev_support/version.rb

@@ -2,5 +2,5 @@
 # This defines the NdrDevSupport version. If you change it, rebuild and commit the gem.
 # Use "rake build" to build the gem, see rake -T for all bundler rake tasks (and our own).
 module NdrDevSupport
-  VERSION = '4.2.0'.freeze
+  VERSION = '4.2.1'.freeze
 end

data/lib/tasks/ci/minitest.rake

@@ -5,7 +5,6 @@ namespace :ci do
 
     # Run the tests
     test_cmd = 'bundle exec rake ci:minitest:setup ci:simplecov:setup default'
-    test_cmd += ' 2>&1 >/dev/null' if ENV['RAKECI_HEADLESS']
     system test_cmd
 
     Rake::Task['ci:minitest:process'].invoke

data/lib/tasks/ci/redmine.rake

@@ -2,6 +2,8 @@ namespace :ci do
   namespace :redmine do
     desc 'Set up Redmine'
     task :setup do
+      require 'highline/import'
+
       ENV['REDMINE_HOSTNAME'] ||= ask('Redmine URL: ')
       ENV['REDMINE_HOSTNAME'] = nil if ENV['REDMINE_HOSTNAME'] == ''
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: ndr_dev_support
 version: !ruby/object:Gem::Version
-  version: 4.2.0
+  version: 4.2.1
 platform: ruby
 authors:
 - NCRS Development Team
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-11-28 00:00:00.000000000 Z
+date: 2018-12-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: pry