shipit-engine 0.20.1 → 0.21.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 32d85f27b707a0fd01fd1772e5803af8663d195b
-  data.tar.gz: 3f5df442e05fc1048307226080695c438fce46aa
+SHA256:
+  metadata.gz: f6940a4cd6a4608c60f6349020a01ef276049cf5aa78ff784abfe2cdf99a920f
+  data.tar.gz: f7056eb044a29a5646a349bcfe9fc8c6614fcbd7a199797275657c6c3598bb2c
 SHA512:
-  metadata.gz: d7a9dbec229ef543d4e4b884c75539abf59f143d67fd068cee822c609a4fcd5232dae511215d54674064a52a912645036ed7f4bdcfeb2263343a4e57d5559965
-  data.tar.gz: 3a02db736fb43949135fdada6d039c32f1a810204a20152c049c775d9bc17d5de526bf783bee98d9f2f892e753cf6bd329994881282344b40079329b6aaac809
+  metadata.gz: 4d61eb2c4fe287478d9ef7589b59ac268abbac149dc1783f15cf98d22230463aaeea4caf13060da7f0b2038742585e5df00cd29d05edd571a2326c030bbefbc0
+  data.tar.gz: 6d94a0ca00022f7727d7ecc9dd965f60ae9b92867c44434fa5057cd036a512ed04a685f0dd0e93a8886bd4df370a01682e6e2c213b5c8d65c1db4339efd79f2e

data/README.md

@@ -122,8 +122,9 @@ The settings in the `shipit.yml` file relate to the different things you can do
 * [Deployment](#deployment) (`deploy`, `rollback`, `fetch`)
 * [Environment](#environment) (`machine.environment`, `machine.directory`, `machine.cleanup`)
 * [CI](#ci) (`ci.require`, `ci.hide`, `ci.allow_failures`)
-* [Custom Tasks](#custom-tasks) (`restart`, `unlock`)
-* [Review Process](#review-process) (`monitor`, `checklist`, `checks`)
+* [Merge Queue](#merge-queue) (`merge.revalidate_after`, `merge.require`, `merge.ignore`, `merge.max_divergence`)
+* [Custom Tasks](#custom-tasks) (`tasks`)
+* [Review Process](#review-process) (`review.checklist`, `review.monitoring`, `review.checks`)
 
 All the settings in `shipit.yml` are optional. Most applications can be deployed from Shipit without any configuration.
 
@@ -269,7 +270,7 @@ deploy:
 ```
 <br>
 
-**<code>deploy.max_commits</code>** define the maximum number of commits that should be shipped per deploys. Defaults to `8`.
+**<code>deploy.max_commits</code>** defines the maximum number of commits that should be shipped per deploys. Defaults to `8`.
 
 Human users will be warned that they are not respecting the recommendation, but allowed to continue.
 However continuous delivery will respect this limit. If there is no deployable commits in this range, a human intervention will be required.
@@ -282,6 +283,15 @@ deploy:
 ```
 <br>
 
+**<code>deploy.interval</code>** defines the interval between the end of a deploy and the next deploy, when continuous delivery is enabled. You can use s, m, h, d as units for seconds, minutes, hours, and days. Defaults to 0, which means a new deploy will start as soon as the current one finishes.
+
+For example, this will wait 5 minutes after the end of a deploy before starting a new one:
+
+```yaml
+deploy:
+  interval: 5m
+```
+
 **<code>rollback.override</code>** contains an array of the shell commands required to rollback the application to a previous state. Shipit will try to infer it from the repository structure, but you can change the default inference. This key defaults to `disabled` unless Capistrano is detected.
 
 For example:
@@ -369,7 +379,7 @@ machine:
 
 <h3 id="ci">CI</h3>
 
-**<code>ci.require</code>** contains an array of the [statuses context](https://developer.github.com/v3/repos/statuses/) you want Shipit to disallow deploys if any of them is missing.
+**<code>ci.require</code>** contains an array of the [statuses context](https://developer.github.com/v3/repos/statuses/) you want Shipit to disallow deploys if any of them is missing on the commit being deployed.
 
 For example:
 ```yml
@@ -396,6 +406,15 @@ ci:
     - ci/circleci
 ```
 
+**<code>ci.blocking</code>** contains an array of the [statuses context](https://developer.github.com/v3/repos/statuses/) you want to disallow deploys if any of them is missing or failing on any of the commits being deployed.
+
+For example:
+```yml
+ci:
+  blocking:
+    - soc/compliance
+```
+
 <h3 id="merge-queue">Merge Queue</h3>
 
 The merge queue allow to register pull requests for them to be merged by Shipit once the stack is clear (no lock, no failing CI, no backlog). It can be enabled on a per stack basis via the settings page.
@@ -428,11 +447,29 @@ merge:
     - codeclimate
 ```
 
+**<code>merge.max_divergence.commits</code>** the maximum number of commits a pull request can be behind its merge base, after which pull requests are rejected from the merge queue.
+
+For example:
+```yml
+merge:
+  max_divergence:
+    commits: 50
+```
+
+**<code>merge.max_divergence.age</code>** a duration after the commit date of the merge base, after which pull requests will be rejected from the merge queue.
+
+For example:
+```yml
+merge:
+  max_divergence:
+    age: 72h
+```
+
 <h3 id="custom-tasks">Custom tasks</h3>
 
 You can create custom tasks that users execute directly from a stack's overview page in Shipit. To create a new custom task, specify its parameters in the `tasks` section of the `shipit.yml` file. For example:
 
-**<code>tasks</code>** restarts the application.
+**<code>tasks.restart</code>** restarts the application.
 
 ```yml
 tasks:
@@ -649,7 +686,7 @@ These variables are accessible only during deploys and rollback:
 
 ### Heroku
 
-To use Heroku integration (`lib/snippets/push-to-heroku`), make sure that the environment has [Heroku toolbelt](https://devcenter.heroku.com/articles/heroku-cli) available.
+To use Heroku integration (`lib/snippets/push-to-heroku`), make sure that the environment has [Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli) available.
 
 ### Kubernetes
 

data/app/assets/stylesheets/_base/_base.scss

@@ -106,6 +106,10 @@ a {
   text-decoration: none;
   cursor: pointer;
   color: $blue;
+
+  &.disabled {
+    cursor: default;
+  }
 }
 
 .more {

data/app/assets/stylesheets/_pages/_commits.scss

@@ -281,7 +281,9 @@
 }
 
 .status--error,
-[data-deploy-status='error'] {
+[data-deploy-status='error'],
+.status--timedout,
+[data-deploy-status='timedout'] {
   border-color: #333;
 
   .status__icon {

data/app/assets/stylesheets/_pages/_deploy.scss

@@ -93,7 +93,8 @@
   }
 
   &[data-status="failure"],
-  &[data-status="error"] {
+  &[data-status="error"],
+  &[data-status="timedout"] {
     .deploy-banner-status {
       background-color: $bright-red;
       width: 100%;
@@ -196,7 +197,8 @@
 
   &[data-status="failed"],
   &[data-status="failure"],
-  &[data-status="error"] {
+  &[data-status="error"],
+  &[data-status="timedout"] {
     border-color: $bright-red;
   }
 

data/app/controllers/concerns/shipit/authentication.rb

@@ -32,7 +32,7 @@ module Shipit
     end
 
     def find_current_user
-      session[:user_id].present? && User.find_by_id(session[:user_id])
+      session[:user_id].present? && User.find_by(id: session[:user_id])
     end
   end
 end

data/app/controllers/shipit/api/base_controller.rb

@@ -9,6 +9,7 @@ module Shipit
       rescue_from ApiClient::InsufficientPermission, with: :insufficient_permission
       rescue_from EnvironmentVariables::NotPermitted, with: :validation_error
       rescue_from TaskDefinition::NotFound, with: :not_found
+      rescue_from Task::ConcurrentTaskRunning, with: :conflict
 
       class << self
         def require_permission(operation, scope, options = {})
@@ -46,7 +47,7 @@ module Shipit
 
       def identify_user
         user_login = request.headers['X-Shipit-User'].presence
-        User.find_by(login: user_login) if user_login
+        User.where('lower(login) = ?', user_login.downcase).first if user_login
       end
 
       def stacks
@@ -72,6 +73,10 @@ module Shipit
       def not_found(_error)
         render status: :not_found, json: {status: '404', error: 'Not Found'}
       end
+
+      def conflict(_error)
+        render status: :conflict, json: {status: '409', error: 'Conflict'}
+      end
     end
   end
 end

data/app/controllers/shipit/api/pull_requests_controller.rb

@@ -9,7 +9,7 @@ module Shipit
       end
 
       def show
-        render_resource stack.pull_requests.find_by_number!(params[:id])
+        render_resource stack.pull_requests.find_by!(number: params[:id])
       end
 
       def update

data/app/controllers/shipit/commit_checks_controller.rb

@@ -23,7 +23,7 @@ module Shipit
     delegate :checks, to: :commit
 
     def commit
-      @commit ||= stack.commits.find_by_sha!(params[:sha])
+      @commit ||= stack.commits.find_by!(sha: params[:sha])
     end
 
     def stack

data/app/controllers/shipit/shipit_controller.rb

@@ -11,7 +11,7 @@ module Shipit
     helper Shipit::Engine.routes.url_helpers
     include Shipit::Engine.routes.url_helpers
 
-    before_action :toogle_bootstrap_feature, :ensure_required_settings
+    before_action :ensure_required_settings
 
     include Shipit::Authentication
 
@@ -24,10 +24,6 @@ module Shipit
 
     private
 
-    def toogle_bootstrap_feature
-      prepend_view_path(Shipit.bootstrap_view_path) if Shipit.feature_bootstrap?
-    end
-
     def ensure_required_settings
       return if Shipit.all_settings_present?
 

data/app/controllers/shipit/stacks_controller.rb

@@ -66,11 +66,13 @@ module Shipit
 
     def sync_webhooks
       @stack.setup_hooks
+      flash[:success] = 'Webhooks syncing scheduled'
       redirect_to stack_settings_path(@stack)
     end
 
     def clear_git_cache
       ClearGitCacheJob.perform_later(@stack)
+      flash[:success] = 'Git Cache clearing scheduled'
       redirect_to stack_settings_path(@stack)
     end
 

data/app/controllers/shipit/tasks_controller.rb

@@ -42,7 +42,7 @@ module Shipit
     end
 
     def abort
-      task.abort!(rollback_once_aborted: params[:rollback].present?)
+      task.abort!(rollback_once_aborted: params[:rollback].present?, aborted_by: current_user)
       head :ok
     end
 

data/app/controllers/shipit/webhooks_controller.rb

@@ -27,7 +27,7 @@ module Shipit
       end
     end
     def state
-      if commit = stack.commits.find_by_sha(params.sha)
+      if commit = stack.commits.find_by(sha: params.sha)
         commit.create_status_from_github!(params)
       end
       head :ok
@@ -88,7 +88,7 @@ module Shipit
       @webhook ||= if params[:stack_id]
         stack.github_hooks.find_by!(event: event)
       else
-        GithubHook::Organization.find_by!(organization: params.organization.login.downcase, event: event)
+        GithubHook::Organization.find_by!(organization: params.organization.login, event: event)
       end
     end
 

data/app/helpers/shipit/deploys_helper.rb

@@ -24,5 +24,14 @@ module Shipit
         content_tag(:span, "#{panel_spec.inspect} is not a valid monitoring panel spec")
       end
     end
+
+    def deploy_status_in_words(status)
+      case status
+      when 'timedout'
+        "Timed out"
+      else
+        status.capitalize
+      end
+    end
   end
 end

data/app/helpers/shipit/shipit_helper.rb

@@ -10,21 +10,23 @@ module Shipit
     end
 
     def emojify(content)
-      h(content).to_str.gsub(/:([\w+-]+):/) do |match|
-        if emoji = Emoji.find_by_alias($1)
-          %(
-            <img
-              alt="##{$1}"
-              src="#{image_path("emoji/#{emoji.image_filename}")}"
-              style="vertical-align:middle"
-              width="20"
-              height="20"
-            />
-          )
-        else
-          match
-        end
-      end.html_safe if content.present?
+      if content.present?
+        h(content).to_str.gsub(/:([\w+-]+):/) do |match|
+          if emoji = Emoji.find_by_alias($1)
+            %(
+              <img
+                alt="##{$1}"
+                src="#{image_path("emoji/#{emoji.image_filename}")}"
+                style="vertical-align:middle"
+                width="20"
+                height="20"
+              />
+            )
+          else
+            match
+          end
+        end.html_safe
+      end
     end
 
     def include_plugins(stack)

data/app/helpers/shipit/stacks_helper.rb

@@ -21,15 +21,20 @@ module Shipit
     def deploy_button(commit)
       url = new_stack_deploy_path(commit.stack, sha: commit.sha)
       classes = %W(btn btn--primary deploy-action #{commit.state})
+      deploy_state = commit.deploy_state(bypass_safeties?)
       data = {}
+
       if commit.deploy_disallowed?
         classes.push(bypass_safeties? ? 'btn--warning' : 'btn--disabled')
+        if deploy_state == 'blocked'
+          data[:tooltip] = t('deploy_button.hint.blocked')
+        end
       elsif commit.deploy_discouraged?
         classes.push('btn--warning')
         data[:tooltip] = t('deploy_button.hint.max_commits', maximum: commit.stack.maximum_commits_per_deploy)
       end
 
-      link_to(t("deploy_button.caption.#{commit.deploy_state(bypass_safeties?)}"), url, class: classes, data: data)
+      link_to(t("deploy_button.caption.#{deploy_state}"), url, class: classes, data: data)
     end
 
     def github_change_url(commit)

data/app/jobs/shipit/destroy_stack_job.rb

@@ -26,8 +26,10 @@ module Shipit
       Shipit::Hook.where(stack_id: stack.id).delete_all
       Shipit::PullRequest.where(stack_id: stack.id).delete_all
       tasks_ids = Shipit::Task.where(stack_id: stack.id).pluck(:id)
-      Shipit::OutputChunk.where(task_id: tasks_ids).delete_all
-      Shipit::Task.where(id: tasks_ids).delete_all
+      tasks_ids.each_slice(100) do |ids|
+        Shipit::OutputChunk.where(task_id: ids).delete_all
+        Shipit::Task.where(id: ids).delete_all
+      end
       stack.destroy!
     end
   end

data/app/jobs/shipit/fetch_deployed_revision_job.rb

@@ -13,7 +13,7 @@ module Shipit
       rescue DeploySpec::Error
       end
 
-      return unless sha.present?
+      return if sha.blank?
 
       begin
         stack.update_deployed_revision(sha)

data/app/jobs/shipit/github_sync_job.rb

@@ -54,7 +54,7 @@ module Shipit
     end
 
     def lookup_commit(sha)
-      @stack.commits.find_by_sha(sha)
+      @stack.commits.find_by(sha: sha)
     end
   end
 end

data/app/jobs/shipit/merge_pull_requests_job.rb

@@ -8,15 +8,15 @@ module Shipit
       pull_requests.each do |pull_request|
         pull_request.refresh!
         pull_request.reject_unless_mergeable!
-        if pull_request.closed?
-          pull_request.merged_upstream? ? pull_request.complete! : pull_request.cancel!
-        end
+        pull_request.cancel! if pull_request.closed?
+        pull_request.revalidate! if pull_request.need_revalidation?
       end
 
       return false unless stack.allows_merges?
 
       pull_requests.select(&:pending?).each do |pull_request|
         pull_request.refresh!
+        next unless pull_request.all_status_checks_passed?
         begin
           pull_request.merge!
         rescue PullRequest::NotReady

data/app/jobs/shipit/perform_task_job.rb

@@ -19,6 +19,9 @@ module Shipit
       checkout_repository
       perform_task
       @task.complete!
+    rescue Command::TimedOut => error
+      @task.write("\n#{error.message}\n")
+      @task.report_timeout!(error)
     rescue Command::Error => error
       @task.write("\n#{error.message}\n")
       @task.report_failure!(error)

data/app/jobs/shipit/purge_old_deliveries_job.rb

@@ -3,6 +3,7 @@ module Shipit
     include BackgroundJob::Unique
 
     queue_as :low
+    on_duplicate :drop
 
     def perform(hook)
       hook.purge_old_deliveries!

data/app/models/shipit/api_client.rb

@@ -20,7 +20,7 @@ module Shipit
 
     class << self
       def authenticate(token)
-        find_by_id(message_verifier.verify(token).to_i)
+        find_by(id: message_verifier.verify(token).to_i)
       rescue Shipit::SimpleMessageVerifier::InvalidSignature
       end
 

data/app/models/shipit/commit.rb

@@ -23,7 +23,7 @@ module Shipit
 
     scope :reachable, -> { where(detached: false) }
 
-    delegate :broadcast_update, :github_repo_name, :hidden_statuses, :required_statuses,
+    delegate :broadcast_update, :github_repo_name, :hidden_statuses, :required_statuses, :blocking_statuses,
              :soft_failing_statuses, to: :stack
 
     def self.newer_than(commit)
@@ -31,6 +31,11 @@ module Shipit
       where('id > ?', commit.try(:id) || commit)
     end
 
+    def self.older_than(commit)
+      return all unless commit
+      where('id < ?', commit.try(:id) || commit)
+    end
+
     def self.until(commit)
       return all unless commit
       where('id <= ?', commit.try(:id) || commit)
@@ -103,10 +108,18 @@ module Shipit
       @checks ||= CommitChecks.new(self)
     end
 
-    delegate :pending?, :success?, :error?, :failure?, :state, to: :status
+    delegate :pending?, :success?, :error?, :failure?, :blocking?, :state, to: :status
 
     def deployable?
-      !locked? && (success? || stack.ignore_ci?)
+      !locked? && (stack.ignore_ci? || (success? && !blocked?))
+    end
+
+    def blocked?
+      return false if stack.blocking_statuses.empty?
+
+      # TODO: Perfs might be horrible here if the range is big.
+      # We should look at fetching the undeployed commits only once
+      stack.commits.reachable.newer_than(stack.last_deployed_commit).older_than(self).any?(&:blocking?)
     end
 
     def children
@@ -121,7 +134,8 @@ module Shipit
       pull_request_number.present?
     end
 
-    def pull_request_number # TODO: remove in a few versions when it is assumed the commits table was backfilled
+    # TODO: remove in a few versions when it is assumed the commits table was backfilled
+    def pull_request_number
       super || message_parser.pull_request_number
     end
 
@@ -133,7 +147,8 @@ module Shipit
       message.lines.first.strip
     end
 
-    def pull_request_title # TODO: remove in a few versions when it is assumed the commits table was backfilled
+    # TODO: remove in a few versions when it is assumed the commits table was backfilled
+    def pull_request_title
       super || message_parser.pull_request_title
     end
 
@@ -183,7 +198,7 @@ module Shipit
 
     def identify_pull_request
       return unless message_parser.pull_request?
-      if pull_request = stack.pull_requests.find_by_number(message_parser.pull_request_number)
+      if pull_request = stack.pull_requests.find_by(number: message_parser.pull_request_number)
         self.pull_request = pull_request
         self.pull_request_number = pull_request.number
         self.pull_request_title = pull_request.title
@@ -194,6 +209,14 @@ module Shipit
       self.pull_request_title = message_parser.pull_request_title unless self[:pull_request_title]
     end
 
+    def deploy_requested_at
+      if pull_request.try!(:merged?)
+        pull_request.merge_requested_at
+      else
+        created_at
+      end
+    end
+
     private
 
     def message_parser