shipit-engine 0.21.0 → 0.22.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f6940a4cd6a4608c60f6349020a01ef276049cf5aa78ff784abfe2cdf99a920f
-  data.tar.gz: f7056eb044a29a5646a349bcfe9fc8c6614fcbd7a199797275657c6c3598bb2c
+  metadata.gz: '09b8ffb2b4130e05d8da6bfbdc22080becca47dd92ee5a06b4113d3729578226'
+  data.tar.gz: 2223bee54a1d2a084823763876a049668829af0725779b413f0ed6d665ed45d6
 SHA512:
-  metadata.gz: 4d61eb2c4fe287478d9ef7589b59ac268abbac149dc1783f15cf98d22230463aaeea4caf13060da7f0b2038742585e5df00cd29d05edd571a2326c030bbefbc0
-  data.tar.gz: 6d94a0ca00022f7727d7ecc9dd965f60ae9b92867c44434fa5057cd036a512ed04a685f0dd0e93a8886bd4df370a01682e6e2c213b5c8d65c1db4339efd79f2e
+  metadata.gz: 2a2bef860f0dc71622f450e4c52949680c7edca82c48462a48db34beb034b4b022e020228d160f5b86b7792d5938e0fdc09e6fb08e018c8a8e7f147bf22b60e7
+  data.tar.gz: 7cf318e67131f2a0a4bfbd316a8b732a5c8cd57fa49428d1f38a25e978a2e1535d0c5c8ca5e67bb83b4ad3c8f7dd07390e43832ef06ba697a9a2c5671e2df2d6

data/README.md

@@ -20,7 +20,6 @@ This guide aims to help you [set up](#installation-and-setup), [use](#using-ship
 **I. INSTALLATION & SETUP**
 
 * [Installation](#installation)
-* [Configuring shipit.yml and secrets.yml](#configuring-ymls)
 * [Updating an existing installation](#updating-shipit)
 
 **II. USING SHIPIT**
@@ -43,21 +42,7 @@ This guide aims to help you [set up](#installation-and-setup), [use](#using-ship
 
 <h3 id="installation">Installation</h3>
 
-*Shipit requires a database (MySQL, PostgreSQL or SQLite3), Redis, and Ruby 2.1 or superior.*
-
-Shipit provides you with a Rails template. To bootstrap your Shipit installation:
-
-1. If you don't have Rails installed, run this command: `gem install rails -v 5.1`
-2. Run this command:  `rails _5.1_ new shipit --skip-action-cable --skip-turbolinks --skip-action-mailer -m https://raw.githubusercontent.com/Shopify/shipit-engine/master/template.rb`
-3. Enter your **Client ID**, **Client Secret**, and **GitHub API access token** when prompted. These can be found on your application's GitHub page.
-4. To setup the database, run this command: `rake db:setup`
-
-<h3 id="configuring-ymls">Configuring <code>shipit.yml</code> and <code>secrets.yml</code></h3>
-
-Shipit should just work right out of the box &mdash; you probably won't need to alter its configuration files before getting it up and running. But if you want to customize Shipit for your own deployment environment, you'll need to edit the `shipit.yml` and `secrets.yml` files:
-
-* The settings in the `shipit.yml` file are related to the different things you can do within Shipit, such as handling deploys, performing custom tasks, and enforcing deployment checklists. If you want to edit these settings, [start here](#configuring-shipit).
-* The settings in the `secrets.yml` file are related to the ways that Shipit connects with GitHub. If you want to edit these settings, [start here](#configuring-secrets).
+To create a new Shipit installation you can follow the [setup guide](docs/setup.md).
 
 <h3 id="updating-shipit">Updating an existing installation</h3>
 
@@ -65,6 +50,10 @@ Shipit should just work right out of the box &mdash; you probably won't need to
 2. Update the `shipit-engine` gem with `bundle update shipit-engine`.
 3. Install new migrations with `rake shipit:install:migrations db:migrate`.
 
+<h3 id="special-update">Specific updates requiring more steps</h3>
+
+If you are upgrading from `0.21` or older, you will have to update the configuration. Please follow [the dedicated upgrade guide](docs/updates/0.22.md)
+
 * * *
 
 <h2 id="using-shipit">II. USING SHIPIT</h2>
@@ -570,96 +559,6 @@ See also `commands_inactivity_timeout` in `secrets.yml` for a global timeout set
 
 
 ***
-<h2 id="configuring-secrets">Configuring <code>secrets.yml</code></h2>
-
-The settings in the `secrets.yml` file relate to the ways that GitHub connects with Shipit:
-
-**`secret_key_base`** is used to verify the integrity of signed cookies.
-
-For example:
-
-```yml
-production:
-  secret_key_base: s3cr3t # This needs to be a very long, fully random
-```
-<br>
-
-**`github_oauth`** contains the settings required to authenticate users through GitHub.
-
-The value for `id` is your application's  *Client ID*, and the value for `secret` is your application's *Client Secret* &mdash; both of these should appear on your application's GitHub page.
-
-Note: When setting up your application in Github, set the *Authorization callback URL* to `<yourdomain>/github/auth/github/callback`.
-
-The `teams` key is optional, and required only if you want to restrict access to a set of GitHub teams.
-
-If it's missing, the Shipit installation will be public unless you setup another authentication method.
-
-After you change the list of teams, you have to invoke `bin/rake teams:fetch` in production so that a webhook is setup to keep the list of members up to date.
-
-For example:
-
-```yml
-production:
-  github_oauth:
-    id: (your application's Client ID)
-    secret: (your application's Client Secret)
-    teams:
-      - Shipit/team
-      - Shipit/another_team
-```
-<br>
-
-**`github_api`** communicates with the GitHub API about the stacks and setup Hooks. It should reflect the guidelines at  https://github.com/octokit/octokit.rb.
-
-If you specify an `access_token`, you don't need a `login` and `password`. The opposite is also true:  if you specify a `login` and `password`, then you don't need an `access_token`.
-
-For example:
-
-```yml
-production:
-  github_api:
-    access_token: 10da65c687f6degaf5475ce12a980d5vd8c44d2a
-```
-<br>
-
-**`host`**  is the host that hosts Shipit. It's used to generate URLs, and it's the host that GitHub will try to talk to.
-
-For example:
-```yml
-production:
-  host: 'http://localhost:3000'
-```
-<br>
-
-**`redis_url`** is the URL of the redis instance that Shipit uses.
-
-For example:
-
-```yml
-production:
-  redis_url: "redis://127.0.0.1:6379/7"
-```
-
-<br>
-
-If you use GitHub Enterprise, you must also specify the `github_domain`.
-
-For example:
-```yml
-production:
-  github_domain: "github.example.com"
-
-```
-
-<br>
-
-**`commands_inactivity_timeout`** is the duration after which Shipit will terminate a command if no ouput was received. Default is `300` (5 minutes).
-
-For example:
-```yml
-production:
-  commands_inactivity_timeout: 900 # 15 minutes
-```
 
 <h2 id="script-parameters">Script parameters</h2>
 

data/app/assets/images/timedout.svg

@@ -0,0 +1,14 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Generator: Adobe Illustrator 18.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
+	 viewBox="0 0 60.4 71.8" enable-background="new 0 0 60.4 71.8" xml:space="preserve">
+<g>
+	<path d="M34.7,11.7V5.9h3.1c1.6,0,2.9-1.3,2.9-2.9c0-1.6-1.3-2.9-2.9-2.9H22.6c-1.6,0-2.9,1.3-2.9,2.9c0,1.6,1.3,2.9,2.9,2.9h3.1
+		v5.9C11.2,13.9,0,26.5,0,41.6c0,16.7,13.5,30.2,30.2,30.2s30.2-13.5,30.2-30.2C60.4,26.5,49.3,13.9,34.7,11.7z M30.2,64.3
+		c-12.5,0-22.7-10.2-22.7-22.7c0-12.5,10.2-22.7,22.7-22.7s22.7,10.2,22.7,22.7C52.9,54.1,42.7,64.3,30.2,64.3z"/>
+	<path d="M44.9,48.4l-12-8.1V26.4c0-0.3-0.1-0.7-0.4-0.9c-0.2-0.2-0.6-0.4-0.9-0.4h-3.3c-0.3,0-0.7,0.1-0.9,0.4
+		c-0.2,0.2-0.4,0.6-0.4,0.9v16.4c0,0.3,0.1,0.7,0.4,0.9c0.1,0.1,0.2,0.2,0.3,0.2l13.9,9.3c0.2,0.1,0.5,0.2,0.7,0.2
+		c0.1,0,0.2,0,0.3,0c0.3-0.1,0.6-0.3,0.8-0.6l1.8-2.8C45.6,49.5,45.4,48.8,44.9,48.4z"/>
+</g>
+</svg>

data/app/assets/stylesheets/_pages/_commits.scss

@@ -281,13 +281,22 @@
 }
 
 .status--error,
-[data-deploy-status='error'],
+[data-deploy-status='error'] {
+  border-color: #333;
+
+  .status__icon {
+    background-image: asset-data-url("error.svg");
+  }
+}
+
 .status--timedout,
 [data-deploy-status='timedout'] {
   border-color: #333;
 
   .status__icon {
-    background-image: asset-data-url("error.svg");
+    background-image: asset-data-url("timedout.svg");
+    background-position: top 35% left 50%;
+    background-size: 55%;
   }
 }
 

data/app/controllers/shipit/stacks_controller.rb

@@ -1,6 +1,6 @@
 module Shipit
   class StacksController < ShipitController
-    before_action :load_stack, only: %i(update destroy settings sync_webhooks clear_git_cache refresh)
+    before_action :load_stack, only: %i(update destroy settings clear_git_cache refresh)
 
     def new
       @stack = Stack.new
@@ -64,12 +64,6 @@ module Shipit
       redirect_to(params[:return_to].presence || stack_settings_path(@stack), options)
     end
 
-    def sync_webhooks
-      @stack.setup_hooks
-      flash[:success] = 'Webhooks syncing scheduled'
-      redirect_to stack_settings_path(@stack)
-    end
-
     def clear_git_cache
       ClearGitCacheJob.perform_later(@stack)
       flash[:success] = 'Git Cache clearing scheduled'

data/app/controllers/shipit/webhooks_controller.rb

@@ -4,102 +4,138 @@ module Shipit
 
     respond_to :json
 
-    def push
-      branch = params['ref'].gsub('refs/heads/', '')
+    class Handler
+      class << self
+        attr_reader :param_parser
 
-      if branch == stack.branch
-        GithubSyncJob.perform_later(stack_id: stack.id)
+        def params(&block)
+          @param_parser = ExplicitParameters::Parameters.define(&block)
+        end
       end
 
-      head :ok
-    end
+      attr_reader :params, :payload
 
-    params do
-      requires :sha, String
-      requires :state, String
-      accepts :description, String
-      accepts :target_url, String
-      accepts :context, String
-      accepts :created_at, String
+      def initialize(payload)
+        @payload = payload
+        @params = self.class.param_parser.parse!(payload)
+      end
 
-      accepts :branches, Array do
-        requires :name, String
+      def process
+        raise NotImplementedError
       end
-    end
-    def state
-      if commit = stack.commits.find_by(sha: params.sha)
-        commit.create_status_from_github!(params)
+
+      private
+
+      def stacks
+        @stacks ||= Stack.repo(repository_name)
+      end
+
+      def repository_name
+        payload.dig('repository', 'full_name')
       end
-      head :ok
     end
 
-    params do
-      requires :team do
-        requires :id, Integer
-        requires :name, String
-        requires :slug, String
-        requires :url, String
+    class PushHandler < Handler
+      params do
+        requires :ref
       end
-      requires :organization do
-        requires :login, String
+      def process
+        stacks.where(branch: branch).each(&:sync_github)
       end
-      requires :member do
-        requires :login, String
+
+      private
+
+      def branch
+        params.ref.gsub('refs/heads/', '')
       end
     end
-    def membership
-      team = find_or_create_team!
-      member = User.find_or_create_by_login!(params.member.login)
-
-      case membership_action
-      when 'added'
-        team.add_member(member)
-      when 'removed'
-        team.members.delete(member)
-      else
-        raise ArgumentError, "Don't know how to perform action: `#{params.action.inspect}`"
+
+    class StatusHandler < Handler
+      params do
+        requires :sha, String
+        requires :state, String
+        accepts :description, String
+        accepts :target_url, String
+        accepts :context, String
+        accepts :created_at, String
+
+        accepts :branches, Array do
+          requires :name, String
+        end
+      end
+      def process
+        Commit.where(sha: params.sha).each do |commit|
+          commit.create_status_from_github!(params)
+        end
       end
-      head :ok
     end
 
-    def index
-      render text: "working"
+    class MembershipHandler < Handler
+      params do
+        requires :action, String
+        requires :team do
+          requires :id, Integer
+          requires :name, String
+          requires :slug, String
+          requires :url, String
+        end
+        requires :organization do
+          requires :login, String
+        end
+        requires :member do
+          requires :login, String
+        end
+      end
+      def process
+        team = find_or_create_team!
+        member = User.find_or_create_by_login!(params.member.login)
+
+        case params.action
+        when 'added'
+          team.add_member(member)
+        when 'removed'
+          team.members.delete(member)
+        else
+          raise ArgumentError, "Don't know how to perform action: `#{action.inspect}`"
+        end
+      end
+
+      private
+
+      def find_or_create_team!
+        Team.find_or_create_by!(github_id: params.team.id) do |team|
+          team.github_team = params.team
+          team.organization = params.organization.login
+        end
+      end
     end
 
-    private
+    HANDLERS = {
+      'push' => PushHandler,
+      'status' => StatusHandler,
+      'membership' => MembershipHandler,
+    }.freeze
 
-    def find_or_create_team!
-      Team.find_or_create_by!(github_id: params.team.id) do |team|
-        team.github_team = params.team
-        team.organization = params.organization.login
-        team.automatically_setup_hooks = true
+    def create
+      if handler = HANDLERS[event]
+        handler.new(JSON.parse(request.raw_post)).process
       end
+
+      head :ok
     end
 
+    private
+
     def verify_signature
-      head(422) unless webhook.verify_signature(request.headers['X-Hub-Signature'], request.raw_post)
+      head(422) unless Shipit.github.verify_webhook_signature(request.headers['X-Hub-Signature'], request.raw_post)
     end
 
     def check_if_ping
       head :ok if event == 'ping'
     end
 
-    def webhook
-      @webhook ||= if params[:stack_id]
-        stack.github_hooks.find_by!(event: event)
-      else
-        GithubHook::Organization.find_by!(organization: params.organization.login, event: event)
-      end
-    end
-
     def event
-      request.headers['X-Github-Event'] || action_name
-    end
-
-    def membership_action
-      # GitHub send an `action` parameter that is shadowed by Rails url parameters
-      # It's also impossible to pass an `action` parameters from a test case.
-      request.request_parameters['action'] || params[:_action]
+      request.headers.fetch('X-Github-Event')
     end
 
     def stack

data/app/helpers/shipit/github_url_helper.rb

@@ -4,7 +4,7 @@ module Shipit
 
     def github_avatar(user, options = {})
       uri = user.avatar_uri
-      attributes = options.slice(:class).merge(alt: user.try!(:name))
+      attributes = options.slice(:class).merge(alt: user&.name)
       if options[:size]
         uri.query ||= ''
         uri.query += "&s=#{options[:size]}"
@@ -21,7 +21,7 @@ module Shipit
     module_function :github_commit_range_url
 
     def github_user_url(user, *args)
-      [Shipit.github_url, user, *args].join('/')
+      Shipit.github.url(user, *args)
     end
     module_function :github_user_url
 

data/app/helpers/shipit/shipit_helper.rb

@@ -41,38 +41,10 @@ module Shipit
       tags
     end
 
-    def missing_github_oauth_message
+    def missing_github_app_message
+      # TODO: Document how to create an app
       <<-MESSAGE.html_safe
-        Shipit requires a GitHub application to authenticate users.
-        If you haven't created an application on GitHub yet, you can do so in the
-        #{link_to 'Settings', Shipit.github_url('/settings/applications/new'), target: '_blank'}
-        section of your profile. You can also create applications for organizations.
-        When setting up your application in Github, set the Homepage URL to your domain
-        and the Authorization callback URL to '<yourdomain>/github/auth/github/callback'.
-      MESSAGE
-    end
-
-    def missing_github_oauth_id_message
-      <<-MESSAGE.html_safe
-        Copy the Client ID from your GitHub application,
-        and paste it into the secrets.yml file under <code>github_oauth.id</code>.
-       MESSAGE
-    end
-
-    def missing_github_oauth_secret_message
-      <<-MESSAGE.html_safe
-        Copy the Client Secret from your GitHub application,
-        and paste it into the secrets.yml file under <code>github_oauth.secret</code>.
-       MESSAGE
-    end
-
-    def missing_github_api_credentials_message
-      <<-MESSAGE.html_safe
-        Shipit needs API access to GitHub. You can
-        #{link_to 'create an access token', Shipit.github_url('/settings/tokens'), target: '_blank'}
-        with the following permissions:
-        <code>admin:repo_hook</code>, <code>admin:org_hook</code> and <code>repo</code>
-        and add it to the secrets.yml file under the key <code>github_api.access_token</code>.
+        Shipit requires a GitHub App to authenticate users and perform API calls.
       MESSAGE
     end