kubernetes-deploy 0.27.0 → 0.28.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 84961e6f603a285f3a16df6fa04f27201e80cf29476d6fc9a8a6ace4748ed64e
-  data.tar.gz: 484d466ef9e3cf80401edc34dd1a4ddefbd57013c1022b274d78f04456461739
+  metadata.gz: 825f058c113d4ae975ee5937866b639ac4d705f5cf9493ab414b56a183bd862e
+  data.tar.gz: 2a6e43245a95f182e63d9f81d085f62aede3d16c74f4184d2f07bb6f0db55563
 SHA512:
-  metadata.gz: 5a7f78c88ccc77549205df00fda10ef4843f721043182b01a5e00e1059a971de2cad12a2c820c4b79c81c1c6a4942baea69e14b4b443bb17f22dec002edfaedd
-  data.tar.gz: 9e6d96674cc341c65cff8fea18ab0cc18196580f4a5134c04a6c91ac6fdb4d1e9d9825520d6e0110432d3ff87c38623cb5fda8843157d73f039042ad6b360d94
+  metadata.gz: dbd772b8de088a986c62075c1443069be9fed802cf04958144ff377c26f3f594e9e2a6b5e64e1570452e936016e03fdc079b7dad986019cc27473e22ebf3bec1
+  data.tar.gz: ff2824c32e6421056842a190fe8658f0fc903b3c866919b0a83f4bb4446b76d5d5cffad739fcba0c1564bb5079c02c21eb2022df30f379f9876e190157fe1ece

data/.buildkite/pipeline.nightly.yml

@@ -1,4 +1,11 @@
 steps:
+  - name: 'Run Test Suite (:kubernetes: 1.15-latest)'
+    command: bin/ci
+    agents:
+      queue: k8s-ci
+    env:
+      LOGGING_LEVEL: 4
+      KUBERNETES_VERSION: v1.15-latest
   - name: 'Run Test Suite (:kubernetes: 1.14-latest)'
     command: bin/ci
     agents:
@@ -27,10 +34,3 @@ steps:
     env:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.11-latest
-  - name: 'Run Test Suite (:kubernetes: 1.10-latest)'
-    command: bin/ci
-    agents:
-      queue: minikube-ci
-    env:
-      LOGGING_LEVEL: 4
-      KUBERNETES_VERSION: v1.10-latest

data/.shopify-build/kubernetes-deploy.yml

@@ -9,6 +9,13 @@ steps:
       - bundle exec rubocop
     dependencies:
       - bundler
+  - label: 'Run Test Suite (:kubernetes: 1.15-latest)'
+    command: bin/ci
+    agents:
+      queue: k8s-ci
+    env:
+      LOGGING_LEVEL: "4"
+      KUBERNETES_VERSION: v1.15-latest
   - label: 'Run Test Suite (:kubernetes: 1.14-latest)'
     command: bin/ci
     agents:
@@ -37,10 +44,3 @@ steps:
     env:
       LOGGING_LEVEL: "4"
       KUBERNETES_VERSION: v1.11-latest
-  - label: 'Run Test Suite (:kubernetes: 1.10-latest)'
-    command: bin/ci
-    agents:
-      queue: minikube-ci
-    env:
-      LOGGING_LEVEL: "4"
-      KUBERNETES_VERSION: v1.10-latest

data/CHANGELOG.md

@@ -1,6 +1,29 @@
 ## next
 
+*Important!*
+- The next release will be 1.0.0, which means that master will contain breaking changes.
+
+## 0.28.0
+
+*Enhancements*
+- Officially support Kubernetes 1.15 ([#546](https://github.com/Shopify/kubernetes-deploy/pull/546))
+- Make sure that we only declare a Service of type LoadBalancer as deployed after its IP address is published. [#547](https://github.com/Shopify/kubernetes-deploy/pull/547)
+- Add more validations to `RunnerTask`. [#554](https://github.com/Shopify/kubernetes-deploy/pull/554)
+- Validate secrets with `--server-dry-run` on supported clusters. [#553](https://github.com/Shopify/kubernetes-deploy/pull/553) 
+*Bug Fixes*
+- Fix a bug in rendering where we failed to add a yaml doc separator (`---`) to
+  an implicit document if there are multiple documents in the file.
+  ([#551](https://github.com/Shopify/kubernetes-deploy/pull/551))
+
+*Other*
+- Kubernetes 1.10 is no longer officially supported as of this version ([#546](https://github.com/Shopify/kubernetes-deploy/pull/546))
+- We've added a new Krane cli. This code is in alpha. We are providing
+no warranty at this time and reserve the right to make major breaking changes including
+removing it entirely at any time. ([#256](https://github.com/Shopify/kubernetes-deploy/issues/256))
+- Deprecate `kubernetes-deploy.shopify.io` annotations in favour of `krane.shopify.io` ([#539](https://github.com/Shopify/kubernetes-deploy/pull/539))
+
 ## 0.27.0
+
 *Enhancements*
 - (alpha) Introduce a new `-f` flag for `kubernetes-deploy`. Allows passing in of multiple directories and/or filenames. Currently only usable by `kubernetes-deploy`, not `kubernetes-render`. [#514](https://github.com/Shopify/kubernetes-deploy/pull/514)
 - Initial implementation of shared task validation objects. [#533](https://github.com/Shopify/kubernetes-deploy/pull/533)

data/README.md

@@ -1,3 +1,7 @@
+# krane
+
+As this project approaches the v1.0 milestone, we're excited to announce that `kubernetes-deploy` will be [officially renamed as `krane`](https://github.com/Shopify/kubernetes-deploy/issues/30#issuecomment-468750341). Follow the [1.0 requirement label](https://github.com/Shopify/kubernetes-deploy/issues?q=is%3Aissue+is%3Aopen+label%3A%22%3Arocket%3A+1.0+requirement%22) to keep up with the progress.
+
 # kubernetes-deploy [![Build status](https://badge.buildkite.com/61937e40a1fc69754d9d198be120543d6de310de2ba8d3cb0e.svg?branch=master)](https://buildkite.com/shopify/kubernetes-deploy) [![codecov](https://codecov.io/gh/Shopify/kubernetes-deploy/branch/master/graph/badge.svg)](https://codecov.io/gh/Shopify/kubernetes-deploy)
 
 `kubernetes-deploy` is a command line tool that helps you ship changes to a Kubernetes namespace and understand the result. At Shopify, we use it within our much-beloved, open-source [Shipit](https://github.com/Shopify/shipit-engine#kubernetes) deployment app.
@@ -68,7 +72,7 @@ This repo also includes related tools for [running tasks](#kubernetes-run) and [
 ## Prerequisites
 
 * Ruby 2.3+
-* Your cluster must be running Kubernetes v1.10.0 or higher<sup>1</sup>
+* Your cluster must be running Kubernetes v1.11.0 or higher<sup>1</sup>
 * Each app must have a deploy directory containing its Kubernetes templates (see [Templates](#using-templates-and-variables))
 
 <sup>1</sup> We run integration tests against these Kubernetes versions. You can find our
@@ -81,10 +85,11 @@ official compatibility chart below.
 |        1.7         |        0.20.6         |
 |        1.8         |        0.21.1         |
 |        1.9         |        0.24.0         |
+|        1.10        |        0.27.0         |
 
 ## Installation
 
-1. [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl) (requires v1.10.0 or higher) and make sure it is available in your $PATH
+1. [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl) (requires v1.11.0 or higher) and make sure it is available in your $PATH
 2. Set up your [kubeconfig file](https://kubernetes.io/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) for access to your cluster(s).
 3. `gem install kubernetes-deploy`
 
@@ -228,10 +233,10 @@ This is a limitation of the current implementation.
 
 
 ### Customizing behaviour with annotations
-- `kubernetes-deploy.shopify.io/timeout-override`: Override the tool's hard timeout for one specific resource. Both full ISO8601 durations and the time portion of ISO8601 durations are valid. Value must be between 1 second and 24 hours.
+- `krane.shopify.io/timeout-override`: Override the tool's hard timeout for one specific resource. Both full ISO8601 durations and the time portion of ISO8601 durations are valid. Value must be between 1 second and 24 hours.
   - _Example values_: 45s / 3m / 1h / PT0.25H
   - _Compatibility_: all resource types
-- `kubernetes-deploy.shopify.io/required-rollout`: Modifies how much of the rollout needs to finish
+- `krane.shopify.io/required-rollout`: Modifies how much of the rollout needs to finish
 before the deployment is considered successful.
   - _Compatibility_: Deployment
   - `full`: The deployment is successful when all pods in the new `replicaSet` are ready.
@@ -242,11 +247,11 @@ before the deployment is considered successful.
   that use the `RollingUpdate` strategy.
   - Percent (e.g. 90%): The deploy is successful when the number of new pods that are ready is equal to
   `spec.replicas` * Percent.
-- `kubernetes-deploy.shopify.io/prunable`: Allows a Custom Resource to be pruned during deployment.
+- `krane.shopify.io/prunable`: Allows a Custom Resource to be pruned during deployment.
   - _Compatibility_: Custom Resource Definition
   - `true`: The custom resource will be pruned if the resource is not in the deploy directory.
   - All other values: The custom resource will not be pruned.
-- `kubernetes-deploy.shopify.io/predeployed`: Causes a Custom Resource to be deployed in the pre-deploy phase.
+- `krane.shopify.io/predeployed`: Causes a Custom Resource to be deployed in the pre-deploy phase.
   - _Compatibility_: Custom Resource Definition
   - _Default_: `true`
   - `true`: The custom resource will be deployed in the pre-deploy phase.
@@ -335,12 +340,12 @@ This feature is only available on clusters running Kubernetes 1.11+ since it rel
 *Requirements:*
 
 * The custom resource must expose a `status` subresource with an `observedGeneration` field.
-* The `kubernetes-deploy.shopify.io/instance-rollout-conditions` annotation must be present on the CRD that defines the custom resource.
-* (optional) The `kubernetes-deploy.shopify.io/instance-timeout` annotation can be added to the CRD that defines the custom resource to override the global default timeout for all instances of that resource. This annotation can use ISO8601 format or unprefixed ISO8601 time components (e.g. '1H', '60S').
+* The `krane.shopify.io/instance-rollout-conditions` annotation must be present on the CRD that defines the custom resource.
+* (optional) The `krane.shopify.io/instance-timeout` annotation can be added to the CRD that defines the custom resource to override the global default timeout for all instances of that resource. This annotation can use ISO8601 format or unprefixed ISO8601 time components (e.g. '1H', '60S').
 
 #### Specifying pass/fail conditions
 
-The presence of a valid `kubernetes-deploy.shopify.io/instance-rollout-conditions` annotation on a CRD will cause kubernetes-deploy to monitor the rollout of all instances of that custom resource. Its value can either be `"true"` (giving you the defaults described in the next section) or a valid JSON string with the following format:
+The presence of a valid `krane.shopify.io/instance-rollout-conditions` annotation on a CRD will cause kubernetes-deploy to monitor the rollout of all instances of that custom resource. Its value can either be `"true"` (giving you the defaults described in the next section) or a valid JSON string with the following format:
 ```
 '{
   "success_conditions": [
@@ -364,7 +369,7 @@ You **must** ensure that your custom resource controller sets `.status.observedG
 
 #### Example
 
-As an example, the following is the default configuration that will be used if you set `kubernetes-deploy.shopify.io/instance-rollout-conditions: "true"` on the CRD that defines the custom resources you wish to monitor:
+As an example, the following is the default configuration that will be used if you set `krane.shopify.io/instance-rollout-conditions: "true"` on the CRD that defines the custom resources you wish to monitor:
 
 ```
 '{
@@ -462,7 +467,7 @@ Refer to `kubernetes-restart --help` for the authoritative set of options.
 
 ## Prerequisites
 
-* You've already deployed a [`PodTemplate`](https://v1-10.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.10/#podtemplate-v1-core) object with field `template` containing a `Pod` specification that does not include the `apiVersion` or `kind` parameters. An example is provided in this repo in `test/fixtures/hello-cloud/template-runner.yml`.
+* You've already deployed a [`PodTemplate`](https://v1-10.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#podtemplate-v1-core) object with field `template` containing a `Pod` specification that does not include the `apiVersion` or `kind` parameters. An example is provided in this repo in `test/fixtures/hello-cloud/template-runner.yml`.
 * The `Pod` specification in that template has a container named `task-runner`.
 
 Based on this specification `kubernetes-run` will create a new pod with the entrypoint of the `task-runner ` container overridden with the supplied arguments.

data/Rakefile

@@ -23,7 +23,14 @@ Rake::TestTask.new(:unit_test) do |t|
   t.test_files = FileList['test/unit/**/*_test.rb']
 end
 
+desc("Run cli tests")
+Rake::TestTask.new(:cli_test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList['test/exe/**/*_test.rb']
+end
+
 desc("Run all tests")
-task(test: %w(unit_test serial_integration_test integration_test))
+task(test: %w(unit_test serial_integration_test integration_test cli_test))
 
 task(default: :test)

data/bin/setup

@@ -9,8 +9,8 @@ if [ ! -x "$(which minikube)" ]; then
 fi
 
 if [ ! -x "$(which kubectl)" ]; then
-  echo -e "\n\033[0;33mPlease install kubectl version 1.10.0 or higher:\nhttps://kubernetes.io/docs/user-guide/prereqs/\033[0m"
+  echo -e "\n\033[0;33mPlease install kubectl version 1.11.0 or higher:\nhttps://kubernetes.io/docs/user-guide/prereqs/\033[0m"
 else
   KUBECTL_VERSION=$(kubectl version --short --client | grep -oe "v[[:digit:]\.]\+")
-  echo -e "\n\033[0;32mKubectl version $KUBECTL_VERSION is already installed. This gem requires version v1.10.0 or greater.\033[0m"
+  echo -e "\n\033[0;32mKubectl version $KUBECTL_VERSION is already installed. This gem requires version v1.11.0 or greater.\033[0m"
 fi

data/bin/test

@@ -32,6 +32,9 @@ if [[ ${CI:="0"} == "1" ]]; then
   bundle install --jobs 4
 fi
 
+print_header "Run CLI Tests"
+bundle exec rake cli_test
+
 print_header "Run Unit Tests"
 bundle exec rake unit_test
 

data/exe/krane

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'krane/cli/krane'
+Krane::CLI::Krane.start(ARGV)

data/exe/kubernetes-restart

@@ -25,7 +25,7 @@ context = ARGV[1]
 restart = KubernetesDeploy::RestartTask.new(namespace: namespace, context: context,
    max_watch_seconds: max_watch_seconds)
 begin
-  restart.perform!(raw_deployments, selector: selector)
+  restart.run!(raw_deployments, selector: selector)
 rescue KubernetesDeploy::DeploymentTimeoutError
   exit(70)
 rescue KubernetesDeploy::FatalDeploymentError

data/kubernetes-deploy.gemspec

@@ -32,6 +32,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency("oj", "~> 3.0")
   spec.add_dependency("concurrent-ruby", "~> 1.1")
   spec.add_dependency("jsonpath", "~> 0.9.6")
+  spec.add_dependency("thor", "~>  0.20.3")
 
   spec.add_development_dependency("bundler")
   spec.add_development_dependency("rake", "~> 10.0")

data/lib/krane.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+require 'kubernetes-deploy/common'
+
+module Krane
+end

data/lib/krane/cli/krane.rb

@@ -0,0 +1,49 @@
+# frozen_string_literal: true
+
+require 'krane'
+require 'thor'
+require 'krane/cli/version_command'
+require 'krane/cli/restart_command'
+
+module Krane
+  module CLI
+    class Krane < Thor
+      TIMEOUT_EXIT_CODE = 70
+      FAILURE_EXIT_CODE = 1
+
+      package_name "Krane"
+
+      def self.expand_options(task_options)
+        task_options.each { |option_name, config| method_option(option_name, config) }
+      end
+
+      desc("version", "Prints the version")
+      expand_options(VersionCommand::OPTIONS)
+      def version
+        VersionCommand.from_options(options)
+      end
+
+      desc("restart NAMESPACE CONTEXT", "Restart the pods in one or more deployments")
+      expand_options(RestartCommand::OPTIONS)
+      def restart(namespace, context)
+        rescue_and_exit do
+          RestartCommand.from_options(namespace, context, options)
+        end
+      end
+
+      def self.exit_on_failure?
+        true
+      end
+
+      private
+
+      def rescue_and_exit
+        yield
+      rescue KubernetesDeploy::DeploymentTimeoutError
+        exit(TIMEOUT_EXIT_CODE)
+      rescue KubernetesDeploy::FatalDeploymentError
+        exit(FAILURE_EXIT_CODE)
+      end
+    end
+  end
+end

data/lib/krane/cli/restart_command.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Krane
+  module CLI
+    class RestartCommand
+      DEFAULT_RESTART_TIMEOUT = '300s'
+      OPTIONS = {
+        "deployments" => { type: :array, banner: "list of deployments",
+                           desc: "List of workload names to restart" },
+        "global-timeout" => { type: :string, banner: "duration", default: DEFAULT_RESTART_TIMEOUT,
+                              desc: "Max duration to monitor workloads correctly restarted" },
+        "selector" => { type: :string, banner: "'label=value'",
+                        desc: "Select workloads by selector(s)" },
+        "verify-result" => { type: :boolean, default: true,
+                             desc: "Verify workloads correctly restarted" },
+      }
+
+      def self.from_options(namespace, context, options)
+        require 'kubernetes-deploy/restart_task'
+        selector = KubernetesDeploy::LabelSelector.parse(options[:selector]) if options[:selector]
+        restart = KubernetesDeploy::RestartTask.new(
+          namespace: namespace,
+          context: context,
+          max_watch_seconds: KubernetesDeploy::DurationParser.new(options["global-timeout"]).parse!.to_i,
+        )
+        restart.run!(
+          options[:deployments],
+          selector: selector,
+          verify_result: options["verify-result"]
+        )
+      end
+    end
+  end
+end

data/lib/krane/cli/version_command.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+module Krane
+  module CLI
+    class VersionCommand
+      OPTIONS = {}
+
+      def self.from_options(_)
+        puts("krane #{KubernetesDeploy::VERSION}")
+      end
+    end
+  end
+end

data/lib/kubernetes-deploy/common.rb

@@ -19,6 +19,6 @@ require 'kubernetes-deploy/task_config'
 require 'kubernetes-deploy/task_config_validator'
 
 module KubernetesDeploy
-  MIN_KUBE_VERSION = '1.10.0'
+  MIN_KUBE_VERSION = '1.11.0'
   StatsD.build
 end

data/lib/kubernetes-deploy/deploy_task.rb

@@ -24,9 +24,6 @@ require 'kubernetes-deploy/kubernetes_resource'
   service_account
   daemon_set
   resource_quota
-  elasticsearch
-  statefulservice
-  topic
   stateful_set
   cron_job
   job
@@ -255,6 +252,11 @@ module KubernetesDeploy
       KubernetesDeploy::Concurrency.split_across_threads(resources) do |r|
         r.validate_definition(kubectl, selector: @selector)
       end
+
+      resources.select(&:has_warnings?).each do |resource|
+        record_warnings(warning: resource.validation_warning_msg, filename: File.basename(resource.file_path))
+      end
+
       failed_resources = resources.select(&:validation_failed?)
       return unless failed_resources.present?
 
@@ -319,6 +321,12 @@ module KubernetesDeploy
       @logger.summary.add_paragraph(debug_msg)
     end
 
+    def record_warnings(warning:, filename:)
+      warn_msg = "Template warning: #{filename}\n"
+      warn_msg += "> Warning message:\n#{FormattedLogger.indent_four(warning)}"
+      @logger.summary.add_paragraph(ColorizedString.new(warn_msg).yellow)
+    end
+
     def validate_configuration(allow_protected_ns:, prune:)
       errors = []
       errors += kubeclient_builder.validate_config_files

data/lib/kubernetes-deploy/kubernetes_resource.rb

@@ -15,6 +15,8 @@ module KubernetesDeploy
     GLOBAL = false
     TIMEOUT = 5.minutes
     LOG_LINE_COUNT = 250
+    SERVER_DRY_RUN_DISABLED_ERROR =
+      /(unknown flag: --server-dry-run)|(doesn't support dry-run)|(dryRun alpha feature is disabled)/
 
     DISABLE_FETCHING_LOG_INFO = 'DISABLE_FETCHING_LOG_INFO'
     DISABLE_FETCHING_EVENT_INFO = 'DISABLE_FETCHING_EVENT_INFO'
@@ -30,9 +32,12 @@ module KubernetesDeploy
       If you have reason to believe it will succeed, retry the deploy to continue to monitor the rollout.
       MSG
 
-    TIMEOUT_OVERRIDE_ANNOTATION = "kubernetes-deploy.shopify.io/timeout-override"
+    TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX = "timeout-override"
+    TIMEOUT_OVERRIDE_ANNOTATION_DEPRECATED = "kubernetes-deploy.shopify.io/#{TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX}"
+    TIMEOUT_OVERRIDE_ANNOTATION = "krane.shopify.io/#{TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX}"
     LAST_APPLIED_ANNOTATION = "kubectl.kubernetes.io/last-applied-configuration"
     SENSITIVE_TEMPLATE_CONTENT = false
+    SERVER_DRY_RUNNABLE = false
 
     class << self
       def build(namespace:, context:, definition:, logger:, statsd_tags:, crd: nil)
@@ -93,7 +98,8 @@ module KubernetesDeploy
 
     def timeout_override
       return @timeout_override if defined?(@timeout_override)
-      @timeout_override = DurationParser.new(timeout_annotation).parse!.to_i
+
+      @timeout_override = DurationParser.new(krane_annotation_value(TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX)).parse!.to_i
     rescue DurationParser::ParsingError
       @timeout_override = nil
     end
@@ -113,6 +119,7 @@ module KubernetesDeploy
       @statsd_report_done = false
       @disappeared = false
       @validation_errors = []
+      @validation_warnings = []
       @instance_data = {}
     end
 
@@ -122,12 +129,22 @@ module KubernetesDeploy
 
     def validate_definition(kubectl, selector: nil)
       @validation_errors = []
+      @validation_warnings = []
       validate_selector(selector) if selector
       validate_timeout_annotation
+      validate_annotation_version
       validate_spec_with_kubectl(kubectl)
       @validation_errors.present?
     end
 
+    def validation_warning_msg
+      @validation_warnings.join("\n")
+    end
+
+    def has_warnings?
+      @validation_warnings.present?
+    end
+
     def validation_error_msg
       @validation_errors.join("\n")
     end
@@ -329,6 +346,10 @@ module KubernetesDeploy
       self.class::SENSITIVE_TEMPLATE_CONTENT
     end
 
+    def server_dry_runnable?
+      self.class::SERVER_DRY_RUNNABLE
+    end
+
     class Event
       EVENT_SEPARATOR = "ENDEVENT--BEGINEVENT"
       FIELD_SEPARATOR = "ENDFIELD--BEGINFIELD"
@@ -396,20 +417,44 @@ module KubernetesDeploy
     private
 
     def validate_timeout_annotation
-      return if timeout_annotation.nil?
+      timeout_override_value = krane_annotation_value(TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX)
+      timeout_annotation_key = krane_annotation_key(TIMEOUT_OVERRIDE_ANNOTATION_SUFFIX)
+      return if timeout_override_value.nil?
 
-      override = DurationParser.new(timeout_annotation).parse!
+      override = DurationParser.new(timeout_override_value).parse!
       if override <= 0
-        @validation_errors << "#{TIMEOUT_OVERRIDE_ANNOTATION} annotation is invalid: Value must be greater than 0"
+        @validation_errors << "#{timeout_annotation_key} annotation is invalid: Value must be greater than 0"
       elsif override > 24.hours
-        @validation_errors << "#{TIMEOUT_OVERRIDE_ANNOTATION} annotation is invalid: Value must be less than 24h"
+        @validation_errors << "#{timeout_annotation_key} annotation is invalid: Value must be less than 24h"
       end
     rescue DurationParser::ParsingError => e
-      @validation_errors << "#{TIMEOUT_OVERRIDE_ANNOTATION} annotation is invalid: #{e}"
+      @validation_errors << "#{timeout_annotation_key} annotation is invalid: #{e}"
+    end
+
+    def validate_annotation_version
+      return if validation_warning_msg.include?("annotations is deprecated")
+      annotation_keys = @definition.dig("metadata", "annotations")&.keys
+      annotation_keys&.each do |annotation|
+        if annotation.include?("kubernetes-deploy.shopify.io")
+          annotation_prefix = annotation.split('/').first
+          @validation_warnings << "#{annotation_prefix} as a prefix for annotations is deprecated: "\
+            "Use the 'krane.shopify.io' annotation prefix instead"
+          return
+        end
+      end
+    end
+
+    def krane_annotation_value(suffix)
+      @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/#{suffix}") ||
+        @definition.dig("metadata", "annotations", "krane.shopify.io/#{suffix}")
     end
 
-    def timeout_annotation
-      @definition.dig("metadata", "annotations", TIMEOUT_OVERRIDE_ANNOTATION)
+    def krane_annotation_key(suffix)
+      if @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/#{suffix}")
+        "kubernetes-deploy.shopify.io/#{suffix}"
+      elsif @definition.dig("metadata", "annotations", "krane.shopify.io/#{suffix}")
+        "krane.shopify.io/#{suffix}"
+      end
     end
 
     def validate_selector(selector)
@@ -426,9 +471,13 @@ module KubernetesDeploy
     end
 
     def validate_spec_with_kubectl(kubectl)
-      command = ["create", "-f", file_path, "--dry-run", "--output=name"]
-      _, err, st = kubectl.run(*command, log_failure: false, output_is_sensitive: sensitive_template_content?,
-        retry_whitelist: [:client_timeout], attempts: 3)
+      _, err, st = validate_with_dry_run_option(kubectl, "--dry-run")
+      if st.success? && server_dry_runnable?
+        _, err, st = validate_with_dry_run_option(kubectl, "--server-dry-run")
+        if st.success? || err.match(SERVER_DRY_RUN_DISABLED_ERROR)
+          return true
+        end
+      end
 
       return true if st.success?
       @validation_errors << if sensitive_template_content?
@@ -438,6 +487,12 @@ module KubernetesDeploy
       end
     end
 
+    def validate_with_dry_run_option(kubectl, dry_run_option)
+      command = ["apply", "-f", file_path, dry_run_option, "--output=name"]
+      kubectl.run(*command, log_failure: false, output_is_sensitive: sensitive_template_content?,
+                               retry_whitelist: [:client_timeout], attempts: 3)
+    end
+
     def labels
       @definition.dig("metadata", "labels")
     end

data/lib/kubernetes-deploy/kubernetes_resource/custom_resource_definition.rb

@@ -2,8 +2,9 @@
 module KubernetesDeploy
   class CustomResourceDefinition < KubernetesResource
     TIMEOUT = 2.minutes
-    ROLLOUT_CONDITIONS_ANNOTATION = "kubernetes-deploy.shopify.io/instance-rollout-conditions"
-    TIMEOUT_FOR_INSTANCE_ANNOTATION = "kubernetes-deploy.shopify.io/instance-timeout"
+    ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX = "instance-rollout-conditions"
+    ROLLOUT_CONDITIONS_ANNOTATION = "krane.shopify.io/#{ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX}"
+    TIMEOUT_FOR_INSTANCE_ANNOTATION = "krane.shopify.io/instance-timeout"
     GLOBAL = true
 
     def deploy_succeeded?
@@ -19,7 +20,7 @@ module KubernetesDeploy
     end
 
     def timeout_for_instance
-      timeout = @definition.dig("metadata", "annotations", TIMEOUT_FOR_INSTANCE_ANNOTATION)
+      timeout = krane_annotation_value("instance-timeout")
       DurationParser.new(timeout).parse!.to_i
     rescue DurationParser::ParsingError
       nil
@@ -50,20 +51,20 @@ module KubernetesDeploy
     end
 
     def prunable?
-      prunable = @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/prunable")
+      prunable = krane_annotation_value("prunable")
       prunable == "true"
     end
 
     def predeployed?
-      predeployed = @definition.dig("metadata", "annotations", "kubernetes-deploy.shopify.io/predeployed")
+      predeployed = krane_annotation_value("predeployed")
       predeployed.nil? || predeployed == "true"
     end
 
     def rollout_conditions
       return @rollout_conditions if defined?(@rollout_conditions)
 
-      @rollout_conditions = if rollout_conditions_annotation
-        RolloutConditions.from_annotation(rollout_conditions_annotation)
+      @rollout_conditions = if krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX)
+        RolloutConditions.from_annotation(krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX))
       end
     rescue RolloutConditionsError
       @rollout_conditions = nil
@@ -74,12 +75,13 @@ module KubernetesDeploy
 
       validate_rollout_conditions
     rescue RolloutConditionsError => e
-      @validation_errors << "Annotation #{ROLLOUT_CONDITIONS_ANNOTATION} on #{name} is invalid: #{e}"
+      @validation_errors << "Annotation #{krane_annotation_key(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX)} "\
+        "on #{name} is invalid: #{e}"
     end
 
     def validate_rollout_conditions
-      if rollout_conditions_annotation && @rollout_conditions_validated.nil?
-        conditions = RolloutConditions.from_annotation(rollout_conditions_annotation)
+      if krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX) && @rollout_conditions_validated.nil?
+        conditions = RolloutConditions.from_annotation(krane_annotation_value(ROLLOUT_CONDITIONS_ANNOTATION_SUFFIX))
         conditions.validate!
       end
 
@@ -96,9 +98,5 @@ module KubernetesDeploy
     def names_accepted_status
       names_accepted_condition["status"]
     end
-
-    def rollout_conditions_annotation
-      @definition.dig("metadata", "annotations", ROLLOUT_CONDITIONS_ANNOTATION)
-    end
   end
 end

data/lib/kubernetes-deploy/kubernetes_resource/deployment.rb

@@ -4,7 +4,9 @@ require 'kubernetes-deploy/kubernetes_resource/replica_set'
 module KubernetesDeploy
   class Deployment < KubernetesResource
     TIMEOUT = 7.minutes
-    REQUIRED_ROLLOUT_ANNOTATION = 'kubernetes-deploy.shopify.io/required-rollout'
+    REQUIRED_ROLLOUT_ANNOTATION_SUFFIX = "required-rollout"
+    REQUIRED_ROLLOUT_ANNOTATION_DEPRECATED = "kubernetes-deploy.shopify.io/#{REQUIRED_ROLLOUT_ANNOTATION_SUFFIX}"
+    REQUIRED_ROLLOUT_ANNOTATION = "krane.shopify.io/#{REQUIRED_ROLLOUT_ANNOTATION_SUFFIX}"
     REQUIRED_ROLLOUT_TYPES = %w(maxUnavailable full none).freeze
     DEFAULT_REQUIRED_ROLLOUT = 'full'
 
@@ -103,8 +105,8 @@ module KubernetesDeploy
 
       strategy = @definition.dig('spec', 'strategy', 'type').to_s
       if required_rollout.downcase == 'maxunavailable' && strategy.present? && strategy.downcase != 'rollingupdate'
-        @validation_errors << "'#{REQUIRED_ROLLOUT_ANNOTATION}: #{required_rollout}' is incompatible "\
-        "with strategy '#{strategy}'"
+        @validation_errors << "'#{krane_annotation_key(REQUIRED_ROLLOUT_ANNOTATION_SUFFIX)}: #{required_rollout}' "\
+          "is incompatible with strategy '#{strategy}'"
       end
 
       @validation_errors.empty?
@@ -148,7 +150,7 @@ module KubernetesDeploy
     end
 
     def rollout_annotation_err_msg
-      "'#{REQUIRED_ROLLOUT_ANNOTATION}: #{required_rollout}' is invalid. "\
+      "'#{krane_annotation_key(REQUIRED_ROLLOUT_ANNOTATION_SUFFIX)}: #{required_rollout}' is invalid. "\
         "Acceptable values: #{REQUIRED_ROLLOUT_TYPES.join(', ')}"
     end
 
@@ -201,7 +203,7 @@ module KubernetesDeploy
     end
 
     def required_rollout
-      @definition.dig('metadata', 'annotations', REQUIRED_ROLLOUT_ANNOTATION).presence || DEFAULT_REQUIRED_ROLLOUT
+      krane_annotation_value("required-rollout") || DEFAULT_REQUIRED_ROLLOUT
     end
 
     def percent?(value)

data/lib/kubernetes-deploy/kubernetes_resource/secret.rb

@@ -3,6 +3,7 @@ module KubernetesDeploy
   class Secret < KubernetesResource
     TIMEOUT = 30.seconds
     SENSITIVE_TEMPLATE_CONTENT = true
+    SERVER_DRY_RUNNABLE = true
 
     def status
       exists? ? "Available" : "Not Found"

data/lib/kubernetes-deploy/kubernetes_resource/service.rb

@@ -19,6 +19,8 @@ module KubernetesDeploy
     def status
       if !exists?
         "Not found"
+      elsif requires_publishing? && !published?
+        "LoadBalancer IP address is not provisioned yet"
       elsif !requires_endpoints?
         "Doesn't require any endpoints"
       elsif selects_some_pods?
@@ -30,6 +32,7 @@ module KubernetesDeploy
 
     def deploy_succeeded?
       return false unless exists?
+      return published? if requires_publishing?
       return exists? unless requires_endpoints?
       # We can't use endpoints if we want the service to be able to fail fast when the pods are down
       exposes_zero_replica_workload? || selects_some_pods?
@@ -89,5 +92,13 @@ module KubernetesDeploy
     def external_name_svc?
       @definition["spec"]["type"] == "ExternalName"
     end
+
+    def requires_publishing?
+      @definition["spec"]["type"] == "LoadBalancer"
+    end
+
+    def published?
+      @instance_data.dig('status', 'loadBalancer', 'ingress').present?
+    end
   end
 end

data/lib/kubernetes-deploy/render_task.rb

@@ -69,10 +69,10 @@ module KubernetesDeploy
       @logger.info("Rendering #{file_basename}...")
       file_content = File.read(File.join(@template_dir, filename))
       rendered_content = @renderer.render_template(filename, file_content)
-      implicit = true
-      YAML.parse_stream(rendered_content, "<rendered> #{filename}") { |d| implicit = d.implicit }
+      implicit = []
+      YAML.parse_stream(rendered_content, "<rendered> #{filename}") { |d| implicit << d.implicit }
       if rendered_content.present?
-        stream.puts "---\n" if implicit
+        stream.puts "---\n" if implicit.first
         stream.puts rendered_content
         @logger.info("Rendered #{file_basename}")
       else

data/lib/kubernetes-deploy/restart_task.rb

@@ -29,14 +29,15 @@ module KubernetesDeploy
       @max_watch_seconds = max_watch_seconds
     end
 
-    def perform(*args)
+    def run(*args)
       perform!(*args)
       true
     rescue FatalDeploymentError
       false
     end
+    alias_method :perform, :run
 
-    def perform!(deployments_names = nil, selector: nil)
+    def run!(deployments_names = nil, selector: nil, verify_result: true)
       start = Time.now.utc
       @logger.reset
 
@@ -47,16 +48,14 @@ module KubernetesDeploy
       @logger.phase_heading("Triggering restart by touching ENV[RESTARTED_AT]")
       patch_kubeclient_deployments(deployments)
 
-      @logger.phase_heading("Waiting for rollout")
-      resources = build_watchables(deployments, start)
-      ResourceWatcher.new(resources: resources, logger: @logger, operation_name: "restart",
-        timeout: @max_watch_seconds, namespace: @namespace, context: @context).run
-      failed_resources = resources.reject(&:deploy_succeeded?)
-      success = failed_resources.empty?
-      if !success && failed_resources.all?(&:deploy_timed_out?)
-        raise DeploymentTimeoutError
+      if verify_result
+        @logger.phase_heading("Waiting for rollout")
+        resources = build_watchables(deployments, start)
+        verify_restart(resources)
+      else
+        warning = "Result verification is disabled for this task"
+        @logger.summary.add_paragraph(ColorizedString.new(warning).yellow)
       end
-      raise FatalDeploymentError unless success
       StatsD.distribution('restart.duration', StatsD.duration(start), tags: tags('success', deployments))
       @logger.print_summary(:success)
     rescue DeploymentTimeoutError
@@ -69,6 +68,7 @@ module KubernetesDeploy
       @logger.print_summary(:failure)
       raise
     end
+    alias_method :perform!, :run!
 
     private
 
@@ -168,6 +168,17 @@ module KubernetesDeploy
       }
     end
 
+    def verify_restart(resources)
+      ResourceWatcher.new(resources: resources, logger: @logger, operation_name: "restart",
+        timeout: @max_watch_seconds, namespace: @namespace, context: @context).run
+      failed_resources = resources.reject(&:deploy_succeeded?)
+      success = failed_resources.empty?
+      if !success && failed_resources.all?(&:deploy_timed_out?)
+        raise DeploymentTimeoutError
+      end
+      raise FatalDeploymentError unless success
+    end
+
     def verify_config!
       task_config_validator = TaskConfigValidator.new(@task_config, kubectl, kubeclient_builder)
       unless task_config_validator.valid?

data/lib/kubernetes-deploy/runner_task.rb

@@ -8,6 +8,7 @@ require 'kubernetes-deploy/resource_cache'
 require 'kubernetes-deploy/resource_watcher'
 require 'kubernetes-deploy/kubernetes_resource'
 require 'kubernetes-deploy/kubernetes_resource/pod'
+require 'kubernetes-deploy/runner_task_config_validator'
 
 module KubernetesDeploy
   class RunnerTask
@@ -35,7 +36,11 @@ module KubernetesDeploy
       @logger.reset
 
       @logger.phase_heading("Initializing task")
-      validate_configuration(task_template, args)
+
+      @logger.info("Validating configuration")
+      verify_config!(task_template, args)
+      @logger.info("Using namespace '#{@namespace}' in context '#{@context}'")
+
       pod = build_pod(task_template, entrypoint, args, env_vars, verify_result)
       validate_pod(pod)
 
@@ -107,41 +112,14 @@ module KubernetesDeploy
       @logger.summary.add_paragraph(warning)
     end
 
-    def validate_configuration(task_template, args)
-      @logger.info("Validating configuration")
-      errors = []
-
-      if task_template.blank?
-        errors << "Task template name can't be nil"
-      end
-
-      if @namespace.blank?
-        errors << "Namespace can't be empty"
-      end
-
-      if args.blank?
-        errors << "Args can't be nil"
-      end
-
-      begin
-        kubeclient.get_namespace(@namespace) if @namespace.present?
-        @logger.info("Using namespace '#{@namespace}' in context '#{@context}'")
-
-      rescue Kubeclient::ResourceNotFoundError
-        errors << "Namespace was not found"
-      rescue Kubeclient::HttpError
-        errors << "Could not connect to kubernetes cluster"
-      rescue KubernetesDeploy::KubeclientBuilder::ContextMissingError
-        errors << "Could not connect to kubernetes cluster - context invalid"
-      end
-
-      unless errors.empty?
+    def verify_config!(task_template, args)
+      task_config_validator = RunnerTaskConfigValidator.new(task_template, args, @task_config, kubectl,
+        kubeclient_builder)
+      unless task_config_validator.valid?
         @logger.summary.add_action("Configuration invalid")
-        @logger.summary.add_paragraph(errors.map { |err| "- #{err}" }.join("\n"))
-        raise TaskConfigurationError, "Configuration invalid: #{errors.join(', ')}"
+        @logger.summary.add_paragraph([task_config_validator.errors].map { |err| "- #{err}" }.join("\n"))
+        raise KubernetesDeploy::TaskConfigurationError
       end
-
-      TaskConfigValidator.new(@task_config, kubectl, kubeclient_builder, only: [:validate_server_version]).valid?
     end
 
     def get_template(template_name)

data/lib/kubernetes-deploy/runner_task_config_validator.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+module KubernetesDeploy
+  class RunnerTaskConfigValidator < TaskConfigValidator
+    def initialize(template, args, *arguments)
+      super(*arguments)
+      @template = template
+      @args = args
+      @validations += %i(validate_template validate_args)
+    end
+
+    private
+
+    def validate_args
+      if @args.blank?
+        @errors << "Args can't be nil"
+      end
+    end
+
+    def validate_template
+      if @template.blank?
+        @errors << "Task template name can't be nil"
+      end
+    end
+  end
+end

data/lib/kubernetes-deploy/version.rb

@@ -1,4 +1,4 @@
 # frozen_string_literal: true
 module KubernetesDeploy
-  VERSION = "0.27.0"
+  VERSION = "0.28.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: kubernetes-deploy
 version: !ruby/object:Gem::Version
-  version: 0.27.0
+  version: 0.28.0
 platform: ruby
 authors:
 - Katrina Verey
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2019-09-05 00:00:00.000000000 Z
+date: 2019-09-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -143,6 +143,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.9.6
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.20.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.20.3
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -231,6 +245,7 @@ description: Kubernetes deploy scripts
 email:
 - ops-accounts+shipit@shopify.com
 executables:
+- krane
 - kubernetes-deploy
 - kubernetes-render
 - kubernetes-restart
@@ -250,7 +265,6 @@ files:
 - Gemfile
 - ISSUE_TEMPLATE.md
 - LICENSE.txt
-- NO-BINAUTH
 - README.md
 - Rakefile
 - bin/ci
@@ -258,11 +272,16 @@ files:
 - bin/test
 - dev.yml
 - dev/flamegraph-from-tests
+- exe/krane
 - exe/kubernetes-deploy
 - exe/kubernetes-render
 - exe/kubernetes-restart
 - exe/kubernetes-run
 - kubernetes-deploy.gemspec
+- lib/krane.rb
+- lib/krane/cli/krane.rb
+- lib/krane/cli/restart_command.rb
+- lib/krane/cli/version_command.rb
 - lib/kubernetes-deploy.rb
 - lib/kubernetes-deploy/bindings_parser.rb
 - lib/kubernetes-deploy/cluster_resource_discovery.rb
@@ -285,7 +304,6 @@ files:
 - lib/kubernetes-deploy/kubernetes_resource/custom_resource_definition.rb
 - lib/kubernetes-deploy/kubernetes_resource/daemon_set.rb
 - lib/kubernetes-deploy/kubernetes_resource/deployment.rb
-- lib/kubernetes-deploy/kubernetes_resource/elasticsearch.rb
 - lib/kubernetes-deploy/kubernetes_resource/horizontal_pod_autoscaler.rb
 - lib/kubernetes-deploy/kubernetes_resource/ingress.rb
 - lib/kubernetes-deploy/kubernetes_resource/job.rb
@@ -303,8 +321,6 @@ files:
 - lib/kubernetes-deploy/kubernetes_resource/service.rb
 - lib/kubernetes-deploy/kubernetes_resource/service_account.rb
 - lib/kubernetes-deploy/kubernetes_resource/stateful_set.rb
-- lib/kubernetes-deploy/kubernetes_resource/statefulservice.rb
-- lib/kubernetes-deploy/kubernetes_resource/topic.rb
 - lib/kubernetes-deploy/label_selector.rb
 - lib/kubernetes-deploy/oj.rb
 - lib/kubernetes-deploy/options_helper.rb
@@ -316,6 +332,7 @@ files:
 - lib/kubernetes-deploy/restart_task.rb
 - lib/kubernetes-deploy/rollout_conditions.rb
 - lib/kubernetes-deploy/runner_task.rb
+- lib/kubernetes-deploy/runner_task_config_validator.rb
 - lib/kubernetes-deploy/statsd.rb
 - lib/kubernetes-deploy/task_config.rb
 - lib/kubernetes-deploy/task_config_validator.rb

data/lib/kubernetes-deploy/kubernetes_resource/elasticsearch.rb

@@ -1,12 +0,0 @@
-# frozen_string_literal: true
-module KubernetesDeploy
-  class Elasticsearch < KubernetesResource
-    def deploy_succeeded?
-      super # success assumption, with warning
-    end
-
-    def deploy_failed?
-      false
-    end
-  end
-end

data/lib/kubernetes-deploy/kubernetes_resource/statefulservice.rb

@@ -1,12 +0,0 @@
-# frozen_string_literal: true
-module KubernetesDeploy
-  class Statefulservice < KubernetesResource
-    def deploy_succeeded?
-      super # success assumption, with warning
-    end
-
-    def deploy_failed?
-      false
-    end
-  end
-end

data/lib/kubernetes-deploy/kubernetes_resource/topic.rb

@@ -1,12 +0,0 @@
-# frozen_string_literal: true
-module KubernetesDeploy
-  class Topic < KubernetesResource
-    def deploy_succeeded?
-      super # success assumption, with warning
-    end
-
-    def deploy_failed?
-      false
-    end
-  end
-end