tobsch-krane 1.0.0

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,46 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at krane@shopify.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/CONTRIBUTING.md

@@ -0,0 +1,164 @@
+# Contributing to krane
+
+:+1::tada: First off, thanks for taking the time to contribute! :tada::+1:
+
+The following is a set of guidelines for contributing to krane. Please take a moment to read through them before submitting your first PR.
+
+
+#### Table Of Contents
+
+[Code of Conduct](#code-of-conduct)
+
+[What should I know before I get started?](#what-should-i-know-before-i-get-started)
+  * [High-level design decisions](#high-level-design-decisions)
+  * [Feature acceptance policies](#feature-acceptance-policies)
+  * [Adding a new resource type](#contributing-a-new-resource-type)
+  * [Contributor License Agreement](#contributor-license-agreement)
+
+[Development](#development)
+  * [Setup](#setup)
+  * [Running the test suite locally](#running-the-test-suite-locally)
+  * [Releasing a new version (Shopify employees)](#releasing-a-new-version-shopify-employees)
+  * [CI (External contributors)](#ci-external-contributors)
+## Code of Conduct
+
+This project and everyone participating in it are governed by the [Code of Conduct](https://github.com/Shopify/krane/blob/master/CODE_OF_CONDUCT.md).
+By participating, you are expected to uphold this code. Please report unacceptable
+behavior to [krane@shopify.com](mailto:krane@shopify.com).
+
+## Maintainers
+
+This project is currently under the stewardship of the Production Platform group at Shopify.
+The two primary maintainers are @knverey and @dturn. Approval from at least one primary maintainer is
+required for all significant feature proposals and code architecture changes. In general,
+two people must approve all non-trivial PRs.
+
+## What should I know before I get started?
+
+### High-level design decisions
+
+**Logging**
+
+Since we are primarily a CLI tool, logging is our entire user interface. Please think carefully about the information you log. Is it clear? Is it logged at the right time? Is it helpful?
+
+In particular, we need to ensure that every mutation we’ve done to the cluster is clearly described (and is something the operator wanted us to do!).
+
+We handle Kubernetes secrets, so it is critical that changes do not cause the contents of any secrets in the template set to be logged.
+
+**Project architecture**
+
+The main interface of this project is our five tasks: `DeployTask`, `GlobalDeployTask`, `RestartTask`, `RunnerTask`, and `RenderTask`. The code in these classes should be high-level abstractions, with implementation details encapsulated in other classes. The public interface of these tasks is a `run` method (and a `run!` equivalent), the body of which should read like a set of phases and steps. Note that non-task classes are considered internal and we reserve the right to change their API at any time.
+
+An important design principle of the tasks is that they should try to fail fast before touching the cluster if they will not succeed overall. Part of how we achieve this is by separating each task into phases, where the first phase simply gathers information and runs validations to determine what needs to be done and whether that will be able to succeed. In practice, this is the “Initializing <task>” phase for all tasks, plus the “Checking initial resource statuses” phase for DeployTask. Our users should be able to assume that these initial phases never modify their clusters.
+
+**Thread safety**
+
+This tool must be able to run concurrent deploys to different targets safely, including when used as a library. Each of those deploys will internally also use parallelism via ruby threads, as do our integration tests. This means all of our code must be thread-safe. Notably, our code must not modify the global namespace (e.g. environment variables, classes, class variables or constants), and all gems we depend on must also be thread-safe.
+
+ _Note_: Local tests do not run in parallel by default. To enable it, use `PARALLELIZE_ME=1 PARALLELISM=$NUM_THREADS`. Unit tests never run in parallel because they use mocha, which is not thread-safe (mocha cannot be used in integration tests).
+
+**Performance and the sync cycle**
+
+DeployTask must remain performant when given several hundred resources at a time, generating 1000+ pods. This means only `sync` methods can make calls to the Kubernetes API server during result verification. This both limits the number of API calls made and ensures a consistent view of the world within each polling cycle.
+
+
+
+### Feature acceptance policies
+
+**Our mission**
+
+This project's mission is to make it easy to ship changes to a Kubernetes namespace and understand the result. Features that introduce new classes of responsibility to the tool are not usually accepted.
+
+Deploys can be a very tempting place to cram features. Imagine a proposed feature actually fits better elsewhere—where might that be? (Examples: validator in CI, custom controller, initializer, pre-processing step in the CD pipeline, or even Kubernetes core)
+
+**Template rendering**
+
+The basic ERB renderer included with the tool is intended as a convenience feature for a better out-of-the box experience. Providing complex rendering capabilities is outside the scope of this project's mission, and enhancements in this area may be rejected.
+
+**Composability**
+
+This project strives to be composable with other tools in the ecosystem, such as renderers and validators. The deploy task must work with any Kubernetes templates provided to it, no matter how they were generated.
+
+**Universality**
+
+This project is open-source. Features tied to any specific organization (including Shopify) will be rejected.
+
+
+### Contributing a new resource type
+
+The list of fully supported types is effectively the list of classes found in `lib/krane/kubernetes_resource/`.
+
+This gem uses subclasses of `KubernetesResource` to implement custom success/failure detection logic for each resource type. If no subclass exists for a type you're deploying, the gem simply assumes `kubectl apply` succeeded (and prints a warning about this assumption). We're always looking to support more types! Here are the basic steps for contributing a new one:
+
+1. Create a file for your type in `lib/krane/kubernetes_resource/`
+2. Create a new class that inherits from `KubernetesResource`. Minimally, it should implement the following methods:
+    * `sync` -- Gather the data you'll need to determine `deploy_succeeded?` and `deploy_failed?`. The superclass's implementation fetches the corresponding resource, parses it and stores it in `@instance_data`. You can define your own implementation if you need something else.
+    * `deploy_succeeded?`
+    * `deploy_failed?`
+3. Adjust the `TIMEOUT` constant to an appropriate value for this type.
+4. Add the new class to list of resources in
+   [`deploy_task.rb`](https://github.com/Shopify/krane/blob/master/lib/krane/deploy_task.rb#L8)
+5. Add the new resource to the [prune whitelist](https://github.com/Shopify/krane/blob/master/lib/krane/deploy_task.rb#L81)
+6. Add a basic example of the type to the hello-cloud [fixture set](https://github.com/Shopify/krane/tree/master/test/fixtures/hello-cloud) and appropriate assertions to `#assert_all_up` in [`hello_cloud.rb`](https://github.com/Shopify/krane/blob/master/test/helpers/fixture_sets/hello_cloud.rb). This will get you coverage in several existing tests, such as `test_full_hello_cloud_set_deploy_succeeds`.
+7. Add tests for any edge cases you foresee.
+
+### Contributor License Agreement
+
+ New contributors will be required to sign [Shopify's Contributor License Agreement (CLA)](https://cla.shopify.com/).
+ There are two versions of the CLA: one for individuals and one for organizations.
+
+# Development
+
+## Setup
+
+If you work for Shopify, just run `dev up`, but otherwise:
+
+1. [Install kubectl version 1.10.0 or higher](https://kubernetes.io/docs/user-guide/prereqs/) and make sure it is in your path
+2. [Install minikube](https://kubernetes.io/docs/getting-started-guides/minikube/#installation) (required to run the test suite)
+3. [Install any required minikube drivers](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md) (on OS X, you may need the [hyperkit driver](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#hyperkit-driver)
+4. Check out the repo
+5. Run `bin/setup` to install dependencies
+
+To install this gem onto your local machine, run `bundle exec rake install`.
+
+
+
+## Running the test suite locally
+
+Using minikube:
+
+1. Start [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/#installation) (`minikube start [options]`).
+2. Make sure you have a context named "minikube" in your kubeconfig. Minikube adds this context for you when you run `minikube start`. You can check for it using `kubectl config get-contexts`.
+3. Run `bundle exec rake test` (or `dev test` if you work for Shopify).
+
+Using another local cluster:
+
+1. Start your cluster.
+2. Put the name of the context you want to use in a file named `.local-context` in the root of this project. For example: `echo "dind" > .local-context`.
+3. Run `bundle exec rake test` (or `dev test` if you work for Shopify).
+
+To make StatsD log what it would have emitted, run a test with `STATSD_ENV=development`.
+
+To see the full-color output of a specific integration test, you can use `PRINT_LOGS=1`. For example: `PRINT_LOGS=1 bundle exec ruby -I test test/integration/krane_deploy_test.rb -n/test_name/`.
+
+
+![test-output](screenshots/test-output.png)
+
+
+## Releasing a new version (Shopify employees)
+
+1. On a new branch, create a new heading in CHANGELOG.md for your version and move the entries from "Next" under it. Leave the "Next" heading in the file (this helps with the diff for rebases after the release).
+1. Make sure CHANGELOG.md includes all user-facing changes since the last release. Things like test changes or refactors do not need to be included.
+1. Update the version number in `version.rb`.
+1. Commit your changes with message "Version x.y.z" and open a PR.
+1. After merging your PR, deploy via [Shipit](https://shipit.shopify.io/shopify/kubernetes-deploy/rubygems). Shipit will automatically tag the release and upload the gem to [rubygems.org](https://rubygems.org/gems/krane).
+
+## CI (External contributors)
+
+Please make sure you run the tests locally before submitting your PR (see [Running the test suite locally](#running-the-test-suite-locally)). After reviewing your PR, a Shopify employee will trigger CI for you.
+
+#### Employees: Triggering CI for a contributed PR
+
+Go to the [krane pipeline](https://buildkite.com/shopify/kubernetes-deploy) and click "New Build". Use branch `external_contrib_ci` and the specific sha of the commit you want to build. Add `BUILDKITE_REFSPEC="refs/pull/${PR_NUM}/head"` in the Environment Variables section. Since CI is only visible to Shopify employees, you will need to provide any failing tests and output to the the contributor.
+
+<img width="350" alt="build external contrib PR" src="https://screenshot.click/2017-11-07--163728_7ovek-wrpwq.png">

data/Gemfile

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in krane.gemspec
+gemspec
+
+gem 'pry'
+gem 'pry-byebug'
+gem 'rubocop'
+gem 'timecop'
+gem 'byebug'
+gem 'codecov', require: false
+gem 'ruby-prof', require: false
+gem 'ruby-prof-flamegraph', require: false
+gem 'minitest-reporters'
+gem 'yard', require: false

data/ISSUE_TEMPLATE.md

@@ -0,0 +1,25 @@
+<!-- Please erase any parts of this template not applicable to your issue. -->
+
+# Bug report
+
+[Description of the bug]
+
+**Expected behavior:** [What you expected to happen]
+
+**Actual behavior:** [What actually happened]
+
+**Version(s) affected:** [run `krane version`]
+
+### Steps to Reproduce
+
+1. [First Step]
+2. [Second Step]
+3. [and so on...]
+
+# Feature request
+
+- [ ] If the maintainers agree with the feature as described here, I intend to submit a Pull Request myself.<sup>1</sup>
+
+**Proposal:** [provide details on the behaviour you'd like to see and why it would be useful]
+
+<sup><small>1</small></sup> <sub>This is the quickest way to get a new feature! We reserve the right to close feature requests, even ones we like, if the proposer does not intend to contribute to the feature and it doesn't fit in our current roadmap.</sub>

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 Kir Shatrov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,655 @@
+# krane [![Build status](https://badge.buildkite.com/61937e40a1fc69754d9d198be120543d6de310de2ba8d3cb0e.svg?branch=master)](https://buildkite.com/shopify/kubernetes-deploy) [![codecov](https://codecov.io/gh/Shopify/kubernetes-deploy/branch/master/graph/badge.svg)](https://codecov.io/gh/Shopify/kubernetes-deploy)
+
+> This project used to be called `kubernetes-deploy`. Check out our [migration guide](https://github.com/Shopify/krane/blob/master/1.0-Upgrade.md) for more information including details about breaking changes.
+
+
+`krane` is a command line tool that helps you ship changes to a Kubernetes namespace and understand the result. At Shopify, we use it within our much-beloved, open-source [Shipit](https://github.com/Shopify/shipit-engine#kubernetes) deployment app.
+
+Why not just use the standard `kubectl apply` mechanism to deploy? It is indeed a fantastic tool; `krane` uses it under the hood! However, it leaves its users with some burning questions: _What just happened?_ _Did it work?_
+
+Especially in a CI/CD environment, we need a clear, actionable pass/fail result for each deploy. Providing this was the foundational goal of `krane`, which has grown to support the following core features:
+
+​:eyes:  Watches the changes you requested to make sure they roll out successfully.
+
+:interrobang: Provides debug information for changes that failed.
+
+:1234:  Predeploys certain types of resources (e.g. ConfigMap, PersistentVolumeClaim) to make sure the latest version will be available when resources that might consume them (e.g. Deployment) are deployed.
+
+:closed_lock_with_key:  [Creates Kubernetes secrets from encrypted EJSON](#deploying-kubernetes-secrets-from-ejson), which you can safely commit to your repository
+
+​:running: [Running tasks at the beginning of a deploy](#running-tasks-at-the-beginning-of-a-deploy) using bare pods (example use case: Rails migrations)
+
+If you need the ability to render dynamic values in templates before deploying, you can use [krane render](#krane-render). Alongside that, this repo also includes tools for [running tasks](#krane-run) and [restarting deployments](#krane-restart).
+
+
+
+![demo-deploy.gif](screenshots/deploy-demo.gif)
+
+
+
+![missing-secret-fail](screenshots/missing-secret-fail.png)
+
+
+--------
+
+
+
+## Table of contents
+
+**KRANE DEPLOY**
+* [Prerequisites](#prerequisites)
+* [Installation](#installation)
+* [Usage](#usage)
+  * [Using templates and variables](#using-templates-and-variables)
+  * [Customizing behaviour with annotations](#customizing-behaviour-with-annotations)
+  * [Running tasks at the beginning of a deploy](#running-tasks-at-the-beginning-of-a-deploy)
+  * [Deploying Kubernetes secrets (from EJSON)](#deploying-kubernetes-secrets-from-ejson)
+  * [Deploying custom resources](#deploying-custom-resources)
+* [Walk through the steps of a deployment](#deploy-walkthrough)
+
+**KRANE GLOBAL DEPLOY**
+* [Usage](#usage-1)
+
+**KRANE RESTART**
+* [Usage](#usage-2)
+
+**KRANE RUN**
+* [Prerequisites](#prerequisites-1)
+* [Usage](#usage-3)
+
+**KRANE RENDER**
+* [Prerequisites](#prerequisites-2)
+* [Usage](#usage-4)
+
+**CONTRIBUTING**
+* [Contributing](#contributing)
+* [Code of Conduct](#code-of-conduct)
+* [License](#license)
+
+
+----------
+
+
+
+## Prerequisites
+
+* Ruby 2.4+
+* Your cluster must be running Kubernetes v1.11.0 or higher<sup>1</sup>
+
+<sup>1</sup> We run integration tests against these Kubernetes versions. You can find our
+official compatibility chart below.
+
+| Kubernetes version | Last officially supported in gem version |
+| :----------------: | :-------------------: |
+|        1.5         |        0.11.2         |
+|        1.6         |        0.15.2         |
+|        1.7         |        0.20.6         |
+|        1.8         |        0.21.1         |
+|        1.9         |        0.24.0         |
+|        1.10        |        0.27.0         |
+
+## Installation
+
+1. [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl) (requires v1.11.0 or higher) and make sure it is available in your $PATH
+2. Set up your [kubeconfig file](https://kubernetes.io/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) for access to your cluster(s).
+3. `gem install krane`
+
+
+
+
+## Usage
+
+`krane deploy <app's namespace> <kube context>`
+
+*Environment variables:*
+
+- `$KUBECONFIG`: points to one or multiple valid kubeconfig files that include the context you want to deploy to. File names are separated by colon for Linux and Mac, and semi-colon for Windows. If ommitted, will use the Kubernetes default of `~/.kube/config`.
+- `$GOOGLE_APPLICATION_CREDENTIALS`: points to the credentials for an authenticated service account (required if your kubeconfig `user`'s auth provider is GCP)
+
+
+*Options:*
+
+Refer to `krane help` for the authoritative set of options.
+
+
+- `--filenames / -f [PATHS]`: Accepts a list of directories and/or filenames to specify the set of directories/files that will be deployed.
+- `--stdin`: Read from STDIN. Can be combined with `-f` Example: `cat templates_from_stdin/*.yml | krane deploy ns ctx -f path/to/dir path/to/file.yml --stdin`
+- `--no-prune`: Skips pruning of resources that are no longer in your Kubernetes template set. Not recommended, as it allows your namespace to accumulate cruft that is not reflected in your deploy directory.
+- `--global-timeout=duration`: Raise a timeout error if it takes longer than _duration_ for any
+resource to deploy.
+- `--selector`: Instructs krane to only prune resources which match the specified label selector, such as `environment=staging`. If you use this option, all resource templates must specify matching labels. See [Sharing a namespace](#sharing-a-namespace) below.
+- `--no-verify-result`: Skip verification that workloads correctly deployed.
+- `--protected-namespaces=default kube-system kube-public`: Fail validation if a deploy is targeted at a protected namespace.
+- `--verbose-log-prefix`: Add [context][namespace] to the log prefix
+
+
+> **NOTICE**: Deploy Secret resources at your own risk. Although we will fix any reported leak vectors with urgency, we cannot guarantee that sensitive information will never be logged.
+
+### Sharing a namespace
+
+By default, krane will prune any resources in the target namespace which have the `kubectl.kubernetes.io/last-applied-configuration` annotation and are not a result of the current deployment process, on the assumption that there is a one-to-one relationship between application deployment and namespace, and that a deployment provisions all relevant resources in the namespace.
+
+If you need to, you may specify `--no-prune` to disable all pruning behaviour, but this is not recommended.
+
+If you need to share a namespace with resources which are managed by other tools or indeed other krane deployments, you can supply the `--selector` option, such that only resources with labels matching the selector are considered for pruning.
+
+### Using templates
+
+All templates must be YAML formatted.
+We recommended storing each app's templates in a single directory, `{app root}/config/deploy/{env}`. However, you may use multiple directories.
+
+If you want dynamic templates, you may render ERB with `krane render` and then pipe that result to `krane deploy --stdin`. `krane deploy` supports using both `--filenames` and `--stdin` together.
+
+### Customizing behaviour with annotations
+- `krane.shopify.io/timeout-override`: Override the tool's hard timeout for one specific resource. Both full ISO8601 durations and the time portion of ISO8601 durations are valid. Value must be between 1 second and 24 hours.
+  - _Example values_: 45s / 3m / 1h / PT0.25H
+  - _Compatibility_: all resource types
+- `krane.shopify.io/required-rollout`: Modifies how much of the rollout needs to finish
+before the deployment is considered successful.
+  - _Compatibility_: Deployment
+  - `full`: The deployment is successful when all pods in the new `replicaSet` are ready.
+  - `none`: The deployment is successful as soon as the new `replicaSet` is created for the deployment.
+  - `maxUnavailable`: The deploy is successful when minimum availability is reached in the new `replicaSet`.
+  In other words, the number of new pods that must be ready is equal to `spec.replicas` - `strategy.RollingUpdate.maxUnavailable`
+  (converted from percentages by rounding up, if applicable). This option is only valid for deployments
+  that use the `RollingUpdate` strategy.
+  - Percent (e.g. 90%): The deploy is successful when the number of new pods that are ready is equal to
+  `spec.replicas` * Percent.
+- `krane.shopify.io/predeployed`: Causes a Custom Resource to be deployed in the pre-deploy phase.
+  - _Compatibility_: Custom Resource Definition
+  - _Default_: `true`
+  - `true`: The custom resource will be deployed in the pre-deploy phase.
+  - All other values: The custom resource will be deployed in the main deployment phase.
+
+
+### Running tasks at the beginning of a deploy
+
+To run a task in your cluster at the beginning of every deploy, simply include a `Pod` template in your deploy directory. `krane` will first deploy any `ConfigMap` and `PersistentVolumeClaim` resources present in the provided templates, followed by any such pods. If the command run by one of these pods fails (i.e. exits with a non-zero status), the overall deploy will fail at this step (no other resources will be deployed).
+
+*Requirements:*
+
+* The pod's name should include `<%= deployment_id %>` to ensure that a unique name will be used on every deploy (the deploy will fail if a pod with the same name already exists).
+* The pod's `spec.restartPolicy` must be set to `Never` so that it will be run exactly once. We'll fail the deploy if that run exits with a non-zero status.
+* The pod's `spec.activeDeadlineSeconds` should be set to a reasonable value for the performed task (not required, but highly recommended)
+
+A simple example can be found in the test fixtures: [test/fixtures/hello-cloud/unmanaged-pod-1.yml.erb](test/fixtures/hello-cloud/unmanaged-pod-1.yml.erb).
+
+The logs of all pods run in this way will be printed inline. If there is only one pod, the logs will be streamed in real-time. If there are multiple, they will be fetched when the pod terminates.
+
+![migrate-logs](screenshots/migrate-logs.png)
+
+
+
+### Deploying Kubernetes secrets (from EJSON)
+
+**Note: If you're a Shopify employee using our cloud platform, this setup has already been done for you. Please consult the CloudPlatform User Guide for usage instructions.**
+
+Since their data is only base64 encoded, Kubernetes secrets should not be committed to your repository. Instead, `krane` supports generating secrets from an encrypted [ejson](https://github.com/Shopify/ejson) file in your template directory. Here's how to use this feature:
+
+1. Install the ejson gem: `gem install ejson`
+2. Generate a new keypair: `ejson keygen` (prints the keypair to stdout)
+3. Create a Kubernetes secret in your target namespace with the new keypair: `kubectl create secret generic ejson-keys --from-literal=YOUR_PUBLIC_KEY=YOUR_PRIVATE_KEY --namespace=TARGET_NAMESPACE`
+>Warning: Do *not* use `apply` to create the `ejson-keys` secret. krane will fail if `ejson-keys` is prunable. This safeguard is to protect against the accidental deletion of your private keys.
+4. (optional but highly recommended) Back up the keypair somewhere secure, such as a password manager, for disaster recovery purposes.
+5. In your template directory (alongside your Kubernetes templates), create `secrets.ejson` with the format shown below. The `_type` key should have the value “kubernetes.io/tls” for TLS secrets and “Opaque” for all others. The `data` key must be a json object, but its keys and values can be whatever you need.
+
+```json
+{
+  "_public_key": "YOUR_PUBLIC_KEY",
+  "kubernetes_secrets": {
+    "catphotoscom": {
+      "_type": "kubernetes.io/tls",
+      "data": {
+        "tls.crt": "cert-data-here",
+        "tls.key": "key-data-here"
+      }
+    },
+    "monitoring-token": {
+      "_type": "Opaque",
+      "data": {
+        "api-token": "token-value-here"
+      }
+    }
+  }
+}
+```
+
+6. Encrypt the file: `ejson encrypt /PATH/TO/secrets.ejson`
+7. Commit the encrypted file and deploy. The deploy will create secrets from the data in the `kubernetes_secrets` key. The ejson file must be included in the resources passed to `--filenames` it can not be read through stdin.
+
+**Note**: Since leading underscores in ejson keys are used to skip encryption of the associated value, `krane` will strip these leading underscores when it creates the keys for the Kubernetes secret data. For example, given the ejson data below, the `monitoring-token` secret will have keys `api-token` and `property` (_not_ `_property`):
+
+```json
+{
+  "_public_key": "YOUR_PUBLIC_KEY",
+  "kubernetes_secrets": {
+    "monitoring-token": {
+      "_type": "kubernetes.io/tls",
+      "data": {
+        "api-token": "EJ[ENCRYPTED]",
+        "_property": "some unencrypted value"
+      }
+    }
+  }
+```
+
+**A warning about using EJSON secrets with `--selector`**: when using EJSON to generate `Secret` resources and specifying a `--selector` for deployment, the labels from the selector are automatically added to the `Secret`. If _the same_ EJSON file is deployed to the same namespace using different selectors, this will cause the resource to thrash - even if the contents of the secret were the same, the resource has different labels on each deploy.
+
+### Deploying custom resources
+
+By default, krane does not check the status of custom resources; it simply assumes that they deployed successfully. In order to meaningfully monitor the rollout of custom resources, krane supports configuring pass/fail conditions using annotations on CustomResourceDefinitions (CRDs).
+
+*Requirements:*
+
+* The custom resource must expose a `status` subresource with an `observedGeneration` field.
+* The `krane.shopify.io/instance-rollout-conditions` annotation must be present on the CRD that defines the custom resource.
+* (optional) The `krane.shopify.io/instance-timeout` annotation can be added to the CRD that defines the custom resource to override the global default timeout for all instances of that resource. This annotation can use ISO8601 format or unprefixed ISO8601 time components (e.g. '1H', '60S').
+
+#### Specifying pass/fail conditions
+
+The presence of a valid `krane.shopify.io/instance-rollout-conditions` annotation on a CRD will cause krane to monitor the rollout of all instances of that custom resource. Its value can either be `"true"` (giving you the defaults described in the next section) or a valid JSON string with the following format:
+```
+'{
+  "success_conditions": [
+    { "path": <JsonPath expression>, "value": <target value> }
+    ... more success conditions
+  ],
+  "failure_conditions": [
+    { "path": <JsonPath expression>, "value": <target value> }
+    ... more failure conditions
+  ]
+}'
+```
+
+For all conditions, `path` must be a valid JsonPath expression that points to a field in the custom resource's status. `value` is the value that must be present at `path` in order to fulfill a condition. For a deployment to be successful, _all_ `success_conditions` must be fulfilled. Conversely, the deploy will be marked as failed if _any one of_ `failure_conditions` is fulfilled. `success_conditions` are mandatory, but `failure_conditions` can be omitted (the resource will simply time out if it never reaches a successful state).
+
+In addition to `path` and `value`, a failure condition can also contain `error_msg_path` or `custom_error_msg`. `error_msg_path` is a JsonPath expression that points to a field you want to surface when a failure condition is fulfilled. For example, a status condition may expose a `message` field that contains a description of the problem it encountered. `custom_error_msg` is a string that can be used if your custom resource doesn't contain sufficient information to warrant using `error_msg_path`. Note that `custom_error_msg` has higher precedence than `error_msg_path` so it will be used in favor of `error_msg_path` when both fields are present.
+
+**Warning:**
+
+You **must** ensure that your custom resource controller sets `.status.observedGeneration` to match the observed `.metadata.generation` of the monitored resource once its sync is complete. If this does not happen, krane will not check success or failure conditions and the deploy will time out.
+
+#### Example
+
+As an example, the following is the default configuration that will be used if you set `krane.shopify.io/instance-rollout-conditions: "true"` on the CRD that defines the custom resources you wish to monitor:
+
+```
+'{
+  "success_conditions": [
+    {
+      "path": "$.status.conditions[?(@.type == \"Ready\")].status",
+      "value": "True",
+    },
+  ],
+  "failure_conditions": [
+    {
+      "path": '$.status.conditions[?(@.type == \"Failed\")].status',
+      "value": "True",
+      "error_msg_path": '$.status.conditions[?(@.type == \"Failed\")].message',
+    },
+  ],
+}'
+```
+
+The paths defined here are based on the [typical status properties](https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#typical-status-properties) as defined by the Kubernetes community. It expects the `status` subresource to contain a `conditions` array whose entries minimally specify `type`, `status`, and `message` fields.
+
+You can see how these conditions relate to the following resource:
+
+```
+apiVersion: stable.shopify.io/v1
+kind: Example
+metadata:
+  generation: 2
+  name: example
+  namespace: namespace
+spec:
+  ...
+status:
+  observedGeneration: 2
+  conditions:
+  - type: "Ready"
+    status: "False"
+    reason: "exampleNotReady"
+    message: "resource is not ready"
+  - type: "Failed"
+    status: "True"
+    reason: "exampleFailed"
+    message: "resource is failed"
+```
+
+- `observedGeneration == metadata.generation`, so krane will check this resource's success and failure conditions.
+- Since `$.status.conditions[?(@.type == "Ready")].status == "False"`, the resource is not considered successful yet.
+- `$.status.conditions[?(@.type == "Failed")].status == "True"` means that a failure condition has been fulfilled and the resource is considered failed.
+- Since `error_msg_path` is specified, krane will log the contents of `$.status.conditions[?(@.type == "Failed")].message`, which in this case is: `resource is failed`.
+
+### Deploy walkthrough
+
+Let's walk through what happens when you run the `deploy` task with [this directory of templates](https://github.com/Shopify/krane/tree/master/test/fixtures/hello-cloud). This particular example uses ERB templates as well, so we'll use the [krane render](#krane-render) task to achieve that.
+
+You can test this out for yourself by running the following command:
+
+```bash
+krane render -f test/fixtures/hello-cloud --current-sha 1 | krane deploy my-namespace my-k8s-cluster --stdin
+```
+
+As soon as you run this, you'll start seeing some output being streamed to STDERR.
+
+#### Phase 1: Initializing deploy
+
+In this phase, we:
+
+- Perform basic validation to ensure we can proceed with the deploy. This includes checking if we can reach the context, if the context is valid, if the namespace exists within the context, and more. We try to validate as much as we can before trying to ship something because we want to avoid having an incomplete deploy in case of a failure (this is especially important because there's no rollback support).
+- List out all the resources we want to deploy (as described in the template files we used).
+
+#### Phase 2: Checking initial resource statuses
+
+In this phase, we check resource statuses. For each resource listed in the previous step, we check Kubernetes for their status; in the first deploy this might show a bunch of items as "Not Found", but for the deploy of a new version, this is an example of what it could look like:
+
+```
+Certificate/services-foo-tls     Exists
+Cloudsql/foo-production          Provisioned
+Deployment/jobs                  3 replicas, 3 updatedReplicas, 3 availableReplicas
+Deployment/web                   3 replicas, 3 updatedReplicas, 3 availableReplicas
+Ingress/web                      Created
+Memcached/foo-production         Healthy
+Pod/db-migrate-856359            Unknown
+Pod/upload-assets-856359         Unknown
+Redis/foo-production             Healthy
+Service/web                      Selects at least 1 pod
+```
+
+The next phase might be either "Predeploying priority resources" (if there's any) or "Deploying all resources". In this example we'll go through the former, as we do have predeployable resources.
+
+#### Phase 3: Predeploying priority resources
+
+This is the first phase that could modify the cluster.
+
+In this phase we predeploy certain types of resources (e.g. `ConfigMap`, `PersistentVolumeClaim`, `Secret`, ...) to make sure the latest version will be available when resources that might consume them (e.g. `Deployment`) are deployed. This phase will be skipped if the templates don't include any resources that would need to be predeployed.
+
+When this runs, we essentially run `kubectl apply` on those templates and periodically check the cluster for the current status of each resource so we can display error or success information. This will look different depending on the type of resource. If you're running the command described above, you should see something like this in the output:
+
+```
+Deploying ConfigMap/hello-cloud-configmap-data (timeout: 30s)
+Successfully deployed in 0.2s: ConfigMap/hello-cloud-configmap-data
+
+Deploying PersistentVolumeClaim/hello-cloud-redis (timeout: 300s)
+Successfully deployed in 3.3s: PersistentVolumeClaim/hello-cloud-redis
+
+Deploying Role/role (timeout: 300s)
+Don't know how to monitor resources of type Role. Assuming Role/role deployed successfully.
+Successfully deployed in 0.2s: Role/role
+```
+
+As you can see, different types of resources might have different timeout values and different success criteria; in some specific cases (such as with Role) we might not know how to confirm success or failure, so we use a higher timeout value and assume it did work.
+
+#### Phase 4: Deploying all resources
+
+In this phase, we:
+
+- Deploy all resources found in the templates, including resources that were predeployed in the previous step (which should be treated as a no-op by Kubernetes). We deploy everything so the pruning logic (described below) doesn't remove any predeployed resources.
+- Prune resources not found in the templates (you can disable this by using `--no-prune`).
+
+Just like in the previous phase, we essentially run `kubectl apply` on those templates and periodically check the cluster for the current status of each resource so we can display error or success information.
+
+If pruning is enabled (which, again, is the default), any [kind not listed in the blacklist](https://github.com/Shopify/krane/blob/master/lib/krane/cluster_resource_discovery.rb#L20) that we can find in the namespace but not in the templates will be removed. A particular message about pruning will be printed in the next phase if any resource matches this criteria.
+
+#### Result
+
+The result section will show:
+- A global status: if **all** resources were deployed successfully, this will show up as "SUCCESS"; if at least one resource failed to deploy (due to an error or timeout), this will show up as "FAILURE".
+- A list of resources and their individual status: this will show up as something like "Available", "Created", and "1 replica, 1 availableReplica, 1 readyReplica".
+
+At this point the command also returns a status code:
+- If it was a success, `0`
+- If there was a timeout, `70`
+- If any other failure happened, `1`
+
+**On timeouts**: It's important to notice that a single resource timeout or a global deploy timeout doesn't necessarily mean that the operation failed. Since Kubernetes updates are asynchronous, maybe something was just too slow to return in the configured time; in those cases, usually running the deploy again might work (that should be a no-op for most - if not all - resources).
+
+# krane global deploy
+
+Ship non-namespaced resources to a cluster
+
+krane global-deploy (accessible through the Ruby API as Krane::GlobalDeployTask) can deploy global (non-namespaced) resources such as PersistentVolume, Namespace, and CustomResourceDefinition.
+Its interface is very similar to krane deploy.
+
+## Usage
+
+`krane global-deploy <kube context>`
+
+```bash
+$ cat my-template.yml
+    apiVersion: storage.k8s.io/v1
+    kind: StorageClass
+    metadata:
+      name: testing-storage-class
+      labels:
+        app: krane
+    provisioner: kubernetes.io/no-provisioner
+
+$ krane global-deploy my-k8s-context -f my-template.yml --selector app=krane
+```
+
+*Options:*
+
+Refer to `krane global-deploy help` for the authoritative set of options.
+
+- `--filenames / -if [PATHS]`: Accepts a list of directories and/or filenames to specify the set of directories/files that will be deployed
+- `--stdin`: Read from STDIN. Can be combined with `-f`
+- `--no-prune`: Skips pruning of resources that are no longer in your Kubernetes template set. Not recommended, as it allows your namespace to accumulate cruft that is not reflected in your deploy directory.
+- `--selector`: Instructs krane to only prune resources which match the specified label selector, such as `environment=staging`. By using this option, all resource templates must specify matching labels. See [Sharing a namespace](#sharing-a-namespace) below.
+- `--global-timeout=duration`: Raise a timeout error if it takes longer than _duration_ for any
+resource to deploy.
+- `--no-verify-result`: Skip verification that resources correctly deployed.
+
+# krane restart
+
+`krane restart` is a tool for restarting all of the pods in one or more deployments. It triggers the restart by touching the `RESTARTED_AT` environment variable in the deployment's podSpec. The rollout strategy defined for each deployment will be respected by the restart.
+
+## Usage
+
+**Option 1: Specify the deployments you want to restart**
+
+The following command will restart all pods in the `web` and `jobs` deployments:
+
+`krane restart <kube namespace> <kube context> --deployments=web jobs`
+
+
+**Option 2: Annotate the deployments you want to restart**
+
+Add the annotation `shipit.shopify.io/restart` to all the deployments you want to target, like this:
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: web
+  annotations:
+    shipit.shopify.io/restart: "true"
+```
+
+With this done, you can use the following command to restart all of them:
+
+`krane restart <kube namespace> <kube context>`
+
+*Options:*
+
+Refer to `krane help restart` for the authoritative set of options.
+
+- `--selector`: Only restarts Deployments which match the specified Kubernetes resource selector.
+- `--deployments`: Restart specific Deployment resources by name.
+- `--global-timeout=duration`: Raise a timeout error if it takes longer than _duration_ for any
+resource to restart.
+- `--no-verify-result`: Skip verification that workloads correctly restarted.
+
+# krane run
+
+`krane run` is a tool for triggering a one-off job, such as a rake task, _outside_ of a deploy.
+
+
+
+## Prerequisites
+
+* You've already deployed a [`PodTemplate`](https://v1-10.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#podtemplate-v1-core) object with field `template` containing a `Pod` specification that does not include the `apiVersion` or `kind` parameters. An example is provided in this repo in `test/fixtures/hello-cloud/template-runner.yml`.
+* The `Pod` specification in that template has a container named `task-runner`.
+
+Based on this specification `krane run` will create a new pod with the entrypoint of the `task-runner ` container overridden with the supplied arguments.
+
+
+
+## Usage
+
+`krane run <kube namespace> <kube context> --arguments=<arguments> --command=<command> --template=<template name>`
+
+*Options:*
+
+* `--template=TEMPLATE`:  Specifies the name of the PodTemplate to use (default is `task-runner-template` if this option is not set).
+* `--env-vars=ENV_VARS`: Accepts a list of environment variables to be added to the pod template. For example, `--env-vars="ENV=VAL ENV2=VAL2"` will make `ENV` and `ENV2` available to the container.
+* `--command=`: Override the default command in the container image.
+* `--no-verify-result`: Skip verification of pod success
+* `--global-timeout=duration`: Raise a timeout error if the pod runs for longer than the specified duration
+* `--arguments:`: Override the default arguments for the command with a space-separated list of arguments
+
+
+# krane render
+
+`krane render` is a tool for rendering ERB templates to raw Kubernetes YAML. It's useful for outputting YAML that can be passed to other tools, for validation or introspection purposes.
+
+
+## Prerequisites
+
+ * `krane render` does __not__ require a running cluster or an active kubernetes context, which is nice if you want to run it in a CI environment, potentially alongside something like https://github.com/garethr/kubeval to make sure your configuration is sound.
+
+## Usage
+
+To render all templates in your template dir, run:
+
+```
+krane render -f ./path/to/template/dir
+```
+
+To render some templates in a template dir, run krane render with the names of the templates to render:
+
+```
+krane render -f ./path/to/template/dir/this-template.yaml.erb
+```
+
+To render a template in a template dir and output it to a file, run krane render with the name of the template and redirect the output to a file:
+
+```
+krane render -f ./path/to/template/dir/template.yaml.erb > template.yaml
+```
+
+*Options:*
+
+- `--filenames / -f [PATHS]`: Accepts a list of directories and/or filenames to specify the set of directories/files that will be deployed.
+- `--stdin`: Read from STDIN. Can be combined with `-f` Example: `cat templates_from_stdin/*.yml | krane render -f path/to/dir path/to/file.yml --stdin`
+- `--bindings=BINDINGS`: Makes additional variables available to your ERB templates. For example, `krane render --bindings=color=blue size=large -f some-template.yaml.erb` will expose `color` and `size` to `some-template.yaml.erb`.
+- `--current-sha`: Expose SHA `current_sha` in ERB bindings
+
+You can add additional variables using the `--bindings=BINDINGS` option which can be formated as a string, JSON string or path to a JSON or YAML file. Complex JSON or YAML data will be converted to a Hash for use in templates. To load a file the argument should include the relative
+file path prefixed with an `@` sign. An argument error will be raised if the string argument cannot be parsed, the referenced file does not include a
+ valid extension (`.json`, `.yaml` or `.yml`) or the referenced file does not exist.
+
+#### Bindings examples
+
+```
+# Comma separated string. Exposes, 'color' and 'size'
+$ krane render --bindings=color=blue size=large
+
+# JSON string. Exposes, 'color' and 'size'
+$ krane render --bindings='{"color":"blue","size":"large"}'
+
+# Load JSON file from ./config
+$ krane render --bindings='@config/production.json'
+
+# Load YAML file from ./config (.yaml or yml supported)
+$ krane render --bindings='@config/production.yaml'
+```
+
+#### Using partials
+
+`krane` supports composing templates from so called partials in order to reduce duplication in Kubernetes YAML files. Given a directory `DIR`, partials are searched for in `DIR/partials`and in 'DIR/../partials', in that order. They can be embedded in other ERB templates using the helper method `partial`. For example, let's assume an application needs a number of different CronJob resources, one could place a template called `cron` in one of those directories and then use it in the main deployment.yaml.erb like so:
+
+```yaml
+<%= partial "cron", name: "cleanup",   schedule: "0 0 * * *", args: %w(cleanup),    cpu: "100m", memory: "100Mi" %>
+<%= partial "cron", name: "send-mail", schedule: "0 0 * * *", args: %w(send-mails), cpu: "200m", memory: "256Mi" %>
+```
+
+Inside a partial, parameters can be accessed as normal variables, or via a hash called `locals`. Thus, the `cron` template could like this:
+
+```yaml
+---
+apiVersion: batch/v1beta1
+kind: CronJob
+metadata:
+  name: cron-<%= name %>
+spec:
+  schedule: <%= schedule %>
+    successfulJobsHistoryLimit: 3
+    failedJobsHistoryLimit: 3
+    concurrencyPolicy: Forbid
+    jobTemplate:
+      spec:
+        template:
+          spec:
+            containers:
+            - name: cron-<%= name %>
+              image: ...
+              args: <%= args %>
+              resources:
+                requests:
+                  cpu: "<%= cpu %>"
+                  memory: <%= memory %>
+            restartPolicy: OnFailure
+```
+
+Both `.yaml.erb` and `.yml.erb` file extensions are supported. Templates must refer to the bare filename (e.g. use `partial: 'cron'` to reference `cron.yaml.erb`).
+
+##### Limitations when using partials
+
+Partials can be included almost everywhere in ERB templates. Note: when using a partial to insert additional key-value pairs to a map you must use [YAML merge keys](http://yaml.org/type/merge.html). For example, given a partial `p` defining two fields 'a' and 'b',
+
+```yaml
+a: 1
+b: 2
+```
+
+you cannot do this:
+
+```yaml
+x: yz
+<%= partial 'p' %>
+```
+
+hoping to get
+
+```yaml
+x: yz
+a: 1
+b: 2
+
+but you can do:
+
+```yaml
+<<: <%= partial 'p' %>
+x: yz
+```
+
+This is a limitation of the current implementation.
+
+# Contributing
+
+We :heart: contributors! To make it easier for you and us we've written a
+[Contributing Guide](https://github.com/Shopify/krane/blob/master/CONTRIBUTING.md)
+
+
+You can also reach out to us on our slack channel, #krane, at https://kubernetes.slack.com. All are welcome!
+
+## Code of Conduct
+Everyone is expected to follow our [Code of Conduct](https://github.com/Shopify/krane/blob/master/CODE_OF_CONDUCT.md).
+
+
+# License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).