kubernetes-deploy 0.25.0 → 0.26.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2f35ca56d7b0ec844c55ff82157238409f2c21db3a9d83f870a73cc3305ee0b8
-  data.tar.gz: a9d9e566c21f699e9f066cd5bd505cd9c9ef9466a0180bb7563e9004211a8184
+  metadata.gz: 75e707a9039e39eacc8206308b5bd0a8a189480cf389763f3a0667c05c975e9d
+  data.tar.gz: d6746729e5889a4b1667385ee9705b61a78ee86d9079d86ccbd38f0f87ddb113
 SHA512:
-  metadata.gz: 2c3e2b2a8267d997e7d8859d6238d70f8a052bb423c8526f6e0f0490efe9de97da54ec1de0737f229f2682b04ba0b385ed2cc54caea3cc8276536026abb660cd
-  data.tar.gz: 98a21af1679cec75a00a6f81981b5a092888113c81f74d03eb967ba07da8b722db06c1394202a2de0fb943df5d784b3e95de15f653682e9c4555a4b595018a45
+  metadata.gz: 618d63d8f2b30607e8610348e9383355646b5446cce4941bf326eea9e82641c007a13788ced9f2a6dee0e047638a90215cd3c603913271ddc329ebec34604abe
+  data.tar.gz: 6c8cfef5dc1320701046bef897c164703e94a50bf9a44c95b7d3b24016f395297707869e488ce55d8c1920c8f561235433b4029cb58d901070df78ddfd879b14

data/.buildkite/pipeline.nightly.yml

@@ -1,11 +1,5 @@
-shared: &shared
-  retry:
-     automatic:
-       - exit_status: "*"
-         limit: 1
 steps:
   - name: 'Run Test Suite (:kubernetes: 1.13-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: k8s-ci
@@ -13,7 +7,6 @@ steps:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.13-latest
   - name: 'Run Test Suite (:kubernetes: 1.12-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: k8s-ci
@@ -21,7 +14,6 @@ steps:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.12-latest
   - name: 'Run Test Suite (:kubernetes: 1.11-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: k8s-ci
@@ -29,7 +21,6 @@ steps:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.11-latest
   - name: 'Run Test Suite (:kubernetes: 1.10-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: minikube-ci

data/.buildkite/pipeline.yml

@@ -1,11 +1,5 @@
-shared: &shared
-  retry:
-     automatic:
-       - exit_status: "*"
-         limit: 1
 steps:
   - name: 'Run Test Suite (:kubernetes: 1.13-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: k8s-ci
@@ -13,7 +7,6 @@ steps:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.13-latest
   - name: 'Run Test Suite (:kubernetes: 1.12-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: k8s-ci
@@ -21,7 +14,6 @@ steps:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.12-latest
   - name: 'Run Test Suite (:kubernetes: 1.11-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: k8s-ci
@@ -29,7 +21,6 @@ steps:
       LOGGING_LEVEL: 4
       KUBERNETES_VERSION: v1.11-latest
   - name: 'Run Test Suite (:kubernetes: 1.10-latest)'
-    <<: *shared
     command: bin/ci
     agents:
       queue: minikube-ci

data/CHANGELOG.md

@@ -1,3 +1,26 @@
+## next
+
+## 0.26.0
+
+*Enhancements*
+- Add support for NetworkPolicies ([#422](https://github.com/Shopify/kubernetes-deploy/pull/422))
+- Setting the REVISION environment variable is now optional ([#429](https://github.com/Shopify/kubernetes-deploy/pull/429))
+- Defaults KUBECONFIG to `~/.kube/config` ([#429](https://github.com/Shopify/kubernetes-deploy/pull/429))
+- Uses `TASK_ID` environment variable as the `deployment_id` when rendering resource templates for better [Shipit](https://github.com/Shopify/shipit) integration. ([#430](https://github.com/Shopify/kubernetes-deploy/pull/430))
+- Arguments to `--bindings` will now be deep merged. ([#419](https://github.com/Shopify/kubernetes-deploy/pull/419))
+- `kubernetes-deploy` and `kubernetes-render` now support reading templates from STDIN. ([#415](https://github.com/Shopify/kubernetes-deploy/pull/415))
+- Support for specifying a `--selector`, a label with which all deployed resources are expected to have, and by which prunable resources will be filtered. This permits sharing a namespace with resources managed by third-parties, including other kubernetes-deploy deployments. ([#439](https://github.com/Shopify/kubernetes-deploy/pull/439))
+- Lists of resources printed during deployments will now be sorted alphabetically. ([#441](https://github.com/Shopify/kubernetes-deploy/pull/441))
+- Bare / unmanaged pods run as pre-deployment tasks will now stream logs if there is only one of them. ([#436](https://github.com/Shopify/kubernetes-deploy/pull/436))
+
+*Features*
+
+- **[Breaking change]** Support for deploying Secrets from templates ([#424](https://github.com/Shopify/kubernetes-deploy/pull/424)). Non-ejson secrets are now fully supported and therefore **subject to pruning like any other resource**. As a result:
+  * If you previously manually `kubectl apply`'d secrets that are not passed to kubernetes-deploy, your first deploy using this version is going to delete them.
+  * If you previously passed secrets manifests to kubernetes-deploy and they are no longer in the set you pass to the first deploy using this version, it will delete them.
+  * To identify potentially affected secrets in your cluster, run: `kubectl get secrets -o jsonpath='{ range .items[*] }{.metadata.namespace}{ "\t" }{.metadata.name}{ "\t" }{.metadata.annotations}{ "\n" }{ end }' --context=$YOUR_CONTEXT_HERE --all-namespaces | grep -v "kubernetes-deploy.shopify.io/ejson-secret" | grep "last-applied" | cut -f 1,2`. To exclude a secret from kubernetes-deploy (and kubectl apply) management, remove the last-applied annotation `kubectl annotate secret $SECRET_NAME kubectl.kubernetes.io/last-applied-configuration-`.
+  * The secret `ejson-keys` will never be pruned by kubernetes-deploy. Instead, it will fail the deploy at the validation stage (unless `--no-prune` is set). ([#447](https://github.com/Shopify/kubernetes-deploy/pull/447))
+
 ## 0.25.0
 
 *Features*

data/CONTRIBUTING.md

@@ -0,0 +1,164 @@
+# Contributing to kubernetes-deploy
+
+:+1::tada: First off, thanks for taking the time to contribute! :tada::+1:
+
+The following is a set of guidelines for contributing to kubernetes-deploy. Please take a moment to read through them before submitting your first PR.
+
+
+#### Table Of Contents
+
+[Code of Conduct](#code-of-conduct)
+
+[What should I know before I get started?](#what-should-i-know-before-i-get-started)
+  * [High-level design decisions](#high-level-design-decisions)
+  * [Feature acceptance policies](#feature-acceptance-policies)
+  * [Adding a new resource type](#contributing-a-new-resource-type)
+
+[Development](#development)
+  * [Setup](#setup)
+  * [Running the test suite locally](#running-the-test-suite-locally)
+  * [Releasing a new version (Shopify employees)](#releasing-a-new-version-shopify-employees)
+  * [CI (External contributors)](#ci-external-contributors)
+## Code of Conduct
+
+This project and everyone participating in it are governed by the [Code of Conduct](https://github.com/Shopify/kubernetes-deploy/blob/master/CODE_OF_CONDUCT.md).
+By participating, you are expected to uphold this code. Please report unacceptable
+behavior to [kubernetes-deploy@shopify.com](mailto:kubernetes-deploy@shopify.com).
+
+## Maintainers
+
+This project is currently under the stewardship of the Production Platform group at Shopify.
+The two primary maintainers are @knverey and @dturn. Approval from at least one primary maintainer is
+required for all significant feature proposals and code architecture changes. In general,
+two people must approve all non-trivial PRs.
+
+## What should I know before I get started?
+
+### High-level design decisions
+
+**Logging**
+
+Since we are primarily a CLI tool, logging is our entire user interface. Please think carefully about the information you log. Is it clear? Is it logged at the right time? Is it helpful?
+
+In particular, we need to ensure that every mutation we’ve done to the cluster is clearly described (and is something the operator wanted us to do!).
+
+We handle Kubernetes secrets, so it is critical that changes do not cause the contents of any secrets in the template set to be logged.
+
+**Project architecture**
+
+The main interface of this project is our four tasks: `DeployTask`, `RestartTask`, `RunnerTask`, and `RenderTask`. The code in these classes should be high-level abstractions, with implementation details encapsulated in other classes. The public interface of these tasks is a `run` method (and a `run!` equivalent), the body of which should read like a set of phases and steps.
+
+An important design principle of the tasks is that they should try to fail fast before touching the cluster if they will not succeed overall. Part of how we achieve this is by separating each task into phases, where the first phase simply gathers information and runs validations to determine what needs to be done and whether that will be able to succeed. In practice, this is the “Initializing <task>” phase for all tasks, plus the “Checking initial resource statuses” phase for DeployTask. Our users should be able to assume that these initial phases never modify their clusters.
+
+**Thread safety**
+
+This tool must be able to run concurrent deploys to different targets safely, including when used as a library. Each of those deploys will internally also use parallelism via ruby threads, as do our integration tests. This means all of our code must be thread-safe. Notably, our code must not modify the global namespace (e.g. environment variables, classes, class variables or constants), and all gems we depend on must also be thread-safe.
+
+ _Note_: Local tests do not run in parallel by default. To enable it, use `PARALLELIZE_ME=1 PARALLELISM=$NUM_THREADS`. Unit tests never run in parallel because they use mocha, which is not thread-safe (mocha cannot be used in integration tests).
+
+**Performance and the sync cycle**
+
+DeployTask must remain performant when given several hundred resources at a time, generating 1000+ pods. This means only `sync` methods can make calls to the Kubernetes API server during result verification. This both limits the number of API calls made and ensures a consistent view of the world within each polling cycle.
+
+
+
+### Feature acceptance policies
+
+**Our mission**
+
+This project's mission is to make it easy to ship changes to a Kubernetes namespace and understand the result. Features that introduce new classes of responsibility to the tool are not usually accepted.
+
+Deploys can be a very tempting place to cram features. Imagine a proposed feature actually fits better elsewhere—where might that be? (Examples: validator in CI, custom controller, initializer, pre-processing step in the CD pipeline, or even Kubernetes core)
+
+**Global resources**
+
+This project is intended to manage a namespace (or a label-delimited subsection of one). It does not officially support non-namespaced resources. In practice, we do model custom resource definitions specifically, because it helps us better handle custom resources (which typically are namespaced). However, our intent is to keep this the only exception to the rule.
+
+**Template rendering**
+
+The basic ERB renderer included with the tool is intended as a convenience feature for a better out-of-the box experience. Providing complex rendering capabilities is outside the scope of this project's mission, and enhancements in this area may be rejected.
+
+**Composability**
+
+This project strives to be composable with other tools in the ecosystem, such as renderers and validators. The deploy task must work with any Kubernetes templates provided to it, no matter how they were generated.
+
+**Universality**
+
+This project is open-source. Features tied to any specific organization (including Shopify) will be rejected.
+
+
+### Contributing a new resource type
+
+The list of fully supported types is effectively the list of classes found in `lib/kubernetes-deploy/kubernetes_resource/`.
+
+This gem uses subclasses of `KubernetesResource` to implement custom success/failure detection logic for each resource type. If no subclass exists for a type you're deploying, the gem simply assumes `kubectl apply` succeeded (and prints a warning about this assumption). We're always looking to support more types! Here are the basic steps for contributing a new one:
+
+1. Create a file for your type in `lib/kubernetes-deploy/kubernetes_resource/`
+2. Create a new class that inherits from `KubernetesResource`. Minimally, it should implement the following methods:
+    * `sync` -- Gather the data you'll need to determine `deploy_succeeded?` and `deploy_failed?`. The superclass's implementation fetches the corresponding resource, parses it and stores it in `@instance_data`. You can define your own implementation if you need something else.
+    * `deploy_succeeded?`
+    * `deploy_failed?`
+3. Adjust the `TIMEOUT` constant to an appropriate value for this type.
+4. Add the new class to list of resources in
+   [`deploy_task.rb`](https://github.com/Shopify/kubernetes-deploy/blob/master/lib/kubernetes-deploy/deploy_task.rb#L8)
+5. Add the new resource to the [prune whitelist](https://github.com/Shopify/kubernetes-deploy/blob/master/lib/kubernetes-deploy/deploy_task.rb#L81)
+6. Add a basic example of the type to the hello-cloud [fixture set](https://github.com/Shopify/kubernetes-deploy/tree/master/test/fixtures/hello-cloud) and appropriate assertions to `#assert_all_up` in [`hello_cloud.rb`](https://github.com/Shopify/kubernetes-deploy/blob/master/test/helpers/fixture_sets/hello_cloud.rb). This will get you coverage in several existing tests, such as `test_full_hello_cloud_set_deploy_succeeds`.
+7. Add tests for any edge cases you foresee.
+
+# Development
+
+## Setup
+
+If you work for Shopify, just run `dev up`, but otherwise:
+
+1. [Install kubectl version 1.10.0 or higher](https://kubernetes.io/docs/user-guide/prereqs/) and make sure it is in your path
+2. [Install minikube](https://kubernetes.io/docs/getting-started-guides/minikube/#installation) (required to run the test suite)
+3. Check out the repo
+4. Run `bin/setup` to install dependencies
+
+To install this gem onto your local machine, run `bundle exec rake install`.
+
+
+
+## Running the test suite locally
+
+Using minikube:
+
+1. Start [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/#installation) (`minikube start [options]`).
+2. Make sure you have a context named "minikube" in your kubeconfig. Minikube adds this context for you when you run `minikube start`. You can check for it using `kubectl config get-contexts`.
+3. Run `bundle exec rake test` (or `dev test` if you work for Shopify).
+
+Using another local cluster:
+
+1. Start your cluster.
+2. Put the name of the context you want to use in a file named `.local-context` in the root of this project. For example: `echo "dind" > .local-context`.
+3. Run `bundle exec rake test` (or `dev test` if you work for Shopify).
+
+To make StatsD log what it would have emitted, run a test with `STATSD_DEV=1`.
+
+To see the full-color output of a specific integration test, you can use `PRINT_LOGS=1`. For example: `PRINT_LOGS=1 bundle exec ruby -I test test/integration/kubernetes_deploy_test.rb -n/test_name/`.
+
+
+![test-output](screenshots/test-output.png)
+
+
+## Releasing a new version (Shopify employees)
+
+1. Make sure all merged PRs are reflected in the changelog before creating the commit for the new version.
+2. Update the version number in `version.rb` and commit that change with message "Version x.y.z". Don't push yet or you'll confuse Shipit.
+3. Tag the version with `git tag vx.y.z -a -m "Version x.y.z"`
+4. Push both your bump commit and its tag simultaneously with `git push origin master --follow-tags` (note that you can set `git config --global push.followTags true` to turn this flag on by default)
+5. Use the [Shipit Stack](https://shipit.shopify.io/shopify/kubernetes-deploy/rubygems) to build the `.gem` file and upload to [rubygems.org](https://rubygems.org/gems/kubernetes-deploy).
+
+If you push your commit and the tag separately, Shipit usually fails with `You need to create the v0.7.9 tag first.`. To make it find your tag, go to `Settings` > `Resynchronize this stack` > `Clear git cache`.
+
+
+## CI (External contributors)
+
+Please make sure you run the tests locally before submitting your PR (see [Running the test suite locally](#running-the-test-suite-locally)). After reviewing your PR, a Shopify employee will trigger CI for you.
+
+#### Employees: Triggering CI for a contributed PR
+
+Go to the [kubernetes-deploy-gem pipeline](https://buildkite.com/shopify/kubernetes-deploy-gem) and click "New Build". Use branch `external_contrib_ci` and the specific sha of the commit you want to build. Add `BUILDKITE_REFSPEC="refs/pull/${PR_NUM}/head"` in the Environment Variables section. Since CI is only visible to Shopify employees, you will need to provide any failing tests and output to the the contributor.
+
+<img width="350" alt="build external contrib PR" src="https://screenshot.click/2017-11-07--163728_7ovek-wrpwq.png">

data/README.md

@@ -55,12 +55,6 @@ This repo also includes related tools for [running tasks](#kubernetes-run) and [
 * [Prerequisites](#prerequisites-2)
 * [Usage](#usage-3)
 
-**DEVELOPMENT**
-* [Setup](#setup)
-* [Running the test suite locally](#running-the-test-suite-locally)
-* [Releasing a new version (Shopify employees)](#releasing-a-new-version-shopify-employees)
-* [CI (External contributors)](#ci-external-contributors)
-
 **CONTRIBUTING**
 * [Contributing](#contributing)
 * [Code of Conduct](#code-of-conduct)
@@ -76,14 +70,10 @@ This repo also includes related tools for [running tasks](#kubernetes-run) and [
 * Ruby 2.3+
 * Your cluster must be running Kubernetes v1.10.0 or higher<sup>1</sup>
 * Each app must have a deploy directory containing its Kubernetes templates (see [Templates](#using-templates-and-variables))
-* You must remove the` kubectl.kubernetes.io/last-applied-configuration` annotation from any resources in the namespace that are not included in your deploy directory. This annotation is added automatically when you create resources with `kubectl apply`. `kubernetes-deploy` will prune any resources that have this annotation and are not in the deploy directory.<sup>2</sup>
-* Each app managed by `kubernetes-deploy` must have its own exclusive Kubernetes namespace.
 
 <sup>1</sup> We run integration tests against these Kubernetes versions. You can find our
 offical compatibility chart below.
 
-<sup>2</sup> This requirement can be bypassed with the `--no-prune` option, but it is not recommended.
-
 | Kubernetes version | Last officially supported in gem version |
 | :----------------: | :-------------------: |
 |        1.5         |        0.11.2         |
@@ -106,8 +96,9 @@ offical compatibility chart below.
 
 *Environment variables:*
 
-- `$REVISION` **(required)**: the SHA of the commit you are deploying. Will be exposed to your ERB templates as `current_sha`.
-- `$KUBECONFIG`  **(required)**: points to one or multiple valid kubeconfig files that include the context you want to deploy to. File names are separated by colon for Linux and Mac, and semi-colon for Windows.
+- `$REVISION`: the SHA of the commit you are deploying. Will be exposed to your ERB templates as `current_sha`.
+- `$KUBECONFIG`: points to one or multiple valid kubeconfig files that include the context you want to deploy to. File names are separated by colon for Linux and Mac, and semi-colon for Windows. If ommitted, will use the Kubernetes default of `~/.kube/config`.
+- `$TASK_ID`: used as the ID of the deployment for resource naming.
 - `$ENVIRONMENT`: used to set the deploy directory to `config/deploy/$ENVIRONMENT`. You can use the `--template-dir=DIR` option instead if you prefer (**one or the other is required**).
 - `$GOOGLE_APPLICATION_CREDENTIALS`: points to the credentials for an authenticated service account (required if your kubeconfig `user`'s auth provider is GCP)
 
@@ -116,12 +107,20 @@ offical compatibility chart below.
 
 Refer to `kubernetes-deploy --help` for the authoritative set of options.
 
-- `--template-dir=DIR`: Used to set the deploy directory. Set `$ENVIRONMENT` instead to use `config/deploy/$ENVIRONMENT`.
+- `--template-dir=DIR`: Used to set the deploy directory. Set `$ENVIRONMENT` instead to use `config/deploy/$ENVIRONMENT`. This flag also supports reading from STDIN. You can do this by using `--template-dir=-`. Example: `cat templates_from_stdin/*.yml | kubernetes-deploy ns ctx --template-dir=-`.
 - `--bindings=BINDINGS`: Makes additional variables available to your ERB templates. For example, `kubernetes-deploy my-app cluster1 --bindings=color=blue,size=large` will expose `color` and `size`.
 - `--no-prune`: Skips pruning of resources that are no longer in your Kubernetes template set. Not recommended, as it allows your namespace to accumulate cruft that is not reflected in your deploy directory.
 - `--max-watch-seconds=seconds`: Raise a timeout error if it takes longer than _seconds_ for any
 resource to deploy.
+- `--selector`: Instructs kubernetes-deploy to only prune resources which match the specified label selector, such as `environment=staging`. If you use this option, all resource templates must specify matching labels. See [Sharing a namespace](#sharing-a-namespace) below.
+
+### Sharing a namespace
 
+By default, kubernetes-deploy will prune any resources in the target namespace which have the `kubectl.kubernetes.io/last-applied-configuration` annotation and are not a result of the current deployment process, on the assumption that there is a one-to-one relationship between application deployment and namespace, and that a deployment provisions all relevant resources in the namespace.
+
+If you need to, you may specify `--no-prune` to disable all pruning behaviour, but this is not recommended.
+
+If you need to share a namespace with resources which are managed by other tools or indeed other kubernetes-deploy deployments, you can supply the `--selector` option, such that only resources with labels matching the selector are considered for pruning.
 
 ### Using templates and variables
 
@@ -130,7 +129,7 @@ Each app's templates are expected to be stored in a single directory. If this is
 All templates must be YAML formatted. You can also use ERB. The following local variables will be available to your ERB templates by default:
 
 * `current_sha`: The value of `$REVISION`
-* `deployment_id`:  A randomly generated identifier for the deploy. Useful for creating unique names for task-runner pods (e.g. a pod that runs rails migrations at the beginning of deploys).
+* `deployment_id`: The value of `$TASK_ID`, or in its absence, a randomly generated identifier for the deploy. Useful for creating unique names for task-runner pods (e.g. a pod that runs rails migrations at the beginning of deploys).
 
 You can add additional variables using the `--bindings=BINDINGS` option which can be formated as comma separated string, JSON string or path to a JSON or YAML file. Complex JSON or YAML data will be converted to a Hash for use in templates. To load a file the argument should include the relative file path prefixed with an `@` sign. An argument error will be raised if the string argument cannot be parsed, the referenced file does not include a valid extension (`.json`, `.yaml` or `.yml`) or the referenced file does not exist.
 
@@ -256,7 +255,7 @@ To run a task in your cluster at the beginning of every deploy, simply include a
 
 A simple example can be found in the test fixtures: test/fixtures/hello-cloud/unmanaged-pod.yml.erb.
 
-The logs of all pods run in this way will be printed inline.
+The logs of all pods run in this way will be printed inline. If there is only one pod, the logs will be streamed in real-time. If there are multiple, they will be fetched when the pod terminates.
 
 ![migrate-logs](screenshots/migrate-logs.png)
 
@@ -271,6 +270,7 @@ Since their data is only base64 encoded, Kubernetes secrets should not be commit
 1. Install the ejson gem: `gem install ejson`
 2. Generate a new keypair: `ejson keygen` (prints the keypair to stdout)
 3. Create a Kubernetes secret in your target namespace with the new keypair: `kubectl create secret generic ejson-keys --from-literal=YOUR_PUBLIC_KEY=YOUR_PRIVATE_KEY --namespace=TARGET_NAMESPACE`
+>Warning: Do *not* use `apply` to create the `ejson-keys` secret. kubernetes-deploy will fail if `ejson-keys` is prunable. This safeguard is to protect against the accidental deletion of your private keys.
 4. (optional but highly recommended) Back up the keypair somewhere secure, such as a password manager, for disaster recovery purposes.
 5. In your template directory (alongside your Kubernetes templates), create `secrets.ejson` with the format shown below. The `_type` key should have the value “kubernetes.io/tls” for TLS secrets and “Opaque” for all others. The `data` key must be a json object, but its keys and values can be whatever you need.
 
@@ -299,6 +299,7 @@ Since their data is only base64 encoded, Kubernetes secrets should not be commit
 7. Commit the encrypted file and deploy as usual. The deploy will create secrets from the data in the `kubernetes_secrets` key.
 
 **Note**: Since leading underscores in ejson keys are used to skip encryption of the associated value, `kubernetes-deploy` will strip these leading underscores when it creates the keys for the Kubernetes secret data. For example, given the ejson data below, the `monitoring-token` secret will have keys `api-token` and `property` (_not_ `_property`):
+
 ```json
 {
   "_public_key": "YOUR_PUBLIC_KEY",
@@ -313,6 +314,8 @@ Since their data is only base64 encoded, Kubernetes secrets should not be commit
   }
 ```
 
+**A warning about using EJSON secrets with `--selector`**: when using EJSON to generate `Secret` resources and specifying a `--selector` for deployment, the labels from the selector are automatically added to the `Secret`. If _the same_ EJSON file is deployed to the same namespace using different selectors, this will cause the resource to thrash - even if the contents of the secret were the same, the resource has different labels on each deploy.
+
 ### Deploying custom resources
 
 By default, kubernetes-deploy does not check the status of custom resources; it simply assumes that they deployed successfully. In order to meaningfully monitor the rollout of custom resources, kubernetes-deploy supports configuring pass/fail conditions using annotations on CustomResourceDefinitions (CRDs).
@@ -435,6 +438,13 @@ With this done, you can use the following command to restart all of them:
 
 `kubernetes-restart <kube namespace> <kube context>`
 
+*Options:*
+
+Refer to `kubernetes-restart --help` for the authoritative set of options.
+
+- `--selector`: Only restarts Deployments which match the specified Kubernetes resource selector.
+- `--deployments`: Restart specific Deployment resources by name.
+
 # kubernetes-run
 
 `kubernetes-run` is a tool for triggering a one-off job, such as a rake task, _outside_ of a deploy.
@@ -490,102 +500,17 @@ kubernetes-render --template-dir=./path/to/template/dir this-template.yaml.erb t
 
 *Options:*
 
-- `--template-dir=DIR`: Used to set the directory to interpret template names relative to. This is often the same directory passed as `--template-dir` when running `kubernetes-deploy` to actually deploy templates. Set `$ENVIRONMENT` instead to use `config/deploy/$ENVIRONMENT`.
+- `--template-dir=DIR`: Used to set the directory to interpret template names relative to. This is often the same directory passed as `--template-dir` when running `kubernetes-deploy` to actually deploy templates. Set `$ENVIRONMENT` instead to use `config/deploy/$ENVIRONMENT`. This flag also supports reading from STDIN. You can do this by using `--template-dir=-`.
 - `--bindings=BINDINGS`: Makes additional variables available to your ERB templates. For example, `kubernetes-render --bindings=color=blue,size=large some-template.yaml.erb` will expose `color` and `size` to `some-template.yaml.erb`.
 
 
-# Development
-
-## Setup
-
-If you work for Shopify, just run `dev up`, but otherwise:
-
-1. [Install kubectl version 1.10.0 or higher](https://kubernetes.io/docs/user-guide/prereqs/) and make sure it is in your path
-2. [Install minikube](https://kubernetes.io/docs/getting-started-guides/minikube/#installation) (required to run the test suite)
-3. Check out the repo
-4. Run `bin/setup` to install dependencies
-
-To install this gem onto your local machine, run `bundle exec rake install`.
-
-
-
-## Running the test suite locally
-
-Using minikube:
-
-1. Start [minikube](https://kubernetes.io/docs/getting-started-guides/minikube/#installation) (`minikube start [options]`).
-2. Make sure you have a context named "minikube" in your kubeconfig. Minikube adds this context for you when you run `minikube start`. You can check for it using `kubectl config get-contexts`.
-3. Run `bundle exec rake test` (or `dev test` if you work for Shopify).
-
-Using another local cluster:
-
-1. Start your cluster.
-2. Put the name of the context you want to use in a file named `.local-context` in the root of this project. For example: `echo "dind" > .local-context`.
-3. Run `bundle exec rake test` (or `dev test` if you work for Shopify).
-
-To make StatsD log what it would have emitted, run a test with `STATSD_DEV=1`.
-
-To see the full-color output of a specific integration test, you can use `PRINT_LOGS=1`. For example: `PRINT_LOGS=1 bundle exec ruby -I test test/integration/kubernetes_deploy_test.rb -n/test_name/`.
-
-
-
-
-![test-output](screenshots/test-output.png)
-
-
-
-## Releasing a new version (Shopify employees)
-
-1. Make sure all merged PRs are reflected in the changelog before creating the commit for the new version.
-2. Update the version number in `version.rb` and commit that change with message "Version x.y.z". Don't push yet or you'll confuse Shipit.
-3. Tag the version with `git tag vx.y.z -a -m "Version x.y.z"`
-4. Push both your bump commit and its tag simultaneously with `git push origin master --follow-tags` (note that you can set `git config --global push.followTags true` to turn this flag on by default)
-5. Use the [Shipit Stack](https://shipit.shopify.io/shopify/kubernetes-deploy/rubygems) to build the `.gem` file and upload to [rubygems.org](https://rubygems.org/gems/kubernetes-deploy).
-
-If you push your commit and the tag separately, Shipit usually fails with `You need to create the v0.7.9 tag first.`. To make it find your tag, go to `Settings` > `Resynchronize this stack` > `Clear git cache`.
-
-
-## CI (External contributors)
-
-Please make sure you run the tests locally before submitting your PR (see [Running the test suite locally](#running-the-test-suite-locally)). After reviewing your PR, a Shopify employee will trigger CI for you.
-
-#### Employees: Triggering CI for a contributed PR
-
-Go to the [kubernetes-deploy-gem pipeline](https://buildkite.com/shopify/kubernetes-deploy-gem) and click "New Build". Use branch `external_contrib_ci` and the specific sha of the commit you want to build. Add `BUILDKITE_REFSPEC="refs/pull/${PR_NUM}/head"` in the Environment Variables section.
-
-<img width="350" alt="build external contrib PR" src="https://screenshot.click/2017-11-07--163728_7ovek-wrpwq.png">
-
 # Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/Shopify/kubernetes-deploy.
-
-Contributions to help us support additional resource types or increase the sophistication of our success heuristics for an existing type are especially encouraged! (See tips below)
-
-## Feature acceptance guidelines
-
-- This project's mission is to make it easy to ship changes to a Kubernetes namespace and understand the result. Features that introduce new classes of responsibility to the tool are not usually accepted.
-  - Deploys can be a very tempting place to cram features. Imagine a proposed feature actually fits better elsewhere—where might that be? (Examples: validator in CI, custom controller, initializer, pre-processing step in the CD pipeline, or even Kubernetes core)
-  - The basic ERB renderer included with the tool is intended as a convenience feature for a better out-of-the box experience. Providing complex rendering capabilities is out of scope of this project's mission, and enhancements in this area may be rejected.
-  - The deploy command does not officially support non-namespaced resource types.
-- This project strives to be composable with other tools in the ecosystem, such as renderers and validators. The deploy command must work with any Kubernetes templates provided to it, no matter how they were generated.
-- This project is open-source. Features tied to any specific organization (including Shopify) will be rejected.
-- The deploy command must remain performant when given several hundred resources at a time, generating 1000+ pods. (Technical note: This means only `sync` methods can make calls to the Kuberentes API server during result verification. This both limits the number of API calls made and ensures a consistent view of the world within each polling cycle.)
-- This tool must be able to run concurrent deploys to different targets safely, including when used as a library.
-
-## Contributing a new resource type
-
-The list of fully supported types is effectively the list of classes found in `lib/kubernetes-deploy/kubernetes_resource/`.
+We :heart: contributors! To make it easier for you and us we've written a
+[Contributing Guide](https://github.com/Shopify/kubernetes-deploy/blob/master/CONTRIBUTING.md)
 
-This gem uses subclasses of `KubernetesResource` to implement custom success/failure detection logic for each resource type. If no subclass exists for a type you're deploying, the gem simply assumes `kubectl apply` succeeded (and prints a warning about this assumption). We're always looking to support more types! Here are the basic steps for contributing a new one:
 
-1. Create a the file for your type in `lib/kubernetes-deploy/kubernetes_resource/`
-2. Create a new class that inherits from `KubernetesResource`. Minimally, it should implement the following methods:
-    * `sync` -- Gather the data you'll need to determine `deploy_succeeded?` and `deploy_failed?`. The superclass's implementation fetches the corresponding resource, parses it and stores it in `@instance_data`. You can define your own implementation if you need something else.
-    * `deploy_succeeded?`
-    * `deploy_failed?`
-3. Adjust the `TIMEOUT` constant to an appropriate value for this type.
-4. Add the a basic example of the type to the hello-cloud [fixture set](https://github.com/Shopify/kubernetes-deploy/tree/master/test/fixtures/hello-cloud) and appropriate assertions to `#assert_all_up` in [`hello_cloud.rb`](https://github.com/Shopify/kubernetes-deploy/blob/master/test/helpers/fixture_sets/hello_cloud.rb). This will get you coverage in several existing tests, such as `test_full_hello_cloud_set_deploy_succeeds`.
-5. Add tests for any edge cases you foresee.
+You can also reach out to us on our slack channel, #krane, at https://kubernetes.slack.com. All are welcome!
 
 ## Code of Conduct
 Everyone is expected to follow our [Code of Conduct](https://github.com/Shopify/kubernetes-deploy/blob/master/CODE_OF_CONDUCT.md).