kubes 0.3.1 → 0.4.0

data/docs/_docs/config/kubectl.md

@@ -1,61 +1,10 @@
 ---
-title: Kubectl Config
+title: Kubectl Customizations
 ---
 
 ## General
 
 Kubes calls out the `kubectl` command. You can customize the command.
 
-## Args
-
-Here are some examples of customizing the kubectl args.
-
-.kubes/config/kubectl/args.rb
-
-```ruby
-command("apply",
-  args: ["--validate=true"],
-)
-
-command("delete",
-  args: ["--grace-period=-1"],
-)
-```
-
-## Hooks
-
-Here are some examples of running custom hooks before and after the kubectl commands.
-
-.kubes/config/kubectl/hooks.rb
-
-```ruby
-before("apply",
-  execute: "kubectl apply -f .kubes/shared/namespace.yaml",
-)
-
-after("delete",
-  execute: "echo 'delete hook',
-)
-```
-
-You can use hooks to do things that may not make sense to do in the `.kubes/resources` definition. Here's an example of automatically creating the namespace.
-
-.kubes/shared/namespace.yaml
-
-```yaml
-apiVersion: v1
-kind: Namespace
-metadata:
-  name: demo
-```
-
-### exit on fail
-
-By default, if the hook commands fail, then terraspace will exit with the original hook error code.  You can change this behavior with the `exit_on_fail` option.
-
-```ruby
-before("apply"
-  execute: "/command/will/fail/but/will/continue",
-  exit_on_fail: false,
-)
-```
+* [Args]({% link _docs/config/args/kubectl.md %}): Customize the CLI args.
+* [Hooks]({% link _docs/config/hooks/kubectl.md %}): Run hooks before and after the kubectl commands.

data/docs/_docs/config/reference.md

@@ -0,0 +1,20 @@
+---
+title: Config Reference
+---
+
+Name | Description | Default
+---|---|---
+auto_prune | Prune and delete old hashed resources like Secret and ConfigMap. | true
+builder | What docker build command to use. Can use `docker` or `gcloud` to build the Docker image. | docker
+kubectl.context | What kubectl context to auto-switch to. | nil
+kubectl.context_keep | Whether or not to keep the context switched | true
+kubectl.exit_on_fail.apply  | Whether or not continue if the `kubectl apply` fails. Note, can use `KUBES_EXIT_ON_FAIL=0` env var to set to false. | true
+kubectl.exit_on_fail.delete | Whether or not continue if the `kubectl delete` fails. | false
+kubectl.order.kinds | Change ordering for Kubernetes Kinds. | See [source code](https://github.com/boltops-tools/kubes/blob/master/lib/kubes/config.rb#L52)
+kubectl.order.roles | Change ordering for Kubes Roles. | See [source code](https://github.com/boltops-tools/kubes/blob/master/lib/kubes/config.rb#L44)
+logger | Logger object | Logger.new($stdout)
+logger.level | Logger level. Can also be set with `KUBES_LOG_LEVEL` env var | info
+repo | The Docker repo to use. Required to be set. | nil
+skip | List of resources to skip. Can also be set with the `KUBES_SKIP` env var. `KUBES_SKIP` should be a list of strings separated by spaces. It adds onto the `config.skip` option. | []
+state.docker_image_path | Where to store the state file with the last build Docker image. | .kubes/state/docker_image.txt
+suffix_hash | Whether or not to append suffix hash to ConfigMap and Secret | true

data/docs/_docs/config/skip.md

@@ -0,0 +1,58 @@
+---
+title: Skip Option
+---
+
+You can tell Kubes to skip resources to deploy. This can useful if you want to still resources with Kubes and have it compile `.kubes/output` files, but wish to deploy them outside of Kubes manually.
+
+## Example
+
+Here's an example with a Job.
+
+.kubes/resources/cleanup/job.yaml:
+
+```yaml
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: cleanup
+spec:
+  template:
+    spec:
+      containers:
+      - name: cleanup
+        image: <%= built_image %>
+        command: ["bin/cleanup.sh"]
+      restartPolicy: Never
+```
+
+To skip the cleanup job, use the `config.skip` option:
+
+```ruby
+Kubes.configure do |config|
+  config.skip = ["cleanup/job"]
+end
+```
+
+Now when you deploy, the `cleanup/job` resource will not be deployed:
+
+    kubes deploy # deploys everything except cleanup/job
+
+## Deploy Outside of Kubes
+
+Then to deploy outside of kubes.
+
+    $ kubes compile # not necessary if already ran: kubes deploy
+    Compiled  .kubes/resources files to .kubes/output
+    $ kubectl apply -f .kubes/output/cleanup/job.yaml
+    job.batch/cleanup created
+    $ kubectl delete -f .kubes/output/cleanup/job.yaml
+    job.batch "cleanup" deleted
+    $
+
+## Env Var KUBES_SKIP
+
+You can also us ethe `KUBES_SKIP` env var. It takes list of strings separated by a space. It adds onto the `config.skip` option. Example:
+
+    KUBES_SKIP="cleanup/job" kubes delete
+
+This can be useful for one-off use cases.

data/docs/_docs/dsl/resources.md

@@ -6,7 +6,7 @@ Here's a list of the resources supported by the Kubes DSL.
 
 {% assign docs = site.docs | where: "categories","dsl" %}
 {% for doc in docs -%}
-* [{{ doc.title }}]({{ doc.url }})
+* [{{ doc.nav_text }}]({{ doc.url }})
 {% endfor %}
 
 For resources, that are not supported, you can use the [Generic resource]({% link _docs/dsl/resources/generic.md %}) or use [YAML]({% link _docs/yaml.md %}) instead. You can use a mix of DSL and YAML definitions in the `.kubes/resources` folder.

data/docs/_docs/dsl/resources/backend_config.md

@@ -3,7 +3,7 @@ title: BackendConfig
 categories: dsl
 ---
 
-A BackendConfig is [custom resource definitions (CRDs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) that allow you to further customize the load balancer.
+A [BackendConfig](https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#create_backendconfig) is [custom resource definitions (CRDs)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) that allow you to further customize the load balancer.
 
 Here's an example of a BackendConfig.
 

data/docs/_docs/intro.md

@@ -4,12 +4,15 @@ title: What is Kubes?
 
 {% include reference.md %}
 
+<div class="video-box"><div class="video-container"><iframe src="https://www.youtube.com/embed/M4zHL0mfKNU" frameborder="0" allowfullscreen=""></iframe></div></div>
+
 ## Features:
 
 * Automation: [Builds the Docker image]({% link _docs/config/docker.md %}) and updates the compiled YAML files
 * Syntactic Sugar: Use an [ERB/YAML]({% link _docs/yaml.md %}) or a [DSL]({% link _docs/dsl.md %}) to write your Kubernetes YAML files. You can use a mix of DSL and YAML definitions in the `.kubes/resources` folder.
 * Layering: Use the same Kubernetes YAML to build multiple environments like dev and prod with [layering]({% link _docs/layering.md %}).
-* CLI Customizations: You can customize the [cli args]({% link _docs/config/kubectl.md %}). You can also run hooks before and after kubectl commands.
+* CLI Customizations: You can customize the [cli args]({% link _docs/config/args/kubectl.md %}). You can also run [hooks]({% link _docs/config/hooks/kubectl.md %}) before and after kubectl commands.
 * Automated Suffix Hashes: Automatically appends a suffix hash to ConfigMap and Secret resources. More details in [ConfigMap]({% link _docs/dsl/resources/config_map.md %}) and [Secret]({% link _docs/dsl/resources/secret.md %}) docs.
-* Kustomize Support: If you’re a kustomization user, you can use it with Kubes. More details in [Kustomize Support Docs]({% link _docs/kustomize.md %}).
-* Auto Context Switching: Map dev to a specific kubectl context and prod to another kubectl context and Kubes can switch them automatically so you won't have to remember. More details in [Auto Context Docs]({% link _docs/auto-context.md %}).
+* Kustomize Support: If you’re a kustomization user, you can use it with Kubes. More details in [Kustomize Support Docs]({% link _docs/misc/kustomize.md %}).
+* Auto Context Switching: Map dev to a specific kubectl context and prod to another kubectl context and Kubes can switch them automatically so you won't have to remember. More details in [Auto Context Docs]({% link _docs/misc/auto-context.md %}).
+* Ordering: Kubes run kubectl apply to create resources in the [correct order]({% link _docs/intro/ordering.md %}). For deleting, it kubes will run `kubectl delete` in the correct reverse order. The order is also [customizable]({% link _docs/intro/ordering/custom.md %}).

data/docs/_docs/learn/yaml/new-project.md

@@ -11,7 +11,6 @@ If you already a project with an existing Dockerfile, you can use that. If you d
 
 Let's generate a starter project:
 
-    $ REPO=$(aws ecr describe-repositories --repository-name demo | jq -r '.repositories[].repositoryUri')
     $ kubes init --app demo --repo $REPO
           create  Dockerfile
           create  .kubes/config.rb

data/docs/_docs/misc/separate-steps.md

@@ -0,0 +1,21 @@
+---
+title: Separate Steps
+---
+
+Sometimes you may want to run the 3 separate kubes steps directly. This may be useful if you are setting up CI/CD and need more control over the build process. Here are the 3 main steps:
+
+To build and push the docker image:
+
+    kubes docker build
+    kubes docker push
+
+Note, you must run a `kubes docker build` at least once. As the build step will store the image name in a `.kubes/state/docker_image.txt ` file for later use.
+
+To compile the Kubernetes YAML files.
+
+    kubes compile
+
+To apply the Kubernetes YAML files in the correct order and create resources on the cluster:
+
+    kubes apply
+

data/docs/_docs/patterns/migrations.md

@@ -0,0 +1,121 @@
+---
+title: Database Migrations
+---
+
+A common task is to run database migrations. You can use Kubes hooks to achieve this as part of the `kubes deploy` process.
+
+1. Create Migrate Job YAML
+2. Set up Kubes Hooks
+
+## 1. Create Migrate Job YAML
+
+First, let's create the migrate job YAML. Here's a starter example:
+
+.kubes/resources/migrate/job.yaml
+
+```yaml
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: migrate
+spec:
+  template:
+    spec:
+      containers:
+      - name: migrate
+        image: <%= built_image %>
+        command: ["bin/job/migrate.sh"]
+      restartPolicy: Never
+  backoffLimit: 4
+```
+
+The Kubernetes job calls a `job/migrate.sh` script. Something like this:
+
+bin/job/migrate.sh
+
+    #!/bin/bash
+    rails db:migrate
+
+## 2. Set up Kubes Hooks
+
+Set up the [kubes hooks]({% link _docs/config/hooks/kubectl.md %}) to help the migrate job run properly.
+
+.kubes/config/hooks/kubectl.rb
+
+```ruby
+before("apply",
+  on: "migrate/job",
+  execute: "bin/hooks/migrate/delete.sh",
+  exit_on_fail: false,
+)
+
+after("apply",
+  on: "migrate/job",
+  execute: "bin/hooks/migrate/wait.sh",
+)
+```
+
+Here's what the `bin/hook/migrate` scripts could look like:
+
+bin/hooks/migrate/delete.sh
+
+    #!/bin/bash
+    kubectl delete job/migrate
+
+bin/hooks/migrate/wait.sh
+
+    #!/bin/bash
+    kubectl wait --for=condition=Complete job/migrate --timeout=300s
+
+The `migrate/delete.sh` script first cleans up old migrate jobs that may have been previously created.
+
+The `migrate/wait.sh` script waits until the migration job finishes before continuing. Note, the default timeout is 30s, which may not be long enough for your migrations to finish, so we set it to 300s.  The `kubectl wait` only returns if the migrate job finishes successfully. If the job fails after it exhausts all its retries, default 6, then you'll see an error like this:
+
+    + kubectl wait --for=condition=Complete job/migrate --timeout=30s
+    error: timed out waiting for the condition on jobs/migrate
+    ERROR: running bin/hooks/migrate.sh
+
+There is also an [migration-example](https://github.com/boltops-tools/kubes-examples/tree/master/yaml/migration-example) repo with a smarter version of the wait script.
+
+## Example Deploy
+
+Once that is set up, a `kubes deploy` will automatically run migrations. Here's an example deploy:
+
+    $ kubes deploy
+    => kubectl apply -f .kubes/output/shared/namespace.yaml
+    => bin/hooks/migrate/delete.sh
+    job.batch "migrate" deleted
+    => kubectl apply -f .kubes/output/migrate/job.yaml
+    job.batch/migrate created
+    Running hook: after apply on: migrate/job
+    => bin/hooks/migrate/wait.sh
+    Sun Oct 11 03:22:35 UTC 2020
+    Migration complete
+    => kubectl apply -f .kubes/output/web/service.yaml
+    service/web unchanged
+    => kubectl apply -f .kubes/output/web/deployment.yaml
+    deployment.apps/web configured
+    $
+
+## To Couple or Not to Couple?
+
+While some companies prefer running the migration step as a part of the app deploy, some prefer to separate it out as a discrete step. Usually, the separate step is still called as part of a pipeline.
+
+In practice, the decision usually comes down to:
+
+* The size of your database. If your database is large and the migrations take a long time to run. It makes sense to separate it out.
+* The risk tolerance of database migration operations. If it's quite risky to run DB migrations, you may want to separate it as discrete step so a human can review it.
+
+For small apps and databases, it's often pragmatic to just run everything in a single step for simplicity.
+
+## Migration as Separate Step
+
+If you would like it to run it as a discrete step, remove the hook in `.kubes/config/hooks/kubectl.rb`, and run it as a separate script like so:
+
+bin/run/migrate.sh
+
+    #!/bin/bash
+    kubes compile
+    bin/hooks/migrate/delete.sh
+    bin/job/migrate.sh
+    bin/hooks/migrate/wait.sh

data/docs/_includes/commands.html

@@ -20,6 +20,8 @@ kubes compile
 kubes deploy
 kubes apply
 kubes get
+kubes exec
+kubes logs
 kubes delete -y
 {% endhighlight %}
               </div>

data/docs/_includes/config/hooks/options.md

@@ -0,0 +1,20 @@
+## General Form
+
+```ruby
+before(COMMAND_NAME, OPTIONS)
+````
+
+The command name corresponds to the `{{ include.command }}` commands: apply, delete, etc.
+
+## Hook Options
+
+Name | Description
+---|---
+label | A human-friendly label so you can see what hooks is being run during `kubes apply`
+execute | The script or command to run. IE: path/to/some/script.sh
+exit_on_fail | Whether or not to continue process if the script returns an failed exit code.
+{% if include.command == "kubectl" %}on | What resource to run the hook on. IE: shared/namespace, web/deployment, web/service. Note: This option is only used by kubectl hooks.{% endif %}
+
+## Ruby Hooks
+
+Instead of using a script for the hook `execute` option, you can also use a Ruby object. This provides some more control over the current process. See: [Ruby Hooks]({% link _docs/config/hooks/ruby.md %})

data/docs/_includes/sidebar.html

@@ -53,13 +53,37 @@
       </li>
       <li><a href="{% link _docs/config.md %}">Config</a>
         <ul>
+          <li><a href="{% link _docs/config/args.md %}">Args</a>
+            <ul>
+            {% assign docs = site.docs | where: "categories","args" %}
+            {% for doc in docs -%}
+              <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+            {% endfor %}
+            </ul>
+          </li>
+          <li><a href="{% link _docs/config/hooks.md %}">Hooks</a>
+            <ul>
+            {% assign docs = site.docs | where: "categories","hooks" %}
+            {% for doc in docs -%}
+              <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+            {% endfor %}
+            </ul>
+          </li>
           <li><a href="{% link _docs/config/docker.md %}">Docker</a></li>
           <li><a href="{% link _docs/config/env.md %}">Env</a></li>
-          <li><a href="{% link _docs/config/kubectl.md %}">Kubectl</a></li>
           <li><a href="{% link _docs/config/builder.md %}">Builder</a></li>
+          <li><a href="{% link _docs/config/skip.md %}">Skip Option</a></li>
+          <li><a href="{% link _docs/config/reference.md %}">Reference</a></li>
         </ul>
       </li>
       <li><a href="{% link _docs/yaml.md %}">YAML</a></li>
+      <li><a href="{% link _docs/layering.md %}">Layering</a>
+        <ul>
+          <li><a href="{% link _docs/layering/yaml.md %}">YAML</a></li>
+          <li><a href="{% link _docs/layering/dsl.md %}">DSL</a></li>
+          <li><a href="{% link _docs/layering/merge.md %}">Merge Behavior</a></li>
+        </ul>
+      </li>
       <li><a href="{% link _docs/dsl.md %}">DSL</a>
         <ul>
           <li><a href="{% link _docs/dsl/resources.md %}">Resources</a>
@@ -73,16 +97,10 @@
           <li><a href="{% link _docs/dsl/multiple-resources.md %}">Multiple Resources</a>
         </ul>
       </li>
-      <li><a href="{% link _docs/layering.md %}">Layering</a>
-        <ul>
-          <li><a href="{% link _docs/layering/yaml.md %}">YAML</a></li>
-          <li><a href="{% link _docs/layering/dsl.md %}">DSL</a></li>
-          <li><a href="{% link _docs/layering/merge.md %}">Merge Behavior</a></li>
-        </ul>
-      </li>
       <li><a href="{% link _docs/helpers.md %}">Helpers</a></li>
       <li><a href="{% link _docs/patterns.md %}">Patterns</a>
         <ul>
+          <li><a href="{% link _docs/patterns/migrations.md %}">Database Migrations</a></li>
           <li><a href="{% link _docs/patterns/clock-web-worker.md %}">Clock Web Worker</a></li>
         </ul>
       </li>
@@ -92,8 +110,12 @@
           <li><a href="{% link _docs/extra-env/dsl.md %}">DSL</a></li>
         </ul>
       </li>
-      <li><a href="{% link _docs/kustomize.md %}">Kustomize Support</a></li>
-      <li><a href="{% link _docs/auto-context.md %}">Auto Context</a></li>
+      <li>Misc
+        <ul>
+          <li><a href="{% link _docs/misc/kustomize.md %}">Kustomize Support</a></li>
+          <li><a href="{% link _docs/misc/separate-steps.md %}">Separate Steps</a></li>
+          <li><a href="{% link _docs/misc/auto-context.md %}">Auto Context</a></li>
+        </ul>
       <li>CI/CD
         <ul>
           <li><a href="{% link _docs/ci/cloudbuild.md %}">CloudBuild</a></li>

data/docs/_reference/kubes-exec.md

@@ -21,13 +21,20 @@ The exec command finds the latest pod from the deployment and runs `kubectl exec
 
 ## Multiple Deployments
 
-If you have have multiple deployments in your `.kubes/resources` then the command will use the first deployment by default. You can specify the specfic deployment with the `--name` option.  Examples:
+If you have have multiple deployments in your `.kubes/resources` then the command will use the first deployment by default. You can specify the specfic deployment with the `--name` or `-n` option.  Examples:
 
-    kubes exec --name demo-web
-    kubes exec --name demo-clock
-    kubes exec --name demo-worker
-    kubes exec --name demo-web sh
-    kubes exec --name demo-web ls -l
+    kubes exec --name web
+    kubes exec -n web
+    kubes exec -n clock
+    kubes exec -n worker
+    kubes exec -n web sh
+    kubes exec -n web ls -l
+
+## Multiple Pod Containers
+
+If you have have multiple containers in your pod. You can specify the specfic container with the `--container` or `-c` option.  Examples:
+
+    kubes exec --name web
 
 
 ## Options
@@ -36,6 +43,7 @@ If you have have multiple deployments in your `.kubes/resources` then the comman
     [--compile], [--no-compile]  # whether or not to compile the .kube/resources
                                  # Default: true
 n, [--name=NAME]                 # deployment name to use. IE: demo-web
+c, [--container=CONTAINER]       # Container name. If omitted, the first container in the pod will be chosen
     [--verbose], [--no-verbose]  
     [--noop], [--no-noop]        
 ```