kubes 0.3.5 → 0.4.0

data/docs/_docs/config/kubectl.md

@@ -6,5 +6,5 @@ title: Kubectl Customizations
 
 Kubes calls out the `kubectl` command. You can customize the command.
 
-* [Args]({% link _docs/config/kubectl/args.md %}): Customize the CLI args.
-* [Hooks]({% link _docs/config/kubectl/hooks.md %}): Run hooks before and after the kubectl commands.
+* [Args]({% link _docs/config/args/kubectl.md %}): Customize the CLI args.
+* [Hooks]({% link _docs/config/hooks/kubectl.md %}): Run hooks before and after the kubectl commands.

data/docs/_docs/config/reference.md

@@ -0,0 +1,20 @@
+---
+title: Config Reference
+---
+
+Name | Description | Default
+---|---|---
+auto_prune | Prune and delete old hashed resources like Secret and ConfigMap. | true
+builder | What docker build command to use. Can use `docker` or `gcloud` to build the Docker image. | docker
+kubectl.context | What kubectl context to auto-switch to. | nil
+kubectl.context_keep | Whether or not to keep the context switched | true
+kubectl.exit_on_fail.apply  | Whether or not continue if the `kubectl apply` fails. Note, can use `KUBES_EXIT_ON_FAIL=0` env var to set to false. | true
+kubectl.exit_on_fail.delete | Whether or not continue if the `kubectl delete` fails. | false
+kubectl.order.kinds | Change ordering for Kubernetes Kinds. | See [source code](https://github.com/boltops-tools/kubes/blob/master/lib/kubes/config.rb#L52)
+kubectl.order.roles | Change ordering for Kubes Roles. | See [source code](https://github.com/boltops-tools/kubes/blob/master/lib/kubes/config.rb#L44)
+logger | Logger object | Logger.new($stdout)
+logger.level | Logger level. Can also be set with `KUBES_LOG_LEVEL` env var | info
+repo | The Docker repo to use. Required to be set. | nil
+skip | List of resources to skip. Can also be set with the `KUBES_SKIP` env var. `KUBES_SKIP` should be a list of strings separated by spaces. It adds onto the `config.skip` option. | []
+state.docker_image_path | Where to store the state file with the last build Docker image. | .kubes/state/docker_image.txt
+suffix_hash | Whether or not to append suffix hash to ConfigMap and Secret | true

data/docs/_docs/config/skip.md

@@ -0,0 +1,58 @@
+---
+title: Skip Option
+---
+
+You can tell Kubes to skip resources to deploy. This can useful if you want to still resources with Kubes and have it compile `.kubes/output` files, but wish to deploy them outside of Kubes manually.
+
+## Example
+
+Here's an example with a Job.
+
+.kubes/resources/cleanup/job.yaml:
+
+```yaml
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: cleanup
+spec:
+  template:
+    spec:
+      containers:
+      - name: cleanup
+        image: <%= built_image %>
+        command: ["bin/cleanup.sh"]
+      restartPolicy: Never
+```
+
+To skip the cleanup job, use the `config.skip` option:
+
+```ruby
+Kubes.configure do |config|
+  config.skip = ["cleanup/job"]
+end
+```
+
+Now when you deploy, the `cleanup/job` resource will not be deployed:
+
+    kubes deploy # deploys everything except cleanup/job
+
+## Deploy Outside of Kubes
+
+Then to deploy outside of kubes.
+
+    $ kubes compile # not necessary if already ran: kubes deploy
+    Compiled  .kubes/resources files to .kubes/output
+    $ kubectl apply -f .kubes/output/cleanup/job.yaml
+    job.batch/cleanup created
+    $ kubectl delete -f .kubes/output/cleanup/job.yaml
+    job.batch "cleanup" deleted
+    $
+
+## Env Var KUBES_SKIP
+
+You can also us ethe `KUBES_SKIP` env var. It takes list of strings separated by a space. It adds onto the `config.skip` option. Example:
+
+    KUBES_SKIP="cleanup/job" kubes delete
+
+This can be useful for one-off use cases.

data/docs/_docs/dsl/resources.md

@@ -6,7 +6,7 @@ Here's a list of the resources supported by the Kubes DSL.
 
 {% assign docs = site.docs | where: "categories","dsl" %}
 {% for doc in docs -%}
-* [{{ doc.title }}]({{ doc.url }})
+* [{{ doc.nav_text }}]({{ doc.url }})
 {% endfor %}
 
 For resources, that are not supported, you can use the [Generic resource]({% link _docs/dsl/resources/generic.md %}) or use [YAML]({% link _docs/yaml.md %}) instead. You can use a mix of DSL and YAML definitions in the `.kubes/resources` folder.

data/docs/_docs/intro.md

@@ -4,12 +4,14 @@ title: What is Kubes?
 
 {% include reference.md %}
 
+<div class="video-box"><div class="video-container"><iframe src="https://www.youtube.com/embed/M4zHL0mfKNU" frameborder="0" allowfullscreen=""></iframe></div></div>
+
 ## Features:
 
 * Automation: [Builds the Docker image]({% link _docs/config/docker.md %}) and updates the compiled YAML files
 * Syntactic Sugar: Use an [ERB/YAML]({% link _docs/yaml.md %}) or a [DSL]({% link _docs/dsl.md %}) to write your Kubernetes YAML files. You can use a mix of DSL and YAML definitions in the `.kubes/resources` folder.
 * Layering: Use the same Kubernetes YAML to build multiple environments like dev and prod with [layering]({% link _docs/layering.md %}).
-* CLI Customizations: You can customize the [cli args]({% link _docs/config/kubectl/args.md %}). You can also run [hooks]({% link _docs/config/kubectl/hooks.md %}) before and after kubectl commands.
+* CLI Customizations: You can customize the [cli args]({% link _docs/config/args/kubectl.md %}). You can also run [hooks]({% link _docs/config/hooks/kubectl.md %}) before and after kubectl commands.
 * Automated Suffix Hashes: Automatically appends a suffix hash to ConfigMap and Secret resources. More details in [ConfigMap]({% link _docs/dsl/resources/config_map.md %}) and [Secret]({% link _docs/dsl/resources/secret.md %}) docs.
 * Kustomize Support: If you’re a kustomization user, you can use it with Kubes. More details in [Kustomize Support Docs]({% link _docs/misc/kustomize.md %}).
 * Auto Context Switching: Map dev to a specific kubectl context and prod to another kubectl context and Kubes can switch them automatically so you won't have to remember. More details in [Auto Context Docs]({% link _docs/misc/auto-context.md %}).

data/docs/_docs/patterns/migrations.md

@@ -0,0 +1,121 @@
+---
+title: Database Migrations
+---
+
+A common task is to run database migrations. You can use Kubes hooks to achieve this as part of the `kubes deploy` process.
+
+1. Create Migrate Job YAML
+2. Set up Kubes Hooks
+
+## 1. Create Migrate Job YAML
+
+First, let's create the migrate job YAML. Here's a starter example:
+
+.kubes/resources/migrate/job.yaml
+
+```yaml
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: migrate
+spec:
+  template:
+    spec:
+      containers:
+      - name: migrate
+        image: <%= built_image %>
+        command: ["bin/job/migrate.sh"]
+      restartPolicy: Never
+  backoffLimit: 4
+```
+
+The Kubernetes job calls a `job/migrate.sh` script. Something like this:
+
+bin/job/migrate.sh
+
+    #!/bin/bash
+    rails db:migrate
+
+## 2. Set up Kubes Hooks
+
+Set up the [kubes hooks]({% link _docs/config/hooks/kubectl.md %}) to help the migrate job run properly.
+
+.kubes/config/hooks/kubectl.rb
+
+```ruby
+before("apply",
+  on: "migrate/job",
+  execute: "bin/hooks/migrate/delete.sh",
+  exit_on_fail: false,
+)
+
+after("apply",
+  on: "migrate/job",
+  execute: "bin/hooks/migrate/wait.sh",
+)
+```
+
+Here's what the `bin/hook/migrate` scripts could look like:
+
+bin/hooks/migrate/delete.sh
+
+    #!/bin/bash
+    kubectl delete job/migrate
+
+bin/hooks/migrate/wait.sh
+
+    #!/bin/bash
+    kubectl wait --for=condition=Complete job/migrate --timeout=300s
+
+The `migrate/delete.sh` script first cleans up old migrate jobs that may have been previously created.
+
+The `migrate/wait.sh` script waits until the migration job finishes before continuing. Note, the default timeout is 30s, which may not be long enough for your migrations to finish, so we set it to 300s.  The `kubectl wait` only returns if the migrate job finishes successfully. If the job fails after it exhausts all its retries, default 6, then you'll see an error like this:
+
+    + kubectl wait --for=condition=Complete job/migrate --timeout=30s
+    error: timed out waiting for the condition on jobs/migrate
+    ERROR: running bin/hooks/migrate.sh
+
+There is also an [migration-example](https://github.com/boltops-tools/kubes-examples/tree/master/yaml/migration-example) repo with a smarter version of the wait script.
+
+## Example Deploy
+
+Once that is set up, a `kubes deploy` will automatically run migrations. Here's an example deploy:
+
+    $ kubes deploy
+    => kubectl apply -f .kubes/output/shared/namespace.yaml
+    => bin/hooks/migrate/delete.sh
+    job.batch "migrate" deleted
+    => kubectl apply -f .kubes/output/migrate/job.yaml
+    job.batch/migrate created
+    Running hook: after apply on: migrate/job
+    => bin/hooks/migrate/wait.sh
+    Sun Oct 11 03:22:35 UTC 2020
+    Migration complete
+    => kubectl apply -f .kubes/output/web/service.yaml
+    service/web unchanged
+    => kubectl apply -f .kubes/output/web/deployment.yaml
+    deployment.apps/web configured
+    $
+
+## To Couple or Not to Couple?
+
+While some companies prefer running the migration step as a part of the app deploy, some prefer to separate it out as a discrete step. Usually, the separate step is still called as part of a pipeline.
+
+In practice, the decision usually comes down to:
+
+* The size of your database. If your database is large and the migrations take a long time to run. It makes sense to separate it out.
+* The risk tolerance of database migration operations. If it's quite risky to run DB migrations, you may want to separate it as discrete step so a human can review it.
+
+For small apps and databases, it's often pragmatic to just run everything in a single step for simplicity.
+
+## Migration as Separate Step
+
+If you would like it to run it as a discrete step, remove the hook in `.kubes/config/hooks/kubectl.rb`, and run it as a separate script like so:
+
+bin/run/migrate.sh
+
+    #!/bin/bash
+    kubes compile
+    bin/hooks/migrate/delete.sh
+    bin/job/migrate.sh
+    bin/hooks/migrate/wait.sh

data/docs/_includes/config/hooks/options.md

@@ -0,0 +1,20 @@
+## General Form
+
+```ruby
+before(COMMAND_NAME, OPTIONS)
+````
+
+The command name corresponds to the `{{ include.command }}` commands: apply, delete, etc.
+
+## Hook Options
+
+Name | Description
+---|---
+label | A human-friendly label so you can see what hooks is being run during `kubes apply`
+execute | The script or command to run. IE: path/to/some/script.sh
+exit_on_fail | Whether or not to continue process if the script returns an failed exit code.
+{% if include.command == "kubectl" %}on | What resource to run the hook on. IE: shared/namespace, web/deployment, web/service. Note: This option is only used by kubectl hooks.{% endif %}
+
+## Ruby Hooks
+
+Instead of using a script for the hook `execute` option, you can also use a Ruby object. This provides some more control over the current process. See: [Ruby Hooks]({% link _docs/config/hooks/ruby.md %})

data/docs/_includes/sidebar.html

@@ -53,18 +53,37 @@
       </li>
       <li><a href="{% link _docs/config.md %}">Config</a>
         <ul>
-          <li><a href="{% link _docs/config/docker.md %}">Docker</a></li>
-          <li><a href="{% link _docs/config/env.md %}">Env</a></li>
-          <li><a href="{% link _docs/config/kubectl.md %}">Kubectl</a>
+          <li><a href="{% link _docs/config/args.md %}">Args</a>
             <ul>
-              <li><a href="{% link _docs/config/kubectl/args.md %}">Args</a></li>
-              <li><a href="{% link _docs/config/kubectl/hooks.md %}">Hooks</a></li>
+            {% assign docs = site.docs | where: "categories","args" %}
+            {% for doc in docs -%}
+              <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+            {% endfor %}
             </ul>
           </li>
+          <li><a href="{% link _docs/config/hooks.md %}">Hooks</a>
+            <ul>
+            {% assign docs = site.docs | where: "categories","hooks" %}
+            {% for doc in docs -%}
+              <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+            {% endfor %}
+            </ul>
+          </li>
+          <li><a href="{% link _docs/config/docker.md %}">Docker</a></li>
+          <li><a href="{% link _docs/config/env.md %}">Env</a></li>
           <li><a href="{% link _docs/config/builder.md %}">Builder</a></li>
+          <li><a href="{% link _docs/config/skip.md %}">Skip Option</a></li>
+          <li><a href="{% link _docs/config/reference.md %}">Reference</a></li>
         </ul>
       </li>
       <li><a href="{% link _docs/yaml.md %}">YAML</a></li>
+      <li><a href="{% link _docs/layering.md %}">Layering</a>
+        <ul>
+          <li><a href="{% link _docs/layering/yaml.md %}">YAML</a></li>
+          <li><a href="{% link _docs/layering/dsl.md %}">DSL</a></li>
+          <li><a href="{% link _docs/layering/merge.md %}">Merge Behavior</a></li>
+        </ul>
+      </li>
       <li><a href="{% link _docs/dsl.md %}">DSL</a>
         <ul>
           <li><a href="{% link _docs/dsl/resources.md %}">Resources</a>
@@ -78,16 +97,10 @@
           <li><a href="{% link _docs/dsl/multiple-resources.md %}">Multiple Resources</a>
         </ul>
       </li>
-      <li><a href="{% link _docs/layering.md %}">Layering</a>
-        <ul>
-          <li><a href="{% link _docs/layering/yaml.md %}">YAML</a></li>
-          <li><a href="{% link _docs/layering/dsl.md %}">DSL</a></li>
-          <li><a href="{% link _docs/layering/merge.md %}">Merge Behavior</a></li>
-        </ul>
-      </li>
       <li><a href="{% link _docs/helpers.md %}">Helpers</a></li>
       <li><a href="{% link _docs/patterns.md %}">Patterns</a>
         <ul>
+          <li><a href="{% link _docs/patterns/migrations.md %}">Database Migrations</a></li>
           <li><a href="{% link _docs/patterns/clock-web-worker.md %}">Clock Web Worker</a></li>
         </ul>
       </li>

data/docs/_sass/theme.scss

@@ -290,4 +290,28 @@ ul.toc {
     text-align: center;
     padding: 20px;
   }
-}
+}
+
+// https://coolestguidesontheplanet.com/videodrome/youtube/
+// I added another box wrapper to control the width
+.video-box {
+  max-width: 600px;
+  text-align: left;
+  margin: 0 auto 0 0;
+  padding-bottom: 20px;
+  .video-container {
+    position:relative;
+    padding-bottom:56.25%;
+    padding-top:30px;
+    height:0;
+    overflow:hidden;
+  }
+
+  .video-container iframe, .video-container object, .video-container embed {
+    position:absolute;
+    top:0;
+    left:0;
+    width:100%;
+    height:100%;
+  }
+}

data/lib/kubes/cli.rb

@@ -9,12 +9,18 @@ module Kubes
     compile_option = Proc.new {
       option :compile, type: :boolean, default: true, desc: "whether or not to compile the .kube/resources"
     }
-    name_option = Proc.new {
-      option :name, aliases: %w[n], desc: "deployment name to use. IE: demo-web"
+    pod_option = Proc.new {
+      option :pod, aliases: %w[p], desc: "pod to use. IE: web"
+    }
+    deployment_option = Proc.new {
+      option :deployment, aliases: %w[d], desc: "deployment name to use. IE: demo-web"
     }
     container_option = Proc.new {
       option :container, aliases: %w[c], desc: "Container name. If omitted, the first container in the pod will be chosen"
     }
+    yes_option = Proc.new {
+      option :yes, aliases: %w[y], type: :boolean, desc: "Skip are you sure prompt"
+    }
 
     desc "docker SUBCOMMAND", "Docker subcommands"
     long_desc Help.text(:docker)
@@ -45,7 +51,7 @@ module Kubes
     desc "delete [ROLE] [RESOURCE]", "Delete Kubernetes resources within the app folder"
     long_desc Help.text(:delete)
     image_option.call
-    option :yes, aliases: %w[y], type: :boolean, desc: "Skip are you sure prompt"
+    yes_option.call
     def delete(role=nil, resource=nil)
       Delete.new(options.merge(role: role, resource: resource)).run
     end
@@ -69,7 +75,8 @@ module Kubes
     desc "exec", "Exec into the latest container from the deployment"
     long_desc Help.text(:exec)
     compile_option.call
-    name_option.call
+    pod_option.call
+    deployment_option.call
     container_option.call
     def exec(*cmd)
       Exec.new(options.merge(cmd: cmd)).run
@@ -88,13 +95,21 @@ module Kubes
     desc "logs", "logs from all deployment pods"
     long_desc Help.text(:logs)
     compile_option.call
-    name_option.call
+    pod_option.call
+    deployment_option.call
     container_option.call
     option :follow, aliases: %w[f], type: :boolean, default: true, desc: "Follow logs"
     def logs(*cmd)
       Logs.new(options.merge(cmd: cmd)).run
     end
 
+    desc "prune", "Prune old resources like secret and config maps"
+    long_desc Help.text(:prune)
+    yes_option.call
+    def prune
+      Prune.new(options).run
+    end
+
     long_desc Help.text(:init)
     Init.options.each { |args| option(*args) }
     register(Init, "init", "init", "Init project")

data/lib/kubes/cli/apply.rb

@@ -3,7 +3,8 @@ class Kubes::CLI
     def run
       compile
       logger.info "Deploying kubes resources"
-      Kubes::Kubectl::Decider.new(:apply, @options).run
+      Kubes::Kubectl::Dispatcher.new(:apply, @options).run
+      Prune.new(@options.merge(yes: true, quiet: true)).run if Kubes.config.auto_prune # prune old secrets and config maps
     end
   end
 end

data/lib/kubes/cli/base.rb

@@ -9,5 +9,16 @@ class Kubes::CLI
     def compile
       Compile.new(@options).run unless @options[:compile] == false
     end
+
+    def pod_name
+      return unless @options[:pod]
+
+      pods = Kubes::Kubectl::Fetch::Pods.new(@options)
+      items = pods.fetch(:pod)
+      metas = items.map { |i| i['metadata'] }
+      metas.select! { |i| i['name'].include?(@options[:pod]) }
+      meta = metas.sort { i['creationTimestamp'] }.last
+      meta['name'] if meta
+    end
   end
 end

data/lib/kubes/cli/delete.rb

@@ -10,7 +10,7 @@ class Kubes::CLI
     end
 
     def perform(preview: false)
-      Kubes::Kubectl::Decider.new(:delete, @options.merge(preview: preview)).run
+      Kubes::Kubectl::Dispatcher.new(:delete, @options.merge(preview: preview)).run
     end
   end
 end