kubes 0.5.1 → 0.6.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 19bc1aa1219b83ad85fb8ac5e2b5e8ed463e97e546927658127b75afc212fd77
-  data.tar.gz: 9cc42229b38fbc1d3eaa3c289133b2bb17fe4677fb6c8603dfd793c6c377ed33
+  metadata.gz: ce0d27f928a17e1013e97242990213cd4b375aee110dc14d3effc9aa8df2d490
+  data.tar.gz: 983312b9f64da2641ec8286dc76b9e7582dac9a575f77222ab4e9f9a145d26dd
 SHA512:
-  metadata.gz: 566c1e5d878bebdd41d8eba2639e2b176896a33d103388b6380ad9524d52613dfbe55ee8e9b328cdab43cc732d9194d7908a1de4543e1e24f6daf3b83d50274b
-  data.tar.gz: 1ee99b1c620161b5b02b3443327c0757a8bce94de80e5f6c8c3d8bced6bfdb116035f71188eb7fec6b535bb228577a3705e9a52866af8134cf642ea9ba7937f4
+  metadata.gz: 36dc2af17c87d9c58eca05f8f63848f57d95cccdfcd51894ddb467c7424c7eefade4aed9133c871f400f05997143d1f537aeb38c1d8288815ab05558f25ae31c
+  data.tar.gz: 3ed8687d0f1af629a7645a002536e7a187057ea2da504e52fb3227da933b4a0c98db8ad0f1170a2a720248cbe517f7206ad8941299467695772d5fca289c1160

data/.gcloudignore

@@ -0,0 +1,22 @@
+*.gem
+*.rbc
+/.bundle
+/.config
+/.yardoc
+/_yardoc
+/coverage
+/doc/
+/Gemfile.lock
+/InstalledFiles
+/lib/bundler/man
+/pkg
+/rdoc
+/spec/reports
+/test/tmp
+/test/version_tmp
+/tmp
+
+.git
+pkg
+docs
+spec

data/.gitignore

@@ -14,4 +14,4 @@
 /spec/reports
 /test/tmp
 /test/version_tmp
-/tmp
+/tmp

data/CHANGELOG.md

@@ -3,6 +3,28 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.6.4] - 2020-11-11
+- [#38](https://github.com/boltops-tools/kubes/pull/38) fix auto auth for docker login to registry, docs for secret base64, update dependencies
+
+## [0.6.3] - 2020-11-11
+- [#37](https://github.com/boltops-tools/kubes/pull/37) Dockerfile for ci and hook updates
+
+## [0.6.2]
+- [#36](https://github.com/boltops-tools/kubes/pull/36) add plugin hooks support
+
+## [0.6.1]
+- update gemspec dependency to plugins that provide the secrets helpers
+
+## [0.6.0]
+- [#35](https://github.com/boltops-tools/kubes/pull/35) mix layering support: evaluate DSL so layering can be mixed between YAML and DSL docs: https://kubes.guru/docs/layering/mix/
+- custom variables support: docs https://kubes.guru/docs/variables/basic/
+- custom helpers support: docs https://kubes.guru/docs/helpers/custom/
+- plugins helpers support
+- generators: new resource, new helper, new variable
+- setup autoloader earlier. removes need for shims
+- auth login for gcr also
+- fix cli -h when not within Kubes project
+
 ## [0.5.1]
 - fix deployment generator
 

data/Dockerfile

@@ -1,10 +1,10 @@
-FROM ruby:2.7-alpine
+FROM ruby:2.7
 
-RUN apk add --no-cache docker
-RUN apk add --no-cache build-base ruby ruby-dev
-
-RUN wget https://storage.googleapis.com/kubernetes-release/release/v1.18.6/bin/linux/amd64/kubectl
-RUN chmod u+x kubectl && mv kubectl /bin/kubectl
+COPY docker docker
+RUN docker/install/docker.sh
+RUN docker/install/gcloud.sh
+ENV PATH=/opt/google/google-cloud-sdk/bin/:$PATH
+RUN docker/install/kubectl.sh
 
 WORKDIR /app
 ADD . /app

data/Dockerfile.alpine

@@ -0,0 +1,20 @@
+FROM ruby:2.7-alpine
+
+# This Dockerfile is much lighter but won't work with gke whitelisting. Getting this error when the google gke sdk is called:
+#
+#    Error loading shared library ld-linux-x86-64.so.2: No such file or directory #986
+#
+# If you don't need gke whitelisting, then this image should work and is lighter.
+
+RUN apk add --no-cache docker
+RUN apk add --no-cache build-base ruby ruby-dev
+
+RUN wget https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl
+RUN chmod u+x kubectl && mv kubectl /bin/kubectl
+
+WORKDIR /app
+ADD . /app
+RUN bundle install
+RUN rake install
+
+ENTRYPOINT ["/usr/local/bundle/bin/kubes"]

data/README.md

@@ -18,13 +18,6 @@ Kubes will:
 2. Compile Kubernetes YAML files from YAML/ERB or a DSL and adjusts the Docker build image
 3. Deploy via kubectl apply on the compiled Kubernetes YAML files
 
-Features:
-
-* Automation: [Builds the Docker image](docs/docker.md) and updates the compiled YAML files
-* Syntactic Sugar: Use an optional [ERB/YAML](docs/yaml.md) or [DSL](docs/dsl.md) to write your Kubernetes YAML files. You can use a mix of DSL and YAML definitions in the `.kubes/resources` folder.
-* Layering: Use the same Kubernetes YAML to build multiple environments like dev and prod with [layering](docs/layering.md).
-* CLI Customizations: You can customize the [cli args](docs/kubectl.md). You can also run hooks before and after kubectl commands.
-
 ## Usage
 
     kubes init # creates .kubes structure
@@ -55,10 +48,63 @@ The deploy command, does all 3 steps: builds the docker image, compiles the `.ku
 
     kubes deploy
 
+## Multiple Enviroments
+
+You can easily create multiple environments with the same YAML configs. Example:
+
+    KUBES_ENV=dev  kubes deploy
+    KUBES_ENV=prod kubes deploy
+
+See: [Multiple Enviroments Pattern](https://kubes.guru/docs/patterns/multiple-envs/)
+
+## Generators: Stop Writing Boilerplate
+
+Your time is precious. Why are we copying and pasting boilerplate structure in this day and age?
+
+Kubes provides generators to help you get going right away.
+
+Resources examples:
+
+    $ kubes new resource secret
+          create  .kubes/resources/shared/secret.yaml
+    $ kubes new resource service_account
+          create  .kubes/resources/shared/service_account.yaml
+
+Kubes components examples:
+
+    $ kubes new helper
+          create  .kubes/helpers/custom_helper.rb
+    $ kubes new variable
+          create  .kubes/variables/dev.rb
+    $
+
+## Features
+
+* Automation: [Builds the Docker image](https://kubes.guru/docs/config/docker/) and updates the compiled YAML files
+* Syntactic Sugar: Use an [ERB/YAML](https://kubes.guru/docs/yaml/) or a [DSL](https://kubes.guru/docs/dsl/) to write your Kubernetes YAML files. You can use a mix of DSL and YAML definitions in the `.kubes/resources` folder.
+* Layering: Use the same Kubernetes YAML to build multiple environments like dev and prod with [layering](https://kubes.guru/docs/layering/).
+* Secrets: Use helpers like [aws_secret](https://kubes.guru/docs/helpers/aws/secrets/), [aws_ssm](https://kubes.guru/docs/helpers/aws/ssm/), and [google_secret](https://kubes.guru/docs/helpers/google/secrets/) to build Kubernetes secrets.yaml from secret providers designed for it.
+* Generators: Kubes ships with a few generators to help you get building with Kubernetes quickly. See: [Generator Docs](https://kubes.guru/docs/generators/).
+* CLI Customizations: You can customize the [cli args](https://kubes.guru/docs/config/args/kubectl/).
+* Hooks: You can also run [hooks](https://kubes.guru/docs/config/hooks/) before and after [kubes](https://kubes.guru/docs/config/hooks/kubes/) and [kubectl](https://kubes.guru/docs/config/hooks/kubectl/) commands.
+* Automated Suffix Hashes: Automatically appends a suffix hash to ConfigMap and Secret resources. More details in [ConfigMap](https://kubes.guru/docs/dsl/resources/config_map/) and [Secret](https://kubes.guru/docs/dsl/resources/secret/) docs.
+* Kustomize Support: If you're a kustomization user, you can use it with Kubes. More details in [Kustomize Support Docs](https://kubes.guru/docs/misc/kustomize/).
+* Auto Context Switching: Map dev to a specific kubectl context and prod to another kubectl context and Kubes can switch them automatically so you won't have to remember. More details in [Auto Context Docs](https://kubes.guru/docs/misc/auto-context/).
+* Ordering: Kubes run kubectl apply to create resources in the [correct order](https://kubes.guru/docs/intro/ordering/). For deleting, it kubes will run `kubectl delete` in the correct reverse order. The order is also [customizable](https://kubes.guru/docs/intro/ordering/custom/).
+
 ## Installation
 
 Install with:
 
     gem install kubes
 
+## Comparison
+
+Here are some useful comparisons to help you compare Kubes vs other tools in the ecosystem:
+
+* Blog Post: [Kustomize vs Helm vs Kubes: Kubernetes Deploy Tools](https://blog.boltops.com/2020/11/05/kustomize-vs-helm-vs-kubes-kubernetes-deploy-tools)
+* [Kubes vs Custom Solution](https://kubes.guru/docs/vs/custom/)
+* [Kubes vs Helm](https://kubes.guru/docs/vs/helm/)
+* [Kubes vs Kustomize](https://kubes.guru/docs/vs/kustomize/)
+
 For more info: [kubes.guru](https://kubes.guru)

data/docker/install/docker.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+apt-get update
+apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y
+curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
+add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
+apt-get update
+apt-get install docker-ce docker-ce-cli containerd.io -y

data/docker/install/gcloud.sh

@@ -0,0 +1,18 @@
+#!/bin/bash -eu
+
+[ -e /opt/google ] && exit
+
+mkdir -p /opt/google
+
+cd /opt/google
+wget https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-318.0.0-linux-x86_64.tar.gz
+tar zxf google-cloud-sdk*.tar.gz
+rm -f google-cloud-sdk*.tar.gz
+
+/opt/google/google-cloud-sdk/install.sh -q
+
+cat << FOE >> ~/.bash_profile
+
+source /opt/google/google-cloud-sdk/completion.bash.inc
+source /opt/google/google-cloud-sdk/path.bash.inc
+FOE

data/docker/install/kubectl.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+wget https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl
+chmod u+x kubectl && mv kubectl /bin/kubectl

data/docs/_docs/config/hooks/kubes.md

@@ -13,6 +13,7 @@ Hook | Description
 compile | When kubes compiles the `.kubes/resources` to `.kubes/output`.
 apply | When kubes runs all the `kubectl apply` commands.
 delete | When kubes runs all the `kubectl delete` commands.
+prune | When kubes prunes. IE: To clean old secrets.
 
 ## Lifecycle At Kubes Level
 

data/docs/_docs/config/reference.md

@@ -16,6 +16,7 @@ kubectl.order.roles | Change ordering for Kubes Roles. | See [source code](https
 logger | Logger object | Logger.new($stdout)
 logger.level | Logger level. Can also be set with `KUBES_LOG_LEVEL` env var | info
 repo | The Docker repo to use. Required to be set. | nil
+repo_auto_auth | Whether or not to try to auth authorize docker repo registry if not yet logged in. Can also be set with env var `KUBES_REPO_AUTO_AUTO` | true
 skip | List of resources to skip. Can also be set with the `KUBES_SKIP` env var. `KUBES_SKIP` should be a list of strings separated by spaces. It adds onto the `config.skip` option. | []
 state.docker_image_path | Where to store the state file with the last build Docker image. | .kubes/state/docker_image.txt
 suffix_hash | Whether or not to append suffix hash to ConfigMap and Secret | true

data/docs/_docs/dsl/multiple-resources.md

@@ -39,7 +39,9 @@ Using multiple files is the general recommended approach.
 
 ## Multiple Resources: Block Form
 
-You can also use a block form to create multiple resources.  You name the resource files with plural names. An example helps explain:
+You can also use a block form to create multiple resources.  The multiple resources block form is an experimental feature.
+
+You name the resource files with plural names. An example helps explain:
 
     .kubes
     └── resources

data/docs/_docs/dsl/resources/secret.md

@@ -19,7 +19,7 @@ data(
 
 Produces:
 
-.kubes/output/shared/service.yaml
+.kubes/output/shared/secret.yaml
 
 ```yaml
 apiVersion: v1
@@ -38,6 +38,23 @@ data:
 
 {% include dsl/rolling_deployment.md kind="Secret" %}
 
+.kubes/resources/web/deployment.yaml:
+
+```yaml
+# ..
+spec:
+  template:
+    spec:
+      containers:
+      - name: demo
+        image: nginx
+        envFrom:
+        - secretRef:
+            name: demo-secret
+```
+
+Produces:
+
 .kubes/output/web/deployment.yaml:
 
 ```yaml
@@ -46,7 +63,7 @@ spec:
   template:
     spec:
       containers:
-      - name: demo-shared
+      - name: demo
         image: nginx
         envFrom:
         - secretRef:

data/docs/_docs/generators.md

@@ -8,15 +8,15 @@ Kubes ships with a few generators to help you get building with Kubernetes quick
 
 Here are a few examples:
 
-    $ kubes new ingress
+    $ kubes new resource ingress
           create  .kubes/resources/web/ingress.yaml
-    $ kubes new service_account
+    $ kubes new resource service_account
           create  .kubes/resources/shared/service_account.yaml
     $
 
 Use `-h` to see the cli options:
 
-    kubes new -h
+    kubes new resource -h
 
 ## Supported Resources
 
@@ -38,4 +38,4 @@ Here's a list of some of the supported resources.
     service_account
     service
 
-Refer to the [source code](https://github.com/boltops-tools/kubes/blob/master/lib/templates/new/yaml) to all the resources that the generator supports.
+Refer to the [source code](https://github.com/boltops-tools/kubes/blob/master/lib/templates/new/resource/yaml) to all the resources that the generator supports.

data/docs/_docs/helpers.md

@@ -6,10 +6,10 @@ Kubes provides some helper methods to help write Kubernetes YAML files.  Here's
 
 Helper | Description
 --- | ---
-decode64 | Basey64 decode a string.
+decode64 | Base64 decode a string.
 docker_image | Method refers to the latest Docker image built by Kubes. This spares you from having to update the image manually in the deployment resource. Note, this can be overridden with the `--image` cli option or the `Kubes.config.image` setting. See: [Docker Image]({% link _docs/intro/docker-image.md %})
 dockerfile_port	| Exposed port extracted from the Dockerfile of the project.
-encode64 | Basey64 encode a string. Also available as `base64` method.
+encode64 | Base64 encode a string. Also available as `base64` method.
 extra | The `KUBES_EXTRA` value.
 with_extra | Appends the `KUBES_EXTRA` value to a string if it's set. It's covered in the [Extra Env Docs]({% link _docs/extra-env.md %}).
 
@@ -25,3 +25,5 @@ There are also provider-specific helpers:
 
 * [AWS Helpers]({% link _docs/helpers/aws.md %})
 * [Google Helpers]({% link _docs/helpers/google.md %})
+
+{% include helpers/generator.md %}

data/docs/_docs/helpers/aws/advanced.md

@@ -0,0 +1,10 @@
+---
+title: Advanced AWS Helpers
+nav_text: Advanced
+categories: helpers-aws
+---
+
+{% assign docs = site.docs | where: "categories","advanced-helpers-aws" %}
+{% for doc in docs -%}
+  * [{{ doc.nav_text }}]({{ doc.url }})
+{% endfor %}

data/docs/_docs/helpers/aws/advanced/secrets.md

@@ -0,0 +1,131 @@
+---
+title: AWS Secrets Advanced
+nav_text: Secrets
+categories: advanced-helpers-aws
+---
+
+This covers an advanced way so that Kubernetes Secrets are created from AWS Secrets Manager in a conventional way.
+
+## Simple Values
+
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_user | jq '.SecretString'
+    user
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_pass | jq '.SecretString'
+    pass
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+secrets = KubesAws::Secrets.new(upcase: true, prefix: "demo/dev/")
+before("compile",
+  label: "Get secrets from AWS Secrets Manager",
+  execute: secrets,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesAws::Secrets.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## JSON Values
+
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/k2 | jq '.SecretString'
+    {\"a\":1,\"b\":2}"
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+secrets = KubesAws::Secrets.new(prefix: "rails/dev/")
+before("compile",
+  label: "Get secrets from AWS Secrets Manager",
+  execute: secrets,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% k2 = JSON.load(KubesAws::Secrets.data["k2"]) %>
+  a: <%= base64(k2["a"]) %>
+  b: <%= base64(k2["b"]) %>
+```
+
+Produces:
+
+```yaml
+metadata:
+  namespace: demo-dev
+  name: demo-a4cd604a95
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  a: MQ==
+  b: Mg==
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+AWS_SECRET_PREFIX | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}