kubes 0.5.1 → 0.6.4

data/docs/_docs/helpers/aws/advanced/ssm.md

@@ -0,0 +1,78 @@
+---
+title: AWS SSM Parameters Advanced
+nav_text: SSM
+categories: advanced-helpers-aws
+---
+
+This covers an advanced way so that Kubernetes Secrets are created from AWS SSM Parameter Store in a conventional way.
+
+For example if you have these secret values:
+
+    $ aws ssm get-parameter --name /demo/development/db_user --with-decryption | jq '.Parameter.Value'
+    user
+    $ aws ssm get-parameter --name /demo/development/db_pass --with-decryption | jq '.Parameter.Value'
+    pass
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+ssm = KubesAws::SSM.new(upcase: true, prefix: "/demo/development/")
+before("compile",
+  label: "Get secrets from AWS SSM Manager",
+  execute: ssm,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesAws::SSM.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+AWS_SSM_PREFIX | Prefixed used to list and filter AWS SSM Parameters. IE: `demo/dev/`.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/aws/secrets.md

@@ -4,28 +4,9 @@ nav_text: Secrets
 categories: helpers-aws
 ---
 
-## Simple Values
+The `aws_secret` helper fetches secret data from AWS Secrets Manager.
 
-For example if you have these secret values:
-
-    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_user | jq '.SecretString'
-    user
-    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_pass | jq '.SecretString'
-    pass
-
-Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
-
-.kubes/config/hooks/kubes.rb
-
-```ruby
-secrets = KubesAws::Secrets.new(upcase: true, prefix: "demo/dev/")
-before("compile",
-  label: "Get secrets from AWS Secrets Manager",
-  execute: secrets,
-)
-```
-
-Then set the secrets in the YAML:
+## Example
 
 .kubes/resources/shared/secret.yaml
 
@@ -37,12 +18,17 @@ metadata:
   labels:
     app: demo
 data:
-<% KubesAws::Secrets.data.each do |k,v| -%>
-  <%= k %>: <%= base64(v) %>
-<% end -%>
+  PASS: <%= aws_secret("demo-#{Kubes.env}-PASS") %>
+  USER: <%= aws_secret("demo-#{Kubes.env}-USER") %>
 ```
 
-This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo-dev-PASS | jq '.SecretString'
+    test1
+    $ aws secretsmanager get-secret-value --secret-id demo-dev-USER | jq '.SecretString'
+    test2
+    $
 
 .kubes/output/shared/secret.yaml
 
@@ -55,75 +41,31 @@ metadata:
 apiVersion: v1
 kind: Secret
 data:
-  db_pass: dGVzdDEK
-  db_user: dGVzdDIK
+  PASS: dGVzdDEK
+  USER: dGVzdDIK
 ```
 
-## JSON Values
+By default, the values are automatically base64 encoded.
 
-For example if you have these secret values:
-
-    $ aws secretsmanager get-secret-value --secret-id demo/dev/k2 | jq '.SecretString'
-    {\"a\":1,\"b\":2}"
+## Base64 Option
 
-Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+By default, the values are automatically base64 encoded. You can change the default behavior with a config option.
 
-.kubes/config/hooks/kubes.rb
+.kubes/config.rb
 
 ```ruby
-secrets = KubesAws::Secrets.new(prefix: "rails/dev/")
-before("compile",
-  label: "Get secrets from AWS Secrets Manager",
-  execute: secrets,
-)
+KubesAws.configure do |config|
+  config.base64_secrets = false
+end
 ```
 
-Then set the secrets in the YAML:
+Note: The use of `KubesAws.configure` instead of `Kubes.configure` here.
 
-.kubes/resources/shared/secret.yaml
+You can also set the `base64` option to turn on and off the automated base64 encoding on a per secret basis.
 
-```yaml
-apiVersion: v1
-kind: Secret
-metadata:
-  name: demo
-  labels:
-    app: demo
-data:
-<% k2 = JSON.load(KubesAws::Secrets.data["k2"]) %>
-  a: <%= base64(k2["a"]) %>
-  b: <%= base64(k2["b"]) %>
-```
-
-Produces:
-
-```yaml
-metadata:
-  namespace: demo-dev
-  name: demo-a4cd604a95
-  labels:
-    app: demo
-apiVersion: v1
-kind: Secret
-data:
-  a: MQ==
-  b: Mg==
+```ruby
+aws_secret("demo-#{Kubes.env}-USER", base64: true)  # default is base64=true
+aws_secret("demo-#{Kubes.env}-PASS", base64: false)
 ```
 
-## Variables
-
-These environment variables can be set:
-
-Name | Description
----|---
-AWS_SECRET_PREFIX | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`.
-
-Secrets#initialize options:
-
-Variable | Description | Default
----|---|---
-base64 | Automatically base64 encode the values. | false
-upcase | Automatically upcase the Kubernetes secret data keys. | false
-prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
-
 {% include helpers/base64.md %}

data/docs/_docs/helpers/aws/ssm.md

@@ -4,26 +4,9 @@ nav_text: SSM
 categories: helpers-aws
 ---
 
-For example if you have these secret values:
+The `aws_ssm` helper fetches data from AWS SSM Parameter Store.
 
-    $ aws ssm get-parameter --name /demo/development/db_user --with-decryption | jq '.Parameter.Value'
-    user
-    $ aws ssm get-parameter --name /demo/development/db_pass --with-decryption | jq '.Parameter.Value'
-    pass
-
-Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
-
-.kubes/config/hooks/kubes.rb
-
-```ruby
-ssm = KubesAws::SSM.new(upcase: true, prefix: "/demo/development/")
-before("compile",
-  label: "Get secrets from AWS SSM Manager",
-  execute: ssm,
-)
-```
-
-Then set the secrets in the YAML:
+## Example
 
 .kubes/resources/shared/secret.yaml
 
@@ -35,12 +18,16 @@ metadata:
   labels:
     app: demo
 data:
-<% KubesAws::SSM.data.each do |k,v| -%>
-  <%= k %>: <%= base64(v) %>
-<% end -%>
+  PASS: <%= aws_ssm("/demo/#{Kubes.env}/PASS") %>
+  USER: <%= aws_ssm("/demo/#{Kubes.env}/USER") %>
 ```
 
-This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+For example if you have these ssm parameter values:
+
+    $ aws ssm get-parameter --name /demo/dev/PASS --with-decryption | jq '.Parameter.Value'
+    test1
+    $ aws ssm get-parameter --name /demo/dev/USER --with-decryption | jq '.Parameter.Value'
+    test2
 
 .kubes/output/shared/secret.yaml
 
@@ -53,24 +40,19 @@ metadata:
 apiVersion: v1
 kind: Secret
 data:
-  db_pass: dGVzdDEK
-  db_user: dGVzdDIK
+  PASS: dGVzdDEK
+  USER: dGVzdDIK
 ```
 
-## Variables
-
-These environment variables can be set:
+The values are base64 encoded based on the SSM parameter type. When the type is a `SecureString`, Kubes base64 encodes it. Other types are not base64 encoded.  You can override this behavior with the base64 option, described next.
 
-Name | Description
----|---
-AWS_SSM_PREFIX | Prefixed used to list and filter AWS SSM Parameters. IE: `demo/dev/`.
+## Base64 Option
 
-Secrets#initialize options:
+The value is automatically base64 encoded based on whether or not the SSM parameter type is a `SecureString`. You can explicitly the `base64` option if needed though. Example:
 
-Variable | Description | Default
----|---|---
-base64 | Automatically base64 encode the values. | false
-upcase | Automatically upcase the Kubernetes secret data keys. | false
-prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+```ruby
+aws_ssm("/demo/#{Kubes.env}/USER", base64: true)  # default is base64=true
+aws_ssm("/demo/#{Kubes.env}/PASS", base64: false)
+```
 
-{% include helpers/base64.md %}
+{% include helpers/base64.md %}

data/docs/_docs/helpers/custom.md

@@ -38,3 +38,4 @@ data:
   DATABASE_ENDPOINT: <%= database_endpoint %>
 ```
 
+{% include helpers/generator.md %}

data/docs/_docs/helpers/google/advanced.md

@@ -0,0 +1,10 @@
+---
+title: Advanced Google Helpers
+nav_text: Advanced
+categories: helpers-google
+---
+
+{% assign docs = site.docs | where: "categories","advanced-helpers-google" %}
+{% for doc in docs -%}
+  * [{{ doc.nav_text }}]({{ doc.url }})
+{% endfor %}

data/docs/_docs/helpers/google/advanced/secrets.md

@@ -0,0 +1,78 @@
+---
+title:  Advanced Google Secrets
+nav_text: Secrets
+categories: advanced-helpers-google
+---
+
+This covers an advanced way so that Kubernetes Secrets are created from Google Secrets in a conventional way.
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+before("compile",
+  execute: KubesGoogle::Secrets.new(upcase: true, prefix: 'projects/686010496118/secrets/demo-dev-')
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesGoogle::Secrets.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in Google secrets with the prefix the `demo-dev-` being added to the Kubernetes secret data.  The values are automatically base64 encoded.
+
+For example if you have these secret values:
+
+    $ gcloud secrets versions access latest --secret demo-dev-db_user
+    test1
+    $ gcloud secrets versions access latest --secret demo-dev-db_pass
+    test2
+    $
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+GCP_SECRET_PREFIX | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`.
+GOOGLE_PROJECT | Google project id.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`. Can also be set with the `GCP_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/google/gke.md

@@ -0,0 +1,92 @@
+---
+title: GKE Whitelisting
+nav_text: GKE
+categories: helpers-google
+---
+
+This page covers how to enable GKE IP Whitelisting. This feature is useful for deploying from a CloudBuild with GKE Private Clusters.
+
+GKE Private Clusters whitelist and only allow authorized IPs to communicate with the Kubernetes control plane.  An issue with CloudBuild is that the IP address is not well-known.  Google creates a VM to run the CI scripts and throws it away when finished.  Kubes can detect the IP of the CloudBuild machine, add it to the cluster, deploy, and remove the IP afterward.
+
+## Setup
+
+To enable the GKE IP whitelisting feature, it's a single line:
+
+.kubes/config/env/dev.rb
+
+```ruby
+KubesGoogle.configure do |config|
+  config.gke.cluster_name = "dev-cluster"
+  config.gke.google_region = ENV['GOOGLE_REGION']
+  config.gke.google_project = ENV['GOOGLE_PROJECT']
+  config.gke.enable_get_credentials = true # enable hook to call: gcloud container clusters get-credentials
+end
+```
+
+Note: The use of `KubesGoogle.configure` instead of `Kubes.configure` here.
+
+This enables `kubes apply` before and after hooks to add and remove the current machine IP.
+
+## Options
+
+Here are the `config.gke` settings:
+
+Name | Description | Default
+---|---|---
+{% include plugins/gke-config.md %}
+
+## Build Docker Image
+
+To build kubes as a Docker image entrypoint for [Google CloudBuild Custom Builder](https://cloud.google.com/cloud-build/docs/configuring-builds/use-community-and-custom-builders).
+
+    git clone http://github.com/boltops-tools/kubes
+    cd kubes
+    gcloud builds submit --tag gcr.io/$GOOGLE_PROJECT/kubes
+
+Be sure to set GOOGLE_PROJECT to your own project id.
+
+## Example Codebuild YAML
+
+cloudbuild.yaml:
+
+```yaml
+steps:
+- name: 'gcr.io/$PROJECT_ID/kubes'
+  args: ['deploy']
+  env:
+  - 'DOCKER_REPO=gcr.io/$PROJECT_ID/demo'
+  - 'GOOGLE_PROJECT=$PROJECT_ID' # .kubes/config.rb: config.repo
+  - 'KUBES_ENV=$_KUBES_ENV'
+  - 'KUBES_EXTRA=$_KUBES_EXTRA'
+  - 'KUBES_REPO_AUTH=0'
+
+substitutions:
+  _KUBES_ENV: dev
+  _KUBES_EXTRA: ''
+options:
+    substitution_option: 'ALLOW_LOOSE'
+```
+
+Make sure to replace the substitutions with your own values. IE: _GCP_REGION, _GKE_CLUSTER, _KUBES_ENV, etc.
+
+## Google CloudBuild IAM Permissions
+
+In order to update the GKE cluster master authorized IP and whitelist the CloudBuild IP, you'll need to allow the CloudBuild IAM role permissions.
+
+Important: The "Kubernetes Engine Developer" that is available in the Cloud Build Settings page as described in [Configuring access for Cloud Build Service Account](https://cloud.google.com/cloud-build/docs/securing-builds/configure-access-for-cloud-build-service-account) does not suffice. You'll need to add the "Kubernetes Engine Cluster Admin" role. Here are the steps:
+
+1. Go to the Google IAM Console and search "cloudbuild"
+2. Click "Edit Member"
+3. Add the "Kubernetes Engine Cluster Admin" role
+
+## Run CloudBuild
+
+Run cloudbuild to deploy the dev env:
+
+    gcloud builds submit --config cloudbuild.yaml
+
+To deploy the prod env:
+
+    gcloud builds submit --config cloudbuild.yaml --substitutions _KUBES_ENV=prod
+
+See [gcloud builds submit](https://cloud.google.com/sdk/gcloud/reference/builds/submit) reference docs for more options.