kubes 0.4.4 → 0.4.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9bfe400824a8ee2877643cba9abfbd10a5e57b058e88977313457836a490c7f0
-  data.tar.gz: 98bff5118293f84161ed5b10b9d0162df5181b75a6dbc09dc88ed2ecd5e7029e
+  metadata.gz: bbdaabc58ed6ae8e1c2941431c61a3e7edf7708b7782f1d456b4bcf2b01ef777
+  data.tar.gz: '04290becf20d593ba00d214dd2e790eac5501e3e8f5e3b62d636b3e882b7a5e1'
 SHA512:
-  metadata.gz: 0f3a2a36e27e2c64d5821f429368bd515c259045a26d24b8ee16713a356f53d6370938a2748bf1e65f95779936ea654547117f49bef8da2c0b435461f5456099
-  data.tar.gz: 9e603c401b2e9a56fc38cd1aec28d3ceaf9ff821d5add2b22bf9d8d2e5d209d7ab5609b9ab0116bd1189d5594a0e6ee00a071984605b235408ac6a0fffbeee1b
+  metadata.gz: 302a4a216bc5007de836e335cc38e4fcd8d4c4db8d873f4333d862f84dbcfd32c302ba72c8123392f2a1679ca419be15ef660818f2b75a06b05cb0250a469198
+  data.tar.gz: dbbf8a825b893490963db61c6b08d2adf45ef62be0bad19d44235b1b822077ebb4e8728d45d119dddfb6f33e39ecb3d69ccb89ceda938275b8371306ca72efaa

data/CHANGELOG.md

@@ -3,6 +3,9 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.4.5]
+- #31 kubes AWS helpers
+
 ## [0.4.4]
 - #30 friendly message for rendered erb yaml and dsl errors
 - fix backtrace_reject pattern

data/docs/_docs/helpers.md

@@ -17,4 +17,11 @@ Here's also the source code with most of the helpers: [helpers.rb](https://githu
 
 ## DSL Specific Methods
 
-Each DSL resource has it's own specific methods. Refer to the [DSL Docs]({% link _docs/dsl.md %}) for their methods.
+Each DSL resource has it's own specific methods. Refer to the [DSL Docs]({% link _docs/dsl.md %}) for their methods.
+
+## Provider Helpers
+
+There are also provider-specific helpers:
+
+* [AWS Helpers]({% link _docs/helpers/aws.md %})
+* [Google Helpers]({% link _docs/helpers/google.md %})

data/docs/_docs/helpers/aws.md

@@ -0,0 +1,14 @@
+---
+title: AWS Helpers
+---
+
+List of AWS helpers:
+
+{% assign docs = site.docs | where: "categories","helpers-aws" %}
+{% for doc in docs -%}
+  * [{{ doc.nav_text }}]({{ doc.url }})
+{% endfor %}
+
+## Notes
+
+* By default, `KubeGoogle.logger = Kubes.logger`. This means, you can set `logger.level = "debug"` in `.kubes/config.rb` to see more details.

data/docs/_docs/helpers/aws/iam-role.md

@@ -0,0 +1,91 @@
+---
+title: AWS IAM Role
+nav_text: IAM Role
+categories: helpers-aws
+---
+
+You can automatically create the IAM Role associated with the Kubernetes Service Account, covered in [Introducing fine-grained IAM roles for service accounts](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/).
+
+Here's a Kubes hook that creates an IAM Role:
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+iam_role = KubesAws::IamRole.new(
+  app: "demo",
+  namespace: "demo-#{Kubes.env}", # defaults to APP-ENV when not set. IE: demo-dev
+  managed_policies: ["AmazonS3ReadOnlyAccess", "AmazonSSMReadOnlyAccess"], # defaults to empty when not set
+  inline_policies: [:secrets_read_only], # See Secrets Read Only Inline Policy at the bottom
+)
+before("apply",
+  label: "create iam role",
+  execute: iam_role,
+)
+KubesAws::IamRole.role_arn = iam_role.arn # used in .kubes/resources/shared/service_account.yaml
+```
+
+The corresponding Kubernetes Service account looks like this:
+
+.kubes/resources/shared/service_account.yaml
+
+```yaml
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  annotations:
+    eks.amazonaws.com/role-arn: <%= KubesAws::IamRole.role_arn %>
+  name: demo
+  labels:
+    app: demo
+```
+
+The role policy permissions are currently always added to the existing permissions. So removing roles that were previously added does not remove them.
+
+IamRole#initialize options:
+
+Variable | Description | Default
+---|---|---
+app | The app name. It's used to set other variables conventionally. This is required. | nil
+ksa | The Kubernetes Service Account name. The conventional name is APP. IE: demo | APP
+namespace | The Kubernetes namespace. Defaults to the APP-ENV. IE: demo-dev. | APP-ENV
+policies | IAM policies to add. This adds permissions to the IAM Role. | []
+role_name | The IAM Role name. The conventional name is APP-ENV. IE: demo-dev. | APP-ENV
+
+## OpenID Connect Provider
+
+The `KubesAws::IamRole` class also automatically creates the OpenID Connect Provider if it doesn't already exist.
+
+## Secrets Read-Only Inline Policy
+
+Note the the `:secrets_read_only` is a way to generate an Inline Policy that represents read-only access for Secrets. Kubes does this since there's no managed policy for this yet. For example:
+
+```ruby
+inline_policies: [:secrets_read_only]
+```
+
+Is the same as:
+
+```ruby
+inline_secrets_read_only = {
+  policy_document: {
+    Version: "2012-10-17",
+    Statement: {
+      Effect: "Allow",
+      Action: [
+        "secretsmanager:Describe*",
+        "secretsmanager:Get*",
+        "secretsmanager:List*"
+      ],
+      Resource: "*"
+    }
+  },
+  policy_name: "SecretsReadOnly",
+}
+iam_role = KubesAws::IamRole.new(
+  app: "rails",
+  cluster: "dev-cluster",
+  namespace: "rails-#{Kubes.env}", # defaults to APP-ENV when not set. IE: rails-dev
+  managed_policies: ["AmazonS3ReadOnlyAccess", "AmazonSSMReadOnlyAccess"], # defaults to empty when not set
+  inline_policies: [inline_secrets_read_only],
+)
+```

data/docs/_docs/helpers/aws/secrets.md

@@ -0,0 +1,129 @@
+---
+title: AWS Secrets
+nav_text: Secrets
+categories: helpers-aws
+---
+
+## Simple Values
+
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_user | jq '.SecretString'
+    user
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_pass | jq '.SecretString'
+    pass
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+secrets = KubesAws::Secrets.new(upcase: true, prefix: "demo/dev/")
+before("compile",
+  label: "Get secrets from AWS Secrets Manager",
+  execute: secrets,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesAws::Secrets.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## JSON Values
+
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/k2 | jq '.SecretString'
+    {\"a\":1,\"b\":2}"
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+secrets = KubesAws::Secrets.new(prefix: "rails/dev/")
+before("compile",
+  label: "Get secrets from AWS Secrets Manager",
+  execute: secrets,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% k2 = JSON.load(KubesAws::Secrets.data["k2"]) %>
+  a: <%= base64(k2["a"]) %>
+  b: <%= base64(k2["b"]) %>
+```
+
+Produces:
+
+```yaml
+metadata:
+  namespace: demo-dev
+  name: demo-a4cd604a95
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  a: MQ==
+  b: Mg==
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+AWS_SECRET_PREFIX | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/aws/ssm.md

@@ -0,0 +1,76 @@
+---
+title: AWS SSM Parameters
+nav_text: SSM
+categories: helpers-aws
+---
+
+For example if you have these secret values:
+
+    $ aws ssm get-parameter --name /demo/development/db_user --with-decryption | jq '.Parameter.Value'
+    user
+    $ aws ssm get-parameter --name /demo/development/db_pass --with-decryption | jq '.Parameter.Value'
+    pass
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+ssm = KubesAws::SSM.new(upcase: true, prefix: "/demo/development/")
+before("compile",
+  label: "Get secrets from AWS SSM Manager",
+  execute: ssm,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesAws::SSM.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+AWS_SSM_PREFIX | Prefixed used to list and filter AWS SSM Parameters. IE: `demo/dev/`.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/google.md

@@ -0,0 +1,17 @@
+---
+title: Google Helpers
+---
+
+List of Google helpers:
+
+{% assign docs = site.docs | where: "categories","helpers-aws" %}
+{% for doc in docs -%}
+  * [{{ doc.nav_text }}]({{ doc.url }})
+{% endfor %}
+
+## Notes
+
+* By default, `KubeGoogle.logger = Kubes.logger`. This means, you can set `logger.level = "debug"` in `.kubes/config.rb` to see more details.
+* The `gcloud` cli is used to create IAM roles. So `gcloud` is required.
+* Note: Would like to use the google sdk, but it wasn't obvious how to do so. PRs are welcomed.
+

data/docs/_docs/helpers/google/secrets.md

@@ -0,0 +1,76 @@
+---
+title: Google Secrets
+nav_text: Secrets
+categories: helpers-google
+---
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+before("compile",
+  execute: KubesGoogle::Secrets.new(upcase: true, prefix: 'projects/686010496118/secrets/demo-dev-')
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesGoogle::Secrets.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in Google secrets with the prefix the `demo-dev-` being added to the Kubernetes secret data.  The values are automatically base64 encoded.
+
+For example if you have these secret values:
+
+    $ gcloud secrets versions access latest --secret demo-dev-db_user
+    test1
+    $ gcloud secrets versions access latest --secret demo-dev-db_pass
+    test2
+    $
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+GCP_SECRET_PREFIX | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`.
+GOOGLE_PROJECT | Google project id.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`. Can also be set with the `GCP_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/google/service-account.md

@@ -0,0 +1,52 @@
+---
+title: Google Service Account
+nav_text: Service Account
+categories: helpers-google
+---
+
+## Service Accounts
+
+You can automatically create the Google Service Account associated with the [GKE Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity).
+
+Here's a Kubes hook that creates a service account:
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+service_account = KubesGoogle::ServiceAccount.new(
+  app: "demo",
+  namespace: "demo-#{Kubes.env}", # defaults to APP-ENV when not set. IE: demo-dev
+  roles: ["cloudsql.client", "secretmanager.viewer"], # defaults to empty when not set
+)
+before("apply",
+  label: "create service account",
+  execute: service_account,
+)
+```
+
+The corresponding Kubernetes Service account looks like this:
+
+.kubes/resources/shared/service_account.yaml
+
+```yaml
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  annotations:
+    iam.gke.io/gcp-service-account: demo-<%= Kubes.env %>@<%= ENV['GOOGLE_PROJECT'] %>.iam.gserviceaccount.com
+  name: demo
+  labels:
+    app: demo
+```
+
+The role permissions are currently always added to the existing permissions. So removing roles that were previously added does not remove them.
+
+ServiceAccount#initialize options:
+
+Variable | Description | Default
+---|---|---
+app | The app name. It's used to set other variables conventionally. This is required. | nil
+gsa | The Google Service Account name. The conventional name is APP-ENV. IE: demo-dev. | APP-ENV
+ksa | The Kubernetes Service Account name. The conventional name is APP. IE: demo | APP
+namespace | The Kubernetes namespace. Defaults to the APP-ENV. IE: demo-dev. | APP-ENV
+roles | Google IAM roles to add. This adds permissions to the Google service account. | []

data/docs/_includes/helpers/base64.md

@@ -0,0 +1 @@
+Note, Kubernetes secrets are only base64 encoded. So users who have access to read Kubernetes secrets will be able to decode and get the value trivially. Depending on your security posture requirements, this may or may not suffice.

data/docs/_includes/sidebar.html

@@ -97,7 +97,26 @@
           <li><a href="{% link _docs/dsl/multiple-resources.md %}">Multiple Resources</a>
         </ul>
       </li>
-      <li><a href="{% link _docs/helpers.md %}">Helpers</a></li>
+      <li><a href="{% link _docs/helpers.md %}">Helpers</a>
+        <ul>
+          <li><a href="{% link _docs/helpers/aws.md %}">AWS</a>
+            <ul>
+              {% assign docs = site.docs | where: "categories","helpers-aws" %}
+              {% for doc in docs -%}
+                <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+              {% endfor %}
+            </ul>
+          </li>
+          <li><a href="{% link _docs/helpers/google.md %}">Google</a>
+            <ul>
+              {% assign docs = site.docs | where: "categories","helpers-google" %}
+              {% for doc in docs -%}
+                <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+              {% endfor %}
+            </ul>
+          </li>
+        </ul>
+      </li>
       <li><a href="{% link _docs/patterns.md %}">Patterns</a>
         <ul>
           {% assign docs = site.docs | where: "categories","patterns" %}

data/kubes.gemspec

@@ -29,6 +29,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency "zeitwerk"
 
   # core helper libs
+  spec.add_dependency "kubes_aws"
   spec.add_dependency "kubes_google"
 
   spec.add_development_dependency "bundler"

data/lib/kubes.rb

@@ -15,6 +15,7 @@ require "rainbow/ext/string"
 require "yaml"
 
 # core helper libraries
+require "kubes_aws"
 require "kubes_google"
 
 DslEvaluator.backtrace_reject = "lib/kubes"

data/lib/kubes/compiler/shared/helpers.rb

@@ -25,7 +25,7 @@ module Kubes::Compiler::Shared
     end
 
     def encode64(v)
-      Base64.strict_encode64(v).strip
+      Base64.strict_encode64(v.to_s).strip
     end
     alias_method :base64, :encode64
 

data/lib/kubes/version.rb

@@ -1,3 +1,3 @@
 module Kubes
-  VERSION = "0.4.4"
+  VERSION = "0.4.5"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kubes
 version: !ruby/object:Gem::Version
-  version: 0.4.4
+  version: 0.4.5
 platform: ruby
 authors:
 - Tung Nguyen
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-10-26 00:00:00.000000000 Z
+date: 2020-10-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -136,6 +136,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: kubes_aws
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: kubes_google
   requirement: !ruby/object:Gem::Requirement
@@ -285,6 +299,13 @@ files:
 - docs/_docs/extra-env/dsl.md
 - docs/_docs/extra-env/yaml.md
 - docs/_docs/helpers.md
+- docs/_docs/helpers/aws.md
+- docs/_docs/helpers/aws/iam-role.md
+- docs/_docs/helpers/aws/secrets.md
+- docs/_docs/helpers/aws/ssm.md
+- docs/_docs/helpers/google.md
+- docs/_docs/helpers/google/secrets.md
+- docs/_docs/helpers/google/service-account.md
 - docs/_docs/intro.md
 - docs/_docs/intro/concepts.md
 - docs/_docs/intro/how-kubes-works.md
@@ -339,6 +360,7 @@ files:
 - docs/_includes/footer.html
 - docs/_includes/google_analytics.html
 - docs/_includes/header.html
+- docs/_includes/helpers/base64.md
 - docs/_includes/intro/install.md
 - docs/_includes/js.html
 - docs/_includes/kubes-steps.md