kubes 0.4.2 → 0.4.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 70ab0b9bdb58186064b2ba54324059ec1944811a021f158a30f219cbd4a19711
-  data.tar.gz: cdccd15d027f75f1c439d1facff1ed9d0b2ef58b4600b289d6f9d7e8e63917cb
+  metadata.gz: 1b2e59aafa5ea2023bdf23dc43ba38c5279dcd719c3db793eca0abf24c8e1da2
+  data.tar.gz: ed004f619cba70d0d6257ca9559b3fd1b1fa7cb6c48943c64c3abcb8cddf3d38
 SHA512:
-  metadata.gz: 23c2dddfd2608971f2d59a52f2df79cd23355e98ab1f3de1aa6611e94ea19c75d54dd744892148bbb27a0e6610fe1a952895fa0a9571cead2ef729c072f15821
-  data.tar.gz: b088738e2a3fa44157f2c340508ff1d87634550ebe63608c4b7a9c4241dd385019af03c132400381db4f4ea36057526afb832e1101860b745a8670b130b7ffa2
+  metadata.gz: 7be4d6f0bd67e15b9ba3e3594f1b98d36c3fb5a4cc68c67fd603a75a973e4b90f8800a0a8fddf68951d0f3f09291c154bae5066830ec792aa167eb8675f5acca
+  data.tar.gz: 6187e6b6bc29e1a038f8c5a20530483a120cc6107854f718f8570648334f27cb7b4afb1a39d4d45ede4bb3cda97e8cc66d63c1cbbdde27a059997f65bd547704

data/CHANGELOG.md

@@ -3,6 +3,22 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.4.7]
+- #33 improve switch context: earlier and only when needed
+
+## [0.4.6]
+- #32 custom helpers support
+
+## [0.4.5]
+- #31 kubes AWS helpers
+
+## [0.4.4]
+- #30 friendly message for rendered erb yaml and dsl errors
+- fix backtrace_reject pattern
+
+## [0.4.3]
+- #29 fix edge case when user provides hook on option for non-kubectl hooks
+
 ## [0.4.2]
 - #28 base64 helper
 

data/docs/_docs/helpers.md

@@ -17,4 +17,11 @@ Here's also the source code with most of the helpers: [helpers.rb](https://githu
 
 ## DSL Specific Methods
 
-Each DSL resource has it's own specific methods. Refer to the [DSL Docs]({% link _docs/dsl.md %}) for their methods.
+Each DSL resource has it's own specific methods. Refer to the [DSL Docs]({% link _docs/dsl.md %}) for their methods.
+
+## Provider Helpers
+
+There are also provider-specific helpers:
+
+* [AWS Helpers]({% link _docs/helpers/aws.md %})
+* [Google Helpers]({% link _docs/helpers/google.md %})

data/docs/_docs/helpers/aws.md

@@ -0,0 +1,15 @@
+---
+title: AWS Helpers
+---
+
+List of AWS helpers:
+
+{% assign docs = site.docs | where: "categories","helpers-aws" %}
+{% for doc in docs -%}
+  * [{{ doc.nav_text }}]({{ doc.url }})
+{% endfor %}
+
+## Notes
+
+* By default, `KubeGoogle.logger = Kubes.logger`. This means, you can set `logger.level = "debug"` in `.kubes/config.rb` to see more details.
+* The AWS helpers are provided by the [boltops-tools/kubes_aws](https://github.com/boltops-tools/kubes_aws) library.

data/docs/_docs/helpers/aws/iam-role.md

@@ -0,0 +1,91 @@
+---
+title: AWS IAM Role
+nav_text: IAM Role
+categories: helpers-aws
+---
+
+You can automatically create the IAM Role associated with the Kubernetes Service Account, covered in [Introducing fine-grained IAM roles for service accounts](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/).
+
+Here's a Kubes hook that creates an IAM Role:
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+iam_role = KubesAws::IamRole.new(
+  app: "demo",
+  namespace: "demo-#{Kubes.env}", # defaults to APP-ENV when not set. IE: demo-dev
+  managed_policies: ["AmazonS3ReadOnlyAccess", "AmazonSSMReadOnlyAccess"], # defaults to empty when not set
+  inline_policies: [:secrets_read_only], # See Secrets Read Only Inline Policy at the bottom
+)
+before("apply",
+  label: "create iam role",
+  execute: iam_role,
+)
+KubesAws::IamRole.role_arn = iam_role.arn # used in .kubes/resources/shared/service_account.yaml
+```
+
+The corresponding Kubernetes Service account looks like this:
+
+.kubes/resources/shared/service_account.yaml
+
+```yaml
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  annotations:
+    eks.amazonaws.com/role-arn: <%= KubesAws::IamRole.role_arn %>
+  name: demo
+  labels:
+    app: demo
+```
+
+The role policy permissions are currently always added to the existing permissions. So removing roles that were previously added does not remove them.
+
+IamRole#initialize options:
+
+Variable | Description | Default
+---|---|---
+app | The app name. It's used to set other variables conventionally. This is required. | nil
+ksa | The Kubernetes Service Account name. The conventional name is APP. IE: demo | APP
+namespace | The Kubernetes namespace. Defaults to the APP-ENV. IE: demo-dev. | APP-ENV
+policies | IAM policies to add. This adds permissions to the IAM Role. | []
+role_name | The IAM Role name. The conventional name is APP-ENV. IE: demo-dev. | APP-ENV
+
+## OpenID Connect Provider
+
+The `KubesAws::IamRole` class also automatically creates the OpenID Connect Provider if it doesn't already exist.
+
+## Secrets Read-Only Inline Policy
+
+Note the the `:secrets_read_only` is a way to generate an Inline Policy that represents read-only access for Secrets. Kubes does this since there's no managed policy for this yet. For example:
+
+```ruby
+inline_policies: [:secrets_read_only]
+```
+
+Is the same as:
+
+```ruby
+inline_secrets_read_only = {
+  policy_document: {
+    Version: "2012-10-17",
+    Statement: {
+      Effect: "Allow",
+      Action: [
+        "secretsmanager:Describe*",
+        "secretsmanager:Get*",
+        "secretsmanager:List*"
+      ],
+      Resource: "*"
+    }
+  },
+  policy_name: "SecretsReadOnly",
+}
+iam_role = KubesAws::IamRole.new(
+  app: "rails",
+  cluster: "dev-cluster",
+  namespace: "rails-#{Kubes.env}", # defaults to APP-ENV when not set. IE: rails-dev
+  managed_policies: ["AmazonS3ReadOnlyAccess", "AmazonSSMReadOnlyAccess"], # defaults to empty when not set
+  inline_policies: [inline_secrets_read_only],
+)
+```

data/docs/_docs/helpers/aws/secrets.md

@@ -0,0 +1,129 @@
+---
+title: AWS Secrets
+nav_text: Secrets
+categories: helpers-aws
+---
+
+## Simple Values
+
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_user | jq '.SecretString'
+    user
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/db_pass | jq '.SecretString'
+    pass
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+secrets = KubesAws::Secrets.new(upcase: true, prefix: "demo/dev/")
+before("compile",
+  label: "Get secrets from AWS Secrets Manager",
+  execute: secrets,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesAws::Secrets.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## JSON Values
+
+For example if you have these secret values:
+
+    $ aws secretsmanager get-secret-value --secret-id demo/dev/k2 | jq '.SecretString'
+    {\"a\":1,\"b\":2}"
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+secrets = KubesAws::Secrets.new(prefix: "rails/dev/")
+before("compile",
+  label: "Get secrets from AWS Secrets Manager",
+  execute: secrets,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% k2 = JSON.load(KubesAws::Secrets.data["k2"]) %>
+  a: <%= base64(k2["a"]) %>
+  b: <%= base64(k2["b"]) %>
+```
+
+Produces:
+
+```yaml
+metadata:
+  namespace: demo-dev
+  name: demo-a4cd604a95
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  a: MQ==
+  b: Mg==
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+AWS_SECRET_PREFIX | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/aws/ssm.md

@@ -0,0 +1,76 @@
+---
+title: AWS SSM Parameters
+nav_text: SSM
+categories: helpers-aws
+---
+
+For example if you have these secret values:
+
+    $ aws ssm get-parameter --name /demo/development/db_user --with-decryption | jq '.Parameter.Value'
+    user
+    $ aws ssm get-parameter --name /demo/development/db_pass --with-decryption | jq '.Parameter.Value'
+    pass
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+ssm = KubesAws::SSM.new(upcase: true, prefix: "/demo/development/")
+before("compile",
+  label: "Get secrets from AWS SSM Manager",
+  execute: ssm,
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesAws::SSM.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in AWS secrets with the prefix the `demo/dev/` being added to the Kubernetes secret data.  The values are automatically base64 encoded. Produces:
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+AWS_SSM_PREFIX | Prefixed used to list and filter AWS SSM Parameters. IE: `demo/dev/`.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter AWS secrets. IE: `demo/dev/`. Can also be set with the `AWS_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/custom.md

@@ -0,0 +1,40 @@
+---
+title: Custom Helpers
+---
+
+Kubes ships with several built-in helpers. On top of this, you can define your own custom helpers.  This allows you to define new methods and customize Kubes further.
+
+## Example
+
+You define custom helpers in the `.kubes/helpers` folder.
+
+.kubes/helpers/my_helpers.rb
+
+```ruby
+module MyHelpers
+  def database_endpoint
+    case Kubes.env
+    when "dev"
+      "dev-db.cbuqdmc3nqvb.us-west-2.rds.amazonaws.com"
+    when "prod"
+      "prod-db.cbuqdmc3nqvb.us-west-2.rds.amazonaws.com"
+    end
+  end
+end
+```
+
+The `database_endpoint` will be available to use in the `.kubes/resources` YAML files. IE:
+
+.kubes/helpers/resources/shared/config_map.yaml
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+  DATABASE_ENDPOINT: <%= database_endpoint %>
+```
+

data/docs/_docs/helpers/google.md

@@ -0,0 +1,17 @@
+---
+title: Google Helpers
+---
+
+List of Google helpers:
+
+{% assign docs = site.docs | where: "categories","helpers-aws" %}
+{% for doc in docs -%}
+  * [{{ doc.nav_text }}]({{ doc.url }})
+{% endfor %}
+
+## Notes
+
+* By default, `KubeGoogle.logger = Kubes.logger`. This means, you can set `logger.level = "debug"` in `.kubes/config.rb` to see more details.
+* The `gcloud` cli is used to create IAM roles. So `gcloud` is required.
+* Note: Would like to use the google sdk, but it wasn't obvious how to do so. PRs are welcomed.
+* The Google helpers are provided by the [boltops-tools/kubes_google](https://github.com/boltops-tools/kubes_google) library.

data/docs/_docs/helpers/google/secrets.md

@@ -0,0 +1,76 @@
+---
+title: Google Secrets
+nav_text: Secrets
+categories: helpers-google
+---
+
+Set up a [Kubes hook](https://kubes.guru/docs/config/hooks/kubes/).
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+before("compile",
+  execute: KubesGoogle::Secrets.new(upcase: true, prefix: 'projects/686010496118/secrets/demo-dev-')
+)
+```
+
+Then set the secrets in the YAML:
+
+.kubes/resources/shared/secret.yaml
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: demo
+  labels:
+    app: demo
+data:
+<% KubesGoogle::Secrets.data.each do |k,v| -%>
+  <%= k %>: <%= base64(v) %>
+<% end -%>
+```
+
+This results in Google secrets with the prefix the `demo-dev-` being added to the Kubernetes secret data.  The values are automatically base64 encoded.
+
+For example if you have these secret values:
+
+    $ gcloud secrets versions access latest --secret demo-dev-db_user
+    test1
+    $ gcloud secrets versions access latest --secret demo-dev-db_pass
+    test2
+    $
+
+.kubes/output/shared/secret.yaml
+
+```yaml
+metadata:
+  namespace: demo
+  name: demo-2a78a13682
+  labels:
+    app: demo
+apiVersion: v1
+kind: Secret
+data:
+  db_pass: dGVzdDEK
+  db_user: dGVzdDIK
+```
+
+## Variables
+
+These environment variables can be set:
+
+Name | Description
+---|---
+GCP_SECRET_PREFIX | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`.
+GOOGLE_PROJECT | Google project id.
+
+Secrets#initialize options:
+
+Variable | Description | Default
+---|---|---
+base64 | Automatically base64 encode the values. | false
+upcase | Automatically upcase the Kubernetes secret data keys. | false
+prefix | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`. Can also be set with the `GCP_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
+
+{% include helpers/base64.md %}

data/docs/_docs/helpers/google/service-account.md

@@ -0,0 +1,52 @@
+---
+title: Google Service Account
+nav_text: Service Account
+categories: helpers-google
+---
+
+## Service Accounts
+
+You can automatically create the Google Service Account associated with the [GKE Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity).
+
+Here's a Kubes hook that creates a service account:
+
+.kubes/config/hooks/kubes.rb
+
+```ruby
+service_account = KubesGoogle::ServiceAccount.new(
+  app: "demo",
+  namespace: "demo-#{Kubes.env}", # defaults to APP-ENV when not set. IE: demo-dev
+  roles: ["cloudsql.client", "secretmanager.viewer"], # defaults to empty when not set
+)
+before("apply",
+  label: "create service account",
+  execute: service_account,
+)
+```
+
+The corresponding Kubernetes Service account looks like this:
+
+.kubes/resources/shared/service_account.yaml
+
+```yaml
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  annotations:
+    iam.gke.io/gcp-service-account: demo-<%= Kubes.env %>@<%= ENV['GOOGLE_PROJECT'] %>.iam.gserviceaccount.com
+  name: demo
+  labels:
+    app: demo
+```
+
+The role permissions are currently always added to the existing permissions. So removing roles that were previously added does not remove them.
+
+ServiceAccount#initialize options:
+
+Variable | Description | Default
+---|---|---
+app | The app name. It's used to set other variables conventionally. This is required. | nil
+gsa | The Google Service Account name. The conventional name is APP-ENV. IE: demo-dev. | APP-ENV
+ksa | The Kubernetes Service Account name. The conventional name is APP. IE: demo | APP
+namespace | The Kubernetes namespace. Defaults to the APP-ENV. IE: demo-dev. | APP-ENV
+roles | Google IAM roles to add. This adds permissions to the Google service account. | []

data/docs/_docs/patterns/secrets.md

@@ -78,5 +78,3 @@ upcase | Automatically upcase the Kubernetes secret data keys. | false
 prefix | Prefixed used to list and filter Google secrets. IE: `projects/686010496118/secrets/demo-dev-`. Can also be set with the `GCP_SECRET_PREFIX` env variable. The env variable takes the highest precedence. | nil
 
 Note, Kubernetes secrets are only base64 encoded. So users who have access to read Kubernetes secrets will be able to decode and get the value trivially. Depending on your security posture requirements, this may or may not suffice.
-
-The Google helpers are provided by the [boltops-tools/kubes_google](https://github.com/boltops-tools/kubes_google) library. For more details, check out its README.

data/docs/_includes/helpers/base64.md

@@ -0,0 +1 @@
+Note, Kubernetes secrets are only base64 encoded. So users who have access to read Kubernetes secrets will be able to decode and get the value trivially. Depending on your security posture requirements, this may or may not suffice.

data/docs/_includes/layering/layers.md

@@ -27,7 +27,7 @@ To explain the layering, here's the general processing order that Kubes takes.
 
 Note, both YAML and DSL forms support layering.
 
-Layering only combines resources definitions with the same form. For example, `base/all.rb` will not be combined with `web/deployment.yaml`.
+Layering only combines resources definitions with the same form. For example, the DSL form `base/all.rb` will not be combined with YAML form `web/deployment.yaml`.
 
 ## Full Layering
 

data/docs/_includes/sidebar.html

@@ -97,7 +97,27 @@
           <li><a href="{% link _docs/dsl/multiple-resources.md %}">Multiple Resources</a>
         </ul>
       </li>
-      <li><a href="{% link _docs/helpers.md %}">Helpers</a></li>
+      <li><a href="{% link _docs/helpers.md %}">Helpers</a>
+        <ul>
+          <li><a href="{% link _docs/helpers/custom.md %}">Custom</a></li>
+          <li><a href="{% link _docs/helpers/aws.md %}">AWS</a>
+            <ul>
+              {% assign docs = site.docs | where: "categories","helpers-aws" %}
+              {% for doc in docs -%}
+                <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+              {% endfor %}
+            </ul>
+          </li>
+          <li><a href="{% link _docs/helpers/google.md %}">Google</a>
+            <ul>
+              {% assign docs = site.docs | where: "categories","helpers-google" %}
+              {% for doc in docs -%}
+                <li><a href="{{ doc.url }}">{{ doc.nav_text }}</a></li>
+              {% endfor %}
+            </ul>
+          </li>
+        </ul>
+      </li>
       <li><a href="{% link _docs/patterns.md %}">Patterns</a>
         <ul>
           {% assign docs = site.docs | where: "categories","patterns" %}

data/kubes.gemspec

@@ -29,6 +29,7 @@ Gem::Specification.new do |spec|
   spec.add_dependency "zeitwerk"
 
   # core helper libs
+  spec.add_dependency "kubes_aws"
   spec.add_dependency "kubes_google"
 
   spec.add_development_dependency "bundler"

data/lib/kubes.rb

@@ -15,9 +15,10 @@ require "rainbow/ext/string"
 require "yaml"
 
 # core helper libraries
+require "kubes_aws"
 require "kubes_google"
 
-DslEvaluator.backtrace_reject = ".kubes"
+DslEvaluator.backtrace_reject = "lib/kubes"
 
 require "kubes/autoloader"
 Kubes::Autoloader.setup

data/lib/kubes/autoloader.rb

@@ -14,8 +14,17 @@ module Kubes
         loader = Zeitwerk::Loader.new
         loader.inflector = Inflector.new
         loader.push_dir(File.dirname(__dir__)) # lib
+
+        helpers = "#{kubes_root}/.kubes/helpers"
+        loader.push_dir(helpers) if File.exist?(helpers) # project helpers
+
         loader.setup
       end
+
+      # Autoloader runs so early that Kubes.root is not available, so we must declare it here
+      def kubes_root
+        ENV['KUBES_ROOT'] || '.'
+      end
     end
   end
 end

data/lib/kubes/compiler/shared/custom_helpers.rb

@@ -0,0 +1,17 @@
+module Kubes::Compiler::Shared
+  module CustomHelpers
+    # Load custom helper methods from project
+    @@custom_helpers_loaded = false
+    def load_custom_helpers
+      return if @@custom_helpers_loaded
+      paths = Dir.glob("#{Kubes.root}/.kubes/helpers/**/*.rb")
+      paths.sort_by! { |p| p.size } # so namespaces are loaded first
+      paths.each do |path|
+        filename = path.sub(%r{.*.kubes/helpers/},'').sub('.rb','')
+        module_name = filename.camelize
+        self.class.send :include, module_name.constantize
+      end
+      @@custom_helpers_loaded = true
+    end
+  end
+end

data/lib/kubes/compiler/shared/helpers.rb

@@ -25,7 +25,7 @@ module Kubes::Compiler::Shared
     end
 
     def encode64(v)
-      Base64.strict_encode64(v).strip
+      Base64.strict_encode64(v.to_s).strip
     end
     alias_method :base64, :encode64
 

data/lib/kubes/compiler/strategy/base.rb

@@ -1,13 +1,13 @@
 class Kubes::Compiler::Strategy
   class Base
     include Kubes::Logging
+    include Kubes::Compiler::Util::SaveFile
+    include Kubes::Compiler::Shared::CustomHelpers
 
     def initialize(options={})
       @options = options
       @path = options[:path]
-
-      @filename = @path.sub(%r{.*\.kubes/resources/},'') # IE: web/deployment.rb or web/deployment.yaml
-      @save_file = @filename.sub('.yml','.yaml').sub('.rb','.yaml')
+      @save_file = save_file(@path)
     end
   end
 end

data/lib/kubes/compiler/strategy/dsl.rb

@@ -3,6 +3,7 @@ class Kubes::Compiler::Strategy
     include Kubes::Compiler::Util::Normalize
 
     def run
+      load_custom_helpers
       dsl = dsl_class.new(@options) # Deployment, Service, etc
       data = dsl.run
       Result.new(@save_file, data)
@@ -17,7 +18,7 @@ class Kubes::Compiler::Strategy
     end
 
     def syntax_class
-      klass_name = normalize_kind(@filename)
+      klass_name = normalize_kind(@save_file)
       "Kubes::Compiler::Dsl::Syntax::#{klass_name}".constantize
     rescue NameError
       logger.debug "Using default resource for: #{klass_name}"
@@ -25,7 +26,7 @@ class Kubes::Compiler::Strategy
     end
 
     def block_form?
-      type = extract_type(@filename)
+      type = extract_type(@save_file)
       type.pluralize == type
     end
   end

data/lib/kubes/compiler/strategy/erb.rb

@@ -4,10 +4,12 @@ class Kubes::Compiler::Strategy
   class Erb < Base
     extend Kubes::Compiler::Dsl::Core::Fields
     include Kubes::Compiler::Dsl::Core::Helpers
+    include Kubes::Compiler::Shared::CustomHelpers
     include Kubes::Compiler::Shared::Helpers
     include Kubes::Compiler::Layering
 
     def run
+      load_custom_helpers
       @data = {}
 
       render_files(pre_layers)
@@ -32,11 +34,18 @@ class Kubes::Compiler::Strategy
     def render_result(path)
       if File.exist?(path)
         yaml = RenderMePretty.result(path, context: self)
-        result = YAML.load(yaml)
+        result = yaml_load(path, yaml)
         result.is_a?(Hash) ? result : {} # in case of blank yaml doc a Boolean false is returned
       else
         {}
       end
     end
+
+    def yaml_load(path, yaml)
+      YAML.load(yaml)
+    rescue Psych::SyntaxError
+      YamlError.new(path, yaml).show
+      exit 1
+    end
   end
 end

data/lib/kubes/compiler/strategy/erb/yaml_error.rb

@@ -0,0 +1,60 @@
+class Kubes::Compiler::Strategy::Erb
+  class YamlError
+    include Kubes::Compiler::Util::SaveFile
+
+    def initialize(path, rendered_yaml)
+      @path, @rendered_yaml = path, rendered_yaml
+    end
+
+    def show
+      FileUtils.mkdir_p(File.dirname(dest))
+      IO.write(dest, @rendered_yaml)
+      show_error(dest)
+    end
+
+    def dest
+      save_file = save_file(@path)
+      "#{Kubes.root}/.kubes/output/#{save_file}"
+    end
+
+    def show_error(path)
+      text = IO.read(path)
+      begin
+        YAML.load(text)
+      rescue Psych::SyntaxError => e
+        handle_yaml_syntax_error(e, path)
+      end
+    end
+
+    def handle_yaml_syntax_error(e, path)
+      io = StringIO.new
+      io.puts "Invalid yaml. Output written for debugging: #{path}".color(:red)
+      io.puts "ERROR: #{e.message}".color(:red)
+
+      # Grab line info.  Example error:
+      #   ERROR: (<unknown>): could not find expected ':' while scanning a simple key at line 2 column 1
+      md = e.message.match(/at line (\d+) column (\d+)/)
+      line = md[1].to_i
+
+      lines = IO.read(path).split("\n")
+      context = 5 # lines of context
+      top, bottom = [line-context-1, 0].max, line+context-1
+      spacing = lines.size.to_s.size
+      lines[top..bottom].each_with_index do |line_content, index|
+        line_number = top+index+1
+        if line_number == line
+          io.printf("%#{spacing}d %s\n".color(:red), line_number, line_content)
+        else
+          io.printf("%#{spacing}d %s\n", line_number, line_content)
+        end
+      end
+
+      if ENV['KUBES_TEST']
+        io.string
+      else
+        puts io.string
+        exit 1
+      end
+    end
+  end
+end

data/lib/kubes/compiler/util/normalize.rb

@@ -5,7 +5,7 @@ module Kubes::Compiler::Util
     end
 
     def extract_type(path)
-      File.basename(path).sub('.rb','').sub(/-.*/,'')
+      File.basename(path).sub('.yaml','').sub('.yml','').sub('.rb','').sub(/-.*/,'')
     end
   end
 end

data/lib/kubes/compiler/util/save_file.rb

@@ -0,0 +1,8 @@
+module Kubes::Compiler::Util
+  module SaveFile
+    def save_file(path)
+      filename = path.sub(%r{.*\.kubes/resources/},'') # IE: web/deployment.rb or web/deployment.yaml
+      filename.sub('.yml','.yaml').sub('.rb','.yaml')
+    end
+  end
+end

data/lib/kubes/hooks/builder.rb

@@ -50,7 +50,7 @@ module Kubes::Hooks
     def run?(hook)
       return false unless hook["execute"]
       return true  unless hook["on"]
-      @output_file.include?(hook["on"])
+      @output_file && @output_file.include?(hook["on"]) # output file is only passed
     end
   end
 end

data/lib/kubes/kubectl.rb

@@ -17,13 +17,11 @@ module Kubes
       params = args.flatten.join(' ')
       args = "#{@name} #{params}" # @name: apply or delete
 
-      switch_context do
-        run_hooks("kubectl.rb", name: @name, file: @options[:file]) do
-          if options[:capture]
-            self.class.capture(args, options) # already includes kubectl
-          else
-            self.class.execute(args, options)
-          end
+      run_hooks("kubectl.rb", name: @name, file: @options[:file]) do
+        if options[:capture]
+          self.class.capture(args, options) # already includes kubectl
+        else
+          self.class.execute(args, options)
         end
       end
     end
@@ -44,22 +42,6 @@ module Kubes
       Kubes.config.kubectl.exit_on_fail[@name]
     end
 
-    def switch_context(&block)
-      kubectl = Kubes.config.kubectl
-      context = kubectl.context
-      if context
-        previous_context = capture("kubectl config current-context")
-        sh("kubectl config use-context #{context}", mute: true)
-        result = block.call
-        if !previous_context.blank? && !kubectl.context_keep
-          sh("kubectl config use-context #{previous_context}", mute: true)
-        end
-        result
-      else
-        block.call
-      end
-    end
-
     def args
       # base at end in case of redirection. IE: command > /path
       custom.args + default.args

data/lib/kubes/kubectl/batch.rb

@@ -3,6 +3,7 @@ class Kubes::Kubectl
     include Kubes::Hooks::Concern
     include Kubes::Logging
     include Kubes::Util::Consider
+    include Kubes::Util::Sh
     include Ordering
 
     def initialize(name, options={})
@@ -12,17 +13,43 @@ class Kubes::Kubectl
     def run
       # @options[:preview] is really only used for kubectl delete
       logger.info "Will run:" if @options[:preview]
-      run_hooks("kubes.rb", name: @name) do
-        sorted_files.each do |file|
-          if @options[:preview]
-            logger.info "    kubectl #{@name} -f #{file}"
-          else
-            Kubes::Kubectl.run(@name, @options.merge(file: file))
+      switch_context do
+        run_hooks("kubes.rb", name: @name) do
+          sorted_files.each do |file|
+            if @options[:preview]
+              logger.info "    kubectl #{@name} -f #{file}"
+            else
+              Kubes::Kubectl.run(@name, @options.merge(file: file))
+            end
           end
         end
       end
     end
 
+    def switch_context(&block)
+      kubectl = Kubes.config.kubectl
+      context = kubectl.context
+
+      unless context
+        block.call
+        return
+      end
+
+      previous_context = sh_capture("kubectl config current-context")
+      if previous_context == context
+        block.call
+        return
+      end
+
+      logger.debug "Switching kubectl context to: #{context}"
+      sh("kubectl config use-context #{context}", mute: true)
+      result = block.call
+      if !previous_context.blank? && !kubectl.context_keep
+        sh("kubectl config use-context #{previous_context}", mute: true)
+      end
+      result
+    end
+
     # kubes apply                   # {role: nil, resource: nil}
     # kubes apply clock             # {role: "clock", resource: nil}
     # kubes apply clock deployment  # {role: "clock", resource: "deployment"}

data/lib/kubes/version.rb

@@ -1,3 +1,3 @@
 module Kubes
-  VERSION = "0.4.2"
+  VERSION = "0.4.7"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kubes
 version: !ruby/object:Gem::Version
-  version: 0.4.2
+  version: 0.4.7
 platform: ruby
 authors:
 - Tung Nguyen
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-10-14 00:00:00.000000000 Z
+date: 2020-10-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -136,6 +136,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: kubes_aws
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: kubes_google
   requirement: !ruby/object:Gem::Requirement
@@ -285,6 +299,14 @@ files:
 - docs/_docs/extra-env/dsl.md
 - docs/_docs/extra-env/yaml.md
 - docs/_docs/helpers.md
+- docs/_docs/helpers/aws.md
+- docs/_docs/helpers/aws/iam-role.md
+- docs/_docs/helpers/aws/secrets.md
+- docs/_docs/helpers/aws/ssm.md
+- docs/_docs/helpers/custom.md
+- docs/_docs/helpers/google.md
+- docs/_docs/helpers/google/secrets.md
+- docs/_docs/helpers/google/service-account.md
 - docs/_docs/intro.md
 - docs/_docs/intro/concepts.md
 - docs/_docs/intro/how-kubes-works.md
@@ -339,6 +361,7 @@ files:
 - docs/_includes/footer.html
 - docs/_includes/google_analytics.html
 - docs/_includes/header.html
+- docs/_includes/helpers/base64.md
 - docs/_includes/intro/install.md
 - docs/_includes/js.html
 - docs/_includes/kubes-steps.md
@@ -535,14 +558,17 @@ files:
 - lib/kubes/compiler/dsl/syntax/service.rb
 - lib/kubes/compiler/dsl/syntax/service_account.rb
 - lib/kubes/compiler/layering.rb
+- lib/kubes/compiler/shared/custom_helpers.rb
 - lib/kubes/compiler/shared/helpers.rb
 - lib/kubes/compiler/strategy.rb
 - lib/kubes/compiler/strategy/base.rb
 - lib/kubes/compiler/strategy/dsl.rb
 - lib/kubes/compiler/strategy/erb.rb
+- lib/kubes/compiler/strategy/erb/yaml_error.rb
 - lib/kubes/compiler/strategy/pass.rb
 - lib/kubes/compiler/strategy/result.rb
 - lib/kubes/compiler/util/normalize.rb
+- lib/kubes/compiler/util/save_file.rb
 - lib/kubes/compiler/util/yaml_dump.rb
 - lib/kubes/completer.rb
 - lib/kubes/completer/script.rb