kubes 0.6.8 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 18a7d3b8deec0ae4cd5d7e8ba2be00309e8fbfe1be70b063c452335c7278c96d
-  data.tar.gz: a0d70be2e034c42a7cc916f919f1de42cd50af7941dc7058875825d0d9cd95d9
+  metadata.gz: c708370f432e482b105a766bd174dd52f89c790256ea91063a8045af1d970f39
+  data.tar.gz: 3b4aef7f7450776cf35b9db54ff7d326a6444e948ab7cf81fac1a7659fca9649
 SHA512:
-  metadata.gz: fd2109ea8ab7141bd2c58b4c95a11a352851cf6ff9cf6512aded1236bb58e4344372ba444ce6a21870654113e80f208ee2d9908c6a14f775c6abfee9eaa33c1f
-  data.tar.gz: 3215ffdb98eda928dd6bb4975e1b7f309db8a41ca70b6869a77df76cb828589753a8d1d54ad8c3d76195dbc925ee7ebb15337b07ac111c91eed78fedaf6a5080
+  metadata.gz: f157bfaa51b24e2f23327ab02075abdeb331564c28cdcbe0710eeefb8ba2cd7910ea2719e28e1cdedb3b5b2873be902734c27254089ba5e1bc4083e13d33a10f
+  data.tar.gz: 48594a55badaba62e2ef7cfe436bc32640a5723221278f7d42c307144a00c300ca9a9c07d1501bc9a10204ff629929272171d8a4fc1cfe878b91bc951f46ba96

data/CHANGELOG.md

@@ -3,6 +3,9 @@
 All notable changes to this project will be documented in this file.
 This project *loosely tries* to adhere to [Semantic Versioning](http://semver.org/), even before v1.0.
 
+## [0.7.0] - 2020-11-16
+- [#41](https://github.com/boltops-tools/kubes/pull/41) multiple resources yaml support
+
 ## [0.6.8] - 2020-11-14
 - [#40](https://github.com/boltops-tools/kubes/pull/40) fix version check
 

data/docs/_docs/dsl/multiple-resources.md

@@ -1,5 +1,5 @@
 ---
-title: Multiple Resources
+title: DSL Multiple Resources
 ---
 
 Kubes encourages a structure with files that matches the resource kind. Example:
@@ -35,7 +35,7 @@ deployment-2.rb | Deployment
 service-1.rb | Service
 service-2.rb | Service
 
-Using multiple files is the general recommended approach.
+Using multiple files is the general recommended approach when using the DSL.
 
 ## Multiple Resources: Block Form
 
@@ -86,8 +86,9 @@ You can declare deployment, service, and other resource kinds multiple times.
 
 Layering works for both simple and block form. Just create a folder with the corresponding name.
 
-* The layering definitions with the simple form only merge with other simple form layers.
-* The layering definitions for block forms only merge with other block form layers.
+* The layering definitions for the prelayers are in singular form.
+* The layering definitions for the post layers are in a folder with plural form.
+* Resources in the main "middle" layer are the only ones that are allowed multiple resource definitions.
 
 Simple form layering:
 
@@ -105,8 +106,8 @@ Block form layering:
 
     .kubes/resources/
     ├── base
-    │   ├── alls.rb
-    │   └── deployments.rb
+    │   ├── all.rb
+    │   └── deployment.rb
     └── web
         ├── deployments
         │   ├── dev.rb

data/docs/_docs/yaml/multiple-files.md

@@ -0,0 +1,23 @@
+---
+title: YAML Multiple Resources with Multiple Files
+---
+
+You can also create multiple resources of same kind by appending a dash followed by anything. Example:
+
+    .kubes
+    └── resources
+        └── web
+            ├── deployment-1.yaml
+            ├── deployment-2.yaml
+            ├── service-1.yaml
+            └── service-2.yaml
+
+Only words before the dash are used to infer the resource kind.
+
+Filename | Resource Kind
+--- | ---
+deployment-1.yaml | Deployment
+deployment-2.yaml | Deployment
+service-1.yaml | Service
+service-2.yaml | Service
+

data/docs/_docs/yaml/multiple-resources.md

@@ -0,0 +1,90 @@
+---
+title: YAML Multiple Resources
+---
+
+Kubes encourages a structure with files that matches the resource kind. Example:
+
+    .kubes
+    └── resources
+        └── web
+            ├── deployment.yaml
+            └── service.yaml
+
+This structure is nicely organized and covers 80% of use cases. An astute user may point out that this struture assumes one resource of each kind.
+
+Next, we'll cover how to create multiple resource of the same kinds.
+
+## Multiple Resources in Same YAML
+
+You can simply define multiple resources in th same YAML file. Conventionally, you should name the resource files with plural names. An example helps explain:
+
+    .kubes
+    └── resources
+        └── deployments.yaml
+
+.kubes/resources/web/deployments.yaml
+
+```yaml
+- apiVersion: apps/v1
+  kind: Deployment
+  metadata:
+    name: web-1
+    labels:
+      role: web
+  spec:
+    selector:
+      matchLabels:
+        role: web
+    template:
+      metadata:
+        labels:
+          role: web
+      spec:
+        containers:
+        - name: web
+          image: <%= docker_image %>
+- apiVersion: apps/v1
+  kind: Deployment
+  metadata:
+    name: web-2
+    labels:
+      role: web
+  spec:
+    selector:
+      matchLabels:
+        role: web
+    template:
+      metadata:
+        labels:
+          role: web
+      spec:
+        containers:
+        - name: web
+          image: <%= docker_image %>
+```
+
+Notice that the YAML contains an Array of definitions now.
+
+## Layering
+
+Layering works just fine with multiple resource definitions. The layering is processed on each item of the Array.
+
+Notes:
+
+* The layering definitions for the pre layers must be in singular form.
+* The layering definitions for the post layers must be in a folder with plural form, but define a singlar resource override.
+* Resources in the main "middle" layer is the only one that allows for multiple resource definitions.
+
+Multiple resources layering structure.
+
+    .kubes/resources/
+    ├── base
+    │   ├── all.rb
+    │   └── deployment.rb  # SINGULAR
+    └── web
+        ├── deployments    # PLURAL
+        │   ├── dev.rb
+        │   └── prod.rb
+        └── deployments.rb # PLURAL
+
+The main difference is the pluralized filenames.

data/docs/_includes/layering/layers.md

@@ -21,11 +21,14 @@ Here's an example structure, so we can understand how layering works.
 
 To explain the layering, here's the general processing order that Kubes takes.
 
-1. The `.kubes/resources/base` folder is treated as a base layer.  It gets processed as pre-layers by Kubes.
-2. Then Kubes will process your `.kubes/resources/ROLE` definitions.
-3. Lastly, Kubes processes any post-layers in the `.kubes/resources/ROLE/KIND` folders.
+1. **Pre Layers**: The `.kubes/resources/base` folder is treated as a base layer.  It gets processed as pre-layers by Kubes.
+2. **Main Layer**: Then Kubes will process your `.kubes/resources/ROLE` definitions.
+3. **Post Layers**: Lastly, Kubes processes any post-layers in the `.kubes/resources/ROLE/KIND` folders.
 
-Note, both YAML and DSL forms support layering. They can be mixed together.
+Notes
+
+* Both YAML and DSL forms support layering. They can be mixed together.
+* In the Main Layer you can define single or multiple resource definitions.
 
 ## Full Layering
 

data/docs/_includes/sidebar.html

@@ -86,7 +86,12 @@
         </ul>
       </li>
       <li><a href="{% link _docs/generators.md %}">Generators</a></li>
-      <li><a href="{% link _docs/yaml.md %}">YAML</a></li>
+      <li><a href="{% link _docs/yaml.md %}">YAML</a>
+        <ul>
+          <li><a href="{% link _docs/yaml/multiple-resources.md %}">Multiple Resources</a></li>
+          <li><a href="{% link _docs/yaml/multiple-files.md %}">Multiple Files</a></li>
+        </ul>
+      </li>
       <li><a href="{% link _docs/layering.md %}">Layering</a>
         <ul>
           <li><a href="{% link _docs/layering/yaml.md %}">YAML</a></li>

data/lib/kubes/compiler/dsl/core/base.rb

@@ -18,7 +18,7 @@ module Kubes::Compiler::Dsl::Core
 
     def run
       evaluate_file(@path) # main resource definition
-      result if respond_to?(:result) # block form doesnt have result
+      result
     end
   end
 end

data/lib/kubes/compiler/dsl/core/blocks.rb

@@ -5,7 +5,11 @@ module Kubes::Compiler::Dsl::Core
       @results = {} # Hash key is the name of resource, using it so we can keep a map to handle layering
       @block_form = true # pluralizes the layer names
       super # handles layering and evaluating the main DSL file
-      self
+      result # Array
+    end
+
+    def result
+      @results.values # Array
     end
 
     def syntax_instance(meth, name)

data/lib/kubes/compiler/dsl/syntax/endpoint.rb

@@ -0,0 +1,34 @@
+module Kubes::Compiler::Dsl::Syntax
+  class Endpoint < Resource
+    fields :subsets
+
+    # kubectl explain endpoints.subsets
+    fields :addresses,         # <[]Object>
+           :notReadyAddresses, # <[]Object>
+           :ports              # <[]Object>
+
+    def default_kind
+      return @kind_from_block if @kind_from_block
+      "Endpoints" # always plural
+    end
+
+    def default_apiVersion
+      "v1"
+    end
+
+    def default_top
+      top = super
+      top.merge(
+        subsets: subsets
+      )
+    end
+
+    def default_subsets
+      [{
+        addresses: addresses,
+        notReadyAddresses: notReadyAddresses,
+        ports: ports,
+      }]
+    end
+  end
+end

data/lib/kubes/compiler/layering.rb

@@ -5,14 +5,10 @@ class Kubes::Compiler
 
       ext = File.extname(@path)
       kind = File.basename(@path).sub(ext,'') # IE: deployment
-      all = "all"
-      if @block_form
-        kind = kind.pluralize
-        all = all.pluralize
-      end
+      kind = kind.pluralize if @block_form
       layers = [
-        "#{all}",
-        "#{all}/#{Kubes.env}",
+        "all",
+        "all/#{Kubes.env}",
         "#{kind}",
         "#{kind}/#{Kubes.env}",
       ]

data/lib/kubes/compiler/strategy.rb

@@ -6,23 +6,9 @@ class Kubes::Compiler
     end
 
     def compile
-      klass = strategy_class
-      return false unless klass
-
-      strategy = klass.new(@options.merge(path: @path)) # Dsl or Erb
-      result = strategy.run
+      result = Dispatcher.new(@options.merge(path: @path)).dispatch
       result.decorate!(:pre) # compile pre phase decoration
       result
     end
-
-    def strategy_class
-      ext = File.extname(@path).sub('.','').to_sym
-      map = {
-        rb: Dsl,
-        yaml: Erb,
-        yml: Erb,
-      }
-      map[ext]
-    end
   end
 end

data/lib/kubes/compiler/strategy/base.rb

@@ -9,62 +9,6 @@ class Kubes::Compiler::Strategy
       @options = options
       @path = options[:path]
       @save_file = save_file(@path)
-      @data = @options[:data] || {}
-    end
-
-    def run
-      render_files(pre_layers)
-      render(@path) # main resource definition
-      render_files(post_layers)
-
-      Result.new(@save_file, @data)
-    end
-
-    def render_files(paths)
-      paths.each do |path|
-        next unless File.exist?(path) # layers may not exist
-        render(path)
-      end
-    end
-
-    # render and merge
-    def render(path)
-      result = render_strategy(path)
-      if result.is_a?(Kubes::Compiler::Dsl::Core::Blocks)
-        result = result.results
-      end
-      @data.deeper_merge!(result)
-    end
-
-    # Delegate to Dsl or Erb strategy again and pass @data down to allow rendering of a mix of yaml and rb files.
-    def render_strategy(path)
-      if path.include?('.rb')
-        dsl_class.new(@options.merge(path: path, data: @data)).run
-      else
-        Erb.new(@options.merge(data: @data)).render_result(path)
-      end
-    end
-
-    # Must be defined here in case coming from Kubes::Compiler::Strategy::Erb#render_strategy
-    def dsl_class
-      if block_form?
-        Kubes::Compiler::Dsl::Core::Blocks
-      else
-        syntax_class
-      end
-    end
-
-    def syntax_class
-      klass_name = normalize_kind(@save_file) # IE: @save_file: web/service.yaml
-      "Kubes::Compiler::Dsl::Syntax::#{klass_name}".constantize
-    rescue NameError
-      logger.debug "Using default resource for: #{klass_name}"
-      Kubes::Compiler::Dsl::Syntax::Resource # default
-    end
-
-    def block_form?
-      type = extract_type(@save_file)
-      type.pluralize == type
     end
   end
 end

data/lib/kubes/compiler/strategy/dispatcher.rb

@@ -0,0 +1,59 @@
+class Kubes::Compiler::Strategy
+  class Dispatcher < Base
+    def dispatch
+      result = render(@path) # main
+      results = [result].flatten # ensure array
+      data = results.map! do |main|
+        hash = pre_layer.deeper_merge!(main) # need the ! or deep_merge returns original hash
+        hash.deeper_merge!(post_layer)
+      end
+      Result.new(@save_file, data)
+    end
+
+    # Render via to Erb or one of the DSL syntax classes or Core/Blocks class
+    def render(path)
+      if path.include?('.rb')
+        klass = dsl_class(path) # IE: Kubes::Compiler::Dsl::Syntax::Deployment or Kubes::Compiler::Dsl::Core::Blocks
+        klass.new(@options.merge(path: path, data: @data)).run
+      else
+        Erb.new(@options.merge(data: @data)).render_result(path)
+      end
+    end
+
+    # Must be defined here in case coming from Kubes::Compiler::Strategy::Erb#render
+    def dsl_class(path)
+      if block_form?(path)
+        Kubes::Compiler::Dsl::Core::Blocks
+      else
+        syntax_class
+      end
+    end
+
+    def syntax_class
+      klass_name = normalize_kind(@save_file) # IE: @save_file: web/service.yaml
+      "Kubes::Compiler::Dsl::Syntax::#{klass_name}".constantize
+    rescue NameError
+      logger.debug "Using default resource for: #{klass_name}"
+      Kubes::Compiler::Dsl::Syntax::Resource # default
+    end
+
+    def block_form?(path)
+      type = extract_type(path)
+      type.pluralize == type
+    end
+
+    def pre_layer
+      merge_layers(pre_layers)
+    end
+
+    def post_layer
+      merge_layers(post_layers)
+    end
+
+    def merge_layers(layers)
+      layers.inject({}) do |hash, layer|
+        render(layer)
+      end
+    end
+  end
+end

data/lib/kubes/compiler/strategy/erb.rb

@@ -20,13 +20,12 @@ class Kubes::Compiler::Strategy
     end
 
     def render_result(path)
-      if File.exist?(path)
-        yaml = RenderMePretty.result(path, context: self)
-        result = yaml_load(path, yaml)
-        result.is_a?(Hash) ? result : {} # in case of blank yaml doc a Boolean false is returned
-      else
-        {}
-      end
+      return unless File.exist?(path)
+
+      yaml = RenderMePretty.result(path, context: self)
+      result = yaml_load(path, yaml)
+      # in case of blank yaml doc a Boolean false is returned. else Hash or Array is returned
+      %w[Array Hash].include?(result.class.to_s) ? result : {}
     end
 
     def yaml_load(path, yaml)

data/lib/kubes/compiler/strategy/result.rb

@@ -14,11 +14,15 @@ class Kubes::Compiler::Strategy
     # decorate(:pre) or decorate(:post)
     def decorate!(phase)
       klass = "Kubes::Compiler::Decorator::#{phase.to_s.camelize}".constantize
-      @data = klass.new(@data).result
+      results = [@data].flatten
+      results.map! do |r|
+        klass.new(r).result
+      end
     end
 
     def content
-      yaml_dump(@data)
+      result = @data.size == 1 ? @data.first : @data
+      yaml_dump(result)
     end
   end
 end

data/lib/kubes/compiler/util/normalize.rb

@@ -1,12 +1,12 @@
 module Kubes::Compiler::Util
   module Normalize
     def normalize_kind(path)
-      info = path.sub(%r{.*/.kubes/resources/}, '')
-      extract_type(info).underscore.camelize # Deployment, Service, Ingress, ManagedCertificate, etc
+      extract_type(path).underscore.camelize # Deployment, Service, Ingress, ManagedCertificate, etc
     end
 
     # info: web/service.yaml
     def extract_type(info)
+      info = info.sub(%r{.*/.kubes/resources/}, '')
       _, kind = info.split('/')
       kind.sub('.yaml','').sub('.yml','').sub('.rb','').sub(/-.*/,'')
     end

data/lib/kubes/compiler/util/yaml_dump.rb

@@ -4,12 +4,15 @@ require "yaml"
 module Kubes::Compiler::Util
   module YamlDump
     def yaml_dump(data)
-      if data.key?("kind")  # single resource in YAML
+      case data
+      when Array
+        items = data.map do |i|
+          standardize_yaml(i)
+        end
+        items.map(&:to_yaml).join("")
+      else # single resource in YAML
         data = standardize_yaml(data)
         data.to_yaml
-      else
-        items = data.map { |k,v| standardize_yaml(v) }
-        items.map(&:to_yaml).join("")
       end
     end
 

data/lib/kubes/version.rb

@@ -1,3 +1,3 @@
 module Kubes
-  VERSION = "0.6.8"
+  VERSION = "0.7.0"
 end

data/lib/templates/new/resource/dsl/endpoint.rb

@@ -0,0 +1,3 @@
+name "<%= app %>"
+addresses(["1.1.1.1"])
+ports(["80"])

data/lib/templates/new/resource/yaml/endpoint.yaml

@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: Endpoints
+metadata:
+  name: <%= app %>
+subsets:
+  - addresses:
+      - ip: 192.0.2.42
+    ports:
+      - port: 9376

data/spec/kubes/compiler/strategy/{dsl_spec.rb → dispatcher_spec.rb}

@@ -1,12 +1,13 @@
-describe Kubes::Compiler::Strategy::Dsl do
-  let(:dsl) { described_class.new(options) }
-  let(:options) { {path: fixture(resource) } }
+describe Kubes::Compiler::Strategy::Dispatcher do
+  let(:dispatcher) { described_class.new(options) }
+  let(:options) { {path: path } }
+  let(:path) { fixture(resource) }
 
   context "standard" do
     let(:resource) { "project/.kubes/resources/web/deployment" }
     it "run" do
-      result = dsl.run
-      expect(dsl.dsl_class).to eq(Kubes::Compiler::Dsl::Syntax::Deployment)
+      result = dispatcher.dispatch
+      expect(dispatcher.dsl_class(path)).to eq(Kubes::Compiler::Dsl::Syntax::Deployment)
       data = YAML.load(result.content)
       expect(data['kind']).to eq "Deployment"
     end
@@ -15,8 +16,8 @@ describe Kubes::Compiler::Strategy::Dsl do
   context "blocks" do
     let(:resource) { "blocks/deployments" }
     it "run" do
-      result = dsl.run
-      expect(dsl.dsl_class).to eq(Kubes::Compiler::Dsl::Core::Blocks)
+      result = dispatcher.dispatch
+      expect(dispatcher.dsl_class(path)).to eq(Kubes::Compiler::Dsl::Core::Blocks)
       resource = result.content.split('---').last
       data = YAML.load(resource)
       expect(data['kind']).to eq "Deployment"
@@ -27,8 +28,8 @@ describe Kubes::Compiler::Strategy::Dsl do
   context "multiple files" do
     let(:resource) { "multiple-files/.kubes/resources/web/deployment-1" }
     it "run" do
-      result = dsl.run
-      expect(dsl.dsl_class).to eq(Kubes::Compiler::Dsl::Syntax::Deployment)
+      result = dispatcher.dispatch
+      expect(dispatcher.dsl_class(path)).to eq(Kubes::Compiler::Dsl::Syntax::Deployment)
       data = YAML.load(result.content)
       expect(data['kind']).to eq "Deployment"
     end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kubes
 version: !ruby/object:Gem::Version
-  version: 0.6.8
+  version: 0.7.0
 platform: ruby
 authors:
 - Tung Nguyen
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-11-14 00:00:00.000000000 Z
+date: 2020-11-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -377,6 +377,8 @@ files:
 - docs/_docs/vs/helm.md
 - docs/_docs/vs/kustomize.md
 - docs/_docs/yaml.md
+- docs/_docs/yaml/multiple-files.md
+- docs/_docs/yaml/multiple-resources.md
 - docs/_includes/commands.html
 - docs/_includes/config/hooks/options.md
 - docs/_includes/content.html
@@ -593,6 +595,7 @@ files:
 - lib/kubes/compiler/dsl/syntax/config_map.rb
 - lib/kubes/compiler/dsl/syntax/daemon_set.rb
 - lib/kubes/compiler/dsl/syntax/deployment.rb
+- lib/kubes/compiler/dsl/syntax/endpoint.rb
 - lib/kubes/compiler/dsl/syntax/ingress.rb
 - lib/kubes/compiler/dsl/syntax/job.rb
 - lib/kubes/compiler/dsl/syntax/managed_certificate.rb
@@ -613,7 +616,7 @@ files:
 - lib/kubes/compiler/shared/plugin_helpers.rb
 - lib/kubes/compiler/strategy.rb
 - lib/kubes/compiler/strategy/base.rb
-- lib/kubes/compiler/strategy/dsl.rb
+- lib/kubes/compiler/strategy/dispatcher.rb
 - lib/kubes/compiler/strategy/erb.rb
 - lib/kubes/compiler/strategy/erb/yaml_error.rb
 - lib/kubes/compiler/strategy/pass.rb
@@ -674,6 +677,7 @@ files:
 - lib/templates/new/resource/dsl/config_map.rb
 - lib/templates/new/resource/dsl/daemon_set.rb
 - lib/templates/new/resource/dsl/deployment.rb
+- lib/templates/new/resource/dsl/endpoint.rb
 - lib/templates/new/resource/dsl/ingress.rb
 - lib/templates/new/resource/dsl/job.rb
 - lib/templates/new/resource/dsl/managed_certificate.rb
@@ -689,6 +693,7 @@ files:
 - lib/templates/new/resource/yaml/config_map.yaml
 - lib/templates/new/resource/yaml/daemon_set.yaml
 - lib/templates/new/resource/yaml/deployment.yaml
+- lib/templates/new/resource/yaml/endpoint.yaml
 - lib/templates/new/resource/yaml/ingress.yaml
 - lib/templates/new/resource/yaml/job.yaml
 - lib/templates/new/resource/yaml/managed_certificate.yaml
@@ -748,7 +753,7 @@ files:
 - spec/kubes/cli/prune_spec.rb
 - spec/kubes/compiler/decorator/post/deployment_spec.rb
 - spec/kubes/compiler/decorator/post/pod_spec.rb
-- spec/kubes/compiler/strategy/dsl_spec.rb
+- spec/kubes/compiler/strategy/dispatcher_spec.rb
 - spec/kubes/compiler_spec.rb
 - spec/kubes/dsl/daemon_set.rb
 - spec/kubes/dsl/deployment_spec.rb
@@ -823,7 +828,7 @@ test_files:
 - spec/kubes/cli/prune_spec.rb
 - spec/kubes/compiler/decorator/post/deployment_spec.rb
 - spec/kubes/compiler/decorator/post/pod_spec.rb
-- spec/kubes/compiler/strategy/dsl_spec.rb
+- spec/kubes/compiler/strategy/dispatcher_spec.rb
 - spec/kubes/compiler_spec.rb
 - spec/kubes/dsl/daemon_set.rb
 - spec/kubes/dsl/deployment_spec.rb

data/lib/kubes/compiler/strategy/dsl.rb

@@ -1,4 +0,0 @@
-class Kubes::Compiler::Strategy
-  class Dsl < Base
-  end
-end