kubernetes-deploy 0.26.7 → 0.27.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (38) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +4 -0
  3. data/CHANGELOG.md +16 -0
  4. data/CONTRIBUTING.md +1 -1
  5. data/README.md +1 -0
  6. data/exe/kubernetes-deploy +22 -6
  7. data/exe/kubernetes-render +7 -6
  8. data/exe/kubernetes-restart +3 -3
  9. data/exe/kubernetes-run +1 -3
  10. data/lib/kubernetes-deploy.rb +3 -27
  11. data/lib/kubernetes-deploy/common.rb +24 -0
  12. data/lib/kubernetes-deploy/deferred_summary_logging.rb +2 -0
  13. data/lib/kubernetes-deploy/deploy_task.rb +58 -65
  14. data/lib/kubernetes-deploy/duration_parser.rb +2 -0
  15. data/lib/kubernetes-deploy/ejson_secret_provisioner.rb +7 -14
  16. data/lib/kubernetes-deploy/errors.rb +0 -8
  17. data/lib/kubernetes-deploy/formatted_logger.rb +1 -0
  18. data/lib/kubernetes-deploy/kubeclient_builder.rb +2 -2
  19. data/lib/kubernetes-deploy/kubectl.rb +1 -0
  20. data/lib/kubernetes-deploy/kubernetes_resource.rb +3 -1
  21. data/lib/kubernetes-deploy/kubernetes_resource/custom_resource_definition.rb +0 -2
  22. data/lib/kubernetes-deploy/kubernetes_resource/deployment.rb +2 -0
  23. data/lib/kubernetes-deploy/kubernetes_resource/pod_set_base.rb +2 -0
  24. data/lib/kubernetes-deploy/kubernetes_resource/replica_set.rb +1 -0
  25. data/lib/kubernetes-deploy/kubernetes_resource/service.rb +21 -8
  26. data/lib/kubernetes-deploy/options_helper.rb +25 -12
  27. data/lib/kubernetes-deploy/render_task.rb +4 -4
  28. data/lib/kubernetes-deploy/resource_watcher.rb +4 -0
  29. data/lib/kubernetes-deploy/restart_task.rb +17 -13
  30. data/lib/kubernetes-deploy/runner_task.rb +9 -5
  31. data/lib/kubernetes-deploy/task_config.rb +16 -0
  32. data/lib/kubernetes-deploy/task_config_validator.rb +96 -0
  33. data/lib/kubernetes-deploy/template_sets.rb +135 -0
  34. data/lib/kubernetes-deploy/version.rb +1 -1
  35. metadata +7 -7
  36. data/lib/kubernetes-deploy/kubernetes_resource/memcached.rb +0 -43
  37. data/lib/kubernetes-deploy/kubernetes_resource/redis.rb +0 -56
  38. data/lib/kubernetes-deploy/template_discovery.rb +0 -15
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e0b06a9aef9e685727b66e55ab51169559c2420ac758cd567fc5274118149d14
4
- data.tar.gz: a50dd8b0579b2e87789a24b346efe7cbc7a440ecc1e8120d42b6734e7acee398
3
+ metadata.gz: 84961e6f603a285f3a16df6fa04f27201e80cf29476d6fc9a8a6ace4748ed64e
4
+ data.tar.gz: 484d466ef9e3cf80401edc34dd1a4ddefbd57013c1022b274d78f04456461739
5
5
  SHA512:
6
- metadata.gz: 01fa4ba1c7eb02c788c5787d7d50e0909776537caae16542fa973adc20f91596cf55698345f3e59bc3fb299244c1753a53b7ac8e4990cdeb7f7f4b9a3e0d7f39
7
- data.tar.gz: ee08ed90bed9a485987528914443d4e85209113575bdef800a93989668a9fd1728e6f6bb262dfdada2c0bd8c37a079d65545330bf9bd7dfc4d139fa5d3466a09
6
+ metadata.gz: 5a7f78c88ccc77549205df00fda10ef4843f721043182b01a5e00e1059a971de2cad12a2c820c4b79c81c1c6a4942baea69e14b4b443bb17f22dec002edfaedd
7
+ data.tar.gz: 9e6d96674cc341c65cff8fea18ab0cc18196580f4a5134c04a6c91ac6fdb4d1e9d9825520d6e0110432d3ff87c38623cb5fda8843157d73f039042ad6b360d94
@@ -11,3 +11,7 @@ Naming/FileName:
11
11
 
12
12
  Sorbet/ConstantsFromStrings:
13
13
  Enabled: false
14
+
15
+ Layout/Tab:
16
+ Exclude:
17
+ - test/integration/kubernetes_deploy_test.rb
@@ -1,5 +1,21 @@
1
1
  ## next
2
2
 
3
+ ## 0.27.0
4
+ *Enhancements*
5
+ - (alpha) Introduce a new `-f` flag for `kubernetes-deploy`. Allows passing in of multiple directories and/or filenames. Currently only usable by `kubernetes-deploy`, not `kubernetes-render`. [#514](https://github.com/Shopify/kubernetes-deploy/pull/514)
6
+ - Initial implementation of shared task validation objects. [#533](https://github.com/Shopify/kubernetes-deploy/pull/533)
7
+ - Restructure `require`s so that requiring a given task actually gives you the dependencies you need, and doesn't give what you don't need. [#487](https://github.com/Shopify/kubernetes-deploy/pull/487)
8
+ - **[Breaking change]** Added ServiceAccount, PodTemplate, ReplicaSet, Role, and RoleBinding to the prune whitelist.
9
+ * To see what resources may be affected, run `kubectl get $RESOURCE -o jsonpath='{ range .items[*] }{.metadata.namespace}{ "\t" }{.metadata.name}{ "\t" }{.metadata.annotations}{ "\n" }{ end }' --all-namespaces | grep "last-applied"`
10
+ * To exclude a resource from kubernetes-deploy (and kubectl apply) management, remove the last-applied annotation `kubectl annotate $RESOURCE $SECRET_NAME kubectl.kubernetes.io/last-applied-configuration-`.
11
+
12
+ *Bug Fixes*
13
+ - StatefulSets with 0 replicas explicitly specified don't fail deploy. [#540](https://github.com/Shopify/kubernetes-deploy/pull/540)
14
+ - Search all workloads if a Pod selector doesn't match any workloads when deploying a Service. [#541](https://github.com/Shopify/kubernetes-deploy/pull/541)
15
+
16
+ *Other*
17
+ - `EjsonSecretProvisioner#new` signature has changed. `EjsonSecretProvisioner` objects no longer have access to `kubectl`. Rather, the `ejson-keys` secret used for decryption is now passed in via the calling task. Note that we only consider the `new` and `run(!)` methods of tasks (render, deploy, etc) to have inviolable APIs, so we do not consider this change breaking. [#514](https://github.com/Shopify/kubernetes-deploy/pull/514)
18
+
3
19
  ## 0.26.7
4
20
 
5
21
  *Other*
@@ -47,7 +47,7 @@ We handle Kubernetes secrets, so it is critical that changes do not cause the co
47
47
 
48
48
  **Project architecture**
49
49
 
50
- The main interface of this project is our four tasks: `DeployTask`, `RestartTask`, `RunnerTask`, and `RenderTask`. The code in these classes should be high-level abstractions, with implementation details encapsulated in other classes. The public interface of these tasks is a `run` method (and a `run!` equivalent), the body of which should read like a set of phases and steps.
50
+ The main interface of this project is our four tasks: `DeployTask`, `RestartTask`, `RunnerTask`, and `RenderTask`. The code in these classes should be high-level abstractions, with implementation details encapsulated in other classes. The public interface of these tasks is a `run` method (and a `run!` equivalent), the body of which should read like a set of phases and steps. Note that non-task classes are considered internal and we reserve the right to change their API at any time.
51
51
 
52
52
  An important design principle of the tasks is that they should try to fail fast before touching the cluster if they will not succeed overall. Part of how we achieve this is by separating each task into phases, where the first phase simply gathers information and runs validations to determine what needs to be done and whether that will be able to succeed. In practice, this is the “Initializing <task>” phase for all tasks, plus the “Checking initial resource statuses” phase for DeployTask. Our users should be able to assume that these initial phases never modify their clusters.
53
53
 
data/README.md CHANGED
@@ -109,6 +109,7 @@ official compatibility chart below.
109
109
  Refer to `kubernetes-deploy --help` for the authoritative set of options.
110
110
 
111
111
  - `--template-dir=DIR`: Used to set the deploy directory. Set `$ENVIRONMENT` instead to use `config/deploy/$ENVIRONMENT`. This flag also supports reading from STDIN. You can do this by using `--template-dir=-`. Example: `cat templates_from_stdin/*.yml | kubernetes-deploy ns ctx --template-dir=-`.
112
+ - (alpha feature) `-f [PATHS]`: Accepts a comma-separated list of directories and/or filenames to specify the set of directories/files that will be deployed (use `-` to read from STDIN). Can be invoked multiple times. Cannot be combined with `--template-dir`. Example: `cat templates_from_stdin/*.yml | kubernetes-deploy ns ctx -f -,path/to/dir,path/to/file.yml`
112
113
  - `--bindings=BINDINGS`: Makes additional variables available to your ERB templates. For example, `kubernetes-deploy my-app cluster1 --bindings=color=blue,size=large` will expose `color` and `size`.
113
114
  - `--no-prune`: Skips pruning of resources that are no longer in your Kubernetes template set. Not recommended, as it allows your namespace to accumulate cruft that is not reflected in your deploy directory.
114
115
  - `--max-watch-seconds=seconds`: Raise a timeout error if it takes longer than _seconds_ for any
@@ -1,11 +1,16 @@
1
1
  #!/usr/bin/env ruby
2
2
  # frozen_string_literal: true
3
3
 
4
- require 'kubernetes-deploy'
4
+ require 'kubernetes-deploy/deploy_task'
5
+ require 'kubernetes-deploy/options_helper'
6
+ require 'kubernetes-deploy/bindings_parser'
7
+ require 'kubernetes-deploy/label_selector'
8
+
5
9
  require 'optparse'
6
10
 
7
11
  skip_wait = false
8
- template_dir = nil
12
+ template_dir = nil # deprecated
13
+ template_paths = []
9
14
  allow_protected_ns = false
10
15
  prune = true
11
16
  bindings = {}
@@ -22,9 +27,13 @@ ARGV.options do |opts|
22
27
  prot_ns = KubernetesDeploy::DeployTask::PROTECTED_NAMESPACES.join(', ')
23
28
  opts.on("--allow-protected-ns", "Enable deploys to #{prot_ns}; requires --no-prune") { allow_protected_ns = true }
24
29
  opts.on("--no-prune", "Disable deletion of resources that do not appear in the template dir") { prune = false }
25
- opts.on("--template-dir=DIR", "Set the template dir (default: config/deploy/$ENVIRONMENT).") do |d|
26
- template_dir = d
30
+ opts.on("--template-dir=DIR", "Set the template dir (default: config/deploy/$ENVIRONMENT).") do |dir|
31
+ template_dir = dir
32
+ end
33
+ opts.on("-f [PATHS]", Array, "comma separated list of template directories and/or filenames") do |paths|
34
+ template_paths += paths
27
35
  end
36
+
28
37
  opts.on("--verbose-log-prefix", "Add [context][namespace] to the log prefix") { verbose_log_prefix = true }
29
38
  opts.on("--max-watch-seconds=seconds",
30
39
  "Timeout error is raised if it takes longer than the specified number of seconds") do |t|
@@ -51,13 +60,20 @@ namespace = ARGV[0]
51
60
  context = ARGV[1]
52
61
  logger = KubernetesDeploy::FormattedLogger.build(namespace, context, verbose_prefix: verbose_log_prefix)
53
62
 
63
+ # Deprecation path: this can be removed when --template-dir is fully replaced by -f
64
+ if template_dir && !template_paths.empty?
65
+ logger.error("Error: --template-dir and -f flags cannot be combined")
66
+ exit(1)
67
+ end
68
+ template_paths = [template_dir] if template_paths.empty? && template_dir
69
+
54
70
  begin
55
- KubernetesDeploy::OptionsHelper.with_validated_template_dir(template_dir) do |dir|
71
+ KubernetesDeploy::OptionsHelper.with_processed_template_paths(template_paths) do |paths|
56
72
  runner = KubernetesDeploy::DeployTask.new(
57
73
  namespace: namespace,
58
74
  context: context,
59
75
  current_sha: ENV["REVISION"],
60
- template_dir: dir,
76
+ template_paths: paths,
61
77
  bindings: bindings,
62
78
  logger: logger,
63
79
  max_watch_seconds: max_watch_seconds,
@@ -1,11 +1,13 @@
1
1
  #!/usr/bin/env ruby
2
2
  # frozen_string_literal: true
3
3
 
4
- require 'kubernetes-deploy'
5
4
  require 'kubernetes-deploy/render_task'
5
+ require 'kubernetes-deploy/options_helper'
6
+ require 'kubernetes-deploy/bindings_parser'
7
+
6
8
  require 'optparse'
7
9
 
8
- template_dir = nil
10
+ template_dir = []
9
11
  bindings = {}
10
12
 
11
13
  ARGV.options do |opts|
@@ -13,7 +15,7 @@ ARGV.options do |opts|
13
15
  opts.on("--bindings=BINDINGS", "Expose additional variables to ERB templates " \
14
16
  "(format: k1=v1,k2=v2, JSON string or file (JSON or YAML) path prefixed by '@')") { |b| parser.add(b) }
15
17
  opts.on("--template-dir=DIR", "Set the template dir (default: config/deploy/$ENVIRONMENT).") do |d|
16
- template_dir = d
18
+ template_dir = [d]
17
19
  end
18
20
  opts.parse!
19
21
  bindings = parser.parse
@@ -23,11 +25,10 @@ templates = ARGV
23
25
  logger = KubernetesDeploy::FormattedLogger.build(verbose_prefix: false)
24
26
 
25
27
  begin
26
- KubernetesDeploy::OptionsHelper.with_validated_template_dir(template_dir) do |dir|
28
+ KubernetesDeploy::OptionsHelper.with_processed_template_paths(template_dir) do |dir|
27
29
  runner = KubernetesDeploy::RenderTask.new(
28
- logger: logger,
29
30
  current_sha: ENV["REVISION"],
30
- template_dir: dir,
31
+ template_dir: dir.first,
31
32
  bindings: bindings,
32
33
  )
33
34
 
@@ -3,8 +3,9 @@
3
3
 
4
4
  require 'optparse'
5
5
 
6
- require 'kubernetes-deploy'
7
6
  require 'kubernetes-deploy/restart_task'
7
+ require 'kubernetes-deploy/options_helper'
8
+ require 'kubernetes-deploy/label_selector'
8
9
 
9
10
  raw_deployments = nil
10
11
  max_watch_seconds = nil
@@ -20,9 +21,8 @@ end
20
21
 
21
22
  namespace = ARGV[0]
22
23
  context = ARGV[1]
23
- logger = KubernetesDeploy::FormattedLogger.build(namespace, context)
24
24
 
25
- restart = KubernetesDeploy::RestartTask.new(namespace: namespace, context: context, logger: logger,
25
+ restart = KubernetesDeploy::RestartTask.new(namespace: namespace, context: context,
26
26
  max_watch_seconds: max_watch_seconds)
27
27
  begin
28
28
  restart.perform!(raw_deployments, selector: selector)
@@ -1,8 +1,8 @@
1
1
  #!/usr/bin/env ruby
2
2
  # frozen_string_literal: true
3
3
 
4
- require 'kubernetes-deploy'
5
4
  require 'kubernetes-deploy/runner_task'
5
+ require 'kubernetes-deploy/options_helper'
6
6
  require 'optparse'
7
7
 
8
8
  template = "task-runner-template"
@@ -25,12 +25,10 @@ end
25
25
 
26
26
  namespace = ARGV[0]
27
27
  context = ARGV[1]
28
- logger = KubernetesDeploy::FormattedLogger.build(namespace, context)
29
28
 
30
29
  runner = KubernetesDeploy::RunnerTask.new(
31
30
  namespace: namespace,
32
31
  context: context,
33
- logger: logger,
34
32
  max_watch_seconds: max_watch_seconds
35
33
  )
36
34
 
@@ -1,30 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require 'active_support/core_ext/object/blank'
4
- require 'active_support/core_ext/hash/reverse_merge'
5
- require 'active_support/core_ext/hash/slice'
6
- require 'active_support/core_ext/numeric/time'
7
- require 'active_support/core_ext/string/inflections'
8
- require 'active_support/core_ext/string/strip'
9
- require 'active_support/core_ext/hash/keys'
10
- require 'active_support/core_ext/array/conversions'
11
- require 'active_support/duration'
12
- require 'colorized_string'
13
-
14
- require 'kubernetes-deploy/version'
15
- require 'kubernetes-deploy/oj'
16
- require 'kubernetes-deploy/errors'
17
- require 'kubernetes-deploy/formatted_logger'
18
- require 'kubernetes-deploy/options_helper'
19
- require 'kubernetes-deploy/statsd'
20
3
  require 'kubernetes-deploy/deploy_task'
21
- require 'kubernetes-deploy/concurrency'
22
- require 'kubernetes-deploy/bindings_parser'
23
- require 'kubernetes-deploy/duration_parser'
24
- require 'kubernetes-deploy/resource_cache'
25
- require 'kubernetes-deploy/label_selector'
26
-
27
- module KubernetesDeploy
28
- MIN_KUBE_VERSION = '1.10.0'
29
- StatsD.build
30
- end
4
+ require 'kubernetes-deploy/render_task'
5
+ require 'kubernetes-deploy/restart_task'
6
+ require 'kubernetes-deploy/runner_task'
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'active_support/core_ext/object/blank'
4
+ require 'active_support/core_ext/hash/reverse_merge'
5
+ require 'active_support/core_ext/hash/slice'
6
+ require 'active_support/core_ext/numeric/time'
7
+ require 'active_support/core_ext/string/inflections'
8
+ require 'active_support/core_ext/string/strip'
9
+ require 'active_support/core_ext/hash/keys'
10
+ require 'active_support/core_ext/array/conversions'
11
+ require 'colorized_string'
12
+
13
+ require 'kubernetes-deploy/version'
14
+ require 'kubernetes-deploy/oj'
15
+ require 'kubernetes-deploy/errors'
16
+ require 'kubernetes-deploy/formatted_logger'
17
+ require 'kubernetes-deploy/statsd'
18
+ require 'kubernetes-deploy/task_config'
19
+ require 'kubernetes-deploy/task_config_validator'
20
+
21
+ module KubernetesDeploy
22
+ MIN_KUBE_VERSION = '1.10.0'
23
+ StatsD.build
24
+ end
@@ -1,4 +1,6 @@
1
1
  # frozen_string_literal: true
2
+ require 'colorized_string'
3
+
2
4
  module KubernetesDeploy
3
5
  # Adds the methods kubernetes-deploy requires to your logger class.
4
6
  # These methods include helpers for logging consistent headings, as well as facilities for
@@ -1,9 +1,12 @@
1
1
  # frozen_string_literal: true
2
- require 'open3'
3
2
  require 'yaml'
4
3
  require 'shellwords'
5
4
  require 'tempfile'
6
5
  require 'fileutils'
6
+
7
+ require 'kubernetes-deploy/common'
8
+ require 'kubernetes-deploy/concurrency'
9
+ require 'kubernetes-deploy/resource_cache'
7
10
  require 'kubernetes-deploy/kubernetes_resource'
8
11
  %w(
9
12
  custom_resource
@@ -13,9 +16,7 @@ require 'kubernetes-deploy/kubernetes_resource'
13
16
  ingress
14
17
  persistent_volume_claim
15
18
  pod
16
- redis
17
19
  network_policy
18
- memcached
19
20
  service
20
21
  pod_template
21
22
  pod_disruption_budget
@@ -41,7 +42,7 @@ require 'kubernetes-deploy/kubeclient_builder'
41
42
  require 'kubernetes-deploy/ejson_secret_provisioner'
42
43
  require 'kubernetes-deploy/renderer'
43
44
  require 'kubernetes-deploy/cluster_resource_discovery'
44
- require 'kubernetes-deploy/template_discovery'
45
+ require 'kubernetes-deploy/template_sets'
45
46
 
46
47
  module KubernetesDeploy
47
48
  class DeployTask
@@ -58,7 +59,6 @@ module KubernetesDeploy
58
59
  # core/v1/Endpoints -- managed by services
59
60
  # core/v1/PersistentVolumeClaim -- would delete data
60
61
  # core/v1/ReplicationController -- superseded by deployments/replicasets
61
- # extensions/v1beta1/ReplicaSet -- managed by deployments
62
62
 
63
63
  def predeploy_sequence
64
64
  before_crs = %w(
@@ -85,7 +85,10 @@ module KubernetesDeploy
85
85
  core/v1/Service
86
86
  core/v1/ResourceQuota
87
87
  core/v1/Secret
88
+ core/v1/ServiceAccount
89
+ core/v1/PodTemplate
88
90
  batch/v1/Job
91
+ extensions/v1beta1/ReplicaSet
89
92
  extensions/v1beta1/DaemonSet
90
93
  extensions/v1beta1/Deployment
91
94
  extensions/v1beta1/Ingress
@@ -94,6 +97,8 @@ module KubernetesDeploy
94
97
  autoscaling/v1/HorizontalPodAutoscaler
95
98
  policy/v1beta1/PodDisruptionBudget
96
99
  batch/v1beta1/CronJob
100
+ rbac.authorization.k8s.io/v1/Role
101
+ rbac.authorization.k8s.io/v1/RoleBinding
97
102
  )
98
103
  wl + cluster_resource_discoverer.crds.select(&:prunable?).map(&:group_version_kind)
99
104
  end
@@ -102,22 +107,21 @@ module KubernetesDeploy
102
107
  kubectl.server_version
103
108
  end
104
109
 
105
- def initialize(namespace:, context:, current_sha:, template_dir:, logger:, kubectl_instance: nil, bindings: {},
106
- max_watch_seconds: nil, selector: nil)
110
+ def initialize(namespace:, context:, current_sha:, logger: nil, kubectl_instance: nil, bindings: {},
111
+ max_watch_seconds: nil, selector: nil, template_paths: [], template_dir: nil)
112
+ template_dir = File.expand_path(template_dir) if template_dir
113
+ template_paths = (template_paths.map { |path| File.expand_path(path) } << template_dir).compact
114
+
115
+ @logger = logger || KubernetesDeploy::FormattedLogger.build(namespace, context)
116
+ @template_sets = TemplateSets.from_dirs_and_files(paths: template_paths, logger: @logger)
117
+ @task_config = KubernetesDeploy::TaskConfig.new(context, namespace, @logger)
118
+ @bindings = bindings
107
119
  @namespace = namespace
108
120
  @namespace_tags = []
109
121
  @context = context
110
122
  @current_sha = current_sha
111
- @template_dir = File.expand_path(template_dir)
112
- @logger = logger
113
123
  @kubectl = kubectl_instance
114
124
  @max_watch_seconds = max_watch_seconds
115
- @renderer = KubernetesDeploy::Renderer.new(
116
- current_sha: @current_sha,
117
- template_dir: @template_dir,
118
- logger: @logger,
119
- bindings: bindings,
120
- )
121
125
  @selector = selector
122
126
  end
123
127
 
@@ -204,15 +208,18 @@ module KubernetesDeploy
204
208
  )
205
209
  end
206
210
 
207
- def ejson_provisioner
208
- @ejson_provisioner ||= EjsonSecretProvisioner.new(
209
- namespace: @namespace,
210
- context: @context,
211
- template_dir: @template_dir,
212
- logger: @logger,
213
- statsd_tags: @namespace_tags,
214
- selector: @selector,
215
- )
211
+ def ejson_provisioners
212
+ @ejson_provisoners ||= @template_sets.ejson_secrets_files.map do |ejson_secret_file|
213
+ EjsonSecretProvisioner.new(
214
+ namespace: @namespace,
215
+ context: @context,
216
+ ejson_keys_secret: ejson_keys_secret,
217
+ ejson_file: ejson_secret_file,
218
+ logger: @logger,
219
+ statsd_tags: @namespace_tags,
220
+ selector: @selector,
221
+ )
222
+ end
216
223
  end
217
224
 
218
225
  def deploy_has_priority_resources?(resources)
@@ -267,54 +274,37 @@ module KubernetesDeploy
267
274
  measure_method(:check_initial_status, "initial_status.duration")
268
275
 
269
276
  def secrets_from_ejson
270
- ejson_provisioner.resources
277
+ ejson_provisioners.flat_map(&:resources)
271
278
  end
272
279
 
273
280
  def discover_resources
281
+ @logger.info("Discovering resources:")
274
282
  resources = []
275
- crds = cluster_resource_discoverer.crds.group_by(&:kind)
276
- @logger.info("Discovering templates:")
277
-
278
- TemplateDiscovery.new(@template_dir).templates.each do |filename|
279
- split_templates(filename) do |r_def|
280
- crd = crds[r_def["kind"]]&.first
281
- r = KubernetesResource.build(namespace: @namespace, context: @context, logger: @logger, definition: r_def,
282
- statsd_tags: @namespace_tags, crd: crd)
283
- resources << r
284
- @logger.info(" - #{r.id}")
285
- end
283
+ crds_by_kind = cluster_resource_discoverer.crds.group_by(&:kind)
284
+ @template_sets.with_resource_definitions(render_erb: true,
285
+ current_sha: @current_sha, bindings: @bindings) do |r_def|
286
+ crd = crds_by_kind[r_def["kind"]]&.first
287
+ r = KubernetesResource.build(namespace: @namespace, context: @context, logger: @logger, definition: r_def,
288
+ statsd_tags: @namespace_tags, crd: crd)
289
+ resources << r
290
+ @logger.info(" - #{r.id}")
286
291
  end
292
+
287
293
  secrets_from_ejson.each do |secret|
288
294
  resources << secret
289
295
  @logger.info(" - #{secret.id} (from ejson)")
290
296
  end
297
+
291
298
  if (global = resources.select(&:global?).presence)
292
299
  @logger.warn("Detected non-namespaced #{'resource'.pluralize(global.count)} which will never be pruned:")
293
300
  global.each { |r| @logger.warn(" - #{r.id}") }
294
301
  end
295
302
  resources.sort
296
- end
297
- measure_method(:discover_resources)
298
-
299
- def split_templates(filename)
300
- file_content = File.read(File.join(@template_dir, filename))
301
- rendered_content = @renderer.render_template(filename, file_content)
302
- YAML.load_stream(rendered_content, "<rendered> #{filename}") do |doc|
303
- next if doc.blank?
304
- unless doc.is_a?(Hash)
305
- raise InvalidTemplateError.new("Template is not a valid Kubernetes manifest",
306
- filename: filename, content: doc)
307
- end
308
- yield doc
309
- end
310
303
  rescue InvalidTemplateError => e
311
- e.filename ||= filename
312
304
  record_invalid_template(err: e.message, filename: e.filename, content: e.content)
313
305
  raise FatalDeploymentError, "Failed to render and parse template"
314
- rescue Psych::SyntaxError => e
315
- record_invalid_template(err: e.message, filename: filename, content: rendered_content)
316
- raise FatalDeploymentError, "Failed to render and parse template"
317
306
  end
307
+ measure_method(:discover_resources)
318
308
 
319
309
  def record_invalid_template(err:, filename:, content: nil)
320
310
  debug_msg = ColorizedString.new("Invalid template: #{filename}\n").red
@@ -332,12 +322,7 @@ module KubernetesDeploy
332
322
  def validate_configuration(allow_protected_ns:, prune:)
333
323
  errors = []
334
324
  errors += kubeclient_builder.validate_config_files
335
-
336
- if !File.directory?(@template_dir)
337
- errors << "Template directory `#{@template_dir}` doesn't exist"
338
- elsif Dir.entries(@template_dir).none? { |file| file =~ /(\.ya?ml(\.erb)?)$|(secrets\.ejson)$/ }
339
- errors << "`#{@template_dir}` doesn't contain valid templates (secrets.ejson or postfix .yml, .yml.erb)"
340
- end
325
+ errors += @template_sets.validate
341
326
 
342
327
  if @namespace.blank?
343
328
  errors << "Namespace must be specified"
@@ -546,9 +531,7 @@ module KubernetesDeploy
546
531
  end
547
532
  end
548
533
  raise FatalDeploymentError, "Failed to reach server for #{@context}" unless success
549
- if kubectl.server_version < Gem::Version.new(MIN_KUBE_VERSION)
550
- @logger.warn(KubernetesDeploy::Errors.server_version_warning(server_version))
551
- end
534
+ TaskConfigValidator.new(@task_config, kubectl, kubeclient_builder, only: [:validate_server_version]).valid?
552
535
  end
553
536
 
554
537
  def confirm_namespace_exists
@@ -568,8 +551,7 @@ module KubernetesDeploy
568
551
 
569
552
  # make sure to never prune the ejson-keys secret
570
553
  def confirm_ejson_keys_not_prunable
571
- secret = ejson_provisioner.ejson_keys_secret
572
- return unless secret.dig("metadata", "annotations", KubernetesResource::LAST_APPLIED_ANNOTATION)
554
+ return unless ejson_keys_secret.dig("metadata", "annotations", KubernetesResource::LAST_APPLIED_ANNOTATION)
573
555
 
574
556
  @logger.error("Deploy cannot proceed because protected resource " \
575
557
  "Secret/#{EjsonSecretProvisioner::EJSON_KEYS_SECRET} would be pruned.")
@@ -588,6 +570,17 @@ module KubernetesDeploy
588
570
  @kubectl ||= Kubectl.new(namespace: @namespace, context: @context, logger: @logger, log_failure_by_default: true)
589
571
  end
590
572
 
573
+ def ejson_keys_secret
574
+ @ejson_keys_secret ||= begin
575
+ out, err, st = kubectl.run("get", "secret", EjsonSecretProvisioner::EJSON_KEYS_SECRET, output: "json",
576
+ raise_if_not_found: true, attempts: 3, output_is_sensitive: true, log_failure: true)
577
+ unless st.success?
578
+ raise EjsonSecretError, "Error retrieving Secret/#{EjsonSecretProvisioner::EJSON_KEYS_SECRET}: #{err}"
579
+ end
580
+ JSON.parse(out)
581
+ end
582
+ end
583
+
591
584
  def statsd_tags
592
585
  %W(namespace:#{@namespace} sha:#{@current_sha} context:#{@context}) | @namespace_tags
593
586
  end