kubernetes-deploy 0.22.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +5 -5
- data/.rubocop.yml +8 -0
- data/CHANGELOG.md +16 -0
- data/README.md +32 -0
- data/exe/kubernetes-deploy +2 -15
- data/exe/kubernetes-render +32 -0
- data/kubernetes-deploy.gemspec +5 -3
- data/lib/kubernetes-deploy.rb +5 -3
- data/lib/kubernetes-deploy/cluster_resource_discovery.rb +34 -0
- data/lib/kubernetes-deploy/container_logs.rb +25 -13
- data/lib/kubernetes-deploy/deploy_task.rb +68 -50
- data/lib/kubernetes-deploy/errors.rb +1 -0
- data/lib/kubernetes-deploy/formatted_logger.rb +16 -2
- data/lib/kubernetes-deploy/kubeclient_builder/google_friendly_config.rb +4 -6
- data/lib/kubernetes-deploy/kubectl.rb +20 -9
- data/lib/kubernetes-deploy/kubernetes_resource.rb +5 -6
- data/lib/kubernetes-deploy/kubernetes_resource/cloudsql.rb +3 -4
- data/lib/kubernetes-deploy/kubernetes_resource/daemon_set.rb +4 -5
- data/lib/kubernetes-deploy/kubernetes_resource/deployment.rb +7 -8
- data/lib/kubernetes-deploy/kubernetes_resource/memcached.rb +4 -5
- data/lib/kubernetes-deploy/kubernetes_resource/pod.rb +7 -5
- data/lib/kubernetes-deploy/kubernetes_resource/pod_set_base.rb +12 -6
- data/lib/kubernetes-deploy/kubernetes_resource/redis.rb +5 -6
- data/lib/kubernetes-deploy/kubernetes_resource/replica_set.rb +23 -5
- data/lib/kubernetes-deploy/kubernetes_resource/role.rb +22 -0
- data/lib/kubernetes-deploy/kubernetes_resource/service.rb +8 -4
- data/lib/kubernetes-deploy/kubernetes_resource/stateful_set.rb +2 -3
- data/lib/kubernetes-deploy/oj.rb +4 -0
- data/lib/kubernetes-deploy/options_helper.rb +27 -0
- data/lib/kubernetes-deploy/remote_logs.rb +10 -4
- data/lib/kubernetes-deploy/render_task.rb +119 -0
- data/lib/kubernetes-deploy/renderer.rb +1 -1
- data/lib/kubernetes-deploy/resource_cache.rb +64 -0
- data/lib/kubernetes-deploy/resource_watcher.rb +27 -6
- data/lib/kubernetes-deploy/restart_task.rb +5 -6
- data/lib/kubernetes-deploy/runner_task.rb +6 -10
- data/lib/kubernetes-deploy/statsd.rb +60 -7
- data/lib/kubernetes-deploy/template_discovery.rb +15 -0
- data/lib/kubernetes-deploy/version.rb +1 -1
- data/pull_request_template.md +8 -0
- metadata +47 -5
- data/lib/kubernetes-deploy/resource_discovery.rb +0 -19
- data/lib/kubernetes-deploy/sync_mediator.rb +0 -80
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 01ddead06a28a399d7364d5789c2bcb1cee0e0aa4ae471a4af1a90c32973050b
|
4
|
+
data.tar.gz: ebcedef932871bd542db69bc0b48fb6f47f2482c712aa9380a79cb872b6c3e82
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 837dcaf96f89beae9f9a57ceb207595d602f1e039217007750112828a8076d0337c17bb6a75bcc2d5a5a8c531f8b0db363a19293d5dc6ce011cdd70440c888a7
|
7
|
+
data.tar.gz: e8e6484dbfae1b867542a02c39f7c96ed4434ab3f900f49fea72523466ea9be23ce2d1f2209d9fb609b7e71af63d1c047ab6f98f756ec28eef8bb47245fb9180
|
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,19 @@
|
|
1
|
+
## 0.23.0
|
2
|
+
|
3
|
+
*Features*
|
4
|
+
- New command: `kubernetes-render` is a tool for rendering ERB templates to raw Kubernetes YAML. It's useful for seeing what `kubernetes-deploy` does before actually invoking `kubectl` on the rendered YAML. It's also useful for outputting YAML that can be passed to other tools, for validation or introspection purposes. ([#375](https://github.com/Shopify/kubernetes-deploy/pull/375/files))
|
5
|
+
- **[Breaking change]** This release completes the conversion of `kubernetes-deploy` StatsD metrics to `distribution`s, which was done for `kubernetes-restart` and `kubernetes-run` in v0.22.0.
|
6
|
+
- Several new distribution metrics are available to give insight into the timing of each step of the deploy process: `KubernetesDeploy.validate_configuration.duration`, `KubernetesDeploy.discover_resources.duration`, `KubernetesDeploy.validate_resources.duration`, `KubernetesDeploy.initial_status.duration`, `KubernetesDeploy.create_ejson_secrets.duration`, `KubernetesDeploy.apply_all.duration`, `KubernetesDeploy.sync.duration`
|
7
|
+
- **[Breaking change]** `KubernetesDeploy.resource.duration` no longer includes `sha` or `resource` tags. ([#392](https://github.com/Shopify/kubernetes-deploy/pull/392))
|
8
|
+
|
9
|
+
*Enhancements*
|
10
|
+
- Roles are now predeployed before RoleBindings ([#380](https://github.com/Shopify/kubernetes-deploy/pull/380))
|
11
|
+
- Several performance enhancements for deploys to namespaces with hundreds of resources.
|
12
|
+
- KubernetesDeploy no longer modifies the global StatsD configuration when used as a gem ([#384](https://github.com/Shopify/kubernetes-deploy/pull/384))
|
13
|
+
|
14
|
+
*Bug fixes*
|
15
|
+
- Handle out-of-order arrival of entries from different streams when processing logs ([#401](https://github.com/Shopify/kubernetes-deploy/pull/401))
|
16
|
+
|
1
17
|
## 0.22.0
|
2
18
|
|
3
19
|
*Features*
|
data/README.md
CHANGED
@@ -50,6 +50,10 @@ This repo also includes related tools for [running tasks](#kubernetes-run) and [
|
|
50
50
|
* [Prerequisites](#prerequisites-1)
|
51
51
|
* [Usage](#usage-2)
|
52
52
|
|
53
|
+
**KUBERNETES-RENDER**
|
54
|
+
* [Prerequisites](#prerequisites-2)
|
55
|
+
* [Usage](#usage-3)
|
56
|
+
|
53
57
|
**DEVELOPMENT**
|
54
58
|
* [Setup](#setup)
|
55
59
|
* [Running the test suite locally](#running-the-test-suite-locally)
|
@@ -371,6 +375,34 @@ Based on this specification `kubernetes-run` will create a new pod with the entr
|
|
371
375
|
|
372
376
|
|
373
377
|
|
378
|
+
# kubernetes-render
|
379
|
+
|
380
|
+
`kubernetes-render` is a tool for rendering ERB templates to raw Kubernetes YAML. It's useful for seeing what `kubernetes-deploy` does before actually invoking `kubectl` on the rendered YAML. It's also useful for outputting YAML that can be passed to other tools, for validation or introspection purposes.
|
381
|
+
|
382
|
+
|
383
|
+
## Prerequisites
|
384
|
+
|
385
|
+
* `kubernetes-render` does __not__ require a running cluster or an active kubernetes context, which is nice if you want to run it in a CI environment, potentially alongside something like https://github.com/garethr/kubeval to make sure your configuration is sound.
|
386
|
+
* Like the other `kubernetes-deploy` commands, `kubernetes-render` requires the `$REVISION` environment variable to be set, and will make it available as `current_sha` in your ERB templates.
|
387
|
+
|
388
|
+
## Usage
|
389
|
+
|
390
|
+
To render all templates in your template dir, run:
|
391
|
+
|
392
|
+
```
|
393
|
+
kubernetes-render --template-dir=./path/to/template/dir
|
394
|
+
```
|
395
|
+
|
396
|
+
To render some templates in a template dir, run kubernetes-render with the names of the templates to render:
|
397
|
+
|
398
|
+
```
|
399
|
+
kubernetes-render --template-dir=./path/to/template/dir this-template.yaml.erb that-template.yaml.erb
|
400
|
+
```
|
401
|
+
|
402
|
+
*Options:*
|
403
|
+
|
404
|
+
- `--template-dir=DIR`: Used to set the directory to interpret template names relative to. This is often the same directory passed as `--template-dir` when running `kubernetes-deploy` to actually deploy templates. Set `$ENVIRONMENT` instead to use `config/deploy/$ENVIRONMENT`.
|
405
|
+
- `--bindings=BINDINGS`: Makes additional variables available to your ERB templates. For example, `kubernetes-render --bindings=color=blue,size=large some-template.yaml.erb` will expose `color` and `size` to `some-template.yaml.erb`.
|
374
406
|
|
375
407
|
|
376
408
|
# Development
|
data/exe/kubernetes-deploy
CHANGED
@@ -40,23 +40,10 @@ ARGV.options do |opts|
|
|
40
40
|
opts.parse!
|
41
41
|
end
|
42
42
|
|
43
|
-
if !template_dir && ENV.key?("ENVIRONMENT")
|
44
|
-
template_dir = "config/deploy/#{ENV['ENVIRONMENT']}"
|
45
|
-
end
|
46
|
-
|
47
|
-
if !template_dir || template_dir.empty?
|
48
|
-
puts "Template directory is unknown. " \
|
49
|
-
"Either specify --template-dir argument or set $ENVIRONMENT to use config/deploy/$ENVIRONMENT as a default path."
|
50
|
-
exit 1
|
51
|
-
end
|
52
|
-
|
53
|
-
revision = ENV.fetch('REVISION') do
|
54
|
-
puts "ENV['REVISION'] is missing. Please specify the commit SHA"
|
55
|
-
exit 1
|
56
|
-
end
|
57
|
-
|
58
43
|
namespace = ARGV[0]
|
59
44
|
context = ARGV[1]
|
45
|
+
template_dir = KubernetesDeploy::OptionsHelper.default_and_check_template_dir(template_dir)
|
46
|
+
revision = KubernetesDeploy::OptionsHelper.revision_from_environment
|
60
47
|
logger = KubernetesDeploy::FormattedLogger.build(namespace, context, verbose_prefix: verbose_log_prefix)
|
61
48
|
|
62
49
|
runner = KubernetesDeploy::DeployTask.new(
|
@@ -0,0 +1,32 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# frozen_string_literal: true
|
3
|
+
|
4
|
+
require 'kubernetes-deploy'
|
5
|
+
require 'kubernetes-deploy/render_task'
|
6
|
+
require 'optparse'
|
7
|
+
|
8
|
+
template_dir = nil
|
9
|
+
bindings = {}
|
10
|
+
|
11
|
+
ARGV.options do |opts|
|
12
|
+
opts.on("--bindings=BINDINGS", "Expose additional variables to ERB templates " \
|
13
|
+
"(format: k1=v1,k2=v2, JSON string or file (JSON or YAML) path prefixed by '@')") do |binds|
|
14
|
+
bindings.merge!(KubernetesDeploy::BindingsParser.parse(binds))
|
15
|
+
end
|
16
|
+
opts.on("--template-dir=DIR", "Set the template dir (default: config/deploy/$ENVIRONMENT)") { |v| template_dir = v }
|
17
|
+
opts.parse!
|
18
|
+
end
|
19
|
+
|
20
|
+
templates = ARGV
|
21
|
+
logger = KubernetesDeploy::FormattedLogger.build(verbose_prefix: false)
|
22
|
+
revision = KubernetesDeploy::OptionsHelper.revision_from_environment
|
23
|
+
|
24
|
+
runner = KubernetesDeploy::RenderTask.new(
|
25
|
+
logger: logger,
|
26
|
+
current_sha: revision,
|
27
|
+
template_dir: template_dir,
|
28
|
+
bindings: bindings,
|
29
|
+
)
|
30
|
+
|
31
|
+
success = runner.run(STDOUT, templates)
|
32
|
+
exit 1 unless success
|
data/kubernetes-deploy.gemspec
CHANGED
@@ -15,12 +15,12 @@ Gem::Specification.new do |spec|
|
|
15
15
|
spec.homepage = "https://github.com/Shopify/kubernetes-deploy"
|
16
16
|
spec.license = "MIT"
|
17
17
|
|
18
|
-
spec.files =
|
18
|
+
spec.files = %x(git ls-files -z).split("\x0").reject do |f|
|
19
19
|
f.match(%r{^(test|spec|features)/})
|
20
20
|
end
|
21
21
|
spec.bindir = "exe"
|
22
22
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
23
|
-
spec.require_paths =
|
23
|
+
spec.require_paths = %w(lib)
|
24
24
|
|
25
25
|
spec.required_ruby_version = '>= 2.3.0'
|
26
26
|
spec.add_dependency "activesupport", ">= 5.0"
|
@@ -28,7 +28,9 @@ Gem::Specification.new do |spec|
|
|
28
28
|
spec.add_dependency "googleauth", "~> 0.6.6" # https://github.com/google/google-auth-library-ruby/issues/153
|
29
29
|
spec.add_dependency "ejson", "~> 1.0"
|
30
30
|
spec.add_dependency "colorize", "~> 0.8"
|
31
|
-
spec.add_dependency "statsd-instrument",
|
31
|
+
spec.add_dependency "statsd-instrument", '~> 2.3', '>= 2.3.2'
|
32
|
+
spec.add_dependency "oj", "~> 3.7"
|
33
|
+
spec.add_dependency "concurrent-ruby", "~> 1.1"
|
32
34
|
|
33
35
|
spec.add_development_dependency "bundler"
|
34
36
|
spec.add_development_dependency "rake", "~> 10.0"
|
data/lib/kubernetes-deploy.rb
CHANGED
@@ -11,16 +11,18 @@ require 'active_support/duration'
|
|
11
11
|
require 'colorized_string'
|
12
12
|
|
13
13
|
require 'kubernetes-deploy/version'
|
14
|
+
require 'kubernetes-deploy/oj'
|
14
15
|
require 'kubernetes-deploy/errors'
|
15
16
|
require 'kubernetes-deploy/formatted_logger'
|
16
|
-
require 'kubernetes-deploy/
|
17
|
+
require 'kubernetes-deploy/options_helper'
|
17
18
|
require 'kubernetes-deploy/statsd'
|
19
|
+
require 'kubernetes-deploy/deploy_task'
|
18
20
|
require 'kubernetes-deploy/concurrency'
|
19
21
|
require 'kubernetes-deploy/bindings_parser'
|
20
22
|
require 'kubernetes-deploy/duration_parser'
|
21
|
-
require 'kubernetes-deploy/
|
23
|
+
require 'kubernetes-deploy/resource_cache'
|
22
24
|
|
23
25
|
module KubernetesDeploy
|
24
26
|
MIN_KUBE_VERSION = '1.9.0'
|
25
|
-
|
27
|
+
StatsD.build
|
26
28
|
end
|
@@ -0,0 +1,34 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module KubernetesDeploy
|
4
|
+
class ClusterResourceDiscovery
|
5
|
+
def initialize(namespace:, context:, logger:, namespace_tags:)
|
6
|
+
@namespace = namespace
|
7
|
+
@context = context
|
8
|
+
@logger = logger
|
9
|
+
@namespace_tags = namespace_tags
|
10
|
+
end
|
11
|
+
|
12
|
+
def crds
|
13
|
+
@crds ||= fetch_crds.map do |cr_def|
|
14
|
+
CustomResourceDefinition.new(namespace: @namespace, context: @context, logger: @logger,
|
15
|
+
definition: cr_def, statsd_tags: @namespace_tags)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
private
|
20
|
+
|
21
|
+
def fetch_crds
|
22
|
+
raw_json, _, st = kubectl.run("get", "CustomResourceDefinition", "-a", "--output=json", attempts: 5)
|
23
|
+
if st.success?
|
24
|
+
JSON.parse(raw_json)["items"]
|
25
|
+
else
|
26
|
+
[]
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
def kubectl
|
31
|
+
@kubectl ||= Kubectl.new(namespace: @namespace, context: @context, logger: @logger, log_failure_by_default: true)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
@@ -5,18 +5,20 @@ module KubernetesDeploy
|
|
5
5
|
|
6
6
|
DEFAULT_LINE_LIMIT = 250
|
7
7
|
|
8
|
-
def initialize(parent_id:, container_name:, logger:)
|
8
|
+
def initialize(parent_id:, container_name:, namespace:, context:, logger:)
|
9
9
|
@parent_id = parent_id
|
10
10
|
@container_name = container_name
|
11
|
+
@namespace = namespace
|
12
|
+
@context = context
|
11
13
|
@logger = logger
|
12
14
|
@lines = []
|
13
15
|
@next_print_index = 0
|
14
16
|
end
|
15
17
|
|
16
|
-
def sync
|
17
|
-
new_logs = fetch_latest
|
18
|
+
def sync
|
19
|
+
new_logs = fetch_latest
|
18
20
|
return unless new_logs.present?
|
19
|
-
@lines +=
|
21
|
+
@lines += sort_and_deduplicate(new_logs)
|
20
22
|
end
|
21
23
|
|
22
24
|
def empty?
|
@@ -39,7 +41,7 @@ module KubernetesDeploy
|
|
39
41
|
|
40
42
|
private
|
41
43
|
|
42
|
-
def fetch_latest
|
44
|
+
def fetch_latest
|
43
45
|
cmd = ["logs", @parent_id, "--container=#{container_name}", "--timestamps"]
|
44
46
|
cmd << if @last_timestamp.present?
|
45
47
|
"--since-time=#{rfc3339_timestamp(@last_timestamp)}"
|
@@ -50,22 +52,32 @@ module KubernetesDeploy
|
|
50
52
|
out.split("\n")
|
51
53
|
end
|
52
54
|
|
55
|
+
def kubectl
|
56
|
+
@kubectl ||= Kubectl.new(namespace: @namespace, context: @context, logger: @logger, log_failure_by_default: false)
|
57
|
+
end
|
58
|
+
|
53
59
|
def rfc3339_timestamp(time)
|
54
60
|
time.strftime("%FT%T.%N%:z")
|
55
61
|
end
|
56
62
|
|
57
|
-
def
|
58
|
-
|
59
|
-
|
63
|
+
def sort_and_deduplicate(logs)
|
64
|
+
parsed_lines = logs.map { |line| split_timestamped_line(line) }
|
65
|
+
sorted_lines = parsed_lines.sort do |(timestamp1, _msg1), (timestamp2, _msg2)|
|
66
|
+
if timestamp1.nil?
|
67
|
+
-1
|
68
|
+
elsif timestamp2.nil?
|
69
|
+
1
|
70
|
+
else
|
71
|
+
timestamp1 <=> timestamp2
|
72
|
+
end
|
73
|
+
end
|
60
74
|
|
61
|
-
|
62
|
-
|
63
|
-
next if
|
64
|
-
check_for_duplicate = false # logs are ordered, so once we've seen a new one, assume all subsequent logs are new
|
75
|
+
deduped = []
|
76
|
+
sorted_lines.each do |timestamp, msg|
|
77
|
+
next if likely_duplicate?(timestamp)
|
65
78
|
@last_timestamp = timestamp if timestamp
|
66
79
|
deduped << msg
|
67
80
|
end
|
68
|
-
|
69
81
|
deduped
|
70
82
|
end
|
71
83
|
|
@@ -38,11 +38,13 @@ require 'kubernetes-deploy/kubectl'
|
|
38
38
|
require 'kubernetes-deploy/kubeclient_builder'
|
39
39
|
require 'kubernetes-deploy/ejson_secret_provisioner'
|
40
40
|
require 'kubernetes-deploy/renderer'
|
41
|
-
require 'kubernetes-deploy/
|
41
|
+
require 'kubernetes-deploy/cluster_resource_discovery'
|
42
|
+
require 'kubernetes-deploy/template_discovery'
|
42
43
|
|
43
44
|
module KubernetesDeploy
|
44
45
|
class DeployTask
|
45
46
|
include KubeclientBuilder
|
47
|
+
extend KubernetesDeploy::StatsD::MeasureMethods
|
46
48
|
|
47
49
|
PREDEPLOY_SEQUENCE = %w(
|
48
50
|
ResourceQuota
|
@@ -52,6 +54,7 @@ module KubernetesDeploy
|
|
52
54
|
ConfigMap
|
53
55
|
PersistentVolumeClaim
|
54
56
|
ServiceAccount
|
57
|
+
Role
|
55
58
|
RoleBinding
|
56
59
|
Pod
|
57
60
|
)
|
@@ -85,7 +88,7 @@ module KubernetesDeploy
|
|
85
88
|
policy/v1beta1/PodDisruptionBudget
|
86
89
|
batch/v1beta1/CronJob
|
87
90
|
)
|
88
|
-
wl + cluster_resource_discoverer.crds
|
91
|
+
wl + cluster_resource_discoverer.crds.select(&:prunable?).map(&:group_version_kind)
|
89
92
|
end
|
90
93
|
|
91
94
|
def server_version
|
@@ -108,7 +111,6 @@ module KubernetesDeploy
|
|
108
111
|
logger: @logger,
|
109
112
|
bindings: bindings,
|
110
113
|
)
|
111
|
-
@sync_mediator = SyncMediator.new(namespace: @namespace, context: @context, logger: @logger)
|
112
114
|
end
|
113
115
|
|
114
116
|
def run(*args)
|
@@ -124,33 +126,16 @@ module KubernetesDeploy
|
|
124
126
|
|
125
127
|
@logger.phase_heading("Initializing deploy")
|
126
128
|
validate_configuration(allow_protected_ns: allow_protected_ns, prune: prune)
|
127
|
-
confirm_context_exists
|
128
|
-
confirm_namespace_exists
|
129
|
-
@namespace_tags |= tags_from_namespace_labels
|
130
129
|
resources = discover_resources
|
131
|
-
|
130
|
+
validate_resources(resources)
|
132
131
|
|
133
132
|
@logger.phase_heading("Checking initial resource statuses")
|
134
|
-
|
135
|
-
|
136
|
-
|
137
|
-
ejson = EjsonSecretProvisioner.new(
|
138
|
-
namespace: @namespace,
|
139
|
-
context: @context,
|
140
|
-
template_dir: @template_dir,
|
141
|
-
logger: @logger,
|
142
|
-
prune: prune,
|
143
|
-
)
|
144
|
-
if ejson.secret_changes_required?
|
145
|
-
@logger.phase_heading("Deploying kubernetes secrets from #{EjsonSecretProvisioner::EJSON_SECRETS_FILE}")
|
146
|
-
ejson.run
|
147
|
-
end
|
133
|
+
check_initial_status(resources)
|
134
|
+
create_ejson_secrets(prune)
|
148
135
|
|
149
136
|
if deploy_has_priority_resources?(resources)
|
150
137
|
@logger.phase_heading("Predeploying priority resources")
|
151
|
-
start_priority_resource = Time.now.utc
|
152
138
|
predeploy_priority_resources(resources)
|
153
|
-
::StatsD.measure('priority_resources.duration', StatsD.duration(start_priority_resource), tags: statsd_tags)
|
154
139
|
end
|
155
140
|
|
156
141
|
@logger.phase_heading("Deploying all resources")
|
@@ -159,9 +144,7 @@ module KubernetesDeploy
|
|
159
144
|
end
|
160
145
|
|
161
146
|
if verify_result
|
162
|
-
|
163
|
-
deploy_resources(resources, prune: prune, verify: true)
|
164
|
-
::StatsD.measure('normal_resources.duration', StatsD.duration(start_normal_resource), tags: statsd_tags)
|
147
|
+
deploy_all_resources(resources, prune: prune, verify: true)
|
165
148
|
failed_resources = resources.reject(&:deploy_succeeded?)
|
166
149
|
success = failed_resources.empty?
|
167
150
|
if !success && failed_resources.all?(&:deploy_timed_out?)
|
@@ -169,7 +152,7 @@ module KubernetesDeploy
|
|
169
152
|
end
|
170
153
|
raise FatalDeploymentError unless success
|
171
154
|
else
|
172
|
-
|
155
|
+
deploy_all_resources(resources, prune: prune, verify: false)
|
173
156
|
@logger.summary.add_action("deployed #{resources.length} #{'resource'.pluralize(resources.length)}")
|
174
157
|
warning = <<~MSG
|
175
158
|
Deploy result verification is disabled for this deploy.
|
@@ -177,32 +160,37 @@ module KubernetesDeploy
|
|
177
160
|
MSG
|
178
161
|
@logger.summary.add_paragraph(ColorizedString.new(warning).yellow)
|
179
162
|
end
|
180
|
-
|
163
|
+
StatsD.event("Deployment of #{@namespace} succeeded",
|
181
164
|
"Successfully deployed all #{@namespace} resources to #{@context}",
|
182
165
|
alert_type: "success", tags: statsd_tags << "status:success")
|
183
|
-
|
166
|
+
StatsD.distribution('all_resources.duration', StatsD.duration(start), tags: statsd_tags << "status:success")
|
184
167
|
@logger.print_summary(:success)
|
185
168
|
rescue DeploymentTimeoutError
|
186
169
|
@logger.print_summary(:timed_out)
|
187
|
-
|
170
|
+
StatsD.event("Deployment of #{@namespace} timed out",
|
188
171
|
"One or more #{@namespace} resources failed to deploy to #{@context} in time",
|
189
172
|
alert_type: "error", tags: statsd_tags << "status:timeout")
|
190
|
-
|
173
|
+
StatsD.distribution('all_resources.duration', StatsD.duration(start), tags: statsd_tags << "status:timeout")
|
191
174
|
raise
|
192
175
|
rescue FatalDeploymentError => error
|
193
176
|
@logger.summary.add_action(error.message) if error.message != error.class.to_s
|
194
177
|
@logger.print_summary(:failure)
|
195
|
-
|
178
|
+
StatsD.event("Deployment of #{@namespace} failed",
|
196
179
|
"One or more #{@namespace} resources failed to deploy to #{@context}",
|
197
180
|
alert_type: "error", tags: statsd_tags << "status:failed")
|
198
|
-
|
181
|
+
StatsD.distribution('all_resources.duration', StatsD.duration(start), tags: statsd_tags << "status:failed")
|
199
182
|
raise
|
200
183
|
end
|
201
184
|
|
202
185
|
private
|
203
186
|
|
204
187
|
def cluster_resource_discoverer
|
205
|
-
|
188
|
+
@cluster_resource_discoverer ||= ClusterResourceDiscovery.new(
|
189
|
+
namespace: @namespace,
|
190
|
+
context: @context,
|
191
|
+
logger: @logger,
|
192
|
+
namespace_tags: @namespace_tags
|
193
|
+
)
|
206
194
|
end
|
207
195
|
|
208
196
|
def deploy_has_priority_resources?(resources)
|
@@ -219,7 +207,7 @@ module KubernetesDeploy
|
|
219
207
|
fail_count = failed_resources.length
|
220
208
|
if fail_count > 0
|
221
209
|
KubernetesDeploy::Concurrency.split_across_threads(failed_resources) do |r|
|
222
|
-
r.sync_debug_info(
|
210
|
+
r.sync_debug_info(kubectl)
|
223
211
|
end
|
224
212
|
failed_resources.each { |r| @logger.summary.add_paragraph(r.debug_message) }
|
225
213
|
raise FatalDeploymentError, "Failed to deploy #{fail_count} priority #{'resource'.pluralize(fail_count)}"
|
@@ -227,8 +215,9 @@ module KubernetesDeploy
|
|
227
215
|
@logger.blank_line
|
228
216
|
end
|
229
217
|
end
|
218
|
+
measure_method(:predeploy_priority_resources, 'priority_resources.duration')
|
230
219
|
|
231
|
-
def
|
220
|
+
def validate_resources(resources)
|
232
221
|
KubernetesDeploy::Concurrency.split_across_threads(resources) { |r| r.validate_definition(kubectl) }
|
233
222
|
failed_resources = resources.select(&:validation_failed?)
|
234
223
|
return unless failed_resources.present?
|
@@ -239,14 +228,35 @@ module KubernetesDeploy
|
|
239
228
|
end
|
240
229
|
raise FatalDeploymentError, "Template validation failed"
|
241
230
|
end
|
231
|
+
measure_method(:validate_resources)
|
232
|
+
|
233
|
+
def check_initial_status(resources)
|
234
|
+
cache = ResourceCache.new(@namespace, @context, @logger)
|
235
|
+
KubernetesDeploy::Concurrency.split_across_threads(resources) { |r| r.sync(cache) }
|
236
|
+
resources.each { |r| @logger.info(r.pretty_status) }
|
237
|
+
end
|
238
|
+
measure_method(:check_initial_status, "initial_status.duration")
|
239
|
+
|
240
|
+
def create_ejson_secrets(prune)
|
241
|
+
ejson = EjsonSecretProvisioner.new(
|
242
|
+
namespace: @namespace,
|
243
|
+
context: @context,
|
244
|
+
template_dir: @template_dir,
|
245
|
+
logger: @logger,
|
246
|
+
prune: prune,
|
247
|
+
)
|
248
|
+
return unless ejson.secret_changes_required?
|
249
|
+
|
250
|
+
@logger.phase_heading("Deploying kubernetes secrets from #{EjsonSecretProvisioner::EJSON_SECRETS_FILE}")
|
251
|
+
ejson.run
|
252
|
+
end
|
253
|
+
measure_method(:create_ejson_secrets)
|
242
254
|
|
243
255
|
def discover_resources
|
244
256
|
resources = []
|
245
257
|
@logger.info("Discovering templates:")
|
246
258
|
|
247
|
-
|
248
|
-
next unless filename.end_with?(".yml.erb", ".yml", ".yaml", ".yaml.erb")
|
249
|
-
|
259
|
+
TemplateDiscovery.new(@template_dir).templates.each do |filename|
|
250
260
|
split_templates(filename) do |r_def|
|
251
261
|
r = KubernetesResource.build(namespace: @namespace, context: @context, logger: @logger,
|
252
262
|
definition: r_def, statsd_tags: @namespace_tags)
|
@@ -260,11 +270,12 @@ module KubernetesDeploy
|
|
260
270
|
end
|
261
271
|
resources
|
262
272
|
end
|
273
|
+
measure_method(:discover_resources)
|
263
274
|
|
264
275
|
def split_templates(filename)
|
265
276
|
file_content = File.read(File.join(@template_dir, filename))
|
266
277
|
rendered_content = @renderer.render_template(filename, file_content)
|
267
|
-
YAML.load_stream(rendered_content) do |doc|
|
278
|
+
YAML.load_stream(rendered_content, "<rendered> #{filename}") do |doc|
|
268
279
|
next if doc.blank?
|
269
280
|
unless doc.is_a?(Hash)
|
270
281
|
raise InvalidTemplateError.new("Template is not a valid Kubernetes manifest",
|
@@ -283,15 +294,11 @@ module KubernetesDeploy
|
|
283
294
|
|
284
295
|
def record_invalid_template(err:, filename:, content:)
|
285
296
|
debug_msg = ColorizedString.new("Invalid template: #{filename}\n").red
|
286
|
-
debug_msg += "> Error message:\n#{indent_four(err)}"
|
287
|
-
debug_msg += "\n> Template content:\n#{indent_four(content)}"
|
297
|
+
debug_msg += "> Error message:\n#{FormattedLogger.indent_four(err)}"
|
298
|
+
debug_msg += "\n> Template content:\n#{FormattedLogger.indent_four(content)}"
|
288
299
|
@logger.summary.add_paragraph(debug_msg)
|
289
300
|
end
|
290
301
|
|
291
|
-
def indent_four(str)
|
292
|
-
" " + str.gsub("\n", "\n ")
|
293
|
-
end
|
294
|
-
|
295
302
|
def validate_configuration(allow_protected_ns:, prune:)
|
296
303
|
errors = []
|
297
304
|
if ENV["KUBECONFIG"].blank?
|
@@ -340,8 +347,12 @@ module KubernetesDeploy
|
|
340
347
|
raise FatalDeploymentError, "Configuration invalid"
|
341
348
|
end
|
342
349
|
|
350
|
+
confirm_context_exists
|
351
|
+
confirm_namespace_exists
|
352
|
+
@namespace_tags |= tags_from_namespace_labels
|
343
353
|
@logger.info("All required parameters and files are present")
|
344
354
|
end
|
355
|
+
measure_method(:validate_configuration)
|
345
356
|
|
346
357
|
def deploy_resources(resources, prune: false, verify:, record_summary: true)
|
347
358
|
return if resources.empty?
|
@@ -388,12 +399,17 @@ module KubernetesDeploy
|
|
388
399
|
apply_all(applyables, prune)
|
389
400
|
|
390
401
|
if verify
|
391
|
-
watcher = ResourceWatcher.new(resources: resources,
|
392
|
-
|
402
|
+
watcher = ResourceWatcher.new(resources: resources, logger: @logger, deploy_started_at: deploy_started_at,
|
403
|
+
timeout: @max_watch_seconds, namespace: @namespace, context: @context, sha: @current_sha)
|
393
404
|
watcher.run(record_summary: record_summary)
|
394
405
|
end
|
395
406
|
end
|
396
407
|
|
408
|
+
def deploy_all_resources(resources, prune: false, verify:, record_summary: true)
|
409
|
+
deploy_resources(resources, prune: prune, verify: verify, record_summary: record_summary)
|
410
|
+
end
|
411
|
+
measure_method(:deploy_all_resources, 'normal_resources.duration')
|
412
|
+
|
397
413
|
def apply_all(resources, prune)
|
398
414
|
return unless resources.present?
|
399
415
|
command = %w(apply)
|
@@ -421,6 +437,7 @@ module KubernetesDeploy
|
|
421
437
|
end
|
422
438
|
end
|
423
439
|
end
|
440
|
+
measure_method(:apply_all)
|
424
441
|
|
425
442
|
def log_pruning(kubectl_output)
|
426
443
|
pruned = kubectl_output.scan(/^(.*) pruned$/)
|
@@ -446,8 +463,9 @@ module KubernetesDeploy
|
|
446
463
|
end
|
447
464
|
|
448
465
|
if unidentified_errors.present?
|
449
|
-
|
450
|
-
|
466
|
+
heading = ColorizedString.new('Unidentified error(s):').red
|
467
|
+
msg = FormattedLogger.indent_four(unidentified_errors.join)
|
468
|
+
@logger.summary.add_paragraph("#{heading}\n#{msg}")
|
451
469
|
end
|
452
470
|
end
|
453
471
|
|