kumo_dockercloud 3.0.0 → 3.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # KumoDockerCloud [![Build status](https://badge.buildkite.com/e9ebd06f4732bbb2a914228ac8816a2bbbeaf8bf0444ea00b4.svg)](https://buildkite.com/redbubble/kumo-docker-cloud)
2
2
 
3
- This is the Redbubble wrapper around creating environments in docker cloud. It is built into our rbdevltools container and can then be used during `apply-env` and `deploy` tasks.
3
+ This is the Redbubble wrapper around creating environments in docker cloud. It is built into our rbdevltools container and can then be used during `apply-env` and `deploy` tasks.
4
4
 
5
5
  ## Installation
6
6
 
@@ -8,26 +8,97 @@ This is installed into the rbdevtools container.
8
8
 
9
9
  ## Usage
10
10
 
11
- Apply env example
11
+ ### Apply env example
12
+
13
+ By default apply-env will check that all services within the stack are running:
14
+
12
15
  ```ruby
13
16
  KumoDockerCloud::Environment.new(
14
17
  name: environment_name,
15
18
  env_vars: env_vars,
16
19
  app_name: 'your-app-here',
17
20
  config_path: File.join('/app', 'config'),
18
- stack_template_path: File.join('/app', 'docker-cloud, 'stack.yml.erb')
19
- ).apply
21
+ stack_template_path: File.join('/app', 'docker-cloud', 'stack.yml.erb')
22
+ ).apply()
23
+ ```
24
+
25
+ This is not desirable for stacks with services that are not permanently running.
26
+
27
+ #### StackChecker
28
+
29
+ For these situations you can pass in a stack checker object which maps services to a custom set of checks as follows:
30
+
31
+ ```ruby
32
+ require 'kumo_dockercloud'
33
+
34
+ custom_checks = {
35
+ 'transitory_service' => [
36
+ lambda { |container| container.state == 'Stopped' },
37
+ lambda { |container| container.exit_code == 0 }
38
+ ]
39
+ }
40
+
41
+ stack_checker = KumoDockerCloud::StackChecker.new(custom_checks)
42
+
43
+ KumoDockerCloud::Environment.new(
44
+ name: environment_name,
45
+ env_vars: env_vars,
46
+ app_name: 'your-app-here',
47
+ config_path: File.join('/app', 'env', 'config'),
48
+ stack_template_path: File.join('/app', 'env', 'docker-cloud', 'stack.yml.erb'),
49
+ timeout: 600
50
+ ).apply(stack_checker)
51
+ ```
52
+
53
+ The `StackChecker` will execute default checks for all services in your stack which are not listed
54
+ in the custom checks. The default check is that the service is running. You can override this by
55
+ passing in your own set of default checks as follows:
56
+
57
+ ```ruby
58
+ default_checks = [
59
+ lambda { |container| container.state == 'Awesome' },
60
+ lambda { |container| container.logs.contain? 'Ready' }
61
+ ]
62
+
63
+ stack_checker = KumoDockerCloud::StackChecker.new(custom_checks, default_checks, 120)
20
64
  ```
21
65
 
22
- Deploy example
66
+ The third parameter in the line above is the timeout, by default is 300 seconds.
67
+
68
+ ### Deploy example
69
+
70
+ The deploy method will deploy your docker image to a service within an existing stack
71
+ (i.e. you've created the `Environment` as above):
72
+
23
73
  ```ruby
24
74
  begin
25
- KumoDockerCloud::Stack.new(app_name, env_name).deploy(version)
75
+ KumoDockerCloud::Stack.new(app_name, env_name).deploy(service_name, version)
26
76
  rescue KumoDockerCloud::Deployment::DeploymentError, TimeoutError
27
77
  exit 1
28
78
  end
29
79
  ```
30
80
 
81
+ The `version` is your Docker Hub tag on an image name that matches what is in your
82
+ Docker Cloud stackfile.
83
+
84
+ #### ServiceChecker
85
+
86
+ By default deploy will not run any checks on the services that you deploy. You can override
87
+ this by passing in a service checker:
88
+
89
+ ```ruby
90
+ custom_service_checks = [
91
+ lambda { |container| container.state == 'Stopped' },
92
+ lambda { |container| container.exit_code == 0 }
93
+ ]
94
+
95
+ service_checker = KumoDockerCloud::ServiceChecker.new(custom_service_checks, 120)
96
+
97
+ KumoDockerCloud::Stack.new(app_name, env_name).deploy(service_name, version, service_checker)
98
+
99
+ ```
100
+ As for the `StackChecker`, the third parameter is a timeout which defaults to 300 seconds.
101
+
31
102
  ## Testing changes
32
103
 
33
104
  Changes to the gem can be manually tested end to end in a project that uses the gem (i.e. http-wala).
@@ -26,4 +26,5 @@ Gem::Specification.new do |spec|
26
26
  spec.add_development_dependency 'rake', '~> 10.0'
27
27
  spec.add_development_dependency 'rspec', '~> 3.4'
28
28
  spec.add_development_dependency 'webmock', '~> 1.22'
29
+ spec.add_development_dependency 'rubocop', '~> 0.40'
29
30
  end
@@ -1,6 +1,7 @@
1
1
  require 'kumo_dockercloud/environment'
2
2
  require 'kumo_dockercloud/deployment'
3
3
  require 'kumo_dockercloud/stack'
4
+ require 'kumo_dockercloud/stack_checker'
4
5
  require 'kumo_dockercloud/service'
5
6
  require 'kumo_dockercloud/service_checker'
6
7
  require 'kumo_dockercloud/errors'
@@ -8,8 +8,8 @@ require_relative 'docker_cloud_api'
8
8
  require_relative 'environment_config'
9
9
  require_relative 'stack_file'
10
10
  require_relative 'state_validator'
11
+ require_relative 'stack_checker'
11
12
 
12
- #TODO refactor this to use the new checker inside Service
13
13
  module KumoDockerCloud
14
14
  class Environment
15
15
  extend ::Forwardable
@@ -22,13 +22,13 @@ module KumoDockerCloud
22
22
  @timeout = params.fetch(:timeout, 120)
23
23
  @confirmation_timeout = params.fetch(:confirmation_timeout, 30)
24
24
 
25
- app_name = params.fetch(:app_name)
26
- @config = EnvironmentConfig.new(app_name: app_name, env_name: @env_name, config_path: params.fetch(:config_path))
25
+ @app_name = params.fetch(:app_name)
26
+ @config = EnvironmentConfig.new(app_name: @app_name, env_name: @env_name, config_path: params.fetch(:config_path))
27
27
  end
28
28
 
29
- def apply
29
+ def apply(stack_checker = StackChecker.new)
30
30
  if @config.image_tag == 'latest'
31
- puts 'WARNING: Deploying latest. The deployed container version may arbitrarily change'
31
+ ConsoleJockey.write_line 'WARNING: Deploying latest. The deployed container version may arbitrarily change'
32
32
  end
33
33
 
34
34
  stack_file = write_stack_config_file(configure_stack(stack_template))
@@ -37,7 +37,13 @@ module KumoDockerCloud
37
37
 
38
38
  run_command("docker-cloud stack redeploy #{stack_name}")
39
39
 
40
- wait_for_running(@timeout)
40
+ stack = Stack.new(@app_name, @env_name)
41
+
42
+ begin
43
+ stack_checker.verify(stack)
44
+ rescue StackCheckError
45
+ raise EnvironmentApplyError.new("The stack is not in the expected state.")
46
+ end
41
47
  end
42
48
 
43
49
  def destroy
@@ -52,33 +58,6 @@ module KumoDockerCloud
52
58
  StackFile.create_from_template(stack_template, @config, @env_vars)
53
59
  end
54
60
 
55
- def wait_for_running(timeout)
56
- StateValidator.new(stack_state_provider).wait_for_state('Redeploying', timeout)
57
- StateValidator.new(stack_state_provider).wait_for_state(expected_state, timeout)
58
- StateValidator.new(service_state_provider).wait_for_state('Running', timeout)
59
- end
60
-
61
- def expected_state
62
- env_name == 'production' ? 'Partly running' : 'Running'
63
- end
64
-
65
- def stack_state_provider
66
- docker_cloud_api = DockerCloudApi.new
67
- lambda {
68
- stack = docker_cloud_api.stack_by_name(stack_name)
69
- { name: stack.name, state: stack.state }
70
- }
71
- end
72
-
73
- def service_state_provider
74
- docker_cloud_api = DockerCloudApi.new
75
- lambda {
76
- services = docker_cloud_api.services_by_stack_name(stack_name)
77
- services.select! { |service| service.name != 'geckoboardwidget' }
78
- { name: 'services', state: services.map { |s| s.state }.uniq.join }
79
- }
80
- end
81
-
82
61
  def run_command(cmd)
83
62
  puts "Executing -> #{cmd}"
84
63
  puts `#{cmd}`
@@ -1,4 +1,7 @@
1
1
  module KumoDockerCloud
2
2
  class Error < RuntimeError; end
3
3
  class ServiceDeployError < RuntimeError; end
4
+ class EnvironmentApplyError < RuntimeError; end
5
+ class StackCheckError < RuntimeError; end
6
+ class InvalidStackError < RuntimeError; end
4
7
  end
@@ -18,6 +18,11 @@ module KumoDockerCloud
18
18
  checker.verify(service)
19
19
  end
20
20
 
21
+ def services
22
+ services = docker_cloud_api.services_by_stack_name(stack_name)
23
+ services.map { |service| Service.new(stack_name, service.name) }
24
+ end
25
+
21
26
  def deploy_blue_green(options)
22
27
  service_names = options[:service_names]
23
28
  version = options[:version]
@@ -48,5 +53,9 @@ module KumoDockerCloud
48
53
  raise KumoDockerCloud::Error.new("#{param_name} cannot be nil") unless param_value
49
54
  raise KumoDockerCloud::Error.new("#{param_name} cannot be empty") if param_value.empty?
50
55
  end
56
+
57
+ def docker_cloud_api
58
+ @docker_cloud_api ||= DockerCloudApi.new
59
+ end
51
60
  end
52
61
  end
@@ -0,0 +1,34 @@
1
+ module KumoDockerCloud
2
+ class StackChecker
3
+ def initialize(specific_checks = {}, common_check = nil, timeout = 300)
4
+ @checks = specific_checks
5
+ @default_check = common_check
6
+ @timeout = timeout
7
+ end
8
+
9
+ def verify(stack)
10
+ raise InvalidStackError.new("The stack being verified is not a valid KumoDockerCloud::Stack.") unless stack.instance_of? KumoDockerCloud::Stack
11
+
12
+ services = stack.services
13
+
14
+ default_checks = services.reduce({}) { |result, service| result.merge(service.name => default_check) }
15
+ service_checks = default_checks.merge(@checks)
16
+
17
+ begin
18
+ service_checker_threads = services.map do |service|
19
+ Thread.new { ServiceChecker.new(service_checks[service.name], @timeout).verify(service) }
20
+ end
21
+ service_checker_threads.each(&:join)
22
+ true
23
+ rescue ServiceDeployError
24
+ raise StackCheckError.new("The stack is not in the expected state.")
25
+ end
26
+ end
27
+
28
+ private
29
+
30
+ def default_check
31
+ @default_check ||= [lambda { |container| container.state == 'Running' }]
32
+ end
33
+ end
34
+ end
@@ -1,3 +1,3 @@
1
1
  module KumoDockerCloud
2
- VERSION = '3.0.0'
2
+ VERSION = '3.1.0'
3
3
  end
@@ -1,11 +1,12 @@
1
1
  describe KumoDockerCloud::Environment do
2
2
  let(:env_vars) { {app_name => {'KEY' => 'VALUE'}} }
3
3
  let(:app_name) { 'application-stack-name' }
4
- let(:config) { KumoDockerCloud::EnvironmentConfig.new(app_name: app_name, env_name: 'test', config_path: 'a path') }
4
+ let(:env_name) { 'test' }
5
+ let(:config) { KumoDockerCloud::EnvironmentConfig.new(app_name: app_name, env_name: env_name, config_path: 'a path') }
5
6
 
6
7
  let(:stack_file) {
7
8
  {
8
- 'application-stack-name': {
9
+ 'application-stack-name' => {
9
10
  image: 'a-thing',
10
11
  environment: {
11
12
  TEST_ENV: 'FAKE',
@@ -19,7 +20,7 @@ describe KumoDockerCloud::Environment do
19
20
  let(:confirmation_timeout) { 0.5 }
20
21
  let(:stack_template_path) { File.join(__dir__, '../fixtures/stack.yml.erb') }
21
22
 
22
- let(:params) { {name: 'test', env_vars: env_vars, app_name: app_name, config_path: 'a path', stack_template_path: stack_template_path, confirmation_timeout: confirmation_timeout} }
23
+ let(:params) { {name: env_name, env_vars: env_vars, app_name: app_name, config_path: 'a path', stack_template_path: stack_template_path, confirmation_timeout: confirmation_timeout} }
23
24
 
24
25
  subject(:env) { described_class.new(params) }
25
26
 
@@ -30,13 +31,15 @@ describe KumoDockerCloud::Environment do
30
31
 
31
32
  describe "#apply" do
32
33
  subject { env.apply }
34
+ let(:stack) { {"#{full_stack_name}" => 'stack stuff'} }
35
+ let(:stack_checker) { instance_double(KumoDockerCloud::StackChecker, :stack_checker, verify: true) }
33
36
  before do
34
37
  allow(config).to receive(:image_tag).and_return('latest')
35
38
  allow(env).to receive(:evaluate_command).and_return app_name
36
39
  allow(env).to receive(:run_command)
37
- docker_cloud_api = double("DockerCloudApi", stack_by_name: {"#{full_stack_name}": 'stack stuff'})
38
- allow(KumoDockerCloud::DockerCloudApi).to receive(:new).and_return docker_cloud_api
39
- allow_any_instance_of(KumoDockerCloud::StateValidator).to receive(:wait_for_state)
40
+ allow(KumoDockerCloud::StackChecker).to receive(:new).and_return(stack_checker)
41
+ allow(KumoDockerCloud::Stack).to receive(:new).with(app_name, env_name).and_return(stack)
42
+
40
43
  end
41
44
 
42
45
  it "writes a stack file" do
@@ -57,37 +60,58 @@ describe KumoDockerCloud::Environment do
57
60
  subject
58
61
  end
59
62
 
60
- describe "waiting for running" do
61
- let(:state_validator) { double(KumoDockerCloud::StateValidator, wait_for_state: nil) }
63
+ it 'uses the StackFile class' do
64
+ expect(KumoDockerCloud::StackFile).to receive(:create_from_template).with(File.read(stack_template_path), config, env_vars)
65
+
66
+ subject
67
+ end
68
+
69
+ it 'passs a KumoDockerCloud::Stack to the stack_checker' do
70
+ expect(stack_checker).to receive(:verify).with(stack).and_return(true)
71
+
72
+ subject
73
+ end
74
+
75
+ context 'with specific stack checker passed in' do
76
+ let(:stack_checker_passed_in) { instance_double(KumoDockerCloud::StackChecker, :passed_in_stack_checker, verify: true) }
77
+ subject { env.apply(stack_checker_passed_in) }
62
78
 
63
- before do
64
- allow(KumoDockerCloud::StateValidator).to receive(:new).exactly(3).times.and_return(state_validator)
79
+ it 'returns true when status check is successful' do
80
+ expect(subject).to be true
65
81
  end
66
82
 
67
- it "makes sure it waits until it's running" do
68
- expect(state_validator).to receive(:wait_for_state).with(anything, 120).exactly(3).times
69
- subject
83
+ it 'raise and stack_apply_exception when status check is not successful' do
84
+ expect(stack_checker_passed_in).to receive(:verify).with(stack).and_raise(KumoDockerCloud::StackCheckError)
85
+ expect{subject}.to raise_error(KumoDockerCloud::EnvironmentApplyError, "The stack is not in the expected state." )
70
86
  end
71
87
 
72
- context "setting a different timeout value" do
73
- let(:params) { {name: 'test', env_vars: env_vars, app_name: app_name, config_path: 'a path', stack_template_path: stack_template_path, timeout: 240} }
88
+ it 'uses the user passed in stack checker' do
89
+ expect(stack_checker_passed_in).to receive(:verify).with(stack).and_return(true)
90
+ subject
91
+ end
92
+ end
74
93
 
75
- it "sends the timeout value to the StateValidator" do
76
- expect(state_validator).to receive(:wait_for_state).with(anything, 240).exactly(3).times
77
- subject
78
- end
94
+ context 'without specific stack checker passed in' do
95
+ subject { env.apply }
96
+ it 'returns true when status check is successful' do
97
+ expect(subject).to be true
79
98
  end
80
99
 
81
- end
100
+ it 'raises a stack_apply_exception when status check is not successful' do
101
+ expect(stack_checker).to receive(:verify).with(stack).and_raise(KumoDockerCloud::StackCheckError)
102
+ expect{subject}.to raise_error(KumoDockerCloud::EnvironmentApplyError, "The stack is not in the expected state." )
103
+ end
82
104
 
83
- it 'uses the StackFile class' do
84
- expect(KumoDockerCloud::StackFile).to receive(:create_from_template).with(File.read(stack_template_path), config, env_vars)
105
+ it 'does create a new instance of stack checker' do
106
+ expect(KumoDockerCloud::StackChecker).to receive(:new).and_return(stack_checker)
107
+ subject
108
+ end
85
109
 
86
- subject
87
110
  end
88
111
  end
89
112
 
90
113
 
114
+
91
115
  describe "#destroy" do
92
116
  subject { env.destroy }
93
117
  before do
@@ -56,7 +56,7 @@ describe KumoDockerCloud::ServiceChecker do
56
56
 
57
57
  context "second time is the charm" do
58
58
  let(:mutating_state) { [] }
59
- let(:mutating_check) { lambda { |container| mutating_state << 1; mutating_state.size > 1 } }
59
+ let(:mutating_check) { lambda { |_container| mutating_state << 1; mutating_state.size > 1 } }
60
60
  let(:checks) { [mutating_check] }
61
61
 
62
62
  it "runs without incident" do
@@ -0,0 +1,125 @@
1
+ describe KumoDockerCloud::StackChecker do
2
+
3
+ let(:stack) { instance_double(KumoDockerCloud::Stack, :stack, stack_name: 'stack' )}
4
+ let(:service) { double(:service, name: 'redbubble', state: 'Running') }
5
+ let(:services) { [service]}
6
+ let(:failed_services) { double(:service_api, name: 'redbubble', state: 'Stopped')}
7
+ let(:service_checker) { instance_double(KumoDockerCloud::ServiceChecker, verify: nil)}
8
+ let(:default_service_check) { [double(:default_check)] }
9
+ let(:specific_service_check) { { "redbubble" => [double(:specific_check)] } }
10
+
11
+ subject { described_class.new.verify(stack) }
12
+
13
+ before do
14
+ allow(stack).to receive(:services).and_return(services)
15
+ allow(stack).to receive(:instance_of?).with(KumoDockerCloud::Stack).and_return(true)
16
+ end
17
+
18
+ describe '#verify' do
19
+
20
+ before do
21
+ allow(KumoDockerCloud::ServiceChecker).to receive(:new).and_return(service_checker)
22
+ end
23
+
24
+ context 'invalid stack passed in' do
25
+ let(:invalid_stack) { double(:stack) }
26
+ subject{ described_class.new.verify(invalid_stack) }
27
+
28
+ it 'raises invalid stack exception when received a non KumoDockerCloud::Stack' do
29
+ expect { subject }.to raise_error( KumoDockerCloud::InvalidStackError, "The stack being verified is not a valid KumoDockerCloud::Stack." )
30
+ end
31
+ end
32
+
33
+
34
+ context 'single service' do
35
+ context 'without passing services checks' do
36
+ before { allow_any_instance_of(KumoDockerCloud::StackChecker).to receive(:default_check).and_return(default_service_check) }
37
+ it 'returns true when verify successful' do
38
+ expect(subject).to be true
39
+ end
40
+
41
+ it 'uses default check' do
42
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with(default_service_check, 300)
43
+ subject
44
+ end
45
+
46
+
47
+ it 'uses user specify timeout value' do
48
+ user_timeout = 1
49
+ allow_any_instance_of(KumoDockerCloud::StackChecker).to receive(:default_check).and_return(default_service_check)
50
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with(default_service_check,user_timeout)
51
+ described_class.new({}, nil, user_timeout).verify(stack)
52
+ end
53
+
54
+ it 'raise StackCheckError when verify unsuccessful' do
55
+ allow(service_checker).to receive(:verify).with(service).and_raise KumoDockerCloud::ServiceDeployError
56
+ expect {subject}.to raise_error(KumoDockerCloud::StackCheckError, "The stack is not in the expected state." )
57
+ end
58
+ end
59
+
60
+ context 'with user specified common checks' do
61
+ it 'uses user specified common check as the default' do
62
+ common_checks = [double(:common_check)]
63
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with(common_checks, 300)
64
+ described_class.new({}, common_checks).verify(stack)
65
+ end
66
+ end
67
+
68
+ context 'with specific services checks passing in' do
69
+ subject { described_class.new(specific_service_check).verify(stack) }
70
+
71
+ it 'returns true when verify successful' do
72
+ expect(subject).to be true
73
+ end
74
+
75
+ it 'raise StackCheckError when verify unsuccessful' do
76
+ allow(service_checker).to receive(:verify).with(service).and_raise KumoDockerCloud::ServiceDeployError
77
+ expect {subject}.to raise_error(KumoDockerCloud::StackCheckError, "The stack is not in the expected state." )
78
+ end
79
+
80
+ it 'uses specific check' do
81
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with(specific_service_check[service.name], 300)
82
+ subject
83
+ end
84
+ end
85
+ end
86
+
87
+ context 'multiple services' do
88
+ let(:services) {[double(:redbubble_service, name: 'redbubble', state: 'Running'), double(:nginx_service, name: 'nginx', state: 'Running')]}
89
+ let(:redbubble_check) { double(:redbubble_check) }
90
+ let(:nginx_check) { double(:nginx_check) }
91
+ let(:specific_service_check) { { "redbubble" => [redbubble_check], "nginx" => [nginx_check] } }
92
+ let(:single_override_service_check) { { "redbubble" => [redbubble_check] } }
93
+
94
+ subject { described_class.new(specific_service_check).verify(stack) }
95
+
96
+ context 'default checks' do
97
+ before { allow_any_instance_of(KumoDockerCloud::StackChecker).to receive(:default_check).and_return(default_service_check) }
98
+
99
+ it 'does the correct checking for each service' do
100
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with([redbubble_check], 300)
101
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with([nginx_check], 300)
102
+ subject
103
+ end
104
+
105
+ it 'uses default checking if one of the service check is not provided' do
106
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with([redbubble_check], 300)
107
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with(default_service_check, 300)
108
+ described_class.new(single_override_service_check).verify(stack)
109
+ end
110
+ end
111
+
112
+ context 'passing common checks' do
113
+ let(:specific_service_check) { { "redbubble" => [redbubble_check] } }
114
+ let(:common_service_checks) { [double(:common_service_check)] }
115
+ subject { described_class.new(specific_service_check, common_service_checks).verify(stack) }
116
+
117
+ it 'overrides the default check with the common checks for non-specified services' do
118
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with([redbubble_check], 300)
119
+ expect(KumoDockerCloud::ServiceChecker).to receive(:new).with(common_service_checks, 300)
120
+ subject
121
+ end
122
+ end
123
+ end
124
+ end
125
+ end