broadside 1.4.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 1de9e1e538f010d28635f3fa1729f4ea8325583f
4
- data.tar.gz: c7c2c7d1a3dc580a76a9e4793509ad1eff6f9e68
3
+ metadata.gz: fc1d69217accd3c37cde3f4a27955021c62b7b75
4
+ data.tar.gz: 4806175370a4a5254a3da35d33bcd6cad2b7e504
5
5
  SHA512:
6
- metadata.gz: 73310a4a5dbd7bd54cf6a9dfbd93933acd91ba8b050c947b1726d89f45ac19f4155f15e82cdd556416168919fd15336dbd6946b99ac24caaf1d82d69c49bf8f9
7
- data.tar.gz: 3ab6ffad553c01486c59ce3e27fa7446ace9e9ec9a5124aad6c168049be839fe5e0ab0994e42a7841a4749aeb1e91516d3b07f05c84036023f5a0b07aeaf0549
6
+ metadata.gz: b123680b578397bc830e20099d47c364071ad2810f71594433fea4eef52280e23baa144c186cc166f9e13eff01799a3e0bc7efbaf39ccef9545efa65747c810f
7
+ data.tar.gz: fa3b8ee635ca551c74ee41e2f0f57198b8a97469f1c1298c2a2af827fdb4adaa06085f274faf8e9a82037930cacd97ef00b97223c9c8938f9953dfc83aad6ac6
data/CHANGELOG.md CHANGED
@@ -1,5 +1,12 @@
1
+ # 2.0.0
2
+ - **BREAKING CHANGE** `rake db:migrate` is no longer the default `predeploy_command`
3
+ - `Target` is a first class object
4
+ - `Deploy` is composed of a `Target` plus command line options
5
+ - There is no more `base` configuration - the main `Configuration` object holds all the `base` config. You can still call `Broadside.config.base` though you will get a deprecation warning.
6
+ - There is no more `deploy` configuration - most of that is handled in the main `Configuration` object and in `targets=`. You can still call `Broadside.config.deploy` though you will get a deprecation warning.
7
+
1
8
  # 1.4.0
2
- = [#42](https://github.com/lumoslabs/broadside/pull/42/files): Update the task definition when running bootstrap
9
+ - [#42](https://github.com/lumoslabs/broadside/pull/42/files): Update the task definition when running bootstrap
3
10
 
4
11
  # 1.3.0
5
12
  - [#41](https://github.com/lumoslabs/broadside/pull/41/files): Introduce the concept of bootstrap commands, which are designed to be run when setting up a new server or environment.
data/README.md CHANGED
@@ -11,9 +11,9 @@ Broadside offers a simple command-line interface to perform deployments on ECS.
11
11
 
12
12
  ```ruby
13
13
  Broadside.configure do |config|
14
- config.base.application = 'hello_world'
15
- config.base.docker_image = 'lumoslabs/hello_world'
16
- config.deploy.type = 'ecs'
14
+ config.application = 'hello_world'
15
+ config.docker_image = 'lumoslabs/hello_world'
16
+ config.type = 'ecs'
17
17
  config.ecs.cluster = 'micro-cluster'
18
18
  config.deploy.targets = {
19
19
  production_web: {
@@ -21,7 +21,7 @@ Broadside.configure do |config|
21
21
  command: ['bundle', 'exec', 'unicorn', '-c', 'config/unicorn.conf.rb'],
22
22
  env_file: '../.env.production'
23
23
  predeploy_commands: [
24
- ['bundle', 'exec', 'rake', 'db:migrate'],
24
+ Broadside::Predeploy::RAKE_DB_MIGRATE, # RAKE_DB_MIGRATE is just a constant for your convenience
25
25
  ['bundle', 'exec', 'rake', 'data:migrate']
26
26
  ]
27
27
  },
@@ -31,6 +31,7 @@ Broadside.configure do |config|
31
31
  env_file: '../.env.production'
32
32
  },
33
33
  staging_web: {
34
+ cluster: 'staging-cluster', # Overrides config.ecs.cluster
34
35
  scale: 1,
35
36
  command: ['bundle', 'exec', 'puma'],
36
37
  env_file: '../.env.staging'
@@ -68,13 +69,16 @@ end
68
69
  ```
69
70
 
70
71
  From here, developers can use broadside's command-line interface to initiate a basic deployment:
71
- ```
72
+
73
+ ```bash
72
74
  broadside deploy short --target production_web --tag $GIT_SHA
73
75
  ```
74
76
  or run
75
- ```
77
+
78
+ ```bash
76
79
  broadside deploy full --target production_web --tag $GIT_SHA
77
80
  ```
81
+
78
82
  which will run the listed `predeploy_commands` listed in the config above prior to the deployment.
79
83
 
80
84
  In the case of an error or timeout during a deploy, broadside will automatically rollback to the latest stable version. You can perform manual rollbacks as well through the command-line.
@@ -84,12 +88,12 @@ See the complete command-line reference in the wiki.
84
88
 
85
89
  ## Setup
86
90
  First, install broadside by adding it to your application gemfile:
87
- ```
91
+ ```ruby
88
92
  gem 'broadside'
89
93
  ```
90
94
 
91
95
  Then run
92
- ```
96
+ ```bash
93
97
  bundle install
94
98
  bundle binstubs broadside
95
99
  ```
@@ -97,7 +101,7 @@ bundle binstubs broadside
97
101
  It's recommended that you specify broadside as a development gem so it doesn't inflate your production image.
98
102
 
99
103
  You can now run the executable in your app directory:
100
- ```
104
+ ```bash
101
105
  bin/broadside --help
102
106
  ```
103
107
 
data/bin/broadside CHANGED
@@ -26,8 +26,10 @@ def add_shared_deploy_configs(subcmd)
26
26
  subcmd.flag [:t, :target], type: Symbol
27
27
 
28
28
  subcmd.action do |global_options, options, args|
29
- _DeployObj = Kernel.const_get("Broadside::#{Broadside.config.deploy.type.capitalize}Deploy")
30
- _DeployObj.new(options).public_send(subcmd.name)
29
+ _DeployObj = Kernel.const_get("Broadside::#{Broadside.config.type.capitalize}Deploy")
30
+ _target = Broadside.config.targets.select { |t| t.name == options[:target] }.first
31
+ raise "Bad target: #{options[:target]}" unless _target
32
+ _DeployObj.new(_target, options).public_send(subcmd.name)
31
33
  end
32
34
  end
33
35
 
@@ -44,9 +46,9 @@ end
44
46
 
45
47
  desc 'Bootstrap your service and task definition from the configured definition.'
46
48
  command :bootstrap do |b|
47
- subcmd.desc 'Docker tag for application container'
48
- subcmd.arg_name 'TAG'
49
- subcmd.flag [:tag]
49
+ b.desc 'Docker tag for application container'
50
+ b.arg_name 'TAG'
51
+ b.flag [:tag]
50
52
 
51
53
  add_shared_deploy_configs(b)
52
54
  end
@@ -144,7 +146,8 @@ command :deploy do |d|
144
146
  end
145
147
 
146
148
  def call_hook(type, command)
147
- hook = Broadside.config.base.send(type)
149
+ hook = Broadside.config.send(type)
150
+
148
151
  if hook.is_a?(Proc)
149
152
  hook_args =
150
153
  if command.parent.is_a?(GLI::Command)
data/broadside.gemspec CHANGED
@@ -6,7 +6,7 @@ require 'broadside/version'
6
6
  Gem::Specification.new do |spec|
7
7
  spec.name = 'broadside'
8
8
  spec.version = Broadside::VERSION
9
- spec.authors = ['Matthew Leung']
9
+ spec.authors = ['Matthew Leung', 'Lumos Labs, Inc.']
10
10
  spec.email = ['leung.mattp@gmail.com']
11
11
 
12
12
  spec.summary = 'A command-line tool for EC2 Container Service deployment.'
@@ -23,7 +23,7 @@ Gem::Specification.new do |spec|
23
23
  spec.add_dependency 'gli', '~> 2.13'
24
24
  spec.add_dependency 'rainbow', '~> 2.1'
25
25
 
26
- spec.add_development_dependency 'rspec'
26
+ spec.add_development_dependency 'rspec', '~> 3.4.0'
27
27
  spec.add_development_dependency 'bundler', '~> 1.9'
28
28
  spec.add_development_dependency 'fakefs'
29
29
  end
data/lib/broadside.rb CHANGED
@@ -1,14 +1,14 @@
1
1
  require 'broadside/error'
2
2
  require 'broadside/utils'
3
+ require 'broadside/configuration/verify_instance_variables'
3
4
  require 'broadside/configuration'
4
- require 'broadside/configuration/config_struct'
5
5
  require 'broadside/configuration/aws_config'
6
- require 'broadside/configuration/base_config'
7
- require 'broadside/configuration/deploy_config'
8
6
  require 'broadside/configuration/ecs_config'
7
+ require 'broadside/target'
9
8
  require 'broadside/deploy'
10
- require 'broadside/deploy/ecs_deploy'
11
- require 'broadside/deploy/ecs_manager'
9
+ require 'broadside/predeploy_commands'
10
+ require 'broadside/ecs/ecs_deploy'
11
+ require 'broadside/ecs/ecs_manager'
12
12
  require 'broadside/version'
13
13
 
14
14
  module Broadside
@@ -29,8 +29,8 @@ module Broadside
29
29
  end
30
30
 
31
31
  begin
32
- load config_file
33
32
  config.file = config_file
33
+ load config_file
34
34
  rescue LoadError => e
35
35
  error "Encountered an error loading required configuration file '#{config_file}' !"
36
36
  raise e
@@ -1,15 +1,30 @@
1
+ require 'logger'
2
+
1
3
  module Broadside
2
4
  class Configuration
5
+ extend Gem::Deprecate
6
+ include VerifyInstanceVariables
3
7
  include Utils
4
8
 
5
- attr_accessor :base, :deploy, :ecs, :aws, :file
9
+ attr_accessor(
10
+ :application,
11
+ :docker_image,
12
+ :file,
13
+ :git_repo,
14
+ :logger,
15
+ :prehook,
16
+ :posthook,
17
+ :ssh,
18
+ :timeout,
19
+ :type
20
+ )
21
+ attr_reader :targets
6
22
 
7
23
  def initialize
8
- @base = BaseConfig.new
9
- end
10
-
11
- def deploy
12
- @deploy ||= DeployConfig.new
24
+ @logger = ::Logger.new(STDOUT)
25
+ @logger.level = ::Logger::DEBUG
26
+ @logger.datetime_format = '%Y-%m-%d_%H:%M:%S'
27
+ @timeout = 600
13
28
  end
14
29
 
15
30
  def aws
@@ -20,13 +35,24 @@ module Broadside
20
35
  @ecs ||= EcsConfig.new
21
36
  end
22
37
 
23
- def verify
24
- @base.verify(:application, :docker_image)
38
+ def targets=(_targets)
39
+ raise ArgumentError, "Targets must be a hash" unless _targets.is_a?(Hash)
40
+ @targets = _targets.map { |name, config| Target.new(name, config) }
41
+ end
42
+
43
+ def verify(*args)
44
+ super(*([:application, :docker_image] + args))
45
+ end
46
+
47
+ # Maintain backward compatibility
48
+ def deploy
49
+ self
25
50
  end
51
+ deprecate :deploy, 'config.deploy.option should be configured directly as config.option', 2017, 4
26
52
 
27
- def method_missing(m, *args, &block)
28
- warn "Unknown configuration '#{m}' provided, ignoring. Check your version of broadside?"
29
- ConfigStruct.new
53
+ def base
54
+ self
30
55
  end
56
+ deprecate :base, 'config.base.option should be configured directly as config.option', 2017, 4
31
57
  end
32
58
  end
@@ -1,14 +1,14 @@
1
1
  require 'aws-sdk'
2
2
 
3
3
  module Broadside
4
- class Configuration
5
- class AwsConfig < ConfigStruct
6
- attr_accessor :region, :credentials
4
+ class AwsConfig
5
+ include VerifyInstanceVariables
7
6
 
8
- def initialize
9
- @region = 'us-east-1'
10
- @credentials = Aws::SharedCredentials.new.credentials
11
- end
7
+ attr_accessor :region, :credentials
8
+
9
+ def initialize
10
+ @region = 'us-east-1'
11
+ @credentials = Aws::SharedCredentials.new.credentials
12
12
  end
13
13
  end
14
14
  end
@@ -1,12 +1,13 @@
1
1
  module Broadside
2
- class Configuration
3
- class EcsConfig < ConfigStruct
4
- attr_accessor :cluster, :poll_frequency
2
+ class EcsConfig
3
+ include VerifyInstanceVariables
5
4
 
6
- def initialize
7
- @cluster = nil
8
- @poll_frequency = 2
9
- end
5
+ # Cluster can be overridden in a Target
6
+ attr_accessor :cluster, :poll_frequency
7
+
8
+ def initialize
9
+ @cluster = nil
10
+ @poll_frequency = 2
10
11
  end
11
12
  end
12
13
  end
@@ -0,0 +1,11 @@
1
+ module Broadside
2
+ module VerifyInstanceVariables
3
+ def verify(*args)
4
+ args.each do |var|
5
+ if self.send(var).nil?
6
+ raise Broadside::MissingVariableError, "Missing required #{self.class.to_s.split("::").last} variable '#{var}' !"
7
+ end
8
+ end
9
+ end
10
+ end
11
+ end
@@ -1,21 +1,24 @@
1
1
  module Broadside
2
2
  class Deploy
3
3
  include Utils
4
-
5
- attr_accessor :deploy_config
6
-
7
- def initialize(opts)
8
- @deploy_config = Broadside.config.deploy.dup
9
- @deploy_config.tag = opts[:tag] || @deploy_config.tag
10
- @deploy_config.target = opts[:target] || @deploy_config.target
11
- @deploy_config.verify(:target, :targets)
12
- @deploy_config.load_target!
13
-
14
- @deploy_config.scale = opts[:scale] || @deploy_config.scale
15
- @deploy_config.rollback = opts[:rollback] || @deploy_config.rollback
16
- @deploy_config.instance = opts[:instance] || @deploy_config.instance
17
- @deploy_config.command = opts[:command] || @deploy_config.command
18
- @deploy_config.lines = opts[:lines] || @deploy_config.lines
4
+ include VerifyInstanceVariables
5
+
6
+ attr_reader(
7
+ :command,
8
+ :instance,
9
+ :lines,
10
+ :tag,
11
+ :target
12
+ )
13
+
14
+ def initialize(target, opts = {})
15
+ @target = target
16
+ @command = opts[:command] || @target.command
17
+ @instance = opts[:instance] || @target.instance
18
+ @lines = opts[:lines] || 10
19
+ @rollback = opts[:rollback] || 1
20
+ @scale = opts[:scale] || @target.scale
21
+ @tag = opts[:tag]
19
22
  end
20
23
 
21
24
  def short
@@ -23,44 +26,44 @@ module Broadside
23
26
  end
24
27
 
25
28
  def full
26
- run_predeploy
29
+ config.verify(:ssh)
30
+ verify(:tag)
31
+
32
+ info "Running predeploy commands for #{family}..."
33
+ run_commands(@target.predeploy_commands)
34
+ info 'Predeploy complete.'
35
+
27
36
  deploy
28
37
  end
29
38
 
30
39
  def deploy
31
- @deploy_config.verify(:tag)
40
+ verify(:tag)
41
+
32
42
  info "Deploying #{image_tag} to #{family}..."
33
43
  yield
34
44
  info 'Deployment complete.'
35
45
  end
36
46
 
37
- def rollback(count = @deploy_config.rollback)
38
- @deploy_config.verify(:rollback)
39
- info "Rolling back #{@deploy_config.rollback} release for #{family}..."
47
+ def rollback(count = @rollback)
48
+ info "Rolling back #{count} release for #{family}..."
40
49
  yield
41
50
  info 'Rollback complete.'
42
51
  end
43
52
 
44
53
  def scale
45
- info "Rescaling #{family} with scale=#{@deploy_config.scale}"
54
+ info "Rescaling #{family} with scale=#{@scale}"
46
55
  yield
47
56
  info 'Rescaling complete.'
48
57
  end
49
58
 
50
59
  def run
51
- @deploy_config.verify(:tag, :ssh, :command)
52
- info "Running command [#{@deploy_config.command}] for #{family}..."
60
+ config.verify(:ssh)
61
+ verify(:tag, :command)
62
+ info "Running command [#{@command}] for #{family}..."
53
63
  yield
54
64
  info 'Complete.'
55
65
  end
56
66
 
57
- def run_predeploy
58
- @deploy_config.verify(:tag, :ssh)
59
- info "Running predeploy commands for #{family}..."
60
- yield
61
- info 'Predeploy complete.'
62
- end
63
-
64
67
  def status
65
68
  info "Getting status information about #{family}"
66
69
  yield
@@ -68,32 +71,33 @@ module Broadside
68
71
  end
69
72
 
70
73
  def logtail
71
- @deploy_config.verify(:instance)
74
+ verify(:instance)
72
75
  yield
73
76
  end
74
77
 
75
78
  def ssh
76
- @deploy_config.verify(:instance)
79
+ verify(:instance)
77
80
  yield
78
81
  end
79
82
 
80
83
  def bash
81
- @deploy_config.verify(:instance)
84
+ verify(:instance)
82
85
  yield
83
86
  end
84
87
 
85
- protected
88
+ private
86
89
 
87
90
  def family
88
- "#{config.base.application}_#{@deploy_config.target}"
91
+ "#{config.application}_#{@target.name}"
89
92
  end
90
93
 
91
94
  def image_tag
92
- "#{config.base.docker_image}:#{@deploy_config.tag}"
95
+ raise ArgumentError, "Missing tag" unless @tag
96
+ "#{config.docker_image}:#{@tag}"
93
97
  end
94
98
 
95
99
  def gen_ssh_cmd(ip, options = { tty: false })
96
- opts = @deploy_config.ssh || {}
100
+ opts = config.ssh || {}
97
101
  cmd = 'ssh -o StrictHostKeyChecking=no'
98
102
  cmd << ' -t -t' if options[:tty]
99
103
  cmd << " -i #{opts[:keyfile]}" if opts[:keyfile]
@@ -12,14 +12,14 @@ module Broadside
12
12
  memory: 1000
13
13
  }
14
14
 
15
- def initialize(opts)
16
- super(opts)
15
+ def initialize(target, opts = {})
16
+ super
17
17
  config.ecs.verify(:cluster, :poll_frequency)
18
18
  end
19
19
 
20
20
  def deploy
21
21
  super do
22
- unless EcsManager.service_exists?(config.ecs.cluster, family)
22
+ unless EcsManager.service_exists?(@target.cluster, family)
23
23
  exception "No service for #{family}! Please bootstrap or manually configure the service."
24
24
  end
25
25
  unless EcsManager.get_latest_task_definition_arn(family)
@@ -46,49 +46,36 @@ module Broadside
46
46
  end
47
47
 
48
48
  def bootstrap
49
- if EcsManager.get_latest_task_definition_arn(family)
50
- info("Task definition for #{family} already exists.")
51
- run_bootstrap_commands
52
- else
53
- unless @deploy_config.task_definition_config
49
+ unless EcsManager.get_latest_task_definition_arn(family)
50
+ unless @target.task_definition_config
54
51
  raise ArgumentError, "No first task definition and no :task_definition_config in '#{family}' configuration"
55
52
  end
56
53
 
57
54
  info "Creating an initial task definition for '#{family}' from the config..."
58
55
 
59
56
  EcsManager.ecs.register_task_definition(
60
- @deploy_config.task_definition_config.merge(
57
+ @target.task_definition_config.merge(
61
58
  family: family,
62
59
  container_definitions: [DEFAULT_CONTAINER_DEFINITION.merge(container_definition)]
63
60
  )
64
61
  )
65
-
66
- run_bootstrap_commands
67
62
  end
68
63
 
69
- if EcsManager.service_exists?(config.ecs.cluster, family)
64
+ run_commands(@target.bootstrap_commands)
65
+
66
+ if EcsManager.service_exists?(@target.cluster, family)
70
67
  info("Service for #{family} already exists.")
71
68
  else
72
- unless @deploy_config.service_config
69
+ unless @target.service_config
73
70
  raise ArgumentError, "Service doesn't exist and no :service_config in '#{family}' configuration"
74
71
  end
75
72
 
76
73
  info "Service '#{family}' doesn't exist, creating..."
77
- EcsManager.create_service(config.ecs.cluster, family, @deploy_config.service_config)
74
+ EcsManager.create_service(@target.cluster, family, @target.service_config)
78
75
  end
79
76
  end
80
77
 
81
- def run_bootstrap_commands
82
- update_task_revision
83
-
84
- begin
85
- @deploy_config.bootstrap_commands.each { |command| run_command(command) }
86
- ensure
87
- EcsManager.deregister_last_n_tasks_definitions(family, 1)
88
- end
89
- end
90
-
91
- def rollback(count = @deploy_config.rollback)
78
+ def rollback(count = @rollback)
92
79
  super do
93
80
  begin
94
81
  EcsManager.deregister_last_n_tasks_definitions(family, count)
@@ -108,32 +95,13 @@ module Broadside
108
95
 
109
96
  def run
110
97
  super do
111
- update_task_revision
112
-
113
- begin
114
- run_command(@deploy_config.command)
115
- ensure
116
- EcsManager.deregister_last_n_tasks_definitions(family, 1)
117
- end
118
- end
119
- end
120
-
121
- # runs before deploy commands using the latest task definition
122
- def run_predeploy
123
- super do
124
- update_task_revision
125
-
126
- begin
127
- @deploy_config.predeploy_commands.each { |command| run_command(command) }
128
- ensure
129
- EcsManager.deregister_last_n_tasks_definitions(family, 1)
130
- end
98
+ run_commands(@command)
131
99
  end
132
100
  end
133
101
 
134
102
  def status
135
103
  super do
136
- ips = EcsManager.get_running_instance_ips(config.ecs.cluster, family)
104
+ ips = EcsManager.get_running_instance_ips(@target.cluster, family)
137
105
  info "\n---------------",
138
106
  "\nDeployed task definition information:\n",
139
107
  Rainbow(PP.pp(EcsManager.get_latest_task_definition(family), '')).blue,
@@ -149,7 +117,7 @@ module Broadside
149
117
  ip = get_running_instance_ip
150
118
  debug "Tailing logs for running container at ip #{ip}..."
151
119
  search_pattern = Shellwords.shellescape(family)
152
- cmd = "docker logs -f --tail=#{@deploy_config.lines} `docker ps -n 1 --quiet --filter name=#{search_pattern}`"
120
+ cmd = "docker logs -f --tail=#{@lines} `docker ps -n 1 --quiet --filter name=#{search_pattern}`"
153
121
  tail_cmd = gen_ssh_cmd(ip) + " '#{cmd}'"
154
122
  exec tail_cmd
155
123
  end
@@ -177,7 +145,7 @@ module Broadside
177
145
  private
178
146
 
179
147
  def get_running_instance_ip
180
- EcsManager.get_running_instance_ips(config.ecs.cluster, family).fetch(@deploy_config.instance)
148
+ EcsManager.get_running_instance_ips(@target.cluster, family).fetch(@target.instance)
181
149
  end
182
150
 
183
151
  # Creates a new task revision using current directory's env vars, provided tag, and configured options.
@@ -194,7 +162,7 @@ module Broadside
194
162
 
195
163
  # Deep merge doesn't work well with arrays (e.g. :container_definitions), so build the container first.
196
164
  updatable_container_definitions.first.merge!(container_definition)
197
- revision.deep_merge!((@deploy_config.task_definition_config || {}).except(:container_definitions))
165
+ revision.deep_merge!((@target.task_definition_config || {}).except(:container_definitions))
198
166
 
199
167
  task_definition = EcsManager.ecs.register_task_definition(revision).task_definition
200
168
  debug "Successfully created #{task_definition.task_definition_arn}"
@@ -203,21 +171,21 @@ module Broadside
203
171
  # reloads the service using the latest task definition
204
172
  def update_service
205
173
  task_definition_arn = EcsManager.get_latest_task_definition_arn(family)
206
- debug "Updating #{family} with scale=#{@deploy_config.scale} using task #{task_definition_arn}..."
174
+ debug "Updating #{family} with scale=#{@target.scale} using task #{task_definition_arn}..."
207
175
 
208
176
  update_service_response = EcsManager.ecs.update_service({
209
- cluster: config.ecs.cluster,
210
- desired_count: @deploy_config.scale,
177
+ cluster: @target.cluster,
178
+ desired_count: @target.scale,
211
179
  service: family,
212
180
  task_definition: task_definition_arn
213
- }.deep_merge(@deploy_config.service_config || {}))
181
+ }.deep_merge(@target.service_config || {}))
214
182
 
215
183
  unless update_service_response.successful?
216
184
  exception('Failed to update service during deploy.', update_service_response.pretty_inspect)
217
185
  end
218
186
 
219
- EcsManager.ecs.wait_until(:services_stable, { cluster: config.ecs.cluster, services: [family] }) do |w|
220
- w.max_attempts = @deploy_config.timeout ? @deploy_config.timeout / config.ecs.poll_frequency : nil
187
+ EcsManager.ecs.wait_until(:services_stable, { cluster: @target.cluster, services: [family] }) do |w|
188
+ w.max_attempts = config.timeout ? config.timeout / config.ecs.poll_frequency : nil
221
189
  w.delay = config.ecs.poll_frequency
222
190
  seen_event = nil
223
191
 
@@ -232,36 +200,46 @@ module Broadside
232
200
  end
233
201
  end
234
202
 
235
- def run_command(command)
236
- command_name = command.join(' ')
237
- run_task_response = EcsManager.run_task(config.ecs.cluster, family, command)
203
+ def run_commands(commands)
204
+ return if commands.nil? || commands.empty?
238
205
 
239
- unless run_task_response.successful? && run_task_response.tasks.try(:[], 0)
240
- exception("Failed to run #{command_name} task.", run_task_response.pretty_inspect)
241
- end
206
+ update_task_revision
242
207
 
243
- task_arn = run_task_response.tasks[0].task_arn
244
- debug "Launched #{command_name} task #{task_arn}, waiting for completion..."
208
+ begin
209
+ Array.wrap(commands).each do |command|
210
+ command_name = command.join(' ')
211
+ run_task_response = EcsManager.run_task(@target.cluster, family, command)
245
212
 
246
- EcsManager.ecs.wait_until(:tasks_stopped, { cluster: config.ecs.cluster, tasks: [task_arn] }) do |w|
247
- w.max_attempts = nil
248
- w.delay = config.ecs.poll_frequency
249
- w.before_attempt do |attempt|
250
- debug "Attempt #{attempt}: waiting for #{command_name} to complete..."
251
- end
252
- end
213
+ unless run_task_response.successful? && run_task_response.tasks.try(:[], 0)
214
+ exception("Failed to run #{command_name} task.", run_task_response.pretty_inspect)
215
+ end
253
216
 
254
- info "#{command_name} task container logs:\n#{get_container_logs(task_arn)}"
217
+ task_arn = run_task_response.tasks[0].task_arn
218
+ debug "Launched #{command_name} task #{task_arn}, waiting for completion..."
255
219
 
256
- if (code = EcsManager.get_task_exit_code(config.ecs.cluster, task_arn, family)) == 0
257
- debug "#{command_name} task #{task_arn} exited with status code 0"
258
- else
259
- exception "#{command_name} task #{task_arn} exited with a non-zero status code #{code}!"
220
+ EcsManager.ecs.wait_until(:tasks_stopped, { cluster: @target.cluster, tasks: [task_arn] }) do |w|
221
+ w.max_attempts = nil
222
+ w.delay = config.ecs.poll_frequency
223
+ w.before_attempt do |attempt|
224
+ debug "Attempt #{attempt}: waiting for #{command_name} to complete..."
225
+ end
226
+ end
227
+
228
+ info "#{command_name} task container logs:\n#{get_container_logs(task_arn)}"
229
+
230
+ if (code = EcsManager.get_task_exit_code(@target.cluster, task_arn, family)) == 0
231
+ debug "#{command_name} task #{task_arn} exited with status code 0"
232
+ else
233
+ exception "#{command_name} task #{task_arn} exited with a non-zero status code #{code}!"
234
+ end
235
+ end
236
+ ensure
237
+ EcsManager.deregister_last_n_tasks_definitions(family, 1)
260
238
  end
261
239
  end
262
240
 
263
241
  def get_container_logs(task_arn)
264
- ip = EcsManager.get_running_instance_ips(config.ecs.cluster, family, task_arn).first
242
+ ip = EcsManager.get_running_instance_ips(@target.cluster, family, task_arn).first
265
243
  debug "Found ip of container instance: #{ip}"
266
244
 
267
245
  find_container_id_cmd = "#{gen_ssh_cmd(ip)} \"docker ps -aqf 'label=com.amazonaws.ecs.task-arn=#{task_arn}'\""
@@ -279,15 +257,15 @@ module Broadside
279
257
  end
280
258
 
281
259
  def container_definition
282
- configured_containers = (@deploy_config.task_definition_config || {})[:container_definitions]
260
+ configured_containers = (@target.task_definition_config || {})[:container_definitions]
283
261
  if configured_containers && configured_containers.size > 1
284
262
  raise ArgumentError, 'Creating > 1 container definition not supported yet'
285
263
  end
286
264
 
287
265
  (configured_containers.try(:first) || {}).merge(
288
266
  name: family,
289
- command: @deploy_config.command,
290
- environment: @deploy_config.env_vars,
267
+ command: @command,
268
+ environment: @target.env_vars,
291
269
  image: image_tag
292
270
  )
293
271
  end
File without changes
@@ -0,0 +1,7 @@
1
+ # Here rest some commonly used predeploy commands, so they can be included by constant name instead of
2
+ # having to retype them in every config file.
3
+ module Broadside
4
+ module PredeployCommands
5
+ RAKE_DB_MIGRATE = ['bundle', 'exec', 'rake', '--trace', 'db:migrate']
6
+ end
7
+ end
@@ -0,0 +1,110 @@
1
+ require 'dotenv'
2
+ require 'pathname'
3
+
4
+ module Broadside
5
+ class Target
6
+ include VerifyInstanceVariables
7
+ include Utils
8
+
9
+ attr_accessor(
10
+ :bootstrap_commands,
11
+ :command,
12
+ :env_files,
13
+ :env_vars,
14
+ :instance,
15
+ :name,
16
+ :predeploy_commands,
17
+ :scale,
18
+ :service_config,
19
+ :tag,
20
+ :task_definition_config
21
+ )
22
+
23
+ DEFAULT_INSTANCE = 0
24
+
25
+ TARGET_ATTRIBUTE_VALIDATIONS = {
26
+ bootstrap_commands: ->(target_attribute) { validate_commands(target_attribute) },
27
+ command: ->(target_attribute) { validate_types([Array, NilClass], target_attribute) },
28
+ env_files: ->(target_attribute) { validate_types([String, Array], target_attribute) },
29
+ predeploy_commands: ->(target_attribute) { validate_commands(target_attribute) },
30
+ scale: ->(target_attribute) { validate_types([Integer], target_attribute) },
31
+ service_config: ->(target_attribute) { validate_types([Hash, NilClass], target_attribute) },
32
+ task_definition_config: ->(target_attribute) { validate_types([Hash, NilClass], target_attribute) }
33
+ }
34
+
35
+ def initialize(name, options = {})
36
+ @name = name
37
+ @config = options
38
+
39
+ @bootstrap_commands = @config[:bootstrap_commands] || []
40
+ @cluster = @config[:cluster]
41
+ @command = @config[:command]
42
+ _env_files = @config[:env_files] || @config[:env_file]
43
+ @env_files = _env_files ? [*_env_files] : nil
44
+ @env_vars = {}
45
+ @instance = DEFAULT_INSTANCE || @config[:instance]
46
+ @predeploy_commands = @config[:predeploy_commands]
47
+ @scale = @config[:scale]
48
+ @service_config = @config[:service_config]
49
+ @tag = @config[:tag]
50
+ @task_definition_config = @config[:task_definition_config]
51
+
52
+ validate!
53
+ load_env_vars!
54
+ end
55
+
56
+ def cluster
57
+ @cluster || config.ecs.cluster
58
+ end
59
+
60
+ private
61
+
62
+ def validate!
63
+ invalid_messages = TARGET_ATTRIBUTE_VALIDATIONS.map do |var, validation|
64
+ message = validation.call(instance_variable_get('@' + var.to_s))
65
+ message.nil? ? nil : "Deploy target '#{@name}' parameter '#{var}' is invalid: #{message}"
66
+ end.compact
67
+
68
+ unless invalid_messages.empty?
69
+ raise ArgumentError, invalid_messages.join("\n")
70
+ end
71
+ end
72
+
73
+ def load_env_vars!
74
+ @env_files.flatten.each do |env_path|
75
+ env_file = Pathname.new(env_path)
76
+
77
+ unless env_file.absolute?
78
+ dir = config.file.nil? ? Dir.pwd : Pathname.new(config.file).dirname
79
+ env_file = env_file.expand_path(dir)
80
+ end
81
+
82
+ if env_file.exist?
83
+ vars = Dotenv.load(env_file)
84
+ @env_vars.merge!(vars)
85
+ else
86
+ raise ArgumentError, "Could not find file '#{env_file}' for loading environment variables !"
87
+ end
88
+ end
89
+
90
+ # convert env vars to format ecs expects
91
+ @env_vars = @env_vars.map { |k, v| { 'name' => k, 'value' => v } }
92
+ end
93
+
94
+ def self.validate_types(types, target_attribute)
95
+ return nil if types.any? { |type| target_attribute.is_a?(type) }
96
+
97
+ "'#{target_attribute}' must be of type [#{types.join('|')}], got '#{target_attribute.class}' !"
98
+ end
99
+
100
+ def self.validate_commands(commands)
101
+ return nil if commands.nil?
102
+ return 'predeploy_commands must be an array' unless commands.is_a?(Array)
103
+
104
+ messages = commands.reject { |cmd| cmd.is_a?(Array) }.map do |command|
105
+ "predeploy_command '#{command}' must be an array" unless command.is_a?(Array)
106
+ end
107
+ messages.empty? ? nil : messages.join(', ')
108
+ end
109
+ end
110
+ end
@@ -1,19 +1,19 @@
1
1
  module Broadside
2
2
  module Utils
3
3
  def debug(*args)
4
- config.base.logger.debug(args.join(' '))
4
+ config.logger.debug(args.join(' '))
5
5
  end
6
6
 
7
7
  def info(*args)
8
- config.base.logger.info(args.join(' '))
8
+ config.logger.info(args.join(' '))
9
9
  end
10
10
 
11
11
  def warn(*args)
12
- config.base.logger.warn(args.join(' '))
12
+ config.logger.warn(args.join(' '))
13
13
  end
14
14
 
15
15
  def error(*args)
16
- config.base.logger.error(args.join(' '))
16
+ config.logger.error(args.join(' '))
17
17
  end
18
18
 
19
19
  def exception(*args)
@@ -1,3 +1,3 @@
1
1
  module Broadside
2
- VERSION = '1.4.0'
2
+ VERSION = '2.0.0'
3
3
  end
metadata CHANGED
@@ -1,14 +1,15 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: broadside
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.4.0
4
+ version: 2.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Matthew Leung
8
+ - Lumos Labs, Inc.
8
9
  autorequire:
9
10
  bindir: bin
10
11
  cert_chain: []
11
- date: 2017-01-09 00:00:00.000000000 Z
12
+ date: 2017-01-13 00:00:00.000000000 Z
12
13
  dependencies:
13
14
  - !ruby/object:Gem::Dependency
14
15
  name: activesupport
@@ -90,16 +91,16 @@ dependencies:
90
91
  name: rspec
91
92
  requirement: !ruby/object:Gem::Requirement
92
93
  requirements:
93
- - - ">="
94
+ - - "~>"
94
95
  - !ruby/object:Gem::Version
95
- version: '0'
96
+ version: 3.4.0
96
97
  type: :development
97
98
  prerelease: false
98
99
  version_requirements: !ruby/object:Gem::Requirement
99
100
  requirements:
100
- - - ">="
101
+ - - "~>"
101
102
  - !ruby/object:Gem::Version
102
- version: '0'
103
+ version: 3.4.0
103
104
  - !ruby/object:Gem::Dependency
104
105
  name: bundler
105
106
  requirement: !ruby/object:Gem::Requirement
@@ -150,14 +151,14 @@ files:
150
151
  - lib/broadside.rb
151
152
  - lib/broadside/configuration.rb
152
153
  - lib/broadside/configuration/aws_config.rb
153
- - lib/broadside/configuration/base_config.rb
154
- - lib/broadside/configuration/config_struct.rb
155
- - lib/broadside/configuration/deploy_config.rb
156
154
  - lib/broadside/configuration/ecs_config.rb
155
+ - lib/broadside/configuration/verify_instance_variables.rb
157
156
  - lib/broadside/deploy.rb
158
- - lib/broadside/deploy/ecs_deploy.rb
159
- - lib/broadside/deploy/ecs_manager.rb
157
+ - lib/broadside/ecs/ecs_deploy.rb
158
+ - lib/broadside/ecs/ecs_manager.rb
160
159
  - lib/broadside/error.rb
160
+ - lib/broadside/predeploy_commands.rb
161
+ - lib/broadside/target.rb
161
162
  - lib/broadside/utils.rb
162
163
  - lib/broadside/version.rb
163
164
  homepage: https://github.com/lumoslabs/broadside
@@ -180,9 +181,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
180
181
  version: '0'
181
182
  requirements: []
182
183
  rubyforge_project:
183
- rubygems_version: 2.2.3
184
+ rubygems_version: 2.2.5
184
185
  signing_key:
185
186
  specification_version: 4
186
187
  summary: A command-line tool for EC2 Container Service deployment.
187
188
  test_files: []
188
- has_rdoc:
@@ -1,19 +0,0 @@
1
- require 'logger'
2
-
3
- module Broadside
4
- class Configuration
5
- class BaseConfig < ConfigStruct
6
- attr_accessor :application, :git_repo, :docker_image, :logger, :loglevel, :prehook, :posthook
7
-
8
- def initialize
9
- @application = nil
10
- @docker_image = nil
11
- @logger = Logger.new(STDOUT)
12
- @logger.level = Logger::DEBUG
13
- @logger.datetime_format = '%Y-%m-%d_%H:%M:%S'
14
- @prehook = nil
15
- @posthook = nil
16
- end
17
- end
18
- end
19
- end
@@ -1,24 +0,0 @@
1
- module Broadside
2
- class Configuration
3
- class ConfigStruct
4
- def verify(*args)
5
- args.each do |var|
6
- if self.send(var).nil?
7
- raise Broadside::MissingVariableError, "Missing required #{self.class.to_s.split("::").last} variable '#{var}' !"
8
- end
9
- end
10
- end
11
-
12
- def to_h
13
- self.instance_variables.inject({}) do |h, var|
14
- h[var] = self.instance_variable_get(var)
15
- h
16
- end
17
- end
18
-
19
- def method_missing(m, *args, &block)
20
- warn "Unknown configuration '#{m}' provided, ignoring. Check your version of broadside?"
21
- end
22
- end
23
- end
24
- end
@@ -1,148 +0,0 @@
1
- require 'dotenv'
2
- require 'pathname'
3
-
4
- module Broadside
5
- class Configuration
6
- class DeployConfig < ConfigStruct
7
- include Utils
8
-
9
- DEFAULT_PREDEPLOY_COMMANDS = [
10
- ['bundle', 'exec', 'rake', '--trace', 'db:migrate']
11
- ]
12
-
13
- attr_accessor(
14
- :type,
15
- :tag,
16
- :ssh,
17
- :rollback,
18
- :timeout,
19
- :target,
20
- :targets,
21
- :scale,
22
- :env_vars,
23
- :command,
24
- :instance,
25
- :lines,
26
- :predeploy_commands,
27
- :bootstrap_commands,
28
- :service_config,
29
- :task_definition_config
30
- )
31
-
32
- TARGET_ATTRIBUTE_VALIDATIONS = {
33
- scale: ->(target_attribute) { validate_types([Fixnum], target_attribute) },
34
- env_file: ->(target_attribute) { validate_types([String, Array], target_attribute) },
35
- command: ->(target_attribute) { validate_types([Array, NilClass], target_attribute) },
36
- predeploy_commands: ->(target_attribute) { validate_predeploy_commands(target_attribute) },
37
- bootstrap_commands: ->(target_attribute) { validate_bootstrap_commands(target_attribute) },
38
- service_config: ->(target_attribute) { validate_types([Hash, NilClass], target_attribute) },
39
- task_definition_config: ->(target_attribute) { validate_types([Hash, NilClass], target_attribute) }
40
- }
41
-
42
- def initialize
43
- @type = 'ecs'
44
- @ssh = nil
45
- @tag = nil
46
- @rollback = 1
47
- @timeout = 600
48
- @target = nil
49
- @targets = nil
50
- @scale = nil
51
- @env_vars = nil
52
- @command = nil
53
- @predeploy_commands = DEFAULT_PREDEPLOY_COMMANDS
54
- @bootstrap_commands = []
55
- @instance = 0
56
- @service_config = nil
57
- @task_definition_config = nil
58
- @lines = 10
59
- end
60
-
61
- # Validates format of deploy targets
62
- # Checks existence of provided target
63
- def validate_targets!
64
- @targets.each do |target, configuration|
65
- invalid_messages = TARGET_ATTRIBUTE_VALIDATIONS.map do |var, validation|
66
- message = validation.call(configuration[var])
67
- message.nil? ? nil : "Deploy target '#{@target}' parameter '#{var}' is invalid: #{message}"
68
- end.compact
69
-
70
- unless invalid_messages.empty?
71
- raise ArgumentError, invalid_messages.join("\n")
72
- end
73
- end
74
-
75
- unless @targets.has_key?(@target)
76
- raise ArgumentError, "Could not find deploy target #{@target} in configuration !"
77
- end
78
- end
79
-
80
- # Loads deploy target data using provided target
81
- def load_target!
82
- validate_targets!
83
- load_env_vars!
84
-
85
- @scale ||= @targets[@target][:scale]
86
- @command = @targets[@target][:command]
87
- @predeploy_commands = @targets[@target][:predeploy_commands] if @targets[@target][:predeploy_commands]
88
- @bootstrap_commands = @targets[@target][:bootstrap_commands] if @targets[@target][:bootstrap_commands]
89
- @service_config = @targets[@target][:service_config]
90
- @task_definition_config = @targets[@target][:task_definition_config]
91
- end
92
-
93
- def load_env_vars!
94
- @env_vars ||= {}
95
-
96
- [@targets[@target][:env_file]].flatten.each do |env_path|
97
- env_file = Pathname.new(env_path)
98
-
99
- unless env_file.absolute?
100
- dir = config.file.nil? ? Dir.pwd : Pathname.new(config.file).dirname
101
- env_file = env_file.expand_path(dir)
102
- end
103
-
104
- if env_file.exist?
105
- vars = Dotenv.load(env_file)
106
- @env_vars.merge!(vars)
107
- else
108
- raise ArgumentError, "Could not find file '#{env_file}' for loading environment variables !"
109
- end
110
- end
111
-
112
- # convert env vars to format ecs expects
113
- @env_vars = @env_vars.map { |k, v| { 'name' => k, 'value' => v } }
114
- end
115
-
116
- private
117
-
118
- class << self
119
- def validate_types(types, target_attribute)
120
- if types.include?(target_attribute.class)
121
- nil
122
- else
123
- "'#{target_attribute}' must be of type [#{types.join('|')}], got '#{target_attribute.class}' !"
124
- end
125
- end
126
-
127
- def validate_predeploy_commands(commands)
128
- validate_commands(commands, 'predeploy_commands')
129
- end
130
-
131
- def validate_bootstrap_commands(commands)
132
- validate_commands(commands, 'bootstrap_commands')
133
- end
134
-
135
- def validate_commands(commands, attribute_name)
136
- return nil if commands.nil?
137
- return "#{attribute_name} must be an array" unless commands.is_a?(Array)
138
-
139
- messages = commands.reject { |cmd| cmd.is_a?(Array) }.map do |command|
140
- "#{attribute_name} '#{command}' must be an array" unless command.is_a?(Array)
141
- end
142
-
143
- messages.empty? ? nil : messages.join(', ')
144
- end
145
- end
146
- end
147
- end
148
- end