ecs_deploy 0.3.2 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.gitignore +1 -0
- data/.travis.yml +5 -0
- data/CHANGELOG.md +117 -0
- data/README.md +269 -23
- data/Rakefile +4 -0
- data/ecs_deploy.gemspec +9 -3
- data/lib/ecs_deploy.rb +1 -1
- data/lib/ecs_deploy/auto_scaler.rb +105 -339
- data/lib/ecs_deploy/auto_scaler/auto_scaling_group_config.rb +209 -0
- data/lib/ecs_deploy/auto_scaler/cluster_resource_manager.rb +149 -0
- data/lib/ecs_deploy/auto_scaler/config_base.rb +16 -0
- data/lib/ecs_deploy/auto_scaler/instance_drainer.rb +134 -0
- data/lib/ecs_deploy/auto_scaler/service_config.rb +222 -0
- data/lib/ecs_deploy/auto_scaler/spot_fleet_request_config.rb +102 -0
- data/lib/ecs_deploy/auto_scaler/trigger_config.rb +36 -0
- data/lib/ecs_deploy/capistrano.rb +70 -1
- data/lib/ecs_deploy/configuration.rb +3 -2
- data/lib/ecs_deploy/instance_fluctuation_manager.rb +173 -0
- data/lib/ecs_deploy/scheduled_task.rb +15 -3
- data/lib/ecs_deploy/service.rb +89 -7
- data/lib/ecs_deploy/task_definition.rb +22 -8
- data/lib/ecs_deploy/version.rb +1 -1
- metadata +110 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: c88001bfd80f75aaeb09c2af4ce4a6e4c44976f465a2812ce1691d4c19e1acd6
|
4
|
+
data.tar.gz: 1bfad93623edc78868040517a1a41006110e8881d661597b6d9b3416449f7b90
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 95f74dd98955021f9280feac7df80a44af75228b0c6460fb807d0b6f8d9976b981a0a253234b36ebd4e5a5f4a10770ba9f77d6782907a9d453e5a4be7bb5c913
|
7
|
+
data.tar.gz: 837b94c1bf33af16303e2cf29720480fa17d4aa165eee2d7b92f9f17fc7facd1637d0475af0b003c5c14ba186042a267a192a46cd9f17dd489909c5839eef83e
|
data/.gitignore
CHANGED
data/.travis.yml
ADDED
data/CHANGELOG.md
ADDED
@@ -0,0 +1,117 @@
|
|
1
|
+
# v1.0
|
2
|
+
|
3
|
+
## Release v1.0.0 - 2019/12/24
|
4
|
+
|
5
|
+
### New feature
|
6
|
+
|
7
|
+
* Add tasks to deploy the application faster
|
8
|
+
https://github.com/reproio/ecs_deploy/pull/57
|
9
|
+
|
10
|
+
### Enhancement
|
11
|
+
|
12
|
+
* Add parameters `ecs_wait_until_services_stable_max_attempts` and `ecs_wait_until_services_stable_delay`
|
13
|
+
https://github.com/reproio/ecs_deploy/pull/30
|
14
|
+
* Detect region automatically according to AWS SDK
|
15
|
+
https://github.com/reproio/ecs_deploy/pull/31
|
16
|
+
* Support new features of ECS to support Fargate
|
17
|
+
https://github.com/reproio/ecs_deploy/pull/32
|
18
|
+
* Ignore running tasks which don't belong to the ECS services on deregistering container instances
|
19
|
+
https://github.com/reproio/ecs_deploy/pull/33
|
20
|
+
* Drop AWS SDK 2 support
|
21
|
+
https://github.com/reproio/ecs_deploy/pull/34
|
22
|
+
* Support scheduling_strategy option
|
23
|
+
https://github.com/reproio/ecs_deploy/pull/35
|
24
|
+
* Support execution_role_arn on task_definition
|
25
|
+
https://github.com/reproio/ecs_deploy/pull/36
|
26
|
+
* Support spot fleet requests and container instance draining
|
27
|
+
https://github.com/reproio/ecs_deploy/pull/40
|
28
|
+
* Add network_configuration paramters to ScheduledTask
|
29
|
+
https://github.com/reproio/ecs_deploy/pull/46
|
30
|
+
* Support tagging ECS resources
|
31
|
+
https://github.com/reproio/ecs_deploy/pull/48
|
32
|
+
https://github.com/reproio/ecs_deploy/pull/49
|
33
|
+
* Wait for stopping tasks until tasks stop
|
34
|
+
https://github.com/reproio/ecs_deploy/pull/50
|
35
|
+
* Improve performance when start tasks
|
36
|
+
https://github.com/reproio/ecs_deploy/pull/53
|
37
|
+
* Improve stability of auto scaling groups managed by ecs_auto_scaler
|
38
|
+
https://github.com/reproio/ecs_deploy/pull/55
|
39
|
+
|
40
|
+
### Bug fixes
|
41
|
+
|
42
|
+
* Fix infinite loop that occurs when there are more than 100 container instances
|
43
|
+
https://github.com/reproio/ecs_deploy/pull/38
|
44
|
+
* Fix errors that occur on decreasing more than 20 container instances
|
45
|
+
https://github.com/reproio/ecs_deploy/pull/39
|
46
|
+
|
47
|
+
# Ancient releases
|
48
|
+
|
49
|
+
## Release v0.3.2 - 2017/23/10
|
50
|
+
|
51
|
+
### Enhancement
|
52
|
+
|
53
|
+
* Remove execution feature
|
54
|
+
https://github.com/reproio/ecs_deploy/pull/24
|
55
|
+
* Support container overrides in scheduled tasks
|
56
|
+
https://github.com/reproio/ecs_deploy/pull/26
|
57
|
+
|
58
|
+
### Bug fixes
|
59
|
+
|
60
|
+
* Fix deployment errors that occur when `ecs_scheduled_tasks` is not set
|
61
|
+
https://github.com/reproio/ecs_deploy/pull/27
|
62
|
+
|
63
|
+
## Release v0.3.1 - 2017/04/08
|
64
|
+
|
65
|
+
### Bug fixes
|
66
|
+
|
67
|
+
* Fix block parameter name
|
68
|
+
|
69
|
+
## Release v0.3.0 - 2017/03/08
|
70
|
+
|
71
|
+
### New feature
|
72
|
+
|
73
|
+
* Support ScheduledTask deployment
|
74
|
+
https://github.com/reproio/ecs_deploy/pull/22
|
75
|
+
|
76
|
+
### Enhancement
|
77
|
+
|
78
|
+
* Support network_mode and placement_constraints
|
79
|
+
* Introduce `ecs_registered_tasks` capistrano variable
|
80
|
+
https://github.com/reproio/ecs_deploy/pull/23
|
81
|
+
|
82
|
+
### Bug fixes
|
83
|
+
|
84
|
+
* Filter inactive services
|
85
|
+
https://github.com/reproio/ecs_deploy/pull/19
|
86
|
+
* Wait 10 services at once
|
87
|
+
https://github.com/reproio/ecs_deploy/pull/20
|
88
|
+
https://github.com/reproio/ecs_deploy/pull/21
|
89
|
+
|
90
|
+
## Release v0.2.0 - 2016/31/10
|
91
|
+
|
92
|
+
### Enhancement
|
93
|
+
|
94
|
+
* Support task role arn
|
95
|
+
https://github.com/reproio/ecs_deploy/pull/13
|
96
|
+
* Make the scale-in process safe
|
97
|
+
https://github.com/reproio/ecs_deploy/pull/14
|
98
|
+
* Support ALB
|
99
|
+
https://github.com/reproio/ecs_deploy/pull/15
|
100
|
+
|
101
|
+
## Release v0.1.2 - 2016/28/07
|
102
|
+
|
103
|
+
### Bug fixes
|
104
|
+
|
105
|
+
* Fix rollback bug
|
106
|
+
https://github.com/reproio/ecs_deploy/pull/11
|
107
|
+
|
108
|
+
## Release v0.1.1 - 2016/03/07
|
109
|
+
|
110
|
+
### Bug fixes
|
111
|
+
|
112
|
+
* Add missing desired_count for backend services
|
113
|
+
https://github.com/reproio/ecs_deploy/pull/9
|
114
|
+
|
115
|
+
## Release v0.1.0 - 2016/27/06
|
116
|
+
|
117
|
+
Initial release.
|
data/README.md
CHANGED
@@ -31,6 +31,8 @@ set :ecs_default_cluster, "ecs-cluster-name"
|
|
31
31
|
set :ecs_region, %w(ap-northeast-1) # optional, if nil, use environment variable
|
32
32
|
set :ecs_service_role, "customEcsServiceRole" # default: ecsServiceRole
|
33
33
|
set :ecs_deploy_wait_timeout, 600 # default: 300
|
34
|
+
set :ecs_wait_until_services_stable_max_attempts, 40 # optional
|
35
|
+
set :ecs_wait_until_services_stable_delay, 15 # optional
|
34
36
|
|
35
37
|
set :ecs_tasks, [
|
36
38
|
{
|
@@ -92,7 +94,7 @@ set :ecs_scheduled_tasks, [
|
|
92
94
|
{
|
93
95
|
cluster: "default", # Unless this key, use fetch(:ecs_default_cluster)
|
94
96
|
rule_name: "schedule_name",
|
95
|
-
schedule_expression: "cron(0 12 * * ? *"
|
97
|
+
schedule_expression: "cron(0 12 * * ? *)",
|
96
98
|
description: "schedule_description", # Optional
|
97
99
|
target_id: "task_name", # Unless this key, use task_definition_name
|
98
100
|
task_definition_name: "myapp-#{fetch(:rails_env)}",
|
@@ -187,7 +189,16 @@ And rollback
|
|
187
189
|
|
188
190
|
## Autoscaler
|
189
191
|
|
190
|
-
|
192
|
+
The autoscaler of `ecs_deploy` supports auto scaling of ECS services and clusters.
|
193
|
+
|
194
|
+
### Prerequisits
|
195
|
+
|
196
|
+
* You use a ECS cluster whose instances belong to either an auto scaling group or a spot fleet request
|
197
|
+
* You have CloudWatch alarms and you want to scale services when their state changes
|
198
|
+
|
199
|
+
### How to use autoscaler
|
200
|
+
|
201
|
+
First, write a configuration file (YAML format) like below:
|
191
202
|
|
192
203
|
```yaml
|
193
204
|
# ポーリング時にupscale_triggersに指定した状態のalarmがあればstep分serviceとinstanceを増やす (max_task_countまで)
|
@@ -200,37 +211,272 @@ polling_interval: 60
|
|
200
211
|
auto_scaling_groups:
|
201
212
|
- name: ecs-cluster-nodes
|
202
213
|
region: ap-northeast-1
|
203
|
-
buffer: 1 # タスク数に対する余剰のインスタンス数
|
204
|
-
|
205
|
-
services:
|
206
|
-
- name: repro-api-production
|
207
214
|
cluster: ecs-cluster
|
215
|
+
# autoscaler will set the capacity to (buffer + desired_tasks * required_capacity).
|
216
|
+
# Adjust this value if it takes much time to prepare ECS instances and launch new tasks.
|
217
|
+
buffer: 1
|
218
|
+
services:
|
219
|
+
- name: repro-api-production
|
220
|
+
step: 1
|
221
|
+
idle_time: 240
|
222
|
+
max_task_count: [10, 25]
|
223
|
+
scheduled_min_task_count:
|
224
|
+
- {from: "1:45", to: "4:30", count: 8}
|
225
|
+
cooldown_time_for_reach_max: 600
|
226
|
+
min_task_count: 0
|
227
|
+
# Required capacity per task (default: 1)
|
228
|
+
# You should specify "binpack" as task placement strategy if the value is less than 1 and you use an auto scaling group.
|
229
|
+
required_capacity: 0.5
|
230
|
+
upscale_triggers:
|
231
|
+
- alarm_name: "ECS [repro-api-production] CPUUtilization"
|
232
|
+
state: ALARM
|
233
|
+
- alarm_name: "ELB repro-api-a HTTPCode_Backend_5XX"
|
234
|
+
state: ALARM
|
235
|
+
step: 2
|
236
|
+
downscale_triggers:
|
237
|
+
- alarm_name: "ECS [repro-api-production] CPUUtilization (low)"
|
238
|
+
state: OK
|
239
|
+
|
240
|
+
spot_fleet_requests:
|
241
|
+
- id: sfr-354de735-2c17-4565-88c9-10ada5b957e5
|
208
242
|
region: ap-northeast-1
|
209
|
-
|
210
|
-
|
211
|
-
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
|
224
|
-
|
225
|
-
|
243
|
+
cluster: ecs-cluster-for-worker
|
244
|
+
buffer: 1
|
245
|
+
services:
|
246
|
+
- name: repro-worker-production
|
247
|
+
step: 1
|
248
|
+
idle_time: 240
|
249
|
+
cooldown_time_for_reach_max: 600
|
250
|
+
min_task_count: 0
|
251
|
+
# Required capacity per task (default: 1)
|
252
|
+
# The capacity assumes that WeightedCapacity is equal to the number of vCPUs.
|
253
|
+
required_capacity: 2
|
254
|
+
upscale_triggers:
|
255
|
+
- alarm_name: "ECS [repro-worker-production] CPUUtilization"
|
256
|
+
state: ALARM
|
257
|
+
downscale_triggers:
|
258
|
+
- alarm_name: "ECS [repro-worker-production] CPUUtilization (low)"
|
259
|
+
state: OK
|
260
|
+
|
261
|
+
# If you specify `spot_instance_intrp_warns_queue_urls` as SQS queue for spot instance interruption warnings,
|
262
|
+
# autoscaler will polls them and set the state of instances to be intrrupted to "DRAINING".
|
263
|
+
# autoscaler will also waits for the capacity of active instances in the cluster being decreased
|
264
|
+
# when the capacity of spot fleet request is decreased,
|
265
|
+
# so you should specify URLs or set the state of the instances to "DRAINING" manually.
|
266
|
+
spot_instance_intrp_warns_queue_urls:
|
267
|
+
- https://sqs.ap-northeast-1.amazonaws.com/<account-id>/spot-instance-intrp-warns
|
226
268
|
```
|
227
269
|
|
270
|
+
Then, execute the following command:
|
271
|
+
|
228
272
|
```sh
|
229
273
|
ecs_auto_scaler <config yaml>
|
230
274
|
```
|
231
275
|
|
232
276
|
I recommends deploy `ecs_auto_scaler` on ECS too.
|
233
277
|
|
278
|
+
### Signals
|
279
|
+
|
280
|
+
Signal | Description
|
281
|
+
-----------|------------------------------------------------------------
|
282
|
+
TERM, INT | Shutdown gracefully
|
283
|
+
CONT | Resume auto scaling
|
284
|
+
TSTP | Pause auto scaling (Run only container instance draining)
|
285
|
+
|
286
|
+
### IAM policy for autoscaler
|
287
|
+
|
288
|
+
The following permissions are required for the preceding configuration of "repro-api-production" service:
|
289
|
+
|
290
|
+
```
|
291
|
+
{
|
292
|
+
"Version": "2012-10-17",
|
293
|
+
"Statement": [
|
294
|
+
{
|
295
|
+
"Effect": "Allow",
|
296
|
+
"Action": [
|
297
|
+
"autoscaling:DescribeAutoScalingGroups",
|
298
|
+
"cloudwatch:DescribeAlarms",
|
299
|
+
"ec2:DescribeInstances",
|
300
|
+
"ec2:TerminateInstances",
|
301
|
+
"ecs:ListTasks"
|
302
|
+
],
|
303
|
+
"Resource": "*"
|
304
|
+
},
|
305
|
+
{
|
306
|
+
"Effect": "Allow",
|
307
|
+
"Action": [
|
308
|
+
"ecs:DescribeServices",
|
309
|
+
"ecs:UpdateService"
|
310
|
+
],
|
311
|
+
"Resource": [
|
312
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:service/ecs-cluster/repro-api-production"
|
313
|
+
]
|
314
|
+
},
|
315
|
+
{
|
316
|
+
"Effect": "Allow",
|
317
|
+
"Action": [
|
318
|
+
"ecs:DescribeTasks"
|
319
|
+
],
|
320
|
+
"Resource": [
|
321
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:task/ecs-cluster/*"
|
322
|
+
]
|
323
|
+
},
|
324
|
+
{
|
325
|
+
"Effect": "Allow",
|
326
|
+
"Action": [
|
327
|
+
"autoscaling:DetachInstances",
|
328
|
+
"autoscaling:UpdateAutoScalingGroup"
|
329
|
+
],
|
330
|
+
"Resource": [
|
331
|
+
"arn:aws:autoscaling:ap-northeast-1:<account-id>:autoScalingGroup:<group-id>:autoScalingGroupName/ecs-cluster-nodes"
|
332
|
+
]
|
333
|
+
},
|
334
|
+
{
|
335
|
+
"Effect": "Allow",
|
336
|
+
"Action": [
|
337
|
+
"ecs:DescribeContainerInstances"
|
338
|
+
],
|
339
|
+
"Resource": [
|
340
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:container-instance/ecs-cluster/*"
|
341
|
+
]
|
342
|
+
},
|
343
|
+
{
|
344
|
+
"Effect": "Allow",
|
345
|
+
"Action": [
|
346
|
+
"ecs:DeregisterContainerInstance",
|
347
|
+
"ecs:ListContainerInstances"
|
348
|
+
],
|
349
|
+
"Resource": [
|
350
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:cluster/ecs-cluster"
|
351
|
+
]
|
352
|
+
}
|
353
|
+
]
|
354
|
+
}
|
355
|
+
```
|
356
|
+
|
357
|
+
If you use spot instances, additional permissions are required like below:
|
358
|
+
|
359
|
+
```
|
360
|
+
{
|
361
|
+
"Version": "2012-10-17",
|
362
|
+
"Statement": [
|
363
|
+
{
|
364
|
+
"Effect": "Allow",
|
365
|
+
"Action": "ecs:UpdateContainerInstancesState",
|
366
|
+
"Resource": "arn:aws:ecs:ap-northeast-1:<account-id>:container-instance/ecs-cluster/*"
|
367
|
+
},
|
368
|
+
{
|
369
|
+
"Effect": "Allow",
|
370
|
+
"Action": [
|
371
|
+
"sqs:DeleteMessage",
|
372
|
+
"sqs:DeleteMessageBatch",
|
373
|
+
"sqs:ReceiveMessage"
|
374
|
+
],
|
375
|
+
"Resource": "arn:aws:sqs:ap-northeast-1:<account-id>:spot-instance-intrp-warns"
|
376
|
+
}
|
377
|
+
]
|
378
|
+
}
|
379
|
+
```
|
380
|
+
|
381
|
+
The following permissions are required for the preceding configuration of "repro-worker-production" service:
|
382
|
+
|
383
|
+
```
|
384
|
+
{
|
385
|
+
"Version": "2012-10-17",
|
386
|
+
"Statement": [
|
387
|
+
{
|
388
|
+
"Effect": "Allow",
|
389
|
+
"Action": [
|
390
|
+
"sqs:DeleteMessage",
|
391
|
+
"sqs:DeleteMessageBatch",
|
392
|
+
"sqs:ReceiveMessage"
|
393
|
+
],
|
394
|
+
"Resource": "arn:aws:sqs:ap-northeast-1:<account-id>:spot-instance-intrp-warns"
|
395
|
+
},
|
396
|
+
{
|
397
|
+
"Effect": "Allow",
|
398
|
+
"Action": [
|
399
|
+
"cloudwatch:DescribeAlarms",
|
400
|
+
"ec2:DescribeInstances",
|
401
|
+
"ec2:DescribeSpotFleetInstances",
|
402
|
+
"ec2:DescribeSpotFleetRequests",
|
403
|
+
"ec2:ModifySpotFleetRequest",
|
404
|
+
"ec2:TerminateInstances",
|
405
|
+
"ecs:ListTasks"
|
406
|
+
],
|
407
|
+
"Resource": "*"
|
408
|
+
},
|
409
|
+
{
|
410
|
+
"Effect": "Allow",
|
411
|
+
"Action": [
|
412
|
+
"ecs:DescribeServices",
|
413
|
+
"ecs:UpdateService"
|
414
|
+
],
|
415
|
+
"Resource": [
|
416
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:service/ecs-cluster-for-worker/repro-worker-production"
|
417
|
+
]
|
418
|
+
},
|
419
|
+
{
|
420
|
+
"Effect": "Allow",
|
421
|
+
"Action": [
|
422
|
+
"ecs:DescribeTasks"
|
423
|
+
],
|
424
|
+
"Resource": [
|
425
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:task/ecs-cluster-for-worker/*"
|
426
|
+
]
|
427
|
+
},
|
428
|
+
{
|
429
|
+
"Effect": "Allow",
|
430
|
+
"Action": [
|
431
|
+
"ecs:DescribeContainerInstances",
|
432
|
+
"ecs:UpdateContainerInstancesState"
|
433
|
+
],
|
434
|
+
"Resource": [
|
435
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:container-instance/ecs-cluster-for-worker/*"
|
436
|
+
]
|
437
|
+
},
|
438
|
+
{
|
439
|
+
"Effect": "Allow",
|
440
|
+
"Action": [
|
441
|
+
"ecs:ListContainerInstances"
|
442
|
+
],
|
443
|
+
"Resource": [
|
444
|
+
"arn:aws:ecs:ap-northeast-1:<account-id>:cluster/ecs-cluster-for-worker"
|
445
|
+
]
|
446
|
+
}
|
447
|
+
]
|
448
|
+
}
|
449
|
+
```
|
450
|
+
|
451
|
+
### How to deploy faster with Auto Scaling Group
|
452
|
+
|
453
|
+
Add following configuration to your deploy.rb and hooks if you need.
|
454
|
+
|
455
|
+
```ruby
|
456
|
+
# deploy.rb
|
457
|
+
set :ecs_instance_fluctuation_manager_configs, [
|
458
|
+
{
|
459
|
+
region: "ap-northeast-1",
|
460
|
+
cluster: "CLUSTER_NAME",
|
461
|
+
auto_scaling_group_name: "AUTO_SCALING_GROUP_NAME",
|
462
|
+
desired_capacity: 20, # original capacity of auto scaling group
|
463
|
+
}
|
464
|
+
]
|
465
|
+
```
|
466
|
+
|
467
|
+
This configuration enables tasks `ecs:increase_instances_to_max_size` and `ecs:terminate_redundant_instances`.
|
468
|
+
If this configuration is not set, the above tasks do nothing.
|
469
|
+
The task `ecs:increase_instances_to_max_size` will increase ECS instances.
|
470
|
+
The task `ecs:terminate_redundant_instances` will decrease ECS instances considering AZ balance.
|
471
|
+
|
472
|
+
Hook configuration example:
|
473
|
+
|
474
|
+
```ruby
|
475
|
+
after "deploy:updating", "ecs:increase_instances_to_max_size"
|
476
|
+
after "deploy:finished", "ecs:terminate_redundant_instances"
|
477
|
+
after "deploy:failed", "ecs:terminate_redundant_instances"
|
478
|
+
```
|
479
|
+
|
234
480
|
## Development
|
235
481
|
|
236
482
|
After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|