simplygenius-atmos 0.9.4 → 0.10.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +30 -0
- data/README.md +15 -15
- data/lib/simplygenius/atmos/cli.rb +14 -3
- data/lib/simplygenius/atmos/commands/container.rb +110 -1
- data/lib/simplygenius/atmos/commands/secret.rb +5 -1
- data/lib/simplygenius/atmos/config.rb +5 -42
- data/lib/simplygenius/atmos/exceptions.rb +4 -0
- data/lib/simplygenius/atmos/generator.rb +1 -1
- data/lib/simplygenius/atmos/plugin.rb +5 -6
- data/lib/simplygenius/atmos/plugin_manager.rb +1 -1
- data/lib/simplygenius/atmos/plugins/core_plugin.rb +23 -0
- data/lib/simplygenius/atmos/plugins/json_diff.rb +95 -0
- data/lib/simplygenius/atmos/plugins/lock_detection.rb +37 -0
- data/lib/simplygenius/atmos/plugins/plan_summary.rb +45 -0
- data/lib/simplygenius/atmos/plugins.rb +11 -0
- data/lib/simplygenius/atmos/providers/aws/container_manager.rb +62 -0
- data/lib/simplygenius/atmos/providers/aws/provider.rb +10 -1
- data/lib/simplygenius/atmos/providers/aws/s3_secret_manager.rb +6 -3
- data/lib/simplygenius/atmos/providers/aws/ssm_secret_manager.rb +70 -0
- data/lib/simplygenius/atmos/settings_hash.rb +114 -0
- data/lib/simplygenius/atmos/version.rb +1 -1
- data/templates/new/config/atmos/runtime.yml +14 -13
- data/templates/new/config/atmos.yml +8 -1
- metadata +65 -17
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 150cd6186165ef42f3e4375c403b83e2ad3e7151e3650c723502bc65200f5938
|
4
|
+
data.tar.gz: 7e0272eede1da6dcf9a44dbce9ca5b626ede5f2ca4b74d7d0163d7cdb7dd2a70
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f193baa7ed6a0d9c817aaefb3247f0b2e79e9f7a3c99ecf237240ffba37aef07af883fe4e01a01a2a1d94031f7626f7d668625c9f8af62b00bf18938b0114fca
|
7
|
+
data.tar.gz: 59560c7e104520aab60a5363508960f55b4a516b859b74c4d1740d3fc3bb8dce1a7dd7607811b5a4383da756f7693965d252d146d2b9a68253e2e91faa0ecbe0
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,33 @@
|
|
1
|
+
0.10.0 (09/13/2019)
|
2
|
+
-------------------
|
3
|
+
|
4
|
+
#### Notes on breaking changes
|
5
|
+
|
6
|
+
* Made AWS SSM Parameter Store the default secrets store for atmos. The s3 secrets will still work unless you overwrite your provider/aws.yml with the changes
|
7
|
+
* Interpolations in the yml config now look up from the top level only if missing in the current level (hashes of hashes). Previously it looked up values from the top level before the current level, which was causing a number of issues. One can force a lookup from the top level by prefixing the key in the interpolations with `_root_`, e.g. The [config/provider/aws.yml](https://github.com/simplygenius/atmos-recipes/blob/master/aws/scaffold/config/atmos/providers/aws.yml#L20) in the atmos aws scaffold. You will know you forgot to do this when you get s circular reference error when running atmos.
|
8
|
+
* Atmos pro has been merged into core Atmos, so you should stop using of the atmos-pro-recipes repository as well as the atmos-pro-plugins gem as they will go away soon.
|
9
|
+
|
10
|
+
#### Full changelog
|
11
|
+
* add ability to push or activate instead of always doing both as deploy does [dfac7ad](https://github.com/simplygenius/atmos/commit/dfac7ad)
|
12
|
+
* add ruby 2.6 [a3b8aff](https://github.com/simplygenius/atmos/commit/a3b8aff)
|
13
|
+
* add ruby 2.6 [db3469f](https://github.com/simplygenius/atmos/commit/db3469f)
|
14
|
+
* running deploy as deployer user needs the deployer specific role to be specified [06a34b0](https://github.com/simplygenius/atmos/commit/06a34b0)
|
15
|
+
* move client out of ctor to fix tests [b9d8c02](https://github.com/simplygenius/atmos/commit/b9d8c02)
|
16
|
+
* make s3 consistent with ssm for trying to set an existing secret add force option when setting secret to cause ssm (and s3) to overwrite existing secret [b05f189](https://github.com/simplygenius/atmos/commit/b05f189)
|
17
|
+
* add aws ssm for secret management, update aws gem version dependencies [8cb4350](https://github.com/simplygenius/atmos/commit/8cb4350)
|
18
|
+
* Merge pull request #3 from nirvdrum/readme-improvements [d82710b](https://github.com/simplygenius/atmos/commit/d82710b)
|
19
|
+
* interpolations in config should only lookup values from root only if they don't exit in current level of hash. One can force a root lookup with the _root_ prefix. This is a breaking change [cc925d3](https://github.com/simplygenius/atmos/commit/cc925d3)
|
20
|
+
* Minor README clean-ups. [16778b1](https://github.com/simplygenius/atmos/commit/16778b1)
|
21
|
+
* Homebrew is now available on Linux, so no need to limit the docs to just macOS. [6885a35](https://github.com/simplygenius/atmos/commit/6885a35)
|
22
|
+
* Update the package names to be installed via Homebrew. [68f3cde](https://github.com/simplygenius/atmos/commit/68f3cde)
|
23
|
+
* friendlier output for cycles in interpolation, better exception handling [dea7766](https://github.com/simplygenius/atmos/commit/dea7766)
|
24
|
+
* fix whitespace [870f4c2](https://github.com/simplygenius/atmos/commit/870f4c2)
|
25
|
+
* allow running multiple organizations from a single ops account [baef6c3](https://github.com/simplygenius/atmos/commit/baef6c3)
|
26
|
+
* Fix config interpolation to allow one path to refer to another that also needs interpolation [202fd85](https://github.com/simplygenius/atmos/commit/202fd85)
|
27
|
+
* Inline all the atmos pro functionality/recipes to make atmos fully open source [b906d7e](https://github.com/simplygenius/atmos/commit/b906d7e)
|
28
|
+
* mention setting of secret for example app [382db44](https://github.com/simplygenius/atmos/commit/382db44)
|
29
|
+
* upgrade bundler [667c7a6](https://github.com/simplygenius/atmos/commit/667c7a6)
|
30
|
+
|
1
31
|
0.9.4 (03/20/2019)
|
2
32
|
------------------
|
3
33
|
|
data/README.md
CHANGED
@@ -6,7 +6,7 @@
|
|
6
6
|
|
7
7
|
Atmos(phere) - Breathe easier with terraform
|
8
8
|
|
9
|
-
Atmos provides a layer of organization on top of terraform for creating cloud system architectures with Amazon Web Services. It handles the plumbing so you can focus on your application. The core atmos runtime is free and open-source, with a business friendly license (Apache). It provides some basic recipes to help get you going with a service oriented architecture implemented with AWS Elastic Container Services.
|
9
|
+
Atmos provides a layer of organization on top of terraform for creating cloud system architectures with Amazon Web Services. It handles the plumbing so you can focus on your application. The core atmos runtime is free and open-source, with a business friendly license (Apache). It provides some basic recipes to help get you going with a service oriented architecture implemented with AWS Elastic Container Services.
|
10
10
|
|
11
11
|
In a nutshell, you provide the green, atmos delivers the rest:
|
12
12
|
|
@@ -14,7 +14,7 @@ In a nutshell, you provide the green, atmos delivers the rest:
|
|
14
14
|
|
15
15
|
## Goals
|
16
16
|
|
17
|
-
* The whole is greater than the sum of its parts. Assist in creating a cloud infrastructure _system_ rather than just discrete infrastructure components. Learning aws and terraform is a lot to bite off when getting started. It's much easier to start with a working system
|
17
|
+
* The whole is greater than the sum of its parts. Assist in creating a cloud infrastructure _system_ rather than just discrete infrastructure components. Learning aws and terraform is a lot to bite off when getting started. It's much easier to start with a working system, and learn incrementally as you go by making changes to it.
|
18
18
|
|
19
19
|
* The command line is king. Using a CLI to iterate on and manage core infrastructure has always been more effective for me, so I aim to make things as convenient and usable as possible from there.
|
20
20
|
|
@@ -39,8 +39,8 @@ In a nutshell, you provide the green, atmos delivers the rest:
|
|
39
39
|
|
40
40
|
First install the dependencies:
|
41
41
|
* [Install docker](https://www.docker.com/community-edition) for deploying containers
|
42
|
-
* Install terraform (optional if running atmos as a docker image): e.g. `brew install terraform` on
|
43
|
-
* Install the aws cli (optional, useful for managing aws credentials): e.g. `brew install
|
42
|
+
* Install terraform (optional if running atmos as a docker image): e.g. `brew install terraform@0.11` on macOS or Linux
|
43
|
+
* Install the aws cli (optional, useful for managing aws credentials): e.g. `brew install awscli` on macOS or Linux
|
44
44
|
|
45
45
|
Then install atmos:
|
46
46
|
|
@@ -107,11 +107,11 @@ aws configure --profile <user_profile_name>
|
|
107
107
|
|
108
108
|
If you supply the "-m" flag, it will automatically create and activate a virtual MFA device with the user, and prompt you to save the secret to the atmos mfa keystore for integrated usage. You can skip saving the secret and instead just copy/paste it into your MFA device of choice. The "user create" command can also act in more of an upsert fashion, so to do something like reset a user's password and keys, you could do `atmos user create --force -l -m -k your@email.address`
|
109
109
|
|
110
|
-
Login to the aws console as that user, change your password and setup MFA there if you prefer doing it that way. Make sure you log out and back in again with MFA before you try setting up the [role switcher](#per-user-role-switcher-in-console)
|
110
|
+
Login to the aws console as that user, change your password and setup MFA there if you prefer doing it that way. Make sure you log out and back in again with MFA before you try setting up the [role switcher](#per-user-role-switcher-in-console).
|
111
111
|
|
112
|
-
Now that a non-root user is created, you should be able to do everything as that user, so you can remove the root access keys if desired. Keeping them around can be useful though, as there are some AWS operations that can only be done as the root user. Leaving them in your shared credential store, but deactivating them in the AWS console
|
112
|
+
Now that a non-root user is created, you should be able to do everything as that user, so you can remove the root access keys if desired. Keeping them around can be useful though, as there are some AWS operations that can only be done as the root user. Leaving them in your shared credential store, but deactivating them in the AWS console until needed is a reasonable compromise.
|
113
113
|
|
114
|
-
While you can do everything in a single account,
|
114
|
+
While you can do everything in a single account, I've found a better practice is to use a new account for each env (dev, staging, prod, etc), and leave the ops account providing authentication duties and acting as a jumping off point to the others. This allows for easier role/permission management down the line as well as better isolation between environments, thereby enabling safe iteration in dev environments without risking production.
|
115
115
|
|
116
116
|
Create a new `dev` account, and bootstrap it to work with atmos
|
117
117
|
|
@@ -127,6 +127,10 @@ Use the 'aws/service' template to setup an ECS Fargate based service, then apply
|
|
127
127
|
|
128
128
|
```
|
129
129
|
atmos generate --force aws/service
|
130
|
+
# If you setup a db for your service, add its password to the secret store.
|
131
|
+
# Otherwise the service container will fail to start if it is using the
|
132
|
+
# ATMOS_SECRET_KEYS mechanism like the example app is using.
|
133
|
+
# atmos -e dev secret set service_<service_name>_db_password sekret!
|
130
134
|
atmos -e dev apply
|
131
135
|
|
132
136
|
```
|
@@ -146,10 +150,10 @@ Then use atmos to push and deploy that image to the ECR repo:
|
|
146
150
|
atmos -e dev container deploy -c services <service_name>
|
147
151
|
```
|
148
152
|
|
149
|
-
The atmos aws scaffold also sets up a user named
|
153
|
+
The atmos aws scaffold also sets up a user named _deployer_, with restricted permissions sufficient to do the deploy. Add the [key/secret](https://github.com/simplygenius/atmos-recipes/blob/master/aws/scaffold/recipes/atmos-permissions.tf#L159) to the environment for your CI to get your CI to auto-deploy on successful build.
|
150
154
|
|
151
155
|
```
|
152
|
-
AWS_ACCESS_KEY_ID=<deployer_key> AWS_SECRET_ACCESS_KEY=<deployer_secret> atmos -e <env_based_on_branch> container deploy -c services <service_name>
|
156
|
+
AWS_ACCESS_KEY_ID=<deployer_key> AWS_SECRET_ACCESS_KEY=<deployer_secret> atmos -e <env_based_on_branch> container deploy -r <env>-deployer -c services <service_name>
|
153
157
|
```
|
154
158
|
|
155
159
|
To clean it all up:
|
@@ -188,14 +192,14 @@ These are separate commands so that day-day usage where you want to tear down ev
|
|
188
192
|
|
189
193
|
If you are following the account-per-environment pattern, you will need to setup a role switcher for each account in the AWS console for your user. The AWS console seems to store these in cookies, so if you make a mistake its easy to fix by clearing them. First, login to the AWS console with your personal aws user that was created in the ops account. Select the dropdown with your email at top right of the page, Switch Role. Fill the details for the environment you want to be able to access from the console:
|
190
194
|
|
191
|
-
* Account number for the environment (
|
195
|
+
* Account number for the environment (see environments->`<env>`-> account_id in `config/atmos.yml`).
|
192
196
|
* Role `<env>-admin` - this is the role you assume in the destination account.
|
193
197
|
* Pick a name (e.g. DevAdmin)
|
194
198
|
* Pick a color that you like (e.g. Red=production, Yellow=staging, Green=dev)
|
195
199
|
|
196
200
|
## Managing secrets
|
197
201
|
|
198
|
-
Secrets are stored in a
|
202
|
+
Secrets are stored in a S3 bucket unique to each environment, and automatically passed into terraform when it is executed by atmos. The secret key should be the same as a terraform variable name defined in your terraform recipes, and if the secret exists, it will override whatever default value you have setup for the terraform variable.
|
199
203
|
|
200
204
|
To set a secret:
|
201
205
|
|
@@ -213,7 +217,3 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/simply
|
|
213
217
|
## License
|
214
218
|
|
215
219
|
The gem is available as open source under the terms of the [Apache 2.0 License](https://opensource.org/licenses/apache-2.0).
|
216
|
-
|
217
|
-
# About Us
|
218
|
-
|
219
|
-
[Simply Genius LLC](https://simplygenius.com) is an independently run organization in Boston, MA USA. Its Chief Everything is Matt Conway, a software engineer and executive with more than 20 years of experience in the Boston Tech Startup scene. Atmos is his attempt at providing the world with the same tools, techniques, mindset and philosophy that he strives for when building a system architecture for a new startup.
|
@@ -1,6 +1,5 @@
|
|
1
1
|
require_relative '../atmos'
|
2
2
|
require_relative '../atmos/ui'
|
3
|
-
require_relative '../atmos/plugins/prompt_notify'
|
4
3
|
require 'clamp'
|
5
4
|
require 'sigdump/setup'
|
6
5
|
|
@@ -44,7 +43,7 @@ module SimplyGenius
|
|
44
43
|
'GROUP', "The atmos working group\n for selecting recipe groups\n",
|
45
44
|
default: 'default'
|
46
45
|
|
47
|
-
option ["-
|
46
|
+
option ["-p", "--load-path"],
|
48
47
|
"PATH", "adds additional paths to ruby load path",
|
49
48
|
multivalued: true
|
50
49
|
|
@@ -109,11 +108,23 @@ module SimplyGenius
|
|
109
108
|
UI.color_enabled = color?
|
110
109
|
|
111
110
|
Atmos.config.add_user_load_path(*load_path_list)
|
112
|
-
Atmos.config.plugin_manager.register_output_filter(:stdout, Plugins::PromptNotify)
|
113
111
|
Atmos.config.plugin_manager.load_plugins
|
114
112
|
end
|
115
113
|
end
|
116
114
|
|
115
|
+
# Hook into clamp lifecycle to globally handle errors
|
116
|
+
class << self
|
117
|
+
def run(*args, **opts, &blk)
|
118
|
+
begin
|
119
|
+
super
|
120
|
+
rescue Exception => e
|
121
|
+
logger.log_exception(e, "Unhandled exception", level: :debug)
|
122
|
+
logger.error(e.message)
|
123
|
+
exit!
|
124
|
+
end
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
117
128
|
end
|
118
129
|
|
119
130
|
end
|
@@ -1,5 +1,6 @@
|
|
1
1
|
require_relative 'base_command'
|
2
2
|
require_relative '../../atmos/settings_hash'
|
3
|
+
require_relative '../../atmos/ui'
|
3
4
|
require 'climate_control'
|
4
5
|
|
5
6
|
module SimplyGenius
|
@@ -7,12 +8,13 @@ module SimplyGenius
|
|
7
8
|
module Commands
|
8
9
|
|
9
10
|
class Container < BaseCommand
|
11
|
+
include UI
|
10
12
|
|
11
13
|
def self.description
|
12
14
|
"Manages containers in the cloud provider"
|
13
15
|
end
|
14
16
|
|
15
|
-
subcommand "
|
17
|
+
subcommand "push", "Only push a container image without activating it" do
|
16
18
|
|
17
19
|
option ["-c", "--cluster"],
|
18
20
|
"CLUSTER", "The cluster name\n",
|
@@ -35,6 +37,113 @@ module SimplyGenius
|
|
35
37
|
end
|
36
38
|
|
37
39
|
def execute
|
40
|
+
Atmos.config.provider.auth_manager.authenticate(ENV, role: role) do |auth_env|
|
41
|
+
ClimateControl.modify(auth_env) do
|
42
|
+
mgr = Atmos.config.provider.container_manager
|
43
|
+
|
44
|
+
primary_name = name_list.first
|
45
|
+
|
46
|
+
result = mgr.push(primary_name, image, revision: revision)
|
47
|
+
|
48
|
+
logger.info "Container pushed:\n #{display result}"
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
53
|
+
end
|
54
|
+
|
55
|
+
subcommand "activate", "Activate a container that has already been pushed" do
|
56
|
+
|
57
|
+
option ["-c", "--cluster"],
|
58
|
+
"CLUSTER", "The cluster name\n",
|
59
|
+
required: true
|
60
|
+
|
61
|
+
option ["-r", "--role"],
|
62
|
+
"ROLE", "The role to assume when deploying\n"
|
63
|
+
|
64
|
+
option ["-v", "--revision"],
|
65
|
+
"REVISION", "Use the given revision of the pushed image to activate\n"
|
66
|
+
|
67
|
+
option ["-l", "--list"],
|
68
|
+
:flag, "List the most recent pushed images\n"
|
69
|
+
|
70
|
+
option ["-t", "--tagcount"],
|
71
|
+
"N", "Only show the last N items when listing\n",
|
72
|
+
default: 10 do |s|
|
73
|
+
Integer(s)
|
74
|
+
end
|
75
|
+
|
76
|
+
parameter "NAME ...",
|
77
|
+
"The name of the service (or task) to activate\nWhen multiple, the first is the primary, and\nthe rest get activated with its image"
|
78
|
+
|
79
|
+
def execute
|
80
|
+
Atmos.config.provider.auth_manager.authenticate(ENV, role: role) do |auth_env|
|
81
|
+
ClimateControl.modify(auth_env) do
|
82
|
+
mgr = Atmos.config.provider.container_manager
|
83
|
+
primary_name = name_list.first
|
84
|
+
|
85
|
+
if list?
|
86
|
+
tags = mgr.list_image_tags(cluster, primary_name)
|
87
|
+
logger.info "Recent revisions:\n"
|
88
|
+
tags[:tags].last(tagcount).each {|t| logger.info "\t#{t}"}
|
89
|
+
logger.info("")
|
90
|
+
logger.info("Currently active revision is: #{tags[:current]}") if tags[:current]
|
91
|
+
logger.info("The revision associated with the `latest` tag is: #{tags[:latest]}") if tags[:latest]
|
92
|
+
return
|
93
|
+
end
|
94
|
+
|
95
|
+
rev = revision
|
96
|
+
if rev.nil? && !list?
|
97
|
+
tags = mgr.list_image_tags(cluster, primary_name)
|
98
|
+
logger.info("Currently active revision is: #{tags[:current]}") if tags[:current]
|
99
|
+
logger.info("The revision associated with the `latest` tag is: #{tags[:latest]}") if tags[:latest]
|
100
|
+
logger.info("")
|
101
|
+
|
102
|
+
rev = choose do |menu|
|
103
|
+
menu.prompt = "Which revision would you like to activate? "
|
104
|
+
menu.choices(* tags[:tags].last(tagcount))
|
105
|
+
end
|
106
|
+
end
|
107
|
+
|
108
|
+
remote_image = mgr.remote_image(primary_name, rev)
|
109
|
+
result = {}
|
110
|
+
name_list.each do |name|
|
111
|
+
resp = mgr.deploy(cluster, name, remote_image)
|
112
|
+
result[:task_definitions] ||= []
|
113
|
+
result[:task_definitions] << resp[:task_definition]
|
114
|
+
end
|
115
|
+
|
116
|
+
logger.info "Container activated:\n #{display result}"
|
117
|
+
end
|
118
|
+
end
|
119
|
+
end
|
120
|
+
|
121
|
+
end
|
122
|
+
|
123
|
+
subcommand "deploy", "Push and activate a container" do
|
124
|
+
|
125
|
+
option ["-c", "--cluster"],
|
126
|
+
"CLUSTER", "The cluster name\n",
|
127
|
+
required: true
|
128
|
+
|
129
|
+
option ["-r", "--role"],
|
130
|
+
"ROLE", "The role to assume when deploying\n"
|
131
|
+
|
132
|
+
option ["-i", "--image"],
|
133
|
+
"IMAGE", "The local container image to deploy\nDefaults to service/task name"
|
134
|
+
|
135
|
+
option ["-v", "--revision"],
|
136
|
+
"REVISION", "Use as the remote image revision\n"
|
137
|
+
|
138
|
+
parameter "NAME ...",
|
139
|
+
"The name of the service (or task) to deploy\nWhen multiple, the first is the primary, and\nthe rest get deployed with its image"
|
140
|
+
|
141
|
+
def default_image
|
142
|
+
name_list.first
|
143
|
+
end
|
144
|
+
|
145
|
+
def execute
|
146
|
+
# TODO: use local_name_prefix for cluster name and repo/service name?
|
38
147
|
Atmos.config.provider.auth_manager.authenticate(ENV, role: role) do |auth_env|
|
39
148
|
ClimateControl.modify(auth_env) do
|
40
149
|
mgr = Atmos.config.provider.container_manager
|
@@ -31,6 +31,10 @@ module SimplyGenius
|
|
31
31
|
|
32
32
|
subcommand "set", "Sets the secret value" do
|
33
33
|
|
34
|
+
option ["-f", "--force"],
|
35
|
+
:flag, "forces updates for pre-existing secret\n",
|
36
|
+
default: false
|
37
|
+
|
34
38
|
parameter "KEY",
|
35
39
|
"The secret key"
|
36
40
|
|
@@ -41,7 +45,7 @@ module SimplyGenius
|
|
41
45
|
|
42
46
|
Atmos.config.provider.auth_manager.authenticate(ENV) do |auth_env|
|
43
47
|
ClimateControl.modify(auth_env) do
|
44
|
-
Atmos.config.provider.secret_manager.set(key, value)
|
48
|
+
Atmos.config.provider.secret_manager.set(key, value, force: force?)
|
45
49
|
logger.info "Secret set for #{key}"
|
46
50
|
end
|
47
51
|
end
|
@@ -212,7 +212,6 @@ module SimplyGenius
|
|
212
212
|
|
213
213
|
logger.debug("Atmos env: #{atmos_env}")
|
214
214
|
|
215
|
-
|
216
215
|
if ! File.exist?(config_file)
|
217
216
|
logger.warn "Could not find an atmos config file at: #{config_file}"
|
218
217
|
@full_config = SettingsHash.new
|
@@ -237,7 +236,11 @@ module SimplyGenius
|
|
237
236
|
atmos_working_group: working_group,
|
238
237
|
atmos_version: VERSION
|
239
238
|
}, ["builtins"])
|
240
|
-
|
239
|
+
|
240
|
+
conf.error_resolver = ->(statement) { find_config_error(statement) }
|
241
|
+
conf.enable_expansion = true
|
242
|
+
conf
|
243
|
+
|
241
244
|
end
|
242
245
|
end
|
243
246
|
|
@@ -260,46 +263,6 @@ module SimplyGenius
|
|
260
263
|
config
|
261
264
|
end
|
262
265
|
|
263
|
-
def expand(config, obj)
|
264
|
-
case obj
|
265
|
-
when Hash
|
266
|
-
SettingsHash.new(Hash[obj.collect {|k, v| [k, expand(config, v)] }])
|
267
|
-
when Array
|
268
|
-
result = obj.collect {|i| expand(config, i) }
|
269
|
-
# HACK: accounting for the case when someone wants to force an override using '^' as the first list item, when
|
270
|
-
# there is no upstream to override (i.e. merge proc doesn't get triggered as key is unique, so just added verbatim)
|
271
|
-
result.delete_at(0) if result[0] == "^"
|
272
|
-
result
|
273
|
-
when String
|
274
|
-
result = obj
|
275
|
-
result.scan(INTERP_PATTERN).each do |substr, statement|
|
276
|
-
# TODO: check for cycles
|
277
|
-
if statement =~ /^[\w\.\[\]]$/
|
278
|
-
val = config.notation_get(statement)
|
279
|
-
else
|
280
|
-
# TODO: be consistent with dot notation between eval and
|
281
|
-
# notation_get. eval ends up calling Hashie method_missing,
|
282
|
-
# which returns nil if a key doesn't exist, causing a nil
|
283
|
-
# exception for next item in chain, while notation_get returns
|
284
|
-
# nil gracefully for the entire chain (preferred)
|
285
|
-
begin
|
286
|
-
val = eval(statement, config.instance_eval("binding"))
|
287
|
-
rescue => e
|
288
|
-
file, line = find_config_error(substr)
|
289
|
-
file_msg = file.nil? ? "" : " in #{File.basename(file)}:#{line}"
|
290
|
-
raise RuntimeError.new("Failing config statement '#{substr}'#{file_msg} => #{e.class} #{e.message}")
|
291
|
-
end
|
292
|
-
end
|
293
|
-
result = result.sub(substr, expand(config, val).to_s)
|
294
|
-
end
|
295
|
-
result = true if result == 'true'
|
296
|
-
result = false if result == 'false'
|
297
|
-
result
|
298
|
-
else
|
299
|
-
obj
|
300
|
-
end
|
301
|
-
end
|
302
|
-
|
303
266
|
def find_config_error(statement)
|
304
267
|
filename = nil
|
305
268
|
line = 0
|
@@ -1,4 +1,5 @@
|
|
1
1
|
require_relative '../atmos'
|
2
|
+
require 'clamp'
|
2
3
|
|
3
4
|
module SimplyGenius
|
4
5
|
module Atmos
|
@@ -6,6 +7,9 @@ module SimplyGenius
|
|
6
7
|
module Exceptions
|
7
8
|
class UsageError < Clamp::UsageError
|
8
9
|
|
10
|
+
end
|
11
|
+
class ConfigInterpolationError < ::ScriptError
|
12
|
+
|
9
13
|
end
|
10
14
|
end
|
11
15
|
|
@@ -136,7 +136,7 @@ module SimplyGenius
|
|
136
136
|
end
|
137
137
|
|
138
138
|
def track_context(varname, value)
|
139
|
-
varname.blank? || value.nil? ? nil: tmpl.scoped_context[varname] = value
|
139
|
+
varname.blank? || value.nil? ? nil : tmpl.scoped_context[varname] = value
|
140
140
|
end
|
141
141
|
|
142
142
|
def respond_to_missing?(method_name, *args)
|
@@ -1,9 +1,7 @@
|
|
1
1
|
require_relative '../atmos'
|
2
2
|
require 'active_support/core_ext/class'
|
3
3
|
|
4
|
-
|
5
|
-
require_relative "plugins/#{File.basename(f).sub(/\.rb$/, "")}"
|
6
|
-
end
|
4
|
+
require_relative "plugins/output_filter"
|
7
5
|
|
8
6
|
module SimplyGenius
|
9
7
|
module Atmos
|
@@ -11,14 +9,15 @@ module SimplyGenius
|
|
11
9
|
class Plugin
|
12
10
|
include GemLogger::LoggerSupport
|
13
11
|
|
14
|
-
attr_reader :config
|
12
|
+
attr_reader :plugin_manager, :config
|
15
13
|
|
16
|
-
def initialize(config)
|
14
|
+
def initialize(plugin_manager, config)
|
15
|
+
@plugin_manager = plugin_manager
|
17
16
|
@config = config
|
18
17
|
end
|
19
18
|
|
20
19
|
def register_output_filter(type, clazz)
|
21
|
-
|
20
|
+
plugin_manager.register_output_filter(type, clazz)
|
22
21
|
end
|
23
22
|
|
24
23
|
end
|
@@ -45,7 +45,7 @@ module SimplyGenius
|
|
45
45
|
begin
|
46
46
|
if ! @plugin_classes.include?(plugin_class)
|
47
47
|
@plugin_classes << plugin_class
|
48
|
-
@plugin_instances << plugin_class.new(plugin)
|
48
|
+
@plugin_instances << plugin_class.new(self, plugin)
|
49
49
|
end
|
50
50
|
rescue StandardError => e
|
51
51
|
logger.log_exception e, "Failed to initialize plugin: #{plugin_class}"
|
@@ -0,0 +1,23 @@
|
|
1
|
+
require_relative '../../atmos'
|
2
|
+
require_relative "prompt_notify"
|
3
|
+
require_relative "lock_detection"
|
4
|
+
require_relative "plan_summary"
|
5
|
+
require_relative "json_diff"
|
6
|
+
|
7
|
+
module SimplyGenius
|
8
|
+
module Atmos
|
9
|
+
module Plugins
|
10
|
+
|
11
|
+
class CorePlugin < SimplyGenius::Atmos::Plugin
|
12
|
+
def initialize(plugin_manager, config)
|
13
|
+
super
|
14
|
+
register_output_filter(:stdout, Plugins::PromptNotify) unless config[:disable_prompt_notify]
|
15
|
+
register_output_filter(:stderr, Plugins::LockDetection) unless config[:disable_lock_detection]
|
16
|
+
register_output_filter(:stdout, Plugins::PlanSummary) unless config[:disable_plan_summary]
|
17
|
+
register_output_filter(:stdout, Plugins::JsonDiff) unless config[:disable_json_diff]
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
@@ -0,0 +1,95 @@
|
|
1
|
+
require_relative '../../atmos'
|
2
|
+
require_relative 'output_filter'
|
3
|
+
require 'deepsort'
|
4
|
+
require 'diffy'
|
5
|
+
require 'json'
|
6
|
+
require 'yaml'
|
7
|
+
|
8
|
+
module SimplyGenius
|
9
|
+
module Atmos
|
10
|
+
module Plugins
|
11
|
+
|
12
|
+
class JsonDiff < SimplyGenius::Atmos::Plugins::OutputFilter
|
13
|
+
|
14
|
+
def initialize(context)
|
15
|
+
super
|
16
|
+
@plan_detected = false
|
17
|
+
@json_data = ""
|
18
|
+
end
|
19
|
+
|
20
|
+
def filter(data, flushing: false)
|
21
|
+
|
22
|
+
# If we are flushing and never saw a json end, then flush the data we have buffered
|
23
|
+
if flushing && @saving_json
|
24
|
+
buffer = @json_data + data
|
25
|
+
@json_data = ""
|
26
|
+
return buffer
|
27
|
+
end
|
28
|
+
|
29
|
+
# TODO: roll up plan detection so we don't have many plugins doing the same regexp match
|
30
|
+
if data =~ /^[\e\[\dm\s]*Terraform will perform the following actions:[\e\[\dm\s]*$/
|
31
|
+
@plan_detected = true
|
32
|
+
end
|
33
|
+
|
34
|
+
if @plan_detected
|
35
|
+
|
36
|
+
if data =~ /^.*:\s*"[\[\{]/
|
37
|
+
@saving_json = true
|
38
|
+
end
|
39
|
+
|
40
|
+
if @saving_json
|
41
|
+
@json_data << data
|
42
|
+
|
43
|
+
if data =~ /[\]\}][\s\\n]*[^\\]"[^"]*$/
|
44
|
+
@saving_json = false
|
45
|
+
with_diff = @json_data.sub(/^(.*:\s*)"([\[\{].*[\]\}])[\s\\n]*"\s*=>\s*"([\[\{].*[\]\}])[\s\\n]*"(.*)$/) do |m|
|
46
|
+
begin
|
47
|
+
"#{$1}\n#{jsondiff($2, $3)}\n#{$4}"
|
48
|
+
rescue JSON::ParserError => e
|
49
|
+
logger.warn("Failed to parse JSON for diff: #{e.message}")
|
50
|
+
"#{$1}\n\n#{$2}\n\n=>\n\n#{$3}\n\n#{$4}"
|
51
|
+
end
|
52
|
+
end
|
53
|
+
@json_data = ""
|
54
|
+
return with_diff
|
55
|
+
end
|
56
|
+
|
57
|
+
return ""
|
58
|
+
end
|
59
|
+
|
60
|
+
end
|
61
|
+
|
62
|
+
data
|
63
|
+
end
|
64
|
+
|
65
|
+
def unescape(s)
|
66
|
+
YAML.load(%Q(---\n"#{s}"\n))
|
67
|
+
end
|
68
|
+
|
69
|
+
def jsondiff(lhs, rhs)
|
70
|
+
lhs = unescape(lhs)
|
71
|
+
rhs = unescape(rhs)
|
72
|
+
|
73
|
+
jl = JSON.parse(lhs).deep_sort
|
74
|
+
jr = JSON.parse(rhs).deep_sort
|
75
|
+
|
76
|
+
sl = JSON.pretty_generate(jl)
|
77
|
+
sr = JSON.pretty_generate(jr)
|
78
|
+
|
79
|
+
if sl == sr
|
80
|
+
result = "No differences"
|
81
|
+
else
|
82
|
+
result = Diffy::Diff.new("#{sl}\n", "#{sr}\n").to_s
|
83
|
+
end
|
84
|
+
|
85
|
+
result
|
86
|
+
end
|
87
|
+
|
88
|
+
|
89
|
+
end
|
90
|
+
|
91
|
+
end
|
92
|
+
end
|
93
|
+
end
|
94
|
+
|
95
|
+
|
@@ -0,0 +1,37 @@
|
|
1
|
+
require_relative '../../atmos'
|
2
|
+
require_relative 'output_filter'
|
3
|
+
require_relative '../../atmos/terraform_executor'
|
4
|
+
|
5
|
+
module SimplyGenius
|
6
|
+
module Atmos
|
7
|
+
module Plugins
|
8
|
+
|
9
|
+
class LockDetection < SimplyGenius::Atmos::Plugins::OutputFilter
|
10
|
+
|
11
|
+
def filter(data, flushing: false)
|
12
|
+
if data =~ /^[\e\[\dm\s]*Lock Info:[\e\[\dm\s]*$/
|
13
|
+
@lock_detected = true
|
14
|
+
end
|
15
|
+
if data =~ /^[\e\[\dm\s]*ID:\s*([a-f0-9\-]+)[\e\[\dm\s]*$/
|
16
|
+
@lock_id = $1
|
17
|
+
end
|
18
|
+
data
|
19
|
+
end
|
20
|
+
|
21
|
+
def close
|
22
|
+
if @lock_detected && @lock_id.present?
|
23
|
+
clear_lock = agree("Terraform lock detected, would you like to clear it? ") {|q| q.default = 'n' }
|
24
|
+
if clear_lock
|
25
|
+
logger.info "Clearing terraform lock with id: #{@lock_id}"
|
26
|
+
te = TerraformExecutor.new(process_env: context[:process_env])
|
27
|
+
te.run("force-unlock", "-force", @lock_id)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
end
|
33
|
+
|
34
|
+
end
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
@@ -0,0 +1,45 @@
|
|
1
|
+
require_relative '../../atmos'
|
2
|
+
require_relative 'output_filter'
|
3
|
+
|
4
|
+
module SimplyGenius
|
5
|
+
module Atmos
|
6
|
+
module Plugins
|
7
|
+
|
8
|
+
class PlanSummary < SimplyGenius::Atmos::Plugins::OutputFilter
|
9
|
+
|
10
|
+
def initialize(context)
|
11
|
+
super
|
12
|
+
@plan_detected = false
|
13
|
+
@summary_data = ""
|
14
|
+
end
|
15
|
+
|
16
|
+
def filter(data, flushing: false)
|
17
|
+
summary_saved = false
|
18
|
+
if data =~ /^[\e\[\dm\s]*Terraform will perform the following actions:[\e\[\dm\s]*$/
|
19
|
+
@plan_detected = true
|
20
|
+
@summary_data = data.sub(/.*Terraform will perform the following actions:[^\n]*\n/m, "")
|
21
|
+
summary_saved = true
|
22
|
+
end
|
23
|
+
|
24
|
+
if @plan_detected
|
25
|
+
@summary_data << data unless summary_saved
|
26
|
+
data.sub!(/^[\e\[\dm\s]*Plan:.*$/) do |m|
|
27
|
+
summary = summarize(@summary_data)
|
28
|
+
m + "\n\n#{summary}\n"
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
data
|
33
|
+
end
|
34
|
+
|
35
|
+
def summarize(data)
|
36
|
+
lines = data.lines.select {|l| l =~ /^[\e\[\dm\s]*[~+\-<]/ }.collect(&:chomp)
|
37
|
+
lines = lines.reject {|l| l =~ /-----/ }
|
38
|
+
"Plan Summary:\n#{lines.join("\n")}"
|
39
|
+
end
|
40
|
+
|
41
|
+
end
|
42
|
+
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
@@ -97,6 +97,68 @@ module SimplyGenius
|
|
97
97
|
return result
|
98
98
|
end
|
99
99
|
|
100
|
+
def remote_image(name, tag)
|
101
|
+
ecr = ::Aws::ECR::Client.new
|
102
|
+
|
103
|
+
resp = ecr.get_authorization_token
|
104
|
+
endpoint = resp.authorization_data.first.proxy_endpoint
|
105
|
+
|
106
|
+
ecs_image="#{endpoint.sub(/https?:\/\//, '')}/#{name}"
|
107
|
+
tagged_image = "#{ecs_image}:#{tag}"
|
108
|
+
|
109
|
+
return tagged_image
|
110
|
+
end
|
111
|
+
|
112
|
+
def list_image_tags(cluster, name)
|
113
|
+
result = {tags: [], latest: nil, current: nil}
|
114
|
+
latest_digest = nil
|
115
|
+
|
116
|
+
ecs = ::Aws::ECS::Client.new
|
117
|
+
ecr = ::Aws::ECR::Client.new
|
118
|
+
|
119
|
+
resp = ecs.describe_services(services: [name], cluster: cluster)
|
120
|
+
if resp.services.size == 1
|
121
|
+
task_def = resp.services.first.task_definition
|
122
|
+
resp = ecs.describe_task_definition(task_definition: task_def)
|
123
|
+
image = resp.task_definition.container_definitions.first.image
|
124
|
+
result[:current] = image.sub(/^.*:/, '')
|
125
|
+
else
|
126
|
+
raise "No services found for '#{name}' in cluster '#{cluster}'"
|
127
|
+
end
|
128
|
+
|
129
|
+
# TODO: handle pagination?
|
130
|
+
resp = ecr.list_images(repository_name: name, filter: {tag_status: "TAGGED"}, max_results: 1000)
|
131
|
+
if resp.image_ids.size > 0
|
132
|
+
|
133
|
+
images = resp.image_ids
|
134
|
+
|
135
|
+
images.each do |i|
|
136
|
+
if i.image_tag == 'latest'
|
137
|
+
latest_digest = i.image_digest
|
138
|
+
end
|
139
|
+
|
140
|
+
result[:tags] << i.image_tag
|
141
|
+
end
|
142
|
+
|
143
|
+
if latest_digest
|
144
|
+
images.each do |i|
|
145
|
+
if i.image_digest == latest_digest && i.image_tag != 'latest'
|
146
|
+
result[:latest] = i.image_tag
|
147
|
+
break
|
148
|
+
end
|
149
|
+
end
|
150
|
+
# Handle if latest tag isn't some other tag
|
151
|
+
result[:latest] = 'latest' if result[:latest].nil?
|
152
|
+
end
|
153
|
+
|
154
|
+
result[:tags].sort!
|
155
|
+
else
|
156
|
+
raise "No images found for '#{name}'"
|
157
|
+
end
|
158
|
+
|
159
|
+
return result
|
160
|
+
end
|
161
|
+
|
100
162
|
private
|
101
163
|
|
102
164
|
def run(*args, **opts)
|
@@ -36,7 +36,16 @@ module SimplyGenius
|
|
36
36
|
|
37
37
|
def secret_manager
|
38
38
|
@secret_manager ||= begin
|
39
|
-
|
39
|
+
conf = Atmos.config[:secret]
|
40
|
+
logger.debug("Secrets config is: #{conf}")
|
41
|
+
manager_type = conf[:type] || "ssm"
|
42
|
+
if manager_type !~ /::/
|
43
|
+
manager_type += "_secret_manager"
|
44
|
+
manager_type = "#{self.class.name.deconstantize}::#{manager_type.camelize}"
|
45
|
+
end
|
46
|
+
manager = manager_type.constantize
|
47
|
+
logger.debug("Using secrets manager #{manager}")
|
48
|
+
manager.new(self)
|
40
49
|
end
|
41
50
|
end
|
42
51
|
|
@@ -11,16 +11,19 @@ module SimplyGenius
|
|
11
11
|
|
12
12
|
def initialize(provider)
|
13
13
|
@provider = provider
|
14
|
-
logger.debug("Secrets config is: #{Atmos.config[:secret]}")
|
15
14
|
@bucket_name = Atmos.config[:secret][:bucket]
|
16
15
|
@bucket_prefix = "#{Atmos.config[:secret][:prefix]}"
|
17
16
|
@encrypt = Atmos.config[:secret][:encrypt]
|
18
17
|
end
|
19
18
|
|
20
|
-
def set(key, value)
|
19
|
+
def set(key, value, force: false)
|
21
20
|
opts = {}
|
22
21
|
opts[:server_side_encryption] = "AES256" if @encrypt
|
23
|
-
bucket.object(@bucket_prefix + key)
|
22
|
+
obj = bucket.object(@bucket_prefix + key)
|
23
|
+
if obj.exists? && ! force
|
24
|
+
raise "A value already exists for the given key, force overwrite or delete first"
|
25
|
+
end
|
26
|
+
obj.put(body: value, **opts)
|
24
27
|
end
|
25
28
|
|
26
29
|
def get(key)
|
@@ -0,0 +1,70 @@
|
|
1
|
+
require_relative '../../../atmos'
|
2
|
+
require 'aws-sdk-ssm'
|
3
|
+
|
4
|
+
module SimplyGenius
|
5
|
+
module Atmos
|
6
|
+
module Providers
|
7
|
+
module Aws
|
8
|
+
|
9
|
+
class SsmSecretManager
|
10
|
+
include GemLogger::LoggerSupport
|
11
|
+
|
12
|
+
def initialize(provider)
|
13
|
+
@provider = provider
|
14
|
+
@path_prefix = "#{Atmos.config[:secret][:prefix]}"
|
15
|
+
@encrypt = Atmos.config[:secret][:encrypt]
|
16
|
+
end
|
17
|
+
|
18
|
+
def set(key, value, force: false)
|
19
|
+
opts = {}
|
20
|
+
|
21
|
+
param_name = param_name(key)
|
22
|
+
param_type = @encrypt ? "SecureString" : "String"
|
23
|
+
param_value = value
|
24
|
+
|
25
|
+
if value.is_a?(Array)
|
26
|
+
raise "AWS SSM Parameter Store cannot encrypt lists directly" if @encrypt
|
27
|
+
param_type = "StringList"
|
28
|
+
param_value = value.join(",")
|
29
|
+
end
|
30
|
+
|
31
|
+
client.put_parameter(name: param_name, value: param_value, type: param_type, overwrite: force)
|
32
|
+
end
|
33
|
+
|
34
|
+
def get(key)
|
35
|
+
resp = client.get_parameter(name: param_name(key), with_decryption: @encrypt)
|
36
|
+
resp.parameter.value
|
37
|
+
end
|
38
|
+
|
39
|
+
def delete(key)
|
40
|
+
client.delete_parameter(name: param_name(key))
|
41
|
+
end
|
42
|
+
|
43
|
+
def to_h
|
44
|
+
result = {}
|
45
|
+
resp = client.get_parameters_by_path(path: param_name(""), recursive: true, with_decryption: @encrypt)
|
46
|
+
resp.parameters.each do |p|
|
47
|
+
key = p.name.gsub(/^#{param_name("")}/, '')
|
48
|
+
result[key] = p.value
|
49
|
+
end
|
50
|
+
|
51
|
+
return result
|
52
|
+
end
|
53
|
+
|
54
|
+
private
|
55
|
+
|
56
|
+
def param_name(key)
|
57
|
+
param_name = "/#{@path_prefix}/#{key}"
|
58
|
+
param_name.gsub!(/\/{2,}/, '/')
|
59
|
+
param_name
|
60
|
+
end
|
61
|
+
|
62
|
+
def client
|
63
|
+
@client ||= ::Aws::SSM::Client.new
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
end
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
@@ -1,4 +1,5 @@
|
|
1
1
|
require 'hashie'
|
2
|
+
require_relative "../atmos/exceptions"
|
2
3
|
|
3
4
|
module SimplyGenius
|
4
5
|
module Atmos
|
@@ -7,9 +8,119 @@ module SimplyGenius
|
|
7
8
|
include GemLogger::LoggerSupport
|
8
9
|
include Hashie::Extensions::DeepMerge
|
9
10
|
include Hashie::Extensions::DeepFetch
|
11
|
+
include SimplyGenius::Atmos::Exceptions
|
10
12
|
disable_warnings
|
11
13
|
|
12
14
|
PATH_PATTERN = /[\.\[\]]/
|
15
|
+
INTERP_PATTERN = /(\#\{([^\}]+)\})/
|
16
|
+
|
17
|
+
attr_accessor :_root_, :error_resolver, :enable_expansion
|
18
|
+
|
19
|
+
alias orig_reader []
|
20
|
+
|
21
|
+
def expand_results(name, &blk)
|
22
|
+
# NOTE: we lookup locally first, then globally if a value is missing
|
23
|
+
# locally. To force a global lookup, use the explicit qualifier like
|
24
|
+
# "_root_.path.to.config"
|
25
|
+
|
26
|
+
value = blk.call(name)
|
27
|
+
|
28
|
+
if value.nil? && _root_ && enable_expansion
|
29
|
+
value = _root_[name]
|
30
|
+
end
|
31
|
+
|
32
|
+
if value.kind_of?(self.class) && value._root_.nil?
|
33
|
+
value._root_ = _root_ || self
|
34
|
+
end
|
35
|
+
|
36
|
+
enable_expansion ? expand(value) : value
|
37
|
+
end
|
38
|
+
|
39
|
+
def expanding_reader(key)
|
40
|
+
expand_results(key) {|k| orig_reader(k) }
|
41
|
+
end
|
42
|
+
|
43
|
+
def fetch(key, *args)
|
44
|
+
expand_results(key) {|k| super(k, *args) }
|
45
|
+
end
|
46
|
+
|
47
|
+
def error_resolver
|
48
|
+
@error_resolver || _root_.try(:error_resolver)
|
49
|
+
end
|
50
|
+
|
51
|
+
def enable_expansion
|
52
|
+
@enable_expansion.nil? ? _root_.try(:enable_expansion) : @enable_expansion
|
53
|
+
end
|
54
|
+
|
55
|
+
alias [] expanding_reader
|
56
|
+
|
57
|
+
# allows expansion when iterating
|
58
|
+
def each
|
59
|
+
each_key do |key|
|
60
|
+
yield key, self[key]
|
61
|
+
end
|
62
|
+
end
|
63
|
+
|
64
|
+
# allows expansion for to_a (which doesn't use each)
|
65
|
+
def to_a
|
66
|
+
self.collect {|k, v| [k, v]}
|
67
|
+
end
|
68
|
+
|
69
|
+
def format_error(msg, expr, ex=nil)
|
70
|
+
file, line = nil, nil
|
71
|
+
if error_resolver
|
72
|
+
file, line = error_resolver.call(expr)
|
73
|
+
end
|
74
|
+
file_msg = file.nil? ? "" : " in #{File.basename(file)}:#{line}"
|
75
|
+
msg = "#{msg} '#{expr}'#{file_msg}"
|
76
|
+
if ex
|
77
|
+
msg += " => #{ex.class} #{ex.message}"
|
78
|
+
end
|
79
|
+
return msg
|
80
|
+
end
|
81
|
+
|
82
|
+
def expand_string(obj)
|
83
|
+
result = obj
|
84
|
+
result.scan(INTERP_PATTERN).each do |substr, statement|
|
85
|
+
# TODO: add an explicit check for cycles instead of relying on Stack error
|
86
|
+
begin
|
87
|
+
# TODO: be consistent with dot notation between eval and
|
88
|
+
# notation_get. eval ends up calling Hashie method_missing,
|
89
|
+
# which returns nil if a key doesn't exist, causing a nil
|
90
|
+
# exception for next item in chain, while notation_get returns
|
91
|
+
# nil gracefully for the entire chain (preferred)
|
92
|
+
val = eval(statement, binding, __FILE__)
|
93
|
+
rescue SystemStackError => e
|
94
|
+
raise ConfigInterpolationError.new(format_error("Cycle in interpolated config", substr))
|
95
|
+
rescue StandardError => e
|
96
|
+
raise ConfigInterpolationError.new(format_error("Failing config statement", substr, e))
|
97
|
+
end
|
98
|
+
result = result.sub(substr, val.to_s)
|
99
|
+
end
|
100
|
+
|
101
|
+
result = true if result == "true"
|
102
|
+
result = false if result == "false"
|
103
|
+
|
104
|
+
result
|
105
|
+
end
|
106
|
+
|
107
|
+
def expand(value)
|
108
|
+
result = value
|
109
|
+
case value
|
110
|
+
when Hash
|
111
|
+
value
|
112
|
+
when String
|
113
|
+
expand_string(value)
|
114
|
+
when Enumerable
|
115
|
+
value.map! {|v| expand(v)}
|
116
|
+
# HACK: accounting for the case when someone wants to force an override using '^' as the first list item, when
|
117
|
+
# there is no upstream to override (i.e. merge proc doesn't get triggered as key is unique, so just added verbatim)
|
118
|
+
value.delete_at(0) if value[0] == "^"
|
119
|
+
value
|
120
|
+
else
|
121
|
+
value
|
122
|
+
end
|
123
|
+
end
|
13
124
|
|
14
125
|
def notation_get(key)
|
15
126
|
path = key.to_s.split(PATH_PATTERN).compact
|
@@ -70,6 +181,9 @@ module SimplyGenius
|
|
70
181
|
comment_places["<EOF>"] = comment_lines
|
71
182
|
|
72
183
|
orig_config = SettingsHash.new((YAML.load_file(yml_file) rescue {}))
|
184
|
+
# expansion disabled by default, but being explicit since we don't want
|
185
|
+
# expansion when mutating config files from generators
|
186
|
+
orig_config.enable_expansion = false
|
73
187
|
orig_config.notation_put(key, value, additive: additive)
|
74
188
|
new_config_no_comments = YAML.dump(orig_config.to_hash)
|
75
189
|
new_config_no_comments.sub!(/\A---\n/, "")
|
@@ -2,7 +2,7 @@
|
|
2
2
|
atmos:
|
3
3
|
|
4
4
|
# Configure the location of a user config file. This is where secrets are
|
5
|
-
# stored, e.g. OTP secrets
|
5
|
+
# stored, e.g. OTP secrets
|
6
6
|
#
|
7
7
|
user_config_file: ~/.atmos.yml
|
8
8
|
|
@@ -14,24 +14,25 @@ atmos:
|
|
14
14
|
template_sources:
|
15
15
|
- name: atmos-recipes
|
16
16
|
location: https://github.com/simplygenius/atmos-recipes/archive/v#{atmos_version}.zip#atmos-recipes-#{atmos_version}
|
17
|
-
# To access the Atmos Pro recipes, add your atmos pro auth token to
|
18
|
-
# ~/.atmos.yml and uncomment below
|
19
|
-
#- name: atmos-pro-recipes
|
20
|
-
# location: https://#{atmos.pro_auth_token}@releases.simplygenius.com/recipes/v#{atmos_version}.zip#atmos-pro-recipes-#{atmos_version}
|
21
17
|
|
22
|
-
#
|
23
|
-
# gem sources -a https://ATMOS_PRO_AUTH_TOKEN@releases.simplygenius.com/gems/
|
24
|
-
# then install the gem:
|
25
|
-
# gem install simplygenius-atmos-pro
|
26
|
-
# then add simplygenius-atmos-pro to the plugins key below
|
18
|
+
# The list of plugins to load when running atmos
|
27
19
|
#
|
28
|
-
# The
|
20
|
+
# The core atmos plugin is named "simplygenius-atmos-plugins" and is enabled
|
21
|
+
# by default below. Remove/comment to disable it completely, or disable parts
|
22
|
+
# of it by editing its config. You can also add to the list for custom
|
23
|
+
# plugins, and can have plugins inline in your repo by adding a relative path
|
24
|
+
# to load_path
|
29
25
|
#
|
30
26
|
plugins:
|
31
|
-
|
27
|
+
- name: simplygenius-atmos-plugins
|
28
|
+
disable_prompt_notify: false
|
29
|
+
disable_lock_detection: false
|
30
|
+
disable_plan_summary: false
|
31
|
+
disable_json_diff: false
|
32
32
|
|
33
33
|
# Allows one to add a custom ruby load path list for extending atmos without
|
34
|
-
# having to publish a gem
|
34
|
+
# having to publish a gem. This can be a relative or shell (~) path as it
|
35
|
+
# gets expanded at runtime
|
35
36
|
load_path:
|
36
37
|
|
37
38
|
# Configure the mechanism that allows terraform to callback into atmos
|
@@ -37,6 +37,13 @@ global_name_prefix: "#{org}-#{atmos_env}-"
|
|
37
37
|
#
|
38
38
|
local_name_prefix:
|
39
39
|
|
40
|
+
# The prefix used for the resources global to the organization. Setting this
|
41
|
+
# to something like "#{org}-" will allow you to manage multiple orgs from the same
|
42
|
+
# ops account without name conflicts.
|
43
|
+
# Most people should leave this blank.
|
44
|
+
#
|
45
|
+
org_prefix:
|
46
|
+
|
40
47
|
# Environment specific overrides. When adding new environments place them
|
41
48
|
# after the existing ones so that you don't end up with permission issues when
|
42
49
|
# bootstrapping the new account. You can also add overrides for each
|
@@ -60,4 +67,4 @@ atmos:
|
|
60
67
|
# to the directory containing atmos.yml
|
61
68
|
#
|
62
69
|
config_sources:
|
63
|
-
|
70
|
+
- atmos/*.y{,a}ml
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: simplygenius-atmos
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.10.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matt Conway
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-09-13 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -16,14 +16,14 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - "~>"
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
19
|
+
version: '2.0'
|
20
20
|
type: :development
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - "~>"
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
26
|
+
version: '2.0'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: rake
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -324,72 +324,72 @@ dependencies:
|
|
324
324
|
requirements:
|
325
325
|
- - "~>"
|
326
326
|
- !ruby/object:Gem::Version
|
327
|
-
version: 3.
|
327
|
+
version: 3.66.0
|
328
328
|
type: :runtime
|
329
329
|
prerelease: false
|
330
330
|
version_requirements: !ruby/object:Gem::Requirement
|
331
331
|
requirements:
|
332
332
|
- - "~>"
|
333
333
|
- !ruby/object:Gem::Version
|
334
|
-
version: 3.
|
334
|
+
version: 3.66.0
|
335
335
|
- !ruby/object:Gem::Dependency
|
336
336
|
name: aws-sdk-iam
|
337
337
|
requirement: !ruby/object:Gem::Requirement
|
338
338
|
requirements:
|
339
339
|
- - "~>"
|
340
340
|
- !ruby/object:Gem::Version
|
341
|
-
version: 1.
|
341
|
+
version: 1.29.0
|
342
342
|
type: :runtime
|
343
343
|
prerelease: false
|
344
344
|
version_requirements: !ruby/object:Gem::Requirement
|
345
345
|
requirements:
|
346
346
|
- - "~>"
|
347
347
|
- !ruby/object:Gem::Version
|
348
|
-
version: 1.
|
348
|
+
version: 1.29.0
|
349
349
|
- !ruby/object:Gem::Dependency
|
350
350
|
name: aws-sdk-organizations
|
351
351
|
requirement: !ruby/object:Gem::Requirement
|
352
352
|
requirements:
|
353
353
|
- - "~>"
|
354
354
|
- !ruby/object:Gem::Version
|
355
|
-
version: 1.
|
355
|
+
version: 1.32.0
|
356
356
|
type: :runtime
|
357
357
|
prerelease: false
|
358
358
|
version_requirements: !ruby/object:Gem::Requirement
|
359
359
|
requirements:
|
360
360
|
- - "~>"
|
361
361
|
- !ruby/object:Gem::Version
|
362
|
-
version: 1.
|
362
|
+
version: 1.32.0
|
363
363
|
- !ruby/object:Gem::Dependency
|
364
364
|
name: aws-sdk-s3
|
365
365
|
requirement: !ruby/object:Gem::Requirement
|
366
366
|
requirements:
|
367
367
|
- - "~>"
|
368
368
|
- !ruby/object:Gem::Version
|
369
|
-
version: 1.
|
369
|
+
version: 1.48.0
|
370
370
|
type: :runtime
|
371
371
|
prerelease: false
|
372
372
|
version_requirements: !ruby/object:Gem::Requirement
|
373
373
|
requirements:
|
374
374
|
- - "~>"
|
375
375
|
- !ruby/object:Gem::Version
|
376
|
-
version: 1.
|
376
|
+
version: 1.48.0
|
377
377
|
- !ruby/object:Gem::Dependency
|
378
|
-
name: aws-sdk-
|
378
|
+
name: aws-sdk-ssm
|
379
379
|
requirement: !ruby/object:Gem::Requirement
|
380
380
|
requirements:
|
381
381
|
- - "~>"
|
382
382
|
- !ruby/object:Gem::Version
|
383
|
-
version: 1.
|
383
|
+
version: 1.55.0
|
384
384
|
type: :runtime
|
385
385
|
prerelease: false
|
386
386
|
version_requirements: !ruby/object:Gem::Requirement
|
387
387
|
requirements:
|
388
388
|
- - "~>"
|
389
389
|
- !ruby/object:Gem::Version
|
390
|
-
version: 1.
|
390
|
+
version: 1.55.0
|
391
391
|
- !ruby/object:Gem::Dependency
|
392
|
-
name: aws-sdk-
|
392
|
+
name: aws-sdk-ecr
|
393
393
|
requirement: !ruby/object:Gem::Requirement
|
394
394
|
requirements:
|
395
395
|
- - "~>"
|
@@ -402,6 +402,20 @@ dependencies:
|
|
402
402
|
- - "~>"
|
403
403
|
- !ruby/object:Gem::Version
|
404
404
|
version: 1.20.0
|
405
|
+
- !ruby/object:Gem::Dependency
|
406
|
+
name: aws-sdk-ecs
|
407
|
+
requirement: !ruby/object:Gem::Requirement
|
408
|
+
requirements:
|
409
|
+
- - "~>"
|
410
|
+
- !ruby/object:Gem::Version
|
411
|
+
version: 1.49.0
|
412
|
+
type: :runtime
|
413
|
+
prerelease: false
|
414
|
+
version_requirements: !ruby/object:Gem::Requirement
|
415
|
+
requirements:
|
416
|
+
- - "~>"
|
417
|
+
- !ruby/object:Gem::Version
|
418
|
+
version: 1.49.0
|
405
419
|
- !ruby/object:Gem::Dependency
|
406
420
|
name: os
|
407
421
|
requirement: !ruby/object:Gem::Requirement
|
@@ -444,6 +458,34 @@ dependencies:
|
|
444
458
|
- - "~>"
|
445
459
|
- !ruby/object:Gem::Version
|
446
460
|
version: 1.1.2
|
461
|
+
- !ruby/object:Gem::Dependency
|
462
|
+
name: deepsort
|
463
|
+
requirement: !ruby/object:Gem::Requirement
|
464
|
+
requirements:
|
465
|
+
- - ">="
|
466
|
+
- !ruby/object:Gem::Version
|
467
|
+
version: '0'
|
468
|
+
type: :runtime
|
469
|
+
prerelease: false
|
470
|
+
version_requirements: !ruby/object:Gem::Requirement
|
471
|
+
requirements:
|
472
|
+
- - ">="
|
473
|
+
- !ruby/object:Gem::Version
|
474
|
+
version: '0'
|
475
|
+
- !ruby/object:Gem::Dependency
|
476
|
+
name: diffy
|
477
|
+
requirement: !ruby/object:Gem::Requirement
|
478
|
+
requirements:
|
479
|
+
- - ">="
|
480
|
+
- !ruby/object:Gem::Version
|
481
|
+
version: '0'
|
482
|
+
type: :runtime
|
483
|
+
prerelease: false
|
484
|
+
version_requirements: !ruby/object:Gem::Requirement
|
485
|
+
requirements:
|
486
|
+
- - ">="
|
487
|
+
- !ruby/object:Gem::Version
|
488
|
+
version: '0'
|
447
489
|
description: Atmos provides a terraform scaffold for creating cloud system architectures
|
448
490
|
email:
|
449
491
|
- matt@simplygenius.com
|
@@ -486,7 +528,12 @@ files:
|
|
486
528
|
- lib/simplygenius/atmos/otp.rb
|
487
529
|
- lib/simplygenius/atmos/plugin.rb
|
488
530
|
- lib/simplygenius/atmos/plugin_manager.rb
|
531
|
+
- lib/simplygenius/atmos/plugins.rb
|
532
|
+
- lib/simplygenius/atmos/plugins/core_plugin.rb
|
533
|
+
- lib/simplygenius/atmos/plugins/json_diff.rb
|
534
|
+
- lib/simplygenius/atmos/plugins/lock_detection.rb
|
489
535
|
- lib/simplygenius/atmos/plugins/output_filter.rb
|
536
|
+
- lib/simplygenius/atmos/plugins/plan_summary.rb
|
490
537
|
- lib/simplygenius/atmos/plugins/prompt_notify.rb
|
491
538
|
- lib/simplygenius/atmos/provider_factory.rb
|
492
539
|
- lib/simplygenius/atmos/providers/aws/account_manager.rb
|
@@ -494,6 +541,7 @@ files:
|
|
494
541
|
- lib/simplygenius/atmos/providers/aws/container_manager.rb
|
495
542
|
- lib/simplygenius/atmos/providers/aws/provider.rb
|
496
543
|
- lib/simplygenius/atmos/providers/aws/s3_secret_manager.rb
|
544
|
+
- lib/simplygenius/atmos/providers/aws/ssm_secret_manager.rb
|
497
545
|
- lib/simplygenius/atmos/providers/aws/user_manager.rb
|
498
546
|
- lib/simplygenius/atmos/settings_hash.rb
|
499
547
|
- lib/simplygenius/atmos/source_path.rb
|
@@ -526,7 +574,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
526
574
|
- !ruby/object:Gem::Version
|
527
575
|
version: '0'
|
528
576
|
requirements: []
|
529
|
-
rubygems_version: 3.0.
|
577
|
+
rubygems_version: 3.0.3
|
530
578
|
signing_key:
|
531
579
|
specification_version: 4
|
532
580
|
summary: Atmos provides a terraform scaffold for creating cloud system architectures
|