stack_master 1.14.0-x64-mingw32 → 1.17.1-x64-mingw32
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +63 -2
- data/lib/stack_master.rb +1 -0
- data/lib/stack_master/parameter_resolvers/ejson.rb +48 -0
- data/lib/stack_master/stack.rb +1 -1
- data/lib/stack_master/stack_definition.rb +16 -0
- data/lib/stack_master/stack_differ.rb +1 -1
- data/lib/stack_master/template_compiler.rb +21 -10
- data/lib/stack_master/template_compilers/cfndsl.rb +2 -1
- data/lib/stack_master/template_compilers/json.rb +2 -1
- data/lib/stack_master/template_compilers/sparkle_formation.rb +15 -5
- data/lib/stack_master/template_compilers/yaml.rb +2 -1
- data/lib/stack_master/validator.rb +1 -2
- data/lib/stack_master/version.rb +1 -1
- metadata +17 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 118dcd91cc2b5b9ae3c7dc9911fb59714c8562af
|
4
|
+
data.tar.gz: 46c2fc8ba4d25944d958b243c8b7e298ca39e53c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: a0b5bc48a5b7fd17d7f379a5852f33dcddcd5fae2350f72bb17c1320bb3711c38f192399e8768dae66b221da7d13e61569903be244b4b5e102ccaee281955a4b
|
7
|
+
data.tar.gz: d13c6bcc6c47c28547e49e06768fa155f2d6d39e3a5f53dd0305d250f7bb8293c261436a9307f00203573771da022133c5fe6c69a9200ad650d86ed9c8f76c85
|
data/README.md
CHANGED
@@ -175,7 +175,7 @@ key_name: myapp-us-east-1
|
|
175
175
|
|
176
176
|
### Compile Time Parameters
|
177
177
|
|
178
|
-
Compile time parameters can be used for [SparkleFormation](http://www.sparkleformation.io) templates. It conforms and
|
178
|
+
Compile time parameters can be used for [SparkleFormation](http://www.sparkleformation.io) templates. It conforms and
|
179
179
|
allows you to use the [Compile Time Parameters](http://www.sparkleformation.io/docs/sparkle_formation/compile-time-parameters.html) feature.
|
180
180
|
|
181
181
|
A simple example looks like this
|
@@ -268,6 +268,7 @@ you will likely want to set the parameter to NoEcho in your template.
|
|
268
268
|
db_password:
|
269
269
|
parameter_store: ssm_parameter_name
|
270
270
|
```
|
271
|
+
|
271
272
|
### 1Password Lookup
|
272
273
|
An Alternative to the alternative secret store is accessing 1password secrets using the 1password cli (`op`).
|
273
274
|
You declare a 1password lookup with the following parameters in your parameters file:
|
@@ -286,6 +287,44 @@ Currently we support two types of secrets, `password`s and `secureNote`s. All va
|
|
286
287
|
|
287
288
|
For more information on 1password cli please see [here](https://support.1password.com/command-line-getting-started/)
|
288
289
|
|
290
|
+
### EJSON Store
|
291
|
+
|
292
|
+
[ejson](https://github.com/Shopify/ejson) is a tool to manage asymmetrically encrypted values in JSON format.
|
293
|
+
This allows you to keep secrets securely in git/Github and gives anyone the ability the capability to add new
|
294
|
+
secrets without requiring access to the private key. [ejson_wrapper](https://github.com/envato/ejson_wrapper)
|
295
|
+
encrypts the underlying EJSON private key with KMS and stores it in the ejson file as `_private_key_enc`. Each
|
296
|
+
time an ejson secret is required, the underlying EJSON private key is first decrypted before passing it onto
|
297
|
+
ejson to decrypt the file.
|
298
|
+
|
299
|
+
First, generate an ejson file with ejson_wrapper, specifying the KMS key ID to be used:
|
300
|
+
|
301
|
+
```shell
|
302
|
+
gem install ejson_wrapper
|
303
|
+
ejson_wrapper generate --region us-east-1 --kms-key-id [key_id] --file secrets/production.ejson
|
304
|
+
```
|
305
|
+
|
306
|
+
Then, add the `ejson_file` argument to your stack in stack_master.yml:
|
307
|
+
|
308
|
+
```yaml
|
309
|
+
stacks:
|
310
|
+
us-east-1:
|
311
|
+
my_app:
|
312
|
+
template: my_app.json
|
313
|
+
ejson_file: production.ejson
|
314
|
+
```
|
315
|
+
|
316
|
+
finally refer to the secret key in the parameter file, i.e. parameters/my_app.yml:
|
317
|
+
|
318
|
+
```yaml
|
319
|
+
my_param:
|
320
|
+
ejson: "my_secret"
|
321
|
+
```
|
322
|
+
|
323
|
+
Additional configuration options:
|
324
|
+
|
325
|
+
- `ejson_file_region` The AWS region to attempt to decrypt private key with
|
326
|
+
- `ejson_file_kms` Default: true. Set to false to use ejson without KMS.
|
327
|
+
|
289
328
|
### Security Group
|
290
329
|
|
291
330
|
Looks up a security group by name and returns the ARN.
|
@@ -531,7 +570,7 @@ stacks:
|
|
531
570
|
|
532
571
|
```yaml
|
533
572
|
stacks:
|
534
|
-
us-east-1
|
573
|
+
us-east-1:
|
535
574
|
my-stack:
|
536
575
|
template: my-stack-with-dynamic.rb
|
537
576
|
compiler_options:
|
@@ -549,6 +588,28 @@ end
|
|
549
588
|
|
550
589
|
Note though that if a dynamic with the same name exists in your `templates/dynamics/` directory it will get loaded since it has higher precedence.
|
551
590
|
|
591
|
+
Templates can be also loaded from sparkle packs by defining `sparkle_pack_template`. The name corresponds to the registered symbol rather than specific name. That means for a sparkle pack containing:
|
592
|
+
|
593
|
+
```ruby
|
594
|
+
SparkleFormation.new(:template_name) do
|
595
|
+
...
|
596
|
+
end
|
597
|
+
```
|
598
|
+
|
599
|
+
we can use stack defined as follows:
|
600
|
+
|
601
|
+
```yaml
|
602
|
+
stacks:
|
603
|
+
us-east-1:
|
604
|
+
my-stack:
|
605
|
+
template: template_name
|
606
|
+
compiler: sparkle_formation
|
607
|
+
compiler_options:
|
608
|
+
sparkle_packs:
|
609
|
+
- some-sparkle-pack
|
610
|
+
sparkle_pack_template: true
|
611
|
+
```
|
612
|
+
|
552
613
|
## Allowed accounts
|
553
614
|
|
554
615
|
The AWS account the command is executing in can be restricted to a specific list of allowed accounts. This is useful in reducing the possibility of applying non-production changes in a production account. Each stack definition can specify the `allowed_accounts` property with an array of AWS account IDs the stack is allowed to work with.
|
data/lib/stack_master.rb
CHANGED
@@ -68,6 +68,7 @@ module StackMaster
|
|
68
68
|
autoload :AcmCertificate, 'stack_master/parameter_resolvers/acm_certificate'
|
69
69
|
autoload :AmiFinder, 'stack_master/parameter_resolvers/ami_finder'
|
70
70
|
autoload :StackOutput, 'stack_master/parameter_resolvers/stack_output'
|
71
|
+
autoload :Ejson, 'stack_master/parameter_resolvers/ejson'
|
71
72
|
autoload :Secret, 'stack_master/parameter_resolvers/secret'
|
72
73
|
autoload :SnsTopicName, 'stack_master/parameter_resolvers/sns_topic_name'
|
73
74
|
autoload :SecurityGroup, 'stack_master/parameter_resolvers/security_group'
|
@@ -0,0 +1,48 @@
|
|
1
|
+
require 'ejson_wrapper'
|
2
|
+
|
3
|
+
module StackMaster
|
4
|
+
module ParameterResolvers
|
5
|
+
class Ejson < Resolver
|
6
|
+
SecretNotFound = Class.new(StandardError)
|
7
|
+
|
8
|
+
def initialize(config, stack_definition)
|
9
|
+
@config = config
|
10
|
+
@stack_definition = stack_definition
|
11
|
+
end
|
12
|
+
|
13
|
+
def resolve(secret_key)
|
14
|
+
validate_ejson_file_specified
|
15
|
+
secrets = decrypt_ejson_file
|
16
|
+
secrets.fetch(secret_key.to_sym) do
|
17
|
+
raise SecretNotFound, "Unable to find key #{secret_key} in file #{@stack_definition.ejson_file}"
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
private
|
22
|
+
|
23
|
+
def validate_ejson_file_specified
|
24
|
+
if @stack_definition.ejson_file.nil?
|
25
|
+
raise ArgumentError, "No ejson_file defined for stack definition #{@stack_definition.stack_name} in #{@stack_definition.region}"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
def decrypt_ejson_file
|
30
|
+
@decrypt_ejson_file ||= EJSONWrapper.decrypt(ejson_file_path,
|
31
|
+
use_kms: @stack_definition.ejson_file_kms,
|
32
|
+
region: ejson_file_region)
|
33
|
+
end
|
34
|
+
|
35
|
+
def ejson_file_region
|
36
|
+
@stack_definition.ejson_file_region || StackMaster.cloud_formation_driver.region
|
37
|
+
end
|
38
|
+
|
39
|
+
def ejson_file_path
|
40
|
+
@ejson_file_path ||= File.join(@config.base_dir, secret_path_relative_to_base)
|
41
|
+
end
|
42
|
+
|
43
|
+
def secret_path_relative_to_base
|
44
|
+
@secret_path_relative_to_base ||= File.join('secrets', @stack_definition.ejson_file)
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
data/lib/stack_master/stack.rb
CHANGED
@@ -65,7 +65,7 @@ module StackMaster
|
|
65
65
|
parameter_hash = ParameterLoader.load(stack_definition.parameter_files)
|
66
66
|
template_parameters = ParameterResolver.resolve(config, stack_definition, parameter_hash[:template_parameters])
|
67
67
|
compile_time_parameters = ParameterResolver.resolve(config, stack_definition, parameter_hash[:compile_time_parameters])
|
68
|
-
template_body = TemplateCompiler.compile(config, stack_definition.
|
68
|
+
template_body = TemplateCompiler.compile(config, stack_definition.compiler, stack_definition.template_dir, stack_definition.template, compile_time_parameters, stack_definition.compiler_options)
|
69
69
|
template_format = TemplateUtils.identify_template_format(template_body)
|
70
70
|
stack_policy_body = if stack_definition.stack_policy_file_path
|
71
71
|
File.read(stack_definition.stack_policy_file_path)
|
@@ -10,12 +10,17 @@ module StackMaster
|
|
10
10
|
:base_dir,
|
11
11
|
:template_dir,
|
12
12
|
:secret_file,
|
13
|
+
:ejson_file,
|
14
|
+
:ejson_file_region,
|
15
|
+
:ejson_file_kms,
|
13
16
|
:stack_policy_file,
|
14
17
|
:additional_parameter_lookup_dirs,
|
15
18
|
:s3,
|
16
19
|
:files,
|
17
20
|
:compiler_options
|
18
21
|
|
22
|
+
attr_reader :compiler
|
23
|
+
|
19
24
|
include Utils::Initializable
|
20
25
|
|
21
26
|
def initialize(attributes = {})
|
@@ -25,6 +30,8 @@ module StackMaster
|
|
25
30
|
@s3 = {}
|
26
31
|
@files = []
|
27
32
|
@allowed_accounts = nil
|
33
|
+
@ejson_file_kms = true
|
34
|
+
@compiler = nil
|
28
35
|
super
|
29
36
|
@template_dir ||= File.join(@base_dir, 'templates')
|
30
37
|
@allowed_accounts = Array(@allowed_accounts)
|
@@ -41,13 +48,22 @@ module StackMaster
|
|
41
48
|
@notification_arns == other.notification_arns &&
|
42
49
|
@base_dir == other.base_dir &&
|
43
50
|
@secret_file == other.secret_file &&
|
51
|
+
@ejson_file == other.ejson_file &&
|
52
|
+
@ejson_file_region == other.ejson_file_region &&
|
53
|
+
@ejson_file_kms == other.ejson_file_kms &&
|
44
54
|
@stack_policy_file == other.stack_policy_file &&
|
45
55
|
@additional_parameter_lookup_dirs == other.additional_parameter_lookup_dirs &&
|
46
56
|
@s3 == other.s3 &&
|
57
|
+
@compiler == other.compiler &&
|
47
58
|
@compiler_options == other.compiler_options
|
48
59
|
end
|
49
60
|
|
61
|
+
def compiler=(compiler)
|
62
|
+
@compiler = compiler.&to_sym
|
63
|
+
end
|
64
|
+
|
50
65
|
def template_file_path
|
66
|
+
return unless template
|
51
67
|
File.expand_path(File.join(template_dir, template))
|
52
68
|
end
|
53
69
|
|
@@ -75,7 +75,7 @@ module StackMaster
|
|
75
75
|
|
76
76
|
def single_param_update?(param_name)
|
77
77
|
return false if param_name.blank? || @current_stack.blank? || body_different?
|
78
|
-
differences =
|
78
|
+
differences = Hashdiff.diff(@current_stack.parameters_with_defaults, @proposed_stack.parameters_with_defaults)
|
79
79
|
return false if differences.count != 1
|
80
80
|
diff = differences[0]
|
81
81
|
diff[0] == "~" && diff[1] == param_name
|
@@ -2,12 +2,16 @@ module StackMaster
|
|
2
2
|
class TemplateCompiler
|
3
3
|
TemplateCompilationFailed = Class.new(RuntimeError)
|
4
4
|
|
5
|
-
def self.compile(config,
|
6
|
-
compiler =
|
5
|
+
def self.compile(config, template_compiler, template_dir, template, compile_time_parameters, compiler_options = {})
|
6
|
+
compiler = if template_compiler
|
7
|
+
find_compiler(template_compiler)
|
8
|
+
else
|
9
|
+
template_compiler_for_stack(template, config)
|
10
|
+
end
|
7
11
|
compiler.require_dependencies
|
8
|
-
compiler.compile(
|
12
|
+
compiler.compile(template_dir, template, compile_time_parameters, compiler_options)
|
9
13
|
rescue StandardError => e
|
10
|
-
raise TemplateCompilationFailed.new("Failed to compile #{
|
14
|
+
raise TemplateCompilationFailed.new("Failed to compile #{template} with error #{e}.\n#{e.backtrace}")
|
11
15
|
end
|
12
16
|
|
13
17
|
def self.register(name, klass)
|
@@ -16,15 +20,22 @@ module StackMaster
|
|
16
20
|
end
|
17
21
|
|
18
22
|
# private
|
19
|
-
def self.
|
20
|
-
|
21
|
-
|
23
|
+
def self.template_compiler_for_stack(template, config)
|
24
|
+
ext = file_ext(template)
|
25
|
+
compiler_name = config.template_compilers.fetch(ext)
|
26
|
+
find_compiler(compiler_name)
|
22
27
|
end
|
23
|
-
private_class_method :
|
28
|
+
private_class_method :template_compiler_for_stack
|
24
29
|
|
25
|
-
def self.file_ext(
|
26
|
-
File.extname(
|
30
|
+
def self.file_ext(template)
|
31
|
+
File.extname(template).gsub('.', '').to_sym
|
27
32
|
end
|
28
33
|
private_class_method :file_ext
|
34
|
+
|
35
|
+
def self.find_compiler(name)
|
36
|
+
@compilers.fetch(name.to_sym) do
|
37
|
+
raise "Unknown compiler #{name.inspect}"
|
38
|
+
end
|
39
|
+
end
|
29
40
|
end
|
30
41
|
end
|
@@ -4,10 +4,11 @@ module StackMaster::TemplateCompilers
|
|
4
4
|
require 'cfndsl'
|
5
5
|
end
|
6
6
|
|
7
|
-
def self.compile(
|
7
|
+
def self.compile(template_dir, template, compile_time_parameters, _compiler_options = {})
|
8
8
|
CfnDsl.disable_binding
|
9
9
|
CfnDsl::ExternalParameters.defaults.clear # Ensure there's no leakage across invocations
|
10
10
|
CfnDsl::ExternalParameters.defaults(compile_time_parameters.symbolize_keys)
|
11
|
+
template_file_path = File.join(template_dir, template)
|
11
12
|
::CfnDsl.eval_file_with_extras(template_file_path).to_json
|
12
13
|
end
|
13
14
|
|
@@ -7,7 +7,8 @@ module StackMaster::TemplateCompilers
|
|
7
7
|
require 'json'
|
8
8
|
end
|
9
9
|
|
10
|
-
def self.compile(
|
10
|
+
def self.compile(template_dir, template, _compile_time_parameters, _compiler_options = {})
|
11
|
+
template_file_path = File.join(template_dir, template)
|
11
12
|
template_body = File.read(template_file_path)
|
12
13
|
if template_body.size > MAX_TEMPLATE_SIZE
|
13
14
|
# Parse the json and rewrite compressed
|
@@ -12,8 +12,8 @@ module StackMaster::TemplateCompilers
|
|
12
12
|
require 'stack_master/sparkle_formation/template_file'
|
13
13
|
end
|
14
14
|
|
15
|
-
def self.compile(
|
16
|
-
sparkle_template = compile_sparkle_template(
|
15
|
+
def self.compile(template_dir, template, compile_time_parameters, compiler_options = {})
|
16
|
+
sparkle_template = compile_sparkle_template(template_dir, template, compiler_options)
|
17
17
|
definitions = sparkle_template.parameters
|
18
18
|
validate_definitions(definitions)
|
19
19
|
validate_parameters(definitions, compile_time_parameters)
|
@@ -27,9 +27,12 @@ module StackMaster::TemplateCompilers
|
|
27
27
|
|
28
28
|
private
|
29
29
|
|
30
|
-
def self.compile_sparkle_template(
|
31
|
-
sparkle_path = compiler_options['sparkle_path']
|
32
|
-
|
30
|
+
def self.compile_sparkle_template(template_dir, template, compiler_options)
|
31
|
+
sparkle_path = if compiler_options['sparkle_path']
|
32
|
+
File.expand_path(compiler_options['sparkle_path'])
|
33
|
+
else
|
34
|
+
template_dir
|
35
|
+
end
|
33
36
|
|
34
37
|
collection = ::SparkleFormation::SparkleCollection.new
|
35
38
|
root_pack = ::SparkleFormation::Sparkle.new(
|
@@ -44,6 +47,13 @@ module StackMaster::TemplateCompilers
|
|
44
47
|
end
|
45
48
|
end
|
46
49
|
|
50
|
+
if compiler_options['sparkle_pack_template']
|
51
|
+
raise ArgumentError.new("Template #{template.inspect} not found in any sparkle pack") unless collection.templates['aws'].include? template
|
52
|
+
template_file_path = collection.templates['aws'][template].top['path']
|
53
|
+
else
|
54
|
+
template_file_path = File.join(template_dir, template)
|
55
|
+
end
|
56
|
+
|
47
57
|
sparkle_template = compile_template_with_sparkle_path(template_file_path, sparkle_path)
|
48
58
|
sparkle_template.sparkle.apply(collection)
|
49
59
|
sparkle_template
|
@@ -5,7 +5,8 @@ module StackMaster::TemplateCompilers
|
|
5
5
|
require 'json'
|
6
6
|
end
|
7
7
|
|
8
|
-
def self.compile(
|
8
|
+
def self.compile(template_dir, template, _compile_time_parameters, _compiler_options = {})
|
9
|
+
template_file_path = File.join(template_dir, template)
|
9
10
|
File.read(template_file_path)
|
10
11
|
end
|
11
12
|
|
@@ -14,7 +14,7 @@ module StackMaster
|
|
14
14
|
compile_time_parameters = ParameterResolver.resolve(@config, @stack_definition, parameter_hash[:compile_time_parameters])
|
15
15
|
|
16
16
|
StackMaster.stdout.print "#{@stack_definition.stack_name}: "
|
17
|
-
template_body = TemplateCompiler.compile(@config, @stack_definition.
|
17
|
+
template_body = TemplateCompiler.compile(@config, @stack_definition.compiler, @stack_definition.template_dir, @stack_definition.template, compile_time_parameters, @stack_definition.compiler_options)
|
18
18
|
cf.validate_template(template_body: TemplateUtils.maybe_compressed_template_body(template_body))
|
19
19
|
StackMaster.stdout.puts "valid"
|
20
20
|
true
|
@@ -28,6 +28,5 @@ module StackMaster
|
|
28
28
|
def cf
|
29
29
|
@cf ||= StackMaster.cloud_formation_driver
|
30
30
|
end
|
31
|
-
|
32
31
|
end
|
33
32
|
end
|
data/lib/stack_master/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: stack_master
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.17.1
|
5
5
|
platform: x64-mingw32
|
6
6
|
authors:
|
7
7
|
- Steve Hodgkiss
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2019-
|
12
|
+
date: 2019-10-03 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: bundler
|
@@ -377,6 +377,20 @@ dependencies:
|
|
377
377
|
version: '0'
|
378
378
|
- !ruby/object:Gem::Dependency
|
379
379
|
name: hashdiff
|
380
|
+
requirement: !ruby/object:Gem::Requirement
|
381
|
+
requirements:
|
382
|
+
- - "~>"
|
383
|
+
- !ruby/object:Gem::Version
|
384
|
+
version: '1'
|
385
|
+
type: :runtime
|
386
|
+
prerelease: false
|
387
|
+
version_requirements: !ruby/object:Gem::Requirement
|
388
|
+
requirements:
|
389
|
+
- - "~>"
|
390
|
+
- !ruby/object:Gem::Version
|
391
|
+
version: '1'
|
392
|
+
- !ruby/object:Gem::Dependency
|
393
|
+
name: ejson_wrapper
|
380
394
|
requirement: !ruby/object:Gem::Requirement
|
381
395
|
requirements:
|
382
396
|
- - ">="
|
@@ -441,6 +455,7 @@ files:
|
|
441
455
|
- lib/stack_master/parameter_resolver.rb
|
442
456
|
- lib/stack_master/parameter_resolvers/acm_certificate.rb
|
443
457
|
- lib/stack_master/parameter_resolvers/ami_finder.rb
|
458
|
+
- lib/stack_master/parameter_resolvers/ejson.rb
|
444
459
|
- lib/stack_master/parameter_resolvers/env.rb
|
445
460
|
- lib/stack_master/parameter_resolvers/latest_ami.rb
|
446
461
|
- lib/stack_master/parameter_resolvers/latest_ami_by_tags.rb
|