stack_master 1.14.0-x64-mingw32 → 1.17.1-x64-mingw32

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: a0c3168b5e279ae1b4944be5eb01d4fa799e5794
4
- data.tar.gz: d4da17e1b54e40434c60060204a4d9e96f720833
3
+ metadata.gz: 118dcd91cc2b5b9ae3c7dc9911fb59714c8562af
4
+ data.tar.gz: 46c2fc8ba4d25944d958b243c8b7e298ca39e53c
5
5
  SHA512:
6
- metadata.gz: d2b34d6a2f518904738c2cbfb18c1f57bbbe89637fb322f7e4706ff3683f8986f700ff440196168664689d5fd4082e200109df6d89e13e3a9928565fe3c5082c
7
- data.tar.gz: 5797d1e8cc5e595ff7f91b86ba97296bdcd68862c30e0e64ab3ca5ac11bd9d4a68417b814831703aa292c48d7cb7ec3f40d1199ed5553b2bb0d3618edbb27f09
6
+ metadata.gz: a0b5bc48a5b7fd17d7f379a5852f33dcddcd5fae2350f72bb17c1320bb3711c38f192399e8768dae66b221da7d13e61569903be244b4b5e102ccaee281955a4b
7
+ data.tar.gz: d13c6bcc6c47c28547e49e06768fa155f2d6d39e3a5f53dd0305d250f7bb8293c261436a9307f00203573771da022133c5fe6c69a9200ad650d86ed9c8f76c85
data/README.md CHANGED
@@ -175,7 +175,7 @@ key_name: myapp-us-east-1
175
175
 
176
176
  ### Compile Time Parameters
177
177
 
178
- Compile time parameters can be used for [SparkleFormation](http://www.sparkleformation.io) templates. It conforms and
178
+ Compile time parameters can be used for [SparkleFormation](http://www.sparkleformation.io) templates. It conforms and
179
179
  allows you to use the [Compile Time Parameters](http://www.sparkleformation.io/docs/sparkle_formation/compile-time-parameters.html) feature.
180
180
 
181
181
  A simple example looks like this
@@ -268,6 +268,7 @@ you will likely want to set the parameter to NoEcho in your template.
268
268
  db_password:
269
269
  parameter_store: ssm_parameter_name
270
270
  ```
271
+
271
272
  ### 1Password Lookup
272
273
  An Alternative to the alternative secret store is accessing 1password secrets using the 1password cli (`op`).
273
274
  You declare a 1password lookup with the following parameters in your parameters file:
@@ -286,6 +287,44 @@ Currently we support two types of secrets, `password`s and `secureNote`s. All va
286
287
 
287
288
  For more information on 1password cli please see [here](https://support.1password.com/command-line-getting-started/)
288
289
 
290
+ ### EJSON Store
291
+
292
+ [ejson](https://github.com/Shopify/ejson) is a tool to manage asymmetrically encrypted values in JSON format.
293
+ This allows you to keep secrets securely in git/Github and gives anyone the ability the capability to add new
294
+ secrets without requiring access to the private key. [ejson_wrapper](https://github.com/envato/ejson_wrapper)
295
+ encrypts the underlying EJSON private key with KMS and stores it in the ejson file as `_private_key_enc`. Each
296
+ time an ejson secret is required, the underlying EJSON private key is first decrypted before passing it onto
297
+ ejson to decrypt the file.
298
+
299
+ First, generate an ejson file with ejson_wrapper, specifying the KMS key ID to be used:
300
+
301
+ ```shell
302
+ gem install ejson_wrapper
303
+ ejson_wrapper generate --region us-east-1 --kms-key-id [key_id] --file secrets/production.ejson
304
+ ```
305
+
306
+ Then, add the `ejson_file` argument to your stack in stack_master.yml:
307
+
308
+ ```yaml
309
+ stacks:
310
+ us-east-1:
311
+ my_app:
312
+ template: my_app.json
313
+ ejson_file: production.ejson
314
+ ```
315
+
316
+ finally refer to the secret key in the parameter file, i.e. parameters/my_app.yml:
317
+
318
+ ```yaml
319
+ my_param:
320
+ ejson: "my_secret"
321
+ ```
322
+
323
+ Additional configuration options:
324
+
325
+ - `ejson_file_region` The AWS region to attempt to decrypt private key with
326
+ - `ejson_file_kms` Default: true. Set to false to use ejson without KMS.
327
+
289
328
  ### Security Group
290
329
 
291
330
  Looks up a security group by name and returns the ARN.
@@ -531,7 +570,7 @@ stacks:
531
570
 
532
571
  ```yaml
533
572
  stacks:
534
- us-east-1
573
+ us-east-1:
535
574
  my-stack:
536
575
  template: my-stack-with-dynamic.rb
537
576
  compiler_options:
@@ -549,6 +588,28 @@ end
549
588
 
550
589
  Note though that if a dynamic with the same name exists in your `templates/dynamics/` directory it will get loaded since it has higher precedence.
551
590
 
591
+ Templates can be also loaded from sparkle packs by defining `sparkle_pack_template`. The name corresponds to the registered symbol rather than specific name. That means for a sparkle pack containing:
592
+
593
+ ```ruby
594
+ SparkleFormation.new(:template_name) do
595
+ ...
596
+ end
597
+ ```
598
+
599
+ we can use stack defined as follows:
600
+
601
+ ```yaml
602
+ stacks:
603
+ us-east-1:
604
+ my-stack:
605
+ template: template_name
606
+ compiler: sparkle_formation
607
+ compiler_options:
608
+ sparkle_packs:
609
+ - some-sparkle-pack
610
+ sparkle_pack_template: true
611
+ ```
612
+
552
613
  ## Allowed accounts
553
614
 
554
615
  The AWS account the command is executing in can be restricted to a specific list of allowed accounts. This is useful in reducing the possibility of applying non-production changes in a production account. Each stack definition can specify the `allowed_accounts` property with an array of AWS account IDs the stack is allowed to work with.
@@ -68,6 +68,7 @@ module StackMaster
68
68
  autoload :AcmCertificate, 'stack_master/parameter_resolvers/acm_certificate'
69
69
  autoload :AmiFinder, 'stack_master/parameter_resolvers/ami_finder'
70
70
  autoload :StackOutput, 'stack_master/parameter_resolvers/stack_output'
71
+ autoload :Ejson, 'stack_master/parameter_resolvers/ejson'
71
72
  autoload :Secret, 'stack_master/parameter_resolvers/secret'
72
73
  autoload :SnsTopicName, 'stack_master/parameter_resolvers/sns_topic_name'
73
74
  autoload :SecurityGroup, 'stack_master/parameter_resolvers/security_group'
@@ -0,0 +1,48 @@
1
+ require 'ejson_wrapper'
2
+
3
+ module StackMaster
4
+ module ParameterResolvers
5
+ class Ejson < Resolver
6
+ SecretNotFound = Class.new(StandardError)
7
+
8
+ def initialize(config, stack_definition)
9
+ @config = config
10
+ @stack_definition = stack_definition
11
+ end
12
+
13
+ def resolve(secret_key)
14
+ validate_ejson_file_specified
15
+ secrets = decrypt_ejson_file
16
+ secrets.fetch(secret_key.to_sym) do
17
+ raise SecretNotFound, "Unable to find key #{secret_key} in file #{@stack_definition.ejson_file}"
18
+ end
19
+ end
20
+
21
+ private
22
+
23
+ def validate_ejson_file_specified
24
+ if @stack_definition.ejson_file.nil?
25
+ raise ArgumentError, "No ejson_file defined for stack definition #{@stack_definition.stack_name} in #{@stack_definition.region}"
26
+ end
27
+ end
28
+
29
+ def decrypt_ejson_file
30
+ @decrypt_ejson_file ||= EJSONWrapper.decrypt(ejson_file_path,
31
+ use_kms: @stack_definition.ejson_file_kms,
32
+ region: ejson_file_region)
33
+ end
34
+
35
+ def ejson_file_region
36
+ @stack_definition.ejson_file_region || StackMaster.cloud_formation_driver.region
37
+ end
38
+
39
+ def ejson_file_path
40
+ @ejson_file_path ||= File.join(@config.base_dir, secret_path_relative_to_base)
41
+ end
42
+
43
+ def secret_path_relative_to_base
44
+ @secret_path_relative_to_base ||= File.join('secrets', @stack_definition.ejson_file)
45
+ end
46
+ end
47
+ end
48
+ end
@@ -65,7 +65,7 @@ module StackMaster
65
65
  parameter_hash = ParameterLoader.load(stack_definition.parameter_files)
66
66
  template_parameters = ParameterResolver.resolve(config, stack_definition, parameter_hash[:template_parameters])
67
67
  compile_time_parameters = ParameterResolver.resolve(config, stack_definition, parameter_hash[:compile_time_parameters])
68
- template_body = TemplateCompiler.compile(config, stack_definition.template_file_path, compile_time_parameters, stack_definition.compiler_options)
68
+ template_body = TemplateCompiler.compile(config, stack_definition.compiler, stack_definition.template_dir, stack_definition.template, compile_time_parameters, stack_definition.compiler_options)
69
69
  template_format = TemplateUtils.identify_template_format(template_body)
70
70
  stack_policy_body = if stack_definition.stack_policy_file_path
71
71
  File.read(stack_definition.stack_policy_file_path)
@@ -10,12 +10,17 @@ module StackMaster
10
10
  :base_dir,
11
11
  :template_dir,
12
12
  :secret_file,
13
+ :ejson_file,
14
+ :ejson_file_region,
15
+ :ejson_file_kms,
13
16
  :stack_policy_file,
14
17
  :additional_parameter_lookup_dirs,
15
18
  :s3,
16
19
  :files,
17
20
  :compiler_options
18
21
 
22
+ attr_reader :compiler
23
+
19
24
  include Utils::Initializable
20
25
 
21
26
  def initialize(attributes = {})
@@ -25,6 +30,8 @@ module StackMaster
25
30
  @s3 = {}
26
31
  @files = []
27
32
  @allowed_accounts = nil
33
+ @ejson_file_kms = true
34
+ @compiler = nil
28
35
  super
29
36
  @template_dir ||= File.join(@base_dir, 'templates')
30
37
  @allowed_accounts = Array(@allowed_accounts)
@@ -41,13 +48,22 @@ module StackMaster
41
48
  @notification_arns == other.notification_arns &&
42
49
  @base_dir == other.base_dir &&
43
50
  @secret_file == other.secret_file &&
51
+ @ejson_file == other.ejson_file &&
52
+ @ejson_file_region == other.ejson_file_region &&
53
+ @ejson_file_kms == other.ejson_file_kms &&
44
54
  @stack_policy_file == other.stack_policy_file &&
45
55
  @additional_parameter_lookup_dirs == other.additional_parameter_lookup_dirs &&
46
56
  @s3 == other.s3 &&
57
+ @compiler == other.compiler &&
47
58
  @compiler_options == other.compiler_options
48
59
  end
49
60
 
61
+ def compiler=(compiler)
62
+ @compiler = compiler.&to_sym
63
+ end
64
+
50
65
  def template_file_path
66
+ return unless template
51
67
  File.expand_path(File.join(template_dir, template))
52
68
  end
53
69
 
@@ -75,7 +75,7 @@ module StackMaster
75
75
 
76
76
  def single_param_update?(param_name)
77
77
  return false if param_name.blank? || @current_stack.blank? || body_different?
78
- differences = HashDiff.diff(@current_stack.parameters_with_defaults, @proposed_stack.parameters_with_defaults)
78
+ differences = Hashdiff.diff(@current_stack.parameters_with_defaults, @proposed_stack.parameters_with_defaults)
79
79
  return false if differences.count != 1
80
80
  diff = differences[0]
81
81
  diff[0] == "~" && diff[1] == param_name
@@ -2,12 +2,16 @@ module StackMaster
2
2
  class TemplateCompiler
3
3
  TemplateCompilationFailed = Class.new(RuntimeError)
4
4
 
5
- def self.compile(config, template_file_path, compile_time_parameters, compiler_options = {})
6
- compiler = template_compiler_for_file(template_file_path, config)
5
+ def self.compile(config, template_compiler, template_dir, template, compile_time_parameters, compiler_options = {})
6
+ compiler = if template_compiler
7
+ find_compiler(template_compiler)
8
+ else
9
+ template_compiler_for_stack(template, config)
10
+ end
7
11
  compiler.require_dependencies
8
- compiler.compile(template_file_path, compile_time_parameters, compiler_options)
12
+ compiler.compile(template_dir, template, compile_time_parameters, compiler_options)
9
13
  rescue StandardError => e
10
- raise TemplateCompilationFailed.new("Failed to compile #{template_file_path} with error #{e}.\n#{e.backtrace}")
14
+ raise TemplateCompilationFailed.new("Failed to compile #{template} with error #{e}.\n#{e.backtrace}")
11
15
  end
12
16
 
13
17
  def self.register(name, klass)
@@ -16,15 +20,22 @@ module StackMaster
16
20
  end
17
21
 
18
22
  # private
19
- def self.template_compiler_for_file(template_file_path, config)
20
- compiler_name = config.template_compilers.fetch(file_ext(template_file_path))
21
- @compilers.fetch(compiler_name)
23
+ def self.template_compiler_for_stack(template, config)
24
+ ext = file_ext(template)
25
+ compiler_name = config.template_compilers.fetch(ext)
26
+ find_compiler(compiler_name)
22
27
  end
23
- private_class_method :template_compiler_for_file
28
+ private_class_method :template_compiler_for_stack
24
29
 
25
- def self.file_ext(template_file_path)
26
- File.extname(template_file_path).gsub('.', '').to_sym
30
+ def self.file_ext(template)
31
+ File.extname(template).gsub('.', '').to_sym
27
32
  end
28
33
  private_class_method :file_ext
34
+
35
+ def self.find_compiler(name)
36
+ @compilers.fetch(name.to_sym) do
37
+ raise "Unknown compiler #{name.inspect}"
38
+ end
39
+ end
29
40
  end
30
41
  end
@@ -4,10 +4,11 @@ module StackMaster::TemplateCompilers
4
4
  require 'cfndsl'
5
5
  end
6
6
 
7
- def self.compile(template_file_path, compile_time_parameters, _compiler_options = {})
7
+ def self.compile(template_dir, template, compile_time_parameters, _compiler_options = {})
8
8
  CfnDsl.disable_binding
9
9
  CfnDsl::ExternalParameters.defaults.clear # Ensure there's no leakage across invocations
10
10
  CfnDsl::ExternalParameters.defaults(compile_time_parameters.symbolize_keys)
11
+ template_file_path = File.join(template_dir, template)
11
12
  ::CfnDsl.eval_file_with_extras(template_file_path).to_json
12
13
  end
13
14
 
@@ -7,7 +7,8 @@ module StackMaster::TemplateCompilers
7
7
  require 'json'
8
8
  end
9
9
 
10
- def self.compile(template_file_path, _compile_time_parameters, _compiler_options = {})
10
+ def self.compile(template_dir, template, _compile_time_parameters, _compiler_options = {})
11
+ template_file_path = File.join(template_dir, template)
11
12
  template_body = File.read(template_file_path)
12
13
  if template_body.size > MAX_TEMPLATE_SIZE
13
14
  # Parse the json and rewrite compressed
@@ -12,8 +12,8 @@ module StackMaster::TemplateCompilers
12
12
  require 'stack_master/sparkle_formation/template_file'
13
13
  end
14
14
 
15
- def self.compile(template_file_path, compile_time_parameters, compiler_options = {})
16
- sparkle_template = compile_sparkle_template(template_file_path, compiler_options)
15
+ def self.compile(template_dir, template, compile_time_parameters, compiler_options = {})
16
+ sparkle_template = compile_sparkle_template(template_dir, template, compiler_options)
17
17
  definitions = sparkle_template.parameters
18
18
  validate_definitions(definitions)
19
19
  validate_parameters(definitions, compile_time_parameters)
@@ -27,9 +27,12 @@ module StackMaster::TemplateCompilers
27
27
 
28
28
  private
29
29
 
30
- def self.compile_sparkle_template(template_file_path, compiler_options)
31
- sparkle_path = compiler_options['sparkle_path'] ?
32
- File.expand_path(compiler_options['sparkle_path']) : File.dirname(template_file_path)
30
+ def self.compile_sparkle_template(template_dir, template, compiler_options)
31
+ sparkle_path = if compiler_options['sparkle_path']
32
+ File.expand_path(compiler_options['sparkle_path'])
33
+ else
34
+ template_dir
35
+ end
33
36
 
34
37
  collection = ::SparkleFormation::SparkleCollection.new
35
38
  root_pack = ::SparkleFormation::Sparkle.new(
@@ -44,6 +47,13 @@ module StackMaster::TemplateCompilers
44
47
  end
45
48
  end
46
49
 
50
+ if compiler_options['sparkle_pack_template']
51
+ raise ArgumentError.new("Template #{template.inspect} not found in any sparkle pack") unless collection.templates['aws'].include? template
52
+ template_file_path = collection.templates['aws'][template].top['path']
53
+ else
54
+ template_file_path = File.join(template_dir, template)
55
+ end
56
+
47
57
  sparkle_template = compile_template_with_sparkle_path(template_file_path, sparkle_path)
48
58
  sparkle_template.sparkle.apply(collection)
49
59
  sparkle_template
@@ -5,7 +5,8 @@ module StackMaster::TemplateCompilers
5
5
  require 'json'
6
6
  end
7
7
 
8
- def self.compile(template_file_path, _compile_time_parameters, _compiler_options = {})
8
+ def self.compile(template_dir, template, _compile_time_parameters, _compiler_options = {})
9
+ template_file_path = File.join(template_dir, template)
9
10
  File.read(template_file_path)
10
11
  end
11
12
 
@@ -14,7 +14,7 @@ module StackMaster
14
14
  compile_time_parameters = ParameterResolver.resolve(@config, @stack_definition, parameter_hash[:compile_time_parameters])
15
15
 
16
16
  StackMaster.stdout.print "#{@stack_definition.stack_name}: "
17
- template_body = TemplateCompiler.compile(@config, @stack_definition.template_file_path, compile_time_parameters, @stack_definition.compiler_options)
17
+ template_body = TemplateCompiler.compile(@config, @stack_definition.compiler, @stack_definition.template_dir, @stack_definition.template, compile_time_parameters, @stack_definition.compiler_options)
18
18
  cf.validate_template(template_body: TemplateUtils.maybe_compressed_template_body(template_body))
19
19
  StackMaster.stdout.puts "valid"
20
20
  true
@@ -28,6 +28,5 @@ module StackMaster
28
28
  def cf
29
29
  @cf ||= StackMaster.cloud_formation_driver
30
30
  end
31
-
32
31
  end
33
32
  end
@@ -1,3 +1,3 @@
1
1
  module StackMaster
2
- VERSION = "1.14.0"
2
+ VERSION = "1.17.1"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: stack_master
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.14.0
4
+ version: 1.17.1
5
5
  platform: x64-mingw32
6
6
  authors:
7
7
  - Steve Hodgkiss
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2019-07-03 00:00:00.000000000 Z
12
+ date: 2019-10-03 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: bundler
@@ -377,6 +377,20 @@ dependencies:
377
377
  version: '0'
378
378
  - !ruby/object:Gem::Dependency
379
379
  name: hashdiff
380
+ requirement: !ruby/object:Gem::Requirement
381
+ requirements:
382
+ - - "~>"
383
+ - !ruby/object:Gem::Version
384
+ version: '1'
385
+ type: :runtime
386
+ prerelease: false
387
+ version_requirements: !ruby/object:Gem::Requirement
388
+ requirements:
389
+ - - "~>"
390
+ - !ruby/object:Gem::Version
391
+ version: '1'
392
+ - !ruby/object:Gem::Dependency
393
+ name: ejson_wrapper
380
394
  requirement: !ruby/object:Gem::Requirement
381
395
  requirements:
382
396
  - - ">="
@@ -441,6 +455,7 @@ files:
441
455
  - lib/stack_master/parameter_resolver.rb
442
456
  - lib/stack_master/parameter_resolvers/acm_certificate.rb
443
457
  - lib/stack_master/parameter_resolvers/ami_finder.rb
458
+ - lib/stack_master/parameter_resolvers/ejson.rb
444
459
  - lib/stack_master/parameter_resolvers/env.rb
445
460
  - lib/stack_master/parameter_resolvers/latest_ami.rb
446
461
  - lib/stack_master/parameter_resolvers/latest_ami_by_tags.rb