secret_config 0.8.0 → 0.10.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8df953d1a074c81118be3cc9458c9790b81fb72fbda9b075ae2a925393be657d
4
- data.tar.gz: e1605d3af8e236b458f25e3d5e6b6bd801c9be1e7767d1925d851fc778478a98
3
+ metadata.gz: 41c6e3154517787c40c77442238b3d4f62d03503924f51db586ff562a0975d2f
4
+ data.tar.gz: 985e416b9d9e480c8dc83e5988d896605329a93f5c854d0a05331209fb1e6fd7
5
5
  SHA512:
6
- metadata.gz: 0d2527c9397af668b648e1cb4f49994db83bfc9ec0619435392f66b59556e5a67ffa2a1cbbff102d23dc5e23000f5cdd3e2344dd385e876b72951ac25fe70b52
7
- data.tar.gz: c0a9f535b97e9e2cd2a1ea429b1464365a24d85ac269213548eda15318916490f7d59a2e5fb8eefc33b88a774c1e01ddff4ca6f4112451d61a09f440b9161cdb
6
+ metadata.gz: 417a5673810b55d39c9ae17c144da9528724ebf142a46ccdc5b02abf60a69bc6fe4ed8455007583d4bdada03fd2d27dc6e03207649df42495b84718c7bb84bd0
7
+ data.tar.gz: 3adae9bd474b45b0d4cc4b4780b3aaef41cdb5a52ea7dff2f4ea0261d34198f3ac7b8158dabe765b366b6a83d7e6dcc5c1a3fc798f7bec11c2366efa141ce012
data/README.md CHANGED
@@ -5,816 +5,33 @@ Centralized Configuration and Secrets Management for Ruby and Rails applications
5
5
 
6
6
  Securely store configuration information centrally, supporting multiple tenants of the same application.
7
7
 
8
- ## Overview
8
+ ## Documentation
9
9
 
10
- Securely store centralized configuration information such as:
11
- * Settings
12
- * Passwords
13
- * Encryption keys and certificates
10
+ * [Secret Config](https://config.rocketjob.io/)
14
11
 
15
- ## Features
12
+ ## Support
16
13
 
17
- Supports storing configuration information in:
18
- * File
19
- * Development and testing use.
20
- * Environment Variables
21
- * Environment Variables take precedence and can be used to override any setting.
22
- * AWS System Manager Parameter Store
23
- * Encrypt and securely store secrets such as passwords centrally.
14
+ * Questions? Join the chat room on Gitter for [rocketjob support](https://gitter.im/rocketjob/support)
15
+ * [Report bugs](https://github.com/rocketjob/secret_config/issues)
24
16
 
25
- Since all values are stored as strings in the central directory or config file, the following type conversions
26
- are supported:
27
- * integer
28
- * float
29
- * string
30
- * boolean
31
- * symbol
32
- * json
17
+ ## v0.10 Upgrade Notes
33
18
 
34
- Supported conversions:
35
- * base64
19
+ String interpolation has been changed to use `$` instead of `%`. Please change
20
+ all interpolated strings to use `$` before upgrading.
36
21
 
37
- Arrays are also supported when the value contains a known separator by which to break down the values.
22
+ Example: `%{date}` needs to be changed to `${date}`
38
23
 
39
- ## Benefits
24
+ ## v0.9 Upgrade Notes
40
25
 
41
- Benefits of moving sensitive configuration information into AWS System Manager Parameter Store:
42
-
43
- * Hierarchical structure is maintained.
44
- * Environment variables force all config into a single level.
45
- * Reduces the number of environment variables.
46
- * In a large application the number of secrets can grow dramatically.
47
- * Replaces sensitive data stored in local yaml or configuration files.
48
- * Including securing and managing encryption keys.
49
- * When encryption keys change, such as during a key rotation, config files don't have to be changed.
50
- * Removes security concerns with placing passwords in the clear into environment variables.
51
- * AWS System Manager Parameter Store does not charge for parameters.
52
- * Still recommend using a custom KMS key that charges only $1 per month.
53
- * Amounts as of 4/2019. Confirm what AWS charges you for these services.
54
- * AWS Secrets Manager charges for every secret being managed, which can accumulate quickly with large projects.
55
- * Configure multiple distinct application instances to support multiple tenants.
56
- * For example, use separate databases with unique credentials for each tenant.
57
- * Separation of responsibilities is achieved since operations can manage production configuration.
58
- * Developers do not need to be involved with production configuration such as host names and passwords.
59
- * All values are encrypted by default when stored in the AWS Parameter Store.
60
- * Prevents accidentally not encrypting sensitive data.
61
-
62
- ## Introduction
63
-
64
- When Secret Config starts up it reads all configuration entries into memory for all keys under the configured path.
65
- This means that once Secret Config has initialized all calls to Secret Config are extremely fast.
66
-
67
- The in-memory copy of the registry can be refreshed at any time by calling `SecretConfig.refresh!`. It can be refreshed
68
- via a process signal, or by calling it through an event, or via a messaging system.
69
-
70
- It is suggested that any programmatic lookup to values stored in Secret Config are called every time a value is
71
- being used, rather than creating a local copy of the value. This ensures that a refresh of the registry will take effect
72
- immediately for any code reading from Secret Config.
73
-
74
- ## API
75
-
76
- When Secret Config starts up it reads all configuration entries immediately for all keys under the configured path.
77
- This means that once Secret Config has initialized all calls to Secret Config are extremely fast.
78
-
79
- Secret Config supports the following programmatic interface:
80
-
81
- ### Read values
82
-
83
- Fetch the value for the supplied key, returning nil if not found:
84
-
85
- ~~~ruby
86
- # Key is present:
87
- SecretConfig["logger/level"]
88
- # => "info"
89
-
90
- # Key is missing:
91
- SecretConfig["logger/blah"]
92
- # => nil
93
- ~~~
94
-
95
- Fetch the value for the supplied key, raising `SecretConfig::MissingMandatoryKey` if not found:
96
-
97
- ~~~ruby
98
- # Key is present:
99
- SecretConfig.fetch("logger/level")
100
- # => "info"
101
-
102
- # Key is missing:
103
- SecretConfig.fetch("logger/blah")
104
- # => SecretConfig::MissingMandatoryKey (Missing configuration value for /development/logger/blah)
105
- ~~~
106
-
107
- A default value can be supplied when the key is not found in the registry:
108
-
109
- ~~~ruby
110
- SecretConfig.fetch("logger/level", default: "info")
111
- # => "info"
112
- ~~~
113
-
114
- Since AWS SSM Parameter store and environment variables only support string values,
115
- it is neccessary to convert the string back to the type required by the program.
116
-
117
- The following types are supported:
118
- `:integer`
119
- `:float`
120
- `:string`
121
- `:boolean`
122
- `:symbol`
123
- `:json`
124
-
125
- ~~~ruby
126
- # Without type conversion:
127
- SecretConfig.fetch("symmetric_encryption/version")
128
- # => "0"
129
-
130
- # With type conversion:
131
- SecretConfig.fetch("symmetric_encryption/version", type: :integer)
132
- # => 0
133
- ~~~
134
-
135
- Sometimes it is useful to store arrays of values as a single key.
136
-
137
- ~~~ruby
138
- # Example: A list of host names could be stored as: "primary.example.net,secondary.example.net,backup.example.net"
139
- # To extract it as an array of strings:
140
- SecretConfig.fetch("address_services/hostnames", separator: ",")
141
- # => ["primary.example.net", "secondary.example.net", "backup.example.net"]
142
-
143
- # Example: A list of ports could be stored as: "12345,5343,26815"
144
- # To extract it as an array of Integers:
145
- SecretConfig.fetch("address_services/ports", type: :integer, separator: ",")
146
- # => [12345, 5343, 26815]
147
- ~~~
148
-
149
- When storing binary data, it should be encoded with strict base64 encoding. To automatically convert it back to binary
150
- specify the encoding as `:base64`
151
-
152
- ~~~ruby
153
- # Return a value that was stored in Base64 encoding format:
154
- SecretConfig.fetch("symmetric_encryption/iv")
155
- # => "FW+/wLubAYM+ZU0bWQj59Q=="
156
-
157
- # Base64 decode a value that was stored in Base64 encoding format:
158
- SecretConfig.fetch("symmetric_encryption/iv", encoding: :base64)
159
- # => "\x15o\xBF\xC0\xBB\x9B\x01\x83>eM\eY\b\xF9\xF5"
160
- ~~~
161
-
162
- ### Key presence
163
-
164
- Returns whether a key is present in the registry:
165
-
166
- ~~~ruby
167
- SecretConfig.key?("logger/level")
168
- # => true
169
- ~~~
170
-
171
- ### Write values
172
-
173
- When Secret Config is configured to use the AWS SSM Parameter store, its values can be modified:
174
-
175
- ~~~ruby
176
- SecretConfig["logger/level"] = "debug"
177
- ~~~
178
-
179
- ~~~ruby
180
- SecretConfig.set("logger/level", "debug")
181
- ~~~
182
-
183
- ### Configuration
184
-
185
- Returns a Hash copy of the configuration as a tree:
186
-
187
- ~~~ruby
188
- SecretConfig.configuration
189
- ~~~
190
-
191
- ### Refresh Configuration
192
-
193
- Tell Secret Config to refresh its in-memory copy of the configuration settings.
194
-
195
- ~~~ruby
196
- SecretConfig.refresh!
197
- ~~~
198
-
199
- Example, refresh the registry any time a SIGUSR2 is raised, add the following code on startup:
200
-
201
- ~~~ruby
202
- Signal.trap('USR2') do
203
- SecretConfig.refresh!
204
- end
205
- ~~~
206
-
207
- Then to make the process refresh it registry:
208
- ~~~shell
209
- kill -SIGUSR2 1234
210
- ~~~
211
-
212
- Where `1234` above is the process PID.
213
-
214
- ## Development and Test use
215
-
216
- In the development environment create the file `config/application.yml` within which to store local development credentials.
217
- Depending on your team setup you may want to use the same file for all developers so can check it into you change control system.
218
-
219
- For example: `config/application.yml`
220
-
221
- ~~~yaml
222
- development:
223
- mysql:
224
- database: secret_config_development
225
- username: secret_config
226
- password: secret_configrules
227
- host: 127.0.0.1
228
-
229
- mongo:
230
- database: secret_config_development
231
- primary: 127.0.0.1:27017
232
- secondary: 127.0.0.1:27018
233
-
234
- secrets:
235
- secret_key_base: somereallylongstring
236
-
237
- test:
238
- mysql:
239
- database: secret_config_test
240
- username: secret_config
241
- password: secret_configrules
242
- host: 127.0.0.1
243
-
244
- mongo:
245
- database: secret_config_test
246
- primary: 127.0.0.1:27017
247
- secondary: 127.0.0.1:27018
248
-
249
- secrets:
250
- secret_key_base: somereallylongteststring
251
- ~~~
252
-
253
- Note how the hierarchical nature of configuration values is maintained. Typical environment variable approaches have
254
- to flatten everything into a single level.
255
-
256
- Note: Do not put any production credentials into this file.
257
-
258
- ### Environment Variables
259
-
260
- Any of the above values can be overridden with an environment variable, unless explicitly configured `SecretConfig.check_env_var = false`.
261
-
262
- To overwrite any of these settings with an environment variable:
263
-
264
- * Join the keys together with an '_'
265
- * Convert to uppercase
266
-
267
- For example, `mysql/host` can be overridden with the env var:
268
-
269
- export MYSQL_HOST=test.server
270
-
271
- ### Applying to existing config files
272
-
273
- Go through all the configuration files and look for sensitive data such as passwords:
274
-
275
- Example, an unchanged common `database.yml`:
276
-
277
- ~~~yaml
278
- defaults: &defaults
279
- encoding: utf8
280
- adapter: mysql2
281
-
282
- development:
283
- <<: *defaults
284
- database: secure_config_development
285
- username: jack
286
- password: jackrules
287
- host: localhost
288
-
289
- test:
290
- <<: *defaults
291
- database: secure_config_test
292
- username: tester
293
- password: khjsdjhdsjhdsr32
294
- host: test.server
295
-
296
- production:
297
- <<: *defaults
298
- database: secure_config_production
299
- username: product
300
- password: donotexpose45
301
- host: production.server
302
- ~~~
303
-
304
- Replace the sensitive data with a `SecureConfig.fetch`:
305
-
306
- Updated `database.yml`:
307
-
308
- ~~~yaml
309
- configuration: &configuration
310
- database: <%= SecretConfig.fetch("mysql/database") %>
311
- username: <%= SecretConfig.fetch("mysql/username") %>
312
- password: <%= SecretConfig.fetch("mysql/password") %>
313
- host: <%= SecretConfig.fetch("mysql/host") %>
314
- encoding: utf8
315
- adapter: mysql2
316
-
317
- development:
318
- <<: *configuration
319
-
320
- test:
321
- <<: *configuration
322
-
323
- production:
324
- <<: *configuration
325
- ~~~
326
-
327
- Since the secrets are externalized the configuration between environments is simpler.
328
-
329
- ### Replacing custom config files
330
-
331
- When writing new components or gems, instead of requiring a proprietary config file, refer
332
- to the settings programmatically:
333
-
334
- For example, somewhere in your codebase you need a persistent http connection:
335
-
336
- ~~~ruby
337
- def http_client
338
- @http_client ||=
339
- PersistentHTTP.new(
340
- name: 'HTTPClient',
341
- url: SecretConfig.fetch('http_client/url'),
342
- logger: logger,
343
- pool_size: SecretConfig.fetch('http_client/pool_size', type: :integer, default: 10),
344
- warn_timeout: SecretConfig.fetch('http_client/warn_timeout', type: :float, default: 0.25),
345
- open_timeout: SecretConfig.fetch('http_client/open_timeout', type: :float, default: 30),
346
- read_timeout: SecretConfig.fetch('http_client/read_timeout', type: :float, default: 30),
347
- force_retry: true
348
- )
349
- end
350
- ~~~
351
-
352
- Then the application that uses the above library / gem just needs to add the relevant entries to their
353
- `application.yml` file:
354
-
355
- ~~~yaml
356
- http_client:
357
- url: https://test.example.com
358
- pool_size: 20
359
- read_timeout: 300
360
- ~~~
361
-
362
- This avoids a custom config file just for the above library.
363
-
364
- Additionally the values can be overridden with environment variables at any time:
365
-
366
- export HTTP_CLIENT_URL=https://production.example.com
367
-
368
- ## Configuration
369
-
370
- Add the following line to Gemfile
371
-
372
- gem "secret_config"
373
-
374
- Out of the box Secret Config will look in the local file system for the file `config/application.yml`
375
- as covered above. By default it will use env var `RAILS_ENV` to define the path to look under for settings.
376
-
377
- The default settings are great for getting started in development and test, but should not be used in production.
378
-
379
- To ensure Secret Config is configured and available for use within any of the config files, add
380
- the following lines to the very top of `application.rb` under the line `class Application < Rails::Application`:
381
-
382
- ~~~ruby
383
- module MyApp
384
- class Application < Rails::Application
385
-
386
- # Add the following lines to configure Secret Config:
387
- if Rails.env.development? || Rails.env.test?
388
- # Use 'config/application.yml'
389
- config.secret_config.use :file
390
- else
391
- # Read configuration from AWS SSM Parameter Store
392
- config.secret_config.use :ssm, path: "/#{Rails.env}/my_app"
393
- end
394
-
395
- # ....
396
- end
397
- end
398
- ~~~
399
-
400
- `path` is the path from which the configuration data will be read. This path uniquely identifies the
401
- configuration for this instance of the application. In the example above it uses the rails env and application name
402
- by default. This can be overridden using the `SECRET_CONFIG_PATH` environment variable when needed.
403
-
404
- By placing the secret config configuration as the very first configuration item, it allows any subsequent
405
- configuration item to access the centralized configuration in AWS System Manager Parameter Store.
406
-
407
- The environment variable `SECRET_CONFIG_PROVIDER` can be used to override the provider when needed.
408
- For example:
409
- `export SECRET_CONFIG_PROVIDER=ssm`
410
- Or,
411
- `export SECRET_CONFIG_PROVIDER=file`
412
-
413
- If we need 2 completely separate instances of the application running in a single AWS account then we could use
414
- multiple paths. For example:
415
-
416
- /production1/my_application
417
- /production2/my_application
418
-
419
- /production/instance1/my_application
420
- /production/instance2/my_application
421
-
422
- The `path` is completely flexible, but must be unique for every AWS account under which the application will run.
423
- The same `path` can be used in different AWS accounts though. It is also not replicated across regions.
424
-
425
- When writing settings to the parameter store, it is recommended to use a custom KMS key to encrypt the values, if you don't specify a key ID, the system uses the default key associated with your AWS account `alias/aws/ssm`.
426
- To supply the key to encrypt the values with, add the `key_id` parameter:
427
-
428
- ~~~ruby
429
- module MyApp
430
- class Application < Rails::Application
431
-
432
- # Add the following lines to configure Secret Config:
433
- if Rails.env.development? || Rails.env.test?
434
- # Use 'config/application.yml'
435
- config.secret_config.use :file
436
- else
437
- # Read configuration from AWS SSM Parameter Store
438
- config.secret_config.use :ssm,
439
- path: "/#{Rails.env}/my_app",
440
- key_id: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
441
- end
442
-
443
- # ....
444
- end
445
- end
446
- ~~~
447
-
448
- Note: The relevant KMS key must be created first prior to using it here.
449
-
450
- `ssm` provider supports various configuration parameters that can be provided as keyword arguments for `config.secret_config.use :ssm, path, **args`
451
-
452
- Args hash:
453
- * **:key_id** (String) - The `key_id` is only used when writing settings to the AWS Parameter store and can be left off when that instance will only read from the parameter store. Can be configred with environment variable `SECRET_CONFIG_KEY_ID`.
454
- * **:retry_count** (Integer, default=10) - Max number of retries in case of execution failure.
455
- * **:retry_max_ms** (Integer, default=3_000) - Interval in ms between retries, `sleep` is used to facilitate throttling.
456
- * any options suported by [Aws::SSM::Client](https://docs.aws.amazon.com/sdkforruby/api/Aws/SSM/Client.html#initialize-instance_method) e.g. **:credentials**:
457
- ~~~ruby
458
- config.secret_config.use :ssm,
459
- path: "/#{Rails.env}/my_app",
460
- key_id: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
461
- credentials: Aws::AssumeRoleCredentials.new(
462
- role_arn: "arn:aws:iam::111111122222222:role/assume_role_name",
463
- role_session_name: "session-name-to-identify-#{SecureRandom.uuid}"
464
- ))
465
- ~~~
466
-
467
- ### Secret Config Environment variables
468
-
469
- Priority describes when environment variable is used as a default value, preceds configuration value or overrides.
470
-
471
- Name | Desctiption | Priority
472
- ------------------------- | --------------------------------------------------------------- | --------
473
- `SECRET_CONFIG_PATH` | path from which the configuration data will be read | precede
474
- `SECRET_CONFIG_PROVIDER` | override the provider configured for `config.secret_config.use` | override
475
- `SECRET_CONFIG_KEY_ID` | encryption `key_id` | default
476
- `SECRET_CONFIG_ACCOUNT_ID`| used in `rspec` to configure AWS Account Id for role assuming | required
477
-
478
- ### Shared configuration for development and test
479
-
480
- When running multiple engines or private "gems" inside the same code repository, the development and test
481
- configuration file `application.yml` can be shared. Update the lines above to:
482
-
483
- ~~~ruby
484
- module MyApp
485
- class Application < Rails::Application
486
-
487
- # Add the following lines:
488
- if Rails.env.development? || Rails.env.test?
489
- # Use 'config/application.yml'
490
- config.secret_config.use :file
491
- else
492
- # Read configuration from AWS SSM Parameter Store
493
- config.secret_config.use :ssm, path: "/#{Rails.env}/my_app"
494
- end
495
-
496
- # ....
497
- end
498
- end
499
- ~~~
500
-
501
- Where `file_name` is the full path and filename for where `application.yml` is located.
502
-
503
- ### Authorization
504
-
505
- The following policy needs to be added to the IAM Group under which the application will be running:
506
-
507
- ~~~json
508
- {
509
- "Version": "2012-10-17",
510
- "Statement": [
511
- {
512
- "Sid": "VisualEditor0",
513
- "Effect": "Allow",
514
- "Action": [
515
- "ssm:GetParametersByPath",
516
- "ssm:PutParameter",
517
- "ssm:DeleteParameter",
518
- ],
519
- "Resource": "*"
520
- }
521
- ]
522
- }
523
- ~~~
524
-
525
- The above policy restricts read and write access to just the Parameter Store capabilities of AWS System Manager.
526
-
527
- These additional Actions are not used by Secret Config, but may be useful for anyone using the AWS Console directly
528
- to view and modify parameters:
529
- - `ssm:DescribeParameters`
530
- - `ssm:GetParameterHistory`
531
- - `ssm:GetParameters`
532
- - `ssm:GetParameter`
533
-
534
- ## String Interpolation
535
-
536
- Values supplied for config settings can be replaced inline with date, time, hostname, pid and random values.
537
-
538
- For example to include the `hostname` in the log file name setting:
539
-
540
- ~~~yaml
541
- development:
542
- logger:
543
- level: info
544
- file_name: /var/log/my_application_%{hostname}.log
545
- ~~~
546
-
547
- Available interpolations:
548
-
549
- * %{date}
550
- * Current date in the format of "%Y%m%d" (CCYYMMDD)
551
- * %{date:format}
552
- * Current date in the supplied format. See strftime
553
- * %{time}
554
- * Current date and time down to ms in the format of "%Y%m%d%Y%H%M%S%L" (CCYYMMDDHHMMSSmmm)
555
- * %{time:format}
556
- * Current date and time in the supplied format. See strftime
557
- * %{env:name}
558
- * Extract value from the named environment variable.
559
- * %{hostname}
560
- * Full name of this host.
561
- * %{hostname:short}
562
- * Short name of this host. Everything up to the first period.
563
- * %{pid}
564
- * Process Id for this process.
565
- * %{random}
566
- * URL safe Random 32 byte value.
567
- * %{random:size}
568
- * URL safe Random value of `size` bytes.
569
-
570
- #### Notes:
571
-
572
- * To prevent interpolation use %%{...}
573
- * %% is not touched, only %{...} is searched for.
574
- * Since these interpolations are only evaluated at load time and
575
- every time the registry is refreshed there is no runtime overhead when keys are fetched.
576
-
577
- ## Command Line Interface
578
-
579
- Secret Config has a command line interface for exporting, importing and copying between paths in the registry.
580
-
581
- ~~~
582
- secret_config [options]
583
- -e, --export [FILE_NAME] Export configuration to a file or stdout if no file_name supplied.
584
- -i, --import [FILE_NAME] Import configuration from a file or stdin if no file_name supplied.
585
- -C, --copy SOURCE_PATH Import configuration from a file or stdin if no file_name supplied.
586
- -D, --diff [FILE_NAME] Compare configuration from a file or stdin if no file_name supplied.
587
- -c, --console Start interactive console.
588
- -p, --path PATH Path to import from / export to.
589
- -P, --provider PROVIDER Provider to use. [ssm | file]. Default: ssm
590
- -U, --no-filter Do not filter passwords and keys.
591
- -d, --prune During import delete all existing keys for which there is no key in the import file.
592
- -k, --key_id KEY_ID AWS KMS Key id or Key Alias to use when importing configuration values. Default: AWS Default key.
593
- -r, --region REGION AWS Region to use. Default: AWS_REGION env var.
594
- -R, --random_size INTEGER Size to use when generating random values. Whenever $random is encountered during an import. Default: 32
595
- -v, --version Display Symmetric Encryption version.
596
- -h, --help Prints this help.
597
- ~~~
598
-
599
- ### CLI Examples
600
-
601
- #### Import from a file into SSM parameters
602
-
603
- To get started it is useful to create a YAML file with all the relevant settings and then import
604
- it into AWS SSM Parameter store. This file is the same as `applcation.yml` except that each file
605
- is just for one environment. I.e. It does not contain the `test` or `development` root level entries.
606
-
607
- For example: `production.yml`
608
-
609
- ~~~yaml
610
- mysql:
611
- database: secret_config_production
612
- username: secret_config
613
- password: secret_configrules
614
- host: mysql_server.example.net
615
-
616
- mongo:
617
- database: secret_config_production
618
- primary: mongo_primary.example.net:27017
619
- secondary: mongo_secondary.example.net:27017
620
-
621
- secrets:
622
- secret_key_base: somereallylongproductionstring
623
- ~~~
624
-
625
- Import a yaml file, into a path in AWS SSM Parameter Store:
626
-
627
- secret_config --import production.yml --path /production/my_application
628
-
629
- Import a yaml file, into a path in AWS SSM Parameter Store, using a custom KMS key to encrypt the values:
630
-
631
- secret_config --import production.yml --path /production/my_application --key_id "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
632
-
633
- #### Diff
634
-
635
- Before importing a new config file into the AWS SSM Parameter store, a diff can be performed to determine
636
- what the differences are that will be applied when the import is run with the `--prune` option.
637
-
638
- secret_config --diff production.yml --path /production/my_application
639
-
640
- Key:
641
-
642
- + Adding a new key to the registry.
643
- - The key will be removed from the registry during the import if --prune is specified.
644
- * The value for that key will change during an import.
645
-
646
- #### Export SSM parameters
647
-
648
- In AWS SSM Parameter store it can be difficult to
649
- Export the values from a specific path into a yaml or json file so that they are easier to read.
650
-
651
- Export from a path in AWS SSM Parameter Store to a yaml file, where passwords are filtered:
652
-
653
- secret_config --export production.yml --path /production/my_application
654
-
655
- Export from a path in AWS SSM Parameter Store to a yaml file, _without_ filtering out passwords:
656
-
657
- secret_config --export production.yml --path /production/my_application --no-filter
658
-
659
- Export from a path in AWS SSM Parameter Store to a json file, where passwords are filtered:
660
-
661
- secret_config --export production.json --path /production/my_application
662
-
663
- #### Copy values between paths in AWS SSM parameter store
664
-
665
- It can be useful to keep a "master" copy of the values for an environment or stack in a custom path
666
- in AWS Parameter Store. Then for each stack or environment that is spun up, copy the "master" / "common" values
667
- into the new path. Once copied the values specific to that path can be updated accordingly.
668
-
669
- Copy configuration from one path in AWS SSM Parameter Store to another path in AWS SSM Parameter Store:
670
-
671
- secret_config --copy /production/my_application --path /tenant73/my_application
672
-
673
- #### Generating random passwords
674
-
675
- In the multi-tenant example above, we may want to generate a secure random password for each tenant.
676
- In the source file or registry, set the value to `$random`, this will ensure that during the `import` or `copy`
677
- that the destination will receive a secure random value.
678
-
679
- By default the length of the randomized value is 32 bytes, use `--random_size` to adjust the length of
680
- the randomized string.
681
-
682
- ## Docker
683
-
684
- Secret Config is at its best when the application is containerized. By externalizing the configuration the same
685
- docker container can be tested in one or more environments and then deployed directly to production without
686
- any changes. The only difference being the path that container uses to read its configuration from.
687
-
688
- Another important benefit is that the docker image does not contain any production or test credentials since
689
- these are all stored in AWS SSM Parameter Store.
690
-
691
- When a Ruby / Rails application is using Secret Config for its configuration settings, it only requires the
692
- following environment variables when starting up the container in for example AWS ECS or AWS Fargate:
693
-
694
- ~~~shell
695
- export SECRET_CONFIG_PATH=/production/my_application
696
- ~~~
697
-
698
- For rails applications, typically the `RAILS_ENV` is also needed, but not required for Secret Config.
699
-
700
- ~~~shell
701
- export RAILS_ENV=production
702
- ~~~
703
-
704
- ### Logging
705
-
706
- When using Semantic Logger, the following code could be added to `application.rb` to facilitate configuration
707
- of the logging output via Secret Config:
708
-
709
- ~~~ruby
710
- # Logging
711
- config.log_level = config.secret_config.fetch("logger/level", default: :info, type: :symbol)
712
- config.semantic_logger.backtrace_level = config.secret_config.fetch("logger/backtrace_level", default: :error, type: :symbol)
713
- config.semantic_logger.application = config.secret_config.fetch("logger/application", default: "my_app")
714
- config.semantic_logger.environment = config.secret_config.fetch("logger/environment", default: Rails.env)
715
- ~~~
716
-
717
- In any environment the log level can be changed, for example set `logger/level` to `debug`. And it can be changed
718
- in the AWS SSM Parameter Store, or directly with the environment variable `export LOGGER_LEVEL=debug`
719
-
720
- `logger/environment` can be used to identify which tenant the log messages are emanating from. By default it is just
721
- the rails environment. For example set `logger/environment` to `tenant73`.
722
-
723
- Additionally the following code can be used with containers to send log output to standard out:
724
-
725
- ~~~ruby
726
- destination = config.secret_config.fetch("logger/destination", default: :file, type: :symbol)
727
- if destination == :stdout
728
- STDOUT.sync = true
729
- config.rails_semantic_logger.add_file_appender = false
730
- config.semantic_logger.add_appender(
731
- io: STDOUT,
732
- level: config.log_level,
733
- formatter: config.secret_config.fetch("logger/formatter", default: :default, type: :symbol)
734
- )
735
- end
736
- ~~~
737
-
738
- Specifically for docker containers it is necessary to turn off file logging and turn on logging to standard out
739
- so that AWS Cloud Watch can pick up the log data.
740
-
741
- To start with `logger/destination` of `stdout` will work with regular non-colorized output. When feeding the
742
- log output into something that can process JSON, set `logger/formatter` to `json`.
743
-
744
- The benefit with the above approach is that a developer can pull the exact same container image that is running
745
- in production and configure it to run locally on their laptop. For example, set `logger/destination` to `file`.
746
-
747
- The above code can be modified as necessary to add any Semantic Logger appender to write directly to external
748
- centralized logging systems, instead of writing to standard out or local files.
749
-
750
- ### Email Server and Assets
751
-
752
- An example of how to setup the email server and the assets for html emails. Add to `application.rb`:
753
-
754
- ~~~ruby
755
- # Emails
756
- application_url = config.secret_config.fetch("emails/asset_host")
757
- uri = URI.parse(application_url)
758
-
759
- config.action_mailer.default_url_options = {host: uri.host, protocol: uri.scheme}
760
- config.action_mailer.asset_host = application_url
761
- config.action_mailer.smtp_settings = {address: config.secret_config.fetch("emails/smtp/address", default: "localhost")}
762
- config.action_mailer.raise_delivery_errors = config.secret_config.fetch("emails/raise_delivery_errors", default: true, type: :boolean)
763
- ~~~
764
-
765
- ### Symmetric Encryption
766
-
767
- An example of how to setup Symmetric Encryption. Add to `application.rb`:
768
-
769
- ~~~ruby
770
- # Encryption
771
- config.symmetric_encryption.cipher =
772
- SymmetricEncryption::Cipher.new(
773
- key: config.secret_config.fetch('symmetric_encryption/key', encoding: :base64),
774
- iv: config.secret_config.fetch('symmetric_encryption/iv', encoding: :base64),
775
- version: config.secret_config.fetch('symmetric_encryption/version', type: :integer),
776
- )
777
-
778
- # Also support one prior encryption key version during key rotation
779
- if config.secret_config.key?('symmetric_encryption/old/key')
780
- SymmetricEncryption.secondary_ciphers = [
781
- SymmetricEncryption::Cipher.new(
782
- key: config.secret_config.fetch('symmetric_encryption/old/key', encoding: :base64),
783
- iv: config.secret_config.fetch('symmetric_encryption/old/iv', encoding: :base64),
784
- version: config.secret_config.fetch('symmetric_encryption/old/version', type: :integer),
785
- ),
786
- ]
787
- end
788
- ~~~
789
-
790
- Using this approach the file `config/symmetric-encryption.yml` can be removed once the keys have been moved to
791
- the registry.
792
-
793
- To extract existing keys from the config file so that they can be imported into the registry,
794
- run the code below inside a console in each of the respective environments.
795
-
796
- ~~~ruby
797
- require "yaml"
798
- require "base64"
799
-
800
- def se_config(cipher)
801
- {
802
- "key" => Base64.strict_encode64(cipher.send(:key)),
803
- "iv" => Base64.strict_encode64(cipher.iv),
804
- "version" => cipher.version
805
- }
806
- end
807
-
808
- config = { "symmetric_encryption" => se_config(SymmetricEncryption.cipher) }
809
- if cipher = SymmetricEncryption.secondary_ciphers.first
810
- config["symmetric_encryption"]["old"] = se_config(cipher)
811
- end
812
- puts config.to_yaml
813
- ~~~
26
+ Note that the command line program name has changed from `secret_config` to `secret-config`.
27
+ Be careful that the arguments have also changed. The arguments are now consistent across operations.
28
+ The command line examples below have also been updated to reflect the changes.
29
+
30
+ Please run `secret-config --help` to see the new arguments and updated operations.
814
31
 
815
32
  ## Versioning
816
33
 
817
- This project adheres to [Semantic Versioning](http://semver.org/).
34
+ This project adheres to [Semantic Versioning](https://semver.org/).
818
35
 
819
36
  ## Author
820
37
 
@@ -822,7 +39,7 @@ This project adheres to [Semantic Versioning](http://semver.org/).
822
39
 
823
40
  ## License
824
41
 
825
- Copyright 2019 Reid Morrison
42
+ Copyright 2020 Reid Morrison
826
43
 
827
44
  Licensed under the Apache License, Version 2.0 (the "License");
828
45
  you may not use this file except in compliance with the License.