hybrid_platforms_conductor 33.2.0 → 33.3.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (26) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +61 -0
  3. data/docs/plugins.md +1 -0
  4. data/docs/plugins/secrets_reader/keepass.md +62 -0
  5. data/lib/hybrid_platforms_conductor/cmd_runner.rb +1 -1
  6. data/lib/hybrid_platforms_conductor/config.rb +0 -35
  7. data/lib/hybrid_platforms_conductor/core_extensions/bundler/without_bundled_env.rb +54 -0
  8. data/lib/hybrid_platforms_conductor/deployer.rb +35 -28
  9. data/lib/hybrid_platforms_conductor/executable.rb +2 -0
  10. data/lib/hybrid_platforms_conductor/hpc_plugins/platform_handler/serverless_chef.rb +4 -3
  11. data/lib/hybrid_platforms_conductor/hpc_plugins/provisioner/proxmox/reserve_proxmox_container +1 -0
  12. data/lib/hybrid_platforms_conductor/hpc_plugins/secrets_reader/keepass.rb +173 -0
  13. data/lib/hybrid_platforms_conductor/plugins.rb +1 -0
  14. data/lib/hybrid_platforms_conductor/safe_merge.rb +37 -0
  15. data/lib/hybrid_platforms_conductor/topographer/plugins/graphviz.rb +5 -3
  16. data/lib/hybrid_platforms_conductor/version.rb +1 -1
  17. data/spec/hybrid_platforms_conductor_test.rb +4 -0
  18. data/spec/hybrid_platforms_conductor_test/api/actions_executor/connectors/ssh/config_dsl_spec.rb +8 -6
  19. data/spec/hybrid_platforms_conductor_test/api/cmd_runner_spec.rb +15 -1
  20. data/spec/hybrid_platforms_conductor_test/api/config_spec.rb +48 -72
  21. data/spec/hybrid_platforms_conductor_test/api/deployer/config_dsl_spec.rb +36 -0
  22. data/spec/hybrid_platforms_conductor_test/api/deployer/secrets_reader_plugins/keepass_spec.rb +719 -0
  23. data/spec/hybrid_platforms_conductor_test/api/nodes_handler/cmdbs_plugins_api_spec.rb +2 -2
  24. data/spec/hybrid_platforms_conductor_test/api/platform_handlers/serverless_chef/services_deployment_spec.rb +1 -1
  25. data/spec/hybrid_platforms_conductor_test/executables/nodes_to_deploy_spec.rb +21 -15
  26. metadata +159 -139
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9f284e0fbe4ad4a3f1731a4c3b1bc61aa0bbe315d37241845bebdfe384e5f366
4
- data.tar.gz: 1f731fef465da4333a435ebfbbc1a4b4f592d85110e02e4ed4f38984275502b0
3
+ metadata.gz: c0707386d30d5e671d5ceac5e1fec9d971cdccd2b7057c6ed3a75cb5d8bb6d85
4
+ data.tar.gz: e9e5c023662e7aa9283905f4ae54f4b8995d14f61bea1d844ef4fef32cba68dc
5
5
  SHA512:
6
- metadata.gz: 8d061127dcc4caeb2e5f5984c4e91b50e46fc9687f4ae4353fb37812232dd69d1289196303f4586c0ad5e6311559679c4937844951ca6daf7a3f902e9b9fe519
7
- data.tar.gz: 5561ebc29c31d61fb0c009faf3b0dc661ce72c7078f55dad2e6e8a5630027137121ea69c536928646f3ac7271a1cf6a2b492d6a3ed0979ee1f1d35e85aed9ac3
6
+ metadata.gz: 1b70dedf6605d48081ead37771b8f94e29d26334fb1c0f7f15ed8ed70cabc24809b145bd681af9e829cd0c968accd18cf87886987391709a7252908a00104096
7
+ data.tar.gz: cbe9033432aae4b83b4153501efad2330651696ba0fc8b487d21f5927430d481661e5d3bf2c3617ffb20c2ba7e254f82328d4225aeba4f98295873c55fe4cca9
data/CHANGELOG.md CHANGED
@@ -1,3 +1,64 @@
1
+ # [v33.3.0](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v33.2.4...v33.3.0) (2021-07-02 17:20:58)
2
+
3
+ ## Global changes
4
+ ### Patches
5
+
6
+ * [[Feature(secrets_reader_keepass)] [#79] Add Keepass secrets reader plugin to get secrets from KeePass databases](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/7ace48653a74f03ed535ee6b41b21242a6454ff3)
7
+
8
+ ## Changes for secrets_reader_keepass
9
+ ### Features
10
+
11
+ * [[Feature(secrets_reader_keepass)] [#79] Add Keepass secrets reader plugin to get secrets from KeePass databases](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/7ace48653a74f03ed535ee6b41b21242a6454ff3)
12
+
13
+ # [v33.2.4](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v33.2.3...v33.2.4) (2021-06-23 15:14:20)
14
+
15
+ ## Global changes
16
+ ### Patches
17
+
18
+ * [[Hotfix(platform_handler_serverless_chef)] Forward environment in sudo commands](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/fd3b58875665c29dd071b5af2055eab0c45c0974)
19
+ * [[Hotfix] Fixed unbundled environment not cleaned + Moved deployer config DSL in deployer.rb](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/2bce6dbc31d98b27f196ed646eb9aa669b0f9a86)
20
+
21
+ ## Changes for platform_handler_serverless_chef
22
+ ### Patches
23
+
24
+ * [[Hotfix(platform_handler_serverless_chef)] Forward environment in sudo commands](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/fd3b58875665c29dd071b5af2055eab0c45c0974)
25
+
26
+ # [v33.2.3](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v33.2.2...v33.2.3) (2021-06-23 13:45:56)
27
+
28
+ ## Global changes
29
+ ### Patches
30
+
31
+ * [[Hotfix(provisioner_proxmox)] Add missing require in synchronization script](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/7a6c71595789c9180e1d95323fc5cf5051f2e2cd)
32
+
33
+ ## Changes for provisioner_proxmox
34
+ ### Patches
35
+
36
+ * [[Hotfix(provisioner_proxmox)] Add missing require in synchronization script](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/7a6c71595789c9180e1d95323fc5cf5051f2e2cd)
37
+
38
+ # [v33.2.2](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v33.2.1...v33.2.2) (2021-06-21 12:41:35)
39
+
40
+ ## Global changes
41
+ ### Patches
42
+
43
+ * [[Hotfix(cmd_runner)] Retain dynamically set environment while executing commands](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/d709d5d2871e43196cc1f5f9eaf5b2155b34ed4e)
44
+
45
+ ## Changes for cmd_runner
46
+ ### Patches
47
+
48
+ * [[Hotfix(cmd_runner)] Retain dynamically set environment while executing commands](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/d709d5d2871e43196cc1f5f9eaf5b2155b34ed4e)
49
+
50
+ # [v33.2.1](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v33.2.0...v33.2.1) (2021-06-21 10:23:51)
51
+
52
+ ## Global changes
53
+ ### Patches
54
+
55
+ * [[Hotfix(platform_handler_serverless_chef)] Corrected dry-run mode not working](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/4800a0f4255c1999eed33651c1e66c445acd17bb)
56
+
57
+ ## Changes for platform_handler_serverless_chef
58
+ ### Patches
59
+
60
+ * [[Hotfix(platform_handler_serverless_chef)] Corrected dry-run mode not working](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/4800a0f4255c1999eed33651c1e66c445acd17bb)
61
+
1
62
  # [v33.2.0](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v33.1.1...v33.2.0) (2021-06-18 23:22:21)
2
63
 
3
64
  ## Global changes
data/docs/plugins.md CHANGED
@@ -196,6 +196,7 @@ Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/sec
196
196
 
197
197
  Plugins shipped by default:
198
198
  * [`cli`](plugins/secrets_reader/cli.md)
199
+ * [`keepass`](plugins/secrets_reader/keepass.md)
199
200
  * [`thycotic`](plugins/secrets_reader/thycotic.md)
200
201
 
201
202
  <a name="test"></a>
@@ -0,0 +1,62 @@
1
+ # Secrets reader plugin: `keepass`
2
+
3
+ The `keepass` secrets reader plugin retrieves secrets from [KeePass](https://keepass.info/) databases, using an actual KeePass installation with the [KPScript plugin](https://keepass.info/plugins.html#kpscript).
4
+
5
+ It is configured by giving the KPScript command-line (using `use_kpscript_from` config DSL method), the KeePass databases to be read (using `secrets_from_keepass` config DSL method) and uses the `keepass` credential ID to authenticate along with extra environment variables for eventual key files or encrypted passwords.
6
+
7
+ ## Config DSL extension
8
+
9
+ ### `use_kpscript_from`
10
+
11
+ Provide the KPScript command-line to be used. If KPScript is already in your path, using `KPScript.exe` or `kpscript` should be enough, otherwise the full path to the command-line will be needed. On Windows it is needed to also include double quotes if the path contains spaces (like `"C:\Program Files\KeePass\KPScript.exe"`).
12
+
13
+ It takes a simple `String` as parameter to get the command line.
14
+
15
+ Example:
16
+ ```ruby
17
+ use_kpscript_from '/path/to/kpscript'
18
+ ```
19
+
20
+ ### `secrets_from_keepass`
21
+
22
+ Define a KeePass database to read secrets from.
23
+ A base group path of the KeePass database can also be specified to only read secrets from this group path.
24
+
25
+ All entries, attachments and sub-groups from the base group will be read as secrets.
26
+
27
+ Can be applied to subset of nodes using the [`for_nodes` DSL method](/docs/config_dsl.md#for_nodes).
28
+
29
+ It takes the following parameters:
30
+ * **database** (`String`): Database file path.
31
+ * **group_path** (`Array<String>`): Group path to extract from [default: `[]`].
32
+
33
+ Example:
34
+ ```ruby
35
+ secrets_from_keepass(
36
+ database: '/path/to/database.kdbx',
37
+ group_path: %w[Secrets Automation]
38
+ )
39
+ ```
40
+
41
+ ## Used credentials
42
+
43
+ | Credential | Usage
44
+ | --- | --- |
45
+ | `keepass` | Used to get the password to the database. No need to be set if the database opens without password. |
46
+
47
+ ## Used Metadata
48
+
49
+ | Metadata | Type | Usage
50
+ | --- | --- | --- |
51
+
52
+ ## Used environment variables
53
+
54
+ | Variable | Usage
55
+ | --- | --- |
56
+ | `hpc_key_file_for_keepass` | Optional path to the key file needed to open the database |
57
+ | `hpc_password_enc_for_keepass` | Optional encrypted password needed to open the database |
58
+
59
+ ## External tools dependencies
60
+
61
+ * [KeePass](https://keepass.info/) to open databases.
62
+ * [KPScript KeePass plugin](https://keepass.info/plugins.html#kpscript) to query KeePass API.
@@ -130,7 +130,7 @@ module HybridPlatformsConductor
130
130
  (log_to_stdout ? [@logger_stderr] : []) +
131
131
  (file_output.nil? ? [] : [file_output])
132
132
  ) do
133
- Bundler.with_unbundled_env do
133
+ Bundler.without_bundled_env do
134
134
  cmd_result = TTY::Command.new(
135
135
  printer: :null,
136
136
  pty: true,
@@ -47,12 +47,6 @@ module HybridPlatformsConductor
47
47
  # Array<Hash,Symbol,Object>
48
48
  attr_reader :expected_failures
49
49
 
50
- # List of retriable errors. Each info has the following properties:
51
- # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by those errors
52
- # * *errors_on_stdout* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stdout
53
- # * *errors_on_stderr* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stderr
54
- attr_reader :retriable_errors
55
-
56
50
  # List of deployment schedules. Each info has the following properties:
57
51
  # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by this rule
58
52
  # * *schedule* (IceCube::Schedule): The deployment schedule
@@ -81,11 +75,6 @@ module HybridPlatformsConductor
81
75
  # * *reason* (String): Reason for this expected failure
82
76
  # Array<Hash,Symbol,Object>
83
77
  @expected_failures = []
84
- # List of retriable errors. Each info has the following properties:
85
- # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by those errors
86
- # * *errors_on_stdout* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stdout
87
- # * *errors_on_stderr* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stderr
88
- @retriable_errors = []
89
78
  # List of deployment schedules. Each info has the following properties:
90
79
  # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by this rule
91
80
  # * *schedule* (IceCube::Schedule): The deployment schedule
@@ -162,30 +151,6 @@ module HybridPlatformsConductor
162
151
  end
163
152
  expose :expect_tests_to_fail
164
153
 
165
- # Mark some errors on stdout to be retriable during a deploy
166
- #
167
- # Parameters::
168
- # * *errors* (String, Regexp or Array<String or Regexp>): Single (or list of) errors matching pattern (either as exact string match or using a regexp).
169
- def retry_deploy_for_errors_on_stdout(errors)
170
- @retriable_errors << {
171
- errors_on_stdout: errors.is_a?(Array) ? errors : [errors],
172
- nodes_selectors_stack: current_nodes_selectors_stack
173
- }
174
- end
175
- expose :retry_deploy_for_errors_on_stdout
176
-
177
- # Mark some errors on stderr to be retriable during a deploy
178
- #
179
- # Parameters::
180
- # * *errors* (String, Regexp or Array<String or Regexp>): Single (or list of) errors matching pattern (either as exact string match or using a regexp).
181
- def retry_deploy_for_errors_on_stderr(errors)
182
- @retriable_errors << {
183
- errors_on_stderr: errors.is_a?(Array) ? errors : [errors],
184
- nodes_selectors_stack: current_nodes_selectors_stack
185
- }
186
- end
187
- expose :retry_deploy_for_errors_on_stderr
188
-
189
154
  # Set a deployment schedule
190
155
  #
191
156
  # Parameters::
@@ -0,0 +1,54 @@
1
+ # Add a way to clean the current env from Bundler variables
2
+ module Bundler
3
+
4
+ class << self
5
+
6
+ # Run block with all bundler-related variables removed from the current environment
7
+ def without_bundled_env(&block)
8
+ with_env(current_unbundled_env, &block)
9
+ end
10
+
11
+ # @return [Hash] Environment with all bundler-related variables removed
12
+ def current_unbundled_env
13
+ env = ENV.clone.to_hash
14
+ %w[
15
+ PATH
16
+ RUBYLIB
17
+ RUBYOPT
18
+ ].each do |env_name|
19
+ if original_env.key?(env_name)
20
+ env[env_name] = original_env[env_name]
21
+ else
22
+ env.delete(env_name)
23
+ end
24
+ end
25
+
26
+ env['MANPATH'] = env['BUNDLER_ORIG_MANPATH'] if env.key?('BUNDLER_ORIG_MANPATH')
27
+
28
+ env.delete_if do |k, _|
29
+ %w[
30
+ GEM_
31
+ BUNDLE_
32
+ BUNDLER_
33
+ ].any? { |prefix| k.start_with?(prefix) }
34
+ end
35
+
36
+ if env.key?('RUBYOPT')
37
+ rubyopt = env['RUBYOPT'].split
38
+ rubyopt.delete("-r#{File.expand_path('bundler/setup', __dir__)}")
39
+ rubyopt.delete('-rbundler/setup')
40
+ env['RUBYOPT'] = rubyopt.join(' ')
41
+ end
42
+
43
+ if env.key?('RUBYLIB')
44
+ rubylib = env['RUBYLIB'].split(File::PATH_SEPARATOR)
45
+ rubylib.delete(File.expand_path(__dir__))
46
+ env['RUBYLIB'] = rubylib.join(File::PATH_SEPARATOR)
47
+ end
48
+
49
+ env
50
+ end
51
+
52
+ end
53
+
54
+ end
@@ -9,6 +9,7 @@ require 'hybrid_platforms_conductor/logger_helpers'
9
9
  require 'hybrid_platforms_conductor/nodes_handler'
10
10
  require 'hybrid_platforms_conductor/services_handler'
11
11
  require 'hybrid_platforms_conductor/plugins'
12
+ require 'hybrid_platforms_conductor/safe_merge'
12
13
 
13
14
  module HybridPlatformsConductor
14
15
 
@@ -18,6 +19,12 @@ module HybridPlatformsConductor
18
19
  # Extend the Config DSL
19
20
  module ConfigDSLExtension
20
21
 
22
+ # List of retriable errors. Each info has the following properties:
23
+ # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by those errors
24
+ # * *errors_on_stdout* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stdout
25
+ # * *errors_on_stderr* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stderr
26
+ attr_reader :retriable_errors
27
+
21
28
  # List of log plugins. Each info has the following properties:
22
29
  # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by this rule.
23
30
  # * *log_plugins* (Array<Symbol>): List of log plugins to be used to store deployment logs.
@@ -36,6 +43,11 @@ module HybridPlatformsConductor
36
43
  # Mixin initializer
37
44
  def init_deployer_config
38
45
  @packaging_timeout_secs = 60
46
+ # List of retriable errors. Each info has the following properties:
47
+ # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by those errors
48
+ # * *errors_on_stdout* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stdout
49
+ # * *errors_on_stderr* (Array<String or Regexp>): List of errors match (as exact string match or using a regexp) to check against stderr
50
+ @retriable_errors = []
39
51
  @deployment_logs = []
40
52
  @secrets_readers = []
41
53
  end
@@ -48,6 +60,28 @@ module HybridPlatformsConductor
48
60
  @packaging_timeout_secs = packaging_timeout_secs
49
61
  end
50
62
 
63
+ # Mark some errors on stdout to be retriable during a deploy
64
+ #
65
+ # Parameters::
66
+ # * *errors* (String, Regexp or Array<String or Regexp>): Single (or list of) errors matching pattern (either as exact string match or using a regexp).
67
+ def retry_deploy_for_errors_on_stdout(errors)
68
+ @retriable_errors << {
69
+ errors_on_stdout: errors.is_a?(Array) ? errors : [errors],
70
+ nodes_selectors_stack: current_nodes_selectors_stack
71
+ }
72
+ end
73
+
74
+ # Mark some errors on stderr to be retriable during a deploy
75
+ #
76
+ # Parameters::
77
+ # * *errors* (String, Regexp or Array<String or Regexp>): Single (or list of) errors matching pattern (either as exact string match or using a regexp).
78
+ def retry_deploy_for_errors_on_stderr(errors)
79
+ @retriable_errors << {
80
+ errors_on_stderr: errors.is_a?(Array) ? errors : [errors],
81
+ nodes_selectors_stack: current_nodes_selectors_stack
82
+ }
83
+ end
84
+
51
85
  # Set the deployment log plugins to be used
52
86
  #
53
87
  # Parameters::
@@ -467,34 +501,7 @@ module HybridPlatformsConductor
467
501
 
468
502
  private
469
503
 
470
- # Safe-merge 2 hashes.
471
- # Safe-merging is done by:
472
- # * Merging values that are hashes.
473
- # * Reporting errors when values conflict.
474
- # When values are conflicting, the initial hash won't modify those conflicting values and will stop the merge.
475
- #
476
- # Parameters::
477
- # * *hash* (Hash): Hash to be modified merging hash_to_merge
478
- # * *hash_to_merge* (Hash): Hash to be merged into hash
479
- # Result::
480
- # * nil or Array<Object>: nil in case of success, or the keys path leading to a conflicting value in case of error
481
- def safe_merge(hash, hash_to_merge)
482
- conflicting_path = nil
483
- hash_to_merge.each do |key, value_to_merge|
484
- if hash.key?(key)
485
- if hash[key].is_a?(Hash) && value_to_merge.is_a?(Hash)
486
- sub_conflicting_path = safe_merge(hash[key], value_to_merge)
487
- conflicting_path = [key] + sub_conflicting_path unless sub_conflicting_path.nil?
488
- elsif hash[key] != value_to_merge
489
- conflicting_path = [key]
490
- end
491
- else
492
- hash[key] = value_to_merge
493
- end
494
- break unless conflicting_path.nil?
495
- end
496
- conflicting_path
497
- end
504
+ include SafeMerge
498
505
 
499
506
  # Get the list of retriable errors a node got from deployment logs.
500
507
  # Useful to know if an error is non-deterministic (due to external and temporary factors).
@@ -1,4 +1,6 @@
1
1
  require 'English'
2
+ require 'bundler'
3
+ require 'hybrid_platforms_conductor/core_extensions/bundler/without_bundled_env'
2
4
  require 'optparse'
3
5
  require 'logger'
4
6
  require 'hybrid_platforms_conductor/config'
@@ -227,13 +227,14 @@ module HybridPlatformsConductor
227
227
  # * Array< Hash<Symbol,Object> >: List of actions to be done
228
228
  def actions_to_deploy_on(node, service, use_why_run: true)
229
229
  package_dir = "#{@repository_path}/dist/#{@local_env ? 'local' : 'prod'}/#{service}"
230
+ gems_to_install = []
230
231
  # Generate the nodes attributes file
231
232
  unless @cmd_runner.dry_run
232
233
  FileUtils.mkdir_p "#{package_dir}/nodes"
233
234
  File.write("#{package_dir}/nodes/#{node}.json", (known_nodes.include?(node) ? metadata_for(node) : {}).merge(@nodes_handler.metadata_of(node)).to_json)
235
+ # Get the gems to be installed
236
+ gems_to_install = JSON.parse(File.read("#{package_dir}/gems.json"))
234
237
  end
235
- # Get the gems to be installed
236
- gems_to_install = JSON.parse(File.read("#{package_dir}/gems.json"))
237
238
  client_options = [
238
239
  '--local-mode',
239
240
  '--chef-license', 'accept',
@@ -262,7 +263,7 @@ module HybridPlatformsConductor
262
263
  raise "Missing file #{chef_versions_file} specifying the Chef Infra Client version to be deployed" unless File.exist?(chef_versions_file)
263
264
 
264
265
  required_chef_client_version = YAML.load_file(chef_versions_file)['client']
265
- sudo = (@actions_executor.connector(:ssh).ssh_user == 'root' ? '' : "#{@nodes_handler.sudo_on(node)} ")
266
+ sudo = (@actions_executor.connector(:ssh).ssh_user == 'root' ? '' : "#{@nodes_handler.sudo_on(node)} -E ")
266
267
  [
267
268
  {
268
269
  # Install dependencies
@@ -34,6 +34,7 @@
34
34
  # * *hpc_password_for_proxmox*: Password to be used to query Proxmox API
35
35
  # * *hpc_realm_for_proxmox*: Realm used to connect to the Proxmox API [default = 'pam']
36
36
 
37
+ require 'English'
37
38
  require 'json'
38
39
 
39
40
  reserved_resource = nil
@@ -0,0 +1,173 @@
1
+ require 'base64'
2
+ require 'nokogiri'
3
+ require 'tempfile'
4
+ require 'keepass_kpscript'
5
+ require 'zlib'
6
+ require 'hybrid_platforms_conductor/credentials'
7
+ require 'hybrid_platforms_conductor/safe_merge'
8
+ require 'hybrid_platforms_conductor/secrets_reader'
9
+
10
+ module HybridPlatformsConductor
11
+
12
+ module HpcPlugins
13
+
14
+ module SecretsReader
15
+
16
+ # Get secrets from a KeePass database
17
+ class Keepass < HybridPlatformsConductor::SecretsReader
18
+
19
+ include SafeMerge
20
+
21
+ # Extend the Config DSL
22
+ module ConfigDSLExtension
23
+
24
+ # List of defined KeePass secrets. Each info has the following properties:
25
+ # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by this rule.
26
+ # * *database* (String): Database file path.
27
+ # * *group_path* (Array<String>): Group path to extract from.
28
+ # Array< Hash<Symbol, Object> >
29
+ attr_reader :keepass_secrets
30
+
31
+ # String: The KPScript command line
32
+ attr_reader :kpscript
33
+
34
+ # Mixin initializer
35
+ def init_keepass_config
36
+ @keepass_secrets = []
37
+ @kpscript = nil
38
+ end
39
+
40
+ # Set the KPScript command line
41
+ #
42
+ # Parameters::
43
+ # * *cmd* (String): KPScript command line
44
+ def use_kpscript_from(cmd)
45
+ @kpscript = cmd
46
+ end
47
+
48
+ # Set a KeePass database configuration
49
+ #
50
+ # Parameters::
51
+ # * *database* (String): Database file path.
52
+ # * *group_path* (Array<String>): Group path to extract from [default: []].
53
+ def secrets_from_keepass(database:, group_path: [])
54
+ @keepass_secrets << {
55
+ nodes_selectors_stack: current_nodes_selectors_stack,
56
+ database: database,
57
+ group_path: group_path
58
+ }
59
+ end
60
+
61
+ end
62
+
63
+ Config.extend_config_dsl_with ConfigDSLExtension, :init_keepass_config
64
+
65
+ # Return secrets for a given service to be deployed on a node.
66
+ # [API] - This method is mandatory
67
+ # [API] - The following API components are accessible:
68
+ # * *@config* (Config): Main configuration API.
69
+ # * *@cmd_runner* (CmdRunner): Command Runner API.
70
+ # * *@nodes_handler* (NodesHandler): Nodes handler API.
71
+ #
72
+ # Parameters::
73
+ # * *node* (String): Node to be deployed
74
+ # * *service* (String): Service to be deployed
75
+ # Result::
76
+ # * Hash: The secrets
77
+ def secrets_for(node, service)
78
+ secrets = {}
79
+ # As we are dealing with global secrets, cache the reading for performance between nodes and services.
80
+ # Keep secrets cache grouped by URL/ID
81
+ @secrets = {} unless defined?(@secrets)
82
+ @nodes_handler.select_confs_for_node(node, @config.keepass_secrets).each do |keepass_secrets_info|
83
+ secret_id = "#{keepass_secrets_info[:database]}:#{keepass_secrets_info[:group_path].join('/')}"
84
+ unless @secrets.key?(secret_id)
85
+ raise 'Missing KPScript configuration. Please use use_kpscript_from to set it.' if @config.kpscript.nil?
86
+
87
+ Credentials.with_credentials_for(:keepass, @logger, @logger_stderr) do |_user, password|
88
+ Tempfile.create('hpc_keepass') do |xml_file|
89
+ key_file = ENV['hpc_key_file_for_keepass']
90
+ password_enc = ENV['hpc_password_enc_for_keepass']
91
+ keepass_credentials = {}
92
+ keepass_credentials[:password] = password if password
93
+ keepass_credentials[:password_enc] = password_enc if password_enc
94
+ keepass_credentials[:key_file] = key_file if key_file
95
+ KeepassKpscript.
96
+ use(@config.kpscript, debug: log_debug?).
97
+ open(keepass_secrets_info[:database], **keepass_credentials).
98
+ export('KeePass XML (2.x)', xml_file.path, group_path: keepass_secrets_info[:group_path].empty? ? nil : keepass_secrets_info[:group_path])
99
+ @secrets[secret_id] = parse_xml_secrets(Nokogiri::XML(xml_file).at_xpath('KeePassFile/Root/Group'))
100
+ end
101
+ end
102
+ end
103
+ conflicting_path = safe_merge(secrets, @secrets[secret_id])
104
+ raise "Secret set at path #{conflicting_path.join('->')} by #{keepass_secrets_info[:database]}#{keepass_secrets_info[:group_path].empty? ? '' : " from group #{keepass_secrets_info[:group_path].join('/')}"} for service #{service} on node #{node} has conflicting values (#{log_debug? ? "#{@secrets[secret_id].dig(*conflicting_path)} != #{secrets.dig(*conflicting_path)}" : 'set debug for value details'})." unless conflicting_path.nil?
105
+ end
106
+ secrets
107
+ end
108
+
109
+ private
110
+
111
+ # List of fields to include in the secrets and their corresponding XML name
112
+ FIELDS = {
113
+ notes: 'Notes',
114
+ password: 'Password',
115
+ url: 'URL',
116
+ user_name: 'UserName'
117
+ }
118
+
119
+ # Parse XML secrets from a Nokogiri XML group node
120
+ #
121
+ # Parameters::
122
+ # * *group* (Nokogiri::XML::Element): The group to parse
123
+ # Result::
124
+ # * Hash: The JSON secrets parsed from this group
125
+ def parse_xml_secrets(group)
126
+ # Parse all entries
127
+ group.xpath('Entry').map do |entry|
128
+ [
129
+ entry.at_xpath('String/Key[contains(.,"Title")]/../Value').text,
130
+ FIELDS.map do |property, field|
131
+ value = entry.at_xpath("String/Key[contains(.,\"#{field}\")]/../Value")&.text
132
+ if value.nil? || value.empty?
133
+ nil
134
+ else
135
+ [
136
+ property.to_s,
137
+ value
138
+ ]
139
+ end
140
+ end.compact.to_h.merge(
141
+ entry.xpath('Binary').map do |binary|
142
+ binary_meta = group.document.at_xpath("KeePassFile/Meta/Binaries/Binary[@ID=#{Integer(binary.xpath('Value').attr('Ref').value)}]")
143
+ binary_content = Base64.decode64(binary_meta.text)
144
+ if binary_meta.attr('Compressed') == 'True'
145
+ gz = Zlib::GzipReader.new(StringIO.new(binary_content))
146
+ binary_content = gz.read
147
+ gz.close
148
+ end
149
+ [
150
+ binary.xpath('Key').text,
151
+ binary_content
152
+ ]
153
+ end.to_h
154
+ )
155
+ ]
156
+ end.to_h.merge(
157
+ # Add children groups
158
+ group.xpath('Group').map do |sub_group|
159
+ [
160
+ sub_group.at_xpath('Name').text,
161
+ parse_xml_secrets(sub_group)
162
+ ]
163
+ end.to_h
164
+ )
165
+ end
166
+
167
+ end
168
+
169
+ end
170
+
171
+ end
172
+
173
+ end