hybrid_platforms_conductor 32.17.0 → 33.0.2

data/docs/plugins/connector/ssh.md

@@ -70,7 +70,7 @@ end
 | `host_keys` | `Array<String>` | The node's host keys used to generate a `known_hosts` file with those to avoid user confirmations when connecting. |
 | `hostname` | `String` | Host name used to connect in case no IP address can be found in metadata. |
 | `private_ips` | `Array<String>` | IP list to connect in case `host_ip` is not defined in metadata. |
-| `ssh_session_exec` | `String` | If set to the string `false`, then consider that the node does not have any SSH SessionExec capabilities. This will make sure that remote command executions is done using stdin piping on interactive sessions instead of SSH commands execution. |
+| `ssh_session_exec` | `Boolean` | If set to `false`, then consider that the node does not have any SSH SessionExec capabilities. This will make sure that remote command executions is done using stdin piping on interactive sessions instead of SSH commands execution. |
 
 ## Used environment variables
 

data/docs/plugins/log/remote_fs.md

@@ -0,0 +1,26 @@
+# Log plugin: `remote_fs`
+
+The `remote_fs` log plugin saves deployment logs in each remote node that has been deployed, in the `/var/log/deployments` directory.
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/secrets_reader/cli.md

@@ -0,0 +1,31 @@
+# Secrets reader plugin: `cli`
+
+The `cli` secrets reader plugin reads secrets from a local JSON file that can be given through the `--secrets` command-line parameter.
+
+Example:
+```bash
+./bin/deploy --node my_node --secrets /path/to/my_secrets.json
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/secrets_reader/thycotic.md

@@ -0,0 +1,46 @@
+# Secrets reader plugin: `thycotic`
+
+The `thycotic` secrets reader plugin retrieves secrets from a [Thycotic secrets server](https://thycotic.com/products/secret-server-vdo/), using its SOAP API.
+
+It is configured using the `secrets_from_thycotic` (see below) config DSL and uses the `thycotic` credential ID to authenticate.
+
+## Config DSL extension
+
+### `secrets_from_thycotic`
+
+Define a Thycotic URL and Thycotic secret ID to fetch from a Thycotic server.
+The Thycotic secret should contain a JSON file that will be retrieved locally to be used as a secrets source. The local copy will then be removed after deployment.
+
+Can be applied to subset of nodes using the [`for_nodes` DSL method](/docs/config_dsl.md#for_nodes).
+
+It takes the following parameters:
+* **thycotic_url** (`String`): The Thycotic server URL.
+* **secret_id** (`Integer`): The Thycotic secret ID containing the secrets file to be used as secrets.
+
+Example:
+```ruby
+secrets_from_thycotic(
+  thycotic_url: 'https://my-thycotic-server.my-domain.com/SecretServer',
+  secret_id: 1107
+)
+```
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+| `thycotic` | Used to authenticate on the Thycotic server's SOAP API |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/test/check_deploy_and_idempotence.md

@@ -49,7 +49,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
-| `root_access_allowed` | `String` | If set to `true`, then skip the test for `root` access being disabled after deployment |
+| `root_access_allowed` | `Boolean` | If set to `true`, then skip the test for `root` access being disabled after deployment |
 
 ## Used environment variables
 

data/docs/plugins/test/connection.md

@@ -16,6 +16,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/deploy_removes_root_access.md

@@ -17,7 +17,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
-| `root_access_allowed` | `String` | If set to `true`, then skip this test |
+| `root_access_allowed` | `Boolean` | If set to `true`, then skip this test |
 
 ## Used environment variables
 

data/docs/plugins/test/file_system.md

@@ -38,6 +38,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/hostname.md

@@ -16,6 +16,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/ip.md

@@ -16,6 +16,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 | `private_ips` | `Array<String>` | List of possible private IPs the node should have |
 
 ## Used environment variables

data/docs/plugins/test/local_users.md

@@ -37,6 +37,7 @@ check_local_users_do_not_exist %w[olduser1 olduser2]
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/mounts.md

@@ -44,6 +44,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/orphan_files.md

@@ -27,6 +27,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/ports.md

@@ -39,6 +39,7 @@ check_closed_ports 25, 110
 | Metadata | Type | Usage
 | --- | --- | --- |
 | `host_ip` | `String` | Host IP address to be tested for port listening |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/spectre.md

@@ -15,6 +15,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/vulnerabilities.md

@@ -54,6 +54,7 @@ None
 | Metadata | Type | Usage
 | --- | --- | --- |
 | `image` | `String` | The name of the OS image to be used. The [configuration](../../config_dsl.md) should define the image and point it to a directory containing a `oval.json` that will contain definition of OVAL files to be checked for this OS(see above). |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/lib/hybrid_platforms_conductor/action.rb

@@ -16,8 +16,8 @@ module HybridPlatformsConductor
     # * *actions_executor* (ActionsExecutor): Actions Executor to be used. [default: ActionsExecutor.new]
     # * *action_info* (Object or nil): Action info needed to setup the action, or nil if none [default: nil]
     def initialize(
-      logger: Logger.new(STDOUT),
-      logger_stderr: Logger.new(STDERR),
+      logger: Logger.new($stdout),
+      logger_stderr: Logger.new($stderr),
       config: Config.new,
       cmd_runner: CmdRunner.new,
       actions_executor: ActionsExecutor.new,
@@ -27,7 +27,7 @@ module HybridPlatformsConductor
       @cmd_runner = cmd_runner
       @actions_executor = actions_executor
       @action_info = action_info
-      setup(@action_info) if self.respond_to?(:setup)
+      setup(@action_info) if respond_to?(:setup)
     end
 
     # Do we need a connector to execute this action on a node?
@@ -53,7 +53,7 @@ module HybridPlatformsConductor
       @timeout = timeout
       @stdout_io = stdout_io
       @stderr_io = stderr_io
-      @connector.prepare_for(@node, @timeout, @stdout_io, @stderr_io) if @connector
+      @connector&.prepare_for(@node, @timeout, @stdout_io, @stderr_io)
     end
 
     private

data/lib/hybrid_platforms_conductor/actions_executor.rb

@@ -35,7 +35,7 @@ module HybridPlatformsConductor
     # * *config* (Config): Config to be used. [default = Config.new]
     # * *cmd_runner* (CmdRunner): Command runner to be used. [default = CmdRunner.new]
     # * *nodes_handler* (NodesHandler): Nodes handler to be used. [default = NodesHandler.new]
-    def initialize(logger: Logger.new(STDOUT), logger_stderr: Logger.new(STDERR), config: Config.new, cmd_runner: CmdRunner.new, nodes_handler: NodesHandler.new)
+    def initialize(logger: Logger.new($stdout), logger_stderr: Logger.new($stderr), config: Config.new, cmd_runner: CmdRunner.new, nodes_handler: NodesHandler.new)
       init_loggers(logger, logger_stderr)
       @config = config
       @cmd_runner = cmd_runner
@@ -74,17 +74,17 @@ module HybridPlatformsConductor
       end
       # Display options connectors might have
       @connector_plugins.each do |connector_name, connector|
-        if connector.respond_to?(:options_parse)
-          options_parser.separator ''
-          options_parser.separator "Connector #{connector_name} options:"
-          connector.options_parse(options_parser)
-        end
+        next unless connector.respond_to?(:options_parse)
+
+        options_parser.separator ''
+        options_parser.separator "Connector #{connector_name} options:"
+        connector.options_parse(options_parser)
       end
     end
 
     # Validate that parsed parameters are valid
     def validate_params
-      @connector_plugins.values.each do |connector|
+      @connector_plugins.each_value do |connector|
         connector.validate_params if connector.respond_to?(:validate_params)
       end
     end
@@ -102,7 +102,14 @@ module HybridPlatformsConductor
     # * *progress_name* (String): Name to display on the progress bar [default: 'Executing actions']
     # Result::
     # * Hash<String, [Integer or Symbol, String, String]>: Exit status code (or Symbol in case of error or dry run), standard output and error for each node.
-    def execute_actions(actions_per_nodes, timeout: nil, concurrent: false, log_to_dir: "#{@config.hybrid_platforms_dir}/run_logs", log_to_stdout: true, progress_name: 'Executing actions')
+    def execute_actions(
+      actions_per_nodes,
+      timeout: nil,
+      concurrent: false,
+      log_to_dir: "#{@config.hybrid_platforms_dir}/run_logs",
+      log_to_stdout: true,
+      progress_name: 'Executing actions'
+    )
       # Keep a list of nodes that will need remote access
       nodes_needing_connectors = []
       # Compute the ordered list of actions per selected node
@@ -117,6 +124,7 @@ module HybridPlatformsConductor
           nodes_actions_set.each do |action_type, action_info|
             raise 'Cannot have concurrent executions for interactive sessions' if concurrent && action_type == :interactive && action_info
             raise "Unknown action type #{action_type}" unless @action_plugins.key?(action_type)
+
             action = @action_plugins[action_type].new(
               logger: @logger,
               logger_stderr: @logger_stderr,
@@ -137,7 +145,7 @@ module HybridPlatformsConductor
           actions_per_node[node].concat(resolved_nodes_actions)
         end
       end
-      result = Hash[actions_per_node.keys.map { |node| [node, nil] }]
+      result = actions_per_node.keys.map { |node| [node, nil] }.to_h
       with_connections_prepared_to(nodes_needing_connectors, no_exception: true) do |connected_nodes|
         missing_nodes = []
         connected_nodes.each do |node, connector|
@@ -151,7 +159,7 @@ module HybridPlatformsConductor
         # Prepare the result (stdout or nil per node)
         unless accessible_nodes.empty?
           # If we run in parallel then clone the connectors, so that each node has its own instance for thread-safe code.
-          connected_nodes = Hash[connected_nodes.map { |node, connector| [node, connector.clone] }] if concurrent
+          connected_nodes = connected_nodes.transform_values(&:clone) if concurrent
           @nodes_handler.for_each_node_in(
             accessible_nodes,
             parallel: concurrent,
@@ -185,10 +193,11 @@ module HybridPlatformsConductor
     #     * *connected_nodes* (Hash<String, Connector or Symbol>): Prepared connectors (or Symbol in case of failure with no_exception), per node name
     def with_connections_prepared_to(nodes, no_exception: false)
       # Make sure every node needing connectors finds a connector
-      nodes_needing_connectors = Hash[nodes.map { |node| [node, nil] }]
-      @connector_plugins.each do |connector_name, connector|
+      nodes_needing_connectors = nodes.map { |node| [node, nil] }.to_h
+      @connector_plugins.each_value do |connector|
         nodes_without_connectors = nodes_needing_connectors.select { |_node, selected_connector| selected_connector.nil? }.keys
         break if nodes_without_connectors.empty?
+
         (connector.connectable_nodes_from(nodes_without_connectors) & nodes_without_connectors).each do |node|
           nodes_needing_connectors[node] = connector if nodes_needing_connectors[node].nil?
         end
@@ -206,23 +215,20 @@ module HybridPlatformsConductor
         if connector_name.nil?
           # All plugins have been prepared.
           # Call our client code.
-          yield Hash[nodes_needing_connectors.map do |node, selected_connector|
-            [
-              node,
-              selected_connector.nil? ? :no_connector : selected_connector
-            ]
-          end]
+          yield(nodes_needing_connectors.transform_values do |selected_connector|
+            selected_connector.nil? ? :no_connector : selected_connector
+          end)
         else
           connector = @connector_plugins[connector_name]
           selected_nodes = nodes_needing_connectors.select { |_node, selected_connector| selected_connector == connector }.keys
           if selected_nodes.empty?
-            preparation_code.call(remaining_plugins_to_prepare[1..-1])
+            preparation_code.call(remaining_plugins_to_prepare[1..])
           else
             connector.with_connection_to(selected_nodes, no_exception: no_exception) do |connected_nodes|
               (selected_nodes - connected_nodes).each do |node_in_error|
                 nodes_needing_connectors[node_in_error] = :connection_error
               end
-              preparation_code.call(remaining_plugins_to_prepare[1..-1])
+              preparation_code.call(remaining_plugins_to_prepare[1..])
             end
           end
         end
@@ -262,8 +268,6 @@ module HybridPlatformsConductor
         if log_to_file
           FileUtils.mkdir_p(File.dirname(log_to_file))
           File.open(log_to_file, 'w')
-        else
-          nil
         end
       stdout_queue = Queue.new
       stderr_queue = Queue.new
@@ -277,28 +281,26 @@ module HybridPlatformsConductor
           (log_to_stdout ? [@logger_stderr] : []) +
           (file_output.nil? ? [] : [file_output])
       ) do
-        begin
-          log_debug "[#{node}] - Execute #{actions.size} actions on #{node}..."
-          actions.each do |action|
-            action.prepare_for(node, connector, remaining_timeout, stdout_queue, stderr_queue)
-            start_time = Time.now
-            action.execute
-            remaining_timeout -= Time.now - start_time unless remaining_timeout.nil?
-          end
-        rescue ConnectionError
-          exit_status = :connection_error
-          stderr_queue << "#{$!}\n"
-        rescue CmdRunner::UnexpectedExitCodeError
-          exit_status = :failed_command
-          stderr_queue << "#{$!}\n"
-        rescue CmdRunner::TimeoutError
-          # Error has already been logged in stderr
-          exit_status = :timeout
-        rescue
-          log_error "Uncaught exception while executing actions on #{node}: #{$!}\n#{$!.backtrace.join("\n")}"
-          stderr_queue << "#{$!}\n"
-          exit_status = :failed_action
+        log_debug "[#{node}] - Execute #{actions.size} actions on #{node}..."
+        actions.each do |action|
+          action.prepare_for(node, connector, remaining_timeout, stdout_queue, stderr_queue)
+          start_time = Time.now
+          action.execute
+          remaining_timeout -= Time.now - start_time unless remaining_timeout.nil?
         end
+      rescue ConnectionError
+        exit_status = :connection_error
+        stderr_queue << "#{$ERROR_INFO}\n"
+      rescue CmdRunner::UnexpectedExitCodeError
+        exit_status = :failed_command
+        stderr_queue << "#{$ERROR_INFO}\n"
+      rescue CmdRunner::TimeoutError
+        # Error has already been logged in stderr
+        exit_status = :timeout
+      rescue
+        log_error "Uncaught exception while executing actions on #{node}: #{$ERROR_INFO}\n#{$ERROR_INFO.backtrace.join("\n")}"
+        stderr_queue << "#{$ERROR_INFO}\n"
+        exit_status = :failed_action
       end
       [exit_status, stdout, stderr]
     end

data/lib/hybrid_platforms_conductor/bitbucket.rb

@@ -38,7 +38,7 @@ module HybridPlatformsConductor
     # * *bitbucket_password* (String): Bitbucket password to be used when querying the API
     # * *logger* (Logger): Logger to be used [default = Logger.new(STDOUT)]
     # * *logger_stderr* (Logger): Logger to be used for stderr [default = Logger.new(STDERR)]
-    def initialize(bitbucket_url, bitbucket_user_name, bitbucket_password, logger: Logger.new(STDOUT), logger_stderr: Logger.new(STDERR))
+    def initialize(bitbucket_url, bitbucket_user_name, bitbucket_password, logger: Logger.new($stdout), logger_stderr: Logger.new($stderr))
       init_loggers(logger, logger_stderr)
       @bitbucket_url = bitbucket_url
       @bitbucket_user_name = bitbucket_user_name
@@ -106,10 +106,11 @@ module HybridPlatformsConductor
       http_response = nil
       loop do
         begin
-          http_response = URI.open(api_url, http_basic_authentication: [@bitbucket_user_name, @bitbucket_password])
+          http_response = URI.parse(api_url).open(http_basic_authentication: [@bitbucket_user_name, @bitbucket_password])
         rescue
-          raise if retries == 0
-          log_warn "Got error #{$!} on #{@bitbucket_user_name}@#{api_url}. Will retry #{retries} times..."
+          raise if retries.zero?
+
+          log_warn "Got error #{$ERROR_INFO} on #{@bitbucket_user_name}@#{api_url}. Will retry #{retries} times..."
           retries -= 1
           sleep 1
         end