hybrid_platforms_conductor 32.16.3 → 33.0.0

data/docs/plugins/secrets_reader/cli.md

@@ -0,0 +1,31 @@
+# Secrets reader plugin: `cli`
+
+The `cli` secrets reader plugin reads secrets from a local JSON file that can be given through the `--secrets` command-line parameter.
+
+Example:
+```bash
+./bin/deploy --node my_node --secrets /path/to/my_secrets.json
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/secrets_reader/thycotic.md

@@ -0,0 +1,46 @@
+# Secrets reader plugin: `thycotic`
+
+The `thycotic` secrets reader plugin retrieves secrets from a [Thycotic secrets server](https://thycotic.com/products/secret-server-vdo/), using its SOAP API.
+
+It is configured using the `secrets_from_thycotic` (see below) config DSL and uses the `thycotic` credential ID to authenticate.
+
+## Config DSL extension
+
+### `secrets_from_thycotic`
+
+Define a Thycotic URL and Thycotic secret ID to fetch from a Thycotic server.
+The Thycotic secret should contain a JSON file that will be retrieved locally to be used as a secrets source. The local copy will then be removed after deployment.
+
+Can be applied to subset of nodes using the [`for_nodes` DSL method](/docs/config_dsl.md#for_nodes).
+
+It takes the following parameters:
+* **thycotic_url** (`String`): The Thycotic server URL.
+* **secret_id** (`Integer`): The Thycotic secret ID containing the secrets file to be used as secrets.
+
+Example:
+```ruby
+secrets_from_thycotic(
+  thycotic_url: 'https://my-thycotic-server.my-domain.com/SecretServer',
+  secret_id: 1107
+)
+```
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+| `thycotic` | Used to authenticate on the Thycotic server's SOAP API |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/test/bitbucket_conf.md

@@ -12,7 +12,7 @@ Define a Bitbucket installation to be targeted.
 It takes the following parameters:
 * **url** (`String`): URL to the Bitbucket server
 * **project** (`String`): Project name from the Bitbucket server, storing repositories
-* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or :all for all [default: :all]
+* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or `:all` for all [default: `:all`]
 * **checks** (`Hash<Symbol, Object>`): Checks definition to be perform on those repositories [default: {}]
   * **branch_permissions** (`Array< Hash<Symbol, Object> >`): List of branch permissions to check [optional]
     * **type** (`String`): Type of branch permissions to check. Examples of values are 'fast-forward-only', 'no-deletes', 'pull-request-only'.

data/docs/plugins/test/check_deploy_and_idempotence.md

@@ -49,7 +49,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
-| `root_access_allowed` | `String` | If set to `true`, then skip the test for `root` access being disabled after deployment |
+| `root_access_allowed` | `Boolean` | If set to `true`, then skip the test for `root` access being disabled after deployment |
 
 ## Used environment variables
 

data/docs/plugins/test/connection.md

@@ -16,6 +16,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/deploy_removes_root_access.md

@@ -17,7 +17,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
-| `root_access_allowed` | `String` | If set to `true`, then skip this test |
+| `root_access_allowed` | `Boolean` | If set to `true`, then skip this test |
 
 ## Used environment variables
 

data/docs/plugins/test/file_system.md

@@ -38,6 +38,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/github_ci.md

@@ -0,0 +1,48 @@
+# Test plugin: `github_ci`
+
+The `github_ci` test plugin checks that the `master` branch of Github repositories has a successful CI result from its [Github Actions](https://github.com/features/actions).
+
+## Config DSL extension
+
+### `github_repos`
+
+Define Github repositories to be targeted.
+
+It takes the following parameters:
+* **url** (`String`): URL to the Github API [default: `'https://api.github.com'`]
+* **user** (`String`): User or organization name, storing repositories
+* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or `:all` for all [default: `:all`]
+
+Example:
+```ruby
+github_repos(
+  # Github's user containing repositories
+  user: 'My-Github-User',
+  # List of repositories to check
+  repos: [
+    'my-platform-repo',
+    'my-chef-repo',
+    'my-hpc-plugins'
+  ]
+)
+```
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+| `github` | Used to connect to the Github API. Password should be the Github API token. |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/test/hostname.md

@@ -16,6 +16,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/ip.md

@@ -16,6 +16,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 | `private_ips` | `Array<String>` | List of possible private IPs the node should have |
 
 ## Used environment variables

data/docs/plugins/test/jenkins_ci_conf.md

@@ -12,7 +12,7 @@ It takes the following parameters:
 * **url** (`String`): URL to the Bitbucket server
 * **project** (`String`): Project name from the Bitbucket server, storing repositories
 * **jenkins_ci_url** (`String` or `nil`): Corresponding Jenkins CI URL, or nil if none.
-* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or :all for all [default: :all]
+* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or `:all` for all [default: `:all`]
 
 Example:
 ```ruby

data/docs/plugins/test/jenkins_ci_masters_ok.md

@@ -12,7 +12,7 @@ It takes the following parameters:
 * **url** (`String`): URL to the Bitbucket server
 * **project** (`String`): Project name from the Bitbucket server, storing repositories
 * **jenkins_ci_url** (`String` or `nil`): Corresponding Jenkins CI URL, or nil if none.
-* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or :all for all [default: :all]
+* **repos** (`Array<String>` or `Symbol`): List of repository names from this project, or `:all` for all [default: `:all`]
 
 Example:
 ```ruby

data/docs/plugins/test/local_users.md

@@ -37,6 +37,7 @@ check_local_users_do_not_exist %w[olduser1 olduser2]
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/mounts.md

@@ -44,6 +44,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/orphan_files.md

@@ -27,6 +27,7 @@ end
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/ports.md

@@ -39,6 +39,7 @@ check_closed_ports 25, 110
 | Metadata | Type | Usage
 | --- | --- | --- |
 | `host_ip` | `String` | Host IP address to be tested for port listening |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/spectre.md

@@ -15,6 +15,7 @@ None
 
 | Metadata | Type | Usage
 | --- | --- | --- |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/docs/plugins/test/vulnerabilities.md

@@ -54,6 +54,7 @@ None
 | Metadata | Type | Usage
 | --- | --- | --- |
 | `image` | `String` | The name of the OS image to be used. The [configuration](../../config_dsl.md) should define the image and point it to a directory containing a `oval.json` that will contain definition of OVAL files to be checked for this OS(see above). |
+| `local_node` | `Boolean` | Skip this test for nodes having this metadata set to `true` |
 
 ## Used environment variables
 

data/lib/hybrid_platforms_conductor/actions_executor.rb

@@ -102,7 +102,14 @@ module HybridPlatformsConductor
     # * *progress_name* (String): Name to display on the progress bar [default: 'Executing actions']
     # Result::
     # * Hash<String, [Integer or Symbol, String, String]>: Exit status code (or Symbol in case of error or dry run), standard output and error for each node.
-    def execute_actions(actions_per_nodes, timeout: nil, concurrent: false, log_to_dir: "#{@config.hybrid_platforms_dir}/run_logs", log_to_stdout: true, progress_name: 'Executing actions')
+    def execute_actions(
+      actions_per_nodes,
+      timeout: nil,
+      concurrent: false,
+      log_to_dir: "#{@config.hybrid_platforms_dir}/run_logs",
+      log_to_stdout: true,
+      progress_name: 'Executing actions'
+    )
       # Keep a list of nodes that will need remote access
       nodes_needing_connectors = []
       # Compute the ordered list of actions per selected node

data/lib/hybrid_platforms_conductor/common_config_dsl/github.rb

@@ -0,0 +1,62 @@
+require 'octokit'
+require 'hybrid_platforms_conductor/credentials'
+
+module HybridPlatformsConductor
+
+  module CommonConfigDsl
+
+    module Github
+
+      # Initialize the DSL 
+      def init_github
+        # List of Github repositories definitions
+        # Array< Hash<Symbol, Object> >
+        # Each definition is just mapping the signature of #github_repos
+        @github_repos = []
+      end
+
+      # Register new Github repositories
+      #
+      # Parameters::
+      # * *url* (String): URL to the Github API [default: 'https://api.github.com']
+      # * *user* (String): User or organization name, storing repositories
+      # * *repos* (Array<String> or Symbol): List of repository names from this project, or :all for all [default: :all]
+      def github_repos(url: 'https://api.github.com', user:, repos: :all)
+        @github_repos << {
+          url: url,
+          user: user,
+          repos: repos
+        }
+      end
+
+      # Iterate over each Github repository
+      #
+      # Parameters::
+      # * Proc: Code called for each Github repository:
+      #   * Parameters::
+      #     * *github* (Octokit::Client): The client instance accessing the Github API
+      #     * *repo_info* (Hash<Symbol, Object>): The repository info:
+      #       * *name* (String): Repository name.
+      #       * *slug* (String): Repository slug.
+      def for_each_github_repo
+        @github_repos.each do |repo_info|
+          Octokit.configure do |c|
+            c.api_endpoint = repo_info[:url]
+          end
+          Credentials.with_credentials_for(:github, @logger, @logger_stderr, url: repo_info[:url]) do |_github_user, github_token|
+            client = Octokit::Client.new(access_token: github_token)
+            (repo_info[:repos] == :all ? client.repositories(repo_info[:user]).map { |repo| repo[:name] } : repo_info[:repos]).each do |name|
+              yield client, {
+                name: name,
+                slug: "#{repo_info[:user]}/#{name}"
+              }
+            end
+          end
+        end
+      end
+
+    end
+
+  end
+
+end

data/lib/hybrid_platforms_conductor/deployer.rb

@@ -11,7 +11,6 @@ require 'hybrid_platforms_conductor/logger_helpers'
 require 'hybrid_platforms_conductor/nodes_handler'
 require 'hybrid_platforms_conductor/services_handler'
 require 'hybrid_platforms_conductor/plugins'
-require 'hybrid_platforms_conductor/thycotic'
 
 module HybridPlatformsConductor
 
@@ -21,12 +20,26 @@ module HybridPlatformsConductor
     # Extend the Config DSL
     module ConfigDSLExtension
 
+      # List of log plugins. Each info has the following properties:
+      # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by this rule.
+      # * *log_plugins* (Array<Symbol>): List of log plugins to be used to store deployment logs.
+      # Array< Hash<Symbol, Object> >
+      attr_reader :deployment_logs
+
+      # List of secrets reader plugins. Each info has the following properties:
+      # * *nodes_selectors_stack* (Array<Object>): Stack of nodes selectors impacted by this rule.
+      # * *secrets_readers* (Array<Symbol>): List of log plugins to be used to store deployment logs.
+      # Array< Hash<Symbol, Object> >
+      attr_reader :secrets_readers
+
       # Integer: Timeout (in seconds) for packaging repositories
       attr_reader :packaging_timeout_secs
 
       # Mixin initializer
       def init_deployer_config
         @packaging_timeout_secs = 60
+        @deployment_logs = []
+        @secrets_readers = []
       end
 
       # Set the packaging timeout
@@ -37,6 +50,28 @@ module HybridPlatformsConductor
         @packaging_timeout_secs = packaging_timeout_secs
       end
 
+      # Set the deployment log plugins to be used
+      #
+      # Parameters::
+      # * *log_plugins* (Symbol or Array<Symbol>): The list of (or single) log plugins to be used
+      def send_logs_to(*log_plugins)
+        @deployment_logs << {
+          nodes_selectors_stack: current_nodes_selectors_stack,
+          log_plugins: log_plugins.flatten
+        }
+      end
+
+      # Set the secrets readers
+      #
+      # Parameters::
+      # * *secrets_readers* (Symbol or Array<Symbol>): The list of (or single) secrets readers plugins to be used
+      def read_secrets_from(*secrets_readers)
+        @secrets_readers << {
+          nodes_selectors_stack: current_nodes_selectors_stack,
+          secrets_readers: secrets_readers.flatten
+        }
+      end
+
     end
 
     include LoggerHelpers
@@ -55,10 +90,6 @@ module HybridPlatformsConductor
     #   Boolean
     attr_accessor :concurrent_execution
 
-    # The list of JSON secrets
-    #   Array<Hash>
-    attr_accessor :secrets
-
     # Are we deploying in a local environment?
     #   Boolean
     attr_accessor :local_environment
@@ -92,8 +123,36 @@ module HybridPlatformsConductor
       @nodes_handler = nodes_handler
       @actions_executor = actions_executor
       @services_handler = services_handler
-      @secrets = []
+      @override_secrets = nil
+      @secrets_readers = Plugins.new(
+        :secrets_reader,
+        logger: @logger,
+        logger_stderr: @logger_stderr,
+        init_plugin: proc do |plugin_class|
+          plugin_class.new(
+            logger: @logger,
+            logger_stderr: @logger_stderr,
+            config: @config,
+            cmd_runner: @cmd_runner,
+            nodes_handler: @nodes_handler
+          )
+        end
+      )
       @provisioners = Plugins.new(:provisioner, logger: @logger, logger_stderr: @logger_stderr)
+      @log_plugins = Plugins.new(
+        :log,
+        logger: @logger,
+        logger_stderr: @logger_stderr,
+        init_plugin: proc do |plugin_class|
+          plugin_class.new(
+            logger: @logger,
+            logger_stderr: @logger_stderr,
+            config: @config,
+            nodes_handler: @nodes_handler,
+            actions_executor: @actions_executor
+          )
+        end
+      )
       # Default values
       @use_why_run = false
       @timeout = nil
@@ -112,30 +171,6 @@ module HybridPlatformsConductor
     def options_parse(options_parser, parallel_switch: true, why_run_switch: false, timeout_options: true)
       options_parser.separator ''
       options_parser.separator 'Deployer options:'
-      options_parser.on(
-        '-e', '--secrets SECRETS_LOCATION',
-        'Specify a secrets location. Can be specified several times. Location can be:',
-        '* Local path to a JSON file',
-        '* URL of the form http[s]://<url>:<secret_id> to get a secret JSON file from a Thycotic Secret Server at the given URL.'
-      ) do |secrets_location|
-        @secrets << JSON.parse(
-          if secrets_location =~ /^(https?:\/\/.+):(\d+)$/
-            url = $1
-            secret_id = $2
-            secret = nil
-            Thycotic.with_thycotic(url, @logger, @logger_stderr) do |thycotic|
-              secret_file_item_id = thycotic.get_secret(secret_id).dig(:secret, :items, :secret_item, :id)
-              raise "Unable to fetch secret file ID #{secrets_location}" if secret_file_item_id.nil?
-              secret = thycotic.download_file_attachment_by_item_id(secret_id, secret_file_item_id)
-              raise "Unable to fetch secret file attachment from #{secrets_location}" if secret.nil?
-            end
-            secret
-          else
-            raise "Missing secret file: #{secrets_location}" unless File.exist?(secrets_location)
-            File.read(secrets_location)
-          end
-        )
-      end
       options_parser.on('-p', '--parallel', 'Execute the commands in parallel (put the standard output in files <hybrid-platforms-dir>/run_logs/*.stdout)') do
         @concurrent_execution = true
       end if parallel_switch
@@ -148,6 +183,14 @@ module HybridPlatformsConductor
       options_parser.on('--retries-on-error NBR', "Number of retries in case of non-deterministic errors (defaults to #{@nbr_retries_on_error})") do |nbr_retries|
         @nbr_retries_on_error = nbr_retries.to_i
       end
+      # Display options secrets readers might have
+      @secrets_readers.each do |secret_reader_name, secret_reader|
+        if secret_reader.respond_to?(:options_parse)
+          options_parser.separator ''
+          options_parser.separator "Secrets reader #{secret_reader_name} options:"
+          secret_reader.options_parse(options_parser)
+        end
+      end
     end
 
     # Validate that parsed parameters are valid
@@ -158,6 +201,16 @@ module HybridPlatformsConductor
     # String: File used as a Futex for packaging
     PACKAGING_FUTEX_FILE = "#{Dir.tmpdir}/hpc_packaging"
 
+    # Override the secrets with a given JSON.
+    # When using this method with a secrets Hash, further deployments will not query secrets readers, but will use those secrets directly.
+    # Useful to override secrets in test conditions when using dummy secrets for example.
+    #
+    # Parameters::
+    # * *secrets* (Hash or nil): Secrets to take into account in place of secrets readers, or nil to cancel a previous overriding and use secrets readers instead.
+    def override_secrets(secrets)
+      @override_secrets = secrets
+    end
+
     # Deploy on a given list of nodes selectors.
     # The workflow is the following:
     # 1. Package the services to be deployed, considering the nodes, services and context (options, secrets, environment...)
@@ -177,10 +230,20 @@ module HybridPlatformsConductor
 
       # Get the secrets to be deployed
       secrets = {}
-      @secrets.each do |secret_json|
-        secrets.merge!(secret_json) do |key, value1, value2|
-          raise "Secret #{key} has conflicting values between different secret JSON files." if value1 != value2
-          value1
+      if @override_secrets
+        secrets = @override_secrets
+      else
+        services_to_deploy.each do |node, services|
+          # If there is no config for secrets, just use cli
+          (@config.secrets_readers.empty? ? [{ secrets_readers: %i[cli] }] : @nodes_handler.select_confs_for_node(node, @config.secrets_readers)).inject([]) do |secrets_readers, secrets_readers_info|
+            secrets_readers + secrets_readers_info[:secrets_readers]
+          end.sort.uniq.each do |secrets_reader|
+            services.each do |service|
+              node_secrets = @secrets_readers[secrets_reader].secrets_for(node, service)
+              conflicting_path = safe_merge(secrets, node_secrets)
+              raise "Secret set at path #{conflicting_path.join('->')} by #{secrets_reader} for service #{service} on node #{node} has conflicting values (#{log_debug? ? "#{node_secrets.dig(*conflicting_path)} != #{secrets.dig(*conflicting_path)}" : 'set debug for value details'})." unless conflicting_path.nil?
+            end
+          end
         end
       end
 
@@ -319,7 +382,7 @@ module HybridPlatformsConductor
         )
         instance.with_running_instance(stop_on_exit: true, destroy_on_exit: !reuse_instance, port: 22) do
           # Test-provisioned nodes have SSH Session Exec capabilities and are not local
-          sub_executable.nodes_handler.override_metadata_of node, :ssh_session_exec, 'true'
+          sub_executable.nodes_handler.override_metadata_of node, :ssh_session_exec, true
           sub_executable.nodes_handler.override_metadata_of node, :local_node, false
           # Test-provisioned nodes use default sudo
           sub_executable.config.sudo_procs.replace(sub_executable.config.sudo_procs.map do |sudo_proc_info|
@@ -338,7 +401,7 @@ module HybridPlatformsConductor
           deployer.local_environment = true
           # Ignore secrets that might have been given: in Docker containers we always use dummy secrets
           dummy_secrets_file = "#{@config.hybrid_platforms_dir}/dummy_secrets.json"
-          deployer.secrets = File.exist?(dummy_secrets_file) ? [JSON.parse(File.read(dummy_secrets_file))] : []
+          deployer.override_secrets(File.exist?(dummy_secrets_file) ? JSON.parse(File.read(dummy_secrets_file)) : {})
           yield deployer, instance
         end
       rescue
@@ -355,72 +418,31 @@ module HybridPlatformsConductor
     # * *nodes* (Array<String>): Nodes to get info from
     # Result::
     # * Hash<String, Hash<Symbol,Object>: The deployed info, per node name.
-    #   Properties are defined by the Deployer#save_logs method, and additionally to them the following properties can be set:
-    #   * *error* (String): Optional property set in case of error
+    #   * *error* (String): Error string in case deployment logs could not be retrieved. If set then further properties will be ignored. [optional]
+    #   * *services* (Array<String>): List of services deployed on the node
+    #   * *deployment_info* (Hash<Symbol,Object>): Deployment metadata
+    #   * *exit_status* (Integer or Symbol): Deployment exit status
+    #   * *stdout* (String): Deployment stdout
+    #   * *stderr* (String): Deployment stderr
     def deployment_info_from(*nodes)
+      nodes = nodes.flatten
       @actions_executor.max_threads = 64
-      Hash[@actions_executor.
-        execute_actions(
-          Hash[nodes.flatten.map do |node|
-            [
-              node,
-              { remote_bash: "cd /var/log/deployments && ls -t | head -1 | xargs sed '/===== STDOUT =====/q'" }
-            ]
-          end],
-          log_to_stdout: false,
-          concurrent: true,
-          timeout: 10,
-          progress_name: 'Getting deployment info'
-        ).
-        map do |node, (exit_status, stdout, stderr)|
-          # Expected format for stdout:
-          # Property1: Value1
-          # ...
-          # PropertyN: ValueN
-          # ===== STDOUT =====
-          # ...
-          deploy_info = {}
-          if exit_status.is_a?(Symbol)
-            deploy_info[:error] = "Error: #{exit_status}\n#{stderr}"
-          else
-            stdout_lines = stdout.split("\n")
-            if stdout_lines.first =~ /No such file or directory/
-              deploy_info[:error] = '/var/log/deployments missing'
-            else
-              stdout_lines.each do |line|
-                if line =~ /^([^:]+): (.+)$/
-                  key_str, value = $1, $2
-                  key = key_str.to_sym
-                  # Type-cast some values
-                  case key_str
-                  when 'date'
-                    # Date and time values
-                    # Thu Nov 23 18:43:01 UTC 2017
-                    deploy_info[key] = Time.parse(value)
-                  when 'debug'
-                    # Boolean values
-                    # Yes
-                    deploy_info[key] = (value == 'Yes')
-                  when /^diff_files_.+$/, 'services'
-                    # Array of strings
-                    # my_file.txt, other_file.txt
-                    deploy_info[key] = value.split(', ')
-                  else
-                    deploy_info[key] = value
-                  end
-                else
-                  deploy_info[:unknown_lines] = [] unless deploy_info.key?(:unknown_lines)
-                  deploy_info[:unknown_lines] << line
-                end
-              end
-            end
-          end
-          [
-            node,
-            deploy_info
-          ]
-        end
-      ]
+      read_actions_results = @actions_executor.execute_actions(
+        Hash[nodes.map do |node|
+          master_log_plugin = @log_plugins[log_plugins_for(node).first]
+          master_log_plugin.respond_to?(:actions_to_read_logs) ? [node, master_log_plugin.actions_to_read_logs(node)] : nil
+        end.compact],
+        log_to_stdout: false,
+        concurrent: true,
+        timeout: 10,
+        progress_name: 'Read deployment logs'
+      )
+      Hash[nodes.map do |node|
+        [
+          node,
+          @log_plugins[log_plugins_for(node).first].logs_for(node, *(read_actions_results[node] || [nil, nil, nil]))
+        ]
+      end]
     end
 
     # Parse stdout and stderr of a given deploy run and get the list of tasks with their status
@@ -442,6 +464,35 @@ module HybridPlatformsConductor
 
     private
 
+    # Safe-merge 2 hashes.
+    # Safe-merging is done by:
+    # * Merging values that are hashes.
+    # * Reporting errors when values conflict.
+    # When values are conflicting, the initial hash won't modify those conflicting values and will stop the merge.
+    #
+    # Parameters::
+    # * *hash* (Hash): Hash to be modified merging hash_to_merge
+    # * *hash_to_merge* (Hash): Hash to be merged into hash
+    # Result::
+    # * nil or Array<Object>: nil in case of success, or the keys path leading to a conflicting value in case of error
+    def safe_merge(hash, hash_to_merge)
+      conflicting_path = nil
+      hash_to_merge.each do |key, value_to_merge|
+        if hash.key?(key)
+          if hash[key].is_a?(Hash) && value_to_merge.is_a?(Hash)
+            sub_conflicting_path = safe_merge(hash[key], value_to_merge)
+            conflicting_path = [key] + sub_conflicting_path unless sub_conflicting_path.nil?
+          elsif hash[key] != value_to_merge
+            conflicting_path = [key]
+          end
+        else
+          hash[key] = value_to_merge
+        end
+        break unless conflicting_path.nil?
+      end
+      conflicting_path
+    end
+
     # Get the list of retriable errors a node got from deployment logs.
     # Useful to know if an error is non-deterministic (due to external and temporary factors).
     #
@@ -494,7 +545,7 @@ module HybridPlatformsConductor
         Hash[services.map do |node, node_services|
           image_id = @nodes_handler.get_image_of(node)
           sudo = (ssh_user == 'root' ? '' : "#{@nodes_handler.sudo_on(node)} ")
-          # Install My_company corporate certificates if present
+          # Install corporate certificates if present
           certificate_actions =
             if @local_environment && ENV['hpc_certificates']
               if File.exist?(ENV['hpc_certificates'])
@@ -584,47 +635,48 @@ module HybridPlatformsConductor
     # * *services* (Hash<String, Array<String>>): List of services that have been deployed, per node
     def save_logs(logs, services)
       section "Saving deployment logs for #{logs.size} nodes" do
-        Dir.mktmpdir('hybrid_platforms_conductor-logs') do |tmp_dir|
-          ssh_user = @actions_executor.connector(:ssh).ssh_user
-          @actions_executor.execute_actions(
-            Hash[logs.map do |node, (exit_status, stdout, stderr)|
-              # Create a log file to be scp with all relevant info
-              now = Time.now.utc
-              log_file = "#{tmp_dir}/#{node}_#{now.strftime('%F_%H%M%S')}_#{ssh_user}"
-              services_info = @services_handler.log_info_for(node, services[node])
-              File.write(
-                log_file,
-                services_info.merge(
-                  date: now.strftime('%F %T'),
-                  user: ssh_user,
-                  debug: log_debug? ? 'Yes' : 'No',
-                  services: services[node].join(', '),
-                  exit_status: exit_status
-                ).map { |property, value| "#{property}: #{value}" }.join("\n") +
-                  "\n===== STDOUT =====\n" +
-                  (stdout || '') +
-                  "\n===== STDERR =====\n" +
-                  (stderr || '')
-              )
-              [
-                node,
-                {
-                  remote_bash: "#{ssh_user == 'root' ? '' : "#{@nodes_handler.sudo_on(node)} "}mkdir -p /var/log/deployments",
-                  scp: {
-                    log_file => '/var/log/deployments',
-                    :sudo => ssh_user != 'root',
-                    :owner => 'root',
-                    :group => 'root'
-                  }
-                }
-              ]
-            end],
-            timeout: 10,
-            concurrent: true,
-            log_to_dir: nil
-          )
-        end
+        ssh_user = @actions_executor.connector(:ssh).ssh_user
+        @actions_executor.execute_actions(
+          Hash[logs.map do |node, (exit_status, stdout, stderr)|
+            [
+              node,
+              log_plugins_for(node).
+                map do |log_plugin|
+                  @log_plugins[log_plugin].actions_to_save_logs(
+                    node,
+                    services[node],
+                    @services_handler.log_info_for(node, services[node]).merge(
+                      date: Time.now.utc.strftime('%F %T'),
+                      user: ssh_user
+                    ),
+                    exit_status,
+                    stdout,
+                    stderr
+                  )
+                end.
+                flatten(1)
+            ]
+          end],
+          timeout: 10,
+          concurrent: true,
+          log_to_dir: nil,
+          progress_name: 'Saving logs'
+        )
+      end
+    end
+
+    # Get the list of log plugins to be used for a given node
+    #
+    # Parameters::
+    # * *node* (String): The node for which log plugins are queried
+    # Result::
+    # * Array<Symbol>: The list of log plugins
+    def log_plugins_for(node)
+      node_log_plugins = @nodes_handler.select_confs_for_node(node, @config.deployment_logs).inject([]) do |log_plugins, deployment_logs_info|
+        log_plugins + deployment_logs_info[:log_plugins]
       end
+      node_log_plugins << :remote_fs if node_log_plugins.empty?
+      node_log_plugins
     end
 
   end