hybrid_platforms_conductor 32.11.1 → 32.13.2

data/docs/plugins.md

@@ -0,0 +1,215 @@
+# Plugins
+
+Hybrid Platforms Conductor ships with plenty of plugins of any type. The type of the plugin is defined by the directory in which the plugin is encountered.
+
+Check [how to create plugins](plugins_create.md) to know how to add your own plugins to this list.
+
+Following are all possible plugin types and the plugins shipped by default with Hybrid Platforms Conductor.
+
+# Table of Contents
+  * [`action`](#action)
+  * [`cmdb`](#cmdb)
+  * [`connector`](#connector)
+  * [`platform_handler`](#platform_handler)
+  * [`provisioner`](#provisioner)
+  * [`report`](#report)
+  * [`test`](#test)
+  * [`test_report`](#test_report)
+
+<a name="action"></a>
+## Actions
+
+Define the kind of actions that can be executed by various processes.
+
+Corresponding plugin type: `action`.
+
+These plugins are meant to define new action types that can be used by the [`ActionsExecutor`](../lib/hybrid_platforms_conductor/actions_executor.rb).
+
+Examples of actions are:
+* Remote bash: Execute remote bash on the node
+* Ruby: Execute Ruby code
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/action/my_action.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`bash`](plugins/action/bash.md)
+* [`interactive`](plugins/action/interactive.md)
+* [`remote_bash`](plugins/action/remote_bash.md)
+* [`ruby`](plugins/action/ruby.md)
+* [`scp`](plugins/action/scp.md)
+
+<a name="cmdb"></a>
+## CMDBs
+
+Retrieve nodes' metadata from various sources.
+
+Corresponding plugin type: `cmdb`.
+
+These plugins allow to retrieve metadata associated to a node, returned by the [`NodesHandler`](../lib/hybrid_platforms_conductor/nodes_handler.rb). New plugins can be used to retrieve new properties that can then be used by Hybrid Platforms Conductor.
+
+Examples of CMDBs are:
+* Host keys: Get host keys associated to nodes
+* Host IPs: Get a node's host IP
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/cmdb/my_cmdb.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`config`](plugins/cmdb/config.md)
+* [`host_ip`](plugins/cmdb/host_ip.md)
+* [`host_keys`](plugins/cmdb/host_keys.md)
+* [`platform_handlers`](plugins/cmdb/platform_handlers.md)
+
+<a name="connector"></a>
+## Connectors
+
+Give a way to execute remote bash or transfer files to nodes.
+
+Corresponding plugin type: `connector`.
+
+These plugins give ways for the [`ActionsExecutor`](../lib/hybrid_platforms_conductor/actions_executor.rb) to connect to nodes when some actions require it (like the remote code executions for example).
+
+Examples of connectors are:
+* SSH: Connect to a node using SSH
+* Docker: Connect using a Docker socket
+* awscli: Connect using awscli
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/connector/my_connector.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`local`](plugins/connector/local.md)
+* [`ssh`](plugins/connector/ssh.md)
+
+<a name="platform_handler"></a>
+## Platform Handlers
+
+Handle repositories of nodes' inventory and services to be deployed.
+
+Corresponding plugin type: `platform_handler`.
+
+These plugins are used to support different types of platforms' repositories, returned by the [`NodesHandler`](../lib/hybrid_platforms_conductor/nodes_handler.rb)
+
+Platforms are registered in the `./hpc_config.rb` file of your project.
+
+Example from a locally checked out platform:
+```ruby
+<platform_type_name>_platform path: '/path/to/platform/to_be_handled_by_your_plugin'
+```
+
+Example from a platform present in a Git repository:
+```ruby
+<platform_type_name>_platform git: '<git_url_to_the_platform_code>'
+```
+
+Examples of platform handlers are:
+* Chef: Handle a platform using Chef
+* Ansible: Handle a platform using Ansible
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/platform_handler/platform_handler_plugin.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`yaml_inventory`](plugins/platform_handler/yaml_inventory.md)
+
+<a name="provisioner"></a>
+## Provisioners
+
+Give a way to provision new nodes.
+
+Corresponding plugin type: `provisioner`.
+
+These plugins add new ways to provision infrastructure, used by the [`Deployer`](../lib/hybrid_platforms_conductor/deployer.rb)
+
+Examples of provisioners are:
+* Docker: Provision Docker containers
+* Podman: Provision Podman pods
+* Terraform: Provision nodes through Terraform
+* Proxmox: Provision containers or VMs using Proxmox
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/provisioner/my_provisioner.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`docker`](plugins/provisioner/docker.md)
+* [`podman`](plugins/provisioner/podman.md)
+* [`proxmox`](plugins/provisioner/proxmox.md)
+
+<a name="report"></a>
+## Reports
+
+Report inventory and metadata information.
+
+Corresponding plugin type: `report`.
+
+These plugins add new ways to publish inventory reports produced by the [`ReportsHandler`](../lib/hybrid_platforms_conductor/reports_handler.rb)
+
+Examples of reports are:
+* stdout: Just dump inventory on stdout
+* Mediawiki: Dump inventory in a Mediawiki page
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/report/my_report_plugin.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`confluence`](plugins/report/confluence.md)
+* [`mediawiki`](plugins/report/mediawiki.md)
+* [`stdout`](plugins/report/stdout.md)
+
+<a name="test"></a>
+## Tests
+
+Perform various tests, on nodes, on platform repositories, and global ones as well.
+
+Corresponding plugin type: `test`.
+
+These plugins add available tests to the [`TestsRunner`](../lib/hybrid_platforms_conductor/tests_runner.rb).
+Depending on the API they implement, they can define tests at global level, at platform level or at node level.
+
+Examples of tests are:
+* Spectre: Test a node against Spectre vulnerability
+* Executables: Test that executables run without errors
+* Divergence: Test that a node has not diverged from the configuration stored in its platform handler
+
+Check the [sample plugin file](../lib/hybrid_platforms_conductor/hpc_plugins/test/my_test_plugin.rb.sample) to know more about the API that needs to be implemented by such plugins.
+
+Plugins shipped by default:
+* [`bitbucket_conf`](plugins/test/bitbucket_conf.md)
+* [`can_be_checked`](plugins/test/can_be_checked.md)
+* [`check_deploy_and_idempotence`](plugins/test/check_deploy_and_idempotence.md)
+* [`check_from_scratch`](plugins/test/check_from_scratch.md)
+* [`connection`](plugins/test/connection.md)
+* [`deploy_freshness`](plugins/test/deploy_freshness.md)
+* [`deploy_from_scratch`](plugins/test/deploy_from_scratch.md)
+* [`deploy_removes_root_access`](plugins/test/deploy_removes_root_access.md)
+* [`divergence`](plugins/test/divergence.md)
+* [`executables`](plugins/test/executables.md)
+* [`file_system_hdfs`](plugins/test/file_system_hdfs.md)
+* [`file_system`](plugins/test/file_system.md)
+* [`hostname`](plugins/test/hostname.md)
+* [`idempotence`](plugins/test/idempotence.md)
+* [`ip`](plugins/test/ip.md)
+* [`jenkins_ci_conf`](plugins/test/jenkins_ci_conf.md)
+* [`jenkins_ci_masters_ok`](plugins/test/jenkins_ci_masters_ok.md)
+* [`linear_strategy`](plugins/test/linear_strategy.md)
+* [`local_users`](plugins/test/local_users.md)
+* [`mounts`](plugins/test/mounts.md)
+* [`orphan_files`](plugins/test/orphan_files.md)
+* [`ports`](plugins/test/ports.md)
+* [`private_ips`](plugins/test/private_ips.md)
+* [`public_ips`](plugins/test/public_ips.md)
+* [`spectre`](plugins/test/spectre.md)
+* [`veids`](plugins/test/veids.md)
+* [`vulnerabilities`](plugins/test/vulnerabilities.md)
+
+<a name="test_report"></a>
+## Test reports
+
+Report testing results on various mediums.
+
+Corresponding plugin type: `test_report`.
+
+These plugins add new ways to publish tests reports, done by the [`TestsRunner`](../lib/hybrid_platforms_conductor/tests_runner.rb).
+
+Examples of tests reports are:
+* stdout: Just dump tests results on stdout
+* Confluence: Dump tests reports in a Confluence page
+
+Plugins shipped by default:
+* [`confluence`](plugins/test_report/confluence.md)
+* [`stdout`](plugins/test_report/stdout.md)

data/docs/plugins/action/bash.md

@@ -0,0 +1,37 @@
+# Action plugin: `bash`
+
+The `bash` action plugin executes a bash command.
+It takes a simple `String` as parameter that will be executed in a local bash shell.
+
+Exit status, stdout and stderr of the execution can be accessed as a result of the call.
+
+Example:
+```ruby
+require 'hybrid_platforms_conductor/executable'
+
+HybridPlatformsConductor::Executable.new.actions_executor.execute_actions('my_node' => { bash: 'hostname' })
+# => { 'my_node' => [0, "my_hostname\n", '' ] }
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/action/interactive.md

@@ -0,0 +1,37 @@
+# Action plugin: `interactive`
+
+The `interactive` action plugin executes an interactive shell on a node (using a [connector](../connector)).
+It takes no argument, so any value will do (`nil`, `true`...)
+
+Example:
+```ruby
+require 'hybrid_platforms_conductor/executable'
+
+actions_executor = HybridPlatformsConductor::Executable.new.actions_executor
+
+# Launch an interactive shell
+actions_executor.execute_actions('my_node' => { interactive: true })
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/action/remote_bash.md

@@ -0,0 +1,67 @@
+# Action plugin: `remote_bash`
+
+The `remote_bash` action plugin executes bash commands on a node (using a [connector](../connector)).
+It takes various kinds of arguments:
+* `String`: The bash command to execute.
+* `Array<String>`: A list of bash commands to execute, sequentially.
+* `Hash<Symbol, Object>`: A hash of properties describing the commands to execute in detail:
+  * **commands** (`Array<String>` or `String`): List of bash commands to execute (can be a single one). This is the default property also that allows to not use the Hash form for brevity.
+  * **file** (`String`): Name of a file from which commands should be taken.
+  * **env** (`Hash<String, String>`): Environment variables to be set before executing those commands.
+
+Exit status, stdout and stderr of the execution can be accessed as a result of the call.
+
+Example:
+```ruby
+require 'hybrid_platforms_conductor/executable'
+
+actions_executor = HybridPlatformsConductor::Executable.new.actions_executor
+
+# Execute 1 command on the node
+actions_executor.execute_actions('my_node' => { remote_bash: 'hostname' })
+# => { 'my_node' => [0, "my_node\n", '' ] }
+# In case of connection error:
+# => { 'my_node' => [:connection_error, '', 'Unable to get a connector to my_node'] }
+
+# Execute several commands
+actions_executor.execute_actions('my_node' => { remote_bash: [
+  'echo Hello',
+  'hostname',
+  'ls'
+]})
+
+# Execute commands from a file
+actions_executor.execute_actions('my_node' => { remote_bash: { file: '/path/to/my/file.cmds' } })
+
+# Execute commands with environment variables set
+actions_executor.execute_actions('my_node' => { remote_bash: {
+  commands: 'echo Hello ${world}',
+  env: {
+    'world' => 'my World'
+  }
+} })
+
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/action/ruby.md

@@ -0,0 +1,69 @@
+# Action plugin: `ruby`
+
+The `ruby` action plugin executes Ruby code.
+It takes various kinds of arguments:
+* `Proc`: The Ruby block to execute
+* `Hash<Symbol, Object>`: A hash of properties describing how Ruby code is to be executed:
+  * **code** (`Proc`): Ruby code to be executed. This is the default property, and can be given directly without using a Hash.
+  * **need_remote** (`Boolean`): Do we need a remote connection to the node for this code to run? [default = false]
+
+The Ruby code block has the following signature:
+* **stdout** (`IO`): Stream in which stdout of this action should be written.
+* **stderr** (`IO`): Stream in which stderr of this action should be written.
+* **action** (`Action`): Action we can use to access other context-specific methods, such as run_cmd.
+* **connector** (`Connector` or `nil`): The connector to the node, or nil if none.
+
+Example:
+```ruby
+require 'hybrid_platforms_conductor/executable'
+
+actions_executor = HybridPlatformsConductor::Executable.new.actions_executor
+
+# Execute a simple Ruby block
+actions_executor.execute_actions('my_node' => { ruby: proc do
+  puts 'Hello'
+end })
+# => Hello
+
+# Execute a Ruby block that logs on stdout and stderr (those are returned by the action execution)
+actions_executor.execute_actions('my_node' => { ruby: proc do |stdout, stderr|
+  stdout << 'Hello'
+  stderr << 'Hello on stderr'
+end })
+# => { 'my_node' => [0, 'Hello', 'Hello on stderr'] }
+
+# Execute a Ruby block that needs a connection to the node, and uses it
+actions_executor.execute_actions('my_node' => { ruby: {
+  code: proc do |stdout, stderr, action, connector|
+    # If we are connecting to my_node using SSH, change the user
+    if connector.is_a?(HybridPlatformsConductor::HpcPlugins::Connector::Ssh)
+      stdout << "The SSH user is #{connector.ssh_user}"
+    end
+  end,
+  need_remote: true
+} })
+# => { 'my_node' => [0, 'The SSH user is my_remote_user', ''] }
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None

data/docs/plugins/action/scp.md

@@ -0,0 +1,61 @@
+# Action plugin: `scp`
+
+The `scp` action plugin transfers a local file to a remote node (using a [connector](../connector)).
+It takes a `Hash` as argument, as a set of source => destination_dir to copy files or directories from the local file system to the remote file system.
+The hash can also contain the following properties:
+* **sudo** (`Boolean`): Do we use sudo on the remote to make the copy? [default: false]
+* **owner** (`String` or `nil`): Owner to use for files, or nil to use current one [default: nil]
+* **group** (`String` or `nil`): Group to use for files, or nil to use current one [default: nil]
+
+Example:
+```ruby
+require 'hybrid_platforms_conductor/executable'
+
+actions_executor = HybridPlatformsConductor::Executable.new.actions_executor
+
+# Copy 1 file
+actions_executor.execute_actions('my_node' => { scp: { '/path/to/file' => '/path/to/remote_dir' } })
+
+# Copy several files
+actions_executor.execute_actions('my_node' => { scp: {
+  '/path/to/file1' => '/path/to/remote_dir1',
+  '/path/to/file2' => '/path/to/remote_dir1',
+  '/path/to/file1' => '/path/to/remote_dir2',
+} })
+
+# Copy a file using sudo on my_node
+actions_executor.execute_actions('my_node' => { scp: {
+  '/path/to/file' => '/path/to/remote_dir',
+  sudo: true
+} })
+
+# Copy a file and set it as a specific owner and group on my_node
+actions_executor.execute_actions('my_node' => { scp: {
+  '/path/to/file' => '/path/to/remote_dir',
+  owner: 'remote_user',
+  group: 'remote_group'
+} })
+```
+
+## Config DSL extension
+
+None
+
+## Used credentials
+
+| Credential | Usage
+| --- | --- |
+
+## Used Metadata
+
+| Metadata | Type | Usage
+| --- | --- | --- |
+
+## Used environment variables
+
+| Variable | Usage
+| --- | --- |
+
+## External tools dependencies
+
+None