hybrid_platforms_conductor 32.13.0 → 32.13.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 33b6e3bc5318de72c2f4d6c9ee0d092ba1cf078befdf07836765b10398dc0513
-  data.tar.gz: '02068aaed645f9897f265da954817ae489dcff55f59e8ede8a179905594230db'
+  metadata.gz: 727927dd5825fdf3350a93fbc918d4d21fe5e24c89138bfc28315eb59b859054
+  data.tar.gz: 854722e9cfe5fe96df75c2bc1ef37afb659365809e9a6c4eb06fdc5295deb100
 SHA512:
-  metadata.gz: 8307851f71ca547872c97772a7890cd3c7db399260e458abb74a226e96b23f51baad3905e74f586c20cf9b8c0ecd658a3b9df1f28c250112476d1e0328f01364
-  data.tar.gz: 129f0b7f6c06c15465a3922374531e4bed7cd1fcc67081dcfe04ead71d5b600c247dcaa4803b05ecca8be26df5c745b4b4cbf7cab9e95d40accd16096e22dbe3
+  metadata.gz: 81e5c8928a3f03b1379bfb9136e927d8f39d6fd193068f24cddb90e671e4d9db21538dd87a566d7541e52b68e565f5a336e6afde98565ffd00e75b1ebf5d9ff8
+  data.tar.gz: dcb51c4dce6027f6153556a71671f4ae2cdfa29ad89d4c5ebca317a03455dd5c155ac0c00489a6c88b83fb222689ecf7fba05ef0706a33960970ef782e671a28

data/CHANGELOG.md

@@ -1,3 +1,9 @@
+# [v32.13.1](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v32.13.0...v32.13.1) (2021-05-06 14:03:16)
+
+### Patches
+
+* [Add tutorial files in examples](https://github.com/sweet-delights/hybrid-platforms-conductor/commit/dfb456f031d48ba1ed6922a5f84ed659fe587ab8)
+
 # [v32.13.0](https://github.com/sweet-delights/hybrid-platforms-conductor/compare/v32.12.0...v32.13.0) (2021-05-06 11:20:57)
 
 ### Features

data/README.md

@@ -246,7 +246,7 @@ So with the concepts described above, the process described as deploying **servi
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/README.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/check-node.md

@@ -12,7 +12,7 @@ It will:
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/check-node.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/deploy.md

@@ -12,7 +12,7 @@ It will:
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/deploy.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/free_ips.md

@@ -10,7 +10,7 @@ Useful to assign new IPs.
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/free_ips.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/free_veids.md

@@ -10,7 +10,7 @@ Useful to assign unused VEIDs to new VMs to be created.
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/free_veids.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/get_impacted_nodes.md

@@ -8,7 +8,7 @@ This is especially useful to know which nodes have to be tested against a given
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/get_impacted_nodes.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/last_deploys.md

@@ -7,7 +7,7 @@ The `last_deploys` executable will fetch the last deployments information for a
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/last_deploys.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/nodes_to_deploy.md

@@ -11,7 +11,7 @@ The list of nodes is given in standard output.
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/nodes_to_deploy.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/report.md

@@ -8,7 +8,7 @@ This executable is using [report plugins](../plugins.md#report) so that the tool
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/report.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/run.md

@@ -8,7 +8,7 @@ It will use connectors (SSH...) to do so.
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/run.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/setup.md

@@ -10,7 +10,7 @@ It is intended to be run only for the initial setup or when such dependencies ch
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/setup.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/ssh_config.md

@@ -11,7 +11,7 @@ This executable is also used internally by other tools of Hybrid Platforms Condu
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/ssh_config.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/executables/test.md

@@ -12,7 +12,7 @@ This executable is perfectly suited to be integrated in a continuous integration
 <!-- Mermaid generator - Section start -->
 ![Mermaid diagram](/docs/gen/mermaid/docs/executables/test.md-0.png)
 <details>
-<summary>See diagram Mermaid code</summary>
+<summary><sub><sup>Diagram code</sup></sub></summary>
 
 ```mermaid
 sequenceDiagram

data/docs/tutorial.md

@@ -43,6 +43,10 @@ The tutorial assumes that all of the Hybrid Platforms Conductor commands will be
 All steps, command lines and files creations are explicitly detailed in the tutorial steps.
 No previous knowledge of Ruby is required to get through this tutorial.
 
+If in doubt, you can always refer to the [tutorial example folder](../examples/tutorial) to check files content.
+Please note however that the content of this example tutorial folder maps the content of the files at the end of each section of the tutorial (numberd from 01 to 05), and intermediate steps in the tutorial can modify such files.
+So the tutorial example folders may not reflect the expected content at an intermediate tutorial step.
+
 **[Next >> Let's start by the first step: installation of your Hybrid Platforms Conductor's environment](tutorial/01_installation.md)**
 
 ---

data/docs/tutorial/04_test.md

@@ -299,10 +299,19 @@ And now we try again the tests:
 Here we see that 3 nodes failed, but 1 of them is expected to fail, and is not counted in the failures summaries.
 Expected success is now down to 90 %.
 
-Let's bring back the 2 nodes that are expected to succeed and check tests again:
+Let's bring back the 2 nodes that are expected to succeed and check tests again (we will also make sure to update their IP address in `/etc/hosts` as they could have changed):
 ```bash
+# Restart only web3 and web5 - keep web1 down
 docker container start web3 web5
 
+# Regenerate IPs in /etc/hosts
+sed '/hpc_tutorial.org/d' /etc/hosts >/tmp/hosts
+cp /tmp/hosts /etc/hosts
+for ((i=2;i<=10;i++));
+do
+   echo "$(docker container inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' web$i)        web$i.hpc_tutorial.org" >>/etc/hosts
+done
+
 ./bin/test --all --test connection
 # ===== Run 11 connected tests ==== Begin...
 #   ===== Run test commands on 11 connected nodes (timeout to 25 secs) ==== Begin...
@@ -625,9 +634,10 @@ And now we run all the tests:
 
 All tests are green!
 
-Before going further, let's bring back `web1` online:
+Before going further, let's bring back `web1` online, and add its IP in `/etc/hosts` as well:
 ```bash
 docker container start web1
+echo "$(docker container inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' web1)        web1.hpc_tutorial.org" >>/etc/hosts
 ```
 
 We'll see later how easy to add you own test plugins to complement those, but now it's time to see other kind of tests.

data/docs/tutorial/05_extend_with_plugins.md

@@ -352,8 +352,8 @@ Let's see what does it take to integrate this new platform repository into Hybri
 ### Write a simple platform handler that can handle your existing repository
 
 A [`platform_handler` plugin](/docs/plugins.md#platform_handler) handles a given kind of platform repository, and has basically 2 roles:
-* Provide inventory information (nodes defined, their metadata, the services they are hosting...).
-* Provide services information (how to check/deploy services on a node).
+* Provide **inventory** information (nodes defined, their metadata, the services they are hosting...).
+* Provide **services** information (how to check/deploy services on a node).
 
 So let's write a new plugin handling your repository.
 Like any plugin, we create a file named `lib/<gem_name>/hpc_plugins/<plugin_type>/<plugin_name>.rb` that define a simple class inherting from a plugin's class.
@@ -999,8 +999,6 @@ For that we'll create a report plugin that will publish to our `web10` instance.
 
 Here is the code of our report plugin:
 ```ruby
-# This file is an example of a Reports plugin that can be used to dump information about the platforms.
-# The MyReportPlugin example contains example of code that could be used to write a plugin for a new kind of report.
 require 'hybrid_platforms_conductor/report'
 
 module MyHpcPlugins
@@ -1042,7 +1040,7 @@ module MyHpcPlugins
             end.join("\n")
           )
           # Upload the file on our web10 instance
-          system 'scp -o StrictHostKeyChecking=no /tmp/web_report.txt web10.hpc_tutorial.org:/root/hello_world.txt'
+          system 'scp -o StrictHostKeyChecking=no /tmp/web_report.txt root@web10.hpc_tutorial.org:/root/hello_world.txt'
           out 'Upload successful'
         end
 

data/examples/tutorial/01_installation/my-platforms/Gemfile

@@ -0,0 +1,3 @@
+source 'http://rubygems.org'
+
+gem 'hybrid_platforms_conductor'

data/examples/tutorial/02_first_node/my-platforms/Gemfile

@@ -0,0 +1,3 @@
+source 'http://rubygems.org'
+
+gem 'hybrid_platforms_conductor'

data/examples/tutorial/02_first_node/my-platforms/hpc_config.rb

@@ -0,0 +1 @@
+yaml_inventory_platform path: "#{Dir.home}/hpc_tutorial/my-service-conf-repo"

data/examples/tutorial/02_first_node/my-service-conf-repo/inventory.yaml

@@ -0,0 +1,13 @@
+---
+local:
+  metadata:
+    # This is a simple description of the node
+    description: The local environment
+    # This node is localhost, so here we'll use the local connector, not ssh.
+    local_node: true
+    # Some other metadata, specific to this node and that can be used by later processes
+    service_port: 1107
+    service_timeout: 60
+  # The list of service names this node should have
+  services:
+    - my-service

data/examples/tutorial/02_first_node/my-service-conf-repo/my-service.conf.erb

@@ -0,0 +1,3 @@
+service-port: <%= @service_port %>
+service-timeout: <%= @service_timeout %>
+service-logs: stdout

data/examples/tutorial/02_first_node/my-service-conf-repo/service_my-service.rb

@@ -0,0 +1,58 @@
+# Get the wanted content of the configuration file as a String, based on the node's metadata
+#
+# Parameters::
+# * *node* (String): Node for which we configure our service
+# Result::
+# * String: The wanted content
+def wanted_conf_for(node)
+  # We will access the node's metadata using the NodesHandler API, through the @nodes_handler object
+  @service_port = @nodes_handler.get_service_port_of(node)
+  @service_timeout = @nodes_handler.get_service_timeout_of(node)
+  # We use erubis to generate the configuration from our eRuby template, and return it directly
+  Erubis::Eruby.new(File.read("#{@platform_handler.repository_path}/my-service.conf.erb")).result(binding)
+end
+
+# Get actions to check the node's service against the wanted content
+#
+# Parameters::
+# * *node* (String): Node on which we check the service
+# Result::
+# * Array< Hash<Symbol,Object> >: The list of actions
+def check(node)
+  # We first dump the wanted content in a temporary file and then we diff it.
+  [
+    {
+      remote_bash: <<~EOS
+        cat <<EOF >/tmp/my-service.conf.wanted
+        #{wanted_conf_for(node)}
+        EOF
+        echo Diffs on my-service.conf:
+        if test -f ~/hpc_tutorial/node/my-service.conf; then
+          diff ~/hpc_tutorial/node/my-service.conf /tmp/my-service.conf.wanted || true
+        else
+          echo "Create file from scratch"
+          cat /tmp/my-service.conf.wanted
+        fi
+      EOS
+    }
+  ]
+end
+
+# Get actions to deploy the node's service against the wanted content
+#
+# Parameters::
+# * *node* (String): Node on which we deploy the service
+# Result::
+# * Array< Hash<Symbol,Object> >: The list of actions
+def deploy(node)
+  # We first check, as this will display diffs and prepare the file to be copied.
+  # And then we really deploy the file on our node.
+  check(node) + [
+    {
+      remote_bash: <<~EOS
+        mkdir -p ~/hpc_tutorial/node
+        cp /tmp/my-service.conf.wanted ~/hpc_tutorial/node/my-service.conf
+      EOS
+    }
+  ]
+end

data/examples/tutorial/02_first_node/node/my-service.conf

@@ -0,0 +1,4 @@
+service-port: 1107
+service-timeout: 60
+service-logs: stdout
+

data/examples/tutorial/03_scale/my-platforms/Gemfile

@@ -0,0 +1,3 @@
+source 'http://rubygems.org'
+
+gem 'hybrid_platforms_conductor'

data/examples/tutorial/03_scale/my-platforms/hpc_config.rb

@@ -0,0 +1 @@
+yaml_inventory_platform path: "#{Dir.home}/hpc_tutorial/my-service-conf-repo"

data/examples/tutorial/03_scale/my-platforms/my_commands.bash

@@ -0,0 +1,2 @@
+echo Hostname here is
+hostname

data/examples/tutorial/03_scale/my-service-conf-repo/inventory.yaml

@@ -0,0 +1,90 @@
+---
+local:
+  metadata:
+    description: The local environment
+    local_node: true
+    service_port: 1107
+    service_timeout: 60
+  services:
+    - my-service
+web1:
+  metadata:
+    description: Web service nbr 1
+    hostname: web1.hpc_tutorial.org
+    planet: Mercury
+    service_port: 1201
+    service_timeout: 60
+  services:
+    - web-hello
+    - my-service
+web2:
+  metadata:
+    description: Web service nbr 2
+    hostname: web2.hpc_tutorial.org
+    planet: Venus
+    service_port: 1202
+    service_timeout: 60
+  services:
+    - web-hello
+    - my-service
+web3:
+  metadata:
+    description: Web service nbr 3
+    hostname: web3.hpc_tutorial.org
+    planet: Earth
+    service_port: 1203
+    service_timeout: 60
+  services:
+    - web-hello
+    - my-service
+web4:
+  metadata:
+    description: Web service nbr 4
+    hostname: web4.hpc_tutorial.org
+    planet: Mars
+    service_port: 1204
+    service_timeout: 60
+  services:
+    - web-hello
+    - my-service
+web5:
+  metadata:
+    description: Web service nbr 5
+    hostname: web5.hpc_tutorial.org
+    planet: Jupiter
+    service_port: 1205
+    service_timeout: 60
+  services:
+    - web-hello
+    - my-service
+web6:
+  metadata:
+    description: Web service nbr 6
+    hostname: web6.hpc_tutorial.org
+  services:
+    - web-hello
+web7:
+  metadata:
+    description: Web service nbr 7
+    hostname: web7.hpc_tutorial.org
+  services:
+    - web-hello
+web8:
+  metadata:
+    description: Web service nbr 8
+    hostname: web8.hpc_tutorial.org
+  services:
+    - web-hello
+web9:
+  metadata:
+    description: Web service nbr 9
+    hostname: web9.hpc_tutorial.org
+  services:
+    - web-hello
+web10:
+  metadata:
+    description: Web service nbr 10
+    hostname: web10.hpc_tutorial.org
+  # The list of service names this node should have
+  services:
+    - web-hello

data/examples/tutorial/03_scale/my-service-conf-repo/my-service.conf.erb

@@ -0,0 +1,3 @@
+service-port: <%= @service_port %>
+service-timeout: <%= @service_timeout %>
+service-logs: stdout

data/examples/tutorial/03_scale/my-service-conf-repo/service_my-service.rb

@@ -0,0 +1,58 @@
+# Get the wanted content of the configuration file as a String, based on the node's metadata
+#
+# Parameters::
+# * *node* (String): Node for which we configure our service
+# Result::
+# * String: The wanted content
+def wanted_conf_for(node)
+  # We will access the node's metadata using the NodesHandler API, through the @nodes_handler object
+  @service_port = @nodes_handler.get_service_port_of(node)
+  @service_timeout = @nodes_handler.get_service_timeout_of(node)
+  # We use erubis to generate the configuration from our eRuby template, and return it directly
+  Erubis::Eruby.new(File.read("#{@platform_handler.repository_path}/my-service.conf.erb")).result(binding)
+end
+
+# Get actions to check the node's service against the wanted content
+#
+# Parameters::
+# * *node* (String): Node on which we check the service
+# Result::
+# * Array< Hash<Symbol,Object> >: The list of actions
+def check(node)
+  # We first dump the wanted content in a temporary file and then we diff it.
+  [
+    {
+      remote_bash: <<~EOS
+        cat <<EOF >/tmp/my-service.conf.wanted
+        #{wanted_conf_for(node)}
+        EOF
+        echo Diffs on my-service.conf:
+        if test -f ~/hpc_tutorial/node/my-service.conf; then
+          diff ~/hpc_tutorial/node/my-service.conf /tmp/my-service.conf.wanted || true
+        else
+          echo "Create file from scratch"
+          cat /tmp/my-service.conf.wanted
+        fi
+      EOS
+    }
+  ]
+end
+
+# Get actions to deploy the node's service against the wanted content
+#
+# Parameters::
+# * *node* (String): Node on which we deploy the service
+# Result::
+# * Array< Hash<Symbol,Object> >: The list of actions
+def deploy(node)
+  # We first check, as this will display diffs and prepare the file to be copied.
+  # And then we really deploy the file on our node.
+  check(node) + [
+    {
+      remote_bash: <<~EOS
+        mkdir -p ~/hpc_tutorial/node
+        cp /tmp/my-service.conf.wanted ~/hpc_tutorial/node/my-service.conf
+      EOS
+    }
+  ]
+end