hybrid_platforms_conductor 32.12.0 → 32.13.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +1103 -0
- data/LICENSE.md +31 -0
- data/README.md +395 -0
- data/bin/setup +1 -1
- data/docs/api.md +349 -0
- data/docs/config_dsl.md +315 -0
- data/docs/executables.md +226 -0
- data/docs/executables/check-node.md +155 -0
- data/docs/executables/deploy.md +198 -0
- data/docs/executables/dump_nodes_json.md +110 -0
- data/docs/executables/free_ips.md +93 -0
- data/docs/executables/free_veids.md +73 -0
- data/docs/executables/get_impacted_nodes.md +94 -0
- data/docs/executables/last_deploys.md +114 -0
- data/docs/executables/nodes_to_deploy.md +139 -0
- data/docs/executables/report.md +159 -0
- data/docs/executables/run.md +126 -0
- data/docs/executables/setup.md +92 -0
- data/docs/executables/ssh_config.md +151 -0
- data/docs/executables/test.md +213 -0
- data/docs/executables/topograph.md +139 -0
- data/docs/gen/mermaid/README.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/check-node.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/deploy.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/free_ips.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/free_veids.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/get_impacted_nodes.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/last_deploys.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/nodes_to_deploy.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/report.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/run.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/setup.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/ssh_config.md-0.png +0 -0
- data/docs/gen/mermaid/docs/executables/test.md-0.png +0 -0
- data/docs/install.md +161 -0
- data/docs/plugins.md +215 -0
- data/docs/plugins/action/bash.md +37 -0
- data/docs/plugins/action/interactive.md +37 -0
- data/docs/plugins/action/remote_bash.md +67 -0
- data/docs/plugins/action/ruby.md +69 -0
- data/docs/plugins/action/scp.md +61 -0
- data/docs/plugins/cmdb/config.md +46 -0
- data/docs/plugins/cmdb/host_ip.md +33 -0
- data/docs/plugins/cmdb/host_keys.md +33 -0
- data/docs/plugins/cmdb/platform_handlers.md +33 -0
- data/docs/plugins/connector/local.md +28 -0
- data/docs/plugins/connector/ssh.md +95 -0
- data/docs/plugins/platform_handler/yaml_inventory.md +105 -0
- data/docs/plugins/provisioner/docker.md +27 -0
- data/docs/plugins/provisioner/podman.md +27 -0
- data/docs/plugins/provisioner/proxmox.md +115 -0
- data/docs/plugins/report/confluence.md +49 -0
- data/docs/plugins/report/mediawiki.md +28 -0
- data/docs/plugins/report/stdout.md +32 -0
- data/docs/plugins/test/bitbucket_conf.md +97 -0
- data/docs/plugins/test/can_be_checked.md +27 -0
- data/docs/plugins/test/check_deploy_and_idempotence.md +61 -0
- data/docs/plugins/test/check_from_scratch.md +28 -0
- data/docs/plugins/test/connection.md +27 -0
- data/docs/plugins/test/deploy_freshness.md +27 -0
- data/docs/plugins/test/deploy_from_scratch.md +28 -0
- data/docs/plugins/test/deploy_removes_root_access.md +29 -0
- data/docs/plugins/test/divergence.md +41 -0
- data/docs/plugins/test/executables.md +26 -0
- data/docs/plugins/test/file_system.md +49 -0
- data/docs/plugins/test/file_system_hdfs.md +65 -0
- data/docs/plugins/test/hostname.md +27 -0
- data/docs/plugins/test/idempotence.md +56 -0
- data/docs/plugins/test/ip.md +28 -0
- data/docs/plugins/test/jenkins_ci_conf.md +54 -0
- data/docs/plugins/test/jenkins_ci_masters_ok.md +54 -0
- data/docs/plugins/test/linear_strategy.md +26 -0
- data/docs/plugins/test/local_users.md +48 -0
- data/docs/plugins/test/mounts.md +55 -0
- data/docs/plugins/test/orphan_files.md +38 -0
- data/docs/plugins/test/ports.md +50 -0
- data/docs/plugins/test/private_ips.md +27 -0
- data/docs/plugins/test/public_ips.md +27 -0
- data/docs/plugins/test/spectre.md +26 -0
- data/docs/plugins/test/veids.md +27 -0
- data/docs/plugins/test/vulnerabilities.md +65 -0
- data/docs/plugins/test_report/confluence.md +43 -0
- data/docs/plugins/test_report/stdout.md +26 -0
- data/docs/plugins_create.md +135 -0
- data/docs/tutorial.md +57 -0
- data/docs/tutorial/01_installation.md +129 -0
- data/docs/tutorial/02_first_node.md +466 -0
- data/docs/tutorial/03_scale.md +876 -0
- data/docs/tutorial/04_test.md +965 -0
- data/docs/tutorial/05_extend_with_plugins.md +1132 -0
- data/examples/bare/Gemfile +4 -0
- data/examples/bare/hpc_config.rb +2 -0
- data/examples/localhost/Gemfile +4 -0
- data/examples/localhost/hpc_config.rb +2 -0
- data/examples/localhost/inventory.yaml +4 -0
- data/lib/hybrid_platforms_conductor/actions_executor.rb +1 -0
- data/lib/hybrid_platforms_conductor/deployer.rb +3 -2
- data/lib/hybrid_platforms_conductor/hpc_plugins/action/remote_bash.rb +29 -13
- data/lib/hybrid_platforms_conductor/hpc_plugins/action/scp.rb +1 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/connector/local.rb +98 -0
- data/lib/hybrid_platforms_conductor/hpc_plugins/connector/my_connector.rb.sample +2 -2
- data/lib/hybrid_platforms_conductor/hpc_plugins/connector/ssh.rb +7 -3
- data/lib/hybrid_platforms_conductor/hpc_plugins/platform_handler/platform_handler_plugin.rb.sample +5 -5
- data/lib/hybrid_platforms_conductor/hpc_plugins/platform_handler/yaml_inventory.rb +140 -0
- data/lib/hybrid_platforms_conductor/hpc_plugins/provisioner/proxmox.rb +5 -2
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/bitbucket_conf.rb +4 -4
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/deploy_freshness.rb +1 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/deploy_removes_root_access.rb +19 -17
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/divergence.rb +3 -0
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/hostname.rb +2 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/ip.rb +2 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/local_users.rb +2 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/mounts.rb +4 -3
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/orphan_files.rb +2 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/spectre.rb +1 -1
- data/lib/hybrid_platforms_conductor/hpc_plugins/test/vulnerabilities.rb +8 -7
- data/lib/hybrid_platforms_conductor/hpc_plugins/test_report/confluence.rb +1 -1
- data/lib/hybrid_platforms_conductor/json_dumper.rb +1 -1
- data/lib/hybrid_platforms_conductor/platform_handler.rb +1 -1
- data/lib/hybrid_platforms_conductor/services_handler.rb +18 -16
- data/lib/hybrid_platforms_conductor/tests_runner.rb +0 -1
- data/lib/hybrid_platforms_conductor/topographer.rb +0 -1
- data/lib/hybrid_platforms_conductor/version.rb +1 -1
- data/spec/hybrid_platforms_conductor_test/api/actions_executor/actions/remote_bash_spec.rb +16 -0
- data/spec/hybrid_platforms_conductor_test/api/actions_executor/connectors/local/connectable_nodes_spec.rb +30 -0
- data/spec/hybrid_platforms_conductor_test/api/actions_executor/connectors/local/remote_actions_spec.rb +113 -0
- data/spec/hybrid_platforms_conductor_test/api/actions_executor/connectors/ssh/cli_options_spec.rb +6 -2
- data/spec/hybrid_platforms_conductor_test/api/actions_executor/connectors/ssh/global_helpers_spec.rb +38 -1
- data/spec/hybrid_platforms_conductor_test/api/actions_executor/connectors/ssh/remote_actions_spec.rb +8 -8
- data/spec/hybrid_platforms_conductor_test/docs_spec.rb +10 -0
- data/tools/check_md +89 -0
- data/tools/generate_mermaid +75 -0
- metadata +207 -12
@@ -0,0 +1,93 @@
|
|
1
|
+
# `free_ips`
|
2
|
+
|
3
|
+
The `free_ips` executable will output all free IP ranges for any used range.
|
4
|
+
Useful to assign new IPs.
|
5
|
+
|
6
|
+
***This executable is still in alpha version: not properly tested, no clear process, no stable interface. Pending [this ticket](https://github.com/sweet-delights/hybrid-platforms-conductor/issues/45).***
|
7
|
+
|
8
|
+
## Process
|
9
|
+
|
10
|
+
<!-- Mermaid generator - Section start -->
|
11
|
+
![Mermaid diagram](/docs/gen/mermaid/docs/executables/free_ips.md-0.png)
|
12
|
+
<details>
|
13
|
+
<summary>See diagram Mermaid code</summary>
|
14
|
+
|
15
|
+
```mermaid
|
16
|
+
sequenceDiagram
|
17
|
+
participant Main as ./bin/free_ips
|
18
|
+
participant CMDB as CMDB
|
19
|
+
|
20
|
+
Main->>+CMDB: Get private IP metadata for nodes
|
21
|
+
CMDB-->>-Main: Private IPs
|
22
|
+
Main->>Main: Display free private IPs
|
23
|
+
```
|
24
|
+
</details>
|
25
|
+
<!-- Mermaid generator - Section end -->
|
26
|
+
|
27
|
+
## Usage
|
28
|
+
|
29
|
+
```
|
30
|
+
Usage: ./bin/free_ips [options]
|
31
|
+
|
32
|
+
Main options:
|
33
|
+
-d, --debug Activate debug mode
|
34
|
+
-h, --help Display help and exit
|
35
|
+
|
36
|
+
Nodes handler options:
|
37
|
+
-o, --show-nodes Display the list of possible nodes and exit
|
38
|
+
|
39
|
+
Command runner options:
|
40
|
+
-s, --show-commands Display the commands that would be run instead of running them
|
41
|
+
```
|
42
|
+
|
43
|
+
## Examples
|
44
|
+
|
45
|
+
```bash
|
46
|
+
./bin/free_ips
|
47
|
+
```
|
48
|
+
|
49
|
+
Example of output:
|
50
|
+
```
|
51
|
+
=> ./bin/free_ips
|
52
|
+
Free IPs for 172.16.0: [11, 20, 23..29, 31, 34, 37..40, 42..45, 48, 51..58, 60, 63..72, 76, 79..80, 82..83, 87, 90..95, 97..101, 103..104, 107..109, 111..113, 115, 117, 119..120, 123..124, 127, 129, 131, 139, 142..149, 153..169, 171..180, 182..189, 191, 193..209, 211..221, 223..251, 253..255]
|
53
|
+
Free IPs for 172.16.1: [21..100, 102, 106, 108, 110..112, 114, 116..119, 121..124, 126, 132..177, 179..200, 205..209, 211..221, 223..252, 254..255]
|
54
|
+
Free IPs for 172.16.2: [102, 110..111, 113, 120..121, 127, 133, 142..150, 194..200, 204..209, 214..221, 223..251, 253..255]
|
55
|
+
Free IPs for 172.16.3: [11..99, 102, 106..199, 201..255]
|
56
|
+
Free IPs for 172.16.4: [12..100, 102, 106, 109, 112..255]
|
57
|
+
Free IPs for 172.16.5: [102, 105..255]
|
58
|
+
Free IPs for 172.16.6: [41..64, 85..102, 104..109, 116..128, 134..255]
|
59
|
+
Free IPs for 172.16.7: [28..102, 104..106, 113..255]
|
60
|
+
Free IPs for 172.16.8: [18..100, 102, 104, 106, 113..119, 121..122, 124..159, 166..170, 196..200, 204..255]
|
61
|
+
Free IPs for 172.16.9: [86..102, 104..106, 113..122, 125, 128..255]
|
62
|
+
Free IPs for 172.16.10: [5..104, 106..255]
|
63
|
+
Free IPs for 172.16.16: [8..19, 23..25, 28, 30, 32..33, 35..36, 39, 41..42, 44..47, 49..51, 54..62, 64..69, 74, 78, 81, 83, 93, 96..98, 104, 110, 114..116, 118, 125..128, 131..136, 138..152, 156..169, 171..177, 179, 181, 183..185, 187, 189, 191, 193..209, 212..255]
|
64
|
+
Free IPs for 172.16.110: [43..54, 56..57, 60..82, 84..105, 108..118, 120..203, 208..255]
|
65
|
+
Free IPs for 172.16.111: [22..23, 25..26, 28..30, 32..46, 48..49, 54..255]
|
66
|
+
Free IPs for 172.16.132: [16..47, 49, 51..181, 183..187, 189..216, 218..225, 227..229, 231..252, 255]
|
67
|
+
Free IPs for 172.16.133: [18..50, 52, 54..95, 97, 99..120, 122..123, 125..128, 130..134, 136..155, 157..163, 166..252, 254..255]
|
68
|
+
Free IPs for 172.16.134: [2, 4..54, 56..65, 67..210, 212..222, 224..228, 231..255]
|
69
|
+
Free IPs for 172.16.135: [61, 63, 65, 67, 72..89, 93, 95..96, 98..104, 107..110, 115..122, 124..126, 131..255]
|
70
|
+
Free IPs for 172.16.139: [99..255]
|
71
|
+
Free IPs for 172.30.14: [227..255]
|
72
|
+
Free IPs for 192.168.0: [3..255]
|
73
|
+
```
|
74
|
+
|
75
|
+
## Used credentials
|
76
|
+
|
77
|
+
| Credential | Usage
|
78
|
+
| --- | --- |
|
79
|
+
|
80
|
+
## Used Metadata
|
81
|
+
|
82
|
+
| Metadata | Type | Usage
|
83
|
+
| --- | --- | --- |
|
84
|
+
| `private_ips` | `Array<String>` | List of private IPs |
|
85
|
+
|
86
|
+
## Used environment variables
|
87
|
+
|
88
|
+
| Variable | Usage
|
89
|
+
| --- | --- |
|
90
|
+
|
91
|
+
## External tools dependencies
|
92
|
+
|
93
|
+
None
|
@@ -0,0 +1,73 @@
|
|
1
|
+
# `free_veids`
|
2
|
+
|
3
|
+
The `free_veids` executable will output all free VEIDs (smaller than 10000).
|
4
|
+
Useful to assign unused VEIDs to new VMs to be created.
|
5
|
+
|
6
|
+
***This executable is still in alpha version: not properly tested, no clear process, no stable interface. Pending [this ticket](https://github.com/sweet-delights/hybrid-platforms-conductor/issues/45).***
|
7
|
+
|
8
|
+
## Process
|
9
|
+
|
10
|
+
<!-- Mermaid generator - Section start -->
|
11
|
+
![Mermaid diagram](/docs/gen/mermaid/docs/executables/free_veids.md-0.png)
|
12
|
+
<details>
|
13
|
+
<summary>See diagram Mermaid code</summary>
|
14
|
+
|
15
|
+
```mermaid
|
16
|
+
sequenceDiagram
|
17
|
+
participant Main as ./bin/free_veids
|
18
|
+
participant CMDB as CMDB
|
19
|
+
|
20
|
+
Main->>+CMDB: Get VEID metadata for nodes
|
21
|
+
CMDB-->>-Main: VEIDs
|
22
|
+
Main->>Main: Display free VEIDs
|
23
|
+
```
|
24
|
+
</details>
|
25
|
+
<!-- Mermaid generator - Section end -->
|
26
|
+
|
27
|
+
## Usage
|
28
|
+
|
29
|
+
```
|
30
|
+
Usage: ./bin/free_veids [options]
|
31
|
+
|
32
|
+
Main options:
|
33
|
+
-d, --debug Activate debug mode
|
34
|
+
-h, --help Display help and exit
|
35
|
+
|
36
|
+
Nodes handler options:
|
37
|
+
-o, --show-nodes Display the list of possible nodes and exit
|
38
|
+
|
39
|
+
Command runner options:
|
40
|
+
-s, --show-commands Display the commands that would be run instead of running them
|
41
|
+
```
|
42
|
+
|
43
|
+
## Examples
|
44
|
+
|
45
|
+
```bash
|
46
|
+
./bin/free_veids
|
47
|
+
```
|
48
|
+
|
49
|
+
Example of output:
|
50
|
+
```
|
51
|
+
=> ./bin/free_veids
|
52
|
+
Free VEIDs: [420, 426, 428, 430, 434, 437..438, 445..446, 449..450, 453, 456..457, 459, 464, 466..467, 471, 475..476, 484, 488, 490, 493, 500..502, 504..513, 523, 525, 536, 544, 546, 554..555, 560..566, 578, 589, 594, 642..659, 668..9999]
|
53
|
+
```
|
54
|
+
|
55
|
+
## Used credentials
|
56
|
+
|
57
|
+
| Credential | Usage
|
58
|
+
| --- | --- |
|
59
|
+
|
60
|
+
## Used Metadata
|
61
|
+
|
62
|
+
| Metadata | Type | Usage
|
63
|
+
| --- | --- | --- |
|
64
|
+
| `veid` | `String` or `Integer` | VEID associated to the node |
|
65
|
+
|
66
|
+
## Used environment variables
|
67
|
+
|
68
|
+
| Variable | Usage
|
69
|
+
| --- | --- |
|
70
|
+
|
71
|
+
## External tools dependencies
|
72
|
+
|
73
|
+
None
|
@@ -0,0 +1,94 @@
|
|
1
|
+
# `get_impacted_nodes`
|
2
|
+
|
3
|
+
The `get_impacted_nodes` executable reports nodes impacted by a git diff in one of the platforms.
|
4
|
+
This is especially useful to know which nodes have to be tested against a given PR or local code difference.
|
5
|
+
|
6
|
+
## Process
|
7
|
+
|
8
|
+
<!-- Mermaid generator - Section start -->
|
9
|
+
![Mermaid diagram](/docs/gen/mermaid/docs/executables/get_impacted_nodes.md-0.png)
|
10
|
+
<details>
|
11
|
+
<summary>See diagram Mermaid code</summary>
|
12
|
+
|
13
|
+
```mermaid
|
14
|
+
sequenceDiagram
|
15
|
+
participant Main as ./bin/get_impacted_nodes
|
16
|
+
participant PlatformHandler as Platform Handler
|
17
|
+
participant PlatformRepo as Platform repository
|
18
|
+
|
19
|
+
Main->>+PlatformHandler: Get the list of nodes and services impacted by a git diff
|
20
|
+
PlatformHandler->>+PlatformRepo: Perform git diff
|
21
|
+
PlatformRepo-->>-PlatformHandler: Git diff details
|
22
|
+
PlatformHandler-->>-Main: List of nodes and services impacted by a git diff
|
23
|
+
Main->>Main: Display the list of impacted nodes and services
|
24
|
+
```
|
25
|
+
</details>
|
26
|
+
<!-- Mermaid generator - Section end -->
|
27
|
+
|
28
|
+
## Usage
|
29
|
+
|
30
|
+
```
|
31
|
+
Usage: ./bin/get_impacted_nodes [options]
|
32
|
+
|
33
|
+
Main options:
|
34
|
+
-d, --debug Activate debug mode
|
35
|
+
-h, --help Display help and exit
|
36
|
+
-f, --from-commit COMMIT_ID Specify the GIT commit from which we look for diffs. Defaults to master.
|
37
|
+
-p, --platform PLATFORM_NAME Specify the repository on which to perform the diff. Possible values are chef-repo, ansible-repo
|
38
|
+
--smallest-test-sample Display the minimal set of nodes to check that would validate all modifications.
|
39
|
+
-t, --to-commit COMMIT_ID Specify the GIT commit to which we look for diffs. Defaults to current checked out files.
|
40
|
+
|
41
|
+
Nodes handler options:
|
42
|
+
-o, --show-nodes Display the list of possible nodes and exit
|
43
|
+
|
44
|
+
Command runner options:
|
45
|
+
-s, --show-commands Display the commands that would be run instead of running them
|
46
|
+
```
|
47
|
+
|
48
|
+
## Examples
|
49
|
+
|
50
|
+
```bash
|
51
|
+
# Get nodes impacted by current modifications in the chef-repo repository
|
52
|
+
./bin/get_impacted_nodes --platform chef-repo
|
53
|
+
|
54
|
+
# Get nodes impacted by modifications between 2 commit IDs in the chef-repo repository
|
55
|
+
./bin/get_impacted_nodes --platform chef-repo --from-commit 6e45ff --to-commit aac411
|
56
|
+
|
57
|
+
# Get the smallest set of nodes to be tested to validate changes from a branch compared to master
|
58
|
+
./bin/get_impacted_nodes --platform chef-repo --to-commit my_branch --smallest-test-sample
|
59
|
+
```
|
60
|
+
|
61
|
+
Here is an example of output:
|
62
|
+
```
|
63
|
+
|
64
|
+
* 1 impacted services:
|
65
|
+
websql
|
66
|
+
|
67
|
+
* 2 impacted nodes (directly):
|
68
|
+
node12hst-nn6
|
69
|
+
node12hst-nn2
|
70
|
+
|
71
|
+
* 3 impacted nodes (total):
|
72
|
+
node12hst-nn6
|
73
|
+
node12hst-nn1
|
74
|
+
node12hst-nn2
|
75
|
+
```
|
76
|
+
|
77
|
+
## Used credentials
|
78
|
+
|
79
|
+
| Credential | Usage
|
80
|
+
| --- | --- |
|
81
|
+
|
82
|
+
## Used Metadata
|
83
|
+
|
84
|
+
| Metadata | Type | Usage
|
85
|
+
| --- | --- | --- |
|
86
|
+
|
87
|
+
## Used environment variables
|
88
|
+
|
89
|
+
| Variable | Usage
|
90
|
+
| --- | --- |
|
91
|
+
|
92
|
+
## External tools dependencies
|
93
|
+
|
94
|
+
None
|
@@ -0,0 +1,114 @@
|
|
1
|
+
# `last_deploys`
|
2
|
+
|
3
|
+
The `last_deploys` executable will fetch the last deployments information for a given list of nodes and display it.
|
4
|
+
|
5
|
+
## Process
|
6
|
+
|
7
|
+
<!-- Mermaid generator - Section start -->
|
8
|
+
![Mermaid diagram](/docs/gen/mermaid/docs/executables/last_deploys.md-0.png)
|
9
|
+
<details>
|
10
|
+
<summary>See diagram Mermaid code</summary>
|
11
|
+
|
12
|
+
```mermaid
|
13
|
+
sequenceDiagram
|
14
|
+
participant Main as ./bin/last_deploys
|
15
|
+
participant Connector as Connector
|
16
|
+
participant Node as Node
|
17
|
+
|
18
|
+
Main->>+Connector: Connect to nodes to read deployment logs
|
19
|
+
Connector->>+Node: Read deployment logs
|
20
|
+
Node-->>-Connector: Deployment logs
|
21
|
+
Connector-->>-Main: Close connection
|
22
|
+
Main->>Main: Display deployment info
|
23
|
+
```
|
24
|
+
</details>
|
25
|
+
<!-- Mermaid generator - Section end -->
|
26
|
+
|
27
|
+
## Usage
|
28
|
+
|
29
|
+
```
|
30
|
+
Usage: ./bin/last_deploys [options]
|
31
|
+
|
32
|
+
Main options:
|
33
|
+
-d, --debug Activate debug mode
|
34
|
+
-h, --help Display help and exit
|
35
|
+
--sort-by SORT Specify a sort. Possible values are: commit_comment, date, node, repo_name, user. Each value can append _desc to specify a reverse sorting. Defaults to node.
|
36
|
+
|
37
|
+
Nodes handler options:
|
38
|
+
-o, --show-nodes Display the list of possible nodes and exit
|
39
|
+
|
40
|
+
Nodes selection options:
|
41
|
+
-a, --all-nodes Select all nodes
|
42
|
+
-b, --nodes-platform PLATFORM Select nodes belonging to a given platform name. Available platforms are: ansible-repo, chef-repo (can be used several times)
|
43
|
+
-l, --nodes-list LIST Select nodes defined in a nodes list (can be used several times)
|
44
|
+
-n, --node NODE Select a specific node. Can be a regular expression to select several nodes if used with enclosing "/" characters. (can be used several times).
|
45
|
+
-r, --nodes-service SERVICE Select nodes implementing a given service (can be used several times)
|
46
|
+
--nodes-git-impact GIT_IMPACT
|
47
|
+
Select nodes impacted by a git diff from a platform (can be used several times).
|
48
|
+
GIT_IMPACT has the format PLATFORM:FROM_COMMIT:TO_COMMIT:FLAGS
|
49
|
+
* PLATFORM: Name of the platform to check git diff from. Available platforms are: ansible-repo, chef-repo
|
50
|
+
* FROM_COMMIT: Commit ID or refspec from which we perform the diff. If ommitted, defaults to master
|
51
|
+
* TO_COMMIT: Commit ID ot refspec to which we perform the diff. If ommitted, defaults to the currently checked-out files
|
52
|
+
* FLAGS: Extra comma-separated flags. The following flags are supported:
|
53
|
+
- min: If specified then each impacted service will select only 1 node implementing this service. If not specified then all nodes implementing the impacted services will be selected.
|
54
|
+
|
55
|
+
Command runner options:
|
56
|
+
-s, --show-commands Display the commands that would be run instead of running them
|
57
|
+
|
58
|
+
Actions Executor options:
|
59
|
+
-m, --max-threads NBR Set the number of threads to use for concurrent queries (defaults to 64)
|
60
|
+
|
61
|
+
Connector ssh options:
|
62
|
+
-g, --ssh-gateway-user USER Name of the gateway user to be used by the gateways. Can also be set from environment variable hpc_ssh_gateway_user. Defaults to ubradm.
|
63
|
+
-j, --ssh-no-control-master If used, don't create SSH control masters for connections.
|
64
|
+
-q, --ssh-no-host-key-checking If used, don't check for SSH host keys.
|
65
|
+
-u, --ssh-user USER Name of user to be used in SSH connections (defaults to hpc_ssh_user or USER environment variables)
|
66
|
+
-w, --password If used, then expect SSH connections to ask for a password.
|
67
|
+
-y GATEWAYS_CONF, Name of the gateways configuration to be used. Can also be set from environment variable hpc_ssh_gateways_conf.
|
68
|
+
--ssh-gateways-conf
|
69
|
+
```
|
70
|
+
|
71
|
+
## Examples
|
72
|
+
|
73
|
+
```bash
|
74
|
+
# Check deployments for all nodes
|
75
|
+
./bin/last_deploys --all-nodes
|
76
|
+
|
77
|
+
# Check deployments for all nodes, sorted by date descending
|
78
|
+
./bin/last_deploys --all-nodes --sort-by date_desc
|
79
|
+
```
|
80
|
+
|
81
|
+
Example of output:
|
82
|
+
```
|
83
|
+
+----------------+---------------------+-----------+---------------+------------------+-------------+----------------------------+-------+
|
84
|
+
| Hostname | Date | Admin | Git artefact | Git branch | Chef commit | Chef comment | Error |
|
85
|
+
+----------------+---------------------+-----------+---------------+------------------+-------------+----------------------------+-------+
|
86
|
+
| node10 | 2017-11-22 09:50:47 | a_usernme | 172.16.0.46 | v20171122T110551 | 73c2017a2a8 | Added sorting capabilities | |
|
87
|
+
| node12had43 | 2017-11-22 10:07:37 | a_usernme | 172.16.110.42 | v20171122T110551 | 73c2017a2a8 | Added sorting capabilities | |
|
88
|
+
| node12hst-nn6 | 2017-11-22 10:07:35 | a_usernme | 172.16.110.42 | v20171122T110551 | 73c2017a2a8 | Added sorting capabilities | |
|
89
|
+
| node12hst-nn9 | 2017-11-23 18:08:59 | root | 172.16.110.42 | v20171123T190837 | 73c2017a2a8 | Added sorting capabilities | |
|
90
|
+
| node12hst-nn2 | 2017-11-22 10:07:37 | a_usernme | 172.16.110.42 | v20171122T110551 | 73c2017a2a8 | Added sorting capabilities | |
|
91
|
+
| node12hst-nn3 | 2017-11-22 10:07:37 | a_usernme | 172.16.110.42 | v20171122T110551 | 73c2017a2a8 | Added sorting capabilities | |
|
92
|
+
| node12lnx10 | 2017-11-22 11:07:33 | a_usernme | 172.16.110.42 | v20171122T110551 | 73c2017a2a8 | Added sorting capabilities | |
|
93
|
+
| xaeprjcttlbd01 | 2017-11-23 18:43:01 | a_usernme | 172.16.0.46 | v20171123T194235 | 73c2017a2a8 | Added sorting capabilities | |
|
94
|
+
+----------------+---------------------+-----------+---------------+------------------+-------------+----------------------------+-------+
|
95
|
+
```
|
96
|
+
|
97
|
+
## Used credentials
|
98
|
+
|
99
|
+
| Credential | Usage
|
100
|
+
| --- | --- |
|
101
|
+
|
102
|
+
## Used Metadata
|
103
|
+
|
104
|
+
| Metadata | Type | Usage
|
105
|
+
| --- | --- | --- |
|
106
|
+
|
107
|
+
## Used environment variables
|
108
|
+
|
109
|
+
| Variable | Usage
|
110
|
+
| --- | --- |
|
111
|
+
|
112
|
+
## External tools dependencies
|
113
|
+
|
114
|
+
None
|
@@ -0,0 +1,139 @@
|
|
1
|
+
# `nodes_to_deploy`
|
2
|
+
|
3
|
+
The `nodes_to_deploy` executable is used to know which nodes need to be deployed, considering:
|
4
|
+
* the deployment schedule allocated to the nodes,
|
5
|
+
* the last level of code that has been deployed on the node.
|
6
|
+
The deployment schedule is defined by the configuration DSL, with the [`deployment_schedule` method](../config_dsl.md).
|
7
|
+
The list of nodes is given in standard output.
|
8
|
+
|
9
|
+
## Process
|
10
|
+
|
11
|
+
<!-- Mermaid generator - Section start -->
|
12
|
+
![Mermaid diagram](/docs/gen/mermaid/docs/executables/nodes_to_deploy.md-0.png)
|
13
|
+
<details>
|
14
|
+
<summary>See diagram Mermaid code</summary>
|
15
|
+
|
16
|
+
```mermaid
|
17
|
+
sequenceDiagram
|
18
|
+
participant Main as ./bin/nodes_to_deploy
|
19
|
+
participant CMDB as CMDB
|
20
|
+
participant PlatformHandler as Platform Handler
|
21
|
+
participant PlatformRepo as Platform repository
|
22
|
+
participant Connector as Connector
|
23
|
+
participant Node as Node
|
24
|
+
|
25
|
+
Main->>+CMDB: Get services to be deployed on nodes
|
26
|
+
CMDB->>+PlatformHandler: Get metadata from the platform
|
27
|
+
PlatformHandler->>+PlatformRepo: Read platform info
|
28
|
+
PlatformRepo-->>-PlatformHandler: Platform info
|
29
|
+
PlatformHandler-->>-CMDB: Platform metadata, including services
|
30
|
+
CMDB-->>-Main: Services to be deployed
|
31
|
+
Main->>+Connector: Connect to nodes to read deployment logs
|
32
|
+
Connector->>+Node: Read deployment logs
|
33
|
+
Node-->>-Connector: Deployment logs
|
34
|
+
Connector-->>-Main: Close connection
|
35
|
+
Main->>Main: Display nodes to be deployed that have divergent services
|
36
|
+
```
|
37
|
+
</details>
|
38
|
+
<!-- Mermaid generator - Section end -->
|
39
|
+
|
40
|
+
## Usage
|
41
|
+
|
42
|
+
```
|
43
|
+
Usage: ./bin/nodes_to_deploy [options]
|
44
|
+
|
45
|
+
Main options:
|
46
|
+
-d, --debug Activate debug mode
|
47
|
+
-h, --help Display help and exit
|
48
|
+
--deployment-time DATETIME Set the deployment time to be considered while matching the schedules. Defaults to now.
|
49
|
+
--ignore-deployed-info Ignore the current deployed information.
|
50
|
+
--ignore-schedule Ignore the deployment schedules.
|
51
|
+
|
52
|
+
Nodes handler options:
|
53
|
+
-o, --show-nodes Display the list of possible nodes and exit
|
54
|
+
|
55
|
+
Nodes selection options:
|
56
|
+
-a, --all-nodes Select all nodes
|
57
|
+
-b, --nodes-platform PLATFORM Select nodes belonging to a given platform name. Available platforms are: ansible-repo, chef-repo (can be used several times)
|
58
|
+
-l, --nodes-list LIST Select nodes defined in a nodes list (can be used several times)
|
59
|
+
-n, --node NODE Select a specific node. Can be a regular expression to select several nodes if used with enclosing "/" characters. (can be used several times).
|
60
|
+
-r, --nodes-service SERVICE Select nodes implementing a given service (can be used several times)
|
61
|
+
--nodes-git-impact GIT_IMPACT
|
62
|
+
Select nodes impacted by a git diff from a platform (can be used several times).
|
63
|
+
GIT_IMPACT has the format PLATFORM:FROM_COMMIT:TO_COMMIT:FLAGS
|
64
|
+
* PLATFORM: Name of the platform to check git diff from. Available platforms are: ansible-repo, chef-repo
|
65
|
+
* FROM_COMMIT: Commit ID or refspec from which we perform the diff. If ommitted, defaults to master
|
66
|
+
* TO_COMMIT: Commit ID ot refspec to which we perform the diff. If ommitted, defaults to the currently checked-out files
|
67
|
+
* FLAGS: Extra comma-separated flags. The following flags are supported:
|
68
|
+
- min: If specified then each impacted service will select only 1 node implementing this service. If not specified then all nodes implementing the impacted services will be selected.
|
69
|
+
|
70
|
+
Command runner options:
|
71
|
+
-s, --show-commands Display the commands that would be run instead of running them
|
72
|
+
|
73
|
+
Actions Executor options:
|
74
|
+
-m, --max-threads NBR Set the number of threads to use for concurrent queries (defaults to 16)
|
75
|
+
|
76
|
+
Connector ssh options:
|
77
|
+
-g, --ssh-gateway-user USER Name of the gateway user to be used by the gateways. Can also be set from environment variable hpc_ssh_gateway_user. Defaults to ubradm.
|
78
|
+
-j, --ssh-no-control-master If used, don't create SSH control masters for connections.
|
79
|
+
-q, --ssh-no-host-key-checking If used, don't check for SSH host keys.
|
80
|
+
-u, --ssh-user USER Name of user to be used in SSH connections (defaults to hpc_ssh_user or USER environment variables)
|
81
|
+
-w, --password If used, then expect SSH connections to ask for a password.
|
82
|
+
-y GATEWAYS_CONF, Name of the gateways configuration to be used. Can also be set from environment variable hpc_ssh_gateways_conf.
|
83
|
+
--ssh-gateways-conf
|
84
|
+
```
|
85
|
+
|
86
|
+
## Examples
|
87
|
+
|
88
|
+
```bash
|
89
|
+
# Get nodes to be deployed now
|
90
|
+
./bin/nodes_to_deploy
|
91
|
+
|
92
|
+
# Get nodes to be deployed now among a given set of nodes
|
93
|
+
./bin/nodes_to_deploy --node /node12.+/
|
94
|
+
|
95
|
+
# Get nodes to be deployed now, ignoring their current deployed information
|
96
|
+
./bin/nodes_to_deploy --ignore-deployed-info
|
97
|
+
|
98
|
+
# Get nodes to be deployed only considering their deployed information
|
99
|
+
./bin/nodes_to_deploy --ignore-schedule
|
100
|
+
|
101
|
+
# Get nodes that would be deployed on October 1st 2020 at 10:00:00 UTC
|
102
|
+
./bin/nodes_to_deploy --deployment-time "2020-10-01 10:00:00"
|
103
|
+
```
|
104
|
+
|
105
|
+
Example of output:
|
106
|
+
```
|
107
|
+
===== Nodes to deploy =====
|
108
|
+
node12had50
|
109
|
+
node12had51
|
110
|
+
node12had52
|
111
|
+
node12had53
|
112
|
+
node12had54
|
113
|
+
node12had55
|
114
|
+
node12had57
|
115
|
+
node12had58
|
116
|
+
node12had59
|
117
|
+
node12hst-nn5
|
118
|
+
node12hst-nn3
|
119
|
+
node12lnx18
|
120
|
+
```
|
121
|
+
|
122
|
+
## Used credentials
|
123
|
+
|
124
|
+
| Credential | Usage
|
125
|
+
| --- | --- |
|
126
|
+
|
127
|
+
## Used Metadata
|
128
|
+
|
129
|
+
| Metadata | Type | Usage
|
130
|
+
| --- | --- | --- |
|
131
|
+
|
132
|
+
## Used environment variables
|
133
|
+
|
134
|
+
| Variable | Usage
|
135
|
+
| --- | --- |
|
136
|
+
|
137
|
+
## External tools dependencies
|
138
|
+
|
139
|
+
None
|