octocatalog-diff 1.4.1 → 1.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.version +1 -1
- data/README.md +1 -0
- data/doc/CHANGELOG.md +9 -0
- data/doc/advanced-environment-variables.md +70 -0
- data/doc/configuration-hiera.md +29 -10
- data/doc/dev/releasing.md +6 -17
- data/doc/optionsref.md +92 -11
- data/lib/octocatalog-diff/catalog-util/builddir.rb +78 -22
- data/lib/octocatalog-diff/catalog-util/fileresources.rb +17 -10
- data/lib/octocatalog-diff/catalog/computed.rb +4 -24
- data/lib/octocatalog-diff/cli.rb +10 -3
- data/lib/octocatalog-diff/cli/options/hiera_config.rb +13 -5
- data/lib/octocatalog-diff/cli/options/hiera_path.rb +34 -19
- data/lib/octocatalog-diff/cli/options/hiera_path_strip.rb +24 -12
- data/lib/octocatalog-diff/util/parallel.rb +3 -12
- data/lib/octocatalog-diff/util/scriptrunner.rb +3 -8
- data/lib/octocatalog-diff/util/util.rb +47 -0
- metadata +3 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 0667ff8feb2bda55fc108e5d4809e953f53a7585
|
|
4
|
+
data.tar.gz: 1699635ec3113a19650cda0e7beeb1558d7eacc8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b10067b2f2f1ca1ea93f1fbdd2dd41385f3e71a14825c87701105e94a84e681d5327aea8401423dd6ae7e9152dd120a7d744b6bfccc70848ee16b894d842f38f
|
|
7
|
+
data.tar.gz: 3f4b1dd759791c9271ba02858834e7efef426a2a448a16089e7486c1922befb3f505f60f09ee486bf59ad1c4ea4d5f86b6dfd457a4ace44d085e3eba7278c8b0
|
data/.version
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
1.
|
|
1
|
+
1.5.0
|
data/README.md
CHANGED
|
@@ -53,6 +53,7 @@ The example above reflects the changes in the Puppet catalog from switching an u
|
|
|
53
53
|
- [Requirements](/doc/requirements.md)
|
|
54
54
|
- [Limitations](/doc/limitations.md)
|
|
55
55
|
- [List of all command line options](/doc/optionsref.md)
|
|
56
|
+
- [Environment variables](/doc/advanced-environment-variables.md)
|
|
56
57
|
|
|
57
58
|
### Project
|
|
58
59
|
|
data/doc/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,15 @@
|
|
|
8
8
|
</tr>
|
|
9
9
|
</thead><tbody>
|
|
10
10
|
<tr valign=top>
|
|
11
|
+
<td>1.5.0</td>
|
|
12
|
+
<td>2017-10-18</td>
|
|
13
|
+
<td>
|
|
14
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/151">#151</a>: (Enhancement) Support text differences in files where `source` is an array</li>
|
|
15
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/153">#153</a>: (Enhancement) Support for hiera 5</li>
|
|
16
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/152">#152</a>: (Internal) Better temporary directory handling</li>
|
|
17
|
+
</td>
|
|
18
|
+
</tr>
|
|
19
|
+
<tr valign=top>
|
|
11
20
|
<td>1.4.1</td>
|
|
12
21
|
<td>2017-10-02</td>
|
|
13
22
|
<td>
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Environment variables
|
|
2
|
+
|
|
3
|
+
The following environment variables have special meaning to octocatalog-diff:
|
|
4
|
+
|
|
5
|
+
### `OCTOCATALOG_DIFF_CONFIG_FILE`
|
|
6
|
+
|
|
7
|
+
This environment variable is used to locate the configuration file for the CLI. The use of configuration files is described generally in:
|
|
8
|
+
|
|
9
|
+
- [Configuration](/doc/configuration.md)
|
|
10
|
+
|
|
11
|
+
### `OCTOCATALOG_DIFF_CUSTOM_VERSION`
|
|
12
|
+
|
|
13
|
+
When set, the `octocatalog-diff` CLI will display this as the version number within debugging, instead of the version number in the package. This is most useful if you want to use or include a git SHA in the version number.
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
$ export OCTOCATALOG_DIFF_CUSTOM_VERSION="@$(git rev-parse HEAD)"
|
|
17
|
+
$ octocatalog-diff -d ...
|
|
18
|
+
D, [2017-10-12T08:57:46.454738 #35205] DEBUG -- : Running octocatalog-diff @504d7f3c91267e5193beb103caae5d4d8cebfee3 with ruby 2.3.1
|
|
19
|
+
...
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### `OCTOCATALOG_DIFF_DEVELOPER_PATH`
|
|
23
|
+
|
|
24
|
+
When set, instead of loading libraries from the system or bundler location, libraries will be loaded from the specified value of this environment variable. This is used internally for development as we point users to unreleased code for debugging or testing.
|
|
25
|
+
|
|
26
|
+
```
|
|
27
|
+
$ export OCTOCATALOG_DIFF_DEVELOPER_PATH=$HOME/git-checkouts/octocatalog-diff
|
|
28
|
+
$ octocatalog-diff ...
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### `OCTOCATALOG_DIFF_TEMPDIR`
|
|
32
|
+
|
|
33
|
+
When set:
|
|
34
|
+
|
|
35
|
+
- `octocatalog-diff` will create all of its temporary directories within the specified directory.
|
|
36
|
+
- `octocatalog-diff` will not attempt to remove any temporary directories it creates.
|
|
37
|
+
|
|
38
|
+
This is useful in the following situations:
|
|
39
|
+
|
|
40
|
+
- You are calling `octocatalog-diff` from within another program which is highly parallelized, and `at_exit` handlers are difficult to implement. Instead of figuring that all out (if it can even be figured out), you create a temporary directory before the parallelized logic, and remove it afterwards.
|
|
41
|
+
|
|
42
|
+
- You wish to debug intermediate output. For example, you may be instructed to set this variable and send some of the output to the project maintainers if you request assistance.
|
|
43
|
+
|
|
44
|
+
This variable is used internally for the parallelized logic for catalog compilation, but the value set from the environment will override any internal usage.
|
|
45
|
+
|
|
46
|
+
### `OCTOCATALOG_DIFF_VERSION`
|
|
47
|
+
|
|
48
|
+
This variable is used when building the gem, to override the default version. This is used for internal testing of `octocatalog-diff` before public releases. This variable is not useful outside the build context.
|
|
49
|
+
|
|
50
|
+
### `PUPPETDB_HOST`
|
|
51
|
+
|
|
52
|
+
This variable specifies the fully qualified domain name or IP address of the PuppetDB server.
|
|
53
|
+
|
|
54
|
+
Note: If `PUPPETDB_URL` is specified, then `PUPPETDB_HOST` is not consulted.
|
|
55
|
+
|
|
56
|
+
### `PUPPETDB_PORT`
|
|
57
|
+
|
|
58
|
+
This variable specifies the port number of the PuppetDB server.
|
|
59
|
+
|
|
60
|
+
Note: If `PUPPETDB_URL` is specified, then `PUPPETDB_PORT` is not consulted.
|
|
61
|
+
|
|
62
|
+
### `PUPPETDB_URL`
|
|
63
|
+
|
|
64
|
+
This variable specifies the URL to the PuppetDB server.
|
|
65
|
+
|
|
66
|
+
Example: `https://puppetdb.example.net:8081`
|
|
67
|
+
|
|
68
|
+
### `PUPPET_FACT_DIR`
|
|
69
|
+
|
|
70
|
+
This variable specifies the directory path where puppet fact files are stored. (Fact files must be named `<fqdn>.yaml` where `<fqdn>` is specified when running `octocatalog-diff`.)
|
data/doc/configuration-hiera.md
CHANGED
|
@@ -1,6 +1,28 @@
|
|
|
1
1
|
# Configuring octocatalog-diff to use Hiera
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Hiera 5
|
|
4
|
+
|
|
5
|
+
Hiera 5 is included with Puppet 4.9 and higher.
|
|
6
|
+
|
|
7
|
+
If there is a `hiera.yaml` file in the base directory of the environment that is in hiera 5 format, and you are running Puppet 4.9 or higher, then that file will be recognized by Puppet (and therefore, by octocatalog-diff). There is no special configuration for octocatalog-diff needed to make this work. Similarly, there is no command line option or setting to changed this behavior, because there is no corresponding option to change Puppet's behavior.
|
|
8
|
+
|
|
9
|
+
If you are running Puppet 4.8 or lower, then the `hiera.yaml` file in the base directory of the environment will be ignored (unless you use `--hiera-config` to specify it as your global configuration file).
|
|
10
|
+
|
|
11
|
+
If you have no global hiera configuration and you wish to rely on a `hiera.yaml` file in the base directory of your environment, make sure that you are *not* using any of the following command line options or [configuration settings](/doc/configuration.md):
|
|
12
|
+
|
|
13
|
+
- `--hiera-path` or `settings[:hiera_path]`
|
|
14
|
+
- `--hiera-path-strip` or `settings[:hiera_path_strip]`
|
|
15
|
+
- `--hiera-config` or `settings[:hiera_config]`
|
|
16
|
+
|
|
17
|
+
There is more information about Hiera 5 in Puppet's documentation:
|
|
18
|
+
|
|
19
|
+
- [Enable the environment layer for existing Hiera data](https://puppet.com/docs/puppet/5.3/hiera_migrate_environments.html)
|
|
20
|
+
|
|
21
|
+
## Hiera global configuration
|
|
22
|
+
|
|
23
|
+
If you are using Hiera 5 with a global configuration, or you are using Hiera 3 or before, then you must already have a [`hiera.yaml`](https://docs.puppet.com/puppet/latest/reference/config_file_hiera.html) file to configure it. These instructions will guide you through pointing octocatalog-diff at that configuration file.
|
|
24
|
+
|
|
25
|
+
octocatalog-diff will automatically determine the version of your Hiera configuration file and treat it accordingly. (Hiera 5 configuration files are identified as such by a `version: 5` line in the file itself.)
|
|
4
26
|
|
|
5
27
|
Before you start, please understand how octocatalog-diff compiles a catalog:
|
|
6
28
|
|
|
@@ -10,9 +32,9 @@ Before you start, please understand how octocatalog-diff compiles a catalog:
|
|
|
10
32
|
- It compiles the catalog, based on the temporary directory, for environment=production
|
|
11
33
|
- It removes the temporary directory
|
|
12
34
|
|
|
13
|
-
|
|
35
|
+
### Configuring the location of global hiera.yaml
|
|
14
36
|
|
|
15
|
-
The command line option `--hiera-config PATH` allows you to set the path to hiera.yaml.
|
|
37
|
+
The command line option `--hiera-config PATH` allows you to set the path to the global hiera.yaml.
|
|
16
38
|
|
|
17
39
|
You may specify this as either an absolute or a relative path.
|
|
18
40
|
|
|
@@ -24,12 +46,9 @@ You may specify this as either an absolute or a relative path.
|
|
|
24
46
|
bin/octocatalog-diff --hiera-config hiera.yaml ...
|
|
25
47
|
```
|
|
26
48
|
|
|
27
|
-
The path is relative to a checkout of your Puppet repository. With the setting above, it will use the file named `hiera.yaml` that is at the top level
|
|
28
|
-
of your Puppet checkout.
|
|
49
|
+
The path is relative to a checkout of your Puppet repository. With the setting above, it will use the file named `hiera.yaml` that is at the top level of your Puppet checkout.
|
|
29
50
|
|
|
30
|
-
Perhaps your hiera.yaml file is in a subdirectory of your Puppet checkout. In that case, just use the relative directory path. Be sure not to add a leading `/` though,
|
|
31
|
-
because you don't want octocatalog-diff to treat it as an absolute path. In the following example, suppose you have a top level directory called `config` and your
|
|
32
|
-
`hiera.yaml` file is contained within it. You could then use:
|
|
51
|
+
Perhaps your hiera.yaml file is in a subdirectory of your Puppet checkout. In that case, just use the relative directory path. Be sure not to add a leading `/` though, because you don't want octocatalog-diff to treat it as an absolute path. In the following example, suppose you have a top level directory called `config` and your `hiera.yaml` file is contained within it. You could then use:
|
|
33
52
|
|
|
34
53
|
```
|
|
35
54
|
bin/octocatalog-diff --hiera-config config/hiera.yaml ...
|
|
@@ -65,7 +84,7 @@ You may specify this as either an absolute or a relative path.
|
|
|
65
84
|
|
|
66
85
|
We strongly recommend that you version-control your hiera.yaml file within your Puppet repository, and use the relative path option described above.
|
|
67
86
|
|
|
68
|
-
|
|
87
|
+
### Configuring the directory in your repository in which hiera data files are found
|
|
69
88
|
|
|
70
89
|
The command line option `--hiera-path PATH` allows you to set the directory path, relative to the checkout of your Puppet repository, of your Hiera YAML/JSON data files.
|
|
71
90
|
|
|
@@ -88,7 +107,7 @@ If you are specifying the Hiera data path in the [configuration file](/doc/confi
|
|
|
88
107
|
|
|
89
108
|
octocatalog-diff will fail if you specify a path that is not a directory.
|
|
90
109
|
|
|
91
|
-
|
|
110
|
+
### Configuring the prefix path to strip
|
|
92
111
|
|
|
93
112
|
This is a different, and potentially more complex, alternative to `--hiera-path` / `settings[:hiera_path]` described in the prior section. Unless you have a very good reason, you should prefer to use the instructions above.
|
|
94
113
|
|
data/doc/dev/releasing.md
CHANGED
|
@@ -6,24 +6,13 @@ The project maintainers are responsible for bumping the version number, regenera
|
|
|
6
6
|
|
|
7
7
|
*This procedure is performed by a GitHubber.*
|
|
8
8
|
|
|
9
|
-
To test the new version of `octocatalog-diff` in the GitHub Puppet repository:
|
|
9
|
+
To test the new version of `octocatalog-diff` in the GitHub Puppet repository, check out `github/puppet` and:
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
```
|
|
18
|
-
OCTOCATALOG_DIFF_VERSION=0.6.0a rake gem:force-build
|
|
19
|
-
```
|
|
20
|
-
- Run the task to install the gem into your Puppet checkout:
|
|
21
|
-
|
|
22
|
-
```
|
|
23
|
-
OCTOCATALOG_DIFF_VERSION=0.6.0a rake gem:localinstall
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
0. Back in the Puppet checkout, ensure that the changes are as expected (updates to Gemfile / Gemfile.lock, addition of new gem). Push the change and build appropriate CI job(s) to validate the changes.
|
|
11
|
+
- Start a new branch based off master
|
|
12
|
+
- Run `script/update-octocatalog-diff -r <ocd_branch_name>`
|
|
13
|
+
- Confirm and commit the result
|
|
14
|
+
- Make sure all CI jobs pass
|
|
15
|
+
- Run the `puppet-catalog-diff` CI job and make sure it passes and shows expected results
|
|
27
16
|
|
|
28
17
|
## Merging one PR
|
|
29
18
|
|
data/doc/optionsref.md
CHANGED
|
@@ -43,11 +43,18 @@ Usage: octocatalog-diff [command line options]
|
|
|
43
43
|
--master-cache-branch BRANCH Branch to cache
|
|
44
44
|
--safe-to-delete-cached-master-dir PATH
|
|
45
45
|
OK to delete cached master directory at this path
|
|
46
|
-
--hiera-config
|
|
46
|
+
--hiera-config STRING Full or relative path to global Hiera configuration file globally
|
|
47
|
+
--to-hiera-config STRING Full or relative path to global Hiera configuration file for the to branch
|
|
48
|
+
--from-hiera-config STRING Full or relative path to global Hiera configuration file for the from branch
|
|
47
49
|
--no-hiera-config Disable hiera config file installation
|
|
48
|
-
--hiera-path
|
|
50
|
+
--hiera-path STRING Path to hiera data directory, relative to top directory of repository globally
|
|
51
|
+
--to-hiera-path STRING Path to hiera data directory, relative to top directory of repository for the to branch
|
|
52
|
+
--from-hiera-path STRING Path to hiera data directory, relative to top directory of repository for the from branch
|
|
49
53
|
--no-hiera-path Do not use any default hiera path settings
|
|
50
|
-
--hiera-path-strip
|
|
54
|
+
--hiera-path-strip STRING Path prefix to strip when munging hiera.yaml globally
|
|
55
|
+
--to-hiera-path-strip STRING Path prefix to strip when munging hiera.yaml for the to branch
|
|
56
|
+
--from-hiera-path-strip STRING
|
|
57
|
+
Path prefix to strip when munging hiera.yaml for the from branch
|
|
51
58
|
--no-hiera-path-strip Do not use any default hiera path strip settings
|
|
52
59
|
--ignore-attr "attr1,attr2,..."
|
|
53
60
|
Attributes to ignore
|
|
@@ -81,10 +88,10 @@ Usage: octocatalog-diff [command line options]
|
|
|
81
88
|
--from-puppet-binary STRING Full path to puppet binary for the from branch
|
|
82
89
|
--facts-terminus STRING Facts terminus: one of yaml, facter
|
|
83
90
|
--puppetdb-ssl-ca FILENAME CA certificate that signed the PuppetDB certificate
|
|
84
|
-
--puppetdb-ssl-client-password PASSWORD
|
|
85
|
-
Password for SSL client key to connect to PuppetDB
|
|
86
91
|
--puppetdb-ssl-client-cert FILENAME
|
|
87
92
|
SSL client certificate to connect to PuppetDB
|
|
93
|
+
--puppetdb-ssl-client-password PASSWORD
|
|
94
|
+
Password for SSL client key to connect to PuppetDB
|
|
88
95
|
--puppetdb-ssl-client-key FILENAME
|
|
89
96
|
SSL client key to connect to PuppetDB
|
|
90
97
|
--puppetdb-ssl-client-password-file FILENAME
|
|
@@ -643,6 +650,43 @@ by permitting a data type specification as well. (<a href="../lib/octocatalog-di
|
|
|
643
650
|
</td>
|
|
644
651
|
</tr>
|
|
645
652
|
|
|
653
|
+
<tr>
|
|
654
|
+
<td valign=top>
|
|
655
|
+
<pre><code>--from-hiera-config STRING</code></pre>
|
|
656
|
+
</td>
|
|
657
|
+
<td valign=top>
|
|
658
|
+
Full or relative path to global Hiera configuration file for the from branch
|
|
659
|
+
</td>
|
|
660
|
+
<td valign=top>
|
|
661
|
+
Specify a relative path to the Hiera yaml file (<a href="../lib/octocatalog-diff/cli/options/hiera_config.rb">hiera_config.rb</a>)
|
|
662
|
+
</td>
|
|
663
|
+
</tr>
|
|
664
|
+
|
|
665
|
+
<tr>
|
|
666
|
+
<td valign=top>
|
|
667
|
+
<pre><code>--from-hiera-path STRING</code></pre>
|
|
668
|
+
</td>
|
|
669
|
+
<td valign=top>
|
|
670
|
+
Path to hiera data directory, relative to top directory of repository for the from branch
|
|
671
|
+
</td>
|
|
672
|
+
<td valign=top>
|
|
673
|
+
Specify the path to the Hiera data directory (relative to the top level Puppet checkout). For Puppet Enterprise and the
|
|
674
|
+
Puppet control repo template, the value of this should be 'hieradata', which is the default. (<a href="../lib/octocatalog-diff/cli/options/hiera_path.rb">hiera_path.rb</a>)
|
|
675
|
+
</td>
|
|
676
|
+
</tr>
|
|
677
|
+
|
|
678
|
+
<tr>
|
|
679
|
+
<td valign=top>
|
|
680
|
+
<pre><code>--from-hiera-path-strip STRING</code></pre>
|
|
681
|
+
</td>
|
|
682
|
+
<td valign=top>
|
|
683
|
+
Path prefix to strip when munging hiera.yaml for the from branch
|
|
684
|
+
</td>
|
|
685
|
+
<td valign=top>
|
|
686
|
+
Specify the path to strip off the datadir to munge hiera.yaml file (<a href="../lib/octocatalog-diff/cli/options/hiera_path_strip.rb">hiera_path_strip.rb</a>)
|
|
687
|
+
</td>
|
|
688
|
+
</tr>
|
|
689
|
+
|
|
646
690
|
<tr>
|
|
647
691
|
<td valign=top>
|
|
648
692
|
<pre><code>--from-puppet-binary STRING</code></pre>
|
|
@@ -773,10 +817,10 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
|
|
|
773
817
|
|
|
774
818
|
<tr>
|
|
775
819
|
<td valign=top>
|
|
776
|
-
<pre><code>--hiera-config
|
|
820
|
+
<pre><code>--hiera-config STRING</code></pre>
|
|
777
821
|
</td>
|
|
778
822
|
<td valign=top>
|
|
779
|
-
|
|
823
|
+
Full or relative path to global Hiera configuration file globally
|
|
780
824
|
</td>
|
|
781
825
|
<td valign=top>
|
|
782
826
|
Specify a relative path to the Hiera yaml file (<a href="../lib/octocatalog-diff/cli/options/hiera_config.rb">hiera_config.rb</a>)
|
|
@@ -785,10 +829,10 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
|
|
|
785
829
|
|
|
786
830
|
<tr>
|
|
787
831
|
<td valign=top>
|
|
788
|
-
<pre><code>--hiera-path
|
|
832
|
+
<pre><code>--hiera-path STRING</code></pre>
|
|
789
833
|
</td>
|
|
790
834
|
<td valign=top>
|
|
791
|
-
Path to hiera data directory, relative to top directory of repository
|
|
835
|
+
Path to hiera data directory, relative to top directory of repository globally
|
|
792
836
|
</td>
|
|
793
837
|
<td valign=top>
|
|
794
838
|
Specify the path to the Hiera data directory (relative to the top level Puppet checkout). For Puppet Enterprise and the
|
|
@@ -798,10 +842,10 @@ Puppet control repo template, the value of this should be 'hieradata', which is
|
|
|
798
842
|
|
|
799
843
|
<tr>
|
|
800
844
|
<td valign=top>
|
|
801
|
-
<pre><code>--hiera-path-strip
|
|
845
|
+
<pre><code>--hiera-path-strip STRING</code></pre>
|
|
802
846
|
</td>
|
|
803
847
|
<td valign=top>
|
|
804
|
-
Path prefix to strip when munging hiera.yaml
|
|
848
|
+
Path prefix to strip when munging hiera.yaml globally
|
|
805
849
|
</td>
|
|
806
850
|
<td valign=top>
|
|
807
851
|
Specify the path to strip off the datadir to munge hiera.yaml file (<a href="../lib/octocatalog-diff/cli/options/hiera_path_strip.rb">hiera_path_strip.rb</a>)
|
|
@@ -1541,6 +1585,43 @@ by permitting a data type specification as well. (<a href="../lib/octocatalog-di
|
|
|
1541
1585
|
</td>
|
|
1542
1586
|
</tr>
|
|
1543
1587
|
|
|
1588
|
+
<tr>
|
|
1589
|
+
<td valign=top>
|
|
1590
|
+
<pre><code>--to-hiera-config STRING</code></pre>
|
|
1591
|
+
</td>
|
|
1592
|
+
<td valign=top>
|
|
1593
|
+
Full or relative path to global Hiera configuration file for the to branch
|
|
1594
|
+
</td>
|
|
1595
|
+
<td valign=top>
|
|
1596
|
+
Specify a relative path to the Hiera yaml file (<a href="../lib/octocatalog-diff/cli/options/hiera_config.rb">hiera_config.rb</a>)
|
|
1597
|
+
</td>
|
|
1598
|
+
</tr>
|
|
1599
|
+
|
|
1600
|
+
<tr>
|
|
1601
|
+
<td valign=top>
|
|
1602
|
+
<pre><code>--to-hiera-path STRING</code></pre>
|
|
1603
|
+
</td>
|
|
1604
|
+
<td valign=top>
|
|
1605
|
+
Path to hiera data directory, relative to top directory of repository for the to branch
|
|
1606
|
+
</td>
|
|
1607
|
+
<td valign=top>
|
|
1608
|
+
Specify the path to the Hiera data directory (relative to the top level Puppet checkout). For Puppet Enterprise and the
|
|
1609
|
+
Puppet control repo template, the value of this should be 'hieradata', which is the default. (<a href="../lib/octocatalog-diff/cli/options/hiera_path.rb">hiera_path.rb</a>)
|
|
1610
|
+
</td>
|
|
1611
|
+
</tr>
|
|
1612
|
+
|
|
1613
|
+
<tr>
|
|
1614
|
+
<td valign=top>
|
|
1615
|
+
<pre><code>--to-hiera-path-strip STRING</code></pre>
|
|
1616
|
+
</td>
|
|
1617
|
+
<td valign=top>
|
|
1618
|
+
Path prefix to strip when munging hiera.yaml for the to branch
|
|
1619
|
+
</td>
|
|
1620
|
+
<td valign=top>
|
|
1621
|
+
Specify the path to strip off the datadir to munge hiera.yaml file (<a href="../lib/octocatalog-diff/cli/options/hiera_path_strip.rb">hiera_path_strip.rb</a>)
|
|
1622
|
+
</td>
|
|
1623
|
+
</tr>
|
|
1624
|
+
|
|
1544
1625
|
<tr>
|
|
1545
1626
|
<td valign=top>
|
|
1546
1627
|
<pre><code>--to-puppet-binary STRING</code></pre>
|
|
@@ -4,6 +4,7 @@ require 'yaml'
|
|
|
4
4
|
|
|
5
5
|
require_relative '../facts'
|
|
6
6
|
require_relative 'enc'
|
|
7
|
+
require_relative '../util/util'
|
|
7
8
|
|
|
8
9
|
module OctocatalogDiff
|
|
9
10
|
module CatalogUtil
|
|
@@ -38,9 +39,7 @@ module OctocatalogDiff
|
|
|
38
39
|
# @param options [Hash] Options for class; see above description
|
|
39
40
|
def initialize(options = {}, logger = nil)
|
|
40
41
|
@options = options.dup
|
|
41
|
-
@tempdir =
|
|
42
|
-
at_exit { FileUtils.rm_rf(@tempdir) if File.directory?(@tempdir) }
|
|
43
|
-
|
|
42
|
+
@tempdir = OctocatalogDiff::Util::Util.temp_dir('ocd-builddir-')
|
|
44
43
|
@factdir = nil
|
|
45
44
|
@enc = nil
|
|
46
45
|
@fact_file = nil
|
|
@@ -237,25 +236,11 @@ module OctocatalogDiff
|
|
|
237
236
|
|
|
238
237
|
# Munge datadir in hiera config file
|
|
239
238
|
obj = YAML.load_file(file_src)
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
obj[key.to_sym][:datadir].sub!(rexp1, @tempdir)
|
|
246
|
-
elsif options[:hiera_path].is_a?(String)
|
|
247
|
-
obj[key.to_sym][:datadir] = File.join(@tempdir, 'environments', environment, options[:hiera_path])
|
|
248
|
-
end
|
|
249
|
-
rexp2 = Regexp.new('%{(::)?environment}')
|
|
250
|
-
obj[key.to_sym][:datadir].sub!(rexp2, environment)
|
|
251
|
-
|
|
252
|
-
# Make sure the dirctory exists. If not, log a warning. This is *probably* a setup error, but we don't
|
|
253
|
-
# want it to be fatal in case (for example) someone is doing an octocatalog-diff to verify moving this
|
|
254
|
-
# directory around or even setting up Hiera for the very first time.
|
|
255
|
-
unless File.directory?(obj[key.to_sym][:datadir])
|
|
256
|
-
message = "WARNING: Hiera datadir for #{key} doesn't seem to exist at #{obj[key.to_sym][:datadir]}"
|
|
257
|
-
logger.warn message
|
|
258
|
-
end
|
|
239
|
+
version = obj['version'] || obj[:version] || 3
|
|
240
|
+
if version.to_i == 5
|
|
241
|
+
update_hiera_config_v5(logger, options, obj)
|
|
242
|
+
else
|
|
243
|
+
update_hiera_config_v3(logger, options, obj)
|
|
259
244
|
end
|
|
260
245
|
|
|
261
246
|
# Write properly formatted hiera config file into temporary directory
|
|
@@ -292,6 +277,77 @@ module OctocatalogDiff
|
|
|
292
277
|
|
|
293
278
|
private
|
|
294
279
|
|
|
280
|
+
# Jump-off for hiera v3 (or earlier)
|
|
281
|
+
# @param logger [Logger] Logger object
|
|
282
|
+
# @param options [Hash] Options hash
|
|
283
|
+
# @param obj [Hash] Parsed hiera.yaml file
|
|
284
|
+
def update_hiera_config_v3(logger, options, obj)
|
|
285
|
+
([obj[:backends]].flatten || %w(yaml json)).each do |key|
|
|
286
|
+
next unless obj.key?(key.to_sym) && !obj[key.to_sym][:datadir].nil?
|
|
287
|
+
obj[key.to_sym][:datadir] = hiera_munge(options, obj[key.to_sym][:datadir])
|
|
288
|
+
|
|
289
|
+
# Make sure the directory exists. If not, log a warning. This is *probably* a setup error, but we don't
|
|
290
|
+
# want it to be fatal in case (for example) someone is doing an octocatalog-diff to verify moving this
|
|
291
|
+
# directory around or even setting up Hiera for the very first time.
|
|
292
|
+
unless File.directory?(obj[key.to_sym][:datadir])
|
|
293
|
+
message = "WARNING: Hiera datadir for #{key} doesn't seem to exist at #{obj[key.to_sym][:datadir]}"
|
|
294
|
+
logger.warn message
|
|
295
|
+
end
|
|
296
|
+
end
|
|
297
|
+
end
|
|
298
|
+
|
|
299
|
+
# Jump-off for hiera v5
|
|
300
|
+
# @param logger [Logger] Logger object
|
|
301
|
+
# @param options [Hash] Options hash
|
|
302
|
+
# @param obj [Hash] Parsed hiera.yaml file
|
|
303
|
+
def update_hiera_config_v5(_logger, options, obj)
|
|
304
|
+
defaults_key = obj.key?(:defaults) ? :defaults : 'defaults'
|
|
305
|
+
hierarchy_key = obj.key?(:hierarchy) ? :hierarchy : 'hierarchy'
|
|
306
|
+
|
|
307
|
+
# Fix defaults:datadir
|
|
308
|
+
if obj[defaults_key].is_a?(Hash)
|
|
309
|
+
[:datadir, 'datadir'].each do |key|
|
|
310
|
+
next unless obj[defaults_key].key?(key)
|
|
311
|
+
obj[defaults_key][key] = hiera_munge(options, obj[defaults_key][key])
|
|
312
|
+
end
|
|
313
|
+
end
|
|
314
|
+
|
|
315
|
+
# Fix hierarchy:datadir
|
|
316
|
+
if obj[hierarchy_key].is_a?(Array)
|
|
317
|
+
obj[hierarchy_key].each do |level|
|
|
318
|
+
[:datadir, 'datadir'].each do |key|
|
|
319
|
+
next unless level.key?(key)
|
|
320
|
+
if options[:hiera_path_strip].is_a?(String)
|
|
321
|
+
level[key] = hiera_munge(options, level[key])
|
|
322
|
+
elsif options[:hiera_path].is_a?(String)
|
|
323
|
+
message = [
|
|
324
|
+
"Hierarchy item #{level.inspect} has a datadir.",
|
|
325
|
+
'--hiera-path is not supported in this situation.',
|
|
326
|
+
'Please use --hiera-path-strip.'
|
|
327
|
+
].join(' ')
|
|
328
|
+
raise ArgumentError, message
|
|
329
|
+
end
|
|
330
|
+
end
|
|
331
|
+
end
|
|
332
|
+
end
|
|
333
|
+
end
|
|
334
|
+
|
|
335
|
+
# Hiera munge - shared method to apply :hiera_path_strip and :hiera_path
|
|
336
|
+
def hiera_munge(options, current_value)
|
|
337
|
+
return if current_value.nil?
|
|
338
|
+
|
|
339
|
+
if options[:hiera_path_strip].is_a?(String)
|
|
340
|
+
rexp1 = Regexp.new('^' + options[:hiera_path_strip])
|
|
341
|
+
current_value.sub!(rexp1, @tempdir)
|
|
342
|
+
elsif options[:hiera_path].is_a?(String)
|
|
343
|
+
current_value = File.join(@tempdir, 'environments', environment, options[:hiera_path])
|
|
344
|
+
end
|
|
345
|
+
rexp2 = Regexp.new('%{(::)?environment}')
|
|
346
|
+
current_value.sub!(rexp2, environment)
|
|
347
|
+
|
|
348
|
+
current_value
|
|
349
|
+
end
|
|
350
|
+
|
|
295
351
|
# Install SSL certificate authority certificate
|
|
296
352
|
# @param logger [Logger] Logger object
|
|
297
353
|
# @param options [Hash] Options hash
|
|
@@ -27,18 +27,20 @@ module OctocatalogDiff
|
|
|
27
27
|
# module path that is specified within the environment.conf file (assuming the default 'modules'
|
|
28
28
|
# directory doesn't exist or the module isn't found in there). If the file can't be found then
|
|
29
29
|
# this returns nil which may trigger an error.
|
|
30
|
-
# @param
|
|
30
|
+
# @param src_in [String|Array] A file reference: puppet:///modules/xxx/yyy
|
|
31
31
|
# @param modulepaths [Array] Cached module path
|
|
32
32
|
# @return [String] File system path to referenced file
|
|
33
|
-
def self.file_path(
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
end
|
|
33
|
+
def self.file_path(src_in, modulepaths)
|
|
34
|
+
valid_sources = [src_in].flatten.select { |line| line =~ %r{\Apuppet:///modules/([^/]+)/(.+)} }
|
|
35
|
+
return unless valid_sources.any?
|
|
37
36
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
37
|
+
valid_sources.each do |src|
|
|
38
|
+
src =~ %r{\Apuppet:///modules/([^/]+)/(.+)}
|
|
39
|
+
path = File.join(Regexp.last_match(1), 'files', Regexp.last_match(2))
|
|
40
|
+
modulepaths.each do |mp|
|
|
41
|
+
file = File.join(mp, path)
|
|
42
|
+
return file if File.exist?(file)
|
|
43
|
+
end
|
|
42
44
|
end
|
|
43
45
|
|
|
44
46
|
nil
|
|
@@ -125,9 +127,14 @@ module OctocatalogDiff
|
|
|
125
127
|
!resource['parameters'].nil? && \
|
|
126
128
|
resource['parameters'].key?('source') && \
|
|
127
129
|
!resource['parameters'].key?('content') && \
|
|
128
|
-
resource
|
|
130
|
+
valid_sources?(resource)
|
|
131
|
+
|
|
129
132
|
false
|
|
130
133
|
end
|
|
134
|
+
|
|
135
|
+
def self.valid_sources?(resource)
|
|
136
|
+
[resource['parameters']['source']].flatten.select { |line| line =~ %r{\Apuppet:///modules/([^/]+)/(.+)} }.any?
|
|
137
|
+
end
|
|
131
138
|
end
|
|
132
139
|
end
|
|
133
140
|
end
|
|
@@ -11,6 +11,7 @@ require_relative '../catalog-util/command'
|
|
|
11
11
|
require_relative '../catalog-util/facts'
|
|
12
12
|
require_relative '../util/puppetversion'
|
|
13
13
|
require_relative '../util/scriptrunner'
|
|
14
|
+
require_relative '../util/util'
|
|
14
15
|
|
|
15
16
|
module OctocatalogDiff
|
|
16
17
|
class Catalog
|
|
@@ -69,23 +70,6 @@ module OctocatalogDiff
|
|
|
69
70
|
OctocatalogDiff::CatalogUtil::FileResources.convert_file_resources(self, environment)
|
|
70
71
|
end
|
|
71
72
|
|
|
72
|
-
private
|
|
73
|
-
|
|
74
|
-
# Private method: Clean up a checkout directory, if it exists
|
|
75
|
-
def cleanup_checkout_dir(checkout_dir, logger)
|
|
76
|
-
return unless File.directory?(checkout_dir)
|
|
77
|
-
logger.debug("Cleaning up temporary directory #{checkout_dir}")
|
|
78
|
-
# Sometimes this seems to break when handling the recursive removal when running under
|
|
79
|
-
# a parallel environment. Trap and ignore the errors here if we don't care about them.
|
|
80
|
-
begin
|
|
81
|
-
FileUtils.remove_entry_secure checkout_dir
|
|
82
|
-
# :nocov:
|
|
83
|
-
rescue Errno::ENOTEMPTY, Errno::ENOENT => exc
|
|
84
|
-
logger.debug "cleanup_checkout_dir(#{checkout_dir}) logged #{exc.class} - this can be ignored"
|
|
85
|
-
# :nocov:
|
|
86
|
-
end
|
|
87
|
-
end
|
|
88
|
-
|
|
89
73
|
# Private method: Bootstrap a directory
|
|
90
74
|
def bootstrap(logger)
|
|
91
75
|
return if @builddir
|
|
@@ -99,9 +83,7 @@ module OctocatalogDiff
|
|
|
99
83
|
tmphash[:basedir] = @options[:bootstrapped_dir]
|
|
100
84
|
elsif @options[:branch] == '.'
|
|
101
85
|
if @options[:bootstrap_current]
|
|
102
|
-
tmphash[:basedir] =
|
|
103
|
-
at_exit { cleanup_checkout_dir(tmphash[:basedir], logger) }
|
|
104
|
-
|
|
86
|
+
tmphash[:basedir] = OctocatalogDiff::Util::Util.temp_dir('ocd-bootstrap-basedir-')
|
|
105
87
|
FileUtils.cp_r File.join(@options[:basedir], '.'), tmphash[:basedir]
|
|
106
88
|
|
|
107
89
|
o = @options.reject { |k, _v| k == :branch }.merge(path: tmphash[:basedir])
|
|
@@ -110,10 +92,8 @@ module OctocatalogDiff
|
|
|
110
92
|
tmphash[:basedir] = @options[:basedir]
|
|
111
93
|
end
|
|
112
94
|
else
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
tmphash[:basedir] = checkout_dir
|
|
116
|
-
OctocatalogDiff::CatalogUtil::Bootstrap.bootstrap_directory(@options.merge(path: checkout_dir), logger)
|
|
95
|
+
tmphash[:basedir] = OctocatalogDiff::Util::Util.temp_dir('ocd-bootstrap-checkout-')
|
|
96
|
+
OctocatalogDiff::CatalogUtil::Bootstrap.bootstrap_directory(@options.merge(path: tmphash[:basedir]), logger)
|
|
117
97
|
end
|
|
118
98
|
|
|
119
99
|
# Create and populate the temporary directory
|
data/lib/octocatalog-diff/cli.rb
CHANGED
|
@@ -7,6 +7,7 @@ require_relative 'cli/options'
|
|
|
7
7
|
require_relative 'cli/printer'
|
|
8
8
|
require_relative 'errors'
|
|
9
9
|
require_relative 'util/catalogs'
|
|
10
|
+
require_relative 'util/util'
|
|
10
11
|
require_relative 'version'
|
|
11
12
|
|
|
12
13
|
require 'logger'
|
|
@@ -53,7 +54,7 @@ module OctocatalogDiff
|
|
|
53
54
|
# @return [Integer] Exit code: 0=no diffs, 1=something went wrong, 2=worked but there are diffs
|
|
54
55
|
def self.cli(argv = ARGV, logger = Logger.new(STDERR), opts = {})
|
|
55
56
|
# Save a copy of argv to print out later in debugging
|
|
56
|
-
argv_save = argv
|
|
57
|
+
argv_save = OctocatalogDiff::Util::Util.deep_dup(argv)
|
|
57
58
|
|
|
58
59
|
# Are there additional ARGV to munge, e.g. that have been supplied in the options from a
|
|
59
60
|
# configuration file?
|
|
@@ -70,8 +71,14 @@ module OctocatalogDiff
|
|
|
70
71
|
# Note: do NOT use 'options[k] ||= v' here because if the value of options[k] is boolean(false)
|
|
71
72
|
# it will then be overridden. Whereas the intent is to define values only for those keys that don't exist.
|
|
72
73
|
opts.each { |k, v| options[k] = v unless options.key?(k) }
|
|
73
|
-
veto_options = %w(enc header
|
|
74
|
+
veto_options = %w(enc header include_tags)
|
|
74
75
|
veto_options.each { |x| options.delete(x.to_sym) if options["no_#{x}".to_sym] }
|
|
76
|
+
if options[:no_hiera_config]
|
|
77
|
+
vetoes = %w[hiera_config to_hiera_config from_hiera_config]
|
|
78
|
+
vetoes.each do |key|
|
|
79
|
+
options.delete(key.to_sym)
|
|
80
|
+
end
|
|
81
|
+
end
|
|
75
82
|
options[:ignore].concat opts.fetch(:additional_ignores, [])
|
|
76
83
|
|
|
77
84
|
# Incorporate default options where needed.
|
|
@@ -125,7 +132,7 @@ module OctocatalogDiff
|
|
|
125
132
|
# @param argv [Array] Command line arguments (MUST be specified)
|
|
126
133
|
# @return [Hash] Options
|
|
127
134
|
def self.parse_opts(argv)
|
|
128
|
-
options = { ignore: DEFAULT_IGNORES
|
|
135
|
+
options = { ignore: OctocatalogDiff::Util::Util.deep_dup(DEFAULT_IGNORES) }
|
|
129
136
|
Options.parse_options(argv, options)
|
|
130
137
|
end
|
|
131
138
|
|
|
@@ -7,13 +7,21 @@ OctocatalogDiff::Cli::Options::Option.newoption(:hiera_config) do
|
|
|
7
7
|
has_weight 180
|
|
8
8
|
|
|
9
9
|
def parse(parser, options)
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
options
|
|
13
|
-
|
|
10
|
+
OctocatalogDiff::Cli::Options.option_globally_or_per_branch(
|
|
11
|
+
parser: parser,
|
|
12
|
+
options: options,
|
|
13
|
+
cli_name: 'hiera-config',
|
|
14
|
+
option_name: 'hiera_config',
|
|
15
|
+
desc: 'Full or relative path to global Hiera configuration file',
|
|
16
|
+
post_process: lambda do |opts|
|
|
17
|
+
raise ArgumentError, '--no-hiera-config incompatible with --hiera-config' if opts[:no_hiera_config]
|
|
18
|
+
end
|
|
19
|
+
)
|
|
14
20
|
|
|
15
21
|
parser.on('--no-hiera-config', 'Disable hiera config file installation') do
|
|
16
|
-
|
|
22
|
+
if options[:to_hiera_config] || options[:from_hiera_config]
|
|
23
|
+
raise ArgumentError, '--no-hiera-config incompatible with --hiera-config'
|
|
24
|
+
end
|
|
17
25
|
options[:no_hiera_config] = true
|
|
18
26
|
end
|
|
19
27
|
end
|
|
@@ -8,31 +8,46 @@ OctocatalogDiff::Cli::Options::Option.newoption(:hiera_path) do
|
|
|
8
8
|
has_weight 181
|
|
9
9
|
|
|
10
10
|
def parse(parser, options)
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
11
|
+
OctocatalogDiff::Cli::Options.option_globally_or_per_branch(
|
|
12
|
+
parser: parser,
|
|
13
|
+
options: options,
|
|
14
|
+
cli_name: 'hiera-path',
|
|
15
|
+
option_name: 'hiera_path',
|
|
16
|
+
desc: 'Path to hiera data directory, relative to top directory of repository',
|
|
17
|
+
validator: lambda do |path|
|
|
18
|
+
if path.start_with?('/')
|
|
19
|
+
raise ArgumentError, '--hiera-path PATH must be a relative path not an absolute path'
|
|
20
|
+
end
|
|
21
|
+
end,
|
|
22
|
+
translator: lambda do |path|
|
|
23
|
+
result = path.sub(%r{/+$}, '')
|
|
24
|
+
raise ArgumentError, '--hiera-path must not be empty' if result.empty?
|
|
25
|
+
result
|
|
26
|
+
end,
|
|
27
|
+
post_process: lambda do |opts|
|
|
28
|
+
if opts.key?(:to_hiera_path_strip) && opts[:to_hiera_path_strip] != :none
|
|
29
|
+
if opts.key?(:to_hiera_path) && opts[:to_hiera_path] != :none
|
|
30
|
+
raise ArgumentError, '--hiera-path and --hiera-path-strip are mutually exclusive'
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
if opts.key?(:from_hiera_path_strip) && opts[:from_hiera_path_strip] != :none
|
|
34
|
+
if opts.key?(:from_hiera_path) && opts[:from_hiera_path] != :none
|
|
35
|
+
raise ArgumentError, '--hiera-path and --hiera-path-strip are mutually exclusive'
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
if opts[:to_hiera_path] == :none || opts[:from_hiera_path] == :none
|
|
39
|
+
raise ArgumentError, '--hiera-path and --no-hiera-path are mutually exclusive'
|
|
40
|
+
end
|
|
14
41
|
end
|
|
15
|
-
|
|
16
|
-
if options[:hiera_path] == :none
|
|
17
|
-
raise ArgumentError, '--hiera-path and --no-hiera-path are mutually exclusive'
|
|
18
|
-
end
|
|
19
|
-
|
|
20
|
-
options[:hiera_path] = path_in
|
|
21
|
-
|
|
22
|
-
if options[:hiera_path].start_with?('/')
|
|
23
|
-
raise ArgumentError, '--hiera-path PATH must be a relative path not an absolute path'
|
|
24
|
-
end
|
|
25
|
-
|
|
26
|
-
options[:hiera_path].sub!(%r{/+$}, '')
|
|
27
|
-
raise ArgumentError, '--hiera-path must not be empty' if options[:hiera_path].empty?
|
|
28
|
-
end
|
|
42
|
+
)
|
|
29
43
|
|
|
30
44
|
parser.on('--no-hiera-path', 'Do not use any default hiera path settings') do
|
|
31
|
-
if options[:
|
|
45
|
+
if options[:to_hiera_path].is_a?(String) || options[:from_hiera_path].is_a?(String)
|
|
32
46
|
raise ArgumentError, '--hiera-path and --no-hiera-path are mutually exclusive'
|
|
33
47
|
end
|
|
34
48
|
|
|
35
|
-
options[:
|
|
49
|
+
options[:from_hiera_path] = :none
|
|
50
|
+
options[:to_hiera_path] = :none
|
|
36
51
|
end
|
|
37
52
|
end
|
|
38
53
|
end
|
|
@@ -7,23 +7,35 @@ OctocatalogDiff::Cli::Options::Option.newoption(:hiera_path_strip) do
|
|
|
7
7
|
has_weight 182
|
|
8
8
|
|
|
9
9
|
def parse(parser, options)
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
10
|
+
OctocatalogDiff::Cli::Options.option_globally_or_per_branch(
|
|
11
|
+
parser: parser,
|
|
12
|
+
options: options,
|
|
13
|
+
cli_name: 'hiera-path-strip',
|
|
14
|
+
option_name: 'hiera_path_strip',
|
|
15
|
+
desc: 'Path prefix to strip when munging hiera.yaml',
|
|
16
|
+
post_process: lambda do |opts|
|
|
17
|
+
if opts.key?(:to_hiera_path) && opts[:to_hiera_path] != :none
|
|
18
|
+
if opts.key?(:to_hiera_path_strip) && opts[:to_hiera_path_strip] != :none
|
|
19
|
+
raise ArgumentError, '--hiera-path and --hiera-path-strip are mutually exclusive'
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
if opts.key?(:from_hiera_path) && opts[:from_hiera_path] != :none
|
|
23
|
+
if opts.key?(:from_hiera_path_strip) && opts[:from_hiera_path_strip] != :none
|
|
24
|
+
raise ArgumentError, '--hiera-path and --hiera-path-strip are mutually exclusive'
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
if opts[:to_hiera_path_strip] == :none || opts[:from_hiera_path_strip] == :none
|
|
28
|
+
raise ArgumentError, '--hiera-path-strip and --no-hiera-path-strip are mutually exclusive'
|
|
29
|
+
end
|
|
13
30
|
end
|
|
14
|
-
|
|
15
|
-
if options[:hiera_path_strip] == :none
|
|
16
|
-
raise ArgumentError, '--hiera-path-strip and --no-hiera-path-strip are mutually exclusive'
|
|
17
|
-
end
|
|
18
|
-
|
|
19
|
-
options[:hiera_path_strip] = path_in
|
|
20
|
-
end
|
|
31
|
+
)
|
|
21
32
|
|
|
22
33
|
parser.on('--no-hiera-path-strip', 'Do not use any default hiera path strip settings') do
|
|
23
|
-
if options[:
|
|
34
|
+
if options[:to_hiera_path_strip].is_a?(String) || options[:from_hiera_path_strip].is_a?(String)
|
|
24
35
|
raise ArgumentError, '--hiera-path-strip and --no-hiera-path-strip are mutually exclusive'
|
|
25
36
|
end
|
|
26
|
-
options[:
|
|
37
|
+
options[:to_hiera_path_strip] = :none
|
|
38
|
+
options[:from_hiera_path_strip] = :none
|
|
27
39
|
end
|
|
28
40
|
end
|
|
29
41
|
end
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
# this instead executes the tasks serially, but provides the same API as the parallel tasks.
|
|
7
7
|
|
|
8
8
|
require 'stringio'
|
|
9
|
+
require_relative 'util'
|
|
9
10
|
|
|
10
11
|
module OctocatalogDiff
|
|
11
12
|
module Util
|
|
@@ -107,13 +108,14 @@ module OctocatalogDiff
|
|
|
107
108
|
# @return [Exception] First exception encountered by a child process; returns nil if no exceptions encountered.
|
|
108
109
|
def self.run_tasks_parallel(result, task_array, logger)
|
|
109
110
|
pidmap = {}
|
|
110
|
-
ipc_tempdir =
|
|
111
|
+
ipc_tempdir = OctocatalogDiff::Util::Util.temp_dir('ocd-ipc-')
|
|
111
112
|
|
|
112
113
|
# Child process forking
|
|
113
114
|
task_array.each_with_index do |task, index|
|
|
114
115
|
# simplecov doesn't see this because it's forked
|
|
115
116
|
# :nocov:
|
|
116
117
|
this_pid = fork do
|
|
118
|
+
ENV['OCTOCATALOG_DIFF_TEMPDIR'] ||= ipc_tempdir
|
|
117
119
|
task_result = execute_task(task, logger)
|
|
118
120
|
File.open(File.join(ipc_tempdir, "#{Process.pid}.dat"), 'w') { |f| f.write Marshal.dump(task_result) }
|
|
119
121
|
Kernel.exit! 0 # Kernel.exit! avoids at_exit from parents being triggered by children exiting
|
|
@@ -157,17 +159,6 @@ module OctocatalogDiff
|
|
|
157
159
|
# If the process doesn't exist, that's fine.
|
|
158
160
|
end
|
|
159
161
|
end
|
|
160
|
-
|
|
161
|
-
retries = 0
|
|
162
|
-
while File.directory?(ipc_tempdir) && retries < 10
|
|
163
|
-
retries += 1
|
|
164
|
-
begin
|
|
165
|
-
FileUtils.remove_entry_secure ipc_tempdir
|
|
166
|
-
rescue Errno::ENOTEMPTY, Errno::ENOENT # rubocop:disable Lint/HandleExceptions
|
|
167
|
-
# Errno::ENOTEMPTY will trigger a retry because the directory exists
|
|
168
|
-
# Errno::ENOENT will break the loop because the directory won't exist next time it's checked
|
|
169
|
-
end
|
|
170
|
-
end
|
|
171
162
|
end
|
|
172
163
|
|
|
173
164
|
# Utility method! Not intended to be called from outside this class.
|
|
@@ -6,6 +6,8 @@ require 'fileutils'
|
|
|
6
6
|
require 'open3'
|
|
7
7
|
require 'shellwords'
|
|
8
8
|
|
|
9
|
+
require_relative 'util'
|
|
10
|
+
|
|
9
11
|
module OctocatalogDiff
|
|
10
12
|
module Util
|
|
11
13
|
# This is a utility class to execute a built-in script.
|
|
@@ -91,14 +93,7 @@ module OctocatalogDiff
|
|
|
91
93
|
# @return [String] Path to tempfile containing script
|
|
92
94
|
def temp_script(script)
|
|
93
95
|
raise Errno::ENOENT, "Script '#{script}' not found" unless File.file?(script)
|
|
94
|
-
temp_dir =
|
|
95
|
-
at_exit do
|
|
96
|
-
begin
|
|
97
|
-
FileUtils.remove_entry_secure temp_dir
|
|
98
|
-
rescue Errno::ENOENT # rubocop:disable Lint/HandleExceptions
|
|
99
|
-
# OK if the directory doesn't exist since we're trying to remove it anyway
|
|
100
|
-
end
|
|
101
|
-
end
|
|
96
|
+
temp_dir = OctocatalogDiff::Util::Util.temp_dir('ocd-scriptrunner')
|
|
102
97
|
temp_file = File.join(temp_dir, File.basename(script))
|
|
103
98
|
File.open(temp_file, 'w') { |f| f.write(File.read(script)) }
|
|
104
99
|
FileUtils.chmod 0o755, temp_file
|
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
# Handy methods that are not tied to one particular class
|
|
4
4
|
|
|
5
|
+
require 'fileutils'
|
|
6
|
+
|
|
5
7
|
module OctocatalogDiff
|
|
6
8
|
module Util
|
|
7
9
|
# Helper class to construct catalogs, performing all necessary steps such as
|
|
@@ -43,6 +45,51 @@ module OctocatalogDiff
|
|
|
43
45
|
safe_dup(object)
|
|
44
46
|
end
|
|
45
47
|
end
|
|
48
|
+
|
|
49
|
+
# Utility Method!
|
|
50
|
+
# This creates a temporary directory. If the base directory is specified, then we
|
|
51
|
+
# do not remove the temporary directory at exit, because we assume that something
|
|
52
|
+
# else will remove the base directory.
|
|
53
|
+
#
|
|
54
|
+
# prefix - A String with the prefix for the temporary directory
|
|
55
|
+
# basedir - A String with the directory in which to make the tempdir
|
|
56
|
+
#
|
|
57
|
+
# Returns the full path to the temporary directory.
|
|
58
|
+
def self.temp_dir(prefix = 'ocd-', basedir = ENV['OCTOCATALOG_DIFF_TEMPDIR'])
|
|
59
|
+
# If the base directory is specified, make sure it exists, and then create the
|
|
60
|
+
# temporary directory within it.
|
|
61
|
+
if basedir
|
|
62
|
+
unless File.directory?(basedir)
|
|
63
|
+
raise Errno::ENOENT, "temp_dir: Base dir #{basedir.inspect} does not exist!"
|
|
64
|
+
end
|
|
65
|
+
return Dir.mktmpdir(prefix, basedir)
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# If the base directory was not specified, then create a temporary directory, and
|
|
69
|
+
# send the `at_exit` to clean it up at the conclusion.
|
|
70
|
+
the_dir = Dir.mktmpdir(prefix)
|
|
71
|
+
at_exit { remove_temp_dir(the_dir) }
|
|
72
|
+
the_dir
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
# Utility method!
|
|
76
|
+
# Remove a directory recursively that has been used as a temporary directory. This
|
|
77
|
+
# should be called within an `at_exit` handler, and is only intended to be called via the
|
|
78
|
+
# `temp_dir` method above.
|
|
79
|
+
#
|
|
80
|
+
# dir - A String with the directory to remove.
|
|
81
|
+
def self.remove_temp_dir(dir)
|
|
82
|
+
retries = 0
|
|
83
|
+
while File.directory?(dir) && retries < 10
|
|
84
|
+
retries += 1
|
|
85
|
+
begin
|
|
86
|
+
FileUtils.remove_entry_secure(dir)
|
|
87
|
+
rescue Errno::ENOTEMPTY, Errno::ENOENT # rubocop:disable Lint/HandleExceptions
|
|
88
|
+
# Errno::ENOTEMPTY will trigger a retry because the directory exists
|
|
89
|
+
# Errno::ENOENT will break the loop because the directory won't exist next time it's checked
|
|
90
|
+
end
|
|
91
|
+
end
|
|
92
|
+
end
|
|
46
93
|
end
|
|
47
94
|
end
|
|
48
95
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: octocatalog-diff
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.
|
|
4
|
+
version: 1.5.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- GitHub, Inc.
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: bin
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2017-10-
|
|
12
|
+
date: 2017-10-18 00:00:00.000000000 Z
|
|
13
13
|
dependencies:
|
|
14
14
|
- !ruby/object:Gem::Dependency
|
|
15
15
|
name: diffy
|
|
@@ -229,6 +229,7 @@ files:
|
|
|
229
229
|
- doc/advanced-catalog-validation.md
|
|
230
230
|
- doc/advanced-ci.md
|
|
231
231
|
- doc/advanced-dynamic-ignores.md
|
|
232
|
+
- doc/advanced-environment-variables.md
|
|
232
233
|
- doc/advanced-environments.md
|
|
233
234
|
- doc/advanced-filter.md
|
|
234
235
|
- doc/advanced-future-parser.md
|