octocatalog-diff 1.5.1 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +5 -5
- data/.version +1 -1
- data/README.md +4 -4
- data/doc/CHANGELOG.md +49 -0
- data/doc/advanced-filter.md +23 -0
- data/doc/advanced-ignores.md +10 -0
- data/doc/advanced-puppet-master.md +23 -5
- data/doc/configuration-puppetdb.md +11 -0
- data/doc/dev/api/v1/calls/catalog-diff.md +6 -2
- data/doc/dev/api/v1/objects/diff.md +3 -3
- data/doc/dev/releasing.md +1 -1
- data/doc/limitations.md +9 -9
- data/doc/optionsref.md +167 -11
- data/doc/requirements.md +6 -2
- data/lib/octocatalog-diff/catalog-diff/differ.rb +29 -4
- data/lib/octocatalog-diff/catalog-diff/filter.rb +2 -1
- data/lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb +29 -25
- data/lib/octocatalog-diff/catalog-diff/filter/single_item_array.rb +44 -0
- data/lib/octocatalog-diff/catalog-util/builddir.rb +3 -3
- data/lib/octocatalog-diff/catalog-util/command.rb +25 -3
- data/lib/octocatalog-diff/catalog-util/fileresources.rb +1 -1
- data/lib/octocatalog-diff/catalog.rb +22 -4
- data/lib/octocatalog-diff/catalog/computed.rb +2 -1
- data/lib/octocatalog-diff/catalog/puppetmaster.rb +43 -5
- data/lib/octocatalog-diff/cli.rb +36 -5
- data/lib/octocatalog-diff/cli/options.rb +39 -3
- data/lib/octocatalog-diff/cli/options/hostname.rb +13 -2
- data/lib/octocatalog-diff/cli/options/pe_enc_token_file.rb +1 -1
- data/lib/octocatalog-diff/cli/options/puppet_master_api_version.rb +2 -2
- data/lib/octocatalog-diff/cli/options/puppet_master_token.rb +20 -0
- data/lib/octocatalog-diff/cli/options/puppet_master_token_file.rb +35 -0
- data/lib/octocatalog-diff/cli/options/puppet_master_update_catalog.rb +20 -0
- data/lib/octocatalog-diff/cli/options/puppet_master_update_facts.rb +20 -0
- data/lib/octocatalog-diff/cli/options/puppetdb_package_inventory.rb +18 -0
- data/lib/octocatalog-diff/cli/options/puppetdb_token.rb +17 -0
- data/lib/octocatalog-diff/cli/options/puppetdb_token_file.rb +21 -0
- data/lib/octocatalog-diff/facts/puppetdb.rb +43 -2
- data/lib/octocatalog-diff/puppetdb.rb +5 -1
- data/lib/octocatalog-diff/util/parallel.rb +20 -16
- data/lib/octocatalog-diff/util/util.rb +2 -0
- data/scripts/env/env.sh +1 -1
- data/scripts/git-extract/git-extract.sh +1 -1
- data/scripts/puppet/puppet.sh +1 -1
- metadata +37 -30
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: b99c2e5da4209a2779ec55c745b43a274c0983f00bc5ec41452bf19636818f9a
|
4
|
+
data.tar.gz: 1ebb9bc86ff2ab9dab9e6529a2f862ddc15447c859d7e1e2120ef8e177220c58
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: b9c7f405b7d219817088a1f29126335270245d5d1f8084a07e26b50b817adfb5b6019ed92edac55250ce611253cc418fee9a53a0cf9f065b73e3b89be1523809
|
7
|
+
data.tar.gz: 0054faec251c287be47947f73314fbcd30d5d3b4af240488fe97665247e5af7100561b908027a84dd9994853036a05330d82ab8da12ab2519437c5ec9a5101bc
|
data/.version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
|
1
|
+
2.0.0
|
data/README.md
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
# octocatalog-diff
|
1
|
+
# octocatalog-diff ![CI](https://github.com/github/octocatalog-diff/workflows/CI/badge.svg)
|
2
2
|
|
3
3
|
#### Compile Puppet catalogs from 2 branches, versions, etc., and compare them <img src="/doc/images/octocatolog-diff-logo.png" align="right" height=126 width=240>
|
4
4
|
|
@@ -23,7 +23,7 @@ There are some [limitations](doc/limitations.md) to a catalog-based approach, me
|
|
23
23
|
`octocatalog-diff` is currently able to get catalogs by the following methods:
|
24
24
|
- Compile catalog via the command line with a Puppet agent on your machine (as GitHub uses the tool internally)
|
25
25
|
- Obtain catalog over the network from PuppetDB
|
26
|
-
- Obtain catalog over the network using the API to query a Puppet Master / PuppetServer (Puppet 3.x
|
26
|
+
- Obtain catalog over the network using the API to query a Puppet Master / PuppetServer (Puppet 3.x through 6.x supported)
|
27
27
|
- Read catalog from a JSON file
|
28
28
|
|
29
29
|
## Example
|
@@ -81,6 +81,6 @@ If you have a problem or suggestion, please [open an issue](https://github.com/g
|
|
81
81
|
|
82
82
|
It requires 3rd party ruby gems found [here](/vendor/cache). It also includes portions of other open source projects [here](/lib/octocatalog-diff/external/pson), [here](/spec/octocatalog-diff/fixtures/repos/default/modules/stdlib), [here](/spec/octocatalog-diff/support/httparty) and [here](/spec/octocatalog-diff/tests/external/pson). All 3rd party code and required gems are licensed either as MIT or Apache 2.0.
|
83
83
|
|
84
|
-
## Authors
|
84
|
+
## Authors / Owners
|
85
85
|
|
86
|
-
`octocatalog-diff` was designed and authored by [Kevin Paulisse](https://github.com/kpaulisse)
|
86
|
+
`octocatalog-diff` was originally designed and authored by [Kevin Paulisse](https://github.com/kpaulisse). It is now maintained by the Site Reliability Engineering team at GitHub.
|
data/doc/CHANGELOG.md
CHANGED
@@ -8,6 +8,55 @@
|
|
8
8
|
</tr>
|
9
9
|
</thead><tbody>
|
10
10
|
|
11
|
+
<tr valign=top>
|
12
|
+
<td>2.0.0</td>
|
13
|
+
<td>2020-01-12</td>
|
14
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/226">#226</a>: (Enhancement) Add Puppet 6 support</li>
|
15
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/228">#228</a>: (Enhancement) Add Puppetserver catalog v4 API support</li>
|
16
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/229">#229</a>: (Enhancement) Add support for PE package inventory facts</li>
|
17
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/230">#230</a>: (Enhancement) Add set quality comparison to ignore filter</li>
|
18
|
+
</td>
|
19
|
+
</tr>
|
20
|
+
|
21
|
+
<tr valign=top>
|
22
|
+
<td>1.6.0</td>
|
23
|
+
<td>2019-10-31</td>
|
24
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/216">#216</a>: (Enhancement) Hide sensitive parameters</li>
|
25
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/204">#204</a>: (Enhancement) Add glob support for modulepath</li>
|
26
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/206">#206</a>: (Bug Fix) Fix multi-node list with parallel mode</li>
|
27
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/215">#215</a>: (Bug Fix) Add Support for Hashdiff 1.0.0</li>
|
28
|
+
</td>
|
29
|
+
</tr>
|
30
|
+
|
31
|
+
<tr valign=top>
|
32
|
+
<td>1.5.4</td>
|
33
|
+
<td>2018-12-11</td>
|
34
|
+
<td>
|
35
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/190">#190</a>: (Enhancement) Additional filtered out cases for compilation directory</li>
|
36
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/195">#195</a>: (Enhancement) Parallel catalog-diff when multiple hostnames are passed</li>
|
37
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/198">#198</a>: (Bug Fix) Portability fixes</li>
|
38
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/200">#200</a>: (Bug Fix) Support name parameter when validating references</li>
|
39
|
+
</td>
|
40
|
+
</tr>
|
41
|
+
|
42
|
+
<tr valign=top>
|
43
|
+
<td>1.5.3</td>
|
44
|
+
<td>2018-03-05</td>
|
45
|
+
<td>
|
46
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/176">#176</a>: (Enhancement) Normalize file resource titles in reference checks</li>
|
47
|
+
</td>
|
48
|
+
</tr>
|
49
|
+
|
50
|
+
<tr valign=top>
|
51
|
+
<td>1.5.2</td>
|
52
|
+
<td>2017-12-19</td>
|
53
|
+
<td>
|
54
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/169">#169</a>: (Enhancement) Puppet Enterprise RBAC token to authenticate to PuppetDB</li>
|
55
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/170">#170</a>: (Enhancement) Filter to treat an object the same as a single array containing that object</li>
|
56
|
+
<li><a href="https://github.com/github/octocatalog-diff/pull/165">#165</a>: (Bug Fix) Override of fact file via CLI now has precedence over value set in configuration file</li>
|
57
|
+
</td>
|
58
|
+
</tr>
|
59
|
+
|
11
60
|
<tr valign=top>
|
12
61
|
<td>1.5.1</td>
|
13
62
|
<td>2017-11-16</td>
|
data/doc/advanced-filter.md
CHANGED
@@ -11,6 +11,7 @@ Here is the list of available filters and an explanation of each:
|
|
11
11
|
|
12
12
|
- [Absent File](/doc/advanced-filter.md#absent-file) - Ignore parameter changes of a file that is declared to be absent
|
13
13
|
- [JSON](/doc/advanced-filter.md#json) - Ignore whitespace differences if JSON parses to the same object
|
14
|
+
- [SingleItemArray](/doc/advanced-filter.md#SingleItemArray) - Ignore differences between object and array containing only that object
|
14
15
|
- [YAML](/doc/advanced-filter.md#yaml) - Ignore whitespace/comment differences if YAML parses to the same object
|
15
16
|
|
16
17
|
## Absent File
|
@@ -84,6 +85,28 @@ If a file resource has extension `.json` and a difference in its content is obse
|
|
84
85
|
|
85
86
|
This allows you to ignore changes in whitespace, comments, etc., that are not meaningful to a machine parsing the file. Note that changes to files may still trigger Puppet to restart services even though these changes are not displayed in the octocatalog-diff output.
|
86
87
|
|
88
|
+
## Single Item Array
|
89
|
+
|
90
|
+
#### Usage
|
91
|
+
|
92
|
+
```
|
93
|
+
--filters SingleItemArray
|
94
|
+
```
|
95
|
+
|
96
|
+
#### Description
|
97
|
+
|
98
|
+
When enabling the future parser or upgrading between certain versions of Puppet, the internal structure of the catalog for certain parameters can change as shown in the following example:
|
99
|
+
|
100
|
+
```
|
101
|
+
Old: { "notify": "Service[foo]" }
|
102
|
+
New: { "notify": [ "Service[foo]" ] }
|
103
|
+
```
|
104
|
+
|
105
|
+
This filter will suppress differences for the value of a parameter when:
|
106
|
+
|
107
|
+
- The value in one catalog is an object, AND
|
108
|
+
- The value in the other catalog is an array containing *only* that same object
|
109
|
+
|
87
110
|
## YAML
|
88
111
|
|
89
112
|
#### Usage
|
data/doc/advanced-ignores.md
CHANGED
@@ -222,3 +222,13 @@ File[/tmp/foo] =>
|
|
222
222
|
In this case, the very important line was removed from the catalog, and you want to know about this. Ignoring `File[/tmp/foo]::parameters::content` would have suppressed this (because all changes to that attribute are ignored). Also ignoring `File[/tmp/foo]::parameters::content=~>This is the line in the new catalog that I do not care about$` would have also suppressed this (because the regular expression was matched for *one* of the lines). However, the two examples with `=&>` in this section would *not* have suppressed this change, because it is no longer the case that *all* changes in the file matched the regular expression.
|
223
223
|
|
224
224
|
:warning: All lines are stripped of leading and trailing spaces before the regular expression match is tried. This stripping of whitespace is done *only* for this comparison stage, and does not affect the display of any results.
|
225
|
+
|
226
|
+
#### Ignoring attributes which have identical elements but in arbitrary order
|
227
|
+
|
228
|
+
You can ignore attributes where both the values in both the old and new catalogs are arrays and the arrays
|
229
|
+
contain identical elements but in arbitrary order. Basically, you can ignore a parameter where the values
|
230
|
+
have set equality.
|
231
|
+
|
232
|
+
To ignore any parameters named `foo` with values having set equality, you would use:
|
233
|
+
|
234
|
+
--ignore 'My::Custom::Resource[*]::parameters::foo=s>='
|
@@ -8,7 +8,11 @@ Please note the following caveats:
|
|
8
8
|
|
9
9
|
0. You will need to deploy your Puppet code to an environment on your Puppet Master prior to running `octocatalog-diff` for that environment. `octocatalog-diff` does not deploy code for you.
|
10
10
|
|
11
|
-
0. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below.
|
11
|
+
0. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below. If you are using Puppet Enterprise and use
|
12
|
+
the Puppet Master v4 API you may also use a Puppet Enterprise RBAC token. The user owning the token will need the "Puppet Server Compile catalogs for remote nodes" permission.
|
13
|
+
See the [PE RBAC Token Authorization](#pe-rbac-token-authorization) section below.
|
14
|
+
|
15
|
+
0. If you are using the v2 or v3 PuppetServer APIs with Octocatalog-Diff to compile catalogs, then those catalogs and facts will be automatically stored in PuppetDB. However, when using the v4 PuppetServer API with Octocatalog-Diff, facts and catalogs are *not* automatically stored in PuppetDB - persistence is optional and may be enabled with the appropriate Octocatalog-Diff CLI flag. If your environment depends on the accuracy of exported resources or facts in PuppetDB, you may wish to upgrade and use the V4 API, to avoid unintentional side-effects.
|
12
16
|
|
13
17
|
## Command line options
|
14
18
|
|
@@ -18,11 +22,15 @@ The following command line options are used to retrieve a catalog from a Puppet
|
|
18
22
|
| ------ | ----------- |
|
19
23
|
| `-f ENVIRONMENT` | Environment name to use for the "from" catalog |
|
20
24
|
| `-t ENVIRONMENT` | Environment name to use for the "to" catalog |
|
21
|
-
| `--puppet-master HOSTNAME:PORT | The hostname and port number of the Puppet Master. (By default the port used by Puppet Master is 8140.) |
|
22
|
-
| `--puppet-master-api-version VERSION | The API version used by the Puppet Master. API versions 2 and
|
25
|
+
| `--puppet-master HOSTNAME:PORT` | The hostname and port number of the Puppet Master. (By default the port used by Puppet Master is 8140.) |
|
26
|
+
| `--puppet-master-api-version VERSION` | The API version used by the Puppet Master. API versions 2, 3,and 4 are supported. Puppet Master 3.x uses API version 2, and the PuppetServer for Puppet 4.x uses API version 3. PuppetServer 6.3.0 introduced the optional use of the v4 API but still fully supports the v3 API. By default, API version 3 is used, so you only need to set this option if you are using Puppet Master 3.x or wish to use the newer v4 API with PuppetServer 6. |
|
23
27
|
| `--puppet-master-ssl-ca PATH` | Path to the CA certificate (public portion of certificate only) for your Puppet Master. This file will be on your Puppet Master and all Puppet agents. You can find it by running `puppet config print cacert` on any Puppet-managed host. |
|
24
|
-
| `--puppet-master-ssl-client-cert PATH` | Path to the client certificate. Please see the section below on certificate authentication. |
|
25
|
-
| `--puppet-master-ssl-client-key PATH` | Path to the client private key. Please see the section below on certificate authentication. |
|
28
|
+
| `--puppet-master-ssl-client-cert PATH` | Path to the client certificate. Please see the section below on certificate authentication. This can be omitted if using PE RBAC token based auth with the v4 API. |
|
29
|
+
| `--puppet-master-ssl-client-key PATH` | Path to the client private key. Please see the section below on certificate authentication. This can be omitted if using PE RBAC token based auth with the v4 API. |
|
30
|
+
| `--puppet-master-token STRING` | A PE RBAC token used to authenticate a v4 catalog compile, in lieu of using certificate authentication. Please see the section below on token authentication. |
|
31
|
+
| `--puppet-master-token-file PATH` | A path to a file containing a PE RBAC token used to authenticate a v4 catalog compile, in lieu of using certificate authentication. If this and `--puppet-master-token` are both specified, `--puppet-master-token` will be used instead. Please see the section below on token authentication. |
|
32
|
+
| `--puppet-master-update-catalog` | When using the v4 API, instruct the PuppetServer to update the catalog generated from the compile in its PuppetDB instance. When using v2 and v3 APIs the catalog is always updated and this option is ignored. |
|
33
|
+
| `--puppet-master-update-facts` | When using the v4 API, instruct the PuppetServer to update the facts used during the compile in its PuppetDB instance. When using v2 and v3 APIs the facts are always updated and this option is ignored. |
|
26
34
|
|
27
35
|
If you wish to use a different Puppet Master to compile the "to" and "from" catalogs, you may prefix any of the `--puppet-master...` options with `to` or `from`. For example, perhaps you are testing an upgrade from Puppet 3.x to 4.x. You could use:
|
28
36
|
|
@@ -48,3 +56,13 @@ allow $1
|
|
48
56
|
```
|
49
57
|
|
50
58
|
Please follow the instructions for the version of Puppet Master, PuppetServer, or Puppet Enterprise that you are using in order to generate and authorize the certificates.
|
59
|
+
|
60
|
+
## PE RBAC Token authorization
|
61
|
+
|
62
|
+
In newer versions of Puppet Enterprise you can authenticate using a valid PE RBAC token with appropriate permissions as long as it is authorized in the PuppetServer `auth.conf` file.
|
63
|
+
|
64
|
+
By default this permission is enabled and controlled by the `puppet_enterprise::master::tk_authz::allow_rbac_catalog_compile` Hiera setting.
|
65
|
+
|
66
|
+
The user the token was issued to must have the `puppetserver:compile_catalogs:*` permission.
|
67
|
+
|
68
|
+
Note: A Puppet catalog may contain unencrypted secrets, even ones marked as `Sensitive`. In order to perform its job, Octocatalog-Diff needs access to the catalog. By granting a user the above RBAC permission you are granting them the ability to retrieve and view the complete catalog resulting from a compile, including any included secrets.
|
@@ -36,6 +36,7 @@ The following settings can be used in a [configuration file](/doc/configuration.
|
|
36
36
|
| `settings[:puppetdb_ssl_client_key]` | TEXT of the private key of the client SSL keypair used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
|
37
37
|
| `settings[:puppetdb_ssl_client_pem]` | Concatenation of the text of `puppetdb_ssl_client_key` and `puppetdb_ssl_client_cert` as previously described. This is a good alternative if your certificate chain is complex and it's easier just to put everything in a single place. Note: this option is second in precedence; if `settings[:puppetdb_ssl_client_cert]` and `settings[:puppetdb_ssl_client_key]` are both set, this will be ignored. |
|
38
38
|
| `settings[:puppetdb_ssl_client_password]` | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required and should be left undefined. |
|
39
|
+
| `settings[:puppetdb_token]` | TEXT containing the PE RBAC token used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
|
39
40
|
|
40
41
|
## Supplying necessary information via the command line
|
41
42
|
|
@@ -48,6 +49,8 @@ The following arguments can be used on the command line.
|
|
48
49
|
| --puppetdb-ssl-client-cert FILENAME | Path to the certificate of the client SSL keypair. |
|
49
50
|
| --puppetdb-ssl-client-key FILENAME | Path to the private key of the client SSL keypair. |
|
50
51
|
| --puppetdb-ssl-client-password PASSWORD_STRING | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required. |
|
52
|
+
| --puppetdb-token STRING | String containing the PE RBAC token used to authenticate to PuppetDB. |
|
53
|
+
| --puppetdb-token-file FILENAME | Path to the PE RBAC token file used to authenticate to PuppetDB. |
|
51
54
|
|
52
55
|
## Supplying necessary information via the environment
|
53
56
|
|
@@ -64,3 +67,11 @@ SSL support is enabled via any of the `--puppetdb-ssl-...` command line options
|
|
64
67
|
- The CA certificate should be the public certificate of the CA that signed your PuppetDB server's certificate. This file can be found in `/etc/puppetlabs/puppetdb/ssl/ca.pem` on a PuppetDB server. Since this is a public certificate, it is safe (and recommended) to distribute this file to any clients that may connect to this PuppetDB instance.
|
65
68
|
|
66
69
|
- The client keypair (key, certificate, and optionally password) should be generated individually for each client. You should NOT copy SSL keypairs from your PuppetDB server (or anywhere else) to your clients. If you are using `octocatalog-diff` on a system that is managed by Puppet, you may wish to use the same SSL credentials that the system uses to authenticate to Puppet. With recent versions of the Puppet agent, those certificates are found in `/etc/puppetlabs/puppet/ssl`.
|
70
|
+
|
71
|
+
# Puppet Enterprise PuppetDB Package Inventory
|
72
|
+
|
73
|
+
Puppet Enterprise customers have an optional package inventory feature which can be enabled. When this feature is enabled an inventory of all system packages
|
74
|
+
is performed and uploaded as a fact which is then processed and stored independently of the normal Facter data in PuppetDB. Most environments won't need
|
75
|
+
to replicate the package inventory facts for testing with Octocatalog-Diff but if you want the package inventory data (if present) to be included
|
76
|
+
in the facts retrieved from PuppetDB by Octocatalog-Diff you should specify the `--puppetdb-package-inventory` flag. When enabled, this flag will instruct
|
77
|
+
Octocatalog-Diff to retrieve any package data found for a node from PuppetDB and include it in the facts used during the Octocatalog-Diff compile.
|
@@ -167,12 +167,16 @@ In this case, "owner", "notify", and "content" are nested under "parameters". In
|
|
167
167
|
|
168
168
|
```
|
169
169
|
# Ignore all changes to the `owner` attribute of a file.
|
170
|
-
[ { type: Regexp.new('\AFile\z'), attr: Regexp.new("
|
170
|
+
[ { type: Regexp.new('\AFile\z'), attr: Regexp.new("\\Aparameters\fowner\\z" } ]
|
171
171
|
|
172
172
|
# Ignore changes to `owner` or `group` for a file or an exec.
|
173
|
-
[ { type: Regexp.new('\A(File|Exec)\z'), attr: Regexp.new("
|
173
|
+
[ { type: Regexp.new('\A(File|Exec)\z'), attr: Regexp.new("\\Aparameters\f(owner|group)\\z" } ]
|
174
174
|
```
|
175
175
|
|
176
|
+
When using regular expressions, `\f` (form feed character) is used to separate the structure (e.g. `parameters\fowner` refers to the `parameters` hash, `owner` key).
|
177
|
+
|
178
|
+
:bulb: Note that `\A` in Ruby matches the beginning of the string and `\z` matches the end, but these are not actual characters. Therefore, if you are using `\A` or `\z` in double quotes (`"`), be sure to heed the examples above and write your expression like: `Regexp.new("\\Aparameters\fowner\\z")`.
|
179
|
+
|
176
180
|
#### `:validate_references` (Array<String>, Optional)
|
177
181
|
|
178
182
|
Invoke the [catalog validation](/doc/advanced-catalog-validation.md) feature to ensure resources targeted by `before`, `notify`, `require`, and/or `subscribe` exist in the catalog. If this parameter is not defined, no reference validation occurs.
|
@@ -94,9 +94,9 @@ Returns the value of the resource from the new catalog.
|
|
94
94
|
}
|
95
95
|
}
|
96
96
|
|
97
|
-
# Demonstrates structure and
|
97
|
+
# Demonstrates structure and new_value
|
98
98
|
diff.structure #=> ['parameters', 'content']
|
99
|
-
diff.
|
99
|
+
diff.new_value #=> 'This is the NEW FILE!!!!!'
|
100
100
|
```
|
101
101
|
|
102
102
|
#### `#old_file` (String)
|
@@ -107,7 +107,7 @@ Note that this is a pass-through of information provided in the Puppet catalog,
|
|
107
107
|
|
108
108
|
Note also that if the diff represents addition of a resource, this will return `nil`, because the resource does not exist in the old catalog.
|
109
109
|
|
110
|
-
#### `#
|
110
|
+
#### `#old_line` (String)
|
111
111
|
|
112
112
|
Returns the line number within the Puppet manifest giving rise to the resource as it exists in the old catalog. (See `#old_file` for the filename of the Puppet manifest.)
|
113
113
|
|
data/doc/dev/releasing.md
CHANGED
@@ -9,7 +9,7 @@ The project maintainers are responsible for bumping the version number, regenera
|
|
9
9
|
To test the new version of `octocatalog-diff` in the GitHub Puppet repository, check out `github/puppet` and:
|
10
10
|
|
11
11
|
- Start a new branch based off master
|
12
|
-
- Run `script/update-octocatalog-diff
|
12
|
+
- Run `script/update-octocatalog-diff <ocd_branch_name>`
|
13
13
|
- Confirm and commit the result
|
14
14
|
- Make sure all CI jobs pass
|
15
15
|
- Run the `puppet-catalog-diff` CI job and make sure it passes and shows expected results
|
data/doc/limitations.md
CHANGED
@@ -2,23 +2,23 @@
|
|
2
2
|
|
3
3
|
Testing of Puppet catalogs is faster than running the agent, but you need to be careful of the following limitations:
|
4
4
|
|
5
|
-
|
5
|
+
1. Facts are not taken from a live agent run
|
6
6
|
|
7
|
-
|
7
|
+
octocatalog-diff by default uses the facts reported from a node's more recent Puppet run. If you have made changes to custom facts, catalog testing will **NOT** be an adequate test of whether your custom facts worked. (You can still use octocatalog-diff to help predict changes to nodes based on changes to facts, by overriding facts on the command line.)
|
8
8
|
|
9
|
-
|
9
|
+
1. Agents handle depenency ordering and implementation details
|
10
10
|
|
11
|
-
|
11
|
+
The catalog defines the state of the system, but it's up to the agent to determine how to bring the system to a point that matches the catalog. The agent is responsible for order of operations and actually making the change.
|
12
12
|
|
13
|
-
|
13
|
+
Two specific situations that catalog testing does **NOT** detect are:
|
14
14
|
|
15
|
-
|
15
|
+
- Dependency loops (e.g., you have made A require B, B require C, and C require A).
|
16
16
|
|
17
|
-
|
17
|
+
- Operations not supported by the provider. For example, assume that in your current Puppet manifests, you set the size of a file system to 100 GB. You change this in your new branch to 50 GB. octocatalog-diff will dutifully report this change to you. However, the agent will fail to make the change, because it is not possible to shrink a file system from 100 GB to 50 GB.
|
18
18
|
|
19
|
-
|
19
|
+
1. Changes in underlying providers may not be noticed
|
20
20
|
|
21
|
-
|
21
|
+
Consider that you are using a Puppet module that creates a file system. The current implementation of that module checks to see if *any* file system is present on the device, and creates a new file system there if no file system was present. You upgrade the module, and the new version checks to see if *the specified* file system is present on the device, and reformats the device with the specified file system (regardless of whether there was no file system or if there was an existing file system of a different type). There would be no catalog changes (hence octocatalog-diff would report nothing) because the catalog simply instructs the agent to create a file system of the specified type at the defined location. However, the actual implementation of those instructions has changed dramatically.
|
22
22
|
|
23
23
|
In general catalog testing is great for:
|
24
24
|
|
data/doc/optionsref.md
CHANGED
@@ -9,7 +9,8 @@
|
|
9
9
|
|
10
10
|
```
|
11
11
|
Usage: octocatalog-diff [command line options]
|
12
|
-
-n,
|
12
|
+
-n HOSTNAME1[,HOSTNAME2[,...]], Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames
|
13
|
+
--hostname
|
13
14
|
--basedir DIRNAME Use an alternate base directory (git checkout of puppet repository)
|
14
15
|
-f, --from FROM_BRANCH Branch you are coming from
|
15
16
|
-t, --to TO_BRANCH Branch you are going to
|
@@ -36,6 +37,8 @@ Usage: octocatalog-diff [command line options]
|
|
36
37
|
--fact-file STRING Override fact globally
|
37
38
|
--to-fact-file STRING Override fact for the to branch
|
38
39
|
--from-fact-file STRING Override fact for the from branch
|
40
|
+
--[no-]puppetdb-package-inventory
|
41
|
+
Include Puppet Enterprise package inventory data, if found
|
39
42
|
--save-catalog STRING Save intermediate catalogs into files globally
|
40
43
|
--to-save-catalog STRING Save intermediate catalogs into files for the to branch
|
41
44
|
--from-save-catalog STRING Save intermediate catalogs into files for the from branch
|
@@ -86,7 +89,20 @@ Usage: octocatalog-diff [command line options]
|
|
86
89
|
--puppet-binary STRING Full path to puppet binary globally
|
87
90
|
--to-puppet-binary STRING Full path to puppet binary for the to branch
|
88
91
|
--from-puppet-binary STRING Full path to puppet binary for the from branch
|
92
|
+
--puppet-master-token-file STRING
|
93
|
+
File containing PE RBAC token to authenticate to the Puppetserver API v4 globally
|
94
|
+
--to-puppet-master-token-file STRING
|
95
|
+
File containing PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
|
96
|
+
--from-puppet-master-token-file STRING
|
97
|
+
File containing PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
|
89
98
|
--facts-terminus STRING Facts terminus: one of yaml, facter
|
99
|
+
--puppet-master-token STRING PE RBAC token to authenticate to the Puppetserver API v4 globally
|
100
|
+
--to-puppet-master-token STRING
|
101
|
+
PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
|
102
|
+
--from-puppet-master-token STRING
|
103
|
+
PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
|
104
|
+
--puppetdb-token TOKEN Token to access the PuppetDB API
|
105
|
+
--puppetdb-token-file PATH Path containing token for PuppetDB API, relative or absolute
|
90
106
|
--puppetdb-url URL PuppetDB base URL
|
91
107
|
--puppetdb-ssl-ca FILENAME CA certificate that signed the PuppetDB certificate
|
92
108
|
--puppetdb-ssl-client-cert FILENAME
|
@@ -108,11 +124,23 @@ Usage: octocatalog-diff [command line options]
|
|
108
124
|
--to-puppet-master STRING Hostname or Hostname:PortNumber for Puppet Master for the to branch
|
109
125
|
--from-puppet-master STRING Hostname or Hostname:PortNumber for Puppet Master for the from branch
|
110
126
|
--puppet-master-api-version STRING
|
111
|
-
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) globally
|
127
|
+
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) globally
|
112
128
|
--to-puppet-master-api-version STRING
|
113
|
-
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the to branch
|
129
|
+
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the to branch
|
114
130
|
--from-puppet-master-api-version STRING
|
115
|
-
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the from branch
|
131
|
+
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the from branch
|
132
|
+
--[no-]puppet-master-update-catalog
|
133
|
+
Update catalog in PuppetDB when using Puppetmaster API version 4 globally
|
134
|
+
--[no-]to-puppet-master-update-catalog
|
135
|
+
Update catalog in PuppetDB when using Puppetmaster API version 4 for the to branch
|
136
|
+
--[no-]from-puppet-master-update-catalog
|
137
|
+
Update catalog in PuppetDB when using Puppetmaster API version 4 for the from branch
|
138
|
+
--[no-]puppet-master-update-facts
|
139
|
+
Update facts in PuppetDB when using Puppetmaster API version 4 globally
|
140
|
+
--[no-]to-puppet-master-update-facts
|
141
|
+
Update facts in PuppetDB when using Puppetmaster API version 4 for the to branch
|
142
|
+
--[no-]from-puppet-master-update-facts
|
143
|
+
Update facts in PuppetDB when using Puppetmaster API version 4 for the from branch
|
116
144
|
--puppet-master-ssl-ca STRING
|
117
145
|
Full path to CA certificate that signed the Puppet Master certificate globally
|
118
146
|
--to-puppet-master-ssl-ca STRING
|
@@ -716,7 +744,7 @@ Puppet control repo template, the value of this should be 'hieradata', which is
|
|
716
744
|
<pre><code>--from-puppet-master-api-version STRING</code></pre>
|
717
745
|
</td>
|
718
746
|
<td valign=top>
|
719
|
-
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the from branch
|
747
|
+
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the from branch
|
720
748
|
</td>
|
721
749
|
<td valign=top>
|
722
750
|
Specify the API version to use for the Puppet Master. This makes it possible to authenticate to a
|
@@ -778,6 +806,32 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
|
|
778
806
|
</td>
|
779
807
|
</tr>
|
780
808
|
|
809
|
+
<tr>
|
810
|
+
<td valign=top>
|
811
|
+
<pre><code>--from-puppet-master-token STRING</code></pre>
|
812
|
+
</td>
|
813
|
+
<td valign=top>
|
814
|
+
PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
|
815
|
+
</td>
|
816
|
+
<td valign=top>
|
817
|
+
Specify a PE RBAC token used to authenticate to Puppetserver for v4
|
818
|
+
catalog API calls. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token.rb">puppet_master_token.rb</a>)
|
819
|
+
</td>
|
820
|
+
</tr>
|
821
|
+
|
822
|
+
<tr>
|
823
|
+
<td valign=top>
|
824
|
+
<pre><code>--from-puppet-master-token-file STRING</code></pre>
|
825
|
+
</td>
|
826
|
+
<td valign=top>
|
827
|
+
File containing PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
|
828
|
+
</td>
|
829
|
+
<td valign=top>
|
830
|
+
Specify a path to a file containing a PE RBAC token used to authenticate to the
|
831
|
+
Puppetserver for a v4 catalog API call. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token_file.rb">puppet_master_token_file.rb</a>)
|
832
|
+
</td>
|
833
|
+
</tr>
|
834
|
+
|
781
835
|
<tr>
|
782
836
|
<td valign=top>
|
783
837
|
<pre><code>--from-puppetdb
|
@@ -854,14 +908,17 @@ Puppet control repo template, the value of this should be 'hieradata', which is
|
|
854
908
|
|
855
909
|
<tr>
|
856
910
|
<td valign=top>
|
857
|
-
<pre><code>-n
|
858
|
-
--hostname
|
911
|
+
<pre><code>-n HOSTNAME1[,HOSTNAME2[,...]]
|
912
|
+
--hostname HOSTNAME1[,HOSTNAME2[,...]]</code></pre>
|
859
913
|
</td>
|
860
914
|
<td valign=top>
|
861
|
-
Use PuppetDB facts from last run of hostname
|
915
|
+
Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames
|
862
916
|
</td>
|
863
917
|
<td valign=top>
|
864
|
-
Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display.
|
918
|
+
Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display.
|
919
|
+
This option can recieve a single hostname, or a comma separated list of
|
920
|
+
multiple hostnames, which are split into an Array. Multiple hostnames do not
|
921
|
+
work with the `catalog-only` or `bootstrap-then-exit` options. (<a href="../lib/octocatalog-diff/cli/options/hostname.rb">hostname.rb</a>)
|
865
922
|
</td>
|
866
923
|
</tr>
|
867
924
|
|
@@ -1233,7 +1290,7 @@ to work correctly. (<a href="../lib/octocatalog-diff/cli/options/preserve_enviro
|
|
1233
1290
|
<pre><code>--puppet-master-api-version STRING</code></pre>
|
1234
1291
|
</td>
|
1235
1292
|
<td valign=top>
|
1236
|
-
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) globally
|
1293
|
+
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) globally
|
1237
1294
|
</td>
|
1238
1295
|
<td valign=top>
|
1239
1296
|
Specify the API version to use for the Puppet Master. This makes it possible to authenticate to a
|
@@ -1295,6 +1352,32 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
|
|
1295
1352
|
</td>
|
1296
1353
|
</tr>
|
1297
1354
|
|
1355
|
+
<tr>
|
1356
|
+
<td valign=top>
|
1357
|
+
<pre><code>--puppet-master-token STRING</code></pre>
|
1358
|
+
</td>
|
1359
|
+
<td valign=top>
|
1360
|
+
PE RBAC token to authenticate to the Puppetserver API v4 globally
|
1361
|
+
</td>
|
1362
|
+
<td valign=top>
|
1363
|
+
Specify a PE RBAC token used to authenticate to Puppetserver for v4
|
1364
|
+
catalog API calls. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token.rb">puppet_master_token.rb</a>)
|
1365
|
+
</td>
|
1366
|
+
</tr>
|
1367
|
+
|
1368
|
+
<tr>
|
1369
|
+
<td valign=top>
|
1370
|
+
<pre><code>--puppet-master-token-file STRING</code></pre>
|
1371
|
+
</td>
|
1372
|
+
<td valign=top>
|
1373
|
+
File containing PE RBAC token to authenticate to the Puppetserver API v4 globally
|
1374
|
+
</td>
|
1375
|
+
<td valign=top>
|
1376
|
+
Specify a path to a file containing a PE RBAC token used to authenticate to the
|
1377
|
+
Puppetserver for a v4 catalog API call. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token_file.rb">puppet_master_token_file.rb</a>)
|
1378
|
+
</td>
|
1379
|
+
</tr>
|
1380
|
+
|
1298
1381
|
<tr>
|
1299
1382
|
<td valign=top>
|
1300
1383
|
<pre><code>--puppetdb-api-version N</code></pre>
|
@@ -1308,6 +1391,23 @@ the default. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_api_version.
|
|
1308
1391
|
</td>
|
1309
1392
|
</tr>
|
1310
1393
|
|
1394
|
+
<tr>
|
1395
|
+
<td valign=top>
|
1396
|
+
<pre><code>--puppetdb-package-inventory
|
1397
|
+
--no-puppetdb-package-inventory </code></pre>
|
1398
|
+
</td>
|
1399
|
+
<td valign=top>
|
1400
|
+
Include Puppet Enterprise package inventory data, if found
|
1401
|
+
</td>
|
1402
|
+
<td valign=top>
|
1403
|
+
When pulling facts from PuppetDB in a Puppet Enterprise environment, also include
|
1404
|
+
the Puppet Enterprise Package Inventory data in the fact results, if available.
|
1405
|
+
Generally you should not need to specify this, but including the package inventory
|
1406
|
+
data will produce a more accurate set of input facts for environments using
|
1407
|
+
package inventory. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_package_inventory.rb">puppetdb_package_inventory.rb</a>)
|
1408
|
+
</td>
|
1409
|
+
</tr>
|
1410
|
+
|
1311
1411
|
<tr>
|
1312
1412
|
<td valign=top>
|
1313
1413
|
<pre><code>--puppetdb-ssl-ca FILENAME</code></pre>
|
@@ -1374,6 +1474,36 @@ the text of the password won't appear in the process list. (<a href="../lib/octo
|
|
1374
1474
|
</td>
|
1375
1475
|
</tr>
|
1376
1476
|
|
1477
|
+
<tr>
|
1478
|
+
<td valign=top>
|
1479
|
+
<pre><code>--puppetdb-token TOKEN</code></pre>
|
1480
|
+
</td>
|
1481
|
+
<td valign=top>
|
1482
|
+
Token to access the PuppetDB API
|
1483
|
+
</td>
|
1484
|
+
<td valign=top>
|
1485
|
+
Specify the PE RBAC token to access the PuppetDB API. Refer to
|
1486
|
+
https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
|
1487
|
+
for details on generating and obtaining a token. Use this option to specify the text
|
1488
|
+
of the token. (Use --puppetdb-token-file to read the content of the token from a file.) (<a href="../lib/octocatalog-diff/cli/options/puppetdb_token.rb">puppetdb_token.rb</a>)
|
1489
|
+
</td>
|
1490
|
+
</tr>
|
1491
|
+
|
1492
|
+
<tr>
|
1493
|
+
<td valign=top>
|
1494
|
+
<pre><code>--puppetdb-token-file PATH</code></pre>
|
1495
|
+
</td>
|
1496
|
+
<td valign=top>
|
1497
|
+
Path containing token for PuppetDB API, relative or absolute
|
1498
|
+
</td>
|
1499
|
+
<td valign=top>
|
1500
|
+
Specify the PE RBAC token to access the PuppetDB API. Refer to
|
1501
|
+
https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
|
1502
|
+
for details on generating and obtaining a token. Use this option to specify the text
|
1503
|
+
in a file, to read the content of the token from the file. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_token_file.rb">puppetdb_token_file.rb</a>)
|
1504
|
+
</td>
|
1505
|
+
</tr>
|
1506
|
+
|
1377
1507
|
<tr>
|
1378
1508
|
<td valign=top>
|
1379
1509
|
<pre><code>--puppetdb-url URL</code></pre>
|
@@ -1651,7 +1781,7 @@ Puppet control repo template, the value of this should be 'hieradata', which is
|
|
1651
1781
|
<pre><code>--to-puppet-master-api-version STRING</code></pre>
|
1652
1782
|
</td>
|
1653
1783
|
<td valign=top>
|
1654
|
-
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the to branch
|
1784
|
+
Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the to branch
|
1655
1785
|
</td>
|
1656
1786
|
<td valign=top>
|
1657
1787
|
Specify the API version to use for the Puppet Master. This makes it possible to authenticate to a
|
@@ -1713,6 +1843,32 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
|
|
1713
1843
|
</td>
|
1714
1844
|
</tr>
|
1715
1845
|
|
1846
|
+
<tr>
|
1847
|
+
<td valign=top>
|
1848
|
+
<pre><code>--to-puppet-master-token STRING</code></pre>
|
1849
|
+
</td>
|
1850
|
+
<td valign=top>
|
1851
|
+
PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
|
1852
|
+
</td>
|
1853
|
+
<td valign=top>
|
1854
|
+
Specify a PE RBAC token used to authenticate to Puppetserver for v4
|
1855
|
+
catalog API calls. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token.rb">puppet_master_token.rb</a>)
|
1856
|
+
</td>
|
1857
|
+
</tr>
|
1858
|
+
|
1859
|
+
<tr>
|
1860
|
+
<td valign=top>
|
1861
|
+
<pre><code>--to-puppet-master-token-file STRING</code></pre>
|
1862
|
+
</td>
|
1863
|
+
<td valign=top>
|
1864
|
+
File containing PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
|
1865
|
+
</td>
|
1866
|
+
<td valign=top>
|
1867
|
+
Specify a path to a file containing a PE RBAC token used to authenticate to the
|
1868
|
+
Puppetserver for a v4 catalog API call. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token_file.rb">puppet_master_token_file.rb</a>)
|
1869
|
+
</td>
|
1870
|
+
</tr>
|
1871
|
+
|
1716
1872
|
<tr>
|
1717
1873
|
<td valign=top>
|
1718
1874
|
<pre><code>--to-save-catalog STRING</code></pre>
|