octocatalog-diff 1.5.1 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (44) hide show
  1. checksums.yaml +5 -5
  2. data/.version +1 -1
  3. data/README.md +4 -4
  4. data/doc/CHANGELOG.md +49 -0
  5. data/doc/advanced-filter.md +23 -0
  6. data/doc/advanced-ignores.md +10 -0
  7. data/doc/advanced-puppet-master.md +23 -5
  8. data/doc/configuration-puppetdb.md +11 -0
  9. data/doc/dev/api/v1/calls/catalog-diff.md +6 -2
  10. data/doc/dev/api/v1/objects/diff.md +3 -3
  11. data/doc/dev/releasing.md +1 -1
  12. data/doc/limitations.md +9 -9
  13. data/doc/optionsref.md +167 -11
  14. data/doc/requirements.md +6 -2
  15. data/lib/octocatalog-diff/catalog-diff/differ.rb +29 -4
  16. data/lib/octocatalog-diff/catalog-diff/filter.rb +2 -1
  17. data/lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb +29 -25
  18. data/lib/octocatalog-diff/catalog-diff/filter/single_item_array.rb +44 -0
  19. data/lib/octocatalog-diff/catalog-util/builddir.rb +3 -3
  20. data/lib/octocatalog-diff/catalog-util/command.rb +25 -3
  21. data/lib/octocatalog-diff/catalog-util/fileresources.rb +1 -1
  22. data/lib/octocatalog-diff/catalog.rb +22 -4
  23. data/lib/octocatalog-diff/catalog/computed.rb +2 -1
  24. data/lib/octocatalog-diff/catalog/puppetmaster.rb +43 -5
  25. data/lib/octocatalog-diff/cli.rb +36 -5
  26. data/lib/octocatalog-diff/cli/options.rb +39 -3
  27. data/lib/octocatalog-diff/cli/options/hostname.rb +13 -2
  28. data/lib/octocatalog-diff/cli/options/pe_enc_token_file.rb +1 -1
  29. data/lib/octocatalog-diff/cli/options/puppet_master_api_version.rb +2 -2
  30. data/lib/octocatalog-diff/cli/options/puppet_master_token.rb +20 -0
  31. data/lib/octocatalog-diff/cli/options/puppet_master_token_file.rb +35 -0
  32. data/lib/octocatalog-diff/cli/options/puppet_master_update_catalog.rb +20 -0
  33. data/lib/octocatalog-diff/cli/options/puppet_master_update_facts.rb +20 -0
  34. data/lib/octocatalog-diff/cli/options/puppetdb_package_inventory.rb +18 -0
  35. data/lib/octocatalog-diff/cli/options/puppetdb_token.rb +17 -0
  36. data/lib/octocatalog-diff/cli/options/puppetdb_token_file.rb +21 -0
  37. data/lib/octocatalog-diff/facts/puppetdb.rb +43 -2
  38. data/lib/octocatalog-diff/puppetdb.rb +5 -1
  39. data/lib/octocatalog-diff/util/parallel.rb +20 -16
  40. data/lib/octocatalog-diff/util/util.rb +2 -0
  41. data/scripts/env/env.sh +1 -1
  42. data/scripts/git-extract/git-extract.sh +1 -1
  43. data/scripts/puppet/puppet.sh +1 -1
  44. metadata +37 -30
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
- SHA1:
3
- metadata.gz: 42b664636f99089e0838f35914f5cd0a89a60fe6
4
- data.tar.gz: a6e77f9e6f1d50c6f9ea8219b580891574dbc31e
2
+ SHA256:
3
+ metadata.gz: b99c2e5da4209a2779ec55c745b43a274c0983f00bc5ec41452bf19636818f9a
4
+ data.tar.gz: 1ebb9bc86ff2ab9dab9e6529a2f862ddc15447c859d7e1e2120ef8e177220c58
5
5
  SHA512:
6
- metadata.gz: 223c4d52d3d7e424d28c42fb52e5185fe1dac46d9ea2dea9be7eebbace60b7d13ae376367abc29e58341f3a82f2c84cf640e540c389badf0c9e4811065b9e2a4
7
- data.tar.gz: 0a821cf5bd95a83df9309de24141e21d3fadc21cb4d74b2344edcdb194c501bdb8c13bcd0db808d32ec07359fba5df6a559bdfc5c45077f7681e0d130d187b9c
6
+ metadata.gz: b9c7f405b7d219817088a1f29126335270245d5d1f8084a07e26b50b817adfb5b6019ed92edac55250ce611253cc418fee9a53a0cf9f065b73e3b89be1523809
7
+ data.tar.gz: 0054faec251c287be47947f73314fbcd30d5d3b4af240488fe97665247e5af7100561b908027a84dd9994853036a05330d82ab8da12ab2519437c5ec9a5101bc
data/.version CHANGED
@@ -1 +1 @@
1
- 1.5.1
1
+ 2.0.0
data/README.md CHANGED
@@ -1,4 +1,4 @@
1
- # octocatalog-diff
1
+ # octocatalog-diff ![CI](https://github.com/github/octocatalog-diff/workflows/CI/badge.svg)
2
2
 
3
3
  #### Compile Puppet catalogs from 2 branches, versions, etc., and compare them <img src="/doc/images/octocatolog-diff-logo.png" align="right" height=126 width=240>
4
4
 
@@ -23,7 +23,7 @@ There are some [limitations](doc/limitations.md) to a catalog-based approach, me
23
23
  `octocatalog-diff` is currently able to get catalogs by the following methods:
24
24
  - Compile catalog via the command line with a Puppet agent on your machine (as GitHub uses the tool internally)
25
25
  - Obtain catalog over the network from PuppetDB
26
- - Obtain catalog over the network using the API to query a Puppet Master / PuppetServer (Puppet 3.x and 4.x supported)
26
+ - Obtain catalog over the network using the API to query a Puppet Master / PuppetServer (Puppet 3.x through 6.x supported)
27
27
  - Read catalog from a JSON file
28
28
 
29
29
  ## Example
@@ -81,6 +81,6 @@ If you have a problem or suggestion, please [open an issue](https://github.com/g
81
81
 
82
82
  It requires 3rd party ruby gems found [here](/vendor/cache). It also includes portions of other open source projects [here](/lib/octocatalog-diff/external/pson), [here](/spec/octocatalog-diff/fixtures/repos/default/modules/stdlib), [here](/spec/octocatalog-diff/support/httparty) and [here](/spec/octocatalog-diff/tests/external/pson). All 3rd party code and required gems are licensed either as MIT or Apache 2.0.
83
83
 
84
- ## Authors
84
+ ## Authors / Owners
85
85
 
86
- `octocatalog-diff` was designed and authored by [Kevin Paulisse](https://github.com/kpaulisse) and is now maintained, reviewed, and tested by Kevin and the rest of the Site Reliability Engineering team at GitHub.
86
+ `octocatalog-diff` was originally designed and authored by [Kevin Paulisse](https://github.com/kpaulisse). It is now maintained by the Site Reliability Engineering team at GitHub.
@@ -8,6 +8,55 @@
8
8
  </tr>
9
9
  </thead><tbody>
10
10
 
11
+ <tr valign=top>
12
+ <td>2.0.0</td>
13
+ <td>2020-01-12</td>
14
+ <li><a href="https://github.com/github/octocatalog-diff/pull/226">#226</a>: (Enhancement) Add Puppet 6 support</li>
15
+ <li><a href="https://github.com/github/octocatalog-diff/pull/228">#228</a>: (Enhancement) Add Puppetserver catalog v4 API support</li>
16
+ <li><a href="https://github.com/github/octocatalog-diff/pull/229">#229</a>: (Enhancement) Add support for PE package inventory facts</li>
17
+ <li><a href="https://github.com/github/octocatalog-diff/pull/230">#230</a>: (Enhancement) Add set quality comparison to ignore filter</li>
18
+ </td>
19
+ </tr>
20
+
21
+ <tr valign=top>
22
+ <td>1.6.0</td>
23
+ <td>2019-10-31</td>
24
+ <li><a href="https://github.com/github/octocatalog-diff/pull/216">#216</a>: (Enhancement) Hide sensitive parameters</li>
25
+ <li><a href="https://github.com/github/octocatalog-diff/pull/204">#204</a>: (Enhancement) Add glob support for modulepath</li>
26
+ <li><a href="https://github.com/github/octocatalog-diff/pull/206">#206</a>: (Bug Fix) Fix multi-node list with parallel mode</li>
27
+ <li><a href="https://github.com/github/octocatalog-diff/pull/215">#215</a>: (Bug Fix) Add Support for Hashdiff 1.0.0</li>
28
+ </td>
29
+ </tr>
30
+
31
+ <tr valign=top>
32
+ <td>1.5.4</td>
33
+ <td>2018-12-11</td>
34
+ <td>
35
+ <li><a href="https://github.com/github/octocatalog-diff/pull/190">#190</a>: (Enhancement) Additional filtered out cases for compilation directory</li>
36
+ <li><a href="https://github.com/github/octocatalog-diff/pull/195">#195</a>: (Enhancement) Parallel catalog-diff when multiple hostnames are passed</li>
37
+ <li><a href="https://github.com/github/octocatalog-diff/pull/198">#198</a>: (Bug Fix) Portability fixes</li>
38
+ <li><a href="https://github.com/github/octocatalog-diff/pull/200">#200</a>: (Bug Fix) Support name parameter when validating references</li>
39
+ </td>
40
+ </tr>
41
+
42
+ <tr valign=top>
43
+ <td>1.5.3</td>
44
+ <td>2018-03-05</td>
45
+ <td>
46
+ <li><a href="https://github.com/github/octocatalog-diff/pull/176">#176</a>: (Enhancement) Normalize file resource titles in reference checks</li>
47
+ </td>
48
+ </tr>
49
+
50
+ <tr valign=top>
51
+ <td>1.5.2</td>
52
+ <td>2017-12-19</td>
53
+ <td>
54
+ <li><a href="https://github.com/github/octocatalog-diff/pull/169">#169</a>: (Enhancement) Puppet Enterprise RBAC token to authenticate to PuppetDB</li>
55
+ <li><a href="https://github.com/github/octocatalog-diff/pull/170">#170</a>: (Enhancement) Filter to treat an object the same as a single array containing that object</li>
56
+ <li><a href="https://github.com/github/octocatalog-diff/pull/165">#165</a>: (Bug Fix) Override of fact file via CLI now has precedence over value set in configuration file</li>
57
+ </td>
58
+ </tr>
59
+
11
60
  <tr valign=top>
12
61
  <td>1.5.1</td>
13
62
  <td>2017-11-16</td>
@@ -11,6 +11,7 @@ Here is the list of available filters and an explanation of each:
11
11
 
12
12
  - [Absent File](/doc/advanced-filter.md#absent-file) - Ignore parameter changes of a file that is declared to be absent
13
13
  - [JSON](/doc/advanced-filter.md#json) - Ignore whitespace differences if JSON parses to the same object
14
+ - [SingleItemArray](/doc/advanced-filter.md#SingleItemArray) - Ignore differences between object and array containing only that object
14
15
  - [YAML](/doc/advanced-filter.md#yaml) - Ignore whitespace/comment differences if YAML parses to the same object
15
16
 
16
17
  ## Absent File
@@ -84,6 +85,28 @@ If a file resource has extension `.json` and a difference in its content is obse
84
85
 
85
86
  This allows you to ignore changes in whitespace, comments, etc., that are not meaningful to a machine parsing the file. Note that changes to files may still trigger Puppet to restart services even though these changes are not displayed in the octocatalog-diff output.
86
87
 
88
+ ## Single Item Array
89
+
90
+ #### Usage
91
+
92
+ ```
93
+ --filters SingleItemArray
94
+ ```
95
+
96
+ #### Description
97
+
98
+ When enabling the future parser or upgrading between certain versions of Puppet, the internal structure of the catalog for certain parameters can change as shown in the following example:
99
+
100
+ ```
101
+ Old: { "notify": "Service[foo]" }
102
+ New: { "notify": [ "Service[foo]" ] }
103
+ ```
104
+
105
+ This filter will suppress differences for the value of a parameter when:
106
+
107
+ - The value in one catalog is an object, AND
108
+ - The value in the other catalog is an array containing *only* that same object
109
+
87
110
  ## YAML
88
111
 
89
112
  #### Usage
@@ -222,3 +222,13 @@ File[/tmp/foo] =>
222
222
  In this case, the very important line was removed from the catalog, and you want to know about this. Ignoring `File[/tmp/foo]::parameters::content` would have suppressed this (because all changes to that attribute are ignored). Also ignoring `File[/tmp/foo]::parameters::content=~>This is the line in the new catalog that I do not care about$` would have also suppressed this (because the regular expression was matched for *one* of the lines). However, the two examples with `=&>` in this section would *not* have suppressed this change, because it is no longer the case that *all* changes in the file matched the regular expression.
223
223
 
224
224
  :warning: All lines are stripped of leading and trailing spaces before the regular expression match is tried. This stripping of whitespace is done *only* for this comparison stage, and does not affect the display of any results.
225
+
226
+ #### Ignoring attributes which have identical elements but in arbitrary order
227
+
228
+ You can ignore attributes where both the values in both the old and new catalogs are arrays and the arrays
229
+ contain identical elements but in arbitrary order. Basically, you can ignore a parameter where the values
230
+ have set equality.
231
+
232
+ To ignore any parameters named `foo` with values having set equality, you would use:
233
+
234
+ --ignore 'My::Custom::Resource[*]::parameters::foo=s>='
@@ -8,7 +8,11 @@ Please note the following caveats:
8
8
 
9
9
  0. You will need to deploy your Puppet code to an environment on your Puppet Master prior to running `octocatalog-diff` for that environment. `octocatalog-diff` does not deploy code for you.
10
10
 
11
- 0. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below.
11
+ 0. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below. If you are using Puppet Enterprise and use
12
+ the Puppet Master v4 API you may also use a Puppet Enterprise RBAC token. The user owning the token will need the "Puppet Server Compile catalogs for remote nodes" permission.
13
+ See the [PE RBAC Token Authorization](#pe-rbac-token-authorization) section below.
14
+
15
+ 0. If you are using the v2 or v3 PuppetServer APIs with Octocatalog-Diff to compile catalogs, then those catalogs and facts will be automatically stored in PuppetDB. However, when using the v4 PuppetServer API with Octocatalog-Diff, facts and catalogs are *not* automatically stored in PuppetDB - persistence is optional and may be enabled with the appropriate Octocatalog-Diff CLI flag. If your environment depends on the accuracy of exported resources or facts in PuppetDB, you may wish to upgrade and use the V4 API, to avoid unintentional side-effects.
12
16
 
13
17
  ## Command line options
14
18
 
@@ -18,11 +22,15 @@ The following command line options are used to retrieve a catalog from a Puppet
18
22
  | ------ | ----------- |
19
23
  | `-f ENVIRONMENT` | Environment name to use for the "from" catalog |
20
24
  | `-t ENVIRONMENT` | Environment name to use for the "to" catalog |
21
- | `--puppet-master HOSTNAME:PORT | The hostname and port number of the Puppet Master. (By default the port used by Puppet Master is 8140.) |
22
- | `--puppet-master-api-version VERSION | The API version used by the Puppet Master. API versions 2 and 3 are supported. Puppet Master 3.x uses API version 2, and the PuppetServer for Puppet 4.x uses API version 3. By default, API version 3 is used, so you only need to set this option if you are using Puppet Master 3.x. |
25
+ | `--puppet-master HOSTNAME:PORT` | The hostname and port number of the Puppet Master. (By default the port used by Puppet Master is 8140.) |
26
+ | `--puppet-master-api-version VERSION` | The API version used by the Puppet Master. API versions 2, 3,and 4 are supported. Puppet Master 3.x uses API version 2, and the PuppetServer for Puppet 4.x uses API version 3. PuppetServer 6.3.0 introduced the optional use of the v4 API but still fully supports the v3 API. By default, API version 3 is used, so you only need to set this option if you are using Puppet Master 3.x or wish to use the newer v4 API with PuppetServer 6. |
23
27
  | `--puppet-master-ssl-ca PATH` | Path to the CA certificate (public portion of certificate only) for your Puppet Master. This file will be on your Puppet Master and all Puppet agents. You can find it by running `puppet config print cacert` on any Puppet-managed host. |
24
- | `--puppet-master-ssl-client-cert PATH` | Path to the client certificate. Please see the section below on certificate authentication. |
25
- | `--puppet-master-ssl-client-key PATH` | Path to the client private key. Please see the section below on certificate authentication. |
28
+ | `--puppet-master-ssl-client-cert PATH` | Path to the client certificate. Please see the section below on certificate authentication. This can be omitted if using PE RBAC token based auth with the v4 API. |
29
+ | `--puppet-master-ssl-client-key PATH` | Path to the client private key. Please see the section below on certificate authentication. This can be omitted if using PE RBAC token based auth with the v4 API. |
30
+ | `--puppet-master-token STRING` | A PE RBAC token used to authenticate a v4 catalog compile, in lieu of using certificate authentication. Please see the section below on token authentication. |
31
+ | `--puppet-master-token-file PATH` | A path to a file containing a PE RBAC token used to authenticate a v4 catalog compile, in lieu of using certificate authentication. If this and `--puppet-master-token` are both specified, `--puppet-master-token` will be used instead. Please see the section below on token authentication. |
32
+ | `--puppet-master-update-catalog` | When using the v4 API, instruct the PuppetServer to update the catalog generated from the compile in its PuppetDB instance. When using v2 and v3 APIs the catalog is always updated and this option is ignored. |
33
+ | `--puppet-master-update-facts` | When using the v4 API, instruct the PuppetServer to update the facts used during the compile in its PuppetDB instance. When using v2 and v3 APIs the facts are always updated and this option is ignored. |
26
34
 
27
35
  If you wish to use a different Puppet Master to compile the "to" and "from" catalogs, you may prefix any of the `--puppet-master...` options with `to` or `from`. For example, perhaps you are testing an upgrade from Puppet 3.x to 4.x. You could use:
28
36
 
@@ -48,3 +56,13 @@ allow $1
48
56
  ```
49
57
 
50
58
  Please follow the instructions for the version of Puppet Master, PuppetServer, or Puppet Enterprise that you are using in order to generate and authorize the certificates.
59
+
60
+ ## PE RBAC Token authorization
61
+
62
+ In newer versions of Puppet Enterprise you can authenticate using a valid PE RBAC token with appropriate permissions as long as it is authorized in the PuppetServer `auth.conf` file.
63
+
64
+ By default this permission is enabled and controlled by the `puppet_enterprise::master::tk_authz::allow_rbac_catalog_compile` Hiera setting.
65
+
66
+ The user the token was issued to must have the `puppetserver:compile_catalogs:*` permission.
67
+
68
+ Note: A Puppet catalog may contain unencrypted secrets, even ones marked as `Sensitive`. In order to perform its job, Octocatalog-Diff needs access to the catalog. By granting a user the above RBAC permission you are granting them the ability to retrieve and view the complete catalog resulting from a compile, including any included secrets.
@@ -36,6 +36,7 @@ The following settings can be used in a [configuration file](/doc/configuration.
36
36
  | `settings[:puppetdb_ssl_client_key]` | TEXT of the private key of the client SSL keypair used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
37
37
  | `settings[:puppetdb_ssl_client_pem]` | Concatenation of the text of `puppetdb_ssl_client_key` and `puppetdb_ssl_client_cert` as previously described. This is a good alternative if your certificate chain is complex and it's easier just to put everything in a single place. Note: this option is second in precedence; if `settings[:puppetdb_ssl_client_cert]` and `settings[:puppetdb_ssl_client_key]` are both set, this will be ignored. |
38
38
  | `settings[:puppetdb_ssl_client_password]` | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required and should be left undefined. |
39
+ | `settings[:puppetdb_token]` | TEXT containing the PE RBAC token used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
39
40
 
40
41
  ## Supplying necessary information via the command line
41
42
 
@@ -48,6 +49,8 @@ The following arguments can be used on the command line.
48
49
  | --puppetdb-ssl-client-cert FILENAME | Path to the certificate of the client SSL keypair. |
49
50
  | --puppetdb-ssl-client-key FILENAME | Path to the private key of the client SSL keypair. |
50
51
  | --puppetdb-ssl-client-password PASSWORD_STRING | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required. |
52
+ | --puppetdb-token STRING | String containing the PE RBAC token used to authenticate to PuppetDB. |
53
+ | --puppetdb-token-file FILENAME | Path to the PE RBAC token file used to authenticate to PuppetDB. |
51
54
 
52
55
  ## Supplying necessary information via the environment
53
56
 
@@ -64,3 +67,11 @@ SSL support is enabled via any of the `--puppetdb-ssl-...` command line options
64
67
  - The CA certificate should be the public certificate of the CA that signed your PuppetDB server's certificate. This file can be found in `/etc/puppetlabs/puppetdb/ssl/ca.pem` on a PuppetDB server. Since this is a public certificate, it is safe (and recommended) to distribute this file to any clients that may connect to this PuppetDB instance.
65
68
 
66
69
  - The client keypair (key, certificate, and optionally password) should be generated individually for each client. You should NOT copy SSL keypairs from your PuppetDB server (or anywhere else) to your clients. If you are using `octocatalog-diff` on a system that is managed by Puppet, you may wish to use the same SSL credentials that the system uses to authenticate to Puppet. With recent versions of the Puppet agent, those certificates are found in `/etc/puppetlabs/puppet/ssl`.
70
+
71
+ # Puppet Enterprise PuppetDB Package Inventory
72
+
73
+ Puppet Enterprise customers have an optional package inventory feature which can be enabled. When this feature is enabled an inventory of all system packages
74
+ is performed and uploaded as a fact which is then processed and stored independently of the normal Facter data in PuppetDB. Most environments won't need
75
+ to replicate the package inventory facts for testing with Octocatalog-Diff but if you want the package inventory data (if present) to be included
76
+ in the facts retrieved from PuppetDB by Octocatalog-Diff you should specify the `--puppetdb-package-inventory` flag. When enabled, this flag will instruct
77
+ Octocatalog-Diff to retrieve any package data found for a node from PuppetDB and include it in the facts used during the Octocatalog-Diff compile.
@@ -167,12 +167,16 @@ In this case, "owner", "notify", and "content" are nested under "parameters". In
167
167
 
168
168
  ```
169
169
  # Ignore all changes to the `owner` attribute of a file.
170
- [ { type: Regexp.new('\AFile\z'), attr: Regexp.new("\Aparameters\fowner\z" } ]
170
+ [ { type: Regexp.new('\AFile\z'), attr: Regexp.new("\\Aparameters\fowner\\z" } ]
171
171
 
172
172
  # Ignore changes to `owner` or `group` for a file or an exec.
173
- [ { type: Regexp.new('\A(File|Exec)\z'), attr: Regexp.new("\Aparameters\f(owner|group)\z" } ]
173
+ [ { type: Regexp.new('\A(File|Exec)\z'), attr: Regexp.new("\\Aparameters\f(owner|group)\\z" } ]
174
174
  ```
175
175
 
176
+ When using regular expressions, `\f` (form feed character) is used to separate the structure (e.g. `parameters\fowner` refers to the `parameters` hash, `owner` key).
177
+
178
+ :bulb: Note that `\A` in Ruby matches the beginning of the string and `\z` matches the end, but these are not actual characters. Therefore, if you are using `\A` or `\z` in double quotes (`"`), be sure to heed the examples above and write your expression like: `Regexp.new("\\Aparameters\fowner\\z")`.
179
+
176
180
  #### `:validate_references` (Array&lt;String&gt;, Optional)
177
181
 
178
182
  Invoke the [catalog validation](/doc/advanced-catalog-validation.md) feature to ensure resources targeted by `before`, `notify`, `require`, and/or `subscribe` exist in the catalog. If this parameter is not defined, no reference validation occurs.
@@ -94,9 +94,9 @@ Returns the value of the resource from the new catalog.
94
94
  }
95
95
  }
96
96
 
97
- # Demonstrates structure and old_value
97
+ # Demonstrates structure and new_value
98
98
  diff.structure #=> ['parameters', 'content']
99
- diff.old_value #=> 'This is the NEW FILE!!!!!'
99
+ diff.new_value #=> 'This is the NEW FILE!!!!!'
100
100
  ```
101
101
 
102
102
  #### `#old_file` (String)
@@ -107,7 +107,7 @@ Note that this is a pass-through of information provided in the Puppet catalog,
107
107
 
108
108
  Note also that if the diff represents addition of a resource, this will return `nil`, because the resource does not exist in the old catalog.
109
109
 
110
- #### `#old_file` (String)
110
+ #### `#old_line` (String)
111
111
 
112
112
  Returns the line number within the Puppet manifest giving rise to the resource as it exists in the old catalog. (See `#old_file` for the filename of the Puppet manifest.)
113
113
 
@@ -9,7 +9,7 @@ The project maintainers are responsible for bumping the version number, regenera
9
9
  To test the new version of `octocatalog-diff` in the GitHub Puppet repository, check out `github/puppet` and:
10
10
 
11
11
  - Start a new branch based off master
12
- - Run `script/update-octocatalog-diff -r <ocd_branch_name>`
12
+ - Run `script/update-octocatalog-diff <ocd_branch_name>`
13
13
  - Confirm and commit the result
14
14
  - Make sure all CI jobs pass
15
15
  - Run the `puppet-catalog-diff` CI job and make sure it passes and shows expected results
@@ -2,23 +2,23 @@
2
2
 
3
3
  Testing of Puppet catalogs is faster than running the agent, but you need to be careful of the following limitations:
4
4
 
5
- 0. Facts are not taken from a live agent run
5
+ 1. Facts are not taken from a live agent run
6
6
 
7
- octocatalog-diff by default uses the facts reported from a node's more recent Puppet run. If you have made changes to custom facts, catalog testing will **NOT** be an adequate test of whether your custom facts worked. (You can still use octocatalog-diff to help predict changes to nodes based on changes to facts, by overriding facts on the command line.)
7
+ octocatalog-diff by default uses the facts reported from a node's more recent Puppet run. If you have made changes to custom facts, catalog testing will **NOT** be an adequate test of whether your custom facts worked. (You can still use octocatalog-diff to help predict changes to nodes based on changes to facts, by overriding facts on the command line.)
8
8
 
9
- 0. Agents handle depenency ordering and implementation details
9
+ 1. Agents handle depenency ordering and implementation details
10
10
 
11
- The catalog defines the state of the system, but it's up to the agent to determine how to bring the system to a point that matches the catalog. The agent is responsible for order of operations and actually making the change.
11
+ The catalog defines the state of the system, but it's up to the agent to determine how to bring the system to a point that matches the catalog. The agent is responsible for order of operations and actually making the change.
12
12
 
13
- Two specific situations that catalog testing does **NOT** detect are:
13
+ Two specific situations that catalog testing does **NOT** detect are:
14
14
 
15
- - Dependency loops (e.g., you have made A require B, B require C, and C require A).
15
+ - Dependency loops (e.g., you have made A require B, B require C, and C require A).
16
16
 
17
- - Operations not supported by the provider. For example, assume that in your current Puppet manifests, you set the size of a file system to 100 GB. You change this in your new branch to 50 GB. octocatalog-diff will dutifully report this change to you. However, the agent will fail to make the change, because it is not possible to shrink a file system from 100 GB to 50 GB.
17
+ - Operations not supported by the provider. For example, assume that in your current Puppet manifests, you set the size of a file system to 100 GB. You change this in your new branch to 50 GB. octocatalog-diff will dutifully report this change to you. However, the agent will fail to make the change, because it is not possible to shrink a file system from 100 GB to 50 GB.
18
18
 
19
- 0. Changes in underlying providers may not be noticed
19
+ 1. Changes in underlying providers may not be noticed
20
20
 
21
- Consider that you are using a Puppet module that creates a file system. The current implementation of that module checks to see if *any* file system is present on the device, and creates a new file system there if no file system was present. You upgrade the module, and the new version checks to see if *the specified* file system is present on the device, and reformats the device with the specified file system (regardless of whether there was no file system or if there was an existing file system of a different type). There would be no catalog changes (hence octocatalog-diff would report nothing) because the catalog simply instructs the agent to create a file system of the specified type at the defined location. However, the actual implementation of those instructions has changed dramatically.
21
+ Consider that you are using a Puppet module that creates a file system. The current implementation of that module checks to see if *any* file system is present on the device, and creates a new file system there if no file system was present. You upgrade the module, and the new version checks to see if *the specified* file system is present on the device, and reformats the device with the specified file system (regardless of whether there was no file system or if there was an existing file system of a different type). There would be no catalog changes (hence octocatalog-diff would report nothing) because the catalog simply instructs the agent to create a file system of the specified type at the defined location. However, the actual implementation of those instructions has changed dramatically.
22
22
 
23
23
  In general catalog testing is great for:
24
24
 
@@ -9,7 +9,8 @@
9
9
 
10
10
  ```
11
11
  Usage: octocatalog-diff [command line options]
12
- -n, --hostname HOSTNAME Use PuppetDB facts from last run of hostname
12
+ -n HOSTNAME1[,HOSTNAME2[,...]], Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames
13
+ --hostname
13
14
  --basedir DIRNAME Use an alternate base directory (git checkout of puppet repository)
14
15
  -f, --from FROM_BRANCH Branch you are coming from
15
16
  -t, --to TO_BRANCH Branch you are going to
@@ -36,6 +37,8 @@ Usage: octocatalog-diff [command line options]
36
37
  --fact-file STRING Override fact globally
37
38
  --to-fact-file STRING Override fact for the to branch
38
39
  --from-fact-file STRING Override fact for the from branch
40
+ --[no-]puppetdb-package-inventory
41
+ Include Puppet Enterprise package inventory data, if found
39
42
  --save-catalog STRING Save intermediate catalogs into files globally
40
43
  --to-save-catalog STRING Save intermediate catalogs into files for the to branch
41
44
  --from-save-catalog STRING Save intermediate catalogs into files for the from branch
@@ -86,7 +89,20 @@ Usage: octocatalog-diff [command line options]
86
89
  --puppet-binary STRING Full path to puppet binary globally
87
90
  --to-puppet-binary STRING Full path to puppet binary for the to branch
88
91
  --from-puppet-binary STRING Full path to puppet binary for the from branch
92
+ --puppet-master-token-file STRING
93
+ File containing PE RBAC token to authenticate to the Puppetserver API v4 globally
94
+ --to-puppet-master-token-file STRING
95
+ File containing PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
96
+ --from-puppet-master-token-file STRING
97
+ File containing PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
89
98
  --facts-terminus STRING Facts terminus: one of yaml, facter
99
+ --puppet-master-token STRING PE RBAC token to authenticate to the Puppetserver API v4 globally
100
+ --to-puppet-master-token STRING
101
+ PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
102
+ --from-puppet-master-token STRING
103
+ PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
104
+ --puppetdb-token TOKEN Token to access the PuppetDB API
105
+ --puppetdb-token-file PATH Path containing token for PuppetDB API, relative or absolute
90
106
  --puppetdb-url URL PuppetDB base URL
91
107
  --puppetdb-ssl-ca FILENAME CA certificate that signed the PuppetDB certificate
92
108
  --puppetdb-ssl-client-cert FILENAME
@@ -108,11 +124,23 @@ Usage: octocatalog-diff [command line options]
108
124
  --to-puppet-master STRING Hostname or Hostname:PortNumber for Puppet Master for the to branch
109
125
  --from-puppet-master STRING Hostname or Hostname:PortNumber for Puppet Master for the from branch
110
126
  --puppet-master-api-version STRING
111
- Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) globally
127
+ Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) globally
112
128
  --to-puppet-master-api-version STRING
113
- Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the to branch
129
+ Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the to branch
114
130
  --from-puppet-master-api-version STRING
115
- Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the from branch
131
+ Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the from branch
132
+ --[no-]puppet-master-update-catalog
133
+ Update catalog in PuppetDB when using Puppetmaster API version 4 globally
134
+ --[no-]to-puppet-master-update-catalog
135
+ Update catalog in PuppetDB when using Puppetmaster API version 4 for the to branch
136
+ --[no-]from-puppet-master-update-catalog
137
+ Update catalog in PuppetDB when using Puppetmaster API version 4 for the from branch
138
+ --[no-]puppet-master-update-facts
139
+ Update facts in PuppetDB when using Puppetmaster API version 4 globally
140
+ --[no-]to-puppet-master-update-facts
141
+ Update facts in PuppetDB when using Puppetmaster API version 4 for the to branch
142
+ --[no-]from-puppet-master-update-facts
143
+ Update facts in PuppetDB when using Puppetmaster API version 4 for the from branch
116
144
  --puppet-master-ssl-ca STRING
117
145
  Full path to CA certificate that signed the Puppet Master certificate globally
118
146
  --to-puppet-master-ssl-ca STRING
@@ -716,7 +744,7 @@ Puppet control repo template, the value of this should be 'hieradata', which is
716
744
  <pre><code>--from-puppet-master-api-version STRING</code></pre>
717
745
  </td>
718
746
  <td valign=top>
719
- Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the from branch
747
+ Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the from branch
720
748
  </td>
721
749
  <td valign=top>
722
750
  Specify the API version to use for the Puppet Master. This makes it possible to authenticate to a
@@ -778,6 +806,32 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
778
806
  </td>
779
807
  </tr>
780
808
 
809
+ <tr>
810
+ <td valign=top>
811
+ <pre><code>--from-puppet-master-token STRING</code></pre>
812
+ </td>
813
+ <td valign=top>
814
+ PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
815
+ </td>
816
+ <td valign=top>
817
+ Specify a PE RBAC token used to authenticate to Puppetserver for v4
818
+ catalog API calls. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token.rb">puppet_master_token.rb</a>)
819
+ </td>
820
+ </tr>
821
+
822
+ <tr>
823
+ <td valign=top>
824
+ <pre><code>--from-puppet-master-token-file STRING</code></pre>
825
+ </td>
826
+ <td valign=top>
827
+ File containing PE RBAC token to authenticate to the Puppetserver API v4 for the from branch
828
+ </td>
829
+ <td valign=top>
830
+ Specify a path to a file containing a PE RBAC token used to authenticate to the
831
+ Puppetserver for a v4 catalog API call. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token_file.rb">puppet_master_token_file.rb</a>)
832
+ </td>
833
+ </tr>
834
+
781
835
  <tr>
782
836
  <td valign=top>
783
837
  <pre><code>--from-puppetdb
@@ -854,14 +908,17 @@ Puppet control repo template, the value of this should be 'hieradata', which is
854
908
 
855
909
  <tr>
856
910
  <td valign=top>
857
- <pre><code>-n HOSTNAME
858
- --hostname HOSTNAME</code></pre>
911
+ <pre><code>-n HOSTNAME1[,HOSTNAME2[,...]]
912
+ --hostname HOSTNAME1[,HOSTNAME2[,...]]</code></pre>
859
913
  </td>
860
914
  <td valign=top>
861
- Use PuppetDB facts from last run of hostname
915
+ Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames
862
916
  </td>
863
917
  <td valign=top>
864
- Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display. (<a href="../lib/octocatalog-diff/cli/options/hostname.rb">hostname.rb</a>)
918
+ Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display.
919
+ This option can recieve a single hostname, or a comma separated list of
920
+ multiple hostnames, which are split into an Array. Multiple hostnames do not
921
+ work with the `catalog-only` or `bootstrap-then-exit` options. (<a href="../lib/octocatalog-diff/cli/options/hostname.rb">hostname.rb</a>)
865
922
  </td>
866
923
  </tr>
867
924
 
@@ -1233,7 +1290,7 @@ to work correctly. (<a href="../lib/octocatalog-diff/cli/options/preserve_enviro
1233
1290
  <pre><code>--puppet-master-api-version STRING</code></pre>
1234
1291
  </td>
1235
1292
  <td valign=top>
1236
- Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) globally
1293
+ Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) globally
1237
1294
  </td>
1238
1295
  <td valign=top>
1239
1296
  Specify the API version to use for the Puppet Master. This makes it possible to authenticate to a
@@ -1295,6 +1352,32 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
1295
1352
  </td>
1296
1353
  </tr>
1297
1354
 
1355
+ <tr>
1356
+ <td valign=top>
1357
+ <pre><code>--puppet-master-token STRING</code></pre>
1358
+ </td>
1359
+ <td valign=top>
1360
+ PE RBAC token to authenticate to the Puppetserver API v4 globally
1361
+ </td>
1362
+ <td valign=top>
1363
+ Specify a PE RBAC token used to authenticate to Puppetserver for v4
1364
+ catalog API calls. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token.rb">puppet_master_token.rb</a>)
1365
+ </td>
1366
+ </tr>
1367
+
1368
+ <tr>
1369
+ <td valign=top>
1370
+ <pre><code>--puppet-master-token-file STRING</code></pre>
1371
+ </td>
1372
+ <td valign=top>
1373
+ File containing PE RBAC token to authenticate to the Puppetserver API v4 globally
1374
+ </td>
1375
+ <td valign=top>
1376
+ Specify a path to a file containing a PE RBAC token used to authenticate to the
1377
+ Puppetserver for a v4 catalog API call. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token_file.rb">puppet_master_token_file.rb</a>)
1378
+ </td>
1379
+ </tr>
1380
+
1298
1381
  <tr>
1299
1382
  <td valign=top>
1300
1383
  <pre><code>--puppetdb-api-version N</code></pre>
@@ -1308,6 +1391,23 @@ the default. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_api_version.
1308
1391
  </td>
1309
1392
  </tr>
1310
1393
 
1394
+ <tr>
1395
+ <td valign=top>
1396
+ <pre><code>--puppetdb-package-inventory
1397
+ --no-puppetdb-package-inventory </code></pre>
1398
+ </td>
1399
+ <td valign=top>
1400
+ Include Puppet Enterprise package inventory data, if found
1401
+ </td>
1402
+ <td valign=top>
1403
+ When pulling facts from PuppetDB in a Puppet Enterprise environment, also include
1404
+ the Puppet Enterprise Package Inventory data in the fact results, if available.
1405
+ Generally you should not need to specify this, but including the package inventory
1406
+ data will produce a more accurate set of input facts for environments using
1407
+ package inventory. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_package_inventory.rb">puppetdb_package_inventory.rb</a>)
1408
+ </td>
1409
+ </tr>
1410
+
1311
1411
  <tr>
1312
1412
  <td valign=top>
1313
1413
  <pre><code>--puppetdb-ssl-ca FILENAME</code></pre>
@@ -1374,6 +1474,36 @@ the text of the password won't appear in the process list. (<a href="../lib/octo
1374
1474
  </td>
1375
1475
  </tr>
1376
1476
 
1477
+ <tr>
1478
+ <td valign=top>
1479
+ <pre><code>--puppetdb-token TOKEN</code></pre>
1480
+ </td>
1481
+ <td valign=top>
1482
+ Token to access the PuppetDB API
1483
+ </td>
1484
+ <td valign=top>
1485
+ Specify the PE RBAC token to access the PuppetDB API. Refer to
1486
+ https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
1487
+ for details on generating and obtaining a token. Use this option to specify the text
1488
+ of the token. (Use --puppetdb-token-file to read the content of the token from a file.) (<a href="../lib/octocatalog-diff/cli/options/puppetdb_token.rb">puppetdb_token.rb</a>)
1489
+ </td>
1490
+ </tr>
1491
+
1492
+ <tr>
1493
+ <td valign=top>
1494
+ <pre><code>--puppetdb-token-file PATH</code></pre>
1495
+ </td>
1496
+ <td valign=top>
1497
+ Path containing token for PuppetDB API, relative or absolute
1498
+ </td>
1499
+ <td valign=top>
1500
+ Specify the PE RBAC token to access the PuppetDB API. Refer to
1501
+ https://puppet.com/docs/pe/latest/rbac/rbac_token_auth_intro.html#generate-a-token-using-puppet-access
1502
+ for details on generating and obtaining a token. Use this option to specify the text
1503
+ in a file, to read the content of the token from the file. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_token_file.rb">puppetdb_token_file.rb</a>)
1504
+ </td>
1505
+ </tr>
1506
+
1377
1507
  <tr>
1378
1508
  <td valign=top>
1379
1509
  <pre><code>--puppetdb-url URL</code></pre>
@@ -1651,7 +1781,7 @@ Puppet control repo template, the value of this should be 'hieradata', which is
1651
1781
  <pre><code>--to-puppet-master-api-version STRING</code></pre>
1652
1782
  </td>
1653
1783
  <td valign=top>
1654
- Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x) for the to branch
1784
+ Puppet Master API version (2 for Puppet 3.x, 3 for Puppet 4.x, 4 for Puppet Server >= 6.3.0) for the to branch
1655
1785
  </td>
1656
1786
  <td valign=top>
1657
1787
  Specify the API version to use for the Puppet Master. This makes it possible to authenticate to a
@@ -1713,6 +1843,32 @@ This timeout is specified in seconds. (<a href="../lib/octocatalog-diff/cli/opti
1713
1843
  </td>
1714
1844
  </tr>
1715
1845
 
1846
+ <tr>
1847
+ <td valign=top>
1848
+ <pre><code>--to-puppet-master-token STRING</code></pre>
1849
+ </td>
1850
+ <td valign=top>
1851
+ PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
1852
+ </td>
1853
+ <td valign=top>
1854
+ Specify a PE RBAC token used to authenticate to Puppetserver for v4
1855
+ catalog API calls. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token.rb">puppet_master_token.rb</a>)
1856
+ </td>
1857
+ </tr>
1858
+
1859
+ <tr>
1860
+ <td valign=top>
1861
+ <pre><code>--to-puppet-master-token-file STRING</code></pre>
1862
+ </td>
1863
+ <td valign=top>
1864
+ File containing PE RBAC token to authenticate to the Puppetserver API v4 for the to branch
1865
+ </td>
1866
+ <td valign=top>
1867
+ Specify a path to a file containing a PE RBAC token used to authenticate to the
1868
+ Puppetserver for a v4 catalog API call. (<a href="../lib/octocatalog-diff/cli/options/puppet_master_token_file.rb">puppet_master_token_file.rb</a>)
1869
+ </td>
1870
+ </tr>
1871
+
1716
1872
  <tr>
1717
1873
  <td valign=top>
1718
1874
  <pre><code>--to-save-catalog STRING</code></pre>