octocatalog-diff 2.0.0 → 2.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b99c2e5da4209a2779ec55c745b43a274c0983f00bc5ec41452bf19636818f9a
4
- data.tar.gz: 1ebb9bc86ff2ab9dab9e6529a2f862ddc15447c859d7e1e2120ef8e177220c58
3
+ metadata.gz: 07e53efa197ece3d868ab453f200b49d6fc0718fcd40e8201ab92d04e4787871
4
+ data.tar.gz: e60db7eb374631ec5a4f430b58b16f57772e081aeea3b12d7576e3265df4ae6b
5
5
  SHA512:
6
- metadata.gz: b9c7f405b7d219817088a1f29126335270245d5d1f8084a07e26b50b817adfb5b6019ed92edac55250ce611253cc418fee9a53a0cf9f065b73e3b89be1523809
7
- data.tar.gz: 0054faec251c287be47947f73314fbcd30d5d3b4af240488fe97665247e5af7100561b908027a84dd9994853036a05330d82ab8da12ab2519437c5ec9a5101bc
6
+ metadata.gz: 2b6ce06f9d1d91f8bfeb12f731a8b8d901b794396dd34a6b3e2d29b8e4e7e59d02ceb1f1bdce89ac73d6d3ad14beb2883188810c778ed6ccb431a30b05a9d7b3
7
+ data.tar.gz: 2f712011fd71e32caf2bf3e453ce16cae41c2be79a6bdfa12ed0d3bdc1ce62751e86c6f4eea500d243f8a69adf6642f1009250c88b5b85cb1af2ddee67e3a7a0
data/.version CHANGED
@@ -1 +1 @@
1
- 2.0.0
1
+ 2.1.0
data/doc/CHANGELOG.md CHANGED
@@ -8,6 +8,17 @@
8
8
  </tr>
9
9
  </thead><tbody>
10
10
 
11
+ <tr valign=top>
12
+ <td>2.1.0</td>
13
+ <td>2020-02-18</td>
14
+
15
+ <li><a href="https://github.com/github/octocatalog-diff/pull/240">#240</a>: (Enhancement) Run CI against Puppet 7 and Ruby 3</li>
16
+ <li><a href="https://github.com/github/octocatalog-diff/pull/241">#241</a>: (Bug Fix) Fix indent and numbering in several docs</li>
17
+ <li><a href="https://github.com/github/octocatalog-diff/pull/242">#242</a>: (Enhancement) Handle JSON facts structured as name/values</li>
18
+ <li><a href="https://github.com/github/octocatalog-diff/pull/244">#244</a>: (Enhancement) Update capability and docs of --compare-file-text</li>
19
+ </td>
20
+ </tr>
21
+
11
22
  <tr valign=top>
12
23
  <td>2.0.0</td>
13
24
  <td>2020-01-12</td>
@@ -0,0 +1,79 @@
1
+ # Compare file text
2
+
3
+ `octocatalog-diff` contains functionality to detect changes in file content for file resources that use the `source` attribute like this:
4
+
5
+ ```
6
+ file { '/etc/logrotate.d/my-service.conf':
7
+ source => 'puppet:///modules/logrotate/etc/logrotate.d/my-service.conf',
8
+ }
9
+ ```
10
+
11
+ When the `source` attribute is used, the catalog contains the value of the attribute (in the example above, `source` = `puppet:///modules/logrotate/etc/logrotate.d/my-service.conf`). However, the catalog does not contain the content of the file. When an agent applies the catalog, the file found on the server or in the code base at `modules/logrotate/file/etc/logrotate.d/my-service.conf` will be installed into `/etc/logrotate.d/my-service.conf`.
12
+
13
+ If a developer creates a change to this file, the catalog will not indicate this change. This means that so long as the `source` location does not change, any tool analyzing just the catalog would not detect a "diff" resulting from changes in the underlying file itself.
14
+
15
+ However, since applying the catalog could change the content of a file on the target node, `octocatalog-diff` has a feature that will do the following for any `source => 'puppet:///modules/...'` files:
16
+
17
+ - Locate the source file in the Puppet code
18
+ - Substitute the content of the file into the generated catalog (remove `source` and populate `content`)
19
+ - Display the "diff" in the now-populated `content` field
20
+
21
+ This feature is available only when the catalogs are being compiled from local code. This feature is not available, and will be automatically disabled, when pulling catalogs from PuppetDB or a Puppet server.
22
+
23
+ Note: In Puppet >= 4.4 there is an option in Puppet itself called "static catalogs" which if enabled will cause the checksum of the file to be included in the catalog. However, the `octocatalog-diff` feature described here is still useful because it can be used to display a "diff" of the change rather than just displaying a "diff" of a checksum.
24
+ ## Command line options
25
+
26
+ ### `--compare-file-text` and `--no-compare-file-text`
27
+
28
+ The feature described above is enabled by default, and no special setup is required.
29
+
30
+ To disable this feature for a specific run, add `--no-compare-file-text` to the command line.
31
+
32
+ To disable this feature by default, add the following to a [configuration](/doc/configuration.md) file:
33
+
34
+ ```ruby
35
+ settings[:compare_file_text] = false
36
+ ```
37
+
38
+ If this feature is disabled by default in a configuration file, add `--compare-file-text` to enable the feature for this specific run.
39
+
40
+ Note that the feature will be automatically disabled, regardless of configuration or command line options, if catalogs are being pulled from PuppetDB or a Puppet server.
41
+
42
+ ### `--compare-file-text-ignore-tags`
43
+
44
+ To disable this feature for specific `file` resources, set a tag on the resources for which the comparison is undesired. For example:
45
+
46
+ ```
47
+ file { '/etc/logrotate.d/my-service.conf':
48
+ source => 'puppet:///modules/logrotate/etc/logrotate.d/my-service.conf',
49
+ }
50
+
51
+ file { '/etc/logrotate.d/other-service.conf':
52
+ tag => ['compare-file-text-disable'],
53
+ source => 'puppet:///modules/logrotate/etc/logrotate.d/other-service.conf',
54
+ }
55
+ ```
56
+
57
+ Inform `octocatalog-diff` of the name of the tag either via the command line (`--compare-file-text-ignore-tags "compare-file-text-disable"`) or via a [configuration](/doc/configuration.md) file:
58
+
59
+ ```ruby
60
+ settings[:compare_file_text_ignore_tags] = %w(compare-file-text-disable)
61
+ ```
62
+
63
+ With this example setup, the file text would be compared for `/etc/logrotate.d/my-service.conf` but would NOT be compared for `/etc/logrotate.d/other-service.conf`.
64
+
65
+ Notes:
66
+
67
+ 1. `--compare-file-text-ignore-tags` can take comma-separated arguments if there are multiple tags, e.g.: `--compare-file-text-ignore-tags tag1,tag2`.
68
+
69
+ 1. When defining values in the configuration file, `settings[:compare_file_text_ignore_tags]` must be an array. (The default value is an empty array, `[]`.)
70
+
71
+ 1. "compare-file-text-disable" is used as the name of the tag in the example above, but it is not a magical value. Any valid tag name can be used.
72
+
73
+ ## Notes
74
+
75
+ 1. If the underlying file source cannot be found when building the "to" catalog, an exception will be raised. This will display the resource type and title, the value of `source` that is invalid, and the file name and line number of the Puppet manifest that declared the resource.
76
+
77
+ 1. If the underlying file source cannot be found when building the "from" catalog, no exception will be raised. This behavior allows `octocatalog-diff` to work correctly even if there is a problem in the "from" catalog that is being corrected in the "to" catalog. (Of course, if the underlying file source can't be found in the "to" catalog either, an exception is raised -- see #1.)
78
+
79
+ 1. No processing takes place for file `source` whose values do not start with `puppet:///modules/`.
@@ -15,13 +15,13 @@ If you are using environment names to control the behavior of Puppet, this defau
15
15
  When you supply the command line argument `--preserve-environments` (or set `settings[:preserve_environments] = true` in your [configuration file](/doc/configuration.md)), `octocatalog-diff` will instead do the following:
16
16
 
17
17
  1. Create a temporary directory
18
- 2. Create the following symlinks from `<temporary directory>` to the corresponding directories in the checkout of your code:
18
+ 1. Create the following symlinks from `<temporary directory>` to the corresponding directories in the checkout of your code:
19
19
 
20
- - `environments`
21
- - `manifests`
22
- - `modules`
20
+ - `environments`
21
+ - `manifests`
22
+ - `modules`
23
23
 
24
- 3. Run Puppet using an environment you specify via the command line
24
+ 1. Run Puppet using an environment you specify via the command line
25
25
 
26
26
  Note that you must have set `--preserve-environments` in order for the `--environment` and/or `--create-symlinks` options (described below) to have any effect.
27
27
 
@@ -4,15 +4,15 @@
4
4
 
5
5
  Please note the following caveats:
6
6
 
7
- 0. This method will put some load on your Puppet Master to build the catalog. Depending on how you use `octocatalog-diff` you should ensure that this extra load will not overwhelm your Puppet Master (especially if you create a "thundering herd" by launching several instances of `octocatalog-diff` simultaneously).
7
+ 1. This method will put some load on your Puppet Master to build the catalog. Depending on how you use `octocatalog-diff` you should ensure that this extra load will not overwhelm your Puppet Master (especially if you create a "thundering herd" by launching several instances of `octocatalog-diff` simultaneously).
8
8
 
9
- 0. You will need to deploy your Puppet code to an environment on your Puppet Master prior to running `octocatalog-diff` for that environment. `octocatalog-diff` does not deploy code for you.
9
+ 1. You will need to deploy your Puppet code to an environment on your Puppet Master prior to running `octocatalog-diff` for that environment. `octocatalog-diff` does not deploy code for you.
10
10
 
11
- 0. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below. If you are using Puppet Enterprise and use
11
+ 1. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below. If you are using Puppet Enterprise and use
12
12
  the Puppet Master v4 API you may also use a Puppet Enterprise RBAC token. The user owning the token will need the "Puppet Server Compile catalogs for remote nodes" permission.
13
13
  See the [PE RBAC Token Authorization](#pe-rbac-token-authorization) section below.
14
14
 
15
- 0. If you are using the v2 or v3 PuppetServer APIs with Octocatalog-Diff to compile catalogs, then those catalogs and facts will be automatically stored in PuppetDB. However, when using the v4 PuppetServer API with Octocatalog-Diff, facts and catalogs are *not* automatically stored in PuppetDB - persistence is optional and may be enabled with the appropriate Octocatalog-Diff CLI flag. If your environment depends on the accuracy of exported resources or facts in PuppetDB, you may wish to upgrade and use the V4 API, to avoid unintentional side-effects.
15
+ 1. If you are using the v2 or v3 PuppetServer APIs with Octocatalog-Diff to compile catalogs, then those catalogs and facts will be automatically stored in PuppetDB. However, when using the v4 PuppetServer API with Octocatalog-Diff, facts and catalogs are *not* automatically stored in PuppetDB - persistence is optional and may be enabled with the appropriate Octocatalog-Diff CLI flag. If your environment depends on the accuracy of exported resources or facts in PuppetDB, you may wish to upgrade and use the V4 API, to avoid unintentional side-effects.
16
16
 
17
17
  ## Command line options
18
18
 
@@ -4,33 +4,33 @@ The most common use of `octocatalog-diff` is to use `puppet` locally to compile
4
4
 
5
5
  In order to successfully use Puppet to compile catalogs:
6
6
 
7
- 0. Puppet must be installed on the system.
7
+ 1. Puppet must be installed on the system.
8
8
 
9
- It is the goal of `octocatalog-diff` to support Puppet version 3.8 and higher, installed via any means supported by Puppet. This includes the [All-In-One agent package](https://docs.puppet.com/puppet/4.0/reference/release_notes.html#all-in-one-packaging) or installed as a Ruby gem.
9
+ It is the goal of `octocatalog-diff` to support Puppet version 3.8 and higher, installed via any means supported by Puppet. This includes the [All-In-One agent package](https://docs.puppet.com/puppet/4.0/reference/release_notes.html#all-in-one-packaging) or installed as a Ruby gem.
10
10
 
11
- By default, `octocatalog-diff` will look for the Puppet binary in several common system locations.
11
+ By default, `octocatalog-diff` will look for the Puppet binary in several common system locations.
12
12
 
13
- For maximum reliability, you can specify the full path to the Puppet binary in the configuration file. For example:
13
+ For maximum reliability, you can specify the full path to the Puppet binary in the configuration file. For example:
14
14
 
15
- ```
16
- ##############################################################################################
17
- # puppet_binary
18
- # This is the full path to the puppet binary on your system. If you don't specify this,
19
- # the tool will just run 'puppet' and hope to find it in your path.
20
- ##############################################################################################
15
+ ```
16
+ ##############################################################################################
17
+ # puppet_binary
18
+ # This is the full path to the puppet binary on your system. If you don't specify this,
19
+ # the tool will just run 'puppet' and hope to find it in your path.
20
+ ##############################################################################################
21
21
 
22
- # settings[:puppet_binary] = '/usr/bin/puppet'
23
- settings[:puppet_binary] = '/opt/puppetlabs/puppet/bin/puppet'
24
- ```
22
+ # settings[:puppet_binary] = '/usr/bin/puppet'
23
+ settings[:puppet_binary] = '/opt/puppetlabs/puppet/bin/puppet'
24
+ ```
25
25
 
26
- 0. Applies if you are using [exported resources](https://docs.puppet.com/puppet/latest/reference/lang_exported.html) from PuppetDB (i.e., the octocatalog-diff `--storeconfigs` option enabled):
26
+ 1. Applies if you are using [exported resources](https://docs.puppet.com/puppet/latest/reference/lang_exported.html) from PuppetDB (i.e., the octocatalog-diff `--storeconfigs` option enabled):
27
27
 
28
- Your Puppet installation must have the `puppetdb-termini` feature available. This feature may not be included by default with the Puppet agent package.
28
+ Your Puppet installation must have the `puppetdb-termini` feature available. This feature may not be included by default with the Puppet agent package.
29
29
 
30
- Consult the [Connecting Puppet masters to PuppetDB](https://docs.puppet.com/puppetdb/latest/connect_puppet_master.html#step-1-install-plug-ins) documentation for instructions on installing the `puppetdb-termini` gem.
30
+ Consult the [Connecting Puppet masters to PuppetDB](https://docs.puppet.com/puppetdb/latest/connect_puppet_master.html#step-1-install-plug-ins) documentation for instructions on installing the `puppetdb-termini` gem.
31
31
 
32
- :warning: Attention Mac OS users: the [documentation](https://docs.puppet.com/puppet/latest/reference/puppet_collections.html#os-x-systems) states:
32
+ :warning: Attention Mac OS users: the [documentation](https://docs.puppet.com/puppet/latest/reference/puppet_collections.html#os-x-systems) states:
33
33
 
34
- > While the puppet-agent package is the only component of a Puppet Collection available on OS X, you can still use Puppet Collections to ensure the version of package-agent you install is compatible with the Puppet Collection powering your infrastructure.
34
+ > While the puppet-agent package is the only component of a Puppet Collection available on OS X, you can still use Puppet Collections to ensure the version of package-agent you install is compatible with the Puppet Collection powering your infrastructure.
35
35
 
36
- Unfortunately this means that you won't be able to enable `--storeconfigs` with the All-In-One Puppet Agent on Mac OS X, unless you manually install a gem-packaged version of `puppetdb-terminus`. The procedure for this is beyond the scope of this documentation.
36
+ Unfortunately this means that you won't be able to enable `--storeconfigs` with the All-In-One Puppet Agent on Mac OS X, unless you manually install a gem-packaged version of `puppetdb-terminus`. The procedure for this is beyond the scope of this documentation.
data/doc/configuration.md CHANGED
@@ -2,51 +2,51 @@
2
2
 
3
3
  `octocatalog-diff` may require configuration to work correctly with your Puppet setup.
4
4
 
5
- 0. Download the [sample configuration file](https://raw.githubusercontent.com/github/octocatalog-diff/master/examples/octocatalog-diff.cfg.rb) and save it into one of the following directories.
5
+ 1. Download the [sample configuration file](https://raw.githubusercontent.com/github/octocatalog-diff/master/examples/octocatalog-diff.cfg.rb) and save it into one of the following directories.
6
6
 
7
- - In the base directory of your Puppet repository checkout (i.e., your current working directory):
7
+ - In the base directory of your Puppet repository checkout (i.e., your current working directory):
8
8
 
9
- ```
10
- .octocatalog-diff.cfg.rb
11
- ```
9
+ ```
10
+ .octocatalog-diff.cfg.rb
11
+ ```
12
12
 
13
- - In your home directory:
13
+ - In your home directory:
14
14
 
15
- ```
16
- $HOME/.octocatalog-diff.cfg.rb
17
- ```
15
+ ```
16
+ $HOME/.octocatalog-diff.cfg.rb
17
+ ```
18
18
 
19
- - In one of the following system locations:
19
+ - In one of the following system locations:
20
20
 
21
- ```
22
- /usr/local/etc/octocatalog-diff.cfg.rb
23
- /opt/puppetlabs/octocatalog-diff/octocatalog-diff.cfg.rb
24
- /etc/octocatalog-diff.cfg.rb
25
- ```
21
+ ```
22
+ /usr/local/etc/octocatalog-diff.cfg.rb
23
+ /opt/puppetlabs/octocatalog-diff/octocatalog-diff.cfg.rb
24
+ /etc/octocatalog-diff.cfg.rb
25
+ ```
26
26
 
27
- Note: If more than one of the above files is present, the first one found will be used (proceeding from top to bottom in that list). If you set an environment variable `OCTOCATALOG_DIFF_CONFIG_FILE` that will supersede all of the above paths, and allow you to specify the configuration file location however you wish.
27
+ Note: If more than one of the above files is present, the first one found will be used (proceeding from top to bottom in that list). If you set an environment variable `OCTOCATALOG_DIFF_CONFIG_FILE` that will supersede all of the above paths, and allow you to specify the configuration file location however you wish.
28
28
 
29
- 0. Open the file in a text editor, and follow the comments within the file to guide yourself through configuration. The configuration file is pure ruby, allowing substantial flexibility in the configuration.
29
+ 1. Open the file in a text editor, and follow the comments within the file to guide yourself through configuration. The configuration file is pure ruby, allowing substantial flexibility in the configuration.
30
30
 
31
- To be minimally functional, you will almost certainly need to define at least the following settings:
31
+ To be minimally functional, you will almost certainly need to define at least the following settings:
32
32
 
33
- - `settings[:hiera_config]` as the absolute or relative path to your hiera configuration file
34
- - `settings[:hiera_path_strip]` as the prefix to strip when munging the hiera configuration file
35
- - `settings[:puppetdb_url]` as the URL to your PuppetDB instance so facts can be obtained
33
+ - `settings[:hiera_config]` as the absolute or relative path to your hiera configuration file
34
+ - `settings[:hiera_path_strip]` as the prefix to strip when munging the hiera configuration file
35
+ - `settings[:puppetdb_url]` as the URL to your PuppetDB instance so facts can be obtained
36
36
 
37
- For more information on these settings:
37
+ For more information on these settings:
38
38
 
39
- - [Configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md)
40
- - [Configuring octocatalog-diff to use ENC](/doc/configuration-enc.md)
41
- - [Configuring octocatalog-diff to use PuppetDB](/doc/configuration-puppetdb.md)
42
- - [Configuring octocatalog-diff to use Puppet](/doc/configuration-puppet.md)
39
+ - [Configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md)
40
+ - [Configuring octocatalog-diff to use ENC](/doc/configuration-enc.md)
41
+ - [Configuring octocatalog-diff to use PuppetDB](/doc/configuration-puppetdb.md)
42
+ - [Configuring octocatalog-diff to use Puppet](/doc/configuration-puppet.md)
43
43
 
44
- 0. Test the configuration, which will indicate the location of the configuration file and validate the contents thereof.
44
+ 1. Test the configuration, which will indicate the location of the configuration file and validate the contents thereof.
45
45
 
46
- ```
47
- octocatalog-diff --config-test
48
- ```
46
+ ```
47
+ octocatalog-diff --config-test
48
+ ```
49
49
 
50
- Note: If you [installed](/doc/installation.md) octocatalog-diff as a gem, the `octocatalog-diff` binary should be in your $PATH and the above command should work correctly. If you installed in a different way, you may need to provide the full path to where the `octocatalog-diff` binary was actually installed.
50
+ Note: If you [installed](/doc/installation.md) octocatalog-diff as a gem, the `octocatalog-diff` binary should be in your $PATH and the above command should work correctly. If you installed in a different way, you may need to provide the full path to where the `octocatalog-diff` binary was actually installed.
51
51
 
52
52
  Now that you have entered your configuration and confirmed proper reading of your configuration file, proceed to [Basic usage](/doc/basic.md) to see if it works!
@@ -58,6 +58,6 @@ end
58
58
 
59
59
  ## Hints for writing an integration test
60
60
 
61
- 0. If your integration test deals only with the calculation or display of differences, and not catalog compilation, and you are compiling catalogs multiple times, prefer `spec_catalog_old` and `spec_catalog_new` to pass in pre-compiled catalogs. This will make the test run faster.
61
+ 1. If your integration test deals only with the calculation or display of differences, and not catalog compilation, and you are compiling catalogs multiple times, prefer `spec_catalog_old` and `spec_catalog_new` to pass in pre-compiled catalogs. This will make the test run faster.
62
62
 
63
- 0. It's a good idea to check the exit code in a test of its own, and then `pending` any subsequent tests if that exit code doesn't match. This way only one test, and not all the tests, will fail if the catalog compilation doesn't work.
63
+ 1. It's a good idea to check the exit code in a test of its own, and then `pending` any subsequent tests if that exit code doesn't match. This way only one test, and not all the tests, will fail if the catalog compilation doesn't work.
data/doc/dev/releasing.md CHANGED
@@ -18,65 +18,65 @@ To test the new version of `octocatalog-diff` in the GitHub Puppet repository, c
18
18
 
19
19
  This section is useful when releasing a new version based on one PR submitted by a contributor. Following this workflow is important so that the contributor gets appropriate credit in the revision history for his or her work.
20
20
 
21
- 0. In your local checkout, start a new branch based off master.
22
- 0. Add the contributor's repository as a remote. For example:
21
+ 1. In your local checkout, start a new branch based off master.
22
+ 1. Add the contributor's repository as a remote. For example:
23
23
 
24
- ```
25
- git remote add octocat https://github.com/octocat/octocatalog-diff.git
26
- ```
24
+ ```
25
+ git remote add octocat https://github.com/octocat/octocatalog-diff.git
26
+ ```
27
27
 
28
- 0. Merge in the contributor's branch into your own. For example:
28
+ 1. Merge in the contributor's branch into your own. For example:
29
29
 
30
- ```
31
- git merge octocat/some-branch
32
- ```
30
+ ```
31
+ git merge octocat/some-branch
32
+ ```
33
33
 
34
- 0. Update `.version` and `doc/CHANGELOG.md` appropriately. In CHANGELOG you should link to the PR submitted by the contributor (not the PR you're creating now).
35
- 0. Commit your changes to `.version` and `doc/CHANGELOG.md`.
36
- 0. If necessary, auto-generate the build documentation, and commit the changes to your branch.
34
+ 1. Update `.version` and `doc/CHANGELOG.md` appropriately. In CHANGELOG you should link to the PR submitted by the contributor (not the PR you're creating now).
35
+ 1. Commit your changes to `.version` and `doc/CHANGELOG.md`.
36
+ 1. If necessary, auto-generate the build documentation, and commit the changes to your branch.
37
37
 
38
- ```
39
- rake doc:build
40
- ```
38
+ ```
39
+ rake doc:build
40
+ ```
41
41
 
42
- 0. Open a Pull Request based on your branch. Confirm that the history is correct, showing the contributor's commits first, and then your commit(s) updating the version file, change log, and/or auto-generated documentation.
43
- 0. Ensure that CI tests are all passing.
44
- 0. Ensure that you've performed "local testing" within GitHub (typically, ~1 day) to confirm the changes.
45
- 0. Merge your PR and delete your branch.
46
- 0. Confirm that the contributor's PR now appears as merged, and any associated issues have been closed.
42
+ 1. Open a Pull Request based on your branch. Confirm that the history is correct, showing the contributor's commits first, and then your commit(s) updating the version file, change log, and/or auto-generated documentation.
43
+ 1. Ensure that CI tests are all passing.
44
+ 1. Ensure that you've performed "local testing" within GitHub (typically, ~1 day) to confirm the changes.
45
+ 1. Merge your PR and delete your branch.
46
+ 1. Confirm that the contributor's PR now appears as merged, and any associated issues have been closed.
47
47
 
48
48
  ## Merging multiple PRs
49
49
 
50
50
  If multiple PRs will constitute a release, it's generally easier to merge each such PR individually, and then create a separate PR afterwards to update the necessary files.
51
51
 
52
- 0. Merge all constituent PRs and ensure that any associated issues have been closed.
53
- 0. Create your own branch based off master.
54
- 0. Update `.version` and `doc/CHANGELOG.md` appropriately. In CHANGELOG you should link to the PR submitted by the contributor (not the PR you're creating now).
55
- 0. Commit your changes to `.version` and `doc/CHANGELOG.md`.
56
- 0. If necessary, auto-generate the build documentation, and commit the changes to your branch.
52
+ 1. Merge all constituent PRs and ensure that any associated issues have been closed.
53
+ 1. Create your own branch based off master.
54
+ 1. Update `.version` and `doc/CHANGELOG.md` appropriately. In CHANGELOG you should link to the PR submitted by the contributor (not the PR you're creating now).
55
+ 1. Commit your changes to `.version` and `doc/CHANGELOG.md`.
56
+ 1. If necessary, auto-generate the build documentation, and commit the changes to your branch.
57
57
 
58
- ```
59
- rake doc:build
60
- ```
58
+ ```
59
+ rake doc:build
60
+ ```
61
61
 
62
- 0. Open a Pull Request based on your branch.
63
- 0. Ensure that CI tests are all passing.
64
- 0. Ensure that you've performed "local testing" within GitHub (typically, ~1 day) to confirm the changes.
65
- 0. Merge your PR and delete your branch.
62
+ 1. Open a Pull Request based on your branch.
63
+ 1. Ensure that CI tests are all passing.
64
+ 1. Ensure that you've performed "local testing" within GitHub (typically, ~1 day) to confirm the changes.
65
+ 1. Merge your PR and delete your branch.
66
66
 
67
67
  ## Releasing
68
68
 
69
69
  Generally, a new release will correspond to a merge to master of one or more Pull Requests.
70
70
 
71
- 0. Ensure that all changes associated with the release have been merged to master.
72
- - Merge all Pull Requests associated with release, including the version number bump, change log update, etc.
73
- - If necessary (for significant changes), complete a Pull Request to update the top-level README file.
74
- 0. Ensure the the master branch is checked out on your system.
75
- 0. Run the release procedure:
71
+ 1. Ensure that all changes associated with the release have been merged to master.
72
+ - Merge all Pull Requests associated with release, including the version number bump, change log update, etc.
73
+ - If necessary (for significant changes), complete a Pull Request to update the top-level README file.
74
+ 1. Ensure the the master branch is checked out on your system.
75
+ 1. Run the release procedure:
76
76
 
77
- ```
78
- rake gem:release
79
- ```
77
+ ```
78
+ rake gem:release
79
+ ```
80
80
 
81
81
  This rake task handles the following:
82
82
 
@@ -8,38 +8,38 @@ This document is intended for people who may not be familiar with git, GitHub, a
8
8
 
9
9
  1. Determine the branch name. If there's an open Pull Request, you can see the branch name near the top of the page.
10
10
 
11
- ![Pull Request branch](/doc/images/pull-request-identify-branch.png)
11
+ ![Pull Request branch](/doc/images/pull-request-identify-branch.png)
12
12
 
13
- 2. Clone the `octocatalog-diff` repository in your home directory. From the command line:
13
+ 1. Clone the `octocatalog-diff` repository in your home directory. From the command line:
14
14
 
15
- ```
16
- cd $HOME
17
- git clone https://github.com/github/octocatalog-diff.git
18
- ```
15
+ ```
16
+ cd $HOME
17
+ git clone https://github.com/github/octocatalog-diff.git
18
+ ```
19
19
 
20
- 3. Change into the directory created by your checkout:
20
+ 1. Change into the directory created by your checkout:
21
21
 
22
- ```
23
- cd $HOME/octocatalog-diff
24
- ```
22
+ ```
23
+ cd $HOME/octocatalog-diff
24
+ ```
25
25
 
26
- 4. Check out the branch you wish to use, filling in the branch name you determined in the first step:
26
+ 1. Check out the branch you wish to use, filling in the branch name you determined in the first step:
27
27
 
28
- ```
29
- git checkout BRANCH_NAME_FROM_STEP_1
30
- ```
28
+ ```
29
+ git checkout BRANCH_NAME_FROM_STEP_1
30
+ ```
31
31
 
32
- 5. Bootstrap the repository to pull in dependencies:
32
+ 1. Bootstrap the repository to pull in dependencies:
33
33
 
34
- ```
35
- ./script/bootstrap
36
- ```
34
+ ```
35
+ ./script/bootstrap
36
+ ```
37
37
 
38
- 6. Optional but recommended - run the test suite:
38
+ 1. Optional but recommended - run the test suite:
39
39
 
40
- ```
41
- rake
42
- ```
40
+ ```
41
+ rake
42
+ ```
43
43
 
44
44
  ## Use
45
45
 
@@ -53,7 +53,7 @@ We've created a wrapper script to make this easier for you.
53
53
  cd /etc/puppetlabs/code
54
54
  ```
55
55
 
56
- 2. Run the `script/octocatalog-diff-wrapper` script from *this* checkout. For example, if you checked out `octocatalog-diff` to your home directory, you could use:
56
+ 1. Run the `script/octocatalog-diff-wrapper` script from *this* checkout. For example, if you checked out `octocatalog-diff` to your home directory, you could use:
57
57
 
58
58
  ```
59
59
  $HOME/octocatalog-diff/script/octocatalog-diff-wrapper <options>
data/doc/installation.md CHANGED
@@ -25,26 +25,26 @@ For general information on installing gems, see: [RubyGems Basics](http://guides
25
25
 
26
26
  To install from source, you'll need a git client and internet access.
27
27
 
28
- 0. Clone the repository
28
+ 1. Clone the repository
29
29
 
30
- ```
31
- git clone https://github.com/github/octocatalog-diff.git
32
- ```
30
+ ```
31
+ git clone https://github.com/github/octocatalog-diff.git
32
+ ```
33
33
 
34
- 0. Bootstrap the repository (this will install dependent gems in the project)
34
+ 1. Bootstrap the repository (this will install dependent gems in the project)
35
35
 
36
- ```
37
- cd octocatalog-diff
38
- ./script/bootstrap
39
- ```
36
+ ```
37
+ cd octocatalog-diff
38
+ ./script/bootstrap
39
+ ```
40
40
 
41
- 0. RECOMMENDED: Make sure the tests pass on your machine
41
+ 1. RECOMMENDED: Make sure the tests pass on your machine
42
42
 
43
- ```
44
- rake
45
- ```
43
+ ```
44
+ rake
45
+ ```
46
46
 
47
- Note: If tests fail on your machine with a clean checkout of the master branch, we would definitely appreciate if you would report it. Please [open an issue](https://github.com/github/octocatalog-diff/issues/new) with the output and some information about your system (e.g. OS, ruby version, etc.) to let us know.
47
+ Note: If tests fail on your machine with a clean checkout of the master branch, we would definitely appreciate if you would report it. Please [open an issue](https://github.com/github/octocatalog-diff/issues/new) with the output and some information about your system (e.g. OS, ruby version, etc.) to let us know.
48
48
 
49
49
  Once the code is downloaded and bootstrapped, please proceed to [Configuration](/doc/configuration.md).
50
50
 
data/doc/optionsref.md CHANGED
@@ -74,6 +74,7 @@ Usage: octocatalog-diff [command line options]
74
74
  --from-enc PATH Path to ENC script (for the from catalog only)
75
75
  --to-enc PATH Path to ENC script (for the to catalog only)
76
76
  --[no-]display-detail-add Display parameters and other details for added resources
77
+ --[no-]use-lcs Use the LCS algorithm to determine differences in arrays
77
78
  --[no-]truncate-details Truncate details with --display-detail-add
78
79
  --no-header Do not print a header
79
80
  --default-header Print default header with output
@@ -184,6 +185,8 @@ Usage: octocatalog-diff [command line options]
184
185
  --no-ignore-tags Disable ignoring based on tags
185
186
  --ignore-tags STRING1[,STRING2[,...]]
186
187
  Specify tags to ignore
188
+ --compare-file-text-ignore-tags STRING1[,STRING2[,...]]
189
+ Tags that exclude a file resource from text comparison
187
190
  --[no-]preserve-environments Enable or disable environment preservation
188
191
  --environment STRING Environment for catalog compilation globally
189
192
  --to-environment STRING Environment for catalog compilation for the to branch
@@ -373,6 +376,21 @@ what is most often desired. (<a href="../lib/octocatalog-diff/cli/options/compar
373
376
  </td>
374
377
  </tr>
375
378
 
379
+ <tr>
380
+ <td valign=top>
381
+ <pre><code>--compare-file-text-ignore-tags STRING1[,STRING2[,...]]</code></pre>
382
+ </td>
383
+ <td valign=top>
384
+ Tags that exclude a file resource from text comparison
385
+ </td>
386
+ <td valign=top>
387
+ When a file is specified with `source => 'puppet:///modules/something/foo.txt'`, remove
388
+ the 'source' attribute and populate the 'content' attribute with the text of the file.
389
+ This allows for a diff of the content, rather than a diff of the location, which is
390
+ what is most often desired. (<a href="../lib/octocatalog-diff/cli/options/compare_file_text.rb">compare_file_text.rb</a>)
391
+ </td>
392
+ </tr>
393
+
376
394
  <tr>
377
395
  <td valign=top>
378
396
  <pre><code>--create-symlinks STRING1[,STRING2[,...]]</code></pre>
@@ -1896,6 +1914,19 @@ has no effect when `--display-detail-add` is not used. (<a href="../lib/octocata
1896
1914
  </td>
1897
1915
  </tr>
1898
1916
 
1917
+ <tr>
1918
+ <td valign=top>
1919
+ <pre><code>--use-lcs
1920
+ --no-use-lcs </code></pre>
1921
+ </td>
1922
+ <td valign=top>
1923
+ Use the LCS algorithm to determine differences in arrays
1924
+ </td>
1925
+ <td valign=top>
1926
+ Configures using the Longest common subsequence (LCS) algorithm to determine differences in arrays (<a href="../lib/octocatalog-diff/cli/options/use_lcs.rb">use_lcs.rb</a>)
1927
+ </td>
1928
+ </tr>
1929
+
1899
1930
  <tr>
1900
1931
  <td valign=top>
1901
1932
  <pre><code>--validate-references
@@ -535,9 +535,11 @@ module OctocatalogDiff
535
535
  catalog2_resources = catalog2_in[:catalog]
536
536
 
537
537
  @logger.debug "Entering hashdiff_initial; catalog sizes: #{catalog1_resources.size}, #{catalog2_resources.size}"
538
+ use_lcs = @opts.fetch(:use_lcs, true)
539
+ @logger.debug "HashDiff configuration: (use_lcs: #{use_lcs})"
538
540
  result = []
539
541
  hashdiff_add_remove = Set.new
540
- hashdiff_result = HashDiff.diff(catalog1_resources, catalog2_resources, delimiter: "\f")
542
+ hashdiff_result = HashDiff.diff(catalog1_resources, catalog2_resources, delimiter: "\f", use_lcs: use_lcs)
541
543
  hashdiff_result.each do |obj|
542
544
  # Regular change
543
545
  if obj[0] == '~'
@@ -13,9 +13,16 @@ module OctocatalogDiff
13
13
  # Public method: Convert file resources to text. See the description of the class
14
14
  # just above for details.
15
15
  # @param obj [OctocatalogDiff::Catalog] Catalog object (will be modified)
16
+ # @param environment [String] Environment (defaults to production)
16
17
  def self.convert_file_resources(obj, environment = 'production')
17
18
  return unless obj.valid? && obj.compilation_dir.is_a?(String) && !obj.compilation_dir.empty?
18
- _convert_file_resources(obj.resources, obj.compilation_dir, environment)
19
+ _convert_file_resources(
20
+ obj.resources,
21
+ obj.compilation_dir,
22
+ environment,
23
+ obj.options[:compare_file_text_ignore_tags],
24
+ obj.options[:tag]
25
+ )
19
26
  begin
20
27
  obj.catalog_json = ::JSON.generate(obj.catalog)
21
28
  rescue ::JSON::GeneratorError => exc
@@ -70,8 +77,11 @@ module OctocatalogDiff
70
77
  # Internal method: Static method to convert file resources. The compilation directory is
71
78
  # required, or else this is a no-op. The passed-in array of resources is modified by this method.
72
79
  # @param resources [Array<Hash>] Array of catalog resources
73
- # @param compilation_dir [String] Compilation directory (so files can be looked up)
74
- def self._convert_file_resources(resources, compilation_dir, environment = 'production')
80
+ # @param compilation_dir [String] Compilation directory
81
+ # @param environment [String] Environment
82
+ # @param ignore_tags [Array<String>] Tags that exempt a resource from conversion
83
+ # @param to_from_tag [String] Either "to" or "from" for catalog type
84
+ def self._convert_file_resources(resources, compilation_dir, environment, ignore_tags, to_from_tag)
75
85
  # Calculate compilation directory. There is not explicit error checking here because
76
86
  # there is on-demand, explicit error checking for each file within the modification loop.
77
87
  return unless compilation_dir.is_a?(String) && compilation_dir != ''
@@ -88,14 +98,10 @@ module OctocatalogDiff
88
98
  resources.map! do |resource|
89
99
  if resource_convertible?(resource)
90
100
  path = file_path(resource['parameters']['source'], modulepaths)
91
- if path.nil?
92
- # Pass this through as a wrapped exception, because it's more likely to be something wrong
93
- # in the catalog itself than it is to be a broken setup of octocatalog-diff.
94
- message = "Errno::ENOENT: Unable to resolve '#{resource['parameters']['source']}'!"
95
- raise OctocatalogDiff::Errors::CatalogError, message
96
- end
97
101
 
98
- if File.file?(path)
102
+ if resource['tags'] && ignore_tags && (resource['tags'] & ignore_tags).any?
103
+ # Resource tagged not to be converted -- do nothing.
104
+ elsif path && File.file?(path)
99
105
  # If the file is found, read its content. If the content is all ASCII, substitute it into
100
106
  # the 'content' parameter for easier comparison. If not, instead populate the md5sum.
101
107
  # Delete the 'source' attribute as well.
@@ -103,23 +109,40 @@ module OctocatalogDiff
103
109
  is_ascii = content.force_encoding('UTF-8').ascii_only?
104
110
  resource['parameters']['content'] = is_ascii ? content : '{md5}' + Digest::MD5.hexdigest(content)
105
111
  resource['parameters'].delete('source')
106
- elsif File.exist?(path)
112
+ elsif path && File.exist?(path)
107
113
  # We are not handling recursive file installs from a directory or anything else.
108
114
  # However, the fact that we found *something* at this location indicates that the catalog
109
115
  # is probably correct. Hence, the very general .exist? check.
116
+ elsif to_from_tag == 'from'
117
+ # Don't raise an exception for an invalid source in the "from"
118
+ # catalog, because the developer may be fixing this in the "to"
119
+ # catalog. If it's broken in the "to" catalog as well, the
120
+ # exception will be raised when this code runs on that catalog.
110
121
  else
111
- # This is probably a bug
112
- # :nocov:
113
- raise "Unable to find '#{resource['parameters']['source']}' at #{path}!"
114
- # :nocov:
122
+ # Pass this through as a wrapped exception, because it's more likely to be something wrong
123
+ # in the catalog itself than it is to be a broken setup of octocatalog-diff.
124
+ #
125
+ # Example error: <OctocatalogDiff::Errors::CatalogError: Unable to resolve
126
+ # source=>'puppet:///modules/test/tmp/bar' in File[/tmp/bar]
127
+ # (/x/modules/test/manifests/init.pp:46)>
128
+ source = resource['parameters']['source']
129
+ type = resource['type']
130
+ title = resource['title']
131
+ file = resource['file'].sub(Regexp.new('^' + Regexp.escape(env_dir) + '/'), '')
132
+ line = resource['line']
133
+ message = "Unable to resolve source=>'#{source}' in #{type}[#{title}] (#{file}:#{line})"
134
+ raise OctocatalogDiff::Errors::CatalogError, message
115
135
  end
116
136
  end
137
+
117
138
  resource
118
139
  end
119
140
  end
120
141
 
121
142
  # Internal method: Determine if a resource is convertible. It is convertible if it
122
143
  # is a file resource with no declared 'content' and with a declared and parseable 'source'.
144
+ # It is not convertible if the resource is tagged with one of the tags declared by
145
+ # the option `--compare-file-text-ignore-tags`.
123
146
  # @param resource [Hash] Resource to check
124
147
  # @return [Boolean] True of resource is convertible, false if not
125
148
  def self.resource_convertible?(resource)
@@ -44,7 +44,8 @@ module OctocatalogDiff
44
44
  display_datatype_changes: true,
45
45
  parallel: true,
46
46
  suppress_absent_file_details: true,
47
- hiera_path: 'hieradata'
47
+ hiera_path: 'hieradata',
48
+ use_lcs: true
48
49
  }.freeze
49
50
 
50
51
  # This method is the one to call externally. It is possible to specify alternate
@@ -15,3 +15,21 @@ OctocatalogDiff::Cli::Options::Option.newoption(:compare_file_text) do
15
15
  end
16
16
  end
17
17
  end
18
+
19
+ # Sometimes there is a particular file resource for which the file text
20
+ # comparison is not desired, while not disabling that option globally. Similar
21
+ # to --ignore_tags, it's possible to tag the file resource and exempt it from
22
+ # the --compare_file_text checks.
23
+ # @param parser [OptionParser object] The OptionParser argument
24
+ # @param options [Hash] Options hash being constructed; this is modified in this method.
25
+ OctocatalogDiff::Cli::Options::Option.newoption(:compare_file_text_ignore_tags) do
26
+ has_weight 415
27
+
28
+ def parse(parser, options)
29
+ description = 'Tags that exclude a file resource from text comparison'
30
+ parser.on('--compare-file-text-ignore-tags STRING1[,STRING2[,...]]', Array, description) do |x|
31
+ options[:compare_file_text_ignore_tags] ||= []
32
+ options[:compare_file_text_ignore_tags].concat x
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,14 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Configures using the Longest common subsequence (LCS) algorithm to determine differences in arrays
4
+ # @param parser [OptionParser object] The OptionParser argument
5
+ # @param options [Hash] Options hash being constructed; this is modified in this method.
6
+ OctocatalogDiff::Cli::Options::Option.newoption(:use_lcs) do
7
+ has_weight 250
8
+
9
+ def parse(parser, options)
10
+ parser.on('--[no-]use-lcs', 'Use the LCS algorithm to determine differences in arrays') do |x|
11
+ options[:use_lcs] = x
12
+ end
13
+ end
14
+ end
@@ -14,8 +14,19 @@ module OctocatalogDiff
14
14
  # @return [Hash] Facts
15
15
  def self.fact_retriever(options = {}, node = '')
16
16
  facts = ::JSON.parse(options.fetch(:fact_file_string))
17
- node = facts.fetch('fqdn', 'unknown.node') if node.empty?
18
- { 'name' => node, 'values' => facts }
17
+
18
+ if facts.keys.include?('name') && facts.keys.include?('values') && facts['values'].is_a?(Hash)
19
+ # If you saved the output of something like
20
+ # `puppet facts find $(hostname)` the structure will already be a
21
+ # {'name' => <fqdn>, 'values' => <hash of facts>}. We do nothing
22
+ # here because we don't want to double-encode.
23
+ else
24
+ facts = { 'name' => node, 'values' => facts }
25
+ end
26
+
27
+ facts['name'] = node unless node.empty?
28
+ facts['name'] = facts['values'].fetch('fqdn', 'unknown.node') if facts['name'].empty?
29
+ facts
19
30
  end
20
31
  end
21
32
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: octocatalog-diff
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.0.0
4
+ version: 2.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - GitHub, Inc.
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2021-01-12 00:00:00.000000000 Z
12
+ date: 2021-02-18 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: diffy
@@ -214,6 +214,7 @@ files:
214
214
  - doc/advanced-catalog-only.md
215
215
  - doc/advanced-catalog-validation.md
216
216
  - doc/advanced-ci.md
217
+ - doc/advanced-compare-file-text.md
217
218
  - doc/advanced-dynamic-ignores.md
218
219
  - doc/advanced-environment-variables.md
219
220
  - doc/advanced-environments.md
@@ -380,6 +381,7 @@ files:
380
381
  - lib/octocatalog-diff/cli/options/suppress_absent_file_details.rb
381
382
  - lib/octocatalog-diff/cli/options/to_from_branch.rb
382
383
  - lib/octocatalog-diff/cli/options/truncate_details.rb
384
+ - lib/octocatalog-diff/cli/options/use_lcs.rb
383
385
  - lib/octocatalog-diff/cli/options/validate_references.rb
384
386
  - lib/octocatalog-diff/cli/printer.rb
385
387
  - lib/octocatalog-diff/errors.rb