octocatalog-diff 0.5.1

data/doc/advanced-output-formats.md

@@ -0,0 +1,96 @@
+# Output formats
+
+By default, when you run `octocatalog-diff`, it will display the results in color on STDOUT.
+
+An example of the output display is shown on the [main page](/README.md#example).
+
+Log messages will be written to STDERR.
+
+## Writing to a file
+
+If you would like to write the results to a file instead, specify:
+
+      octocatalog-diff -o OUTPUT_FILE_PATH [other options]
+
+When you output to a file, the default output format is the same as the on-screen format, but without the color.
+
+If you would like to redirect the debug/log output to a file, you can use shell redirection:
+
+      octocatalog-diff [options] 2>/var/log/octocatalog-diff.log
+
+## Alternate output formats
+
+You can specify the `--output-format FORMAT` command line options with one of the following values for FORMAT:
+
+| FORMAT | Explanation |
+| ------ | ----------- |
+| text   | (Default) Traditional human readable output |
+| json   | JSON output of the difference array |
+
+## Color
+
+As previously noted, color text is enabled for on-screen display and disabled for file output.
+
+You can override the automatic selection via:
+
+- `--color` to turn colored text on
+- `--no-color` to turn colored text off
+
+Note: There is no color used for JSON output, regardless of whether the display is on-screen or redirected to a file, and regardless of whether you specify the `--color` command line option.
+
+## JSON format
+
+### Top level format
+
+The JSON result is in the following format:
+
+```
+{
+  "diff": [
+    DIFFERENCE_ARRAY_1,
+    DIFFERENCE_ARRAY_2,
+    ...
+  ],
+  "header": "HEADER STRING"
+}
+```
+
+### Format of difference arrays
+
+Each difference array is in one of the following formats:
+
+#### Addition or removal
+
+```
+[
+  (String) Change Type (+ or -),
+  (String) Type, Title separated by \f,
+  (Hash) Object as it exists in old or new catalog,
+  (Hash) { file: "MANIFEST FILENAME", line: LINE_NUMBER }
+]
+```
+
+When the change type is '+' this indicates that the resource was added (i.e., it exists in the new catalog and not the old). '-' indicates that the resource was removed (i.e., it exists in the old catalog, but not the new). It is important to note that a removed resource does not necessarily cause cleanup on the target system.
+
+#### Change
+
+```
+[
+  (String) Change Type (~ or !),
+  (String) Type, Title, Attribute separated by \f,
+  (?) Object as it exists in old catalog,
+  (?) Object as it exists in new catalog,
+  (Hash) { file: "MANIFEST FILENAME", line: LINE_NUMBER } from old catalog,
+  (Hash) { file: "MANIFEST FILENAME", line: LINE_NUMBER } from new catalog
+]
+```
+
+Note that `~` and `!` are used internally to signify different types of changes, but for display purposes they should be considered equivalent.
+
+The objects as they appear in the old and new catalogs can be many different data types. Strings, integers, and booleans are the most common. `null` represents that the attribute was not present in the catalog or that it was set to Puppet's `undef`. (`null` in JSON is equivalent to Ruby's `nil`.)
+
+### Notes
+
+- In the second array field, the type, title, and attribute are separated by the form feed (`\f`) character. This was chosen because it is unlikely to be encountered in actual naming. You are guaranteed that when splitting on `\f`, the first item is the type, the second item is the title, and the third (and subsequent) items represent the attribute, with each key in the data structure also separated by `\f`.
+
+- If the manifest name and line number are not reported in the catalog, the hashes may have `nil` values for the file and line keys.

data/doc/advanced-output-hacks.md

@@ -0,0 +1,45 @@
+# Output hacks
+
+This document describes command line options that are useful to format the output of the human readable text format (which is the default). Note that if you are [outputting in JSON](/doc/advanced-output-formats.md#json-format), none of these options will have any effect.
+
+See also [Output formats](/doc/advanced-output-formats.md) for details on text vs. JSON and color vs. non-color.
+
+## Displaying the detail of added resources (`--display-detail-add`)
+
+By default, `octocatalog-diff` will display an added resource on one line, without displaying all of the parameters. For example:
+
+```
++ File[/tmp/my-file]
+```
+
+If you would like to see the detail (all parameters and other settings), provide the `--display-detail-add` command line argument. Then the display will look more like:
+
+```
++ File[/tmp/my-file] =>
+   parameters =>
+     "content": "This is my amazing new file",
+     "ensure": "file",
+     "group": "root",
+     "mode": "0755",
+     "owner": "root"
+```
+
+If any line is too long, it will be abbreviated with `...`.
+
+## Displaying the file and line giving rise to a resource (`--display-source`)
+
+To get help tracking down the file and line giving rise to a resource, provide the `--display-source` command line argument.
+
+If the file name and line number are known in the catalog, they will be displayed with the resource.
+
+If this information is the same between the old and new catalogs, it will be displayed just once. If the information differs, it will be displayed twice in a `diff` format: the `-` represents the location in the old catalog, and `+` represents the location in the new catalog.
+
+## Tuning the level of debugging (`--debug`, `--quiet`)
+
+`octocatalog-diff` comes with 3 levels of log output. From least to most:
+
+| Command line option | Description |
+| ------------------- | ----------- |
+| `--quiet` (or `-q`) | Only critical errors are displayed to STDERR |
+| (default)           | Critical errors and informational messages (statistics mainly) are displayed to STDERR |
+| `--debug` (or `-d`) | Full debugging messages are displayed to STDERR |

data/doc/advanced-override-facts.md

@@ -0,0 +1,67 @@
+# Overriding facts
+
+One powerful feature of `octocatalog-diff` allows you to override facts when compiling the catalogs, to predict the effect of a fact change on the catalog. This is useful to simulate a change in agent node configuration without actually setting up an agent to do so.
+
+## Usage
+
+To override a fact in the "to" catalog:
+
+```
+--to-fact-override factname=value
+```
+
+To override a fact in the "from" catalog:
+
+```
+--from-fact-override factname=value
+```
+
+You may use as many of these arguments as you wish to adjust as many facts as you wish.
+
+## Examples
+
+Simulate a change to its IP address in the "to" branch:
+
+```
+octocatalog-diff -n some-node.example.com -f master -t master \
+  --to-fact-override ipaddress=10.0.0.1
+```
+
+Simulate a change in operating system version (in this case, from Ubuntu trusty to xenial):
+
+```
+octocatalog-diff -n some-node.example.com -f master -t master \
+  --from-fact-override lsbdistcodename=trusty --to-fact-override lsbdistcodename=xenial
+```
+
+Simulate changes to multiple facts, in this case the effect of moving a physical machine into EC2:
+
+```
+octocatalog-diff -n some-node.example.com -f master -t master \
+  --to-fact-override ec2=true --to-fact-override ec2_ami_id=ami-abcdef01 \
+  --to-fact-override ec2_hostname=ip-172-16-0-1.internal --to-fact-override ec2_instance_id=i-ba987654 \
+  --to-fact-override ec2_instance_type=c4.2xlarge ...... \
+  --to-fact-override virtual=xenhvm
+```
+
+Note that each of the examples specified the from branch and to branch to be `master`. There is no requirement that you do this, but you can generally obtain the most accurate test results by changing only one variable at a time.
+
+## Advanced usage
+
+The `octocatalog-diff` parser will attempt to guess the data type based on the input. However, you can force the data type using the following syntax:
+
+```
+octocatalog-diff -n some-node.example.com -f master -t master \
+  --to-fact-override some_fact='(string)42' fact_to_delete='(nil)'
+```
+
+The following data types in parentheses are supported:
+
+| Data type in parentheses | Description |
+| ------------------------ | ------------|
+| `(string)` | Treat the input as a string |
+| `(fixnum)` | Treat the input as an integer (calls the `.to_i` method in ruby) |
+| `(float)` | Treat the input as an integer (calls the `.to_f` method in ruby) |
+| `(json)` | Treat the input as a JSON string (calls `JSON.parse` in ruby) |
+| `(boolean)` | Treat the input as a boolean -- it must be `true` or `false`, case-insensitive |
+| `(nil)` | Ignore any characters after `(nil)` and deletes the fact if the fact exists |

data/doc/advanced-pe-enc.md

@@ -0,0 +1,52 @@
+# Puppet Enterprise node classification service
+
+If you are using Puppet Enterprise, `octocatalog-diff` can use the node classifier service API instead of an external node classifier.
+
+## Basics
+
+To use the Puppet Enterprise node classifier service instead of an ENC, you must supply the URL to your node classifier service API endpoint, and either an authentication token or a whilelisted SSL client keypair. It is recommended to supply the SSL Certificate Authority (CA) file as well, to verify the identity of the server to which you are connecting.
+
+To use Puppet Enterprise node classifier service with an authentication token:
+
+```
+octocatalog-diff \
+  --pe-enc-url https://your.pe.console.server:4433/classifier-api \
+  --pe-enc-token-file /path/to/token/file.txt \
+  --pe-enc-ssl-ca /path/to/ca.crt \
+  [other options]
+```
+
+To use Puppet Enterprise node classifier service with a whitelisted SSL client keypair:
+
+```
+octocatalog-diff \
+  --pe-enc-url https://your.pe.console.server:4433/classifier-api \
+  --pe-enc-ssl-ca /path/to/ca.crt \
+  --pe-enc-ssl-client-cert /path/to/client.crt \
+  --pe-enc-ssl-client-key /path/to/client.key \
+  [other options]
+```
+
+## Details
+
+### Requirements
+
+- Your PE console server must be accessible to the machine from which you are running `octocatalog-diff` on the appropriate port (by default, 4433). This port is *not* required to run Puppet agents, so it's possible that you have it firewalled off.
+
+### Authentication token
+
+Please see [Authentication token](https://docs.puppet.com/pe/latest/nc_forming_requests.html#authentication-token) in the official Puppet documentation.
+
+The referenced document contains links to generate a token with the `puppet-access` command.
+
+Note that if you wish to hard-code an authentication token in your [configuration file](/doc/configuration.md), the internal variable key is `:pe_enc_token` and the content is a string containing the entire token. (The `--pe-enc-token-file` option simply reads the provided file and stores the content in the `:pe_enc_token` key. See [source](/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_token_file.rb).)
+
+### SSL client keypair
+
+If you wish to use a SSL client keypair instead of a token, please see [Whilelisted certificate](https://docs.puppet.com/pe/latest/nc_forming_requests.html#whitelisted-certificate) in the official Puppet documentation.
+
+The referenced document contains instructions to add the certificate name to the whitelist file and restart the necessary services.
+
+### Further reading
+
+[Puppet documentation on node classifier endpoints](https://docs.puppet.com/pe/latest/nc_index.html)

data/doc/advanced-puppet-master.md

@@ -0,0 +1,50 @@
+# Fetching catalogs from Puppet Master / PuppetServer
+
+`octocatalog-diff` can fetch catalogs from a Puppet Master, PuppetServer, or Puppet Enterprise server by calling their HTTPS API, just as a node would when it fetches its catalog. For simplicity, this document will refer only to "Puppet Master" but unless otherwise noted, the instructions apply equally to the open source PuppetServer and Puppet Enterprise PuppetServer as well.
+
+Please note the following caveats:
+
+0. This method will put some load on your Puppet Master to build the catalog. Depending on how you use `octocatalog-diff` you should ensure that this extra load will not overwhelm your Puppet Master (especially if you create a "thundering herd" by launching several instances of `octocatalog-diff` simultaneously).
+
+0. You will need to deploy your Puppet code to an environment on your Puppet Master prior to running `octocatalog-diff` for that environment. `octocatalog-diff` does not deploy code for you.
+
+0. You will need to configure authorization for one or more whitelisted certificates on your Puppet Master. The default permissions allow a node to retrieve its own catalog via the API, but you need a certificate for `octocatalog-diff` that permits it to retrieve any catalog. See the [Certificate authorization](#certificate-authorization) section below.
+
+## Command line options
+
+The following command line options are used to retrieve a catalog from a Puppet Master:
+
+| Option | Description |
+| ------ | ----------- |
+| `-f ENVIRONMENT` | Environment name to use for the "from" catalog |
+| `-t ENVIRONMENT` | Environment name to use for the "to" catalog |
+| `--puppet-master HOSTNAME:PORT | The hostname and port number of the Puppet Master. (By default the port used by Puppet Master is 8140.) |
+| `--puppet-master-api-version VERSION | The API version used by the Puppet Master. API versions 2 and 3 are supported. Puppet Master 3.x uses API version 2, and the PuppetServer for Puppet 4.x uses API version 3. By default, API version 3 is used, so you only need to set this option if you are using Puppet Master 3.x. |
+| `--puppet-master-ssl-ca PATH` | Path to the CA certificate (public portion of certificate only) for your Puppet Master. This file will be on your Puppet Master and all Puppet agents. You can find it by running `puppet config print cacert` on any Puppet-managed host. |
+| `--puppet-master-ssl-client-cert PATH` | Path to the client certificate. Please see the section below on certificate authentication. |
+| `--puppet-master-ssl-client-key PATH` | Path to the client private key. Please see the section below on certificate authentication. |
+
+If you wish to use a different Puppet Master to compile the "to" and "from" catalogs, you may prefix any of the `--puppet-master...` options with `to` or `from`. For example, perhaps you are testing an upgrade from Puppet 3.x to 4.x. You could use:
+
+```
+... --from-puppet-master puppet3-x.yourdomain.com:8140 --from-puppet-master-api-version 2 --to-puppet-master puppet4-x.yourdomain.com:8140 ...
+```
+
+It is possible to "mix and match" catalog generation methods. For example, you could retrieve a "from" catalog from a Puppet Master using `--from-puppet-master` while compiling a "to" catalog from local code. Please note that some enhanced options of `octocatalog-diff`, such as comparing file text instead of file source location, may not be available for all such combinations.
+
+## Certificate authorization
+
+In order to use `octocatalog-diff` you will need to create one or more certificates that are empowered to retrieve all catalogs. This requires both creating the certificate, and reconfiguring your Puppet Master to expand the scope of authorization for that certificate.
+
+Puppet Masters use the [legacy auth.conf file](https://docs.puppet.com/puppet/latest/reference/config_file_auth.html) and/or [PuppetServer auth.conf file](https://docs.puppet.com/puppetserver/latest/config_file_auth.html) to control access to HTTPS API.
+
+In particular, the following entry in the legacy auth.conf permits a particular agent to retrieve its own catalog:
+
+```
+# allow nodes to retrieve their own catalog (ie their configuration)
+path ~ ^/catalog/([^/]+)$
+method find
+allow $1
+```
+
+Please follow the instructions for the version of Puppet Master, PuppetServer, or Puppet Enterprise that you are using in order to generate and authorize the certificates.

data/doc/advanced-puppet-versions.md

@@ -0,0 +1,9 @@
+# Compiling the same catalog with different Puppet versions
+
+`octocatalog-diff` can be a valuable tool when upgrading from one Puppet version to another. By instructing `octocatalog-diff` to compile the "from" catalog with one version of Puppet, and the "to" catalog with another version of Puppet, you can look for changes that arise due to different Puppet versions.
+
+To use this feature, simply point `octocatalog-diff` at the Puppet binary you wish to use for each catalog.
+
+- `--from-puppet-binary DIRECTORY_PATH/puppet` will compile the "from" catalog with the specified Puppet binary
+- `--to-puppet-binary DIRECTORY_PATH/puppet` will compile the "to" catalog with the specified Puppet binary
+- `--puppet-binary DIRECTORY_PATH/puppet` will compile both catalogs with the specified Puppet binary

data/doc/advanced-storeconfigs.md

@@ -0,0 +1,72 @@
+# Enabling storeconfigs for exported resources in PuppetDB
+
+The "storeconfigs" setting in Puppet is a feature related to [exported resources](https://docs.puppet.com/puppet/latest/reference/lang_exported.html).
+
+It is possible to enable the collection of exported resources when `octocatalog-diff` compiles catalogs, to give the most accurate representation possible of the catalogs before they are compared.
+
+## Usage
+
+When you provide the `--storeconfigs` command line option, or set `settings[:storeconfigs] = true` in the [configuration file](/doc/configuration.md), the following behavior is triggered:
+
+  - `octocatalog-diff` will create a `puppetdb.conf` file in its temporary compilation directory, using the [PuppetDB configuration settings](/doc/configuration-puppetdb.md) that you have specified, either as command line parameters or in a configuration file.
+
+  - `octocatalog-diff` will install the SSL client certificates you have provided for PuppetDB, if any, in its temporary compilation directory, so that Puppet will pick these up and use them to connect to PuppetDB. This allows SSL client authentication to PuppetDB. (Please note: Puppet *must* connect to PuppetDB over an SSL connection, although not necessarily an authenticated SSL connection.)
+
+  - `octocatalog-diff` will create a `routes.yaml` file in its temporary compilation directory so that Puppet does not try to send fact data, resource data, or reports back to PuppetDB. We have done our best to make this connection be "read only" although we do encourage you to set up a separate, read-only port for PuppetDB to ensure this.
+
+## Caveats
+
+  - Beware of load this may cause on PuppetDB, especially if you run `octocatalog-diff` simultaneously in a CI environment. At GitHub, we run `octocatalog-diff` distributed across 8 CI nodes, each of which is capable of performing 16 simultaneous catalog compilations. We have noticed performance degradation or outages when a "thundering herd" of 128 catalog compilations hit PuppetDB at the same time, leading us to implement a layer of caching on top of PuppetDB.
+
+  - `octocatalog-diff` compiles a "before" and "after" catalog in mostly parallel fashion, but it is possible that the order of operations happens as follows: (a) "before" catalog compiles; (b) something else writes updated data to PuppetDB; (c) "after" catalog compiles. In this case, there can be false differences reported in the output.
+
+## Advanced configuration
+
+This section contains tips on setting up a proxy in front of PuppetDB to control access and/or enable caching. You are not required to do this in order to use `--storeconfigs` but you may find these tips useful if simultaneous runs of `octocatalog-diff` put too much load on your PuppetDB instance.
+
+### Making a read-only PuppetDB port
+
+It is possible to create a read-only endpoint to PuppetDB by setting up a proxy that only allows the desired URLs. This ensures that the Puppet runs cannot submit fact data, resource data, or reports back to PuppetDB from `octocatalog-diff` runs. (As noted previously, we do set up `routes.yaml` to prevent this, but the strategy in this section provides an extra layer of security.)
+
+To allow only the desired traffic, you should configure a port on your proxy that will pass to these URLs only:
+
+  - /pdb/query
+  - /pdb/meta
+
+### Caching
+
+To reduce the impact of a "thundering herd" of simultaneous `octocatalog-diff` runs, you can set up a caching proxy in front of the `/pdb/query` endpoint.
+
+Here is a portion of the nginx configuration that has provided the best balance between performance and accuracy. We have chosen a 1 minute TTL on results, and incorporated the request body into the cache key. You will need to adjust and incorporate this configuration into your own nginx proxy.
+
+```
+upstream puppetdb {
+    server localhost:8080;
+}
+
+proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=pdb_cache:10m max_size=25g inactive=2m;
+
+server {
+    listen 10.0.0.1:8082;
+
+    # Some SSL settings omitted. Configure as per your own needs.
+    ssl_client_certificate /etc/nginx/ca.crt
+    ssl_verify_client on;
+
+    proxy_cache_key "$scheme$proxy_host$uri$is_args$args|$request_body";
+
+    deny all;
+
+    location /pdb/query {
+        proxy_cache pdb_cache;
+        proxy_ignore_headers Cache-Control;
+        proxy_cache_valid any 1m;
+        proxy_pass http://puppetdb;
+        proxy_redirect off;
+        add_header X-Cache-Status $upstream_cache_status;
+        allow all;
+    }
+
+    # Other settings omitted.
+}
+```

data/doc/advanced-using-without-git.md

@@ -0,0 +1,15 @@
+# Using `octocatalog-diff` without git
+
+`octocatalog-diff` is designed to be used with git, and is developed by GitHub. This means that on a daily basis, it is run thousands of times on repositories backed by GitHub for CI jobs reported to GitHub.
+
+If you do not manage your Puppet code with git, it is still possible to use `octocatalog-diff`, but you will not be able to take advantage of the git integrations and will need to do some things manually.
+
+## Usage
+
+You will need to use these command line options (at minimum):
+
+  - `--bootstrapped-from-dir`: The directory containing your checked out "from" Puppet code, and if there is any [bootstrapping](/doc/advanced-bootstrap.md) it must already be done.
+
+  - `--bootstrapped-to-dir`: The directory containing your checked out "to" Puppet code, and if there is any [bootstrapping](/doc/advanced-bootstrap.md) it must already be done.
+
+You are responsible for the process by which you check out code from your version control system into two separate directories and run any necessary bootstrapping. You will need to script that step *before* invoking `octocatalog-diff`.

data/doc/advanced.md

@@ -0,0 +1,43 @@
+# Advanced usage
+
+With `--octocatalog-diff` supporting over 75 command line options (and counting), there's a little something for everyone. On this page, we document some interesting use cases that can be accomplished with creative combinations of options.
+
+If you find a creative use of `octocatalog-diff` that we haven't thought of, we encourage you to create a document named `advanced-SOMETHING.md` and link to it from here!
+
+See also:
+
+- [Basic usage](/doc/basic.md) - Common use cases to get you started
+- [Command line options reference](/doc/optionsref.md) - A list of *all* the options
+- [How to add new command line options](/doc/dev/how-to-add-options.md) - If you'd like to add an option of your own
+
+## Advanced usage documentation
+
+### Building catalogs
+
+- [Bootstrapping your Puppet checkout](/doc/advanced-bootstrap.md)
+- [Building catalogs instead of diffing catalogs](/doc/advanced-catalog-only.md)
+- [Enabling storeconfigs for exported resources in PuppetDB](/doc/advanced-storeconfigs.md)
+- [Fetching catalogs from Puppet Master / PuppetServer](/doc/advanced-puppet-master.md)
+- [Overriding facts](/doc/advanced-override-facts.md)
+- [Puppet Enterprise node classification service](/doc/advanced-pe-enc.md)
+- [Using `octocatalog-diff` without git](/doc/advanced-using-without-git.md)
+
+### Controlling output
+
+- [Ignoring certain changes via command line options](/doc/advanced-ignores.md)
+- [Dynamic ignoring of changes via tags in Puppet manifests](/doc/advanced-dynamic-ignores.md)
+- [Output formats](/doc/advanced-output-formats.md)
+- [Useful output hacks](/doc/advanced-output-hacks.md)
+
+### Using `octocatalog-diff` in CI
+
+- [Using `octocatalog-diff` in CI](/doc/advanced-ci.md)
+
+### Using `octocatalog-diff` on your workstation
+
+- [Enabling the cache directory](/doc/advanced-cache-dir.md)
+
+### Using `octocatalog-diff` to help you upgrade
+
+- [Compiling the same catalog with different Puppet versions](/doc/advanced-puppet-versions.md)
+- [Enabling the future parser](/doc/advanced-future-parser.md)