octocatalog-diff 0.5.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (143) hide show
  1. checksums.yaml +7 -0
  2. data/.version +1 -0
  3. data/LICENSE +20 -0
  4. data/README.md +82 -0
  5. data/bin/octocatalog-diff +75 -0
  6. data/doc/advanced-bootstrap.md +33 -0
  7. data/doc/advanced-cache-dir.md +24 -0
  8. data/doc/advanced-catalog-only.md +37 -0
  9. data/doc/advanced-ci.md +13 -0
  10. data/doc/advanced-dynamic-ignores.md +123 -0
  11. data/doc/advanced-future-parser.md +11 -0
  12. data/doc/advanced-ignores.md +224 -0
  13. data/doc/advanced-output-formats.md +96 -0
  14. data/doc/advanced-output-hacks.md +45 -0
  15. data/doc/advanced-override-facts.md +67 -0
  16. data/doc/advanced-pe-enc.md +52 -0
  17. data/doc/advanced-puppet-master.md +50 -0
  18. data/doc/advanced-puppet-versions.md +9 -0
  19. data/doc/advanced-storeconfigs.md +72 -0
  20. data/doc/advanced-using-without-git.md +15 -0
  21. data/doc/advanced.md +43 -0
  22. data/doc/basic.md +70 -0
  23. data/doc/configuration-enc.md +69 -0
  24. data/doc/configuration-hiera.md +103 -0
  25. data/doc/configuration-puppetdb.md +49 -0
  26. data/doc/configuration.md +51 -0
  27. data/doc/dev/README.md +1 -0
  28. data/doc/dev/coverage.md +34 -0
  29. data/doc/dev/how-to-add-options.md +83 -0
  30. data/doc/dev/integration-tests.md +63 -0
  31. data/doc/dev/releasing.md +19 -0
  32. data/doc/installation.md +49 -0
  33. data/doc/limitations.md +34 -0
  34. data/doc/optionsref.md +947 -0
  35. data/doc/requirements.md +16 -0
  36. data/doc/roadmap.md +26 -0
  37. data/doc/similar.md +17 -0
  38. data/doc/troubleshooting.md +54 -0
  39. data/lib/octocatalog-diff.rb +12 -0
  40. data/lib/octocatalog-diff/bootstrap.rb +53 -0
  41. data/lib/octocatalog-diff/catalog-diff/cli.rb +205 -0
  42. data/lib/octocatalog-diff/catalog-diff/cli/catalogs.rb +240 -0
  43. data/lib/octocatalog-diff/catalog-diff/cli/diffs.rb +145 -0
  44. data/lib/octocatalog-diff/catalog-diff/cli/helpers/fact_override.rb +99 -0
  45. data/lib/octocatalog-diff/catalog-diff/cli/options.rb +173 -0
  46. data/lib/octocatalog-diff/catalog-diff/cli/options/basedir.rb +14 -0
  47. data/lib/octocatalog-diff/catalog-diff/cli/options/bootstrap_environment.rb +18 -0
  48. data/lib/octocatalog-diff/catalog-diff/cli/options/bootstrap_script.rb +14 -0
  49. data/lib/octocatalog-diff/catalog-diff/cli/options/bootstrap_then_exit.rb +12 -0
  50. data/lib/octocatalog-diff/catalog-diff/cli/options/bootstrapped_dirs.rb +18 -0
  51. data/lib/octocatalog-diff/catalog-diff/cli/options/cached_master_dir.rb +21 -0
  52. data/lib/octocatalog-diff/catalog-diff/cli/options/catalog_only.rb +14 -0
  53. data/lib/octocatalog-diff/catalog-diff/cli/options/color.rb +13 -0
  54. data/lib/octocatalog-diff/catalog-diff/cli/options/compare_file_text.rb +15 -0
  55. data/lib/octocatalog-diff/catalog-diff/cli/options/debug.rb +12 -0
  56. data/lib/octocatalog-diff/catalog-diff/cli/options/display_datatype_changes.rb +16 -0
  57. data/lib/octocatalog-diff/catalog-diff/cli/options/display_detail_add.rb +12 -0
  58. data/lib/octocatalog-diff/catalog-diff/cli/options/display_source_file_line.rb +12 -0
  59. data/lib/octocatalog-diff/catalog-diff/cli/options/enc.rb +31 -0
  60. data/lib/octocatalog-diff/catalog-diff/cli/options/existing_catalogs.rb +25 -0
  61. data/lib/octocatalog-diff/catalog-diff/cli/options/fact_file.rb +23 -0
  62. data/lib/octocatalog-diff/catalog-diff/cli/options/fact_override.rb +19 -0
  63. data/lib/octocatalog-diff/catalog-diff/cli/options/facts_terminus.rb +16 -0
  64. data/lib/octocatalog-diff/catalog-diff/cli/options/from_puppetdb.rb +13 -0
  65. data/lib/octocatalog-diff/catalog-diff/cli/options/header.rb +24 -0
  66. data/lib/octocatalog-diff/catalog-diff/cli/options/hiera_config.rb +18 -0
  67. data/lib/octocatalog-diff/catalog-diff/cli/options/hiera_path_strip.rb +12 -0
  68. data/lib/octocatalog-diff/catalog-diff/cli/options/hostname.rb +13 -0
  69. data/lib/octocatalog-diff/catalog-diff/cli/options/ignore.rb +24 -0
  70. data/lib/octocatalog-diff/catalog-diff/cli/options/ignore_attr.rb +16 -0
  71. data/lib/octocatalog-diff/catalog-diff/cli/options/ignore_tags.rb +23 -0
  72. data/lib/octocatalog-diff/catalog-diff/cli/options/include_tags.rb +12 -0
  73. data/lib/octocatalog-diff/catalog-diff/cli/options/master_cache_branch.rb +12 -0
  74. data/lib/octocatalog-diff/catalog-diff/cli/options/output_file.rb +15 -0
  75. data/lib/octocatalog-diff/catalog-diff/cli/options/output_format.rb +15 -0
  76. data/lib/octocatalog-diff/catalog-diff/cli/options/parallel.rb +12 -0
  77. data/lib/octocatalog-diff/catalog-diff/cli/options/parser.rb +48 -0
  78. data/lib/octocatalog-diff/catalog-diff/cli/options/pass_env_vars.rb +19 -0
  79. data/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_ssl_ca.rb +15 -0
  80. data/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_ssl_client_cert.rb +14 -0
  81. data/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_ssl_client_key.rb +14 -0
  82. data/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_token.rb +15 -0
  83. data/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_token_file.rb +17 -0
  84. data/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_url.rb +19 -0
  85. data/lib/octocatalog-diff/catalog-diff/cli/options/puppet_binary.rb +16 -0
  86. data/lib/octocatalog-diff/catalog-diff/cli/options/puppet_master.rb +16 -0
  87. data/lib/octocatalog-diff/catalog-diff/cli/options/puppet_master_api_version.rb +20 -0
  88. data/lib/octocatalog-diff/catalog-diff/cli/options/puppet_master_ssl_ca.rb +19 -0
  89. data/lib/octocatalog-diff/catalog-diff/cli/options/puppet_master_ssl_client_cert.rb +19 -0
  90. data/lib/octocatalog-diff/catalog-diff/cli/options/puppet_master_ssl_client_key.rb +19 -0
  91. data/lib/octocatalog-diff/catalog-diff/cli/options/puppetdb_ssl_ca.rb +15 -0
  92. data/lib/octocatalog-diff/catalog-diff/cli/options/puppetdb_ssl_client_cert.rb +14 -0
  93. data/lib/octocatalog-diff/catalog-diff/cli/options/puppetdb_ssl_client_key.rb +14 -0
  94. data/lib/octocatalog-diff/catalog-diff/cli/options/puppetdb_ssl_client_password.rb +14 -0
  95. data/lib/octocatalog-diff/catalog-diff/cli/options/puppetdb_ssl_client_password_file.rb +13 -0
  96. data/lib/octocatalog-diff/catalog-diff/cli/options/puppetdb_url.rb +18 -0
  97. data/lib/octocatalog-diff/catalog-diff/cli/options/quiet.rb +12 -0
  98. data/lib/octocatalog-diff/catalog-diff/cli/options/retry_failed_catalog.rb +13 -0
  99. data/lib/octocatalog-diff/catalog-diff/cli/options/safe_to_delete_cached_master_dir.rb +15 -0
  100. data/lib/octocatalog-diff/catalog-diff/cli/options/storeconfigs.rb +12 -0
  101. data/lib/octocatalog-diff/catalog-diff/cli/options/suppress_absent_file_details.rb +14 -0
  102. data/lib/octocatalog-diff/catalog-diff/cli/options/to_from_branch.rb +16 -0
  103. data/lib/octocatalog-diff/catalog-diff/cli/printer.rb +52 -0
  104. data/lib/octocatalog-diff/catalog-diff/differ.rb +615 -0
  105. data/lib/octocatalog-diff/catalog-diff/display.rb +125 -0
  106. data/lib/octocatalog-diff/catalog-diff/display/json.rb +25 -0
  107. data/lib/octocatalog-diff/catalog-diff/display/text.rb +452 -0
  108. data/lib/octocatalog-diff/catalog-util/bootstrap.rb +145 -0
  109. data/lib/octocatalog-diff/catalog-util/builddir.rb +289 -0
  110. data/lib/octocatalog-diff/catalog-util/cached_master_directory.rb +169 -0
  111. data/lib/octocatalog-diff/catalog-util/command.rb +96 -0
  112. data/lib/octocatalog-diff/catalog-util/enc.rb +77 -0
  113. data/lib/octocatalog-diff/catalog-util/enc/noop.rb +22 -0
  114. data/lib/octocatalog-diff/catalog-util/enc/pe.rb +99 -0
  115. data/lib/octocatalog-diff/catalog-util/enc/pe/v1.rb +61 -0
  116. data/lib/octocatalog-diff/catalog-util/enc/script.rb +88 -0
  117. data/lib/octocatalog-diff/catalog-util/facts.rb +89 -0
  118. data/lib/octocatalog-diff/catalog-util/fileresources.rb +83 -0
  119. data/lib/octocatalog-diff/catalog-util/git.rb +65 -0
  120. data/lib/octocatalog-diff/catalog.rb +209 -0
  121. data/lib/octocatalog-diff/catalog/computed.rb +205 -0
  122. data/lib/octocatalog-diff/catalog/json.rb +30 -0
  123. data/lib/octocatalog-diff/catalog/noop.rb +19 -0
  124. data/lib/octocatalog-diff/catalog/puppetdb.rb +82 -0
  125. data/lib/octocatalog-diff/catalog/puppetmaster.rb +121 -0
  126. data/lib/octocatalog-diff/external/pson/LICENSE +17 -0
  127. data/lib/octocatalog-diff/external/pson/README.md +20 -0
  128. data/lib/octocatalog-diff/external/pson/common.rb +370 -0
  129. data/lib/octocatalog-diff/external/pson/pure.rb +15 -0
  130. data/lib/octocatalog-diff/external/pson/pure/generator.rb +395 -0
  131. data/lib/octocatalog-diff/external/pson/pure/parser.rb +307 -0
  132. data/lib/octocatalog-diff/external/pson/version.rb +8 -0
  133. data/lib/octocatalog-diff/facts.rb +125 -0
  134. data/lib/octocatalog-diff/facts/json.rb +20 -0
  135. data/lib/octocatalog-diff/facts/puppetdb.rb +59 -0
  136. data/lib/octocatalog-diff/facts/yaml.rb +29 -0
  137. data/lib/octocatalog-diff/puppetdb.rb +163 -0
  138. data/lib/octocatalog-diff/util/colored.rb +20 -0
  139. data/lib/octocatalog-diff/util/httparty.rb +158 -0
  140. data/lib/octocatalog-diff/util/parallel.rb +170 -0
  141. data/lib/octocatalog-diff/util/puppetversion.rb +24 -0
  142. data/lib/octocatalog-diff/version.rb +7 -0
  143. metadata +386 -0
@@ -0,0 +1,70 @@
1
+ # Basic usage
2
+
3
+ The most basic usage of octocatalog-diff is to compare catalogs built from two different git branches, for a node of your choosing.
4
+
5
+ You should be aware of these defaults, all of which are [configurable](/doc/configuration.md).
6
+
7
+ - octocatalog-diff will default to compiling catalogs based on the assumption that your Puppet code resides in a git repository. If your Puppet code does not reside in a git repository, head over to the [advanced instructions](/doc/advanced.md) for workarounds.
8
+
9
+ - octocatalog-diff will compile the catalog produced from the `origin/master` branch of your repository as the "from" catalog, and the catalog produced from your current working directory as the "to" catalog. You can override these defaults with the `-f BRANCH` and `-t BRANCH` arguments, for the "from" and "to" branches, respectively.
10
+
11
+ - octocatalog-diff will assume you are not using hiera or an external node classifier, unless you [configure](/doc/configuration.md) it accordingly, or use the appropriate command line arguments to point it at your hiera configuration and/or ENC script.
12
+
13
+ You are required to provide the following information, either as a command line argument, in the [configuration](/doc/configuration.md), or in some cases, via the environment:
14
+
15
+ - The node name whose catalogs you wish to compile. Use `-n HOSTNAME` on the command line.
16
+
17
+ - Facts, which can either be retrieved from [PuppetDB](/doc/configuration-puppetdb.md) or via the `--fact-file` command line option. See the usage examples below.
18
+
19
+ ## Examples
20
+
21
+ ### From git repository with facts from PuppetDB
22
+
23
+ ```
24
+ export PUPPETDB_URL="http://puppetdb.yourdomain.com:8080"
25
+ cd Puppet_Checkout_Directory
26
+ git checkout master
27
+ git pull
28
+ octocatalog-diff -n SomeNodeName.yourdomain.com
29
+ ```
30
+
31
+ ### Using a fact file
32
+
33
+ You can retrieve the fact file from your Puppet Master (3.x) typically in `/var/lib/puppet/yaml/facts/<node>.yaml`, or your Puppet Server (4.x) typically in `/opt/puppetlabs/server/data/puppetserver/yaml/facts/<node>.yaml`. We recommend using PuppetDB as a more convenient fact source, but you can copy the fact file for a node from your Puppet server onto the machine running octocatalog-diff for testing purposes.
34
+
35
+ ```
36
+ # Copy the fact file for SomeNodeName.yourdomain.com into /tmp/SomeNodeName.yourdomain.com.yaml
37
+ cd Puppet_Checkout_Directory
38
+ git checkout master
39
+ git pull
40
+ octocatalog-diff -n SomeNodeName.yourdomain.com --fact-file /tmp/SomeNodeName.yourdomain.com.yaml
41
+ ```
42
+
43
+ ## Using hiera
44
+
45
+ This example demonstrates how to point octocatalog-diff at your Hiera configuration file. The Hiera configuration file for your site might be found in `/etc/puppet/hiera.yaml` (for Puppet 3.x) or `/etc/puppetlabs/puppet/hiera.yaml` (for Puppet 4.x).
46
+
47
+ Note that you will either need to configure the PuppetDB URL or specify a `--fact-file` for this to work.
48
+
49
+ ```
50
+ # Copy the fact file for SomeNodeName.yourdomain.com into /tmp/SomeNodeName.yourdomain.com.yaml
51
+ # (or)
52
+ # Set the PUPPETDB_URL variable as shown in the first example
53
+ #
54
+ # Also copy hiera.yaml from your Puppet master into the /tmp directory
55
+
56
+ cd Puppet_Checkout_Directory
57
+ git checkout master
58
+ git pull
59
+ octocatalog-diff -n SomeNodeName.yourdomain.com --hiera-config /tmp/hiera.yaml
60
+ ```
61
+
62
+ Depending on your hiera configuration, you may also need to supply the `--hiera-path-strip` option (or set that option in your [configuration](/doc/configuration.md)). Consult the [configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md) document for details on this option.
63
+
64
+ ## Next steps
65
+
66
+ If you're ready to learn about additional command line flags to customize your experience, head to [Advanced usage](/doc/advanced.md).
67
+
68
+ If you experience problems running octocatalog-diff even with these most basic arguments, please see [Troubleshooting](/doc/troubleshooting.md).
69
+
70
+ If you are not using git to manage your Puppet source code, you will need to see the [Advanced usage](/doc/advanced.md) instructions to get your directories manually bootstrapped for use, or use one of the other supported methods to build catalogs.
@@ -0,0 +1,69 @@
1
+ # Configuring octocatalog-diff to use ENC
2
+
3
+ If you use an [External Node Classifier (ENC)](https://docs.puppet.com/guides/external_nodes.html) on your Puppet master, you should configure octocatalog-diff to use the same ENC when it compiles catalogs.
4
+
5
+ Before you start, please understand how octocatalog-diff compiles a catalog:
6
+
7
+ - It creates a temporary directory (e.g. `/var/tmp/puppet-compile-dir-92347829847`)
8
+ - It copies or creates hiera configuration, ENC, PuppetDB configuration, etc., in the temporary directory
9
+ - It symlinks `/var/tmp/puppet-compile-dir-92347829847/environments/production` to your code
10
+ - It compiles the catalog, based on the temporary directory, for environment=production
11
+ - It removes the temporary directory
12
+
13
+ NOTE: If you are using the built-in node classification in Puppet Enterprise, you don't need to worry about any of this. Instead, please read about [Puppet Enterprise as your ENC](/doc/advanced-pe-enc.md).
14
+
15
+ ## Configuring the path to the ENC
16
+
17
+ The command line option `--enc PATH` allows you to set the path to your ENC script.
18
+
19
+ You may specify this as either an absolute or a relative path.
20
+
21
+ - As a relative path
22
+
23
+ octocatalog-diff knows to use a relative path when the supplied path for `--enc` does not start with a `/`.
24
+
25
+ ```
26
+ bin/octocatalog-diff --enc config/enc.sh ...
27
+ ```
28
+
29
+ The path is relative to a checkout of your Puppet repository. As per the example in the introduction, say that octocatalog-diff is using a temporary directory of `/var/tmp/puppet-compile-dir-92347829847` when compiling a Puppet catalog. With the setting above, it will copy `config/enc.sh` (relative to your Puppet checkout) into the temporary directory.
30
+
31
+ If you are specifying the ENC path in the [configuration file](/doc/configuration.md), you will instead set the variable like this:
32
+
33
+ ```
34
+ settings[:enc] = 'config/enc.sh'
35
+ ```
36
+
37
+ octocatalog-diff will fail if you specify a ENC location that cannot be opened.
38
+
39
+ - As an absolute path
40
+
41
+ octocatalog-diff knows to use a relative path when the supplied path for `--enc` starts with a `/`.
42
+
43
+ For example:
44
+
45
+ ```
46
+ bin/octocatalog-diff --enc /etc/puppetlabs/puppet/enc.sh ...
47
+ ```
48
+
49
+ If you are specifying the ENC path in the [configuration file](/doc/configuration.md), you will instead set the variable like this:
50
+
51
+ ```
52
+ settings[:enc] = '/etc/puppetlabs/puppet/enc.sh'
53
+ ```
54
+
55
+ Please note that octocatalog-diff will copy the file from the specified location into the compile directory. Since this ENC file is not copied from your Puppet repo, there is no way to compile the "to" and "from" branches using different ENC scripts. Furthermore, you are responsible for getting this file into place on any machine that will run octocatalog-diff.
56
+
57
+ We strongly recommend that you version-control your ENC script within your Puppet repository, and use the relative path option described above.
58
+
59
+ ## Executing the ENC
60
+
61
+ When Puppet runs the ENC, it will do so with one argument (the node name for which you are compiling the catalog).
62
+
63
+ For example, when compiling the catalog for `some-node.github.net`, Puppet will effectively execute this command:
64
+
65
+ ```
66
+ /etc/puppetlabs/puppet/enc.sh some-node.github.net
67
+ ```
68
+
69
+ Sometimes the ENC script requires credentials or makes other assumptions about the system on which it is running. To be able to run the ENC script on systems other than your Puppet master, you will need to ensure that any such credentials are supplied and other assumptions are met.
@@ -0,0 +1,103 @@
1
+ # Configuring octocatalog-diff to use Hiera
2
+
3
+ If you are using Hiera with Puppet, then you must already have a [`hiera.yaml`](https://docs.puppet.com/puppet/latest/reference/config_file_hiera.html) file to configure it. These instructions will guide you through pointing octocatalog-diff at that configuration file.
4
+
5
+ Before you start, please understand how octocatalog-diff compiles a catalog:
6
+
7
+ - It creates a temporary directory (e.g. `/var/tmp/puppet-compile-dir-92347829847`)
8
+ - It copies or creates hiera configuration, ENC, PuppetDB configuration, etc., in the temporary directory
9
+ - It symlinks `/var/tmp/puppet-compile-dir-92347829847/environments/production` to your code
10
+ - It compiles the catalog, based on the temporary directory, for environment=production
11
+ - It removes the temporary directory
12
+
13
+ ## Configuring the path to hiera.yaml
14
+
15
+ The command line option `--hiera-config PATH` allows you to set the path to hiera.yaml.
16
+
17
+ You may specify this as either an absolute or a relative path.
18
+
19
+ - As a relative path
20
+
21
+ octocatalog-diff knows to use a relative path when the supplied path for `--hiera-config` does not start with a `/`.
22
+
23
+ ```
24
+ bin/octocatalog-diff --hiera-config config/hiera.yaml ...
25
+ ```
26
+
27
+ The path is relative to a checkout of your Puppet repository. As per the example in the introduction, say that octocatalog-diff is using a temporary directory of `/var/tmp/puppet-compile-dir-92347829847` when compiling a Puppet catalog. With the setting above, it will copy `config/hiera.yaml` (relative to your Puppet checkout) into the temporary directory.
28
+
29
+ If you use Puppet to manage your hiera.yaml file on Puppet masters, perhaps it is found in one of the modules in your code. In that case, you may use syntax like:
30
+
31
+ ```
32
+ bin/octocatalog-diff --hiera-config modules/puppet/files/hiera.yaml ...
33
+ ```
34
+
35
+ If you are specifying the hiera.yaml path in the [configuration file](/doc/configuration.md), you will instead set the variable like this:
36
+
37
+ ```
38
+ settings[:hiera_config] = 'config/hiera.yaml'
39
+ settings[:hiera_config] = 'modules/puppet/files/hiera.yaml'
40
+ ```
41
+
42
+ octocatalog-diff will fail if you specify a hiera configuration location that cannot be opened.
43
+
44
+ - As an absolute path
45
+
46
+ octocatalog-diff knows to use a relative path when the supplied path for `--hiera-config` starts with a `/`.
47
+
48
+ For example:
49
+
50
+ ```
51
+ bin/octocatalog-diff --hiera-config /etc/puppetlabs/puppet/hiera.yaml ...
52
+ ```
53
+
54
+ If you are specifying the hiera.yaml path in the [configuration file](/doc/configuration.md), you will instead set the variable like this:
55
+
56
+ ```
57
+ settings[:hiera_config] = '/etc/puppetlabs/puppet/hiera.yaml'
58
+ ```
59
+
60
+ Please note that octocatalog-diff will copy the file from the specified location into the compile directory. Since this hiera.yaml file is not copied from your Puppet repo, there is no way to compile the "to" and "from" branches using different hiera.yaml files. Furthermore, you are responsible for getting this file into place on any machine that will run octocatalog-diff.
61
+
62
+ We strongly recommend that you version-control your hiera.yaml file within your Puppet repository, and use the relative path option described above.
63
+
64
+ ## Configuring the prefix path to strip
65
+
66
+ The command line option `--hiera-path-strip PATH` allows you to manipulate directory paths for the JSON or YAML hiera backends. This setting only has an effect on the copy of hiera.yaml that is copied into the temporary compilation directory. This does not make any changes to the actual source hiera.yaml file on your system or in your checkout.
67
+
68
+ For example, perhaps your production hiera.yaml file has entries such as the following:
69
+
70
+ ```
71
+ :backends:
72
+ - yaml
73
+ :yaml:
74
+ :datadir: /var/lib/puppet/environments/%{::environment}/hieradata
75
+ :hierarchy:
76
+ - servers/%{::fqdn}
77
+ - platform/%{::virtual}
78
+ - datacenter/%{::datacenter}
79
+ - os/%{::operatingsystem}
80
+ - common
81
+ ```
82
+
83
+ However, when you run octocatalog-diff, the hiera data will not actually be found in `/var/lib/puppet/environments/production/hieradata`, but rather in a directory called `environments/production/hieradata` relative to the checkout of your Puppet code.
84
+
85
+ Specifying `--hiera-path-strip PATH` causes octocatalog-diff will rewrite the datadir for the YAML and JSON configuration. In the example above, the correct setting is `--hiera-path-strip /var/lib/puppet`, which will result in the following configuration in the hiera.yaml file:
86
+
87
+ ```
88
+ bin/octocatalog-diff --hiera-config environments/production/config/hiera.yaml --hiera-path-strip /var/lib/puppet
89
+ ```
90
+
91
+ ```
92
+ # This is the temporary hiera.yaml file used for octocatalog-diff catalog compilation
93
+ :backends:
94
+ - yaml
95
+ :yaml:
96
+ :datadir: /var/tmp/puppet-compile-dir-92347829847/environments/%{::environment}/hieradata
97
+ :hierarchy:
98
+ - servers/%{::fqdn}
99
+ - platform/%{::virtual}
100
+ - datacenter/%{::datacenter}
101
+ - os/%{::operatingsystem}
102
+ - common
103
+ ```
@@ -0,0 +1,49 @@
1
+ # Configuring octocatalog-diff to use PuppetDB
2
+
3
+ octocatalog-diff can interact with PuppetDB in the following ways:
4
+
5
+ - Retrieving the most recent set of facts for a node
6
+ - Query for exported resources during a Puppet run with storeconfigs enabled
7
+ - Retrieving a catalog for a node
8
+
9
+ For this to work, you will need to configure or provide information about your PuppetDB server to octocatalog-diff. You can provide this information via a [configuration file](/doc/configuration.md), via environment variables, or via command line parameters.
10
+
11
+ ## Required information
12
+
13
+ - **Version of PuppetDB**: octocatalog-diff supports PuppetDB's query API v4, which requires that you be running PuppetDB 2.3 or higher.
14
+
15
+ - **URL to PuppetDB**: This is the URL with the host name and port number to reach your PuppetDB instance. If you have already set up your Puppet master to communicate with PuppetDB, you can see the URL by reviewing `/etc/puppetlabs/puppet/puppetdb.conf` (on Puppet Server) or `/etc/puppet/puppetdb.conf` (on Puppet Master 3.x). The URL (or URLs) to your PuppetDB installation are visible in the `server_urls` configuration setting.
16
+
17
+ - **SSL Authentication Information**: Whether your PuppetDB instance requires clients to authenticate via SSL certificates. Unless you have made a special effort to configure your PuppetDB instance not to require client certificates, it is likely that client certificate authentication is required.
18
+
19
+ ## Supplying necessary information via configuration files
20
+
21
+ The following settings can be used in a [configuration file](/doc/configuration.md).
22
+
23
+ | Setting | Description |
24
+ | --- | --- |
25
+ | `settings[:puppetdb_url]` | PuppetDB URL settings. If this is a string, it will set a single PuppetDB URL. If it is an array, it will set multiple URLs, which will be tried in a random order until one responds. |
26
+ | `settings[:puppetdb_ssl_ca]` | Path to the certificate of the CA that signed PuppetDB's certificate. This file is typically found in `/etc/puppetlabs/puppetdb/ssl/ca.pem` on your PuppetDB server. This file should contain only the public certificate, so it is safe to distribute to developer workstations or CI environments. |
27
+ | `settings[:puppetdb_ssl_client_cert]` | Path to the certificate of the client SSL keypair. You should generate a keypair specifically for this client (or if you are running this on a machine managed by Puppet, you may be able to use the keypair for the client machine). You should **NOT** copy the certificate from your PuppetDB server itself. |
28
+ | `settings[:puppetdb_ssl_client_key]` | Path to the private key of the client SSL keypair. You should generate a keypair specifically for this client (or if you are running this on a machine managed by Puppet, you may be able to use the keypair for the client machine). You should **NOT** copy the private key from your PuppetDB server itself. |
29
+ | `settings[:puppetdb_ssl_client_password]` | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required and should be left undefined. |
30
+
31
+ ## Supplying necessary information via the command line
32
+
33
+ The following arguments can be used on the command line.
34
+
35
+ | Setting | Description |
36
+ | --- | --- |
37
+ | --puppetdb-url https://puppetdb.example.net:8081 | PuppetDB URL. The argument should match the `server_urls` configuration setting as described previously. Please note that only one URL is supported via the command line method, so if you have multiple `server_urls` URLs specified, you can only choose one. To use multiple URLs for failover purposes, please configure via configuration files. |
38
+ | --puppetdb-ssl-ca FILENAME | Path to the certificate of the CA that signed PuppetDB's certificate. This file is typically found in `/etc/puppetlabs/puppetdb/ssl/ca.pem` on your PuppetDB server. This file should contain only the public certificate, so it is safe to distribute to developer workstations or CI environments. |
39
+ | --puppetdb-ssl-client-cert FILENAME | Path to the certificate of the client SSL keypair. You should generate a keypair specifically for this client (or if you are running this on a machine managed by Puppet, you may be able to use the keypair for the client machine). You should **NOT** copy the certificate from your PuppetDB server itself. |
40
+ | --puppetdb-ssl-client-key FILENAME | Path to the private key of the client SSL keypair. You should generate a keypair specifically for this client (or if you are running this on a machine managed by Puppet, you may be able to use the keypair for the client machine). You should **NOT** copy the private key from your PuppetDB server itself. |
41
+ | --puppetdb-ssl-client-password PASSWORD_STRING | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required. |
42
+
43
+ ## Supplying necessary information via the environment
44
+
45
+ :warning: While this method of configuring octocatalog-diff for use with PuppetDB is currently supported, we recommend that configuration is done via the command line or in configuration files.
46
+
47
+ Set the environment variable `PUPPETDB_URL` to match the `server_urls` configuration setting as described previously. Please note that only one URL is supported via the environment variable method, so if you have multiple `server_urls` URLs specified, you can only choose one. To use multiple URLs for failover purposes, please configure via configuration files.
48
+
49
+ Environment variable support is not currently available for SSL client authentication settings.
@@ -0,0 +1,51 @@
1
+ # Configuration
2
+
3
+ `octocatalog-diff` may require configuration to work correctly with your Puppet setup.
4
+
5
+ 0. Download the [sample configuration file](https://raw.githubusercontent.com/github/octocatalog-diff/master/examples/octocatalog-diff.cfg.rb) and save it into one of the following directories.
6
+
7
+ - In the base directory of your Puppet repository checkout (i.e., your current working directory):
8
+
9
+ ```
10
+ .octocatalog-diff.cfg.rb
11
+ ```
12
+
13
+ - In your home directory:
14
+
15
+ ```
16
+ $HOME/.octocatalog-diff.cfg.rb
17
+ ```
18
+
19
+ - In one of the following system locations:
20
+
21
+ ```
22
+ /usr/local/etc/octocatalog-diff.cfg.rb
23
+ /opt/puppetlabs/octocatalog-diff/octocatalog-diff.cfg.rb
24
+ /etc/octocatalog-diff.cfg.rb
25
+ ```
26
+
27
+ Note: If more than one of the above files is present, the first one found will be used (proceeding from top to bottom in that list). If you set an environment variable `OCTOCATALOG_DIFF_CONFIG_FILE` that will supersede all of the above paths, and allow you to specify the configuration file location however you wish.
28
+
29
+ 0. Open the file in a text editor, and follow the comments within the file to guide yourself through configuration. The configuration file is pure ruby, allowing substantial flexibility in the configuration.
30
+
31
+ To be minimally functional, you will almost certainly need to define at least the following settings:
32
+
33
+ - `settings[:hiera_yaml_file]` as the absolute or relative path to your hiera configuration file
34
+ - `settings[:hiera_path_strip]` as the prefix to strip when munging the hiera configuration file
35
+ - `settings[:puppetdb_url]` as the URL to your PuppetDB instance so facts can be obtained
36
+
37
+ For more information on these settings:
38
+
39
+ - [Configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md)
40
+ - [Configuring octocatalog-diff to use ENC](/doc/configuration-enc.md)
41
+ - [Configuring octocatalog-diff to use PuppetDB](/doc/configuration-puppetdb.md)
42
+
43
+ 0. Test the configuration, which will indicate the location of the configuration file and validate the contents thereof.
44
+
45
+ ```
46
+ octocatalog-diff --config-test
47
+ ```
48
+
49
+ Note: If you [installed](/doc/installation.md) octocatalog-diff as a gem, the `octocatalog-diff` binary should be in your $PATH and the above command should work correctly. If you installed in a different way, you may need to provide the full path to where the `octocatalog-diff` binary was actually installed.
50
+
51
+ Now that you have entered your configuration and confirmed proper reading of your configuration file, proceed to [Basic usage](/doc/basic.md) to see if it works!
@@ -0,0 +1 @@
1
+ # octocatalog-diff developer documentation
@@ -0,0 +1,34 @@
1
+ # Test coverage
2
+
3
+ We provide two types of tests for octocatalog-diff:
4
+
5
+ - Unit tests (found in [/spec/octocatalog-diff/tests](/spec/octocatalog-diff/tests))
6
+ - Integration tests (found in [/spec/octocatalog-diff/integration](/spec/octocatalog-diff/integration))
7
+
8
+ The difference between these is as follows. Unit tests are designed to test the smallest bit of code that's practical to test. Integration tests are designed to run from end-to-end, starting via the invocation from the command line, exercising internals, and checking output from the end result.
9
+
10
+ It's our goal to have as much test coverage as we can provide from both sides, so that we can have confidence in anything we release.
11
+
12
+ ## Coverage report
13
+
14
+ The `simplecov` gem is bundled with octocatalog-diff to produce coverage reports.
15
+
16
+ To build a coverage report for unit tests only:
17
+
18
+ ```
19
+ rake coverage:spec
20
+ ```
21
+
22
+ To build a coverage report for integration tests only:
23
+
24
+ ```
25
+ rake coverage:integration
26
+ ```
27
+
28
+ To build a coverage report combining the results of unit tests and integration test:
29
+
30
+ ```
31
+ rake coverage:all
32
+ ```
33
+
34
+ Running any of these tests creates a `coverage` directory in the root of the project, and this contains an `index.html` file with a graphical report. Open this file in a browser to see the results.
@@ -0,0 +1,83 @@
1
+ # How to add new command line options
2
+
3
+ This document contains a checklist and guidance to adding new command line options.
4
+
5
+ ## Checklist
6
+
7
+ Please copy and paste this text into your Pull Request. This will create boxes for each step along the way, which you can then check off when complete.
8
+
9
+ ```
10
+ - [ ] REQUIRED: Add new file in `lib/octocatalog-diff/catalog-diff/cli/options`
11
+ - [ ] REQUIRED: Add corresponding test in `spec/octocatalog-diff/tests/catalog-diff/cli/options`
12
+ - [ ] OPTIONAL: Add default value in `lib/octocatalog-diff/catalog-diff/cli.rb`
13
+ - [ ] OPTIONAL: Add configuration example in `examples/octocatalog-diff.cfg.rb`
14
+ - [ ] REQUIRED: Add code to implement your option in `lib`
15
+ - [ ] REQUIRED: Add corresponding tests for code to implement your option in `spec/octocatalog-diff/tests`
16
+ - [ ] OPTIONAL: Add an integration test to test your option in `spec/octocatalog-diff/integration`
17
+ ```
18
+
19
+ ## Procedure
20
+
21
+ ### Create option parser
22
+
23
+ Option parsers are created in [`lib/octocatalog-diff/catalog-diff/cli/options`](/lib/octocatalog-diff/catalog-diff/cli/options).
24
+
25
+ Your option should have a "long form" that contains dashes and not underscores. For example, you should prefer `--your-new-option` and NOT use `--your_new_option`.
26
+
27
+ The file you create should reflect the name of your option. Generally the file name is the command line flag, with `-` converted to `_`.
28
+
29
+ The key that your option creates in the options hash should generally be a symbol, named the same as the command line flag, with `-` converted to `_`. In this example, `:your_new_option`.
30
+
31
+ If you are creating a binary (yes-no) option, please recognize both `--your-new-option` and `--no-your-new-option`.
32
+
33
+ We recommend copying prior art as a template:
34
+
35
+ - For a binary (yes-no) option, look at [`quiet.rb`](/lib/octocatalog-diff/catalog-diff/cli/options/quiet.rb).
36
+
37
+ - For an option that takes an integer parameter, look at [`retry_failed_catalog.rb`](/lib/octocatalog-diff/catalog-diff/cli/options/retry_failed_catalog.rb).
38
+
39
+ - For an option that takes an string parameter, look at [`bootstrap_script.rb`](/lib/octocatalog-diff/catalog-diff/cli/options/bootstrap_script.rb).
40
+
41
+ - For an option that takes an array or can be specified more than once, look at [`bootstrap_environment.rb`](/lib/octocatalog-diff/catalog-diff/cli/options/bootstrap_environment.rb).
42
+
43
+ If you can do simple validation of the argument, such as making sure the argument (if specified) matches a particular regular expression or is one of a particular set of values, please do that within the option file. For example, look at [`facts_terminus.rb`](/lib/octocatalog-diff/catalog-diff/cli/options/facts_terminus.rb).
44
+
45
+ ### Create test for option parser
46
+
47
+ Option parser tests are created in [`spec/octocatalog-diff/tests/catalog-diff/cli/options`](/spec/octocatalog-diff/tests/catalog-diff/cli/options).
48
+
49
+ If you used an existing option as a reference for your new code, consider using that option's test as a reference for your test. We have some methods, e.g. `test_with_true_false_option`, to avoid repetitive code for common patterns.
50
+
51
+ If you have handled any edge cases, e.g. input validation, please add a test that expects an error when input is provided that does not match your validation. For example, look at [`parser_spec.rb`](/spec/octocatalog-diff/tests/catalog-diff/cli/options/parser_spec.rb).
52
+
53
+ ### Add default value (OPTIONAL)
54
+
55
+ Unless specifically cleared with the project maintainers, adding a new option should not change the behavior of the program when that option isn't specified, and the new option should not be required.
56
+
57
+ In other words, if someone invokes the program *without* specifying your option, it should behave in the same way as it did before your option was ever added.
58
+
59
+ If you need to set a default value for your option, do so in [`lib/octocatalog-diff/catalog-diff/cli.rb`](/lib/octocatalog-diff/catalog-diff/cli.rb). Items should be added to DEFAULT_OPTIONS *only if* a value is required even if your option is not provided, or if you are defaulting something to *true* but providing an option to make it false.
60
+
61
+ ### Add configuration example (OPTIONAL)
62
+
63
+ If you believe your option is of general use to other users of octocatalog-diff such that they may wish to add it to their configuration file, update [`examples/octocatalog-diff.cfg.rb`](/examples/octocatalog-diff.cfg.rb) with some comments and example code for your option.
64
+
65
+ Only the most commonly used options have entries in the example configuration file. If you are unsure whether or not to include your option there, please [open an issue](https://github.com/github/octocatalog-diff/issues/new) to discuss.
66
+
67
+ As described in the default value section above, if your option is not configured in the template, the program should still work as it did before your new option was added. We want to avoid forcing users to update their configuration file unless there is a major update.
68
+
69
+ ### Add code to implement your option
70
+
71
+ This is for you to figure out! :smile_cat:
72
+
73
+ ### Add corresponding tests for code
74
+
75
+ Please add unit tests in [`spec/octocatalog-diff/tests`](/spec/octocatalog-diff/tests) that test the new behavior of any methods impacted by your changes.
76
+
77
+ ### Add an integration test (OPTIONAL)
78
+
79
+ Adding an integration test is optional, but very much appreciated.
80
+
81
+ Integration tests are in [`spec/octocatalog-diff/integration`](/spec/octocatalog-diff/integration). Generally, if you're adding a new option, the integration test for that new option will go in its own file.
82
+
83
+ Please see [integration tests](/doc/dev/integration-tests.md) for more.