octocatalog-diff 2.1.0 → 2.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 07e53efa197ece3d868ab453f200b49d6fc0718fcd40e8201ab92d04e4787871
-  data.tar.gz: e60db7eb374631ec5a4f430b58b16f57772e081aeea3b12d7576e3265df4ae6b
+  metadata.gz: 27f9f4a1e03865d74c4fbc3bc3e06a2431d1346f177c515abf6c5e27b44102c5
+  data.tar.gz: a8ebc11ab363214bd9802e0c6f72f2d5f6c6633b5eb14152da0f9e38f2dd100b
 SHA512:
-  metadata.gz: 2b6ce06f9d1d91f8bfeb12f731a8b8d901b794396dd34a6b3e2d29b8e4e7e59d02ceb1f1bdce89ac73d6d3ad14beb2883188810c778ed6ccb431a30b05a9d7b3
-  data.tar.gz: 2f712011fd71e32caf2bf3e453ce16cae41c2be79a6bdfa12ed0d3bdc1ce62751e86c6f4eea500d243f8a69adf6642f1009250c88b5b85cb1af2ddee67e3a7a0
+  metadata.gz: 459315e4f4db1610c4b16d26b1476aff00385692a98ebb1f8328cbec062197ba67be0fb84e92e92a22434b77e28d96dd0b198e54110383222d15b268e67b589e
+  data.tar.gz: 770fe20daffa95e8e2938fa0eebd604d20ef0577ce42722f8d0c0d4527e5838f90a4917d6fc8d9e41402062cc8ac37414abbd3896ad5ea1fe76bfb0d6f331ec9

data/.version

@@ -1 +1 @@
-2.1.0
+2.3.0

data/doc/CHANGELOG.md

@@ -11,7 +11,7 @@
 <tr valign=top>
 <td>2.1.0</td>
 <td>2020-02-18</td>
-
+<td>
 <li><a href="https://github.com/github/octocatalog-diff/pull/240">#240</a>: (Enhancement) Run CI against Puppet 7 and Ruby 3</li>
 <li><a href="https://github.com/github/octocatalog-diff/pull/241">#241</a>: (Bug Fix) Fix indent and numbering in several docs</li>
 <li><a href="https://github.com/github/octocatalog-diff/pull/242">#242</a>: (Enhancement) Handle JSON facts structured as name/values</li>
@@ -22,6 +22,7 @@
 <tr valign=top>
 <td>2.0.0</td>
 <td>2020-01-12</td>
+<td>
 <li><a href="https://github.com/github/octocatalog-diff/pull/226">#226</a>: (Enhancement) Add Puppet 6 support</li>
 <li><a href="https://github.com/github/octocatalog-diff/pull/228">#228</a>: (Enhancement) Add Puppetserver catalog v4 API support</li>
 <li><a href="https://github.com/github/octocatalog-diff/pull/229">#229</a>: (Enhancement) Add support for PE package inventory facts</li>
@@ -32,6 +33,7 @@
 <tr valign=top>
 <td>1.6.0</td>
 <td>2019-10-31</td>
+<td>
 <li><a href="https://github.com/github/octocatalog-diff/pull/216">#216</a>: (Enhancement) Hide sensitive parameters</li>
 <li><a href="https://github.com/github/octocatalog-diff/pull/204">#204</a>: (Enhancement) Add glob support for modulepath</li>
 <li><a href="https://github.com/github/octocatalog-diff/pull/206">#206</a>: (Bug Fix) Fix multi-node list with parallel mode</li>

data/doc/advanced-compare-file-text.md

@@ -21,6 +21,7 @@ However, since applying the catalog could change the content of a file on the ta
 This feature is available only when the catalogs are being compiled from local code. This feature is not available, and will be automatically disabled, when pulling catalogs from PuppetDB or a Puppet server.
 
 Note: In Puppet >= 4.4 there is an option in Puppet itself called "static catalogs" which if enabled will cause the checksum of the file to be included in the catalog. However, the `octocatalog-diff` feature described here is still useful because it can be used to display a "diff" of the change rather than just displaying a "diff" of a checksum.
+
 ## Command line options
 
 ### `--compare-file-text` and `--no-compare-file-text`
@@ -39,6 +40,12 @@ If this feature is disabled by default in a configuration file, add `--compare-f
 
 Note that the feature will be automatically disabled, regardless of configuration or command line options, if catalogs are being pulled from PuppetDB or a Puppet server.
 
+### `--compare-file-text=force`
+
+To force the option to be on even in situations when it would be auto-disabled, set the command line argument `--compare-file-text=force`. When the Puppet source code is available, e.g. when compiling a catalog with `--catalog-only`, this will adjust the resulting catalog.
+
+If the Puppet source code is not available, forcing the feature on anyway may end up causing an exception. Use this option at your own risk.
+
 ### `--compare-file-text-ignore-tags`
 
 To disable this feature for specific `file` resources, set a tag on the resources for which the comparison is undesired. For example:

data/doc/advanced-filter.md

@@ -24,7 +24,7 @@ Here is the list of available filters and an explanation of each:
 
 #### Description
 
-When the `AbsentFile` filter is enabled, if any file is `ensure => absent` in the *new* catalog, then changes to any other parameters will be suppressed.
+When the `AbsentFile` filter is enabled, if any file is `ensure => absent` in the _new_ catalog, then changes to any other parameters will be suppressed.
 
 Consider that a file resource is declared as follows in two catalogs:
 
@@ -71,6 +71,25 @@ Wouldn't it be nice if the meaningless information didn't appear, and all you sa
       + absent
 ```
 
+## Equivalent Array (not considering datatypes)
+
+#### Usage
+
+```
+--filters EquivalentArrayNoDatatypes
+```
+
+#### Description
+
+In an array, ignore changes where the old and new arrays are "equivalent" as described below. This is useful when octocatalog-diff is comparing changes between a catalog with stringified values and a catalog with non-stringified values.
+
+The following are considered equivalent when this filter is engaged:
+
+- Stringified integers (`[0, 1]` and `['0', '1']`)
+- Stringified floats (`[0.0, 1.0]` and `['0.0', '1.0']`)
+- Numerically-equal integers and floats (`[0, 1]` and `[0.0, 1.0]`)
+- Symbols and corresponding strings (`[:foo, :bar]` and `[':foo', ':bar']`)
+
 ## JSON
 
 #### Usage
@@ -105,7 +124,7 @@ New: { "notify": [ "Service[foo]" ] }
 This filter will suppress differences for the value of a parameter when:
 
 - The value in one catalog is an object, AND
-- The value in the other catalog is an array containing *only* that same object
+- The value in the other catalog is an array containing _only_ that same object
 
 ## YAML
 

data/doc/configuration-puppetdb.md

@@ -32,6 +32,7 @@ The following settings can be used in a [configuration file](/doc/configuration.
 | --- | --- |
 | `settings[:puppetdb_url]` | PuppetDB URL settings. If this is a string, it will set a single PuppetDB URL. If it is an array, it will set multiple URLs, which will be tried in a random order until one responds. |
 | `settings[:puppetdb_ssl_ca]` | Path to the certificate of the CA that signed PuppetDB's certificate. This file should contain only the public certificate, so it is safe to distribute to developer workstations or CI environments. |
+| `settings[:puppetdb_ssl_crl]` | Path to the Certificate Revocation List provided by Puppetserver.
 | `settings[:puppetdb_ssl_client_cert]` | TEXT of the certificate of the client SSL keypair used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
 | `settings[:puppetdb_ssl_client_key]` | TEXT of the private key of the client SSL keypair used to authenticate to PuppetDB. Note: This variable is not set to a file path, which means you will likely want to use means you will likely want to use `File.read(...)` if you are configuring this to be read from a file. |
 | `settings[:puppetdb_ssl_client_pem]` | Concatenation of the text of `puppetdb_ssl_client_key` and `puppetdb_ssl_client_cert` as previously described. This is a good alternative if your certificate chain is complex and it's easier just to put everything in a single place. Note: this option is second in precedence; if `settings[:puppetdb_ssl_client_cert]` and `settings[:puppetdb_ssl_client_key]` are both set, this will be ignored. |
@@ -46,6 +47,7 @@ The following arguments can be used on the command line.
 | --- | --- |
 | --puppetdb-url https://puppetdb.example.net:8081 | PuppetDB URL. The argument should match the `server_urls` configuration setting as described previously. Please note that only one URL is supported via the command line method, so if you have multiple `server_urls` URLs specified, you can only choose one. To use multiple URLs for failover purposes, please configure via configuration files. |
 | --puppetdb-ssl-ca FILENAME | Path to the certificate of the CA that signed PuppetDB's certificate. This file should contain only the public certificate, so it is safe to distribute to developer workstations or CI environments. |
+| --puppetdb-ssl-crl FILENAME | Path to the Certificate Revocation List of the CA that signed PuppetDB's certificate. |
 | --puppetdb-ssl-client-cert FILENAME | Path to the certificate of the client SSL keypair. |
 | --puppetdb-ssl-client-key FILENAME | Path to the private key of the client SSL keypair. |
 | --puppetdb-ssl-client-password PASSWORD_STRING | Plain text string containing the password to unlock the private key. For keys generated by the Puppet Master CA, this is not required. |

data/doc/limitations.md

@@ -6,7 +6,7 @@ Testing of Puppet catalogs is faster than running the agent, but you need to be
 
     octocatalog-diff by default uses the facts reported from a node's more recent Puppet run. If you have made changes to custom facts, catalog testing will **NOT** be an adequate test of whether your custom facts worked. (You can still use octocatalog-diff to help predict changes to nodes based on changes to facts, by overriding facts on the command line.)
 
-1. Agents handle depenency ordering and implementation details
+1. Agents handle dependency ordering and implementation details
 
     The catalog defines the state of the system, but it's up to the agent to determine how to bring the system to a point that matches the catalog. The agent is responsible for order of operations and actually making the change.
 

data/doc/optionsref.md

@@ -66,7 +66,10 @@ Usage: octocatalog-diff [command line options]
         --[no-]display-source        Show source file and line for each difference
         --[no-]validate-references "before,require,subscribe,notify"
                                      References to validate
-        --[no-]compare-file-text     Compare text, not source location, of file resources
+        --[no-]compare-file-text[=force]
+                                     Compare text, not source location, of file resources
+        --storeconfigs-backend TERMINUS
+                                     Set the terminus used for storeconfigs
         --[no-]storeconfigs          Enable integration with puppetdb for collected resources
         --retry-failed-catalog N     Retry building a failed catalog N times
         --no-enc                     Disable ENC
@@ -106,6 +109,7 @@ Usage: octocatalog-diff [command line options]
         --puppetdb-token-file PATH   Path containing token for PuppetDB API, relative or absolute
         --puppetdb-url URL           PuppetDB base URL
         --puppetdb-ssl-ca FILENAME   CA certificate that signed the PuppetDB certificate
+        --puppetdb-ssl-crl FILENAME  Certificate Revocation List provided by the Puppetserver
         --puppetdb-ssl-client-cert FILENAME
                                      SSL client certificate to connect to PuppetDB
         --puppetdb-ssl-client-key FILENAME
@@ -372,7 +376,14 @@ diffing activity. The catalog will be printed to STDOUT or written to the output
       When a file is specified with `source => 'puppet:///modules/something/foo.txt'`, remove
 the 'source' attribute and populate the 'content' attribute with the text of the file.
 This allows for a diff of the content, rather than a diff of the location, which is
-what is most often desired. (<a href="../lib/octocatalog-diff/cli/options/compare_file_text.rb">compare_file_text.rb</a>)
+what is most often desired.
+This has historically been a binary option, so --compare-file-text with no argument will
+set this to `true` and --no-compare-file-text will set this to `false`. Note that
+--no-compare-file-text does not accept an argument.
+File text comparison will be auto-disabled in circumstances other than compiling and
+comparing two catalogs. To force file text comparison to be enabled at other times,
+set --compare-file-text=force. This allows the content of the file to be substituted
+in to --catalog-only compilations, for example. (<a href="../lib/octocatalog-diff/cli/options/compare_file_text.rb">compare_file_text.rb</a>)
     </td>
   </tr>
 
@@ -387,7 +398,14 @@ what is most often desired. (<a href="../lib/octocatalog-diff/cli/options/compar
       When a file is specified with `source => 'puppet:///modules/something/foo.txt'`, remove
 the 'source' attribute and populate the 'content' attribute with the text of the file.
 This allows for a diff of the content, rather than a diff of the location, which is
-what is most often desired. (<a href="../lib/octocatalog-diff/cli/options/compare_file_text.rb">compare_file_text.rb</a>)
+what is most often desired.
+This has historically been a binary option, so --compare-file-text with no argument will
+set this to `true` and --no-compare-file-text will set this to `false`. Note that
+--no-compare-file-text does not accept an argument.
+File text comparison will be auto-disabled in circumstances other than compiling and
+comparing two catalogs. To force file text comparison to be enabled at other times,
+set --compare-file-text=force. This allows the content of the file to be substituted
+in to --catalog-only compilations, for example. (<a href="../lib/octocatalog-diff/cli/options/compare_file_text.rb">compare_file_text.rb</a>)
     </td>
   </tr>
 
@@ -1492,6 +1510,18 @@ the text of the password won't appear in the process list. (<a href="../lib/octo
     </td>
   </tr>
 
+  <tr>
+    <td valign=top>
+      <pre><code>--puppetdb-ssl-crl FILENAME</code></pre>
+    </td>
+    <td valign=top>
+      Certificate Revocation List provided by the Puppetserver
+    </td>
+    <td valign=top>
+      Specify the Certificate Revocation List for PuppetDB SSL. (<a href="../lib/octocatalog-diff/cli/options/puppetdb_ssl_crl.rb">puppetdb_ssl_crl.rb</a>)
+    </td>
+  </tr>
+
   <tr>
     <td valign=top>
       <pre><code>--puppetdb-token TOKEN</code></pre>
@@ -1600,6 +1630,18 @@ cached directory). (<a href="../lib/octocatalog-diff/cli/options/safe_to_delete_
     </td>
   </tr>
 
+  <tr>
+    <td valign=top>
+      <pre><code>--storeconfigs-backend TERMINUS</code></pre>
+    </td>
+    <td valign=top>
+      Set the terminus used for storeconfigs
+    </td>
+    <td valign=top>
+      Set storeconfigs (integration with PuppetDB for collected resources) (<a href="../lib/octocatalog-diff/cli/options/storeconfigs_backend.rb">storeconfigs_backend.rb</a>)
+    </td>
+  </tr>
+
   <tr>
     <td valign=top>
       <pre><code>--suppress-absent-file-details

data/lib/octocatalog-diff/api/v1/override.rb

@@ -74,7 +74,7 @@ module OctocatalogDiff
           return value if datatype == 'string'
           return parse_json(value) if datatype == 'json'
           return nil if datatype == 'nil'
-          if datatype == 'fixnum' || datatype == 'integer'
+          if ['fixnum', 'integer'].include? datatype
             return Regexp.last_match(1).to_i if value =~ /^(-?\d+)$/
             raise ArgumentError, "Illegal integer '#{value}'"
           end

data/lib/octocatalog-diff/catalog-diff/differ.rb

@@ -262,7 +262,7 @@ module OctocatalogDiff
           # Process each attribute in the resource
           resource.each do |k, v|
             # Title was pre-processed
-            next if k == 'title' || k == 'type'
+            next if ['title', 'type'].include? k
 
             # Handle parameters
             if k == 'parameters'
@@ -272,7 +272,7 @@ module OctocatalogDiff
               # The order of tags is unimportant. Sort this array to avoid false diffs if order changes.
               # Also if tags is empty, don't add.
               hsh[k] = v.sort if v.is_a?(Array) && v.any?
-            elsif k == 'file' || k == 'line'
+            elsif ['file', 'line'].include? k
               # We don't care, for the purposes of catalog-diff, from which manifest and line this resource originated.
               # However, we may report this to the user, so we will keep it in here for now.
               hsh[k] = v
@@ -354,7 +354,7 @@ module OctocatalogDiff
           elsif operator == '=->'
             # String equality test only of the old value
             matcher = ->(x, _y) { x == value }
-          elsif operator == '=~>' || operator == '=&>'
+          elsif ['=~>', '=&>'].include? operator
             begin
               my_regex = Regexp.new(value, Regexp::IGNORECASE)
             rescue RegexpError => exc
@@ -558,7 +558,7 @@ module OctocatalogDiff
 
           # Added a new key that points to some kind of data structure that we know how
           # to handle.
-          classes = [String, Integer, Float, TrueClass, FalseClass, Array, Hash]
+          classes = [String, Integer, Float, TrueClass, FalseClass, Array, Hash, NilClass]
           if obj[1] =~ /^(.+)\f([^\f]+)$/ && OctocatalogDiff::Util::Util.object_is_any_of?(obj[2], classes)
             hashdiff_add_remove.add(obj[1])
             next

data/lib/octocatalog-diff/catalog-diff/display/text.rb

@@ -307,7 +307,7 @@ module OctocatalogDiff
         # @param string_in [String] Input string, which might contain trailing whitespace
         # @return [String] Modified string
         def self.make_trailing_whitespace_visible(string_in)
-          return string_in unless string_in =~ /\A((?:.|\n)*?)(\s+)(\e\[0m)?\Z/
+          return string_in unless string_in =~ /\A((?:.|\n){1,1000}?)(\s+)(\e\[0m)?\Z/
           beginning = Regexp.last_match(1)
           trailing_space = Regexp.last_match(2)
           end_escape = Regexp.last_match(3)
@@ -457,7 +457,7 @@ module OctocatalogDiff
         def self.stringify_for_diffy(obj)
           return JSON.pretty_generate(obj) if OctocatalogDiff::Util::Util.object_is_any_of?(obj, [Hash, Array])
           return '""' if obj.is_a?(String) && obj == ''
-          return obj if OctocatalogDiff::Util::Util.object_is_any_of?(obj, [String, Fixnum, Integer, Float])
+          return obj if OctocatalogDiff::Util::Util.object_is_any_of?(obj, [String, Integer, Float])
           "#{class_name_for_diffy(obj.class)}: #{obj.inspect}"
         end
 

data/lib/octocatalog-diff/catalog-diff/filter/equivalent_array_no_datatypes.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+require_relative '../filter'
+
+module OctocatalogDiff
+  module CatalogDiff
+    class Filter
+      # Filter out changes in parameters where the elements of an array are the
+      # same values but different data types. For example, this would filter out
+      # the following diffs:
+      #   Exec[some command] =>
+      #    parameters =>
+      #      returns =>
+      #        - ["0", "1"]
+      #        + [0, 1]
+      class EquivalentArrayNoDatatypes < OctocatalogDiff::CatalogDiff::Filter
+        # Public: Implement the filter for arrays that have the same elements
+        # but possibly different data types.
+        #
+        # @param diff [OctocatalogDiff::API::V1::Diff] Difference
+        # @param _options [Hash] Additional options (there are none for this filter)
+        # @return [Boolean] true if this should be filtered out, false otherwise
+        def filtered?(diff, _options = {})
+          # Skip additions or removals - focus only on changes
+          return false unless diff.change?
+          old_value = diff.old_value
+          new_value = diff.new_value
+
+          # Skip unless both the old and new values are arrays.
+          return false unless old_value.is_a?(Array) && new_value.is_a?(Array)
+
+          # Avoid generating comparable values if the arrays are a different
+          # size, because there's no possible way that they are equivalent.
+          return false unless old_value.size == new_value.size
+
+          # Generate and then compare the comparable arrays.
+          old_value.map { |x| comparable_value(x) } == new_value.map { |x| comparable_value(x) }
+        end
+
+        # Private: Get a more easily comparable value for an array element.
+        # Integers, floats, and strings that look like integers or floats become
+        # floats, and symbols are converted to string representation.
+        #
+        # @param input [any] Value to convert
+        # @return [any] "Comparable" value
+        def comparable_value(input)
+          # Any string that looks like a number is converted to a float.
+          if input.is_a?(String) && input =~ /\A-?(([0-9]*\.[0-9]+)|([0-9]+))\z/
+            return input.to_f
+          end
+
+          # Any number is converted to a float
+          return input.to_f if input.is_a?(Integer) || input.is_a?(Float)
+
+          # Symbols are converted to ":xxx" strings.
+          return ":#{input}" if input.is_a?(Symbol)
+
+          # Everything else is unconverted.
+          input
+        end
+      end
+    end
+  end
+end

data/lib/octocatalog-diff/catalog-diff/filter.rb

@@ -1,6 +1,7 @@
 require_relative '../api/v1/diff'
 require_relative 'filter/absent_file'
 require_relative 'filter/compilation_dir'
+require_relative 'filter/equivalent_array_no_datatypes'
 require_relative 'filter/json'
 require_relative 'filter/single_item_array'
 require_relative 'filter/yaml'
@@ -14,7 +15,7 @@ module OctocatalogDiff
       attr_accessor :logger
 
       # List the available filters here (by class name) for use in the validator method.
-      AVAILABLE_FILTERS = %w(AbsentFile CompilationDir JSON SingleItemArray YAML).freeze
+      AVAILABLE_FILTERS = %w(AbsentFile CompilationDir EquivalentArrayNoDatatypes JSON SingleItemArray YAML).freeze
 
       # Public: Determine whether a particular filter exists. This can be used to validate
       # a user-submitted filter.

data/lib/octocatalog-diff/catalog-util/builddir.rb

@@ -33,6 +33,7 @@ module OctocatalogDiff
       # :hiera_path [String] relative path to hiera data files (mutually exclusive with :hiera_path_strip)
       # :hiera_path_strip [String] string to strip off the beginning of :datadir
       # :puppetdb_ssl_ca [String] Path to SSL CA certificate
+      # :puppetdb_ssl_crl [String] Path to Certificate Revocation List
       # :puppetdb_ssl_client_key [String] String representation of SSL client key
       # :puppetdb_ssl_client_cert [String] String representation of SSL client certificate
       # :puppetdb_ssl_client_password [String] Password to unlock SSL private key
@@ -154,7 +155,7 @@ module OctocatalogDiff
         elsif options[:fact_file]
           raise Errno::ENOENT, "Fact file #{options[:fact_file]} does not exist" unless File.file?(options[:fact_file])
           fact_file_opts = { fact_file_string: File.read(options[:fact_file]) }
-          fact_file_opts[:backend] = Regexp.last_match(1).to_sym if options[:fact_file] =~ /.*\.(\w+)$/
+          fact_file_opts[:backend] = Regexp.last_match(1).to_sym if options[:fact_file] =~ /.{1,1000}\.(\w+)$/
           OctocatalogDiff::Facts.new(fact_file_opts)
         else
           raise ArgumentError, 'No facts passed to "install_fact_file" method'
@@ -207,7 +208,7 @@ module OctocatalogDiff
         enc_path = File.join(@tempdir, 'enc.sh')
         File.open(enc_path, 'w') do |f|
           f.write "#!/bin/sh\n"
-          f.write "cat <<-EOF\n"
+          f.write "cat <<-'EOF'\n"
           f.write enc_obj.content
           f.write "\nEOF\n"
         end
@@ -273,6 +274,9 @@ module OctocatalogDiff
 
         # SSL CA provided?
         install_ssl_ca(logger, options) if options[:puppetdb_ssl_ca]
+
+        # SSL CRL provided?
+        install_ssl_crl(logger, options) if options[:puppetdb_ssl_crl]
       end
 
       private
@@ -360,6 +364,18 @@ module OctocatalogDiff
         logger.debug "Installed CA certificate in #{ca_outfile}"
       end
 
+      # Install SSL Certificate Revocation List
+      # @param logger [Logger] Logger object
+      # @param options [Hash] Options hash
+      def install_ssl_crl(logger, options)
+        crl_file = options[:puppetdb_ssl_crl]
+        raise Errno::ENOENT, 'SSL CRL file does not exist' unless File.file?(crl_file)
+        crl_content = File.read(crl_file)
+        crl_outfile = File.join(@tempdir, 'var', 'ssl', 'crl.pem')
+        File.open(crl_outfile, 'w') { |f| f.write(crl_content) }
+        logger.debug "Installed Certificate Revocation List in #{crl_outfile}"
+      end
+
       # Install SSL keypair for client certificate authentication
       # @param logger [Logger] Logger object
       # @param options [Hash] Options hash

data/lib/octocatalog-diff/catalog-util/command.rb

@@ -72,7 +72,12 @@ module OctocatalogDiff
 
         # storeconfigs?
         if @options[:storeconfigs]
-          cmdline.concat %w(--storeconfigs --storeconfigs_backend=puppetdb)
+          if @options[:storeconfigs_backend]
+            cmdline << '--storeconfigs'
+            cmdline << "--storeconfigs_backend=#{Shellwords.escape(@options[:storeconfigs_backend])}"
+          else
+            cmdline.concat %w(--storeconfigs --storeconfigs_backend=puppetdb)
+          end
         else
           cmdline << '--no-storeconfigs'
         end
@@ -91,7 +96,7 @@ module OctocatalogDiff
         facts_terminus = @options.fetch(:facts_terminus, 'yaml')
         if facts_terminus == 'yaml'
           cmdline << "--factpath=#{Shellwords.escape(File.join(@compilation_dir, 'var', 'yaml', 'facts'))}"
-          if @options[:fact_file].is_a?(String) && @options[:fact_file] =~ /.*\.(\w+)$/
+          if @options[:fact_file].is_a?(String) && @options[:fact_file] =~ /.*{1,1000}\.(\w+)$/
             fact_file = File.join(@compilation_dir, 'var', 'yaml', 'facts', "#{@node}.#{Regexp.last_match(1)}")
             FileUtils.cp @options[:fact_file], fact_file unless File.file?(fact_file) || @options[:fact_file] == fact_file
           end

data/lib/octocatalog-diff/cli/options/compare_file_text.rb

@@ -4,14 +4,30 @@
 # the 'source' attribute and populate the 'content' attribute with the text of the file.
 # This allows for a diff of the content, rather than a diff of the location, which is
 # what is most often desired.
+#
+# This has historically been a binary option, so --compare-file-text with no argument will
+# set this to `true` and --no-compare-file-text will set this to `false`. Note that
+# --no-compare-file-text does not accept an argument.
+#
+# File text comparison will be auto-disabled in circumstances other than compiling and
+# comparing two catalogs. To force file text comparison to be enabled at other times,
+# set --compare-file-text=force. This allows the content of the file to be substituted
+# in to --catalog-only compilations, for example.
+#
 # @param parser [OptionParser object] The OptionParser argument
 # @param options [Hash] Options hash being constructed; this is modified in this method.
 OctocatalogDiff::Cli::Options::Option.newoption(:compare_file_text) do
   has_weight 210
 
   def parse(parser, options)
-    parser.on('--[no-]compare-file-text', 'Compare text, not source location, of file resources') do |x|
-      options[:compare_file_text] = x
+    parser.on('--[no-]compare-file-text[=force]', 'Compare text, not source location, of file resources') do |x|
+      if x == 'force'
+        options[:compare_file_text] = :force
+      elsif [true, false].include? x
+        options[:compare_file_text] = x
+      else
+        raise OptionParser::NeedlessArgument("needless argument: --compare-file-text=#{x}")
+      end
     end
   end
 end

data/lib/octocatalog-diff/cli/options/puppetdb_ssl_crl.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+# Specify the Certificate Revocation List for PuppetDB SSL.
+# @param parser [OptionParser object] The OptionParser argument
+# @param options [Hash] Options hash being constructed; this is modified in this method.
+OctocatalogDiff::Cli::Options::Option.newoption(:puppetdb_ssl_crl) do
+  has_weight 310
+  order_within_weight 11
+
+  def parse(parser, options)
+    parser.on('--puppetdb-ssl-crl FILENAME', 'Certificate Revocation List provided by the Puppetserver') do |x|
+      raise Errno::ENOENT, "--puppetdb-ssl-crl #{x} does not point to a valid file" unless File.file?(x)
+      options[:puppetdb_ssl_crl] = x
+    end
+  end
+end

data/lib/octocatalog-diff/cli/options/storeconfigs_backend.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+# Set storeconfigs (integration with PuppetDB for collected resources)
+# @param parser [OptionParser object] The OptionParser argument
+# @param options [Hash] Options hash being constructed; this is modified in this method.
+OctocatalogDiff::Cli::Options::Option.newoption(:storeconfigs) do
+  has_weight 220
+
+  def parse(parser, options)
+    parser.on('--storeconfigs-backend TERMINUS', 'Set the terminus used for storeconfigs') do |x|
+      options[:storeconfigs_backend] = x
+    end
+  end
+end

data/lib/octocatalog-diff/cli/options.rb

@@ -99,7 +99,7 @@ module OctocatalogDiff
       # option will populate any of the 'to' and 'from' variants that are missing.
       # @param :datatype [?] Expected data type
       def self.option_globally_or_per_branch(opts = {})
-        opts[:filename] = caller[0].split(':').first
+        opts[:filename] = opts[:filename] = caller[0].split(':').first
         datatype = opts.fetch(:datatype, '')
         return option_globally_or_per_branch_string(opts) if datatype.is_a?(String)
         return option_globally_or_per_branch_array(opts) if datatype.is_a?(Array)

data/lib/octocatalog-diff/cli.rb

@@ -43,6 +43,7 @@ module OctocatalogDiff
       compare_file_text: true,
       display_datatype_changes: true,
       parallel: true,
+      storeconfigs_backend: 'puppetdb',
       suppress_absent_file_details: true,
       hiera_path: 'hieradata',
       use_lcs: true

data/lib/octocatalog-diff/puppetdb.rb

@@ -37,6 +37,7 @@ module OctocatalogDiff
     # @param :puppetdb_port [Integer] Port number, defaults to 8080 (non-SSL) or 8081 (SSL)
     # @param :puppetdb_ssl [Boolean] defaults to true, because you should use SSL
     # @param :puppetdb_ssl_ca [String] Path to file containing CA certificate
+    # @param :puppetdb_ssl_crl [String] Path to file containing CRL file
     # @param :puppetdb_ssl_verify [Boolean] Override the CA verification setting guessed from parameters
     # @param :puppetdb_ssl_client_pem [String] PEM-encoded client key and certificate
     # @param :puppetdb_ssl_client_p12 [String] pkcs12-encoded client key and certificate

data/lib/octocatalog-diff/util/catalogs.rb

@@ -71,6 +71,12 @@ module OctocatalogDiff
         if @options.fetch(:compare_file_text, false)
           result.each do |_key, builder_obj|
             next if builder_obj.convert_file_resources(true)
+
+            if @options[:compare_file_text] == :force
+              @logger.debug "--compare-file-text is force-enabled even though it is not supported by #{builder_obj.builder}"
+              next
+            end
+
             @logger.debug "Disabling --compare-file-text; not supported by #{builder_obj.builder}"
             @options[:compare_file_text] = false
             catalog_tasks.map! do |x|

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: octocatalog-diff
 version: !ruby/object:Gem::Version
-  version: 2.1.0
+  version: 2.3.0
 platform: ruby
 authors:
 - GitHub, Inc.
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-02-18 00:00:00.000000000 Z
+date: 2024-09-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: diffy
@@ -31,14 +31,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.11.0
+        version: 0.21.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.11.0
+        version: 0.21.0
 - !ruby/object:Gem::Dependency
   name: hashdiff
   requirement: !ruby/object:Gem::Requirement
@@ -81,6 +81,34 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 0.25.0b2
+- !ruby/object:Gem::Dependency
+  name: puppet
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 5.5.22
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 5.5.22
+- !ruby/object:Gem::Dependency
+  name: puppet
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 5.5.22
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 5.5.22
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
@@ -101,14 +129,14 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 12.3.1
+        version: 12.3.3
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 12.3.1
+        version: 12.3.3
 - !ruby/object:Gem::Dependency
   name: parallel_tests
   requirement: !ruby/object:Gem::Requirement
@@ -143,14 +171,14 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.48.1
+        version: 0.49.0
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.48.1
+        version: 0.49.0
 - !ruby/object:Gem::Dependency
   name: simplecov
   requirement: !ruby/object:Gem::Requirement
@@ -179,25 +207,11 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.1.1
-- !ruby/object:Gem::Dependency
-  name: puppet
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 5.5.8
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 5.5.8
-description: |
-  Octocatalog-Diff assists with Puppet development and testing by enabling the user to
-  compile 2 Puppet catalogs and compare them. It is possible to compare different
-  branches, different versions, and different fact values. This is intended to be run
-  from a local development environment or in CI.
+description: |2
+    Octocatalog-Diff assists with Puppet development and testing by enabling the user to
+    compile 2 Puppet catalogs and compare them. It is possible to compare different
+    branches, different versions, and different fact values. This is intended to be run
+    from a local development environment or in CI.
 email: opensource+octocatalog-diff@github.com
 executables:
 - octocatalog-diff
@@ -279,6 +293,7 @@ files:
 - lib/octocatalog-diff/catalog-diff/filter.rb
 - lib/octocatalog-diff/catalog-diff/filter/absent_file.rb
 - lib/octocatalog-diff/catalog-diff/filter/compilation_dir.rb
+- lib/octocatalog-diff/catalog-diff/filter/equivalent_array_no_datatypes.rb
 - lib/octocatalog-diff/catalog-diff/filter/json.rb
 - lib/octocatalog-diff/catalog-diff/filter/single_item_array.rb
 - lib/octocatalog-diff/catalog-diff/filter/yaml.rb
@@ -370,6 +385,7 @@ files:
 - lib/octocatalog-diff/cli/options/puppetdb_ssl_client_key.rb
 - lib/octocatalog-diff/cli/options/puppetdb_ssl_client_password.rb
 - lib/octocatalog-diff/cli/options/puppetdb_ssl_client_password_file.rb
+- lib/octocatalog-diff/cli/options/puppetdb_ssl_crl.rb
 - lib/octocatalog-diff/cli/options/puppetdb_token.rb
 - lib/octocatalog-diff/cli/options/puppetdb_token_file.rb
 - lib/octocatalog-diff/cli/options/puppetdb_url.rb
@@ -378,6 +394,7 @@ files:
 - lib/octocatalog-diff/cli/options/safe_to_delete_cached_master_dir.rb
 - lib/octocatalog-diff/cli/options/save_catalog.rb
 - lib/octocatalog-diff/cli/options/storeconfigs.rb
+- lib/octocatalog-diff/cli/options/storeconfigs_backend.rb
 - lib/octocatalog-diff/cli/options/suppress_absent_file_details.rb
 - lib/octocatalog-diff/cli/options/to_from_branch.rb
 - lib/octocatalog-diff/cli/options/truncate_details.rb
@@ -420,14 +437,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.0.0
+      version: 2.6.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.3
+rubygems_version: 3.5.7
 signing_key:
 specification_version: 4
 summary: Compile Puppet catalogs from 2 branches, versions, etc., and compare them.