octocatalog-diff 1.5.3 → 1.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 1d454b6d3a7fc5ec8ff2f55231e83fc2cad36d2a
4
- data.tar.gz: 10dba5a6f4694bebd11fa01465fdbbdc1cb77774
3
+ metadata.gz: 0ba4dc520ab7e8ae27324beef80edd6c96f601de
4
+ data.tar.gz: d927873135a73e5f1b8c178e0cf4a54b9a29aa46
5
5
  SHA512:
6
- metadata.gz: f4edfbcd8f3d5e4508e7610ba47f93159608f6572c3585fc506eba2c2037a541193fe76d018afee365ddf6c0b6a27e7df32ff0145ded794e3cab5d9ed7d67115
7
- data.tar.gz: 0053a799e8342b2df02d722502c1dc1ead513148ce78e9e1616a715e33af63cb1990b29afe1abae81ef18659295efbc6b14190db498215c8b4438bf5397eb9a3
6
+ metadata.gz: e7a23ff7139158b5092765b190a801fbd670fe2931dde206fad3007fdf03995e3e6f62a881f9b3258d2b91880688664c7f43c947f5a2cfdb9b1c9b36516c94be
7
+ data.tar.gz: 4eae1cff736c29ced07644eac4cd4af812df56ac6c32afade41cf85ba0771d8136d79bf07783bec6a101cec3e5880ac647fbb83db55fe352b1181db6422288d0
data/.version CHANGED
@@ -1 +1 @@
1
- 1.5.3
1
+ 1.5.4
@@ -8,6 +8,17 @@
8
8
  </tr>
9
9
  </thead><tbody>
10
10
 
11
+ <tr valign=top>
12
+ <td>1.5.4</td>
13
+ <td>2018-12-11</td>
14
+ <td>
15
+ <li><a href="https://github.com/github/octocatalog-diff/pull/190">#190</a>: (Enhancement) Additional filtered out cases for compilation directory</li>
16
+ <li><a href="https://github.com/github/octocatalog-diff/pull/195">#195</a>: (Enhancement) Parallel catalog-diff when multiple hostnames are passed</li>
17
+ <li><a href="https://github.com/github/octocatalog-diff/pull/198">#198</a>: (Bug Fix) Portability fixes</li>
18
+ <li><a href="https://github.com/github/octocatalog-diff/pull/200">#200</a>: (Bug Fix) Support name parameter when validating references</li>
19
+ </td>
20
+ </tr>
21
+
11
22
  <tr valign=top>
12
23
  <td>1.5.3</td>
13
24
  <td>2018-03-05</td>
@@ -167,12 +167,16 @@ In this case, "owner", "notify", and "content" are nested under "parameters". In
167
167
 
168
168
  ```
169
169
  # Ignore all changes to the `owner` attribute of a file.
170
- [ { type: Regexp.new('\AFile\z'), attr: Regexp.new("\Aparameters\fowner\z" } ]
170
+ [ { type: Regexp.new('\AFile\z'), attr: Regexp.new("\\Aparameters\fowner\\z" } ]
171
171
 
172
172
  # Ignore changes to `owner` or `group` for a file or an exec.
173
- [ { type: Regexp.new('\A(File|Exec)\z'), attr: Regexp.new("\Aparameters\f(owner|group)\z" } ]
173
+ [ { type: Regexp.new('\A(File|Exec)\z'), attr: Regexp.new("\\Aparameters\f(owner|group)\\z" } ]
174
174
  ```
175
175
 
176
+ When using regular expressions, `\f` (form feed character) is used to separate the structure (e.g. `parameters\fowner` refers to the `parameters` hash, `owner` key).
177
+
178
+ :bulb: Note that `\A` in Ruby matches the beginning of the string and `\z` matches the end, but these are not actual characters. Therefore, if you are using `\A` or `\z` in double quotes (`"`), be sure to heed the examples above and write your expression like: `Regexp.new("\\Aparameters\fowner\\z")`.
179
+
176
180
  #### `:validate_references` (Array&lt;String&gt;, Optional)
177
181
 
178
182
  Invoke the [catalog validation](/doc/advanced-catalog-validation.md) feature to ensure resources targeted by `before`, `notify`, `require`, and/or `subscribe` exist in the catalog. If this parameter is not defined, no reference validation occurs.
@@ -94,9 +94,9 @@ Returns the value of the resource from the new catalog.
94
94
  }
95
95
  }
96
96
 
97
- # Demonstrates structure and old_value
97
+ # Demonstrates structure and new_value
98
98
  diff.structure #=> ['parameters', 'content']
99
- diff.old_value #=> 'This is the NEW FILE!!!!!'
99
+ diff.new_value #=> 'This is the NEW FILE!!!!!'
100
100
  ```
101
101
 
102
102
  #### `#old_file` (String)
@@ -107,7 +107,7 @@ Note that this is a pass-through of information provided in the Puppet catalog,
107
107
 
108
108
  Note also that if the diff represents addition of a resource, this will return `nil`, because the resource does not exist in the old catalog.
109
109
 
110
- #### `#old_file` (String)
110
+ #### `#old_line` (String)
111
111
 
112
112
  Returns the line number within the Puppet manifest giving rise to the resource as it exists in the old catalog. (See `#old_file` for the filename of the Puppet manifest.)
113
113
 
@@ -9,7 +9,7 @@ The project maintainers are responsible for bumping the version number, regenera
9
9
  To test the new version of `octocatalog-diff` in the GitHub Puppet repository, check out `github/puppet` and:
10
10
 
11
11
  - Start a new branch based off master
12
- - Run `script/update-octocatalog-diff -r <ocd_branch_name>`
12
+ - Run `script/update-octocatalog-diff <ocd_branch_name>`
13
13
  - Confirm and commit the result
14
14
  - Make sure all CI jobs pass
15
15
  - Run the `puppet-catalog-diff` CI job and make sure it passes and shows expected results
@@ -9,7 +9,8 @@
9
9
 
10
10
  ```
11
11
  Usage: octocatalog-diff [command line options]
12
- -n, --hostname HOSTNAME Use PuppetDB facts from last run of hostname
12
+ -n HOSTNAME1[,HOSTNAME2[,...]], Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames
13
+ --hostname
13
14
  --basedir DIRNAME Use an alternate base directory (git checkout of puppet repository)
14
15
  -f, --from FROM_BRANCH Branch you are coming from
15
16
  -t, --to TO_BRANCH Branch you are going to
@@ -856,14 +857,17 @@ Puppet control repo template, the value of this should be 'hieradata', which is
856
857
 
857
858
  <tr>
858
859
  <td valign=top>
859
- <pre><code>-n HOSTNAME
860
- --hostname HOSTNAME</code></pre>
860
+ <pre><code>-n HOSTNAME1[,HOSTNAME2[,...]]
861
+ --hostname HOSTNAME1[,HOSTNAME2[,...]]</code></pre>
861
862
  </td>
862
863
  <td valign=top>
863
- Use PuppetDB facts from last run of hostname
864
+ Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames
864
865
  </td>
865
866
  <td valign=top>
866
- Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display. (<a href="../lib/octocatalog-diff/cli/options/hostname.rb">hostname.rb</a>)
867
+ Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display.
868
+ This option can recieve a single hostname, or a comma separated list of
869
+ multiple hostnames, which are split into an Array. Multiple hostnames do not
870
+ work with the `catalog-only` or `bootstrap-then-exit` options. (<a href="../lib/octocatalog-diff/cli/options/hostname.rb">hostname.rb</a>)
867
871
  </td>
868
872
  </tr>
869
873
 
@@ -294,10 +294,13 @@ module OctocatalogDiff
294
294
  # Use diffy to get only the lines that have changed in a text object.
295
295
  # As we iterate through the diff, jump out if we have our answer: either
296
296
  # true if '=~>' finds ANY match, or false if '=&>' fails to find a match.
297
- Diffy::Diff.new(old_val, new_val, context: 0).each do |line|
297
+ diffy_result = Diffy::Diff.new(old_val, new_val, context: 0)
298
+ newline_alerts = diffy_result.count { |line| line.strip == '\' }
299
+ diffy_result.each do |line|
298
300
  if regex.match(line.strip)
299
301
  return true if operator == '=~>'
300
302
  elsif operator == '=&>'
303
+ next if line.strip == '\' && newline_alerts == 2
301
304
  return false
302
305
  end
303
306
  end
@@ -394,6 +397,13 @@ module OctocatalogDiff
394
397
  return false unless rule[:title].casecmp(hsh[:title]).zero?
395
398
  end
396
399
 
400
+ # If rule[:attr] is a regular expression, handle that case here.
401
+ if rule[:attr].is_a?(Regexp)
402
+ return false unless hsh[:attr].is_a?(String)
403
+ return false unless rule[:attr].match(hsh[:attr])
404
+ return ignore_match_true(hsh, rule)
405
+ end
406
+
397
407
  # Special 'attributes': Ignore specific diff types (+ add, - remove, ~ and ! change)
398
408
  if rule[:attr] =~ /\A[\-\+~!]+\Z/
399
409
  return ignore_match_true(hsh, rule) if rule[:attr].include?(diff_type)
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require_relative '../filter'
4
+ require_relative '../../util/util'
4
5
 
5
6
  module OctocatalogDiff
6
7
  module CatalogDiff
@@ -35,43 +36,46 @@ module OctocatalogDiff
35
36
 
36
37
  # Check for a change where the difference in a parameter exactly corresponds to the difference in the
37
38
  # compilation directory.
38
- if diff.change? && (diff.old_value.is_a?(String) || diff.new_value.is_a?(String))
39
- from_before = nil
40
- from_after = nil
41
- from_match = false
42
- to_before = nil
43
- to_after = nil
44
- to_match = false
39
+ if diff.change?
40
+ o = remove_compilation_dir(diff.old_value, dir2)
41
+ n = remove_compilation_dir(diff.new_value, dir1)
45
42
 
46
- if diff.old_value =~ /^(.*)#{dir2}(.*)$/m
47
- from_before = Regexp.last_match(1) || ''
48
- from_after = Regexp.last_match(2) || ''
49
- from_match = true
50
- end
51
-
52
- if diff.new_value =~ /^(.*)#{dir1}(.*)$/m
53
- to_before = Regexp.last_match(1) || ''
54
- to_after = Regexp.last_match(2) || ''
55
- to_match = true
56
- end
57
-
58
- if from_match && to_match && to_before == from_before && to_after == from_after
43
+ if o != diff.old_value || n != diff.new_value
59
44
  message = "Resource key #{diff.type}[#{diff.title}] #{diff.structure.join(' => ')}"
60
- message += ' appears to depend on catalog compilation directory. Suppressed from results.'
45
+ message += ' may depend on catalog compilation directory, but there may be differences.'
46
+ message += ' This is included in results for now, but please verify.'
61
47
  @logger.warn message
62
- return true
63
48
  end
64
49
 
65
- if from_match || to_match
50
+ if o == n
66
51
  message = "Resource key #{diff.type}[#{diff.title}] #{diff.structure.join(' => ')}"
67
- message += ' may depend on catalog compilation directory, but there may be differences.'
68
- message += ' This is included in results for now, but please verify.'
52
+ message += ' appears to depend on catalog compilation directory. Suppressed from results.'
69
53
  @logger.warn message
54
+ return true
70
55
  end
71
56
  end
72
57
 
73
58
  false
74
59
  end
60
+
61
+ def remove_compilation_dir(v, dir)
62
+ value = OctocatalogDiff::Util::Util.deep_dup(v)
63
+ traverse(value) do |e|
64
+ e.gsub!(dir, '') if e.respond_to?(:gsub!)
65
+ end
66
+ value
67
+ end
68
+
69
+ def traverse(a)
70
+ case a
71
+ when Array
72
+ a.map { |v| traverse(v, &Proc.new) }
73
+ when Hash
74
+ traverse(a.values, &Proc.new)
75
+ else
76
+ yield a
77
+ end
78
+ end
75
79
  end
76
80
  end
77
81
  end
@@ -331,8 +331,9 @@ module OctocatalogDiff
331
331
  title = normalized_title(resource['title'], resource['type'])
332
332
  @resource_hash[resource['type']][title] = resource
333
333
 
334
- if resource.key?('parameters') && resource['parameters'].key?('alias')
335
- @resource_hash[resource['type']][resource['parameters']['alias']] = resource
334
+ if resource.key?('parameters')
335
+ @resource_hash[resource['type']][resource['parameters']['alias']] = resource if resource['parameters'].key?('alias')
336
+ @resource_hash[resource['type']][resource['parameters']['name']] = resource if resource['parameters'].key?('name')
336
337
  end
337
338
  end
338
339
  end
@@ -11,6 +11,7 @@ require_relative 'util/util'
11
11
  require_relative 'version'
12
12
 
13
13
  require 'logger'
14
+ require 'parallel'
14
15
  require 'socket'
15
16
 
16
17
  module OctocatalogDiff
@@ -116,16 +117,46 @@ module OctocatalogDiff
116
117
  end
117
118
 
118
119
  # Compile catalogs and do catalog-diff
119
- catalog_diff = OctocatalogDiff::API::V1.catalog_diff(options.merge(logger: logger))
120
+ node_set = options.delete(:node)
121
+ node_set = [node_set] unless node_set.is_a?(Array)
122
+
123
+ # run multiple node diffs in parallel
124
+ catalog_diffs = if node_set.size == 1
125
+ [run_octocatalog_diff(node_set.first, options, logger)]
126
+ else
127
+ ::Parallel.map(node_set, in_threads: 4) { |node| run_octocatalog_diff(node, options, logger) }
128
+ end
129
+
130
+ # Return the resulting diff object if requested (generally for testing)
131
+ # or otherwise return exit code
132
+ return catalog_diffs.first if opts[:INTEGRATION]
133
+
134
+ all_diffs = catalog_diffs.map(&:diffs)
135
+
136
+ all_diffs.each do |diff|
137
+ next unless diff.any?
138
+ return EXITCODE_SUCCESS_WITH_DIFFS
139
+ end
140
+
141
+ EXITCODE_SUCCESS_NO_DIFFS
142
+ end
143
+
144
+ # Run the octocatalog-diff process for a given node. Return the diffs for a contribution to
145
+ # the final exit status.
146
+ # node - String with the node
147
+ # options - All of the currently defined options
148
+ # logger - Logger object
149
+ def self.run_octocatalog_diff(node, options, logger)
150
+ options_copy = options.merge(node: node)
151
+ catalog_diff = OctocatalogDiff::API::V1.catalog_diff(options_copy.merge(logger: logger))
120
152
  diffs = catalog_diff.diffs
121
153
 
122
154
  # Display diffs
123
- printer_obj = OctocatalogDiff::Cli::Printer.new(options, logger)
155
+ printer_obj = OctocatalogDiff::Cli::Printer.new(options_copy, logger)
124
156
  printer_obj.printer(diffs, catalog_diff.from.compilation_dir, catalog_diff.to.compilation_dir)
125
157
 
126
- # Return the resulting diff object if requested (generally for testing) or otherwise return exit code
127
- return catalog_diff if opts[:INTEGRATION]
128
- diffs.any? ? EXITCODE_SUCCESS_WITH_DIFFS : EXITCODE_SUCCESS_NO_DIFFS
158
+ # Return catalog-diff object.
159
+ catalog_diff
129
160
  end
130
161
 
131
162
  # Parse command line options with 'optparse'. Returns a hash with the parsed arguments.
@@ -11,7 +11,7 @@ module OctocatalogDiff
11
11
  # This class contains the option parser. 'parse_options' is the external entry point.
12
12
  class Options
13
13
  # The usage banner.
14
- BANNER = 'Usage: catalog-diff -n <hostname> [-f <from environment>] [-t <to environment>]'.freeze
14
+ BANNER = 'Usage: catalog-diff -n <hostname>[,<hostname>...] [-f <from environment>] [-t <to environment>]'.freeze
15
15
 
16
16
  # An error class specifically for passing information to the document build task.
17
17
  class DocBuildError < RuntimeError; end
@@ -1,6 +1,9 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  # Set hostname, which is used to look up facts in PuppetDB, and in the header of diff display.
4
+ # This option can recieve a single hostname, or a comma separated list of
5
+ # multiple hostnames, which are split into an Array. Multiple hostnames do not
6
+ # work with the `catalog-only` or `bootstrap-then-exit` options.
4
7
  # @param parser [OptionParser object] The OptionParser argument
5
8
  # @param options [Hash] Options hash being constructed; this is modified in this method.
6
9
 
@@ -8,8 +11,16 @@ OctocatalogDiff::Cli::Options::Option.newoption(:hostname) do
8
11
  has_weight 1
9
12
 
10
13
  def parse(parser, options)
11
- parser.on('--hostname HOSTNAME', '-n', 'Use PuppetDB facts from last run of hostname') do |hostname|
12
- options[:node] = hostname
14
+ parser.on(
15
+ '--hostname HOSTNAME1[,HOSTNAME2[,...]]',
16
+ '-n',
17
+ 'Use PuppetDB facts from last run of a hostname or a comma separated list of multiple hostnames'
18
+ ) do |hostname|
19
+ options[:node] = if hostname.include?(',')
20
+ hostname.split(',')
21
+ else
22
+ hostname
23
+ end
13
24
  end
14
25
  end
15
26
  end
@@ -1,4 +1,4 @@
1
- #!/bin/bash
1
+ #!/usr/bin/env bash
2
2
 
3
3
  # This script echoes back the environment. This is used for spec testing
4
4
  # and possible debugging.
@@ -1,4 +1,4 @@
1
- #!/bin/bash
1
+ #!/usr/bin/env bash
2
2
 
3
3
  # This script is called from lib/octocatalog-diff/catalog-util/git.rb and is used to
4
4
  # archive and extract a certain branch of a git repository into a target directory.
@@ -1,4 +1,4 @@
1
- #!/bin/bash
1
+ #!/usr/bin/env bash
2
2
 
3
3
  # Script to run Puppet. The default implementation here is simply to pass
4
4
  # through the command line arguments (which are likely to be numerous when
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: octocatalog-diff
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.5.3
4
+ version: 1.5.4
5
5
  platform: ruby
6
6
  authors:
7
7
  - GitHub, Inc.
@@ -9,7 +9,7 @@ authors:
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2018-03-05 00:00:00.000000000 Z
12
+ date: 2018-12-13 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: diffy
@@ -53,6 +53,20 @@ dependencies:
53
53
  - - ">="
54
54
  - !ruby/object:Gem::Version
55
55
  version: 0.3.0
56
+ - !ruby/object:Gem::Dependency
57
+ name: parallel
58
+ requirement: !ruby/object:Gem::Requirement
59
+ requirements:
60
+ - - ">="
61
+ - !ruby/object:Gem::Version
62
+ version: 1.12.0
63
+ type: :runtime
64
+ prerelease: false
65
+ version_requirements: !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ version: 1.12.0
56
70
  - !ruby/object:Gem::Dependency
57
71
  name: rugged
58
72
  requirement: !ruby/object:Gem::Requirement
@@ -87,14 +101,14 @@ dependencies:
87
101
  requirements:
88
102
  - - '='
89
103
  - !ruby/object:Gem::Version
90
- version: 11.2.2
104
+ version: 12.3.1
91
105
  type: :development
92
106
  prerelease: false
93
107
  version_requirements: !ruby/object:Gem::Requirement
94
108
  requirements:
95
109
  - - '='
96
110
  - !ruby/object:Gem::Version
97
- version: 11.2.2
111
+ version: 12.3.1
98
112
  - !ruby/object:Gem::Dependency
99
113
  name: parallel_tests
100
114
  requirement: !ruby/object:Gem::Requirement
@@ -171,14 +185,14 @@ dependencies:
171
185
  requirements:
172
186
  - - "~>"
173
187
  - !ruby/object:Gem::Version
174
- version: 5.4.0
188
+ version: 5.5.8
175
189
  type: :development
176
190
  prerelease: false
177
191
  version_requirements: !ruby/object:Gem::Requirement
178
192
  requirements:
179
193
  - - "~>"
180
194
  - !ruby/object:Gem::Version
181
- version: 5.4.0
195
+ version: 5.5.8
182
196
  description: |
183
197
  Octocatalog-Diff assists with Puppet development and testing by enabling the user to
184
198
  compile 2 Puppet catalogs and compare them. It is possible to compare different