knife-depsolver 2.1.0 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 20ead315f5ba6d98327f23b83076eb7204362143
-  data.tar.gz: 8431be0109945a59cba7b7e6498a240ff611ec12
+  metadata.gz: 0b190d248ebfddfe1f8ba7c211cf779396daae1e
+  data.tar.gz: a1f372988af3570d26d80aaa5d459bc5193a95f5
 SHA512:
-  metadata.gz: d292509903debe72d884dc499e202782a01bcf51c8251d55542bc4460ddc54e9a4bc51e0873c78272698ed7206cc9b033d9fb399edbe69480e6148e58befbccf
-  data.tar.gz: 7378fc8d7d0b78722f634b670affd99f918c478f964a1f0b4edb51501882168a3da4b7bbb40bc8aff74063c61efba237b4c8b6d056fce5795b7b107995cd694f
+  metadata.gz: 4a8ef5c1d2e361d0bfc6e1ddaeeb1ffd3264c7c7589c4a40bdc941184f99e28a5bcd93023329815b311964a5caeeac021a6dfdf0581b582004f5f73b4791be86
+  data.tar.gz: 9368b84cf5d6de137e8ad00abe7943e6d1a7a174e9766eb8d09059cb685a48576ea0cec15f54d8815ae4e4185365baa582e3b18a5f757762edee3fd36db7a4ab

data/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Change Log
 
+## 2.2.0 (2017-05-19)
+
+* Rename --print-trimmed-universe to --filter-universe
+* Remove --env-constraints-filter-universe option
+* Don't filter cookbook universe by run list when run list is empty
+* Add --version-pin-cookbooks option
+
 ## 2.1.0 (2017-05-15)
 
 * Replace --print-constrained-cookbook-set with --print-trimmed-universe

data/README.md

@@ -75,6 +75,8 @@ In the first post he goes on to talk about the importance of troubleshooting the
 
 knife-depsolver can provide many troubleshooting options by using the depsolver embedded in your workstation's Chef DK to make an identical calculation as the Chef Server.
 
+#### Use the appropriate version of Chef DK
+
 First, you need to make sure that your version of Chef DK is using the same version of the dep_selector gem as your version of Chef Server. If your Chef DK is using a different version of the dep_selector gem then knife-depsolver's calculations will not be identical to the Chef Server's calculations which will confuse troubleshooting efforts.
 
 | dep_selector gem | Chef Server | Chef DK    |
@@ -82,6 +84,8 @@ First, you need to make sure that your version of Chef DK is using the same vers
 | 1.0.3            | <= 12.8.0   | <= 0.16.28 |
 | 1.0.4            | >= 12.9.0   | >= 0.17.17 |
 
+#### --capture
+
 Once you are sure you are using the correct version of Chef DK and you have installed the knife-depsolver plugin you can use the "--capture" option which queries the Chef Server and creates a separate file on the workstation for each of the following pieces of information.
 
 1. environment cookbook version constraints
@@ -98,7 +102,9 @@ OR
 knife depsolver -E production 'role[base],cookbook-B,cookbook-A::foo@3.1.4,cookbook-R' --capture
 ```
 
-Now use the "--env-constraints", "universe" and "expanded-run-list" options to provide all the information required for local depsolver calculations.
+#### --env-constraints, --universe and --expanded-run-list
+
+Now use the "--env-constraints", "--universe" and "--expanded-run-list" options to provide the information required for local depsolver calculations.
 
 For example:
 
@@ -106,7 +112,7 @@ For example:
 knife depsolver --expanded-run-list expanded-run-list-2017-05-01-18.52.40-387d90499514747792a805213c30be13d830d31f.txt --universe my-org-universe-2017-05-01-18.52.40-1c8e59e23530b1e1a8e0b3b3cc5236a29c84e469.txt --env-constraints production-environment-2017-05-01-18.52.40-5f5843d819ecb0b174f308d76d4336bb7bbfacbf.txt
 ```
 
-Now it is easy to modify the environment cookbook version constraints or the cookbook universe input files to see the impact on the depsolver.
+This makes it easy to modify the input files to see the impact on the depsolver.
 
 #### --timeout
 
@@ -116,17 +122,25 @@ Sometimes it can help to give the depsolver more than the default five seconds t
 knife depsolver --expanded-run-list expanded-run-list-2017-05-01-18.52.40-387d90499514747792a805213c30be13d830d31f.txt --universe my-org-universe-2017-05-01-18.52.40-1c8e59e23530b1e1a8e0b3b3cc5236a29c84e469.txt --env-constraints production-environment-2017-05-01-18.52.40-5f5843d819ecb0b174f308d76d4336bb7bbfacbf.txt --timeout 120
 ```
 
-#### --print-trimmed-universe
+#### --filter-universe
 
-Sometimes you want to only see the list of cookbooks, and their cookbook dependency version constraints, that ultimately would be sent to the depsolver without actually triggering the depsolver calculation. This is especially helpful when the depsolver isn't returning any results because it can't calculate a solution in a reasonable amount of time. This can be done by using the `--print-trimmed-universe` option.
+Combine the --universe and the --filter-universe options with --env-constraints and/or --expanded-run-list options to filter the cookbook universe based on the environment cookbook version constraints and/or the expanded run list. This provides the ability to focus only on relevant information that would be sent to the depsolver without actually triggering the depsolver calculation. This is especially helpful when the depsolver isn't returning any results because it can't calculate a solution in a reasonable amount of time.
 
 ```
-knife depsolver --expanded-run-list expanded-run-list-2017-05-01-18.52.40-387d90499514747792a805213c30be13d830d31f.txt --universe my-org-universe-2017-05-01-18.52.40-1c8e59e23530b1e1a8e0b3b3cc5236a29c84e469.txt --env-constraints production-environment-2017-05-01-18.52.40-5f5843d819ecb0b174f308d76d4336bb7bbfacbf.txt --print-trimmed-universe
+knife depsolver --expanded-run-list expanded-run-list-2017-05-01-18.52.40-387d90499514747792a805213c30be13d830d31f.txt --universe my-org-universe-2017-05-01-18.52.40-1c8e59e23530b1e1a8e0b3b3cc5236a29c84e469.txt --env-constraints production-environment-2017-05-01-18.52.40-5f5843d819ecb0b174f308d76d4336bb7bbfacbf.txt --filter-universe > filtered-universe.txt
 ```
 
 Now you can review the output to see if you can find anything that could be causing problems for the depsolver. You can make changes to the input files to see the impact on the list of cookbooks that would be sent to the depsolver.
 
-If necessary you could also use the list of cookbooks as a starting point for setting version constraints in the environment input file. Then you can modify that set of version constraints and run the depsolver in an effort to get a solution in a reasonable amount of time or to isolate the problem.
+If necessary you could also use the results along with the --version-pin-cookbooks option to create an environment input file that pins a version for each cookbook. Then you can modify that set of version constraints and run the depsolver in an effort to get a solution in a reasonable amount of time or to isolate the problem.
+
+#### --version-pin-cookbooks
+
+The --version-pin-cookbooks option takes a filename as an argument. The file must be in the JSON format of a cookbook universe or filtered cookbook universe. The --version-pin-cookbooks option creates an environment JSON file with cookbook version constraints for that pin every cookbook to its latest version. This environment file can then be used with the --env-constraints option to provide a strictly constrained starting point when troubleshooting a depsolver issue.
+
+```
+knife depsolver --version-pin-cookbooks filtered-universe.txt > version-pinned-cookbooks.txt
+```
 
 #### /tmp/DepSelectorDebugOn
 

data/lib/chef/knife/depsolver.rb

@@ -45,59 +45,48 @@ class Chef
              long: '--csv-universe-to-json FILENAME',
              description: 'Convert a CSV cookbook universe, FILENAME, to JSON.'
 
-      option :env_constraints_filter_universe,
-           long: '--env-constraints-filter-universe',
-           description: 'Filter the cookbook universe using the environment cookbook version constraints.'
+      option :filter_universe,
+           long: '--filter-universe',
+           description: 'Filter the cookbook universe by environment cookbook version constraints and an optional run list.'
 
-      option :print_trimmed_universe,
-           long: '--print-trimmed-universe',
+      option :version_pin_cookbooks,
+           long: '--version-pin-cookbooks FILENAME',
            description: 'Print the cookbooks and dependency data that would be sent to the depsolver.'
 
       def run
         begin
           DepSelector::Debug.log.level = Logger::INFO if defined?(DepSelector::Debug)
-          use_local_depsolver = false
-          if config[:env_constraints_filter_universe]
-            if config[:node] || config[:environment] || config[:timeout] || config[:capture] || config[:expanded_run_list] || config[:csv_universe_to_json]
-              msg("ERROR: The --env-constraints-filter-universe option is only compatible with the --env-constraints and --universe options")
+          use_local_depsolver = config[:universe] ? true : false
+          if use_local_depsolver
+            unless name_args.empty?
+              puts "ERROR: Setting a run list on the command line is not compatible with the --universe option"
+              exit!
+            end
+            if config[:node]
+              puts "ERROR: The --node option is not compatible with the --universe option"
               exit!
-            elsif !(config[:env_constraints] && config[:universe])
-              msg("ERROR: The --env-constraints-filter-universe option requires the --env-constraints and --universe options to be set")
+            end
+            if config[:environment]
+              puts "ERROR: The --environment option is not compatible with the --universe option"
+              exit!
+            end
+            if config[:capture]
+              puts "ERROR: The --capture option is not compatible with the --universe option"
               exit!
             end
           else
-            if config[:env_constraints] || config[:universe] || config[:expanded_run_list]
-              if config[:env_constraints] && config[:universe] && config[:expanded_run_list]
-                use_local_depsolver = true
-                unless name_args.empty?
-                  puts "ERROR: Setting a run list on the command line is not compatible with the --env-constraints, --universe or --expanded-run-list options"
-                  exit!
-                end
-                if config[:node]
-                  puts "ERROR: The --node option is not compatible with the --env-constraints, --universe or --expanded-run-list options"
-                  exit!
-                end
-                if config[:environment]
-                  puts "ERROR: The --environment option is not compatible with the --env-constraints, --universe or --expanded-run-list options"
-                  exit!
-                end
-                if config[:capture]
-                  puts "ERROR: The --capture option is not compatible with the --env-constraints, --universe or --expanded-run-list options"
-                  exit!
-                end
-              else
-                puts "ERROR: The --env-constraints, --universe and --expanded-run-list options must be used together to use the local depsolver"
-                exit!
-              end
+            if (config[:env_constraints] || config[:expanded_run_list])
+              puts "ERROR: The --env-constraints and --expanded-run-list options require the --universe option to be set"
+              exit!
+            end
+            if config[:timeout]
+              msg("ERROR: The --timeout option requires the --env-constraints, --universe and --expanded-run-list options to be set")
+              exit!
+            end
+            if config[:filter_universe]
+              msg("ERROR: The --filter-universe option requires the --universe option and optionally the --env-constraints and/or --expanded-run-list options")
+              exit!
             end
-          end
-          if config[:timeout] && !use_local_depsolver
-            msg("ERROR: The --timeout option requires the --env-constraints, --universe and --expanded-run-list options to be set")
-            exit!
-          end
-          if config[:print_constrained_cookbook_set] && !use_local_depsolver
-            msg("ERROR: The --print-constrained-cookbook-set option requires the --env-constraints, --universe and --expanded-run-list options to be set")
-            exit!
           end
 
           timeout = (config[:timeout].to_f * 1000).to_i if config[:timeout]
@@ -121,7 +110,22 @@ class Chef
             exit!
           end
 
-          if config[:node] && !(config[:env_constraints_filter_universe] && config[:env_constraints])
+          if config[:version_pin_cookbooks]
+            unless File.file?(config[:version_pin_cookbooks])
+              msg("ERROR: #{config[:version_pin_cookbooks]} does not exist or is not a file.")
+              exit!
+            end
+            cookbooks = JSON.parse(IO.read(config[:version_pin_cookbooks]))
+            version_pinned_cookbooks = Hash.new
+            cookbooks.each do |cookbook_name, versions|
+              max_version = versions.keys.max_by {|v| DepSelector::Version.new(v) }
+              version_pinned_cookbooks[cookbook_name] = "= #{max_version}" if max_version
+            end
+            puts JSON.pretty_generate({name: "version-pinned-cookbooks", cookbook_versions: version_pinned_cookbooks})
+            exit!
+          end
+
+          if config[:node]
             node = Chef::Node.load(config[:node])
           else
             node = Chef::Node.new
@@ -152,7 +156,11 @@ class Chef
             end
           end
 
+          # node.chef_environment must be set before expanding the run list in case the run list includes
+          # roles with environment specific run lists
           node.chef_environment = config[:environment] if config[:environment]
+          run_list_expansion = node.run_list.expand(node.chef_environment, 'server')
+          expanded_run_list_with_versions = run_list_expansion.recipes.with_version_constraints_strings
 
           if config[:capture]
             if node.chef_environment == '_default'
@@ -179,86 +187,18 @@ class Chef
               puts "WARNING: Try capturing the cookbook universe using the SQL query found in the knife-depsolver README."
               puts "WARNING: Then convert the results using knife-depsolver's --csv-universe-to-json option"
             end
+            exit
           end
 
-          if config[:env_constraints]
-            unless File.file?(config[:env_constraints])
-              msg("ERROR: #{config[:env_constraints]} does not exist or is not a file.")
-              exit!
-            end
-            env = JSON.parse(IO.read(config[:env_constraints]))
-            if env['name'].to_s.empty?
-              msg("ERROR: #{config[:env_constraints]} does not contain an environment name.")
-              exit!
-            else
-              node.chef_environment = env['name']
-            end
-            if !env['cookbook_versions'].is_a?(Hash)
-              msg("ERROR: #{config[:env_constraints]} does not contain a Hash of cookbook version constraints.")
-              exit!
-            else
-              environment_cookbook_versions = env['cookbook_versions']
-            end
-          end
-
-          if config[:universe]
-            unless File.file?(config[:universe])
-              msg("ERROR: #{config[:universe]} does not exist or is not a file.")
-              exit!
-            end
-            universe = JSON.parse(IO.read(config[:universe]))
-            if !universe.is_a?(Hash)
-              msg("ERROR: #{config[:universe]} does not contain a cookbook universe Hash.")
-              exit!
-            end
-          end
-
-          if config[:env_constraints_filter_universe]
-            env_constraints = environment_cookbook_versions.each_with_object({}) do |env_constraint, memo|
-              name, constraint = env_constraint
-              constraint, version = constraint.split
-              memo[name] = DepSelector::VersionConstraint.new(constraint_to_str(constraint, version))
-            end
-            universe.each do |name, versions|
-              versions.delete_if {|version, v| !env_constraints[name].include?(version)} if env_constraints[name]
-            end
-            filtered_universe_json = JSON.pretty_generate(universe)
-            filtered_universe_filename = "filtered-universe-#{Time.now.strftime("%Y-%m-%d-%H.%M.%S")}-#{Digest::SHA1.hexdigest(filtered_universe_json)}.txt"
-            IO.write(filtered_universe_filename, filtered_universe_json)
-            puts "Filtered cookbook universe saved to #{filtered_universe_filename}"
+          if config[:filter_universe]
+            data = get_depsolver_data(node, expanded_run_list_with_versions, timeout)
+            puts JSON.pretty_generate(filter_universe(data))
             exit!
           end
 
-          run_list_expansion = node.run_list.expand(node.chef_environment, 'server')
-          expanded_run_list_with_versions = run_list_expansion.recipes.with_version_constraints_strings
-
-          exit if config[:capture]
-
           depsolver_results = Hash.new
           if use_local_depsolver
-            env_ckbk_constraints = environment_cookbook_versions.map do |ckbk_name, ckbk_constraint|
-              [ckbk_name, ckbk_constraint.split.reverse].flatten
-            end
-
-            all_versions = universe.map do |ckbk_name, ckbk_metadata|
-              ckbk_versions = ckbk_metadata.map do |version, version_metadata|
-                [version, version_metadata['dependencies'].map { |dep_ckbk_name, dep_ckbk_constraint| [dep_ckbk_name, dep_ckbk_constraint.split.reverse].flatten }]
-              end
-              [ckbk_name, ckbk_versions]
-            end
-
-            expanded_run_list_with_split_versions = expanded_run_list_with_versions.map do |run_list_item|
-              name, version = run_list_item.split('@')
-              name.sub!(/::.*/, '')
-              version ? [name, version] : name
-            end
-
-            data = {environment_constraints: env_ckbk_constraints, all_versions: all_versions, run_list: expanded_run_list_with_split_versions, timeout_ms: timeout}
-
-            if config[:print_trimmed_universe]
-              puts JSON.pretty_generate(get_trimmed_universe(data))
-              exit!
-            end
+            data = get_depsolver_data(node, expanded_run_list_with_versions, timeout)
 
             depsolver_start_time = Time.now
 
@@ -338,7 +278,61 @@ class Chef
         end
       end
 
-      def get_trimmed_universe(data)
+      def get_depsolver_data(node, expanded_run_list_with_versions, timeout)
+        if config[:env_constraints]
+          unless File.file?(config[:env_constraints])
+            msg("ERROR: #{config[:env_constraints]} does not exist or is not a file.")
+            exit!
+          end
+          env = JSON.parse(IO.read(config[:env_constraints]))
+          if env['name'].to_s.empty?
+            msg("ERROR: #{config[:env_constraints]} does not contain an environment name.")
+            exit!
+          else
+            node.chef_environment = env['name']
+          end
+          if !env['cookbook_versions'].is_a?(Hash)
+            msg("ERROR: #{config[:env_constraints]} does not contain a Hash of cookbook version constraints.")
+            exit!
+          else
+            environment_cookbook_versions = env['cookbook_versions']
+          end
+        else
+          environment_cookbook_versions = Hash.new
+        end
+
+        env_ckbk_constraints = environment_cookbook_versions.map do |ckbk_name, ckbk_constraint|
+          [ckbk_name, ckbk_constraint.split.reverse].flatten
+        end
+
+        unless File.file?(config[:universe])
+          msg("ERROR: #{config[:universe]} does not exist or is not a file.")
+          exit!
+        end
+        universe = JSON.parse(IO.read(config[:universe]))
+        if !universe.is_a?(Hash)
+          msg("ERROR: #{config[:universe]} does not contain a cookbook universe Hash.")
+          exit!
+        end
+
+        all_versions = universe.map do |ckbk_name, ckbk_metadata|
+          ckbk_versions = ckbk_metadata.map do |version, version_metadata|
+            [version, version_metadata['dependencies'].map { |dep_ckbk_name, dep_ckbk_constraint| [dep_ckbk_name, dep_ckbk_constraint.split.reverse].flatten }]
+          end
+          [ckbk_name, ckbk_versions]
+        end
+
+        expanded_run_list_with_split_versions = expanded_run_list_with_versions.map do |run_list_item|
+          name, version = run_list_item.split('@')
+          name.sub!(/::.*/, '')
+          version ? [name, version] : name
+        end
+
+        depsolver_data = {environment_constraints: env_ckbk_constraints, all_versions: all_versions, run_list: expanded_run_list_with_split_versions, timeout_ms: timeout}
+        depsolver_data
+      end
+
+      def filter_universe(data)
           # create dependency graph from cookbooks
           graph = DepSelector::DependencyGraph.new
 
@@ -384,19 +378,23 @@ class Chef
           timeout_ms = data[:timeout_ms]
           selector = DepSelector::Selector.new(graph, (timeout_ms / 1000.0))
 
-          constrained_cookbook_set = selector.send(:trim_unreachable_packages, selector.dep_graph, run_list)
-          trimmed_universe = Hash.new { |h,k| h[k] = Hash.new { |h,k| h[k] = Hash.new { |h,k| h[k] = Hash.new } } }
+          if run_list.empty?
+            constrained_cookbook_set = selector.dep_graph.packages.values
+          else
+            constrained_cookbook_set = selector.send(:trim_unreachable_packages, selector.dep_graph, run_list)
+          end
+          filtered_universe = Hash.new { |h,k| h[k] = Hash.new { |h,k| h[k] = Hash.new { |h,k| h[k] = Hash.new } } }
           constrained_cookbook_set.each do |ckbk|
             ckbk.versions.each do |ckbk_version|
-              trimmed_universe[ckbk.name][ckbk_version.version]["dependencies"] = Hash.new
+              filtered_universe[ckbk.name][ckbk_version.version]["dependencies"] = Hash.new
               ckbk_version.dependencies.each do |ckbk_version_dep|
                 constraint = ckbk_version_dep.constraint
                 constraint_version = constraint.missing_patch_level ? "#{constraint.version.major}.#{constraint.version.minor}" : "#{constraint.version}"
-                trimmed_universe[ckbk.name][ckbk_version.version]["dependencies"][ckbk_version_dep.package.name] = "#{constraint.op} #{constraint_version}"
+                filtered_universe[ckbk.name][ckbk_version.version]["dependencies"][ckbk_version_dep.package.name] = "#{constraint.op} #{constraint_version}"
               end
             end
           end
-          trimmed_universe
+          filtered_universe
       end
     end
   end

data/lib/knife-depsolver/version.rb

@@ -1,3 +1,3 @@
 module KnifeDepsolver
-  VERSION = "2.1.0"
+  VERSION = "2.2.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: knife-depsolver
 version: !ruby/object:Gem::Version
-  version: 2.1.0
+  version: 2.2.0
 platform: ruby
 authors:
 - Jeremiah Snapp
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-05-15 00:00:00.000000000 Z
+date: 2017-05-19 00:00:00.000000000 Z
 dependencies: []
 description: Knife plugin that uses Chef Server to calculate cookbook dependencies
   for a given run_list.