knife-audit 0.0.1 → 0.2.0

data/README.md

@@ -1,7 +1,7 @@
 knife-audit
 ========
 A knife plugin for determining which cookbooks are in use on which nodes of your Chef server or Opscode organization.
-Allows you to safely maintain a chef cookbook set by determining which cookbooks are currently in use by nodes (included in node runlists).
+Allows you to safely maintain a chef cookbook set by determining which cookbooks are currently in use by nodes - either solely via inclusion in node runlists (available from every node) or via runlist *or* include_recipes (for nodes with knife_audit helper cookbook installed).
 
 
 Installing knife-audit
@@ -9,25 +9,47 @@ Installing knife-audit
 
 #### Script install
 
-Copy the knife-audit script from https://github.com/jbz/knife-audit/blob/master/lib/chef/knife/audit.rb to your .chef/plugins/knife directory.
+Copy the knife-audit script from https://github.com/jbz/knife-audit/blob/master/lib/chef/knife/audit.rb to your .chef/plugins/knife directory.   Note that script-installed knife audit will be unable to install the knife_audit helper cookbook for you.
 
+#### Gem install
+
+knife-audit is available on rubygems.org - if you have that source in your gemrc, you can simply use:
+
+    gem install knife-audit
+
+...if you don't have internet access or just want a local gemfile, you can clone the repo and build/install a working gem from the main repo directory:
+
+    gem build knife-audit.gemspec
+    gem install ./knife-audit-<version>.gem
 
 Usage
 ---------------
 
-    knife audit <COOKBOOK COOKBOOK ...>
+    knife audit [-a|-t] [-s] [-i] <COOKBOOK COOKBOOK ...>
 
 If no cookbooks are specified, knife-audit will return a list of *all* cookbooks available on the currently configured Chef server or Opscode Platform organization, along with a count for each of how many nodes in the current Chef server or Opscode Platform organization explicitly reference that cookbook in their expanded runlist. 
 
-Note that this does *not* include nodes that call the cookbook via 'include' and/or 'depends' statements.  The 'complete runlist' for nodes, which includes all cookbooks pulled in due to includes, is kept in Node.run_state.seen_recipes], but this is an ephemeral attribute and is only populated locally on the node during a client run.  It is not saved to the Chef server, therefore knife-audit cannot 'see' it.
+Note that this does *not* include nodes that call the cookbook via 'include' and/or 'depends' statements.  The 'complete runlist' for nodes, which includes all cookbooks pulled in due to includes, is kept in Node.run_state.seen_recipes, but this is an ephemeral attribute and is only populated locally on the node during a client run.  It is not saved to the Chef server, therefore knife-audit cannot 'see' it 'unless' you have installed the knife_audit helper cookbook to your nodes (see 'Helper Cookbook' below).
 
 If one or more cookbook names are specified on the command line, knife-audit will return a list of only those cookbooks and their counts.  Specifying a cookbook which is not available on the Chef server will result in an error.
 
+The '-a' or '--all-cookbooks' option will cause knife-audit to check on each node for the attribute [:knife_audit][:seen_recipes] (which the helper cookbook saves there).  If it is present, it will use the contents of that attribute to determine which recipes the node is calling.  If it is not present, it will fall back (for that node) to the regular expanded runlist.  The output of knife-audit will, in this case, be in two parts:  The first will be identical to the default output and will display totals for all those nodes which do *not* have the seen_recipes attribute available.  The second part will be totals for all those nodes which *do* have the attribute available.  They are separated so that if the '-s' option is used, the nodes can be differentiated.
+
+The '-t' or '--totals' option will cause knife-audit to present a single output section containing the merged totals of all nodes with and without the helper cookbook.  This is less accurate, but still useful and easier to read.
+
 The '-s' or '--show-nodelist' option will cause knife-audit to include in its output a list of all nodes which reference each cookbook.
 
+The '-i' or '--install-cookbook' option will cause knife-audit to copy the knife_audit helper cookbook into the currently configured Chef cookbook_path.  If there is already a directory or file there with that name, it will abort.  Note that you will need to 'knife cookbook upload knife_audit' once you have done this in order to push the cookbook to your Chef server; in addition, you will need to add the knife_audit cookbook to your node runlists. See 'Helper Cookbook' below for more information.
+
 **NOTE** knife-audit retrieves an array of *all* nodes present on your chef server for each run.  As a result, it is relatively slow; if you have many ( >= 16) nodes, it will take noticeable wallclock time to complete its run.  In addition. it may use lots of memory to hold those node objects.
 
 
+Helper Cookbook
+---------------
+
+The helper cookbook (knife_audit) consists of a single recipe (default) with a single resource in it - a ruby_block which saves node.run_state.seen_recipes to the attribute node[:knife_audit][:seen_recipes].  This preserves the *complete* runlist information from seen_recipes, which chef-client does not save to the chef server after constructing it in the compile phase.  Since the helper cookbook performs this attribute copy in a ruby_block, it will occur during the execute phase, guaranteeing that seen_recipes is complete (unless your runlist contains a cookbook which modifies the node's runlist!)  As a result, knife_audit can be called at any point in your runlist without affecting its function (again, unless your runlist modifies itself; in this case, best to call it first).
+
+
 Disclaimer
 ----------
 
@@ -37,7 +59,9 @@ This is my first knife plugin, and I haven't been using Ruby that long.  Plus, I
 License terms
 -------------
 Authors:: J.B. Zimmerman 
+
 Copyright:: Copyright (c) 2009-2011 J.B. Zimmerman
+
 License:: Apache License, Version 2.0
 
 

data/knife-audit.gemspec

@@ -5,8 +5,8 @@ require "knife-audit/version"
 Gem::Specification.new do |s|
   s.name        = "knife-audit"
   s.version     = Knife::Audit::VERSION
-  s.authors     = ["Jacob Zimmerman"]
-  s.email       = ["jzimmerman@mdsol.com"]
+  s.authors     = ["J.B. Zimmerman"]
+  s.email       = ["jbzimmerman91@gmail.com"]
   s.homepage    = "https://github.com/jbz/knife-audit"
   s.summary     = %q{A Chef plugin for determining which cookbooks are in use on which nodes of your Chef server or Opscode organization.}
   s.description = %q{Allows you to safely maintain a chef cookbook set by determining which cookbooks are currently in use by nodes (included in node runlists).}

data/lib/chef/knife/audit.rb

@@ -1,5 +1,5 @@
 #
-## Author:: Jacob Zimmermann (<jzimmerman@mdsol.com>)
+## Author:: Jacob Zimmermann (<jbzimmerman91@gmail.com>)
 ##
 ## Licensed under the Apache License, Version 2.0 (the "License");
 ## you may not use this file except in compliance with the License.
@@ -36,6 +36,21 @@ module KnifeAudit
       :long => "--show-nodelist",
       :description => "Show all nodes running each cookbook"
 
+    option :all_cookbooks,
+      :short => "-a",
+      :long => "--all-cookbooks",
+      :description => "Show all cookbook references, including those from seen_recipes if available on nodes from helper cookbook"
+
+    option :totals,
+      :short => "-t",
+      :long => "--totals",
+      :description => "Show cookbook count totals for all node types"
+
+    option :install_cookbook,
+      :short => "-i",
+      :long => "--install-cookbook",
+      :description => "Install knife_audit helper cookbook into current chef cookbook directory"
+
     def run
 
       if @name_args.empty? 
@@ -45,6 +60,27 @@ module KnifeAudit
       end
 
       self.config = Chef::Config.merge!(config)
+
+      # If :install_cookbook flag is set, just install the cookbook and return (exit).
+      if config[:install_cookbook]
+
+        unless config[:cookbook_path]
+          ui.msg("No cookbook path set in Chef config, cannot install cookbook.")
+          return
+        end
+      
+        source_path = File.dirname(__FILE__) + "/knife_audit_cookbook"
+        dest_path = config[:cookbook_path].first + "/knife_audit"
+
+        if File.exist?(dest_path)
+          ui.msg("knife_audit cookbook already present in cookbook directory #{config[:cookbook_path].first} - aborting...")
+        else
+          FileUtils.copy_entry(source_path, dest_path)
+          ui.msg("knife-audit cookbook copied to Chef cookbook directory #{config[:cookbook_path].first}")
+        end
+
+        return
+      end
    
       # 1) Get a list (hash, actually, with key of 'name') of cookbooks available on the current server/org
       #    unless we've been given a cookbook/cookbooks on the command line
@@ -71,8 +107,10 @@ module KnifeAudit
 
       # add count => 0 to each cookbook hash
       cookbook_list.each do |name,book|
-        book["count"] = 0 
+        book["count"] = 0
+        book["seen_recipe_count"] = 0
         book["nodes"] = []
+        book["seen_recipe_nodes"] = []
       end
 
 
@@ -89,38 +127,80 @@ module KnifeAudit
 
         # 3a) Get node's runlist
 
-        # using expand!.recipes catches multi-level roles (roles with roles with recipes, etc.)
-        recipes = node.expand!.recipes.to_a
+        # Check to see if we need the seen_recipes total or not; if no, skip.  
+        # If yes use seen_recipes if it's available. If it's not available, fall back 
+        # to the node.recipes contents.
+        if (config[:all_cookbooks] || config[:totals])
+          recipes = (node["knife_audit"] && node["knife_audit"]["seen_recipes"].keys) || node.expand!.recipes.to_a
+          if node["knife_audit"] && node["knife_audit"]["seen_recipes"] 
+            node_seen_recipe_flag = true
+          end
+        else
+          # If not, use node.recipes. Using expand!.recipes catches multi-level roles 
+          # (roles with roles with recipes, etc.)
+          recipes = node.expand!.recipes.to_a
+        end
+
         node_cookbook_list = recipes.map{ |x| x.match(/[^\:]+/)[0] }.uniq
 
         # 3b) For each cookbook in the node runlist, if it's in our cookbook array increment its count and
         #     add the node to its running node array
 
         node_cookbook_list.each do |cookbook|
-	  if cookbook_list.has_key?(cookbook)
-            # Up the cookbook count
-            cookbook_list[cookbook]["count"] += 1
-            # Add the node to the cookbook's nodes array 
-            cookbook_list[cookbook]["nodes"] << node.name
+	        if cookbook_list.has_key?(cookbook)
+            # Up the appropriate ookbook count and add node to appropriate nodes array
+            if node_seen_recipe_flag
+              cookbook_list[cookbook]["seen_recipe_count"] += 1
+              cookbook_list[cookbook]["seen_recipe_nodes"] << node.name
+            else
+              cookbook_list[cookbook]["count"] += 1
+              cookbook_list[cookbook]["nodes"] << node.name
+            end
           end
         end
+      node_seen_recipe_flag = false
 
       end # step 3 iterate end
 
       # 4) Output
 
-      format_cookbook_audit_list_for_display(cookbook_list).each do |line|
-        ui.msg(line)
+      unless config[:totals]
+        ui.msg("Cookbook audit from node runlists:")
+
+        format_cookbook_runlist_list_for_display(cookbook_list).each do |line|
+          ui.msg(line)
+        end
+      end
+
+      if config[:all_cookbooks]
+        puts("\n")
+
+        ui.msg("Cookbook audit from seen_recipes:")
+
+        format_cookbook_seenlist_list_for_display(cookbook_list).each do |line|
+          ui.msg(line)
+        end
       end
 
+      if config[:totals]
+        puts("\n")
+
+        ui.msg("Cookbook audit totals - runlist-only nodes + seen_recipes:")
+
+        format_cookbook_totallist_list_for_display(cookbook_list).each do |line|
+          ui.msg(line)
+        end
+      end
+
+
     end # 'run' def end
 
 
-    def format_cookbook_audit_list_for_display(item)
+    def format_cookbook_runlist_list_for_display(item)
       key_length = item.empty? ? 0 : item.keys.map {|name| name.size }.max + 2
       if config[:show_nodelist]
         item.sort.map do |name, cookbook|
-          "#{name.ljust(key_length)} #{cookbook["count"]}   [ #{cookbook["nodes"].join('  ')} ]"
+          "#{name.ljust(key_length)} #{cookbook["count"]} [ #{cookbook["nodes"].join('  ')} ]"
         end
       else
         item.sort.map do |name, cookbook|
@@ -128,7 +208,39 @@ module KnifeAudit
         end
       end
          
-    end # format_cokbook_audit... def end
+    end # format_cokbook_runlist... def end
+
+    def format_cookbook_seenlist_list_for_display(item)
+      key_length = item.empty? ? 0 : item.keys.map {|name| name.size }.max + 2
+      if config[:show_nodelist]
+        item.sort.map do |name, cookbook|
+          "#{name.ljust(key_length)} #{cookbook["seen_recipe_count"]} [ #{cookbook["seen_recipe_nodes"].join('  ')} ]"
+        end
+      else
+        item.sort.map do |name, cookbook|
+          "#{name.ljust(key_length)} #{cookbook["seen_recipe_count"]}"
+        end
+      end
+         
+    end # format_cokbook_seenlist... def end
+
+    def format_cookbook_totallist_list_for_display(item)
+      key_length = item.empty? ? 0 : item.keys.map {|name| name.size }.max + 2
+      if config[:show_nodelist]
+        item.sort.map do |name, cookbook|
+          cookbook_display = (cookbook["seen_recipe_nodes"] + cookbook["nodes"]).uniq
+          cookbook_count = cookbook["seen_recipe_count"] + cookbook["count"]
+          "#{name.ljust(key_length)} #{cookbook_count} [ #{cookbook_display.join('  ')} ]"
+        end
+      else
+        item.sort.map do |name, cookbook|
+          cookbook_count = cookbook["seen_recipe_count"] + cookbook["count"]
+          "#{name.ljust(key_length)} #{cookbook_count}"
+        end
+      end
+         
+    end # format_cokbook_seenlist... def end
+
 
 
   end #class end

data/lib/chef/knife/knife_audit_cookbook/README.rdoc

@@ -0,0 +1,8 @@
+= DESCRIPTION:
+
+= REQUIREMENTS:
+
+= ATTRIBUTES: 
+
+= USAGE:
+

data/lib/chef/knife/knife_audit_cookbook/metadata.rb

@@ -0,0 +1,6 @@
+maintainer        "J.B. Zimmerman"
+maintainer_email  "jbzimmerman91@gmail.com"
+license           "Apache 2.0"
+description       "A cookbook for saving node auditing data for the knife-audit plugin"
+long_description  IO.read(File.join(File.dirname(__FILE__), 'README.rdoc'))
+version           "0.0.2"

data/lib/chef/knife/knife_audit_cookbook/recipes/default.rb

@@ -0,0 +1,32 @@
+#
+# Cookbook Name:: knife_audit
+# Recipe:: default
+#
+# Copyright 2011, Medidata
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+# Get the :seen_recipes list and save it to a permanent attribute.  This is in a ruby_block so that it runs
+# during execute phase, that way :seen_recipes is complete since it is populated during compile phase.  Once
+# this has been placed in an attribute, the end-of-run node.save will save it so that knife audit can get
+# the node's full runlist from the chef server.
+
+ruby_block "get_seen_recipes" do
+  block do
+    runlist = node.run_state[:seen_recipes]
+    node.set["knife_audit"]["seen_recipes"] = runlist
+  end
+  action :create
+end

data/lib/knife-audit/version.rb

@@ -1,5 +1,5 @@
 module Knife
   module Audit
-    VERSION = "0.0.1"
+    VERSION = "0.2.0"
   end
 end

metadata

@@ -1,27 +1,27 @@
 --- !ruby/object:Gem::Specification 
 name: knife-audit
 version: !ruby/object:Gem::Version 
-  hash: 29
+  hash: 23
   prerelease: 
   segments: 
   - 0
+  - 2
   - 0
-  - 1
-  version: 0.0.1
+  version: 0.2.0
 platform: ruby
 authors: 
-- Jacob Zimmerman
+- J.B. Zimmerman
 autorequire: 
 bindir: bin
 cert_chain: []
 
-date: 2011-08-31 00:00:00 -04:00
+date: 2011-10-07 00:00:00 -04:00
 default_executable: 
 dependencies: []
 
 description: Allows you to safely maintain a chef cookbook set by determining which cookbooks are currently in use by nodes (included in node runlists).
 email: 
-- jzimmerman@mdsol.com
+- jbzimmerman91@gmail.com
 executables: []
 
 extensions: []
@@ -34,6 +34,9 @@ files:
 - Rakefile
 - knife-audit.gemspec
 - lib/chef/knife/audit.rb
+- lib/chef/knife/knife_audit_cookbook/README.rdoc
+- lib/chef/knife/knife_audit_cookbook/metadata.rb
+- lib/chef/knife/knife_audit_cookbook/recipes/default.rb
 - lib/knife-audit/version.rb
 has_rdoc: true
 homepage: https://github.com/jbz/knife-audit