chef 11.12.8 → 11.14.0.alpha.2

data/distro/common/man/man8/chef-solo.8

@@ -1,4 +1,6 @@
-.TH "CHEF-SOLO" "8" "Chef 11.10.0" "" "chef-solo"
+.\" Man page generated from reStructuredText.
+.
+.TH "CHEF-SOLO" "8" "Chef 11.12.0" "" "chef-solo"
 .SH NAME
 chef-solo \- The man page for the chef-solo command line tool.
 .
@@ -28,10 +30,8 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
 .in \\n[rst2man-indent\\n[rst2man-indent-level]]u
 ..
-.\" Man page generated from reStructuredText.
-.
 .sp
-chef\-solo is an open source version of the chef\-client that allows using cookbooks with nodes without requiring access to a server. chef\-solo runs locally and requires that a cookbook (and any of its dependencies) be on the same physical disk as the node. chef\-solo is a limited\-functionality version of the chef\-client and \fBdoes not support\fP the following:
+chef\-solo is an open source version of the chef\-client that allows using cookbooks with nodes without requiring access to a Chef server\&. chef\-solo runs locally and requires that a cookbook (and any of its dependencies) be on the same physical disk as the node. chef\-solo is a limited\-functionality version of the chef\-client and \fBdoes not support\fP the following:
 .INDENT 0.0
 .IP \(bu 2
 Node data storage
@@ -47,16 +47,27 @@ Authentication or authorization
 Persistent attributes
 .UNINDENT
 .sp
-The chef\-solo executable can be run as a command\-line tool.
+\fBNOTE:\fP
+.INDENT 0.0
+.INDENT 3.5
+chef\-solo can be run as a daemon.
+.UNINDENT
+.UNINDENT
+.sp
+The chef\-solo executable is run as a command\-line tool.
 .SH OPTIONS
 .sp
 This command has the following syntax:
+.INDENT 0.0
+.INDENT 3.5
 .sp
 .nf
 .ft C
 chef\-solo OPTION VALUE OPTION VALUE ...
 .ft P
 .fi
+.UNINDENT
+.UNINDENT
 .sp
 This command has the following options:
 .INDENT 0.0
@@ -65,20 +76,20 @@ This command has the following options:
 The configuration file to use.
 .TP
 .B \fB\-d\fP, \fB\-\-daemonize\fP
-Indicates that the executable will be run as a daemon. This option is only available on machines that run in UNIX or Linux environments. For machines that are running Microsoft Windows that require similar functionality, use the \fBchef\-client::service\fP recipe in the \fBchef\-client\fP cookbook: \fI\%http://community.opscode.com/cookbooks/chef-client\fP. This will install a chef\-client service under Microsoft Windows using the Windows Service Wrapper.
+Indicates that the executable will be run as a daemon. This option is only available on machines that run in UNIX or Linux environments. For machines that are running Microsoft Windows that require similar functionality, use the \fBchef\-client::service\fP recipe in the \fBchef\-client\fP cookbook: \fI\%http://community.opscode.com/cookbooks/chef\-client\fP\&. This will install a chef\-client service under Microsoft Windows using the Windows Service Wrapper\&.
 .TP
 .B \fB\-E ENVIRONMENT_NAME\fP, \fB\-\-environment ENVIRONMENT_NAME\fP
 The name of the environment.
 .TP
 .B \fB\-f\fP, \fB\-\-[no\-]fork\fP
-Indicates that a chef\-client run will be contained in a secondary process with dedicated RAM. When the chef\-client run is complete the RAM will be returned to the master process. This option helps ensure that a chef\-client will use a steady amount of RAM over time because the master process will not run recipes. This option will also help prevent memory leaks (such as those that can be introduced by the code contained within a poorly designed cookbook). Use \fB\-\-no\-fork\fP to disable running the chef\-client in fork node. Default value: \fB\-\-fork\fP.
+Indicates that a chef\-client run will be contained in a secondary process with dedicated RAM. When the chef\-client run is complete the RAM will be returned to the master process. This option helps ensure that a chef\-client will use a steady amount of RAM over time because the master process will not run recipes. This option will also help prevent memory leaks (such as those that can be introduced by the code contained within a poorly designed cookbook). Use \fB\-\-no\-fork\fP to disable running the chef\-client in fork node. Default value: \fB\-\-fork\fP\&.
 .TP
 .B \fB\-F FORMAT\fP, \fB\-\-format FORMAT\fP
-The output format: \fBdoc\fP (default) or \fBmin\fP.
+The output format: \fBdoc\fP (default) or \fBmin\fP\&.
 .sp
 Use \fBdoc\fP to print the progress of the chef\-client run using full strings that display a summary of updates as they occur.
 .sp
-Use \fBmin\fP to print the progress of the chef\-client run using single characters. A summary of updates is printed at the end of the chef\-client run. A dot (\fB.\fP) is printed for events that do not have meaningful status information, such as loading a file or synchronizing a cookbook. For resources, a dot (\fB.\fP) is printed when the resource is up to date, an \fBS\fP is printed when the resource is skipped by \fBnot_if\fP or \fBonly_if\fP, and a \fBU\fP is printed when the resource is updated.
+Use \fBmin\fP to print the progress of the chef\-client run using single characters. A summary of updates is printed at the end of the chef\-client run. A dot (\fB\&.\fP) is printed for events that do not have meaningful status information, such as loading a file or synchronizing a cookbook. For resources, a dot (\fB\&.\fP) is printed when the resource is up to date, an \fBS\fP is printed when the resource is skipped by \fBnot_if\fP or \fBonly_if\fP, and a \fBU\fP is printed when the resource is updated.
 .sp
 Other formatting options are available when those formatters are configured in the client.rb file using the \fBadd_formatter\fP option.
 .TP
@@ -107,7 +118,7 @@ The level of logging that will be stored in a log file.
 The location in which log file output files will be saved. If this location is set to something other than \fBSTDOUT\fP, standard output logging will still be performed (otherwise there would be no output other than to a file). This is recommended when starting any executable as a daemon.
 .TP
 .B \fB\-\-[no\-]color\fP
-Indicates whether colored output will be used. Default setting: \fB\-\-color\fP.
+Indicates whether colored output will be used. Default setting: \fB\-\-color\fP\&.
 .TP
 .B \fB\-N NODE_NAME\fP, \fB\-\-node\-name NODE_NAME\fP
 The name of the node.
@@ -125,7 +136,7 @@ A number (in seconds) to add to the \fBinterval\fP that is used to determine the
 The user that owns a process. This is required when starting any executable as a daemon.
 .TP
 .B \fB\-v\fP, \fB\-\-version\fP
-The version of the chef\-client.
+The version of the chef\-client\&.
 .TP
 .B \fB\-W\fP, \fB\-\-why\-run\fP
 Indicates that the executable will be run in why\-run mode, which is a type of chef\-client run that does everything except modify the system. Use why\-run mode to understand why the chef\-client makes the decisions that it makes and to learn more about the current and proposed state of the system.
@@ -133,34 +144,44 @@ Indicates that the executable will be run in why\-run mode, which is a type of c
 .SH EXAMPLES
 .sp
 \fBUse a URL\fP
+.INDENT 0.0
+.INDENT 3.5
 .sp
 .nf
 .ft C
 $ chef\-solo \-c ~/solo.rb \-j ~/node.json \-r http://www.example.com/chef\-solo.tar.gz
 .ft P
 .fi
+.UNINDENT
+.UNINDENT
 .sp
-where \fB\-r\fP uses the \fBremote_file\fP resource to retrieve the tar.gz archive into the \fBfile_cache_path\fP, and then extract it to \fBcookbooks_path\fP.
+The tar.gz archived into the \fBfile_cache_path\fP, and then extracted to \fBcookbooks_path\fP\&.
 .sp
 \fBUse a directory\fP
+.INDENT 0.0
+.INDENT 3.5
 .sp
 .nf
 .ft C
 $ chef\-solo \-c ~/solo.rb \-j ~/node.json
 .ft P
 .fi
+.UNINDENT
+.UNINDENT
 .sp
-where the \fB\-r URL\fP option is not used. chef\-solo will look in the solo.rb file to determine the directory in which cookbooks are located.
+chef\-solo will look in the solo.rb file to determine the directory in which cookbooks are located.
 .sp
 \fBUse a URL for cookbook and JSON data\fP
+.INDENT 0.0
+.INDENT 3.5
 .sp
 .nf
 .ft C
 $ chef\-solo \-c ~/solo.rb \-j http://www.example.com/node.json \-r http://www.example.com/chef\-solo.tar.gz
 .ft P
 .fi
-.sp
-where \fB\-r\fP corresponds to \fBrecipe_url\fP and \fB\-j\fP corresponds to \fBjson_attribs\fP, both of which are configuration options in solo.rb.
+.UNINDENT
+.UNINDENT
 .SH AUTHOR
 Chef
 .\" Generated by docutils manpage writer.

data/lib/chef/application.rb

@@ -35,20 +35,6 @@ class Chef::Application
 
     @chef_client = nil
     @chef_client_json = nil
-    trap("INT") do
-      Chef::Application.fatal!("SIGINT received, stopping", 2)
-    end
-
-    unless Chef::Platform.windows?
-      trap("QUIT") do
-        Chef::Log.info("SIGQUIT received, call stack:\n  " + caller.join("\n  "))
-      end
-
-      trap("HUP") do
-        Chef::Log.info("SIGHUP received, reconfiguring")
-        reconfigure
-      end
-    end
 
     # Always switch to a readable directory. Keeps subsequent Dir.chdir() {}
     # from failing due to permissions when launched as a less privileged user.
@@ -62,11 +48,30 @@ class Chef::Application
 
   # Get this party started
   def run
+    setup_signal_handlers
     reconfigure
     setup_application
     run_application
   end
 
+  def setup_signal_handlers
+    trap("INT") do
+      Chef::Application.fatal!("SIGINT received, stopping", 2)
+    end
+
+    unless Chef::Platform.windows?
+      trap("QUIT") do
+        Chef::Log.info("SIGQUIT received, call stack:\n  " + caller.join("\n  "))
+      end
+
+      trap("HUP") do
+        Chef::Log.info("SIGHUP received, reconfiguring")
+        reconfigure
+      end
+    end
+  end
+
+
   # Parse configuration (options and config file)
   def configure_chef
     parse_options

data/lib/chef/application/client.rb

@@ -219,6 +219,11 @@ class Chef::Application::Client < Chef::Application
     :long         => "--config-file-jail PATH",
     :description  => "Directory under which config files are allowed to be loaded (no client.rb or knife.rb outside this path will be loaded)."
 
+  option :run_lock_timeout,
+    :long         => "--run-lock-timeout SECONDS",
+    :description  => "Set maximum duration to wait for another client run to finish, default is indefinitely.",
+    :proc         => lambda { |s| s.to_i }
+
   if Chef::Platform.windows?
     option :fatal_windows_admin_check,
       :short        => "-A",

data/lib/chef/application/solo.rb

@@ -165,6 +165,11 @@ class Chef::Application::Solo < Chef::Application
     :long         => '--environment ENVIRONMENT',
     :description  => 'Set the Chef Environment on the node'
 
+  option :run_lock_timeout,
+    :long         => "--run-lock-timeout SECONDS",
+    :description  => "Set maximum duration to wait for another client run to finish, default is indefinitely.",
+    :proc         => lambda { |s| s.to_i }
+
   attr_reader :chef_client_json
 
   def initialize

data/lib/chef/application/windows_service_manager.rb

@@ -112,6 +112,9 @@ class Chef
                                  :service_name     => @service_name,
                                  :display_name     => @service_display_name,
                                  :description      => @service_description,
+                                 # Prior to 0.8.5, win32-service creates interactive services by default,
+                                 # and we don't want that, so we need to override the service type.
+                                 :service_type     => ::Win32::Service::SERVICE_WIN32_OWN_PROCESS,
                                  :start_type       => ::Win32::Service::SERVICE_AUTO_START,
                                  :binary_path_name => cmd
                                  )

data/lib/chef/chef_fs/chef_fs_data_store.rb

@@ -23,6 +23,7 @@ require 'chef/chef_fs/file_pattern'
 require 'chef/chef_fs/file_system'
 require 'chef/chef_fs/file_system/not_found_error'
 require 'chef/chef_fs/file_system/memory_root'
+require 'fileutils'
 
 class Chef
   module ChefFS
@@ -43,7 +44,11 @@ class Chef
           @memory_store.create_dir(path, name, *options)
         else
           with_dir(path) do |parent|
-            parent.create_child(chef_fs_filename(path + [name]), nil)
+            begin
+              parent.create_child(chef_fs_filename(path + [name]), nil)
+            rescue Chef::ChefFS::FileSystem::AlreadyExistsError => e
+              raise ChefZero::DataStore::DataAlreadyExistsError.new(to_zero_path(e.entry), e)
+            end
           end
         end
       end
@@ -61,7 +66,11 @@ class Chef
           end
 
           with_dir(path) do |parent|
-            parent.create_child(chef_fs_filename(path + [name]), data)
+            begin
+              parent.create_child(chef_fs_filename(path + [name]), data)
+            rescue Chef::ChefFS::FileSystem::AlreadyExistsError => e
+              raise ChefZero::DataStore::DataAlreadyExistsError.new(to_zero_path(e.entry), e)
+            end
           end
         end
       end
@@ -82,7 +91,13 @@ class Chef
           with_entry(path) do |entry|
             if path[0] == 'cookbooks' && path.length == 3
               # get /cookbooks/NAME/version
-              result = entry.chef_object.to_hash
+              result = nil
+              begin
+                result = entry.chef_object.to_hash
+              rescue Chef::ChefFS::FileSystem::NotFoundError => e
+                raise ChefZero::DataStore::DataNotFoundError.new(to_zero_path(e.entry), e)
+              end
+
               result.each_pair do |key, value|
                 if value.is_a?(Array)
                   value.each do |file|
@@ -102,7 +117,11 @@ class Chef
               JSON.pretty_generate(result)
 
             else
-              entry.read
+              begin
+                entry.read
+              rescue Chef::ChefFS::FileSystem::NotFoundError => e
+                raise ChefZero::DataStore::DataNotFoundError.new(to_zero_path(e.entry), e)
+              end
             end
           end
         end
@@ -121,7 +140,12 @@ class Chef
             write_cookbook(path, data, *options)
           else
             with_dir(path[0..-2]) do |parent|
-              parent.create_child(chef_fs_filename(path), data)
+              child = parent.child(chef_fs_filename(path))
+              if child.exists?
+                child.write(data)
+              else
+                parent.create_child(chef_fs_filename(path), data)
+              end
             end
           end
         end
@@ -132,10 +156,14 @@ class Chef
           @memory_store.delete(path)
         else
           with_entry(path) do |entry|
-            if path[0] == 'cookbooks' && path.length >= 3
-              entry.delete(true)
-            else
-              entry.delete(false)
+            begin
+              if path[0] == 'cookbooks' && path.length >= 3
+                entry.delete(true)
+              else
+                entry.delete(false)
+              end
+            rescue Chef::ChefFS::FileSystem::NotFoundError => e
+              raise ChefZero::DataStore::DataNotFoundError.new(to_zero_path(e.entry), e)
             end
           end
         end
@@ -146,7 +174,11 @@ class Chef
           @memory_store.delete_dir(path, *options)
         else
           with_entry(path) do |entry|
-            entry.delete(options.include?(:recursive))
+            begin
+              entry.delete(options.include?(:recursive))
+            rescue Chef::ChefFS::FileSystem::NotFoundError => e
+              raise ChefZero::DataStore::DataNotFoundError.new(to_zero_path(e.entry), e)
+            end
           end
         end
       end
@@ -172,10 +204,16 @@ class Chef
 
         elsif path[0] == 'cookbooks' && path.length == 2
           if Chef::Config.versioned_cookbooks
-            # list /cookbooks/name = filter /cookbooks/name-version down to name
-            entry.children.map { |child| split_name_version(child.name) }.
-                           select { |name, version| name == path[1] }.
-                           map { |name, version| version }.to_a
+            result = with_entry([ 'cookbooks' ]) do |entry|
+              # list /cookbooks/name = filter /cookbooks/name-version down to name
+              entry.children.map { |child| split_name_version(child.name) }.
+                             select { |name, version| name == path[1] }.
+                             map { |name, version| version }
+            end
+            if result.empty?
+              raise ChefZero::DataStore::DataNotFoundError.new(path)
+            end
+            result
           else
             # list /cookbooks/name = <single version>
             version = get_single_cookbook_version(path)
@@ -186,12 +224,12 @@ class Chef
           with_entry(path) do |entry|
             begin
               entry.children.map { |c| zero_filename(c) }.sort
-            rescue Chef::ChefFS::FileSystem::NotFoundError
+            rescue Chef::ChefFS::FileSystem::NotFoundError => e
               # /cookbooks, /data, etc. never return 404
               if path_always_exists?(path)
                 []
               else
-                raise
+                raise ChefZero::DataStore::DataNotFoundError.new(to_zero_path(e.entry), e)
               end
             end
           end
@@ -223,12 +261,13 @@ class Chef
       end
 
       def write_cookbook(path, data, *options)
-        # Create a little Chef::ChefFS memory filesystem with the data
         if Chef::Config.versioned_cookbooks
-          cookbook_path = "cookbooks/#{path[1]}-#{path[2]}"
+          cookbook_path = File.join('cookbooks', "#{path[1]}-#{path[2]}")
         else
-          cookbook_path = "cookbooks/#{path[1]}"
+          cookbook_path = File.join('cookbooks', path[1])
         end
+
+        # Create a little Chef::ChefFS memory filesystem with the data
         cookbook_fs = Chef::ChefFS::FileSystem::MemoryRoot.new('uploading')
         cookbook = JSON.parse(data, :create_additions => false)
         cookbook.each_pair do |key, value|
@@ -236,15 +275,22 @@ class Chef
             value.each do |file|
               if file.is_a?(Hash) && file.has_key?('checksum')
                 file_data = @memory_store.get(['file_store', 'checksums', file['checksum']])
-                cookbook_fs.add_file("#{cookbook_path}/#{file['path']}", file_data)
+                cookbook_fs.add_file(File.join(cookbook_path, file['path']), file_data)
               end
             end
           end
         end
 
-        # Use the copy/diff algorithm to copy it down so we don't destroy
-        # chefignored data.  This is terribly un-thread-safe.
-        Chef::ChefFS::FileSystem.copy_to(Chef::ChefFS::FilePattern.new("/#{cookbook_path}"), cookbook_fs, chef_fs, nil, {:purge => true})
+        # Create the .uploaded-cookbook-version.json
+        cookbooks = chef_fs.child('cookbooks')
+        if !cookbooks.exists?
+          cookbooks = chef_fs.create_child('cookbooks')
+        end
+        # We are calling a cookbooks-specific API, so get multiplexed_dirs out of the way if it is there
+        if cookbooks.respond_to?(:multiplexed_dirs)
+          cookbooks = cookbooks.write_dir
+        end
+        cookbooks.write_cookbook(cookbook_path, data, cookbook_fs)
       end
 
       def split_name_version(entry_name)
@@ -343,8 +389,10 @@ class Chef
       end
 
       def with_dir(path)
+        # Do not automatically create data bags
+        create = !(path[0] == 'data' && path.size >= 2)
         begin
-          yield get_dir(_to_chef_fs_path(path), true)
+          yield get_dir(_to_chef_fs_path(path), create)
         rescue Chef::ChefFS::FileSystem::NotFoundError => e
           raise ChefZero::DataStore::DataNotFoundError.new(to_zero_path(e.entry), e)
         end

data/lib/chef/chef_fs/file_system/chef_repository_file_system_cookbook_dir.rb

@@ -44,11 +44,17 @@ class Chef
             end
 
             loader.load_cookbooks
-            return loader.cookbook_version
-          rescue
-            Chef::Log.error("Could not read #{path_for_printing} into a Chef object: #{$!}")
+            cb = loader.cookbook_version
+            if !cb
+              Chef::Log.error("Cookbook #{file_path} empty.")
+              raise "Cookbook #{file_path} empty."
+            end
+            cb
+          rescue => e
+            Chef::Log.error("Could not read #{path_for_printing} into a Chef object: #{e}")
+            Chef::Log.error(e.backtrace.join("\n"))
+            raise
           end
-          nil
         end
 
         def children
@@ -66,6 +72,8 @@ class Chef
           if is_dir
             # Only the given directories will be uploaded.
             return CookbookDir::COOKBOOK_SEGMENT_INFO.keys.include?(name.to_sym) && name != 'root_files'
+          elsif name == Chef::Cookbook::CookbookVersionLoader::UPLOADED_COOKBOOK_VERSION_FILE
+            return false
           end
           super(name, is_dir)
         end
@@ -81,6 +89,14 @@ class Chef
           self.class.canonical_cookbook_name(entry_name)
         end
 
+        def uploaded_cookbook_version_path
+          File.join(file_path, Chef::Cookbook::CookbookVersionLoader::UPLOADED_COOKBOOK_VERSION_FILE)
+        end
+
+        def can_upload?
+          File.exists?(uploaded_cookbook_version_path) || children.size > 0
+        end
+
         protected
 
         def make_child(child_name)