puppet 0.24.4 → 0.24.5

data/CHANGELOG

@@ -1,3 +1,173 @@
+0.24.5
+    You can now select the encoding format when transferring the catalog,
+    with 'yaml' still being the default but 'marshal' being an option.
+    This is because testing has shown drastic performance differences
+    between the two, with up to 70% of compile time being spent
+    in YAML code.  Use the 'catalog_format' setting to choose your format,
+    and the setting must be set on the client.
+
+    Fixed #1431 - Provider confines must now specify similar tests in one call.
+    I.e., you can't do confine :operatingsystem => %w{a b} and then
+    confine :operatingsystem => %w{b c}; you'd need to do them in one command.
+    This now-obsolete behaviour does not seem to be used anywhere.
+    The fix for #1431 is actually just removing the tests that exposed
+    this change; the change happened when I refactored how confines work.
+
+    Updated /spec/unit/rails.rb test
+
+    Fix #1426 - services on redhat are restarted again and status is 
+    called from the Red Hat provider
+
+    Fixed #1414 - Return code from waitpid now right shifted 8 bits
+
+    Fixed #174 - a native type type for managing ssh authorized_keys 
+    files is available.
+
+    Further moves from the examples directory and ext directory
+
+    Fixed #1397 One line fix, fail instead of log
+
+    Moved debian to conf and updated examples directory
+
+    Fixed #1368 - updated Red Hat init scripts
+ 
+    Added message referencing ReductiveLabs build library
+
+    Fixed #1396 - Added sha1 function from DavidS to core
+    
+    Fixed #1399 - the ldap user provider now knows it can manage
+    passwords.
+    
+    Fixed #1272 - if you provide a group name as the gid to an ldap
+    user, the name will be converted to a gid.  Note that this only
+    looks up ldap groups, at this point; if you want to set an ldap
+    user's primary group to a local group, you have to specify the GID.
+
+    Fixed #1226 - gems can now specify source repositories.
+
+    Fixed #1232 - the rundir no longer specifies a user/group,
+    and there are now client- and server-specific yaml directories.
+
+    Fixed 1240 - puppet will function more like puppetd if graphing
+    or reporting are enabled.
+    
+    Fixed #1231 - Exceptions during initialization should now be clearer.
+
+    Fixed #1006 - puppetrun --class works again.  I added the class
+    membership testing to the Ldap node terminus, and added tests,
+    so it shouldn't break again.
+
+    Fixed #1114 - Facts in plugin directories should now be autoloaded,
+    as long as you're using Facter 1.5.
+
+    Fixed #1195 - Updated Gentoo init scripts
+
+    Fixed #1367 - Updated Rakefile for new daily builds
+
+    Fixed #1370 - removed test/util/loadedfile.rb tests
+    
+    Fixed #1221 - aliases to titles now work for resources.
+    
+    Fixed #1012 - templates in the templatedir are preferred to module templates.
+
+    Fixed #707 - special '@reboot'-style cron jobs work again.
+
+    Fixed #1360 - allowdupe works on groups again.
+
+    Fixed #1369 - the init service provider now supports HP-UX.
+
+    Removed support for the 'node_name' setting in LDAP and external node 
+    lookups.
+
+    Also removed support for 'default' nodes in external nodes.
+    LDAP nodes now use the certificate name, the short name, and 'default',
+    but external nodes just use the certificate name and any custom terminus
+    types will use just the certificate name.
+    
+    Fixing #1168 (for 0.24.x) -- automatically downcasing the fqdn.
+    Also requiring that passed in certnames be downcased; the setting
+    system isn't currently flexible enough to automatically downcase
+    it for the user.
+
+    Adding a ResourceTemplate class for using templates directly
+    within resources (i.e., client-side templates).  This would really
+    only be used for composite resources that pass the results of the
+    template on to generated resources.
+
+    Exporting or collecting resources no longer raises an exception
+    when no storeconfigs is enabled, it just produces a warning.
+
+    Always using the cert name to store yaml files, which fixes #1178.
+    The Master handler previously provided the support for the :node_name
+    setting, and that functionality has now been moved into the Node
+    class.  At the same time, the names to search through have been
+    changed somewhat:  Previously, the certificate name and the 
+    hostname were both used for searching, but now, the cert name
+    is always searched first (unless node_name == facter), but only
+    the Facter hostname, domain, and fqdn are used otherwise.  We no
+    longer split the cert name, only the hostname/domain/fqdn.
+
+    Fixing transaction support for prefetching generated resources.
+
+    Adding support for settings within the existing Facter provider confines.
+
+    Moving all confine code out of the Provider class, and fixing #1197. 
+    Created a Confiner module for the Provider class methods, enhanced 
+    the interface between it and the Confine class to make sure binary 
+    paths are searched for fresh each time.
+
+    Modified the 'factpath' setting to automatically configure
+    Facter to load facts there if a new enough version of
+    Facter is used.
+
+    Crontab provider: fix a parse error when a line begins with a space 
+    character (fixes #1216)
+
+    Instead of deleting the init scripts (with --del) we should simply 
+    disable it with chkconfig service off, and respectfully do the same 
+    for enable => true;
+ 
+    Added ldap providers for users and groups.
+
+    Added support for the --all option to puppetca --clean.  If
+    puppetca --clean --all is issued then all client certificates
+    are removed.
+ 
+    Resources now return the 'should' value for properties from
+    the [] accessor method (they previously threw an exception when
+    this method was used with properties).  This shouldn't have any
+    affect functionally; it just makes the method equivalent to 'should'
+    for properties, but it works for all attribute types now.
+
+    Modified the 'master' handler to use the Catalog class to
+    compile node configurations, rather than using the Configuration
+    handler, which was never used directly.  I removed the Configuration
+    handler as a result.
+
+    Modified the 'master' handler (responsible for sending configurations
+    to clients) to always return Time.now as its compile date, so
+    configurations will always get recompiled.
+
+    Fixed #1184 -- definitions now autoload correctly all of the time.
+
+    Removed the code from the client that tries to avoid recompiling
+    the catalog. The client will now always recompile, assuming it
+    can reach the server.  It will still use the cached config if
+    there's a failure.
+
+    Fixing #1173 -- classes and definitions can now have the same
+    name as a directory with no failures.
+
+    Saving new facts now expires any cached node information.
+
+    Switching how caching is handled, so that objects now all
+    have an expiration date associated with them.  This makes it
+    much easier to know whether a given cached object should be used
+    or if it should be regenerated.
+
+    Changing the default environment to production.
+
+0.24.4
     Pass source to pkg_add via the PKG_PATH environment variable if
     it ends in a '/' indicating it is a directory. Allows pkg_add
     to resolve dependancies, and make it possible to specify packages

data/Rakefile

@@ -5,7 +5,7 @@ $: << File.expand_path(File.join(File.dirname(__FILE__), 'lib'))
 begin
     require 'rake/reductive'
 rescue LoadError
-    $stderr.puts "You must have the Reductive build library in your RUBYLIB."
+    $stderr.puts "You must have the Reductive build library in your RUBYLIB; see http://github.com/lak/reductive-build/tree/master."
     exit(14)
 end
 
@@ -23,11 +23,12 @@ project = Rake::RedLabProject.new("puppet") do |p|
         'lib/puppet.rb',
         'lib/puppet/**/*.rb',
         'lib/puppet/**/*.py',
-        'test/**/*.rb',
+        'test/**/*',
         'bin/**/*',
         'ext/**/*',
         'examples/**/*',
-        'conf/**/*'
+        'conf/**/*',
+        'man/**/*'
     ]
     p.filelist.exclude("bin/pi")
 
@@ -59,6 +60,7 @@ if project.has?(:gem)
             '--main' << 'README' <<
             '--line-numbers'
         task.test_file = "test/Rakefile"
+        task.author = "Luke Kanies"
     end
 end
 
@@ -101,7 +103,7 @@ def daily(package)
     edir = "/tmp/daily-export"
     Dir.mkdir edir
     Dir.chdir(edir) do
-        sh %{svn export http://reductivelabs.com/svn/#{package}/trunk #{package} >/dev/null}
+        sh %{git clone git://reductivelabs.com/#{package} #{package} >/dev/null}
         sh %{tar cf - #{package} | gzip -c > #{dailyfile(package)}}
     end
     FileUtils.rm_rf(edir)

data/bin/filebucket

@@ -163,8 +163,9 @@ end
 # Now parse the config
 Puppet.parse_config
 
-Puppet.genconfig
-Puppet.genmanifest
+if Puppet.settings.print_configs?
+        exit(Puppet.settings.print_configs ? 0 : 1)
+end
 
 begin
     if options[:local] or options[:bucket]

data/bin/puppet

@@ -141,8 +141,9 @@ if Puppet[:config] and File.exists? Puppet[:config]
     Puppet.settings.parse(Puppet[:config])
 end
 
-Puppet.genconfig
-Puppet.genmanifest
+if Puppet.settings.print_configs?
+    exit(Puppet.settings.print_configs ? 0 : 1)
+end
 
 # If noop is set, then also enable diffs
 if Puppet[:noop]
@@ -186,7 +187,7 @@ facts = Puppet::Node::Facts.find("me")
 facts.name = facts.values["hostname"]
 
 # Find our Node
-node = Puppet::Node.find_by_any_name(facts.name)
+node = Puppet::Node.find(facts.name)
 
 # Merge in the facts.
 node.merge(facts.values)
@@ -206,11 +207,13 @@ end
 
 begin
     # Compile our catalog
-    catalog = Puppet::Node::Catalog.find(node)
+    catalog = Puppet::Node::Catalog.find(node.name, :use_node => node)
 
     # Translate it to a RAL catalog
     catalog = catalog.to_ral
 
+    catalog.host_config = true if Puppet[:graph] or Puppet[:report]
+
     catalog.finalize
 
     # And apply it

data/bin/puppetca

@@ -32,14 +32,16 @@
 # '--genconfig'.
 #
 # all::
-#   Operate on all outstanding requests.  Only makes sense with '--sign',
-#   or '--list'.
+#   Operate on all items.  Currently only makes sense with '--sign',
+#   '--clean', or '--list'.
 #
 # clean::
 #    Remove all files related to a host from puppetca's storage. This is 
 #    useful when rebuilding hosts, since new certificate signing requests
 #    will only be honored if puppetca does not have a copy of a signed
 #    certificate for that host. The certificate of the host remains valid.
+#    If '--all' is specified then all host certificates, both signed and
+#    unsigned, will be removed.
 #
 # debug::
 #   Enable full debugging.
@@ -169,8 +171,9 @@ end
 # Now parse the config
 Puppet.parse_config
 
-Puppet.genconfig
-Puppet.genmanifest
+if Puppet.settings.print_configs?
+    exit(Puppet.settings.print_configs ? 0 : 1)
+end
 
 begin
     ca = Puppet::SSLCertificates::CA.new()
@@ -213,20 +216,35 @@ when :list
         puts ca.list_signed.collect { |cert | cert.sub(/^/,"+ ") }.join("\n")
     end
 when :clean
-    if hosts.empty?
-        $stderr.puts "You must specify one or more hosts to clean"
+    if hosts.empty? and all == false
+        $stderr.puts "You must specify one or more hosts to clean or --all to clean all host certificates"
         exit(24)
     end
+
     cleaned = false
-    hosts.each do |host|
-        cert = ca.getclientcert(host)[0]
-        if cert.nil?
-            $stderr.puts "Could not find client certificate for %s" % host
-            next
-        end
-        ca.clean(host)
+
+    if all
+        certs = ca.list
+            if certs.empty?
+                $stderr.puts "No certificates to clean"
+                exit(24)
+            end
+            certs.each do |c|
+                ca.clean(c)
+            end
         cleaned = true
+    else
+        hosts.each do |host|
+            cert = ca.getclientcert(host)[0]
+                if cert.nil?
+                    $stderr.puts "Could not find client certificate for %s" % host
+                    next
+                end
+            ca.clean(host)
+            cleaned = true
+        end
     end
+ 
     unless cleaned
         exit(27)
     end
@@ -234,7 +252,7 @@ when :sign
     to_sign = ARGV.collect { |h| h.downcase }
     unless to_sign.length > 0 or all
         $stderr.puts(
-            "You must specify to sign all certificates or you must specify hostnames"
+            "You must specify one or more hosts to sign certificates for or --all to sign all certificates"
         )
         exit(24)
     end

data/bin/puppetd

@@ -10,7 +10,7 @@
 #
 #   puppetd  [-D|--daemonize|--no-daemonize] [-d|--debug] [--disable] [--enable]
 #       [-h|--help] [--fqdn <host name>] [-l|--logdest syslog|<file>|console]
-#       [-o|--onetime] [--serve <handler>] [-t|--test]
+#       [-o|--onetime] [--serve <handler>] [-t|--test] [--noop]
 #       [-V|--version] [-v|--verbose] [-w|--waitforcert <seconds>]
 #
 # = Description
@@ -57,7 +57,7 @@
 # parameter, so you can specify '--server <servername>' as an argument.
 #
 # See the configuration file documentation at
-# http://reductivelabs.com/projects/puppet/reference/configref.html for
+# http://reductivelabs.com/trac/puppet/wiki/ConfigurationReference for
 # the full list of acceptable parameters. A commented list of all
 # configuration options can also be generated by running puppetd with
 # '--genconfig'.
@@ -124,6 +124,10 @@
 #   Enable the most common options used for testing.  These are +onetime+,
 #   +verbose+, +ignorecache, and +no-usecacheonfailure+.
 #
+# noop::
+#   Use +noop+ mode where the daemon runs in a no-op or dry-run mode.  This is useful
+#   for seeing what changes Puppet will make without actually executing the changes.
+#
 # verbose::
 #   Turn on verbose reporting.
 #
@@ -158,6 +162,7 @@ trap(:INT) do
 end
 
 require 'puppet'
+require 'puppet/executables/client/certhandler'
 require 'puppet/network/client'
 require 'getoptlong'
 
@@ -294,8 +299,9 @@ unless options[:setdest]
     Puppet::Util::Log.newdestination(:syslog)
 end
 
-Puppet.genconfig
-Puppet.genmanifest
+if Puppet.settings.print_configs?
+    exit(Puppet.settings.print_configs ? 0 : 1)
+end
 
 # If noop is set, then also enable diffs
 if Puppet[:noop]
@@ -338,36 +344,8 @@ if Puppet[:daemonize]
     client.daemonize
 end
 
-unless Puppet::Network::HttpPool.read_cert
-    # If we don't already have the certificate, then create a client to
-    # request one.  Use the special ca stuff, don't use the normal server and port.
-    caclient = Puppet::Network::Client.ca.new()
-    if options[:waitforcert] > 0
-        begin
-            while ! caclient.request_cert do
-                Puppet.notice "Did not receive certificate"
-                sleep options[:waitforcert]
-            end
-        rescue => detail
-            Puppet.err "Could not request certificate: %s" % detail.to_s
-            exit(23)
-        end
-    else
-        unless caclient.request_cert
-            Puppet.notice "No certificates; exiting"
-            exit(1)
-        end
-    end
-
-    # Now read the new cert in.
-    if Puppet::Network::HttpPool.read_cert
-        # If we read it in, then get rid of our existing http connection.
-        client.recycle_connection
-        Puppet.notice "Got signed certificate"
-    else
-        Puppet.err "Could not read certificates after retrieving them"
-        exit(34)
-    end
+unless Puppet::Executables::Client::CertHandler.new(options[:waitforcert], options[:onetime]).read_retrieve
+    client.recycle_connection
 end
 
 objects = []

data/bin/puppetmasterd

@@ -182,8 +182,9 @@ unless options[:setdest]
     Puppet::Util::Log.newdestination(:syslog)
 end
 
-Puppet.genconfig
-Puppet.genmanifest
+if Puppet.settings.print_configs?
+    exit(Puppet.settings.print_configs ? 0 : 1)
+end
 
 # A temporary solution, to at least make the master work for now.
 Puppet::Node::Facts.terminus_class = :yaml

data/bin/puppetrun

@@ -139,51 +139,12 @@ begin
 rescue LoadError
     $stderr.puts "Failed to load ruby LDAP library. LDAP functionality will not be available"
 end
+
 require 'puppet'
 require 'puppet/network/client'
+require 'puppet/util/ldap/connection'
 require 'getoptlong'
 
-
-# Look up all nodes matching a given class in LDAP.
-def ldapnodes(klass, fqdn = true)
-    unless defined? @ldap
-        setupldap()
-    end
-
-    hosts = []
-
-    filter = nil
-    if klass == :all
-        filter = "objectclass=puppetclient"
-    else
-        filter = "puppetclass=#{klass}"
-    end
-    @ldap.search(Puppet[:ldapbase], 2, filter, "cn") do |entry|
-        # Skip the default host entry
-        if entry.dn =~ /cn=default,/
-            $stderr.puts "Skipping default host entry"
-            next
-        end
-
-        if fqdn
-            hosts << entry.dn.sub("cn=",'').sub(/ou=hosts,/i, '').gsub(",dc=",".")
-        else
-            hosts << entry.get_values("cn")[0]
-        end
-    end
-
-    return hosts
-end
-
-def setupldap
-    begin
-        @ldap = Puppet::Parser::Interpreter.ldap()
-    rescue => detail
-        $stderr.puts "Could not connect to LDAP: %s" % detail
-        exit(34)
-    end
-end
-
 flags = [
     [ "--all",      "-a",	        GetoptLong::NO_ARGUMENT ],
     [ "--tag",      "-t",           GetoptLong::REQUIRED_ARGUMENT ],
@@ -278,11 +239,12 @@ Puppet.parse_config
 
 if Puppet[:node_terminus] = "ldap"
     if options[:all]
-        hosts = ldapnodes(:all, options[:fqdn])
+        hosts = Puppet::Node.search("whatever").collect { |node| node.name }
         puts "all: %s" % hosts.join(", ")
     else
+        hosts = []
         classes.each do |klass|
-            list = ldapnodes(klass, options[:fqdn])
+            list = Puppet::Node.search("whatever", :class => klass).collect { |node| node.name }
             puts "%s: %s" % [klass, list.join(", ")]
 
             hosts += list