puppet 0.24.9 → 0.25.0

data/LICENSE

@@ -3,8 +3,8 @@ Puppet - Automating Configuration Management. Copyright (C) 2005 Reductive Labs
 Reductive Labs can be contacted at: info@reductivelabs.com
 
 This program and entire repository is free software; you can
-redistribute it and/or modify it under the terms of the GNU 
-General Public License as published by the Free Software 
+redistribute it and/or modify it under the terms of the GNU
+General Public License as published by the Free Software
 Foundation; either version 2 of the License, or any later version.
 
 This program is distributed in the hope that it will be useful,

data/README

@@ -1,4 +1,4 @@
-Documentation (and detailed install instructions) can be found 
+Documentation (and detailed install instructions) can be found
 online at http://reductivelabs.com/trac/puppet/wiki/DocumentationStart .
 
 Generally, you need the following things installed:

data/README.queueing

@@ -0,0 +1,126 @@
+*PUPPET QUEUEING
+
+Puppet Queueing is a feature which is designed to take some load
+off of the PuppetMaster by transferring the task of updating the
+database to a separate program which is named puppetqd (Puppet
+Queue Daemon).
+
+Currently this is only supported for "Storeconfigs" which is
+documented at:
+
+http://reductivelabs.com/trac/puppet/wiki/UsingStoredConfiguration
+
+In the future this feature can be extended to any new puppet
+data which involves storage in a database.
+
+*OPERATION
+
+In a nutshell:
+
+  puppetmasterd -> stomp -> service -> stomp -> puppetqd -> database
+
+At the moment the only messaging protocol supported is "stomp". Although
+others could be implemented, stomp is considered by many as the
+default queueing mechanism for Ruby and Rails applications. It is
+distributed as a Ruby gem and is easily installed.
+
+(The queueing code inside Puppet has been written so that when other
+interfaces and protocols are implemented they will be easy to use by
+changing settings in puppet.conf).
+
+The "service" in the diagram above is any queueing service that supports
+the Stomp API. For details refer to:
+
+  http://xircles.codehaus.org/projects/stomp
+
+Both puppetmasterd and puppetqd subscribe to the same queueing service
+using the stomp interface. As puppetmasterd posts data to the queue,
+puppetqd receives it and stores it. The details of how to connect to
+the service and the name of the queue to use are set in puppet.conf:
+
+  [main]
+    queue_type = stomp
+    queue_source = stomp://localhost:61613
+  [puppetmasterd]
+    async_storeconfigs = true
+
+Note: since puppetmasterd needs to recover the data being stored at a
+later time, both puppetmasterd and puppetqd need to work with the same
+database as defined in the STORECONFIGS setup.
+
+*QUEUEING SERVICES
+
+As mentioned previously any queueing service that supports the Stomp
+protocol can be used. Which one you use depends on your needs. We have
+tested with two of the most popular services - StompServer and ActiveMQ.
+
++ StompServer
+
+    http://rubyforge.org/projects/stompserver/
+
+StompServer is a lightweight queueing service written in Ruby which is
+suitable for testing or low volume puppet usage. Works well when both
+puppetmasterd and puppetd are running on the same machine that it's running
+on but we encountered some problems when using it from multiple machines.
+
+Just install the stompserver gem and run 'stompserver'.
+
++ Apache ActiveMQ
+
+    http://activemq.apache.org
+
+Considered by many to be the most popular message service in use today,
+ActiveMQ has hundreds of features for scaling, persistence and so on.
+
+Although installation is fairly simple, the configuration can seem quite
+intimidating, but for our use a one line change to the  standard configuration
+is all that is required and is explained at:
+
+    http://activemq.apache.org/stomp.html
+
+Other customization of the internal workings of ActiveMQ, if any, will depend
+on your needs and deployment. A quick skimming of the ActiveMQ documentation
+will give you enough info to decide.
+
+Others
+
+We have looked at but not tried some other queuing services which are
+compatible with the Stomp API:
+
++ POE Component Message Queue
++ JBoss Messaging (with 3rd party support for Stomp)
+
+*SCALING
+
+For StoreConfigs you basically need to have the catalog for a node stored
+in the database before the next time the node connects and asks for a
+new catalog.
+
+If the puppetd on your nodes is set to check every 30 minutes,
+then it would seem that there is no problem. However if you have 3000
+nodes you have a LOT of catalogs to store and it is possible you will
+not get a catalog saved in time.
+
+Running puppetmaster, your queueing service and puppetqd on the same
+machine means that they are all competing for the same CPU cycles. Bumping
+up the power of the server they are running on may be enough to handle
+even fairly large deployments.
+
+However since most queueing services (even StompServer) are designed to
+deliver messages from a "queue" to whoever asks for the next message you
+can split things up between machines:
+
+  puppetmaster1 --\             /-- puppetqd1 -\
+  puppetmaster2 ----> ActiveMQ ---> puppetqd2 ---> database
+  puppetmaster3 --/             \-- puppetqd33 -/
+                                 \- puppetqd4-/
+
+This is, of course a totally contrived example, but it gets the point
+across. As long as the data gets to the database, it doesn't matter
+which machines or services it goes through.
+
+Although for StoreConfigs absolute reliability is not a requirement as
+a new catalog will be sent the next time a node connects, some amount
+of persistence should some process crash may be desirable. Both ActiveMQ
+and MySQL (and other databases) have these kind of features built in
+which can be activated as needed.

data/README.rst

@@ -1,11 +1,11 @@
 Puppet
 ======
 
-Puppet, an automated administrative engine for your Linux and Unix systems, performs 
-administrative tasks (such as adding users, installing packages, and updating server 
+Puppet, an automated administrative engine for your Linux and Unix systems, performs
+administrative tasks (such as adding users, installing packages, and updating server
 configurations) based on a centralized specification.
 
-Documentation (and detailed install instructions) can be found 
+Documentation (and detailed install instructions) can be found
 online at the `Puppet Wiki`_.
 
 Installation
@@ -32,7 +32,7 @@ Generally, you need the following things installed:
   the library.
 
 * Facter => 1.1.1
-  You can get this from your package management system or the `Facter site`_ 
+  You can get this from your package management system or the `Facter site`_
 
 .. _Puppet Wiki: http://reductivelabs.com/trac/puppet/wiki/DocumentationStart
 .. _Facter site: http://reductivelabs.com/projects/facter

data/Rakefile

@@ -1,233 +1,79 @@
 # Rakefile for Puppet -*- ruby -*-
 
+$: << File.expand_path('lib')
 $LOAD_PATH << File.join(File.dirname(__FILE__), 'tasks')
 
-$: << File.expand_path(File.join(File.dirname(__FILE__), 'lib'))
-
-begin
-    require 'rake/reductive'
-rescue LoadError
-    $stderr.puts "You must have the Reductive build library in your RUBYLIB; see http://github.com/lak/reductive-build/tree/master."
-    exit(14)
-end
-
-TESTHOSTS = %w{rh3a fedora1 centos1 freebsd1 culain}
-
-project = Rake::RedLabProject.new("puppet") do |p|
-    p.summary = "System Automation and Configuration Management Software"
-    p.description = "Puppet is a declarative language for expressing system
-        configuration, a client and server for distributing it, and a library
-        for realizing the configuration."
-
-    p.filelist = [
-        'install.rb',
-        '[A-Z]*',
-        'lib/puppet.rb',
-        'lib/puppet/**/*.rb',
-        'lib/puppet/**/*.py',
-        'test/**/*',
-        'spec/**/*',
-        'bin/**/*',
-        'ext/**/*',
-        'examples/**/*',
-        'conf/**/*',
-        'man/**/*'
-    ]
-    p.filelist.exclude("bin/pi")
-
-    p.add_dependency('facter', '1.1.0')
-end
-
-if project.has?(:gem)
-    # Make our gem task.  This actually just fills out the spec.
-    project.mkgemtask do |task|
-
-        task.require_path = 'lib'                         # Use these for libraries.
-
-        task.bindir = "bin"                               # Use these for applications.
-        task.executables = ["puppet", "puppetd", "puppetmasterd", "puppetdoc",
-                         "puppetca", "puppetrun", "ralsh"]
-        task.default_executable = "puppet"
-
-        #### Documentation and testing.
-
-        task.has_rdoc = true
-        #s.extra_rdoc_files = rd.rdoc_files.reject { |fn| fn =~ /\.rb$/ }.to_a
-        task.rdoc_options <<
-            '--title' <<  'Puppet - Configuration Management' <<
-            '--main' << 'README' <<
-            '--line-numbers'
-        task.test_file = "test/Rakefile"
-        task.author = "Luke Kanies"
-    end
-end
-
-rule(/_is_runnable$/) do |t|
-    available = false
-    executable = t.name.sub(/_is_runnable$/, '')
-    ENV['PATH'].split(':').each do |elem|
-        available = true if File.executable? File.join(elem, executable)
-    end
-    
-    unless available
-        puts "You do not have #{executable} available in your path"
-        exit 1
-    end
-end
-
-task :check_build_deps => 'dpkg-checkbuilddeps_is_runnable' do
-    system("dpkg-checkbuilddeps") || exit(1)
-end
-
-task :debian_packages => [ "debian", :check_build_deps, :fakeroot_is_runnable ] do
-    system("fakeroot debian/rules clean") || exit(1)
-    system("fakeroot debian/rules binary") || exit(1)
-end
-
-
-def dailyfile(package)
-    "#{downdir}/#{package}/#{package}-daily-#{stamp}.tgz"
-end
-
-def daily(package)
-    edir = "/tmp/daily-export"
-    Dir.mkdir edir
-    Dir.chdir(edir) do
-        sh %{git clone git://reductivelabs.com/#{package} #{package} >/dev/null}
-        sh %{tar cf - #{package} | gzip -c > #{dailyfile(package)}}
-    end
-    FileUtils.rm_rf(edir)
-end
-
-def downdir
-    ENV['DOWNLOAD_DIR'] || "/opt/rl/docroots/reductivelabs.com/htdocs/downloads"
-end
-
-def stamp
-    [Time.now.year, Time.now.month, Time.now.day].collect { |i| i.to_s}.join
-end
-
-pdaily = dailyfile("puppet")
-fdaily = dailyfile("facter")
-
-file pdaily do
-    daily("puppet")
-end
-
-file fdaily do
-    daily("facter")
-end
-
-task :daily => [pdaily, fdaily]
-
-task :dailyclean do
-    Dir.glob("#{downdir}/*/*daily*.tgz").each do |file|
-        puts "Removing %s" % file
-        File.unlink(file)
-    end
-end
-
-task :tracdocs do
-    require 'puppet'
-    require 'puppet/util/reference'
-    Puppet::Util::Reference.references.each do |ref| 
-        sh "puppetdoc -m trac -r #{ref.to_s}"
-    end
+Dir['tasks/**/*.rake'].each { |t| load t }
+
+require './lib/puppet.rb'
+require 'rake'
+require 'rake/packagetask'
+require 'rake/gempackagetask'
+
+FILES = FileList[
+    '[A-Z]*',
+    'install.rb',
+    'bin/**/*',
+    'sbin/**/*',
+    'lib/**/*',
+    'conf/**/*',
+    'man/**/*',
+    'examples/**/*',
+    'ext/**/*',
+    'test/**/*',
+    'spec/**/*'
+]
+
+task :default do
+    sh %{rake -T}
+end
+
+spec = Gem::Specification.new do |spec|
+    spec.platform = Gem::Platform::RUBY
+    spec.name = 'puppet'
+    spec.files = FILES.to_a
+    spec.version = Puppet::PUPPETVERSION
+    spec.summary = 'Puppet, an automated configuration management tool'
+    spec.author = 'Reductive Labs'
+    spec.email = 'puppet@reductivelabs.com'
+    spec.homepage = 'http://reductivelabs.com'
+    spec.rubyforge_project = 'puppet'
+    spec.has_rdoc = true
+    spec.rdoc_options <<
+        '--title' <<  'Puppet - Configuration Management' <<
+        '--main' << 'README' <<
+        '--line-numbers'
+end
+
+Rake::PackageTask.new("puppet", Puppet::PUPPETVERSION) do |pkg|
+    pkg.package_dir = 'pkg'
+    pkg.need_tar_gz = true
+    pkg.package_files = FILES.to_a
+end
+
+Rake::GemPackageTask.new(spec) do |pkg|
 end
 
 desc "Run the specs under spec/"
 task :spec do
     require 'spec'
     require 'spec/rake/spectask'
-    # require 'rcov'
-    Spec::Rake::SpecTask.new do |t|
-         #   t.rcov = true
-         t.spec_opts = ['--format','s', '--loadby','mtime']
-         t.spec_files = FileList['spec/**/*.rb']
+    begin
+        require 'rcov'
+    rescue LoadError
     end
+
+    Spec::Rake::SpecTask.new do |t|
+        t.spec_opts = ['--format','s', '--loadby','mtime']
+        t.spec_files = FileList['spec/**/*.rb']
+        if defined?(Rcov)
+            t.rcov = true
+            t.rcov_opts = ['--exclude', 'spec/*,test/*,results/*,/usr/lib/*']
+        end
+     end
 end
 
 desc "Run the unit tests"
 task :unit do
     sh "cd test; rake"
 end
-
-namespace :ci do
-
-  desc "Run the CI prep tasks"
-  task :prep do
-    require 'rubygems'
-    gem 'ci_reporter'
-    require 'ci/reporter/rake/rspec'
-    require 'ci/reporter/rake/test_unit'
-    ENV['CI_REPORTS'] = 'results'
-  end
-
-  desc "Run all CI tests"
-  task :all => [:unit, :spec]
-
-  desc "Run CI Unit tests"
-  task :unit => [:prep, 'ci:setup:testunit'] do
-     sh "cd test; rake test; exit 0"
-  end
-
-  desc "Run CI RSpec tests"
-  task :spec => [:prep, 'ci:setup:rspec'] do
-     sh "cd spec; rake all; exit 0"
-  end
-
-end
-
-desc "Send patch information to the puppet-dev list"
-task :mail_patches do
-    if Dir.glob("00*.patch").length > 0
-        raise "Patches already exist matching '00*.patch'; clean up first"
-    end
-
-    unless %x{git status} =~ /On branch (.+)/
-        raise "Could not get branch from 'git status'"
-    end
-    branch = $1
-    
-    unless branch =~ %r{^([^\/]+)/([^\/]+)/([^\/]+)$}
-        raise "Branch name does not follow <type>/<parent>/<name> model; cannot autodetect parent branch"
-    end
-
-    type, parent, name = $1, $2, $3
-
-    # Create all of the patches
-    sh "git format-patch -C -M -s -n #{parent}..HEAD"
-
-    # And then mail them out.
-
-    # If we've got more than one patch, add --compose
-    if Dir.glob("00*.patch").length > 1
-        compose = "--compose"
-    else
-        compose = ""
-    end
-
-    # Now send the mail.
-    sh "git send-email #{compose} --no-chain-reply-to --no-signed-off-by-cc --suppress-from --no-thread --to puppet-dev@googlegroups.com 00*.patch"
-
-    # Finally, clean up the patches
-    sh "rm 00*.patch"
-end
-
-    desc "Create a changelog based on your git commits."
-    task :changelog do
- 
-      CHANGELOG_DIR = "#{Dir.pwd}"
- 
-      mkdir(CHANGELOG_DIR) unless File.directory?(CHANGELOG_DIR)
- 
-      change_body=`git log --pretty=format:'%aD%n%an <%ae>%n%s%n'`
-
-      File.open(File.join(CHANGELOG_DIR, "CHANGELOG.git"), 'w') do |f|
-        f << change_body 
-      end
- 
-      # Changelog commit
-      `git add #{CHANGELOG_DIR}/CHANGELOG.git`
-      `git commit -m "Update CHANGELOG.git"`
-    end