deprec 2.1.19 → 2.2.0

data/CHANGELOG

@@ -1,6 +1,16 @@
 # deprec changelog
 
-= 2.1.19 (Feb 4, 2011)
+= 2.2.1  (Jul 10, 2011)
+
+* First release since rewrite of http://deprec.org
+* Tidy up of various recipes
+* Introduction of deprec:rack namespace
+
+= 2.1.20 (Mar 4, 2011)
+
+* generate apache vhost to current dir if config/ not present
+
+= 2.1.19 (Mar 4, 2011)
 
 * replaced mailx with mailutils
 * cap deprec:apache:vhost # handy vhost generator

data/{README → README.md}

@@ -1,41 +1,39 @@
-------------------------------------------
 deprec - Deployment Recipes for Capistrano
-------------------------------------------
+==========================================
 
-== Introduction
-
-The deprec [1] gem is a set of tasks for Capistrano [2]. These tasks provide
+The [deprec][1] gem is a set of tasks for [Capistrano][2]. These tasks provide
 for the installation, configuration and control of system services on servers
 running Ubuntu linux. Deprec was created in 2006 by Mike Bailey to setup an
 environment for running Ruby on Rails web applications on Ubuntu servers. Since
-then its uses have grown to installing Xen virtualization, mail, monitoring, high
-availability IP failover and other services.
+then its uses have grown to installing Xen virtualization, mail, monitoring, 
+high availability IP failover and other services.
 
 The tasks are run at the command line on your workstation and connect to 
 remote servers via ssh to run commands and copy out files.
 
 Deprec-2.x is a complete rewrite of the project that achieves the following:
 
-- support for Capistrano 2
-- generated config files are stored locally to enable editing and version control
-- support for more services (heartbeat, nagios, nginx, ntp, postfix, etc) 
-- multiple Rails deployment options (Passenger+Apache, Mongrel+Apache/Nginx)
-- creation of a standard base set of task names
-- tasks are cleanly separated into namespaced units (one file per service)
-- interactive prompting for missing config values
+* support for Capistrano 2
+* generated config files are stored locally to enable editing and version control
+* support for more services (heartbeat, nagios, nginx, ntp, postfix, etc) 
+* multiple Rails deployment options (Passenger+Apache, Mongrel+Apache/Nginx)
+* creation of a standard base set of task names
+* tasks are cleanly separated into namespaced units (one file per service)
+* interactive prompting for missing config values
 
-Deprec and Capistrano are written in the Ruby programming language [3] however 
+Deprec and Capistrano are written in the [Ruby programming language][3] however 
 no knowledge of Ruby is required to use it. Users should be able to write 
 new tasks and modify existing options without prior knowledge of Ruby.
 
 
-== Installation
+Installation
+------------
 
-Deprec can be obtained from rubyforge[4] and installed using rubygems[5].
+Deprec can using rubygems[5].
 
 	sudo gem install deprec  # installs deprec and dependancies 
-	cap depify .			 # creates ~/.caprc which you may edit
-	cap -T					 # should list lots of deprec tasks
+	depify -c                # creates ~/.caprc which you may edit
+	cap -T                   # should list lots of deprec tasks
 
 The .caprc file is loaded every time you use Capistrano. It in turn loads 
 the deprec tasks so you always have them available. Editing the .caprc file 
@@ -45,56 +43,23 @@ also put tasks here that you want access to, regardless of the current working
 directory.
 
 
-== Getting a Ruby on Rails app running on a fresh Ubuntu server
-
-This is still what brings people to deprec. You can install a full Rails stack
-and get multiple apps running on it in much less time than it would take to 
-do it manually (e.g. an hour vs. a weekend).
-
-Get your app running on a fresh Ubuntu (7.10, 8.04) server with these commands:
-
-    cd your_rails_app
-    depify . 
-    # Edit config/deploy.rb
-    cap deprec:rails:install_stack
-
-	# WARNING! Don't run the following command if you are using a shared 
-	# database server that has already been installed.
-	cap deprec:db:install 
-
-    cap deploy:setup 
-    cap deploy 
-    cap deploy:migrate
-
-Recorded times for complete install on Ubuntu hardy server (7.10 amd64)
-
-	* Passenger+REE+Apache        : 14m29s
-	* Passenger+ruby-1.8.7+Apache : 17m51s
-	* nginx+mongrel               : 16m25s
-	
-Note you can choose combinations of:
-	
-	* Ruby Enterprise Edition or ruby-1.8.7
-	* Passenger or mongrel (passenger requires you choose to the apache option)
-	* Apache or nginx
-
-You can find documentation on the deprec site. http://www.deprec.org/
-
-
-== Installing other things
+Installing other things
+-----------------------
 
 I plan to document other things I use deprec for on http://www.deprec.org/. 
 Feel free to poke around and see what's there. I use deprec to provision and 
 manage servers so you might find some things in there I haven't documented. Lucky you.
 
 
-== Disclaimer
+Disclaimer
+----------
 
 The tasks run commands that may make changes to your workstation and remote server. 
 You are advised to read the source and use at your own risk.
 
 
-== Credits
+Credits
+-------
 
 Deprec is written and maintained by Mike Bailey <mike@bailey.net.au>. 
 More about me here: [http://mike.bailey.net.au/]
@@ -103,7 +68,8 @@ Deprec was inspired and uses the brilliantly executed Capistrano. Thanks Jamis!
 This gem includes a modified copy of Neil Wilson's very useful vmbuilder_plugins gem.
 
 
-== Thanks
+Thanks
+------
 
 Deprec wouldn't be what it is without the contributions of many people, a few of whom are listed here:
 
@@ -120,7 +86,8 @@ Deprec wouldn't be what it is without the contributions of many people, a few of
     zippy
 
 
-== License
+License
+-------
 
 Deprec is licenced under the GPL. This means that you can use it in commercial 
 or open source applications. More details found here:
@@ -144,11 +111,9 @@ along with this program; if not, write to the Free Software
 Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 
 
-[1] http://www.deprec.org/
-[2] http://www.capify.org/
-[3] http://www.ruby-lang.org/en/
-[4] http://rubyforge.org/
-[5] http://rubygems.org/
-[6] http://www.sct.com.au/
-
-foo
+[1]: http://www.deprec.org/
+[2]: http://www.capify.org/
+[3]: http://www.ruby-lang.org/en/
+[4]: http://rubyforge.org/
+[5]: http://rubygems.org/
+[6]: http://www.sct.com.au/

data/bin/hosts.rb

@@ -0,0 +1,44 @@
+#!/usr/bin/env ruby
+#
+# Generate Nagios hosts files from unix hosts file
+#
+# e.g.
+# 10.1.1.1 gw.failmode.com
+# 10.1.1.5 mail.failmode.com 
+
+require 'erb'
+
+hosts_file = ARGV[0] || '/etc/hosts'
+
+template = ERB.new <<-EOF
+define host{
+  use        generic-host
+  hostgroups linux-servers
+  host_name  <%= host_name %>
+  address    <%= ip %>
+}
+EOF
+
+ValidIpAddressRegex = /^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$/;
+
+ValidHostnameRegex = /^(([a-zA-Z]|[a-zA-Z][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z]|[A-Za-z][A-Za-z0-9\-]*[A-Za-z0-9])$/;
+
+# XXX Add hostgroups to hosts
+# HOSTGROUPS = 'linux-servers', 'windows-servers', 'debian-servers', 'vmware-hosts'
+
+open(hosts_file).each do |line| 
+  ip, host_name = line.sub(/#.*/,'').split[0,2]
+  hostgroups = []
+
+
+  if ip =~ ValidIpAddressRegex and host_name =~ ValidHostnameRegex
+    filename = host_name + '.cfg'
+    File.open(filename, 'w') do |file|
+      file.write template.result(binding)
+      puts "writing #{filename}"
+    end
+  else
+    puts "not writing anything for '#{line}'"
+  end
+
+end

data/docs/nagios.txt

@@ -0,0 +1,28 @@
+Nagios with deprec
+==================
+
+For each command, append HOSTS=target.host.name
+
+# Nagios server
+cap deprec:nagios:install
+cap deprec:nagios:gen_host # One way to add hosts to monitor
+cap deprec:nagios:config
+
+# servers to be monitored
+cap deprec:nrpe:install
+cap deprec:nrpe:test_local  # does nrpe query from local host
+cap deprec:nrpe:test_remote # does nrpe query from nagios server
+
+
+# TODO
+By maintaining a simple hosts file you can generate the nagios
+hosts files. The following hosts file is valid:
+
+# This hosts file is used to generate Nagios hosts configs
+# The first two tokens are used only if they consist of a
+# valid IP address and hostname.
+#
+# hostgroups can follow but must be preceded by a '#'
+# 
+10.1.1.10 xen01 # linux-servers dom0
+10.1.1.11 web01 # linux-servers domU

data/lib/deprec/capistrano_extensions.rb

@@ -12,6 +12,7 @@ module Deprec2
     yield
     ENV['ROLES'] = old_roles.to_s unless ENV['HOSTS']
   end
+
   
   # Temporarily ignore ROLES and HOSTS
   def ignoring_roles_and_hosts
@@ -77,10 +78,10 @@ module Deprec2
     elsif path 
       # render to local file
       full_path = File.join('config', stage, app.to_s, path)
-      path_dir = File.dirname(full_path)
+      path_dir = File.dirname(File.expand_path(full_path))
       if File.exists?(full_path)
         if IO.read(full_path) == rendered_template
-          puts "[skip] File exists and is identical (#{full_path})."
+          puts "[skip] Identical file exists (#{full_path})."
           return false
         elsif overwrite?(full_path, rendered_template)
           File.delete(full_path)
@@ -92,7 +93,7 @@ module Deprec2
       FileUtils.mkdir_p "#{path_dir}" if ! File.directory?(path_dir)
       # added line above to make windows compatible
       # system "mkdir -p #{path_dir}" if ! File.directory?(path_dir) 
-      File.open(full_path, 'w'){|f| f.write rendered_template }
+      File.open(File.expand_path(full_path), 'w'){|f| f.write rendered_template }
       puts "[done] #{full_path} written"
     else
       # render to string
@@ -241,7 +242,7 @@ module Deprec2
   end
   
   # download source pkg if we don't already have it
-  def download_src(src_pkg, src_dir)
+  def download_src(src_pkg, src_dir=src_dir)
     set_pkg_defaults(src_pkg)
     create_src_dir
     # check if file exists and if we have an MD5 hash or bytecount to compare 
@@ -278,7 +279,7 @@ module Deprec2
   end
 
   # unpack src and make it writable by the group
-  def unpack_src(src_pkg, src_dir)
+  def unpack_src(src_pkg, src_dir=src_dir)
     set_pkg_defaults(src_pkg)
     pkg_dir = File.join([src_dir, src_pkg[:dir]].compact)
     case src_pkg[:download_method]
@@ -319,7 +320,8 @@ module Deprec2
   end
 
   # install pkg from source
-  def install_from_src(src_pkg, src_dir)
+  def install_from_src(src_pkg, src_dir=src_dir)
+    install_deps(src_pkg[:deps])
     set_pkg_defaults(src_pkg)
     pkg_dir = File.join([src_dir, src_pkg[:dir]].compact)
     unpack_src(src_pkg, src_dir)
@@ -329,6 +331,10 @@ module Deprec2
     run "cd #{pkg_dir} && #{sudo} #{src_pkg[:install]}" if src_pkg[:install] != ''
     run "cd #{pkg_dir} && #{sudo} #{src_pkg[:post_install]}" if src_pkg[:post_install] != ''
   end
+
+  def install_deps(packages=[])
+    apt.install({:base => Array(packages)}, :stable)
+  end
   
   def read_database_yml
     stage = exists?(:stage) ? fetch(:stage).to_s : ''
@@ -384,32 +390,15 @@ module Deprec2
     end
   end
 
-  # We don't need this. Put 'USER=root' on the command line instead.
-  #
-  # XXX Not working in deprec2
-  # ##
-  # # Run a command using the root account.
-  # #
-  # # Some linux distros/VPS providers only give you a root login when you install.
-  # 
-  # def run_as_root(shell_command)
-  #   std.connect_as_root do |tempuser|
-  #     run shell_command
-  #   end
-  # end
-  # 
-  # ##
-  # # Run a task using root account.
-  # #
-  # # Some linux distros/VPS providers only give you a root login when you install.
-  # #
-  # # tempuser: contains the value replaced by 'root' for the duration of this call
-  # 
-  # def as_root()
-  #   std.connect_as_root do |tempuser|
-  #     yield tempuser
-  #   end
-  # end
+  # deprecated...I don't see any need for this
+  # Set the value if not already set
+  # This method is accessible to all recipe files
+  def self.default(name, *args, &block)
+    puts "********** deprecation warning **********"
+    puts "Could you use set() instead of default()?"
+    puts "*****************************************"
+    set(name, *args, &block) unless exists?(name)
+  end
 
   private
 

data/lib/deprec/recipes/apache.rb

@@ -13,41 +13,12 @@ Capistrano::Configuration.instance(:must_exist).load do
       set :apache_modules_enabled, 
         %w(rewrite ssl proxy_balancer proxy_http deflate expires headers)
 
-      # Start apache vhost extras
-      # These are only used for generating vhost files with: 
-      #
-      #   cap deprec:apache:vhost
-      #
-      set(:apache_vhost_domain) { Capistrano::CLI.ui.ask 'Primary domain' }
-      set(:apache_vhost_server_alii) { 
-        Capistrano::CLI.ui.ask('ServerAlii (space separated)' ).split(' ')
-      }
-      set :apache_vhost_access_log_type, 'combined'
-      set :apache_vhost_canonicalize_hostname, true
-      set(:apache_vhost_access_log) { 
-        File.join(apache_log_dir, "#{apache_vhost_domain}-access.log")
-      }
-      set(:apache_vhost_error_log) { 
-        File.join(apache_log_dir, "#{apache_vhost_domain}-error.log")
-      }
-      set(:apache_vhost_document_root) { 
-        File.join('/var/apps', "#{apache_vhost_domain}", 'public')
-      }
-      set :apache_vhost_rack_env, false
-      # End apache vhost extras
-
-       
-      desc "Install apache"
+      desc "Install Apache"
       task :install do
-        install_deps
-        config
-      end
-      
-      # install dependencies for apache
-      task :install_deps do
         apt.install( 
           {:base => %w(apache2-mpm-prefork apache2-prefork-dev rsync ssl-cert)},
           :stable )
+        config
       end
       
       SYSTEM_CONFIG_FILES[:apache] = [
@@ -83,43 +54,30 @@ Capistrano::Configuration.instance(:must_exist).load do
         # Not required
       ]
 
-      desc "Generate configuration file(s) for apache from template(s)"
+      desc "Generate Apache config files"
       task :config_gen do
         config_gen_system
       end
 
+      # Generate system level apache configs
       task :config_gen_system do
         SYSTEM_CONFIG_FILES[:apache].each do |file|
           deprec2.render_template(:apache, file)
         end
       end
 
+      # Not used here
       task :config_gen_project do
-        PROJECT_CONFIG_FILES[:apache].each do |file|
-          deprec2.render_template(:apache, file)
-        end
-        # XXX Need to flesh out generation of custom certs
-        # In the meantime we'll just use the snakeoil cert
-        #
-        # top.deprec.ssl.config_gen if apache_ssl_enabled
-        top.deprec.ssl.config_gen if apache_ssl_enabled
       end
       
-      desc "Push apache config files to server"
+      desc "Push Apache config files to server"
       task :config, :roles => :web do
         config_system
         enable_modules
         reload
       end
       
-      desc "Generate initial configs and copy direct to server."
-      task :initial_config, :roles => :web do
-        SYSTEM_CONFIG_FILES[:apache].each do |file|
-          deprec2.render_template(:apache, file.merge(:remote => true))
-        end
-      end
-      
-      desc "Push apache config files to server"
+      desc "Push Apache config files to server"
       task :config_system, :roles => :web do
         deprec2.push_configs(:apache, SYSTEM_CONFIG_FILES[:apache])
         run "#{sudo} touch /var/www/check.txt" # Used to test webserver up
@@ -133,58 +91,81 @@ Capistrano::Configuration.instance(:must_exist).load do
         top.deprec.ssl.config if apache_ssl_enabled
       end
 
-      task :vhost do
-        file = { 
-          :template => 'vhost.erb',
-          :path => "/etc/apache2/sites-available/#{apache_vhost_domain}",
-          :mode => 0644,
-          :owner => 'root:root'
-        }
-        deprec2.render_template(:apache, file)
-      end
-      
       task :enable_modules, :roles => :web do
         apache_modules_enabled.each { |mod| sudo "a2enmod #{mod}" }
       end
+
+      #
+      # Control
+      #
       
       desc "Start Apache"
       task :start, :roles => :web do
-        send(run_method, "/etc/init.d/apache2 start")
+        run "#{sudo} /etc/init.d/apache2 start"
       end
 
       desc "Stop Apache"
       task :stop, :roles => :web do
-        send(run_method, "/etc/init.d/apache2 stop")
+        run "#{sudo} /etc/init.d/apache2 stop"
       end
 
       desc "Restart Apache"
       task :restart, :roles => :web do
-        send(run_method, "/etc/init.d/apache2 restart")
+        run "#{sudo} /etc/init.d/apache2 restart"
       end
 
       desc "Reload Apache"
       task :reload, :roles => :web do
-        send(run_method, "/etc/init.d/apache2 force-reload")
+        run "#{sudo} /etc/init.d/apache2 force-reload"
       end
 
-      desc "Set apache to start on boot"
+      desc "Set Apache to start on boot"
       task :activate, :roles => :web do
-        send(run_method, "update-rc.d apache2 defaults")
+        run "#{sudo} update-rc.d apache2 defaults"
       end
       
-      desc "Set apache to not start on boot"
+      desc "Set Apache to not start on boot"
       task :deactivate, :roles => :web do
-        send(run_method, "update-rc.d -f apache2 remove")
+        run "#{sudo} update-rc.d -f apache2 remove"
       end
       
-      task :backup, :roles => :web do
-        # not yet implemented
+      # Apache vhost extras
+      #
+      # These are only used for generating vhost files with: 
+      #
+      #   cap deprec:apache:vhost
+      #
+      set(:apache_vhost_domain) { Capistrano::CLI.ui.ask 'Primary domain' }
+      set(:apache_vhost_server_alii) { 
+        Capistrano::CLI.ui.ask('ServerAlii (space separated)' ).split(' ')
+      }
+      set :apache_vhost_access_log_type, 'combined'
+      set :apache_vhost_canonicalize_hostname, true
+      set(:apache_vhost_access_log) { 
+        File.join(apache_log_dir, "#{apache_vhost_domain}-access.log")
+      }
+      set(:apache_vhost_error_log) { 
+        File.join(apache_log_dir, "#{apache_vhost_domain}-error.log")
+      }
+      set(:apache_vhost_document_root) { 
+        File.join('/var/apps', "#{apache_vhost_domain}", 'public')
+      }
+      set :apache_vhost_rack_env, false
+      # 
+      task :vhost do
+        file = { 
+          :template => 'vhost.erb',
+          :path => "/etc/apache2/sites-available/#{apache_vhost_domain}",
+          :mode => 0644,
+          :owner => 'root:root'
+        }
+        if ! File.exists? 'config'
+          file[:path] = "../../#{apache_vhost_domain}"
+        end    
+        deprec2.render_template(:apache, file)
       end
       
-      task :restore, :roles => :web do
-        # not yet implemented
-      end
-
+      # End apache vhost extras
     end
   end
 end