Empact-ec2onrails 0.9.9

data/README.rdoc

@@ -0,0 +1,232 @@
+= EC2 on Rails
+
+
+== Deploy a Ruby on Rails app on EC2 in five minutes
+
+EC2 on Rails is an Ubuntu Linux server image for
+"Amazon's EC2 hosting service":http://www.amazon.com/b/ref=sc_fe_l_2/102-6342260-7987311?ie=UTF8&node=201590011&no=3435361
+that's ready to run a standard Ruby on Rails application with little or no customization.
+It's a Ruby on Rails "virtual appliance":http://en.wikipedia.org/wiki/Virtual_appliance.
+
+If you have an EC2 account and can start EC2 instances you're five minutes away from deploying
+your Rails app.
+
+EC2 on Rails is "opinionated software":http://gettingreal.37signals.com/ch04_Make_Opinionated_Software.php:
+the opinion is that for many rails apps the server setup can be generalized
+and shared the same way as the web application framework itself. For many people (Twitter, this isn't for you)
+the server image can be treated the same way as other shared libraries. And if the day comes when your needs are
+unique enough that EC2 on Rails can't be configured to work for you then you can bundle your own image from it
+or fork the build source and customize it.
+
+But until then, why spend your time configuring servers?
+
+Features of the EC2 image:
+
+* Ready to deploy a Rails app with little or no configuration of the server required
+* Automatic backup of MySQL database to S3 (full backup nightly + incremental backup using binary logs every 5 minutes)
+* Capistrano tasks to customize the server image, archive and restore the database to/from S3, and more (available as a rubygem)
+* Mongrel_cluster behind Apache 2.2, configured according to
+  "Coda Hale's excellent guide":http://blog.codahale.com/2006/06/19/time-for-a-grown-up-server-rails-mongrel-apache-capistrano-and-you/
+* Ruby on Rails 2.1.0, 2.0.2 and 1.2.6
+* Ruby 1.8.6
+* MySQL 5
+* "memcached":http://www.danga.com/memcached/
+* "monit":http://www.tildeslash.com/monit/ configured to monitor apache, mongrel, mysql, memcached, drive space and system load
+* Ubuntu 8.04 LTS "Hardy" base image built using "Eric Hammond's EC2 Ubuntu script":http://alestic.com/
+* SSL support
+* Amazon AMI tools installed
+* MySQL, Apache, and syslog configured to use /mnt for data and logging so you don't fill up EC2's small root filesystem
+* Automatically archives Rails and Apache logs to S3 nightly.
+* 32-bit and 64-bit images available (supports all instance types, small to extra large).
+* Created using a build file, full source is "available":http://rubyforge.org/scm/?group_id=4552 (the EC2 on Rails script is run from "Eric Hammond's EC2 Ubuntu script":http://alestic.com/)
+* Can be used as a clustered Rails app running on multiple instances
+* Automatically runs hourly, daily, weekly and monthly scripts if they exist in Rails application's script directory
+* Local "Postfix":http://www.postfix.org/ SMTP mail server (only available from within the instance, not listening on external network interfaces)
+
+
+== Using the image
+
+This documentation will be improved soon, for now hopefully this covers the basics.
+
+The current AMI id's are:
+* ami-c9bc58a0 (32-bit)
+* ami-cbbc58a2 (64-bit)
+
+_I will keep these images for as long as possible, they will not be deleted for at least a few years._
+
+
+=== 1. Install the gem
+
+<pre>sudo gem install ec2onrails</pre>
+
+=== 2. Add the config files to your Rails app
+
+Put "Capfile":http://ec2onrails.rubyforge.org/svn/trunk/documentation/examples/Capfile
+in the root of your rails folder, and put
+"deploy.rb":http://ec2onrails.rubyforge.org/svn/trunk/documentation/examples/deploy.rb
+and
+"s3.yml":http://ec2onrails.rubyforge.org/svn/trunk/documentation/examples/s3.yml
+in the config folder.
+
+_Be sure to customize those files and read the comments._
+
+Also, use the hostname "db_primary" in your database.yml file. After running "cap ec2onrails:server:set_roles" it will resolve
+to the instance defined in your Capistrano "db" role.
+
+=== 4. Start up one or more instances of the image.
+
+There is nothing EC2 on Rails-specific here yet (though soon there will be a Capistrano task to do this for you),
+if you've started EC2 instances before you can skip this section. Otherwise, I'm not going to lie, this part is complicated
+and will take a lot more than 5 minutes the first time.
+
+Read the
+"running an instance section":http://docs.amazonwebservices.com/AWSEC2/2007-08-29/GettingStartedGuide/running-an-instance.html
+in Amazon's getting started guide.
+
+For the AMI id's of the current images do <code>cap ec2onrails:ami_ids</code> from within the app that you
+configured in the previous step (they're also listed earlier on this page).
+
+_NOTE: Only use the images that match the current version of the gem._
+
+Please see the "change log":http://ec2onrails.rubyforge.org/svn/trunk/gem/History.txt for release notes, and
+see the "list of open issues":http://rubyforge.org/tracker/?atid=17558&group_id=4552&func=browse.
+
+As is "standard for public AMI's":http://docs.amazonwebservices.com/AWSEC2/2007-08-29/DeveloperGuide/public-ami-guidelines.html,
+password-based logins are disabled. You log in with your own
+"public/private keypair":http://docs.amazonwebservices.com/AWSEC2/2007-08-29/GettingStartedGuide/running-an-instance.html.
+
+Most basic things can be configured automatically by the Capistrano tasks, but if you want to
+you can login by ssh as a user named "admin" (has sudo ability) or as "app" (the user
+that the app runs as, does not have sudo ability). The Capistrano tasks automatically
+use the app user to deploy the app, and the admin user for server admin tasks
+that require sudo.
+
+IMPORTANT: Double-check "your firewall settings":http://docs.amazonwebservices.com/AWSEC2/2007-08-29/GettingStartedGuide/running-an-instance.html.
+Be sure that you haven't allowed public access to any ports other than TCP 22 and TCP 80
+(and possibly TCP 443 if you're going to enable HTTPS).
+If you're using multiple instances, be sure to allow them network access to each other.
+
+
+=== 5. Copy your public key from the server to keep Capistrano happy
+
+This is a workaround for a quirk in Capistrano. Technically all you should need to connect to the server is the private
+key file, the public key is on the server. But for some reason
+"Capistrano requires that you have both the public key and the private key files together on the client":http://groups.google.com/group/capistrano/browse_thread/thread/1102208ff925d18.
+
+There is a Capistrano task that tries to fix this for you. From within the root of your rails app do:
+
+<pre>cap ec2onrails:get_public_key_from_server</pre>
+
+Note, this will only work if you have an external ssh command in the path, it won't work for most Windows users.
+
+
+=== 6. Deploy the app with Capistrano
+
+Now that the gem is installed, your deploy.rb is configured and you can start and stop EC2 instances,
+this is the only thing you'll need to do from now on.
+
+<pre>
+cap ec2onrails:setup
+cap deploy:cold
+</pre>
+
+Yes, it's that easy! The setup task will set the server's timezone, install any
+gems and Ubuntu packages that you specified in the config file, and
+create your database.
+
+That's it, your app is now running on EC2!!
+
+
+== Capistrano tasks
+
+"Capistrano":http://capify.org is the most commonly used Rails deployment tool. It comes with many standard "tasks",
+and the EC2 on Rails gem includes Capistrano tasks specifically for configuring the server instance.
+
+Capistrano is run from the command-line using the "cap" command, with a task name given as an argument.
+
+=== Commonly-used tasks
+
+You'll mostly need just the following Capistrano tasks:
+
+* <code>cap ec2onrails:ami_ids</code>
+Shows the AMI id's of the images that match the current version of the gem.
+
+* <code>cap ec2onrails:setup</code>
+This task configures a newly-launched instance. This is the first thing you should do after
+starting a new instance. It can be run more than once without ill effect. After running
+"cap ec2onrails:setup" the next thing to do is run "cap deploy:cold"
+
+* <code>cap ec2onrails:server:set_roles</code>
+Customizes each instance for it's role(s) (as defined in your Capistrano deploy.rb file).
+Run this after starting or stopping instances.
+For now this just makes sure that only the appropriate services (Apache, Mongrel, and/or MySQL)
+are running. Eventually this will customize settings for the running services also. Note that
+an instance can have more than one role. If there's only one instance it will have all roles.
+
+Note that due to the way that Capistrano works all tasks are run
+against all hosts that are currently defined in the deploy.rb file.
+So if you start a new instance then add it to your deploy.rb you will need to run
+"cap ec2onrails:setup" again which will be run on all existing instances.
+
+
+=== Database management tasks
+
+* <code>cap ec2onrails:db:archive</code>
+Archive the MySQL database to the bucket specified in your deploy.rb. This is for archiving a snapshot of your
+database into any S3 bucket. For example, you might want to do this before deploying.
+
+* <code>cap ec2onrails:db:restore</code>
+Restore the MySQL database from the bucket specified in your deploy.rb
+For example, I use this to restore the current production data (from my actual production backup
+bucket) onto a staging server that has the current production version of my
+app. I then deploy the new version which tests migrations exactly as they'll
+run on the production server.
+
+To get a full list of the Capistrano tasks at any time type <code>cap -T</code> from with your rails app root.
+
+== Building the image
+
+Building the image is not required, most people will simply use the prebuilt public
+image, but there is also a build script that builds the image. It's meant to be called by
+"Eric Hammond's EC2 Ubuntu script":http://alestic.com/.
+
+
+== Mailing lists
+
+There are two Google groups, one for
+"announcements":http://groups.google.com/group/ec2-on-rails-announce
+(usually just new release announcements) and one for
+"discussion":http://groups.google.com/group/ec2-on-rails-discuss.
+
+
+== Comments
+
+Comments are welcome. Send an email to "Paul Dowman":http://pauldowman.com/contact/
+or to the "Google group":http://groups.google.com/group/ec2-on-rails-discuss.
+If you find bugs please file them
+"here":http://rubyforge.org/tracker/?atid=17558&group_id=4552&func=browse
+or send me an "email":http://pauldowman.com/contact/.
+
+
+== Change log
+
+See the "change log":http://ec2onrails.rubyforge.org/svn/trunk/gem/History.txt.
+
+
+== How to submit patches
+
+Pleae read the "8 steps for fixing other people's code":http://drnicwilliams.com/2007/06/01/8-steps-for-fixing-other-peoples-code/.
+The source code can be checked out anonymously using:
+<pre>
+svn checkout http://ec2onrails.rubyforge.org/svn/trunk ec2onrails
+</pre>
+
+Patches can be submitted to the "RubyForge Tracker":http://rubyforge.org/tracker/?atid=17560&group_id=4552&func=browse
+or "emailed directly to me":http://pauldowman.com/contact/ .
+
+== License
+
+This code is free to use under the terms of the GPL v2.
+
+If you find EC2 on Rails useful please "recommend Paul Dowman":http://www.workingwithrails.com/person/10131-paul-dowman
+at Working With Rails.

data/Rakefile

@@ -0,0 +1,34 @@
+require 'rubygems'
+require 'rake/gempackagetask'
+require 'rake/testtask'
+require 'rake/rdoctask'
+require 'spec/rake/spectask'
+
+# read the contents of the gemspec, eval it, and assign it to 'spec'
+# this lets us maintain all gemspec info in one place. Nice and DRY.
+spec = eval(IO.read("ec2onrails.gemspec"))
+
+Rake::GemPackageTask.new(spec) do |pkg|
+  pkg.gem_spec = spec
+end
+
+task :install => [:package] do
+  sh %{sudo gem install pkg/#{GEM}-#{VERSION}}
+end
+
+Rake::TestTask.new do |t|
+  t.libs << "test"
+  t.test_files = FileList['test/test*.rb']
+  t.verbose = true
+end
+
+Spec::Rake::SpecTask.new('spec') do |t|
+  t.spec_files = FileList['test/spec/lib/*.rb']
+end
+
+Rake::RDocTask.new do |rd|
+  rd.main = "README.rdoc"
+  rd.rdoc_files.include("README.rdoc", "lib/**/*.rb")
+  rd.rdoc_dir = 'doc'
+  rd.options = spec.rdoc_options
+end

data/examples/Capfile

@@ -0,0 +1,3 @@
+load 'deploy' if respond_to?(:namespace) # cap2 differentiator
+load 'config/deploy'
+require 'ec2onrails/recipes'

data/examples/deploy.rb

@@ -0,0 +1,85 @@
+# This is a sample Capistrano config file for EC2 on Rails.
+# It should be edited and customized.
+
+set :application, "yourapp"
+
+set :repository, "http://svn.foo.com/svn/#{application}/trunk"
+
+# NOTE: for some reason Capistrano requires you to have both the public and
+# the private key in the same folder, the public key should have the
+# extension ".pub".
+ssh_options[:keys] = ["#{ENV['HOME']}/.ssh/your-ec2-key"]
+
+# Your EC2 instances. Use the ec2-xxx....amazonaws.com hostname, not
+# any other name (in case you have your own DNS alias) or it won't
+# be able to resolve to the internal IP address.
+role :web,      "ec2-12-xx-xx-xx.z-1.compute-1.amazonaws.com"
+role :app,      "ec2-34-xx-xx-xx.z-1.compute-1.amazonaws.com"
+role :db,       "ec2-56-xx-xx-xx.z-1.compute-1.amazonaws.com", :primary => true
+role :memcache, "ec2-12-xx-xx-xx.z-1.compute-1.amazonaws.com"
+
+# Whatever you set here will be taken set as the default RAILS_ENV value
+# on the server. Your app and your hourly/daily/weekly/monthly scripts
+# will run with RAILS_ENV set to this value.
+set :rails_env, "production"
+
+# EC2 on Rails config.
+# NOTE: Some of these should be omitted if not needed.
+set :ec2onrails_config, {
+  # S3 bucket and "subdir" used by the ec2onrails:db:restore task
+  :restore_from_bucket => "your-bucket",
+  :restore_from_bucket_subdir => "database",
+
+  # S3 bucket and "subdir" used by the ec2onrails:db:archive task
+  # This does not affect the automatic backup of your MySQL db to S3, it's
+  # just for manually archiving a db snapshot to a different bucket if
+  # desired.
+  :archive_to_bucket => "your-other-bucket",
+  :archive_to_bucket_subdir => "db-archive/#{Time.new.strftime('%Y-%m-%d--%H-%M-%S')}",
+
+  # Set a root password for MySQL. Run "cap ec2onrails:db:set_root_password"
+  # to enable this. This is optional, and after doing this the
+  # ec2onrails:db:drop task won't work, but be aware that MySQL accepts
+  # connections on the public network interface (you should block the MySQL
+  # port with the firewall anyway).
+  # If you don't care about setting the mysql root password then remove this.
+  :mysql_root_password => "your-mysql-root-password",
+
+  # Any extra Ubuntu packages to install if desired
+  # If you don't want to install extra packages then remove this.
+  :packages => ["logwatch", "imagemagick"],
+
+  # Any extra RubyGems to install if desired: can be "gemname" or if a
+  # particular version is desired "gemname -v 1.0.1"
+  # If you don't want to install extra rubygems then remove this
+  :rubygems => ["rmagick", "rfacebook -v 0.9.7"],
+
+  # Set the server timezone. run "cap -e ec2onrails:server:set_timezone" for
+  # details
+  :timezone => "Canada/Eastern",
+
+  # Files to deploy to the server (they'll be owned by root). It's intended
+  # mainly for customized config files for new packages installed via the
+  # ec2onrails:server:install_packages task. Subdirectories and files inside
+  # here will be placed in the same structure relative to the root of the
+  # server's filesystem.
+  # If you don't need to deploy customized config files to the server then
+  # remove this.
+  :server_config_files_root => "../server_config",
+
+  # If config files are deployed, some services might need to be restarted.
+  # If you don't need to deploy customized config files to the server then
+  # remove this.
+  :services_to_restart => %w(apache2 postfix sysklogd),
+
+  # Set an email address to forward admin mail messages to. If you don't
+  # want to receive mail from the server (e.g. monit alert messages) then
+  # remove this.
+  :admin_mail_forward_address => "you@yourdomain.com",
+
+  # Set this if you want SSL to be enabled on the web server. The SSL cert
+  # and key files need to exist on the server, The cert file should be in
+  # /etc/ssl/certs/default.pem and the key file should be in
+  # /etc/ssl/private/default.key (see :server_config_files_root).
+  :enable_ssl => true
+}

data/examples/s3.yml

@@ -0,0 +1,9 @@
+staging:
+  aws_access_key: ABC123
+  aws_secret_access_key: abc123abc123abc123abc123
+  bucket_base_name: yourbucket
+
+production:
+  aws_access_key: DEF456
+  aws_secret_access_key: def456def456def456def456
+  bucket_base_name: yourbucket

data/lib/ec2onrails.rb

@@ -0,0 +1,20 @@
+#    This file is part of EC2 on Rails.
+#    http://rubyforge.org/projects/ec2onrails/
+#
+#    Copyright 2007 Paul Dowman, http://pauldowman.com/
+#
+#    EC2 on Rails is free software; you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License as published by
+#    the Free Software Foundation; either version 2 of the License, or
+#    (at your option) any later version.
+#
+#    EC2 on Rails is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+$:.unshift File.dirname(__FILE__)

data/lib/ec2onrails/capistrano_utils.rb

@@ -0,0 +1,33 @@
+module Ec2onrails
+  module CapistranoUtils
+    def run_local(command)
+      result = system command
+      raise("error: #{$?}") unless result
+    end
+
+    def run_init_script(script, arg)
+      # since init scripts might have the execute bit unset by the set_roles script we need to check
+      sudo "sh -c 'if [ -x /etc/init.d/#{script} ] ; then /etc/init.d/#{script} #{arg}; fi'"
+    end
+
+    def make_admin_role_for(role)
+      newrole = "#{role.to_s}_admin".to_sym
+      roles[role].each do |srv_def|
+        options = srv_def.options.dup
+        options[:user] = "admin"
+        options[:port] = srv_def.port
+        options[:no_release] = true
+        role newrole, srv_def.host, options
+      end
+    end
+
+    # return hostnames for the role named role_sym that has the specified options
+    def hostnames_for_role(role_sym, options = {})
+      role = roles[role_sym]
+      unless role
+        return []
+      end
+      role.select{|s| s.options == options}.collect{|s| s.host}
+    end
+  end
+end

data/lib/ec2onrails/recipes.rb

@@ -0,0 +1,460 @@
+#    This file is part of EC2 on Rails.
+#    http://rubyforge.org/projects/ec2onrails/
+#
+#    Copyright 2007 Paul Dowman, http://pauldowman.com/
+#
+#    EC2 on Rails is free software; you can redistribute it and/or modify
+#    it under the terms of the GNU General Public License as published by
+#    the Free Software Foundation; either version 2 of the License, or
+#    (at your option) any later version.
+#
+#    EC2 on Rails is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU General Public License for more details.
+#
+#    You should have received a copy of the GNU General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+require 'fileutils'
+include FileUtils
+require 'tmpdir'
+require 'pp'
+require 'zlib'
+require 'archive/tar/minitar'
+include Archive::Tar
+
+require 'ec2onrails/version'
+require 'ec2onrails/capistrano_utils'
+include Ec2onrails::CapistranoUtils
+
+Capistrano::Configuration.instance.load do
+
+  unless ec2onrails_config
+    raise "ec2onrails_config variable not set. (It should be a hash.)"
+  end
+
+  cfg = ec2onrails_config
+
+  set :ec2onrails_version, Ec2onrails::VERSION::STRING
+  set :image_id_32_bit, Ec2onrails::VERSION::AMI_ID_32_BIT
+  set :image_id_64_bit, Ec2onrails::VERSION::AMI_ID_64_BIT
+  set :deploy_to, "/mnt/app"
+  set :use_sudo, false
+  set :user, "app"
+
+  # make an "admin" role for each role, and create arrays containing
+  # the names of admin roles and non-admin roles for convenience
+  set :all_admin_role_names, []
+  set :all_non_admin_role_names, []
+  roles.keys.clone.each do |name|
+    make_admin_role_for(name)
+    all_non_admin_role_names << name
+    all_admin_role_names << "#{name.to_s}_admin".to_sym
+  end
+
+  after "deploy:symlink", "ec2onrails:server:set_roles"
+  after "deploy:cold", "ec2onrails:db:init_backup"
+
+  # override default start/stop/restart tasks
+  namespace :deploy do
+    desc <<-DESC
+      Overrides the default Capistrano deploy:restart, uses \
+      /etc/init.d/mongrel
+    DESC
+    task :start, :roles => :app_admin do
+      run_init_script("mongrel", "start")
+      run "sleep 30" # give the service 30 seconds to start before attempting to monitor it
+      sudo "monit -g app monitor all"
+    end
+
+    desc <<-DESC
+      Overrides the default Capistrano deploy:restart, uses \
+      /etc/init.d/mongrel
+    DESC
+    task :stop, :roles => :app_admin do
+      sudo "monit -g app unmonitor all"
+      run_init_script("mongrel", "stop")
+    end
+
+    desc <<-DESC
+      Overrides the default Capistrano deploy:restart, uses \
+      /etc/init.d/mongrel
+    DESC
+    task :restart, :roles => :app_admin do
+      deploy.stop
+      deploy.start
+    end
+  end
+
+  namespace :ec2onrails do
+    desc <<-DESC
+      Show the AMI id's of the current images for this version of \
+      EC2 on Rails.
+    DESC
+    task :ami_ids do
+      puts "32-bit server image for EC2 on Rails #{ec2onrails_version}: #{image_id_32_bit}"
+      puts "64-bit server image for EC2 on Rails #{ec2onrails_version}: #{image_id_64_bit}"
+    end
+
+    desc <<-DESC
+      Copies the public key from the server using the external "ssh"
+      command because Net::SSH, which is used by Capistrano, needs it.
+      This will only work if you have an ssh command in the path.
+      If Capistrano can successfully connect to your EC2 instance you
+      don't need to do this. It will copy from the first server in the
+      :app role, this can be overridden by specifying the HOST
+      environment variable
+    DESC
+    task :get_public_key_from_server do
+      host = find_servers_for_task(current_task).first.host
+      privkey = ssh_options[:keys][0]
+      pubkey = "#{privkey}.pub"
+      msg = <<-MSG
+      Your first key in ssh_options[:keys] is #{privkey}, presumably that's
+      your EC2 private key. The public key will be copied from the server
+      named '#{host}' and saved locally as #{pubkey}. Continue? [y/n]
+      MSG
+      choice = nil
+      while choice != "y" && choice != "n"
+        choice = Capistrano::CLI.ui.ask(msg).downcase
+        msg = "Please enter 'y' or 'n'."
+      end
+      if choice == "y"
+        run_local "scp -i '#{privkey}' app@#{host}:.ssh/authorized_keys #{pubkey}"
+      end
+    end
+
+    desc <<-DESC
+      Prepare a newly-started instance for a cold deploy.
+    DESC
+    task :setup, :roles => all_admin_role_names do
+      server.set_admin_mail_forward_address
+      server.set_timezone
+      server.install_packages
+      server.install_gems
+      server.deploy_files
+      server.enable_ssl if cfg[:enable_ssl]
+      server.set_rails_env
+      server.restart_services
+      deploy.setup
+      db.create
+    end
+
+    desc <<-DESC
+      Deploy and restore database from S3
+    DESC
+    task :restore_db_and_deploy do
+      db.recreate
+      deploy.update_code
+      deploy.symlink
+      db.restore
+      deploy.migrations
+    end
+
+    namespace :ec2 do
+      desc <<-DESC
+      DESC
+      task :configure_firewall do
+        # TODO
+      end
+    end
+
+    namespace :db do
+      desc <<-DESC
+        [internal] Load configuration info for the database from
+        config/database.yml, and start mysql (it must be running
+        in order to interact with it).
+      DESC
+      task :load_config do
+        unless hostnames_for_role(:db, :primary => true).empty?
+          db_config = YAML::load(ERB.new(File.read("config/database.yml")).result)[rails_env.to_s]
+          cfg[:db_name] = db_config['database']
+          cfg[:db_user] = db_config['username'] || db_config['user']
+          cfg[:db_password] = db_config['password']
+          cfg[:db_host] = db_config['host']
+          cfg[:db_socket] = db_config['socket']
+
+          if (cfg[:db_host].nil? || cfg[:db_host].empty?) && (cfg[:db_socket].nil? || cfg[:db_socket].empty?)
+              raise "ERROR: missing database config. Make sure database.yml contains a '#{rails_env}' section with either 'host: hostname' or 'socket: /var/run/mysqld/mysqld.sock'."
+          end
+
+          [cfg[:db_name], cfg[:db_user], cfg[:db_password]].each do |s|
+            if s.nil? || s.empty?
+              raise "ERROR: missing database config. Make sure database.yml contains a '#{rails_env}' section with a database name, user, and password."
+            elsif s.match(/['"]/)
+              raise "ERROR: database config string '#{s}' contains quotes."
+            end
+          end
+        end
+      end
+
+      desc <<-DESC
+        Create the MySQL database. Assumes there is no MySQL root \
+        password. To create a MySQL root password create a task that's run \
+        after this task using an after hook.
+      DESC
+      task :create, :roles => :db do
+        on_rollback { drop }
+        load_config
+        start
+
+        # For some reason the default db on Hardy contains users with '' as the name.
+        # This causes authentication problems when connecting from localhost
+        run %{mysql -u root -D mysql -e "delete from user where User = ''; flush privileges;"}
+
+        run %{mysql -u root -e "create database if not exists #{cfg[:db_name]};"}
+        run %{mysql -u root -e "grant all on #{cfg[:db_name]}.* to '#{cfg[:db_user]}'@'%' identified by '#{cfg[:db_password]}';"}
+        run %{mysql -u root -e "grant reload on *.* to '#{cfg[:db_user]}'@'%' identified by '#{cfg[:db_password]}';"}
+        run %{mysql -u root -e "grant super on *.* to '#{cfg[:db_user]}'@'%' identified by '#{cfg[:db_password]}';"}
+      end
+
+      desc <<-DESC
+        [internal] Make sure the MySQL server has been started, just in case the db role
+        hasn't been set, e.g. when called from ec2onrails:setup.
+        (But don't enable monitoring on it.)
+      DESC
+      task :start, :roles => :db_admin do
+        sudo "chmod a+x /etc/init.d/mysql"
+        # The mysql init script can fail on the first startup if mysql takes too long
+        # to create the logfiles, so try again
+        sudo "sh -c '/etc/init.d/mysql start || (sleep 10 && /etc/init.d/mysql start)'"
+      end
+
+      desc <<-DESC
+        Drop the MySQL database. Assumes there is no MySQL root \
+        password. If there is a MySQL root password, create a task that removes \
+        it and run that task before this one using a before hook.
+      DESC
+      task :drop, :roles => :db do
+        load_config
+        run %{mysql -u root -e "drop database if exists #{cfg[:db_name]};"}
+      end
+
+      desc <<-DESC
+        db:drop and db:create.
+      DESC
+      task :recreate, :roles => :db do
+        drop
+        create
+      end
+
+      desc <<-DESC
+        Set a root password for MySQL, using the variable mysql_root_password \
+        if it is set. If this is done db:drop won't work.
+      DESC
+      task :set_root_password, :roles => :db do
+        if cfg[:mysql_root_password]
+          run %{mysql -u root -e "UPDATE mysql.user SET Password=PASSWORD('#{cfg[:mysql_root_password]}') WHERE User='root'; FLUSH PRIVILEGES;"}
+        end
+      end
+
+      desc <<-DESC
+        Dump the MySQL database to the S3 bucket specified by \
+        ec2onrails_config[:archive_to_bucket]. The filename will be \
+        "database-archive/<timestamp>/dump.sql.gz".
+      DESC
+      task :archive, :roles => :db do
+        run "/usr/local/ec2onrails/bin/backup_app_db.rb --bucket #{cfg[:archive_to_bucket]} --dir #{cfg[:archive_to_bucket_subdir]}"
+      end
+
+      desc <<-DESC
+        Restore the MySQL database from the S3 bucket specified by \
+        ec2onrails_config[:restore_from_bucket]. The archive filename is \
+        expected to be the default, "mysqldump.sql.gz".
+      DESC
+      task :restore, :roles => :db do
+        run "/usr/local/ec2onrails/bin/restore_app_db.rb --bucket #{cfg[:restore_from_bucket]} --dir #{cfg[:restore_from_bucket_subdir]}"
+      end
+
+      desc <<-DESC
+        [internal] Initialize the default backup folder on S3 (i.e. do a full
+        backup of the newly-created db so the automatic incremental backups
+        make sense).
+      DESC
+      task :init_backup, :roles => :db do
+        run "/usr/local/ec2onrails/bin/backup_app_db.rb --reset"
+      end
+    end
+
+    namespace :server do
+      desc <<-DESC
+        Tell the servers what roles they are in. This configures them with \
+        the appropriate settings for each role, and starts and/or stops the \
+        relevant services.
+      DESC
+      task :set_roles, :roles => all_admin_role_names do
+        # TODO generate this based on the roles that actually exist so arbitrary new ones can be added
+        roles = {
+          :web =>        hostnames_for_role(:web),
+          :app =>        hostnames_for_role(:app),
+          :db_primary => hostnames_for_role(:db, :primary => true),
+          :memcache =>   hostnames_for_role(:memcache)
+        }
+        roles_yml = YAML::dump(roles)
+        put roles_yml, "/tmp/roles.yml"
+        sudo "cp /tmp/roles.yml /etc/ec2onrails"
+        sudo "/usr/local/ec2onrails/bin/set_roles.rb"
+      end
+
+      desc <<-DESC
+        Change the default value of RAILS_ENV on the server. Technically
+        this changes the server's mongrel config to use a different value
+        for "environment". The value is specified in :rails_env.
+        Be sure to do deploy:restart after this.
+      DESC
+      task :set_rails_env, :roles => all_admin_role_names do
+        rails_env = fetch(:rails_env, "production")
+        sudo "/usr/local/ec2onrails/bin/set_rails_env #{rails_env}"
+      end
+
+      desc <<-DESC
+        Upgrade to the newest versions of all Ubuntu packages.
+      DESC
+      task :upgrade_packages, :roles => all_admin_role_names do
+        sudo "aptitude -q update"
+        run "export DEBIAN_FRONTEND=noninteractive; sudo aptitude -q -y safe-upgrade"
+      end
+
+      desc <<-DESC
+        Upgrade to the newest versions of all rubygems.
+      DESC
+      task :upgrade_gems, :roles => all_admin_role_names do
+        sudo "gem update --system --no-rdoc --no-ri"
+        sudo "gem update --no-rdoc --no-ri" do |ch, str, data|
+          ch[:data] ||= ""
+          ch[:data] << data
+          if data =~ />\s*$/
+            puts data
+            choice = Capistrano::CLI.ui.ask("The gem command is asking for a number:")
+            ch.send_data("#{choice}\n")
+          else
+            puts data
+          end
+        end
+
+      end
+
+      desc <<-DESC
+        Install extra Ubuntu packages. Set ec2onrails_config[:packages], it \
+        should be an array of strings.
+        NOTE: the package installation will be non-interactive, if the packages \
+        require configuration either log in as 'admin' and run \
+        'dpkg-reconfigure packagename' or replace the package's config files \
+        using the 'ec2onrails:server:deploy_files' task.
+      DESC
+      task :install_packages, :roles => all_admin_role_names do
+        if cfg[:packages] && cfg[:packages].any?
+          run "export DEBIAN_FRONTEND=noninteractive; sudo aptitude -q -y install #{cfg[:packages].join(' ')}"
+        end
+      end
+
+      desc <<-DESC
+        Install extra rubygems. Set ec2onrails_config[:rubygems], it should \
+        be with an array of strings.
+      DESC
+      task :install_gems, :roles => all_admin_role_names do
+        if cfg[:rubygems]
+          cfg[:rubygems].each do |gem|
+            sudo "gem install #{gem} --no-rdoc --no-ri" do |ch, str, data|
+              ch[:data] ||= ""
+              ch[:data] << data
+              if data =~ />\s*$/
+                puts data
+                choice = Capistrano::CLI.ui.ask("The gem command is asking for a number:")
+                ch.send_data("#{choice}\n")
+              else
+                puts data
+              end
+            end
+          end
+        end
+      end
+
+      desc <<-DESC
+        A convenience task to upgrade existing packages and gems and install \
+        specified new ones.
+      DESC
+      task :upgrade_and_install_all, :roles => all_admin_role_names do
+        upgrade_packages
+        upgrade_gems
+        install_packages
+        install_gems
+      end
+
+      desc <<-DESC
+        Set the timezone using the value of the variable named timezone. \
+        Valid options for timezone can be determined by the contents of \
+        /usr/share/zoneinfo, which can be seen here: \
+        http://packages.ubuntu.com/cgi-bin/search_contents.pl?searchmode=filelist&word=tzdata&version=gutsy&arch=all&page=1&number=all \
+        Remove 'usr/share/zoneinfo/' from the filename, and use the last \
+        directory and file as the value. For example 'Africa/Abidjan' or \
+        'posix/GMT' or 'Canada/Eastern'.
+      DESC
+      task :set_timezone, :roles => all_admin_role_names do
+        if cfg[:timezone]
+          sudo "bash -c 'echo #{cfg[:timezone]} > /etc/timezone'"
+          sudo "cp /usr/share/zoneinfo/#{cfg[:timezone]} /etc/localtime"
+        end
+      end
+
+      desc <<-DESC
+        Deploy a set of config files to the server, the files will be owned by \
+        root. This doesn't delete any files from the server. This is intended
+        mainly for customized config files for new packages installed via the \
+        ec2onrails:server:install_packages task. Subdirectories and files \
+        inside here will be placed within the same directory structure \
+        relative to the root of the server's filesystem.
+      DESC
+      task :deploy_files, :roles => all_admin_role_names do
+        if cfg[:server_config_files_root]
+          begin
+            filename = "config_files.tar"
+            local_file = "#{Dir.tmpdir}/#{filename}"
+            remote_file = "/tmp/#{filename}"
+            FileUtils.cd(cfg[:server_config_files_root]) do
+              File.open(local_file, 'wb') { |tar| Minitar.pack(".", tar) }
+            end
+            put File.read(local_file), remote_file
+            sudo "tar xvf #{remote_file} -o -C /"
+          ensure
+            rm_rf local_file
+            run "rm -f #{remote_file}"
+          end
+        end
+      end
+
+      desc <<-DESC
+        Restart a set of services. Set ec2onrails_config[:services_to_restart] \
+        to an array of strings. It's assumed that each service has a script \
+        in /etc/init.d
+      DESC
+      task :restart_services, :roles => all_admin_role_names do
+        if cfg[:services_to_restart] && cfg[:services_to_restart].any?
+          cfg[:services_to_restart].each do |service|
+            run_init_script(service, "restart")
+          end
+        end
+      end
+
+      desc <<-DESC
+        Set the email address that mail to the admin user forwards to.
+      DESC
+      task :set_admin_mail_forward_address, :roles => all_admin_role_names do
+        put cfg[:admin_mail_forward_address], "/home/admin/.forward" if cfg[:admin_mail_forward_address]
+      end
+
+      desc <<-DESC
+        Enable ssl for the web server. The SSL cert file should be in
+        /etc/ssl/certs/default.pem and the SSL key file should be in
+        /etc/ssl/private/default.key (use the deploy_files task).
+      DESC
+      task :enable_ssl, :roles => :web_admin do
+        sudo "a2enmod ssl"
+        sudo "a2ensite default-ssl"
+        run_init_script("apache2", "restart")
+      end
+    end
+
+  end
+end