vlad 1.0.0

data/History.txt

@@ -0,0 +1,5 @@
+== 1.0.0 / 2007-08-04
+
+* 1 major enhancement
+  * Birthday!
+

data/Manifest.txt

@@ -0,0 +1,19 @@
+History.txt
+Manifest.txt
+README.txt
+Rakefile
+considerations.txt
+doco/getting_started.txt
+doco/migration.txt
+doco/perforce.txt
+doco/variables.txt
+lib/rake_remote_task.rb
+lib/vlad.rb
+lib/vlad/perforce.rb
+lib/vlad/subversion.rb
+lib/vlad_tasks.rb
+test/test_rake_remote_task.rb
+test/test_vlad.rb
+test/test_vlad_perforce.rb
+test/test_vlad_subversion.rb
+test/vlad_test_case.rb

data/README.txt

@@ -0,0 +1,81 @@
+Vlad the Deployer
+    by the Ruby Hit Squad
+    http://rubyhitsquad.com/
+    http://rubyforge.org/projects/hitsquad/
+
+== DESCRIPTION:
+
+Vlad the Deployer is pragmatic application deployment automation,
+without mercy. Much like Capistrano, but with 1/10th the
+complexity. Vlad integrates seamlessly with Rake, and uses familiar
+and standard tools like ssh and rsync.
+
+Impale your application on the heartless spike of the Deployer.
+
+== FEATURES/PROBLEMS:
+
+* Full deployment automation stack.
+* Supports single server deployment with just 4 variables defined.
+* Very few dependencies. All simple.
+* Uses ssh with your ssh settings already in place.
+* Uses rsync for efficient transfers.
+* Run remote commands on one or more servers.
+* Syncs files to one or more servers.
+* Mix and match local and remote tasks.
+* Built on rake. easy.
+* Compatible with all of your tab completion shell script rake-tastic goodness.
+* Ships with tests that actually pass.
+* Engine is under 500 lines of code.
+* Super uper simple.
+* Does NOT support Windows right now. Coming soon in 1.1.
+* This is 1.0.0... expect rough edges.
+
+== SYNOPSIS:
+
+    rake vlad:setup   # first time only
+    rake vlad:update
+    rake vlad:migrate # optional
+    rake vlad:start
+
+== REQUIREMENTS:
+
+* Rake
+* Hoe
+* Rubyforge
+* open4
+
+== INSTALL:
+
+* sudo gem install -y vlad
+
+== SPECIAL THANKS:
+
+* First, of course, to Capistrano. For coming up with the idea and
+  providing a lot of meat for the recipes.
+* Scott Baron for coming up with one of the best project names evar.
+* Bradley Taylor for giving us permission to use RailsMachine recipes sans-LGPL.
+
+== LICENSE:
+
+(The MIT License)
+
+Copyright (c) 2007 Ryan Davis and the rest of the Ruby Hit Squad
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,27 @@
+# -*- ruby -*-
+
+require 'rubygems'
+require 'hoe'
+$: << 'lib'
+require 'vlad'
+
+Hoe.new('vlad', Vlad::VERSION) do |p|
+  p.rubyforge_name = 'hitsquad'
+  p.author = ["Ryan Davis", "Eric Hodel", "Wilson Bilkovich"]
+  p.email = "ryand-ruby@zenspider.com"
+  p.url = p.paragraphs_of('README.txt', 0).first.split(/\n/).map { |s| s.strip }[2..-1]
+  p.description = p.paragraphs_of('README.txt', 2..5).join("\n\n")
+  p.changes = p.paragraphs_of('History.txt', 0..1).join("\n\n")
+  p.extra_deps << 'rake'
+  p.extra_deps << 'open4'
+end
+
+task :flog do
+  sh 'find lib -name \*.rb | grep -v vlad_tasks | xargs flog | head -1'
+end
+
+task :flog_full do
+  sh 'find lib -name \*.rb | xargs flog -a'
+end
+
+# vim: syntax=Ruby

data/considerations.txt

@@ -0,0 +1,91 @@
+* might we want per connection values?
+
+* :except => {:no_release => true}
+
+  It is common to configure tasks to 'announce' deployments in IRC, Campfire,
+  etc.  If you have 6 app servers, you don't want to see 6 announcements.  In
+  Capistrano, this is handled via the :no_release => true flag.  Various tasks
+  only execute on the 'release' servers.
+
+  An easier way to meet this would be to introduce a :release role in the
+  default setup
+
+    role :release, "app1.example.com"
+    
+    remote_task :announce_in_irc, :roles => :release ...
+
+  Drawback: Yet another thing to change when you migrate a project from cap to
+  vlad
+
+* 'dynamic deployments'
+
+    role :app, "app1.example.com"
+    role :app, "app2.example.com"
+
+  Let's say that app1 and app2 need slightly different monit configurations.
+
+  In Capistrano, you might approach this by making two additional roles, and 
+  splitting your 'push a monit config' task into two. This sucks.
+
+  Vlad makes the 'execution context' of a task available. In Vlad, you would:
+
+    remote_task :update_monit, :roles => :app
+      rsync "templates/#{target_host}.monitrc", "/etc/monitrc"
+    end
+
+* fine-grained tasks
+
+    remote_task :update
+    remote_task :symlink
+    remote_task :migrate
+    remote_task :deploy => [:update, :symlink, :migrate, :restart]
+
+  Let's assume that this is a multi-server config with shared deploy path.
+  The user wants to do only a single checkout. If we make "update" be one big
+  task body that includes the update, symlink, and migrate steps,
+  it is difficult for the user to override the roles for the particular steps
+  they need to change.
+
+  If we break these into separate tasks, they can say:
+
+    Rake::Task["migrate"].options[:roles] = :master_db
+
+  and the migrations will only run on the master db
+
+* sudo / via how? and if we call it via I will stab ppl. "user" is sufficient.
+
+* handling 'use_sudo'
+
+  1. Check for this inside the 'run' command, and preface the command
+     with 'sudo' if necessary
+
+  2. Default this to 'false' in the reset method, and check for it
+     in the default tasks that we provide:
+       if use_sudo then
+         sudo "blah"
+       else
+         run "blah"
+       end
+
+  Option 2 has fewer moving parts, but clutters up the tasks that care about
+  this.
+
+* Dependencies
+
+  Task dependencies aren't settable when creating a Rake::RemoteTask.
+
+* Apache configuration
+
+  Pull in railsmachine/rails/recipes/apache.rb's apache configuration.  Needs
+  erb to work.
+
+* I really like tasks with naming <cmd>_<role> (eg setup_app,
+  start_web). We could easily make the front end remote_task command
+  look for such a convention and apply the :role => x automatically.
+
+* from bousquet: get a couple of server environment recipes that prepare your
+  machine that would be the golden ticket:
+
+    rake vlad:prepare TYPE=accelerator | ubuntu | osx | osxserver | site5 | ...
+
+  and have people maintaining those setups who depend on them

data/doco/getting_started.txt

@@ -0,0 +1,27 @@
+
+== Quick Start for a 1-Server Solution:
+
+=== Setup
+
+* Create a deploy file, usually in "config/deploy.rb":
+
+    set :application, "project"
+    set :domain, "example.com"
+    set :deploy_to, "/path/to/install"
+    set :repository, 'http://svn.example.com/project/branches/stable/'
+    
+This defaults to using 'svn export' from +repository+, and a single
+server for +app+, +db+, and +www+. If you need to tweak these things,
+refer to the variable documentation.
+
+* Add the following to your Rakefile:
+
+    require 'vlad'
+    Vlad.load 'config/deploy.rb'
+
+* <tt>rake vlad:setup</tt>
+
+=== Launch
+
+* <tt>rake vlad:update vlad:migrate vlad:start</tt>
+

data/doco/migration.txt

@@ -0,0 +1,21 @@
+== Converting from Capistrano
+
+* 'task' blocks are renamed to 'remote_task'.
+* Most variables are the same. See variables.txt for details.
+* No +with_command+ / +sudo+ / +via+ wonkiness
+* Uses real ssh so env vars and the like are not a problem
+  - no +with_env+ as a result.
+* Vlad doesn't use ':no_release' or ':primary'.
+  - If you have a task that needs to run on only one host from a role,
+    you should declare a new role for that host:
+
+      role :master_db, "master.example.com"
+
+    ..and then override the role for the task you want to limit:
+
+      Rake::Task["mytask"].options[:roles] = :master_db
+
+* The 'host' method can be used to consolidate multiple 'role' calls.
+  - host "www.example.com", :app, :web, :db
+    specifies a host with three roles.
+* migrate_env is now migrate_args.

data/doco/perforce.txt

@@ -0,0 +1,5 @@
+# Using Perforce
+
+TODO: write real doco.
+
+Set up a .p4config file and put it in the client's scm directories.

data/doco/variables.txt

@@ -0,0 +1,44 @@
+
+== Variables
+
+application::          REQUIRED: Name of your application. e.g. hitsquad
+repository::           REQUIRED: Repository path: e.g. http://repo.example.com/svn
+deploy_to::            REQUIRED: Deploy path on target machines. e.g. /var/www/app
+domain::               REQUIRED: Used for the common case of a single target
+                       server. e.g. example.com
+current_path::         The full path on the remote host that will be symlinked
+                       as 'current'. Defaults to "#{deploy_to}/current".
+current_release::      The full path to the current release's actual location.
+                       Defaults to "#{releases_path}/#{releases.last}".
+deploy_timestamped::   Create timestamped release directories instead of using
+                       revision numbers. Defaults to true.
+deploy_via::           Which SCM command should be used when deploying the app.
+                       Defaults to "export".
+latest_release::       The most recent release, which may not yet have been
+                       symlinked. Defaults to release_path.
+migrate_args::         Set this to change the RAILS_ENV that 'rake db:migrate'
+                       will run under. Defaults to "".
+migrate_target::       Set this if you need to specify a particular migration
+                       'VERSION' number. Defaults to "latest".
+rails_env::            Specifies the RAILS_ENV environment variable that will
+                       be used. Defaults to "production".
+rake::                 Set this if you need to specify an alternate path to
+                       'rake'. Defaults to "rake".
+release_name::         Name of the release directory, if deploy_timestamped is
+                       true. Defaults to timestamp: "YYYYMMDDHHMMSS".
+release_path::         Path to this release, which may not have been created
+                       yet. Defaults to "#{releases_path}/#{release_name}".
+releases::             An array of all existing releases, oldest first.
+                       Defaults to latest release directory name.
+releases_path::        Full path to the 'releases' directory on the remote host.
+                       Defaults to "#{deploy_to}/releases".
+scm::                  Which SCM module to use. Valid options are :subversion
+                       and :perforce as of 1.0. Defaults to "subversion"
+scm_path::             Path on the remote host that will be used as 'working
+                       space' for SCM tasks. Defaults to "#{deploy_to}/scm".
+sudo_password::        Asks for password when referenced.
+source::               Read-Only: An SCM worker instance defined by scm.
+shared_path::          Full path to remote 'shared' directory, symlinked into
+                       your app by default. Defaults to "#{deploy_to}/shared".
+web_command::          Command to execute when controlling the web server.
+                       Defaults to "apachectl".

data/lib/rake_remote_task.rb

@@ -0,0 +1,399 @@
+require 'rubygems'
+require 'open4'
+require 'vlad'
+
+##
+# Rake::RemoteTask is a subclass of Rake::Task that adds remote_actions that
+# execute in parallel on multiple hosts via ssh.
+
+class Rake::RemoteTask < Rake::Task
+
+  include Open4
+
+  ##
+  # Options for execution of this task.
+
+  attr_accessor :options
+
+  ##
+  # The host this task is running on during execution.
+
+  attr_accessor :target_host
+
+  ##
+  # An Array of Actions this host will perform during execution.  Use enhance
+  # to add new actions to a task.
+
+  attr_reader :remote_actions
+
+  ##
+  # Create a new task named +task_name+ attached to Rake::Application +app+.
+
+  def initialize(task_name, app)
+    super
+    @remote_actions = []
+  end
+
+  ##
+  # Add a local action to this task.  This calls Rake::Task#enhance.
+
+  alias_method :original_enhance, :enhance
+
+  ##
+  # Add remote action +block+ to this task with dependencies +deps+.  See
+  # Rake::Task#enhance.
+
+  def enhance(deps=nil, &block)
+    original_enhance(deps) # can't use super because block passed regardless.
+    @remote_actions << Action.new(self, block) if block_given?
+    self
+  end
+
+  ##
+  # Execute this action.  Local actions will be performed first, then remote
+  # actions will be performed in parallel on each host configured for this
+  # RemoteTask.
+
+  def execute
+    raise Vlad::ConfigurationError, "No target hosts specified for task: #{self.name}" if target_hosts.empty?
+    super
+    @remote_actions.each { |act| act.execute(target_hosts) }
+  end
+
+  ##
+  # Use rsync to send +local+ to +remote+ on target_host.
+
+  def rsync local, remote
+    cmd = ['rsync', '-azP', '--delete', local, "#{@target_host}:#{remote}"]
+
+    success = system(*cmd)
+
+    unless success then
+      raise Vlad::CommandFailedError, "execution failed: #{cmd.join ' '}"
+    end
+  end
+
+  ##
+  # Use ssh to execute +command+ on target_host.  If +command+ uses sudo, the
+  # sudo password will be prompted for then saved for subsequent sudo commands.
+
+  def run command
+    cmd = ["ssh", target_host, command]
+    result = []
+
+    puts cmd.join(' ') if Rake.application.options.trace
+
+    status = popen4(*cmd) do |pid, inn, out, err|
+      inn.sync = true
+
+      until out.eof? and err.eof? do
+        unless err.eof? then
+          data = err.readpartial(1024)
+          result << data
+          $stderr.write data
+
+          if data =~ /^Password:/ then
+            inn.puts sudo_password
+            result << "\n"
+            $stderr.write "\n"
+          end
+        end
+
+        unless out.eof? then
+          data = out.readpartial(1024)
+          result << data
+          $stdout.write data
+        end
+      end
+    end
+
+    unless status.success? then
+      raise Vlad::CommandFailedError, "execution failed with status #{status.exitstatus}: #{cmd.join ' '}"
+    end
+
+    result.join
+  end
+
+  ##
+  # Execute +command+ under sudo using run.
+
+  def sudo command
+    run "sudo #{command}"
+  end
+
+  ##
+  # The hosts this task will execute on.  The hosts are determined from the
+  # role this task belongs to.
+  #
+  # The target hosts may be overridden by providing a comma-separated list of
+  # commands to the HOSTS environment variable:
+  #
+  #   rake my_task HOSTS=app1.example.com,app2.example.com
+
+  def target_hosts
+    if hosts = ENV["HOSTS"] then
+      hosts.strip.gsub(/\s+/, '').split(",")
+    else
+      roles = options[:roles]
+      roles ? Rake::RemoteTask.hosts_for(roles) : Rake::RemoteTask.all_hosts
+    end
+  end
+
+  ##
+  # Returns an Array with every host configured.
+
+  def self.all_hosts
+    hosts_for(roles.keys)
+  end
+
+  ##
+  # Fetches environment variable +name+ from the environment using default
+  # +default+.
+
+  def self.fetch name, default = nil
+    name = name.to_s if Symbol === name
+    if @@env.has_key? name then
+      protect_env(name) do
+        v = @@env[name]
+        v = @@env[name] = v.call if Proc === v
+        v
+      end
+    elsif default
+      v = @@env[name] = default
+    else
+      raise Vlad::FetchError
+    end
+  end
+
+  ##
+  # Add host +host_name+ that belongs to +roles+.  Extra arguments may be
+  # specified for the host as a hash as the last argument.
+  #
+  # host is the inversion of role:
+  #
+  #   host 'db1.example.com', :db, :master_db
+  #
+  # Is equivalent to:
+  #
+  #   role :db, 'db1.example.com'
+  #   role :master_db, 'db1.example.com'
+
+  def self.host host_name, *roles
+    opts = Hash === roles.last ? roles.pop : {}
+
+    roles.each do |role_name|
+      role role_name, host_name, opts.dup
+    end
+  end
+
+  ##
+  # Returns an Array of all hosts in +roles+.
+
+  def self.hosts_for *roles
+    roles.flatten.map { |r|
+      self.roles[r].keys
+    }.flatten.uniq.sort
+  end
+
+  ##
+  # Ensures exclusive access to +name+.
+
+  def self.protect_env name # :nodoc:
+    @@env_locks[name.to_s].synchronize do
+      yield
+    end
+  end
+
+  ##
+  # Ensures +name+ does not conflict with an existing method.
+
+  def self.reserved_name? name # :nodoc:
+    !@@env.has_key?(name.to_s) && self.respond_to?(name)
+  end
+
+  ##
+  # The Rake::RemoteTask executing in this Thread.
+
+  def self.task
+    Thread.current[:task]
+  end
+
+  ##
+  # The configured roles.
+
+  def self.roles
+    host domain, :app, :web, :db if @@roles.empty?
+
+    @@roles
+  end
+
+  ##
+  # The configured Rake::RemoteTasks.
+
+  def self.tasks
+    @@tasks
+  end
+
+  ##
+  # The vlad environment.
+
+  def self.env
+    @@env
+  end
+
+  ##
+  # Resets vlad, restoring all roles, tasks and environment variables to the
+  # defaults.
+
+  def self.reset
+    @@roles = Hash.new { |h,k| h[k] = {} }
+    @@env = {}
+    @@tasks = {}
+    @@env_locks = Hash.new { |h,k| h[k] = Mutex.new }
+
+    # mandatory
+    set(:application) { raise(Vlad::ConfigurationError,
+                              "Please specify the name of the application") }
+    set(:repository)  { raise(Vlad::ConfigurationError,
+                              "Please specify the repository path") }
+    set(:deploy_to)   { raise(Vlad::ConfigurationError,
+                              "Please specify the deploy path") }
+    set(:domain)      { raise(Vlad::ConfigurationError,
+                              "Please specify the server domain") }
+
+    # optional
+    set(:current_path)    { File.join(deploy_to, "current") }
+    set(:current_release) { File.join(releases_path, releases[-1]) }
+    set :keep_releases, 5
+    set :deploy_timestamped, true
+    set :deploy_via, :export
+    set(:latest_release)  { deploy_timestamped ? release_path : current_release }
+    set :migrate_args, ""
+    set :migrate_target, :latest
+    set(:previous_release){ File.join(releases_path, releases[-2]) }
+    set :rails_env, "production"
+    set :rake, "rake"
+    set(:release_name)    { Time.now.utc.strftime("%Y%m%d%H%M%S") }
+    set(:release_path)    { File.join(releases_path, release_name) }
+    set(:releases)        { task.run("ls -x #{releases_path}").split.sort }
+    set(:releases_path)   { File.join(deploy_to, "releases") }
+    set :scm, :subversion
+    set(:scm_path)        { File.join(deploy_to, "scm") }
+    set(:shared_path)     { File.join(deploy_to, "shared") }
+
+    set(:sudo_password) do
+      state = `stty -g`
+
+      raise Vlad::Error, "stty(1) not found" unless $?.success?
+
+      begin
+        system "stty -echo"
+        $stdout.print "sudo password: "
+        $stdout.flush
+        sudo_password = $stdin.gets
+        $stdout.puts
+      ensure
+        system "stty #{state}"
+      end
+      sudo_password
+    end
+
+    set(:source) do
+      require "vlad/#{scm}"
+      Vlad.const_get(scm.to_s.capitalize).new
+    end
+  end
+
+  ##
+  # Adds role +role_name+ with +host+ and +args+ for that host.
+
+  def self.role role_name, host, args = {}
+    raise ArgumentError, "invalid host" if host.nil? or host.empty?
+    @@roles[role_name][host] = args
+  end
+
+  ##
+  # Adds a remote task named +name+ with options +options+ that will execute
+  # +block+.
+
+  def self.remote_task name, options = {}, &block
+    t = Rake::RemoteTask.define_task(name, &block)
+    t.options = options
+    roles = options[:roles]
+    t
+  end
+
+  ##
+  # Set environment variable +name+ to +value+ or +default_block+.
+  #
+  # If +default_block+ is defined, the block will be executed the first time
+  # the variable is fetched, and the value will be used for every subsequent
+  # fetch.
+
+  def self.set name, value = nil, &default_block
+    raise ArgumentError, "cannot provide both a value and a block" if
+      value and default_block
+    raise ArgumentError, "cannot set reserved name: '#{name}'" if
+      Rake::RemoteTask.reserved_name?(name)
+
+    Rake::RemoteTask.env[name.to_s] = value || default_block
+
+    Object.send :define_method, name do
+      Rake::RemoteTask.fetch name
+    end
+  end
+
+  ##
+  # Action is used to run a task's remote_actions in parallel on each of its
+  # hosts.  Actions are created automatically in Rake::RemoteTask#enhance.
+
+  class Action
+
+    ##
+    # The task this action is attached to.
+
+    attr_reader :task
+
+    ##
+    # The block this action will execute.
+
+    attr_reader :block
+
+    ##
+    # An Array of threads, one for each host this action executes on.
+
+    attr_reader :workers
+
+    ##
+    # Creates a new Action that will run +block+ for +task+.
+
+    def initialize task, block
+      @task  = task
+      @block = block
+      @workers = []
+    end
+
+    def == other # :nodoc:
+      return false unless Action === other
+      block == other.block && task == other.task
+    end
+
+    ##
+    # Execute this action on +hosts+ in parallel.  Returns when block has
+    # completed for each host.
+
+    def execute hosts
+      hosts.each do |host|
+        t = task.clone
+        t.target_host = host
+        thread = Thread.new(t) do |task|
+          Thread.current[:task] = task
+          block.call
+        end
+        @workers << thread
+      end
+      @workers.each { |w| w.join }
+    end
+  end
+end
+