ey-deploy 0.2.4.pre

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2010 Engine Yard, Inc
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,66 @@
+== ey-deploy
+
+We've been listening to your feedback here on the Engine Yard Cloud and we've heard that the current web deploy is not optimal. People like to see the deploy steps as they happen. not wait until the end to go through the logs to see what went wrong. It is also clear that running the full set of chef recipes every time you deploy is doing a lot of redundant work and taking a lot of time each deploy. So I say down and came up with a new deployment strategy that meets these needs and makes deploying to our cloud a much nicer experience. So today I am releasing a beta of our new git deployment strategy for you to use. This new system is 100% backwards and forwards compatible with the current web deploy. If you have a server already deployed you can start using this new system right on top of it with a few minutes of setup.
+
+So here is the steps needed to get a new git push based deploy up and running on your cloud system. Let's assume that you have an environment called 'production' with an ip address of 123.123.123.123 and a unix user called 'deploy' with an application attached called 'railsapp'. In the instructions below please replace these items with the correct info for your own setup.
+
+
+Once you have a  server fully up and running, follow these steps:
+
+1. SSH into each of the servers in your cluster, or just the single server if you have only one and run the following commands:
+  $ sudo  gem install ey-deploy --source http://gems.engineyard.com
+  Successfully installed ey-deploy-0.0.7
+  1 gem installed
+  $ sudo ey-deploy-install
+  engineyard git hooks installed
+
+Now back on your local machine, 
+
+  cd railsapp
+  git remote add production deploy@123.123.123.123:/data/railsapp/shared/cached-copy
+
+# NOTE  make sure to replace 'deploy' with the username you created for this env
+# NOTE  also make sure to change the ip address to the address of your app master
+# NOTE also make sure to replace 'railsapp' in the path with the name of your own application
+
+If you have only one server then you can go ahead and deploy new code like this:
+    $ git commit -m "made some changes"
+    $ git push production master
+
+If you have a cluster of servers you need to setup an ssh keypair named 'internal' for them to use to talk to each other.
+
+    $ cd ~/.ssh
+    $ ssh-keygen -t rsa                                                                                                 
+    Generating public/private rsa key pair.
+    Enter file in which to save the key (/Users/ez/.ssh/id_rsa): internal
+    Enter passphrase (empty for no passphrase): 
+    Enter same passphrase again: 
+    Your identification has been saved in /Users/ez/.ssh/internal.
+    Your public key has been saved in /Users/ez/.ssh/foo.pub.
+    The key fingerprint is:
+    79:40:8a:18:84:ff:96:39:c7:3a:3d:f2:e9:20:a1:ea ez@ezra-zygmuntowiczs-macbook-pro.local
+    The key's randomart image is:
+    +--[ RSA 2048]----+
+    | oo     .        |
+    |.  o . o         |
+    | .. . . .        |
+    |  .      o       |
+    |  .. +  S .      |
+    | . .* o  .       |
+    |. ...=           |
+    |.  .+.o.         |
+    |oE   =+.         |
+    +-----------------+
+
+Now you need to put the ssh keys in place on your servers. Run the following command once for each server in your cluster.
+
+    $ cd ~/.ssh
+    # do this for each server in your cluster
+    $ cat internal.pub | ssh deploy@server "cat >> ~/.ssh/authorized_keys && chmod 0600 ~/.ssh/authorized_keys"
+    $ cat internal | ssh deploy@server "cat > ~/.ssh/internal && chmod 0600 ~/.ssh/internal"
+
+
+Once you have those keys in place you can start deploying to your cluster via git push. A few things to keep in mind with this new strategy. Your cluster does not run a git server, so you will still keep the canonical git repo on github or on your own git server somewhere. You can push any branch you want but you will always push to master on the far end as that is the branch that will get deployed on the server. But pushing another branch form your local repo to the remote master will work fine.
+
+
+Thats it! you are now deploying with git directly to your instances. Please give this a try and give me some feedback! I'm hoping to hear from you if you use this new deploy strategy. You can reach me by responding to this post or email me at ez+git@engineyard.com.

data/Rakefile

@@ -0,0 +1,63 @@
+require 'rubygems'
+require 'rake/gempackagetask'
+require 'rubygems/specification'
+require 'date'
+require 'spec/rake/spectask'
+require File.expand_path("lib/ey-deploy/version", File.dirname(__FILE__))
+
+GEM = "ey-deploy"
+GEM_VERSION = EY::VERSION
+AUTHOR = "EY Cloud Team"
+EMAIL = "cloud@engineyard.com"
+HOMEPAGE = "http://engineyard.com"
+SUMMARY = "A gem that deploys ruby applications on EY Cloud instances"
+
+spec = Gem::Specification.new do |s|
+  s.name = GEM
+  s.version = GEM_VERSION
+  s.platform = Gem::Platform::RUBY
+  s.has_rdoc = true
+  s.extra_rdoc_files = ["README.rdoc", "LICENSE", 'TODO']
+  s.summary = SUMMARY
+  s.description = s.summary
+  s.author = AUTHOR
+  s.email = EMAIL
+  s.homepage = HOMEPAGE
+  s.bindir       = "bin"
+  s.executables  = %w(eysd)
+
+  bundle = Bundler::Definition.from_gemfile('Gemfile')
+  bundle.dependencies.each do |dep|
+    next unless dep.groups.include?(:runtime)
+    s.add_dependency(dep.name, dep.version_requirements.to_s)
+  end
+
+  s.require_path = 'lib'
+  s.autorequire = GEM
+  s.files = %w(LICENSE README.rdoc Rakefile TODO) + Dir.glob("{lib,spec}/**/*")
+end
+
+task :default => :spec
+
+desc "Run specs"
+Spec::Rake::SpecTask.new do |t|
+  t.spec_files = FileList['spec/**/*_spec.rb']
+  t.spec_opts = %w(-fs --color)
+end
+
+
+Rake::GemPackageTask.new(spec) do |pkg|
+  pkg.gem_spec = spec
+end
+
+desc "install the gem locally"
+task :install => [:package] do
+  sh %{sudo gem install pkg/#{GEM}-#{GEM_VERSION}}
+end
+
+desc "create a gemspec file"
+task :make_spec do
+  File.open("#{GEM}.gemspec", "w") do |file|
+    file.puts spec.to_ruby
+  end
+end

data/TODO

@@ -0,0 +1,4 @@
+TODO:
+Fix LICENSE with your name
+Fix Rakefile with your name and contact info
+Add your code to lib/<%= name %>.rb

data/bin/eysd

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+begin
+  require 'rubygems'
+rescue LoadError
+end
+
+$LOAD_PATH.push(File.expand_path("../lib", File.dirname(__FILE__)))
+require 'ey-deploy'
+
+EY::CLI.start

data/lib/ey-deploy.rb

@@ -0,0 +1,16 @@
+$LOAD_PATH.push(File.expand_path("ey-deploy", File.dirname(__FILE__)))
+
+require 'version'
+require 'compatibility'
+require 'strategies/git'
+require 'task'
+require 'server'
+require 'deploy'
+require 'cli'
+require 'configuration'
+
+module EY
+  def self.node
+    @node ||= JSON.parse(`sudo cat /etc/chef/dna.json`)
+  end
+end

data/lib/ey-deploy/cli.rb

@@ -0,0 +1,58 @@
+$:.unshift File.expand_path('../vendor/thor/lib', File.dirname(__FILE__))
+
+require 'thor'
+
+module EY
+  class CLI < Thor
+    method_option :migrate, :type     => :string,
+                            :desc     => "Run migrations with this deploy",
+                            :aliases  => ["-m"]
+
+    method_option :branch,  :type     => :string,
+                            :desc     => "Branch to deploy from, defaults to master",
+                            :aliases  => ["-b"]
+
+    method_option :repo,    :type     => :string,
+                            :desc     => "Remote repo to deploy",
+                            :aliases  => ["-r"]
+
+    method_option :app,     :type     => :string,
+                            :required => true,
+                            :desc     => "Application to deploy",
+                            :aliases  => ["-a"]
+
+    method_option :config,  :type     => :string,
+                            :desc     => "Additional configuration"
+
+    desc "deploy", "Deploy code from /data/<app>"
+    def deploy(default_task=:deploy)
+      EY::Deploy.run(options.merge("default_task" => default_task))
+    end
+
+
+    desc "check", "Check whether the client gem is compatible with the server gem"
+    def check(client_version, server_requirement)
+      compat = EY::Compatibility.new(client_version, server_requirement)
+      return if compat.compatible?
+
+      if compat.server_newer?
+        puts "Server library is newer than supported by the engineyard gem"
+        puts "Please upgrade the engineyard gem"
+        exit(1)
+      end
+
+      system("gem install ey-deploy -v '#{server_requirement}' > /dev/null 2>&1")
+      case $?.exitstatus
+      when 2
+        puts "Incompatible server component detected"
+        puts "Please contact us at http://cloud-support.engineyard.com"
+        exit 2
+      when 0
+        puts "Upgraded server component"
+        exit
+      else
+        exit 3
+      end
+    end
+  end
+end

data/lib/ey-deploy/compatibility.rb

@@ -0,0 +1,24 @@
+module EY
+  class Compatibility
+    attr_reader :server_version, :client_version, :server_required
+
+    def initialize(client_version, server_required)
+      require 'rubygems'
+      @client_version = Gem::Version.new(client_version)
+      @server_required = Gem::Requirement.new(server_required)
+      @server_version = Gem::Version.new(EY::VERSION)
+    end
+
+    def server_required_version
+      server_required.requirements.first.last
+    end
+
+    def compatible?
+      server_required.satisfied_by?(server_version)
+    end
+
+    def server_newer?
+      server_version > server_required_version
+    end
+  end
+end

data/lib/ey-deploy/configuration.rb

@@ -0,0 +1,111 @@
+require 'json'
+
+module EY
+  class Deploy::Configuration
+    DEFAULT_CONFIG = {
+      "migrate"      => "rake db:migrate",
+      "branch"       => "master",
+      "copy_exclude" => ".git",
+      "strategy"     => "Git",
+    }
+
+    attr_reader :configuration
+    alias :c :configuration
+
+    def initialize(opts={})
+      config = JSON.parse(opts["config"] || "{}")
+      @configuration = DEFAULT_CONFIG.merge(config).merge(opts)
+    end
+
+    # Delegate to the configuration objects
+    def method_missing(meth, *args, &blk)
+      c.key?(meth.to_s) ? c[meth.to_s] : super
+    end
+    def respond_to?(meth)
+      c.key?(meth.to_s) ? true : super
+    end
+
+    def repository_cache
+      configuration['repository_cache'] || File.join(deploy_to, "/shared/cached-copy")
+    end
+
+    def repo
+      configuration['repo'] || EY.node["applications"][app]["repository_name"]
+    end
+
+    def deploy_to
+      configuration['deploy_to'] || "/data/#{app}"
+    end
+
+    def migrate?
+      !!configuration['migrate']
+    end
+
+    def migration_command
+      configuration['migrate'] == "migrate" ? DEFAULT_CONFIG["migrate"] : configuration['migrate']
+    end
+
+    def user
+      EY.node['users'].first['username'] || 'nobody'
+    end
+    alias :group :user
+
+    def role
+      EY.node['instance_role']
+    end
+
+    def copy_exclude
+      @copy_exclude ||= Array(configuration.fetch("copy_exclude", []))
+    end
+
+    def stack
+      EY.node['environment']['stack']
+    end
+
+    def environment
+      EY.node['environment']['framework_env']
+    end
+
+    def latest_release
+      all_releases.last
+    end
+
+    def previous_release(current=latest_release)
+      index = all_releases.index(current)
+      all_releases[index-1]
+    end
+
+    def oldest_release
+      all_releases.first
+    end
+
+    def all_releases
+      Dir.glob("#{release_dir}/*").sort
+    end
+
+    def framework_envs
+      "RAILS_ENV=#{environment} RACK_ENV=#{environment} MERB_ENV=#{environment}"
+    end
+
+    def current_path
+      File.join(deploy_to, "current")
+    end
+
+    def shared_path
+      File.join(deploy_to, "shared")
+    end
+
+    def release_dir
+      File.join(deploy_to, "releases")
+    end
+
+    def release_path
+      @release_path ||= File.join(release_dir, Time.now.utc.strftime("%Y%m%d%H%M%S"))
+    end
+
+    def exclusions
+      copy_exclude.map { |e| %|--exclude="#{e}"| }.join(' ')
+    end
+
+  end
+end

data/lib/ey-deploy/deploy.rb

@@ -0,0 +1,158 @@
+# stolen wholesale from capistrano, thanks Jamis!
+require 'fileutils'
+require 'json'
+
+module EY
+  class DeployBase < Task
+    # default task
+    def deploy
+      update_repository_cache
+      require_custom_tasks
+      push_code
+
+      puts "~> Starting full deploy"
+
+      copy_repository_cache
+      bundle
+      symlink_configs
+      migrate
+      symlink
+      restart
+      cleanup
+
+      puts "~> finalizing deploy"
+    end
+
+    # task
+    def push_code
+      puts "~> Pushing code to all servers"
+      EY::Server.all.each do |server|
+        server.push_code
+      end
+    end
+
+    # task
+    def restart
+      puts "~> Restarting app servers"
+      roles :app_master, :app, :solo do
+        restart_command = case c.stack
+        when "nginx_unicorn"
+          "/etc/init.d/unicorn_#{c.app} restart"
+        when "nginx_mongrel"
+          "monit restart all -g #{c.app}"
+        when "nginx_passenger", "apache_passenger"
+          "touch #{c.latest_release}/tmp/restart.txt"
+        end
+        if restart_command
+          puts "~> restarting app: #{c.latest_release}"
+          sudo("cd #{c.current_path} && INLINEDIR=/tmp #{c.framework_envs} #{restart_command}")
+        end
+        callback(:after_restart)
+      end
+    end
+
+    # task
+    def bundle
+      roles :app_master, :app, :solo do
+        if File.exist?("#{c.latest_release}/Gemfile")
+          puts "~> Gemfile detected, bundling gems"
+          run %|cd #{c.latest_release} && bundle install|
+        end
+      end
+    end
+
+    # task
+    def cleanup
+      puts "~> cleaning up old releases"
+      sudo "ls #{c.release_dir} | head -n -3 | xargs -I{} rm -rf #{c.release_dir}/{}"
+    end
+
+    # task
+    def rollback
+      puts "~> rolling back to previous release"
+      symlink(c.previous_release)
+      FileUtils.rm_rf c.latest_release
+      puts "~> restarting with previous release"
+      restart
+    end
+
+    # task
+    def migrate
+      roles :app_master, :solo do
+        if c.migrate?
+          callback(:before_migrate)
+          puts "~> migrating"
+          cmd = "cd #{c.latest_release} && #{c.framework_envs} #{c.migration_command}"
+          puts "~> Migrating: #{cmd}"
+          run(cmd)
+        end
+      end
+    end
+
+    # task
+    def copy_repository_cache
+      puts "~> copying to #{c.release_path}"
+      sudo("mkdir -p #{c.release_path} && rsync -aq #{c.exclusions} #{c.repository_cache}/* #{c.release_path}")
+
+      puts "~> ensuring proper ownership"
+      sudo("chown -R #{c.user}:#{c.group} #{c.deploy_to}")
+    end
+
+    def symlink_configs(release_to_link=c.latest_release)
+      puts "~> Symlinking configs"
+      sudo [ "chmod -R g+w #{release_to_link}",
+        "rm -rf #{release_to_link}/log #{release_to_link}/public/system #{release_to_link}/tmp/pids",
+        "mkdir -p #{release_to_link}/tmp",
+        "ln -nfs #{c.shared_path}/log #{release_to_link}/log",
+        "mkdir -p #{release_to_link}/public",
+        "mkdir -p #{release_to_link}/config",
+        "ln -nfs #{c.shared_path}/system #{release_to_link}/public/system",
+        "ln -nfs #{c.shared_path}/pids #{release_to_link}/tmp/pids",
+        "ln -nfs #{c.shared_path}/config/database.yml #{release_to_link}/config/database.yml",
+        "chown -R #{c.user}:#{c.group} #{release_to_link}"
+        ].join(" && ")
+    end
+
+    # task
+    def symlink(release_to_link=c.latest_release)
+      callback(:before_symlink)
+      puts "~> symlinking code"
+      begin
+        sudo "rm -f #{c.current_path} && ln -nfs #{release_to_link} #{c.current_path} && chown -R #{c.user}:#{c.group} #{c.current_path}"
+      rescue => e
+        sudo "rm -f #{c.current_path} && ln -nfs #{c.previous_release(release_to_link)} #{c.current_path} && chown -R #{c.user}:#{c.group} #{c.current_path}"
+        sudo "rm -rf #{release_to_link}"
+        raise e
+      end
+    end
+
+    # before_symlink
+    # before_restart
+    def callback(what)
+      if File.exist?("#{c.latest_release}/deploy/#{what}.rb")
+        Dir.chdir(c.latest_release) do
+          puts "~> running deploy hook: deploy/#{what}.rb"
+          instance_eval(IO.read("#{c.latest_release}/deploy/#{what}.rb"))
+        end
+      end
+    end
+
+  end
+
+  class Deploy < DeployBase
+    def self.new(opts={})
+      # include the correct fetch strategy
+      include EY::Strategies.const_get(opts.strategy)::Helpers
+      super
+    end
+
+    def self.run(opts={})
+      conf = EY::Deploy::Configuration.new(opts)
+      EY::Server.config = conf
+
+      dep = new(conf)
+      dep.require_custom_tasks
+      dep.send(opts["default_task"])
+    end
+  end
+end