crapapult 0.0.8 → 0.0.9

data/README.md

@@ -5,25 +5,22 @@ services at [Yammer](http://www.yammer.com).
 
 ## What the hell is going on here
 
-At Yammer, we use
-[Hudson](http://hudson-ci.org/)/[Jenkins](http://jenkins-ci.org/) to build all
-of our JVM-based services. Each time we commit to the various branches of a
-service, Hudson runs all the tests and (if they pass, naturally) builds a fat,
-deployable JAR containing all the various dependencies.
-
-Crapapult allows us to deploy new versions of our services *and* synchronize
-those deploys with changes to application-specific configuration. (We use 
-Puppet to provision and maintain machines with broad roles but use Crapapult to
-generate all the application-specific configuration.)
-
-It does so by using Capistrano to SSH into each of the application servers and
-using `curl` to download the last successful build artifact from the specified
-Hudson job. This tends to be super-fast, especially if your CI box is located in
-the same data center as your app servers.
-
-Once the artifact is staged (by default in `/opt/APPNAME`), it then uploads the
-specified set of asset files to each server. After that, it renders and uploads
-the specified set of template files to each server.
+At Yammer, we use store all our deployable artifacts in a [Maven](http://maven.apache.org)
+repository. Each time we commit to a service, Jenkins runs all the tests and (if they pass,
+naturally) builds a fat, deployable JAR containing all the various dependencies which then gets
+deployed to our [Nexus](http://nexus.sonatype.org) repository.
+
+Crapapult allows us to deploy new versions of our services *and* synchronize those deploys with
+changes to application-specific configuration. (We use Puppet to provision and maintain machines
+with broad roles but use Crapapult to generate all the application-specific configuration.)
+
+It does so by using Capistrano to SSH into each of the application servers and using `curl` to
+download the specified artifact from the Maven repo. This tends to be super-fast, especially if your
+Maven repo is located in the same data center as your app servers.
+
+Once the artifact is staged (by default in `/opt/APPNAME`), it then uploads the specified set of
+asset files to each server. After that, it renders and uploads the specified set of template files
+to each server.
 
 Then it restarts your service and gives you a high-five.
 
@@ -43,6 +40,11 @@ And start slingin' some crap in your `Capfile`:
 ```ruby
 require "crapapult"
 
+# Tell us where and what your application is.
+maven "http://maven.example.com/repo/"
+group_id "com.example.myapp"
+artifact_id "myapp-service"
+
 # Give your application a name.
 application "myapp" do
   # Upload the file in assets/myapp.jvm.conf to each host.
@@ -56,15 +58,8 @@ application "myapp" do
   upstart "myapp.upstart"
 end
 
-# Specify where your Hudson/Jenkins server is.
-hudson "http://build.example.com"
-
-# Specify the various branches and their respective build jobs.
-branch :master, "myapp-release"
-branch :development, "myapp-development"
-
-# Define environments with allowed branches.
-environment :staging, [:master, :development] do
+# Define environments, optionally allowing snapshots.
+environment :staging, :allow_snapshots => true do
   # Set environment-specific data for your templates to work with.
   data :jdbc_url, "jdbc:postgresql://db.example.com/happy_fun_times"
   
@@ -74,8 +69,8 @@ environment :staging, [:master, :development] do
   end
 end
 
-# This will disallow deploying the staging branch to production.
-environment :production, [:master] do
+# This will disallow deploying snapshots to production.
+environment :production do
   # Set environment-specific data for your templates to work with.
   data :jdbc_url, "jdbc:postgresql://db.example.com/happy_fun_times"
 
@@ -92,18 +87,15 @@ end
 
 ## Deploying a thing
 
-    cap deploy           # Deploy the specified branch to the specified environment
-    cap from:development # Deploy the development branch
-    cap from:master      # Deploy the master branch
-    cap to:production    # Deploy to the production environment
-    cap to:staging       # Deploy to the staging environment
+    cap production    # Deploy to the production environment
+    cap staging       # Deploy to the staging environment
 
-So to deploy your development branch to your staging environment, it's just a
-simple:
+So to deploy to your staging environment, it's just a simple:
     
-    cap from:development to:staging deploy
+    cap staging
 
-And you're off to the races.
+You'll be prompted for the version you want to deploy (and, if you're deploying a snapshot, the
+build number), and you're off to the races.
 
 --------------------------------------------------------------------------------
 Copyright (c) 2011 Yammer, Inc. See LICENSE.txt for further details.

data/Rakefile

@@ -11,9 +11,12 @@ Jeweler::Tasks.new do |gem|
   gem.email = "coda.hale@gmail.com"
   gem.authors = ["Coda Hale"]
   
-  gem.add_runtime_dependency "json"
   gem.add_runtime_dependency "capistrano"
   gem.add_runtime_dependency "erubis"
+  gem.add_runtime_dependency "nokogiri"
+  gem.add_runtime_dependency "highline"
+  gem.add_runtime_dependency "patron"
+  gem.add_runtime_dependency "yajl-ruby"
   gem.add_runtime_dependency "always_verify_ssl_certificates"
 end
-Jeweler::RubygemsDotOrgTasks.new
+Jeweler::RubygemsDotOrgTasks.new

data/VERSION

@@ -1 +1 @@
-0.0.8
+0.0.9

data/lib/crapapult.rb

@@ -1,10 +1,64 @@
 require "rubygems"
+require "patron"
+require "nokogiri"
 require "always_verify_ssl_certificates"
 require "capistrano"
 require "net/http"
 require "net/https"
-require "json"
 require "erubis"
+require "yajl/json_gem"
+
+class Maven
+  def initialize(url)
+    @url = url.gsub(/\/$/, '')
+    @session = Patron::Session.new
+    @session.timeout = 10
+    @session.base_url = @url
+  end
+
+  def find_versions(group_id, artifact_id)
+    resp = @session.get("/#{urlify(group_id)}/#{artifact_id}/maven-metadata.xml")
+    if resp.status == 200
+      Nokogiri::XML(resp.body).css("metadata versioning versions version").map { |e| e.text.strip }
+    else
+      raise "unable to find metadata for #{group_id}:#{artifact_id}"
+    end
+  end
+
+  def find_release_versions(group_id, artifact_id)
+    find_versions(group_id, artifact_id).select { |v| v !~ /SNAPSHOT$/ }
+  end
+
+  def find_snapshot_builds(group_id, artifact_id, version)
+    resp = @session.get("/#{urlify(group_id)}/#{artifact_id}/#{version}/maven-metadata.xml")
+    if resp.status == 200
+      Nokogiri::XML(resp.body).css('metadata > versioning > snapshotVersions').map { |e|
+        ext = e.at("extension")
+        if ext && ext.text.strip == "jar"
+          e.at("value").text.strip
+        else
+          []
+        end
+      }.flatten
+    else
+      raise "unable to find metadata for #{group_id}:#{artifact_id}:#{version}"
+    end
+  end
+
+  def release_artifact_url(group_id, artifact_id, version)
+    "#{@url}/#{urlify(group_id)}/#{artifact_id}/#{version}/#{artifact_id}-#{version}.jar"
+  end
+
+  def snapshot_artifact_url(group_id, artifact_id, version, build)
+    "#{@url}/#{urlify(group_id)}/#{artifact_id}/#{version}/#{artifact_id}-#{build}.jar"
+  end
+
+  private
+
+  def urlify(group_id)
+    group_id.gsub('.', '/')
+  end
+end
 
 Capistrano::Configuration.instance(:must_exist).load do
   
@@ -16,8 +70,7 @@ Capistrano::Configuration.instance(:must_exist).load do
   default_run_options[:pty] = true
   @invoke = tasks.delete(:invoke)
   @shell = tasks.delete(:shell)
-  
-  @branches = []
+
   @hosts = {}
   @data = {}
   
@@ -25,51 +78,43 @@ Capistrano::Configuration.instance(:must_exist).load do
   ###################################### API ###################################
   ##############################################################################
   
+  # Defines the artifact's groupId.
+  def group_id(name)
+    set :current_group_id, name
+  end
+
+  # Defines the artifact's artifactId.
+  def artifact_id(name)
+    set :current_artifact_id, name
+  end
+  
   # Defines a host with an optional host-specific configuration block.
   def host(name, &block)
     @hosts[name] = block || lambda {}
     server name, :app
   end
 
-  # Defines an environment with a set of allowed branches and an optional
+  # Defines an environment which might allow snapshots and an optional
   # environment-specific configuration block.
-  def environment(name, branches, &block)
+  def environment(name, opts={}, &block)
     env_block = block || lambda {}
-    
-    for b in branches
-      unless @branches.include?(b)
-        abort "Environment #{name} allows a branch which hasn't been declared: #{b}"
-      end
-    end
-    
-    namespace :to do
-      desc "Deploy to the #{name} environment"
-      task name.to_sym do
-        set :current_environment, name
-        set :allowed_branches, branches
-        env_block.call
-      end
-    end
-  end
-  
-  # Defines a branch with a Hudson job name.
-  def branch(name, job)
-    @branches << name.to_sym
-    namespace :from do
-      desc "Deploy the #{name} branch"
-      task name.to_sym do
-        set :current_branch, name
-        set :current_job, job
-      end
+
+    desc "Deploy to the #{name} environment"
+    task name.to_sym do
+      set :current_environment, name
+      set :allow_snapshots, opts[:allow_snapshots]
+      env_block.call
+      execute_task(find_task("deploy"))
     end
   end
-  
-  # Defines a Hudson server.
-  def hudson(url)
-    if exists?(:current_hudson_url)
-      abort "You're calling #hudson twice. Stop it."
+
+  # Defines a Maven repository.
+  def maven(url)
+    if exists?(:current_repo)
+      abort "You're calling #maven twice. Stop it."
+    else
+      set :current_repo, url
     end
-    set :current_hudson_url, url
   end
   
   # Defines an application with an optional owner, group, and configuration
@@ -99,7 +144,7 @@ Capistrano::Configuration.instance(:must_exist).load do
                 puts(">> #{server.host} is OK")
               else
                 puts(">> #{server.host} is DOWN!!!")
-                puts(resp.body.split("\n").map { |s| ">>> #{s}" }.join)
+                puts(resp.body.split("\n").map { |s| ">>> #{s}" }.join("\n"))
               end
             end
           end
@@ -153,12 +198,12 @@ Capistrano::Configuration.instance(:must_exist).load do
   end
   
   # Post a message to your Yammer network when you deploy.
-  def yammer(token, opts={})    
-    opts = { :host => "www.yammer.com",
-             :body => %Q{The <%= current_branch %> branch of <%= application_name %> (<%= last_ref[0..5] %>) was just deployed to the <%= current_environment %> environment by @<%= `whoami`.strip %>.}
-    }.merge(opts)
-
+  def yammer(token, opts={})
     after :deploy do
+      set :deployed_version, exists?(:build) ? "#{version} (#{build})" : version
+      opts = {:host => "www.yammer.com",
+              :body => %Q{<%= application_name %> v<%= deployed_version %> was just deployed to the <%= current_environment %> environment by @<%= `whoami`.strip %>.}
+      }.merge(opts)
       puts "> Sending notification to Yammer"
       opts[:body] = Erubis::Eruby.new(opts[:body]).result(binding)
       http = Net::HTTP.new(opts.delete(:host), 443)
@@ -180,11 +225,12 @@ Capistrano::Configuration.instance(:must_exist).load do
   ##############################################################################
   
   namespace :deploy do
-    desc "Deploy the specified branch to the specified environment"
     task :default do
       check_required_parameters!
       
-      puts "> Deploying the #{current_branch} branch to #{current_environment}"
+      puts "> Deploying #{application_name} to #{current_environment}"
+
+      artifact_url # queue this up before hand
       
       setup
       stage
@@ -194,7 +240,7 @@ Capistrano::Configuration.instance(:must_exist).load do
       verify
       cleanup
       
-      puts "> Successfully deployed the #{current_branch} branch to #{current_environment}! Congrats!"
+      puts "> Successfully #{application_name} to #{current_environment}! Congrats!"
     end
     
     task :setup, :role => :app do
@@ -257,77 +303,56 @@ Capistrano::Configuration.instance(:must_exist).load do
     "/tmp/#{application_name}-#{rand(10_000_000)}"
   end
   
-  set(:last_build) do
-    build = hudson_api(current_hudson_url + "/job/#{current_job}/lastBuild")
-    unless build["result"] == "SUCCESS"
-      abort("The last build of #{current_job} failed; see #{build["url"]} for details")
-    end
-    
-    build
-  end
-  
-  set(:last_ref) do
-    last_build["actions"].find { |f| f["lastBuiltRevision"] }["lastBuiltRevision"]["SHA1"]
-  end
-  
   set(:artifact_url) do
-    artifacts = last_build["artifacts"]
-    if artifacts.empty?
-      abort("No artifacts exist for this project!")
-    end
-    artifact = if artifacts.length > 1
-      puts "More than one artifact was found. Please choose:"
-      artifacts.each_with_index do |artifact, index|
-        puts "#{index}: #{artifact["relativePath"]}"
-      end
-      print  "? "
-      artifact_index = Capistrano::CLI.ui.ask("choice: ").to_i
-      artifacts[artifact_index]
+    hl = HighLine.new
+    m = Maven.new(current_repo)
+
+    hl.say("Select a version to deploy:")
+    versions = if allow_snapshots
+                 m.find_versions(current_group_id, current_artifact_id).reverse[0..2]
+               else
+                 m.find_release_versions(current_group_id, current_artifact_id).reverse[0..2]
+               end
+
+    set :version, hl.choose(*versions)
+    puts
+
+    if version =~ /SNAPSHOT$/
+      builds = m.find_snapshot_builds(current_group_id, current_artifact_id, version)
+      hl.say("Select a snapshot build to deploy:")
+      set :build, hl.choose(*builds)
+      m.snapshot_artifact_url(current_group_id, current_artifact_id, version, build)
     else
-      artifacts.first
+      m.release_artifact_url(current_group_id, current_artifact_id, version)
     end
-    last_build["url"] + "artifact/" + artifact["relativePath"]
   end
 
   set(:artifact_filename) do
     filename = artifact_url.split("/").last
-    "#{application_name}-#{Time.now.strftime("%Y%m%d%H%M%S")}-#{last_ref[0..5]}#{File.extname(filename)}"
+    "#{application_name}-#{Time.now.strftime("%Y%m%d%H%M%S")}-#{version}#{File.extname(filename)}"
   end
   
   set(:directory) { "/opt/#{application_name}" }
   
-  def hudson_api(url)
-    if url !~ /\/$/
-      url = url + "/api/json"
-    else
-      url = url + "api/json"
-    end
-    
-    logger.debug("getting #{url}...")
-    json = Net::HTTP.get(URI.parse(url))
-    # logger.debug("Received: #{json}")
-    JSON.parse(json)
-  end
-  
   def check_required_parameters!
     unless exists?(:application_name)
       abort("No application defined.")
     end
     
-    unless exists?(:current_hudson_url)
-      abort("No Hudson instance defined.")
+    unless exists?(:current_repo)
+      abort("No Maven repository defined.")
     end
-    
-    unless exists?(:current_environment)
-      abort("No target environment specified. Please use one of the to:* tasks.")
+
+    unless exists?(:current_group_id)
+      abort("No group ID defined.")
     end
-    
-    unless exists?(:current_branch)
-      abort("No source branch specified. Please use one of the from:* tasks.")
+
+    unless exists?(:current_artifact_id)
+      abort("No artifact ID defined.")
     end
     
-    unless allowed_branches.include?(current_branch)
-      abort "The #{current_environment} environment only allows deploys from the following branches: #{allowed_branches.join(", ")}"
+    unless exists?(:current_environment)
+      abort("No target environment specified. Please use one of the to:* tasks.")
     end
   end
   
@@ -336,4 +361,4 @@ Capistrano::Configuration.instance(:must_exist).load do
     tasks[:invoke] = @invoke
     tasks[:shell] = @shell
   end
-end
+end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: crapapult
 version: !ruby/object:Gem::Version 
-  hash: 15
+  hash: 13
   prerelease: false
   segments: 
   - 0
   - 0
-  - 8
-  version: 0.0.8
+  - 9
+  version: 0.0.9
 platform: ruby
 authors: 
 - Coda Hale
@@ -15,11 +15,11 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-05-13 00:00:00 -07:00
+date: 2011-12-21 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
-  name: json
+  name: capistrano
   prerelease: false
   requirement: &id001 !ruby/object:Gem::Requirement 
     none: false
@@ -33,7 +33,7 @@ dependencies:
   type: :runtime
   version_requirements: *id001
 - !ruby/object:Gem::Dependency 
-  name: capistrano
+  name: erubis
   prerelease: false
   requirement: &id002 !ruby/object:Gem::Requirement 
     none: false
@@ -47,7 +47,7 @@ dependencies:
   type: :runtime
   version_requirements: *id002
 - !ruby/object:Gem::Dependency 
-  name: erubis
+  name: nokogiri
   prerelease: false
   requirement: &id003 !ruby/object:Gem::Requirement 
     none: false
@@ -61,7 +61,7 @@ dependencies:
   type: :runtime
   version_requirements: *id003
 - !ruby/object:Gem::Dependency 
-  name: always_verify_ssl_certificates
+  name: highline
   prerelease: false
   requirement: &id004 !ruby/object:Gem::Requirement 
     none: false
@@ -74,6 +74,48 @@ dependencies:
         version: "0"
   type: :runtime
   version_requirements: *id004
+- !ruby/object:Gem::Dependency 
+  name: patron
+  prerelease: false
+  requirement: &id005 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 3
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id005
+- !ruby/object:Gem::Dependency 
+  name: yajl-ruby
+  prerelease: false
+  requirement: &id006 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 3
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id006
+- !ruby/object:Gem::Dependency 
+  name: always_verify_ssl_certificates
+  prerelease: false
+  requirement: &id007 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 3
+        segments: 
+        - 0
+        version: "0"
+  type: :runtime
+  version_requirements: *id007
 description: Yammer's Capistrano-based deploy crap.
 email: coda.hale@gmail.com
 executables: []