rudy 0.4.0 → 0.6.0

data/CHANGES.txt

@@ -1,41 +1,65 @@
 RUDY, CHANGES
 
-TODO: Look into RubyGems error
-    /Library/Ruby/Gems/1.8/gems/rake-0.8.3/lib/rake/gempackagetask.rb:13:Warning: Gem::manage_gems is deprecated and will be removed on or after March 2009.
-    rake aborted!
-    RubyGem version error: rdoc(2.4.0 not ~> 2.3.0)
+* TODO: Remove string messages for non-string output formats
+* TODO: Support for reserved instances
+* TODO: Test for zone and region support
+* TODO: Support for machine image attributes
+* TODO: Support for product codes
+* TODO: HTTPS
 
-TODO: Remove aws_swb
 
+#### 0.6.0 (2009-04-19) ###############################
 
-#### 0.4.1 (2009-03-??) ###############################
+NOTE: This is a significant re-write from 0.5
+
+* CHANGE: Rebuilt Rudy::AWS::*
+* CHANGE: Removed lib/rudy/commands/* and lib/rudy/metadata/*
+* ADDED: rudy-ec2 executable
+* CHANGE: have local copies of Caesars and Drydock to reduce external dependencies. 
+* ADDED: new output formats: yaml, json, csv, tsv
+
+
+#### 0.5.1 (2009-04-10) ###############################
+
+* CHANGE: Removed require 'rubygems'
+* FIXED: Resolved Drydock namespace issue for rudy and ird
+
+
+#### 0.5.0 (2009-04-09) ###############################
+
+NOTE: This is a significant re-write from 0.4
 
 * CHANGE: Recommend keypair config to be in ~/.rudy/config
 * FIX: Rudy now checks for user keys specified by env-role, env, and global
 * FIX: gemspec dependency net-ssh-multi
-* NEW: rerelease command
+* ADDED: --nocolor option 
+* ADDED: Ruby API
+* ADDED: Tests with 55% coverage
+* ADDED: Ruby API (Rudy::Machines, etc...)
+* ADDED: ird, a REPL (experimental)
+* ADDED: Now uses Rye 0.4.1
 
 #### 0.4 (2009-03-12) ###############################
 
 NOTE: This is a significant re-write from 0.3
 
-* CHANGE: Mostly re-written bin/ruby, moving validation to Command classes 
+* CHANGE: Mostly re-written bin/ruby, moving validation to CLI classes 
 * CHANGE: upgrade to Drydock 0.5
 * CHANGE: Moved generic EC2 commands to bin/rudy-ec2
 * CHANGE: Removed ambiguity of pluralized command names.
   * OLD: backups, disks, configs
-  * NEW: backup, disk, config
-* NEW: ssh and scp commands for connecting to and copying files to/from machines
-* NEW: New dependencies (trying out net-ssh)
-* NEW: Domain specific language for configuration
-* NEW: "Routines" for handling common actions. Starting, stopping, releasing, deploying. 
-* NEW: "rudy release" will create a release from the current working copy, start an instance, 
+  * ADDED: backup, disk, config
+* ADDED: ssh and scp commands for connecting to and copying files to/from machines
+* ADDED: New dependencies (trying out net-ssh)
+* ADDED: Domain specific language for configuration
+* ADDED: "Routines" for handling common actions. Starting, stopping, releasing, deploying. 
+* ADDED: "rudy release" will create a release from the current working copy, start an instance, 
 checkout the release, run routines. It needs some work still, but this already functions as
 a single command release process. 
-* NEW: "rudy start|destroy|restart|update|status". Routines allow us to have generic commands 
+* ADDED: "rudy start|destroy|restart|update|status". Routines allow us to have generic commands 
 that can be used for any machine group. These commands relate to starting new instances.
-* NEW: Extra caution when running destructive commands
-* NEW: Default ~/.rudy/config created if it doesn't exist. 
+* ADDED: Extra caution when running destructive commands
+* ADDED: Default ~/.rudy/config created if it doesn't exist. 
 
 
 #### 0.3 (2009-02-26) ###############################
@@ -44,9 +68,9 @@ NOTE: This is a significant re-write from 0.2
 
 * CHANGE: Re-written support/rudy-ec2-startup
 * CHANGE: upgrade to Drydock 0.4
-* NEW: More functionality for disks and backups
-* NEW: config commands
-* NEW: Per machine configuration (via ~/.rudy)
+* ADDED: More functionality for disks and backups
+* ADDED: config commands
+* ADDED: Per machine configuration (via ~/.rudy)
 
 
 #### 0.2 (2009-02-23) ###############################
@@ -55,18 +79,18 @@ NOTE: This is a complete re-write from 0.1
 
 * CHANGE: Added Environment variables
 * CHANGE: upgrade to drydock 0.3.3
-* NEW: All time references are converted to UTC
-* NEW: Safer "Are you sure?". Number of characters to enter is 
+* ADDED: All time references are converted to UTC
+* ADDED: Safer "Are you sure?". Number of characters to enter is 
        commiserate with amount of danger.  
-* NEW: Commands: myaddress, addresses, images, instances, 
+* ADDED: CLIs: myaddress, addresses, images, instances, 
        disks, connect, copy, stage, backups, volumes
-* NEW: Metadata storage to SimpleDB for disks, backups
-* NEW: Creates EBS volumes based on startup from metadata
-* NEW: Automated release process
-* NEW: Automated creation of machine images
-* NEW: Partial support for regions and zones
-* NEW: Manage system based on security groups. 
-* NEW: "rudy groups" overhaul. Display, creates, destroys groups. 
+* ADDED: Metadata storage to SimpleDB for disks, backups
+* ADDED: Creates EBS volumes based on startup from metadata
+* ADDED: Automated release process
+* ADDED: Automated creation of machine images
+* ADDED: Partial support for regions and zones
+* ADDED: Manage system based on security groups. 
+* ADDED: "rudy groups" overhaul. Display, creates, destroys groups. 
 
 
 

data/README.rdoc

@@ -1,34 +1,122 @@
-= Rudy - v0.4 ALPHA!
+= Rudy - v0.6 ALPHA
 
-Rudy is a handy staging and deployment tool for EC2.
+Rudy is a command-line development and deployment tool for Amazon Web Services. It helps you build and manage machines in EC2 by organizing them into groups of _environments_ and _roles_. These are called _machine_ _groups_. <tt>stage</tt> is the default environment and <tt>app</tt> is the default role. You can run multiple machines with the same role. These are called _positions_. Rudy also supports running machine groups across availability _zones_. 
 
-NOTE: Rudy will be ready for general consumption in Q2 2009.
+When you put all this together, you have a unique name for every machine. The default machine is:
+
+             zone     env  role
+              v        v    v   
+        m-us-east-1b-stage-app-01
+        ^                       ^
+     "machine"                position
+
+All configuration is organized into the zones, environments, and roles that you specify (Rudy assumes positions are identical which is important for backups and scaling). And as you'd expect, the defaults can be changed too (in <tt>~/.rudy/config</tt>). See below for usage examples. 
+
+<b>NOTE: This is an ALPHA release. That means Rudy's not ready for production use! See Alpha Warning.</b>
 
 == Installation
 
-* Soon!
-* ruby -ropenssl -e 'puts OpenSSL::OPENSSL_VERSION' (See net/ssh docs)
+=== Step 1 of 2 (get Rudy on to your machine):
+
+Via Rubygems, one of:
+
+    $ sudo gem install rudy
+    $ sudo gem install solutious-rudy --source http://gems.github.com/
+
+or via download:
+* rudy-0.6.0.tar.gz[http://github.com/solutious/rudy/tarball/rudy-0.6.0]
+* rudy-0.6.0.zip[http://github.com/solutious/rudy/zipball/rudy-0.6.0]
+
+NOTE: <em>If you are not installing via RubyGems, you need to make sure the dependencies are in your LOAD_PATH (<tt>$:</tt>). Ryan Tomayko wrote a gist[http://gist.github.com/54177] about it.</em>
+
+=== Step 2 of 2 (initialize your configuration):
+
+    $ rudy generate-config
+      [edit ~/.rudy/config with your Amazon Web Services credentials] 
+    $ rudy init
+
+NOTE: <em>You need an Amazon[http://aws.amazon.com/] account to use Rudy. Sign up for EC2[http://aws.amazon.com/ec2/],  SimpleDB[http://aws.amazon.com/simpledb/], and S3[http://aws.amazon.com/s3/]</em>
+
+
+== Example -- Launch a Machine Group
+
+Machine groups are named after the current environment and role. The default machine group is <tt>stage-app</tt>. In this example, Rudy will create the default machine group from scratch. 
+
+<b>Note: Make sure your <tt>~/.rudy/config</tt> contains your AWS credentials</b>
+
+We need to tell Rudy what this machine group should look like. We can generate a default machine configuration with this command:
+
+    $ rudy config --rudy > Rudyfile
+
+This will generate a config file called <tt>Rudyfile</tt> which contains a default machines and routines configuration. The machine configuration describes the "physical" characteristics of your machine groups. The routine configuration describes what happens when you startup and shutdown a machine group (Rudy refers to these as _routines_).
+    
+We can launch the default machine group (stage-app) with this command:
+
+    $ rudy startup
+
+You now have a new EC2 instance called <tt>m-us-east-1b-stage-app-01</tt> to work with. You can log into it with:
+
+    $ rudy -u root ssh
+    
+The <tt>-u root</tt> tells Rudy to open a connection as the root user. If you create an account on that instance, you can login as that user too. When you don't feel like paying for the instance you can run the following command to destroy it forever:
+
+    $ rudy shutdown
+
+
+== Dependencies:
+
+* OpenSSH[http://www.openssh.com/]
+* Net::SSH[http://net-ssh.rubyforge.org/]
+* Net::SCP[http://net-ssh.rubyforge.org/]
+* amazon-ec2[http://github.com/grempe/amazon-ec2]
+* Rye[http://github.com/delano/rye]
+* Drydock[http://github.com/delano/drydock]
+* Caesars[http://github.com/delano/caesars]
+
+
+== Common Errors
+
+* <tt>"`require': no such file to load -- amazon-ec2 (LoadError)"</tt>
+  * You're probably running a mix of Rubygem and non-Rubygem libraries. Ryan Tomayko has you covered: http://gist.github.com/54177
+* <tt>"The request signature we calculated does not match the signature you provided."</tt>
+  * Check that your Amazon Web Service access key and secret key are correct
+* <tt>"AWS was not able to validate the provided access credentials"</tt>
+  * If it's a new Amazon Account, you need to sign up for EC2, SDB, and S3. 
+
+== Alpha Warning
+
+Although Rudy has been pretty well tested and is currently being used in production in a private beta, it's not ready for general production use. There are a few things to keep in mind while Rudy is in alpha status.
+
+* If you already have private SSH keys, back them up. Rudy is quite safe but hasn't been audited so backup your keys as a precaution.
+* If you're running EC2 instances in production, create a new set of AWS credentials (access key and secret key) to use with Rudy. This will ensure that Rudy has no access to your production instances. 
+* Rudy commands and APIs may change in later releases. This means that if you create an environment with an alpha release and later upgrade Rudy, some things may not work as expected. 
+
 
 == More Info
 
 * GitHub[http://github.com/solutious/rudy]
+* Issues[http://github.com/solutious/rudy/issues]
 * RubyForge[http://rubyforge.org/projects/rudy]
 * Inspiration[http://www.youtube.com/watch?v=CgaiIW5Rzes]
 
 
-== Credits
-
-* Delano Mandelbaum (delano@solutious.com)
-* Keshia Knight Pulliam (rudy@solutious.com)
-
-
 == Thanks
 
-* The Rilli.com team -- for the initial use case, the ongoing feedback and support, and the good times!
+* The Rilli.com[http://rilli.com] team -- for the initial use case, the ongoing feedback and support, and the good times!
   * Adam Bognar
   * Andrew Simpson
   * Caleb Buxton
   * Colin Brumelle
+* Everyone at Utrecht.rb[http://groups.google.com/group/utrecht-rb]
+* Jamis Buck for writing Net::SSH and friends. 
+
+
+== Credits
+
+* Rudy::AWS::SDB adapted from aws_sdb by Tim Dysinger (http://dysinger.net)
+* Escape, Copyright (C) 2006,2007 Tanaka Akira  <akr@fsij.org>
+* Keshia Knight Pulliam (rudy@solutious.com)
+* Delano Mandelbaum (delano@solutious.com)
 
 
 == License

data/Rakefile

@@ -1,15 +1,106 @@
-require 'rubygems'
+
 require 'rake/clean'
 require 'rake/gempackagetask'
 require 'hanna/rdoctask'
+require 'rake/testtask'
+require 'shoulda/tasks'
+require 'rake/runtest'
+require 'monkeyspecdoc'  # http://jgre.org/2008/09/03/monkeyspecdoc/
 require 'fileutils'
 include FileUtils
  
-task :default => :package
+task :default => :test
  
-# SPECS ===============================================================
+# TESTS ===============================================================
  
-# None-yet!
+Rake::TestTask.new(:test_old) do |t|
+  require 'monkeyspecdoc'
+  test_files = FileList['test/**/*_test.rb'] || []
+  t.test_files = test_files
+  t.ruby_opts = ['-rubygems'] if defined? Gem
+  t.verbose = true
+#  t.warning = true
+  
+end
+
+
+namespace :test do
+  desc 'Measures test coverage'
+  task :coverage do
+    rm_f "coverage"
+    rm_f "coverage.data"
+    rcov = "rcov -Itest --aggregate coverage.data -T -x ' rubygems/*,/Library/Ruby/Site/*,gems/*,rcov*'"
+    system("#{rcov} --html test/**/*_test.rb")
+    system("open coverage/index.html") if RUBY_PLATFORM['darwin']
+  end
+  task :list do
+    puts Dir.glob(File.join('test', '**', '*_test.rb'))
+  end
+  
+  # Run individual test groups with: 
+  # rake test:05 
+  # rake test:60
+  # etc...
+  ('00'..'99').each do |group|
+    task group.to_sym do
+      Rake.run_tests "test/#{group}*/*_test.rb"
+    end
+    
+    # And also individual test files
+    # rake test:50:10
+    # etc...
+    ('00'..'99').each do |test|
+      namespace group.to_sym do
+        task test.to_sym do
+          Rake.run_tests "test/#{group}*/{00,#{test}}*_test.rb"
+        end
+      end
+    end
+  end
+end
+
+task :test do
+
+  #all_tests = Dir.glob(File.join('test', '{05,20,50}*', '*_test.rb')) || []
+  #all_tests.sort.each do |file|
+  #  load file
+  #end
+  Rake.run_tests 'test/**/*_test.rb'
+end
+
+
+# From: shoulda/tasks/list_tests.rake
+namespace :shoulda do
+  desc "List the names of the test methods in a specification like format"
+  task :list_fixed do
+    $LOAD_PATH.unshift("test")
+
+    require 'test/unit'
+    require 'active_support'
+
+    # bug in test unit.  Set to true to stop from running.
+    Test::Unit.run = true
+
+    test_files = Dir.glob(File.join('test', '**', '*_test.rb'))
+    
+    test_files.each do |file|
+      load file
+      klass = File.basename(file, '.rb').classify
+      #unless Object.const_defined?(klass.to_s)    # => raises: wrong constant name 00SetupTest
+      unless Object.constants.member?(klass.to_s)  # fixed
+        puts "Skipping #{klass} because it doesn't map to a Class"
+        next
+      end
+      klass = klass.constantize
+
+      puts klass.name.gsub('Test', '')
+
+      test_methods = klass.instance_methods.grep(/^test/).map {|s| s.gsub(/^test: /, '')}.sort
+      test_methods.each {|m| puts "  " + m }
+    end
+  end
+end
+
 
 # PACKAGE =============================================================
 
@@ -22,7 +113,11 @@ Rake::GemPackageTask.new(@spec) do |p|
   p.need_tar = true if RUBY_PLATFORM !~ /mswin/
 end
 
-task :release => [ :rdoc, :package ]
+task :release => [ :rdoc, :package ] do
+  $: << File.join(File.dirname(__FILE__), 'lib')
+  require "rudy"
+  abort if Drydock.debug?
+end
 
 task :install => [ :rdoc, :package ] do
 	sh %{sudo gem install pkg/#{name}-#{version}.gem}
@@ -52,17 +147,17 @@ end
 Rake::RDocTask.new do |t|
 	t.rdoc_dir = 'doc'
 	t.title    = @spec.summary
-	t.options << '--line-numbers' << '--inline-source' << '-A cattr_accessor=object'
+	t.options << '--line-numbers' <<  '-A cattr_accessor=object'
 	t.options << '--charset' << 'utf-8'
 	t.rdoc_files.include('LICENSE.txt')
 	t.rdoc_files.include('README.rdoc')
 	t.rdoc_files.include('CHANGES.txt')
+	#t.rdoc_files.include('Rudyfile')  # why is the formatting f'd?
 	t.rdoc_files.include('bin/*')
-	t.rdoc_files.include('lib/*.rb')
 	t.rdoc_files.include('lib/**/*.rb')
 end
 
-CLEAN.include [ 'pkg', '*.gem', '.config', 'doc' ]
+CLEAN.include [ 'pkg', '*.gem', '.config', 'doc', 'coverage*' ]
 
 
 

data/Rudyfile

@@ -0,0 +1,119 @@
+# = Rudy
+# 
+# This is an example Rudy configuration.
+#
+# Rudy automatically looks for configuration files in the 
+# following locations (in this order):
+#
+#     ~/.rudy/config
+#     ./.rudy/config
+#
+#     ./Rudyfile
+#     ./config/rudy/*.rb
+#     ./.rudy/*.rb
+#     /etc/rudy/*.rb
+#
+# When multuple files are found, the configuration is 
+# NOT OVERRIDDEN. It's ADDED / APPENDED. This means you can 
+# split configuration across many files as you please. 
+#
+# There are three sections: accounts, defaults, machines. 
+# 
+# By convention, accounts and defaults go in ~/.rudy/config or ./.rudy/config 
+# machines configuration goes in ./Rudyfile or ./config/rudy/machines.rb
+
+
+# ---------------------------------------------------------  MACHINES  --------
+# The machines block describes the "physical" characteristics
+# of your environments. 
+machines do
+  
+  users do
+    # If you already have private keys for logging in to your EC2 instances
+    # EC2 instances you can specify them here and Rudy will use these instead.
+    # root :keypair => "/#{Rudy.sysinfo.home}/.rudy/root-private-key"
+  end
+  
+  zone :"us-east-1b" do
+    ami 'ami-235fba4a'    # Amazon Getting Started AMI (US)
+  end
+  zone :"eu-west-1b" do
+    ami 'ami-e40f2790'    # Amazon Getting Started AMI (EU)
+  end
+
+  # We've defined an environment called "stage" with one role: "app". 
+  # The configuration inside the env block is available to all its 
+  # roles. The configuration inside the role blocks is available only
+  # to machines in that specific role. 
+  env :stage do
+    ami "ami-5394733a"    # ec2onrails/ec2onrails-v0_9_9_1-i386.manifest.xml
+    size 'm1.small'
+    
+    role :app do
+      positions 1
+      
+      # We define two disks for the stage-app machines
+      disks do
+        path "/rudy/disk1" do
+          size 2
+          device "/dev/sdr"
+        end
+        path "/rudy/disk2" do
+          size 1 
+          device "/dev/sdm"
+        end
+      end
+      
+    end
+
+    
+    role :SUCCESS do
+      
+    end
+    
+  end  
+
+  # The routines section below contains calls to local and remote
+  # scripts. The config contained in this block is made available
+  # those scripts in the form of a yaml file. The file is called
+  # rudy-config.yml. 
+  config do
+    dbmaster 'localhost'
+    newparam 573114
+  end
+  
+  
+end
+
+# ----------------------------------------------------------- ROUTINES --------
+# The routines block describes the repeatable processes for each machine group.
+routines do
+  
+  env :stage do
+    role :app do
+      
+      startup do
+        before_local Rudy.sysinfo.user => :uname
+        
+        disks do
+          create "/rudy/disk1"
+        end
+        
+        after :root => [:uname, :a]
+        after :root => [:touch, '/rudy/disk1/rudy-was-here']
+        after_local Rudy.sysinfo.user => :date
+      end
+      
+      shutdown do
+        before :root => '/a/nonexistent/script'
+        before :root => :uptime 
+        
+        disks do
+          destroy "/rudy/disk1"
+        end
+      end
+      
+    end
+  end
+end
+