rudy 0.8.5 → 0.9.0

data/CHANGES.txt

@@ -1,18 +1,92 @@
-RUDY, CHANGES
-
-#### 0.9.0 (TBD) ###############################
-
-NOTE: The following changes are planned for the 0.9.0 release. 
-
-* group identifier will be changed to 'grp' from 'g'
-* machine identifier will be changes to 'mach' from 'm'
-* private keys are stored in ~/.ssh/
-* zone is now a single letter: a, b, c, etc... The long form (us-east-1b) will set the region too.
-* Disk syntax in the routines DSL will change. The DSL no longer accepts arguments, just the action (create, destroy, ...). 
-* Disk backups will be removed (temporarily -- probably until on of the early 1.x releases). 
-
-
-#### 0.8.5 (2009-06-??) ###############################
+RUDY, CHANGES 
+
+
+#### 0.9.0 (2009-08-04) ###########################
+
+* FIXED: Better Windows support
+* FIXED: Added rescue for disk handler in reboot routine
+* FIXED: Another fix for resolving region/zone ambiguity
+* FIXED: No Routines config error during shutdown routine
+* CHANGE: Renamed "rudy machines -S" to "rudy machines -A"
+* CHANGE: Renamed "rudy machines -A" to "rudy machines -T"
+* ADDED: Print friendly error, rather then an angry exception when running on win32 with ruby 1.9
+* ADDED: "rudy machines --disassociate" command
+* ADDED: Allow colons in place of dashes (rudy publish:gem -> publish_gem do; ...; end)
+
+
+#### 0.9.0-RC3 (2009-08-03) ###########################
+
+* FIXED: GH-18 (Error when running startup, reboot, shutdown routines in parallel mode)
+* FIXED: Routine::Handler::User.adduser was specifying invalid home directory with multiple machines
+* FIXED: Socket.connect error in JRuby when checking if SSH is available
+* FIXED: Rudy::AWS::EC2.connect now correctly reconnects when run for the second time
+* FIXED: Global CLI option -R (region) was being ignored in lieu of -z (zone)
+* FIXED: Rudy::Disk#archive wasn't storing size, fstype
+* FIXED: Disk restore was balking when no FS type supplied
+* FIXED: Reboot routine now automatically umounts volumes if the routine is configured to mount after the reboot
+* CHANGE: "rudy myaddress" is now "rudy networks -L"
+* CHANGE: Renamed Volume attributes: create_time, attach_time -> created, attached
+* CHANGE: Renamed Instance attribute: launch_time -> created
+* CHANGE: Global CLI option -C (config) can now accept multiple paths
+* ADDED: "rudy machines -A" to display available machines
+* ADDED: "rudy status" to display all EC2 objects
+* ADDED: "rudy machines -U" to update hostnames
+* ADDED: "rudy machines -S" to create static IP addresses
+* ADDED: Rudy::Routines::Handlers::Disks.mount? etc... methods
+
+
+#### 0.9.0-RC2 (2009-08-02) ###########################
+
+* FIXED: Some status lines were printing on the same line during routines
+* FIXED: user value in defaults wasn't being used for remote commands
+* FIXED: Double printing SSH commands in parallel mode
+* FIXED: Startup routine was creating only one machine
+* FIXED: Reboot now executes commands in correct order
+* CHANGE: Renamed config parameter 'yes' to 'auto'
+* CHANGE: Global config parameter processed differently (see Rudy::Config#look_and_load)
+* ADDED: "rudy ssh" can now run commands in parallel
+* ADDED: rudy networks command
+
+
+#### 0.9.0-RC1 (2009-08-01) ###########################
+
+NOTE: This is a significant re-write from 0.8 (primarily the routines, metadata)
+
+* FIXED: Routine blocks now process directives in the order their given (in Ruby 1.8 and 1.9). 
+* FIXED: Region is now based on the zone. Corrects ambiguity when specifying zone and region.
+* FIXED: Reboot routines are now fully supported
+* FIXED: Unless position (-p) is specified, it's assumed that a command applies to the entire machine group. 
+* CHANGE: Rudy::Machine#inspect now prints in a standard Ruby inspect format
+* CHANGE: Routines have been completely rebuilt
+* CHANGE: Now requires Rye 0.8
+* CHANGE: before / after routine actions are now used for dependencies only.
+* CHANGE: Removed script_local, after_local, before_local from passthrough routines
+* CHANGE: Support before_local for startup routines
+* CHANGE: Support before_local and before_remote for reboot routines
+* CHANGE: Support after_local for shutdown routines
+* CHANGE: Removed machines config (now recommend uploading local files)
+* CHANGE: Group names now take the form: grp-us-east-1b-stage-app
+* CHANGE: private keys are now stored in ~/.ssh/
+* CHANGE: Removed create_image and prepare_image actions from rudy-ec2
+* CHANGE: local routine blocks now share a single Rye::Box instance
+* CHANGE: remote routine blocks now share a single Rye::Set instance
+* CHANGE: "privatekey" in accounts config is renamed to "pkey"
+* CHANGE: Reduced Rudy::AWS#execute_request timeout to 15 seconds.
+* CHANGE: Rudy:Disks, Rudy::Machines, and Rudy::Backups now use class methods
+* CHANGE: Rudy::AWS::EC2 now uses class methods
+* CHANGE: Renamed disk action 'snapshot' to 'archive'
+* ADDED: routines command for displaying available routines
+* ADDED: "rudy init" now authorizes public keys for user@localhost
+* ADDED: Parallel processing for remote machines
+* ADDED: Routines now have access to $global and $option (which are frozen)
+* ADDED: xremote and xlocal routine actions for quick skipping
+* ADDED: Dependency Attic 0.4
+* ADDED: New 'objects' and 'query' commands for rudy-sdb
+* ADDED: Execute routines on individual machines in a group
+* ADDED: rudy backups command
+
+
+#### 0.8.5 (2009-06-08) ###############################
 
 * FIXED: Bug in DiskHelper. mount method tried to attach when already attached. 
 * CHANGE: rudy config now displays account info (but keeps secret key hidden)
@@ -78,7 +152,7 @@ NOTE: The following changes are planned for the 0.9.0 release.
 * CHANGE: Moved non-rudy specific modules to separate projects (storable, sysinfo, annoy, 
           console -> Drydock, escape -> Rye, tryouts -> tryouts/).
 * CHANGE: Renamed restart routine to reboot to make it clear it's about the machine
-* CHANGE: Renamed Rudy::Disk to Rudy::MetaData::Disk
+* CHANGE: Renamed Rudy::Disk to Rudy::Metadata::Disk
 * CHANGE: Short alias for rudy-ec2 snapshot is now 's' (instead of 't')
 * CHANGE: Removed highline-1.5.1 from vendor (it's now available as a gem)
 * FIXED: Bug in Huxtable when loading global. Wasn't reloaded after config was updated. 
@@ -132,7 +206,7 @@ NOTE: The following changes are planned for the 0.9.0 release.
 #### 0.6.8 (2009-04-24) ###############################
 
 * CHANGE: Renamed rudy config --rudy option to --project. This was important 
-to maintain consistency in documentation.
+          to maintain consistency in documentation.
 * CHANGE: Cleaned README. See: http://wiki.github.com/solutious/rudy
 
 #### 0.6.7 (2009-04-22) ###############################
@@ -240,7 +314,25 @@ NOTE: This is a complete re-write from 0.1
 * ADDED: "rudy groups" overhaul. Display, creates, destroys groups. 
 
 
-
 #### 0.1 (2009-02-06) ###############################
 
 * Initial public release
+
+
+#### TODO ###########################################
+
+* TODO: Supply dns values in machines config
+* TODO: Rudy::Routines::Arcade. 
+  * install/uninstall: uses package management for the specific os impl
+  * compile: from source. 
+  * e.g. rudy -a experimental install git (for ruby-unix-osx)
+  * How to handle multiple package managers?
+  * How to handle multiple app versions? 
+* TODO: http://opensoul.org/2009/6/24/capistrano-git-and-ssh-keys
+* TODO: look for config file in ./.rudy, then ../.rudy, etc... (git-like)
+* TODO: FIX: When shutting down, warns about disks even when they don't exist (CLI::Routines)
+* TODO: Investigate SSH daemon on Windows
+* TODO: Supply machines config to routines blocks with only the specific machine group context 
+
+
+

data/README.rdoc

@@ -1,27 +1,19 @@
-= Rudy - v0.8 BETA
+= Rudy - v0.9.0 BETA
 
 <b>Not your grandparents' EC2 deployment tool.</b>
 
-Rudy is a command-line development and deployment tool for Amazon Web Services. It helps you build and manage machines in EC2 by organizing them into groups of _environments_ and _roles_. These are called _machine_ _groups_. <tt>stage</tt> is the default environment and <tt>app</tt> is the default role. You can run multiple machines with the same role. These are called _positions_. Rudy also supports running machine groups across availability _zones_. 
+Rudy is a development and deployment tool for Amazon Web Services. It helps you build and infrastructures in EC2 by organizing them into groups of _zones_,  _environments_, and _roles_. You can run multiple machines with the same role. When you put all this together, you have a unique name for every machine. The default machine is called <tt>m-us-east-1b-stage-app-01</tt> but you can also give them custom names. 
 
-When you put all this together, you have a unique name for every machine. The default machine is:
+All configuration is organized into the zones, environments, and roles that you specify (Rudy assumes positions are identical which is important for backups and scaling). And as you'd expect, the defaults can be changed too.
 
-             zone     env  role
-              v        v    v   
-        m-us-east-1b-stage-app-01
-        ^                       ^
-     "machine"                position
-
-All configuration is organized into the zones, environments, and roles that you specify (Rudy assumes positions are identical which is important for backups and scaling). And as you'd expect, the defaults can be changed too (in <tt>~/.rudy/config</tt>).
-
-<b>See Getting-Started[http://solutious.com/products/rudy/getting-started.html] for more info.</b>
+<b>See Getting-Started[http://solutious.com/projects/rudy/getting-started/] for more info.</b>
 
 
 == Project Status
 
 <em>This is a BETA release. That means Rudy's not ready for production use! See Project-Status[http://wiki.github.com/solutious/rudy/project-status].</em>
 
-I'm currently rebuilding[http://github.com/solutious/rudy/tree/metadata-refactor] the metadata implementation. This is the part of Rudy that stores your machine, disk, and backup infoz in SimpleDB so it's pretty important! The implementation in the 0.8.4 release (and this master branch) are functional but frail so I want to re-emphasize not to use Rudy for production right now.
+The 0.9 release is a major re-write and has many changes from previous releases. 0.9-RC1 is the first public release from the 0.9 branch and much of the code has not been tested. DO NOT USE IN PRODUCTION. Try it out have some fun and if you have the time send a patch or pull request!
 
 
 == Configuration
@@ -30,13 +22,13 @@ I'm currently rebuilding[http://github.com/solutious/rudy/tree/metadata-refactor
 
 The machines configuration describes the "physical" characteristics of your infrastructure. 
 
-    env :stage do         # Define an environment
+    env :stage do                   # Define an environment
       ami 'ami-e348af8a'         
                              
-      role :app do        # Define a role
-        addresses '11.22.33.44'  # Use elastic IPs
+      role :app do                  # Define a role
+        addresses '11.22.33.44'     # Use elastic IPs
                              
-        disks do          # Create EBS volumes
+        disks do                    # Define EBS volumes
           path "/rudy/disk1" do  
             size 100             
             device "/dev/sdr"     
@@ -49,18 +41,18 @@ The machines configuration describes the "physical" characteristics of your infr
 
 The routines configuration describes repeatable processes that you can execute on your machines. 
 
-    startup do            # $ rudy startup
+    startup do                      # $ rudy startup
       adduser :rudy       
-      authorize :rudy     # Enable passwordless login
+      authorize :rudy               # Enable passwordless login
                              
-      disks do                   # Create a disk 
-        restore "/rudy/disk1"    # from a backup
+      disks do                   
+        create "/rudy/disk1"        # Create a disk 
       end                       
                             
-      after :rudy do      # Run remote SSH commands
-        mkdir :p, "great"        # $ mkdir -p great
-        touch "great/scott"      # $ touch great/scott
-        ls :l, :a                # $ ls -l -a
+      remote :rudy do               # Run remote commands via SSH
+        mkdir :p, "great"           # $ mkdir -p great
+        touch "great/scott"         # $ touch great/scott
+        mysql_init :start 
       end                          
     end
 
@@ -68,32 +60,35 @@ The routines configuration describes repeatable processes that you can execute o
 
 == Features
 
-* Create complex environments with a single command
+* Create complex IT infrastructures from a simple configuration
   * Launch multiple instances 
   * Assign elastic IP addresses
   * Create EBS volumes, attach to instances, format, and mount
-  * Run scripts locally and remotely automatically
+* Run SSH commands on machine groups in parallel
 * Organize instances into environments and roles
 * Powerful command-line tools
   * <tt>$ rudy -u root ssh</tt>
-  * <tt>$ rudy -e testing -r database startup</tt>
+  * <tt>$ rudy -e testing -r database backup-mysql</tt>
 * Use _any Linux_ Amazon machine image (AMI) (partial Solaris support)
 * Complete command-line interface for EC2. See <tt>bin/rudy-ec2</tt>.
 
 == Installation
 
-Via Rubygems, one of:
+Via Rubygems:
 
     $ sudo gem install rudy
-    $ sudo gem install solutious-rudy --source http://gems.github.com/
 
 or via download:
 * rudy-latest.tar.gz[http://github.com/solutious/rudy/tarball/latest]
 * rudy-latest.zip[http://github.com/solutious/rudy/zipball/latest]
 
+or via git:
+
+    $ git clone git://github.com/solutious/rudy.git
+    
 NOTE: <em>If you are not installing via RubyGems, you need to make sure the dependencies are in your LOAD_PATH (<tt>$:</tt>). Ryan Tomayko wrote a gist[http://gist.github.com/54177] about it.</em>
 
-<b>See Getting-Started[http://solutious.com/products/rudy/getting-started.html] for more info.</b>
+<b>See Getting-Started[http://solutious.com/projects/rudy/getting-started/] for more info.</b>
 
 
 == Pre-Requisites
@@ -102,8 +97,9 @@ NOTE: <em>If you are not installing via RubyGems, you need to make sure the depe
   * EC2[http://aws.amazon.com/ec2/]
   * SimpleDB[http://aws.amazon.com/simpledb/]
   * S3[http://aws.amazon.com/s3/]
+* OpenSSL[http://www.openssl.org/]
 * OpenSSH[http://www.openssh.com/]
-* Ruby[http://ruby-lang.org] 1.8.x, 1.9.x, or JRuby[http://jruby.codehaus.org] 1.2
+* Ruby[http://ruby-lang.org] 1.8.x, 1.9.x, or JRuby[http://jruby.codehaus.org] 1.3
   * Net::SSH[http://net-ssh.rubyforge.org/]
   * Net::SCP[http://net-ssh.rubyforge.org/]
   * amazon-ec2[http://github.com/grempe/amazon-ec2]
@@ -111,20 +107,26 @@ NOTE: <em>If you are not installing via RubyGems, you need to make sure the depe
   * Rye[http://github.com/delano/rye]
   * Drydock[http://github.com/delano/drydock]
   * Caesars[http://github.com/delano/caesars]
+  * Gibbler[http://github.com/delano/gibbler]
 
-
+* Supported Platforms
+  * Linux: Ruby 1.8, 1.9, and JRuby 1.3+
+  * Windows: Ruby 1.8 and JRuby 1.3+ (Net::SSH does not run under Ruby 1.9)
+  * BSD: <i>Needs testing (any takers?)</i>
+  
 == More Info
 
 * Fork at GitHub[http://github.com/solutious/rudy]
-* Read the Documentation[http://wiki.github.com/solutious/rudy]
+* Read the Documentation[http://solutious.com/projects/rudy]
 * Submit issues to the IssueTracker[http://github.com/solutious/rudy/issues]
 * Start a discussion on the GoogleGroup[http://groups.google.com/group/rudy-deployment]
 * Find some Inspiration[http://www.youtube.com/watch?v=CgaiIW5Rzes]
-* For all other inquires, email me directly: delano@solutious.com
+* For all other inquires, email me directly: delano (@solutious.com)
 
 
 == Thanks
 
+* Kalin Harvey for all the feedback
 * The Rilli.com[http://rilli.com] team
   * Adam Bognar
   * Andrew Simpson
@@ -142,10 +144,8 @@ NOTE: <em>If you are not installing via RubyGems, you need to make sure the depe
 
 == Contributions
 
-I'm very open to contributions! 
-
 * For bigger features please fork the git repo and send me a pull request. 
-* For small or single file changes, send me an email with the details: <tt>delano@solutious.com</tt>
+* For small or single file changes, send me an email with the details: Delano (@solutious.com)
 
 
 == Credits
@@ -156,16 +156,12 @@ I'm very open to contributions!
 
 == Related Projects
 
-* Sprinkle -- http://github.com/crafterm/sprinkle/
-* Wakame -- http://wakame.rubyforge.org/
-* Pool Party -- http://www.poolpartyrb.com/
-* Rubber -- http://github.com/wr0ngway/rubber/wikis
-* Moonshine -- http://github.com/railsmachine/moonshine/tree
+* Rake -- http://rake.rubyforge.org/
+* Moonshine -- http://github.com/railsmachine/moonshine/tree/
 * Boto -- http://code.google.com/p/boto/
 * Fabric -- http://www.nongnu.org/fab/
 
 
-
 == License
 
 See: LICENSE.txt

data/Rudyfile

@@ -25,27 +25,28 @@
 #
 
 # ---------------------------------------------------------  MACHINES  --------
-# The machines block describes the "physical" characteristics of your machines. 
+# The machines block describes the 'physical' characteristics of your machines. 
 machines do
   
-  zone :"us-east-1b" do
+  region :'us-east-1' do
     ami 'ami-e348af8a'               # Alestic Debian 5.0, 32-bit (US)
   end
-  zone :"eu-west-1b" do
+  region :'eu-west-1' do
     ami 'ami-6ecde51a'               # Alestic Debian 5.0, 32-bit (EU)
   end
   
-  hostname :rudy                     # One of: :default, :rudy, "your-name"
+  hostname :rudy                     # One of: :default, :rudy, 'your-name'
   
-  # We've defined an environment called "stage" with one role: "app". 
+  # We've defined an environment called 'stage' with one role: 'app'. 
   # The configuration inside the env block is available to all its 
   # roles. The configuration inside the role blocks is available only
   # to machines in that specific role. 
   env :stage, :prod do
+    user :root                       # User to connect as
     size 'm1.small'                  # EC2 machine type for all machines
-                                     # in the "stage" environment
+                                     # in the 'stage' environment
     role :app do
-      positions 1                    # Only 1 machine in stage-app
+      positions 2                    # 2 machines in stage-app
       #addresses '11.22.33.44'       # Define an elastic IP to reuse
       
       disks do                       # Define EBS volumes 
@@ -62,21 +63,13 @@ machines do
     end
     
     users do                         # Specify existing private keys per user
-      rudy do                       
-        keypair "/path/2/private-key"
-      end
+      #rudy do                       
+      #  keypair '/path/2/private-key'
+      #end
     end
     
   end  
 
-  # The routines section below contains calls to local and remote
-  # scripts. The config contained in this block is made available
-  # those scripts in the form of a yaml file. The file is called
-  # rudy-config.yml. 
-  config do                 
-    dbmaster 'localhost'
-    newparam 573114
-  end
 end
 
 
@@ -94,7 +87,8 @@ end
 commands do
   allow :gem_install, '/usr/bin/gem', 'install', :V, '--no-rdoc', '--no-ri'
   allow :apt_get, 'apt-get', :y, :q
-  allow :floo, :time, 1 do
+  allow :rubycode do
+    puts "Some ruby code running in #{self}"
   end
 end
 
@@ -104,17 +98,18 @@ end
 routines do
   
   env :stage, :prod do               # We'll define routines for the stage-app
-    role :app do                     # and prod-app machine groups
+    role :app do                     # and prod-app machine groups.
+      
+      user :root                     # The default remote user
       
       startup do                     # $ rudy startup
         adduser :rudy                # Create a user called 'rudy'
-        authorize :rudy              # Enable passwordless logins as rudy
                                      #
         disks do                     # Define EBS volume routines
           create '/rudy/disk1'       # Create an EBS volume, attach it, give
         end                          # it a filesystem, and mount it.
                                      #
-        after :rudy do               # Run remote SSH commands after startup
+        remote :root do              # Run remote SSH commands after startup
           mkdir :p, 'great'          # $ mkdir -p great
           touch 'great/scott'        # $ touch great/scott
           ls :l, :a                  # $ ls -l -a *
@@ -122,7 +117,7 @@ routines do
       end                            
                                      
       shutdown do                    # $ rudy shutdown
-        before :root do              # Run remote SSH commands before shutdown
+        remote :root do              # Run remote SSH commands before shutdown
           uptime               
         end                               
         disks do                     
@@ -131,27 +126,17 @@ routines do
       end                            
       
       reboot do                      # $ rudy reboot
-        before :root do              # Run any pre-reboot tasks like stopping
+        before_remote do             # Run any pre-reboot tasks like stopping
           uptime                     # web servers and databases. 
         end                          #
-        after :root do               # Run any startup tasks like starting
-          uptime                     # processes or initializing the filesystem
+        remote do                    # Run any startup tasks like starting
+          uname                      # processes or initializing the filesystem
         end
-      end
-      
-      backup do                      # $ rudy backup
-        disks do                     # A simple routine that creates an EBS
-          snapshot '/rudy/disk1'     # snapshot of the specified volume.
+        disks do
+          mount "/rudy/disk1"
         end
       end
       
-      restore do                     # $ rudy restore
-        disks do                     # A contrived example of restoring a 
-          destroy '/rudy/disk1'      # disk from a backup. NOTE: You'll need
-          restore '/rudy/disk1'      # to run 'rudy backup' at least once 
-        end                          # before otherise there are no backups
-      end                            # to restore from. 
-      
     end                              
   end                                
   
@@ -160,7 +145,7 @@ routines do
   # install essential libraries. 
   # See http://github.com/rudy/arcade
   sysupdate do                       # $ rudy sysupdate
-    before :root do                  
+    remote :root do                  
       apt_get 'update'               # Update debian / ubuntu 
       apt_get 'install', 'build-essential', 'sqlite3', 'libsqlite3-dev'
       apt_get 'install', 'apache2-prefork-dev', 'libapr1-dev'
@@ -170,30 +155,30 @@ routines do
   
   anything do                        # $ rudy anything
     before :uptime                   # Specify a dependency
-    script_local do                  # This is Ruby, so any valid syntax
+    local do                         # This is Ruby, so any valid syntax
       ls :l                          # can be used in the definitions. 
     end                              # See: SysInfo gem for more info.
   end
   
   uptime do                          # $ rudy uptime
-    script_local do
-      uptime
-    end
+    local { uptime }                 # Short block syntax
   end
+  
 end
 
 
 # ----------------------------------------------------------- DEFAULTS --------
-# These values are used as defaults for their respective global settings. All
-# non-boolean values are expected to be Symbols. 
+# These values are used as defaults for their respective global settings. They
+# can be overridden by the command-line global options.  
 #
 defaults do
-  region :"us-east-1" 
-  zone :"us-east-1b"
+  zone :'us-east-1d'
   environment :stage
   role :app
-  user Rudy.sysinfo.user.to_sym      # This will be the user running Rudy
-  color false                        # Terminal colors? true/false
-  yes false                          # Auto-confirm? true/false
+  color true                         # Terminal colors? true/false
+  #user 'someuser'                   # The default remote user
+  #localhost 'hostname'              # A local hostname instead of localhost
+  #auto true                         # Skip interactive confirmation?
+  #keydir 'path/2/keys/'             # The path to store SSH keys
 end