rudy 0.9.1 → 0.9.2

data/CHANGES.txt

@@ -1,6 +1,37 @@
 RUDY, CHANGES 
 
 
+#### 0.9.2 (2009-08-25) ###########################
+
+* FIXED: Threading error in Net::HTTP by forcing a mutex during AWS::EC2 requests
+* FIXED: errors when washing backups
+* FIXED: Host.is_available? now checks port 3389 (Remote Desktop) for Windows instances. Closes GH-16
+* FIXED: "rudy machines -T" now checks port 3389 for Windows instances. Closes GH-21
+* FIXED: "rudy disks -l" doesn't display disks from a different zone. Closes GH-24
+* FIXED: Rudy::Machine#refresh! now correctly refreshes metadata
+* FIXED: startup, shutdown, and reboot routines don't run correctly as dependencies. Closes GH-25
+* FIXED: Rogue "No machines configuration" error when accessing current_machine_address
+* CHANGE: Increased AWS timeout to 30 seconds. Closes GH-19
+* CHANGE: Increased boot wait time for windows instances to 500 seconds.
+* CHANGE: Module names have changed from EC2 to AWS::EC2 in amazon-ec2 0.5 [Dominic Cleal]
+* CHANGE: Rudy::Routines::Handlers::Disks.attach now creates a volume if the disk doesn't already have one. 
+* CHANGE: Standardized CLI outut for rudy-ec2 (except groups b/c of weird YAML error)
+* CHANGE: Windows "os" is now referred to by :windows rather than :win32 (SysInfo 0.7)
+* CHANGE: Renamed "rudy status" to "rudy info"
+* ADDED: Rudy::Machine#get_console, Rudy::Machine#get_password
+* ADDED: Rudy::Machine#windows?, Rudy::Machine#linux?, Rudy::Machine#solaris?, Rudy::Machine#os?
+* ADDED: "rudy m -console" and "rudy m --password"
+* ADDED: "rudy keypairs"
+* ADDED: "rudy networks --authorize"
+* ADDED: "rudy networks --revoke"
+* ADDED: "rudy disks --create"
+* ADDED: "rudy disks --destroy"
+* ADDED: "rudy backups --create"
+* ADDED: Support for attaching and formating disks in Windows. 
+* ADDED: "rudy-ec2 info"
+* ADDED: Re-enabled networks action to routines config (see examples/jira.rb)
+
+
 #### 0.9.1 (2009-08-05) ###########################
 
 * FIXED: "rudy init" now exits with 0
@@ -95,7 +126,7 @@ NOTE: This is a significant re-write from 0.8 (primarily the routines, metadata)
 
 #### 0.8.5 (2009-06-08) ###############################
 
-* FIXED: Bug in DiskHelper. mount method tried to attach when already attached. 
+* FIXED: Bug in DiskHandler. mount method tried to attach when already attached. 
 * CHANGE: rudy config now displays account info (but keeps secret key hidden)
 * CHANGE: defaults configuration has been removed from the default ~/.rudy/config. 
   It is now recommended to put it in the same place as machines, routines config.
@@ -326,20 +357,4 @@ NOTE: This is a complete re-write from 0.1
 * Initial public release
 
 
-#### TODO ###########################################
-
-* TODO: Supply dns values in machines config
-* TODO: Rudy::Routines::Arcade. 
-  * install/uninstall: uses package management for the specific os impl
-  * compile: from source. 
-  * e.g. rudy -a experimental install git (for ruby-unix-osx)
-  * How to handle multiple package managers?
-  * How to handle multiple app versions? 
-* TODO: http://opensoul.org/2009/6/24/capistrano-git-and-ssh-keys
-* TODO: look for config file in ./.rudy, then ../.rudy, etc... (git-like)
-* TODO: FIX: When shutting down, warns about disks even when they don't exist (CLI::Routines)
-* TODO: Investigate SSH daemon on Windows
-* TODO: Supply machines config to routines blocks with only the specific machine group context 
-
-
 

data/README.rdoc

@@ -1,63 +1,89 @@
-= Rudy - v0.9.0 BETA
+= Rudy - v0.9 BETA
 
 <b>Not your grandparents' EC2 deployment tool.</b>
 
-Rudy is a development and deployment tool for Amazon Web Services. It helps you build and infrastructures in EC2 by organizing them into groups of _zones_,  _environments_, and _roles_. You can run multiple machines with the same role. When you put all this together, you have a unique name for every machine. The default machine is called <tt>m-us-east-1b-stage-app-01</tt> but you can also give them custom names. 
+Rudy is a development and deployment tool for EC2. It helps you build and maintain infrastructures by organizing them into groups of _zones_,  _environments_, and _roles_. By making it quick and easy to build infrastructures, Rudy makes it feasible to run environments only for the time that you need them. 
 
-All configuration is organized into the zones, environments, and roles that you specify (Rudy assumes positions are identical which is important for backups and scaling). And as you'd expect, the defaults can be changed too.
-
-<b>See Getting-Started[http://solutious.com/projects/rudy/getting-started/] for more info.</b>
+All configuration is organized into this hierarchy so you can define properties (machine image, machine type, IP address, etc...) by role, by environment, or by zone. You can also define routines for running shell commands and scripts, uploading files, creating disks and backups, etc... Routines are organized by hierarchy too so a "startup" routine can run one thing for your database machines and another thing for your application machines. 
 
+Rudy helps with other stuff too. Every machine can be given a hostname based on its zone, environment, role, and position (you can run multiple machines with the same role). These names look like this: <tt>m-us-east-1b-stage-app-01</tt>, but you can also assign your own names. Rudy also automatically creates a private keypair and security for each machine group (e.g. <tt>key-us-east-1b-stage-app</tt> and <tt>grp-us-east-1b-stage-app</tt>).
 
-== Project Status
+<b>Get-Started[http://solutious.com/projects/rudy/getting-started/] with Rudy now.</b>
 
-<em>This is a BETA release. That means Rudy's not ready for production use! See Project-Status[http://wiki.github.com/solutious/rudy/project-status].</em>
 
-The 0.9 release is a major re-write and has many changes from previous releases. 0.9-RC1 is the first public release from the 0.9 branch and much of the code has not been tested. DO NOT USE IN PRODUCTION. Try it out have some fun and if you have the time send a patch or pull request!
+== Configuration
 
+Rudy is configured via several Ruby-based domain specific languages.
 
-== Configuration
 
 === Machines
 
 The machines configuration describes the "physical" characteristics of your infrastructure. 
 
-    env :stage do                   # Define an environment
-      ami 'ami-e348af8a'         
+    machines do
+      env :stage do                   # Define an environment
+        ami 'ami-e348af8a'            # Specify a machine image
+      
+        role :app do                  # Define a role
+          addresses '11.22.33.44'     # Use elastic IPs
                              
-      role :app do                  # Define a role
-        addresses '11.22.33.44'     # Use elastic IPs
-                             
-        disks do                    # Define EBS volumes
-          path "/rudy/disk1" do  
-            size 100             
-            device "/dev/sdr"     
-          end                       
+          disks do                    # Define EBS volumes
+            path "/rudy/disk1" do  
+              size 100
+              device "/dev/sdr"     
+            end                       
+          end
         end
+        
       end
     end
-
+    
 === Routines
 
 The routines configuration describes repeatable processes that you can execute on your machines. 
 
-    startup do                      # $ rudy startup
-      adduser :rudy       
-      authorize :rudy               # Enable passwordless login
+    routines do
+    
+      startup do                      # $ rudy startup
+        adduser :rudy       
+        authorize :rudy               # Enable passwordless login
                              
-      disks do                   
-        create "/rudy/disk1"        # Create a disk 
-      end                       
+        disks do                   
+          create "/rudy/disk1"        # Create, format, and mount a volume 
+        end                       
                             
-      remote :rudy do               # Run remote commands via SSH
-        mkdir :p, "great"           # $ mkdir -p great
-        touch "great/scott"         # $ touch great/scott
-        mysql_init :start 
-      end                          
+        remote :rudy do               # Run remote commands via SSH
+          mkdir :p, "great"           # $ mkdir -p great
+          mysql_init :start 
+          your_script 'arg1', 'arg2'  # Call your own scripts
+        end                          
+      end
+    
     end
+    
+See Rudyfile[http://github.com/solutious/rudy/raw/master/Rudyfile] for a complete configuration example. 
 
 
+== Running Commands
 
+Rudy comes with a command-line tool called <tt>rudy</tt>. It provides commands for describing the infrastructures you've built with Rudy. 
+
+    $ rudy machines
+    m-us-east-1d-stage-app-01: ec2-67-202-0-112.compute-1.amazonaws.com
+    m-us-east-1d-stage-app-02: ec2-174-129-117-30.compute-1.amazonaws.com
+    
+    $ rudy disks
+    disk-us-east-1d-stage-app-01-rudy-disk1
+    disk-us-east-1d-stage-app-02-rudy-disk1
+    
+    $ rudy backups
+    back-us-east-1d-stage-app-01-rudy-disk1-20090803-1857-49
+    back-us-east-1d-stage-app-01-rudy-disk1-20090803-1858-36
+    back-us-east-1d-stage-app-02-rudy-disk1-20090803-1911-05
+    
+See <tt>rudy -h</tt> for more info.     
+    
+    
 == Features
 
 * Create complex IT infrastructures from a simple configuration
@@ -69,9 +95,15 @@ The routines configuration describes repeatable processes that you can execute o
 * Powerful command-line tools
   * <tt>$ rudy -u root ssh</tt>
   * <tt>$ rudy -e testing -r database backup-mysql</tt>
-* Use _any Linux_ Amazon machine image (AMI) (partial Solaris support)
+* Use _any Linux_ Amazon machine image (AMI) (partial Windows and Solaris support)
 * Complete command-line interface for EC2. See <tt>bin/rudy-ec2</tt>.
 
+
+== Project Status
+
+<em>This is a BETA release. That means Rudy is not ready for production use! See Project-Status[http://wiki.github.com/solutious/rudy/project-status].</em>
+
+
 == Installation
 
 Via Rubygems:
@@ -113,6 +145,7 @@ NOTE: <em>If you are not installing via RubyGems, you need to make sure the depe
   * Linux: Ruby 1.8, 1.9, and JRuby 1.3+
   * Windows: Ruby 1.8 and JRuby 1.3+ (Net::SSH does not run under Ruby 1.9)
   * BSD: <i>Needs testing (any takers?)</i>
+ 
   
 == More Info
 
@@ -124,6 +157,14 @@ NOTE: <em>If you are not installing via RubyGems, you need to make sure the depe
 * For all other inquires, email me directly: delano (@solutious.com)
 
 
+== Contributions
+
+We encourage people to find unique and interesting ways to break and extend rudy.
+
+* For bigger features please fork the git repo and send me a pull request. 
+* For small or single file changes, send an email with details to: delano (@solutious.com)
+
+
 == Thanks
 
 * Kalin Harvey for all the feedback
@@ -141,16 +182,9 @@ NOTE: <em>If you are not installing via RubyGems, you need to make sure the depe
 * Marcel Molina Jr. for aws-s3
 * Keshia Knight Pulliam
 
-
-== Contributions
-
-* For bigger features please fork the git repo and send me a pull request. 
-* For small or single file changes, send me an email with the details: Delano (@solutious.com)
-
-
 == Credits
 
-* Delano Mandelbaum (delano@solutious.com)
+* Delano Mandelbaum ( http://solutious.com/ )
 * Rudy::AWS::SDB adapted from aws_sdb by Tim Dysinger (http://dysinger.net)
 
 

data/bin/rudy

@@ -20,6 +20,8 @@ require 'rudy/cli'
 # Command-line interface for /bin/rudy
 class RudyCLI < Rudy::CLI::Base
   
+  debug :off
+  
   default :machines       # when no command is provided
   trawler :passthrough    # unknown command names will forward here.
   
@@ -43,32 +45,53 @@ class RudyCLI < Rudy::CLI::Base
   usage "rudy machines"
   usage "rudy machines -l"
   usage "rudy machines -U"
-  usage "rudy machines -A"
-  usage "rudy machines -S [static ip address]"
-  usage "rudy machines -D"
-  option :l, :all, "Display machines for all environments and roles"
+  usage "rudy machines -A [static ip address]"
+  usage "rudy machines -N"
+  usage "rudy machines -T"
+  usage "rudy machines -P"
+  usage "rudy machines -O"
   action :W, :wash, "Wash machine metadata"
   action :T, :available, "Test availablity"
   action :U, :update, "Update machines based on configuration"
   action :A, :associate, "Associate static IP addresses or display existing ones"
   action :N, :disassociate, "Disassociate static IP addresses"
+  action :P, :password, "Display admin password (Windows only)"
+  action :O, :console, "Display console output"
+  option :l, :all, "Display machines for all environments and roles"
   command :machines => Rudy::CLI::Machines
   command_alias :machines, :m
   
+  about "View Keypairs"
+  usage "rudy keypairs"
+  #usage "rudy keypairs -A"
+  usage "rudy keypairs -S"
+  action :A, :add, "Add a keypair"
+  action :S, :show, "Show a private key"
+  command :keypairs => Rudy::CLI::Keypairs
+  command_alias :keypairs, :k
+  
   about "View Disks"
   usage "rudy disks"
   usage "rudy disks -l"
-  action :W, :wash, "Wash disk metadata"
+  usage "rudy disks -C -s 1 /path/2/mount"
+  option :s, :size, Integer, "Volume size (GB)"
+  option :d, :device, String, "Device ID"
+  option :f, :fstype, String, "Filesystem type"
   option :b, :backups, "Display backups"
   option :l, :all, "Display all disks"
+  action :C, :create, "Create disk"
+  action :D, :destroy, "Destroy disk"
+  action :W, :wash, "Wash disk metadata"
   command :disks => Rudy::CLI::Disks
   command_alias :disks, :d
 
   about "View Backups"
   usage "rudy backups"
   usage "rudy backups -l"
-  action :W, :wash, "Wash backup metadata"
+  usage "rudy backups -C /disk/mount/point"
   option :l, :all, "Display all backups"
+  action :W, :wash, "Wash backup metadata"
+  action :C, :create, "Create backup"
   command :backups => Rudy::CLI::Backups
   command_alias :backups, :b
 
@@ -82,16 +105,27 @@ class RudyCLI < Rudy::CLI::Base
   argv :oid
   command :metadata => Rudy::CLI::Metadata
   
-  about "View Network configuration (local network only)"
+  about "View Network configuration"
   usage "rudy networks"
   usage "rudy networks -L"
   usage "rudy networks -L -i"
   usage "rudy networks -L -e"
   usage "rudy networks -U"
+  usage "rudy networks -A [-p ports] [-a addresses] [-r protocols] "
+  usage "rudy networks -A -p 81,8000-9000 -a 127.0.0.1,127.0.0.2"
+  usage "rudy networks -A -g default -o 123456789012"
+  usage "rudy networks -R -g default"
   option :e, :external, "Display only external IP address"
   option :i, :internal, "Display only internal IP address"
+  option :r, :protocols, Array, "List of protocols. One of: tcp (default), udp, icmp"
+  option :p, :ports, Array, "List of port ranges (default: 22,80,443)"
+  option :a, :addresses, Array, "List of IP addresses (default: your current external IP)"
+  option :g, :group, String, "Other group to authorize or revoke. Use with -o!"
+  option :o, :owner, String, "Other group owner ID (account number). Use with -g!"
   action :U, :update, "Update networks based on configuration"
   action :L, :local, "Show local network configuration"
+  action :A, :authorize, "Authorize a rule for a network security group"
+  action :R, :revoke, "Revoke a rule for a network security group"
   command :networks => Rudy::CLI::Networks
   command_alias :networks, :n
   
@@ -134,7 +168,7 @@ class RudyCLI < Rudy::CLI::Base
   
   about "Display existing environment objects"  
   option :l, :all, "Include all regions"
-  command :status => Rudy::CLI::Status
+  command :info => Rudy::CLI::Info
   
   about "Log in to a machine"
   command :ssh => Rudy::CLI::Machines
@@ -144,8 +178,8 @@ class RudyCLI < Rudy::CLI::Base
   #option :p, :port, Integer, "Port"
   #command :open => Rudy::CLI::Candy
   
-  usage "rudy [-f config-file] config [param-name]"
   about "Check Rudy configuration."
+  usage "rudy [-f config-file] config [param-name]"
   option :l, :all, "Display all configs for all machines"
   option :commands, "Display commands configuration"
   option :defaults, "Display defaults configuration"
@@ -163,7 +197,29 @@ class RudyCLI < Rudy::CLI::Base
   command :print_global => Rudy::CLI::Config
   command_alias :print_global, :globals
   command_alias :print_global, :global
-   
+  
+  about "Create a machine image from a running instance (Windows only)"
+  usage "rudy -b BUCKET-NAME bundle IMAGE-NAME"
+  argv :name
+  command :bundle => Rudy::CLI::Images
+  
+  about "Check status of bundling process (Windows only)"
+  usage "rudy -b BUCKET-NAME bundle-status IMAGE-NAME"
+  argv :name
+  command :bundle_status => Rudy::CLI::Images
+  
+  about "Display machine images"
+  usage "rudy images"
+  usage "rudy images -o amazon"
+  usage "rudy -b BUCKET-NAME images -R IMAGE-NAME"
+  usage "rudy images -D AMI"
+  option :o, :owner, String, "Owner ID (default: self)"
+  action :R, :register, "Register a machine image"
+  action :D, :deregister, "De-register a machine image (does not delete from S3)"
+  argv :name
+  command :images => Rudy::CLI::Images
+  command_alias :images, :im
+  
   about "Initialize Rudy configuration"
   command :init do |obj|
     

data/bin/rudy-ec2

@@ -201,7 +201,9 @@ module RudyCLI_EC2
   command_alias :zones, :zone
   command_alias :zones, :z
   
-
+  about "Display existing environment objects"  
+  option :l, :all, "Include all regions"
+  command :info => Rudy::CLI::AWS::EC2::Info
   
 end
 

data/examples/authorize.rb

@@ -0,0 +1,15 @@
+
+machines do
+  env :stage do                
+    user :jira                 
+    ami 'ami-e348af8a'               # Alestic Debian 5.0, 32-bit (US)
+  end  
+end
+
+routines do  
+  setup do
+    # NOTE: This fails b/c it's trying to login as jira rather than root. 
+    adduser :jira
+    authorize :jira
+  end
+end

data/examples/gem-test.rb

@@ -10,9 +10,11 @@ defaults do
 end
 
 machines do
+  region :'us-east-1' do
+    ami 'ami-e348af8a'               # Alestic Debian 5.0, 32-bit (US)
+  end
   env :test do
     role :rudy do
-      ami 'ami-e348af8a'               # Alestic Debian 5.0, 32-bit (US)
       user :root
     end
   end
@@ -22,15 +24,14 @@ commands do
   allow :apt_get, "apt-get", :y, :q
   allow :gem_install, "/usr/bin/gem", "install", :n, '/usr/bin', :y, :V, "--no-rdoc", "--no-ri"
   allow :gem_sources, "/usr/bin/gem", "sources"
-  allow :gem_uninstall, "/usr/bin/gem", "uninstall", :V, :y
+  allow :gem_uninstall, "/usr/bin/gem", "uninstall", :V
   allow :update_rubygems
   allow :rake
   allow :rm
 end
 
 routines do
-
-    
+  
   install_rubyforge do
     remote :root do
       gem_install 'rudy', :V
@@ -67,13 +68,14 @@ routines do
   end
   
   init_rudy do
+    before :install_gem
     remote do
       disable_safe_mode
       rudy :v, :v, 'init'  # create home directory
       file_upload File.expand_path('~/.rudy/config'), '.rudy/'
       ls :l, '.rudy/config'
       rudy :v, :v, 'init'
-      rudy 'status', :l
+      rudy 'info', :l
     end
   end
   
@@ -88,5 +90,9 @@ routines do
       update_rubygems
     end
   end
+  
+  startup do
+    after :sysupdate
+  end
 end