fog 1.2.0 → 1.3.0

data/Rakefile

@@ -233,6 +233,7 @@ task :changelog do
         'Dylan Egan',
         'geemus',
         'Henry Addison',
+        'Kevin Menard',
         'Lincoln Stoll',
         'Luqman Amjad',
         'Michael Zeng',

data/changelog.txt

@@ -1,3 +1,94 @@
+1.3.0 03/21/2012 f78afe98242a60ae4dbbfcd8c5ab67ba71c6d773
+=========================================================
+
+Stats! { 'collaborators' => 32, 'downloads' => 513974, 'forks' => 387, 'open_issues' => 24, 'watchers' => 1893 }
+
+MVP! Decklin Foster
+
+[aws|cloud_watch]
+  GitHub Edit! s/prodide/provide/ :v:. thanks Dylan Egan
+
+[aws|simpledb]
+  fix region/host for us-east-1. thanks geemus
+
+[ibm]
+  Added Mocking and Tests. thanks Carl Hicks
+  Moar convenience methods for servers. thanks Carl Hicks
+  Update model for Volume. thanks Carl Hicks
+  Added request tests for addresses. thanks Carl Hicks
+  Initial IBM SmartCloud support. thanks Decklin Foster
+  Add Location model and requests. thanks Decklin Foster
+  Rename collection requests get->list so we have get_foo and list_foos. thanks Decklin Foster
+  Don't need json_body, so we can simplify requests; remove unused params. thanks Decklin Foster
+  Pass optional params to requests with hashes. thanks Decklin Foster
+  Add a InstanceType model, returned as part of Images. thanks Decklin Foster
+  Rename model test files into compute, storage dirs. thanks Decklin Foster
+  Remove trailing whitespace. thanks Decklin Foster
+  Update tests to match model arguments (attribs hash) and lower-level request methods. thanks Decklin Foster
+  Update mocks to reflect moving volume models from from compute to storage. thanks Decklin Foster
+  Add some missing mocks, fix list_instances. thanks Decklin Foster
+  Raise NotFound in mock instead of returning a 404. thanks Decklin Foster
+  Fix form_encode to stringify args (e.g. booleans) to URI.escape. thanks Decklin Foster
+  Typos in state names and status vs. state. thanks Decklin Foster
+  Fix key model for passing public_key, add setting/testing whether key is default. thanks Decklin Foster
+  Setting default key returns success, not key name. thanks Decklin Foster
+  Typo, assignment instead of equality, made some tests incorrectly pass. thanks Decklin Foster
+  Flesh out modify_instance and modify_key for different modes. thanks Decklin Foster
+  Restarting should return success, changing expiration should only return time. thanks Decklin Foster
+  Delete instance mock should return success. thanks Decklin Foster
+  Remove IBM from flavors tests. thanks Decklin Foster
+  Make names used in tests unique using current time. thanks Decklin Foster
+  Add vlan_id attrib so that it can be used in creating an instance. thanks Decklin Foster
+  Return nil for public_hostname if primary_ip unset (e.g. state is Failed). thanks Decklin Foster
+  Change default location and image ID. thanks Decklin Foster
+  Rename ibm_user_id -> ibm_username. thanks Decklin Foster
+  Create temporary keys in tests that need to create instances. thanks Decklin Foster
+  servers are not sorted, don't assume we can just take the last. thanks Decklin Foster
+  create_image should do a PUT, not a POST, and needs 'state' param. thanks Decklin Foster
+  Correct spelling of test volume format ('RAW') and fix parameter typo. thanks Decklin Foster
+  Rename data -> body. thanks Decklin Foster
+  Don't reboot or immediately expire, interferes with other tests. thanks Decklin Foster
+  Use Raleigh location for tests. thanks Decklin Foster
+  Nullable formats for attributes that may not be returned. thanks Decklin Foster
+  Set expire a few seconds in the future since it takes a while to process. thanks Decklin Foster
+  Return nil if instance_id is nil. thanks Decklin Foster
+  Can't access Fog::Compute::IBM::Location class from here, just check if ID returned. thanks Decklin Foster
+  servers.length will include already existing servers. thanks Decklin Foster
+  Add state and ready? method to Image. thanks Decklin Foster
+  Change public key format. thanks Decklin Foster
+  vlan is part of primaryIP. thanks Decklin Foster
+  Rename root_only -> is_mini_ephemeral. thanks Decklin Foster
+  Description is mandatory. thanks Decklin Foster
+  Wait for instance to be ready before deleting it or creating image. thanks Decklin Foster
+  Add state/ready? methods for Address. thanks Decklin Foster
+  Expiration time should be epoch in ms. thanks Decklin Foster
+  launched_at convenience method. thanks Decklin Foster
+  Don't set server to nil. thanks Decklin Foster
+  Set a longer timeout on all wait_for calls. thanks Decklin Foster
+  Generated key needs different name, supplied key only returns success. thanks Decklin Foster
+  Invalid instance creation will return a 412 and thus raise PreconditionFailed. thanks Decklin Foster
+  Don't calculate expiration time until ready to make the request. thanks Decklin Foster
+  Fix volume formats and mocks. thanks Decklin Foster
+  Wait for volume to be ready before deleting. thanks Decklin Foster
+  Mark volume attach/detach pending, won't create instance. thanks Decklin Foster
+  Return pending if provisioning times out in the real tests. thanks Decklin Foster
+  adding documentation. thanks Wyatt Walter
+  add bin helpers for storage. thanks geemus
+
+[misc]
+  ovirt added support for 'set vm ticket'.     This api call is needed for openning a console to the server. thanks Amos Benari
+  wrangled security tests into working. thanks Eric Stonfer
+  Changes to the security group handling:     * CreateSecurityGroup now includes the group id in the reply, this       patch makes the code store this     * The patch also changes the delete call to use the group id if       present (since you must use the id when deleting VPC groups). thanks MaF
+  Changes to the security group handling:     * CreateSecurityGroup now includes the group id in the reply, this       patch makes the code store this     * The patch also changes the delete call to use the group id if       present (since you must use the id when deleting VPC groups)     * Fix teh security group mock and test code to handle this new behavior. thanks MaF
+  Removed merge commit. thanks MaF
+
+[release]
+  add Kevin Menard to future MVP exclusion list. thanks geemus
+
+[storage|aws]
+  fix hardcoded host in get_object_http(s)_url methods. thanks geemus
+
+
 1.2.0 03/19/2012 70e0f48fa446dbf233ae31c4f055eb26ea2dadd1
 =========================================================
 

data/fog.gemspec

@@ -6,8 +6,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'fog'
-  s.version           = '1.2.0'
-  s.date              = '2012-03-19'
+  s.version           = '1.3.0'
+  s.date              = '2012-03-21'
   s.rubyforge_project = 'fog'
 
   ## Make sure your summary is short. The description may be as long
@@ -54,7 +54,7 @@ Gem::Specification.new do |s|
   s.add_development_dependency('rdoc')
   s.add_development_dependency('thor')
   s.add_development_dependency('rspec', '~>1.3.1')
-  s.add_development_dependency('rbovirt', '>=0.0.8')
+  s.add_development_dependency('rbovirt', '>=0.0.9')
   s.add_development_dependency('shindo', '~>0.3.4')
   s.add_development_dependency('virtualbox', '~>0.9.1')
   s.add_development_dependency('fission')

data/lib/fog.rb

@@ -3,7 +3,7 @@ require File.join(File.dirname(__FILE__), 'fog', 'core')
 module Fog
 
   unless const_defined?(:VERSION)
-    VERSION = '1.2.0'
+    VERSION = '1.3.0'
   end
 
 end

data/lib/fog/aws/compute.rb

@@ -117,6 +117,7 @@ module Fog
           @data ||= Hash.new do |hash, region|
             hash[region] = Hash.new do |region_hash, key|
               owner_id = Fog::AWS::Mock.owner_id
+              security_group_id = Fog::AWS::Mock.security_group_id
               region_hash[key] = {
                 :deleted_at => {},
                 :addresses  => {},
@@ -135,25 +136,25 @@ module Fog
                   'default' => {
                     'groupDescription'    => 'default group',
                     'groupName'           => 'default',
-                    'groupId'             => 'sg-11223344',
+                    'groupId'             => security_group_id,
                     'ipPermissionsEgress' => [],
                     'ipPermissions'       => [
                       {
-                        'groups'      => [{'groupName' => 'default', 'userId' => owner_id}],
+                        'groups'      => [{'groupName' => 'default', 'userId' => owner_id, 'groupId' => security_group_id }],
                         'fromPort'    => -1,
                         'toPort'      => -1,
                         'ipProtocol'  => 'icmp',
                         'ipRanges'    => []
                       },
                       {
-                        'groups'      => [{'groupName' => 'default', 'userId' => owner_id}],
+                        'groups'      => [{'groupName' => 'default', 'userId' => owner_id, 'groupId' => security_group_id}],
                         'fromPort'    => 0,
                         'toPort'      => 65535,
                         'ipProtocol'  => 'tcp',
                         'ipRanges'    => []
                       },
                       {
-                        'groups'      => [{'groupName' => 'default', 'userId' => owner_id}],
+                        'groups'      => [{'groupName' => 'default', 'userId' => owner_id, 'groupId' => security_group_id}],
                         'fromPort'    => 0,
                         'toPort'      => 65535,
                         'ipProtocol'  => 'udp',

data/lib/fog/aws/models/compute/security_group.rb

@@ -101,7 +101,11 @@ module Fog
         def destroy
           requires :name
 
-          connection.delete_security_group(name)
+          if group_id.nil?
+            connection.delete_security_group(name)
+          else
+            connection.delete_security_group(nil, group_id)
+          end
           true
         end
 
@@ -195,6 +199,8 @@ module Fog
         def save
           requires :description, :name
           data = connection.create_security_group(name, description, vpc_id).body
+          new_attributes = data.reject {|key,value| key == 'requestId'}
+          merge_attributes(new_attributes)
           true
         end
 

data/lib/fog/aws/models/compute/subnet.rb

@@ -44,7 +44,6 @@ module Fog
         def save
           requires :vpc_id, :cidr_block
           data = connection.create_subnet(vpc_id, cidr_block).body['subnetSet'].first
-		puts data
           new_attributes = data.reject {|key,value| key == 'requestId'}
           merge_attributes(new_attributes)
           true

data/lib/fog/aws/parsers/compute/create_security_group.rb

@@ -0,0 +1,24 @@
+module Fog
+  module Parsers
+    module Compute
+      module AWS
+
+        class CreateSecurityGroup < Fog::Parsers::Base
+
+          def end_element(name)
+            case name
+            when 'return'
+              if value == 'true'
+                @response[name] = true
+              else
+                @response[name] = false
+              end
+            when 'requestId', 'groupId'
+              @response[name] = value
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/fog/aws/requests/cloud_watch/get_metric_statistics.rb

@@ -26,7 +26,7 @@ module Fog
         #
         def get_metric_statistics(options={})
           %w{Statistics StartTime EndTime Period MetricName Namespace}.each do |required_parameter|
-            raise ArgumentError, "Must prodide #{required_parameter}" unless options.has_key?(required_parameter)
+            raise ArgumentError, "Must provide #{required_parameter}" unless options.has_key?(required_parameter)
           end
           statistics = options.delete 'Statistics'
           options.merge!(AWS.indexed_param('Statistics.member.%d', [*statistics]))

data/lib/fog/aws/requests/compute/authorize_security_group_ingress.rb

@@ -138,7 +138,7 @@ module Fog
           if !is_vpc && options['IpProtocol'] && !['tcp', 'udp', 'icmp'].include?(options['IpProtocol'])
             raise Fog::Compute::AWS::Error.new("InvalidPermission.Malformed => Unsupported IP protocol \"#{options['IpProtocol']}\"  - supported: [tcp, udp, icmp]")
           end
-          if options['IpProtocol'] && (!options['FromPort'] || !options['ToPort'])
+          if !is_vpc && (options['IpProtocol'] && (!options['FromPort'] || !options['ToPort']))
             raise Fog::Compute::AWS::Error.new("InvalidPermission.Malformed => TCP/UDP port (-1) out of range")
           end
           if options.has_key?('IpPermissions')
@@ -151,14 +151,14 @@ module Fog
 
         def normalize_permissions(options)
           normalized_permissions = []
-
           if options['SourceSecurityGroupName']
+            source_group_id=self.data[:security_groups][options['SourceSecurityGroupName']]['groupId']
             ['tcp', 'udp'].each do |protocol|
               normalized_permissions << {
                 'ipProtocol' => protocol,
                 'fromPort' => 1,
                 'toPort' => 65535,
-                'groups' => [{'groupName' => options['SourceSecurityGroupName'], 'userId' => options['SourceSecurityGroupOwnerId'] || self.data[:owner_id]}],
+                'groups' => [{'groupName' => options['SourceSecurityGroupName'], 'userId' => options['SourceSecurityGroupOwnerId'] || self.data[:owner_id], 'groupId' => source_group_id }],
                 'ipRanges' => []
               }
             end
@@ -166,7 +166,7 @@ module Fog
               'ipProtocol' => 'icmp',
               'fromPort' => -1,
               'toPort' => -1,
-              'groups' => [{'groupName' => options['SourceSecurityGroupName'], 'userId' => options['SourceSecurityGroupOwnerId'] || self.data[:owner_id]}],
+              'groups' => [{'groupName' => options['SourceSecurityGroupName'], 'userId' => options['SourceSecurityGroupOwnerId'] || self.data[:owner_id], 'groupId' => source_group_id }],
               'ipRanges' => []
             }
           elsif options['CidrIp']
@@ -179,13 +179,21 @@ module Fog
             }
           elsif options['IpPermissions']
             options['IpPermissions'].each do |permission|
-              normalized_permissions << {
-                'ipProtocol' => permission['IpProtocol'],
-                'fromPort' => Integer(permission['FromPort']),
-                'toPort' => Integer(permission['ToPort']),
-                'groups' => (permission['Groups'] || []).map {|g| {'groupName' => g['GroupName'], 'userId' => g['UserId'] || self.data[:owner_id]} },
-                'ipRanges' => (permission['IpRanges'] || []).map {|r| { 'cidrIp' => r['CidrIp'] } }
-              }
+              if ['tcp', 'udp', 'icmp'].include?(permission['IpProtocol'])
+                normalized_permissions << {
+                  'ipProtocol' => permission['IpProtocol'],
+                  'fromPort' => Integer(permission['FromPort']),
+                  'toPort' => Integer(permission['ToPort']),
+                  'groups' => (permission['Groups'] || []).map {|g| {'groupName' => g['GroupName'], 'userId' => g['UserId'] || self.data[:owner_id], 'groupId' => self.data[:security_groups][g['GroupName']]['groupId']} },
+                  'ipRanges' => (permission['IpRanges'] || []).map {|r| { 'cidrIp' => r['CidrIp'] } }
+                }
+              else
+                normalized_permissions << {
+                  'ipProtocol' => permission['IpProtocol'],
+                  'groups' => (permission['Groups'] || []).map {|g| {'groupName' => g['GroupName'], 'userId' => g['UserId'] || self.data[:owner_id], 'groupId' => self.data[:security_groups][g['GroupName']]['groupId']} },
+                  'ipRanges' => (permission['IpRanges'] || []).map {|r| { 'cidrIp' => r['CidrIp'] } }
+                }
+              end
             end
           end
 

data/lib/fog/aws/requests/compute/create_security_group.rb

@@ -3,7 +3,7 @@ module Fog
     class AWS
       class Real
 
-        require 'fog/aws/parsers/compute/basic'
+        require 'fog/aws/parsers/compute/create_security_group'
 
         # Create a new security group
         #
@@ -17,6 +17,7 @@ module Fog
         #   * body<~Hash>:
         #     * 'requestId'<~String> - Id of request
         #     * 'return'<~Boolean> - success?
+        #     * 'groupId'<~String> - Id of created group
         #
         # {Amazon API Reference}[http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/ApiReference-query-CreateSecurityGroup.html]
         def create_security_group(name, description, vpc_id=nil)
@@ -24,8 +25,8 @@ module Fog
             'Action'            => 'CreateSecurityGroup',
             'GroupName'         => name,
             'GroupDescription'  => description,
-            :parser             => Fog::Parsers::Compute::AWS::Basic.new,
-            'VpcId'             => vpc_id
+            'VpcId'             => vpc_id,
+            :parser             => Fog::Parsers::Compute::AWS::CreateSecurityGroup.new
           )
         end
 
@@ -48,6 +49,7 @@ module Fog
             self.data[:security_groups][name] = data
             response.body = {
               'requestId' => Fog::AWS::Mock.request_id,
+              'groupId'   => data['groupId'],
               'return'    => true
             }
             response

data/lib/fog/aws/requests/compute/describe_security_groups.rb

@@ -71,7 +71,7 @@ module Fog
           }
           for filter_key, filter_value in filters
             if permission_key = filter_key.split('ip-permission.')[1]
-              if permission_key == 'group-name'
+              if permission_key == 'group-name'	
                 security_group_info = security_group_info.reject{|security_group| !security_group['ipPermissions']['groups'].detect {|group| [*filter_value].include?(group['groupName'])}}
               elsif permission_key == 'group-id'
                 security_group_info = security_group_info.reject{|security_group| !security_group['ipPermissions']['groups'].detect {|group| [*filter_value].include?(group['groupId'])}}

data/lib/fog/aws/requests/storage/get_object_http_url.rb

@@ -12,9 +12,9 @@ module Fog
             raise ArgumentError.new('object_name is required')
           end
           host, path = if bucket_name =~ /^(?:[a-z]|\d(?!\d{0,2}(?:\.\d{1,3}){3}$))(?:[a-z0-9]|\.(?![\.\-])|\-(?![\.])){1,61}[a-z0-9]$/
-            ["#{bucket_name}.s3.amazonaws.com", object_name]
+            ["#{bucket_name}.#{host}", object_name]
           else
-            ['s3.amazonaws.com', "#{bucket_name}/#{object_name}"]
+            [@host, "#{bucket_name}/#{object_name}"]
           end
           http_url({
             :headers  => {},

data/lib/fog/aws/requests/storage/get_object_https_url.rb

@@ -12,9 +12,9 @@ module Fog
             raise ArgumentError.new('object_name is required')
           end
           host, path = if bucket_name =~ /^(?:[a-z]|\d(?!\d{0,2}(?:\.\d{1,3}){3}$))(?:[a-z0-9]|\.(?![\.\-])|\-(?![\.])){1,61}[a-z0-9]$/
-            ["#{bucket_name}.s3.amazonaws.com", object_name]
+            ["#{bucket_name}.#{@host}", object_name]
           else
-            ['s3.amazonaws.com', "#{bucket_name}/#{object_name}"]
+            [@host, "#{bucket_name}/#{object_name}"]
           end
           https_url({
             :headers  => {},

data/lib/fog/aws/simpledb.rb

@@ -76,7 +76,12 @@ module Fog
           @nil_string = options[:nil_string]|| 'nil'
 
           options[:region] ||= 'us-east-1'
-          @host = options[:host] || "sdb.#{options[:region]}.amazonaws.com"
+          @host = options[:host] || case options[:region]
+          when 'us-east-1'
+            'sdb.amazonaws.com'
+          else
+            "sdb.#{options[:region]}.amazonaws.com"
+          end
           @path       = options[:path]        || '/'
           @persistent = options[:persistent]  || false
           @port       = options[:port]        || 443

data/lib/fog/bin.rb

@@ -64,6 +64,7 @@ require 'fog/bin/ecloud'
 require 'fog/bin/glesys'
 require 'fog/bin/go_grid'
 require 'fog/bin/google'
+require 'fog/bin/ibm'
 require 'fog/bin/joyent'
 require 'fog/bin/libvirt'
 require 'fog/bin/linode'

data/lib/fog/bin/ibm.rb

@@ -0,0 +1,34 @@
+class IBM < Fog::Bin
+  class << self
+
+    def class_for(key)
+      case key
+      when :compute
+        Fog::Compute::IBM
+      when :storage
+        Fog::Storage::IBM
+      else
+        raise ArgumentError, "Unsupported #{self} service: #{key}"
+      end
+    end
+
+    def [](service)
+      @@connections ||= Hash.new do |hash, key|
+        hash[key] = case key
+        when :compute
+          Fog::Compute.new(:provider => 'IBM')
+        when :storage
+          Fog::Storage.new(:provider => 'Storage')
+        else
+          raise ArgumentError, "Unrecognized service: #{service}"
+        end
+      end
+      @@connections[service]
+    end
+
+    def services
+      Fog::IBM.services
+    end
+
+  end
+end

data/lib/fog/compute.rb

@@ -32,6 +32,9 @@ module Fog
       when :gogrid
         require 'fog/go_grid/compute'
         Fog::Compute::GoGrid.new(attributes)
+      when :ibm
+        require 'fog/ibm/compute'
+        Fog::Compute::IBM.new(attributes)
       when :joyent
         require 'fog/joyent/compute'
         Fog::Compute::Joyent.new(attributes)

data/lib/fog/core/errors.rb

@@ -84,6 +84,8 @@ An alternate file may be used by placing its path in the FOG_RC environment vari
   :libvirt_password:
   :libvirt_uri:
   :libvirt_ip_command:
+  :ibm_username:
+  :ibm_password:
 #
 # End of Fog Credentials File
 #######################################################