fog 0.7.1 → 2.3.0

data/Gemfile

@@ -1,3 +1,10 @@
-source "http://rubygems.org"
+source "https://rubygems.org"
+
+group :development, :test do
+  # This is here because gemspec doesn't support require: false
+  gem "netrc", :require => false
+  gem "octokit", :require => false
+  gem "rspec", :require => false
+end
 
 gemspec

data/LICENSE.md

@@ -0,0 +1,20 @@
+The MIT License (MIT)
+
+Copyright (c) 2009-2016 [CONTRIBUTORS.md](https://github.com/fog/fog/blob/master/CONTRIBUTORS.md)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,187 @@
+![fog](http://geemus.s3.amazonaws.com/fog.png)
+
+fog is the Ruby cloud services library, top to bottom:
+
+* Collections provide a simplified interface, making clouds easier to work with and switch between.
+* Requests allow power users to get the most out of the features of each individual cloud.
+* Mocks make testing and integrating a breeze.
+
+[![Build Status](https://github.com/fog/fog/actions/workflows/ruby.yml/badge.svg)](https://github.com/fog/fog/actions/workflows/ruby.yml)
+[![Code Climate](https://codeclimate.com/github/fog/fog/badges/gpa.svg)](https://codeclimate.com/github/fog/fog)
+[![Gem Version](https://badge.fury.io/rb/fog.svg)](http://badge.fury.io/rb/fog)
+[![SemVer](https://api.dependabot.com/badges/compatibility_score?dependency-name=fog&package-manager=bundler&version-scheme=semver)](https://dependabot.com/compatibility-score.html?dependency-name=fog&package-manager=bundler&version-scheme=semver)
+
+## Dependency Notice
+
+Currently all fog providers are getting separated into metagems to lower the
+load time and dependency count.
+
+If there's a metagem available for your cloud provider, e.g. `fog-aws`,
+you should be using it instead of requiring the full fog collection to avoid
+unnecessary dependencies.
+
+'fog' should be required explicitly only if the provider you use doesn't yet
+have a metagem available.
+
+## Getting Started
+
+The easiest way to learn fog is to install the gem and use the interactive console.
+Here is an example of wading through server creation for Amazon Elastic Compute Cloud:
+
+```
+$ sudo gem install fog
+[...]
+
+$ fog
+
+  Welcome to fog interactive!
+  :default provides [...]
+
+>> server = Compute[:aws].servers.create
+ArgumentError: image_id is required for this operation
+
+>> server = Compute[:aws].servers.create(:image_id => 'ami-5ee70037')
+<Fog::AWS::EC2::Server [...]>
+
+>> server.destroy # cleanup after yourself or regret it, trust me
+true
+```
+
+## Ruby version
+
+Fog requires Ruby `2.0.0` or later.
+
+Ruby `1.8` and `1.9` support was dropped in `fog-v2.0.0` as a backwards incompatible
+change. Please use the later fog `1.x` versions if you require `1.8.7` or `1.9.x` support.
+
+## Collections
+
+A high level interface to each cloud is provided through collections, such as `images` and `servers`.
+You can see a list of available collections by calling `collections` on the connection object.
+You can try it out using the `fog` command:
+
+    >> Compute[:aws].collections
+    [:addresses, :directories, ..., :volumes, :zones]
+
+Some collections are available across multiple providers:
+
+* compute providers have `flavors`, `images` and `servers`
+* dns providers have `zones` and `records`
+* storage providers have `directories` and `files`
+
+Collections share basic CRUD type operations, such as:
+
+* `all` - fetch every object of that type from the provider.
+* `create` - initialize a new record locally and a remote resource with the provider.
+* `get` - fetch a single object by its identity from the provider.
+* `new` - initialize a new record locally, but do not create a remote resource with the provider.
+
+As an example, we'll try initializing and persisting a Rackspace Cloud server:
+
+```ruby
+require 'fog'
+
+compute = Fog::Compute.new(
+  :provider           => 'Rackspace',
+  :rackspace_api_key  => key,
+  :rackspace_username => username
+)
+
+# boot a gentoo server (flavor 1 = 256, image 3 = gentoo 2008.0)
+server = compute.servers.create(:flavor_id => 1, :image_id => 3, :name => 'my_server')
+server.wait_for { ready? } # give server time to boot
+
+# DO STUFF
+
+server.destroy # cleanup after yourself or regret it, trust me
+```
+
+## Models
+
+Many of the collection methods return individual objects, which also provide common methods:
+
+* `destroy` - will destroy the persisted object from the provider
+* `save` - persist the object to the provider
+* `wait_for` - takes a block and waits for either the block to return true for the object or for a timeout (defaults to 10 minutes)
+
+## Mocks
+
+As you might imagine, testing code using Fog can be slow and expensive, constantly turning on and shutting down instances.
+Mocking allows skipping this overhead by providing an in memory representation of resources as you make requests.
+Enabling mocking is easy to use: before you run other commands, simply run:
+
+```ruby
+Fog.mock!
+```
+
+Then proceed as usual, if you run into unimplemented mocks, fog will raise an error and as always contributions are welcome!
+
+## Requests
+
+Requests allow you to dive deeper when the models just can't cut it.
+You can see a list of available requests by calling `#requests` on the connection object.
+
+For instance, ec2 provides methods related to reserved instances that don't have any models (yet). Here is how you can lookup your reserved instances:
+
+    $ fog
+    >> Compute[:aws].describe_reserved_instances
+    #<Excon::Response [...]>
+
+It will return an [excon](http://github.com/geemus/excon) response, which has `body`, `headers` and `status`. Both return nice hashes.
+
+## Go forth and conquer
+
+Play around and use the console to explore or check out [fog.io](http://fog.io) and the [provider documentation](http://fog.io/about/provider_documentation.html)
+for more details and examples. Once you are ready to start scripting fog, here is a quick hint on how to make connections without the command line thing to help you.
+
+```ruby
+# create a compute connection
+compute = Fog::Compute.new(:provider => 'AWS', :aws_access_key_id => ACCESS_KEY_ID, :aws_secret_access_key => SECRET_ACCESS_KEY)
+# compute operations go here
+
+# create a storage connection
+storage = Fog::Storage.new(:provider => 'AWS', :aws_access_key_id => ACCESS_KEY_ID, :aws_secret_access_key => SECRET_ACCESS_KEY)
+# storage operations go here
+```
+
+geemus says: "That should give you everything you need to get started, but let me know if there is anything I can do to help!"
+
+## Versioning
+
+Fog library aims to adhere to [Semantic Versioning 2.0.0][semver], although it does not
+address challenges of multi-provider libraries. Semantic versioning is only guaranteed for
+the common API, not any provider-specific extensions.  You may also need to update your
+configuration from time to time (even between Fog releases) as providers update or deprecate
+services.
+
+However, we still aim for forwards compatibility within Fog major versions.  As a result of this policy, you can (and
+should) specify a dependency on this gem using the [Pessimistic Version
+Constraint][pvc] with two digits of precision. For example:
+
+```ruby
+spec.add_dependency 'fog', '~> 1.0'
+```
+
+This means your project is compatible with Fog 1.0 up until 2.0.  You can also set a higher minimum version:
+
+```ruby
+spec.add_dependency 'fog', '~> 1.16'
+```
+
+[semver]: http://semver.org/
+[pvc]: http://guides.rubygems.org/patterns/
+
+## Getting Help
+
+* [General Documentation](http://fog.io).
+* [Provider Specific Documentation](http://fog.io/about/provider_documentation.html).
+* Ask specific questions on [Stack Overflow](http://stackoverflow.com/questions/tagged/fog)
+* Report bugs and discuss potential features in [Github issues](https://github.com/fog/fog/issues).
+
+## Contributing
+
+Please refer to [CONTRIBUTING.md](https://github.com/fog/fog/blob/master/CONTRIBUTING.md).
+
+## License
+
+Please refer to [LICENSE.md](https://github.com/fog/fog/blob/master/LICENSE.md).

data/RELEASE.md

@@ -0,0 +1,40 @@
+# Release process
+
+This is fog's current release process, documented so people know what is
+currently done.
+
+## Versioning
+
+fog uses semantic versioning (http://semver.org/)
+
+## When we release
+
+Releases occur monthly and are manually handled by fog's Benevolent
+Dictator Wes (@geemus).
+
+To request a new release please raise an issue.
+
+## Prepare the release
+
+* Ensure the code is passing on the CI server [![Build Status](https://secure.travis-ci.org/fog/fog.png?branch=master)](http://travis-ci.org/fog/fog)
+* Ensure the code is passing for live tests (Requires Credentials for all
+services)
+* Ensure working on **master**
+* Update the version number (`lib/fog/version.rb`)
+* Run `rake changelog` to update `CHANGELOG.md`
+* Run `rake release` to prepare the release which does:
+  * Prepares the release (`rake release:prepare`)
+    * Builds the gem
+    * Tags the commit
+    * Creates commits for version
+  * Publishes the release (`rake release:publish`)
+    * Pushes commit and tag to Github (Requires Credentials)
+    * Pushes gem to Rubygems (Requires Credentials)
+* Run `rake github_release` to add release to [github release feed](https://github.com/fog/fog/releases.atom) (Requires Credentials)
+
+## Announce the release
+
+Once the release is prepared and uploaded it needs to be announced.
+
+* Send an email to https://groups.google.com/forum/?fromgroups#!forum/ruby-fog
+* Tweet as @fog on Twitter (Requires Credentials)

data/Rakefile

@@ -1,6 +1,16 @@
-require 'rubygems'
 require 'bundler/setup'
+require 'rake/testtask'
 require 'date'
+require 'rubygems'
+require 'rubygems/package_task'
+require 'yard'
+require File.dirname(__FILE__) + '/lib/fog'
+
+require "tasks/changelog_task"
+Fog::Rake::ChangelogTask.new
+
+require "tasks/github_release_task"
+Fog::Rake::GithubReleaseTask.new
 
 #############################################################################
 #
@@ -13,18 +23,13 @@ def name
 end
 
 def version
-  line = File.read("lib/#{name}.rb")[/^\s*VERSION\s*=\s*.*/]
-  line.match(/.*VERSION\s*=\s*['"](.*)['"]/)[1]
+  Fog::VERSION
 end
 
 def date
   Date.today.to_s
 end
 
-def rubyforge_project
-  name
-end
-
 def gemspec_file
   "#{name}.gemspec"
 end
@@ -33,8 +38,12 @@ def gem_file
   "#{name}-#{version}.gem"
 end
 
+def package_gem_file
+  "pkg/#{gem_file}"
+end
+
 def replace_header(head, header_name)
-  head.sub!(/(\.#{header_name}\s*= ').*'/) { "#{$1}#{send(header_name)}'"}
+  head.sub!(/(\.#{header_name}\s*= \").*\"/) { "#{$1}#{send(header_name)}\""}
 end
 
 #############################################################################
@@ -43,35 +52,75 @@ end
 #
 #############################################################################
 
+GEM_NAME = "#{name}"
 task :default => :test
+task :travis  => ['test', 'test:travis']
 
-task :examples do
-  sh("export FOG_MOCK=false && bundle exec shindont examples")
-  # some don't provide mocks so we'll leave this out for now
-  # sh("export FOG_MOCK=true  && bundle exec shindont examples")
+Rake::TestTask.new do |t|
+  t.pattern = File.join("spec", "**", "*_spec.rb")
+  t.libs << "spec"
 end
 
-task :test => :examples do
-  sh("export FOG_MOCK=true  && bundle exec spec spec") &&
-  sh("export FOG_MOCK=true  && bundle exec shindont tests") &&
-  sh("export FOG_MOCK=false && bundle exec spec spec") &&
-  sh("export FOG_MOCK=false && bundle exec shindont tests")
+namespace :test do
+  mock = ENV['FOG_MOCK'] || 'true'
+  task :travis do
+      sh("export FOG_MOCK=#{mock} && bundle exec shindont")
+  end
+  task :openvz do
+      sh("export FOG_MOCK=#{mock} && bundle exec shindont tests/openvz")
+  end
+  task :vcloud_director do
+      sh("export FOG_MOCK=#{mock} && bundle exec shindont tests/vcloud_director")
+  end
+  task :vcloud_director_specs do
+    puts "Running vCloud Minitest Suite"
+    Rake::TestTask.new do |t|
+      Dir.glob('./spec/vcloud_director/**/*_spec.rb').each { |file| require file}
+    end
+  end
 end
 
-desc "Generate RCov test coverage and open in your browser"
-task :coverage do
-  require 'rcov'
-  sh "rm -fr coverage"
-  sh "rcov test/test_*.rb"
-  sh "open coverage/index.html"
+desc 'Run mocked tests for a specific provider'
+task :mock, :provider do |t, args|
+  if args.to_a.size != 1
+    fail 'USAGE: rake mock[<provider>]'
+  end
+  provider = args[:provider]
+  sh("export FOG_MOCK=true && bundle exec shindont tests/#{provider}")
 end
 
-require 'rake/rdoctask'
-Rake::RDocTask.new do |rdoc|
-  rdoc.rdoc_dir = 'rdoc'
-  rdoc.title = "#{name} #{version}"
-  rdoc.rdoc_files.include('README*')
-  rdoc.rdoc_files.include('lib/**/*.rb')
+desc 'Run live tests against a specific provider'
+task :live, :provider do |t, args|
+  if args.to_a.size != 1
+    fail 'USAGE: rake live[<provider>]'
+  end
+  provider = args[:provider]
+  sh("export FOG_MOCK=false PROVIDER=#{provider} && bundle exec shindont tests/#{provider}")
+end
+
+task :nuke do
+  Fog.providers.each do |provider|
+    next if ['Vmfusion'].include?(provider)
+    begin
+      compute = Fog::Compute.new(:provider => provider)
+      for server in compute.servers
+        Fog::Formatador.display_line("[#{provider}] destroying server #{server.identity}")
+        server.destroy rescue nil
+      end
+    rescue
+    end
+    begin
+      dns = Fog::DNS.new(:provider => provider)
+      for zone in dns.zones
+        for record in zone.records
+          record.destroy rescue nil
+        end
+        Fog::Formatador.display_line("[#{provider}] destroying zone #{zone.identity}")
+        zone.destroy rescue nil
+      end
+    rescue
+    end
+  end
 end
 
 desc "Open an irb session preloaded with this library"
@@ -85,25 +134,60 @@ end
 #
 #############################################################################
 
-task :release => :build do
-  unless `git branch` =~ /^\* master$/
-    puts "You must be on the master branch to release!"
-    exit!
+task :release => ["release:prepare", "release:publish"]
+
+namespace :release do
+  task :preflight do
+    unless `git branch` =~ /^\* master$/
+      puts "You must be on the master branch to release!"
+      exit!
+    end
+    if `git tag` =~ /^\* v#{version}$/
+      puts "Tag v#{version} already exists!"
+      exit!
+    end
+  end
+
+  task :prepare => :preflight do
+    Rake::Task[:build].invoke
+    sh "gem install #{package_gem_file}"
+    Rake::Task[:git_mark_release].invoke
+  end
+
+  task :publish do
+    Rake::Task[:git_push_release].invoke
+    Rake::Task[:gem_push].invoke
   end
-  sh "gem install pkg/#{name}-#{version}.gem"
+end
+
+task :git_mark_release do
   sh "git commit --allow-empty -a -m 'Release #{version}'"
-  sh "git tag v#{version}"
+end
+
+task :git_push_release do
   sh "git push origin master"
-  sh "git push origin v#{version}"
-  sh "gem push pkg/#{name}-#{version}.gem"
+  ::Rake::Task[:github_release].invoke
+end
+
+task :gem_push do
+  sh "gem push #{package_gem_file}"
 end
 
+desc "Build fog-#{version}.gem"
 task :build => :gemspec do
   sh "mkdir -p pkg"
   sh "gem build #{gemspec_file}"
   sh "mv #{gem_file} pkg"
 end
+task :gem => :build
 
+desc "Install fog-#{version}.gem"
+task "install" do
+  Rake::Task[:build].invoke
+  sh "gem install #{package_gem_file} --no-document"
+end
+
+desc "Updates the gemspec and runs 'validate'"
 task :gemspec => :validate do
   # read spec file and split out manifest section
   spec = File.read(gemspec_file)
@@ -112,15 +196,14 @@ task :gemspec => :validate do
   replace_header(spec, :name)
   replace_header(spec, :version)
   replace_header(spec, :date)
-  #comment this out if your rubyforge_project has a different name
-  replace_header(spec, :rubyforge_project)
 
   File.open(gemspec_file, 'w') { |io| io.write(spec) }
   puts "Updated #{gemspec_file}"
 end
 
+desc "Run before pushing out the code"
 task :validate do
-  libfiles = Dir['lib/*'] - ["lib/#{name}.rb", "lib/#{name}"]
+  libfiles = Dir['lib/*'] - ["lib/#{name}.rb", "lib/#{name}", "lib/tasks"]
   unless libfiles.empty?
     puts "Directory `lib` should only contain a `#{name}.rb` file and `#{name}` dir."
     exit!
@@ -130,3 +213,10 @@ task :validate do
     exit!
   end
 end
+
+# Include Yard tasks for rake yard
+YARDOC_LOCATION = "doc"
+YARD::Rake::YardocTask.new do |t|
+  t.files   = ['lib/**/*.rb', "README"]
+  t.options = ["--output-dir", YARDOC_LOCATION, "--title", "#{name} #{version}"]
+end

data/benchs/each_provider.sh

@@ -0,0 +1,6 @@
+#!/bin/sh
+
+# for each provider
+grep -lr "< Fog::Service$" lib/fog | xargs -L1 dirname | sort | uniq | sed -e 's/lib\///' |
+  # trigger rubygem cost then benchmark the load time
+  xargs -I{} ruby -Ilib -rbenchmark -e 'Gem::Specification._all; puts "{}: #{Benchmark.realtime{require "{}" }}"'

data/benchs/each_service.sh

@@ -0,0 +1,6 @@
+#!/bin/sh
+
+# for each service
+grep -lr "< Fog::Service$" lib/fog | sed -e 's/lib\///' -e 's/.rb$//' |
+  # trigger rubygem cost then benchmark the load time
+  xargs -I{} ruby -Ilib -rbenchmark -e 'Gem::Specification._all; puts "{}: #{Benchmark.realtime{ require "{}" }}"'

data/benchs/load_times.rb

@@ -0,0 +1,37 @@
+require 'benchmark'
+$LOAD_PATH << File.dirname(__FILE__) + '/../lib'
+
+def time_in_fork(&block)
+  read, write = IO.pipe
+  Process.fork do
+    write.puts Benchmark.realtime{ block.call }
+  end
+  Process.wait
+  write.close
+  read.read.tap do
+    read.close
+  end
+end
+
+class Array
+  def avg
+    map(&:to_f).inject(:+) / size
+  end
+end
+
+def report(label, n = 10, &block)
+  puts label
+  puts "%.4f" % n.times.map{ time_in_fork &block }.avg
+  puts
+end
+
+# make rubygems load specifications now so the
+# time is not included below.
+Gem::Specification._all
+
+N = 10
+
+report("require fog:", N)             { require 'fog' }
+report("require fog/aws:", N)         { require 'fog/aws' }
+report("require fog/aws/compute:", N) { require 'fog/aws/compute' }
+report("require fog/aws/core:", N)    { require 'fog/aws/core'}

data/benchs/parse_vs_push.rb

@@ -3,7 +3,6 @@ require 'rubygems'
 require 'nokogiri'
 
 class Parser < Nokogiri::XML::SAX::Document
-
   attr_reader :response
 
   def initialize
@@ -16,6 +15,7 @@ class Parser < Nokogiri::XML::SAX::Document
   end
 
   def characters(string)
+    @value ||= ''
     @value << string.strip
   end
 
@@ -29,10 +29,13 @@ class Parser < Nokogiri::XML::SAX::Document
       @response[:items] << @item
       @item = {}
     when 'key'
-      @item[:key] = @value
+      @item[:key] = value
     end
   end
 
+  def value
+    @value.dup
+  end
 end
 
 data = <<-DATA
@@ -43,18 +46,22 @@ data = <<-DATA
 </items>
 DATA
 
-COUNT = 100
+COUNT = 10_000
 
 Benchmark.bmbm(25) do |bench|
   bench.report('parse') do
-    parser = Parser.new
-    Nokogiri::XML::SAX::Parser.new(parser).parse(data)
-    parser.response
+    COUNT.times do
+      parser = Parser.new
+      Nokogiri::XML::SAX::Parser.new(parser).parse(data)
+      parser.response
+    end
   end
 
   bench.report('push') do
-    parser = Parser.new
-    Nokogiri::XML::SAX::PushParser.new(parser).write(data, true)
-    parser.response
+    COUNT.times do
+      parser = Parser.new
+      Nokogiri::XML::SAX::PushParser.new(parser).write(data, true)
+      parser.response
+    end
   end
 end