nifty 0.0.1 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 144a359133c932d83555b37798b63e89b809efa0
-  data.tar.gz: 25c2ca02a30599c5551dd3f732421144e20ddc05
+  metadata.gz: 34535f6d35b3c8a655fd2e09ba345bed53776e33
+  data.tar.gz: 3da0882d184d3100c510ad0586c0b38c8adb9b7d
 SHA512:
-  metadata.gz: 1438877502e061b80cf124843c1807818d47b6bb3a7967004841221487eb98a14e069bb76b6c177279fc1a0c6c0f89ebc784128aa117001e9f16eca118de09f0
-  data.tar.gz: e579c6241833505e0e3802cde2e21beb0bf7e0a3ba714600c849ed5bcd60434453727c8cec7187b6df5960bb92bfa42196d58d26a89e297a665c40392626dc8e
+  metadata.gz: fdaeac11c89d829767d745b7e33f5290cf02bd9a243fd24a637303d8c8bf79270d94a8a8b5f64236bfa444ca0463c6396eca4ca6c6356bcbc54afc890578c532
+  data.tar.gz: a6d550cf0e9c89b9bfebe07c971892bdee2af05c3479485c33fa0d4597df41839ebdb84d01c10bb9f0ea0b31e8af6eba4f4583d798f1e9be222f964536b065f7

data/README.md

@@ -2,17 +2,33 @@
 
 NIFTY is a tool for uploading and registering cloud appliances in OpenNebula.
 
-[![Build Status](https://secure.travis-ci.org/Misenko/nifty.png)](http://travis-ci.org/Misenko/nifty)
-[![Dependency Status](https://gemnasium.com/Misenko/nifty.png)](https://gemnasium.com/Misenko/nifty)
+[![Build Status](https://secure.travis-ci.org/CESNET/nifty.png)](http://travis-ci.org/CESNET/nifty)
+[![Dependency Status](https://gemnasium.com/CESNET/nifty.png)](https://gemnasium.com/CESNET/nifty)
 [![Gem Version](https://fury-badge.herokuapp.com/rb/nifty.png)](https://badge.fury.io/rb/nifty)
-[![Code Climate](https://codeclimate.com/github/Misenko/nifty.png)](https://codeclimate.com/github/Misenko/nifty)
+[![Code Climate](https://codeclimate.com/github/CESNET/nifty.png)](https://codeclimate.com/github/CESNET/nifty)
+
+##What does NIFY do?
+Basically NIFTY takes a prepared appliance (virtual machine) and its descriptor and automatically uploads, register and prepares template in OpenNebula. Virtual machine can be than easily instantiated. Furthermore, NIFTY can expire an appliance so users will no longer be able to use it (for example due to security reasons). Expired appliances are automatically cleaned after configured time period.
+
+##How does NIFTY work?
+NIFTY cooperates with two other tools [COMFY](https://github.com/CESNET/comfy) and [ITCHY](https://github.com/CESNET/itchy) which can prepare an appliance with a correct descriptor. NIFTY then reads a descriptor and either registers or expires the appliance.   
 
 ##Requirements
 * Ruby >= 1.9.3
 * Rubygems
 
+###OpenNebula backend
+* [OpenNebula](http://opennebula.org/) >= 4.6 (doesn't have to be present on the same machine)
+
 ## Installation
-**Unfortunately, neither gem nor packages are available for NIFTY right now. To try NIFTY, please use the [From source](#from-source-dev) guide below.**
+###From distribution specific packages
+Omnibus packaging for NIFTY will be available soon. Meanwhile, use one of other installation methods.
+
+###From RubyGems.org
+To install the most recent stable version
+```bash
+gem install nifty
+```
 
 ###From source (dev)
 **Installation from source should never be your first choice! Especially, if you are not
@@ -23,7 +39,7 @@ familiar with RVM, Bundler, Rake and other dev tools for Ruby!**
 To build and install the bleeding edge version from master
 
 ```bash
-git clone git://github.com/Misenko/nifty.git
+git clone git://github.com/CESNET/nifty.git
 cd nifty
 gem install bundler
 bundle install
@@ -35,18 +51,69 @@ bundle exec rake spec
 Configuration file can be read by NIFTY from these
 three locations:
 
-* `~/.nifty/conf.yml`
-* `/etc/nifty/conf.yml`
-* `PATH_TO_GEM_DIR/config/conf.yml`
+* `~/.nifty/nifty.yml`
+* `/etc/nifty/nifty.yml`
+* `PATH_TO_GEM_DIR/config/nifty.yml`
 
 The example configuration file can be found at the last location
-`PATH_TO_GEM_DIR/config/conf.yml`.
+`PATH_TO_GEM_DIR/config/nifty.yml`.
+
+##Usage
+NIFTY is run with executable `nifty`. For further assistance run `nifty help`:
+```bash
+$ nifty help
+
+Commands:
+  nifty backends                                                                             # Lists all available backends with their description
+  nifty help [COMMAND]                                                                       # Describe available commands or one specific command
+  nifty opennebula --expiration-interval=EXPIRATION-INTERVAL -d, --datastores=one two three  # Runs NIFTY with backend opennebula
+  nifty opennebula-transfer-methods                                                          # Lists all available transfer methods with their description for opennebula backend
+  nifty version                                                                              # Prints NIFTY's version
+
+Options:
+  -a, --appliance-dir=APPLIANCE-DIR                  # Directory from where appliance descriptors will be loaded
+  -m, --transfer-method=TRANSFER-METHOD              # Transfer method for image upload
+  -t, [--transfer-destination=TRANSFER-DESTINATION]  # Image upload destination
+      --logging-level=LOGGING-LEVEL
+      [--logging-file=LOGGING-FILE]                  # File to write log to
+      [--debug], [--no-debug]                        # Runs nifty in debug mode
+```
+
+###Backends
+OpenNebula is currently the only supported backend for NIFTY. NIFTY can upload and register images and create VM templates in OpenNebula. Image and template generating mechanism can be customized by providing custom templates via `--template-dir` option. Default templates are situated in `PATH_TO_GEM_DIR/config/templates` directory. To list all options for Opennebula backend run `nifty help opennebula`:
+```bash
+$ nifty help opennebula
+
+Usage:
+  nifty opennebula --expiration-interval=EXPIRATION-INTERVAL -d, --datastores=one two three
+
+Options:
+      [--secret=SECRET]                                  # Pair of username and password in form of 'username:password' for accessing OpenNebula
+      [--endpoint=ENDPOINT]                              # OpenNebula's XML RPC endpoint
+      --expiration-interval=EXPIRATION-INTERVAL          # How long should expired images be kept in OpenNebula befor removal, 0 means don't remove
+  -d, --datastores=one two three                         # Names of OpenNebula datastores images will be uploaded to
+      [--template-dir=TEMPLATE-DIR]                      # If set, templates within this directory are used to construct images and templates in OpenNebula
+      [--description-naming], [--no-description-naming]  # If true, uses identifier and version from appliance description as template/image name instead of generated uuids
+      [--disk-expiration], [--no-disk-expiration]        # Will expire old disks before the new one is registered
+      --permissions=PERMISSIONS                          # UNIX-like image and template permissions in OpenNebula
+  -a, --appliance-dir=APPLIANCE-DIR                      # Directory from where appliance descriptors will be loaded
+  -m, --transfer-method=TRANSFER-METHOD                  # Transfer method for image upload
+  -t, [--transfer-destination=TRANSFER-DESTINATION]      # Image upload destination
+  -b, --backend=BACKEND                                  # Backend for registering images
+      --logging-level=LOGGING-LEVEL
+      [--logging-file=LOGGING-FILE]                      # File to write log to
+      [--debug], [--no-debug]                            # Runs nifty in debug mode
+```
+
+###Transfer methods
+NIFTY currently supports two transfer methods:
+* `noop` (no operation) - dummy transfer method, doesn't change image location
+* `cp` (copy) - creates an image copy in `--transfer-destination` directory
 
-## Usage
-NIFTY is run with executable `nifty`.
+To list all available transfer methods for selected backend run `nifty <BACKEND>-transfer-methods`, for example `nifty opennebula-transfer-methods`.
 
 ## Contributing
-1. Fork it ( https://github.com/[my-github-username]/nifty/fork )
+1. Fork it ( https://github.com/CESNET/nifty/fork )
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
 4. Push to the branch (`git push origin my-new-feature`)

data/bin/nifty

@@ -1,2 +1,4 @@
 #!/usr/bin/env ruby
 require 'nifty'
+
+Nifty::CommandExecutioner.start(ARGV)

data/config/nifty.yml

@@ -0,0 +1,18 @@
+production:
+  appliance-dir: /var/run/nifty/appliances # Directory from where appliance descriptors will be loaded.
+  transfer:
+    method: noop # Transfer method for image upload. Available methods can be listed in help.
+    destination: # Image upload destination. Format depends on transfer method.
+  opennebula:
+    secret: oneadmin:opennebula # If not specified, looking for secret in environment variable ONE_AUTH and file ~/.one/one_auth
+    endpoint: http://localhost:2633/RPC2 # If not specified, looking for endpoint in environment variable ONE_XMLRPC and file ~/.one/one_endpoint
+    datastores: # Array of names of OpenNebula datastores images will be uploaded to. Has to contain at least one datastore if opennebula backend is selected
+    expiration-interval: 2w # How long should expired images be kept in OpenNebula before removal, 0 means don't remove
+    template-dir: # If set, templates within this directory are used to construct images and templates in OpenNebula
+    description-naming: false # If true, uses identifier and version from appliance description as template/image name instead of generated uuids
+    disk-expiration: true # Will expire old disks before the new one is registered
+    permissions: "640" # UNIX-like image and template permissions in OpenNebula, MUST be a string
+  logging:
+    level: ERROR # Logging level
+    file: /var/log/nifty/nifty.log # File to write log to. To turn off file logging leave this field empty.
+  debug: false # Debug mode

data/config/templates/image.erb

@@ -0,0 +1,38 @@
+NAME = "<%= name %>"
+
+<% if appliance.title && appliance.description %>
+DESCRIPTION = "<%= "#{appliance.title} - #{appliance.description} created using NIFTY on #{::DateTime.now.readable_inspect}." %>"
+<% else %>
+DESCRIPTION = "<%= "Image created using NIFTY on #{::DateTime.now.readable_inspect}." %>"
+<% end %>
+
+<% case disk.type.upcase %>
+<% when "CDROM" %>
+TYPE = "CDROM"
+DISK_TYPE = "CDROM"
+PATH = "<%= disk.path %>"
+<% when "FILE" %>
+TYPE = "FILE"
+DISK_TYPE = "FILE"
+PATH = "<%= disk.path %>"
+<% else %>
+TYPE = "OS"
+DISK_TYPE = "BLOCK"
+PATH = "<%= disk.path %>"
+<% end %>
+
+<% if disk.device_prefix %>
+DEV_PREFIX = "<%= disk.device_prefix %>"
+<% end %>
+
+<% if disk.target %>
+BUS = "<%= disk.target %>"
+<% end %>
+
+<% if disk.format %>
+DRIVER = "<%= disk.format %>"
+<% end %>
+
+<% appliance.attributes.each do |key, value| %>
+<%= key %> = "<%= value %>"
+<% end %>

data/config/templates/template.erb

@@ -0,0 +1,26 @@
+NAME = "<%= name %>"
+
+<% if appliance.title && appliance.description %>
+DESCRIPTION = "<%= "#{appliance.title} - #{appliance.description} created using NIFTY on #{::DateTime.now.readable_inspect}." %>"
+<% else %>
+DESCRIPTION = "<%= "Template created using NIFTY on #{::DateTime.now.readable_inspect}." %>"
+<% end %>
+
+MEMORY = "<%= appliance.memory || 1024 %>"
+CPU = "<%= appliance.cpu || 0.25 %>"
+VCPU = "<%= appliance.cpu || 1 %>"
+
+OS = [
+ARCH = "<%= appliance.os.arch || "x86_64" %>"
+]
+
+<% disk_names.each do |name| %>
+DISK = [
+IMAGE = "<%= name %>",
+IMAGE_UNAME = "<%= username %>"
+]
+<% end %>
+
+<% appliance.attributes.each do |key, value| %>
+<%= key %> = "<%= value %>"
+<% end %>

data/lib/nifty/backend.rb

@@ -0,0 +1,55 @@
+# Abstract class for all NIFTY backends
+#
+# @author Michal Kimle
+class Nifty::Backend
+  class << self
+    # Helper method to recognize NIFTY backend
+    #
+    # @return [TrueClass, FalseClass] whether or not class is a NIFTY backend
+    def backend?
+      false
+    end
+
+    # Returns textual description of the backend
+    # Used in help messages.
+    #
+    # @abstract
+    # @return [String, nil] textual description of the backend
+    def description
+      nil
+    end
+
+    # Returns hash from which new CLI command for backend is constructed
+    #
+    # @abstract
+    # @return [Hash, nil] hash from which new CLI command for backend is constructed
+    def options
+      nil
+    end
+
+    # Routine run before any events are processed
+    #
+    # @abstract
+    # @param [Hash] parameters to run pre method with
+    def pre(parameters)
+    end
+
+    # Routine run after all the events are processed
+    #
+    # @abstract
+    # @param [Hash] parameters to run post method with
+    def post(parameters)
+    end
+
+    # Creates specific event for the backend
+    #
+    # @abstract
+    # @param [Nifty::Events::Event] event_class class object representing event
+    # @param [Cloud::Appliance::Descriptor::Appliance] appliance description
+    # @param [Nifty::TransferMethod] transfer_method instance of transfer method used in the event
+    # @param [Hash] parameters any additional parameters
+    # @return [Nifty::Events::Event] instance of event
+    def create_event(event_class, appliance, transfer_method, parameters)
+    end
+  end
+end

data/lib/nifty/backends/opennebula.rb

@@ -0,0 +1,109 @@
+require 'time'
+require 'nifty/backends/utils/opennebula'
+require 'chronic_duration'
+
+# OpenNebula backend
+class Nifty::Backends::Opennebula < Nifty::Backend
+  class << self
+    # @see Nifty::Backend#backend?
+    def backend?
+      true
+    end
+
+    # @see Nifty::Backend#description
+    def description
+      "OpenNebula backend"
+    end
+
+    # @see Nifty::Backend#options
+    def options
+      { :secret => {
+          :type => :string,
+          :desc => 'Pair of username and password in form of \'username:password\' for accessing OpenNebula'
+        },
+        :endpoint => {
+          :type => :string,
+          :desc => 'OpenNebula\'s XML RPC endpoint'
+        },
+        :"expiration-interval" => {
+          :required => true,
+          :type => :string,
+          :desc => 'How long should expired images be kept in OpenNebula befor removal, 0 means don\'t remove'
+        },
+        :datastores => {
+          :required => true,
+          :type => :array,
+          :aliases => '-d',
+          :desc => 'Names of OpenNebula datastores images will be uploaded to'
+        },
+        :"template-dir" => {
+          :default => File.join(Nifty::GEM_DIR, 'config', 'templates'),
+          :type => :string,
+          :desc => 'If set, templates within this directory are used to construct images and templates in OpenNebula'
+        },
+        :"description-naming" => {
+          :type => :boolean,
+          :desc => 'If true, uses identifier and version from appliance description as template/image name instead of generated uuids'
+        },
+        :"disk-expiration" => {
+          :type => :boolean,
+          :desc => 'Will expire old disks before the new one is registered'
+        },
+        :permissions => {
+          :required => true,
+          :type => :string,
+          :desc => 'UNIX-like image and template permissions in OpenNebula'
+        }
+      }
+    end
+
+    # @see Nifty::Backend#pre
+    def pre(parameters)
+      expiration_check(parameters)
+    end
+
+    # @see Nifty::Backend#create_event
+    # @param [Hash] parameters
+    # @option parameters [String] secret OpenNebula's secret
+    # @option parameters [String] endpoint OpenNebula's endpoint
+    # @option parameters [Array] datastores array of datastore names
+    # @option parameters [String] template-dir path to dir with image and template templates
+    def create_event(event_class, appliance, transfer_method, parameters)
+      client = Nifty::Backends::Utils::Opennebula::Helper.client(parameters[:secret], parameters[:endpoint])
+      datastores = Nifty::Backends::Utils::Opennebula::DatastoreHandler.new(client).datastores(parameters[:datastores])
+      event_class.new(appliance, transfer_method, client, datastores, parameters)
+    rescue Nifty::Errors::Backends::OpennebulaError => ex
+      fail Nifty::Errors::BackendError, ex
+    end
+
+    private
+
+    # Checks whether there are expired images that should be removed
+    #
+    # @param [Hash] parameters
+    # @option parameters [String] expiration-interval expiration interval in human readable form
+    # @option parameters [String] secret OpenNebula's secret
+    # @option parameters [String] endpoint OpenNebula's endpoint
+    def expiration_check(parameters)
+      interval = ChronicDuration.parse(parameters[:"expiration-interval"])
+      return unless interval
+
+      client = Nifty::Backends::Utils::Opennebula::Helper.client(parameters[:secret], parameters[:endpoint])
+      image_handler = Nifty::Backends::Utils::Opennebula::ImageHandler.new(client)
+      now = Time.now
+
+      logger.debug('Looking for expired images...')
+      image_handler.expired_images.each do |image|
+        image_handler.disable_image image
+
+        image_expiration_date = Time.parse(image['TEMPLATE/NIFTY_EXPIRATION'])
+        # checks whether image is already expired for specified time
+        next if (now - image_expiration_date) < interval
+
+        image_handler.delete_image image
+      end
+    rescue Nifty::Errors::Backends::OpennebulaError => ex
+      logger.error("Cannot check and remove expired images: #{ex.message}")
+    end
+  end
+end

data/lib/nifty/backends/utils/opennebula/datastore_handler.rb

@@ -0,0 +1,36 @@
+# Handler for OpenNebula DatastorePool
+#
+# @author Michal Kimle
+class Nifty::Backends::Utils::Opennebula::DatastoreHandler < Nifty::Backends::Utils::Opennebula::Handler
+
+  # Constructor
+  #
+  # @see Nifty::Backends::Utils::Opennebula::Handler#initialize
+  def initialize(client)
+    super(client)
+    @pool = OpenNebula::DatastorePool.new(client)
+  end
+
+  # Returns list of datastores matching the names
+  # If there is no datastore with given name, it's skipped.
+  #
+  # @param [Array] names array of the names
+  # @return [Array] array of found datastores
+  def datastores(names)
+    reload!
+
+    datastores = []
+    names.each do |name|
+      datastore = pool.find { |ds| ds.name == name }
+
+      unless datastore
+        logger.warn("Datastore #{name.inspect} was not found, skipping.")
+        next
+      end
+
+      datastores << datastore
+    end
+
+    datastores
+  end
+end

data/lib/nifty/backends/utils/opennebula/group_handler.rb

@@ -0,0 +1,23 @@
+# Handler for OpenNebula GroupPool
+#
+# @author Michal Kimle
+class Nifty::Backends::Utils::Opennebula::GroupHandler < Nifty::Backends::Utils::Opennebula::Handler
+
+  # Constructor
+  #
+  # @see Nifty::Backends::Utils::Opennebula::Handler#initialize
+  def initialize(client)
+    super(client)
+    @pool = OpenNebula::GroupPool.new(client)
+  end
+
+  # Returns groupt with specified name
+  #
+  # @param [String] name
+  # @return [OpenNebula::Group] group with specified name
+  def group(name)
+    reload!
+
+    pool.find { |group| group.name == name }
+  end
+end

data/lib/nifty/backends/utils/opennebula/handler.rb

@@ -0,0 +1,60 @@
+# Handler for OpenNebula pools
+#
+# @author Michal Kimle
+# @abstract
+# @attr_reader [OpenNebula::Client] client
+# @attr [OpenNebula::Pool] pool
+class Nifty::Backends::Utils::Opennebula::Handler
+  attr_reader :client
+  attr_accessor :pool
+
+  ONEADMIN_ID = 0
+  LEAVE_AS_IS = -1
+  ATTRIBUTE_APPLIANCE_ID = 'NIFTY_APPLIANCE_ID'
+  OWNER_OCTET = '600'
+
+  # Constructor
+  #
+  # @param [OpenNebula::Client] client
+  def initialize(client)
+    @client = client
+  end
+
+  # Reloads the pool
+  #
+  def reload!
+    return unless pool
+
+    if pool.respond_to?('info_all!')
+      Nifty::Backends::Utils::Opennebula::Helper.handle_opennebula_error { pool.info_all! }
+      return
+    end
+
+    Nifty::Backends::Utils::Opennebula::Helper.handle_opennebula_error { pool.info! }
+  end
+
+  # Change owner of the element
+  #
+  # @param [OpenNebula::PoolElement] element
+  # @param [Fixnum] owner_id
+  # @param [Fixnum] group_id
+  def self.chown(element, owner_id, group_id)
+    Nifty::Backends::Utils::Opennebula::Helper.handle_opennebula_error { element.info! }
+
+    old_owner_id = element['UID'].to_i
+    old_group_id = element.gid
+    owner_id = LEAVE_AS_IS if owner_id == old_owner_id
+    group_id = LEAVE_AS_IS if group_id == old_group_id
+
+    Nifty::Backends::Utils::Opennebula::Helper.handle_opennebula_error { element.chown(owner_id, group_id) }
+  end
+
+  # Change permissions on the element
+  #
+  # @param [OpenNebula::PoolElement] element
+  # @param [String] octet UNIX-like permission octet
+  def self.chmod(element, octet)
+    Nifty::Backends::Utils::Opennebula::Helper.handle_opennebula_error { element.info! }
+    Nifty::Backends::Utils::Opennebula::Helper.handle_opennebula_error { element.chmod_octet(octet) }
+  end
+end

data/lib/nifty/backends/utils/opennebula/helper.rb

@@ -0,0 +1,44 @@
+require 'opennebula'
+
+# Helper class for OpenNebula backend
+#
+# @author Michal Kimle
+class Nifty::Backends::Utils::Opennebula::Helper
+  class << self
+    # Creates client for communication with OpenNebula
+    #
+    # @param [String] secret OpenNebula's authentication secret
+    # @param [String] endpoint OpenNebula's endpoint
+    # @return [OpenNebula::Client] OpenNebula client instance
+    def client(secret, endpoint)
+      OpenNebula::Client.new(secret, endpoint)
+    end
+
+    # Handles OpenNebula error codes and turn them into exceptions
+    #
+    # @raise [Nifty::Errors::Backends::Opennebula::AuthenticationError]
+    # @raise [Nifty::Errors::Backends::Opennebula::UserNotAuthorizedError]
+    # @raise [Nifty::Errors::Backends::Opennebula::ResourceNotFoundError]
+    # @raise [Nifty::Errors::Backends::Opennebula::ResourceStateError]
+    # @raise [Nifty::Errors::Backends::Opennebula::ResourceRetrievalError]
+    def handle_opennebula_error
+      fail Nifty::Errors::Backends::Opennebula::StubError, 'OpenNebula service-wrapper was called without a block!' unless block_given?
+
+      return_value = yield
+      return return_value unless OpenNebula.is_error?(return_value)
+
+      case return_value.errno
+      when OpenNebula::Error::EAUTHENTICATION
+        fail Nifty::Errors::Backends::Opennebula::AuthenticationError, return_value.message
+      when OpenNebula::Error::EAUTHORIZATION
+        fail Nifty::Errors::Backends::Opennebula::UserNotAuthorizedError, return_value.message
+      when OpenNebula::Error::ENO_EXISTS
+        fail Nifty::Errors::Backends::Opennebula::ResourceNotFoundError, return_value.message
+      when OpenNebula::Error::EACTION
+        fail Nifty::Errors::Backends::Opennebula::ResourceStateError, return_value.message
+      else
+        fail Nifty::Errors::Backends::Opennebula::ResourceRetrievalError, return_value.message
+      end
+    end
+  end
+end