fog-libvirt-csem 0.9.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: dfd0bcd881f8bb3ef03f9b584933344a461014c1d0b0c888c44db34f0a2a03e9
+  data.tar.gz: ee3b95c80546536c1cfeae8b1df206860081422b35ee41076e15cc179c4798f3
+SHA512:
+  metadata.gz: b66669f38b82e91336bb765bcd7178f5ee9246ec961892302fdf52e42a8ef63cd87b98249d0ec1db85f55fe620d244dbedecb081ab41fc97358907b7c157a5fa
+  data.tar.gz: 8b62a1cb4b5eb575fe241c75807ad3cda9700b8d8d71ed229bed03c3ca269246baa81a27dba1c88575912ad670cd48383267283cd0715f168d317755410b513e

data/CONTRIBUTORS.md

@@ -0,0 +1,24 @@
+* Amos Benari <abenari@redhat.com>
+* brookemckim <brooke.mckim@gmail.com>
+* Carl Caum <carl@carlcaum.com>
+* Carlos Sanchez <csanchez@maestrodev.com>
+* David Wittman <david@wittman.com>
+* Dominic Cleal <dcleal@redhat.com>
+* Greg Sutcliffe <gsutclif@redhat.com>
+* James Herdman <james.herdman@me.com>
+* Josef Strzibny <strzibny@strzibny.name>
+* Kevin Menard <nirvdrum@gmail.com>
+* Konstantin Haase <konstantin.mailinglists@googlemail.com>
+* Kyle Rames <kyle.rames@rackspace.com>
+* Lance Ivy <lance@cainlevy.net>
+* Ohad Levy <ohadlevy@redhat.com>
+* Patrick Debois <Patrick.Debois@jedi.be>
+* Paul Thornthwaite <tokengeek@gmail.com>
+* Romain Vrignaud <romain@yakaz.com>
+* Ryan Davies <ryan@ryandavies.co.nz>
+* Sergio Rubio <rubiojr@frameos.org>
+* Shlomi Zadok <shlomi@ben-hanna.com>
+* Steve Smith <github@scsworld.co.uk>
+* Vincent Demeester <vincent+git@demeester.fr>
+* Wesley Beary <geemus@gmail.com>
+* Tomer Brisker <tbrisker@redhat.com>

data/Gemfile

@@ -0,0 +1,9 @@
+source "https://rubygems.org"
+
+group :development, :test do
+  # This is here because gemspec doesn't support require: false
+  gem "netrc", :require => false
+  gem "octokit", :require => false
+end
+
+gemspec

data/LICENSE.md

@@ -0,0 +1,20 @@
+The MIT License (MIT)
+
+Copyright (c) 2009-2014 [CONTRIBUTORS.md](https://github.com/fog/fog/blob/master/CONTRIBUTORS.md)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of
+this software and associated documentation files (the "Software"), to deal in
+the Software without restriction, including without limitation the rights to
+use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+the Software, and to permit persons to whom the Software is furnished to do so,
+subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,83 @@
+# Fog::Libvirt
+
+fog-libvirt is a libvirt provider for [fog](https://github.com/fog/fog).
+
+[![Build Status](https://github.com/fog/fog-libvirt/actions/workflows/ruby.yml/badge.svg)](https://github.com/fog/fog-libvirt/actions/workflows/ruby.yml)
+[![Dependency Status](https://gemnasium.com/fog/fog.png)](https://gemnasium.com/fog/fog-libvirt)
+[![Code Climate](https://codeclimate.com/github/fog/fog.png)](https://codeclimate.com/github/fog/fog-libvirt)
+[![Gem Version](https://fury-badge.herokuapp.com/rb/fog.png)](http://badge.fury.io/rb/fog-libvirt)
+[![Gittip](http://img.shields.io/gittip/geemus.png)](https://www.gittip.com/geemus/)
+
+## Installation
+
+fog-libvirt can be used as a module for fog or installed separately as:
+
+```
+$ sudo gem install fog-libvirt
+```
+
+## Usage
+
+Example REPL session:
+
+```
+>> require "fog/libvirt"
+=> true
+>> compute = Fog::Compute.new(provider: :libvirt, libvirt_uri: "qemu:///session")
+=> #<Fog::Libvirt::Compute::Real:46980 @uri=#<Fog::Libvirt::Util::URI:0x0000000002def920 @parsed_uri=#<URI::Generic qemu:/session>, @uri="qemu:///session"...
+>> server = compute.servers.create(name: "test")
+=>
+  <Fog::Libvirt::Compute::Server
+    id="bbb663e4-723b-4165-bc29-c77b54b12bca",
+    cpus=1,
+    cputime=0,
+    os_type="hvm",
+    memory_size=262144,
+    max_memory_size=262144,
+    name="test",
+    arch="x86_64",
+    persistent=true,
+    domain_type="kvm",
+    uuid="bbb663e4-723b-4165-bc29-c77b54b12bca",
+    autostart=false,
+    nics=[    <Fog::Libvirt::Compute::Nic
+      mac="52:54:00:d1:18:23",
+      id=nil,
+      type="network",
+      network="default",
+      bridge=nil,
+      model="virtio"
+    >],
+    volumes=[    <Fog::Libvirt::Compute::Volume
+      id=nil,
+      pool_name="1Download",
+      key=nil,
+      name="test.img",
+      path="/home/lzap/1Download/test.img",
+      capacity="10G",
+      allocation="1G",
+      owner=nil,
+      group=nil,
+      format_type="raw",
+      backing_volume=nil
+    >],
+    active=false,
+    boot_order=["hd", "cdrom", "network"],
+    display={:type=>"vnc", :port=>"-1", :listen=>"127.0.0.1"},
+    cpu={},
+    hugepages=false,
+    guest_agent=true,
+    virtio_rng={},
+    state="shutoff"
+  >
+```
+
+See [README.md](https://github.com/fog/fog-libvirt/blob/master/lib/fog/libvirt/models/compute/README.md).
+
+## Contributing
+
+Please refer to [CONTRIBUTING.md](https://github.com/fog/fog/blob/master/CONTRIBUTING.md).
+
+## License
+
+Please refer to [LICENSE.md](https://github.com/fog/fog-libvirt/blob/master/LICENSE.md).

data/Rakefile

@@ -0,0 +1,127 @@
+require 'bundler/setup'
+require 'rake/testtask'
+require 'rubygems'
+require 'rubygems/package_task'
+require 'yard'
+require File.dirname(__FILE__) + '/lib/fog/libvirt'
+
+#############################################################################
+#
+# Helper functions
+#
+#############################################################################
+
+def name
+  @name ||= Dir['*.gemspec'].first.split('.').first
+end
+
+def version
+  Fog::Libvirt::VERSION
+end
+
+def gemspec_file
+  "#{name}.gemspec"
+end
+
+def gem_file
+  "#{name}-#{version}.gem"
+end
+
+#############################################################################
+#
+# Standard tasks
+#
+#############################################################################
+
+GEM_NAME = "#{name}"
+task :default => [:test, :minitest]
+
+Rake::TestTask.new(:minitest) do |t|
+  t.libs << '.'
+  t.libs << 'lib'
+  t.libs << 'minitests'
+  t.test_files  =  Dir.glob('minitests/**/*_test.rb')
+  t.verbose = true
+end
+
+desc 'Run tests'
+task :test do
+  mock = ENV['FOG_MOCK'] || 'true'
+  sh("export FOG_MOCK=#{mock} && bundle exec shindont tests")
+end
+
+desc 'Run mocked tests'
+task :mock do
+  sh("export FOG_MOCK=true && bundle exec shindont tests")
+end
+
+desc 'Run live tests'
+task :live do
+  sh("export FOG_MOCK=false && bundle exec shindont tests")
+end
+
+desc "Open an irb session preloaded with this library"
+task :console do
+  sh "irb -rubygems -r ./lib/fog/libvirt.rb"
+end
+
+#############################################################################
+#
+# Packaging tasks
+#
+#############################################################################
+
+task :release => ["release:prepare", "release:publish"]
+
+namespace :release do
+  task :preflight do
+    unless `git branch` =~ /^\* master$/
+      puts "You must be on the master branch to release!"
+      exit!
+    end
+    if `git tag` =~ /^\* v#{version}$/
+      puts "Tag v#{version} already exists!"
+      exit!
+    end
+  end
+
+  task :prepare => :preflight do
+    Rake::Task[:build].invoke
+    sh "gem install pkg/#{name}-#{version}.gem"
+    Rake::Task[:git_mark_release].invoke
+  end
+
+  task :publish do
+    Rake::Task[:git_push_release].invoke
+    Rake::Task[:gem_push].invoke
+  end
+end
+
+task :git_mark_release do
+  sh "git commit --allow-empty -a -m 'Release #{version}'"
+  sh "git tag v#{version}"
+end
+
+task :git_push_release do
+  sh "git push origin master"
+  sh "git push origin v#{version}"
+end
+
+task :gem_push do
+  sh "gem push pkg/#{name}-#{version}.gem"
+end
+
+desc "Build #{name}-#{version}.gem"
+task :build do
+  sh "mkdir -p pkg"
+  sh "gem build #{gemspec_file}"
+  sh "mv #{gem_file} pkg"
+end
+task :gem => :build
+
+# Include Yard tasks for rake yard
+YARDOC_LOCATION = "doc"
+YARD::Rake::YardocTask.new do |t|
+  t.files   = ['lib/**/*.rb', "README"]
+  t.options = ["--output-dir", YARDOC_LOCATION, "--title", "#{name} #{version}"]
+end

data/fog-libvirt-csem.gemspec

@@ -0,0 +1,49 @@
+# coding: utf-8
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "fog/libvirt/version"
+
+Gem::Specification.new do |s|
+  s.specification_version = 2 if s.respond_to? :specification_version=
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+
+  s.name              = "fog-libvirt-csem"
+  s.version           = Fog::Libvirt::VERSION
+
+  s.summary     = "Module for the 'fog' gem to support libvirt"
+  s.description = "This library can be used as a module for 'fog' or as standalone libvirt provider."
+
+  s.authors  = ["geemus (Wesley Beary)"]
+  s.email    = "geemus@gmail.com"
+  s.homepage = "http://github.com/csem/fog-libvirt"
+  s.license  = "MIT"
+
+  s.require_paths = %w[lib]
+
+  s.rdoc_options = ["--charset=UTF-8"]
+  s.extra_rdoc_files = %w[README.md]
+
+  s.required_ruby_version = '>= 2.5.0'
+
+  s.add_dependency("fog-core", ">= 1.27.4")
+  s.add_dependency("fog-json")
+  s.add_dependency("fog-xml", "~> 0.1.1")
+  s.add_dependency('ruby-libvirt','>= 0.7.0')
+  s.add_dependency("json")
+
+  s.add_development_dependency("net-ssh")
+  s.add_development_dependency("minitest", "~> 5.0")
+  s.add_development_dependency("minitest-stub-const")
+  s.add_development_dependency("pry")
+  s.add_development_dependency("rake")
+  s.add_development_dependency("rubocop") if RUBY_VERSION > "2.0"
+  s.add_development_dependency("shindo", "~> 0.3.4")
+  s.add_development_dependency("simplecov")
+  s.add_development_dependency("yard")
+  s.add_development_dependency("mocha", "~> 1.13.0")
+
+  # Let's not ship dot files and gemfiles
+  git_files = `git ls-files`.split("\n")
+  s.files = git_files.delete_if{ |f| f =~ /^\..*/ || f =~ /^gemfiles\/*/ }
+  s.test_files = `git ls-files -- {spec,tests}/*`.split("\n")
+end

data/lib/fog/bin/libvirt.rb

@@ -0,0 +1,58 @@
+module Libvirt # deviates from other bin stuff to accommodate gem
+  class << self
+    def class_for(key)
+      case key
+      when :compute
+        Fog::Libvirt::Compute
+      else
+        raise ArgumentError, "Unrecognized service: #{key}"
+      end
+    end
+
+    def [](service)
+      @@connections ||= Hash.new do |hash, key|
+        hash[key] = case key
+        when :compute
+          Fog::Logger.warning("Libvirt[:compute] is not recommended, use Compute[:libvirt] for portability")
+          Fog::Compute.new(:provider => 'Libvirt')
+        else
+          raise ArgumentError, "Unrecognized service: #{key.inspect}"
+        end
+      end
+      @@connections[service]
+    end
+
+    def available?
+      begin
+        availability=true unless Gem::Specification::find_by_name("ruby-libvirt").nil?
+      rescue Gem::LoadError
+        availability=false
+      rescue
+        availability_gem=Gem.available?("ruby-libvirt")
+      end
+
+      if availability
+        for service in services
+          for collection in self.class_for(service).collections
+            unless self.respond_to?(collection)
+              self.class_eval <<-EOS, __FILE__, __LINE__
+                def self.#{collection}
+                  self[:#{service}].#{collection}
+                end
+              EOS
+            end
+          end
+        end
+      end
+      availability
+    end
+
+    def collections
+      services.map {|service| self[service].collections}.flatten.sort_by {|service| service.to_s}
+    end
+
+    def services
+      Fog::Libvirt.services
+    end
+  end
+end

data/lib/fog/libvirt/compute.rb

@@ -0,0 +1,140 @@
+require 'fog/libvirt/models/compute/util/util'
+require 'fog/libvirt/models/compute/util/uri'
+
+module Fog
+  module Libvirt
+    class Compute < Fog::Service
+      requires   :libvirt_uri
+      recognizes :libvirt_username, :libvirt_password
+      recognizes :libvirt_ip_command
+
+      model_path 'fog/libvirt/models/compute'
+      model       :server
+      collection  :servers
+      model       :network
+      collection  :networks
+      model       :interface
+      collection  :interfaces
+      model       :volume
+      collection  :volumes
+      model       :pool
+      collection  :pools
+      model       :node
+      collection  :nodes
+      model       :nic
+      collection  :nics
+
+      request_path 'fog/libvirt/requests/compute'
+      request :list_domains
+      request :create_domain
+      request :define_domain
+      request :vm_action
+      request :list_pools
+      request :list_pool_volumes
+      request :define_pool
+      request :pool_action
+      request :list_volumes
+      request :volume_action
+      request :create_volume
+      request :upload_volume
+      request :clone_volume
+      request :list_networks
+      request :destroy_network
+      request :dhcp_leases
+      request :list_interfaces
+      request :destroy_interface
+      request :get_node_info
+      request :update_autostart
+      request :update_display
+      request :libversion
+
+      module Shared
+        include Fog::Libvirt::Util
+      end
+
+      class Mock
+        include Shared
+        def initialize(options={})
+          # libvirt is part of the gem => ruby-libvirt
+          require 'libvirt'
+        end
+
+        private
+
+        def client
+          return @client if defined?(@client)
+        end
+
+        #read mocks xml
+        def read_xml(file_name)
+          file_path = File.join(File.dirname(__FILE__),"requests","compute","mock_files",file_name)
+          File.read(file_path)
+        end
+      end
+
+      class Real
+        include Shared
+        attr_reader :client
+        attr_reader :uri
+        attr_reader :ip_command
+
+        def initialize(options={})
+          @uri = ::Fog::Libvirt::Util::URI.new(enhance_uri(options[:libvirt_uri]))
+          @ip_command = options[:libvirt_ip_command]
+
+          # libvirt is part of the gem => ruby-libvirt
+          begin
+            require 'libvirt'
+          rescue LoadError => e
+            retry if require('rubygems')
+            raise e.message
+          end
+
+          begin
+            if options[:libvirt_username] and options[:libvirt_password]
+              @client = ::Libvirt::open_auth(uri.uri, [::Libvirt::CRED_AUTHNAME, ::Libvirt::CRED_PASSPHRASE]) do |cred|
+                case cred['type']
+                  when ::Libvirt::CRED_AUTHNAME
+                    options[:libvirt_username]
+                  when ::Libvirt::CRED_PASSPHRASE
+                    options[:libvirt_password]
+                end
+              end
+            else
+              @client = ::Libvirt::open(uri.uri)
+            end
+
+          rescue ::Libvirt::ConnectionError
+            raise Fog::Errors::Error.new("Error making a connection to libvirt URI #{uri.uri}:\n#{$!}")
+          end
+        end
+
+        def terminate
+          @client.close if @client and !@client.closed?
+        end
+
+        def enhance_uri(uri)
+          require 'cgi'
+          append=""
+
+          # on macosx, chances are we are using libvirt through homebrew
+          # the client will default to a socket location based on it's own location (/opt)
+          # we conveniently point it to /var/run/libvirt/libvirt-sock
+          # if no socket option has been specified explicitly and
+          # if the socket exists
+
+          socketpath="/var/run/libvirt/libvirt-sock"
+          if RUBY_PLATFORM =~ /darwin/ && File.exist?(socketpath)
+            querystring=::URI.parse(uri).query
+            if querystring.nil?
+              append="?socket=#{socketpath}"
+            elsif !::CGI.parse(querystring).key?("socket")
+              append="&socket=#{socketpath}"
+            end
+          end
+          uri+append
+        end
+      end
+    end
+  end
+end

data/lib/fog/libvirt/models/compute/README.md

@@ -0,0 +1,92 @@
+This model implements the connection with a libvirt URI.
+A libvirt URI can either be local or remote.
+
+To learn more on the specific libvirt URI syntax see:
+
+- [http://libvirt.org/uri.html](http://libvirt.org/uri.html)
+- [http://libvirt.org/remote.html#Remote_URI_reference](http://libvirt.org/remote.html#Remote_URI_reference)
+
+Only ssh is supported as the transport for remote URI's. TLS is NOT supported, as we can't easily login to the server
+
+## Dependencies
+
+- the interaction with libvirt is done through the official libvirt gem called 'ruby-libvirt'.
+- be aware that there is another libvirt gem called 'libvirt', which is not compatible
+- If this gem is not installed the models for libvirt will not be available
+
+- libvirt needs to be setup so that it can be used
+- for a remote ssh connection this requires to be member of the libvirt group before you can use the libvirt commands
+- verify if you can execute virsh command to see if you have correct access
+
+## Libvirt on Macosx
+
+- There is a libvirt client for Macosx, available via homebrew
+- By default this will install things in /usr/local/somewhere
+- This means that also the default locations of the libvirt-socket are assumed to be in /usr/local
+- To check the connection you need to override your libvirt socket location in the URI
+  - "qemu+ssh://patrick@myserver/system?socket=/var/run/libvirt/libvirt-sock"
+
+## Ceph RBD volumes
+To configure Ceph RBD volumes, the file ``/etc/foreman/ceph.conf`` is used.
+After adding the authentication key to a libvirt secret, it can be configured as follows:
+```
+monitor=mon001.example.com,mon002.example.com,mon003.example.com
+port=6789
+libvirt_ceph_pool=rbd_pool_name
+auth_username=libvirt
+auth_uuid=uuid_of_libvirt_secret
+bus_type=virtio
+```
+For more recent versions of libvirt which support using the secret by name (`usage` attribute in the `secret` tag),
+you can also drop `auth_uuid` and specify `auth_usage` instead. If both are specified, `auth_uuid` will be preferred for maximum compatibility.
+
+The `bus_type` can be set to `virtio` or `scsi`. If it is ommited, the default is `scsi`.
+
+## Configuration
+
+The URI can be configured in two ways:
+1) via the .fog file
+:default
+  :libvirt_uri: "qemu+ssh://patrick@myserver/system?socket=/var/run/libvirt/libvirt-sock"
+
+2) you can also pass it during creation :
+f=Fog::Compute.new(:provider => "Libvirt", :libvirt_uri => "qemu+ssh://patrick@myserver/system")
+
+## IP-addresses of guests
+Libvirt does not provide a way to query guests for Ip-addresses.
+The way we solve this problem is by installing arpwatch: this watches an interface for new mac-addresses and ip-addresses requested by DHCP
+We query that logfile for the mac-address and can retrieve the ip-address
+
+vi /etc/rsyslog.d/30-arpwatch.conf
+#:msg, contains, "arpwatch:" -/var/log/arpwatch.log
+#& ~
+if $programname =='arpwatch' then /var/log/arpwatch.log
+& ~
+
+This log files needs to be readable for the users of libvirt
+
+## SSh-ing into the guests
+Once we have retrieved the ip-address of the guest we can ssh into it. This works great if the URI is local.
+But when the URI is remote our machine can't ssh directly into the guest sometimes (due to NAT or firewall issues)
+
+Luckily libvirt over ssh requires netcat to be installed on the libvirt server.
+We use this to proxy our ssh requests to the guest over the ssh connection to the libvirt server.
+Thanks to the requirement that you need ssh login to work to a libvirt server, we can login and tunnel the ssh to the guest.
+
+## Bridge configuration (slowness)
+We had noticed that sometimes it takes about 30 seconds before the server gets a DHCP response from the server.
+In our case it was because the new machine Mac-address was not allowed immediately by the bridge.
+Adding the flag 'bridge_fd 0' solved that problem.
+
+/etc/network/interfaces
+auto br0
+iface br0 inet static
+address 10.247.4.13
+netmask 255.255.255.0
+network 10.247.4.0
+broadcast 10.247.4.255
+bridge_ports eth0.4
+bridge_stp on
+bridge_maxwait 0
+bridge_fd 0
+

data/lib/fog/libvirt/models/compute/interface.rb

@@ -0,0 +1,25 @@
+require 'fog/core/model'
+
+module Fog
+  module Libvirt
+    class Compute
+      class Interface < Fog::Model
+        identity :name
+        attribute :mac
+        attribute :active
+
+        def save
+          raise Fog::Errors::Error.new('Creating a new interface is not yet implemented. Contributions welcome!')
+        end
+
+        def shutdown
+          service.destroy_interface(mac)
+        end
+
+        def active?
+          active
+        end
+      end
+    end
+  end
+end

data/lib/fog/libvirt/models/compute/interfaces.rb

@@ -0,0 +1,20 @@
+require 'fog/core/collection'
+require 'fog/libvirt/models/compute/interface'
+
+module Fog
+  module Libvirt
+    class Compute
+      class Interfaces < Fog::Collection
+        model Fog::Libvirt::Compute::Interface
+
+        def all(filter={})
+          load(service.list_interfaces(filter))
+        end
+
+        def get(name)
+          self.all(:name => name).first
+        end
+      end
+    end
+  end
+end