vagrant-libvirt 0.4.1 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8cce5cc2c14c33a541cb2c0a446dd76692685131abbff67992976e736cd0f04e
-  data.tar.gz: 4abe7566cc6269cb8c9fa4cac79cdfe6a1d5919e7305ffec1a214e937853c4ea
+  metadata.gz: af89387fcdeb6518bd31879ac2eaa00373c3af1c02613b389868eb6c656b5f2b
+  data.tar.gz: 82fa3560eb88abac51088e894d8e475bbf4dbcda4b10596b43e7fc41d158945a
 SHA512:
-  metadata.gz: 2eb5a19c0aa21eccd856d8056270669d201ceec6f3f6cfb209ec7e6185bfcca0a25fd1b60556bef4ea92ed261b6069c17011c2c0083999c146065dc216656beb
-  data.tar.gz: c3ffc56e5e8d680f87541be65a9990fd77a0e8235c070a413eb5d44d63f7cb1b5bbc712ea6c8b5fb8fe45ef85d72a967e31f241ea0461a9b943f7cd79eb6151b
+  metadata.gz: 53d1714279eaa3ed9c0434b891bad6bc03b4f52a0c2a5aed6e0736cefa89a3f66243ec3d04e2bf1bb707f76169e2f799ffa98e63f3cfdc3c91ee0d56e5332695
+  data.tar.gz: 15ea6d1ce672545f14f3886a6d945f2349c85cca61a63604abf30f5a716b9665c65ea181fd2dcb96c39321567725911b230d22d1c589c888a2b05ec9c7715b13

data/README.md

@@ -55,6 +55,7 @@ can help a lot :-)
 * [No box and PXE boot](#no-box-and-pxe-boot)
 * [SSH Access To VM](#ssh-access-to-vm)
 * [Forwarded Ports](#forwarded-ports)
+  * [Forwarding the ssh-port](#forwarding-the-ssh-port)
 * [Synced Folders](#synced-folders)
 * [QEMU Session Support](#qemu-session-support)
 * [Customized Graphics](#customized-graphics)
@@ -62,7 +63,9 @@ can help a lot :-)
 * [Memory balloon](#memory-balloon)
 * [Libvirt communication channels](#libvirt-communication-channels)
 * [Custom command line arguments and environment variables](#custom-command-line-arguments-and-environment-variables)
-* [Box Format](#box-format)
+* [Box Formats](#box-formats)
+  * [Version 1](#version-1)
+  * [Version 2 (Experimental)](#version-2-experimental)
 * [Create Box](#create-box)
 * [Package Box from VM](#package-box-from-vm)
 * [Troubleshooting VMs](#troubleshooting-vms)
@@ -83,7 +86,7 @@ can help a lot :-)
 * SSH into domains.
 * Setup hostname and network interfaces.
 * Provision domains with any built-in Vagrant provisioner.
-* Synced folder support via `rsync`, `nfs` or `9p`.
+* Synced folder support via `rsync`, `nfs`, `9p` or `virtiofs`.
 * Snapshots via [sahara](https://github.com/jedi4ever/sahara).
 * Package caching via
   [vagrant-cachier](http://fgrehm.viewdocs.io/vagrant-cachier/).
@@ -892,6 +895,8 @@ used by this network are configurable at the provider level.
 * `management_network_pci_slot` -  The slot of the PCI device.
 * `management_network_mac` - MAC address of management network interface.
 * `management_network_domain` - Domain name assigned to the management network.
+* `management_network_mtu` - MTU size of management network. If not specified,
+  the Libvirt default (1500) will be used.
 
 You may wonder how vagrant-libvirt knows the IP address a VM received.  Libvirt
 doesn't provide a standard way to find out the IP address of a running domain.
@@ -1256,6 +1261,8 @@ Vagrant.configure("2") do |config|
     libvirt.hyperv_feature :name => 'relaxed', :state => 'on'
     # Enable virtual APIC
     libvirt.hyperv_feature :name => 'vapic', :state => 'on'
+    # Enable spinlocks (requires retries to be specified)
+    libvirt.hyperv_feature :name => 'spinlocks', :state => 'on', :retries => '8191'
   end
 end
 ```
@@ -1412,40 +1419,157 @@ Default is `eth0`.
 
 `config.vm.network :forwarded_port, guest: 80, host: 2000, host_ip: "0.0.0.0"`
 
+### Forwarding the ssh-port
+
+Vagrant-libvirt now supports forwarding the standard ssh-port on port 2222 from
+the localhost to allow for consistent provisioning steps/ports to be used when
+defining across multiple providers.
+
+To enable, set the following:
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    # Enable forwarding of forwarded_port with id 'ssh'.
+    libvirt.forward_ssh_port = true
+  end
+end
+```
+
+Previously by default libvirt skipped the forwarding of the ssh-port because
+you can access the machine directly. In the future it is expected that this
+will be enabled by default once autocorrect support is added to handle port
+collisions for multi machine environments gracefully.
+
 ## Synced Folders
 
-Vagrant automatically syncs the project folder on the host to `/vagrant` in the guest. You can also configure
-additional synced folders.
+Vagrant automatically syncs the project folder on the host to `/vagrant` in
+the guest. You can also configure additional synced folders.
 
-`vagrant-libvirt` supports bidirectional synced folders via [NFS](https://en.wikipedia.org/wiki/Network_File_System) or [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p or Plan 9](https://en.wikipedia.org/wiki/9P_(protocol))) and
-unidirectional via rsync. The default is NFS. Difference between NFS and 9p is explained [here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
+public network interface and you must connect to Libvirt via ssh.
 
-You can change the synced folder type for `/vagrant` by explicity configuring
-it an setting the type, e.g.
+**NFS**
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: 'rsync'
-```
+`vagrant-libvirt` supports
+[NFS](https://www.vagrantup.com/docs/synced-folders/nfs) as default with
+bidirectional synced folders.
 
-or
+Example with NFS:
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "squash", owner: "1000"
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant"
+end
 ```
 
-or
+**RSync**
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "mapped", mount: false
+`vagrant-libvirt` supports
+[rsync](https://www.vagrantup.com/docs/synced-folders/rsync) with
+unidirectional synced folders.
+
+Example with rsync:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "rsync"
+end
 ```
 
+**9P**
+
+`vagrant-libvirt` supports [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p
+or Plan 9](https://en.wikipedia.org/wiki/9P_\(protocol\))) with bidirectional
+synced folders.
+
+Difference between NFS and 9p is explained
+[here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+
 For 9p shares, a `mount: false` option allows to define synced folders without
 mounting them at boot.
 
-Further documentation on using 9p can be found in [kernel docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in [QEMU wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly). Please do note that 9p depends on support in the guest and not all distros come with the 9p module by default.
+Example for `accessmode: "squash"` with 9p:
 
-**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
-public network interface and you must connect to Libvirt via ssh.
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "squash", owner: "1000"
+end
+```
+
+Example for `accessmode: "mapped"` with 9p:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "mapped", mount: false
+end
+```
+
+Further documentation on using 9p can be found in [kernel
+docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in
+[QEMU
+wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly).
+
+Please do note that 9p depends on support in the guest and not all distros
+come with the 9p module by default.
+
+**Virtio-fs**
+
+`vagrant-libvirt` supports [Virtio-fs](https://virtio-fs.gitlab.io/) with
+bidirectional synced folders.
+
+For virtiofs shares, a `mount: false` option allows to define synced folders
+without mounting them at boot.
+
+So far, passthrough is the only supported access mode and it requires running
+the virtiofsd daemon as root.
+
+QEMU needs to allocate the backing memory for all the guest RAM as shared
+memory, e.g. [Use file-backed
+memory](https://libvirt.org/kbase/virtiofs.html#host-setup) by enable
+`memory_backing_dir` option in `/etc/libvirt/qemu.conf`:
+
+``` shell
+memory_backing_dir = "/dev/shm"
+```
+
+Example for Libvirt \>= 6.2.0 (e.g. Ubuntu 20.10 with Linux 5.8.0 + QEMU 5.0 +
+Libvirt 6.6.0, i.e. NUMA nodes required) with virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.numa_nodes = [{ :cpus => "0-1", :memory => 8192, :memAccess => "shared" }]
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Example for Libvirt \>= 6.9.0 (e.g. Ubuntu 21.04 with Linux 5.11.0 + QEMU 5.2 +
+Libvirt 7.0.0, or Ubuntu 20.04 + [PPA
+enabled](https://launchpad.net/~savoury1/+archive/ubuntu/virtualisation)) with
+virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.memory = 8192
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Further documentation on using virtiofs can be found in [official
+HowTo](https://virtio-fs.gitlab.io/index.html#howto) and in [Libvirt
+KB](https://libvirt.org/kbase/virtiofs.html).
+
+Please do note that virtiofs depends on:
+
+  - Host: Linux \>= 5.4, QEMU \>= 4.2 and Libvirt \>= 6.2 (e.g. Ubuntu 20.10)
+  - Guest: Linux \>= 5.4 (e.g. Ubuntu 20.04)
 
 ## QEMU Session Support
 
@@ -1652,7 +1776,11 @@ Vagrant.configure("2") do |config|
 end
 ```
 
-## Box Format
+## Box Formats
+
+### Version 1
+
+This is the original format that most boxes currently use.
 
 You can view an example box in the
 [`example_box/directory`](https://github.com/vagrant-libvirt/vagrant-libvirt/tree/master/example_box).
@@ -1666,6 +1794,45 @@ The box is a tarball containing:
 * `Vagrantfile` that does default settings for the provider-specific
   configuration for this provider
 
+
+### Version 2 (Experimental)
+
+Due to the limitation of only being able to handle a single disk with the version 1 format, a new
+format was added to support boxes that need to specify multiple disks. This is still currently
+experimental and as such support for packaging has yet to be added. There is a script in the tools
+folder (tools/create_box_with_two_disks.sh) that should provide a guideline on how to create such
+a box for those that wish to experiment and provide early feedback.
+
+At it's most basic, it expects an array of disks to allow a specific order to be presented. Disks
+will be attached in this order and as such assume device names base on this within the VM. The
+'path' attribute is required, and is expected to be relative to the base of the box. This should
+allow placing the disk images within a nested directory within the box if it useful for those
+with a larger number of disks. The name allows overriding the target volume name that will be
+used in the libvirt storage pool. Note that vagrant-libvirt will still prefix the volume name
+with `#{box_name}_vagrant_box_image_#{box_version}_` to avoid accidental clashes with other boxes.
+
+Format and virtual size need no longer be specified as they are now retrieved directly from the
+provided image using `qemu-img info ...`.
+
+Example format:
+```json
+{
+  'disks': [
+      {
+          'path': 'disk1.img'
+      },
+      {
+          'path': 'disk2.img',
+          'name': 'secondary_disk'
+      },
+      {
+          'path': 'disk3.img'
+      }
+  ],
+  'provider': 'libvirt'
+}
+```
+
 ## Create Box
 
 If creating a box from a modified vagrant-libvirt machine, ensure that
@@ -1782,14 +1949,32 @@ $ bundle install
 Once you have the dependencies, verify the unit tests pass with `rspec`:
 
 ```shell
-$ bundle exec rspec spec/
+$ export VAGRANT_HOME=$(mktemp -d)
+$ bundle exec rspec --fail-fast --color --format documentation
 ```
 
-If those pass, you're ready to start developing the plugin. You can test the
-plugin without installing it into your Vagrant environment by just creating a
-`Vagrantfile` in the top level of this directory (it is gitignored) that uses
-it. You can add the following line to your Vagrantfile while in development to
-ensure vagrant checks that the plugin is installed:
+If those pass, you're ready to start developing the plugin.
+
+Setting `VAGRANT_HOME` is to avoid issues with conflicting with other
+plugins/gems or data already present under `~/.vagrant.d`.
+
+Additionally if you wish to test against a specific version of vagrant you
+can control the version using the following before running the tests:
+
+```shell
+$ export VAGRANT_VERSION=v2.2.14
+```
+
+**Note** rvm is used by the maintainers to help provide an environment to test
+against multiple ruby versions that align with the ones used by vagrant for
+their embedded ruby depending on the release. You can see what version is used
+by looking at the current [unit tests](.github/workflows/unit-tests.yml)
+workflow.
+
+You can test the plugin without installing it into your Vagrant environment by
+just creating a `Vagrantfile` in the top level of this directory (it is
+gitignored) that uses it. You can add the following line to your Vagrantfile
+while in development to ensure vagrant checks that the plugin is installed:
 
 ```ruby
 Vagrant.configure("2") do |config|

data/lib/vagrant-libvirt/action.rb

@@ -49,6 +49,7 @@ module VagrantPlugins
 
                 b2.use StartDomain
                 b2.use WaitTillUp
+                b2.use WaitForCommunicator, [:running]
 
                 b2.use ForwardPorts
                 b2.use SetHostname
@@ -107,6 +108,7 @@ module VagrantPlugins
                 # Machine should gain IP address when comming up,
                 # so wait for dhcp lease and store IP into machines data_dir.
                 b3.use WaitTillUp
+                b3.use WaitForCommunicator, [:running]
 
                 b3.use ForwardPorts
                 b3.use PrepareNFSSettings
@@ -179,6 +181,7 @@ module VagrantPlugins
               # Try to remove stale volumes anyway
               b2.use SetNameOfDomain
               b2.use RemoveStaleVolume if env[:machine].config.vm.box
+              b2.use CleanMachineFolder, quiet: true
               b2.use MessageNotCreated unless env[:result]
 
               next
@@ -191,6 +194,7 @@ module VagrantPlugins
                 b3.use PruneNFSExports
                 b3.use DestroyDomain
                 b3.use DestroyNetworks
+                b3.use CleanMachineFolder
               else
                 b3.use MessageWillNotDestroy
               end
@@ -324,6 +328,7 @@ module VagrantPlugins
       autoload :CreateDomainVolume, action_root.join('create_domain_volume')
       autoload :CreateNetworkInterfaces, action_root.join('create_network_interfaces')
       autoload :CreateNetworks, action_root.join('create_networks')
+      autoload :CleanMachineFolder, action_root.join('clean_machine_folder')
       autoload :DestroyDomain, action_root.join('destroy_domain')
       autoload :DestroyNetworks, action_root.join('destroy_networks')
       autoload :ForwardPorts, action_root.join('forward_ports')
@@ -366,6 +371,7 @@ module VagrantPlugins
       autoload :SyncedFolders, 'vagrant/action/builtin/synced_folders'
       autoload :SyncedFolderCleanup, 'vagrant/action/builtin/synced_folder_cleanup'
       autoload :ProvisionerCleanup, 'vagrant/action/builtin/provisioner_cleanup'
+      autoload :WaitForCommunicator, 'vagrant/action/builtin/wait_for_communicator'
     end
   end
 end

data/lib/vagrant-libvirt/action/clean_machine_folder.rb

@@ -0,0 +1,28 @@
+require 'log4r'
+
+module VagrantPlugins
+  module ProviderLibvirt
+    module Action
+      class CleanMachineFolder
+
+        def initialize(app, env, options=nil)
+          @logger = Log4r::Logger.new('vagrant_libvirt::action::create_domain')
+          @app = app
+          @ui = env[:ui]
+          @quiet = (options || {}).fetch(:quiet, false)
+        end
+
+        def call(env)
+          machine_folder = env[:machine].data_dir
+
+          @ui.info("Deleting the machine folder") unless @quiet
+
+          @logger.debug("Recursively removing: #{machine_folder}")
+          FileUtils.rm_rf(machine_folder, :secure => true)
+
+          @app.call(env)
+        end
+      end
+    end
+  end
+end

data/lib/vagrant-libvirt/action/create_domain.rb

@@ -92,6 +92,7 @@ module VagrantPlugins
           # Storage
           @storage_pool_name = config.storage_pool_name
           @snapshot_pool_name = config.snapshot_pool_name
+          @domain_volumes = []
           @disks = config.disks
           @cdroms = config.cdroms
 
@@ -141,19 +142,28 @@ module VagrantPlugins
             else
                 pool_name = @storage_pool_name
             end
-            @logger.debug "Search for volume in pool: #{pool_name}"
-            domain_volume = env[:machine].provider.driver.connection.volumes.all(
-              name: "#{@name}.img"
-            ).find { |x| x.pool_name == pool_name }
-            raise Errors::DomainVolumeExists if domain_volume.nil?
-            @domain_volume_path = domain_volume.path
+            @logger.debug "Search for volumes in pool: #{pool_name}"
+            env[:box_volumes].each_index do |index|
+              suffix_index = index > 0 ? "_#{index}" : ''
+              domain_volume = env[:machine].provider.driver.connection.volumes.all(
+                name: "#{@name}#{suffix_index}.img"
+              ).find { |x| x.pool_name == pool_name }
+              raise Errors::DomainVolumeExists if domain_volume.nil?
+              @domain_volumes.push({
+                :dev => (index+1).vdev.to_s,
+                :cache => @domain_volume_cache,
+                :bus => @disk_bus,
+                :path => domain_volume.path,
+                :virtual_size => env[:box_volumes][index][:virtual_size]
+              })
+              end
           end
 
           # If we have a box, take the path from the domain volume and set our storage_prefix.
           # If not, we dump the storage pool xml to get its defined path.
           # the default storage prefix is typically: /var/lib/libvirt/images/
           if env[:machine].config.vm.box
-            storage_prefix = File.dirname(@domain_volume_path) + '/' # steal
+            storage_prefix = File.dirname(@domain_volumes[0][:path]) + '/' # steal
           else
             storage_prefix = get_disk_storage_prefix(env, @storage_pool_name)
           end
@@ -188,7 +198,7 @@ module VagrantPlugins
                 path: disk[:absolute_path],
                 capacity: disk[:size],
                 owner: storage_uid(env),
-                group: storage_uid(env),
+                group: storage_gid(env),
                 #:allocation => ?,
                 pool_name: disk_pool_name
               )
@@ -227,7 +237,11 @@ module VagrantPlugins
             env[:ui].info(" -- Feature:           #{feature}")
           end
           @features_hyperv.each do |feature|
-            env[:ui].info(" -- Feature (HyperV):  name=#{feature[:name]}, state=#{feature[:state]}")
+            if feature[:name] == 'spinlocks'
+              env[:ui].info(" -- Feature (HyperV):  name=#{feature[:name]}, state=#{feature[:state]}, retries=#{feature[:retries]}")
+            else
+              env[:ui].info(" -- Feature (HyperV):  name=#{feature[:name]}, state=#{feature[:state]}")
+            end
           end
           env[:ui].info(" -- Clock offset:      #{@clock_offset}")
           @clock_timers.each do |timer|
@@ -250,7 +264,9 @@ module VagrantPlugins
             env[:ui].info(" -- Base box:          #{env[:machine].box.name}")
           end
           env[:ui].info(" -- Storage pool:      #{@storage_pool_name}")
-          env[:ui].info(" -- Image:             #{@domain_volume_path} (#{env[:box_virtual_size]}G)")
+          @domain_volumes.each do |volume|
+            env[:ui].info(" -- Image(#{volume[:device]}):     #{volume[:path]}, #{volume[:virtual_size]}G")
+          end
 
           if not @disk_driver_opts.empty?
             env[:ui].info(" -- Disk driver opts:  #{@disk_driver_opts.reject { |k,v| v.nil? }.map { |k,v| "#{k}='#{v}'"}.join(' ')}")