vagrant-libvirt 0.1.2 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 531920cd3f0420b7e66b8524d4999ae76bdd7b72b25112cb7d5d4ad91e3b151c
-  data.tar.gz: fd8e444a917769d361e022bbf0278e5df263decec5e45672f69e1d95e5d68669
+  metadata.gz: af89387fcdeb6518bd31879ac2eaa00373c3af1c02613b389868eb6c656b5f2b
+  data.tar.gz: 82fa3560eb88abac51088e894d8e475bbf4dbcda4b10596b43e7fc41d158945a
 SHA512:
-  metadata.gz: ab2759ad2755cf174a2f4c9f855f2237dabdb2543fe3f3fad54edd70a27cdf6f8b8357279617efb02487ef04d5a5f3aa7b3e1ec047d8a4b2fda15ba84fca9dea
-  data.tar.gz: ad85d67c9f592a481bba252b3c820f677a8635bcb5f11be83be760245edea5d50745228d8034a61efd63897c1089390a1e6459d8f31c3957aaa6cbe61705b32e
+  metadata.gz: 53d1714279eaa3ed9c0434b891bad6bc03b4f52a0c2a5aed6e0736cefa89a3f66243ec3d04e2bf1bb707f76169e2f799ffa98e63f3cfdc3c91ee0d56e5332695
+  data.tar.gz: 15ea6d1ce672545f14f3886a6d945f2349c85cca61a63604abf30f5a716b9665c65ea181fd2dcb96c39321567725911b230d22d1c589c888a2b05ec9c7715b13

data/README.md

@@ -16,54 +16,61 @@ can help a lot :-)
 <!-- note in vim set "let g:vmt_list_item_char='-'" to generate the correct output -->
 <!-- vim-markdown-toc GFM -->
 
-- [Features](#features)
-- [Future work](#future-work)
-- [Installation](#installation)
-  - [Possible problems with plugin installation on Linux](#possible-problems-with-plugin-installation-on-linux)
-- [Vagrant Project Preparation](#vagrant-project-preparation)
-  - [Add Box](#add-box)
-  - [Create Vagrantfile](#create-vagrantfile)
-  - [Start VM](#start-vm)
-  - [How Project Is Created](#how-project-is-created)
-  - [Libvirt Configuration](#libvirt-configuration)
-  - [Provider Options](#provider-options)
-  - [Domain Specific Options](#domain-specific-options)
-    - [Reload behavior](#reload-behavior)
-- [Networks](#networks)
-  - [Private Network Options](#private-network-options)
-  - [Public Network Options](#public-network-options)
-  - [Management Network](#management-network)
-- [Additional Disks](#additional-disks)
-  - [Reload behavior](#reload-behavior-1)
-- [CDROMs](#cdroms)
-- [Input](#input)
-- [PCI device passthrough](#pci-device-passthrough)
-- [Using USB Devices](#using-usb-devices)
-  - [USB Controller Configuration](#usb-controller-configuration)
-  - [USB Device Passthrough](#usb-device-passthrough)
-  - [USB Redirector Devices](#usb-redirector-devices)
-    - [Filter for USB Redirector Devices](#filter-for-usb-redirector-devices)
-- [Random number generator passthrough](#random-number-generator-passthrough)
-- [Watchdog device](#watchdog-device)
-- [Smartcard device](#smartcard-device)
-- [Hypervisor Features](#hypervisor-features)
-- [CPU features](#cpu-features)
-- [Memory Backing](#memory-backing)
-- [No box and PXE boot](#no-box-and-pxe-boot)
-- [SSH Access To VM](#ssh-access-to-vm)
-- [Forwarded Ports](#forwarded-ports)
-- [Synced Folders](#synced-folders)
-- [QEMU Session Support](#qemu-session-support)
-- [Customized Graphics](#customized-graphics)
-- [TPM Devices](#tpm-devices)
-- [Libvirt communication channels](#libvirt-communication-channels)
-- [Custom command line arguments and environment variables](#custom-command-line-arguments-and-environment-variables)
-- [Box Format](#box-format)
-- [Create Box](#create-box)
-- [Package Box from VM](#package-box-from-vm)
-- [Troubleshooting VMs](#troubleshooting-vms)
-- [Development](#development)
-- [Contributing](#contributing)
+* [Features](#features)
+* [Future work](#future-work)
+* [Using Docker based Installation](#using-docker-based-installation)
+* [Installation](#installation)
+  * [Possible problems with plugin installation on Linux](#possible-problems-with-plugin-installation-on-linux)
+  * [Additional Notes for Fedora and Similar Linux Distributions](#additional-notes-for-fedora-and-similar-linux-distributions)
+* [Vagrant Project Preparation](#vagrant-project-preparation)
+  * [Add Box](#add-box)
+  * [Create Vagrantfile](#create-vagrantfile)
+  * [Start VM](#start-vm)
+  * [How Project Is Created](#how-project-is-created)
+  * [Libvirt Configuration](#libvirt-configuration)
+  * [Provider Options](#provider-options)
+  * [Domain Specific Options](#domain-specific-options)
+    * [Reload behavior](#reload-behavior)
+* [Networks](#networks)
+  * [Private Network Options](#private-network-options)
+  * [Public Network Options](#public-network-options)
+  * [Management Network](#management-network)
+* [Additional Disks](#additional-disks)
+  * [Reload behavior](#reload-behavior-1)
+* [CDROMs](#cdroms)
+* [Input](#input)
+* [PCI device passthrough](#pci-device-passthrough)
+* [Using USB Devices](#using-usb-devices)
+  * [USB Controller Configuration](#usb-controller-configuration)
+  * [USB Device Passthrough](#usb-device-passthrough)
+  * [USB Redirector Devices](#usb-redirector-devices)
+    * [Filter for USB Redirector Devices](#filter-for-usb-redirector-devices)
+* [Random number generator passthrough](#random-number-generator-passthrough)
+* [Watchdog device](#watchdog-device)
+* [Smartcard device](#smartcard-device)
+* [Hypervisor Features](#hypervisor-features)
+* [Clock](#clock)
+* [CPU features](#cpu-features)
+* [Memory Backing](#memory-backing)
+* [No box and PXE boot](#no-box-and-pxe-boot)
+* [SSH Access To VM](#ssh-access-to-vm)
+* [Forwarded Ports](#forwarded-ports)
+  * [Forwarding the ssh-port](#forwarding-the-ssh-port)
+* [Synced Folders](#synced-folders)
+* [QEMU Session Support](#qemu-session-support)
+* [Customized Graphics](#customized-graphics)
+* [TPM Devices](#tpm-devices)
+* [Memory balloon](#memory-balloon)
+* [Libvirt communication channels](#libvirt-communication-channels)
+* [Custom command line arguments and environment variables](#custom-command-line-arguments-and-environment-variables)
+* [Box Formats](#box-formats)
+  * [Version 1](#version-1)
+  * [Version 2 (Experimental)](#version-2-experimental)
+* [Create Box](#create-box)
+* [Package Box from VM](#package-box-from-vm)
+* [Troubleshooting VMs](#troubleshooting-vms)
+* [Development](#development)
+* [Contributing](#contributing)
 
 <!-- vim-markdown-toc -->
 
@@ -79,7 +86,7 @@ can help a lot :-)
 * SSH into domains.
 * Setup hostname and network interfaces.
 * Provision domains with any built-in Vagrant provisioner.
-* Synced folder support via `rsync`, `nfs` or `9p`.
+* Synced folder support via `rsync`, `nfs`, `9p` or `virtiofs`.
 * Snapshots via [sahara](https://github.com/jedi4ever/sahara).
 * Package caching via
   [vagrant-cachier](http://fgrehm.viewdocs.io/vagrant-cachier/).
@@ -92,6 +99,61 @@ can help a lot :-)
 * Take a look at [open
   issues](https://github.com/vagrant-libvirt/vagrant-libvirt/issues?state=open).
 
+## Using Docker based Installation
+
+Due to the number of issues encountered around compatibility between the ruby runtime environment
+that is part of the upstream vagrant installation and the library dependencies of libvirt that
+this project requires to communicate with libvirt, there is a docker image build and published.
+
+This should allow users to execute vagrant with vagrant-libvirt without needing to deal with
+the compatibility issues, though you may need to extend the image for your own needs should
+you make use of additional plugins.
+
+To get the image:
+```bash
+docker pull vagrantlibvirt/vagrant-libvirt:latest
+```
+
+Preparing the docker run, only once:
+
+```bash
+mkdir -p ~/.vagrant.d/{boxes,data,tmp}
+```
+
+Running the image:
+```bash
+docker run -it --rm \
+  -e LIBVIRT_DEFAULT_URI \
+  -v /var/run/libvirt/:/var/run/libvirt/ \
+  -v ~/.vagrant.d:/.vagrant.d \
+  -v $(pwd):$(pwd) \
+  -w $(pwd) \
+  --network host \
+  vagrantlibvirt/vagrant-libvirt:latest \
+    vagrant status
+```
+
+It's possible to define an alias in `~/.bashrc`, for example:
+```bash
+alias vagrant='
+  mkdir -p ~/.vagrant.d/{boxes,data,tmp}; \
+  docker run -it --rm \
+    -e LIBVIRT_DEFAULT_URI \
+    -v /var/run/libvirt/:/var/run/libvirt/ \
+    -v ~/.vagrant.d:/.vagrant.d \
+    -v $(pwd):$(pwd) \
+    -w $(pwd) \
+    --network host \
+    vagrantlibvirt/vagrant-libvirt:latest \
+    vagrant'
+```
+
+Note that if you are connecting to a remote system libvirt, you may omit the
+`-v /var/run/libvirt/:/var/run/libvirt/` mount bind. Some distributions patch the local
+vagrant environment to ensure vagrant-libvirt uses `qemu:///session`, which means you
+may need to set the environment variable `LIBVIRT_DEFAULT_URI` to the same value if
+looking to use this in place of your distribution provided installation.
+
 ## Installation
 
 First, you should have both QEMU and Libvirt installed if you plan to run VMs
@@ -197,6 +259,69 @@ If you have issues building ruby-libvirt, try the following:
 ```shell
 CONFIGURE_ARGS='with-ldflags=-L/opt/vagrant/embedded/lib with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib' GEM_HOME=~/.vagrant.d/gems GEM_PATH=$GEM_HOME:/opt/vagrant/embedded/gems PATH=/opt/vagrant/embedded/bin:$PATH vagrant plugin install vagrant-libvirt
 ```
+### Additional Notes for Fedora and Similar Linux Distributions
+
+If you encounter the following load error when using the vagrant-libvirt plugin (note the required by libssh):
+
+```shell
+/opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': /opt/vagrant/embedded/lib64/libcrypto.so.1.1: version `OPENSSL_1_1_1b' not found (required by /lib64/libssh.so.4) - /home/xxx/.vagrant.d/gems/2.4.6/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)
+```
+then the following steps have been found to resolve the problem. Thanks to James Reynolds (see https://github.com/hashicorp/vagrant/issues/11020#issuecomment-540043472). The specific version of libssh will change over time so references to the rpm in the commands below will need to be adjusted accordingly.
+
+```shell
+# Fedora
+dnf download --source libssh
+
+# centos 8 stream, doesn't provide source RPMs, so you need to download like so
+git clone https://git.centos.org/centos-git-common
+# centos-git-common needs its tools in PATH
+export PATH=$(readlink -f ./centos-git-common):$PATH
+git clone https://git.centos.org/rpms/libssh
+cd libssh
+git checkout imports/c8s/libssh-0.9.4-1.el8
+into_srpm.sh -d c8s
+cd SRPMS
+
+# common commands (make sure to adjust verison accordingly)
+rpm2cpio libssh-0.9.0-5.fc30.src.rpm | cpio -imdV
+tar xf libssh-0.9.0.tar.xz
+mkdir build
+cd build
+cmake ../libssh-0.9.0 -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
+make
+sudo cp lib/libssh* /opt/vagrant/embedded/lib64
+```
+
+If you encounter the following load error when using the vagrant-libvirt plugin (note the required by libk5crypto):
+
+```shell
+/opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/rbelgrave/.vagrant.d/gems/2.4.9/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)
+```
+
+then the following steps have been found to resolve the problem. After the steps below are complete, then reinstall the vagrant-libvirt plugin without setting the `CONFIGURE_ARGS`. Thanks to Marco Bevc (see https://github.com/hashicorp/vagrant/issues/11020#issuecomment-625801983):
+
+```shell
+# Fedora
+dnf download --source krb5-libs
+
+# centos 8 stream, doesn't provide source RPMs, so you need to download like so
+git clone https://git.centos.org/centos-git-common
+# centos-git-common needs its tools in PATH
+export PATH=$(readlink -f ./centos-git-common):$PATH
+git clone https://git.centos.org/rpms/krb5
+cd krb5
+git checkout imports/c8s/krb5-1.18.2-8.el8
+into_srpm.sh -d c8s
+cd SRPMS
+
+# common commands (make sure to adjust verison accordingly)
+rpm2cpio krb5-1.18-1.fc32.src.rpm | cpio -imdV
+tar xf krb5-1.18.tar.gz
+cd krb5-1.18/src
+./configure
+make
+sudo cp -P lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/
+```
 
 ## Vagrant Project Preparation
 
@@ -210,7 +335,7 @@ You can find more Libvirt-ready boxes at
 example:
 
 ```shell
-vagrant init fedora/24-cloud-base
+vagrant init fedora/32-cloud-base
 ```
 
 ### Create Vagrantfile
@@ -221,7 +346,7 @@ information where necessary. For example:
 ```ruby
 Vagrant.configure("2") do |config|
   config.vm.define :test_vm do |test_vm|
-    test_vm.vm.box = "fedora/24-cloud-base"
+    test_vm.vm.box = "fedora/32-cloud-base"
   end
 end
 ```
@@ -278,14 +403,32 @@ URI](http://libvirt.org/uri.html):
   Default is `$HOME/.ssh/id_rsa`. Prepends `$HOME/.ssh/` if no directory
 * `socket` - Path to the Libvirt unix socket (e.g.
   `/var/run/libvirt/libvirt-sock`)
+* `proxy_command` - For advanced usage. When connecting to remote libvirt
+  instances, if the default constructed proxy\_command which uses `-W %h:%p`
+  does not work, set this as needed. It performs interpolation using `{key}`
+  and supports only `{host}`, `{username}`, and `{id_ssh_key_file}`. This is
+  to try and avoid issues with escaping `%` and `$` which might be necessary
+  to the ssh command itself. e.g.:
+  `libvirt.proxy_command = "ssh {host} -l {username} -i {id_ssh_key_file} nc %h %p"`
 * `uri` - For advanced usage. Directly specifies what Libvirt connection URI
   vagrant-libvirt should use. Overrides all other connection configuration
   options
 
+In the event that none of these are set (excluding the `driver` option) the
+provider will attempt to retrieve the uri from the environment variable
+`LIBVIRT_DEFAULT_URI` similar to how virsh works. If any of them are set, it
+will ignore the environment variable. The reason the driver option is ignored
+is that it is not uncommon for this to be explicitly set on the box itself
+and there is no easily to determine whether it is being set by the user or
+the box packager.
+
 Connection-independent options:
 
 * `storage_pool_name` - Libvirt storage pool name, where box image and instance
-  snapshots will be stored.
+  snapshots (if `snapshot_pool_name` is not set) will be stored.
+* `snapshot_pool_name` - Libvirt storage pool name. If set, the created
+  snapshot of the instance will be stored at this location instead of
+  `storage_pool_name`.
 
 For example:
 
@@ -299,6 +442,8 @@ end
 
 ### Domain Specific Options
 
+* `title` - A short description of the domain.
+* `description` - A human readable description of the virtual machine.
 * `disk_bus` - The type of disk device to emulate. Defaults to virtio if not
   set. Possible values are documented in Libvirt's [description for
   _target_](http://libvirt.org/formatdomain.html#elementsDisks). NOTE: this
@@ -308,6 +453,14 @@ end
   set, which should be fine for paravirtualized guests, but some fully
   virtualized guests may require hda. NOTE: this option also applies only to
   disks associated with a box image.
+* `disk_driver` - Extra options for the main disk driver ([see Libvirt documentation](http://libvirt.org/formatdomain.html#elementsDisks)).
+  NOTE: this option also applies only to disks associated with a box image. In all cases, the value `nil` can be used to force the hypervisor default behaviour (e.g. to override settings defined in top-level Vagrantfiles). Supported options include:
+  * `:cache` - Controls the cache mechanism. Possible values are "default", "none", "writethrough", "writeback", "directsync" and "unsafe".
+  * `:io` - Controls specific policies on I/O. Possible values are "threads" and "native".
+  * `:copy_on_read` - Controls whether to copy read backing file into the image file. The value can be either "on" or "off".
+  * `:discard` - Controls whether discard requests (also known as "trim" or "unmap") are ignored or passed to the filesystem. Possible values are "unmap" or "ignore".
+    Note: for discard to work, you will likely also need to set `disk_bus = 'scsi'`
+  * `:detect_zeroes` - Controls whether to detect zero write requests. The value can be "off", "on" or "unmap".
 * `nic_model_type` - parameter specifies the model of the network adapter when
   you create a domain value by default virtio KVM believe possible values, see
   the [documentation for
@@ -353,10 +506,6 @@ end
   ]
   ```
 * `loader` - Sets path to custom UEFI loader.
-* `volume_cache` - Controls the cache mechanism. Possible values are "default",
-  "none", "writethrough", "writeback", "directsync" and "unsafe". [See
-  driver->cache in Libvirt
-  documentation](http://libvirt.org/formatdomain.html#elementsDisks).
 * `kernel` - To launch the guest with a kernel residing on host filesystems.
   Equivalent to qemu `-kernel`.
 * `initrd` - To specify the initramfs/initrd to use for the guest. Equivalent
@@ -428,6 +577,7 @@ end
 * `tpm_model` - The model of the TPM to which you wish to connect.
 * `tpm_type` - The type of TPM device to which you are connecting.
 * `tpm_path` - The path to the TPM device on the host system.
+* `tpm_version` - The TPM version to use.
 * `dtb` - The device tree blob file, mostly used for non-x86 platforms. In case
   the device tree isn't added in-line to the kernel, it can be manually
   specified here.
@@ -456,7 +606,7 @@ Vagrant.configure("2") do |config|
       domain.memory = 2048
       domain.cpus = 2
       domain.nested = true
-      domain.volume_cache = 'none'
+      domain.disk_driver :cache => 'none'
     end
   end
 
@@ -503,6 +653,7 @@ defined domain:
 * `tpm_model` - Updated
 * `tpm_type` - Updated
 * `tpm_path` - Updated
+* `tpm_version` - Updated
 
 ## Networks
 
@@ -632,8 +783,8 @@ starts with `libvirt__` string. Here is a list of those options:
   only when dhcp is enabled.By default is the same host that runs the DHCP
   server.
 * `:libvirt__adapter` - Number specifiyng sequence number of interface.
-* `:libvirt__forward_mode` - Specify one of `veryisolated`, `none`, `nat` or
-  `route` options.  This option is used only when creating new network. Mode
+* `:libvirt__forward_mode` - Specify one of `veryisolated`, `none`, `open`, `nat`
+  or `route` options.  This option is used only when creating new network. Mode
   `none` will create isolated network without NATing or routing outside. You
   will want to use NATed forwarding typically to reach networks outside of
   hypervisor. Routed forwarding is typically useful to reach other networks
@@ -712,6 +863,7 @@ virtual network.
 * `:portgroup` - Name of Libvirt portgroup to connect to.
 * `:ovs` - Support to connect to an Open vSwitch bridge device. Default is
   'false'.
+* :ovs_interfaceid - Add Open vSwitch 'interfaceid' parameter.
 * `:trust_guest_rx_filters` - Support trustGuestRxFilters attribute. Details
   are listed [here](http://www.libvirt.org/formatdomain.html#elementsNICSDirect).
   Default is 'false'.
@@ -730,8 +882,8 @@ used by this network are configurable at the provider level.
   connected. Must include the address and subnet mask. If not specified the
   default is '192.168.121.0/24'.
 * `management_network_mode` - Network mode for the Libvirt management network.
-  Specify one of veryisolated, none, nat or route options. Further documented
-  under [Private Networks](#private-network-options)
+  Specify one of veryisolated, none, open, nat or route options. Further
+  documented under [Private Networks](#private-network-options)
 * `management_network_guest_ipv6` - Enable or disable guest-to-guest IPv6
   communication. See
   [here](https://libvirt.org/formatnetwork.html#examplesPrivate6), and
@@ -743,6 +895,8 @@ used by this network are configurable at the provider level.
 * `management_network_pci_slot` -  The slot of the PCI device.
 * `management_network_mac` - MAC address of management network interface.
 * `management_network_domain` - Domain name assigned to the management network.
+* `management_network_mtu` - MTU size of management network. If not specified,
+  the Libvirt default (1500) will be used.
 
 You may wonder how vagrant-libvirt knows the IP address a VM received.  Libvirt
 doesn't provide a standard way to find out the IP address of a running domain.
@@ -764,16 +918,24 @@ It has a number of options:
 * `size` - Size of the disk image. If unspecified, defaults to 10G.
 * `type` - Type of disk image to create. Defaults to *qcow2*.
 * `bus` - Type of bus to connect device to. Defaults to *virtio*.
-* `cache` - Cache mode to use, e.g. `none`, `writeback`, `writethrough` (see
-  the [libvirt documentation for possible
-  values](http://libvirt.org/formatdomain.html#elementsDisks) or
-  [here](https://www.suse.com/documentation/sles11/book_kvm/data/sect1_chapter_book_kvm.html)
-  for a fuller explanation). Defaults to *default*.
 * `allow_existing` - Set to true if you want to allow the VM to use a
   pre-existing disk. If the disk doesn't exist it will be created.
   Disks with this option set to true need to be removed manually.
 * `shareable` - Set to true if you want to simulate shared SAN storage.
 * `serial` - Serial number of the disk device.
+* `wwn` - WWN number of the disk device.
+
+The following disk performance options can also be configured
+(see the [libvirt documentation for possible values](http://libvirt.org/formatdomain.html#elementsDisks)
+or [here](https://www.suse.com/documentation/sles11/book_kvm/data/sect1_chapter_book_kvm.html) for a fuller explanation).
+In all cases, the options use the hypervisor default if not specified, or if set to `nil`.
+
+* `cache` - Cache mode to use. Value may be `default`, `none`, `writeback`, `writethrough`, `directsync` or `unsafe`.
+* `io` - Controls specific policies on I/O. Value may be `threads` or `native`.
+* `copy_on_read` - Controls whether to copy read backing file into the image file. Value may be `on` or `off`.
+* `discard` - Controls whether discard requests (also known as "trim" or "unmap") are ignored or passed to the filesystem. Value may be `unmap` or `ignore`.
+  Note: for discard to work, you will likely also need to set `:bus => 'scsi'`
+* `detect_zeroes` - Controls whether to detect zero write requests. Value may be `off`, `on` or `unmap`.
 
 The following example creates two additional disks.
 
@@ -781,7 +943,7 @@ The following example creates two additional disks.
 Vagrant.configure("2") do |config|
   config.vm.provider :libvirt do |libvirt|
     libvirt.storage :file, :size => '20G'
-    libvirt.storage :file, :size => '40G', :type => 'raw'
+    libvirt.storage :file, :size => '40G', :bus => 'scsi', :type => 'raw', :discard => 'unmap', :detect_zeroes => 'on'
   end
 end
 ```
@@ -1099,6 +1261,29 @@ Vagrant.configure("2") do |config|
     libvirt.hyperv_feature :name => 'relaxed', :state => 'on'
     # Enable virtual APIC
     libvirt.hyperv_feature :name => 'vapic', :state => 'on'
+    # Enable spinlocks (requires retries to be specified)
+    libvirt.hyperv_feature :name => 'spinlocks', :state => 'on', :retries => '8191'
+  end
+end
+```
+
+## Clock
+
+Clock offset can be specified via `libvirt.clock_offset`. (Default is utc)
+
+Additionally timers can be specified via `libvirt.clock_timer`.
+Available options for timers are: name, track, tickpolicy, frequency, mode,  present
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    # Set clock offset to localtime
+    libvirt.clock_offset = 'localtime'
+    # Timers ...
+    libvirt.clock_timer :name => 'rtc', :tickpolicy => 'catchup'
+    libvirt.clock_timer :name => 'pit', :tickpolicy => 'delay'
+    libvirt.clock_timer :name => 'hpet', :present => 'no'
+    libvirt.clock_timer :name => 'hypervclock', :present => 'yes'
   end
 end
 ```
@@ -1234,40 +1419,157 @@ Default is `eth0`.
 
 `config.vm.network :forwarded_port, guest: 80, host: 2000, host_ip: "0.0.0.0"`
 
+### Forwarding the ssh-port
+
+Vagrant-libvirt now supports forwarding the standard ssh-port on port 2222 from
+the localhost to allow for consistent provisioning steps/ports to be used when
+defining across multiple providers.
+
+To enable, set the following:
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    # Enable forwarding of forwarded_port with id 'ssh'.
+    libvirt.forward_ssh_port = true
+  end
+end
+```
+
+Previously by default libvirt skipped the forwarding of the ssh-port because
+you can access the machine directly. In the future it is expected that this
+will be enabled by default once autocorrect support is added to handle port
+collisions for multi machine environments gracefully.
+
 ## Synced Folders
 
-Vagrant automatically syncs the project folder on the host to `/vagrant` in the guest. You can also configure
-additional synced folders.
+Vagrant automatically syncs the project folder on the host to `/vagrant` in
+the guest. You can also configure additional synced folders.
 
-`vagrant-libvirt` supports bidirectional synced folders via [NFS](https://en.wikipedia.org/wiki/Network_File_System) or [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p or Plan 9](https://en.wikipedia.org/wiki/9P_(protocol))) and
-unidirectional via rsync. The default is NFS. Difference between NFS and 9p is explained [here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
+public network interface and you must connect to Libvirt via ssh.
 
-You can change the synced folder type for `/vagrant` by explicity configuring
-it an setting the type, e.g.
+**NFS**
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: 'rsync'
-```
+`vagrant-libvirt` supports
+[NFS](https://www.vagrantup.com/docs/synced-folders/nfs) as default with
+bidirectional synced folders.
 
-or
+Example with NFS:
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "squash", owner: "1000"
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant"
+end
 ```
 
-or
+**RSync**
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "mapped", mount: false
+`vagrant-libvirt` supports
+[rsync](https://www.vagrantup.com/docs/synced-folders/rsync) with
+unidirectional synced folders.
+
+Example with rsync:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "rsync"
+end
 ```
 
+**9P**
+
+`vagrant-libvirt` supports [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p
+or Plan 9](https://en.wikipedia.org/wiki/9P_\(protocol\))) with bidirectional
+synced folders.
+
+Difference between NFS and 9p is explained
+[here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+
 For 9p shares, a `mount: false` option allows to define synced folders without
 mounting them at boot.
 
-Further documentation on using 9p can be found in [kernel docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in [QEMU wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly). Please do note that 9p depends on support in the guest and not all distros come with the 9p module by default.
+Example for `accessmode: "squash"` with 9p:
 
-**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
-public network interface and you must connect to Libvirt via ssh.
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "squash", owner: "1000"
+end
+```
+
+Example for `accessmode: "mapped"` with 9p:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "mapped", mount: false
+end
+```
+
+Further documentation on using 9p can be found in [kernel
+docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in
+[QEMU
+wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly).
+
+Please do note that 9p depends on support in the guest and not all distros
+come with the 9p module by default.
+
+**Virtio-fs**
+
+`vagrant-libvirt` supports [Virtio-fs](https://virtio-fs.gitlab.io/) with
+bidirectional synced folders.
+
+For virtiofs shares, a `mount: false` option allows to define synced folders
+without mounting them at boot.
+
+So far, passthrough is the only supported access mode and it requires running
+the virtiofsd daemon as root.
+
+QEMU needs to allocate the backing memory for all the guest RAM as shared
+memory, e.g. [Use file-backed
+memory](https://libvirt.org/kbase/virtiofs.html#host-setup) by enable
+`memory_backing_dir` option in `/etc/libvirt/qemu.conf`:
+
+``` shell
+memory_backing_dir = "/dev/shm"
+```
+
+Example for Libvirt \>= 6.2.0 (e.g. Ubuntu 20.10 with Linux 5.8.0 + QEMU 5.0 +
+Libvirt 6.6.0, i.e. NUMA nodes required) with virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.numa_nodes = [{ :cpus => "0-1", :memory => 8192, :memAccess => "shared" }]
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Example for Libvirt \>= 6.9.0 (e.g. Ubuntu 21.04 with Linux 5.11.0 + QEMU 5.2 +
+Libvirt 7.0.0, or Ubuntu 20.04 + [PPA
+enabled](https://launchpad.net/~savoury1/+archive/ubuntu/virtualisation)) with
+virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.memory = 8192
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Further documentation on using virtiofs can be found in [official
+HowTo](https://virtio-fs.gitlab.io/index.html#howto) and in [Libvirt
+KB](https://libvirt.org/kbase/virtiofs.html).
+
+Please do note that virtiofs depends on:
+
+  - Host: Linux \>= 5.4, QEMU \>= 4.2 and Libvirt \>= 6.2 (e.g. Ubuntu 20.10)
+  - Guest: Linux \>= 5.4 (e.g. Ubuntu 20.04)
 
 ## QEMU Session Support
 
@@ -1330,13 +1632,14 @@ Modern versions of Libvirt support connecting to TPM devices on the host
 system. This allows you to enable Trusted Boot Extensions, among other
 features, on your guest VMs.
 
-In general, you will only need to modify the `tpm_path` variable in your guest
-configuration. However, advanced usage, such as the application of a Software
-TPM, may require modifying the `tpm_model` and `tpm_type` variables.
+To passthrough a hardware TPM, you will generally only need to modify the
+`tpm_path` variable in your guest configuration. However, advanced usage,
+such as the application of a Software TPM, may require modifying the
+`tpm_model`, `tpm_type` and `tpm_version` variables.
 
-The TPM options will only be used if you specify a TPM path. Declarations of
-any TPM options without specifying a path will result in those options being
-ignored.
+The TPM options will only be used if you specify a TPM path or version.
+Declarations of any TPM options without specifying a path or version will
+result in those options being ignored.
 
 Here is an example of using the TPM options:
 
@@ -1350,6 +1653,41 @@ Vagrant.configure("2") do |config|
 end
 ```
 
+It's also possible for Libvirt to start an emulated TPM device on the host.
+Requires `swtpm` and `swtpm-tools`
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.tpm_model = "tpm-crb"
+    libvirt.tpm_type = "emulator"
+    libvirt.tpm_version = "2.0"
+  end
+end
+```
+
+## Memory balloon
+
+The configuration of the memory balloon device can be overridden. By default,
+libvirt will automatically attach a memory balloon; this behavior is preserved
+by not configuring any memballoon-related options. The memory balloon can be
+explicitly disabled by setting `memballoon_enabled` to `false`. Setting
+`memballoon_enabled` to `true` will allow additional configuration of
+memballoon-related options.
+
+Here is an example of using the memballoon options:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.memballoon_enabled = true
+    libvirt.memballoon_model = 'virtio'
+    libvirt.memballoon_pci_bus = '0x00'
+    libvirt.memballoon_pci_slot = '0x0f'
+  end
+end
+```
+
 ## Libvirt communication channels
 
 For certain functionality to be available within a guest, a private
@@ -1395,7 +1733,7 @@ For example:
 
 ```ruby
 Vagrant.configure(2) do |config|
-  config.vm.box = "fedora/24-cloud-base"
+  config.vm.box = "fedora/32-cloud-base"
   config.vm.provider :libvirt do |libvirt|
     libvirt.channel :type => 'unix', :target_name => 'org.qemu.guest_agent.0', :target_type => 'virtio'
   end
@@ -1438,7 +1776,11 @@ Vagrant.configure("2") do |config|
 end
 ```
 
-## Box Format
+## Box Formats
+
+### Version 1
+
+This is the original format that most boxes currently use.
 
 You can view an example box in the
 [`example_box/directory`](https://github.com/vagrant-libvirt/vagrant-libvirt/tree/master/example_box).
@@ -1452,8 +1794,64 @@ The box is a tarball containing:
 * `Vagrantfile` that does default settings for the provider-specific
   configuration for this provider
 
+
+### Version 2 (Experimental)
+
+Due to the limitation of only being able to handle a single disk with the version 1 format, a new
+format was added to support boxes that need to specify multiple disks. This is still currently
+experimental and as such support for packaging has yet to be added. There is a script in the tools
+folder (tools/create_box_with_two_disks.sh) that should provide a guideline on how to create such
+a box for those that wish to experiment and provide early feedback.
+
+At it's most basic, it expects an array of disks to allow a specific order to be presented. Disks
+will be attached in this order and as such assume device names base on this within the VM. The
+'path' attribute is required, and is expected to be relative to the base of the box. This should
+allow placing the disk images within a nested directory within the box if it useful for those
+with a larger number of disks. The name allows overriding the target volume name that will be
+used in the libvirt storage pool. Note that vagrant-libvirt will still prefix the volume name
+with `#{box_name}_vagrant_box_image_#{box_version}_` to avoid accidental clashes with other boxes.
+
+Format and virtual size need no longer be specified as they are now retrieved directly from the
+provided image using `qemu-img info ...`.
+
+Example format:
+```json
+{
+  'disks': [
+      {
+          'path': 'disk1.img'
+      },
+      {
+          'path': 'disk2.img',
+          'name': 'secondary_disk'
+      },
+      {
+          'path': 'disk3.img'
+      }
+  ],
+  'provider': 'libvirt'
+}
+```
+
 ## Create Box
 
+If creating a box from a modified vagrant-libvirt machine, ensure that
+you have set the `config.ssh.insert_key = false` in the original Vagrantfile
+as otherwise Vagrant will replace the default connection key-pair that is
+required on first boot with one specific to the machine and prevent
+the default key from working on the exported result.
+```ruby
+Vagrant.configure("2") do |config|
+  # this setting is only recommended if planning to export the
+  # resulting machine
+  config.ssh.insert_key = false
+
+  config.vm.define :test_vm do |test_vm|
+    test_vm.vm.box = "fedora/32-cloud-base"
+  end
+end
+```
+
 To create a vagrant-libvirt box from a qcow2 image, run `create_box.sh`
 (located in the tools directory):
 
@@ -1551,17 +1949,49 @@ $ bundle install
 Once you have the dependencies, verify the unit tests pass with `rspec`:
 
 ```shell
-$ bundle exec rspec spec/
+$ export VAGRANT_HOME=$(mktemp -d)
+$ bundle exec rspec --fail-fast --color --format documentation
+```
+
+If those pass, you're ready to start developing the plugin.
+
+Setting `VAGRANT_HOME` is to avoid issues with conflicting with other
+plugins/gems or data already present under `~/.vagrant.d`.
+
+Additionally if you wish to test against a specific version of vagrant you
+can control the version using the following before running the tests:
+
+```shell
+$ export VAGRANT_VERSION=v2.2.14
 ```
 
-If those pass, you're ready to start developing the plugin. You can test the
-plugin without installing it into your Vagrant environment by just creating a
-`Vagrantfile` in the top level of this directory (it is gitignored) that uses
-it. Don't forget to add following line at the beginning of your `Vagrantfile`
-while in development mode:
+**Note** rvm is used by the maintainers to help provide an environment to test
+against multiple ruby versions that align with the ones used by vagrant for
+their embedded ruby depending on the release. You can see what version is used
+by looking at the current [unit tests](.github/workflows/unit-tests.yml)
+workflow.
+
+You can test the plugin without installing it into your Vagrant environment by
+just creating a `Vagrantfile` in the top level of this directory (it is
+gitignored) that uses it. You can add the following line to your Vagrantfile
+while in development to ensure vagrant checks that the plugin is installed:
 
 ```ruby
-Vagrant.require_plugin "vagrant-libvirt"
+Vagrant.configure("2") do |config|
+  config.vagrant.plugins = "vagrant-libvirt"
+end
+```
+Or add the following to the top of the file to ensure that any required plugins
+are installed globally:
+```ruby
+REQUIRED_PLUGINS = %w(vagrant-libvirt)
+exit unless REQUIRED_PLUGINS.all? do |plugin|
+  Vagrant.has_plugin?(plugin) || (
+    puts "The #{plugin} plugin is required. Please install it with:"
+    puts "$ vagrant plugin install #{plugin}"
+    false
+  )
+end
 ```
 
 Now you can use bundler to execute Vagrant: