vagrant-libvirt 0.3.0 → 0.5.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4200a82105f1c9bf2860841eb3c801b103649c76afae39e303f28542316f0bbd
-  data.tar.gz: fd14f23b17262da08eac29401b779cde545913cb9b6bf4240b044a93a35fed52
+  metadata.gz: 318bb5e0023b717064b450108e5a8974386562b88c814cbe22e6ff98c7aebb3a
+  data.tar.gz: 784a85de569e533693e0d4478c013574fddb9dd9c11ba8c1fbae19d3d667d099
 SHA512:
-  metadata.gz: 7c068629389967340010d5d94594111fbbf661e895ede714bc45898bc7d2e21f860a8c8d6e9cdc4b7ae5ee77cdd1b6dd313193cb2d7b634971a251456695959b
-  data.tar.gz: 724c11548cf02a12567bbb87470087539e39ed70aa67dd2d5938a6ed5879667fc03315022dfc1c2c6ca367b2da2552cb408e264c9fd1b2ab090506723f67a688
+  metadata.gz: 133fb557db9fdfe87c5b81b2f0875b436ec60c71ac9d86199c02b23f44024d720331b8615507a3c8394c6b3172ca11959677e30bc283d1a75baccdbb444c0e84
+  data.tar.gz: '0873ca30d8422dc71070e69652a8f8edea1d85a0bd1708ce0ad8a6b0c5df66a0726dfc636a34d3ff690d5e0346fefab9cec5437d01ff05b3228797dfc4564b9a'

data/README.md

@@ -21,6 +21,7 @@ can help a lot :-)
 * [Using Docker based Installation](#using-docker-based-installation)
 * [Installation](#installation)
   * [Possible problems with plugin installation on Linux](#possible-problems-with-plugin-installation-on-linux)
+  * [Additional Notes for Fedora and Similar Linux Distributions](#additional-notes-for-fedora-and-similar-linux-distributions)
 * [Vagrant Project Preparation](#vagrant-project-preparation)
   * [Add Box](#add-box)
   * [Create Vagrantfile](#create-vagrantfile)
@@ -48,18 +49,23 @@ can help a lot :-)
 * [Watchdog device](#watchdog-device)
 * [Smartcard device](#smartcard-device)
 * [Hypervisor Features](#hypervisor-features)
+* [Clock](#clock)
 * [CPU features](#cpu-features)
 * [Memory Backing](#memory-backing)
 * [No box and PXE boot](#no-box-and-pxe-boot)
 * [SSH Access To VM](#ssh-access-to-vm)
 * [Forwarded Ports](#forwarded-ports)
+  * [Forwarding the ssh-port](#forwarding-the-ssh-port)
 * [Synced Folders](#synced-folders)
 * [QEMU Session Support](#qemu-session-support)
 * [Customized Graphics](#customized-graphics)
 * [TPM Devices](#tpm-devices)
+* [Memory balloon](#memory-balloon)
 * [Libvirt communication channels](#libvirt-communication-channels)
 * [Custom command line arguments and environment variables](#custom-command-line-arguments-and-environment-variables)
-* [Box Format](#box-format)
+* [Box Formats](#box-formats)
+  * [Version 1](#version-1)
+  * [Version 2 (Experimental)](#version-2-experimental)
 * [Create Box](#create-box)
 * [Package Box from VM](#package-box-from-vm)
 * [Troubleshooting VMs](#troubleshooting-vms)
@@ -80,7 +86,7 @@ can help a lot :-)
 * SSH into domains.
 * Setup hostname and network interfaces.
 * Provision domains with any built-in Vagrant provisioner.
-* Synced folder support via `rsync`, `nfs` or `9p`.
+* Synced folder support via `rsync`, `nfs`, `9p` or `virtiofs`.
 * Snapshots via [sahara](https://github.com/jedi4ever/sahara).
 * Package caching via
   [vagrant-cachier](http://fgrehm.viewdocs.io/vagrant-cachier/).
@@ -108,6 +114,12 @@ To get the image:
 docker pull vagrantlibvirt/vagrant-libvirt:latest
 ```
 
+Preparing the docker run, only once:
+
+```bash
+mkdir -p ~/.vagrant.d/{boxes,data,tmp}
+```
+
 Running the image:
 ```bash
 docker run -it --rm \
@@ -116,10 +128,26 @@ docker run -it --rm \
   -v ~/.vagrant.d:/.vagrant.d \
   -v $(pwd):$(pwd) \
   -w $(pwd) \
+  --network host \
   vagrantlibvirt/vagrant-libvirt:latest \
     vagrant status
 ```
 
+It's possible to define an alias in `~/.bashrc`, for example:
+```bash
+alias vagrant='
+  mkdir -p ~/.vagrant.d/{boxes,data,tmp}; \
+  docker run -it --rm \
+    -e LIBVIRT_DEFAULT_URI \
+    -v /var/run/libvirt/:/var/run/libvirt/ \
+    -v ~/.vagrant.d:/.vagrant.d \
+    -v $(pwd):$(pwd) \
+    -w $(pwd) \
+    --network host \
+    vagrantlibvirt/vagrant-libvirt:latest \
+    vagrant'
+```
+
 Note that if you are connecting to a remote system libvirt, you may omit the
 `-v /var/run/libvirt/:/var/run/libvirt/` mount bind. Some distributions patch the local
 vagrant environment to ensure vagrant-libvirt uses `qemu:///session`, which means you
@@ -155,6 +183,7 @@ vagrant-libvirt. This depends on your distro. An overview:
 apt-get build-dep vagrant ruby-libvirt
 apt-get install qemu libvirt-daemon-system libvirt-clients ebtables dnsmasq-base
 apt-get install libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev
+apt-get install libguestfs-tools
 ```
 
 * Ubuntu 18.04, Debian 8 and older:
@@ -162,23 +191,24 @@ apt-get install libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev
 apt-get build-dep vagrant ruby-libvirt
 apt-get install qemu libvirt-bin ebtables dnsmasq-base
 apt-get install libxslt-dev libxml2-dev libvirt-dev zlib1g-dev ruby-dev
+apt-get install libguestfs-tools
 ```
 
 (It is possible some users will already have libraries from the third line installed, but this is the way to make it work OOTB.)
 
 * CentOS 6, 7, Fedora 21:
 ```shell
-yum install qemu libvirt libvirt-devel ruby-devel gcc qemu-kvm
+yum install qemu libvirt libvirt-devel ruby-devel gcc qemu-kvm libguestfs-tools
 ```
 
 * Fedora 22 and up:
 ```shell
-dnf install -y gcc libvirt libvirt-devel libxml2-devel make ruby-devel
+dnf install -y gcc libvirt libvirt-devel libxml2-devel make ruby-devel libguestfs-tools
 ```
 
 * OpenSUSE leap 15.1:
 ```shell
-zypper install qemu libvirt libvirt-devel ruby-devel gcc qemu-kvm
+zypper install qemu libvirt libvirt-devel ruby-devel gcc qemu-kvm libguestfs
 ```
 
 * Arch Linux: please read the related [ArchWiki](https://wiki.archlinux.org/index.php/Vagrant#vagrant-libvirt) page.
@@ -213,8 +243,7 @@ On Ubuntu, Debian, make sure you are running all three of the `apt` commands abo
 On RedHat, Centos, Fedora, ...
 
 ```shell
-$ sudo dnf install libxslt-devel libxml2-devel libvirt-devel \
-  libguestfs-tools-c ruby-devel gcc
+$ sudo dnf install libxslt-devel libxml2-devel libvirt-devel ruby-devel gcc
 ```
 
 On Arch Linux it is recommended to follow [steps from ArchWiki](https://wiki.archlinux.org/index.php/Vagrant#vagrant-libvirt).
@@ -241,7 +270,20 @@ If you encounter the following load error when using the vagrant-libvirt plugin
 then the following steps have been found to resolve the problem. Thanks to James Reynolds (see https://github.com/hashicorp/vagrant/issues/11020#issuecomment-540043472). The specific version of libssh will change over time so references to the rpm in the commands below will need to be adjusted accordingly.
 
 ```shell
+# Fedora
 dnf download --source libssh
+
+# centos 8 stream, doesn't provide source RPMs, so you need to download like so
+git clone https://git.centos.org/centos-git-common
+# centos-git-common needs its tools in PATH
+export PATH=$(readlink -f ./centos-git-common):$PATH
+git clone https://git.centos.org/rpms/libssh
+cd libssh
+git checkout imports/c8s/libssh-0.9.4-1.el8
+into_srpm.sh -d c8s
+cd SRPMS
+
+# common commands (make sure to adjust verison accordingly)
 rpm2cpio libssh-0.9.0-5.fc30.src.rpm | cpio -imdV
 tar xf libssh-0.9.0.tar.xz
 mkdir build
@@ -260,7 +302,20 @@ If you encounter the following load error when using the vagrant-libvirt plugin
 then the following steps have been found to resolve the problem. After the steps below are complete, then reinstall the vagrant-libvirt plugin without setting the `CONFIGURE_ARGS`. Thanks to Marco Bevc (see https://github.com/hashicorp/vagrant/issues/11020#issuecomment-625801983):
 
 ```shell
+# Fedora
 dnf download --source krb5-libs
+
+# centos 8 stream, doesn't provide source RPMs, so you need to download like so
+git clone https://git.centos.org/centos-git-common
+# centos-git-common needs its tools in PATH
+export PATH=$(readlink -f ./centos-git-common):$PATH
+git clone https://git.centos.org/rpms/krb5
+cd krb5
+git checkout imports/c8s/krb5-1.18.2-8.el8
+into_srpm.sh -d c8s
+cd SRPMS
+
+# common commands (make sure to adjust verison accordingly)
 rpm2cpio krb5-1.18-1.fc32.src.rpm | cpio -imdV
 tar xf krb5-1.18.tar.gz
 cd krb5-1.18/src
@@ -349,10 +404,25 @@ URI](http://libvirt.org/uri.html):
   Default is `$HOME/.ssh/id_rsa`. Prepends `$HOME/.ssh/` if no directory
 * `socket` - Path to the Libvirt unix socket (e.g.
   `/var/run/libvirt/libvirt-sock`)
+* `proxy_command` - For advanced usage. When connecting to remote libvirt
+  instances, if the default constructed proxy\_command which uses `-W %h:%p`
+  does not work, set this as needed. It performs interpolation using `{key}`
+  and supports only `{host}`, `{username}`, and `{id_ssh_key_file}`. This is
+  to try and avoid issues with escaping `%` and `$` which might be necessary
+  to the ssh command itself. e.g.:
+  `libvirt.proxy_command = "ssh {host} -l {username} -i {id_ssh_key_file} nc %h %p"`
 * `uri` - For advanced usage. Directly specifies what Libvirt connection URI
   vagrant-libvirt should use. Overrides all other connection configuration
   options
 
+In the event that none of these are set (excluding the `driver` option) the
+provider will attempt to retrieve the uri from the environment variable
+`LIBVIRT_DEFAULT_URI` similar to how virsh works. If any of them are set, it
+will ignore the environment variable. The reason the driver option is ignored
+is that it is not uncommon for this to be explicitly set on the box itself
+and there is no easily to determine whether it is being set by the user or
+the box packager.
+
 Connection-independent options:
 
 * `storage_pool_name` - Libvirt storage pool name, where box image and instance
@@ -384,6 +454,14 @@ end
   set, which should be fine for paravirtualized guests, but some fully
   virtualized guests may require hda. NOTE: this option also applies only to
   disks associated with a box image.
+* `disk_driver` - Extra options for the main disk driver ([see Libvirt documentation](http://libvirt.org/formatdomain.html#elementsDisks)).
+  NOTE: this option also applies only to disks associated with a box image. In all cases, the value `nil` can be used to force the hypervisor default behaviour (e.g. to override settings defined in top-level Vagrantfiles). Supported options include:
+  * `:cache` - Controls the cache mechanism. Possible values are "default", "none", "writethrough", "writeback", "directsync" and "unsafe".
+  * `:io` - Controls specific policies on I/O. Possible values are "threads" and "native".
+  * `:copy_on_read` - Controls whether to copy read backing file into the image file. The value can be either "on" or "off".
+  * `:discard` - Controls whether discard requests (also known as "trim" or "unmap") are ignored or passed to the filesystem. Possible values are "unmap" or "ignore".
+    Note: for discard to work, you will likely also need to set `disk_bus = 'scsi'`
+  * `:detect_zeroes` - Controls whether to detect zero write requests. The value can be "off", "on" or "unmap".
 * `nic_model_type` - parameter specifies the model of the network adapter when
   you create a domain value by default virtio KVM believe possible values, see
   the [documentation for
@@ -429,10 +507,6 @@ end
   ]
   ```
 * `loader` - Sets path to custom UEFI loader.
-* `volume_cache` - Controls the cache mechanism. Possible values are "default",
-  "none", "writethrough", "writeback", "directsync" and "unsafe". [See
-  driver->cache in Libvirt
-  documentation](http://libvirt.org/formatdomain.html#elementsDisks).
 * `kernel` - To launch the guest with a kernel residing on host filesystems.
   Equivalent to qemu `-kernel`.
 * `initrd` - To specify the initramfs/initrd to use for the guest. Equivalent
@@ -504,6 +578,7 @@ end
 * `tpm_model` - The model of the TPM to which you wish to connect.
 * `tpm_type` - The type of TPM device to which you are connecting.
 * `tpm_path` - The path to the TPM device on the host system.
+* `tpm_version` - The TPM version to use.
 * `dtb` - The device tree blob file, mostly used for non-x86 platforms. In case
   the device tree isn't added in-line to the kernel, it can be manually
   specified here.
@@ -532,7 +607,7 @@ Vagrant.configure("2") do |config|
       domain.memory = 2048
       domain.cpus = 2
       domain.nested = true
-      domain.volume_cache = 'none'
+      domain.disk_driver :cache => 'none'
     end
   end
 
@@ -579,6 +654,7 @@ defined domain:
 * `tpm_model` - Updated
 * `tpm_type` - Updated
 * `tpm_path` - Updated
+* `tpm_version` - Updated
 
 ## Networks
 
@@ -820,6 +896,8 @@ used by this network are configurable at the provider level.
 * `management_network_pci_slot` -  The slot of the PCI device.
 * `management_network_mac` - MAC address of management network interface.
 * `management_network_domain` - Domain name assigned to the management network.
+* `management_network_mtu` - MTU size of management network. If not specified,
+  the Libvirt default (1500) will be used.
 
 You may wonder how vagrant-libvirt knows the IP address a VM received.  Libvirt
 doesn't provide a standard way to find out the IP address of a running domain.
@@ -841,11 +919,6 @@ It has a number of options:
 * `size` - Size of the disk image. If unspecified, defaults to 10G.
 * `type` - Type of disk image to create. Defaults to *qcow2*.
 * `bus` - Type of bus to connect device to. Defaults to *virtio*.
-* `cache` - Cache mode to use, e.g. `none`, `writeback`, `writethrough` (see
-  the [libvirt documentation for possible
-  values](http://libvirt.org/formatdomain.html#elementsDisks) or
-  [here](https://www.suse.com/documentation/sles11/book_kvm/data/sect1_chapter_book_kvm.html)
-  for a fuller explanation). Defaults to *default*.
 * `allow_existing` - Set to true if you want to allow the VM to use a
   pre-existing disk. If the disk doesn't exist it will be created.
   Disks with this option set to true need to be removed manually.
@@ -853,13 +926,25 @@ It has a number of options:
 * `serial` - Serial number of the disk device.
 * `wwn` - WWN number of the disk device.
 
+The following disk performance options can also be configured
+(see the [libvirt documentation for possible values](http://libvirt.org/formatdomain.html#elementsDisks)
+or [here](https://www.suse.com/documentation/sles11/book_kvm/data/sect1_chapter_book_kvm.html) for a fuller explanation).
+In all cases, the options use the hypervisor default if not specified, or if set to `nil`.
+
+* `cache` - Cache mode to use. Value may be `default`, `none`, `writeback`, `writethrough`, `directsync` or `unsafe`.
+* `io` - Controls specific policies on I/O. Value may be `threads` or `native`.
+* `copy_on_read` - Controls whether to copy read backing file into the image file. Value may be `on` or `off`.
+* `discard` - Controls whether discard requests (also known as "trim" or "unmap") are ignored or passed to the filesystem. Value may be `unmap` or `ignore`.
+  Note: for discard to work, you will likely also need to set `:bus => 'scsi'`
+* `detect_zeroes` - Controls whether to detect zero write requests. Value may be `off`, `on` or `unmap`.
+
 The following example creates two additional disks.
 
 ```ruby
 Vagrant.configure("2") do |config|
   config.vm.provider :libvirt do |libvirt|
     libvirt.storage :file, :size => '20G'
-    libvirt.storage :file, :size => '40G', :type => 'raw'
+    libvirt.storage :file, :size => '40G', :bus => 'scsi', :type => 'raw', :discard => 'unmap', :detect_zeroes => 'on'
   end
 end
 ```
@@ -1177,6 +1262,29 @@ Vagrant.configure("2") do |config|
     libvirt.hyperv_feature :name => 'relaxed', :state => 'on'
     # Enable virtual APIC
     libvirt.hyperv_feature :name => 'vapic', :state => 'on'
+    # Enable spinlocks (requires retries to be specified)
+    libvirt.hyperv_feature :name => 'spinlocks', :state => 'on', :retries => '8191'
+  end
+end
+```
+
+## Clock
+
+Clock offset can be specified via `libvirt.clock_offset`. (Default is utc)
+
+Additionally timers can be specified via `libvirt.clock_timer`.
+Available options for timers are: name, track, tickpolicy, frequency, mode,  present
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    # Set clock offset to localtime
+    libvirt.clock_offset = 'localtime'
+    # Timers ...
+    libvirt.clock_timer :name => 'rtc', :tickpolicy => 'catchup'
+    libvirt.clock_timer :name => 'pit', :tickpolicy => 'delay'
+    libvirt.clock_timer :name => 'hpet', :present => 'no'
+    libvirt.clock_timer :name => 'hypervclock', :present => 'yes'
   end
 end
 ```
@@ -1282,6 +1390,24 @@ Name of network "foreman_managed" is key for define boot order
       end
 ```
 
+An example VM that is PXE booted from the `br1` device (which must already be configured in the host machine), and if that fails, is booted from the disk:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.define :pxeclient do |pxeclient|
+    pxeclient.vm.network :public_network,
+      dev: 'br1',
+      auto_config: false
+    pxeclient.vm.provider :libvirt do |domain|
+      boot_network = {'dev' => 'br1'}
+      domain.storage :file, :size => '100G'
+      domain.boot boot_network
+      domain.boot 'hd'
+    end
+  end
+end
+```
+
 ## SSH Access To VM
 
 vagrant-libvirt supports vagrant's [standard ssh
@@ -1312,40 +1438,157 @@ Default is `eth0`.
 
 `config.vm.network :forwarded_port, guest: 80, host: 2000, host_ip: "0.0.0.0"`
 
+### Forwarding the ssh-port
+
+Vagrant-libvirt now supports forwarding the standard ssh-port on port 2222 from
+the localhost to allow for consistent provisioning steps/ports to be used when
+defining across multiple providers.
+
+To enable, set the following:
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    # Enable forwarding of forwarded_port with id 'ssh'.
+    libvirt.forward_ssh_port = true
+  end
+end
+```
+
+Previously by default libvirt skipped the forwarding of the ssh-port because
+you can access the machine directly. In the future it is expected that this
+will be enabled by default once autocorrect support is added to handle port
+collisions for multi machine environments gracefully.
+
 ## Synced Folders
 
-Vagrant automatically syncs the project folder on the host to `/vagrant` in the guest. You can also configure
-additional synced folders.
+Vagrant automatically syncs the project folder on the host to `/vagrant` in
+the guest. You can also configure additional synced folders.
 
-`vagrant-libvirt` supports bidirectional synced folders via [NFS](https://en.wikipedia.org/wiki/Network_File_System) or [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p or Plan 9](https://en.wikipedia.org/wiki/9P_(protocol))) and
-unidirectional via rsync. The default is NFS. Difference between NFS and 9p is explained [here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
+public network interface and you must connect to Libvirt via ssh.
 
-You can change the synced folder type for `/vagrant` by explicity configuring
-it an setting the type, e.g.
+**NFS**
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: 'rsync'
-```
+`vagrant-libvirt` supports
+[NFS](https://www.vagrantup.com/docs/synced-folders/nfs) as default with
+bidirectional synced folders.
 
-or
+Example with NFS:
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "squash", owner: "1000"
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant"
+end
 ```
 
-or
+**RSync**
 
-```shell
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "mapped", mount: false
+`vagrant-libvirt` supports
+[rsync](https://www.vagrantup.com/docs/synced-folders/rsync) with
+unidirectional synced folders.
+
+Example with rsync:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "rsync"
+end
 ```
 
+**9P**
+
+`vagrant-libvirt` supports [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p
+or Plan 9](https://en.wikipedia.org/wiki/9P_\(protocol\))) with bidirectional
+synced folders.
+
+Difference between NFS and 9p is explained
+[here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+
 For 9p shares, a `mount: false` option allows to define synced folders without
 mounting them at boot.
 
-Further documentation on using 9p can be found in [kernel docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in [QEMU wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly). Please do note that 9p depends on support in the guest and not all distros come with the 9p module by default.
+Example for `accessmode: "squash"` with 9p:
 
-**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
-public network interface and you must connect to Libvirt via ssh.
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "squash", owner: "1000"
+end
+```
+
+Example for `accessmode: "mapped"` with 9p:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "mapped", mount: false
+end
+```
+
+Further documentation on using 9p can be found in [kernel
+docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in
+[QEMU
+wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly).
+
+Please do note that 9p depends on support in the guest and not all distros
+come with the 9p module by default.
+
+**Virtio-fs**
+
+`vagrant-libvirt` supports [Virtio-fs](https://virtio-fs.gitlab.io/) with
+bidirectional synced folders.
+
+For virtiofs shares, a `mount: false` option allows to define synced folders
+without mounting them at boot.
+
+So far, passthrough is the only supported access mode and it requires running
+the virtiofsd daemon as root.
+
+QEMU needs to allocate the backing memory for all the guest RAM as shared
+memory, e.g. [Use file-backed
+memory](https://libvirt.org/kbase/virtiofs.html#host-setup) by enable
+`memory_backing_dir` option in `/etc/libvirt/qemu.conf`:
+
+``` shell
+memory_backing_dir = "/dev/shm"
+```
+
+Example for Libvirt \>= 6.2.0 (e.g. Ubuntu 20.10 with Linux 5.8.0 + QEMU 5.0 +
+Libvirt 6.6.0, i.e. NUMA nodes required) with virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.numa_nodes = [{ :cpus => "0-1", :memory => 8192, :memAccess => "shared" }]
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Example for Libvirt \>= 6.9.0 (e.g. Ubuntu 21.04 with Linux 5.11.0 + QEMU 5.2 +
+Libvirt 7.0.0, or Ubuntu 20.04 + [PPA
+enabled](https://launchpad.net/~savoury1/+archive/ubuntu/virtualisation)) with
+virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.memory = 8192
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Further documentation on using virtiofs can be found in [official
+HowTo](https://virtio-fs.gitlab.io/index.html#howto) and in [Libvirt
+KB](https://libvirt.org/kbase/virtiofs.html).
+
+Please do note that virtiofs depends on:
+
+  - Host: Linux \>= 5.4, QEMU \>= 4.2 and Libvirt \>= 6.2 (e.g. Ubuntu 20.10)
+  - Guest: Linux \>= 5.4 (e.g. Ubuntu 20.04)
 
 ## QEMU Session Support
 
@@ -1408,13 +1651,14 @@ Modern versions of Libvirt support connecting to TPM devices on the host
 system. This allows you to enable Trusted Boot Extensions, among other
 features, on your guest VMs.
 
-In general, you will only need to modify the `tpm_path` variable in your guest
-configuration. However, advanced usage, such as the application of a Software
-TPM, may require modifying the `tpm_model` and `tpm_type` variables.
+To passthrough a hardware TPM, you will generally only need to modify the
+`tpm_path` variable in your guest configuration. However, advanced usage,
+such as the application of a Software TPM, may require modifying the
+`tpm_model`, `tpm_type` and `tpm_version` variables.
 
-The TPM options will only be used if you specify a TPM path. Declarations of
-any TPM options without specifying a path will result in those options being
-ignored.
+The TPM options will only be used if you specify a TPM path or version.
+Declarations of any TPM options without specifying a path or version will
+result in those options being ignored.
 
 Here is an example of using the TPM options:
 
@@ -1428,6 +1672,41 @@ Vagrant.configure("2") do |config|
 end
 ```
 
+It's also possible for Libvirt to start an emulated TPM device on the host.
+Requires `swtpm` and `swtpm-tools`
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.tpm_model = "tpm-crb"
+    libvirt.tpm_type = "emulator"
+    libvirt.tpm_version = "2.0"
+  end
+end
+```
+
+## Memory balloon
+
+The configuration of the memory balloon device can be overridden. By default,
+libvirt will automatically attach a memory balloon; this behavior is preserved
+by not configuring any memballoon-related options. The memory balloon can be
+explicitly disabled by setting `memballoon_enabled` to `false`. Setting
+`memballoon_enabled` to `true` will allow additional configuration of
+memballoon-related options.
+
+Here is an example of using the memballoon options:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.memballoon_enabled = true
+    libvirt.memballoon_model = 'virtio'
+    libvirt.memballoon_pci_bus = '0x00'
+    libvirt.memballoon_pci_slot = '0x0f'
+  end
+end
+```
+
 ## Libvirt communication channels
 
 For certain functionality to be available within a guest, a private
@@ -1516,7 +1795,11 @@ Vagrant.configure("2") do |config|
 end
 ```
 
-## Box Format
+## Box Formats
+
+### Version 1
+
+This is the original format that most boxes currently use.
 
 You can view an example box in the
 [`example_box/directory`](https://github.com/vagrant-libvirt/vagrant-libvirt/tree/master/example_box).
@@ -1530,8 +1813,64 @@ The box is a tarball containing:
 * `Vagrantfile` that does default settings for the provider-specific
   configuration for this provider
 
+
+### Version 2 (Experimental)
+
+Due to the limitation of only being able to handle a single disk with the version 1 format, a new
+format was added to support boxes that need to specify multiple disks. This is still currently
+experimental and as such support for packaging has yet to be added. There is a script in the tools
+folder (tools/create_box_with_two_disks.sh) that should provide a guideline on how to create such
+a box for those that wish to experiment and provide early feedback.
+
+At it's most basic, it expects an array of disks to allow a specific order to be presented. Disks
+will be attached in this order and as such assume device names base on this within the VM. The
+'path' attribute is required, and is expected to be relative to the base of the box. This should
+allow placing the disk images within a nested directory within the box if it useful for those
+with a larger number of disks. The name allows overriding the target volume name that will be
+used in the libvirt storage pool. Note that vagrant-libvirt will still prefix the volume name
+with `#{box_name}_vagrant_box_image_#{box_version}_` to avoid accidental clashes with other boxes.
+
+Format and virtual size need no longer be specified as they are now retrieved directly from the
+provided image using `qemu-img info ...`.
+
+Example format:
+```json
+{
+  'disks': [
+      {
+          'path': 'disk1.img'
+      },
+      {
+          'path': 'disk2.img',
+          'name': 'secondary_disk'
+      },
+      {
+          'path': 'disk3.img'
+      }
+  ],
+  'provider': 'libvirt'
+}
+```
+
 ## Create Box
 
+If creating a box from a modified vagrant-libvirt machine, ensure that
+you have set the `config.ssh.insert_key = false` in the original Vagrantfile
+as otherwise Vagrant will replace the default connection key-pair that is
+required on first boot with one specific to the machine and prevent
+the default key from working on the exported result.
+```ruby
+Vagrant.configure("2") do |config|
+  # this setting is only recommended if planning to export the
+  # resulting machine
+  config.ssh.insert_key = false
+
+  config.vm.define :test_vm do |test_vm|
+    test_vm.vm.box = "fedora/32-cloud-base"
+  end
+end
+```
+
 To create a vagrant-libvirt box from a qcow2 image, run `create_box.sh`
 (located in the tools directory):
 
@@ -1629,17 +1968,49 @@ $ bundle install
 Once you have the dependencies, verify the unit tests pass with `rspec`:
 
 ```shell
-$ bundle exec rspec spec/
+$ export VAGRANT_HOME=$(mktemp -d)
+$ bundle exec rspec --fail-fast --color --format documentation
 ```
 
-If those pass, you're ready to start developing the plugin. You can test the
-plugin without installing it into your Vagrant environment by just creating a
-`Vagrantfile` in the top level of this directory (it is gitignored) that uses
-it. Don't forget to add following line at the beginning of your `Vagrantfile`
-while in development mode:
+If those pass, you're ready to start developing the plugin.
 
+Setting `VAGRANT_HOME` is to avoid issues with conflicting with other
+plugins/gems or data already present under `~/.vagrant.d`.
+
+Additionally if you wish to test against a specific version of vagrant you
+can control the version using the following before running the tests:
+
+```shell
+$ export VAGRANT_VERSION=v2.2.14
+```
+
+**Note** rvm is used by the maintainers to help provide an environment to test
+against multiple ruby versions that align with the ones used by vagrant for
+their embedded ruby depending on the release. You can see what version is used
+by looking at the current [unit tests](.github/workflows/unit-tests.yml)
+workflow.
+
+You can test the plugin without installing it into your Vagrant environment by
+just creating a `Vagrantfile` in the top level of this directory (it is
+gitignored) that uses it. You can add the following line to your Vagrantfile
+while in development to ensure vagrant checks that the plugin is installed:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vagrant.plugins = "vagrant-libvirt"
+end
+```
+Or add the following to the top of the file to ensure that any required plugins
+are installed globally:
 ```ruby
-Vagrant.require_plugin "vagrant-libvirt"
+REQUIRED_PLUGINS = %w(vagrant-libvirt)
+exit unless REQUIRED_PLUGINS.all? do |plugin|
+  Vagrant.has_plugin?(plugin) || (
+    puts "The #{plugin} plugin is required. Please install it with:"
+    puts "$ vagrant plugin install #{plugin}"
+    false
+  )
+end
 ```
 
 Now you can use bundler to execute Vagrant: